/usr/bin/tqdm in filesystem NOT owned by any package? (python tqdm)
Devs, I was surprised trying to install a python-tqdm package to get the error: (2/2) checking for file conflicts [######################################] 100% error: failed to commit transaction (conflicting files) python-tqdm: /usr/bin/tqdm exists in filesystem Errors occurred, no packages were upgraded. The package is there: $ l /usr/bin/tqdm -rwxr-xr-x 1 root root 206 Jul 8 2021 /usr/bin/tqdm I check with pacman to find the owning package -- and there is none: $ pacman -Qo /usr/bin/tqdm error: No package owns /usr/bin/tqdm How can I check how it got there and what may depend on it? I don't install stray packages, and especially not to /usr/bin, so this is a surprise. I don't want to just willy-nilly remove it either and have something unknown break. Best course of action? -- David C. Rankin, J.D.,P.E.
Aug 10, 2024 20:18:48 David C. Rankin <drankinatty@gmail.com>:
Devs,
I was surprised trying to install a python-tqdm package to get the error:
(2/2) checking for file conflicts [######################################] 100% error: failed to commit transaction (conflicting files) python-tqdm: /usr/bin/tqdm exists in filesystem Errors occurred, no packages were upgraded.
The package is there:
$ l /usr/bin/tqdm -rwxr-xr-x 1 root root 206 Jul 8 2021 /usr/bin/tqdm
I check with pacman to find the owning package -- and there is none:
$ pacman -Qo /usr/bin/tqdm error: No package owns /usr/bin/tqdm
How can I check how it got there and what may depend on it? I don't install stray packages, and especially not to /usr/bin, so this is a surprise. I don't want to just willy-nilly remove it either and have something unknown break.
Best course of action?
-- David C. Rankin, J.D.,P.E. You can probably just delete it and any other file that conflicts for this. It was likely installed through `pip install -g` which is why pacman isn't tracking it or what package it's from, if you're replacing it with python-tqdm from the pacman repository, whatever installed it through pip shouldn't be able to tell the difference.
-- Kusoneko GPG: https://kusoneko.moe/gpg.txt https://kusoneko.moe
On Sat, 2024-08-10 at 20:22 -0400, Kusoneko wrote:
Aug 10, 2024 20:18:48 David C. Rankin <drankinatty@gmail.com>:
Devs,
I was surprised trying to install a python-tqdm package to get the error:
(2/2) checking for file conflicts [######################################] 100% error: failed to commit transaction (conflicting files) python-tqdm: /usr/bin/tqdm exists in filesystem Errors occurred, no packages were upgraded.
The package is there:
$ l /usr/bin/tqdm -rwxr-xr-x 1 root root 206 Jul 8 2021 /usr/bin/tqdm
I check with pacman to find the owning package -- and there is none:
$ pacman -Qo /usr/bin/tqdm error: No package owns /usr/bin/tqdm
How can I check how it got there and what may depend on it? I don't install stray packages, and especially not to /usr/bin, so this is a surprise. I don't want to just willy-nilly remove it either and have something unknown break.
Best course of action?
-- David C. Rankin, J.D.,P.E. You can probably just delete it and any other file that conflicts for this. It was likely installed through `pip install -g` which is why pacman isn't tracking it or what package it's from, if you're replacing it with python-tqdm from the pacman repository, whatever installed it through pip shouldn't be able to tell the difference.
-- Kusoneko GPG: https://kusoneko.moe/gpg.txt https://kusoneko.moe
And if you're really worried about something breaking, you can always rename the file, install python-tqdm, and see if anything breaks. If anything does then you can easily revert by just renaming the binaries. Perhaps the tqdm versions are the same and you won't even need to do anything. -- Best regards, Brian
On 8/10/24 7:22 PM, Kusoneko wrote:
You can probably just delete it and any other file that conflicts for this. It was likely installed through `pip install -g` which is why pacman isn't tracking it or what package it's from, if you're replacing it with python-tqdm from the pacman repository, whatever installed it through pip shouldn't be able to tell the difference.
Thank you! I also found https://bbs.archlinux.org/viewtopic.php?id=285543 after I sent the e-mail. I've just moved it to tqdm-orphan and installed python-tqdm. The files are identical. It's some damn pip file that got pulled in from some app. I hate that insecure python software supply-chain. I attempted to remove it with pip and it gave a horrible warning: <quote> # pip uninstall /usr/bin/tqdm ERROR: Invalid requirement: '/usr/bin/tqdm': Expected package name at the start of dependency specifier /usr/bin/tqdm ^ Hint: It looks like a path. The path does exist. [19:21 valkyrie:.../pkg/bld/zchunk-git] # pip uninstall python-tqdm error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try 'pacman -S python-xyz', where xyz is the package you are trying to install. If you wish to install a non-Arch-packaged Python package, create a virtual environment using 'python -m venv path/to/venv'. Then use path/to/venv/bin/python and path/to/venv/bin/pip. If you wish to install a non-Arch packaged Python application, it may be easiest to use 'pipx install xyz', which will manage a virtual environment for you. Make sure you have python-pipx installed via pacman. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. </quote> I'll have to find and remove the rest of the trash that slithered into my box through pip. -- David C. Rankin, J.D.,P.E.
All, I would like to defend Python a little bit here. On Sunday, August 11th, 2024 at 02:48, David C. Rankin <drankinatty@gmail.com> wrote:
I hate that insecure python software supply-chain.
In my opinion, it is the responsibility of the system administrator to avoid installing software other than by using the package manager. Using `pip install` is only one such installation method; one could have also just unpacked an archive into /usr/bin, or created their own shell script there -- all with the same result, but none of the Python's fault. Arguably, Python should not even offer this option. However, at the time Python was created, this was pretty much normal behaviour (e.g. Perl and Tex package managers would eagerly install packages globally), and removing it now would probably be extremely tedious. However, Python already improves on that by allowing to mark an environment as "externally managed"[1], which is what the error message is about. Finally, if you would like to find all files that are not installed by Pacman, consider trying lostfiles[2]. Best regards, -- Edward [1] https://packaging.python.org/en/latest/specifications/externally-managed-env... [2] https://archlinux.org/packages/extra/any/lostfiles/
El Sun, 11 Aug 2024 07:28:08 +0000 Edward Toroshchyn <edward.toroshchyn@pm.me> escribió:
All,
I would like to defend Python a little bit here.
On Sunday, August 11th, 2024 at 02:48, David C. Rankin <drankinatty@gmail.com> wrote:
I hate that insecure python software supply-chain.
In my opinion, it is the responsibility of the system administrator to avoid installing software other than by using the package manager. Using `pip install` is only one such installation method; one could have also just unpacked an archive into /usr/bin, or created their own shell script there -- all with the same result, but none of the Python's fault.
Arguably, Python should not even offer this option. However, at the time Python was created, this was pretty much normal behaviour (e.g. Perl and Tex package managers would eagerly install packages globally), and removing it now would probably be extremely tedious. However, Python already improves on that by allowing to mark an environment as "externally managed"[1], which is what the error message is about.
Finally, if you would like to find all files that are not installed by Pacman, consider trying lostfiles[2].
Best regards, -- Edward
[1] https://packaging.python.org/en/latest/specifications/externally-managed-env... [2] https://archlinux.org/packages/extra/any/lostfiles/
I don't like python too much, but in this case Edward Toroshchyn is right. Normally I only install python packages with pacman when they are dependencies of antoher program. When it's a isolate Python applications I use the wonderful virtual enviorement of Python. The correct way is to create a virtual enviorment in a user space create the variables in a .bashrc o whatever you use like shell. I recommend pipx for all that because is a very power tool.
On 8/11/24 2:45 PM, Maxxcan Fox wrote:
I don't like python too much, but in this case Edward Toroshchyn is right.
Normally I only install python packages with pacman when they are dependencies of antoher program. When it's a isolate Python applications I use the wonderful virtual enviorement of Python.
The correct way is to create a virtual enviorment in a user space create the variables in a .bashrc o whatever you use like shell.
I recommend pipx for all that because is a very power tool.
Maxxcan and Edward, Thank you both. My frustration with python is its proliferation of included packages that have grown like weeds in a vacant lot for version 3. Seems every app tries to pull in 20 other pieces that it just grabbed off the shelf somewhere instead of writing itself. Granted if I kept up more with python 3 I'd probably be more comfortable with that. For I all know it's Jia Tan authoring the pulled in packages. If I recall correctly, it was a pacman package I installed in 2021 that when installing or on first run required additional packages from pip or whatever installed tqdm. That is the frustrating part. I don't like allowing any other packages on the system other than those I write or those I install with pacman. The number of supply-chain compromises over the past 2-3 years with pip, etc.. has really been a turn-off. I'll check out pipx and see if I can make better friends with it. I would prefer apps were written in a compiled language, but the world loves shortcuts, so I guess we are stuck with python. If there was just a way of separating the quality, well designed parts from the cruft some 15 yr. old cobbled together with Llama I'd feel better. -- David C. Rankin, J.D.,P.E.
David, I understand you perfectly. Now, programmers don't program. They make programs that combine different blocks of code from other people. This makes the programming easier but harder to maintain, and too much insecure. You know the Python world, but the Javascript/Typescript world is worse. When you want to install any javascript program thanks to npm install to many library. But with the new language is the same, Clojure, Rust, Java, etc. But it seems that Pacman manages it well even if it is always improvable. It is the world that we have had to live. El dom, 11 de ago de 2024 a las 15:16:09 PM, David C. Rankin <drankinatty@gmail.com> escribió:
On 8/11/24 2:45 PM, Maxxcan Fox wrote:
I don't like python too much, but in this case Edward Toroshchyn is right.
Normally I only install python packages with pacman when they are dependencies of antoher program. When it's a isolate Python applications I use the wonderful virtual enviorement of Python.
The correct way is to create a virtual enviorment in a user space create the variables in a .bashrc o whatever you use like shell.
I recommend pipx for all that because is a very power tool.
Maxxcan and Edward, Thank you both.
My frustration with python is its proliferation of included packages that have grown like weeds in a vacant lot for version 3. Seems every app tries to pull in 20 other pieces that it just grabbed off the shelf somewhere instead of writing itself. Granted if I kept up more with python 3 I'd probably be more comfortable with that. For I all know it's Jia Tan authoring the pulled in packages.
If I recall correctly, it was a pacman package I installed in 2021 that when installing or on first run required additional packages from pip or whatever installed tqdm. That is the frustrating part.
I don't like allowing any other packages on the system other than those I write or those I install with pacman. The number of supply-chain compromises over the past 2-3 years with pip, etc.. has really been a turn-off.
I'll check out pipx and see if I can make better friends with it. I would prefer apps were written in a compiled language, but the world loves shortcuts, so I guess we are stuck with python. If there was just a way of separating the quality, well designed parts from the cruft some 15 yr. old cobbled together with Llama I'd feel better.
-- David C. Rankin, J.D.,P.E.
Hi everyone, for one, this was a thread about an unmanaged file in /usr/bin but has evolved first to discussing supply-chain attacks and then to incompetent programmers.
"Anyone can code" is one of the most ridiculous statements so far this millenia
“Anyone can code” is an educational statement so those who have potenial problem solving can learn it.
If software development is reduced to button-clicking and YAML manifests, then there's no incentive to learn that stuff.
It’s not as if it would replace proper programming; I have yet to see a functional app built without code. The art of programming is problem solving, not coding. -- tippfehlr
I would like to share my perspective. The practice of programming by combining and building upon code that others have written is fundamental to the ethos of open source and collaboration in general, whether in public or private projects. It is not a mark against anyone that they haven’t written every single line of code themselves. Leveraging the strengths of others is often a wise approach, and it is at the heart of effective teamwork. There are various skill levels in coding, and I believe it's important to respect each of them rather than dismissing anyone as 'not a programmer.' We should also be cautious with phrases like 'Back in my days...' as they can often reflect a longing for the past rather than an objective assessment of the present. Every generation brings its own strengths and challenges, and it’s essential to view them with clarity rather than nostalgia. On Mon, 12 Aug 2024 at 08:28, tippfehlr <tippfehlr@tippfehlr.eu> wrote:
Hi everyone,
for one, this was a thread about an unmanaged file in /usr/bin but has evolved first to discussing supply-chain attacks and then to incompetent programmers.
"Anyone can code" is one of the most ridiculous statements so far this millenia
“Anyone can code” is an educational statement so those who have potenial problem solving can learn it.
If software development is reduced to button-clicking and YAML manifests, then there's no incentive to learn that stuff.
It’s not as if it would replace proper programming; I have yet to see a functional app built without code.
The art of programming is problem solving, not coding.
-- tippfehlr
“Anyone can code” is an educational statement so those who have potenial problem solving can learn it. Educational statement or not, it attracted a lot of mediocre people to the field. If software development is reduced to button-clicking and YAMLmanifests, then there's no incentive to learn that stuff.It’s not as if it would replace proper programming; I have yet to see a functional app built without code.
Best regards, Brian Sent with Spark On Aug 12, 2024 at 2:27 AM -0500, tippfehlr <tippfehlr@tippfehlr.eu>, wrote:
Hi everyone,
for one, this was a thread about an unmanaged file in /usr/bin but has evolved first to discussing supply-chain attacks and then to incompetent programmers.
"Anyone can code" is one of the most ridiculous statements so far this millenia
“Anyone can code” is an educational statement so those who have potenial problem solving can learn it.
If software development is reduced to button-clicking and YAML manifests, then there's no incentive to learn that stuff.
It’s not as if it would replace proper programming; I have yet to see a functional app built without code.
The art of programming is problem solving, not coding.
-- tippfehlr
Sorry, new email client.
It’s not as if it would replace proper programming; I have yet to see a functional app built without code. Agreed. I wasn't implying that it would. The art of programming is problem solving, not coding. 100% agree.
Best regards, Brian On Aug 12, 2024 at 10:03 AM -0500, Brian Thompson <brianrobt@pm.me>, wrote:
“Anyone can code” is an educational statement so those who have potenial problem solving can learn it. Educational statement or not, it attracted a lot of mediocre people to the field. If software development is reduced to button-clicking and YAMLmanifests, then there's no incentive to learn that stuff.It’s not as if it would replace proper programming; I have yet to see a functional app built without code.
Best regards,
Brian Sent with Spark On Aug 12, 2024 at 2:27 AM -0500, tippfehlr <tippfehlr@tippfehlr.eu>, wrote:
Hi everyone,
for one, this was a thread about an unmanaged file in /usr/bin but has evolved first to discussing supply-chain attacks and then to incompetent programmers.
"Anyone can code" is one of the most ridiculous statements so far this millenia
“Anyone can code” is an educational statement so those who have potenial problem solving can learn it.
If software development is reduced to button-clicking and YAML manifests, then there's no incentive to learn that stuff.
It’s not as if it would replace proper programming; I have yet to see a functional app built without code.
The art of programming is problem solving, not coding.
-- tippfehlr
Hello everyone I don't want to make this thread any longer on this subject. On the one hand, you are right those who say that we should not belittle anyone, and not fall into the "in my day people were prettier and smarter", but we can not ignore if because of erroneous educational systems now people are more mediocre and it is not their fault cause those systems. About the problem of education I don't say it myself, there are many studies of experts on the subject talking about it. On the other hand, I am a sysadmin, not a programmer and I respect them a lot, but sometimes because of them I have problems in my job. It's good not to offend anyone, but sometimes our decisions affect others. Imagine that your car breaks down. You take it to the garage and because of a bad mechanic, the car stops on the highway. Objectively, won't you think that the mechanic is bad and to blame for your misfortune? Then we can discuss, if his training was good or bad, if he is a bad mechanic because he was not well supervised by his boss, or if the poor guy had too much work. But we are in the middle of the road As I say, I don't want to make this topic longer but to be more positive I think that pacman manages well all those programs with a lot of dependencies although I tell you, it can be improved and I hate to have to install 50 python packages for one program, but it also happens to me with haskell and pandoc. Regards El Mon, 12 Aug 2024 15:05:50 +0000 Brian Thompson <brianrobt@pm.me> escribió:
Sorry, new email client.
It’s not as if it would replace proper programming; I have yet to see a functional app built without code. Agreed. I wasn't implying that it would. The art of programming is problem solving, not coding. 100% agree.
Best regards, Brian On Aug 12, 2024 at 10:03 AM -0500, Brian Thompson <brianrobt@pm.me>, wrote:
“Anyone can code” is an educational statement so those who have potenial problem solving can learn it. Educational statement or not, it attracted a lot of mediocre people to the field. If software development is reduced to button-clicking and YAMLmanifests, then there's no incentive to learn that stuff.It’s not as if it would replace proper programming; I have yet to see a functional app built without code.
Best regards,
Brian Sent with Spark On Aug 12, 2024 at 2:27 AM -0500, tippfehlr <tippfehlr@tippfehlr.eu>, wrote:
Hi everyone,
for one, this was a thread about an unmanaged file in /usr/bin but has evolved first to discussing supply-chain attacks and then to incompetent programmers.
"Anyone can code" is one of the most ridiculous statements so far this millenia
“Anyone can code” is an educational statement so those who have potenial problem solving can learn it.
If software development is reduced to button-clicking and YAML manifests, then there's no incentive to learn that stuff.
It’s not as if it would replace proper programming; I have yet to see a functional app built without code.
The art of programming is problem solving, not coding.
-- tippfehlr
participants (8)
-
Andy Pieters
-
Brian Thompson
-
David C. Rankin
-
Edward Toroshchyn
-
Kusoneko
-
Maxxcan Fox
-
Patricio Martínez
-
tippfehlr