Simply by posting this, there's a big chance you'll say "Hey! Didn't you know there's already a well-known script that does this? Better." Or you'll say "Hey! That'll save me hundreds of seconds per year!"
The problem
Suppose you have a requirements.in file that is used, by pip-compile to generate the requirements.txt that you actually install in your Dockerfile or whatever server deployment. The requirements.in is meant to be the human-readable file and the requirements.txt is for the computers. You manually edit the version numbers in the requirements.in and then run pip-compile --generate-hashes requirements.in to generate a new requirements.txt. But the "first-class" packages in the requirements.in aren't the only packages that get installed. For example:
▶ cat requirements.in | rg '==' | wc -l
54
▶ cat requirements.txt | rg '==' | wc -l
102
In other words, in this particular example, there are 76 "second-class" packages that get installed. There might actually be more stuff installed that you didn't describe. That's why pip list | wc -l can be even higher. For example, you might have locally and manually done pip install ipython for a nicer interactive prompt.
The solution
The command pip list --outdated will list packages based on the requirements.txt not the requirements.in. To mitigate that, I wrote a quick Python CLI script that combines the output of pip list --outdated with the packages mentioned in requirements.in:
#!/usr/bin/env python
import subprocess
def main(*args):
if not args:
requirements_in = "requirements.in"
else:
requirements_in = args[0]
required = {}
with open(requirements_in) as f:
for line in f:
if "==" in line:
package, version = line.strip().split("==")
package = package.split("[")[0]
required[package] = version
res = subprocess.run(["pip", "list", "--outdated"], capture_output=True)
if res.returncode:
raise Exception(res.stderr)
lines = res.stdout.decode("utf-8").splitlines()
relevant = [line for line in lines if line.split()[0] in required]
longest_package_name = max([len(x.split()[0]) for x in relevant]) if relevant else 0
for line in relevant:
p, installed, possible, *_ = line.split()
if p in required:
print(
p.ljust(longest_package_name + 2),
"INSTALLED:",
installed.ljust(9),
"POSSIBLE:",
possible,
)
if __name__ == "__main__":
import sys
sys.exit(main(*sys.argv[1:]))
Installation
To install this, you can just download the script and run it in any directory that contains a requirements.in file.
Or you can install it like this:
curl -L https://gist.github.com/peterbe/099ad364657b70a04b1d65aa29087df7/raw/23fb1963b35a2559a8b24058a0a014893c4e7199/Pip-Outdated.py > ~/bin/Pip-Outdated.py chmod +x ~/bin/Pip-Outdated.py Pip-Outdated.py
Comments