9

Given requirements.txt and a virtualenv environment, what is the best way to check from a script whether requirements are met and possibly provide details in case of mismatch?

Pip changes it's internal API with major releases, so I seen advices not to use it's parse_requirements method.

There is a way of pkg_resources.require(dependencies), but then how to parse requirements file with all it's fanciness, like github links, etc.?

This should be something pretty simple, but can't find any pointers.

UPDATE: programmatic solution is needed.

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Roman Susi
  • 4,135
  • 2
  • 32
  • 47
  • If those requirements can be installed using pip, you could try to install them all and if the user already has them installed he will get a "Requirement already satisfied" message and will continue with the next requirements installation. You could also inside your code catch a missing dependency in a try-except statement and if missing ask the user if he wants to install it. – coder Aug 19 '16 at 20:39

3 Answers3

11

You can save your virtualenv's current installed packages with pip freeze to a file, say current.txt

pip freeze > current.txt

Then you can compare this to requirements.txt with difflib using a script like this:

import difflib

req = open('requirements.txt')
current = open('current.txt')

diff = difflib.ndiff(req.readlines(), current.readlines())
delta = ''.join([x for x in diff if x.startswith('-')])

print(delta)

This should display only the packages that are in 'requirements.txt' that aren't in 'current.txt'.

Community
  • 1
  • 1
mohrtw
  • 366
  • 2
  • 7
  • 1
    This is very noisy way: specifiers aren't compared properly and reported as changes (eg, >=8.2 and ==8.2 are reported, so it is no violation). Also, the check should not require human when there are no problems (I bolded that in the question and added update to make it even clearer). – Roman Susi Aug 19 '16 at 21:39
  • 8
    Also, hint for those who can be satisfied with the answer: use `pip freeze -r requirements.txt > current.txt` to preserve the order. It also reports some case mismatches. – Roman Susi Aug 19 '16 at 21:40
  • I use `pip freeze | diff requirements.txt -` (note the `-` at the end) to check in the command line, but you can put it in a shell script and it works to output a diff. – drkvogel Mar 07 '22 at 16:38
  • This has issues with transitive dependencies? Freeze will list them, but requirements.txt likely won't. Can try something like this `python -m pipdeptree | grep -v " "` which should give only highest level installed modules. Though better would be to install requirements.txt in a clean environment. – Annan Yearian Nov 09 '22 at 13:18
0

Got tired of the discrepancies between requirements.txt and the actually installed packages (e.g. when deploying to Heroku, I'd often get ModuleNotFoundError for forgetting to add a module to requirements.)

This helps:

Use compare-requirements (GitHub)
(you'll need to pip install pipdeptree to use it.)

It's then as simple as...

cmpreqs --pipdeptree

...to show you (in "Input 2") which modules are installed, but missing from requirements.txt.

You can then examine the list and see which ones should in fact be added to requirements.txt.

Fabien Snauwaert
  • 4,995
  • 5
  • 52
  • 70
0

No script option => If you use VS Code you can open both files in compare mode and check where they are different and where they are the same:

Step one: open files & folders side bar in VS Code and right click on file you would like to compare, click Select to Compare

enter image description here

Then you select second file, right click on Compare with Selected

enter image description here

And you will get something like this: black one are the same, red, green diffrent

enter image description here

Ilija
  • 11
  • 1
  • 4