How to lock requirements using pip-tools

Simple instructions for locking requirements in a python project using pip-tools

Locking Dependencies with pip-compile

Opinions vary on how one should make use of lock files, depending on whether the project is the main application, or the project is actually a library that is meant to be consumed by an application or another library.

Lock files are unquestionably useful if you build any application. Python has no concept of lock files, equally it can be argued python has no package dependency files at all and that's why we have many options outside the core python team like setup.py, Pipfile, and the most common requirements.txt as a pattern for Pip. This blog will show you how to lock requirements with command pip-compile (provided by pip-tools).

First, we need a clean virtual environment created with virtualenv.

Install the virtualenv package

The virtualenv package is required to create virtual environments. You can install it with pip:

pip install virtualenv

Create the virtual environment

To create a virtual environment, you must specify a path.

For example to create one in the local directory called ‘python3env’, type the following:

virtualenv python3env

Then You need to activate the python environment by running the following command:

Mac OS / Linux

source python3env/bin/activate

Windows

python3env\Scripts\activate

Any python commands you use will now work with your virtual environment.

Install pip-tools and lock requirements

Now you need to activate the virtual environment to install pip-tools

pip install pip-tools

Once the package has been installed, you need to create a requirements.in file. This file is where you define your project's top-level dependencies (similar to pipenv's Pipfile or pyproject.toml in poetry). A basic example might look something like this:

1Django==5.0.*
2psycopg2
3celery>5.1

To "lock" these dependencies, you can run:

pip-compile --output-file=requirements.txt requirements.in

This generates the standard requirements.txt file with all dependencies. Here's the file:

1#
2# This file is autogenerated by pip-compile with Python 3.10
3# by the following command:
4#
5#    pip-compile
6#
7
8amqp==5.2.0
9    # via kombu
10asgiref==3.7.2
11    # via django
12billiard==4.2.0
13    # via celery
14celery==5.3.6
15    # via -r requirements.in
16click==8.1.7
17    # via
18    #   celery
19    #   click-didyoumean
20    #   click-plugins
21    #   click-repl
22click-didyoumean==0.3.0
23    # via celery
24click-plugins==1.1.1
25    # via celery
26click-repl==0.3.0
27    # via celery
28django==5.0
29    # via -r requirements.in
30kombu==5.3.5
31    # via celery
32prompt-toolkit==3.0.43
33    # via click-repl
34psycopg2==2.9.9
35    # via -r requirements.in
36python-dateutil==2.9.0.post0
37    # via celery
38six==1.16.0
39    # via python-dateutil
40sqlparse==0.4.4
41    # via django
42typing-extensions==4.10.0
43    # via asgiref
44tzdata==2024.1
45    # via celery
46vine==5.1.0
47    # via
48    #   amqp
49    #   celery
50    #   kombu
51wcwidth==0.2.13
52    # via prompt-toolkit
53

We didn't have pytz in our requirements.in, but it's included in requirements.txt because it is required by django (which the pip-compile is kind enough to output in the file).

MAKEFILE allows you to run make requirements.txt and it will be updated if and only if the requirements.in file has changed since requirements.txt was last generated.

Example:

1requirements.txt: requirements.in
2pip-compile --upgrade --output-file=$@ requirements.in

Installing the dependencies is as simple as:

pip install -r requirements.txt