How to lock requirements using pip-tools

Simple instructions for locking requirements in a python project using pip-tools

Locking Dependencies with pip-compile

Opinions vary on how one should make use of lock files, depending on whether the project is the main application, or the project is actually a library that is meant to be consumed by an application or another library.

Lock files are unquestionably useful if you build any application. Python has no concept of lock files, equally it can be argued python has no package dependency files at all and that's why we have many options outside the core python team like setup.py, Pipfile, and the most common requirements.txt as a pattern for Pip. This blog will show you how to lock requirements with command pip-compile (provided by pip-tools).

First, we need a clean virtual environment created with virtualenv.

Install the virtualenv package

The virtualenv package is required to create virtual environments. You can install it with pip:

pip install virtualenv

Create the virtual environment

To create a virtual environment, you must specify a path.

For example to create one in the local directory called ‘python3env’, type the following:

virtualenv python3env

Then You need to activate the python environment by running the following command:

Mac OS / Linux

source python3env/bin/activate

Windows

python3env\Scripts\activate

Any python commands you use will now work with your virtual environment.

Install pip-tools and lock requirements

Now you need to activate the virtual environment to install pip-tools

pip install pip-tools

Once the package has been installed, you need to create a requirements.in file. This file is where you define your project's top-level dependencies (similar to pipenv's Pipfile or pyproject.toml in poetry). A basic example might look something like this:

1Django==2.2.*
2psycopg2
3celery>4.4

To "lock" these dependencies, you can run:

pip-compile --output-file=requirements.txt requirements.in

This generates the standard requirements.txt file with all dependencies. Here's the file:

1#
2# This file is autogenerated by pip-compile
3# To update, run:
4#
5#    pip-compile requirements.in
6#
7--trusted-host pypi.python.org
8--trusted-host pypi.org
9--trusted-host files.pythonhosted.org
10
11amqp==5.0.6
12    # via kombu
13billiard==3.6.4.0
14    # via celery
15celery==5.1.2
16    # via -r requirements.in
17click-didyoumean==0.0.3
18    # via celery
19click-plugins==1.1.1
20    # via celery
21click-repl==0.2.0
22    # via celery
23click==7.1.2
24    # via
25    #   celery
26    #   click-didyoumean
27    #   click-plugins
28    #   click-repl
29django==2.2.24
30    # via -r requirements.in
31kombu==5.1.0
32    # via celery
33prompt-toolkit==3.0.19
34    # via click-repl
35psycopg2==2.9.1
36    # via -r requirements.in
37pytz==2021.1
38    # via
39    #   celery
40    #   django
41six==1.16.0
42    # via click-repl
43sqlparse==0.4.1
44    # via django
45vine==5.0.0
46    # via
47    #   amqp
48    #   celery
49    #   kombu
50wcwidth==0.2.5
51    # via prompt-toolkit
52
53# The following packages are considered to be unsafe in a requirements file:
54# setuptools

We didn't have pytz in our requirements.in, but it's included in requirements.txt because it is required by django (which the pip-compile is kind enough to output in the file).

MAKEFILE allows you to run make requirements.txt and it will be updated if and only if the requirements.in file has changed since requirements.txt was last generated.

Example:

1requirements.txt: requirements.in
2pip-compile --upgrade --output-file=$@ requirements.in

Installing the dependencies is as simple as:

pip install -r requirements.txt