Python Requirements¶
requirements.in
files contain the first-order requirements. It is split
into requirements required for production, and those used in development. If
you want to change requirements, change this file.
requirements.txt
includes the exact versions and package hashes for the
first-order requirements as well as the requirements-of-requirements. This is
the file used with pip install
to install packages in the Docker image.
pip-compile
, from pip-tools, generates and maintains this file using
requirements.in
.
requirements-docs.txt
has the requirements used on ReadTheDocs to build
the Ichnaea documentation on each merge to main. Since this is a
non-production environment, we neither pin nor hash the requirements.
Compiling requirements.txt¶
After making changes, the .in
files can be compiled to .txt
output
files by running:
make update-reqs
This will start a Docker container and run pip-compile
with the proper
options. Running in the container ensures that the correct dependencies are
chosen for the Docker environment, rather than your host environment.
There will be a warnings at the end of the process:
The generated requirements file may be rejected by pip install. See # WARNING lines for details.
This is expected. pip
and setuptools
are provided by the container, and
should not be pinned.
To apply the new requirements, rebuild your Docker image:
make build
Automated Updates¶
Dependabot opens PRs for updates around the first of the month.
It also opens PRs for security updates when they are available.
It seems to have some support for pip-tools
, but it may be
necessary to manually run make update-reqs
to
correctly regenerate the requirements.
paul-mclendahand is useful for packaging several PRs into a single PR, and avoiding the rebase / rebuild / test cycle when merging one Dependabot PR at a time.
Manually upgrading requirements.txt¶
To upgrade all the requirements, run make shell
to enter the Docker
environment, and run
CUSTOM_COMPILE_COMMAND="make update-reqs" pip-compile --generate-hashes --upgrade
To upgrade a single package, run this instead:
CUSTOM_COMPILE_COMMAND="make update-reqs" pip-compile --generate-hashes --upgrade-package <package-name>
You’ll need to exit the Docker environment and run make build
to recreate
the Docker image with your changes.
Other tools¶
pipdeptree displays the requirements tree, which can be useful to determine which package required an unknown package.