Welcome to dh-virtualenv’s documentation!¶
Contents:
What is dh-virtualenv¶
dh-virtualenv
is a tool that aims to combine Debian packaging with
self-contained virtualenv based Python deployments. To do this, the
package extends debhelper’s sequence by providing a new command in sequence,
dh_virtualenv
, which effectively replaces following commands
from the sequence:
dh_auto_install
dh_python2
dh_pycentral
dh_pysupport
In the sequence the dh_virtualenv
is inserted right after
dh_perl
.
Changelog¶
Following list contains most notable changes by version. For full list consult the git history of the project.
0.11¶
- Allow passing explicit filename for requirements.txt using –requirements option. Thanks to Eric Larson for implementing!
- Ensure that venv is configured before starting any daemons. Thanks to Chris Lamb for fixing this!
- Make sure fix_activate_path updates all activate scripts. Thanks to walrusVision for fixing this!
0.10¶
- Backwards incompatible Fix installation using the built-in virtual environment on 3.4. This might break installation on Python versions prior to 3.4 when using –builtin-venv flag. Thanks to Elonen for fixing!
- Honor
DH_VIRTUALENV_INSTALL_ROOT
in build system. Thanks to Ludwig Hähne for implementing! - Allow overriding virtualenv arguments by using the
DH_VIRTUALENV_ARGUMENTS
environment variable when using the build system. Thanks to Ludwig Hähne for implementing! - Add option to skip installation of the actual project. In other
words using
--skip-install
installs only the dependencies of the project found in requirements.txt. Thanks to Phillip O’Donnell for implementing! - Support custom installation suffix instead of the package name via
--install-suffix
. Thanks to Phillip O’Donnell for implementing!
0.9¶
- Support using system packages via a command line flag
--use-system-packages
. Thanks to Wes Mason for implementing this feature! - Introduce a new, experimental, more modular build system. See the Building packages with dh-virtualenv for documentation.
- Respect the
DEB_NO_CHECK
environment variable.
0.8¶
- Support for running triggers upon host interpreter update. This new feature makes it possible to upgrade the host Python interpreter and avoid breakage of all the virtualenvs installed with virtualenv. For usage, see the the Tutorial. Huge thanks to Jürgen Hermann for implementing this long wanted feature!
- Add support for the built-in
venv
module. Thanks to Petri Lehtinen! - Allow custom
pip
flags to be passed via the--extra-pip-args
flag. Thanks to @labeneator for the feature.
0.7¶
- Backwards incompatible Support running tests. This change
breaks builds that use distutils. For those cases a flag
--no-test
needs to be passed. - Add tutorial to documentation
- Don’t crash on debbuild parameters
-i
and-a
- Support custom source directory (debhelper’s flag
-D
)
0.6¶
First public release of dh-virtualenv
Tutorial¶
This tutorial will guide you through setting up your first project using dh-virtualenv. Having some knowledge on how Debian packages work won’t hurt, but it is not necessary a mandatory requirement. You also need some basic build tools, so it is recommended to install build-essential and devscripts packages.
Step 1: Install dh-virtualenv¶
In order to use it, you need to install the dh-virtualenv. If you run Debian Jessie (testing), Debian Sid (unstable) or Ubuntu 14.04 LTS (Trusty), you can install dh-virtualenv simply with apt-get:
sudo apt-get install dh-virtualenv
For more recent packages there is [an unofficial PPA maintained](https://launchpad.net/~spotify-jyrki/+archive/ubuntu/dh-virtualenv) by the author.
For other systems the only way is to build and install it yourself. Steps to do that, after you have cloned the repository are:
sudo apt-get install devscripts python-virtualenv git equivs # Install needed packages
git clone https://github.com/spotify/dh-virtualenv.git # Clone Git repository
cd dh-virtualenv # Move into the repository
sudo mk-build-deps -ri # This will install build dependencies
dpkg-buildpackage -us -uc -b # Build the *dh-virtualenv* package
# and finally, install it (you might have to solve some
# dependencies when doing this):
sudo dpkg -i ../dh-virtualenv_<version>.deb
Step 2: Setup the Debian packaging¶
Grab your favourite Python project you want to use dh-virtualenv
with and set it up. Only requirement is that your project has a
somewhat sane setup.py
and requirements listed in a
requirements.txt
file. Note however that defining any requirements
is not mandatory.
Next you need to define the Debian packaging for your software. To do
this, create a directory called debian
in the project root.
To be able to build a debian package, a few files are needed. First, we need to define the compatibility level of the project. For this, do:
echo "9" > debian/compat
The 9 is a magic number for latest compatibility level, but we don’t
need to worry about that. Next we need a file that tells what our
project is about, a file called control
. Enter a following
debian/control
file:
Source: my-awesome-python-software
Section: python
Priority: extra
Maintainer: Matt Maintainer <matt@example.com>
Build-Depends: debhelper (>= 9), python, dh-virtualenv (>= 0.8)
Standards-Version: 3.9.5
Package: my-awesome-python-software
Architecture: any
Pre-Depends: dpkg (>= 1.16.1), python2.7 | python2.6, ${misc:Pre-Depends}
Depends: ${python:Depends}, ${misc:Depends}
Description: really neat package!
second line can contain extra information about it.
The control
file is used to define the build dependencies, so if you
are building a package that requires for example lxml
, make sure
you define libxml2-dev
in Build-Depends etc.
Depends in the lower section is used to define run-time dependencies.
Following the example above, in case of lxml you would add libxml2
in to the Depends field.
To help keeping your installed virtualenv in sync with the host’s Python
interpreter in case of updates, create a file named
debian/«pkgname».triggers
, where «pkgname»
is what you
named your package in the control
file. It triggers a special script
whenever the Python binary changes; don’t worry, that script is provided
by dh-virtualenv
automatically.
# Register interest in Python interpreter changes (Python 2 for now); and
# don't make the Python package dependent on the virtualenv package
# processing (noawait)
interest-noawait /usr/bin/python2.6
interest-noawait /usr/bin/python2.7
# Also provide a symbolic trigger for all dh-virtualenv packages
interest dh-virtualenv-interpreter-update
Note that if you provide a custom postinst
script with your package,
then don’t forget to put the #DEBHELPER#
marker into it, else the trigger
script will be missing.
Next, we need a changelog file. It is basically a documentation of
changes in your package plus the source for version number for Debian
package builder. Here’s a short sample changelog to be entered in
debian/changelog
:
my-awesome-python-software (0.1-1) unstable; urgency=low
* Initial public release
-- Matt Maintainer <matt@example.com> Fri, 01 Nov 2013 17:00:00 +0200
You don’t need to create this file by hand, a handy tool called
dch
exists for entering new changelog entries.
Now, last bit is left, which is the debian/rules
file. This file
is basically a Makefile that Debian uses to build the package. Content
for that is fairly straightforward:
#!/usr/bin/make -f
%:
dh $@ --with python-virtualenv
And there we go, debianization of your new package is ready!
Step 3: Build your project¶
Now you can just build your project by running dpkg-buildpackage -us
-uc
. Enjoy your newly baked dh-virtualenv backed project! :)
Building packages with dh-virtualenv¶
Building packages with dh-virtualenv is relatively easy to start with but it also supports lot of customization to fit in your general needs.
By default, dh-virtualenv installs your packages under
/usr/share/python/<packagename>
. The package name is provided by
the debian/control
file.
To use an alternative install prefix, add a line like
export DH_VIRTUALENV_INSTALL_ROOT=</your/custom/install/dir>
to the top of your debian/rules
file. dh_virtualenv will use
DH_VIRTUALENV_INSTALL_ROOT instead of /usr/share/python
when it
constructs the install path.
To use an install suffix other than the package name, call the
dh_virtualenv
command using with the --install-suffix
command line option. See Advanced Usage for further information
on passing options.
Simple usecase¶
To signal debhelper to use dh-virtualenv for building your
package, you need to pass --with python-virtualenv
to debhelper
sequencer.
In a nutshell, the simplest debian/rules
file to build using
dh-virtualenv looks like this:
#!/usr/bin/make -f
%:
dh $@ --with python-virtualenv
However, the tool makes a few assumptions of your project’s structure:
- For installing requirements, you need to have a file called
requirements.txt
in the root directory of your project. The requirements file is not mandatory.- The project must have a
setup.py
file in the root of the project. Sequencer will runsetup.py install
to install the package inside the virtualenv.
After these are place, you can just build the package with your favorite tool!
Command line options¶
To change the default behavior the dh_virtualenv
command accepts a
few command line options:
-
-p
<package>
,
--package
<package>
¶ Act on the package named <package>
-
-N
<package>
,
--no-package
<package>
¶ Do not act on the specified package
-
-v
,
--verbose
¶
Turn on verbose mode. This has a few effects: it sets root logger level to
DEBUG
and passes verbose flag topip
when installing packages. This can also be provided using the standardDH_VERBOSE
environment variable.
-
--install-suffix
<suffix>
¶ Override virtualenv installation suffix. The suffix is appended to
/usr/share/python
, or theDH_VIRTUALENV_INSTALL_ROOT
environment variable if specified, to construct the installation path.
-
--extra-index-url
<url>
¶ Use extra index url <url> when running
pip
to install packages. This can be provided multiple times to pass multiple URLs topip
. This is useful if you for example have a private Python Package Index.
-
--preinstall
<package>
¶ Package to install before processing the requirements. This flag can be used to provide a package that is installed by
pip
before processing requirements file. This is handy if you need to install for example a custom setup script or other packages needed to parsesetup.py
. This flag can be provided multiple times to pass multiple packages for pre-install.
-
--pypi-url
<URL>
¶ Base URL of the PyPI server. This flag can be used to pass in a custom URL to a PyPI mirror. It’s useful if you for example have an internal mirror of the PyPI or you run a special instance that only exposes selected packages of PyPI. If this is not provided, the default will be whatever
pip
uses as default (usuallyhttp://pypi.python.org/simple
).
-
--extra-pip-arg
<PIP ARG>
¶ Extra parameters to pass to the pip executable. This is useful if you need to change the behaviour of pip during the packaging process. You can use this flag multiple times to pass in different pip flags. As an example passing in –extra-pip-arg “–no-compile” to the override_dh_virtualenv section of the debian/rules file will disable the generation of pyc files.
-
--requirements
<REQUIREMENTS FILE>
¶ Use a different requirements file when installing. Some packages such as pbr expect the
requirements.txt
file to be a simple list of requirements that can be copied verbatim into theinstall_requires
list. This command option allows specifying a differentrequirements.txt
file that may include pip specific flags such as-i
,-r-
and-e
.
-
--setuptools
¶
Use setuptools instead of distribute in the virtualenv
-
--no-test
¶
Skip running
python setup.py test
after dependencies and the package is installed. This is useful if the Python code is packaged using distutils and not setuptools.
-
--python
<path>
¶ Use a specific Python interpreter found in
path
as the interpreter for the virtualenv. Default is to use the system default, usually/usr/bin/python
.
-
--builtin-venv
¶
Enable the use of the build-in
venv
module, i.e. usepython -m venv
to create the virtualenv. For this to work, requires Python 3.4 or later to be used, e.g. by using the option--python /usr/bin/python3.4
. (Python 3.3 has thevenv
module, but virtualenvs created with Python 3.3 are not bootstrapped with setuptools or pip.)
-
-S
,
--use-system-packages
¶
Enable the use of system site-packages in the created virtualenv by passing the
--system-site-packages
flag tovirtualenv
.
-
--skip-install
¶
Skip running
pip install .
after dependencies have been installed. This will result in anything specified in setup.py being ignored. If this package is intended to install a virtualenv and a program that uses the supplied virtualenv, it is up to the user to ensure that if setup.py exists, any installation logic or dependencies contained therein are handled.This option is useful for web application deployments where the package is expected contain the virtual environment to support an application which itself may be installed via some other means – typically, by the packages
./debian/<packagename>.install
file, possibly into a directory structure unrelated to the location of the virtual environment.
Advanced usage¶
To provide command line options to dh_virtualenv
sequence the
override mechanism of the debhelper is the best tool.
Following debian/rules
will provide http://example.com as
additional Python Package Index URI:
#!/usr/bin/make -f
%:
dh $@ --with python-virtualenv
override_dh_virtualenv:
dh_virtualenv --extra-index-url http://example.com
Experimental buildsystem support¶
Important: Following chapters describe a completely experimental functionality of dh-virtualenv.
Starting with version 0.9 of dh-virtualenv, there is a buildsystem alternative.
The main difference in use is that instead of the --with python-virtualenv
option, --buildsystem=dh_virtualenv
is passed to debhelper. The debian rules
file should look like this:
#!/usr/bin/make -f
%:
dh $@ --buildsystem=dh_virtualenv
Using the buildsystem instead of the part of the sequence (in other
words, instead of the --with python-virtualenv
) one can get more
flexibility into the build process.
Flexibility comes from the fact that buildsystem will have individual
steps for configure, build, test and install and those can be
overridden by adding override_dh_auto_<STEP>
target into the
debian/rules
file. For example:
#!/usr/bin/make -f
%:
dh $@ --buildsystem=dh_virtualenv
override_dh_auto_test:
py.test test/
In addition the separation of build and install steps makes it
possible to use debian/install
files to include built files into
the Debian package. This is not possible with the sequencer addition.
The build system honors the DH_VIRTUALENV_INSTALL_ROOT
environment
variable. Arguments can be passed to virtualenv by setting
DH_VIRTUALENV_ARGUMENTS
. For example:
export DH_VIRTUALENV_ARGUMENTS=--no-site-packages --always-copy
The default is to create the virtual environment with --no-site-packages
.
Known incompabilities of the buildsystem¶
This section defines the known incompabilities with the sequencer approach. There are no guarantees that these all get addressed, but most of them, if not all, probably will.
- No custom Python interpreter supported
Pyvenv
of Python 3.x is not supported- No custom arguments outside requirements.txt can be passed to
pip