How to run a command inside a virtualenv using systemd

I believe this should be simple but I can’t get it to work properly.

These are the commands I can run on command line:

cd /home/debian/ap

# Start a virtualenv
source venv-ap/bin/activate

# This needs to happen inside the virtualenv and takes ~20 seconds
crossbar start

# Outside the virtualenv, perhaps in a different command line window
python3 /home/debian/myscript.py

These commands have to be done in this order. Due to the virtualenv, the non-executable for crossbar, and the separate python script afterwards, I haven’t been able to figure out the best way to get this to work. My current work-in-progress:

[Unit]
Description=Start CB
After=network.target

[Service]
Type=simple
User=debian
ExecStartPre=source /home/debian/ap/venv-ap/bin/activate
ExecStart=cd /home/debian/ap/ && crossbar start
Restart=always

[Install]
WantedBy=multi-user.target

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

This doesn’t work because source is a shell command, so systemd’s ExecStart= or ExecStartPre= won’t understand them directly… (BTW, the same is true for cd and the &&.)

You could achieve that by running a shell explicitly and running all your commands together there:

ExecStart=/bin/sh -c 'cd /home/debian/ap/ && source venv-ap/bin/activate && crossbar start'

But a better approach is, instead of sourcing the “activate” script, to use the python executable in the bin/ of your virtualenv directly.

If you look at virtualenv’s usage document, you’ll notice it says:

ENV/bin is created, where executables live – noticeably a new python. Thus running a script with #! /path/to/ENV/bin/python would run that script under this virtualenv’s python.

In other words, assuming crossbar is the Python script you want to run that requires the venv-ap virtualenv, simply begin crossbar with:

#!/home/debian/ap/venv-ap/bin/python

And it will automatically use the virtualenv whenever invoked.

Also possible, invoking the Python interpreter from the virtualenv directly, with:

ExecStart=/home/debian/ap/venv-ap/bin/python /path/to/crossbar start

(Also, regarding running in a specific directory, setting WorkingDirectory=/home/debian/ap is better than using a cd command. You don’t need a shell that way, and systemd can do better error handling for you.)

Method 2

I want to iterate on @filbranden’s answer second part

But a better approach is, instead of sourcing the “activate” script, to use the python executable in the bin/ of your virtualenv directly.

If you look at virtualenv’s usage document, you’ll notice it says:

ENV/bin is created, where executables live – noticeably a new python. Thus running a script with #! /path/to/ENV/bin/python would run that script under this virtualenv’s python.

Don’t run stuff in venv without activating it! To be fair, the documentation is also misleading. Activating the venv makes you able to separate the working directory from the python (and venv-ed packages’) path which is a must quite often: you don’t want to run/place something in the venv bin directly, but you installed some script’s dependencies to it.

Consider the following (which I bet is one of the most common) scenario:

• you need to run a script from a git repo with but it has its dependencies incompatible with your python packages
• you created a virtualenv and pip installed the dependencies there
• you cloned the repo somewhere, but the script doesn’t lies in venv’s bin.

If you even want to work on the app from the git repo, you want its folder to be pristine, except your meaningful code modifications. You wouldn’t want to change shebang. You can put the whole venv into a subdir in the cloned repo and gitignore it to keep everything organized, but then:

• you would set the systemd working dir to the cloned folder
• you would run with script with the subfolder’s python full path
• script will not find the pip-ed deps, as they are not in the working folder, neither in the path
  The problem remains if you reverse the folder hierarchy (clone the repo to the venv’s subfolder).

Also, maybe you don’t want to run python directly, but e.g.: gunicorn, and you placed the webapp to /srv while the venv is in /var/lib.

Not activating a venv is not a good habit to pick up, I wonder how that moronic paragraph made it into the official venv docs. Yeah, activating it is not needed in some cases except in quite a lot of real-world scenarios! – I argue.

ExecStartPre won’t work for sourcing venv anyway, as that is executed in a different shell than ExecStart‘s command, so environment variables won’t be right in the latter.

I’d advise setting the venv PATH with Environment

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0