pip ignores dependency_links in setup.py

I have dependency_links in my setup.py:

...
dependency_links = ['http://github.com/robot-republic/python-s3/tarball/master.tar.gz#egg=python-s3'],
...

But it doesn’t work. However install_requires works fine.
Maybe there are another method to set up git repo as required for setup.py?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

This answer should help. In a nutshell, you need to specify the version (or “dev”) for the #egg=python-s3 so it looks like #egg=python-s3-1.0.0.

Updates based on @Cerin’s comment:

• Pip 1.5.x has a flag to enable dependency-links processing: --process-dependency-links. I haven’t tested it because I agree with the point below.
• This discussion seems to indicate that using dependency-links for pip is a bad practice. Although this feature was enlisted for deprecation, it’s not anymore. There’s a valid use case for private packages.

Method 2

since pip version 18.1 PEP 508 URL is supported. That means you don’t need the deprecated dependency_links anymore.
You write the dependency directly in the install_requires list instead.
The example from @Chad looks like this:

setup(
    name='yourpackage',
    version='1.7.5',
    packages=[],
    url='',
    license='',
    author='',
    author_email='',
    description='',
    install_requires=[
        'somepackage==1.2.0',
        'repo @ https://github.com/user/archive/master.zip#egg=repo-1.0.0',
        'anotherpackage==4.2.1'
    ],
)

To install your package you can simply write:

pip install yourpackage

(without –process-dependency-links)

Method 3

I realize this is an old question, but, just in case you find yourself here like I did, this is what worked for me.

I’ve got a package on GitHub (not registered with pypi) that relies on other GitHub (non-pypi) packages. I spent an inordinate amount of time trying to figure out how to get pip to handle this correctly. I will include what I did to fix it here.

Putting dependencies in a requirements.txt file is the preferred method of listing dependencies. However, you also need to populate install_requires in setup. It was at this stage that I ran into a roadblock with pip not wanting to install dependencies from GitHub.

Most places, including answers to this question, tell you to populate the dependency_links section of setup. However, you also need to populate the install_requires field with the name of the package referenced in dependency_links.

For example, if your requirements.txt contains the following.

somepackage==1.2.0
https://github.com/user/repo/tarball/master#egg=repo-1.0.0
anotherpackage==4.2.1

Then, your setup call should look like this:

setup(
    name='yourpackage',
    version='1.7.5',
    packages=[],
    url='',
    license='',
    author='',
    author_email='',
    description='',
    install_requires=[
        'somepackage==1.2.0',
        'repo==1.0.0',
        'anotherpackage==4.2.1'
    ],
    dependency_links=[
        'https://github.com/user/repo/tarball/master#egg=repo-1.0.0'
    ]
)

Ok, so now we’ve got our package configured; installing it is the next task. This is where I spent a lot of time. I could not figure out why specifying dependency_links apparently did nothing. The trick is that in some cases, you need to set the allow-all-external (can be more specific) flag for pip. For example:

pip install git+https://github.com/user/anotherrepo.git
--process-dependency-links --allow-all-external

You’re done and it works!

DISCLAIMER: dependency_links and the flags process-dependency-links and allow-all-external are deprecated, so they will be removed soon. In the time I spent, I could not locate a better, prefered method and still have pip function properly.

Method 4

A couple of notes on some issues I found, in particular for installing from private repos.

Installing from pip & setuptools have some subtle differences; but this way should work for both.

from setuptools import setup
import os
# get deploy key from https://help.github.com/articles/git-automation-with-oauth-tokens/
github_token = os.environ['GITHUB_TOKEN']

setup(
    # ...
    install_requires='package',
    dependency_links = [
    'git+https://{github_token}@github.com/user/{package}.git/@{version}#egg={package}-0'
        .format(github_token=github_token, package=package, version=master)
        ]

A couple of notes here:

• For private repos, you need to authenticate with GitHub; the simplest way I found is to create an oauth token, drop that into your environment, and then include it with the URL
• You need to include some version number (here is 0) at the end of the link, even if there’s no package on PyPI. This has to be a actual number, not a word.
• You need to preface with git+ to tell setuptools it’s to clone the repo, rather than pointing at a zip / tarball
• version can be a branch, a tag, or a commit hash
• You need to supply --process-dependency-links if installing from pip

Method 5

First upgrade your pip version as this is a new syntax.

pip install pip --upgrade

Then do the following:

install_requires=[
    'bleualign-git @ https://github.com/rsennrich/Bleualign/archive/<commit-hash or branch-name>.zip#egg=bleualign-git-1.0.0'
]

• Version 1.0.0 and name “bluealign-git” are randomly chosen.
• The version number is necessary.
• We advisedly used the name “bleualign-git” to distinguish it from the main repository version.

Hope this helps.

Comments:

Good answer (upvoted). Also want to mention that the actual result would likely depend on the platform and/or pip version, though. I’ve seen when it works, or seemingly works but did not really pull the dependency from the specified link, or simply being rejected. So I would probably just use this as a short time workaround only. Stick with the mainstream way whenever possible.

Method 6

None of the above solutions worked for me in their respective explicit forms. I am adding this as another solution for certain specific cases. OP has a tarball in their remote repo, but I ended up here for a related case: only source code in the remote repo.

I have a dependency that is a remote private repository on Azure DevOps. I would like to use pip to install a local python package and also automatically install the remote dependency. The remote repository is a python package with a setup.py and some source code. The repo doesn’t have build artifacts like eggs/wheels/tarballs, just .py files. I am able to manually install the remote dependency with pip install git+https://..., hence I suspected correctly that the remote repository doesn’t need to contain eggs/wheels/tarballs.

Here is the relevant line of my setup.py file

setup.py

...

install_requires=["packagename @ git+https://dev.azure.com/.../_git/packagename"]

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0