Skip to content
Stéphane Robert
Français
Account: not signed in, sync available My learning
Select theme
Français
Lire en français

Publish and consume artifacts with Pulp

By updated

Improve this page

What would you like to report?

The issue opens pre-filled with this page title and URL.

Pulp logo

This guide walks through the complete lifecycle of an artifact in Pulp: uploading it, publishing it, exposing it, then retrieving it from a client machine. Intended audience: teams that want to host their own packages or cache a public repository. Prerequisites: a working Pulp instance and the pulp CLI configured (see Install Pulp). You will do two laps: a file repository that you fill by hand, then a PyPI proxy consumed by a real pip install.

A reminder of the model: in Pulp, an artifact goes through three objects, Repository → Publication → Distribution, fed either by direct upload or by synchronisation from a remote. Both modes are illustrated.

What you will learn

  • Create a repository and upload an artifact into it.
  • Publish then distribute that repository at a stable URL.
  • Retrieve the artifact with a simple HTTP request.
  • Build, upload (with --relative-path) and install your own Python package from another machine, checking the import.
  • Synchronise a subset of PyPI and install it with pip.

Upload an artifact

You start with the pulp_file plugin, the simplest one: it hosts any binary file. The scenario is an internal artifact, a deliverable or an archive, that you want to distribute to your machines.

  1. Create the repository:

    Terminal window
    pulp file repository create --name livrables

  2. Upload a file into this repository:

    Terminal window
    echo "Livrable interne v1.0" > mon-livrable-1.0.txt
    pulp file content upload \
      --repository livrables \
      --file mon-livrable-1.0.txt \
      --relative-path mon-livrable-1.0.txt

    Pulp computes the file's sha256, stores it only once through digest deduplication, and adds it to the repository. Every upload creates a new immutable version of the repository.

Publish and distribute

A filled repository cannot be browsed yet. You have to publish it (freeze a version in a servable format) then create a distribution (the access URL).

  1. Publish the latest version of the repository:

    Terminal window
    pulp file publication create --repository livrables

  2. Distribute this publication under a readable base path:

    Terminal window
    pulp file distribution create \
      --name livrables \
      --base-path livrables \
      --repository livrables

    By linking the distribution to the repository, and not to a frozen publication, it always serves the latest publication: a new upload followed by a publication is exposed automatically.

Retrieve the artifact

The artifact is now reachable at a predictable URL, under /pulp/content/<base-path>/. From any authorised machine:

Terminal window
curl https://pulp.exemple.com/pulp/content/livrables/mon-livrable-1.0.txt

The output shows Livrable interne v1.0. You have just gone through the repository → publication → distribution → retrieval cycle from end to end.

How do I host my own Python package in Pulp?

To host an in-house package, you build it into a Python distribution, upload it to a python repository with --relative-path, publish then distribute that repository, and install it from another machine with pip. The procedure differs from the file repository on one point: pip does not read a folder, it reads an index (an HTML page that lists packages by name), and Pulp generates it at publication time. The scenario below is a library called bibliotheque-equipe, which exists on no public repository.

Build the distribution

A Python distribution is the file that pip installs: a wheel (.whl, a ready-to-install archive) or an sdist (.tar.gz, the sources). All it takes is a short pyproject.toml and a module folder, here bibliotheque_equipe/, which contains an __init__.py:

[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"


[project]
name = "bibliotheque-equipe"
version = "1.0.0"
description = "Bibliothèque interne de l'équipe"
requires-python = ">=3.9"

The package name (with hyphens) and the name of the module to import (with underscores) are two different things: pip normalises them, but import only accepts the second. The module bibliotheque_equipe/__init__.py contains a function bonjour() here. The build is run from the project folder:

Terminal window
python3 -m build
ls dist/

The command writes two files to dist/, the wheel bibliotheque_equipe-1.0.0-py3-none-any.whl and the sdist bibliotheque_equipe-1.0.0.tar.gz. This build was replayed for this guide with build and setuptools, and does produce these two files. Without access to PyPI, python3 -m build fails because it downloads setuptools into an isolated environment: add --no-isolation after installing python3-build, python3-setuptools and python3-wheel.

Upload the package with --relative-path

You first create a python repository, then upload the wheel into it. The --relative-path option gives the exact name under which Pulp will serve the file:

Terminal window
pulp python repository create --name equipe
pulp python content create \
  --repository equipe \
  --file dist/bibliotheque_equipe-1.0.0-py3-none-any.whl \
  --relative-path bibliotheque_equipe-1.0.0-py3-none-any.whl

The create command uploads the file transparently, then adds it to the repository. It replaces pulp python content upload, which still exists but has been deprecated since version 0.38.0 of pulp-cli: the online help marks it [deprecated], and you will come across it in older tutorials. No removal date has been announced.

Remember the --relative-path trap. The option is mandatory: the CLI refuses the command if it is missing. However, Pulp does not check that its value matches the file. A value with a valid extension but the wrong name is accepted, and the package is published under that name, which pip does not find. Always give the exact file name of the wheel. This behaviour is read from the pulp_python code; it was not replayed on an instance for this guide. Every upload creates a new version of the repository, as with pulp_file.

To check the content, list the packages. The content list command has no --repository option (that one exists only for create), so here you filter on the package name; the --repository-version option exists to target a specific repository version:

Terminal window
pulp python content list --name bibliotheque-equipe

The name, the version 1.0.0 and the package file name appear. This filter does not say which repository the package is in: the repository version, read from pulp python repository list, must have gone from 0 to 1.

Publish and distribute the repository

The repository contains the package, but the index does not exist yet: the publication builds it. The distribution then gives it an address:

Terminal window
pulp python publication create --repository equipe
pulp python distribution create \
  --name equipe \
  --base-path equipe \
  --repository equipe

Check that the index offers the package. A published index first answers with a 302 redirect to /pypi/<base-path>/simple/, and only if the distribution has a publication: follow it with -L.

Terminal window
curl -sSL https://pulp.exemple.com/pypi/equipe/simple/

The page must contain the link bibliotheque-equipe. If it is empty or in error, the cause is either the absence of a publication, or a --relative-path value that does not match the exact name of the wheel.

Install from another machine and prove the import

The proof is made from a host other than the Pulp server, in a fresh virtual environment. Done from the server, it would succeed without anything being served; done in an environment that already has the package, it would prove nothing.

Terminal window
python3 -m venv /tmp/essai
/tmp/essai/bin/pip install --no-cache-dir \
  --index-url https://pulp.exemple.com/pypi/equipe/simple/ \
  bibliotheque-equipe
/tmp/essai/bin/python -c 'import bibliotheque_equipe; print(bibliotheque_equipe.bonjour())'

--no-cache-dir stops pip from reusing an already downloaded file. Since the package name exists on no public repository, a successful installation can only come from your Pulp. The last line counts as much as the installation: a package whose wheel is empty or badly built installs without error and cannot be imported. It must print bonjour depuis le depot interne. The client must trust Pulp's certificate, as in the next section.

The second lab on this page replays this procedure end to end on machines, with a different package name: you write the project yourself, without copying this one.

Cache PyPI and install from it with pip

Now for proxy mode, the one that replaces a public repository with a local cache. You synchronise a package from PyPI, then install it with pip pointing at Pulp. This is the use case that makes Pulp a direct competitor to the proxy repositories of Nexus and Artifactory.

  1. Describe the source with a remote, fetching only one targeted package and actually storing the files (immediate):

    Terminal window
    pulp python remote create \
      --name pypi-proxy \
      --url https://pypi.org/ \
      --includes '["shellingham"]' \
      --policy immediate

    The --includes filter limits the synchronisation; it is also a first control lever (see Securing Pulp). The immediate policy downloads and keeps the artifacts, which guarantees a complete cache and consistent digests.

  2. Create a Python repository and synchronise it:

    Terminal window
    pulp python repository create --name pypi-cache
    pulp python repository sync --name pypi-cache --remote pypi-proxy

  3. Distribute the repository: the Python plugin then exposes a PEP 503-compatible index:

    Terminal window
    pulp python distribution create \
      --name pypi-local \
      --base-path pypi-local \
      --repository pypi-cache

  4. Install the package from Pulp on a client machine:

    Terminal window
    pip install \
      --index-url https://pulp.exemple.com/pypi/pypi-local/simple/ \
      shellingham

    pip queries the index served by Pulp over HTTPS, downloads the wheel from the cache and installs it. Your machines no longer depend directly on PyPI. The client must trust Pulp's certificate: a public certificate, or a self-signed authority added to the trust store (see Pulp air-gapped). Do not resort to --trusted-host, which disables TLS verification. One case is an exception: on an HTTP lab with no TLS, pip only accepts an unencrypted origin for localhost, 127.0.0.0/8, ::1 and file. For a repository's IP address, it refuses the index unless you pass --trusted-host <host:port>, an option that lifts the origin check and the certificate check for that host only. Reserve it for the lab: in production, serve the index over HTTPS.

The same pattern for every format

This cycle is identical whatever the format. You simply replace file or python with the plugin you want:

  • pulp rpm for an internal YUM/DNF repository.
  • pulp deb for an APT mirror.
  • pulp ansible for a private collections repository.
  • pulp npm, pulp maven, pulp gem for the matching ecosystems.

This consistency of model is what makes Pulp productive: once the remote / repository / publication / distribution pattern is acquired, every format is driven in the same way.

Hands-on practice

The first lab puts a proxy repository in front of PyPI, then cuts the repository's network egress and requires an installation that still succeeds. It is the only proof that counts: a cache that serves while the source is reachable proves nothing, since the client could have bypassed it. The validation measures the cut instead of assuming it.

The second lab has you build a Python package, upload it to a hosted repository, and install it from another machine. Its name exists on no public repository: this is deliberate, and it is what makes the proof indisputable, because pip cannot have found it elsewhere. The validation also checks that the module can be imported: a package that installs without importing has proven nothing.

Key points

  • An artifact follows the repository → publication → distribution → retrieval cycle.
  • You feed a repository by direct upload (content upload for a file, content create for a Python package) or by synchronisation from a remote.
  • Linking a distribution to the repository always exposes the latest publication.
  • An in-house Python package is built (python3 -m build), uploaded with --relative-path (mandatory, and a wrong name is accepted but pip will not find it), published, distributed, then proven by an installation and an import from another machine.
  • A Python remote in proxy mode gives you a real pip install from Pulp, without depending on PyPI.
  • The same model applies to RPM, DEB, Ansible, npm, Maven: only the plugin name changes.

Knowledge check

Eight questions on the full cycle of an artifact: uploading with its relative path, publishing, distributing, and proving an install from another machine down to the import.

Knowledge check

Check what you have learned with this interactive quiz

8 questions
8 min
70% to pass

How it works

  • The timer starts when you click Start
  • Multiple choice, true/false and short answer questions
  • You can move between questions
  • Detailed results are shown at the end

Is this project useful to you?

The guides, courses, quizzes and labs stay free. If you use them regularly, you can contribute to their upkeep and development.

Become a supporting member (new tab) One-time support (new tab) Learn more

Follow the project on LinkedIn