Technical Tutorials

For years, Python developers relied on a fragmented ecosystem of packaging tools, often juggling between setup.py, setup.cfg, and various build backends. This lack of standardization led to confusion and technical debt. However, with the adoption of PEP 517 and PEP 518, the landscape has stabilized. Today, Python packaging is defined by the pyproject.toml file, a standardized configuration format that simplifies both building and distributing packages.

This guide explores the modern best practices for creating, testing, and distributing Python packages, targeting developers who want to ensure their libraries are robust, reproducible, and easy for the community to install.

The Core Configuration: pyproject.toml

The modern Python packaging workflow revolves entirely around the pyproject.toml file. This single source of truth replaces the need for multiple configuration files. It defines the build system and project metadata, ensuring that any tool following PEP 517 can build your project consistently.

A typical pyproject.toml for a modern project looks like this:

[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "my-awesome-library"
version = "0.1.0"
description = "A sample library demonstrating modern Python packaging"
readme = "README.md"
license = {text = "MIT"}
authors = [
    {name = "Jane Doe", email = "jane@example.com"}
]
requires-python = ">=3.8"
dependencies = [
    "requests>=2.28.0",
    "click>=8.0"
]

[project.optional-dependencies]
dev = [
    "pytest>=7.0",
    "black"
]

[project.urls]
Homepage = "https://github.com/janedoe/my-awesome-library"
Documentation = "https://my-awesome-library.readthedocs.io"

Note the use of static metadata in the [project] section. Unlike dynamic metadata in setup.py, static metadata allows faster parsing by tools like PyPI and ensures reproducibility across different environments. Dependencies are strictly versioned to avoid breaking changes in upstream libraries.

Building and Distributing Artifacts

Once your configuration is set, the next step is generating the distribution artifacts. The industry standard has shifted from source distributions (.tar.gz) and legacy wheels (.egg) to modern wheels (.whl). Modern wheels include pre-built binaries and metadata, significantly speeding up installation times for end-users.

To build your project, use the build package, which is the recommended tool for creating source and wheel distributions. First, install the build tool:

python -m pip install build

Then, run the build command:

python -m build

This command creates a dist/ directory containing both the source distribution (sdist) and the wheel. The sdist includes all source files and allows users to build the package from source on their systems, while the wheel provides a fast, installable format for most users.

Uploading to PyPI

With your artifacts ready, the final step is distribution. The Trusted publishing feature introduced in twine version 4.0+ simplifies the upload process by removing the need for long-lived API tokens. Instead, it uses OIDC (OpenID Connect) for authentication, which is more secure and convenient for CI/CD pipelines.

Install twine:

python -m pip install twine

Upload your distributions:

python -m twine upload dist/*

Always test your uploads on TestPyPI first to catch errors before hitting the main repository. You can do this by specifying the --repository testpypi flag and configuring your .pypirc or using environment variables for credentials.

Best Practices for Long-Term Maintenance

Packaging is not a one-time task. To maintain a healthy project, consider the following practices:

Version Management: Use semantic versioning (SemVer) to communicate breaking changes.
Dependency Pinning: Pin dependencies in your pyproject.toml to ensure consistent builds, but use flexible ranges (e.g., >=2.0) to allow for security patches.
CI/CD Integration: Automate testing and publishing using GitHub Actions or GitLab CI to ensure every release is tested across multiple Python versions.
Documentation: Keep your README.md and documentation updated, as these are the first things users see on PyPI.

Conclusion

Python packaging has matured into a robust, standardized process. By leveraging pyproject.toml, modern build tools, and secure distribution methods, developers can focus on writing code rather than troubleshooting installation issues. Embracing these modern practices ensures that your contributions to the Python ecosystem are accessible, reliable, and maintainable for years to come.