• History of Python package managers
• Problems
• How it done in other languages
• PyPA initiative
• poetry – Python pacakging and depndency management made easy
• poetry Pros & Cons
• What’s next for poetry?

$ curl -O package.tar.gz
$ tar -xzf package.tar.gz --directory=lib/package
$ PYTHONPATH=lib python app.py

import sys
sys.path.append('../lib')

$ curl -O package.tar.gz
$ tar -xzf package.tar.gz
$ python package/setup.py

Part of setuptools, now is deprecated

$ curl -O package.tar.gz
$ easy_install package.tar.gz

Is able to install packages from PyPI:

$ easy_install package

pip install packages

$ pip install package
$ pip install -r requirements.txt

• Active development started at 2008
• 1.0 version released 8 years ago, 4th of April, 2011
• Current release: 19.0.3

requirements.txt

Django

$ pip install -r requirements.txt
...
Successfully installed Django-2.1.7 pytz-2018.9

requirements-dev.txt

awsebcli==3.14.13

$ pip install -r requirements-dev.txt
...
Successfully installed PyYAML-3.13 awsebcli-3.14.13 blessed-1.15.0
botocore-1.12.120 cached-property-1.5.1 cement-2.8.2 certifi-2019.3.9
chardet-3.0.4 colorama-0.3.9 docker-3.7.1 docker-compose-1.23.2
docker-pycreds-0.4.0 dockerpty-0.4.1 docopt-0.6.2 docutils-0.14 future-0.16.0
idna-2.7 jmespath-0.9.4 jsonschema-2.6.0 pathspec-0.5.9 python-dateutil-2.8.0
requests-2.20.1 semantic-version-2.5.0 six-1.11.0 termcolor-1.1.0
texttable-0.9.1 urllib3-1.24.1 wcwidth-0.1.7 websocket-client-0.56.0

$ python -m venv .venv
$ .venv/bin/activate
(.venv)$ pip install ...

• Few of the other problems…
• Hash check not enabled by default
• Adding requirements via pip add?
• Updating requirements via pip update?

• cargo new to create new package
• cargo install CRATE to install external crate (package)
• cargo build to build the package
• And…
• cargo run
• cargo test
• cargo doc
• cargo publish
• Behind the scene cargo stores everything in Cargo.toml and Cargo.lock

• Default package manager for JavaScript
• Before npm@5 did not have lock file
• Had problem with dependency resolving, which partially fixed by pinning deps

• Package manager for JavaScript from Facebook devs
• Introduced lock file: yarn.lock
• Do not care about node_modules, while managing deps
• Cause of yarn, npm got package-lock.json
• Cause of yarn, npm@6 fixed many issues, but not all

• Wheels
• Environment markers (PEP 508)
• Build dependencies (PEP 518)

• Attempt to make something new
• Pipfile to store package metadata
• Pipfile.lock to store dependencies versions
• Many additional commands

• Official site
• Fix issues with pip & pipenv
• Have tools for packaging Python libraries
• Better dependency resolver management
• Better virtual environment management
• Originally inspired by composer & cargo

Install with:

curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python

Upgrade with:

poetry self:update

Create new package at path

shep:Projects playpauseandstop$ poetry new test-project
Created package test-project in test-project
shep:Projects playpauseandstop$ exa -l test-project/
.rw-r--r-- 297 playpauseandstop 23 Mar 12:20 pyproject.toml
.rw-r--r--   0 playpauseandstop 23 Mar 12:20 README.rst
drwxr-xr-x   - playpauseandstop 23 Mar 12:20 test_project
drwxr-xr-x   - playpauseandstop 23 Mar 12:20 tests

Add pyproject.toml to the existing project

shep:Projects playpauseandstop$ mkdir test-project
shep:Projects playpauseandstop$ cd test-project/
shep:test-project playpauseandstop$ poetry init

This command will guide you through creating your pyproject.toml config.

...

Would you like to define your dependencies (require) interactively? (yes/no) [yes] no
Would you like to define your dev dependencies (require-dev) interactively (yes/no) [yes] no

Generated file

...

Do you confirm generation? (yes/no) [yes] yes

shep:test-project playpauseandstop$ exa -l .
.rw-r--r-- 297 playpauseandstop 23 Mar 12:30 pyproject.toml

Add dependency or dev dependency to the project

shep:test-project playpauseandstop$ poetry add aiohttp
Creating virtualenv test-project-py3.7 in /Users/playpauseandstop/Projects/test-project/.venv
Using version ^3.5 for aiohttp

...

Writing lock file

  - Installing idna (2.8)
  - Installing multidict (4.5.2)
  - Installing async-timeout (3.0.1)
  - Installing attrs (19.1.0)
  - Installing chardet (3.0.4)
  - Installing yarl (1.3.0)
  - Installing aiohttp (3.5.4)
shep:test-project playpauseandstop$ exa -l
.rw-r--r-- 6.4k playpauseandstop 23 Mar 12:33 poetry.lock
.rw-r--r--  314 playpauseandstop 23 Mar 12:32 pyproject.toml

Dependency resolver included

shep:test-project playpauseandstop$ poetry add async-timeout==1.1.0

Updating dependencies
Resolving dependencies... (15.1s)

[SolverProblemError]
Because no versions of aiohttp match >3.5,<3.5.1 || >3.5.1,<3.5.2 || >3.5.2,<3.5.3 || >3.5.3,<3.5.4 || >3.5.4,<4.0
 and aiohttp (3.5.0) depends on async-timeout (>=3.0,<4.0), aiohttp (>=3.5,<3.5.1
 || >3.5.1,<3.5.2 || >3.5.2,<3.5.3 || >3.5.3,<3.5.4 || >3.5.4,<4.0) requires
 async-timeout (>=3.0,<4.0).
And because aiohttp (3.5.1) depends on async-timeout (>=3.0,<4.0)
 and aiohttp (3.5.2) depends on async-timeout (>=3.0,<4.0), aiohttp (>=3.5,<3.5.3
 || >3.5.3,<3.5.4 || >3.5.4,<4.0) requires async-timeout (>=3.0,<4.0).
And because aiohttp (3.5.3) depends on async-timeout (>=3.0,<4.0)
 and aiohttp (3.5.4) depends on async-timeout (>=3.0,<4.0), aiohttp (>=3.5,<4.0) requires async-timeout (>=3.0,<4.0).
So, because test-project depends on both aiohttp (^3.5) and async-timeout (=1.1.0), version solving failed.

Remove dependency and all its dependencies

shep:test-project playpauseandstop$ poetry remove aiohttp
Updating dependencies
Resolving dependencies... (0.1s)

Package operations: 0 installs, 0 updates, 7 removals

Writing lock file

  - Removing aiohttp (3.5.4)
  - Removing async-timeout (3.0.1)
  - Removing attrs (19.1.0)
  - Removing chardet (3.0.4)
  - Removing idna (2.8)
  - Removing multidict (4.5.2)
  - Removing yarl (1.3.0)

Update dependencies

shep:project playpauseandstop$ poetry show -o
decorator           4.3.2   4.4.0   Better living through Python with decorators
ipython             7.3.0   7.4.0   IPython: Productive Interactive Computing
uvloop              0.12.1  0.12.2  Fast implementation of asyncio event loop on top of libuv
shep:project playpauseandstop$ poetry update
Updating dependencies
Resolving dependencies... (3.2s)

Package operations: 0 installs, 3 updates, 0 removals

Writing lock file

  - Updating decorator (4.3.2 -> 4.4.0)
  - Updating ipython (7.3.0 -> 7.4.0)
  - Updating uvloop (0.12.1 -> 0.12.2)

Run command inside virtual environment

shep:test-project playpauseandstop$ $ poetry run python -m aiohttp.web app:create_app
DEBUG:asyncio:Using selector: KqueueSelector
======== Running on http://localhost:8080 ========
(Press CTRL+C to quit)

• poetry build to build package to wheel or sdist
• poetry publish to publish package to PyPI or private repository
• poetry shell to spawns a shell within a virtual environment

• Using dependency lock is a real thing
• It is slower then pip or pipenv, but it resolve dependencies
• You will not install a wrong dependency
• poetry update also cares about resolving dependencies

• poetry update rules

dotpoetry

#!/bin/bash
#
# Run poetry as usual, but before load values from dotenv file(s).
#

set -e

dotenv () {
    filename=$1
    if [ -f "$filename" ]; then
        set -a
        source "$filename"
        set +a
    fi
}

LEVEL=${LEVEL:-dev}

dotenv .env
dotenv ".${LEVEL}.env"
dotenv ".env.${LEVEL}"

$HOME/.pyenv/versions/`cat .python-version`/bin/python $HOME/.poetry/bin/poetry "$@"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

You don’t need twine to publish packages

• poetry build rules

You need to have setup.py & docs/requirements.txt

update-setup-py: .install
  rm -rf dist/
  $(POETRY) build
  tar -xzf dist/$(PROJECT)-*.tar.gz --directory dist/
  cp dist/$(PROJECT)-*/setup.py .
  rm -rf dist/

You need to generate requirements.txt

generate-requirements:
  -rm -rf .venv/ .install
  $(POETRY) install --no-dev
  .venv/bin/pip uninstall -y $(PROJECT)
  .venv/bin/pip freeze > requirements.txt
  rm -rf .venv/

poetry not available at python official image

FROM michalmazurek/python-poetry

poetry not yet available on GitHub deps page