Oh my py

😔

Type hints
in Python

And how to use them

Brief History aka PEP-484

  • Designed on top of PEP-3107 (Function Annotations)
  • Created on September 2014
  • Became part of standard library in Python 3.5
  • Endorsed by Guido van Rossum

Basics

def hello(text: str) -> str:
    return 'Hello, {0}'.format(text)


def sum(a: int, b: int) -> int:
    return a + b

Basics

from decimal import Decimal, ROUND_UP


PRECISION = Decimal('.01')


def quantize_payment(hours: float, rate: Decimal) -> Decimal:
    return (Decimal(hours) * rate).quantize(PRECISION, ROUND_UP)

str, bytes, Text, AnyStr

from typing import AnyStr, Text


def response(body: bytes=None, text: str=None) -> bytes:
    ...


def response(content: AnyStr=None) -> bytes:
    ...


def render_template(path: Text) -> str:
    ...

list, List, Tuple, Set, Sequence

from typing import List, Sequence, Set, Tuple


def avg(data: list) -> float:
    return sum(data, 0.) / len(data)


def avg(data: List[float]) -> float:
    return sum(data, 0.) / len(data)


def split_name(value: str) -> Tuple[str, str]:
    return tuple(value.split(' ', 1))


def avg_tuple(data: Tuple[float, ...]) -> float:
    return sum(data, 0.) / len(data)


def unique(data: List[str]) -> Set[str]:
    return set(data)


def avg(data: Sequence[float]) -> float:
    return sum(data, 0.) / len(data)

dict, Dict, Mapping

from typing import Dict, Mapping


def read_data(slug: str) -> dict:
    ...


def read_data(slug: str) -> Dict[str, str]:
    ...


def process_data(data: Mapping[str, str]) -> None:
    for key, value in data.items():
        ...

Any, Union, Optional

from typing import Any, Optional, Sequence, Union


def read_data(slug: str) -> Dict[str, Union[int, str]]:
    ...


def read_data(slug: str) -> Dict[str, Any]:
    ...


def avg(data: Sequence[float]) -> Optional[float]:
    if len(data):
        return sum(data, 0.) / len(data)
    return None

Callable

from random import randint
from typing import Callable


def random_int_factory(min: int=0, max: int=100) -> Callable[[], int]:
    def random_int() -> int:
        return randint(min, max)
    return random_int

Aliases

from aiohttp import web


View = Callable[[web.Request], web.Response]


def middleware() -> Callable[[web.Application, View], Awaitable[View]]:
  async def factory(app: web.Application, handler: View) -> View:
      async def middleware(request: web.Request) -> web.Response:
          ...

Classes

class Schema(object):

    name = None  # type: str

    def __init__(self, name: str=None) -> None:
        self.name = name or self.name

    def clone(self) -> 'Schema':
        return Schema(self.name)

    def load(self) -> bool:
        with open(SCHEMA_PATH / '{0}.json'.format(self.name)) as handler:
            return json.loads(handler.read())

    def validate(self, data: Dict[str, Any]) -> Dict[str, Any]:
        json_schema = self.load()
        return fastjsonschema.compile(json_schema).validate(data)

Types

from typing import NewType


UserID = NewType('UserID', int)


def fetch_user(user_id: UserID) -> Any:
    ...


fetch_user(UserID(1))  # OK
fetch_user(1)  # Type check will fail

Inline Type Hints

WEEKS = defaultdict(lambda: {
    False: list(range(8)),
    True: list(range(16)),
})  # type: Dict[str, Dict[bool, List[int]]]

Stubs

package/module.py

def avg(data):
    if not len(data):
        return None
    return sum(data, 0.) / len(data)


def avg_unique(data):
    return avg(unique(data))


def unique(data):
    return set(data)

Stubs

package/module.pyi

def avg(data: Sequence[float]) -> Optional[float]: ...
def avg_unique(data: Sequence[float]) -> Optional[float]: ...
def unique(data: Sequence[float]) -> Set[float]: ...

Typeshed

  • python/typeshed
  • Provide stubs for standard library
  • And some widely-used shared libraries
  • Have stubs for Python 2 & Python 3
  • Curated by PSF

Python 2 Type Hints

def hello(name):  # type: (name: str) -> str
    return 'Hello, {0}'.format(name)


def multi_line_annotations(address,  # type: Union[str, List[str]]
                           sender,  # type: str
                           subject,  # type: str
                           body  # type: str
                           ):
    # type: (...) -> bool
    ...

More info at mypy docs.

mypy

Introduction

  • Static type checker for Python
  • Works with Python 3 & Python 2 code
  • Still experimental
  • Developed at Dropbox
  • Again. Endorsed by Guido

Usage

pip install mypy-lang typed-ast
mypy ...
mypy --fast-parser ...

Configuration. Step 1

mypy.ini

[mypy]
fast_parser = True
check_untyped_defs = True
warn_redundant_casts = True

Configuration. Step 2

mypy.ini

[mypy]
fast_parser = True
silent_imports = True
check_untyped_defs = True
warn_redundant_casts = True

Configuration. Step 3

mypy.ini

[mypy]
fast_parser = True
silent_imports = True
check_untyped_defs = True
disallow_untyped_defs = True
warn_redundant_casts = True

Output

$ mypy project/
project/signals.py: note: In function "cache_api_urls":
project/signals.py:25: error: Argument 2 to "api_url" has incompatible type "str"; expected "int"

Problems

No Hype

  • mypy is not widely used
  • No viable benefits for users
  • Hard to migrate large codebase to type hints

Long Lines / Ugly Code

Before

async def retrieve_tweets(pool, count=50):
    ...

After

async def retrieve_tweets(
    pool: Pool,
    count: int=50
) -> Sequence[Mapping[str, Any]]:
    ...

Stubs

  • Hard to maintain
  • Easy to get into situation, when implementation != stub
  • Completely same problems as with tests & documentation

Incomplete Stubs

import asyncio

def main() -> int:
    loop = asyncio.get_event_loop()
    tasks = [
        asyncio.ensure_future(some_async_def),
        asyncio.ensure_future(some_other_async_def),
    ]
    loop.run_until_complete(asyncio.gather(*tasks, loop=loop))
    loop.close()
    return 0

Incomplete Stubs

mypy -c '...'

<string>: note: In function "main":
<string>:6: error: "module" has no attribute "ensure_future"
<string>:6: error: Name 'some_async_def' is not defined
<string>:7: error: "module" has no attribute "ensure_future"
<string>:7: error: Name 'some_other_async_def' is not defined
<string>:9: error: "module" has no attribute "gather"

Incomplete Stubs

from lxml import etree


def fetch_data(url: str) -> etree._Element:
    ...


def use_data():
    data = fetch_data(URL)
    for item in data.iterfind(...):  # Will fail with "has no attribute" error
        ...

# noqa: F401

from typing import Dict, List, Set


def process_data(data: Dict[str, str]) -> List[int]:
    uniques = set()  # type: Set[int]
    for value in data.values():
        uniques.add(int(value))
    return list(uniques)

module.py:1:1: F401 'typing.Set' imported but unused

Solution to # noqa: F401

PEP-526 implements syntax for variable annotations.

uniques: Set[int] = set()

number: int  # Works even without assignment

class Schema(object):
    name: str
    data: Dict[str, str] = {}

Included in Python 3.6

Circular Imports

project/models.py

from .utils import some_func


class Model(object):

    def shortcut_for_some_func(self) -> int:
        return some_func(self)

project/utils.py

import typing

if typing.TYPE_CHECKING:
    from .models import Model

def some_func(model: 'Model') -> int:
    ...

# type: ignore

Sooner or later, but you'll need to use # type: ignore

asyncio.gather(*tasks, loop=loop)  # type: ignore

for item in data.iterfind('...'):  # type: ignore

mypy is still experimental and you'll need to use # type: ignore after yelling WTF 😔

Additional Notes

pytype

  • google/pytype
  • Type checker from Google
  • Needs Python 2.7 to run
  • Able to type check Python 3 code though

enforce

  • RussBaz/enforce
  • Runtime type checking
  • Designed to use at tests or for data validation
import enforce

@enforce.runtime_validation
def hello(name: str) -> str:
    return 'Hello, {0}!'.format(name)

hello('world')
hello(1)  # Will fail with RuntimeTypeError

mypy-django

Conclusion
🤔 🤓 🤓 😳 😫 😕 😢

Questions?

Twitter: @playpausenstop
GitHub: @playpauseandstop