How type annotations
make your code better?

I am…

from dataclasses import dataclass, field
from typing import Dict

@dataclass
class Speaker:

    name: str
    github: str
    twitter: str
    experience: Dict[str, int] = field(default_factory=dict)

    def knows(self, **languages: int) -> None:
        self.experience.update(**languages)

    @property
    def github_url(self) -> str:
        return f'https://github.com/{self.github}'

    @property
    def twitter_url(self) -> str:
        return f'https://twitter.com/{self.twitter}'

igor = Speaker('Igor Davydenko', 'playpauseandstop', 'playpausenstop')
igor.knows(python=2007, javascript=2011)

Agenda, kind of

from typing import List, Optional

@dataclass
class Topic:

    name: str
    description: Optional[str]

@dataclass
class Agenda:

    topics: List[Topic]

pycon_de_2018 = Agenda([
    Topic('Maintainability'),
    Topic('Predictability'),
    Topic('Developer Experience'),
    Topic('Toolset'),
])

Maintainability

Python code without type annotations

Is not:

  • Self-documented
  • Understandable by other teammates
  • Easy to refactor
  • Easy to cover with tests

def generate_token(email, expired_at, secret_key):
    """Generate valid JWT token for given email."""
    encoded = jwt.encode({'email': email, 'exp': expired_at}, secret_key)
    return encoded.decode('utf-8')

Make it better

def generate_token(email: str, expired_at: float, secret_key: bytes) -> str:
    """Generate valid JWT token for given email."""
    encoded = jwt.encode({'email': email, 'exp': expired_at}, secret_key)
    return encoded.decode('utf-8')

Adding type annotations to the project

Best case:

  1. Start new project
  2. Add type annotations for all code

Realistic case:

  1. Gradually add type annotations
  2. Start with core / myproject
  3. Continue in nested code

Adding type annotations to pet project

  1. LOC before type annotations: 2675
  2. Time for adding type annotations: about 3 hrs
  3. Commit with type annotations: +531/-925
  4. LOC after type annotations: 2642

Adding type annotations to the library

Hail to PEP-561

  1. Add type annotations to the code or stubs (*.pyi)
  2. Add empty py.typed file to each typed package
  3. Ensure including py.typed (and stubs) files as package_data
  4. PROFIT!

Examples: rororo, aiohttp-middlewares

typeshed and friends

Predictability

Before type annotations

Trust all the data!

  • Settings dictionaries
  • Request / response data
  • Working with database via proxies

And validating data doesn't fix the problem

Before type annotations

async def add_project(request):
    valid_data = validate_add_project(await request.json())
    async with request.app['db'].acquire() as conn:
        await conn.execute(
            projects_table
            .insert()
            .values(
                name=valid_data['nam'],
                description=valid_data['description'],
            )
        )
    return web.json_response(status=201)

Before type annotations

  • KeyError
  • AttributeError
  • You really need to test your code!

from typing import NamedTuple

class AddProjectStruct(NamedTuple):

    name: str
    description: Optional[str] = None

...

def validate_add_project(data: Dict[str, str]) -> AddProjectStruct:
    ...

Documentation

from mypy_extensions import TypedDict

AddProjectDict = TypedDict('AddProject', {
    'name': str,
    'description': Optional[str],
})

Documentation

from dataclasses import dataclass

@dataclass
class AddProjectData:

    name: str
    description: Optional[str] = None

Documentation

from pydantic.dataclasses import dataclass

@dataclass
class AddProjectData:

    name: str
    description: Optional[str] = None

...

async def add_project(request: web.Request) -> web.Response:
    data = AddProjectData(**await request.json())
    ...

Documentation

After type annotations

More structs for the god of structs!

  • Your code became more predictable
  • Simpler understanding of possible data flows
  • Better tests for the code

And more…

  • No var redifinition
  • Avoiding changing method signatures
  • Avoiding complex data structures
  • Avoiding Any data

Developer Experience

No docs? No problem!

def build_secret(email: str, secret_key: bytes) -> str:
    ...

def generate_otp(email: str,
                 period: int,
                 secret_key: bytes) -> Tuple[int, float]:
    ...

def generate_token(email: str, expired_at: float, secret_key: bytes) -> str:
    ...

def send_email(app: web.Application, email: str) -> float:
    ...

def validate_otp(app: web.Application,
                 otp: int,
                 email: str,
                 timestamp: float) -> bool:
    ...

def validate_token(app: web.Application,
                   token: str) -> Tuple[bool, Optional[Mapping[str, Any]]]:
    ...

Editor is your friend

build_secret autocomplete

Fact: Your start typing much faster with annotated code

No tests? Still problem, but…

  • You still need to write tests!
  • But mypy allows you to find additional problems with your code
  • Deploys became more robust

Better understanding other languages

Rust

fn add(x: i32, y: i32) -> i32 {
    x + y
}

struct DebugWriter {
    sha: Writer<Blake2b512>,
    data: Opt<Vec<u8>>,
}

Better understanding other languages

Flow (JavaScript)

// @flow
const add = (x: number, y: number): number => x + y

type Props = {
    id: number,
    text: str,
    topics: Array<str>
}

Better understanding other languages

Elm

add : Int -> Int -> Int
add x y =
    x + y

type alias TweetId =
    Int

type alias Tweet =
    { id : TweetId
    , text : String
    , url : String
    , createdAt : String
    }

Toolset

Cornerstone for type annotations in Python

When you need faster type checker

  • Faster type checker from Facebook
  • Written in OCaml
  • I still not tried it on real project

When you need to guess type annotations

  • Collect runtime types, and generate stub files
  • For example, run test suite - get stubs for your code
  • MonkeyType is from Instagram
  • PyAnnotate is from Dropbox

When you need to validate your data

  • BaseModel for basic validation
  • @dataclass for better integration with mypy
  • BaseSettings for working with settings

When you want to generate Python C Extensions

  • Generate Python C Extensions annotated code
  • Useful for speedup things
  • Product from mypy authors
  • Is not a replacement for Cython

Conclusion?

Annotate your code

for all good things!

Questions?

Twitter: @playpausenstop
GitHub: @playpauseandstop