A couple of days ago I came across Python type hints. I’ve decided to try it out in practice on some of my Python scripts I use to automate some of my daily tasks.

After applying the type hints as well as using mypy tool for a while, I’ve decided to share a couple of thoughts on how it worked and how it supposed to work, to get some true value out of it.

Oh no, again few words on types in Python

Just reminding - Python is a dynamically typed language which means you don’t have to statically declare the type of a variables you are using to hold your values and object references. The type of a variable is automatically determined at the time when an assignment operation is executed. Yes - we all know that…

That mechanism brings some convenience. And at the same time can hit back hard.

I’ve struggled for far too many times to get my Python code refactored correctly partially also because there is no other feedback on weather you’ve made things right then the actual application run-time. It is quite easy to get lost in what are the expected types of arguments to functions and methods, especially when the code size keeps growing and re-factoring starts to get some real momentum.

OK, one may say - have your unit tests covering all of your code. True in theory. I find this approach a bit overhead for smaller and ‘not-so-important’ - yet still practical and useful Python utilities.

For a quite a time I was missing the feature that statically typed languages like C++/C#/Java (… and many others - you name it) have, so that when e.g. you change an argument type of a function argument to something different, you will get an immediate (or at least an early) feedback from a compiler yelling at you and complaining about other source files in your code you have to visit and adopt for a change.

I was quite pleased to discover a Python 3.5 feature - an initial support for expressing and propagating type information in Python code, which can be used to validate the type consistency across the code.

It’s worth mentioning here that the type hints available in Python 3 have a form of a really ‘soft’ feature - the interpreter will completely ignore them all during the run-time.

There are two things you can use Python types hints for:

• having self-documented function and method prototypes, so you no longer have to write and explain expected argument types in plain text python doc,
• using an external tool like mypy to analyse your source code define you with potential warning error messages.

I will focus here on the latter case.

Let’s have a look at the example code below. First, good, old-fashioned Python 3 code:

def foo(arg):
    print (arg * 1.1)

# Let's call it with string argument
foo('bar')

An attempt to run it (with python3) not surprisingly gives a run-time error:

Traceback (most recent call last):
  File "example3.py", line 5, in <module>
    foo('bar')
  File "example3.py", line 2, in foo
    print (arg * 1.1)
TypeError: can't multiply sequence by non-int of type 'float'

Clearly, foo() method is not expecting a string as an valid input.

Now, with Python type hints the same code can look like this:

def foo(arg : float):
    print (arg * 1.1)

# Let's call it with string argument
foo('bar')

Notice the : float syntax in the function argument - that’s where the type hint goes.

As I’ve mentioned earlier, Python interpreter will ignore that extra syntax, so the execution will be by no mean different than before - you’ll end up with exactly the same run-time error.

However the difference is that the latter code can be checked with the nice and helpful mypy tool giving you an early feedback on that things have been messed up.

Assuming our type-hinted code is located in example.py file, we can now run:

mypy example.py

and we will get:

example.py:5: error: Argument 1 to "foo" has incompatible type "str"; expected "float"

Sweet!

Not all sweet actually

I got some excitement and after applying type checking to some of my favourite Python tools, I’ve quickly realised practical type checking gets a little bit trickier than just following the pattern from the example above.

Python’s dynamic typing system practically means, that you can be a lot more flexible in the way you use underlying types of arguments passed to functions and methods.

What you’re actually using in Python (and passing in the run-time of your code) are not just values and objects whose behaviour is determined by their type (nor their type inheritance hierarchy - that’s true, but not the whole truth out there), but rather a vague behavioural concepts, abstract concepts, or abstract interfaces which don’t really have any formal form of declaration in Python.

That is what makes a type checking in Python code base a bit complex and not always straightforward. So when you introduce type hints, what you should be taking care of is not really declarations of a concrete arguments type, but rather a check whether argument has a certain nature or in other words whether it implements certain run-time available functionality.

Python is known for its duck-typing (or as someone phrased it “if it looks like a duck and it quacks like a duck, it’s a duck”).

Let’s have a look at an example. Let’s have a function that calculates a sum of float numbers passed to it in a collection, e.g. a list.

The implementations for that is fairly straightforward. Let’s be lazy and for the sake of this exercise let’s reuse built in sum function to actually do the job. Obviously, we add the type hint straight away, using some imports from typing module, where representation of List type is available for that purpose.

from typing import List

def get_total(numbers : List[float]) -> float:
    return sum(numbers)

Looks all good. Now let’s use this function too calculate a total price of some items from a hypothetical shopping basket, represented by a simple Python dictionary data structure, e.g. with two items foo and bar, and some (randomly chosen) prices as floats:

basket = {'foo': 10.22, 'bar': 23.13}

There are possibly many ways to go from having our basket dictionary towards using our get_total function here, let’s look at two of them:

• first - using a list comprehension to get the prices from the basket,
• second - using values() method of the dictionary.

Both are fully valid Python code.

example1.py:

from typing import List

def get_total(numbers : List[float]) -> float:
    return sum(numbers)

basket = {'foo': 10.22, 'bar': 23.13}

prices = [v for k,v in basket.items()]
print (get_total(prices))

and example2.py:

from typing import List

def get_total(numbers : List[float]) -> float:
    return sum(numbers)

basket = {'foo': 10.22, 'bar': 23.13}

prices = basket.values()
print (get_total(prices))

They both run without any errors (python3 example1.py or python3 example2.py), printing the result of 33.35 on the console.

OK. Now let’s check what mypy is about to say.

Executing:

mypy example1.py

passes without any errors. At the same time:

mypy example2.py

ends with non-zero exit code and shouts with:

example2.py:9: error: Argument 1 to "get_total" has incompatible type ValuesView[float]; expected List[float]

The error message reveals the problem. As a Python developer you would consider keys() dictionary method as fitting for the purpose too, while the mypy tells it finds it being a type mismatch.

The Python 3 dictionary keys() method is not returning a list, but a dynamic view object (a generator object actually). This object is neither derived from a built-in list type, nor has any common ancestor type in the inheritance hierarchy with the built-in list that we could use here. On the other hand, it is however designed to fit well as a run-time substitution of a list-like objects in many situations, definitely like this one here.

Expect the ducks, not types

Let’s look again to our get_total function and how we use the argument numbers there.

Our implementation wants to ‘iterate’ over all numbers given in it and calculate the total sum. All that we expect from argument given is iterability over the numbers. So here is the real catch. The Python type hint here should focus on behavioural aspect like ability to iterate over whatever numbers happen to be.

Let’s correct our function accordingly then.

To make things work we need to replace the type hint to be Iterable rather than a List:

example2b.py:

from typing import Iterable

def get_total(numbers : Iterable[float]) -> float:
    return sum(numbers)

basket = {'foo': 10.22, 'bar': 23.13}

prices = basket.values()
print (get_total(prices))

Now, mypy has nothing to complain about. We’ve made it clear for the tool (as well as for other people too) what we’re really expecting to get when get_total is called.

Toolbox

If you want to have a painless experience and start using Python type hints, get familiar with these abstract base classes for containers, because most of them have matching type in typing module that you will be using for type hints.

How about Python 2.7?

Type hints in the form as I’ve present here are available in Python >= 3.5. You can still successful a presented technique in Python 2, but the syntax will be a bit different, and you will be relying on placing the type hints inside a comment blocks, rather than regular code, bit like this:

# Python 2.7 equivalent 
from typing import Iterable

def get_total(numbers): # type: (Iterable[float]) -> float
    return sum(numbers)

basket = {'foo': 10.22, 'bar': 23.13}

prices = basket.values()
print get_total(prices)

and the checking with mypy goes like this:

mypy --python-version 2.7 example27b.py

More on this to be found on this mypy documentation page.