Using Python's Type Annotations
Daniel Starner

Daniel Starner @dan_starner

About: I am a curious person who enjoys figuring out the building blocks of the world, and rearranging them to build something even better.

Location:
Brooklyn, New York
Joined:
Feb 23, 2017

Using Python's Type Annotations

Publish Date: Aug 18 '18
225 30

Python is known for being a Wild West language where anything goes. Indentation aside, code style and documentation are mostly left to the developer's opinion writing the application, which can lead to some messy, unreadable code.

This vague styling structure comes partially from Python being a dynamic typed language, meaning that types are associated with the variable's value at a point in time, not the variable itself. This language attribute means that variables can take on any value at any point and are only type checked when an attribute or method is accessed.

Consider the following code. In Python, this is acceptable.

age = 21
print(age)  # 21
age = 'Twenty One'
print(age)  # Twenty One
Enter fullscreen mode Exit fullscreen mode

In the code above, the value of age is first an int (integer), but then we change it to a str (string) later on. Every variable can represent any value at any point in the program. That is the power of dynamic typing!

Let's do the same thing in a statically typed language, like Java.

int age = 21;
System.out.print(age);
age = "Twenty One";
System.out.print(age);
Enter fullscreen mode Exit fullscreen mode

We end up with the following error because we are trying to assign "Twenty One" (a String) to the variable age that was declared as an int.

Error: incompatible types: String cannot be converted to int
Enter fullscreen mode Exit fullscreen mode

To work in a statically typed language, we would have to use two separate variables and use some assistive type-conversion method, such as the standard toString() method.

int ageNum = 21;
System.out.print(ageNum);
String ageStr = ageNum.toString();
System.out.print(ageStr);
Enter fullscreen mode Exit fullscreen mode

This conversion works, but I really like the flexibility of Python, and I don't want to sacrifice its positive attributes as a dynamic, readable, and beginner-friendly language just because types are difficult to reason about in most cases. With this said, I also enjoy the readability of statically typed languages for other programmers to know what type a specific variable should be! So, to get the best of both worlds, Python 3.5 introduced type annotations.

What Are Type Annotations?

Type Annotating is a new feature added in PEP 484 that allows adding type hints to variables. They are used to inform someone reading the code what the type of a variable should be expected. This hinting brings a sense of statically typed control to the dynamically typed Python. This is accomplished by adding a given type declaration after initializing/declaring a variable or method.

Why & How to Use Type Annotations

A helpful feature of statically typed languages is that the value of a variable can always be known within a specific domain. For instance, we know string variables can only be strings, ints can only be ints, and so on. With dynamically typed languages, its basically anyone's guess as to what the value of a variable is or should be.

Annotating Variables

When annotating variables, it can be defined in the form

my_var: <type> = <value>
Enter fullscreen mode Exit fullscreen mode

to create a variable named my_var of the given type <type> with the given value.

An example is shown below, which adds the : int when we declare the variable to show that age should be of type int.

age: int = 5
print(age)
# 5
Enter fullscreen mode Exit fullscreen mode

It is important to note that type annotations do not affect the program's runtime in any way. These hints are ignored by the interpreter and are solely used to increase the readability for other programmers and yourself. But again, these type hints are not enforced are runtime, so it is still up to the caller method/function/block to ensure proper types are used.

Annotating Functions & Methods

We can use the expected variable's type when writing and calling functions to ensure we are passing and using parameters correctly. If we pass a str when the function expects an int, then it most likely will not work in the way we expected.

Consider the following code below:

def mystery_combine(a, b, times):
    return (a + b) * times
Enter fullscreen mode Exit fullscreen mode

We can see what that function is doing, but do we know what a, b, or times are supposed to be? Look at the following code, especially at the two lines where we call the mystery_combine with different types of arguments. Observe each version's output, which is shown in the comments below each block.

# Our original function
def mystery_combine(a, b, times):
    return (a + b) * times

print(mystery_combine(2, 3, 4))
# 20

print(mystery_combine('Hello ', 'World! ', 4))
# Hello World! Hello World! Hello World! Hello World!
Enter fullscreen mode Exit fullscreen mode

Hmm, based on what we pass the function, we get two totally different results. With integers we get some nice PEMDAS math, but when we pass strings to the function, we can see that the first two arguments are concatenated, and that resulting string is multiplied times times.

It turns out that the developer who wrote the function actually anticipated the second version to be the use case of mystery_combine! Using type annotations, we can clear up this confusion.

def mystery_combine(a: str, b: str, times: int) -> str:
    return (a + b) * times
Enter fullscreen mode Exit fullscreen mode

We have added : str, : str, and : int to the function's parameters to show what types they should be. This will hopefully make the code clearer to read, and reveal it's purpose a little more.

We also added the -> str to show that this function will return a str. Using -> <type>, we can more easily show the return value types of any function or method, to avoid confusion by future developers!

Again, we can still call our code in the first, incorrect way, but hopefully with a good review, a programmer will see that they are using the function in a way it was not intended. Type annotations and hints are incredibly useful for teams and multi-developer Python applications. It removes most of the guesswork from reading code!

We can extend this one step further to handle default argument values. We have adapted mystery_combine below to use 2 as the default argument value of the times parameter. This default value gets placed after the type hint.

def mystery_combine(a: str, b: str, times: int = 2) -> str:
    return (a + b) * times
Enter fullscreen mode Exit fullscreen mode

Type Hints with Methods

Type hints work very similarly with methods, although it's pretty common - in my experience anyways - to leave off the type hint for self, since that is implied to be an instance of the containing class itself.

class WordBuilder:

    suffix = 'World'

    def mystery_combine(self, a: str, times: int) -> str:
        return (a, self.suffix) * times
Enter fullscreen mode Exit fullscreen mode

You can see above that the code is very similar to the previous function-based example, except we have dropped the b parameter for a suffix attribute that is on the WordBuilder class. Note that we don't need to explicitly add : str to the suffix definition because most code editors will look at the default value for the expected type.


Available Types

The previous section handles many basic use cases of type annotations, but nothing is ever just basic, so let's break down some more complex cases and show the common types.

Basic Types

The most basic way to annotate objects is with the class types themselves. You can provide anything that satisfies a type in Python.

# Built-in class examples
an_int: int = 3
a_float: float = 1.23
a_str: str = 'Hello'
a_bool: bool = False
a_list: list = [1, 2, 3]
a_set: set = set([1, 2, 3])  # or {1, 2, 3}
a_dict: dict = {'a': 1, 'b': 2}

# Works with defined classes as well
class SomeClass:
    pass

instance: SomeClass = SomeClass()
Enter fullscreen mode Exit fullscreen mode

Complex Types

Use the typing module for anything more than a primitive in Python. It describes types to hint any variable of any type more detailed. It comes preloaded with type annotations such as Dict, Tuple, List, Set, and more! In the example above, we have a list-hinted variable, but nothing defines what should be in that list. The typing containers provided by the typing module allow us to specify the desired types more correctly.

Then you can expand your type hints into use cases like the example below.

from typing import Sequence

def print_names(names: Sequence[str]) -> None:
    for student in names:
        print(student)
Enter fullscreen mode Exit fullscreen mode

This will tell the reader that names should be a Sequence of strs, such as a list, set, or tuple of strings.

Dictionaries work in a similar fashion.

from typing import Dict

def print_name_and_grade(grades: Dict[str, float]) -> None:
    for student, grade in grades.items():
        print(student, grade)
Enter fullscreen mode Exit fullscreen mode

The Dict[str, float] type hint tells us that grades should be a dictionary where the keys are strings and the values are floats.

Type Aliases

If you want to work with custom type names, you can use type aliases. For example, let's say you are working with a group of \[x, y\] points as Tuples, then we could use an alias to map the Tuple type to a Point type.

from typing import List, Tuple


# Declare a point type annotation using a tuple of ints of [x, y]
Point = Tuple[int, int]


# Create a function designed to take in a list of Points
def print_points(points: List[Point]):
    for point in points:
        print("X:", point[0], "  Y:", point[1])
Enter fullscreen mode Exit fullscreen mode

Multiple Return Values

If your function returns multiple values as a tuple, wrap the expected output as a typing.Tuple[<type 1>, <type 2>, ...]

from typing import Tuple

def get_api_response() -> Tuple[int, int]:
    successes, errors = ... # Some API call
    return successes, errors
Enter fullscreen mode Exit fullscreen mode

The code above returns a tuple of the number of successes and errors from the API call, where both values are integers. By using Tuple[int, int], we are indicating to a developer reading this that the function does return multiple int values.

Multiple Possible Return Types

If your function has a value that can take on a different number of forms, you can use the typing.Optional or typing.Union types.

Use Optional when the value will be be either of the given type or None, exclusively.

from typing import Optional

def try_to_print(some_num: Optional[int]):
    if some_num:
        print(some_num)
    else:
        print('Value was None!')
Enter fullscreen mode Exit fullscreen mode

The above code indicates that some_num can either be of type int or None.

Use Union when the value can take on more specific types.

from typing import Union

def print_grade(grade: Union[int, str]):
    if isinstance(grade, str):
        print(grade + ' percent')
    else:
        print(str(grade) + '%')
Enter fullscreen mode Exit fullscreen mode

The above code indicates that grade can either be of type int or str. This is helpful in our example of printing grades so that we can print either 98% or Ninety Eight Percent, with no unexpected consequences.

Working with Dataclasses

Dataclasses are a convenience class that provide automatically generated __init__ and __repr__ methods to an appropriate class. It reduces the amount of boilerplate code needed to create new classes that take in multiple keyword arguments to their constructor. These dataclasses use type hints and class-level attribute definitions to determine what keyword arguments and associated values can be passed to __init__ and printed by __repr__.

The following code is directly from the dataclasses documentation. It defines an InventoryItem that has three attributes defined on it, all using type hints; a name, unit_price, and quantity_on_hand .

from dataclasses import dataclass

@dataclass
class InventoryItem:
    """Class for keeping track of an item in inventory."""
    name: str
    unit_price: float
    quantity_on_hand: int = 0

    def total_cost(self) -> float:
        return self.unit_price * self.quantity_on_hand
Enter fullscreen mode Exit fullscreen mode

Using the type hints and @dataclass decorator, new InventoryItems can be created with the following code, and the dataclass will take care of mapping the keyword arguments to attributes.

common_item = InventoryItem(name='My Item', unit_price=2.99, quantity_on_hand=60)
other_item = InventoryItem(name='My Item', unit_price=2.99)  # uses default value of 10 quantity
Enter fullscreen mode Exit fullscreen mode

An important note to @dataclasses is that any class attribute defined with a default value must be declared after any attributes without a default value. This means quantity_on_hand has to be declared after name and unit_price. This can get interesting when working with dataclasses that extend from a parent dataclass, so be careful, but the Python interpreter should catch these issues for you.

More Examples

For more examples, check out the official Python documentation for the typing module. They have a ton of different variations of examples that you can check out. I just hit the tip of the iceberg here, but hopefully, I have piqued your interest in making cleaner, easier-to-read code using type annotations in Python.

As always, please reach out, like, comment, or share if you have any comments or questions!

Comments 30 total

  • Paddy3118
    Paddy3118Aug 19, 2018

    " Python is known for being a Wild West language where anything goes. Aside from indentation, code style and documentation is left mostly up to the opinion of the developer writing the application, which can lead to some messy, unreadable code."

    Hyperbolic twaddle! The author starts with such rubbish. Don't read further

    • Daniel Starner
      Daniel StarnerAug 19, 2018

      Sorry you're not a fan. I respect your opinion. I'll be the first to admit I'm not a good writer, but hopefully through writing and feedback like this, I can get better!

      • Paddy3118
        Paddy3118Aug 26, 2018

        It is just that there is a preponderance of articles out there on what language to learn first; or what language to learn next; that don't have such an attitude about Python. I would have thought that a writer would have picked up on this and either go with it; or put forward their argument for why Python should be classed as you described.

  • KhoPhi
    KhoPhiAug 19, 2018

    After using Typescript and now using Dart, I think I'm gonna try this Python typings too.

    Thanks for the overview.

  • Thomas van Latum
    Thomas van LatumAug 19, 2018

    The problem with statically typed languages is that you tend to spend an enormous amount of time finding out what type of variable you've used somewhere in your code.

    For me dynamic typed code works best, I just write variable and function names that are self explanatory.

    greetingString = "Hello World!";

    • AkashGanesan
      AkashGanesanAug 19, 2018

      Actually, when the language had type inferences, it gets a lot easier and the intent is almost always made clear with type annotations. Haskell is a prime example of how easy it is to use the type system to guide program correctness. Oh, And the thing can look like a dynamically typed language because of the inference engine doing most of the heavy lifting for you.

    • vishal
      vishalAug 19, 2018

      Not if you are coding in C#. var keyword does all the heavy lifting for you and you still have all the good compile-time safety.

    • Rudra
      RudraAug 21, 2018

      greetingString = "Hello World!";

      A pinch of JavaScript camel-case in Python.

      Humans are messy !! but that's how they keep themselves evolving

    • yucer
      yucerAug 23, 2018

      The problem with statically-typed languages is that you tend to spend an enormous amount of time finding out what type of variable you've used somewhere in your code.

      Why would you do that ? Given that is statically-typed language the compiler and the editor have the info of the variable type in the moment that you are programming.

      They can not do that in a dynamically-typed language without stuff like the one described in this article, or something like the method documentations.

      By the way, we can think about this like another way to document the method parameters and result.

      I guess it is like this:

      1. statically-typed language: You spent more time selecting the types to define / choose in every context.

      2. dynamically-typed language: You spent more time guessing the types when you are using it.

      Take this example:

      You are going to use a python function called compute for the first time.

      def compute(foo, bar):
         # .... 20 lines here
      

      If the method has no documentation string, you need to read the documentation or read the code to infer the type.

  • edA‑qa mort‑ora‑y
    edA‑qa mort‑ora‑yAug 19, 2018

    Type annotations that have no compile-time or runtime effect are worse than not having annotations at all. They will mislead readers of the code, giving them a false sense of security. Debugging will be made harder since we'll start relying on these annotations being true, even if they are not.

    Sorry, it's a bad language that allows:

    number : int = "Hello"
    
    Enter fullscreen mode Exit fullscreen mode

    This doesn't help anything at all.

    It would have been better off to just allow inline comments and you'd get the desired effect without the artificial "safety"

    foo( number /*int*/, name /*string*/ )
    
    Enter fullscreen mode Exit fullscreen mode
    • rhymes
      rhymesAug 19, 2018

      Type annotations that have no compile-time or runtime effect are worse than not having annotations at all

      I agree, that's why I don't use them yet. I might start looking into them if I were to adopt mypy which is a compile time static type checker...

    • Dave Cridland
      Dave CridlandAug 20, 2018

      Actually it's much cleverer than having a comment:

      • Your IDE can pick up the difference much more easily and warn you, plus the typing information itself is syntax-checked.
      • Tools (like MyPy) can enforce it.
      • Yes, it can be ignored or overridden by a developer - that could be useful inside library internals.
      • User-defined metatypes, like the UserId example given in the docs, aren't possible with your comments.

      For an example of the latter point:

      from typing import NewType
      
      UserId = NewType('UserId', int)
      user1 = UserId(1)
      user2 = UserId(2)
      
      # user1 and user2 are simply integer values, but with typing info.
      
      user = get_user(user1) # OK
      user = get_user(user2) # OK
      user3 = user1 + user2 # OK, these are ints!
      user = get_user(user3) # Not OK! That's not a UserId
      
      
    • Augusto Santos
      Augusto SantosAug 22, 2018

      Wait, it's just to turn the code more clear and avoid have to guess types to deal with function/methods parameters. Python is not a compiled language, all python programmers know that and won't use this feature like compile time checking for type security.

    • Vedran Čačić
      Vedran ČačićAug 25, 2018

      By the same thinking, why do we name variables meaningful names? If a compiler lets me write num_tries = 'five', does it follow that we should only name our variables generic names like x and y? After all, num_tries might give readers of the code false sense of security. :-P

    • Maik Nowak
      Maik NowakApr 18, 2019

      Ofc it's always easyer to tell why something is bad, than trying to see how it could add improvement. Just use MyPy as part of your Unit test suite and you get immediate value for your CI pipeline. See it like adding some "intentions" (that also your IDE understands #refactorings) to your code and let them be checked for consistency. From a small project point of view that may not seem like a big plus, especially compared to the additional work. But from a more professional point of view this was missing for way to long.

      plus: what Dave Cridland said.

      • edA‑qa mort‑ora‑y
        edA‑qa mort‑ora‑yApr 18, 2019

        I'm not saying it doesn't have uses, I'm saying that the basic syntax makes the code worse than it was before. The lying about the types will create more bugs.

        This type of behaviour is very unusual for a language feature. Most features add value of some kind, or at least sit at zero value. This one actively reduces the ability of somebody to write proper code by misleading them about what is happening.

        The fact it has some benefits does not outweigh these guaranteed negatives.

        • Jiri Novak
          Jiri NovakDec 20, 2019

          After several months your comment seems wrong. Large number of questions on /r/learnpython wouldn't have been even asked if using type annotations + mypy.

          Typical mistake: missing int() when using input() to read int values.

  • David McKay
    David McKayAug 19, 2018

    Interesting addition. As an intermediate level coder seeing this for the first time I can see it being useful of every developer looking at the code well read on the syntax. edA-qa makes a good point that it could lead to some people thinking they're making a runtime change when really they're not. The following syntax would make more sense to me:
    number = 7 : int
    Then the definition is separated from the annotation similarly to an end-of-line comment, which I often use, i.e.,
    number = 7 # an integer

  • 8uurg
    8uurgAug 19, 2018

    If you want to have these types checked, mypy-lang.org exists and is pretty cool.

  • Tomer
    TomerAug 19, 2018

    Pretty sure type annotations were introduced in py3.5

    • Daniel Starner
      Daniel StarnerAug 19, 2018

      Oops, my bad! Will edit. That's what I get for trying to remember from memory. Thanks for the feedback! 👍

  • Ikem Krueger
    Ikem KruegerAug 21, 2018

    It looks like Swift. :D

  • Mayank Singh
    Mayank SinghJan 28, 2019

    Thanks, this discussion, and the article both were highly useful.

  • abhyvyth
    abhyvythJun 24, 2019

    Hi!

    What do I do if my dictionary can have values of multiple types?

    Dict[str, Any] is throwing this error: Implicit generic "Any". Use "typing.List" and specify generic parameters

  • Roberto Torres
    Roberto TorresJul 11, 2019

    wow! interesting...

  • Mohammed El-Afifi
    Mohammed El-AfifiFeb 3, 2020

    I'm going to write my honest opinion about this. I got to learn about type hinting in python a couple months ago(; I knew it existed long before that but didn't bother to learn about it until recently). While the idea seems promising at the first glance, I think the implementation could be much better, to say the least.

    Adding type hints to python was performed in a way that got in the way of developers writing code instead of helping them to do so. It looks like a completely new language was suddenly embedded inside your python code, and makes the developer to write what seems like two languages interspersed on the same line.

    First it has a steep learning curve(, and yes I mean the learning curve not the effort to apply the rules to an existing codebase, which is another story). Just consider the different types in the typing module the developer has to learn about to annotate variables instantiated from other popular types(built-in ones or otherwise) like list, tuple, re.Match, ...etc. You can't use the original types because they don't support the generic class syntax like list[int]; you have to instead rely on the equivalent ones from the typing module like List[int]. Even the syntax for specifying a generic class with a type parameter is inconvenient, relying on the indexing operator instead of some other clearer one. So you end up using two families of type names, one for constructing objects and another for annotating. Luckily this doesn't happen with user-defined types as they're supported to be used for both purposes.

    And then consider the code changes. I should be honest to say that these changes are needed if you want a static type checker like mypy(which is the only one I've used so far) against your code, which seems the rational and right thing to do after you decide to add type hints(otherwise they'll turn out over time to be just another type of comments). Once you decide to use something like mypy, all hell gates are open. Suddenly some expressions that looked just fine and logical need restructuring and changes. Take for example containers or strings that we can use as boolean expressions; this no longer works because the type checker needs to see that you really mean to test their truth value(especially evident in return statements with functions returning bool). Another example is the need to convert many lambdas passed as parameters to functions to explicit standalone functions, just because the type checker can't sanitize the lambda syntax sufficiently(especially evident with third-party libraries, which I discuss later).

    Sometimes the type checker is completely unable to understand what's written and seems like a very clear code snippet, like this question stackoverflow.com/questions/600296... I asked on stackoverflow about how to satisfy mypy without writing a lot of boilerplate code. Please don't get me wrong: I do appreciate the effort that has gone over the years to bring mypy and other type checkers to how they look now, but the deficiencies and shortcomings of these tools adversely affects the full picture of type hinting.

    Now consider the conformance and adoption of third-party libraries and packages for type hinting. Very few libraries have done so, and even some very popular ones haven't put the effort to do(like the attrs library for example). This imposes that you must instruct your type checker to ignore uses and references to utilities from these third-party libraries which scanning your code, which still introduces many weaknesses contrary to what type hinting claimed and was intended for in the first place.

    I'm still keen and intending to use type hints with my projects, but honestly the status quo of type hinting in python discourages developers from adopting and applying it as a mainstream practice while coding.

  • Shiva Krishna
    Shiva KrishnaMar 22, 2020

    json_data: {str, ...} = None

    I'm new to python. What is "{str, ...}" from the above the syntax? Please help me to understand that.

  • 5elenay
    5elenayMay 17, 2021

    I was actually using tuple but i learned typing.Union with this. Thanks for this post!

Add comment