Get the top HN stories in your inbox every day.
linsomniac
totony
>My editor now has a fairly deep understanding of my code, and can tell me of all sorts of surprising errors I'm making before I run the code.
I agree this is useful, but to me the most useful feature of types are the self-documentation. It's crazy the difference between a non-typed library with shitty documentation vs a typed library with shitty documentation.
kadoban
The types on stitty-documented code often do depend on the types actually being checked though. Otherwise you need to author to be diligent in their types, but if they're bad in their other docs, that seems unlikely.
tjoff
You don't think that "the author thought that this function would return X, but I got Y" is useful?
kixiQu
Some people are willing to be meticulous about references even when they're Less Good Than One Might Wish at picking names or explaining things.
pvillano
type hints are documentation, and machine readable documentation to boot!
mountainriver
Not typing is kinda a crazy way to write programs when you think about it. It can be beneficial in certain niche use cases like data science where code is always terrible. For everything else, types are a huge boon for the developer and everyone consuming their work
waitlist
I think data science is a perfect example in favor of types -- the code is often terrible because of the lack of typing. Pandas has notoriously poor developer ergonomics, and I recall painfully poring over type errors across the board -- lists, dataframes, numpy arrays, etc. are all iterables, so they can be interchanged in some contexts, but not in others.
Had I had MyPy back when I was working in data science, I would've saved countless hours and headaches.
bayesian_horse
What Pandas does is notoriously hard to fit into a compile-time type system. Certainly too hard to go into the brains of scientists who didn't grow up coding.
No, the code in data science isn't bad because of the lack of typing. The code is "bad" mostly because those writing it are relatively fresh from starting to program. Also there is more pressure to make things possible, often just to run it once, and neglect repeatability or scaling to larger code bases. Different emphasis. That doesn't mean an experienced full stack developer would do Data Science better, because he might lack a lot of skills that matter more in that domain.
mountainriver
Totally agree, I would love for data scientists to use types and a lot of other good practices. I work primarily in mlops and have been trying to do this for some time, it just mostly feels like a losing game.
The output of most analytics is a report. For longer lived processes it seems easier to just hand off
bayesian_horse
Have you tried it? Python has gained most of its popularity before type hints, and I would (wildly) guess 99% of Python programmers don't even use type checking at all.
Static typing is completely viable and I hear those arguments mostly from those who didn't use Python for a long time. The added productivity makes up for a little more debugging while the program is running.
I'd also posit that many Python codebases are somewhat less "untyped" than one might assume. Django for example does a lot in its ORM and Form classes to make sure the right stuff goes in the right slot.
smohare
If you mean “Python programmers” to be random sysadmins that write hack code residing on a random EC2 instance then maybe you’re right. But most I get the sense you haven’t looked at many Python libs lately.
The added productivity of non-typed Python is such a ridiculous myth to anyone who has to maintain significant Python code bases. Sure, it makes you more productive for one-off exercises but the moment you’re having someone else inspect your code the productivity gains vanish. This largely true of dynamic languages more generally except in niche caches (e.g., ORMs) where types are strongly implicit.
I say all this as a mathematician that is used to reading papers full of symbols packed with implicit information. The code I’ve read from gung-ho dynamic typing advocates is some of the most painful drivel I’ve ever had to wade through.
saila
I'm not sure I'm following your comment correctly, but you seem to be implying that adding type hints in Python results in some lost productivity?
If so, I would counter that adding types isn't particularly onerous and adds very little overhead to development time, and over time they'll tend to increase your productivity since you'll make less errors, IDEs can give you better hints, etc.
As to the percent of Python developers who use types / type checking, it seems to me that's been changing somewhat rapidly over the past few years. It's certainly greater than 1%.
mountainriver
I have absolutely tried it and worked on many code bases big and small before typing happened. I've also worked on other languages and paradigms and seen what works and doesn't.
In my experience its the other way around. Most people who are a fan of dynamic typing have worked in systems with types. The world is largely coming around to the view that types make more consumable and safe software
antipaul
As a data scientist, you make a strong point.
For me, python is mostly a calculator, and it seems faster to try a calculation and immediately see if it runs and/or looks correct, rather than try to implement type hints et al (though I still use type hints occasionally).
wheelerof4te
Yeah, especially types like:
List[Tuple[Tuple[str, int, str], Tuple[int, float, float]]]
How about no?
animal_spirits
If you find that you are returning silly types like that, then it is an indicator you are writing your function badly and should refactor it to be simpler
icedchai
Yep, that's gross. Having come from a position where people where forced to actually add typing like this to make it through code review, I feel the pain.
Jcowell
Seems to me you have a Data Structure problem. What type of data is this suppose to represent and why can a Class not fix it.
nmeagent
LOL, just assign that beast of a type to a name and roll with it:
Thing = List[Tuple[Tuple[str, int, str], Tuple[int, float, float]]]
def foo(t: Thing) -> Thing:
...eweise
Totally agree. I'm on a project using python for the first time and really can't believe how horrible it is to not have type checking, especially given that it takes time to run the code as a glue job.
undefined
gkhartman
I use them for the same reason. I've started sprinkling them into projects where I can. Other devs on the team seem to like seeing them, but have higher inertia when it comes to writing them.
yowlingcat
Have you (or anyone else on this thread) used retype or MonkeyType? We have a codebase right now that's mostly lacking type hints. But, now that we have added automatic flaking into our build (and pre-commit hooks), I'd like to get some more mileage out of that toolchain we just invested into, but it would be a hassle to go back and add type hinting to everything.
What I'd like to do is integrate something into our pre-commit hooks.
bageljr
Actually depending on the LSP, you are using a type checker since Pyright is its own type checker and Pylsp has a mypy plugin
np_tedious
> I'm not even using mypy often, if ever.
Hopefully your CI does?
Spivak
It's not really as necessary as you think -- making mypy/pyright pass your CI ends up being a lot of busy work to make the type checker happy when, to me, all the value is realized when it's used as a development aid. If your test suite passes that's honestly more than good enough.
icedchai
You have the right attitude. I have worked with delusional individuals who one day "soon" expect to mypy check their entire several hundred thousand line legacy code base. This is supposed to be a boon to developer productivity. I hope it outweighs what was lost getting to this point.
linsomniac
Not yet, but that might be coming. I have one personal project set up to run mypy as a pre-commit, but I'm not (quite) ready to force that on my coworkers.
kgbcia
same, started using Objects and Classes more to get type hint support
kortex
This article is so hopelessly mis-informed, that it's practically hard to read without screaming "that's not how any of this works!" Most of their main arguments - type hints can be wrong, they can be ignored, and they don't inform you in a useful way about program state because of this - all evaporate as soon as you start using the correct tooling.
My stack is pycharm, mypy, pydantic, sqlalchemy stub, several mypy plugins, black, isort, tabNine completion, and a bunch of custom scripts/Make to automate the common tasks. I'm not even sure I'm on the same planet as the author. I can practically spot incorrect type hints, mismatched types, weakly defined interfaces, from orbit, that's how well they are highlighted. Since every type is known, between pycharm and tabnine I can type 2-5 keys and spit out entire lines of code. And since it's all typed, automatic refactor is a breeze.
I remember the dark ages, before type hints. It was rough. I lived in the REPL. Write 3 lines, run, Exception. Write 3 more lines, run, NoneType error. Ad nauseum. Now I write for hours on end and don't wince when I hit execute. And I don't have to go bug coworkers with questions like "hey what keys are in this dict when we redirect from the oauth endpoint".
Sure, I have my gripes, but it's mostly with the lack of power of the type system, like lack of higher kinded types. But I'm confident it'll get there.
Come to the typed side. We have less mental fatigue and screaming at neverending runtime oopsies.
tonyg
> My stack is pycharm, mypy, pydantic, sqlalchemy stub, several mypy plugins, black, isort, tabNine completion, and a bunch of custom scripts/Make to automate the common tasks.
Christ. What a mess. With a stack like that, it's a stretch to say you're "writing in Python" anymore.
serial_dev
> Most of their main arguments (...) evaporate as soon as you start using the correct tooling.
You silly! Just use these carefully curated list of 28 different tools (that I got to after years of trial and error) and you'll have a somewhat acceptable tooling and type support in Python.
danmur
Anything that can use an LSP server will give you most/all of those benefits. Your favourite editor + LSP, that's the tooling. It's very much worth it.
Spivak
That list is an editor, a type checker, a package to enforce type checks at runtime, 3rd party type annotations for a popular SQL ORM, some plugins for the type checker (probably the pydantic one), a code formatter, an import sorter, and (optionally) a Github co-pilot alternative.
avianlyric
To be fair, I don’t think you need that entire stack. Just using VSCode + pyright (which is a one button install in VSCode) + type stubs for libraries your using without native hints, gets you 99% of the benefits.
If you really want, you can even throw in a little pre-commit hit hook that runs pyright and prevents you from commuting code with type errors.
nerdponx
That's such a disingenuous argument.
First of all, Black and Isort are literally code formatters that have nothing to do with type hints.
Second, the whole point is that you can actually develop powerful dev tooling for Python now, whereas in the past it was difficult or impossible.
john-radio
I think the person you're responding to is just trying to be thorough in case it is helpful to someone else setting up python tooling; the only parts of what they wrote that are relevant to type checking are mypy and pydantic.
rstuart4133
And pydantic isn't really about type checking either. In fact it says that in the first few lines of the it's docs https://pydantic-docs.helpmanual.io/usage/models/ :
> pydantic is primarily a parsing library, not a validation library. Validation is a means to an end: building a model which conforms to the types and constraints provided.
Anyone whose used Typescript or even strong typed languages knew what to expect from Python + mypy. No surprises there. But pydantic's use of type hinting to create a remarkably dense serialisation library, and then fastapi's leveraging that to create REST interfaces along with automagic documentation generation - well that wasn't when I was thinking would happen when I saw the type hinting proposal.
ramraj07
Wow gatekeeping types out of python are we? Of course the language looks different when typing is enforced, let it evolve.
kortex
It's more so "this is my toolchain I use for my own ergonomics, and what you can achieve with the right kit".
Who cares if I'm "writing in python" at all? I'm solving problems. Honestly a good IDE (pycharm or vscode) will give you 80% of the ergonomics.
qayxc
As long as you don't have to rely on 3rd party libraries, sure, type hints are fine.
Many popular libraries, however, don't have type hints or use them inconsistently (looking at you numpy, pytorch, tensorflow, scipy, ...).
Always great fun to see function signatures that use type hints for the parameters and return Any.
So it really depends what kind of code you write - web frameworks might be fine, data science and ML is definitely not.
Mehdi2277
You can write type stubs for other libraries. Over past year my team had a 100k line codebase that we’ve gotten mostly typed. But we still use a lot of other untyped libraries like tensorflow. So in past couple months we’ve started adding type stubs to the parts of the api we use. While tensorflow/numpy/etc are massive libraries most people only use a small subset of api. You can write type stubs for the subset you use. It definitely takes time but I find it very helpful to have useful type checking of tensorflow so I bit bullet and started writing stubs. I explored upstreaming them to type shed but got lazy on dealing with CI issue although I hope to revisit it later.
zhfliz
https://github.com/python/typeshed also provides community maintained stub packages for packages that are lacking upstream type hints
wheelerof4te
Type hints are good if you stick to the std + your own code that mostly relies on std.
Otherwise, don't bother.
Spivak
If you expect type hints to be perfect and completely disallow you to write type unsound code then this is right, if you expect type hints to be useful when developing then this is very wrong. The automatic type hints produced by pyright on completely untyped code are fantastic.
hummingn3rd
I was curious of seeing if pydantic was mentioned, thanks! We use most of the same tooling as you, I'm always interested in improving our tooling, would you be willing to share your mypy plug-ins names or maybe most useful tools for inspiration?
igortg
My thoughts exactly. Obviously, Python Type hints are an attempt to add type-check to a language "after the fact" and keep it optional. So it's unfair to compare that to a static typed language and conclude that it's bad.
I work with large code bases (100K-1M lines) and can't look back to the time we didn't use Type Hints. Would maybe dismiss it only for scripts or very small projects.
ok_dad
I use this very strict config with mypy and it's effectively a typed language. I rarely have any errors in runtime now, at least from type-related problems like assigning 'vals = 3' and later trying to call 'vals.append(4)'. I sometimes use 'Any' because it's basically required for things like loading JSON or retrieving the results from a GET request, but I have several strictness settings that ensure I never pass around wild Any values and I try to keep their use to within a single function or method and always return a well-typed value. Sure it's all in my IDE right now, but if I import 1 or 2 packages suddenly my well-typed Python code starts to act more like a typed language in runtime, too!
disallow_any_unimported = true
disallow_any_expr = true
disallow_any_decorated = false # true if you never plan to use Any
disallow_any_explicit = false # true if you never plan to use Any
disallow_any_generics = false # true if you never plan to use Any
disallow_subclassing_any = true
disallow_untyped_calls = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
no_warn_no_return = true
warn_return_any = true
warn_unreachable = true
strict_equality = true
strict = truestackedinserter
What opensource project is closest to "doing python the right way", in your opinion?
I'm in "python is not suitable for production" camp, would be happy to change my mind.
> I remember the dark ages, before type hints. It was rough. I lived in the REPL. Write 3 lines, run, Exception. Write 3 more lines, run, NoneType error.
That's why I switched (back) to typed compiled languages and never went back.
nine_zeros
Python types just work. Just use these 30 libraries, spice up your IDE with settings and voila /s.
This is garbage and moving backwards as a programmer. I loved python when it was easy to use with any editor and with few plugins. Now I'm just going to switch to a statically typed language where I don't have to install a zillion plugins to get first class typing.
cjalmeida
As many already said, 99% is just a modern IDE with a one-click-install plugin.
Also, I don't know the grip with productivity tools.
bspammer
I think the difference is largely a culture problem. Most python devs think of typing as a “nice to have” extra, whereas someone using a typed-at-compile-time language see it as an essential feature of good code. You can see the results of this in the fact that a lot of popular 3rd party python libraries still don’t have type hints.
Even with typing, how often do you see Any in python vs Object in Java?
nine_zeros
But a language relying so heavily on IDE means that the language is severely lacking.
People not using said "modern" IDEs are just stuck out of using the language. Is this progress? Even with modern IDEs, there is a difference between what different IDEs do. So now the language is stuck with different IDEs doing different things. Is the progress?
It also means that anyone developing a future IDE can no longer support these languages without support. Is this progress?
God forbid someone tries python on a new architecture where new the entire complex toolchain is not going to be available. They just can't use this language. Is this progress?
Spivak
Python type annotations in the language are nothing more then allowing you put any Python expression in the "type slot" and have it be syntatically valid. They don't do anything. All the functionality is 3rd party packages taking advantage of their existence.
nine_zeros
My codebase is more complex than str, text, dict, int, bool.
So developers are creating complex custom types with union[custom_type_1, custom_type_2] where each custom type could have more unions of other custom types. These custom types then get imported everywhere. It is utter garbage.
wheelerof4te
This is the crust of the problem.
Python type-hints bind you to a lot of other sister projects of Python, which makes you question if they are even official addition to the language.
I think that's also what the OP's article tries to convey.
twelve40
Definitely types, but it's still weaker than in other languages. Just the other day found a maddening bug, where Django ORM query accepted `date` type comparison against a datetime field, and of course that resulted in crazy things in production. Other languages would have caught this compile-time.
jsmeaton
Adding type hints to Python has increased my productivity by at least a factor of 10. They allow you to reason about code in a local function without having to track back up dozens of call sites to ensure you're getting what you think you're getting. That alone is worth the price of admission. Both when editing code or reviewing someone else's. It's fantastic, particularly in a very large code base.
The editor experience only increases that factor as you can navigate a lot more freely. Even for small code bases type hints are a revelation. Knowing if something is nullable or not, alone, has caught 100's of bugs in waiting.
Most of the complaints here feel like attacking a strawman, particularly the ones arguing what's the point if you're not running mypy. Run mypy! It's like saying what's the point of writing unit tests if you don't run the test suite. For any sufficiently large code base you absolutely should be running mypy with any plugins for your framework of choice added.
For heterogeneous dictionaries use `dict[str, object]` and then you're forced to use the existing duck-typing mechanisms you're used to. Or, you refactor into better types and/or structures.
Is there friction? Sometimes. You don't have to annotate everything. If the cost is too prohibitive for a particular structure or function, ignore it. Sometimes there are mypy bugs that make writing correct code impossible. # type: ignore[reason] that sucker. Don't throw the baby out with the bath water.
We use a flake8 linter to enforce that all new or modified functions at least annotate their return type which makes interacting with functions quite a bit nicer, and encourages all devs to at least do the minimum. Usually you find that most args are appropriately typed at the same time.
eweise
Why not just use a statically type language? I really don't understand why python is so popular outside of a few specialized areas.
Spivak
Because ecosystems are what makes people productive and languages are just a means to access them. Python's ecosystem is absolutely massive and high-quality.
eweise
Most popular languages have massive ecosystems. Don't see Python's being a differentiator except for a few specialized areas. In fact, the project I'm on runs on spark wrapped by python libraries, so really its the scala libraries that provide all the heavy lifting.
michaelt
Ah, but what if I'm a masochist and I want all the verbosity of a statically typed language with none of the benefits?
mbrodersen
Try F#: Strongly typed but 100% inference of types => best of both worlds.
ok123456
It's basically a lisp with better syntax.
tootie
I'm baffled by people loving type hints in python when strongly typed languages have been widely available for so long. This is why people liked java and c#.
coder543
There are way more differences between Python and Java than just "having explicit types". If that was the only difference, your comment would make more sense.
I would even go so far as to say that Java's type system is the very one that left such a bad taste in people's mouth that many people swore off explicitly typed languages for a decade or two. It really was that bad, especially before the last few years. It added a lot of extra boilerplate for minimal practical benefit. People used Java because it was fast, cross-platform, and less likely to go horribly wrong than unmanaged languages like C and C++. I doubt very many people used it because they just loved typing tons of boilerplate or because they thought the type system was good enough to catch tons of errors.
The type systems in Rust, TypeScript, and other more recent languages are far more expressive (and capable of catching real-world bugs) than what Java offered. If you think Java has a good type system... things have come a long way since then. Java gets updated continually, and some people will surely jump at my comment and claim things are great these days, but, no. Java is not up to par, even today. C# has done a better job of keeping up, in my opinion, but even it still lacks extremely useful features like proper Sum Types.
I don't have much experience with Python's type hinting, so I can't comment as directly there.
cocoa19
And many of us prefer simplicity in languages.
When a language like typescript supports too many obscure features, or multiple ways to do the same thing, the curve becomes too steep to be proficient.
jve
> C# has done a better job of keeping up, in my opinion, but even it still lacks extremely useful features like proper Sum Types
Does type pattern matching comes close? https://dotnetfiddle.net/Oz2Qyd
Yeah, it doesn't prevent passing unexpected type as object to Print func and getting runtime exceptions. And returning "object" type isn't helpful if we want to prevent runtime errors. But then again, it can be written to spit out compile time errors: https://dotnetfiddle.net/XfKVaa
deterministic
F# is a great example of a strongly typed language where you almost never have to write type’s explicitly. The best of both worlds IMHO.
smohare
You misunderstand the ecosystem. Many folk program in Python because it’s largely forced upon them. It’s often the easiest language to work with for data science, ML Eng, or data engineering, despite many frameworks actually running in the JVM. It’s simply more accessible. The appeal of Python has not been its provenance or design for over 15 years, but rather the ecosystem.
whiplash451
Most engineers I work with are very happy to work with python. They would not say it is forced upon them. You are free to provide a much better alternative (that does not exist as far as I know).
minism
Same thing happened to the JS world with typescript (though typescript seems more powerful than python type annotations).
Its fine though - I'm glad the culture is trending towards understanding that strong typing saves you time rather than causes you extra time. This is true even for very small and simple programs, and even if you're the only developer.
goatlover
The typing debate has been ongoing for decades, and it's switched back and forth a few times. JS, Python and Ruby all came out in the 90s, after there were plenty of statically typed languages being used at the time. Alan Kay has argued that types are too restrictive and usually don't describe the kind of data the program is really about. Maybe that's not as true for an advanced typing language like Haskell. And maybe Kay had C and Pascal in mind, when he thought of objects as being the basis for a proper dynamic type system.
andrewingram
TypeScript has a more powerful type system, but arguably it's less powerful because the hints are invisible to runtime code. Python doesn't do any runtime type checking by default, but the hints are accessible in code, which unlocks a fair amount of power. For example, Strawberry is a GraphQL server library that allows you to define your GraphQL schema using the same syntax as dataclasses.
aix1
Python is, and has always been, a strongly typed language.
Do you mean "statically typed"?
jsmeaton
I really like C# and have done a bunch of work with it in the past! But I also love Python for reasons other than just static vs dynamic typing.
To be honest, I think a lot of Python [or arbitrary dynamic language] developers can be quite sloppy, throwing dictionaries around everywhere because it's easy and leading to some really hard to read code. Type hints guide people away from that sloppiness while still allowing access to most of the features that make dynamic languages so useful and expressive.
It's a best-of-both-worlds situation and I'm here for it.
grumpyprole
Many languages also demonstrated Python-like succinctness with strong static types and type inference. E.g. OCaml dates back to the 90's and ML dates back to the 70's.
cttet
If the type system is not powerful enough it will be extra works just to get around the inadequacy.. If c# have some support of sum type and structural subtyping like Python I will be using that... I mean even Haskell does not have proper records.. Python is not definitely the best, but quite nice among the current options.
vegai_
There's a difference between putting on your own seatbelt and the car forcefully tying you into the chair every time you sit.
neves
Nice. How do you force types just in the new code?
Mehdi2277
My team started with an untyped codebase a year ago. I added a rule to CI that every pr that touches files with type errors must decrease the type errors in touched files by at least 5. When we started we had like 30k type errors. A year later and it’s about 5k left. This was a small wrapper script over mypy/pyright that just called them twice once on master and once on your branch to compare error counts.
I did occasionally get pushback on it but stricter checks will be inconvenience at first and I argued it’d help over time. Now that we’ve had it a year it’s pretty well accepted in same way many would accept run formatter.
icedchai
The problem with these sorts of "rules" is they create needless changes outside of the PR's primary focus. The PR's purpose is to do X: add a feature, fix a bug, whatever. Now because I've done X, you want me to do Y, a set of totally unrelated changes that ultimately are a distraction.
Too
Mypy can be configured to ignore modules based on their names.
Though honestly I think it’s worth it to just bite the bullet and spend a full day or two to go through and fix every single error. After this you will have a much easier time navigating code base. I mean, your “new code” is probably modifications to your old code or calling your old code right? So you want to have at least the boundary between new and old typed. It’s quite satisfying anyway, as you will likely learn and discover tons of bugs on the way.
necovek
With any large legacy Python codebase, that's usually a month-long undertaking for one engineer, if not longer!
Python by design encourages duck typing, which means that you'll have plenty of places that simply take multiple different types, and just slapping a union is usually non-trival and not very beneficial either (eg. you'd be randomly patching things with fakes and mocks in tests).
If you think it's a single day job, I am sure my company (and plenty others) would pay you gladly your single daily rate to get our codebase migrated in a day :-D
LtWorf_
> day or two
Your code must be quite small.
icedchai
For any large code base, it'll go on for months, since it gets done between higher priority work. You'll discover some problems, sure. But if they were really important, you'd have hit them earlier. My experience is these sorts of efforts create a lot of busy work and not much more.
jsmeaton
You can pipe the diff into specific flake8 checkers. Disable the check by default so it's not run over the entire codebase, and have a separate line for running specific checks on just the diff. Eg:
git diff -U0 --relative origin/master... | flake8 --diff --select <custom-codes>
I'll leave the flake8 plugin as an exercise to the reader as ours is intertwined with code I can't share right now.
orf
The same (and only) way you’d reasonably be able to do it in any other language that uses a series of text files for an input.
File by file.
chombier
I generally force it on a per-file basis by adding
# mypy: disallow-untyped-defs
nyuszika7h
You can use flakehell for that.
ledauphin
this article is so full of... not-very-educated thoughts on gradual typing in a dynamic language that it's hard to know where to start.
A few concrete criticisms:
1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime. However, good news! Python has always been strongly typed, and _does_ enforce types at runtime!
2. A number of the examples given would be _impossible_ to statically type in most compiled languages (those without dependent types). It's hard to know where to go with a critique saying that you can't fully represent heterogeneous values in a dict, given that you can't do this _at all_ in many statically compiled languages.
It seems to fall back on saying "use of the type system requires too much discipline to be useful". This might be an interesting criticism on its own; however many current users of Python can say, from experience, that the required discipline is not a high enough hurdle that it prevents us from using types consistently and correctly.
There is plenty to critique about Python's static typing, but this article would have been better titled "I'm confused about Python's static type system and want somebody to show me how it's actually used in real-world scenarios".
civilized
> not-very-educated thoughts on gradual typing
This seems like an inaccurate and condescending put-down. What is the definition of "educated" in this context? Is there a book or generally well-known resource?
The author is clearly thoughtful and curious. Near the top of the post he says "what I’m hoping for is that someone will come along and tell me that I got it all wrong. “When you do it as follows, the system works: $explanation” Because, you know, I’d really like to have static typing in Python."
> 1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime. However, good news! Python has always been strongly typed, and _does_ enforce types at runtime!
And yet, the author points out that this is a working program:
foo: int = 'hello'
print(foo)
ImprobableTruth
I think the top comment was less polite than it should be, but I would echo that the post seems to have a bit of a weird understanding of gradual typing?
The Any type is to allow incremental typing. You start out with dynamic code, and progressively 'typify' it by adding more and more annotations, with Any working as a stopgap where for the moment no valid type exists, until you finally have a fully typed program. Inserting Anys because you're lazy is like casting things to Object in Java because you're lazy, abusing it like that is just incredibly sloppy and bad code.
If someone is really struggling with this, simply use a linting rule to ban Any in 'mature' code.
ZephyrBlu
Like the author said, the weird thing is that it goes both ways. I'm pretty sure if you try to return `any` from a function with an explicit return type in TypeScript you'll get pinged for it.
d0mine
Don't confuse strong typing and static typing.
Python is strongly typed. It respects types at runtime e.g., `"1"+1` causes TypeError. Python is not statically typed.
For comparison, C is statically typed but it is weakly typed e.g., printf function has no idea about actual types of its arguments (in particular, errors in the format arg may produce non-sensical results silently).
ledauphin
This one is self-explanatory.
Can a string be printed? Yes! Then Python is (correctly) allowing typesafe behavior here. Your (incorrect) annotation does not in any way contradict the typesafe-ness of printing a string (or an int). They're both equally printable.
This is just a very, very bad example, plain and simple. It 'looks' bad to a superficial reading, but in practice it demonstrates only what is already known about static typing in Python - it is enforced (or not) separately from the interpreter.
civilized
It seems that your point boils down to "Python enforces types at runtime if you define 'enforce types at runtime' not to include 'enforcing that data you declare to be of a certain type actually is of that type'".
Am I the only one baffled by this way of thinking?
the_af
Agreed. Static typing is usually enforced during the compile stage in other languages. It's specifically meant to catch problems before running the code (hence the "static" in its name).
This said, I find the linting/type-hint-checking stage of Python cumbersome and confusing :(
the_af
I agree with most of your comment, but I think there's a miscommunication problem here.
When Python "enforces types at runtime" this is the actual types, not type hints. Type hints are not a runtime artifact. The "actual type" here is "string", and enforcing it means Python would not allow invalid operations on it without error'ing. A programming language that will let you do basically anything to any value because it doesn't enforce type safety is of course C. Python is safer than C.
AnimalMuppet
Yes and no. C will let you do things like:
*(unsigned long*)0xFFFFFF14 = 0x749235f8;
*0xFFFFFF14 = 0x749235f8;
char* s = "abc";
*(unsigned long*)0xFFFFFF14 = s;
Best of all, all of these are enforced at compile time.
Now, C absolutely will let you convert an int to a pointer, or a char to an int, or an array to a pointer, or... well, it will let you convert lots of things to lots of things. Some of those things are unsafe unless you are quite sure what you're doing.
> A programming language that will let you do basically anything to any value because it doesn't enforce type safety is of course C.
False on several grounds. You can't call something that isn't a function. You can't call a function on something that doesn't have it. You can't use an int as a pointer or an array without casting it. You can't use an int as a dict (not that C has any built-in idea of what a dict is...)
> Python is safer than C.
Depends on what you're doing, and what kinds of errors you are more prone to.
linsomniac
Python can't be said to enforce types at runtime. AFAIK, it doesn't say that. In fact, the original article starts off with "I'm expecting to be able to run a static analyzer", then it goes on to say "This isn't caught at runtime".
Yes, but is it caught at static analyzer run? Yes, I typed it into vi and ran mypy on it and it said "error: Incompatible types in assignment (expression has type "str", variable has type "int")."
But I didn't have to run mypy on it, my editor told me: "Expression of type "Literal['hello']" cannot be assigned to declared type "int"".
Maybe the word "educated" wasn't the right choice in this case, can you offer a better one? I honestly couldn't make it through the entire article, despite several attempts. It is full of inconsistencies and what seem like wilful misunderstandings to justify the authors conclusion.
Particularly as the author starts off saying "run a static analysis tool" as if knowing that's how you go about it, and then saying "it doesn't catch it at runtime".
fer
> Python can't be said to enforce types at runtime.
It does: in form of TypeError exceptions. It doesn't enforce type hints, sure, but that's why they're called hints and not declarations.
woodruffw
> In what reasonable sense can Python be said to "enforce types at runtime" here?
Python is both strongly typed and protocol-typed. In the case of `print(...)`, the protocol is that the received parameter responds to `str(...)` (i.e., has `__str__`).
(What Python isn't is statically typed. But "strong" and "static" are completely different typing dimensions.)
reikonomusha
I think the person you're responding to means to ask
> In what reasonable sense can Python be said to "enforce statically declared types at runtime" here?
which is certainly an extremely pertinent question for a language that allows declaring supposedly static qualities of a program.
sevensor
Python enforces its runtime types at runtime. 'hello' is a perfectly valid argument to print. Python will raise a TypeError if you try to use it to index a list though.
ungawatkt
For 1, there's some unfortunate but important semantics. Python does not enforce type _Hints_ at runtime, as Hints are in many ways fancy comments. But as pointed out, python is a strongly typed language (at least on the sliding scale of strong to weak), and types are known at runtime (call `print(type(var))` to see them). And calling `1 + 'a'` will result in a TypeError exception, unlike say JavaScript.
I believe this is the relavent passage in the article the GP is referencing, which is incorrect if you take "runtime" to mean something like when the line is executed: "Python is a dynamically typed language. By definition, you don’t know the real types of variables at runtime."
It _is_ however correct to say you don't know the real type _before_ runtime, at least python does not.
hermitdev
Personally, I took exception to the following from TFA:
> Python is a dynamically typed language. By definition, you don’t know the real types of variables at runtime. This is a feature.
In "you don’t know the real types of variables at runtime", this is just flat wrong. One doesn't know the real type of a variable until runtime. It seems to me the author maybe has a bit of confusion between weak typing and dynamic typing here.
ModernMech
> "you don’t know the real types of variables at runtime", this is just flat wrong. One doesn't know the real type of a variable until runtime.
I read the sentence as
"you don't know the real types of variable at [the start of] runtime"
"you don't know the real types of variable *during* runtimes"
edgyquant
I get what they mean though. In a typed language you know the type of a variable at compile time. In dynamically typed languages there is nothing to enforce this and a variable can be passed with any type. So there wording may be off but the point stands.
umanwizard
In fact that quote is almost exactly the opposite of the truth. “Dynamic” literally means “at runtime”.
tonyg
> One doesn't know the real type of a variable until runtime.
In the program
def x(f):
return f(3)
If your answer is that `f` has type "function-like", then that's a much weaker kind of a type than even type hints offer, let alone a real type system.
faho
> what is the "real type" of `f` at runtime; say, at the instant just before the call to `f`? How can you tell?
We can tell by looking at what python triggers TypeErrors for.
>If your answer is that `f` has type "function-like", then that's a much weaker kind of a type than even type hints offer, let alone a real type system.
f implements the Callable interface - meaning it's either a function/method or an instance of a class with a `__call__` method.
So you can pass e.g. `print` or an object with a `__call__` method. If you try to do `x(5)` that's an immediate runtime TypeError because `5` can't be called.
f must also accept one int argument. It can have additional optional arguments, and it can also accept other types (e.g. `f("foo")` can also work!), but it must work on one int. The one int can already be optional - the function can also work with zero arguments.
Calling e.g. `"foo".join(5)` fails with a TypeError because 5 isn't Iterable, and calling `"foo".join([1], [2])` fails with a TypeError because it gets more arguments than expected.
cardanome
> 1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime. However, good news! Python has always been strongly typed, and _does_ enforce types at runtime!
So PHP is not a dynamically typed programming language according to you? It enforces type hints at runtime just fine.
There is nothing in the definition of dynamically typed languages that says they can't use type hints to perform runtime checks.
Jach
Was going to say the same for some implementations of Common Lisp, notably SBCL, which by default treats type declarations additionally as assertions. Subject to limitations it can use declared types and infer more types at compile time to produce more optimal assembly and give warnings if it detects type errors, or interestingly warnings if it has a chance to use e.g. a fast assembly add on a fixed sized int if you help narrow its inferred type with an explicit declaration one way or another. But unless it can prove a type always holds, it will also by default check declared types at runtime.
the_af
I can only speak about my own experience:
Most people I know writing Python don't use type hints because they are too much of a hurdle for very little payoff. The tooling that pays attention to type hints is slow as molasses or difficult to use or understand. The type hints themselves are of dubious use.
As a fan and advocate of static typing, I find myself advocating for type hints anyway, but I must agree I often can't reply anything to my coworkers' objections, because Python type hints are truly not that useful.
wronglyprepaid
> The tooling that pays attention to type hints is slow as molasses or difficult to use or understand.
I use mypy daily on big codebases, it is fine, not fast, but fine.
> The type hints themselves are of dubious use.
They tell you when you make type errors, they help you understand what type things should be. The same as in literally every other language with static typing. There is nothing special here, nothing different. python with a static type checker running in strict mode is not fundementally different from Java's static type checking.
> As a fan and advocate of static typing, I find myself advocating for type hints anyway, but I must agree I often can't reply anything to my coworkers' objections, because Python type hints are truly not that useful.
What do they lack that would make them useful?
the_af
> The same as in literally every other language with static typing
No, obviously not the same, otherwise I wouldn't be complaining. They are not even on par with Typescript, which I'm not a fan of either.
> [type hints] tell you when you make type errors
Not according to other comments I seem to be getting here. Other people are arguing type hints are not primarily for checking, but a form of notation for documentation. Seems wasteful, and I wish the Python community and tooling decided instead that they are for actually checking them.
> What do they lack that would make them useful?
Standardized, go-to tools that work in the build pipeline and that catch most errors without taking a long time to do so.
I haven't found mypy to fill these requirements. It's so bad I cannot convince my coworkers to make the effort to write more type hints.
seti0Cha
Have you looked at the tools recently? LSP + Pyright is very fast and plugins exist for many editors. Or maybe it's slow on larger codebases or very large files? I don't have that broad an experience yet, but so far it's been very good.
the_af
Sorry, I wasn't clear: I want command-line tooling for the build pipeline, not for the IDE or any individual developer.
theodorejb
> 1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime.
Counterexample: PHP is a dynamic language which enforces static type declarations at runtime.
dgb23
Fun fact: it will still do type coercion in that scenario without strict mode. And you cannot annotate assignments. Otherwise it’s pretty useful.
ledauphin
Meh. you can do the same thing in Python if you really want - there are dynamic interpreter shims that can do this.
The point is that it is not expected by definition, since by definition, static != runtime.
lolinder
You're assuming that type hints are by definition static-only. There's no particular reason why type hints can't be both statically and dynamically enforceable, as they are in PHP.
dtech
It seems to me like Python needs to get Typescript's capabilities (if Python wants to go further this way of course). It solves all these problems very well, and has no problem with 2 and most of the author's objections.
ledauphin
Typescript does absolutely fantastic type inference, which mypy definitely does not. I think that's a huge advantage for Typescript over mypy, and it's really foundational to the value it provides.
However, even in Typescript it is nontrivial to _annotate_ these complex types, and in most cases it's still possible to do in Python - you just need to commit to using something like dataclasses rather than pretending that what you have is a `dict`.
Basically, to get the most out of static typing in the world of Python, you do have to write more boilerplate than you would in Typescript, and in particular you need to avoid dicts and prefer various sorts of class-based data containers (again, dataclasses, attrs, namedtuples, etc).
oefrha
You can type Python dicts, see typing.TypedDict. It’s still lacking though, totality is all or nothing until py311, that is to say, TypedDict’s are either Required<T> or Partial<T> in TypeScript parlance. And you don’t get Pick<T>, Omit<T>, etc.
eyelidlessness
> However, even in Typescript it is nontrivial to _annotate_ these complex types
The only one which stands out to me as non-trivial in TypeScript is thrown exceptions… and even then only because you can only type them in a catch clause, and only with unknown, and they’re invisible to callers. But then this is why people who would care for checked exceptions quite reasonably just use something like a Result type.
But I agree with your conclusion completely, and it’s basically the same one I came to reading the post.
I was curious about how the current state of things compares to TS, as I may soon have a foray back into Python soon, and would be delighted to bring some static types into the mix. Pretty much as I expected, it’ll probably be more effort/less valuable than TS, but more valuable to me than to the author.
edgyquant
The ability to slowly add typed code to an existing database has been a lifesaver. I always hated JS because of callback hell (and the notions around its community) but since I started working with typescript my mind has been changed. I love it
mjn
> If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime.
I don't think this follows by definition. It's common not to check the static hints at runtime, but a dynamic language could choose to treat them as contracts or assertions. Over in Lisp-land, SBCL will enforce static type declarations at runtime [1], contrary to most other Lisp compilers, at least under the default compilation settings (you can force it to skip the checks by compiling with a low "safety" level). If the compiler can prove that a static type declaration always holds, it will omit the runtime check; otherwise it will compile it into a runtime assertion.
moonchild
> If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime
What definition is that? Raku is a dynamically language with gradual typing which will enforce those types at runtime (if it cannot do so at compile time).
phailhaus
Python's type system is a disappointment, but I don't think this article does a great job of explaining why. My biggest gripes:
1. There is no way to get typing like dataclasses without writing your own mypy plugin. The syntax is simply not expressive enough to do it. This means that if you want to be productive with a library like Pydantic, you also have to add the mypy plugin to your dependencies and add it to your mypy config. Otherwise, no typing.
2. You cannot compose types. There is no way to say "hey this type is the same as this dictionary here, except the keys are all optional." If you're trying to type a RESTful API, your only option is to repeat yourself and carefully keep all your types in sync.
3. To this day, there is still no way to express optional keys. Not "Optional" keys, but keys that can be left out of your dictionary. The closest you can get is this weird hack where you can set `total=False` in your TypedDict, which makes all keys optional.
I really wish Python learned from Typescript, but it's too late at this point. Too many half-baked measures have already made it into the spec.
kirbyfan64sos
> There is no way to get typing like dataclasses without writing your own mypy plugin. The syntax is simply not expressive enough to do it. This means that if you want to be productive with a library like Pydantic, you also have to add the mypy plugin to your dependencies and add it to your mypy config. Otherwise, no typing.
This would be solved by PEP 681, which originates from (and is already used in) pyright: https://peps.python.org/pep-0681/
> 3. To this day, there is still no way to express optional keys. Not "Optional" keys, but keys that can be left out of your dictionary. The closest you can get is this weird hack where you can set `total=False` in your TypedDict, which makes all keys optional.
This is solved by PEP 655, which introduces NotRequired: https://peps.python.org/pep-0655/
It's implemented in mypy 0.930: https://mypy-lang.blogspot.com/2021/12/mypy-0930-released.ht...
> I really wish Python learned from Typescript, but it's too late at this point
Personally, I feel like it's partly just entirely different styles: Python relies on a mix of nominal and duck types, whereas JavaScript is much more focused on structural typing. As a result, there are a lot of things like TypedDict that are absolutely essential for JS to even function but a decent bit less so in idiomatic Python code (keeping a significant amount of dicts around uses much more memory than objects, I recall a PyCon talk explicitly mentioning this too).
meatmanek
Even without NotRequired, you've been able to have some-keys-optional-and-some-keys-required for a long time by putting the required keys in one TypedDict, then adding the optional keys in a subclass with total=False: https://mypy.readthedocs.io/en/stable/more_types.html#mixing...
It's not exactly elegant (hence PEP655), but it works just fine.
ledauphin
I'm a big fan of mypy and static typing as a concrete improvement to Python, but this criticism is so superior to the original article that all I can say is bravo.
Typescript's type system, of course, had and has the advantage of being a clean sheet design with an intermediate compilation step, so the comparison is hardly apples to apples. But there's no denying that I wish frequently that Python could have opted for a more powerful static type system.
goodoldneon
Agreed. Also, Mypy's dict inference is too broad, making TypedDict a little less useful. For example, the following code raises a Mypy error even though `foo` conforms to `Foo`:
import typing
class Foo(typing.TypedDict):
a: int
def do_thing(value: Foo) -> None:
return
foo = {"a": 1}
# error: Argument 1 to "do_thing" has incompatible type "Dict[str, int]"; expected "Foo"
do_thing(foo)
interface Foo {
a: number
}
function doThing(value: Foo): void {
return
}
const foo = { a: 1 }
doThing(foo)bayesian_horse
This is a misunderstanding of the Python type system. In the Typescript example you are depending on "foo" being structurally compatible to the "Foo" interface.
In the Python case, "foo" and "Foo" might look structurally compatible, but they aren't. "Foo" isn't just a type, it's an object of type "class". You can for example print(Foo), you can't console.log(Foo). "foo" is not an instance of the Foo class. What you are looking for is "dict[str, object]".
goodoldneon
TypedDicts fall under structural typing, not nominal typing. So Mypy is cool with this:
import typing
class Foo(typing.TypedDict):
a: int
def do_thing(value: Foo) -> None:
return
foo: Foo = {"a": 1}
do_thing(foo)
RhysU
If you have a dictionary with heterogenous, optional keys then I believe you have a struct-like class. If you have any constraints on the keys beyond a homogeneous domain and a homogeneous range then you have a struct-like class.
phailhaus
Avoiding classes has a couple of benefits:
1. It reduces the overhead to use your interface. Your users don't have to import your struct classes, and can just pass in bare dictionaries. If Python's typing was more effective, these dictionaries would also be type checked to make sure you're calling the library's APIs correctly.
2. It deliberately closes off the option of adding state or initialization logic. Classes let you do way more than just encode a struct, and as a codebase grows they have a tendency to become more complex just because they can (like a gas filling its container). This makes APIs harder to maintain and work with. Sometimes, all you need is "raw structs".
RhysU
You want keyword arguments given these constraints. The type checker will confirm anything you like about those.
Humphrey
While I love the idea behind Python's type hints, they are merely a shadow of the success of TypeScript.
Like the author, I've mostly given up on adding type hints in my Python code. I now only use them when I want to help my IDE find autocomplete suggestions.
Whereas TypeScript was a game changer for JavaScript. I used to hate JavaScript, but somehow TypeScript has become one of my favourite languages! How has the advent of Typing has changed my opinion on these two very similar languages?
- JavaScript without types is a mess, whereas Python comparatively was much better, esp since it does runtime duck type checks.
- Python type hints are much similar to Flow type hints in JS, which I tried, but ditched for the same reasons as Python type hints.
- I was hesitant to try TS's all in approach, cause it was harder to introduce into a project, but after having converted a number of projects to TS, I can see that going all-in is a much better approach than just adding hints as you go.
- TS does checks at many more levels. Eg, if a property is optional or could have different types, it is a syntax error if I don't check the value is valid before use.
- TS does an amazing job of auto-detecting types, so most of the time you don't need to specify types, and it enforces these just as if you declared them.
- TS has reached the critical mass were most popular packages now include type definitions, I very rarely have to add @types/* anymore. This means you get full intelisense on all 3rd packages! I spend a lot less time referring to documentation now!
In hindsight, compiling out types is a great work flow. TypeScript is so good that it has made me enjoy Python less. If there was ever a Python equivalent to TS which reached a critical mass of support, I'd jump all-in in a heartbeat.
Frotag
This is exactly my experience. I've also found that Python's type annotations for even basic stuff like are way clunkier to write.
For example an optional requires a typing.Optional import or a an ugly "| None" instead of a question mark like TS has. And good luck trying to annotate some complex / nested json, you'll need a bazillion intermediary classes.
goodoldneon
There's a subtle difference between TS' question mark and union. The question mark means "this argument is optional", which can be different than "this argument can be undefined".
The following code is valid:
function foo1(value?: number) {}
foo1()
function foo2(value: number | undefined) {}
foo2()
Loeffelmann
I really think the second one shouldn't raise a type error.
This is already invalid:
function foo1(arg1?: number, arg2: number) {}
function foo1(arg1: number | undefined, arg2: number) {}
foo1(undefined,1);
TomBombadildoze
There are many complaints in this discussion about inaccuracies in the post but you've actually captured the essence of the problem--type hints just don't solve the problem well.
My team began using type hints and aggressively applied mypy to new code starting a couple years ago and it _has_ helped immensely. However, we also started doing some work in Typescript around the same time and the difference in developer experience and code quality is pretty clear. Typescript is just a better solution to the problem of adding type information to a dynamic language.
What's especially frustrating is living through the design and early life of asyncio (Guido, you should have adopted the gevent model), then getting excited about typing, only to find it is also not very good. What's _especially_ frustrating is that Typescript existed two years before PEP 484 was published, so the core developers had ample time to seek inspiration elsewhere, and they came up with a sub-standard solution.
Python needs a Typescript analogue, with a compiler that targets vanilla Python.
bayesian_horse
In my opinion type checking in Python is a lot less necessary, which throws off the cost/benefit calculation. And the cost/benefit ratio also varies with the size of the codebase, team and other things.
Let's emphasis the "gradual" part. The more libraries are carrying type annotations, the more helpful IDEs will be and the easier it is to type-check your own code.
hannofcart
To be fair, you're comparing a static typed language that transpiles to JS with a type annotation system for a dynamic lang.
Pylint is closer to Flow than Typescript.
Humphrey
Oh of cause -- and I suspect TS only became so good because it had the freedom to be a new language.
fulafel
The realities of the browser platform have indeed created a unique environment where many nice languages have appeared that compile to JS. Besides TypeScript there's Elm, ReasonML, ClojureScript, and many more.
There are a some competing languages for the Python runtime as well: https://github.com/vindarel/languages-that-compile-to-python
remram
The fact that we can't have a type for JSON really does make it look like a toy.
ImprobableTruth
This is (for now) a limitation of mypy, not of the type system. Pyright and Pyre both can type JSON since they support recursive type aliases.
ok123456
Use pydantic then. Then you get type checking and validation for your json blobs.
simonw
"You don’t know if there really is a tool in place to check them."
Every project I interact with that uses type hints runs mypy as part of CI - and I can see that they're setup to do that by looking at the CI configuration (which these days is usually done using GitHub Actions).
I wouldn't add type hints to a project without also configuring mypy for it.
Diggsey
I work on a project that uses type hints but not mypy. To be fair, we used to use mypy, but there was just too much code that it couldn't check (sqlalchemy models, schematics models, some custom models, essentially lots of different data-modelling types!) Maybe it's improved since then, but it just wasn't mature enough at the time.
However, type annotations still make the code so much easier to work with, because they serve as documentation and are understood by the IDE. When I hit "." I actually get correct autocompletion because of the type annotations. When I want to know what a function returns I don't need to dig through multiple levels of function calls.
The article seems to be suggesting that because the type annotations might be wrong that they are worthless, which seems like a ridiculous argument. Any documentation can be wrong, but clearly having documentation that is 99% accurate is better than not having it at all.
nas
I've found that it's pretty easy to slowly enable mypy checking. You can mark modules with "type:ignore" to make mypy ignore them. Then, you can move to marking individual statements with "type:ignore". Finally, you can start marking modules with "mypy: disallow-any-generics" at the top. That requires that functions are annotated with types.
In a large code base, our team has found type annotations and mypy to be quite helpful. If you are writing a quick-and-dirty script, forget about wasting time with type annotations. However, for long-lived and important code, it seems to be worth adding the types.
Another thing I noticed, if your type annotations are hard to write, e.g. lots "Union", "Optional" or deeply nested type expressions, it's probably a sign that your code is poorly designed. Think about what types you actually want to consume and return. Define new containers using dataclasses or attrs to keep the types simple, if required.
Diggsey
Right - I mean we had mypy passing across the whole project, but it just wasn't detecting any issues because so many of our types had to be ignored because mypy didn't understand that the runtime type of a Column field in SQLAlchemy is not Column, and things like that.
Rather than maintain all of the mypy stubs for no benefit, we decided to just scrap mypy.
plonk
> You can mark modules with "type:ignore" to make mypy ignore them.
If I added MyPy to the CI/CD checks, some coworkers would flood the code with these to not have to think about their typing. Too bad that some libraries like SQLAlchemy make this mandatory sometimes. I'd like the option to ignore these "type:ignore", but the type system is too immature and has to be overridden sometimes.
whimsicalism
I disagree with your last line, especially for Optional (Union in python is basically useless IMO).
Bedon292
There are still a lot of libraries without hints, so mypy has to be told to ignore them in the setup.cfg file. Last I checked, sqlalchemy still isn't so good with mypy. So we have it ignored. But mypy is still great overall.
oefrha
MyPy isn’t the only checker of course, there’s also Microsoft’s Pyright, among others. Pyright is fast and natively integrated in Microsoft’s Pylance VSCode extension, so these days I use Pyright’s type checking in real time even when I can’t be bothered to set up MyPy (no difficult, just not important for, say, <1000 line scripts). I default to strict mode for stuff that’s not too dynamic.
https://github.com/microsoft/pyright
https://marketplace.visualstudio.com/items?itemName=ms-pytho...
sevensor
Precisely. We don't merge unless the result passes mypy.
Bedon292
We use pre-commit and you can't even commit until it passes mypy. It can be a bit frustrating sometimes, but overall it has saved us from a lot of issues.
folkrav
I like the validation scripts being available in a repo so I can run them locally before pushing to CI, but I also often use WIP commits and quick fixups and then rebase before opening the PR, so pre-commit hooks are really annoying to my workflow. I more often than not just do `git commit --no-verify` or simply delete the git hook in `.git`, then just run it myself before pushing. Anyway CI will catch it, so I don't see much value in forcing it on individual machines.
evrydayhustling
Yes. The reason type checking is separate from the interpreter is that python is an ecosystem, not a language. Even if you like static typing, 99% of projects benefit from other code that lacks typing. Type hinting puts you in control of where types are enforced while allowing you to use duck-typed code. The cost is that you have an extra step in your build chain - but CI, IDEs and other automation all but negate that cost.
chis
Only facts here. I think most large codebases eventually see type hints drift away from reality as individual contributors are more incentivized to hack in `Any` types to make things compile instead of typing every line properly. This is especially common for handling data objects which come in over the network - often they can have a couple different types but people just type it as one thing for simplicity.
Overall I do think type hints are worth it though, for maybe two benefits. They force you to look at the ridiculous types you are using, like `List[Union[None, str, List[Dict[str, str]]]]` which is the sort of thing that happens in codebases all the time. It adds just enough friction to push people to make explicit dataclasses or simplify function returns, which is good. Secondly they help with tracking functions which return None, which is a pain when following callstacks in big codebases.
bb88
My understanding was large python code bases (think Google) have a large problem. Someone makes a code change and suddenly it becomes difficult to find the scope of type errors in their monorepo. That was the driving force IIRC.
That said, I think pytype makes a lot of sense since it infers types from code, which you can edit by hand and then merge back into the python file when your code is stable.
mountainriver
Yup this is the biggest benefit. Working on a large untyped code base is about as bad as it gets
goodoldneon
You can disallow `Any` with the `--disallow-any-explicit` option
majormajor
> But as a general rule: You don’t know if there really is a tool in place to check them. You especially don’t know this when you join a new project, but it’s also a little hard to tell afterwards. Did the mypy task in your build pipeline silently break? Maybe mypy is misconfigured? Maybe it spits out errors but does not cause build jobs to actually fail? What if mypy has a bug? What if mypy is not even complete and doesn’t cover all cases?
This seems like an extreme overstatement and a sign of a very weird team environment.
Are you not running your linting and validation locally?
Heck, when dealing with some poorly-typed external code, I run mypy manually to help me figure out what the stupid types I need to assign even are. This is far from ideal, but in lieu of a better solution, it works for me.
I just find the idea that people would be adopting typed Python and then completely ignoring the type-related tooling a bit wild.
And sure, libraries can have bugs, you can open yourself up to big holes with Any... and in cases where I'm super worried about things like that, I'm generally not using Python. But I'd rather use Python with type hints + use the tooling in a sane way then not use the type hints at all.
just-ok
Despite this:
> Even if the Python runtime did check all the type hints at runtime, then it would still be too late. I don’t want a fancy type exception at runtime. That already exists (most of the time). I want to know about type mismatches in advance.
You should check out Typeguard [1], which lets you add a @type_checked decorator to anything and get runtime type checking that's far better than e.g. a random blowup when you split() on an integer. It does introduce a significant amount of overhead, but it's still useful during development alongside type hints to avoid some of the pitfalls from this article.
In another vein, Any should definitely be used sparingly. For the foo() function outlined in the article, Union[str, int] offers more precision. In fact, you could even argue that using Any is a code smell.
Overall, though, I agree with most of the sentiment in this article. I find myself using type hints more as documentation than anything else, since their true ability to prevent type-related runtime bugs is very limited. Documentation has the same qualms the author outlines, anyway:
> You can not be sure that they are correct. [...] They waste mental energy when reading code, they create new maintenance burdens, and they are potentially deceiving: You cannot trust them.
But they're still slightly better than docstrings...
[1]: https://typeguard.readthedocs.io/en/latest/userguide.html
riyadparvez
There's also bear-type for runtime type checking: https://github.com/beartype/beartype.
pydry
I've been perplexed as to why this hasnt been built in to the python runtime since day 1.
bayesian_horse
Python was first released in 1991.
captainmuon
Do you know if there is a way to do runtime type checking in the whole program with typeguard, beartype or something else? As far as I know you have to go through and add decorators manually. Typeguard had a profiler hook that almost got it right, but is being removed. Ideally I would want to say 'python3 -m typecheck myprogram.py' and it would run typechecking everything in my code (but maybe not in library code).
just-ok
I believe you're describing MyPy? http://mypy-lang.org/
captainmuon
MyPy is great but it checks "offline" or "at compile time" if you know what I mean. It is like a super linter, and doesn't actually run the program (AFAIK).
What I'm looking for is more like a debugger or profiler. It actually runs the program, and then reports if at any time the type annotations deviate from the actual types. (I guess the general case is too costly - think of typechecking a huge list - but for most cases it should be possible.) There are runtime type checkers for python, but the all require modifying the code, and they don't work properly on the main module.
raverbashing
Type hints work fine. They are hints (that are checkable through tools)
> But as a general rule: You don’t know if there really is a tool in place to check them.
Cool. So you just go and do stupid stuff if no one is looking?
Check it yourself. They're probably there for a reason. If you think they're not needed, then sure, take them out.
> Most of this turned out to be wrong and it threw me off the track during my debugging session.
Here's an idea: Then why don't you fix it. If I see code that's wrong or bad, I go and fix it. It's your responsibility as well
Now maybe try acting like they're anything but type hints (and I'm very glad Guido insisted it is optional and flexible - the last thing the world needed is for Python to get consumed by Java type checking pedantry where the compiler needs you to annotate every single thing)
And every time I do something in Python that would be "impossible" in Java I'm glad I'm not bound by the small-minded type pedantry of Java.
wvenable
Can't you just type something as `object` in Java and make anything possible?
raverbashing
It might be possible for some cases, yes. But not for function pointers. And you don't get duck typing.
POiNTx
Don't let perfect be the enemy of good. Type hints are usefull especially in large codebases. I find myself using them with minimal effort and they can catch bugs that otherwise wouldn't be catched.
mindwok
Exactly. The article has some valid criticisms of the weirdness of type hinting, but type hinting is a pretty large paradigm shift added to a 20+ year old language. Yes it will have shortcomings, but in my opinion has made working with large Python codebases actually possible. It’s a good thing.
goodoldneon
Totally agree. Mypy isn't perfect but it's so much better than no type annotations
Get the top HN stories in your inbox every day.
Honestly, I've recently written a few 300-500 line programs in Python using type hints and I'm never going back. And I'm not even using mypy often, if ever.
My editor now has a fairly deep understanding of my code, and can tell me of all sorts of surprising errors I'm making before I run the code. There have been a few times where I found an error, went into the editor and saw I had missed a error message about that line. Shout out: Using LunarVim with LSP and TreeSitter.
The other thing I'm enjoying is that libraries can use them to make things happen more automatically. I believe it was Typer (the CLI argument parsing library) where if you declare an argument to a function as "files: List[Path]", it understands that the argument will take one or more files, or "-" to mean stdin. If you just say "file: Path", it understands it is a singular file.
I was curious about type hints when they first came out, wasn't really expecting to use them but they seemed cool, but now I'm totally sold.