Get the top HN stories in your inbox every day.
dahfizz
bb88
More importantly is the ability to subclass exceptions, so it doesn't break client code.
class MyBaseException(Exception):
...
class MyExistingException(MyBaseException):
...
class MyBrandSpankingNewException(MyBaseException):
...
tialaramex
This design claims MyBrandSpankingNewException was:
* So different we just couldn't represent it as MyExistingException and yet
* So similar that it's fine to just treat it as MyBaseException everywhere
In my experience everywhere I see this pattern all we're doing with MyBaseException is giving it to a human, via logging, stderr, or whatever, which means this is a lot of complexity to just move some text.
Although this strategy didn't formally "break client code" in practice if it matters the code will need to be rewritten to actually handle MyBrandSpankingNewException.
d0mine
Usually a derived exception is a special case of the base exception e.g., OSError -> FileNotFoundError. Sometimes you want to handle specific error differently and sometimes you don't care. btw, it may be the reason to create a new subclass, to distinguish between cases when you do care.
derefr
I don't think typed errors + isinstance checks are really the alternative.
As of Python 3.10, I can just write the same Result-monad-ish destructuring-pattern-matching code in Python that I do in e.g. Elixir:
def foo():
if whatever:
return (True, some_data)
else:
return (False, some_error)
def use_foo():
match foo():
case (True, good_data):
...
case (False, ExpectedErr() as err):
...
case (False, any_unexpected_err):
...
A great example would be an HTTP request-response:
match requests.get(...):
case Response(status_code=code) if code >= 300 and code < 400:
# do a redirect
(I do wish that the default for `match` was to blow up with a ValueError if you run off the end, rather than being a no-op, though. As it stands, you have to explicitly add `case _: raise ValueError(...)` clauses if you want to be strict.)
bb88
The problem is the overloaded "some_data" / "some_error" return value. It's arguably worse semantics than just using exceptions.
try:
data = foo()
except SpecificException as err:
...
except BaseException as err:
...
except Exception as unhandled_exception:
...
Further:
return (False, some_error)
raise NoBarError(f"Bar wasn't found in file {f}")
Speaking of requests I actually like the requests API here. It always gives you a response object, but allows you to do this:
resp = requests.get(...)
resp.raise_for_status() # raises an exception on 400 and above.
Further it gives you:
resp = requests.get(...)
if resp.ok:
derefr
> The problem is the overloaded "some_data" / "some_error" return value.
My point was that it's not "overloaded", because tuples returned for the purposes of matching aren't product-types (i.e. essentially objects but with unnamed positional fields, where the same position has the same meaning regardless of the other positions' values) but rather "mutually-exclusive-pattern-clause-matching conventional sum types" — the pure-data equivalent of the Result monad's `Left(Data) | Right(Error)`. Think Erlang's `{ok, Data} | {error, Error}`.
With tagged tuples, you're not supposed to attempt to work with the tuple's fields independent of the whole — i.e. doing tuple[1] is a design smell, because it works regardless of whether you've got the tuple you expected or not. Rather, you're supposed to recognize the pattern that the tuple represents, including the tuple's "shape", and any specific literal tags; and then destructure the result into variables only if the pattern fully matches. Tuples and pattern-matching go together. (And it's kind of weird that Python got one long before the other.)
Note that there's no real difference between using tuples here, and using plain data-carrier classes. A type-tag as a positional tuple parameter is not trying to do anything different than an object with an explicit type is; tagged tuples just avoid needing to create little bespoke data-carrier classes for every possible little returning-an-enum-of-patterns situation that crops up a thousand different places in a codebase.
> Would be arguably better to write: [raise the error]
Why? Just because a piece of data "represents an error", doesn't mean that it's a programming/logic/operational/runtime error in the semantics of your system. It's just data you're carrying around, that represents a certain condition. Maybe a condition you're modelling in some other system. Maybe a condition that's equally common to the "data" path.
Picture, for example, implementing an HTTP caching gateway, like Varnish. Receiving a 4xx/5xx response from the origin isn't an error for Varnish — rather, it's an entirely expected condition. An "HTTPErrorResponse" is just a type of data that Varnish works with — caching it, serving it, etc.
I recently wrote a Python function called is_domain_trustworthy. It returns True | (False, reason_class, reason_detail). In theory, this function could raise these as errors... but it's not a validation function; it's a predicate. It's answering a question. It wants to answer that question with a yes or no; it's just that in the False case, it also has some extra detail it can pass along about why it decided the domain wasn't trustworthy, which can be used for debugging / audit logging. This "yes or no-and-why" answer to the question is just data. That computed data gets put into a database, even.
You can certainly build a validation function on top of this predicate. You'd do so by writing a pattern-match, where one or more arms of the pattern-match raises an exception.
But personally, I find it extremely useful to have the "plain" version of the function not raise, but just give me data. More flexible in code; much simpler to work with in the Python REPL; etc.
> resp.raise_for_status(); resp.ok
A difference with these, is that these are fundamentally examples of a library author trying to predict what the library's users might want to treat as semantic error conditions.
Mutually-exclusive pattern matching, on the other hand, doesn't require that the data is modelled to "bake in" any knowledge of conditions users might care about. You can arbitrarily group patterns together. For an example:
match resp:
case Response(status_code=code) as long_resp if code in [200, 500, 502, 503, 504]:
# these requests take a nontrivial amount of time, so capture the request in a histogram
case _:
pass
dahfizz
> I don't think typed errors + isinstance checks are really the alternative.
It's the alternative proposed by the article, because the code from the article is 5 years old. The author's claim is that calling isinstance is better than exceptions.
Pattern matching makes error objects nicer, but still doesn't make them compelling for the general case.
There are cases, like HTTP requests, where it makes sense to embed status in the response object. But, critically, every http response has both a status and some content. Every response object is the same type and can be treated in the same way.
The article's proposal is to return either a string, or one of a handful of error objects.
I will grant that some APIs are nicer when you embed status information in the response type. That is a special case. Preferring error objects over exceptions in the general case is wrong, IMO.
derefr
I think things become a lot clearer when you unpack which APIs those "some APIs" are.
The case where sum-typed return types (because that's what we're really talking about here) truly shine, isn't in handling "errors" in the sense of invalidated preconditions that you need the caller to handle.
Rather, the case where sum-typed return types make sense, is in handling condition states more generally.
When I say "state" here, think "finite-state machine." Or actually, maybe think "pushdown automata." Imagine a sum type representing the state of a door, like:
Open | Closed(door_tag: DoorTag() | None) | ClosedAndLocked(required_key_bidding)
Certainly, you could write another function door.open() † that does throw an exception if the door cannot be opened. It might also raise a DoorAlreadyOpenException. But this function makes more sense to implement on top of the first function, than the other way around. Door.try_open is more core to how the door models its state and reacts to input, than Door.open is.
† In other languages, the naming-complexity expectation would be reversed — the non-raising variant would be `Door.open`, while the raising variant would be something like `Door.open!`.
NovemberWhiskey
Yeah; this proposal is pretty painful-looking to me, as a Python programmer.
How do you do finally logic with this pattern?
angarg12
Doesn't Python 3.10 add pattern matching? [1]. Seems to make this much more palatable. Specifically look at [2].
[1] https://peps.python.org/pep-0636/ [2] https://peps.python.org/pep-0636/#adding-a-ui-matching-objec...
dahfizz
Yes it does! It's such a nice addition to the language. It does make error objects more palatable, but I still don't see the benefit of error objects over exceptions. You still lose stack traces, and a universal way to get error messages. You're still taking a risk that the caller will miss a case and not handle the error at all, thereby shrinking the "pit of success".
Also, FWIW, the code in question is 5 years old. It seems to be more of an idealogical question than a pragmatic one.
tialaramex
Exceptions come with program flow. Didn't need/ want that flow? Too bad, you get it anyway and must re-structure your program accordingly.
Suppose there are two consumers of my function F, which usually returns a Digit but sometimes instead has a problem. Should F throw ProblemException or should its return type be a Sum Type with both Digit and Problem ?
Alice uses F in some hot code that doesn't immediately care about the problem or the digits, while Bob is only using F in an edge case and needs the result immediately.
For Alice, needing to try F and handle the exceptions (which Alice doesn't ever care about immediately) is annoying. For Bob it works fine. But, Bob can also write code to handle the Problem next to his code for handling the Digit, we were forcing Alice to handle ProblemException, but Bob can choose where to do something with Problem as it suits him.
CaptainNegative
There are legitimate cases in Python where you want the equivalent of "On Error GoTo Next", which currently requires a blowup of one statement into four (try, f(), except, pass). Returning an error can simplify such cases.
Sure, ignoring exceptions can be (horrid) code smell, but in low-stakes contexts it's not necessarily bad code. You see this especially often in the context of one-off Jupyter notebooks that data scientists like writing, where quick iteration, flexibility, and ease of reading is more important than making robust, production-ready code.
Did my text scraper fail because of a network error or was it just some weird unicode character that broke it? Don't care. I just want some collection of simple english text to train my language model, and there's more than enough else on the internet for me to fuss about corner cases.
NovemberWhiskey
>which currently requires a blowup of one statement into four (try, f(), except, pass).
Or ...
with suppress(Exception):
flaky_thing()
przemub
That's nice, thanks for sharing that.
dragonwriter
> With error objects, you're going to fall through a bunch of isinstance checks and continue on with a result object which isn't the type you think it is.
If you are doing isinstance checks (or pattern matching or something else equivalent) for separating error/success returns where, as in Python, you don't have exhaustiveness checks, your default case should be treating the result as an unknown error, not a success.
Also, this error can be avoided on the called-function side by using a common type (concrete or root) for errors. Using a wrapper for errors, as with a Result type (whether or not successes are also wrapped as they would be with a result) achieves this, for instance.
dahfizz
Yes, having a default case that throws an Unknown Error is the correct thing to do here. But that is still less ergonomic. For an unhandled exception type, the person creating the exception can attach a helpful error message. For an unhandled exception object, you can't do any better than "An Unexpected Error Occurred".
You can do a little better by having multiple error subclasses. But at that point you're literally reinventing exceptions, but with a greater onus of the caller to do the right thing. This seems to go against the "pit of success" idea from the article.
There's no way for the developer to accidentally miss an exception, or misunderstand the subtleties of the error inheritance structure.
kstrauser
In some API code, we adopted and use the heck of out the pattern of making Exception subclasses that mapped to HTTP error codes, like:
class NotFoundError(ClientError):
"""Not Found."""
status = 404
raise NotFoundError(f"The requested object ID {object_id} doesn't exist.")
def handle(request):
validate_signature(request)
check_authorization(request)
person = fetch_from_database(request.object_id)
return {"first_name": person.first_name}
Compare and contrast with:
def handle(request):
if validate_signature(request):
if check_authorization(request):
person = fetch_from_database(request.object_id)
if person:
return {"first_name": person.first_name}
else:
return 404
else:
return 403
else:
return 400
eddd-ddde
What about actually using early returns ? I feel like the implicitness of exceptions is more of a downside than an advantage
def handle(request):
if !validate_signature(request):
return 400
if !check_authorization(request):
return 403
person = fetch_from_database(request.object_id)
if !person:
return 404
return { "first_name":person.first_name }kstrauser
That's twice the code for the same result. Also, `validate_signature` itself might have lots of similar code, like
def validate_signature(request):
signature = sig_from_request(request)
payload = payload_from_request(request)
check_signature(payload, signature)
Yeah, you can do all of that with passing around error objects. A lot of people writing Go do that every day, so obviously it works and it's not overwhelmingly tedious. I highly prefer the Python style of EAFP though.
undefined
yccs27
> That's twice the code for the same result.
Only if you ignore that the exception-throwing solution _needs_ comments like `# this might throw a 404` to understand the control flow. The early-return solution is what I'd call self-documenting code.
the_gipsy
The most ergonomic error handling is a Result<Value,Error> with some syntax sugar to get-value-or-return-early-error like rust's ? operator.
zmmmmm
One thing I miss when the error object pattern is used in preference to exceptions is the stack trace.
With an exception, you get a detailed log of the full stack that led to the error. With an error object, I have to reverse engineer out all the places in the code that could potentially return that type of error. In a perfect world, you don't need to know that, and all the information you need is in the error object. But we don't live in a perfect world and very often the stack trace is the main lead you have to debug some crazy thing happening in production.
akdor1154
Yes i have hit this too.
My planned goto approach if I was on a prod app instead of a toy, is to use something like `result`, but with the `Err` variant modded to generate an internal traceback whenever it's constructed.
With that said i haven't ever done the above - anyone done something similar? Or approached this a different way?
bb88
Given enough digging around in the python runtime system I'm sure you can find the current stack trace and shove that in an error class.
But now it's a lesser error than an exception. Any exception while processing your error will cause the stack to unwind and drop the error variable with the stack trace inside of it.
Now if you raise an exception while in the handling of an exception, python will keep around both stack traces.
bb88
I personally like requests api.
Requests gives you a response, but let's you raise an exception if an error case was caught (404, 500, etc)
And it's only one type (Response) and doesn't force the user into using isinstance to figure out if it succeeded or not.
g_delgado14
Yessss... I'm a big advocate of using error objects rather than throwing exceptions. I think the industry over-uses throwing exceptions and it's basically turned into the new GOTO statement.
I wrote an article about this as well but for TypeScript [0] and I've got a relatively popular npm package that does what I mention in the article [1]
---
[0] - https://gdelgado.ca/type-safe-error-handling-in-typescript.h...
Supermancho
> I think the industry over-uses throwing exceptions and it's basically turned into the new GOTO
This is a misunderstanding of what's bad about GOTO versus jumping to a label. eg Functions are labels you GOTO. The difference between a GOTO instruction and a Function call, for the purposes of differentiating the methodology of code routing, is that a Function is part of a stack that returns in a serial fashion. ie Each element of the stack returns to the parent before disappearing, which can be though of as "unwinding". An exception unwinds a Function stack without restriction (eg no consideration of signatures like args/returns), until it is caught in that stack, making them conceptually and practically different from GOTOs.
throwaway290
I'm not surprised. TS exceptions feel limiting and awkward and I don't like using them, compared with Python's. Add to that, TS's powerful typing makes it easier to precisely type return values compared to Python's annotations, but still you cannot type what function throws in TS.
In Go, I tend to use the return error pattern. In Python, I raise and try. With TS, I do both (but slightly more confident with returning errors).
IMO, rule of thumb: try not to surprise the developer after you, who most likely will be hired for knowing this language not this project (note: that developer may be future yourself). When in Rome...
g_delgado14
... do as the Roman's do! Agreed :)
int_19h
It really depends on the language. In Python, exceptions are idiomatic even for non-fatal failures, the runtime is optimized for that performance-wise, and it's what API clients expect.
Zababa
Interesting, I've seen the same discussion but applied to OCaml and with an option type vs an exception. For example, the module List has both List.nth that raises Failure if the list is too short, and List.nth_opt that returns None of the list is too short. One aspect that I've seen mentioned about OCaml and not with this article is performance. An option will trigger an allocation, and will generay be slower than an exception. One the other hand, the option approach is safer. I'm not sure if that's true with Python too though.
g_delgado14
Performance cost is a legitimate concern but one that should be considered in relative terms.
e.g. yes there are more allocations, but in what context is the code being used? Are we building gigabyte-scale real-time systems? no? ok then let's maybe not worry about performance right now and instead optimize for intuition, maintainability and readability. Obviously while at the same time monitoring CPU and memory consumption of our applications to help us determine when we might want to look to optimize our code.
dahfizz
> then let's maybe not worry about performance right now and instead optimize for intuition, maintainability and readability.
I would argue that throwing exceptions is the intuitive and readable thing for Python. Having to call isinstance for all possible return types of a function is not the "pythonic" way.
dahfizz
That's interesting - coming from C++, exceptions are much slower than returning -1 on error. I suppose this is because C++ doesn't need to allocate a Option.
int_19h
Throwing an exception is much slower, but that only happens on error. With option types, you still have to create and return an instance even if successful, and if that causes a heap allocation, you're worse off than exceptions if the success case is much more likely.
That said, it's really an OCaml implementation deficiency - std::optional in C++ is not heap-allocated, for example (although there's still some overhead due to sizeof being larger & possibly not fitting in a single register).
On the other hand, in (C)Python, exceptions behave more or less like in C++ semantically, but under the hood the VM implements it all as error return codes and thread-local "current exception" pointer, which is closer to std::option perf-wise than it is to real exceptions. Hence why performance is rarely a consideration when deciding whether to throw or return in Python, and why exceptions are so idiomatic there.
tbrlpld
Has anyone tried this: https://returns.readthedocs.io/en/latest/pages/result.html
yccs27
One place where you are pretty much forced to use error objects is queue-based concurrency.
In my case, a seperate thread listened for network events, parsed them and put the results on a queue for the main thread to consume. For parsing errors, I needed to put error objects onto the queue instead of terminating the whole thread with an exception.
In this sense, error objects generalize much better to cases where you don't have an usual function-call structure.
akdor1154
++, i normally use the `result` lib to wrap good/error return values, but your approach without it honestly doesn't lose that much.
I would still use `result` for bigger projects or if I was dubious that consumer would do the right thing though.
Get the top HN stories in your inbox every day.
> If you instead raise exceptions, you are immediately forcing the calling code into a special control flow structure, namely the try/except dance, which can be inconvenient.
I fail to see how a try/catch is more inconvenient than a handful of isinstance calls.
What happens when you want to add a new error case? With exceptions, the exception will always be thrown. With error objects, you're going to fall through a bunch of isinstance checks and continue on with a result object which isn't the type you think it is. Then, some time later, youre going to try and reference a field that doesn't exist. This is the kind of thing which is a nightmare to debug.
I do appreciate the author's point that error objects are nice in functional programming languages. But using them in Python because you like using them in fp is not a good justification.