Uber backup driver indicted in 2018 self-driving crash that killed woman

I’m in complete agreement with your assessment on that. If the processes I mentioned above weren’t in place and someone drowned while the lifeguard was watching a video or reading a magazine, they should be held to account. I’m remiss to completely remove individual agency from scenarios like this. The processes just help make those failure cases less accessible because you get fired before it happens.

That’s really the fundamental point, these processes exist because those failure cases are understood, acknowledged, and chosen to be managed diligently.

SOYA summer was a lot of fun, though an office job still qualifies for the acronym even if you don’t get a great tan and have a lot less fun, heh.

> I think the key difference is the backup driver was watching a video on her phone when the accident happened.

Blaming the individual would not solve the problem. The question that the parent comment tries to answer is "what do we do so the backup driver does not watch his phone?".

That would save lives. To blame the individual and move on will kill more people in the future.

Indeed. But to me the distinction between your experience and the driver's is that you were supervised and compliance was checked. If no-one was (at least occasionally) making sure the driver was always focusing on the road ahead, it was inevitable that the driver would not think the rule was that serious, and lapses were bound to occur.

The point is to not place someone in a position where their attention would likely wander (for example not having active shifts over 30 minutes in the case of a lifeguard).

Safety is about proactively defending against human nature, not blaming people after the fact.

But would that really have made a difference if that person would have starred straight forward instead.

If you don't expect something to happen reaction times can be really slow.

Which is also how it differs from the lifeguard job, there you always expect that from time to time something will happen, but as far as I can tell many people would not expect this when they use a self-driving car. Because you know, the car is already meant to be able to handle this. Or at least this is how many people think about self driving cars I think.

Not to be pedantic but the indictment is 'Negligent Homicide'. I believe this is involuntary manslaughter. The reason I bring this up is because I see people misuse the term 'murder' on HN often, leading their readers to surmise the possibility of intent when that is not at all the charge.

I think the technical solution has merit. It doesn't need to be voice, presumably you could rig something up on a steering wheel with buttons to press to record every time they go past an intersection or something. Something to force them to watch the road and keep their hands in the right area. You're right though, that does dent the image of unattended self-driving but we've already demonstrated that isn't what this was.

People are condemning this person for being on their phone (and fair enough), but if I am sitting in a comfortable vehicle with nothing to do / read then I will fall asleep soon enough. I think this is quite common.

They do. That's why the OP mentioned rotations and redundancy. Uber could easily put two people in the vehicle and have them swap seats every 20-30 minutes. The person in the driver seat would be responsible for watching the road and reacting while the person in the passenger seat supervises.

It's okay self-driving cars aren't yet single seater.

that's the fundamental attribution error again, attributing poor judgement/negligence to the driver, who's the visible actor, rather than the inherent systemic flaws designed by other unseen actors who hold greater responsibility.

Stepping back, I also find it hard to sympathize with someone watching a show on their phone. But there are two parties to this crime, and only one of them is being charged. Uber is responsible for creating this situation - I doubt this "backup driver" watches videos while driving her own car - yet has managed to escape liability. Hence why it's appropriate to describe her as a "fall gal".

Yeah, the problem isn’t that they weren’t paying attention, or that they weren’t even trying, but that they where intentionally NOT paying attention.

That said, I don’t think this gets Uber off the hook. If I were on a jury, I’d likely say Uber was guilty of manslaughter (barring a real look at the evidence).

> you can’t get them to be vigilant for a low probability event.

How about starting by simply removing their phone? Isn't that easy? The driver was looking at her phone at the time when she was supposed to be looking at the road. This is a simple violation. The phone should have been banned and the driver fired after a first violation.

I am pretty sure that while what you say might be hard to do for some people, there are individuals out there for whom sitting and looking at the road for hours would not be such a big problem. If she couldn't handle the job, she shouldn't have worked there.

Corporate equivalent of ablative armor then?

I believe the normal behavior of self-driving cars in these situations is for the car to brake to avoid hitting the obstacle, so the safety driver has ample time to take over and navigate the situation. Didn’t Uber disable the auto-braking because it was oversensitive, choosing not to pause road tests until that issue was fixed?

> Would you blame an airplane's autopilot if it crashes?

If the plane crashed due to an autopilot error, yes, absolutely.

Eerily similar to how the Challenger tragedy happened

The American legal system places much more emphasis on acts you may have committed than omissions, and tends to avoid compelling action.

So yes, in an American court, disabling a proven safety feature is significantly worse than simply purchasing a vehicle without the feature.

The safety driver failed at their job, but the NTSB clearly lays significant blame for that failure on Uber, who should know well that humans are poorly suited to monitoring automated systems, and committed acts and omissions that increased the likelihood of an accident.

She.

I think an interesting variation on this is that even if a safety system is not required by law, but is available - then disabling could constitute criminal negligence. Consider what happens if safety equipment on industrial equipment is disabled and injuries result. I'm fairly sure that criminal charges could result for whoever disabled the safety mechanisms (though the there are likely differences between workplace safety criminal law and road safety).

The system was, by its very nature, not ready for production. That's why it had a safety driver in the first place. It's crazy to argue that the safety driver should have been alerted...the whole point they're there is to handle failures of the system, including ones where the system fails to detect an issue.

If my skydive instructor doesn't deploy the backup parachute because I, the student, didn't alert them that the primary chute failed to deploy, it's entirely their fault if we hit the ground at terminal velocity.

If a lifeguard is working at a public pool watching Netflix on their phone and a kid drowns, you can't argue that the kid should have splashed more.

Of course it was "inadequate system" if you consider it a full self-driving. It is inadequate in the simple fact that it was not complete. It was under development.

But how do you suppose we create FSD cars if we can't try them out before they are ready? There is just no other way to do it than vigilant drivers that watch what these cars do.

> To put it another way, the self-driving system did not alert the driver that it had detected something

Well of course not, who would expect that? If the car could positively identify the collision before it happened, it would have simply stopped, no need for a driver at all. The driver is there to prevent exactly this kinds of accidents, and this driver failed by getting our her phone and distracting herself with streaming videos instead of doing her one job. Plain and simple.

She wasn’t a “user”

This was a training mission. Uber wasn't offering rides to passengers who weren't employees.

I agree. This is where torts get fun in first-year law school. If Uber's training wasn't perfectly clear about the importance of the job of monitoring outside the vehicle, then part of the liability shifts to them.

Consider two training courses:

* instructors who are gung-ho on automation being "nearly there" and encouraging people to relax in the car and let the software do its work!

* instructors who are constantly impressing upon students that they need to be vigilant.

One can see how one party is 90% liable. And the other is 10% liable, etc.

The interesting part of this case comes down to the training. In that if it was lacking, it then makes it very clear to future companies that their training needs to be more rigorous.

Its literally in the name of the job title "Car safety driver"

She was not brand new on the job, right? She knew how often intervention was required and chose to watch TV instead.

The vehicle is designed to ultimately be a level 4 system. She effectively has the job of a test pilot of an experimental airplane. The job requires careful attention and is not easy.

There would be a different blame calculus if this were a production level 2 system like autopilot. In that case, it's not a paid test pilot, it would be the paid purchaser of a certified aircraft.

>>> It was an accident that would have happened if it was a manually operated car with an alert driver, no?

No, the last articles and discussions were quite clear that this accident wouldn't have happened with a real driver. The road conditions were good, the visibility was good. A driver would have seen the woman crossing the road well enough and slow down.

From the footage I've seen (both the footage Uber released, which portrayed the road as much darker than it actually was, and from videos of people driving around in the same area after it happened) I can only conclude that an attentive driver would've been able to prevent the crash.

However, I don't think it's reasonable at all to expect someone to remain attentive while looking at a self-driving car. This is the same problem train drivers face, which has been mitigated with all sorts of methods, least of which a dead-man switch. Some countries let their train drivers mention every signal they come across to themselves, with Japanese train drivers even pointing at signs to ensure they're paying attention.

This was a vehicle that had been modified to reduce certain safety features (because Uber couldn't get them to work properly) with someone at the wheel expected to be 100% focused on the road while giving them nothing to do at the same time. You can only go through so many hours of sitting in a card doing nothing before you go crazy.

From a revenge-seeking perspective it's easy to blame the one person who could've stopped the car for her obvious disregard for safety (streaming video on the job), and I suppose a criminal justice case might be in order. However, I think Uber should be mainly responsible for the loss of life because their flawed design not only made the car less secure but also completely disregarded the human psychology when they designed how their human safeguard driver should do their job. Even human-operated cars will beep and yell at you if you don't pay attention while you're driving in cruise control, if such safety features were omitted in the self-driving design then clearly the driver was set up to take the fall when something bad would happen.

I strongly believe Uber only put that woman in there because local law wouldn't let them test their car without a human at the wheel, not because they wanted to ensure their car didn't kill anyone.

From what I remember the person that was hit was fairly visible. The video uber released was very dark however, giving that impression.

Accidents do happen, but the circumstances in this case are important; she was not paying attention to the road when it happened. If the victim came out of nowhere and she could not act, the case would pan out differently.

It doesn't matter what she was told. She was in charge of the vehicle. She was the licensed operator. The presence of a broken ML autosteer doesn't abdicate responsibility.

This is part of the general attitude that rights and freedoms don't matter if a machine violates them. When you walk out of a store post purchase and the security alarm goes off you have zero obligation to disclose what is now your property to the loss prevention experts. But somehow it's acceptable to assume you're a criminal because a machine said so.

That is, presumably, something that the grand jury considered.

How many have videos of both the victim being clearly hit and the driver being severely negligent leading up to and including the accident? How many are pedestrians that subsequently die?

To any curious readers like me: the charge is negligent homicide (the indictment itself is linked at bottom of article)

Most of the time there isn't a video of both the driver and the incident with detailed telemetry data. Also, commercial drivers are held to a higher standard.

Agreed.

> "Had the vehicle operator been attentive, she would likely have had sufficient time to detect and react to the crossing pedestrian to avoid the crash or mitigate the impact," the November 2019 NTSB report stated.

This isn’t a case of the Uber performing some split-second error that needed immediate correction then blaming the driver when that didn’t happen, the way so many in this thread seem to want to make it out to be. This is a person whose job it was to watch the road watching Hulu instead and as a result failing to react to developing conditions that, if NTSB is to be believed (and I trust them a whole lot more than the randos commenting here), could have easily been prevented by an attentive driver.

The vehicle was travelling the posted speed limit at night, as designed. The NHTSA pointed this out that the law may require you to travel at a "safe" speed, and simply choosing to always move at the posted limit is potentially a moving violation and a design flaw.

The vehicle was travelling 20mph faster than it's own "safe clear ahead" zone allowed for. Simply ignoring it's own safety limits and travelling at a higher rate of speed is a design flaw.

Designing a large public road autonomous vehicle test and not taking into account an easily predicted human vulnerability that's been known in other automation experiments for decades is a project design flaw.

Whether or not the driver is at some fault, I think the court will be capable of determining that. Arizona's willingness to completely disregard the mountains of negligence on part of a giant corporation is disturbing.

> Guy was using his phone not paying attention to the road when testing experimental equipment on a public road. He was there specifically to mitigate potential malfunctions of said experimental equipment, but he was reading reddit or whatever instead of keeping lookout.

> I can't see how this is not clear cut negligence.

1) The driver is a woman.

2) While I think the driver does bear some fault here, they don't bear all the fault. Uber designed an unsafe system that relied on an unnatural amount of vigilance from a single person while simultaneously discouraging that vigilance [1]. They didn't design the car to shut down when one of it's critical safety components (the driver) was not operating correctly, and they didn't even give that driver amphetamines or something to increase their vigilance to artificial levels.

[1] Basically: pay close attention to a boring process while doing absolutely nothing for hours on end. I'm pretty sure that's a classic "humans suck at this" task.

This is the first of many such cases to come.

It's unreasonable to hold someone accountable for a "self-driving car" that suddenly decides at a split-seconds notice that it can't cope driving.

Of course this is extra bad because it's an experimental car, but it's the same in my opinion with those Teslas on the road now that do the same thing.

I give you a weight, you are to hold the weight in front of you with your arm outstretched, for 12 hours straight. If you drop your arm then a random person is shot. If you drop the weight is the death your fault or my fault for putting you in an impossible, for a human, situation?

Humans are very bad at paying attention for a rare event and having nothing else to do 99.999% of the rest of the time. This is surely well known to the people who set up this system in the self driving car.

Sorry! I should know better than to assume gender these days. Fixed.

He can desire reeeeaaaallllyyyy hard to be a lady all he wants, but he'll never be one. Reality does not change just because of human desire. The guy deserves to be treated with the same amount of respect as any human being does, but he haves no authority over me to obligate me to say an airplane is a submarine.

On the subject at hand, he was paid to do one job: look at the road and be the safety net between an experimental car and the other people on the road.

If he was listening to a podcast using headphones it could have been avoided, but he was looking down at his phone for the whole duration of the recorded video we have from the incident.

It's negligent homicide just as it would be to any other machine operator, on any other industry, that did not fulfilled his duty to the company that contracted him would.

It's also a well-known and completely predictable behavior for humans. In just about every other job requiring constant vigilance, there are multiple factors mitigating this failure mode (shift lengths counted in minutes, multiple observers, automated attention checking devices that shut down the machine if not reacted to, other employees ensuring the observers pay attention, etc.).

Uber should absolutely get the blame for creating this situation in the first place; letting a self-driving car out with only a backup driver as safety, and with her phone on her to boot, should not even be allowed.

> "Uber had disconnected the Volvo's factory-installed crash avoidance system. While the Uber vehicle's autonomous system did detect Herzberg before the impact, the vehicle — and Uber — relied on Vasquez to take action if an emergency arose."

The Uber software can detect an imminent collision but relies on the backup driver to act on it. For everything else the car is designed to drive itself. This is why "automation complacency" should not be ignored.

I still haven't heard a good reason for them to not have also been running Volvo's system in parallel. Considering that the system is clearly not so hair trigger that humans hate it it shouldn't hobble their AI and it seems like all upside and no downside to me.

Maybe not a prerequisite, but certainly necessary to prevent certain type of long-tailed events. For example, I can anticipate by the look of a neighborhood that it's more likely that there would be jaywalking and poor infrastructure, I can feel that the driver riding ahead is drunk or on drugs and keep my distance etc.

It remains to be seen if a automated drivers can make-up for increased risk in such scenarios with good performance in areas they excel, such as reaction time, highly distributive attention, near perfect physical predictions and decisions made on that basis, vehicle-to-vehicle coordination and vehicle to road communication etc.

What is clear though is that any practical system in the near term (non-general AI) will approach driving quite differently than a human driver. Hence the jury is still out of lidar vs camera-only approaches (Waymo vs Tesla).

> I completely agree with the assertion that the grey area in between is where a bunch of people are going to wind up dead.

But will more people end up dead than in the scenario where we had no ‘self-driving’ cars at all?

Humans are much more accident prone than robots in all the research on this I’ve seen.

You can probably achieve a safer-than-humans solution without strong/general AI, provided it only operates on a special kind of road (in-highway sensors, no pedestrians, all other vehicles are also self-driving).

FYI it was a bicyclist (singular) but they were walking their bike across a crosswalk at 10pm.

Not to victim blame but if they were not wearing lights or a reflective vest I think a human could have made the same tragic error.

As you point out the system actually DID detect them.

I think your claim is far more theoretical in nature. In a vacuum, sure, fewer deaths is better. In the real world, where people aren't satisfied unless someone can be held accountable in a clear way, we are far better off with humans driving.

By virtue of being a HN reader, you are almost certainly much better educated and much more highly skilled than the person Uber hired to baby sit their vehicles during testing. Set the alarm on your watch to go off in two hours. Start looking at your monitor to see if any pixels fail in the next two hours. Ask yourself how far into that 2 hour task you were able to maintain vigilance. Now try to do it day after day. Was she at fault, yes. Were the Uber engineers who designed this testing system at fault too? Absolutely.

Airline pilots are highly trained, subject to regular physical inspections and testing, and spend much of their careers practicing for emergency procedures to maintain their vigilance.

Auto and truck drivers operating normal vehicles are in a state of at least partial task concentration while operating their vehicles. The task concentration context in which this test driver was seated was completely unlike that of a normal vehicle driver.

> The engineers probably were right to disable that, if the false positive rate was too high.

This isn't a binary. They can also choose to stop the public test of a vehicle they can't adequately control.

I think GP is talking about the OEM system that shipped with the car. Considering that their false positive rates are so low as to not annoy human drivers I don't see why they disabled it.

IF the engineers told the driver, "Hey, we disabled pretty much all the safety features on your vehicle because we can't figure out how to make them work -- so be Extra Careful!!!" --- well, maybe. But shifting the blame onto a low-wage worker to shield Corporate Hubris seems....wrong.

Wasn't considering the phone at all. In a lot of places in the US, using a phone is also a distraction likely to get you jail/ conviction. (In some places it's considered driving impaired).

The accident would just never have gotten the scrutiny it did. Unless the driver came out and mentioned they were using their phone, the whole incident would have just vanished. They certainly wouldn't have bothered getting a subpoena to check the drivers cell record.