Is Google’s AI research about to implode?

The problems they identified have been known for years and there are lots of papers exploring how to mitigate.

The whole paper was a nothing burger wrapped in social justice language with asides about how global warming is Actually Racism because of disparate impact (interesting, not an ML topic).

If the problems aren't novel and you're proposing zero solutions, it shouldn't be a paper.

No, it is not too much to ask in the specific case of Gebru’s bad paper. Several of the arguments are specious, like comparing the total energy consumption for training GPT with car trips, or demanding that NLP researchers have to keep up with rapidly changing activist “woke” vocabulary and ensure their models are respecting it.

These are ridiculous claims, and it’s fair to respond to them by saying, “well, what exactly do you imagine a solution or mitigation looks like?”

Essentially, by the nature of how specious Gebru’s stated problems are, they demand clarity over what an “ethical solution” even is, conceptually, and why everyone would have to agree.

For example, you could discuss economies of scale or train-once-finetune-everywhere approaches with GPT that reduce total energy needs. Or you could discuss how researchers can register the corpus they use and the snapshot of time it was grabbed, with an open understanding that as long as the methods and data are reproducible, there is no research ethical issue with studying that corpus, no matter how much bias or lack of woke vocab a given person believes it has. (And also, nobody is required to just accept activist language as important or valid.)

Gebru did none of this. The article could literally be summed up by Gebru saying, “I think <supposedly shocking evidence> is bad, therefore its connection to something in ML is bad.”

E.g. “I think, subjectively, that the raw energy use to train GPT is bad. Here are some shocking comparisons. Therefore GPT is bad.”

It’s incredibly unrigorous and juvenile. Dean’s comments that it needs to clearly state mitigations is actually a super generous, polite way of saying the paper is just subjective amateur hour.

Isn't that what the "directions for future research" section is for?

Of course. Especially to expect all that to be in each individual published paper.

I think you're very wrong. Yes, there's hype around ML etichs. Trends in ML come and go. Do you see any VC investing in etichs in ML startups? As I said, it's an esoteric academic field.

How do you even compare between the two? MSR closed an entire lab out of the blue of some great researchers who didn't do anything wrong. Here you have two employees going against their company and shitting on it publicly. The only researchers who will not want to work at Google after this saga are the ones Google better off without.

In terms of attracting AI researchers, think of this:

Gebru has very publicly got into fights with Yann LeCunn and now with Jeff Dean. If you are building AI, who would you rather build your team around, Dean/LeCunn or Gebru? If you are an AI researcher, do you want a join a team where one of the team members is in the habit of aggressively accusing other researchers of racism? Would you be worried that your research might fall within their crosshairs for some reason or another? For example, if you are working on natural language research, and your model ends up doing better with Indo-European languages versus those from other families, do you want to be accused of propagating racist power structures on Twitter?

> ethics of ML is possibly its hottest subfield.

Is this really true? I don't see ethics in ML papers getting the same attention in major conferences as theoretical or experimental breakthroughs in deep / reinforcement learning.

Don't get me wrong, ethics could be hot outside the ML academia, but I very much doubt it's something majority of grad students in ML are dying to get into.

i guess you are right that potential employees will consider this behaviour in their calculations. for the most part by adjusting their salary demands with an additional "risk adjustment bonus". as the FAANG can easily swallow that additional cost and are still incredibly attractive i doubt there will be a big effect besides loosing some value-oriented people. i doubt this will make a difference numbers-wise. nonetheless i applaud employees sharing their view of a companies inner workings for us others to have more information to make an informed decision themselves - yeah transparency

Google and other companies should regard rigorous research and discussion about AI ethics as long-term protection of their bottom lines. If they start launching products and selling services that are found to unfairly favor or disfavor certain groups of people, they will be vulnerable to lawsuits, government regulation, and damage to their reputations.

This may not change end of year results, but this kind of research is what gives Google a credible voice when it comes to shaping public discourse and influencing legislative process, for instance.

i agree. Ethics means nothing. It's just an emotional security blanket. They are no more qualified than a 5 year old to invent ethical standards.

But when you write something like this, do you also understand that their actual research is widely considered to be of a high quality and very important? So if you agree that ethics is important, would you leave them off a top 10 list (and who would you put on)?

It's certainly true for most of us but not the names the op mentioned.

I would argue the real ego resistance is in not accepting such people exist.

Replaceability is a vain concept.

Simile: saying “your brain is replaceable”. Beyond the fact that the most likely context is a threat, it is a poor argument: while technically true, what would remain of me would not be meaningfully me. And the surgery is work that would be hard-pressed to generate the expected value, such that the only reason to do it, is either out of anger or as a consequence of irremediable damage.

Companies are stories. The decisions are made internally, but their meaning is narrated externally. If you change the protagonists, the story changes. The case of Uber’s self-driving car division is quite an example of that.

Does the change in Google’s story converge to a positive or a negative light?

>> It hurts to admit that we're not replaceable, but we are

The more people, the less the individual is valued. But that does not make the individual less valuable. Unfortunately, for a few years now, respect for the performance and qualifications of others has been declining more and more. This increases the illusion that everyone could be replaceable. Just ask your family if they see it that way in relation to you; the illusion of replaceability definitely ends here.

  The job might not get done as well, or done in a different way than we'd do it, but it'll still get done.

Excellence can't be replaced as easily. Maybe for certain kinds of jobs yes, but for all jobs? No. If that were the case then we'd be inundated with Einsteins, etc. And we aren't.

That's maybe true for YOUR job. The more a job has a well-defined description, the truer your statement is.

People are somewhere on the scale of greatness. At some point it becomes harder and harder to find replacements that will be able to get that job done. People are very capable to steer projects into failure.

Also this is a management illusion. There is no evidence for this assumption. You don't even know the probability distribution. There is no reason to assume that the percentage is equally distributed across all firms or countries. And anyway, the article is about something else.

And yet so many people consider themselves important enough e.g. to post comments here. It just seems that it is always the others who are dispensable. For people who make it into management, this tendency even seems to intensify (or it was the prerequisite why they wanted to be in management).

> approximately all of us are replaceable

I guess it depends on the purpose for which we are all supposed to be replaceable. Nature probably doesn't care which individuals reproduce or are eaten, as long as the numbers are right. Human society with its elaborate specializations and long training periods has added a few more dimensions.

If A and B made the same discovery independently, that is evidence that A was replaceable, but that C built on D is not evidence that C was replaceable.

The title, which does not mention anyone's firing, has everything to do with the article...

Anyway, you stopped reading in the first sentence? That's essentially the same as not reading it.

It says the high point is 2017, not the last good paper. There are of course other good papers coming out go Google. But the novelty is dropping and the angst is increasing.

>. I don’t want to downplay the deep instutionalised sexism and racism that is at play in Gebru’s firing — that is there for all to see.

This is a very badly written and uniformed article, and sentences like these essentially illustrate the thinking here (It's imploding because I don't like it).

Here is an alternative reading: Google is cleaning house of toxic activists who are not interested in serious ethics research but use it as a vehicle for their ultra-progressive political agendas.

I read the article and i'm not following the argument delivered at all.

There is no real proof to this.

I'm following and reading research @ google (stuff like this https://ai.googleblog.com/ and other sources) for ages now and NOTHING indicates an 'implosion'.

It is strong research with real and constant results.

I have no idea why the autor would even consider using the word 'implode'.

Its not rocket science that data is biased and it just will continue be researched and a solution will be found. For the single reason that biased systems in certain areas will not deliver the results you need to use it properly.

Nothing symptomatic about firing toxic employees. The only thing imploding is AI etichs research. There are good people doing quality research that will now have much harder time finding a job in industry because of the bad rep these Google employees "contributed" to the field.

I read the article. Doesn't change the fact that your chosen title is clickbaity. Even if you disclaim it in the article itself.

Blah blah blah racism blah blah blah lgbtiqp blah blah blah parroting blag blah blah white male redditors.

Very good article.

Time will show; winning the competition was just the first step.

Too subjective; what looks like malice could just be basic misunderstanding.

Simple example: if I reply only the original forum post, but my message is posted after yours, then you might assume like I'm excluding you intentionally, probably because I don't like you personally; you get mad. This assumption is wrong and harmful to productive conversations. The simplest explanation is I began writing my reply before you posted; no apology is warranted. Another possibility is I'm simply a careless or excitable type of person who usually replies quickly before reading other replies; an apology is warranted, but it wasn't a personal attack!

C.f. Sussman attains enlightenment [0]

[0] http://www.catb.org/jargon/html/koans.html#id3141241

Maybe you subconsciously want Google's AI research to collapse? ;)