Get the top HN stories in your inbox every day.
asveikau
gxs
Yeah, as long as it’s intelligible an accent is perfectly fine
It’s also perfectly fine to want to sound like a native speaker - whether it be because they are self conscious, think it will benefit them in some way, or simply want to feel like they are speaking “correctly”
Sorry to pick on you, it’s just amazing to me how sensitive we are to “inclusivity” to the point where we almost discourage people wanting to fit in
matsemann
Being legible also means to cater to your audience. I work in an English-speaking company in a country where English isn't the native language, with loads of non-native speakers from around the world. Sometimes the native/best English speakers are the ones being misunderstood, because they use idioms or advanced words. None of us are bad at English, and I don't mean that I need to "dumb it down" (if anything, verbally I'm one of the worser ones), but I don't feel like I'm missing out on speaking simple with an accent.
dhosek
Generalizing from my own experience, it’s easier for me to understand a non-native Spanish speaker than a native Spanish speaker and I would guess that the same applies with ESL speakers. One thing I found really fascinating is that even though I’d never studied French¹, I actually had an easier time understanding a conversation between my ex-wife and her aunt in French than when they spoke Spanish in which I was functional (my skill in the language has gone up a great deal since then so that I now read fluently, and speak and listen reasonably well, albeit less well than I would like).
⸻
1. Thanks to my kids studying French on Duolingo and my joining them, I can no longer say that I’ve never studied it.
netsharc
I've read of a language, or I guess dialect of English called "European english", i.e. English as spoken Europeans that learnt it as a second language. Another example is how a Brit using too many idioms like "Bob's your uncle" would confuse English speakers that haven't been exposed to Britishisms.
Searching on YouTube just gives me this relevant result, an English that has some new/odd words that's been adapted by the Euro-bureaucrats https://www.youtube.com/watch?v=vf8KxvRYvQU
voidUpdate
I believe I have some kind of auditory processing issue because with some accents, it can be really hard for me to understand what someone is saying, when other people can understand them fine. It's gotten to the point where I avoid going to some shops in my town because I know the staff have quite strong accents and I feel embarrassed having to ask them to repeat themselves all the time
wincy
My wife has this problem. She got chided because she asked for a translator for the doctor at the children’s hospital and they just assumed she was being racist. He was Thai and his English was very difficult to understand.
chpatrick
That kind of implies that there's a "correct" accent for English, even though many countries and regions natively speak it. Someone from Glasgow is just as much of a native speaker as someone from Los Angeles even though the accents are wildly different.
gxs
Hence in quotes, man
orbital-decay
Intelligibility heavily depends on what you expect to hear, and that depends on your native language or even locality. Even a tiny amount of French accent in English makes it sound like gibberish to me (but not others, and I don't have this issue with other thick accents). I'm sure my native accent is also incompatible with someone else's ears. That's the reason people pay accent coaches.
vintermann
I understand the feeling, but I can usually adapt pretty quickly when someone speaks in a very unexpected accent. Sometimes I can even remember the sounds well enough that I can reinterpret it on the fly without having to ask them to repeat.
gxs
Yes, should go without saying that intelligibly is perfectly provided it’s intelligible in whatever context you’re in
anadalakra
"If Victor wanted to move beyond this point, the sound-by-sound phonetic analysis available in the BoldVoice app would allow him to understand the patterns in pronunciation and stress that contribute to Eliza’s accent and teach him how to apply them in his own speech."
Indeed Victor would likely receive a personalized lesson and practice on the NG sound on the app.
screcth
A very important part of people trusting you is them being able to understand what you say without making extra efforts compared to a native speaker.
An easy way to improve intonation and fluency is to imitate a native speaker. Copying things like the intervocalic T and D is a consequence of that. It would be easier for a native Spanish speaker to say the Spanish /t/ and /d/ but intonation and fluency would be impaired.
The sounds don't "flow" as they should.
yodsanklai
> An easy way to improve intonation and fluency is to imitate a native speaker.
There are lots of variations in English pronunciation. Singaporean, Australian or Scottish native speakers do sound very differently. I don't know to what extent they benefit from adjusting their accent if working in a different English speaking country to match the local dialect.
Also, as a non-native speaker I wonder if it's worth practicing my accent considering that everybody has a different accent anyway. Rather than trying to mimic a north american accent (which I'll never be able to do anyway), I'd be more interested to identify and fix the major issues in my prononciation.
asveikau
The specific problem is that American intervocalic /t/ and /d/ is very similar to Spanish /ɾ/. But if they don't get it right it's not perceived as the right phoneme. The Spanish /t/ is more dental and the undervocalic /d/ is more of a [ð] but they will sound correct in English.
JoshTko
Thank you for pinpoints my confusion/disconnect on what lack of improvement that I was sensing. There was an improvement on pacing, and cadence, yes, but that was not the main challenge with Victors accent. Visually I'd say victor improved by at most 5% and not 50% as indicated by the visualization. In some regards it was even harder to understand than the original due to speed and cadence without improvement in core pronunciation.
oscarfree
People have all sorts of motivations for learning languages and accents. Right now, I'm using this tech to work on my accent in Spanish. Honestly I would rather mumble almost unintelligibly with an decent Mexican accent than speak Spanish slowly and clearly with an American accent. There is a difficult but necessary period of learning an accent where intelligibility drops. For a while, I made a strange [ð]-like noise when learning the alveolar trill (rolled R), and it would have been more intelligible to use something like alveolar tap. But, I built up the muscle memory, and can now make the correct sound. Hearing a version of myself (rather than a different speaker) gives me a more useful target to mimic, and the distance metric gives me a useful measure of whether I'm closer or further from the target.
vitus
It's fascinating that "long" is the biggest tip-off you have, given that Mandarin Chinese (based on the mention of "a noticeably strong Chinese accent") does have words that have the same IPA pronunciation (if you set aside tone) [0] as an American whose speech follows the cot-caught merger [1].
Meanwhile the thing that stood out to me in the initial recording were the vowel sounds: for instance, "young" sounded almost like it rhymed with "long" before training. (That makes sense, since Mandarin similarly has a word with that sound, as can be found in the common last name Yang [2].)
Incidentally, Mandarin has words that sound like "lung" (e.g. the word for "cold" [3]), but if you replace the "l" sound at the front with a "y" sound, depending on which of two transformations you use, it turns the vowel sound into a long o [4] (near rhyme with "lone"). (There is another transformation that you can use that results in a leading "y" in pinyin, but in that specific case, the vowel turns into a long e, and the "y" is largely silent (e.g. the word for "solid" [5]).)
In the last recording, Victor is clearly rushing through the sentence, and you can tell that where he previously had a clear "s" ending for the word "days", it's now slurred into a "th" sound. Agreed that that's actually a net negative for intelligibility.
The wiktionary links below have clips of pronunciation. I will note that not all native speakers have a Standard Chinese accent [6,7], so there are assuredly some differences in pronunciation to be expected depending on exactly which region said individual hails from.
[0] https://en.wiktionary.org/wiki/%E6%B5%AA
[1] https://en.wiktionary.org/wiki/long
[2] https://en.wiktionary.org/wiki/y%C3%A1ng#Mandarin
[3] https://en.wiktionary.org/wiki/%E5%86%B7
[4] https://en.wiktionary.org/wiki/%E7%94%A8
[5] https://en.wiktionary.org/wiki/%E7%A1%AC
pjc50
What the vector-space data gets right, and what the human commentary tends not to, is the idea that accents are a complex statistical distribution. You should be careful about the concept of a "default" or "neutral" accent. Telecommunications has spent the 20th century flattening accents together, as has accent discrimination. There's always the tendency for people to say "my accent is the neutral standard against which all others should be measured".
ilyausorov
For sure, and I don't think we ever use the term default or neutral. The "the American English accent of our expert accent coach Eliza" is just that -- it's one accent.
As a learning platform that provides instruction to our users, we do need to set some kind of direction in our pedagogy, but we 100% recognize that there isn't just 1 American English accent, and there's lots of variance.
ahazred8ta
Kids who grow up on US military bases have a very homogenized north american accent.
lurk2
> There's always the tendency for people to say "my accent is the neutral standard against which all others should be measured".
You can measure this by mutual intelligibility with other accent groupings.
dqv
Well, no. That would also measure what accents someone has the most exposure to, which doesn't necessarily reflect its "absolute intelligibility", but rather its popularity. Popularity and optimality are not the same thing. You would first need to measure if an accent's popularity is a result of its optimality to make the claim that your measure is accurate.
lurk2
> Popularity and optimality are not the same thing.
Yes they are. If we lived in a world where Australia was a world superpower we might be having this conversation upside down and with r’s where there shouldn’t be any, but we don’t. Every student wants to learn to speak with an American accent because it has the highest level of intelligibility owing to exposure via cinema, music, expatriate communities, etc.
SamBam
Like others recently, I've been extremely impressed by LLM's ability to play GeoGuessr, or, more generally, to geo-locate random snapshots that you give them, with what seem (to me) to be almost no context clues. (I gave ChatGPT loads of holiday snapshots, screenshotted to remove metadata, and it did amazingly.)
I assume that, with enough training, we could get similarly accurate guesses of a person's linguistic history from their voice data.
Obviously it would be extremely tricky for lots of people. For instance, many people think I sound English or Irish. I grew up in France to American parents who both went to Oxford and spent 15 years in England. I wouldn't be surprised, though, if a well-trained model could do much better on my accent than "you sound kinda Irish."
ilyausorov
We actually did something like this for non-native English speakers a few months back. Check out https://accentoracle.com (most mind-blowing if you're a non native English speaker)
SamBam
Well, it says I'm Finish. But now I have a new game, where I put on my best Italian or Russian or Greek or Australian accent and try to see how close I am.
I'm terrible, according to the program. My Italian is Russian or Hungarian or Swedish, my Australian is English.
New party game unlocked.
ilyausorov
Amazing! If you can make it go viral again too, I will love you!
AJoxo
I've been building that exact game
accentgame.xyz
vunderba
Fun. I have a strongly modulated North American midwestern accent so unsurprisingly it had me read several paragraphs before only being able to say with any certainty that my accent was 83% English with the rest being Spanish/Russian. It couldn't detect the country of origin.
haarolean
Agreed, pretty meh. Tried my usual accent (the one where natives mostly can’t tell where I'm from) — got 78%. Then went full cartoon russian ‘bad neighborhood’ mode — somehow scored 68%.
jfim
Interestingly enough, it thought I was Russian, even though my native language is French. It was tied at 32% with French though.
Edit: Tried it a few times and also got English as an accent. Pretty fun application!
cmcconomy
I would love to be able to explore combinations of X spoken language with Y accent, like for example I've always been curious how French sounds spoken with an Indian accent.
eru
It detects my non-native English accent correctly, but then it makes the mistaken assumption that I want to sound like an American of all things?
nmeofthestate
I'm 42% Arabic apparently! And 20% Russian. Got an 81% American accent level. I guess it is tuned to non-native-English speaker accents.
ilyausorov
Was that right? Or what is the correct native language it should have predicted? Note the %s in the accent breakdown section are prediction probabilities
reactormonk
Swiss-German accent doesn't seem to be on the list, so it guessed mostly Swedish.
owenthejumper
Wow that was actually accurate
nmstoker
Yes, although I believe this is a speaker embedding model here, so not LLM related.
This kind of speech clustering has been possible for years - the exciting point with their model here is how it's highly focused on accents alone. Here's a video of mine from 2020 that demonstrated this kind of voice clustering in the Mozilla TTS repo (sadly the code got broken + dropped after a refactoring). Bokeh made it possible to directly click on points in a cluster and have them play
https://youtu.be/KW3oO7JVa7Q?si=1w-4pU5488WxYL3l
note: take care when listening as the audio level varies a bit (sorry!)
ilyausorov
Correct, not LLM
chris_va
I bet you are right.
I had a forensic linguistics TA during college who was able to identify the island in southeast Asia one of the students grew up on, and where they moved to in the UK as a teenager before coming to the US (if I am remembering this story right).
From what I gather, there are a lot of clues in how we speak that most brains edit out when parsing language.
dhosek
Or the classic scene in Mrs Doubtfire where Pierce Brosnan attempts to locate the origin of Robin Williams’s fake English accent.
dhosek
I’ve seen some online quizzes that based on regional variations in accent (does root rhyme with foot or boot?) and vocabulary (what do you call a sweet fizzy beverage) that did a great job of locating where my Facebook friends back in the day grew up. It got me a bit off largely because while I grew up in Chicago, I had spent most of my adult life in Los Angeles so I tend to prefer “freeway” to “expressway” (changing that answer moved me from Rockford to Chicago).
georgewsinger
This is so cool. Real-time accent feedback is something language learners have never had throughout all of human history, until now.
Along similar lines, it would be useful to map a speaker's vowels in vowel-space (and likewise for consonants?) to compare native to non-native speakers.
I can't wait until something like this is available for Japanese.
yorwba
The approach in the article is roughly equivalent to having someone listen to you speak and then repeating back in their own voice so you can attempt to copy their accent. Certainly nice to have available on demand without needing to coordinate schedules with another human.
A good accent coach would be able to do much better by identifying exactly how you're pronouncing things differently, telling you what you should be doing in your mouth to change that, and giving you targeted exercises to practice.
Presumably a model that predicts the position of various articulators at every timestamp in a recording could be useful for something similar.
pjc50
> something language learners have never had throughout all of human history
.. unless they had access to a native speaker and/or vocal coach? While an automated Henry Higgins is nifty, it's not something humans haven't been able to do themselves.
anadalakra
Native speakers are less helpful at this than you might think. Speech coaches are absolutely the way to go, but they're outside the price range for most people ($200+/hr for a good coach). BoldVoice gives coach-level feedback and instruction at a price point that everyone can access, on demand.
yorwba
Do you have another blog post showing your product giving targeted feedback about individual speech sounds? That's what I would expect from a coach.
astrange
You can take a language class rather than have a personal instructor. Although accents are a sensitive topic so I don't remember mine going into it much.
ilyausorov
That's a fascinating idea! Definitely something to try out for our team. We actively and continuously do all sorts of experiments with our machine learning models to be able to extract the most useful insights. We will definitely share if we find something useful here.
coherentpony
> Real-time accent feedback is something language learners have never had throughout all of human history, until now.
Do you have a source for this? It doesn't seem plausible to me, but I'm not an expert.
fxtentacle
What a great AI use-case! At first, I felt excited ...
But then I read their privacy policy. They want permission to save all of my audio interactions for all eternity. It's so sad that I will never try out their (admittedly super cool) AI tech.
anadalakra
You can reach out and request your data to be deleted at any time.
fxtentacle
"if you wish to opt out of future collection of voice samples, you may do so by disabling voice-related features in the BoldVoice app. Please note that this may limit the functionality of certain services."
Yeah, I can opt out. By not using any voice-related feature in their voice training app.
anadalakra
If you're still actively using the app, the voice will be retained and processed so that you can receive instant feedback, and also so that you receive additional personalized practice items and video lessons based on your speech needs. If you don't want the samples saved "in perpetuity", you can request them to be deleted once you decide that you're done with the application. Hope this helps!
3abiton
I wonder if voice will be treated like facial data in terms of privacy. At least the EU has strong provisions against PII.
runelohrhauge
This is fascinating work. Love seeing how you’re combining machine learning with practical coaching to support real accent improvement. The concept of an “accent fingerprint” is especially clever, and the visualization of progress in latent space really brings it to life. Excited to see where you take this next!
treetalker
This is cool and one of the applications of LLMs that I'm actually looking forward to: accent training when acquiring a new language, particularly hearing what you would sound like without an accent!
That said, I found the recording of Victor's speech after practicing with the recording of his own unaccented voice to be far less intelligible than his original recording.
Looking forward to seeing the developments in this particular application.
ilyausorov
Fair point! When Victor tried to speed up to speak as fast as Coach Eliza, while it sounded somewhat less accented, a few parts of the phrase did get less intelligible. 10 minutes of practice is only a start after all.
Interesting to note that we're also developing a separate measure of intelligibility that will give a separate sense of how intelligible versus accented something is.
ccheever
This is really cool.
Just had an employee at our company start expensing BoldVoice. Being able to be understood more easily is a big deal for global remote employees.
(Note - I am a small investor in BoldVoice)
vessenes
This is super cool.
A suggestion and some surprise: I’m surprised by your assertion that there’s no clustering. I see the representation shows no clustering, and believe you that there is therefore no broad high-dimensional clustering. I also agree that the demo where Victor’s voice moves closer to Eliza’s sounds more native.
But, how can it be that you can show directionality toward “native” without clustering? I would read this as a problem with my embedding, not a feature. Perhaps there are some smaller-dimensional sub-axes that do encode what sort of accent someone has?
Suggestion for the BoldVoice team: if you’d like to go viral, I suggest you dig into American idiolects — two that are hard not to talk about / opine on / retweet are AAVE and Gay male speech (not sure if there’s a more formal name for this, it’s what Wikipedia uses).
I’m in a mixed race family, and we spent a lot of time playing with ChatGPT’s AAVE abilities which have, I think sadly, been completely nerfed over the releases. Chat seems to have no sense of shame when it says speaking like one of my kids is harmful; I imagine the well intentioned OpenAI folks were sort of thinking the opposite when they cut it out. It seems to have a list of “okay” and “bad” idiolects baked in - for instance, it will give you a thick Irish accent, a Boston accent, a NY/Bronx accent, but no Asian/SE Asian accents.
I like the idea of an idiolect-manager, something that could help me move my speech more or less toward a given idiolect. Similarly England is a rich minefield of idiolects, from scouse to highly posh.
I’m guessing you guys are aimed at the call center market based on your demo, but there could be a lot more applications! Voice coaches in Hollywood (the good ones) charge hundreds of dollar per hour, so there’s a valuable if small market out there for much of this. Thanks for the demo and write up. Very cool.
pjc50
> It seems to have a list of “okay” and “bad” idiolects baked in
We're back to "AI safety actually means brand safety": inept pushback against being made into an automated racism factory with their name on it.
vessenes
100%
BalinKing
(Minor nitpick, but I think "dialect" is a more appropriate word than "idiolect" here—at least according to Wikipedia, "idiolect" refers to a single person's way of speaking, whereas AAVE et al. are shared and are therefore considered dialects.)
vessenes
OK, good read for me here. Based on your feedback and some research, I think I should have use ‘sociolect’ for both in that I was less complaining about ChatGPT’s unwillingness to use, say, finna, in a sentence, and more complaining about the vocalized accents. Anyway good catch, thanks!
retrac
Sociolect is the right term for a dialect used by a particular social group. A related idea is "register" when multiple related and mutually understandable standards exist, and are used in different contexts.
adhsu01
Super cool work, congrats BoldVoice team! I've always thought that one of the non-obvious applications of voice cloning/matching is the ability to show a language learner what they would sound like with a more native accent.
ilyausorov
This and more exciting features are coming to the BoldVoice app soon!
oscar120
this^
WhitneyLand
The hear my own voice without an accent thing is a really cool party trick.
I’d consider making this feature available free with super low friction, maybe no signup required, to get some viral traction.
ilyausorov
What if it was already available? Try it out at https://accentfilter.com!
lukeinator42
Is the approach being used to do accented TTS (or just reference recordings), and then a tone color conversion model that just changes the timbre? Because if I say a completely different sentence it still says the original words, haha.
PaulDavisThe1st
Hmmm. Initially impressive but upon retries and reflection ... not that great. It doesn't even maintain timing ... unless that's part of the transform.
ilyausorov
Indeed yeah that’s one of the key weaknesses of the approach that we’re using. It overrides the speakers cadence and accent while keeping their voice profile / timbre in place. Different techniques may not do this but also may not copy over the accent to the resulting clip as effectively. So far we’re using this to support pedagogical (and lead-gen) use cases where we think it works sufficiently enough.
sardines
How's the "accent conversion model" work? Is it all embedding based?
If so—and if you want to transfer-learn new downstream models from embeddings—then seems to me you are onto a very effective way of doing data augmentation. It's expensive to do data augmentation on raw waveforms since you always need to run the STFT again; but if you've pre-computed & cached embeddings and can do data augmentation there, it would be super fast.
Get the top HN stories in your inbox every day.
Victor's problem isn't really the vowels or pacing. The final consonants are soft or not really audible. I am not hearing the /ŋ/ of "long" as the most marked example. It sounds closer to "law". In his "improved" recording he hasn't fixed this.
I sometimes see content on social media encouraging people to sound more native or improve their accent. But IMO it's perfectly ok to have an accent, as long as the speech meets some baseline of intelligibility. (So Victor needs to work on "long" but not "days".) I've even come across people who are trying to mimick a native accent but lose intelligibility, where they'd sound better with their foreign accent. (An example I've seen is a native Spanish speaker trying to imitate the American accent's intervocalic T and D, and I don't understand them. A Spanish /t/ or /d/ would be different from most English language accents, but be way more understandable.)