As part of its broader quest to ‘Connect the World’, Meta has developed a new, AI-based speech-to-speech translation process, which will eventually, ideally, provide more opportunities for people to connect, in real-time, despite language barriers.

As you can see in this example, Meta’s new Universal Speech Translator project converts audio spoken in Hokkien, a primarily spoken language from China, into English audio in real time.

Because Hokkien is a lesser-used language, and doesn’t have a wide corpus of text-based material available, Meta says that this new process will alleviate the need for transcription output in translation – and with almost 3,500 living languages being primarily spoken, that could be hugely impactful for global connection and engagement.

As per Meta:

“While the Hokkien translation model is still a work in progress and can translate only one full sentence at a time, it’s a step toward a future where simultaneous translation between languages is possible. The techniques we pioneered can be extended to many other written and unwritten languages.”

The broader view, then, is that these techniques can be translated to its metaverse future, enabling broader interaction, on a global scale, in the metaverse space.

“We believe spoken communication can bring people together wherever they are located — even in the metaverse.”

I mean, Meta’s apps are banned in China, so not sure that this specific example would apply here. But the broader concept is that audio translation will facilitate a Star Trek-like metaverse experience, where anyone can speak their native language and still take part in the global conversation.

Which is pretty amazing, and it’s amazing to consider how far we’ve come with translation tools and processes, and what that could mean for future opportunities.

That includes business collaboration and marketing opportunities as well. opening up the world to broader connectivity expands market reach, and could facilitate all-new potential for many creators and businesses.

Meta’s open sourcing its Hokkien translation dataset, as well as its SpeechMatrix system, which includes a large collection of speech-to-speech translations developed through its natural language processing toolkit.

It’s another step towards a more open, more connected digital world, which meta hopes will form the backbone of metaverse connectivity.