Introducing Dubbing 2.0

Today we are releasing a significant improvement of Synthesia's dubbing system. Dubbing 2.0 has better lip sync, more natural voices, smarter translations, and a faster editing workflow thanks to an entirely new stack that delivers improvements in every part of the process, from the first translated word to the final frame.

Taken together, these improvements deliver first-pass outputs that are now good enough to publish without the need for heavy correction, external agencies, or weeks of post-production between finishing a video in one language and shipping it in over 130.

Improvements across the entire workflow

The most visible improvement is the lip sync. The previous model would drift across fast cuts and scene transitions, during moments where a viewer's eye is most likely to catch a mismatch. We’ve upgraded our lip-sync model to track micro-movements of the mouth, keeping it frame-accurate even through fast cuts, scene transitions, or multiple people talking at once. This fixes a major challenge in dubbing: translating between languages with entirely different mouth shapes (like English to Japanese).

The voices sound more human now. While the old version was technically correct, sometimes it was emotionally flat. The new audio engine raises fidelity across the board. Pacing follows the natural rhythm of speech, accents are reproduced more faithfully, and the emotional range is far wider, so a product pitch sounds enthusiastic while a compliance-focused update carries a more serious tone. 

Translations now fit the video they belong to. The old system struggled with timing when a translated line ran longer than the original, forcing segments to be extended. The new translation engine respects the original timing and length, so far fewer segments need extension or manual repair. Alongside that, glossary support finally works with dubbing. Terminology that matters to your business such as product names or regular phrases stay consistent across every language, every time, without someone having to adjust it by hand.

Finally, the new workflow lets you edit once and regenerate once. You can review and refine transcripts and translations without burning credits on every pass, and when you find a segment that needs a fix, you correct that one segment and regenerate only that part. There is no longer a penalty for getting it exactly right. Dubbing becomes a process you commit to all at once into one you can iterate on, the way you would iterate on any other piece of content.

Why this matters for businesses

Until now, a company dubbing twenty videos into multiple languages faced a choice that was really no choice at all. You either brought in external support and paid for it in budget and lead time, or you took on heavy post-production internally and paid for it in your team's hours. Dubbing 2.0 removes that trade-off, because the first pass is genuinely usable. 

For learning and development teams, this means compliance training can go live in every market on schedule rather than staggered across weeks as each language catches up. A single source course becomes a full multilingual rollout without a specialist localization project wrapped around it.

For corporate communications, it means a CEO message that actually sounds like the CEO in every language. The tone, the pace, and the intent survive translation, so a message meant to reassure or to rally lands the same way for an employee in Tokyo as it does for one in Toronto.

For marketing teams, it means product explainers and campaign videos ship to every region when they need to ship, not weeks later once localization has run its course. Regional launches can move together instead of waiting on the slowest language in the set.

As Florian Metz, global head of analytics and AI product portfolio at Merck Group put it:

“Dubbing 2.0 has allowed us to scale our multilingual content in a way that was simply not possible before. We are looking forward to what comes next - the product keeps evolving, and so do the possibilities for our teams.”

The through line across all of these is speed without a quality tax. Teams have always been able to move fast if they were willing to accept rough output, or produce polished output if they were willing to wait. Dubbing 2.0 is our attempt to give them both at once, and the results are so good that we are now using the tool for our own videos. We would not put a first-pass output in front of our customers if we were not willing to put it in front of our own audience first.

Dubbing 2.0 is available now

Dubbing 2.0 is live for all customers today. Enterprise plans unlock the frictionless editing workflow and unlimited dubbing options, which is where high-volume localization programs will see the biggest gains. You can try it for free or book a demo to see it run on your own content.

‍