We’re Testing Our Own AI Translator

As you might have noticed, Enonic has released a brand new AI feature.

Juke—the purple alien thing—is both our mascot and your new AI-driven virtual "colleague", whose job it is to help your daily content management tasks directly in Content Studio.

In his first iteration, Juke provides two services related to content:

1. Creation and modification
2. Translation

Just how good is Juke at translating stuff? We're eating our own dogfood and will show you our findings here. Keep on reading!

Test: From English to Norwegian

In our test, we will translate from English to Norwegian. For our non-Norwegian readers, we're sorry, but you'll have to trust our quality judgements on this one.

We have of course set up an English parent layer in Content Studio, with a Norwegian child layer below it. This way, all content items from the English layer is inherited to the Norwegian layer. Here we can freely choose what inherited items to localize, archive, and publish—as well as creating unique content.

Here is our test blog post in English, about spinning:

Now let's translate the blog post to Norwegian by using the layers widget in the context panel. We're first navigating to the Norwegian layer. When we hit "localize", we are asked if we want Juke's assistance in translating to Norwegian. Of course we do!

After a few seconds, all three relevant fields in the blog post—title, intro text, and body text—are translated.

So, what's good and what could be better? For the record, Juke's using Gemini 2.0 via the Google Vertex API. Let's take a closer look.

The English original version had totally fine English language about the benefits of spinning. But how fares the Norwegian translation?

Here is the English title and intro:

And here is the Norwegian title and intro:

It's by no means a terrible translation. The meaning behind the original text and intro is preserved. But is it perfect Norwegian? Nope!

Firstly, the title should have had a small letter in the first word: "7 fordeler med spinning". We Norwegians don't capitalize the first letters of words in titles like in English.

Secondly, parts of the intro is clearly directly translated from English. The sentence "den spennende verdenen av spinningtimer" ("the exhilarating world of spinning cycle group lessons") would sound much more Norwegian as "spinningtimenes spennende verden". It's just a Norwegian thing.

And now the first paragraph in the body text. The English original:

The Norwegian version:

The Norwegian is fine here, but we had a surprise when the translation added an example with a skiing trip in Nordmarka ("the Northern Woodland"), north of Oslo.

We had actually forgotten that we previously had added a custom general instruction to the Juke AI Translator. The instruction specifies that the translations should keep references to Norwegian customs, culture, celebrities, language, and sayings:

The rest of the translation is perfectly serviceable in Norwegian, but at the end there are some really forced and awkward references to Norway. English original:

Norwegian:

Both of these paragraphs cram in desperate references to Norwegian: "som å ha "stå-på-vilje", slik vi sier i Norge" ("like having an "iron will", as we say in Norway") and "det er jo sosialt å trene sammen", som vi nordmenn liker å si" ("working out together is social, like we Norwegians use to say").

These references are quite unnecessary and forced, but we'll bear the full responsibility for their inclusion due to our insistent general instruction to include Norwegian references.

Otherwise, the translation is good enough!

Comparing with Other AI Services

As mentioned, Juke uses the Gemini 2.0 model from Google Vertex AI. Let's see how well the same English source material fares in other AI services.

ChatGPT

Arguably the most famous generative text AI on the market, OpenAI's ChatGPT revolutionized the mainstream entrance into the world of AI. Is it the best in translation, though? We're using the model o1.

ChatGPT insists on capitalizing the words in the title, and also uses the very English-sounding "den spennende verdenen av spinning-gruppetimer" instead of the more Norwegian-sounding s-genitive version.

Overall, the quality of the ChatGPT translation rivals that of Gemini. While none would win any translation prices, they get the job done without too much AI taint.

DeepSeek

Chinese-run DeepSeek made headlines recently due to its highly efficient model. But can it compete with the others in terms of translation? DeepThink (R1) delivers the following:

DeepSeek actually manages to keep the title fully correct in terms of capitalization, but the s-genitive is absent also here. 

Despite its impressive efficiency in energy and cost, DeepSeek delivers a bit weaker translation than Gemini and ChatGPT.

It uses difficult phrases ("Den høye intensiteten i treningen øker metabolisme" – the last word should also be definite singular, and not indefinite singular as in the present example) when easier alternatives exist (stoffskifte or forbrenning).

There are also some spelling mistakes present ("Hop på en sykkel").

Grok

X (formerly known as Twitter) has also skin in the AI game, and their Grok model has been touted as more "open and humorous" than its compatriots. How well does that translate into, well, translations?

Grok 3 reintroduces the capitalized letters, but is still one qualitative step above DeepSeek. There are no silly spelling mistakes, and it even uses definite singular where appropriate.

However, the use of difficult words instead of more easily understood descriptive terms as in Gemini or ChatGPT makes Grok rank slightly below them, but above DeepSeek.

Llama

Venice AI is a service offering what they call "uncensored" results within generative text, image, and code. We're testing out their most intelligent model—Llama 3.1 405B.

It seems we have saved the worst for last. In addition to struggling with the same stuff as the other AI services, Venice/Llama uses some insane words ("eksalterende", "stamina").

It also introduces spelling mistakes ("pedal", "energienivåer") and what we in Norwegian call "særskriving"/"special writing" ("spinning sykkel gruppetimer", "spinning revolusjonen").

This practice cuts compound nouns into its constituent parts, which is correct in English, but not in Norwegian.

Let's just say we're glad we're not using this "intelligent" model for Juke AI! 

Conclusion

All in all, translating content with Juke AI in Enonic is a smooth process, using one of the most powerfull AI models out there, in the guise of Gemini from Google.

The combination of Enonic's innovative content layering system and Juke's AI translation will make it much easier to manage your sites across different markets and geographies.

We can't wait to release new helpful features to Juke in the future, so your content management will be even smarter!

Do you have any questions regarding our AI Translator or content layering system? Please contact us and let us know!