Experiment: German translation 🇩🇪🇨🇭🇦🇹 · mdn · Discussion #741

6 min read Original article ↗

tl;dr

We're launching an experimental German version of MDN, which is targeted to those that prefer to read MDN content in German rather than English. The content is automatically translated from English using OpenAI's latest GPT-4o model, and is now available on (developer.mozilla.org). The sources can be found in the mdn/translated-content-de repository.

Note: This is an experiment, and work in progress. Translations may contain inaccuracies.

Screenshot of the References landing page on MDN in German

Details

MDN was available in German (de) until August 2022, when we had to remove the locale, because it was incomplete, outdated, and no longer maintained. Although we hypothesized that there was still demand for the German locale, we saw no possibility to bring it back at the time: We had evaluated automatic translations as not good enough, and considered manual translation as impossible due to the volume of MDN content, and the pace at which it is updated.

In April 2024, we re-evaluated our options, and identified LLM-based machine translation as a potential solution to revive the German locale, and to maintain it in a semi-automated way going forward.

In May 2024, we ran initial tests with LLM-based machine translation from English to German, which confirmed that the approach yields sufficient results even with a simple prompt.

In June 2024, we ran a survey (see results below) that confirmed interest in the German locale, and informed us about translation criteria, and user concerns.

In September 2024, we launched the new experimental Remember language feature, which we had identified as a blocker for the German locale in the survey. The feature effectively allows users to opt-out of the German locale.

In October 2024, we launched the public preview of the German language in an isolated test environment.

Experiment

In November 2024, we're now launching the German language as an experiment on production (without exposure to search engines). This is the first fully-translated language covering all MDN Web Docs content with over 12'500 pages, translated from the English language using OpenAI's latest gpt-4o-2024-08-06 model. (Note that most of the user interface, MDN Plus, Curriculum, and Blog are not translated.)

In December 2024, we started exposing the German translation to search engines.

To provide feedback, please rate good and bad translations using the thumbs (👍👎) in the "MDN-Feedback-Box" below the articles. If you notice any issues, please use the "Report a problem with this translation" link, write us on Discord, comment below, or send us an email (mdn-german[at]mozilla[dot]com). Please respect the Mozilla Community Participation Guidelines to ensure a healthy and constructive collaboration.

Screenshot of the feedback box

Translation process

As of December 4, 2024, the translation process includes a daily automated workflow that includes the following steps:

  1. The English Markdown file is pre-processed, in particular by replacing code blocks with placeholders, converting selected macros to links, converting the YAML frontmatter to Markdown, and chunking longer documents.
  2. The resulting input is passed to GPT-4o with a system prompt.
  3. The resulting output is post-processed, in particular by cleaning up frontmatter, normalizing links, fixing well-known recurring syntax errors, re-inserting code blocks, and replacing link anchors.
  4. The resulting German Markdown files are committed with automatic markdownlint and prettier fixes to the mdn/translated-content-de repository.

Next steps

We believe that the machine-translated German locale is helpful in its current state, even if it may contain inaccuracies.

For now, we gather user feedback, and aim to iterate on a weekly basis to address issues, and improve the quality of the translation, and the reliability of the translation process.

Our vision is for German to become a regular translated locale, and the first locale to be maintained using a combination of machine translation, and community reviews.

Appendix: Survey

In June 2024, we ran a survey on MDN for 5 days that was shown to all users whose browser preference indicated German as the preferred language.

Goal

The goal of the survey was to test our hypothesis that there is demand for the German language, understand our users' language proficiency and preference, translation priorities, and concerns.

Participation

The survey was seen by over 60'000 users, and over 1'900 users completed it (3%).

Language proficiency and preference

We asked about language proficiency:

  • 90% of participants indicated German as their native language.
  • Of these, 33% rated their English as basic or advanced, not fluent.

(Note: German language preference correlated inversely with English language proficiency.)

We asked what language they would use if MDN was available in German:

  • 62% participants indicated a preference for English (54% would use mostly English, 8% more English),
  • 35% participants indicated a preference for German (24% mostly German, 8% more German), and
  • 3% participants indicated no preference.

Translation priorities

We asked what would be important in a German version of MDN:

  1. Technical correctness (89% Very important)
  2. Consistency of technical terms (73% Very important)
  3. Up-to-date ness (68% Very important)
  4. Completeness (56% Very important)
  5. Comprehensibility (50% Very important)
  6. Linguistic correctness (31% Very important)
  7. User interface in German (14% Very important)
  8. Graphics in German (8.3% Very important)
  9. Code examples in German (7.5% Very important)

Concerns

We asked an open-ended question, and 13% of participants answered it, providing context for their responses, or sharing concerns.

Among users who indicated no preference or a preference for German there were no significant patterns, apart from praise and gratitude (n = 18; thank you!).

Users who indicated a preference for English …

  • … explained their preference with programming languages and technical terms being in English (n = 59), and having better experiences with English documentation, being more complete and up-to-date (n = 44).
  • … rejected automatic or AI translations, criticizing them as inaccurate and incomprehensible (n = 34), and sharing negative experiences with machine-translated technical documentation (n = 19).
  • … emphasized that they don't want to be redirected to the German language (n = 21), even if their browser indicates German as the preferred language, and mentioned the need to permanently switch to the English version (n = 10).
  • … suggested having translations reviewed by the community to improve their quality (n = 12).
  • … acknowledged that some users, especially beginners, could benefit from a German translation to better understand technical concepts (n = 11).

Limitations

The main limitation of the survey is its appearance in the context of non-German content, so potential users who only speak German may be under-represented. On the other hand, users who are interested in the German locale may be more likely to respond to the survey, and therefore over-represented.