Internationalization (i18n)

SecureDrop is translated into a number of languages. We use a web-based collaborative translation platform called Weblate to make it easier. Under the hood, all translation is done using GNU gettext.

With gettext, text to be translated is specially marked in source code. A Python example:

if not (msg or fh):
    flash(gettext("You must enter a message or choose a file to submit."), "error")
    return redirect(url_for('main.lookup'))

In this code, the string You must enter a message or choose a file to submit. can be automatically extracted for translation. The gettext function to which it is passed is used as a marker by pybabel or similar tools to collect the strings to be translated and store them into a .pot file at securedrop/translations/messages.pot. For instance:

#: source_app/main.py:111
msgid "You must enter a message or choose a file to submit."
msgstr ""

The .pot file serves as a template for all the language-specific .po files, which are where Weblate stores the contributed translations. For each language to be translated, a directory is created, such as securedrop/translations/fr_FR, and populated with a .po file derived from the template. For instance, securedrop/translations/fr_FR/LC_MESSAGES/messages.po is almost identical to securedrop/translations/messages.pot except for the msgstr fields, which will contain the French translations, e.g.:

#: source_app/main.py:111
msgid "You must enter a message or choose a file to submit."
msgstr "Vous devez saisir un message ou sélectionner un fichier à envoyer."

There’s one last type of file in the gettext system, a machine-readable version of the .po translations called a .mo file. Applications use these to get translations at runtime. The .po files are compiled to .mo files when the SecureDrop package is built.

The desktop icons installed on SecureDrop workstations are also translated. The icon templates are in the install_files/ansible-base/roles/tails-config/templates directory. Their labels are collected in the desktop.pot file and translated in the corresponding .po files in the same directory (fr.po, de.po etc.). All translations are merged from the *.j2.in files into the corresponding *.j2 file and committed to the SecureDrop repository. They are then installed when configuring Tails with the tasks/create_desktop_shortcuts.yml tasks.

We don’t expect translators to deal with all these files directly. Translation happens on our Weblate server, which is configured to use a fork of the main SecureDrop repository.

As string changes are merged into the develop branch in the main SecureDrop repository, the changes will automatically appear in Weblate, and translation can begin. Translation and review can take place continuously, at any time. Translations for supported languages reviewed and finalized during the release process.

What languages are available where?

  • All languages translated in Weblate are present in the securedrop/translations directory.

  • Supported languages are listed in the supported_locales object in the i18n.json file.

  • Those languages that are both present and supported are available for administrators to configure in securedrop-admin sdconfig.

  • Those languages that are both configured and available on the Application Server are usable for users to select.

Development tasks

Add a new language

See How to add a new language to SecureDrop.

However, SecureDrop only supports a subset of all the languages being worked on in Weblate. New languages are supported according to the Policy on Supported Languages.

Update strings to be translated

Whenever strings are modified in the SecureDrop source, whether in Python code, HTML templates, or desktop icon labels, the translation files should also be updated by running make extract-strings in the root of the SecureDrop working copy.

The extract-strings target gathers source strings, then updates the .pot files for the SecureDrop server code and the desktop icons. (This step is enforced by CI, which will fail if you skip it.)

After running make extract-strings, carefully review the output of git diff. Check securedrop/messages.pot first for updated strings, looking for problems like:

  • overly idiomatic English

  • fragmented text, such as pieces of a sentence intended to be concatenated together, which can be difficult to translate

  • messages that are marked with plain gettext and contain plurals based on numeric placeholder variables – these should generally be marked with ngettext so that they can be translated properly in languages with complex plural forms

Then review the messages.po of one existing translation. There is no need to review multiple languages’ .po files because they are processed in the same way.

Commit and push these changes for review along with your source changes. See Translation Responsibilities for the complete workflow.

Verify translations

Content review is the responsibilty of Localization Lab and their reviewers, in Weblate’s review process. Security review of translations, primarily checking for malicious HTML or interpolation not caught by Weblate’s checks, is the responsibility of SecureDrop maintainers, especially the localization manager and release manager for a given release.

SecureDrop web interfaces (securedrop/securedrop)

After a translation is compiled, the web page in which it appears can be verified visually by starting the SecureDrop development servers and navigating via http://localhost:8080 for the source interface or http://localhost:8081 for the journalist interface. You can start the development servers with:

$ make dev

The translations can be checked automatically by running the SecureDrop page layout tests:

$ export PAGE_LAYOUT_LOCALES="en_US,fr_FR"  # may be set to any supported languages
$ make test TESTFILES=tests/functional/pageslayout
[...]
tests/pageslayout/test_journalist.py::TestJournalistLayout::test_account_edit_hotp_secret[en_US] PASSED
tests/pageslayout/test_journalist.py::TestJournalistLayout::test_account_edit_hotp_secret[fr_FR] PASSED
[...]

Note

if unset, PAGE_LAYOUT_LOCALES defaults to en_US (US English) and ar (Arabic).

After running the tests, screenshots for each locale are available in securedrop/tests/pageslayout/screenshots/<locale>, e.g. securedrop/tests/pageslayout/screenshots/fr_FR. Screenshot filenames can be found in the tests that created them, in securedrop/tests/pageslayout/test_journalist.py or securedrop/tests/pageslayout/test_source.py.

Desktop icons (securedrop/desktop)

The translated templates for the desktop icons are:

  • install_files/ansible-base/roles/tails-config/templates/desktop-journalist-icon.j2

  • install_files/ansible-base/roles/tails-config/templates/desktop-source-icon.j2

Check that each of them contains a Name line for each of SecureDrop’s supported locales.

If there have been new changes to the securedrop/desktop component, CI will fail on the pull request from Weblate with a warning to run make update-desktop-files. Run this command locally and push the changes to the weblate-fpf/securedrop fork, and CI should pass.

Update Weblate screenshots

You can use the script securedrop/upload_screenshots.py to update UI screenshots that are used to illustrate strings in Weblate. The script depends on the existence of up-to-date layout test results, which you can generate using this command in the base directory:

$ LOCALES=en_US make translation-test

Inspect the screenshots in the directory securedrop/tests/pageslayout/screenshots/en_US and make sure that their content corresponds to the expected version of the codebase.

Obtain your API key in Weblate. Export the token to the environment variable WEBLATE_API_TOKEN. You can now run this command to perform an upload:

$ securedrop/upload-screenshots.py

If new screenshots were added as part of this run, make sure to associate them with relevant strings in Weblate, which you can do from the screenshots list.

Release Management

Two weeks before the release: string freeze

Note

If both a Localization Manager and a deputy are assigned for this release, consider pairing on this ceremony, both for knowledge-sharing and so that the intermediate pull requests can be reviewed and merged promptly.

When features for a new SecureDrop release are frozen, so are the source strings. The localization manager should apply any source-string suggestions that have been made by translators before the release branch has been cut, since only translations will be updated for release candidates and the final release.

Then, the localization manager for the release will:

  • Update Weblate screenshots so translators can see new or modified source strings in context.

  • Add a Weblate announcement for the securedrop/securedrop component with the translation timeline for the release.

    • Important: Make sure the Notify users box is checked, so that translators receive an email alert.

    • You can view a history of past announcements in Weblate’s Django admin panel, or use this template:

      Translation for the SecureDrop X.Y.Z release has begun. If you have suggestions for source strings, please get them to us by YYYY-MM-DD. Translation will end on YYYY-MM-DD.

    • Set the Expiry date to release day itself (the day after the translation deadline).

  • Remind all developers about the string freeze in Gitter, for example using this template:

    Hello! We’ve just opened translations for the upcoming SecureDrop 2.3.0 release. If you have suggestions for source strings, please get them to us by 2022-03-20. Translation will end on 2022-03-27.

    Translations are done using Weblate (https://weblate.securedrop.org/projects/securedrop/securedrop/). If you haven’t used it before, <https://developers.securedrop.org/en/latest/translations.html> has instructions on how to get started.

  • Update Localization Lab via the SecureDrop Coordination channel in the TCU Mattermost.

  • During the feedback period, monitor Weblate comments and suggestions, and open a pull request for every source string suggestion coming from translators.

Remember that supported languages are the priority during this period. That is, while translation contributions are welcome for all languages, the pre-release goal is to keep the current set of supported languages at 100% translation in Weblate. Localization Lab can marshal individual translators to help meet this goal.

During QA

Review, merge, and backport Translations update from Weblate pull requests at most once before each release candidate is cut. Coordinate with the release manager.

Release day

Prior to cutting the final release, the localization manager must:

  • Review, merge, and backport the final Translations update from Weblate pull request.

  • Update the documentation screenshots.

  • Provide translator credits to add to the SecureDrop release announcement.

Then, post-release, either same day or day-after, the localization manager should:

  • Remove the Weblate announcement about this release’s translation timeline (if you set an end-date on the original announcement, this may happen automatically)

  • Update the tracking spreadsheet with supported languages’ current translation and review coverage. File a ticket for each new language due either (a) consideration for new support, (b) probation for dropping coverage, or (c) revocation of support.

Translator credits

Correct acknowledgment of translators’ contributions is important, so Weblate makes it easy to list the translators who have contributed in a specific period or at any point in the project’s history. For example, the Communications Manager for a release can generate a “Credits” report since the date of the last minor (X.Y.0) release and copy-paste it into the release notes.

Weblate administration

Note

The privilege escalation workflow is different for code maintainers and translation maintainers.

A translation admin has special permissions on Weblate and the repositories. When someone is willing to become an admin, a thread is started in the translation section of the forum. If there is consensus after a week, the permissions of the new admin are elevated. If there is not yet consensus, a public vote is organized among the current admins.

The privileges of an admin who has not been active for six months or more are revoked, but they can apply again at any time.

The community of SecureDrop translators works very closely with the SecureDrop developers and some of them participate in both groups. However, the translator community has a different set of rules and permissions, and therefore independent policies from SecureDrop itself.

Admin permissions

The full set of admin permissions can be granted at:

Granting reviewer privileges in Weblate

  • Visit https://weblate.securedrop.org/admin/weblate_auth/user/.

  • Click on the user name.

  • In the Groups block:

    • Select Localizationlab in the Available groups list and click on the right arrow to move it to the Chosen groups list.

    • Select Users in the Chosen groups list and click on the left arrow to remove it.

Update the Weblate full text index

Weblate’s full-text index can occasionally get out of sync. When this happens, Weblate’s search may fail to find a word that you know exists in the source strings. You can rebuild the index with:

$ ssh debian@weblate.securedrop.org
$ cd /app/weblate
$ sudo docker-compose run weblate rebuild_index --all --clean

Note that the new index may not be used right away. Some workers may still have the old index open. If the index is holding up translators with a release looming, the server can be rebooted.