# Internationalization (i18n) **Table of Contents** - [Internationalization (i18n)](#internationalization-i18n) - [Using Translations](#using-translations) - [URL Query Parameter](#url-query-parameter) - [Language Selector](#language-selector) - [Translations](#translations) - [The Workflow](#the-workflow) - [Staging Translation Repository](#staging-translation-repository) - [Synchronization](#synchronization) - [Uploading Translatable Texts](#uploading-translatable-texts) - [Downloading Updated Translations](#downloading-updated-translations) - [Weblate Configuration](#weblate-configuration) - [Plugins](#plugins) - [Technical Details](#technical-details) - [The Web Frontend](#the-web-frontend) - [Marking Texts for Translation](#marking-texts-for-translation) - [Singular Form](#singular-form) - [Plural Form](#plural-form) - [Translating Constants](#translating-constants) - [Formatting Texts](#formatting-texts) - [TRANSLATORS Comments](#translators-comments) - [Missing Translations](#missing-translations) - [Testing Language](#testing-language) - [Building POT File](#building-pot-file) - [Cockpit Details](#cockpit-details) - [Development Server](#development-server) - [D-Bus Backend](#d-bus-backend) - [Backend Translations](#backend-translations) - [Troubleshooting](#troubleshooting) --- Internationalization (sometimes abbreviated as "i18n") is a process for supporting different languages and locales in software. This documentation describes how is that handled in Agama. Each Agama part (the web frontend, the D-Bus backend and the command line interface) needs to solve this problem separately, see more details for each part in the details below. ## Using Translations Users have two ways how to change the used language in the Agama interface. ### URL Query Parameter When using a remote installation it is possible to set the used language via an URL query parameter. This is an expert option. To change the language append the `?lang=` query to the URL when accessing the Agama installer. The `locale` string uses the usual Linux locale format, e.g. `cs_CZ`. It is the user responsibility to use a correct locale name. When using a wrong name the translations might be broken, displayed only partially or even not at all. Changing the language causes reloading the page, in some situations this could cause losing some entered values on the current page. Therefore it is recommended to change the language at the very beginning. ### Language Selector *to be done...* ## Translations For translation process Agama uses [Weblate](https://weblate.org/) tool running at the [openSUSE](https://l10n.opensuse.org/) instance. ## The Workflow The basic translation workflow looks like this: 1. The translatable texts are marked for translation in the source code, usually with the `_()` function or something similar 2. The translatable string are collected into a POT file which is uploaded to the staging [agama-weblate](https://github.com/openSUSE/agama-weblate) GitHub repository 3. The POT file is loaded by the Weblate into the [agama](https://l10n.opensuse.org/projects/agama/) project 4. Translators then can translate the English texts into other languages 5. Weblate pushes back the translations in the PO files back to the [agama-weblate](https://github.com/openSUSE/agama-weblate) repository 6. The translations in the PO files are regularly copied to the main repository using pull requests 7. The PO files are processed during build so the translations can be used later at runtime *Note: The Agama workflow is pretty similar to the [Cockpit localization]( https://github.com/cockpit-project/cockpit/blob/main/doc/i18n.md) workflow, we decided to use a similar approach here.* ## Staging Translation Repository The special [agama-weblate](https://github.com/openSUSE/agama-weblate) repository works like a buffer between the Agama sources and the Weblate tool. We do not want to spam the Weblate tool with every trivial change in the texts and the other way round, we do not need to get dozen commits from the Weblate every day with updated translations. This would be especially annoying before releasing a new version where we might want to accept only unnecessary changes to not break something at the very last minute. The agama-weblate repository uses webhooks to notify the Weblate instance of any change. Changes in the repository should be visible in Weblate in matter of seconds. ## Synchronization The content between the main [agama](https://github.com/openSUSE/agama) and the translation [agama-weblate](https://github.com/openSUSE/agama-weblate) GitHub repositories is synchronized automatically using the GitHub Actions. ### Uploading Translatable Texts Collecting and uploading the translatable texts is done by the [weblate-update-pot.yml]( https://github.com/openSUSE/agama/blob/master/.github/workflows/weblate-update-pot.yml) GitHub action. - It checks out both `agama` and `agama-weblate` repositories - It runs a script which extracts the translatable strings in the `agama` repository into a POT file - The POT file is copied to the `agama-weblate` repository and committed to the GitHub repository This action is run daily, but it can be started manually if needed. Go to the [weblate-update-pot.yml]( https://github.com/openSUSE/agama/actions/workflows/weblate-update-pot.yml) action detail and use the "Run workflow" option at the top of the page. The code compares the old POT file with the new one and if there is no change besides the timestamps in the file it is not uploaded to `agama-weblate`. ### Downloading Updated Translations The translations from the `agama-weblate` repository are merged back by the [weblate-merge-po.yml]( https://github.com/openSUSE/agama/blob/master/.github/workflows/weblate-merge-po.yml) GitHub action. - It checks out both `agama` and `agama-weblate` repositories - It copies the PO files from the `agama-weblate` to the `agama` repository - It creates a pull request with the changes If there are no changes besides the timestamps in the PO files the pull request is not created. ## Weblate Configuration The [Agama Weblate](https://l10n.opensuse.org/projects/agama/) project defines a separate translation component for each Agama part (the web frontend, the D-Bus backend and the command line interface). For reading the translations it uses the [agama-weblate]( https://github.com/openSUSE/agama-weblate) GitHub repository, but for the source code locations it uses the original [agama](https://github.com/openSUSE/agama) repository. That means after clicking the source location link in the Weblate you will see the correct source location in the other repository. ### Plugins The Weblate components use the [msgmerge]( https://docs.weblate.org/en/weblate-4.9.1/admin/addons.html#addon-weblate-gettext-msgmerge) plugin which automatically updates all PO files to match the POT file. That means after adding or updating any translatable text the changes are automatically applied to all existing PO files and translators can translate the new or updated texts immediately. ## Technical Details This part describes technical and implementation details. It also describes the translation process for developers. ### The Web Frontend Most of the translatable texts which can user see in Agama are defined in the web frontend. However, the web frontend can display some translated texts coming from the backend part so it is important to set the same language in both parts and make sure the translations are available there. ### Marking Texts for Translation The texts are marked for translation using the usual functions like `_()`, `n_()` and others. It is similar to the GNU gettext style. Currently Agama uses the Cockpit implementation for loading and displaying translations. To make this process transparent for developers and to allow easy change of the implementation there is a simple [i18n.js]( ../web/src/i18n.js) wrapper which provides the translation functions: ```js import { _, n_, N_, Nn_ } from "~/i18n"; ``` It is important to use these functions only on string literals! Do not use them with string templates or string concatenation, that will not work properly. The `_` and `n_` functions can be used on variables or constants, but their content must marked for translation using the `N_` or `Nn_` functions on string literals. See the examples below. #### Singular Form This is the most common way: ```js const title=_("Storage"); ``` For using translated text in React components you need to use curly braces to switch to plain Javascript in the attributes and the content: ```js
``` or ```js ``` For translating long texts you might use multiline string literals: ```js const description = _("Select the device for installing the system. All the \ file systems will be created on the selected device."); ``` If you need to insert some values into the text you should use the [formatting function](#formatting-texts). It is recommended to use [translator comments](#translators-comments) for short or ambiguous texts. #### Plural Form If the translated text contains a number or quantity it is good to use a plural form so the translated texts looks naturally. ```js sprintf( // the first argument is a singular form used when errors.length === 1, // the second argument is used in all other cases n_("%d error found", "%d errors found", errors.length), errors.length ) ``` Although the English language has only a single plural form the translators can use more of them. The translation definition also defines a function that computes which plural form to use for a specific number. See more details about the [plural forms]( https://www.gnu.org/software/gettext/manual/html_node/Plural-forms.html) in the GNU gettext documentation. #### Translating Constants The top-level constants are evaluated too early, at that time the translations are not available yet and the language is not set. The constant value must be marked with `N_()` function which does nothing and then later translated with the usual `_()` function. ```js const LABELS = Object.freeze({ auto: N_("Auto"), fixed: N_("Fixed"), range: N_("Range") }); export default function Foo() { ... ``` #### Formatting Texts For formatting complex texts use the C-like `sprintf()` function. ```js import { sprintf } from "sprintf-js"; sprintf(_("User %s will be created"), user) ``` See the [sprintf-js](https://www.npmjs.com/package/sprintf-js) documentation for more details about the formatting specifiers. *Note: You cannot use HTML tags in the translated texts, React automatically escapes all special HTML characters like `<>&`. That is different to YaST where we can use HTML tags in rich text messages.* It is recommended to avoid building texts from several parts translated separately. This is error prone because the translators will not see the complete final text and might translate some parts inaccurately. ```js // do NOT use this! it is difficult to translate the parts correctly // so they build a nice text after merging return
{_("User ")}{user}{_(" will be created")}
// rather translate a complete text and then split it into parts // TRANSLATORS: %s will be replaced by the user name const [msg1, msg2] = _("User %s will be created").split("%s"); ... return
{msg1}{user}{msg2}
``` Text splitting might be quite complex in some cases, but still should be preferred. ```js // TRANSLATORS: error message, the text in square brackets [] is a clickable link const [msgStart, msgLink, msgEnd] = _("An error occurred. \ Check the [details] for more information.").split(/[[\]]/); ... return

{msgStart}{msgLink}{msgEnd}

; ``` Building sentences from separate parts might be easy in English, but is some other languages it might much more complex. Always assume that the target language has more complex grammar and rules. ```js // do NOT use this! it is difficult to translate "enabled" and "not enabled" // differently in the target language! const msgNot = enabled ? "" : _("not"); sprintf(_("Service is %s enabled"), msgNot); // this is better enabled ? _("Service is enabled") : _("Service is disabled"); ``` #### TRANSLATORS Comments You can use a special `TRANSLATORS` keyword in the comment preceding the text marked for translation. All comments after the keyword are included in the translations file, this should help the translator to correctly translate the text. ```js // this line is NOT included in the PO file // TRANSLATORS: this line is included in the PO file, // this line as well _("Back") ``` The translators comments should be used especially for short texts to better describe the meaning and the context of the message. ```js // TRANSLATORS: button label, going back to the main page _("Back") ``` Also the formatting placeholders should be always described. ```js // TRANSLATORS: %s will be replaced by the user name _("User %s will be created") ``` The JSX code does not support comments natively, you have to write them in curly braces. ```js ``` But in the component attributes you can use usual Javascript comments. ```js ``` The translators can change the order of the arguments if needed using additional positional arguments. For example to change the order of the arguments in `"Foo %s %s"` they can translate it as `"Bar %2$s %1$s"`. ### Missing Translations Here are some code examples which might look correct at the first sight. They even work, no crash. But there are still translation problems with them. - Do not use Javascript string templates, that does not work at all (~~``_(`User ${user} will be created`)``~~), use a string formatting function for that (`sprintf(_("User %s will be created"), user)`). - Use the translation functions only with string literals without any operators, texts like ~~`_("foo" + "bar")`~~ will not be extracted correctly to the POT file. In both cases the strings will not be extracted to the POT file. ### Testing Language The Agama web interface supports special testing `xx` language. That language needs to be selected manually by adding the `?lang=xx` query string to the server URL. With this setting the `_()` and `n_()` translation functions replace all alphabetical characters by the `x` symbol. This can be used for testing to find out the texts which are not marked for translation. If a text is not replaced by the `x` symbols then either it is not marked for translation or it comes from the backend and it should be translated by the backend. However, this is not perfect. It will replace also the texts mentioned in the [missing translations](#missing-translations) section above but in reality the there would be no translations available for them. ### Building POT File The translatable strings are extracted from the source files by using the standard `xgettext` tool. There is a helper script [build_pot](../web/build_pot) which defines the needed parameters for the `xgettext` tool. To build the POT file locally just run the script, it will save the output to the `agama.pot` file. ### Cockpit Details The translations are loaded by the `` HTML code in the [index.html](../web/src/index.html) file. But there is no such file in Agama (or in any Cockpit plugin in general), there are files `po..js` like `po.cs.js`. The trick is that the Cockpit server checks the `CockpitLang` cookie sent in the HTTP request header and returns the content from the respective file. ### Development Server Because Cockpit serves the `po.js` file a bit differently as described in the [Cockpit Details](#cockpit-details) section above we need to implement this logic also in the development server. The [webpack-po-handler.js](../web/src/lib/webpack-po-handler.js) file checks the language cookie and uses HTTP redirection to load the respective language file. It uses redirection because the built translation files are only available in the webpack memory. But the result is the same, the browser gets a different content according to the currently configured language. ### D-Bus Backend The D-Bus service backend implements a writable `UILocale` property which defines the locale used by the service. For debugging purposes you can get the current value or set a new one using command line: ```shell # get the current locale busctl --address=unix:path=/run/agama/bus get-property org.opensuse.Agama1 /org/opensuse/Agama1/Locale org.opensuse.Agama1.Locale UILocale # set a new locale busctl --address=unix:path=/run/agama/bus set-property org.opensuse.Agama1 /org/opensuse/Agama1/Locale org.opensuse.Agama1.Locale UILocale s cs_CZ ``` The `UILocale` property is a single global value shared by all connected clients. That means it is possible to use only one language for all clients, if more users connect to the server and uses a different UI language then there will be race conditions and the other users might see the texts coming from the backend in another language than expected. This is a known limitation, we expect that only one user at a time will access the Agama installer or if multiple users use the same server we expect that they will be from the same team or company using the same language. #### Backend Translations The backend might return texts from external components like `libstorage-ng`. Make sure the translations for those components are also available for Agama, e.g. the `libstorage-ng` translations are stored in the `libstorage-ng-lang` package or the YaST translations are stored in `yast2-trans-` packages. ## Troubleshooting Here are some hints what to do when some untranslated text appears in the Agama UI. 1. Check that the text is marked for translation, for a quick verification you might try setting the [testing language](#testing-language) 2. If the text comes from backend or the other parts check that the appropriate [language package](#backend-translations) is installed 3. The text should be [extracted into the POT file](#building-pot-file) 4. The [agama.pot]( https://github.com/openSUSE/agama-weblate/blob/master/web/agama.pot) in the `agama-weblate` repository is up to date, if not then run the [Weblate Update POT](https://github.com/openSUSE/agama/actions/workflows/weblate-update-pot.yml) Github Action manually 5. The text is translated in the [Weblate repository]( https://l10n.opensuse.org/projects/agama/) 6. The translation is included in the respective PO file in the [agama-weblate]( https://github.com/openSUSE/agama-weblate) repository 7. The PO file in the [agama]( https://github.com/openSUSE/agama/tree/master/web/po) repository is up to date, if not the check whether there is an [open pull request]( https://github.com/openSUSE/agama/pulls?q=is%3Aopen+is%3Apr+label%3Atranslations+label%3Abot) with the change, if yes then check it and merge, if not then run the [Weblate Merge PO](https://github.com/openSUSE/agama/actions/workflows/weblate-merge-po.yml) GitHub Action manually, then check the created pull request 8. The translated string should be present in the built packages, run `npm build` command and check the `dist/po.*.js` files, check the built RPM package 9. The translations are loaded by the browser, check the content loaded for the `po.js` file 10. Check the current language used in the browser, run `cockpit.language` command in the development tools console, check the `CockpitLang` cookie value (run `document.cookie` command in the console) 11. Check the language used by the [service backend](#d-bus-backend)