= Adding i18n/l10n to Trac plugins ^(Trac >= 0.12)^ = [[PageOutline(2-5)]] == Intro and Motivation == If you want to learn about translation for a plugin that already has translated end-user display strings, then the section '[#Dotranslatorswork Do translators work]' and following parts will be useful. === i18n, l10n introduction '''i18n''' stands for '''`i`'''`nternationalizatio`'''`n`''' (count 18 chars between i and n) and is defined as software design for programs with translation support. '''`l`'''`ocalisatio`'''`n`''' that is abbreviated as '''l10n''' could be seen as a follow-up process providing data for one or more locales. It is taking care of feature differences between the original/default (that is English in most cases including Trac) as well as a given locale. Such features are eg sentence structure, including punctuation and formatting of numbers, date/time strings and currencies. Once you did some ground work at the source (i18n), what remains is proper translation work (l10n), preserving the meaning of the original while looking as native locale as possible.^[#a1 1]^ '''NLS''' (National Language Support or Native Language Support) is meant to be the sum of both.^[#a1 1], [#a2 2]^ === Background and concept of i18n/l10n support for Trac plugins It begun with adding [Babel:] to Trac, a powerful translation framework. For one part, it is a message extraction tool: it can extract messages from source code files (in our case, Python and Javascript) as well as from the Genshi templates, and create so-called catalog templates (`.pot` files). It can also create and update the message catalogs (`.po` files), and compile those catalogs (`.mo` files). For the other part, as a Python library used within Trac, it provides the implementation of the message retrieval functions (`gettext` and related). For more information, see [http://babel.edgewall.org/ Babel]. Some plugin maintainers created their own translation module inside each plugin separately. Growing amount of code redundancy and possibility of error within imperfect copies and variants of a translation module was not a desirable situation. And Trac core maintainers took responsibility with adding functions dedicated to i18n/l10n support for Trac plugins. The evolution of this functions has been documented in [comment:11:ticket:7497 ticket 7497]. The final implementation as mentioned there in [comment:12:ticket:7497 comment 12] was introduced to Trac trunk in changeset r7705 and finally done with changeset r7714. Now adding the needed i18n/l10n helper functions is done by importing a set of functions from `trac/util/translation.py` and providing the necessary extra information (''domain'') for storing and fetching the messages from the plugin code into plugin specific message catalogs. During plugin initialization, the dedicated translation domain is created as well and corresponding catalog files holding translated messages are loaded in memory. If everything is setup correctly, when a translatable text is encountered at runtime inside the plugin's code, the i18n/l10n helper functions will try to get the corresponding translation from a message catalog of the plugin's domain. The message catalog selection is done according to the locale setting. Valid settings are a combination of language and country code, optionally extended further by the character encoding used, i.e. to read like ‘de_DE.UTF-8’. Trac uses UTF-8 encoding internally, so there is not much to tell about that. 'C' is a special locale code since it disables all translations and programs use English texts as required by POSIX standard.^[#a3 3]^ == Required workflow == You need to: - specify in your plugin's `setup.py` file on which files the Babel commands will have to operate - create a `setup.cfg` files for adding options to the Babel commands - in your Python source code: - define specializations of the translation functions for your specific domain; there is a helper function for doing that easily - in the "root" `Component` in your plugin (one you're sure is always enabled) and initialize the translation domain in its `__init__` method - use your translation functions appropriately - in your Genshi templates: - be sure to have the necessary namespace declaration and domain directive in place - use the i18n: directive as appropriate - in your Javascript code: - be sure to load your catalog and define your domain specific translation functions - use the translation functions as appropriate === Enable Babel support for your plugin === ==== Add Babel commands to the setup (`setup.py`) Babel by default only extracts from Python scripts. To extract messages from Genshi templates also, you'll have to declare the needed extractors in [=#setup `setup.py`]: {{{#!diff diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -34,6 +35,21 @@ from setuptools import find_packages, setup +extra = {} +try: + from trac.util.dist import get_l10n_cmdclass + cmdclass = get_l10n_cmdclass() + if cmdclass: # Yay, Babel is there, we've got something to do! + extra['cmdclass'] = cmdclass + extractors = [ + ('**.py', 'python', None), + ('**/templates/**.html', 'genshi', None), + ('**/templates/**.txt', 'genshi', { + 'template_class': 'genshi.template:TextTemplate', + }), + ] + extra['message_extractors'] = { + 'foo': extractors, + } +except ImportError: + pass + setup( name = 'foo', version = '0.12', @@ -53,6 +69,7 @@ 'templates/*.txt', 'htdocs/*.*', 'htdocs/css/*.*', + 'locale/*/LC_MESSAGES/*.mo', ] }, install_requires = [ @@ -96,4 +113,5 @@ ] }, test_suite = '.tests', + **extra ) }}} ==== Preset configuration for Babel commands (`setup.cfg`) ==== Add some lines to `setup.cfg` or, if it doesn't exist by now, create it with the following content: {{{#!ini [extract_messages] add_comments = TRANSLATOR: msgid_bugs_address = output_file = /locale/messages.pot # Note: specify as 'keywords' the functions for which the messages # should be extracted. This should match the list of functions # that you've listed in the `domain_functions()` call above. keywords = _ N_ tag_ # Other example: #keywords = _ ngettext:1,2 N_ tag_ width = 72 [init_catalog] input_file = /locale/messages.pot output_dir = /locale domain = foo [compile_catalog] directory = /locale domain = foo [update_catalog] input_file = /locale/messages.pot output_dir = /locale domain = foo }}} Replace `` as appropriate (i.e. the relative path to the folder containing the `locale` directory, for example `mytracplugin`). This will tell Babel where to look for and store message catalog files. In the `extract_messages` section there is just one more lines you may like to change: `msgid_bugs_address`. To allow for direct feedback regarding your i18n work, add a valid e-mail address or a mailing list dedicated to translation issues there. The `add_comments` line simply lists the tags in the comments surrounding the calls to the translation functions in the source code that have to be propagated to the catalogs, see [Babel:wiki:Documentation/0.9/setup.html#extract-messages extract_messages] in Babel's documentation. So you will want to leave that one untouched. ==== Register message catalog files for packaging ==== To include the translated messages into the packaged plugin you need to add the path to the catalog files to `package_data` in the call for function `setup()` in `setup.py`: {{{#!diff diff -u a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -39,6 +39,7 @@ package_data = { : [ 'htdocs/css/*.css', + 'locale/*/LC_MESSAGES/*.mo', ], }, entry_points = { }}} === Make the Python code translation-aware ==== Prepare domain-specific translation helper functions ==== Pick a unique name for the domain, as this will be the basename for the various translation catalog files, eg `foo/locale/fr/LC_MESSAGES/foo.po` for the French catalog. At run-time, the translation functions (typically `_(...)`) have to know in which catalog the translation will be found. Specifying the 'foo' domain in every such call would be tedious, that's why there's a facility for creating partially instantiated domain-aware translation functions: `domain_functions`. This sample helper function should be called at module load time: {{{#!python from trac.util.translation import domain_functions _, tag_, N_, add_domain = \ domain_functions('foo', ('_', 'tag_', 'N_', 'add_domain')) }}} The translation functions which can be bound to a domain are: - `'_'`: extract and translate - `'ngettext'`: extract and translate (singular, plural, num) - `'tgettext'`, `'tag_'`: same as `'_'` but for Markup - `'tngettext'`, `'tagn_'`: same as `'ngettext'` but for Markup - `'gettext'`: translate //only//, don't extract - `'N_'`: extract //only//, don't translate - `'add_domain'`: register the catalog file for the bound domain Note: `N_` and `gettext()` are usually used in tandem. For example, when you have a global dict containing strings that need to extracted, you want to mark those strings for extraction but you don't want to put their //translation// in the dict: use `N_("the string")`; when you later use that dict and want to retrieve the translation for the string corresponding to some key, you don't want to mark anything here: use `gettext(mydict.get(key))`. To inform Trac about where the plugin's message catalogs can be found, call the `add_domain` function obtained via `domain_functions` as shown above. One place to do this is in the `__init__` function of your plugin's main component, like this: {{{#!python def __init__(self): import pkg_resources # here or with the other imports # bind the 'foo' catalog to the specified locale directory try: locale_dir = pkg_resources.resource_filename(__name__, 'locale') except KeyError: pass # no locale directory in plugin if Babel is not installed else: add_domain(self.env.path, locale_dir) }}} assuming that folder `locale` will reside in the same folder as the file containing the code above, referred to as `` below (as can be observed inside the Python egg after packaging). The i18n/l10n helper functions are available inside the plugin now, but if the plugin code contains several python script files and you encounter text for translation in one of them too, you need to import the functions from the main script, say its name is `api.py`, there: {{{#!python from api import _, tag_, N_ }}} ==== Mark text for extraction ==== To mark text for translation by helper programs, wrap the relevant Python code with the translation function `_()`: {{{#!diff --- a//api.py +++ b//api.py @@ -1,1 +1,1 @@ - msg = 'This is a msg text.' + msg = _("This is a msg text.") }}} Note, that quoting of (i18n) message texts should be done in double quotes. Single quotes are reserved for string constants (see commit note for r9751). If you fail to find all desired texts, you will notice this by seeing those messages missing from the message catalog. If the plugin maintainer is unaware of your i18n work or unwilling to support it and he adds more messages without the translation function call, you have to do the wrapping of these new texts too. === Make the Genshi templates translation-aware See the Genshi documentation on this topic, [http://genshi.edgewall.org/wiki/Documentation/0.6.x/i18n.html Internationalization and Localization]. ==== Text extraction from Python code and Genshi templates ==== Message extraction for Genshi templates should be done automatically. However there is the markup `i18n:msg` available to ensure extraction even from less common tags. For a real-world example have a look at [changeset:9542/trunk/trac/ticket/templates/ Trac SVN changeset r9542] for marking previously undetected text in templates. ==== Runtime support Extraction is automatic, however message retrieval at runtime is not. You have to make sure you've specified the appropriate //domain// in your template, by adding a `i18n:domain` directive. Usually you would put it in the top-level element, next to the mandatory `xmlns:i18n` namespace declaration. For example: {{{#!xml ... }}} === Make the Javascript code translation-aware ==== Text extraction from Javascript code ==== #Javascript Adding support for translating the marked strings in the Javascript code is a bit more involved. We currently support only statically deployed Javascript files, which means they can't be translated like template files on the server, but that the translation has to happen dynamically on the client side. To this end, we want to send an additional `.js` file containing a dictionary of the messages that have to be translated, and only those. In order to clearly identify which strings have to be present in this dictionary, we'll extract the messages marked for translation (the usual `_(...)` ways) from the Javascript code into a dedicated catalog template, and from there, we'll create dedicated catalogs for each locale. In the end, the translations present in each compiled catalog will be extracted and placed into a `.js` file containing the messages dictionary and some setup code. The first change is to use `get_l10n_js_cmdclass` in lieu of `get_l10n_cmdclass`. The former adds a few more setup commands for extracting messages strings from Javascript `.js` files and `