Edgewall Software
Modify

Opened 11 years ago

Closed 8 years ago

Last modified 8 years ago

#6353 closed enhancement (fixed)

Translation Interfaces for javascript

Reported by: trac-ja@… Owned by: Jun Omae
Priority: high Milestone: 0.12
Component: i18n Version: devel
Severity: major Keywords: i18n javascript
Cc: trac-ja@…, jun66j5@…, Pedro Algarvio, aka, s0undt3ch, Christopher Lenz
Release Notes:
API Changes:

Description

There are some message strings in javascript:

I'd like to translate these strings, but there are no interfaces for I18N.

Do you have any ideas?

Attachments (5)

trac-jsi18n.patch (5.1 KB ) - added by Jonas Borgström 10 years ago.
A first prototype for i18n support for javascript
babel-l10n-js.diff (7.1 KB ) - added by Jun Omae 9 years ago.
trac-l10n-js.diff (12.4 KB ) - added by Jun Omae 9 years ago.
l10n-js-r9738.diff (23.6 KB ) - added by Jun Omae 9 years ago.
patch for javascript l10n based on r9738
use-presentation-to_json.diff (2.9 KB ) - added by Christian Boos 9 years ago.
setup: remove to_json compatibility code and instead reuse the one from trac.util.presentation

Download all attachments as: .zip

Change History (31)

comment:1 Changed 11 years ago by trac-ja@…

Cc: trac-ja@… added
Milestone: 0.12

I tried to write a patch to use Genshi text template in javascript. But it conflicts with $ between jQuery and Genshi.

comment:3 Changed 10 years ago by Christian Boos

Component: generali18n
Milestone: 0.130.12

Changed 10 years ago by Jonas Borgström

Attachment: trac-jsi18n.patch added

A first prototype for i18n support for javascript

comment:4 Changed 10 years ago by Jonas Borgström

I just attached a first attempt at i18n support for javascript.

Most of it is pretty straight forward and the babel javascript extractor works perfectly.

One problem that is still unsolved is how to reduce the size of the js message catalog. The current implementation includes the entire message catalog which isn't very practical. One option would be to store the js strings in a separate domain but I'm not sure the babel extractor currently can extract messages into more than one domain.

comment:5 Changed 10 years ago by Christian Boos

Keywords: i18n added

comment:6 Changed 9 years ago by Christian Boos

We should use babel:source:tags/0.9.4/contrib/babel.js and a specific domain for .js files.

comment:7 Changed 9 years ago by Christian Boos

Priority: normalhigh

comment:8 Changed 9 years ago by Christian Boos

Keywords: javascript added
Milestone: 0.120.12.1
Severity: normalmajor

We should try to get this for the 0.12 release…

comment:9 Changed 9 years ago by Carsten Klein <carsten.klein@…>

please make all translations part of the server process, do not implement it as part of the client.

by that, make all javascripts, upon adding them to the response, also a genshi text template that then would be processed by genshi in order to resolve translation issues.

for plugins that will not use _(…) for translation in their javascript genshi plain text templates, the author of the plugin must be made responsible.

comment:10 in reply to:  9 Changed 9 years ago by anonymous

Milestone: 0.12.1next-major-0.1X

Replying to Carsten Klein <carsten.klein@…>:

please make all translations part of the server process, do not implement it as part of the client.

by that, make all javascripts, upon adding them to the response, also a genshi text template that then would be processed by genshi in order to resolve translation issues.

for plugins that will not use _(…) for translation in their javascript genshi plain text templates, the author of the plugin must be made responsible.

I was initially skeptical about this approach, thinking that this goes against the existing Javascript support in Babel (comment:6), and that it could be too costly to have to serve the htdocs/js files dynamically…

But since then, we discussed the migration to Jinja2 and in this new context, I think your proposal makes perfectly sense: much simpler and still reasonably fast.

Given that there will be only ${_(...)} expressions in the template, I think it should even be possible to use the same templates indifferently with Jinja2 or Genshi, making a 0.12.x backport possible. But the primary target should be 0.13 or some later milestone.

comment:11 Changed 9 years ago by Christian Boos

(comment above by me)

comment:12 Changed 9 years ago by Jun Omae

Cc: jun66j5@… added

comment:13 Changed 9 years ago by Jun Omae

I attached the patch that supports javascript localization for babel and trac. It works fine for me.

babel-l10n-js.diff

New generate_messages_js distutils.command generates javascript files from compiled MO files. The command is used after compile_catalog command.

Generated javascript files load translations using trunk/contrib/babel.js in babel.

trac-l10n-js.diff

setup.py generate_messages_js command generates trac/htdocs/js/messages/*.js from trac/locale/*/LC_MESSAGES/messages.mo.

trac/htdocs/js/babel.js and trac/htdocs/js/messages/<locale>.js are added in trac/web/chrome.py.

Changed 9 years ago by Jun Omae

Attachment: babel-l10n-js.diff added

Changed 9 years ago by Jun Omae

Attachment: trac-l10n-js.diff added

comment:14 Changed 9 years ago by Christian Boos

Milestone: next-major-0.1X0.12

Looks really promising! Thanks for working on this.

  1. we should use a specific domain here; unless I'm mistaken, with the current patches, the messages/*.js will contain all the strings?
  2. the babel-l10n patch should perhaps be re-targeted to Trac, we could also trigger the generate_messages_js step right after compile_catalog (r9638)
  3. the add_javascript lines should be conditional, it's only needed if Babel is used

Let's see how it goes, eventually it can even make it for 0.12 ;-)

comment:15 in reply to:  14 ; Changed 9 years ago by Jun Omae

  1. we should use a specific domain here; unless I'm mistaken, with the current patches, the messages/*.js will contain all the strings?

generate_messages_js command has -D option that is specified the domain. messages/*.js contain all strings in locale/*/LC_MESSAGES/<domain>.mo. Should the file path be messages/<domain>-<locale>.js?

  1. the babel-l10n patch should perhaps be re-targeted to Trac, we could also trigger the generate_messages_js step right after compile_catalog (r9638)

Ok. I will rework the patch for only trac. Because I expect that trac plugins use generate_messages_js command.

  1. the add_javascript lines should be conditional, it's only needed if Babel is used

Oh…, understand.

comment:16 in reply to:  15 ; Changed 9 years ago by Christian Boos

Cc: Pedro Algarvio, aka, s0undt3ch Christopher Lenz added

Replying to jomae:

  1. we should use a specific domain here; unless I'm mistaken, with the current patches, the messages/*.js will contain all the strings?

generate_messages_js command has -D option that is specified the domain. messages/*.js contain all strings in locale/*/LC_MESSAGES/<domain>.mo. Should the file path be messages/<domain>-<locale>.js?

What I'd like to have is an extraction of the messages in *.js files in trac/locale/messages-js.pot, the corresponding catalogs in locale/*/LC_MESSAGES/messages-js.{po,mo} and the .js dictionary generated in js/messages/<locale>.js.

  1. the babel-l10n patch should perhaps be re-targeted to Trac, we could also trigger the generate_messages_js step right after compile_catalog (r9638)

Ok. I will rework the patch for only trac. Because I expect that trac plugins use generate_messages_js command.

Yes, this ultimately belongs to Babel, but I don't want to depend on a new Babel release for Trac 0.12. But also submit your patch there, so that in a future version of Trac we could upgrade the requirements.

Changed 9 years ago by Jun Omae

Attachment: l10n-js-r9738.diff added

patch for javascript l10n based on r9738

comment:17 in reply to:  16 ; Changed 9 years ago by Jun Omae

Replying to cboos:

What I'd like to have is an extraction of the messages in *.js files in trac/locale/messages-js.pot, the corresponding catalogs in locale/*/LC_MESSAGES/messages-js.{po,mo} and the .js dictionary generated in js/messages/<locale>.js.

That's splendid! It's unacceptable that js/messages/<locale>.js are too large.

Yes, this ultimately belongs to Babel, but I don't want to depend on a new Babel release for Trac 0.12. But also submit your patch there, so that in a future version of Trac we could upgrade the requirements.

Agree. The patch for Babel is for the future.

I reworked the patch for javascript l10n - l10n-js-r9738.diff.

  1. Prepared the below commands for trac/locale/messages-js.pot.
    • extract_messages_js
    • init_catalog_js
    • compile_catalog_js
    • update_catalog_js
  2. generate_messages_js commands is implemented in setup.py.
  3. compile_catalog and generate_message_js commands are run before install_lib command like compile_catalog command.

I tested in python 2.6 and babel 0.9.5.

comment:18 Changed 9 years ago by hasienda <hoff.st@…>

reminder: as soon as you come to a final implementation, please update http://trac.edgewall.org/wiki/CookBook/PluginL10N#i18nforJavascripts, thanks.

comment:19 Changed 9 years ago by Christian Boos

Owner: changed from Jonas Borgström to Jun Omae

I've now tested the patch, but the generate_messages_js command fails for me:

# PYTHONPATH=. python setup.py generate_messages_js  -l fr
running generate_messages_js
generating messages javascript 'trac/locale\\fr\\LC_MESSAGES\\messages-js.mo' to 'trac/htdocs/js/messages\\fr.js'
Traceback (most recent call last):
  File "setup.py", line 333, in <module>
    **extra
  File "C:\Dev\Python261\lib\distutils\core.py", line 152, in setup
    dist.run_commands()
  File "C:\Dev\Python261\lib\distutils\dist.py", line 975, in run_commands
    self.run_command(cmd)
  File "C:\Dev\Python261\lib\distutils\dist.py", line 995, in run_command
    cmd_obj.run()
  File "setup.py", line 108, in run
    t = Translations(infile, self.domain)
  File "c:\dev\python261\lib\site-packages\Babel-0.9.5-py2.6.egg\babel\support.py", line 283, in __init__
    gettext.GNUTranslations.__init__(self, fp=fileobj)
  File "C:\Dev\Python261\lib\gettext.py", line 180, in __init__
    self._parse(fp)
  File "C:\Dev\Python261\lib\gettext.py", line 287, in _parse
    tlen, toff = unpack(ii, buf[transidx:transidx+8])
struct.error: unpack requires a string argument of length 8

It happens on a file with all the translations, but also on a file freshly generated and compiled (init_catalog_js -l fr, compile_catalog_js -l fr), so the issue should be easy to reproduce.

Ok, turns out it was a missing 'b'inary spec while opening the file (the joy of Windows ;-) ).

Now it works perfectly! Great job, I'll integrate this tomorrow.

comment:20 Changed 9 years ago by Christian Boos

Btw, I wanted to tweak the code a bit to re-use our trac.util.presentation.to_json, but thanks to the loading of trac/utils/__init__.py … this doesn't work (chokes on importing genshi.builder, via trac.util.translation and trac.util.text). So it looks like we have to keep this second implementation in place. Other ideas?

comment:21 in reply to:  20 Changed 9 years ago by Remy Blank

Replying to cboos:

(chokes on importing genshi.builder, via trac.util.translation and trac.util.text)

What exception do you get?

comment:22 Changed 9 years ago by Christian Boos

Oh, my bad, I used a Python version which had no Genshi installed. So yes, this could work.

Changed 9 years ago by Christian Boos

setup: remove to_json compatibility code and instead reuse the one from trac.util.presentation

comment:23 Changed 8 years ago by Christian Boos

Resolution: fixed
Status: newclosed

Patch committed in r9758 and several tweaks done in the follow up changesets.

In r9763, I moved most of the distutils related code in source:trunk/trac/util/dist.py. There are two benefits:

  • setup.py is getting even clearer than before (the r9638 changes are also moved there)
  • the plugins can now easily reuse our special tweaks by importing either get_l10n_cmdclass or get_l10n_js_cmdclass from that module (depending whether they need to translate Javascript or not); it remains to be seen if more tweaks are needed in order to have js translation working for plugins, or if everything can be done with the config files (see _js sections in source:trunk/trac/setup.cfg for an example)

Note that in r9764, I adapted the Makefile L10N targets so that they take the messages-js.{po,pot} files into account.

Finally, as an added bonus, in r9766 I added an extractor in order to get a few messages defined in <scripts> tags. Quite mysteriously, it seems that the normal genshi:**.html extraction doesn't catch those messages redundantly… good!

It would be good to have some testing feedback on the changes, and a second check that I didn't forget any Javascript message. See also the updates in TracL10N@104#ForCommitters.

comment:24 in reply to:  23 Changed 8 years ago by Christian Boos

… it seems that the normal genshi:**.html extraction doesn't catch those messages redundantly…

Of course, as the _(...) annotations in <script> tags are not contained within Genshi expressions, so they just go through the Genshi extractor unnoticed, as text.

comment:25 in reply to:  17 ; Changed 8 years ago by Christian Boos

Replying to jomae:

(about babel-l10n-js.diff) The patch for Babel is for the future.

Jun, do you intend to submit this to the Babel project? If so, don't forget to integrate r9762 in the patch.

comment:26 in reply to:  25 Changed 8 years ago by Jun Omae

Replying to cboos:

Jun, do you intend to submit this to the Babel project? If so, don't forget to integrate r9762 in the patch.

No. No need to submit the patch, cause you have implemented get_l10n_cmdclass and get_l10n_js_cmdclass methods in trac/util/dist.py. The plugins for Trac 0.12+ should use these methods in the setup.py.

These get_*_cmdclass methods are excellent! Thanks.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jun Omae.
The resolution will be deleted.
to The owner will be changed from Jun Omae to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.