Multi-language support in the wiki

[Paul Baranowski <paul@…>]

Let trac be the first-ever multi-lingual wiki on the market! Have the ability to translate wiki articles into different languages. A user can select their language when they view the wiki. We have looked around and there isnt a wiki out there that handles this. And if you dont have time to do it, do you mind having some volunteers help you out? :)

[timonator@…]

the wikipedia has a multilanguage feature. every article has a little box on the left side that shows what other languages have this article. i dont know if this is a feature of the ​MediaWiki, tho…

[stephane]

You could check:

​http://moinmoin.wikiwikiweb.de/MultipleLanguagesSupport

but I don't know if its what you want

[stephane]

See also:

​http://moinmoin.wikiwikiweb.de/FeatureRequests/ThisPageInOtherLanguages

PS: MoinMoin is also python based :-)

[anonymous]

maybe this helps the case, tikiwiki also has this feature. please also introduce it in trac.

[Christian Boos]

#5238 was marked as duplicate.

There's also been a very interesting proposal by Shun-Ichi Goto, in ​googlegroups:trac-dev:ab6aa7900a471742 and ​TH:TracWikiNegotiatorPlugin.

[Stephan]

The multi-language feature shouldn´t be limited to the proposals above but should be extended to trac setup, too. Any text showing up in trac (e.g. the context navigation bar, search, "attach file", "edit this page" button) should be changeable. E.g., in trac.ini an entry languagefile=… could be added which points to a file trac_de.ini. This file enlists all text labels used in trac which can be adjusted according to the desired language.

Currently I can only adjust the mainnav and metanav bars. Having German header, footer, content and navigation bars mixed with English context navigation bar and button labels isn´t a maximal sophisticated feature ;)

[Jeroen Ruigrok van der Werven]

Under source:sandbox/i18n there is already an effort underway to do translation of Trac's strings, so that the main UI is localized. For this we make use of ​Babel to handle the i18n parts.

For the translation of Wiki pages and how to handle such things for projects of our users I am still tossing around some ideas to properly facilitate this.

Right now my primary goal is getting the message catalogs filled so that the UI is localized as much as possible. We will get there, one step at a time.

[Jeroen Ruigrok van der Werven]

Part, if not all, of this problem can be solved with the newhelp system.

See TracDev/Proposals/NewHelp.

[Christian Boos]

Milestone 2.0 deleted

[stoecker]

Regarding wiki translations: josm.openstreetmap.de uses translated wiki pages for some time now.

JOSM uses a [[TranslatedPages]] macro, which creates the language selection box and a wiki page ​http://josm.openstreetmap.de/wiki/TracLanguages containing the information about available languages.

Translated webpage get a "code:" prefix before the name (i.e. directly after "wiki/" even for pages containing longer path)

• ​http://josm.openstreetmap.de/wiki/StartupPage gets
• ​http://josm.openstreetmap.de/wiki/De:StartupPage or
• ​http://josm.openstreetmap.de/wiki/Zh_TW:StartupPage

Language code is:

• First char uppercase, second lowercase
• subcodes are added in uppercase with "_" as separation

This method is consistent with other wikis, can be addressed easily from other resources (used as JOSM online help), the page index is grouped by language and users accepted and understood that system fine.

It would be fine when Trac would support this internal (especially regarding the new 0.12 feature to show the clickable path of a wiki page).

[stoecker]

One additional suggestion: For translated pages it is a common problem to keep original and translation in sync. Usually is the english page the base and others are translated. To help here trac could support translation efforts like this:

• define one page/language as base
• Allow to define a version of the base page as "complete" or ok. This means this version is completely represented in current page.
• Display the change differences beween the complete page und the current revision below the translation page currently edited.

This would allow to see changes in the base page in the edit section of a wiki page and thus improve the translations and the update rate. Also this information could be use in the index to show pages which are not up-to-date.

Maybe the concept of base pages could be used for other things as well, not only for translations.

For JOSM we use a different concept where all languages are in one page: ​http://josm.openstreetmap.de/wiki/StartupPageSource This works, but is rather suboptimal. Integrated support would be much better.

[Christian Boos]

Interesting ideas.

[Steffen Hoffmann]

As I've seen today, Russian translations have been done months ago. I'll start German translations now. Let's see, how this even helps to turn ideas here into reality as part of i18n effort for Trac.

I.e. we could quite easily have a macro, that creates a list of translated pages auto-detected by the wiki engine itself.

[[TranslatedPages()]]

example:

[[TranslatedPages(path='Translation',label='available in other languages too:')]]

optional arguments:

• label - custom label to prepend to the auto-generated list, defaults to a standard text per language
• path - additional/alternative search path, defaults to using <wiki page name><language appendix> syntax
• style - optional alternative presentation, i.e. at bottom, as side-bar, etc.

[Dirk Stöcker]

Translated pages macro as used on JOSM pages

[Dirk Stöcker]

I don't think arguments for the macro are helpful. Such things should be unified for a site and thus not passed as macro arguments (maybe except the style argument, which could be useful).

I attached the macro of JOSM site, which uses the above described page design and which works pretty well.

The idea of your path maybe useful (as trac.ini variable thought), but needs to use placeholders. Something like "{lang}:{page}" for JOSM and "Translation{lang}/{page}" for your design started here at the t.e.o. pages and "{page}/{lclang}" for the ​original macro.

{page} referring to page name, {lang} to the lang code as described above} and {lclang} to the fully lower case lang code.

[Dirk Stöcker]

Hello,

I attached an updated macro now containing all features I requested (thought not yet in the style I would like). Christian, could you install that macro?

The Parameter "translatedpages/template" should be set to "Translation{lang}/{page}" for the current translation style on t.e.o.

The TracLanguages page can be copied from JOSM: ​http://josm.openstreetmap.de/wiki/TracLanguages.

Usage: [[TranslatedPages(revision=<rev>)]] where rev is the tested base page revision (Testing results possible with a base revision < current revision in preview mode :-) [[TranslatedPages]] works also, but without revision check.

I contacted the current maintainer of the macro, but had no reaction till now, so I attached the current version.

[Christian Boos]

Thanks, this looks good! I've installed it. Any chance to get a patch for integrating this as an optional component (in source:trunk/tracopt/wiki/translated_pages.py)? Plus the usual TracDev/CodingStyle fixes of course…

I'm also adding Mikhail Gusarov, the current maintainer of the plugin on TracHacks, on CC:, but as there's no explicit license attached to that plugin, I'll assume this is the same license as Trac and therefore OK to swallow.

[Dirk Stöcker]

I added it for one set of pages.

There is a bug, as the base_page_name (and thus links to english page) is wrong for t.e.o. - it uses the sub-page name always. Probably somehow related to the changed structure compared to the JOSM pages.

And we need to change style, as two floating boxes (toc+lang) obviously don't work together well :-)

[Christian Boos]

Replying to dstoecker:

I added it for one set of pages.

There is a bug, as the base_page_name (and thus links to english page) is wrong for t.e.o. - it uses the sub-page name always.

Yes, and the diff link was also supposed to show the diffs on the original page, I guess?

And we need to change style, as two floating boxes (toc+lang) obviously don't work together well :-)

What about this: TranslationDe/TracTickets@3?

[Dirk Stöcker]

Replying to cboos:

There is a bug, as the base_page_name (and thus links to english page) is wrong for t.e.o. - it uses the sub-page name always.

Yes, and the diff link was also supposed to show the diffs on the original page, I guess?

Sure. Same issue - somewhere a bug in _get_page_info, probably the regular expression is not constructed correctly. Thought I have no idea how to debug this :-)

And we need to change style, as two floating boxes (toc+lang) obviously don't work together well :-)

What about this: TranslationDe/TracTickets@3?

If it looks good also when the list gets longer?

[Dirk Stöcker]

Does replacing the second line in _get_page_info with

       regres = re.compile("^"+regstr+"$").search(page_name)

help?

It is more correct, but does it fix the bug?

[Christian Boos]

Replying to dstoecker:

Replying to cboos:

There is a bug, as the base_page_name (and thus links to english page) is wrong for t.e.o. - it uses the sub-page name always.

Yes, and the diff link was also supposed to show the diffs on the original page, I guess?

Sure. Same issue - somewhere a bug in _get_page_info, probably the regular expression is not constructed correctly. Thought I have no idea how to debug this :-)

Ok, I can have a look. Also tell me if you'd like to tackle on the cleanup/integration, just to avoid duplicate work ;-)

And we need to change style, as two floating boxes (toc+lang) obviously don't work together well :-)

What about this: TranslationDe/TracTickets@3?

If it looks good also when the list gets longer?

We could indeed have something like a format=inline|compact|pullout extra option, a bit like the PageOutline macro. pullout would be the current style, inline would output the same content as a comma separated list, and compact could simply show the language codes.

[Christian Boos]

Replying to dstoecker:

Does replacing the second line in _get_page_info with

       regres = re.compile("^"+regstr+"$").search(page_name)

help?

It is more correct, but does it fix the bug?

Well, _get_page_info works as expected (at least with the above fix):

self._get_page_info("TranslationDe/TracTickets") => ('TracTickets', 'De') 

But the problem is still there. Looking for a fix.

I also see:

WARNING: Wrong line syntax while parsing languages list:

(but without further output, I suppose empty lines could be skipped instead)

[Dirk Stöcker]

Found the problem - wiki links are relative to current directory. I suggest to use [wiki:/ in expand_macro and get_lang_link (add the missing "/").

[Christian Boos]

Right ;-) I've also just found it. While that fix will work for now (applied), we should nevertheless find something more general based on the template, so that the macro still works when copying a whole hierarchy for versioning (e.g. 0.12/TranslationDe/TracLinks pointing back to 0.12/TracLinks).

[Dirk Stöcker]

Yes. That would mean a language prefix in the middle of a page-name. I left this out, as it is much more complicated than the current form. The question is how to find the start of the translation area for the English base page (is no problem for the translated ones).

As long as the style used on JOSM pages is still possible I'm happy to accept any changes. I only did this, as nobody else did this work and we needed a good solution for language specific JOSM online help.

Still fine would be an automatic handling of the DIFF part in case a translated page is edited.

[dottedmag@…]

As a maintainer of TranslatedPagesMacro I confirm that the licensed is under the modified BSD, like the Trac itself.

[anonymous]

*that it is licensed under

[anonymous]

2Dirk: did not have time to review your changes yet (especially given they were sent in single patch, not as a patchset).

[Dirk Stöcker]

@cboos: When adding a new regular expression "basepath" which defaults to empty, then this would be possible:

For t.e.o. that would be

basepath = "(0\.\d+/)?"

If you have no better ideas, I would implement this.

Is there a way to modify the layout of the edit page (e.g. with IRequestFilter)? If yes, then I could add the display of diffs there. I would need to get the page diff without using a link thought - I'm sure there is a get_wiki_formatted_diff(text1, text2) functions somewhere, isn't there?

@dm: What do you mean by patch/patchset? I did a svn diff. As this has been developed over a time of approx 1-2 years I have no step-by-step changes, only all-in-one :-) Anyway the current version has moved forward again.

[dottedmag@…]

Replying to dstoecker:

@dm: What do you mean by patch/patchset? I did a svn diff. As this has been developed over a time of approx 1-2 years I have no step-by-step changes, only all-in-one :-) Anyway the current version has moved forward again.

Single logical change per commit (like in git community). But nevermind, svn does not encourage such behaviour.

[Steffen Hoffmann]

As I missed emails of the last ticket changes I was surprised to actually see my preliminary formatting of hand-written translated pages gone and replaced with a new WikiMacro; great! Thanks especially to the contributors of the work done before, that allowed such a jump-start here. And thanks for taking care at all.

[Dirk Stöcker]

@dm: Actually I did a SVN checkout only 2 days ago. Before I simply had a "translatedpages.py" and adapted it to my needs. It was only to speed up solving this ticket that I really did care about giving the changes back. Otherwise you probably would have got a patch set of 50 or 100 changes distributed over a time of two years. I doubt that would have been easier to read.

[Steffen Hoffmann]

Replying to dstoecker:

@dm: Actually I did a SVN checkout only 2 days ago. Before I simply had a "translatedpages.py" and adapted it to my needs. It was only to speed up solving this ticket ![…]

Yeah, considering the ticket has been opened more than 5 ½ years ago, this has been a speedup, considerably. Thank you. This is highly appreciated. I think, anything else could be cleared now.

[Christian Boos]

Replying to dstoecker:

@cboos: When adding a new regular expression "basepath" which defaults to empty, then this would be possible:

For t.e.o. that would be

basepath = "(0\.\d+/)?"

Well, copying below /0.1x is only one possibility. I think we can retrieve this base path by truncating the name from the current path, then re-add the translated name. Basically, in most cases we can auto-detect what you call the basepath and what I call prefix in the patch TranslatedPages-fix-links.diff​. The patch needs to be completed to accept a "canonical" name, in case the translated page is itself a sub-page (e.g. TracTickets/Components, currently we have none of such pages, so the macro works fine for t.e.o at least).

[Christian Boos]

More generic fix, detects the prefix

[Dirk Stöcker]

Ok, I integrated your patch and reworked the whole thing a lot. See e.g. here ​http://josm.openstreetmap.de/wiki/Translations what I mean. Not much left from original code :-)

[Dirk Stöcker]

Your patch does not work as expected - as i feared it fails on the English base page. e.g. for ​http://josm.openstreetmap.de/wiki/Help/SystemRequirements it does not work. I thus disabled it for my site. The infrastrucre is nevertheless the same for the regexp-based method, so no harm done.

Only automatic way would be an autodecting translated pages for all possible subdirs levels.

[Christian Boos]

Replying to dstoecker:

Your patch does not work as expected

It does, I described this limitation for pages like Help/SystemRequirements in comment:41. Additional change needed:

translatedpages/translatedpages.py

@@ -139 +139 @@
         args, kw = parse_args(args)
 
         page_name = formatter.context.resource.id
-        prefix, base_page_name, lang_code = self._get_page_info(page_name)
+        prefix, base_page_name, lang_code = self._get_page_info(page_name,
+            kw.get('canonical'))
 
         lang_link_list = []
         for translation in self._get_translations(prefix, base_page_name):

(considering it's only one more line, I should have finished it along with the first patch but I was already late for helping to prepare the appetizers for this evening ;-) )

The Help/SystemRequirements page should then contain:

[[TranslatedPages(canonical=Help/SystemRequirements)]]

The downside to the basepath regexp approach is that you have to specify the canonical parameter in every source page when it is a SubWiki page. The plus side is that once it's there, you can copy that page anywhere, not relying on a central setting.

---

Now, as you said you reworked the thing a lot, I'll wait before doing further clean-ups and integration in tracopt. Tell me when you're done, either by uploading a new TranslatedPages.tbz2 or your own attempt at the integration.

The missing pages listings look useful, but the presentation could perhaps be simplified by generating a table, languages as columns, pages as lines, each cell containing a 'e'dit link and a color code (green - up to date, yellow - out of date, gray - missing, red - problem detected).

[Dirk Stöcker]

Attached version should now be complete. Please test and report results. It does auto-detection of prefix and thus also works for the form Path/path/De:LastPage, which is used by some Wiki's for translating. Also added the requested status table (I did not drop the others though, as they are useful as well).

[Dirk Stöcker]

To not forget it - Repeating my question from comment 36:

Is there a way to modify the layout of the edit page (e.g. with IRequestFilter)? If yes, then I could add the display of diffs there. I would need to get the page diff without using a link thought - I'm sure there is a get_wiki_formatted_diff(text1, text2) functions somewhere, isn't there?

Adding this would then would be the ultimate solution.

[Christian Boos]

Replying to dstoecker:

To not forget it - Repeating my question from comment 36:

Is there a way to modify the layout of the edit page (e.g. with IRequestFilter)?

The layout of a wiki_edit page is a bit complex due to the coexistence of the two edit modes (normal and side-by-side). Furthermore, there's already provision for showing up a diff in there, the one presented in case of conflicting edits (so called "collision" mode). Maybe the easiest solution could be to "hijack" that somehow (and if not, ITemplateStreamFilter is the way to go rather than IRequestFilter).

If yes, then I could add the display of diffs there. I would need to get the page diff without using a link thought - I'm sure there is a get_wiki_formatted_diff(text1, text2) functions somewhere, isn't there?

Likewise, you should have a look at how the WikiModule._render_editor(... has_collision=True) does it, by calling WikiModule._prepare_diff().

Adding this would then would be the ultimate solution.

Well, opening the diff link in an other window will give you most of the same usefulness, if not more convenience as you could arrange your windows in such a way you won't have to scroll up and down for big pages.

[Dirk Stöcker]

Well, I don't want to add such functionality at the moment (enough other things to do), so feel free to adapt it to your tracopt requirements.

I will monitor if the feature is accepted as I want it for the JOSM pages and judge afterward if improvements are necessary.

[Dirk Stöcker]

Found one little bug: In documentation stands "version=…" instead of "revision=…".

[Dirk Stöcker]

Any progress here? Macro is used for 4 weeks on my pages now and users mostly understand the idea behind.

See e.g. ​http://josm.openstreetmap.de/wiki/Translations.

Would be nice to have this as standard component.

[Steffen Hoffmann]

Replying to dstoecker:

[…] Would be nice to have this as standard component.

Well, works here at t.h.o with the few already translated pages too. Wouldn't miss it for the German translation. I'm sure, it could go into trunk for 0.13, and it shouldn't be trac-opt but standard. Only Remy or Christian could include it into 0.12.2 too, but I fear, the feature set for this is already fixed and frozen according to recent notes on the MailingList.

But nobody stops you from continued use like now, as a single-file macro plugin, until it got finally into, right? I do really appreciate your work for this piece of code, so I assure you that any delay here should not be understood as a vote against it or as critic regarding the code quality. The core devs are just busy as always, so it takes it's time. :-)

[Dirk Stöcker]

I know that I can use it myself. But it would me much more helpful, when it was included in the basic distribution.

@cboos: 1) Can you rename ​http://trac.edgewall.org/wiki/TranslationRu/WikiNewPageRu to ​http://trac.edgewall.org/wiki/TranslationRu/WikiNewPage

2) Could you update the script, so the statistics can be made available somewhere?

[Dirk Stöcker]

Current version after long tests

[Dirk Stöcker]

BTW: ​http://trac.edgewall.org/wiki/TracTermsAM should be ​http://trac.edgewall.org/wiki/TracTermsAm also.

[Dirk Stöcker]

And ​http://trac.edgewall.org/wiki/TranslateRu/TracOnWindowsIis6 should be ​http://trac.edgewall.org/wiki/TranslationRu/TracOnWindowsIis6

[Dirk Stöcker]

And ​http://trac.edgewall.org/wiki/TranslationRu/TracMacros must be ​http://trac.edgewall.org/wiki/TranslationRu/WikiMacros

[Remy Blank]

All renames above done.

[Dirk Stöcker]

Should have been all. The Index looks clean now.

[Dirk Stöcker]

Ok, any progress with integration of this into "tracopt"? What needs to be done for this?

Any chance to install current version for t.e.o.?

[Dirk Stöcker]

@cboos: Hello - any progress here? I really think this should be included in basic distribution. More sites using that would allow much better testing.

If you don't have the time, please give me access to the relevant directory, so I can add it. For JOSM the system has been accepted very nicely. The revisioning method is intuitive and translators now update their pages much better than they did before. I added some other goodies in my local version (like a outdated parameter, which adds a text at page top to indicate that page needs updating).

Have a look at ​http://josm.openstreetmap.de/wiki/Translations for current state (especially the status table).

@dm: When not integrated into basic repository, could we at least update the track-hacks version? Do you still maintain it or should I take over?

[dottedmag@…]

@dstoecker

Feel free to add yourself as a contributor to trac-hacks plugin.

[Christian Boos]

Replying to dstoecker:

@cboos: Hello - any progress here? I really think this should be included in basic distribution. More sites using that would allow much better testing.

I also would like to get it integrated, but there was quite an increase in the size of the code from the one I started working with (your attachment:TranslatedPages.py​) and your next update (attachment:TranslatedPages.tbz2​). The first version was quite adequately sized for inclusion, from what I remember I had the impression the second one was much more heavyweight. Maybe some other opinions would be welcome here (Remy, Simon?).

[Dirk Stöcker]

@dottedmag:

I found not how to request access to existing project, so I added a ticket ​http://trac-hacks.org/ticket/8912 as new user stoecker.

@cboos:

Yes it has increased in size a lot (graphical interface), but is still only this one file.

[anonymous]

Replying to dstoecker: I have request @otaku42 (the site admin) to grant access to you. This is the only way to grant an existing project write access to others now…

@dottedmag:

I found not how to request access to existing project, so I added a ticket ​http://trac-hacks.org/ticket/8912 as new user stoecker.

@cboos:

Yes it has increased in size a lot (graphical interface), but is still only this one file.

[ftofficer]

Uh, comment:63 is written by ftofficer.

Replying to anonymous:

Replying to dstoecker: I have request @otaku42 (the site admin) to grant access to you. This is the only way to grant an existing project write access to others now…

@dottedmag:

I found not how to request access to existing project, so I added a ticket ​http://trac-hacks.org/ticket/8912 as new user stoecker.

@cboos:

Yes it has increased in size a lot (graphical interface), but is still only this one file.

[Dirk Stöcker]

I did an update on track-hacks.org. I updated the documentation there, but lots of new stuff remains to be copied from JOSM docs :-) Will follow the next days.

@cboos: Could you nevertheless update the installed macro here on t.e.o. (maybe using the track-hacks.org version, so I see if everything is right there).

Is there someone with an old macro installation who can verify if my update description from 0.3 is correct?

[Remy Blank]

Preparing for 1.0.

[Christian Boos]

Move feature requests to next-dev.

[Christian Boos]

well, once again… next-dev

[Ryan J Ollos]

Retargetting tickets to narrow focus for milestone:1.2. Please move the ticket back to milestone:next-dev-1.1.x if you intend to resolve it by milestone:1.2.