New Help System

What is the problem?

Having help pages provided as wiki pages has some disadvantages:

• They clutter the project wiki with pages unrelated to the actual project.
• trac-admin wiki upgrade by system admin is needed to update the pages, but very often they are not updated leaving the help pages out of sync with the actual installed version.
• They appear as wiki pages and can be edited, but an upgrade will erase the content.
• It is unecessarily complicated to support and depend on content being available in a project installation (like references from the templates and so on).
• There are sound arguments for moving to rst as standard documentation format for Trac documentation, and keeping things outside the wiki makes that easier.

A large part of the Trac experience is based on enhancements by plugins. Currently there is no support at all for plugins also extending the set of Help pages available. Even if the plugin inserted a wiki page for help content, it would also need to update all the [[TracGuide]] macros in all other wiki pages so that it would display in the menu…

Speaking of, the help menu should be sectioned - not just a list of pages, but perhaps split into sections based on general content, general admin content, ticket system, wiki system, version control and so on.

With the i18n branch in incubation, the natural question is: "What about all the default pages?" How are they to be transated? How are we to switch to localised version of the pages per request depending on who accesses them? If this stays in the wiki, no-one is going to find any project-related wiki pages - 95% would be help pages.

Proposal

The proposal is quite simple:

• Make the help system its own module.
• Have it provide an API to allow content to be provided from anywhere, and in whatever locale.
• Stop using the project wikis for help, and instead make a help page provider that renders the Help pages from resources (files) stored inside the Trac egg.

Implementation

The New Help system does not in itself provide many features except these basic ones:

• A menu item for Help/Guide
• A request handler that responds to /help
• A wiki syntax provider that allows linking to help pages, like [help:HelpMe]
• Some helpers such as req.help_pages callback and format_help_title() for use in templates

The request handler and the syntax provider will have to ask the providers to supply the needed information - the list of pages for builing the help menu, and rendering a requested page for display. The API should be generic enough to allow help pages to be provided from anywhere and in the format of choice.

The basic API currently looks like this:

class IHelpProvider(Interface):
    """Extension point interface for components that provide pages
    to the Help system and its related features.
    """

    def get_help_pages(locale_id):
        """ Returns help pages provided by the plugin as a generator
        of tuples of the form: `(section, title, page_id)` """

    def get_help_page_content(page_id, locale_id):
        """ Based on a page_id and locale_id, return content and mimeview type. """

i18n

As can be seen from the API, New Help accounts for locale and the preference to supply a localised page if such a page exists - using a default locale of '' as fall-back, where '' then means 'available for all locales'.

The help menu (the set of available pages per request) will be composed as a merge between localised pages and pages marked as 'available for all', where a local version of a given page_id will be given preference.

First version is not branched off i18n as it does not depend on it other than to hook them together once they get prepared for branching - essentially all that is needed is to fetch the locale from the request in the i18n preferred manner. Apart from that, the i18n features use a different strategy to make language help plugins totally self-contained.

However, basic gettext support can be added simply by wrapping the various default section and page titles in _() - it would give the illusion of a localised help page with English content for all default pages, and may be a good idea?

Example provider

Modelled after the idea from WikiMacroBase, an abstract AbstractHelpFileProvider class is provided. This is a provider that reads files stored inside any installed egg that provides a help plugin.

If we were to distribute Help pages into the specific modules, here is a very simple one-page example of what a ticket system help provider would look like:

from trac.help.providers import AbstractHelpFileProvider
class TracTicketHelpProvider(AbstractHelpFileProvider):
    """ Providing help pages related to the ticket system. """

    docformat = 'text/x-trac-wiki'   # Not needed if default
    pkg_name = 'trac'                # Not needed if default

    storage_location = 'ticket/guide'

    help_pages = {
        '': [
            ('Ticket System', 'Using Tickets', 'TracTickets'),
            .....
            ]
        }

A Norwegian 'language pack' will similarly just need to include the TracTickets help page id in its list of translated pages, mapped inside the help_pages using no_NB locale specification.

Trac itself will not supply any localised documentation pages - only default pages. Hopefully people will start projects at trac-hacks.org for maintaining localised language packs for major languages, perhaps also including pages for plugins other than Trac as well.

mrelbe:

"Trac itself will not supply any localised documentation pages - only default pages." — Why? Since the core of Trac 0.12 is localised, I would expect help pages to be presented in the same language as I've selected in the preferences panel, without having to add separate plugins.

Shouldn't help pages be handled in a fashion inspired by the way text strings are now? Today, a message.po file defines text strings for each locale.

Some thoughts (for what it's worth): Define a help-page-directory for each locale containing files of translated help pages. The physical name of each file shall be same as for the implementation language (cf. gettext strings).

trac/locale/en_US/help/index.po
trac/locale/en_US/help/Permissions
trac/locale/en_US/help/Tickets

trac/locale/sv/help/index.po
trac/locale/sv/help/Hjälp     # unique help file for Swedes
trac/locale/sv/help/Tickets

index.po contains translations of localised help page names to the physical file name.

Example for Swedish:

• "Permissions" —> "" (not translated)
• "Tickets" —> "Ärenden"

When Swedish language is selected:

• [[help:Ärenden]] shows "sv/help/Tickets".

• [[help:Tickets]] shows "sv/help/Tickets".

• [[help:Permissions]] falls back to show "en_US/help/Permissions" since that page has not been translated.

• [[help:Hjälp]] shows "sv/help/Hjälp" since this is a unique Swedish help page, not present in index.po but a file exists with the referred name for selected locale. (If Unicode characters in filenames is a problem, then it should be possible to define an ASCII-based filename and extend index.po, or similar.)

Tasks and questions

In addition to the general task of getting the New Help system up and running there are some specific issues related to further features or to other parts of Trac.

Done:

• Basic help module with api and default provider for first testing.
• Move navigation item from wiki to new help.
• Remove the trac-admin wiki upgrade command.
• Update all templates and a few code references to point to help location instead of wiki, also with a way to provide localised titles.
• All templates use format_help_title() to fetch localised title where appropriate and if such a localised title exists.

Questions / Open Issues:

• i18n: Provide 3 stage resolving? First see if page exists for specific locale (no_NB). If not, see if any plugins responds to no only - catching both no_NB and no_NY. Lastly, check the default ('')?
• Search: This is not implemented, and full text search could be problematic as that would mean reading the files (or indexing them), but other providers may also fetch from locations outside Trac. Perhaps some smart plugin decides to use the Google APIs to translate help pages on-the-fly, and that would be very costly localised searches. If we are to do something about search, we could just search the page titles and ids. Well, perhaps I'm overestimating it, but I'd rate the ability to search in the Trac Help as quite high. At least the content of the bundled pages could be also stored in a table and the search can be performed there. —cboos
• Timeline: Help events do not show up in the timeline anymore, and no real need for them to do so either.
• Distributed or not? Like for templates, my vote (osimons) goes to using same strategy as for templates, so that major modules provide their own help pages, while maintaining the minor and general help pages as part of the help module. That way, if some installation does trac.versioncontrol.* = disabled, all the help pages will also disappear.
  +1 from cboos
• When/if pages gets translated, there will be some link problems as we use a couple of links directly to anchors on the page. If so we need to use explicit anchors that do not change, and this must be supported by language plugins as well. How do we handle links to anchors?
  By using explicitly the ids from the canonical document, like this:

  == Tâches et questions == #Tasksandquestions
  

• For those upgrading, do we delete all previously existing wiki help pages? Or offer a contrib script that can do it?

To be done:

• Structure and move the pages into their new locations ('*/guide')
• Update all links in the various documentation pages to point to [help:] instead of wiki.
• Likely use the opportunity to convert them to rst format and improve them where we can - leaving the essence for the docs, and the wiki to be contributed specifics. Well, no. Let's dissociate the question of the format in which the documentation should be written, for now, and simply do with what we have at this point. — cboos
• i18n adjustments to pick up the locale when these gets merged.
• Remove the [[TracGuideToc]] macro - still useful during development as a direct comparison between menus, but it should not really be needed anymore.
• Unit tests.

trac.edgewall.org

Ongoing discussions about moving project documentation into the repository is strictly not related to this proposal, but combined with it there should be some concensus on how the project presents standard documentation on the project pages - and how we get contributions back.

For contributions back, perhaps use a variation of the [[Include]] macro like on Genshi and Babel projects, but that more distinctly display that as a referenced source. Then we could encourage users to add comments on this wiki page - as content editing will no longer be possible? Then the project could periodically review comments - migrate to docs the useful bits, remove no longer valid information, and just let useful supplement information live on that page. Perhaps using the [[AddComment]] macro in append-only mode?

References

The proposal resolves or references these tickets:

• #2656: Make built-in documentation independent from the wiki
• #3386: Embed pages from one wiki into another project (partial solution, shared help pages can be done as a help plugin)
• #756: trac-admin option for custom set of default wiki pages (may not be needed anymore as there will only be a handful of wiki pages created - the rest supplied as help)
• #1513: Localized versions of wiki pages, at least in so far as the Trac Guide is concerned.
• #7193: Related to the issue of if/how to remove existing wiki pages in projects that upgrade (as pages will be out-of-date and no longer needed).
• #7124: Not show Trac default pages in TitleIndex.
• #7313: Searching help output provided by macros.
• #7430: Option to exclude Trac documentation from Timeline.
• #9876: wiki:TicketComponent does not exist

This list is to be updated as no doubt other related tickets on this topic exist.

Additionally, it is very much related to the various i18n tickets as it allows plugins to provide 'language packs' also for default and custom help pages.