Please fix documentation about the variables available to customize templates

[hartmut.benz@…]

Hi folks,

I've been digging through this site and a few other (with the help of google) trying to find out how to configure my email notifications (subject). I am not a trac (system/plugin) programmer and do not want to be required to become one just to reconfigure the email subject to say say something like …NEW owner: xxx if and only if the owner change. Even better would be a … assigned to ME… iff the receiver is the new owner.

​Customizing the Mail Subject appears to be a good starting point.

• It points to the general genshi help on how to do commands
• It lists the variables available to put in

But…

• The variable list given here is extremely terse - a non-developer is left pretty clueless of what is in there or how to access it.
• The list is incomplete as looking at the example below the list immediately reveals
  • missing are for instance contents and structure of changes_descr, changes_body, %(author)
  • I have tried to 'guess' that there may be a changes_owner, but that does not work

For trac/plugin developers this may be all pretty obvious, but for a non-python person that just apt-gets trac, it is not.

My suggestion/wish would be to describe 'all' available variables or access patterns giving plenty of examples of how to access them:

• list/pattern of the ticket fields
  • (the linked model.py does not yield that you can access the owner with ${ticketowner}
• pattern of how to access ticket custom fields (old, new, whether-there-was-a-change)
• pattern of how to access settings from trac.ini
  • perhaps the link to env.py is intended to tell me…?
• list/structure of data available about what changed and who
  • must be in the changes_descr etc.
  • but if found elsewhere that {% if action %} [$action] {$ end %} tests if an 'action' occurred without referring to changes*

Thanks for your product and your support

Hartmut

[osimons]

Generally, the variables available in the shipped template should be a quite complete set of meaningful information. You should start with current template and make modifications.

Particularly, what intuitively one may think is available for rendering is not actually there. And certainly not if you read the general site.html template modification information. Notifications render in a subcontext way removed from the request, and hence has very little information to lookup - nor even any way to check permissions for other information even if you got hold of it somehow. The information even arrives pre-made / pre-formatted / pre-wrapped for inclusion in text body… Which is a bummer for those that try to hack HTML support that splits and presents the fields in a nicer way.

Also, note that the email subject and body are just rendered once - it is not rendered for each recipient. So content like "assigned to me" is not possible. And for instance why all emails are obfuscated as we just don't know who can view them or not. Notifications content is currently lowest-common-denominator for what can work for all receivers - and as receivers just arrive as to, cc and bcc lists of emails the context don't know anything meaningful about them.

More documentation? Sure, that would have been nice, but most realistically would be to hope that perhaps someone could post some more elaborate modification example to the CookBook or simiar.

[anonymous]

Replying to osimons:

Generally, the variables available in the shipped template should be a quite complete set of meaningful information. You should start with current template and make modifications.

Thank you for the fast reply, osimons!

Aside from the example on the documentation page, the template I have is exactly this (I am using TracAnnouncer 0.11.1):

Ticket #${ticket.id}: ${ticket['summary']} {% if action %}[${action}]{% end %}

That is pretty scarce to start with. I've been looking at the template rendering the email, but this does not reveal what I am looking for, because it is iterating over (to me) unknown data structures. I.e., still do not know how to test if the owner changed (which is the only thing I would like to know right now…everything else has time ;-)

I am happy to provide my non-python-programmer insights while I go for potential inclusion in the documentation. A new CookBook about it would be nice as well, but I do not think it can replace the documentation entry.

The list I'm asking for can be very simple, something like

• Accessing ticket fields
  • ${ticket.id} - the ticket number
  • ${ticket['summary']} the summary of the ticket (why is this not accessed as ${ticket.summary}
• Other fields
  • $(author) the author of the change
    • I have no clue what ${_('Changes (by %(author)s):', author=change.author)} might do
    • does it, perhaps, create the ${author} variable?
  • action holds "Status → new-status if and only if the ticket's status changed
  • changes_descr but there is also change.author
• Conditionals based on what changed
  • If (status of ticket changed) insert status-transition
    • {% if action %}[${action}]{% end %}
• etc.

The (at least) three types of accessing variables ( ${…} $(…) ${…...} makes it look a bit difficult.

Still mystified ;-)

Harmtut

[osimons]

Replying to anonymous:

(I am using TracAnnouncer 0.11.1):

Then you are not using Trac notification or any template or data context that ships as part of Trac. This is really then a PluginIssue as it is outside Trac itself. Consider asking on the trac-users MailingList or checking the plugin documentation and bug tracker at ​th:AnnouncerPlugin

[anonymous]

Replying to osimons:

Replying to anonymous:

(I am using TracAnnouncer 0.11.1):

Then you are not using Trac notification or any template or data context that ships as part of Trac. This is really then a PluginIssue as it is outside Trac itself. Consider asking on the trac-users MailingList or checking the plugin documentation and bug tracker at ​th:AnnouncerPlugin

I am a bit disappointed by this reaction - just reading a plugin name seems to have triggered a close-ticket reflex. Just ignore my comment that I am using this plugin - I am using it only in one trac instance.

I am re-opening this ticket because I have other 'virgin' instances without any activated plugins, and their documentation (and behavior) is as incomplete as the one with the plugin and both run the same genshi with the same template.

[osimons]

You need to understand the essence of Genshi and some Python to change templates. So ​genshi:wiki:Documentation/index.html would be a good starting point to understand the syntax and what goes on.

Note that Genshi supports 2 template syntaxes, used depending on output needs:

• ​XML Templates that need to ba valid XHTML, and used in Trac for all HTML output.
• ​(New)Text Templates that we use for all email / plain text output.

There is even a ​Genshi tutorial that is quite useful even though it is not based on Trac, but should help you get the general logic:

1. Trac request handlers put various strings, objects and functions into the data available for rendering.
2. Genshi renders the template using the Python data as context.

Here is the problem: Each module in Trac (or plugins) that need to render data put whatever it needs into the data - under whatever structure it wants, containing the information it wants, and naming it as it sees fit.

The upside is that the data available is usually always used, as modules/handlers rarely put data into template rendering context just for fun - that would just be useless overhead. So, starting with an existing template you should see pretty much all the available data with examples of use. And if you have looked at the Genshi docs you should know the essence of why it works and how it works.

The reason you need some basic Python insights is do understand syntax like your example: ${_('Changes (by %(author)s):', author=change.author)}:

• Having read the docs above, you know that ${} calls to Genshi to evaluate the contained expression..
• The expression is just one function call: _() - which is the Trac function for all gettext translations. So this calls to find the string in the right locale based on the current user. So 'Changes (by %(author)s):' may end up returning 'Endringer (av %(author)s):' if you are Norwegian like me.
• As this example shows, translation strings often includes variables - including them inline for substitution makes the strings easier to read, and allows some languages to shuffle text and variables around according to their own language constructs. But author still needs to added, so that is passed in as second argument to that function: author=change.author - essentially saying use change.author as author when translating the string.
• However, that author is valid just for translating the string - the method call will not generate the variable nor change the string.

You need to do some reading. This is not just documentation because there is really quite a fair bit of it around already, particularly when investing time in understanding what Genshi does. When you've done that, feel free to use my text above and use your own new-found knowledge and add it to a wiki page (perhaps in CookBook?).

Enjoy!

[anonymous]

Thanks for the detailed answer. I agree, and disagree :-). From the point of view of a (trac/trac-plugin) developer, you are absolutely right. In order to be and do that, you'd need some and then some more basic understanding of python and genshi and the trac architecture etc. to get started. As with most programming languages and systems, people generally start with the Hello World approach, that is: copy an existing app and start modifying (first small, later bigger) things. And eventually, you might become an expert. Trivial changes require trivial (research and) knowledge, Complex changes require much more work.

What I do have is a perfect Hello World example (showing the status transition in the subject if there is one {% if action %}[${action}]{% end %}) that would like to change into 'show an owner-change in the subject if there is one).

I created this ticket, because the primary spot in the wiki documentation: ​Customizing The Mail Subject, was of no help in and a considerable amount of googeling did not yield more insight.

Because I do think that this might be an issue/a desire of other users as well, I did not just ask for the variable name holding the 'owner'/'owner change', but pointed out the more general issue.

Now if I understand your last post right, you suggest that I should dive into learning genshi, python and trac first, and find out myself and you won't/can't give me the variable name(s) I need directly. Investing that amount of time for the intended trivial change is completely out of proportion for me and I will therefore drop the attempt.

You furthermore seem to imply that you find the documentation (in the particular spot I pointed out) more than sufficient and clear. Naturally, I disagree, but it is your project and documentation. So feel free to close the ticket again.

Enjoy!

[osimons]

Replying to anonymous:

What I do have is a perfect Hello World example (showing the status transition in the subject if there is one {% if action %}[${action}]{% end %}) that would like to change into 'show an owner-change in the subject if there is one).

The subject and body are rendered in different contexts, and the documentation you refer to is an accurate overview of the data available for manipulating subject. action is not one of them, so not sure how your 'Hello, world' actually works for you. This is the actual subject formatting code from trac.ticket.notification.TicketNotifyEmail (with data being the key structure here):

    def format_subj(self, summary):
        template = self.config.get('notification','ticket_subject_template')
        template = NewTextTemplate(template.encode('utf8'))
                                                
        prefix = self.config.get('notification', 'smtp_subject_prefix')
        if prefix == '__default__': 
            prefix = '[%s]' % self.env.project_name
        
        data = {
            'prefix': prefix,
            'summary': summary,
            'ticket': self.ticket,
            'env': self.env,
        }
        
        return template.generate(**data).render('text', encoding=None).strip()

I created this ticket, because the primary spot in the wiki documentation: ​Customizing The Mail Subject, was of no help in and a considerable amount of googeling did not yield more insight.

According to the code for rendering subject, I'd say that the docs you link to are 100% accurate. And no major surprise that searching the web won't give you more variables when there are none.

However, when I closed the ticket by referring to this a PluginIssue, that was also correct: When using the ​th:AnnouncerPlugin it is the plugin that formats the subject, and not Trac. So when I point you elsewhere it is because that is not part of this project - it is not our code, we don't know its internals and cannot document it or support it. Take a look at the code @ ​th:source:/announcerplugin/trunk/announcer/email_decorators.py at line 151:

class TicketSubjectEmailDecorator(Component):
    """Formats ticket announcement subject headers based on the
    ticket_email_subject configuration.
    """

    implements(IAnnouncementEmailDecorator)

    ticket_email_subject = Option('announcer', 'ticket_email_subject',
            "Ticket #${ticket.id}: ${ticket['summary']} " \
                    "{% if action %}[${action}]{% end %}",
            """Format string for ticket email subject.  This is
               a mini genshi template that is passed the ticket
               event and action objects.""")

    def decorate_message(self, event, message, decorates=None):
        if event.realm == 'ticket':
            if event.changes:
                if 'status' in event.changes:
                    action = 'Status -> %s' % (event.target['status'])
            template = NewTextTemplate(self.ticket_email_subject.encode('utf8'))
            subject = template.generate(
                ticket=event.target,
                event=event,
                action=event.category
            ).render('text', encoding=None)

            prefix = self.config.get('announcer', 'email_subject_prefix')
            if prefix == '__default__':
                prefix = '[%s] ' % self.env.project_name
            if prefix:
                subject = "%s%s"%(prefix, subject)
            if event.category != 'created':
                subject = 'Re: %s'%subject
            set_header(message, 'Subject', subject)

        return next_decorator(event, message, decorates)

It renders its subject based on the [announcer] ticket_email_subject template text, and seems to render by passing in data for ticket, event and action….

Now if I understand your last post right, you suggest that I should dive into learning genshi, python and trac first, and find out myself and you won't/can't give me the variable name(s) I need directly. Investing that amount of time for the intended trivial change is completely out of proportion for me and I will therefore drop the attempt.

There is not just some variable to "give you". IF you can get at the information you need, you need to write conditional comparisons and output in right form and syntax to get the result you want. How is it unfair to say that you need to understand the basics of Genshi and Python to accomplish that? Or do you want us to help you research code and documentation from other projects, and write the code for you?

You furthermore seem to imply that you find the documentation (in the particular spot I pointed out) more than sufficient and clear. Naturally, I disagree, but it is your project and documentation. So feel free to close the ticket again.

Well, you never actually reopened the ticket… Do you still think it would be correct to do so?

[anonymous]

Hi osimons, thanks again for the detailed answer … and … me culpa for getting things wrong.

• I had clicked the re-open ticket at some moment, but apparently my browser lost this during a re-load
• The action worked in my clean trac environment because it was not as clean as I thought - the announcer had stayed on partially. Now that it is off, the action does not produce results any more. Therefore, your suggestion to bother the author of the ​AnnouncerPlugin was correct and I will post a ticket there.

Regarding the documentation of vanilla trac, I have been playing some more with then notification and would still like to suggest a few small changes to make things clearer and provide an easier start for newcomers.

The subject and body are rendered in different contexts, a…

That subject and body are rendered in different contexts is not clear to me from the documentation and I'm not sure how to conclude that easily. My suggestion is to make this a bit more clear by explicitly listing the available vars for the e-mail content as well.

Customizing the e-mail subject … explanation… The following variables are available in the template: … Customizing the e-mail content … explanation… The following variables are available in the template: … The default template is:

The symmetry of the section structure and the difference between the vars should make it more clear in which way the contexts differ (I wonder, though, whether it would not be an idea to provide identical contexts for both templates).

… and the documentation you refer to is an accurate overview of the data available for manipulating subject.

As you demonstrated in the code, the documentation accurately describes the available data. To make the content a bit more available to end users (non-programmer trac-admins), I would suggest something along these lines:

prefix: The prefix defined in smtp_subject_prefix. summary: The ticket summary, with the old value if the summary was edited. ticket: Any ticket fields (including custom fields) can be used in the template by appending the field name separated by a dot, e.g. $ticket.milestone (programmers: cf. model.py and (ref to … format_subj)) env: The project environment. You can, for example access please put some examples.(programmers: cf env.py)

Specifically, the env variable needs some explanation what can be accessed there, unless it is only relevant for people writing or extending code. If so, a remark like This variable is only relevant for you if you are modifying or writing code and not for simple modifications of the template text. I cannot offer text for this because I have not been able to access anything from env so far.

If you agree to such changes, please re-open the ticket until they are done. Thanks again

Hartmut

[anonymous]

I agree with the need of a simple variable documentation. I have tried to find out for quite some time now if there is an array with previous updater available, like $ticket.PreviousUpdater or similar

[anonymous]

See #12575.