Uncaught exception when TRAC_ENV_INDEX_TEMPLATE points to file with no extension

[pulkomandy@…]

Hello, I have just migrated my Trac install to Trac 1.5. I had some trouble getting the project index working. The reason was that my template file was named just "template" (without extension).

The new Jinja template system does things differently depending on the file extension (it tests for "html" and a few others). However, if the file has no extension it just raises an exception:

2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:Traceback (most recent call last):
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/local/lib/python3.10/dist-packages/flup/server/fcgi_base.py", line 576, in run
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    protocolStatus, appStatus = self.server.handler(self)
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/local/lib/python3.10/dist-packages/flup/server/fcgi_base.py", line 1175, in handler
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    result = self.application(environ, start_response)
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/lib/python3/dist-packages/trac/web/fcgi_frontend.py", line 36, in __call__
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    return self.application(environ, start_response)
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/lib/python3/dist-packages/trac/web/main.py", line 528, in dispatch_request
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    send_project_index(environ, start_response, env_parent_dir,
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/lib/python3/dist-packages/trac/web/main.py", line 846, in send_project_index
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    tmpl = jenv.get_template(template)
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/local/lib/python3.10/dist-packages/jinja2/environment.py", line 883, in get_template
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    return self._load_template(name, self.make_globals(globals))
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/local/lib/python3.10/dist-packages/jinja2/environment.py", line 857, in _load_template
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    template = self.loader.load(self, name, globals)
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/local/lib/python3.10/dist-packages/jinja2/loaders.py", line 127, in load
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    code = environment.compile(source, name, filename)
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/local/lib/python3.10/dist-packages/jinja2/environment.py", line 629, in compile
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    source = self._generate(source, name, filename, defer_init=defer_init)
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/local/lib/python3.10/dist-packages/jinja2/environment.py", line 586, in _generate
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    return generate(
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/local/lib/python3.10/dist-packages/jinja2/compiler.py", line 87, in generate
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    generator.visit(node)
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/local/lib/python3.10/dist-packages/jinja2/visitor.py", line 32, in visit
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    return f(node, *args, **kwargs)
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/local/lib/python3.10/dist-packages/jinja2/compiler.py", line 714, in visit_Template
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    eval_ctx = EvalContext(self.environment, self.name)
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/local/lib/python3.10/dist-packages/jinja2/nodes.py", line 69, in __init__
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    self.autoescape = environment.autoescape(template_name)
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:  File "/usr/lib/python3/dist-packages/trac/util/text.py", line 60, in autoescape_extensions
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:    return template and template.rsplit('.', 1)[1] in exts
2022-06-05 13:44:31: (mod_fastcgi.c.451) FastCGI-stderr:IndexError: list index out of range

I had to look at the sourcecode to understand what was going wrong. Maybe unknown extensions should default to html, or at least the exception should be caught and an error page shown in the web browser so the info is not hidden in webserver logs.

After renaming my template, things are working fine.

[Jun Omae]

Thanks for the reporting!

trac/util/text.py

@@ -52 +52 @@
     :rtype: `jinja.Environment`
 
     """
-    exts = ('html', 'rss', 'xml')
+    exts = ('.html', '.rss', '.xml')
     def filterout_none(v):
         return '' if v is None else v
     def autoescape_extensions(template):
-        return template and template.rsplit('.', 1)[1] in exts
+        return template and template.endswith(exts)
     defaults = dict(
         variable_start_string='${',
         variable_end_string='}',

[Jun Omae]

Another issue, the project index page sends contents as text/html or text/xml. We should enable autoescape even if the template file has no extension.

trac/web/main.py

@@ -870 +870 @@
 
     data['projects'] = projects
 
-    jenv = jinja2env(loader=FileSystemLoader(loadpaths))
+    jenv = jinja2env(loader=FileSystemLoader(loadpaths), autoescape=True)
     jenv.globals.update(translation.functions)
     tmpl = jenv.get_template(template)
     output = valid_html_bytes(tmpl.render(**data).encode('utf-8'))

[Jun Omae]

Committed and merged in [17642-17643].