Test failures with Pygments 2.1

[Ryan J Ollos]

The automated tests that ran when committing the fix for #12325 revealed some test case failures. Pygments 2.1 was released on 2016-01-17. Test failures were not seen with Pygments 2.0.2.

======================================================================
FAIL: test_extra_mimetypes (trac.mimeview.tests.pygments.PygmentsRendererTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/tests/pygments.py", line 123, in test_extra_mimetypes
    mimeview.get_mimetype('file.ini'))
AssertionError: 'text/x-ini; charset=utf-8' != u'text/inf; charset=utf-8'

======================================================================
FAIL: test_python_hello (trac.mimeview.tests.pygments.PygmentsRendererTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/tests/pygments.py", line 74, in test_python_hello
    self._test('python_hello', result)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/tests/pygments.py", line 62, in _test
    self.assertEqual(exp, res)
AssertionError: '        <span class="k">return</span> <span class="s">"Hello World!"</span>' != '        <span class="k">return</span> <span class="s2">"Hello World!"</span>'

======================================================================
FAIL: test_python_hello_mimeview (trac.mimeview.tests.pygments.PygmentsRendererTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/tests/pygments.py", line 85, in test_python_hello_mimeview
    self._test('python_hello_mimeview', result)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/tests/pygments.py", line 62, in _test
    self.assertEqual(exp, res)
AssertionError: '        <span class="k">return</span> <span class="s">"Hello World!"</span>' != '        <span class="k">return</span> <span class="s2">"Hello World!"</span>'

----------------------------------------------------------------------
Ran 1795 tests in 108.513s

FAILED (failures=3)
make: *** [unit-test] Error 1

[Ryan J Ollos]

First issue seems to be due to addition of a mimetype for ini.

Pygments 2.1:

>>> from pygments.lexers import get_lexer_by_name
>>> lexer = get_lexer_by_name('ini')
>>> lexer.mimetypes
['text/x-ini', 'text/inf']

Pygments 2.0.2:

>>> from pygments.lexers import get_lexer_by_name
>>> lexer = get_lexer_by_name('ini')
>>> lexer.mimetypes
['text/x-ini']

[Ryan J Ollos]

Now I remember looking at this some time ago. When there are multiple mimetypes for a given ​Short name (also referred to as aliases​ and keywords​), the last mimetype overwrites the previously stored mimetype: tags/trac-1.0.9/trac/mimeview/api.py@:932-933#L922​.

So I imagine we can fix this similar to how we fix the Babel tests when the CLDR data is updated:

trac/mimeview/tests/pygments.py

@@ -119 +119 @@
         Pygments supports it.
         """
         mimeview = Mimeview(self.env)
-        self.assertEqual('text/x-ini; charset=utf-8',
-                         mimeview.get_mimetype('file.ini'))
-        self.assertEqual('text/x-ini; charset=utf-8',
-                         mimeview.get_mimetype('file.cfg'))
+        self.assertIn(mimeview.get_mimetype('file.ini'),
+                      ('text/x-ini; charset=utf-8',
+                       'text/inf; charset=utf-8'))  # Pygment 2.1+
+        self.assertIn(mimeview.get_mimetype('file.cfg'),
+                      ('text/x-ini; charset=utf-8',
+                       'text/inf; charset=utf-8'))  # Pygment 2.1+
         self.assertEqual('text/x-ini; charset=utf-8',
                          mimeview.get_mimetype('file.text/x-ini'))

[Ryan J Ollos]

Regarding second and third errors, in Pygments 2.1 strings are being assigned Token.Literal.String.Double rather than Token.Literal.String.

Pygments 2.1:

>>> from pygments.lexers import PythonLexer
>>> pl = PythonLexer()
>>> s = """
... def hello():
...     return "Hello World!"
... """
>>> tokens = pl.get_tokens(s)
>>> for t in tokens:
...   print t
... 
(Token.Keyword, u'def')
(Token.Text, u' ')
(Token.Name.Function, u'hello')
(Token.Punctuation, u'(')
(Token.Punctuation, u')')
(Token.Punctuation, u':')
(Token.Text, u'\n')
(Token.Text, u'    ')
(Token.Keyword, u'return')
(Token.Text, u' ')
(Token.Literal.String.Double, u'"')
(Token.Literal.String.Double, u'Hello World!')
(Token.Literal.String.Double, u'"')
(Token.Text, u'\n')

Pygments 2.0.2:

>>> from pygments.lexers import get_lexer_by_name
>>> lexer = get_lexer_by_name('ini')
>>> lexer.mimetypes
['text/x-ini']
>>> from pygments.lexers import PythonLexer
>>> pl = PythonLexer()
>>> s = """
... def hello():
...     return "Hello World!"
... """
>>> tokens = pl.get_tokens(s)
>>> for t in tokens:
...   print t
... 
(Token.Keyword, u'def')
(Token.Text, u' ')
(Token.Name.Function, u'hello')
(Token.Punctuation, u'(')
(Token.Punctuation, u')')
(Token.Punctuation, u':')
(Token.Text, u'\n')
(Token.Text, u'    ')
(Token.Keyword, u'return')
(Token.Text, u' ')
(Token.Literal.String, u'"')
(Token.Literal.String, u'Hello World!')
(Token.Literal.String, u'"')
(Token.Text, u'\n')

[Ryan J Ollos]

Looks like comment:3 changes are due to ​issue 685.

[Ryan J Ollos]

Proposed changes in log:rjollos.git:t12326_pygments_2.1.

[Ryan J Ollos]

Some additional test failures on the trunk:

======================================================================
FAIL: test_python_with_invalid_arguments (trac.mimeview.tests.pygments.PygmentsRendererTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/tests/pygments.py", line 147, in test_python_with_invalid_arguments
    self._test('python_with_invalid_arguments_1', result)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/tests/pygments.py", line 69, in _test
    self.assertEqual(exp, res)
AssertionError: u'<div class="wiki-code"><table class="code"><thead><tr><th class="lineno" title="Line numbers">Line</th><th class="content">\xa0</th></tr></thead><tbody><tr><th id="a-L1"><a href="#a-L1">1</a></th><td><span class="k">print</span>\xa0<span class="s">\'this is a python sample\'</span></td></tr><tr><th id="a-L2"><a href="#a-L2">2</a></th><td>a <span class="o">=</span>\xa0b<span class="o">+</span><span class="mi">3</span></td></tr><tr><th id="a-L3"><a href="#a-L3">3</a></th><td>z <span class="o">=</span>\xa0<span class="s">"this is a string"</span></td></tr><tr><th id="a-L4"><a href="#a-L4">4</a></th><td><span class="k">print</span>\xa0<span class="s">\'this is the end of the python sample\'</span></td></tr></tbody></table></div>' != u'<div class="wiki-code"><table class="code"><thead><tr><th class="lineno" title="Line numbers">Line</th><th class="content">\xa0</th></tr></thead><tbody><tr><th id="a-L1"><a href="#a-L1">1</a></th><td><span class="k">print</span>\xa0<span class="s1">\'this is a python sample\'</span></td></tr><tr><th id="a-L2"><a href="#a-L2">2</a></th><td>a <span class="o">=</span>\xa0b<span class="o">+</span><span class="mi">3</span></td></tr><tr><th id="a-L3"><a href="#a-L3">3</a></th><td>z <span class="o">=</span>\xa0<span class="s2">"this is a string"</span></td></tr><tr><th id="a-L4"><a href="#a-L4">4</a></th><td><span class="k">print</span>\xa0<span class="s1">\'this is the end of the python sample\'</span></td></tr></tbody></table></div>'
Diff is 2605 characters long. Set self.maxDiff to None to see it.

======================================================================
FAIL: test_python_with_lineno (trac.mimeview.tests.pygments.PygmentsRendererTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/tests/pygments.py", line 110, in test_python_with_lineno
    self._test('python_with_lineno_1', result)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/tests/pygments.py", line 69, in _test
    self.assertEqual(exp, res)
AssertionError: u'<div class="wiki-code"><table class="code"><thead><tr><th class="lineno" title="Line numbers">Line</th><th class="content">\xa0</th></tr></thead><tbody><tr><th id="a-L1"><a href="#a-L1">1</a></th><td><span class="k">print</span>\xa0<span class="s">\'this is a python sample\'</span></td></tr><tr><th id="a-L2"><a href="#a-L2">2</a></th><td>a <span class="o">=</span>\xa0b<span class="o">+</span><span class="mi">3</span></td></tr><tr><th id="a-L3"><a href="#a-L3">3</a></th><td>z <span class="o">=</span>\xa0<span class="s">"this is a string"</span></td></tr><tr><th id="a-L4"><a href="#a-L4">4</a></th><td><span class="k">print</span>\xa0<span class="s">\'this is the end of the python sample\'</span></td></tr></tbody></table></div>' != u'<div class="wiki-code"><table class="code"><thead><tr><th class="lineno" title="Line numbers">Line</th><th class="content">\xa0</th></tr></thead><tbody><tr><th id="a-L1"><a href="#a-L1">1</a></th><td><span class="k">print</span>\xa0<span class="s1">\'this is a python sample\'</span></td></tr><tr><th id="a-L2"><a href="#a-L2">2</a></th><td>a <span class="o">=</span>\xa0b<span class="o">+</span><span class="mi">3</span></td></tr><tr><th id="a-L3"><a href="#a-L3">3</a></th><td>z <span class="o">=</span>\xa0<span class="s2">"this is a string"</span></td></tr><tr><th id="a-L4"><a href="#a-L4">4</a></th><td><span class="k">print</span>\xa0<span class="s1">\'this is the end of the python sample\'</span></td></tr></tbody></table></div>'
Diff is 2605 characters long. Set self.maxDiff to None to see it.

======================================================================
FAIL: test_python_with_lineno_and_markups (trac.mimeview.tests.pygments.PygmentsRendererTestCase)
Python highlighting with Pygments and lineno annotator
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/tests/pygments.py", line 135, in test_python_with_lineno_and_markups
    self._test('python_with_lineno_and_markups', result)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/mimeview/tests/pygments.py", line 69, in _test
    self.assertEqual(exp, res)
AssertionError: u'<div class="wiki-code"><table class="code"><thead><tr><th class="lineno" title="Line numbers">Line</th><th class="content">\xa0</th></tr></thead><tbody><tr><th id="b-L3"><a href="#b-L3">3</a></th><td><span class="k">print</span>\xa0<span class="s">\'this is a python sample\'</span></td></tr><tr class="hilite"><th id="b-L4"><a href="#b-L4">4</a></th><td>a <span class="o">=</span>\xa0b<span class="o">+</span><span class="mi">3</span></td></tr><tr class="hilite"><th id="b-L5"><a href="#b-L5">5</a></th><td>z <span class="o">=</span>\xa0<span class="s">"this is a string"</span></td></tr><tr><th id="b-L6"><a href="#b-L6">6</a></th><td><span class="k">print</span>\xa0<span class="s">\'this is the end of the python sample\'</span></td></tr></tbody></table></div>' != u'<div class="wiki-code"><table class="code"><thead><tr><th class="lineno" title="Line numbers">Line</th><th class="content">\xa0</th></tr></thead><tbody><tr><th id="b-L3"><a href="#b-L3">3</a></th><td><span class="k">print</span>\xa0<span class="s1">\'this is a python sample\'</span></td></tr><tr class="hilite"><th id="b-L4"><a href="#b-L4">4</a></th><td>a <span class="o">=</span>\xa0b<span class="o">+</span><span class="mi">3</span></td></tr><tr class="hilite"><th id="b-L5"><a href="#b-L5">5</a></th><td>z <span class="o">=</span>\xa0<span class="s2">"this is a string"</span></td></tr><tr><th id="b-L6"><a href="#b-L6">6</a></th><td><span class="k">print</span>\xa0<span class="s1">\'this is the end of the python sample\'</span></td></tr></tbody></table></div>'
Diff is 2725 characters long. Set self.maxDiff to None to see it.

----------------------------------------------------------------------
Ran 2035 tests in 156.712s

FAILED (failures=3)

I'll investigate tomorrow.

[Jun Omae]

Unit test failing without Pygments.

python ./trac/test.py --skip-functional-tests 
SKIP: fine-grained permission tests (ConfigObj not installed)
Traceback (most recent call last):
  File "./trac/test.py", line 472, in <module>
    unittest.main(defaultTest='suite')
  File "/usr/lib/python2.5/unittest.py", line 767, in __init__
    self.parseArgs(argv)
  File "/usr/lib/python2.5/unittest.py", line 794, in parseArgs
    self.createTests()
  File "/usr/lib/python2.5/unittest.py", line 800, in createTests
    self.module)
  File "/usr/lib/python2.5/unittest.py", line 565, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python2.5/unittest.py", line 553, in loadTestsFromName
    test = obj()
  File "./trac/test.py", line 428, in suite
    import trac.mimeview.tests
  File "/run/shm/f9fd19921f51b96374766f7218644ce0e335be2b/py25-sqlite/trac/mimeview/tests/__init__.py", line 14, in <module>
    from trac.mimeview.tests import api, patch, pygments
  File "/run/shm/f9fd19921f51b96374766f7218644ce0e335be2b/py25-sqlite/trac/mimeview/tests/pygments.py", line 39, in <module>
    pygments_version = parse_version(get_pkginfo(pygments).get('version'))
NameError: name 'pygments' is not defined
make: *** [unit-test] Error 1

[Christian Boos]

Replying to Ryan J Ollos:

Proposed changes in log:rjollos.git:t12326_pygments_2.1.

One minor thing: id="...pygments_2.1+ should be pygments_2.1plus (no + in ​id). Of course it also works without that and there are anyway other minor validation issues in this file, but this wrong id stood out ;-)

[Ryan J Ollos]

Thanks for the feedback. I revised log:rjollos.git:t12326_pygments_2.1 to address comment:8 and comment:9, and prepared changes for the trunk: log:rjollos.git:t12326_pygments_2.1_trunk.

Tests pass with Pygments 2.1, Pygments 2.0.2 and no Pygments.

[Ryan J Ollos]

I'll push tomorrow if there is no further feedback, so that we have the Travis CI tests passing once again.

[Christian Boos]

t12326_pygments_2.1_trunk works fine for me, thanks!

[Ryan J Ollos]

Committed to 1.0-stable in [14490], merged to trunk in [14491].