PygmentsRenderer produces spurious END token

[Tim Hatch <trac@…>]

Feed it an empty file and you'll get the following while trying to render:

Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/trac/mimeview/api.py", line 475, in render
    return self._annotate(result, annotations, m and Ranges(m))
  File "/usr/lib/python2.4/site-packages/trac/mimeview/api.py", line 512, in _annotate
    tag.tbody(_body_rows())
  File "build/bdist.linux-i686/egg/genshi/builder.py", line 177, in __call__
  File "build/bdist.linux-i686/egg/genshi/builder.py", line 32, in __call__
  File "build/bdist.linux-i686/egg/genshi/builder.py", line 57, in append
  File "/usr/lib/python2.4/site-packages/trac/mimeview/api.py", line 501, in _body_rows
    for idx, line in enumerate(_group_lines(stream)):
  File "/usr/lib/python2.4/site-packages/trac/mimeview/api.py", line 676, in _group_lines
    for kind, data, pos in _generate():
  File "/usr/lib/python2.4/site-packages/trac/mimeview/api.py", line 671, in _generate
    stack.pop()
IndexError: pop from empty list

This is the entirety of the events which causes this traceback, generated by the PygmentsRenderer?:

'END', QName(u'span'), (None, -1, -1)

I believe this is caused by yielding an END token regardless of whether a span was open (lattrs is None). Attempting to correct for this is proving quite confusing because it won't produce the number of \n that seems logical. I think this needs another pair of eyes.

[Tim Hatch <trac@…>]

With the addition of a test suite and reverting local modification to trunk, I am not able to reproduce this issue any longer. If there is independent verification, reopen... otherwise closing as worksforme.