"OverflowError: long int too large to convert to int" - large integer enclosed in brackets

[anonymous]

A user entered an error message containing square brackets around large (64 bit) integers. The ticket was created correctly but now cannot be viewed (or, more significantly, edited to fix it.)

While I believe errors in using wiki formatting is a user error, the system needs to allow the user to recover from it.

This is not an installation issue - Trac 0.12 has been stable for us for several months.

Symptom

Oops…
Trac detected an internal error: OverflowError: long int too large to convert to int

Description line which caused problem

 255,"Computed frequency sweep [start,stop] out of bounds;  [-19980000000,20000000000] outside of range [10000000,20000000000] for  STAR = 0, STOP = 20000000000"

Python Traceback

Most recent call last: File "C:/Python25/lib/site-packages/trac-0.12-py2.5-win32.egg/trac/ticket/templates/ticket_box.html", line 77, in <Expression u'wiki_to_html(context, ticket.description, escape_newlines=preserve_newlines)'> 
Code fragment:
Line  
72         <input type="hidden" name="replyto" value="description" /> 
73         <input type="submit" name="reply" value="${_('Reply')}" title="Reply, quoting this description" /> 
74       </div> 
75     </form> 
76     <div py:if="ticket.description" class="searchable" xml:space="preserve"> 
77       ${wiki_to_html(context, ticket.description, escape_newlines=preserve_newlines)} 
78     </div> 
79     <br py:if="not ticket.description" style="clear: both" /> 
80   </div> 
81 </div> 

Local variables:
Name Value 
__data__ [{'can_edit': True, '_i18n.ngettext': <bound method ... 

File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/wiki/formatter.py", line 1497, in format_to_html 
Code fragment:
Line  
1492 def format_to_html(env, context, wikidom, escape_newlines=None): 
1493     if not wikidom: 
1494         return Markup() 
1495     if escape_newlines is None: 
1496         escape_newlines = context.get_hint('preserve_newlines', False) 
1497     return HtmlFormatter(env, context, wikidom).generate(escape_newlines) 
1498   
1499 def format_to_oneliner(env, context, wikidom, shorten=None): 
1500     if not wikidom: 
1501         return Markup() 
1502     if shorten is None: 

Local variables:
Name Value 
context <Context <Resource u'ticket:289'>> 
env <trac.env.Environment object at 0x052EEF30> 
escape_newlines False 
wikidom u'freq:stop 20e9; cent 10e6\r\n\r\nGenerates this:\r\n\r\n255,"Computed ... 

File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/wiki/formatter.py", line 1452, in generate 
Code fragment:
Line  
1447         newlines in the wikidom will be preserved if `escape_newlines` is set. 
1448         """ 
1449         # FIXME: compatibility code only for now 
1450         out = StringIO() 
1451         Formatter(self.env, self.context).format(self.wikidom, out, 
1452                                                  escape_newlines) 
1453         return Markup(out.getvalue()) 
1454   
1455   
1456 class InlineHtmlFormatter(object): 
1457     """Format parsed wiki text to inline elements HTML. 

Local variables:
Name Value 
escape_newlines False 
out <StringIO.StringIO instance at 0x06DBD210> 
self <trac.wiki.formatter.HtmlFormatter object at 0x06DC8D50> 

File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/wiki/formatter.py", line 1233, in format 
Code fragment:
Line  
1228   
1229             self.in_list_item = False 
1230             self.in_quote = False 
1231             # Throw a bunch of regexps on the problem 
1232             self.line = line 
1233             result = re.sub(self.wikiparser.rules, self.replace, line) 
1234   
1235             if not self.in_list_item: 
1236                 self.close_list() 
1237   
1238             if not self.in_quote: 

Local variables:
Name Value 
block_start_match None 
escape_newlines False 
line u'255,"Computed frequency sweep [start,stop] out of bounds; ... 
out <StringIO.StringIO instance at 0x06DBD210> 
result u'Generates this:' 
self <trac.wiki.formatter.Formatter object at 0x06DC8D90> 
sep '\r\n' 
text [u'freq:stop 20e9; cent 10e6', u'', u'Generates this:', u'', ... 

File "C:/Python25/Lib/re.py", line 150, in sub 
Code fragment:
Line  
145     """Return the string obtained by replacing the leftmost 
146     non-overlapping occurrences of the pattern in string by the 
147     replacement repl.  repl can be either a string or a callable; 
148     if a callable, it's passed the match object and must return 
149     a replacement string to be used.""" 
150     return _compile(pattern, 0).sub(repl, string, count) 
151   
152 def subn(pattern, repl, string, count=0): 
153     """Return a 2-tuple containing (new_string, number). 
154     new_string is the string obtained by replacing the leftmost 
155     non-overlapping occurrences of the pattern in the source 

Local variables:
Name Value 
count 0 
pattern <_sre.SRE_Pattern object at 0x056BEE98> 
repl <bound method Formatter.replace of <trac.wiki.formatter.Formatter object ... 
string u'255,"Computed frequency sweep [start,stop] out of bounds; ... 

File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/wiki/formatter.py", line 1151, in replace 
Code fragment:
Line  
1146                     internal_handler = getattr(self, '_%s_formatter' % itype) 
1147                     return internal_handler(match, fullmatch) 
1148   
1149     def replace(self, fullmatch): 
1150         """Replace one match with its corresponding expansion""" 
1151         replacement = self.handle_match(fullmatch) 
1152         if replacement: 
1153             return _markup_to_unicode(replacement) 
1154   
1155     _normalize_re = re.compile(r'[\v\f]', re.UNICODE) 
1156   

Local variables:
Name Value 
fullmatch <_sre.SRE_Match object at 0x064CFFC8> 
self <trac.wiki.formatter.Formatter object at 0x06DC8D90> 

File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/wiki/formatter.py", line 1144, in handle_match 
Code fragment:
Line  
1139                 # Check for preceding escape character '!' 
1140                 if match[0] == '!': 
1141                     return escape(match[1:]) 
1142                 if itype in self.wikiparser.external_handlers: 
1143                     external_handler = self.wikiparser.external_handlers[itype] 
1144                     return external_handler(self, match, fullmatch) 
1145                 else: 
1146                     internal_handler = getattr(self, '_%s_formatter' % itype) 
1147                     return internal_handler(match, fullmatch) 
1148   
1149     def replace(self, fullmatch): 

Local variables:
Name Value 
external_handler <function <lambda> at 0x057D1070> 
fullmatch <_sre.SRE_Match object at 0x064CFFC8> 
itype u'i6' 
match u'[10000000,20000000000]' 
self <trac.wiki.formatter.Formatter object at 0x06DC8D90> 

File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/versioncontrol/web_ui/log.py", line 332, in <lambda> 
Code fragment:
Line  
327         yield ( 
328             # [...] form, starts with optional intertrac: [T... or [trac ... 
329             r"!?\[(?P<it_log>%s\s*)" % WikiParser.INTERTRAC_SCHEME + 
330             # <from>:<to> + optional path restriction 
331             r"(?P<log_revs>%s)(?P<log_path>[/?][^\]]*)?\]" % self.REV_RANGE, 
332             lambda x, y, z: self._format_link(x, 'log1', y[1:-1], y, z)) 
333         yield ( 
334             # r<from>:<to> form (no intertrac and no path restriction) 
335             r"(?:\b|!)r%s\b" % Ranges.RE_STR, 
336             lambda x, y, z: self._format_link(x, 'log2', '@' + y[1:], y)) 
337   

Local variables:
Name Value 
self <trac.versioncontrol.web_ui.log.LogModule object at 0x067B2990> 
x <trac.wiki.formatter.Formatter object at 0x06DC8D90> 
y u'[10000000,20000000000]' 
z <_sre.SRE_Match object at 0x064CFFC8> 

File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/versioncontrol/web_ui/log.py", line 380, in _format_link 
Code fragment:
Line  
375             revranges = None 
376             if any(c for c in ':-,' if c in revs): 
377                 revranges = self._normalize_ranges(repos, path, revs) 
378                 revs = None 
379             if 'LOG_VIEW' in formatter.perm: 
380                 if revranges: 
381                     href = formatter.href.log(repos.reponame or None, 
382                                               path or '/', revs=str(revranges)) 
383                 else: 
384                     try: 
385                         rev = repos.normalize_rev(revs) 

Local variables:
Name Value 
formatter <trac.wiki.formatter.Formatter object at 0x06DC8D90> 
fragment '' 
fullmatch <_sre.SRE_Match object at 0x064CFFC8> 
groups {u'htext': None, u'wiki_label': None, u'shrefbr': None, u'inline2': None, ... 
intertrac None 
it_log u'' 
label u'[10000000,20000000000]' 
match u'10000000,20000000000' 
ns 'log1' 
path '/' 
query '' 
reponame '' 
repos <trac.versioncontrol.svn_fs.SvnCachedRepository object at 0x06EADB30> 
revranges <trac.util.Ranges object at 0x06E985B0> 
revs None 
rm <trac.versioncontrol.api.RepositoryManager object at 0x067B2110> 
self <trac.versioncontrol.web_ui.log.LogModule object at 0x067B2990> 
target u'/@10000000,20000000000' 
File "C:/Python25/lib/site-packages/trac-0.12-py2.5-win32.egg/trac/ticket/templates/ticket_box.html", line 77, in <Expression u'wiki_to_html(context, ticket.description, escape_newlines=preserve_newlines)'>
  ${wiki_to_html(context, ticket.description, escape_newlines=preserve_newlines)}
File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/wiki/formatter.py", line 1497, in format_to_html
  return HtmlFormatter(env, context, wikidom).generate(escape_newlines)
File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/wiki/formatter.py", line 1452, in generate
  escape_newlines)
File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/wiki/formatter.py", line 1233, in format
  result = re.sub(self.wikiparser.rules, self.replace, line)
File "C:/Python25/Lib/re.py", line 150, in sub
  return _compile(pattern, 0).sub(repl, string, count)
File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/wiki/formatter.py", line 1151, in replace
  replacement = self.handle_match(fullmatch)
File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/wiki/formatter.py", line 1144, in handle_match
  return external_handler(self, match, fullmatch)
File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/versioncontrol/web_ui/log.py", line 332, in <lambda>
  lambda x, y, z: self._format_link(x, 'log1', y[1:-1], y, z))
File "c:/python25/lib/site-packages/Trac-0.12-py2.5-win32.egg/trac/versioncontrol/web_ui/log.py", line 380, in _format_link
  if revranges

System Information

User Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E)

Trac 0.12 
Genshi 0.6 
mod_python 3.3.1 
pysqlite 2.3.2 
Python 2.5.5 (r255:77872, Jan 31 2010, 15:49:35) [MSC v.1310 32 bit (Intel)] 
setuptools 0.6c9 
SQLite 3.3.4 
Subversion 1.6.12 (r955767) 
jQuery 1.4.2 

[Remy Blank]

Thanks for the bug report!

[Remy Blank]

This is due to Python expecting Ranges.__len__() to return an int, and in this case, the value is larger than fits an int. __len__() is actually called in a boolean context, so the following patch fixes the issue:

trac/util/__init__.py

@@ -801 +801 @@
         else:
             return 0
 
+    def __nonzero__(self):
+        """Return True iff the range is not empty.
+        
+        >>> (bool(Ranges()), bool(Ranges('1-2')))
+        (False, True)
+        """
+        return self.a is not None and self.b is not None
+
     def truncate(self, max):
         """Truncate the Ranges by setting a maximal allowed value.
 

However, I'm not sure if we shouldn't rather tweak Ranges.__len__() to return sys.maxint if the value exceeds an int, something like:

trac/util/__init__.py

@@ -797 +797 @@
         (1, 2, 0)
         """
         if self.a is not None and self.b is not None:
-            return self.b - self.a + 1
+            # Result must fit an int
+            return min(self.b - self.a + 1, sys.maxint)
         else:
             return 0
 

Or maybe even both.

[Remy Blank]

There seems to have been a debate on the Python list about having len() silently "lie" and clip lengths at sys.maxint, with the result of keeping the status-quo of raising an OverflowError. As we don't seem to use the actual length of Ranges objects other than to test for an empty range, I have committed the first patch in [10411].

[Thijs Triemstra]

There's a typo in the docstring of the applied patch;

Return True iff the range is not empty. 

[Christian Boos]

Nah, that's a common abbreviation for "if and only if" ;-)

[Christian Boos]

+ regression test added in r10412.

[anonymous]

Thank you rblank and cboos!

[Remy Blank]

#10111 had the same symptom, but for ticket ranges. In that case however, the check made on the range was len(range) == 1, so adding the __nonzero__() method was insufficient. Applying the fix to __len__() fixed the issue.