SQLiteConnector crashes when Mercurial is 4.8rc0

[Jun Omae]

$ /dev/shm/hg4.8rc0/bin/pip list
Package       Version
------------- -------
Genshi        0.7.1
mercurial     4.8rc0
pip           18.1
pkg-resources 0.0.0
setuptools    40.5.0
Trac          1.0.17
wheel         0.32.2

$ /dev/shm/hg4.8rc0/bin/trac-admin /var/trac/1.0-sqlite
...

Trac [/var/trac/1.0-sqlite]> wiki list
2018-11-02 01:19:16,988 Trac[pool] ERROR: Exception caught on create
Traceback (most recent call last):
  File "/dev/shm/hg4.8rc0/local/lib/python2.7/site-packages/trac/db/pool.py", line 105, in get_cnx
    cnx = connector.get_connection(**kwargs)
  File "/dev/shm/hg4.8rc0/local/lib/python2.7/site-packages/trac/db/sqlite_backend.py", line 172, in get_connection
    self._version = get_pkginfo(sqlite).get(
  File "/dev/shm/hg4.8rc0/local/lib/python2.7/site-packages/trac/util/__init__.py", line 790, in get_pkginfo
    metadata = 'METADATA' if dist.has_metadata('METADATA') else 'PKG-INFO'
  File "/dev/shm/hg4.8rc0/local/lib/python2.7/site-packages/hgdemandimport/demandimportpy2.py", line 151, in __getattr__
    return getattr(self._module, attr)
AttributeError: 'module' object has no attribute 'has_metadata'
Error: Unable to get database connection within 0 seconds. (AttributeError: 'module' object has no attribute 'has_metadata')

(tweeted in ​https://twitter.com/shunichigoto/status/1058003791088365569)

[Jun Omae]

We expect a module instance is imported by import sqlite3 but a hgdemandimport instance is imported with Mercurial 4.8rc0.

Mercurial 4.8rc0

>>> from trac.env import Environment
>>> env = Environment('/var/trac/1.0-sqlite')
>>> env.db_query('SELECT * FROM system LIMIT 1')
AttributeError: 'module' object has no attribute 'has_metadata'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/dev/shm/hg4.8rc0/local/lib/python2.7/site-packages/trac/db/api.py", line 123, in execute
    with self as db:
  File "/dev/shm/hg4.8rc0/local/lib/python2.7/site-packages/trac/db/api.py", line 176, in __enter__
    db = self.dbmgr.get_connection(readonly=True)
  File "/dev/shm/hg4.8rc0/local/lib/python2.7/site-packages/trac/db/api.py", line 289, in get_connection
    db = self._cnx_pool.get_cnx(self.timeout or None)
  File "/dev/shm/hg4.8rc0/local/lib/python2.7/site-packages/trac/db/pool.py", line 216, in get_cnx
    return _backend.get_cnx(self._connector, self._kwargs, timeout)
  File "/dev/shm/hg4.8rc0/local/lib/python2.7/site-packages/trac/db/pool.py", line 137, in get_cnx
    raise TimeoutError(errmsg)
trac.db.pool.TimeoutError: Unable to get database connection within 0 seconds. (AttributeError: 'module' object has no attribute 'has_metadata')
>>> 
>>> from mercurial.util import version
>>> version()
'4.8rc0'
>>> import sqlite3
>>> type(sqlite3)
<class 'hgdemandimport.demandimportpy2._demandmod'>
>>> from trac.util import get_pkginfo
>>> get_pkginfo(sqlite3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/dev/shm/hg4.8rc0/local/lib/python2.7/site-packages/trac/util/__init__.py", line 790, in get_pkginfo
    metadata = 'METADATA' if dist.has_metadata('METADATA') else 'PKG-INFO'
  File "/dev/shm/hg4.8rc0/local/lib/python2.7/site-packages/hgdemandimport/demandimportpy2.py", line 151, in __getattr__
    return getattr(self._module, attr)
AttributeError: 'module' object has no attribute 'has_metadata'

Mercurial 4.7.2

>>> from trac.env import Environment
>>> env = Environment('/var/trac/1.0-sqlite')
>>> env.db_query('SELECT * FROM system LIMIT 1')
[(u'database_version', u'29')]
>>> 
>>> from mercurial.util import version
>>> version()
'4.7.2'
>>> import sqlite3
>>> sqlite3
<module 'sqlite3' from '/usr/lib/python2.7/sqlite3/__init__.pyc'>
>>> from trac.util import get_pkginfo
>>> get_pkginfo(sqlite3)
{}

[Jun Omae]

Work around is here but python setup.py test still fails.

diff -r 1b6c6b50f742 tracext/hg/backend.py
--- a/tracext/hg/backend.py     Wed Sep 19 19:57:36 2018 +0900
+++ b/tracext/hg/backend.py     Fri Nov 02 13:21:41 2018 +0900
@@ -67,9 +67,12 @@

     try:
         from mercurial import demandimport
-        demandimport.enable();
     except ImportError, hg_import_error:
         demandimport = None
+    else:
+        demandimport.IGNORES.add('sqlite3')
+        demandimport.IGNORES.add('pysqlite2.dbapi2')
+        demandimport.enable()

     from mercurial import hg
     from mercurial.context import filectx

[Jun Omae]

Proposed changes in t13100.diff​, tested with Mercurial 1.1.2, 1.2, 2.3, 2.3.1, 2.6.3, 2.9.1, 3.7.3, 4.5.3, 4.6.2, 4.7.2 and 4.8rc0.

Root cause is that demandimport.disable() is not invoked when ImportError is raised from from mercurial.revlog import LookupError​. The LookupError has been removed from mercurial.revlog in ​https://www.mercurial-scm.org/repo/hg/rev/974592474dee.

[Jun Omae]

Mercurial 4.8 is already released (​https://www.mercurial-scm.org/repo/hg/rev/4.8) and the issue is reproduced with 4.8. I'm going to push the changes.

[Jun Omae]

Fixed in [afea9958cf7c/mercurial-plugin].