0.12b1 is not working with Pysqlite 1.1.7, SQLite 3.3.6

[aaron@…]

My current SQLite and Pysqlite versions, on a shared server:

>>> import sqlite
>>> sqlite.version
'1.1.7'
>>> sqlite._sqlite.sqlite_version()
'3.3.6'

0.11.7 does this:

>>> import trac.db.sqlite_backend as test
>>> test.have_pysqlite
1
>>> test.sqlite.version
'1.1.7'
>>> test._ver
(3, 3, 6)

But 0.12b1 does this:

>>> import trac.db.sqlite_backend as test
>>> test.have_pysqlite
0

Traceback of the error that 0.12b1 sees:

2010-05-18 17:52:23,454 Trac[__init__] ERROR: Exception caught while checking for upgrade:
Traceback (most recent call last):
  File "build/bdist.linux-i686/egg/trac/env.py", line 673, in open_environment
  File "build/bdist.linux-i686/egg/trac/env.py", line 505, in needs_upgrade
  File "build/bdist.linux-i686/egg/trac/env.py", line 335, in get_db_cnx
  File "build/bdist.linux-i686/egg/trac/db/api.py", line 91, in get_connection
  File "build/bdist.linux-i686/egg/trac/db/api.py", line 136, in get_connector
TracError: Cannot load Python bindings for SQLite

According to everything I can find, SQLite 3.3.6 with Pysqlite 1.1.7 is still supported; only SQLite 2.x and its corresponding Pysqlite 1.0.x bindings have had support dropped in Trac 0.12.

I'd blame this on server configuration, but 0.11.7 works fine. Please advise me what I can do to help diagnose this!

[Remy Blank]

Replying to aaron@…:

According to everything I can find, SQLite 3.3.6 with Pysqlite 1.1.7 is still supported; only SQLite 2.x and its corresponding Pysqlite 1.0.x bindings have had support dropped in Trac 0.12.

Wrong. From TracInstall:

SQLite 2.x is no longer supported. For SQLite 3.x, the pysqlite 1.1.x bindings are also no longer supported, use pysqlite 2.x.

Please upgrade pysqlite to 2.x.

[aaron@…]

Fair enough.

FYI, TracUpgrade says only that SQLite 3.x is required for 0.12, and PySqlite says only that support for the 1.0.x bindings is dropped for 0.12. As somebody who is upgrading rather than installing afresh, that is the documentation I was reading—something up-to-date would be appreciated!

[Christian Boos]

I had a second look on this. Apparently our real intent was to drop support for 1.0.x only, and keep 1.1.7 support. I think there is a mistake in the check done in r8582, ("Need at least PySqlite 1.0.7 or higher"), as this should have been ("Need at least PySqlite 1.1.7 or higher") and the comparison is consistent with this mistake. Because of the wrong "1.0" part, those lines got dropped when merged in 0.12 (r8583), making pysqlite 2.x effectively required for SQLite ≥ 3.3.3, when 1.1.7 would have been OK (well, "sort of" OK).

[aaron@…]

That does make perfect sense. It also explains the differing documentation, and why I couldn't find a ticket where the decision was made to drop 1.1.x support in addition to dropping 1.0.x support.

Thanks a ton for looking into this. I thought I was going crazy!

[Christian Boos]

Wrong test in r8582 fixed in r9726 and support for PySqlite 1.1.7 restored for 0.12 in r9727.

[aaron@…]

Is it possible something was missed when fixing this?

I ask because I just tried upgrading to Trac 0.12 final, and was stymied once more by the same error, and Trac failing the same have_pysqlite test.

Here is the new traceback:

2010-07-23 17:31:11,098 Trac[__init__] ERROR: Exception caught while checking for upgrade:
Traceback (most recent call last):
  File "build/bdist.linux-i686/egg/trac/env.py", line 672, in open_environment
  File "build/bdist.linux-i686/egg/trac/env.py", line 504, in needs_upgrade
  File "build/bdist.linux-i686/egg/trac/env.py", line 335, in get_db_cnx
  File "build/bdist.linux-i686/egg/trac/db/api.py", line 90, in get_read_db
  File "build/bdist.linux-i686/egg/trac/db/api.py", line 150, in get_connection
  File "build/bdist.linux-i686/egg/trac/db/api.py", line 195, in get_connector
TracError: Cannot load Python bindings for SQLite

Any assistance would be appreciated, thanks!

[Remy Blank]

This has nothing to do with this issue. Your Trac installation is unable to even load any SQLite bindings. At least one of the following should succeed on a Python prompt:

import pysqlite2.dbapi
import sqlite3

If it does, make sure the Python installation used by your web server is correct. This can be tricky if you are using mod_python, for example.

[aaron@…]

Replying to rblank:

This has nothing to do with this issue. Your Trac installation is unable to even load any SQLite bindings. At least one of the following should succeed on a Python prompt:

import pysqlite2.dbapi
import sqlite3

If it does, make sure the Python installation used by your web server is correct. This can be tricky if you are using mod_python, for example.

Unfortunately that's incorrect. Both of those fail, because both of those rely on Pysqlite 2; I have Pysqlite 1.1.7.

Pysqlite 2 bundles SQLite 3 as sqlite3; Pysqlite 1 bundles SQLite 3 as sqlite.

import sqlite succeeds at my Python prompt. If Trac would simply try importing sqlite instead of sqlite3, Trac 0.12 would successfully support Pysqlite 1.1.7 again.

HostGator shared environments are stuck on Pysqlite 1.1.7. So until we're able to move to a different hosting environment, we're stuck as well. And until Trac 0.12 is fixed, we're stuck on 0.11.

[Remy Blank]

Replying to aaron@…:

Unfortunately that's incorrect. Both of those fail, because both of those rely on Pysqlite 2; I have Pysqlite 1.1.7.

Sorry, my bad. I'll fix this for 0.12.1.

[aaron@…]

Thank you! And I greatly look forward to the day I get the budget to leave HostGator's shared hosting behind and run Trac on something production-worthy.

[aaron@…]

(Oh, and just remembering to actually re-open the ticket, since I didn't want to be rude and do that originally.)

[Remy Blank]

Aaron, do you happen to have pysqlite-1.1.7.tar.gz at hand? I can't find it anywhere on the net, and without that I can't test my fix…

[Remy Blank]

Restore pysqlite-1.1.7 compatibility in 0.12-stable.

[Remy Blank]

9352-pysqlite-1.patch​ restores pysqlite-1.1.x compatibility. I have made it require 1.1.7, but in theory this should only be necessary for sqlite 3.3.3 or higher. Should I only check for ≥1.1.0?

As I can't find a tarball of pysqlite-1.1.7, I can't test the fix. Aaron, could you please test the patch? Ideally, you should try running the unit tests and functional tests, and if they pass with 1.1.7, we should be in good shape.

[Remy Blank]

(Of course, since we have a copy of the [pysqlite: old pysqlite Trac site] and repository, I could just install from the tag. Or from the tag in the new repository on Google Code.)

[Remy Blank]

I finally managed to install pysqlite 1.1.7, and the bad news is it doesn't support values larger than 2³¹-1, as it converts INTEGER column values to int. This effectively disqualifies it for 0.12, as our microsecond timestamps require larger values.

So I'm going to remove support for pysqlite 1 cleanly in 0.12-stable, and update the documentation. Sorry about that.

[Remy Blank]

The last traces of pysqlite 1.1.x support have been removed in [10064], and the docs have been updated (TracInstall@317, TracUpgrade@80, PySqlite@51).

[aaron@…]

Ah, no worries dude. You tried! Appreciate you looking into this. I'll let you know if I find any other inconsistencies in the docs, too.