1 | | = PySqlite = |
2 | | |
3 | | |
4 | | [http://pysqlite.googlecode.com PySqlite] is a Python binding |
5 | | for the [http://www.sqlite.org SQLite] light-weight database engine, |
6 | | which is Trac's default DatabaseBackend. |
7 | | |
8 | | == Installation == |
9 | | === The short version === |
10 | | |
11 | | {{{ |
12 | | #!div style="background: #f0f0f0; padding: .5em" |
13 | | |
14 | | If you're using Python 2.5 and up, you already have a working version of pysqlite 2, bundled as `sqlite3`. You can stop here ;-) |
15 | | |
16 | | If you are using an older version of Python or you'd like to benefit from the latest and greatest version of pysqlite, grab Windows installer or the source .tar.gz from the official [http://code.google.com/p/pysqlite/downloads/list Downloads]. |
| 1 | = PySqlite |
| 2 | |
| 3 | [https://pypi.python.org/pypi/pysqlite PySqlite] is a Python binding for the [http://www.sqlite.org SQLite] light-weight database engine, which is Trac's default DatabaseBackend. |
| 4 | |
| 5 | == Installation |
| 6 | === The short version |
| 7 | |
| 8 | If you're using Python 2.5 and up, you already have a working version of pysqlite 2, bundled as `sqlite3`. |
| 9 | |
| 10 | If you are using an older version of Python or you'd like to benefit from the latest version of pysqlite, grab Windows installer or the source .tar.gz from the official [https://pypi.python.org/pypi/pysqlite pysqlite site]. |
28 | | === The SQLite library === |
29 | | |
30 | | Trac supports '''[http://www.sqlite.org/version3.html SQLite 3.x]''', |
31 | | like SQLite 3.3.8, 3.5.6, etc. The latest tested as of this writing ('''3.6.12''') works fine. |
32 | | |
33 | | Pay attention to |
34 | | [http://www.sqlite.org/formatchng.html database format changes] |
35 | | when upgrading the SQLite library. |
36 | | See below how to [wiki:PySqlite#UpgradingSQLite upgrade] your database, |
37 | | if needed (SQLite v2.x is no longer supported in 0.12.x, use it at your own risks with earlier versions of Trac). |
38 | | The recent ''[[http://www.sqlite.org/wal.html|WAL]]'' mode introduced with sqlite 3.7.0 also works great with Trac. |
39 | | |
40 | | |
41 | | ==== Downloading SQLite ==== |
42 | | |
43 | | The latest stable version of SQLite can be obtained on the |
44 | | [http://www.sqlite.org/download.html SQLite download] page. |
45 | | |
46 | | But you don't need to download it by yourself, |
47 | | if you install the pysqlite bindings from a package, it |
48 | | will usually come with the SQLite code bundled. |
49 | | |
50 | | If you build the pysqlite bindings by yourself, you also |
51 | | have the possibility to use the `build_static` mode, which |
52 | | will download the latest "amalgamation" code of SQLite and |
53 | | build it together with the extension code. |
54 | | |
55 | | |
56 | | ==== Building SQLite yourself ==== |
57 | | |
58 | | If you really insist in being in full control... |
59 | | |
60 | | '''Note:''' If you want to use Trac in a multi-threaded setup |
61 | | by using either TracModPython or TracStandalone, be sure to build a |
62 | | '''thread-safe version of SQLite''', by using the `--enable-threadsafe` |
63 | | configuration switch. |
64 | | |
65 | | If you use a non thread-safe library, which is unfortunately what you |
66 | | get by default on non-windows platforms, you face the risk to get |
67 | | persistent database locks (see #2170). |
68 | | |
69 | | === The Pysqlite2 bindings === |
70 | | |
71 | | The latest stable version available at pysqlite.googlecode.com as of this writing is '''2.5.6''', and it works fine with Trac. |
72 | | |
73 | | {{{ |
74 | | #!comment |
75 | | For the very adventurous people, note that there's also a few experimental versions: |
76 | | - ctypes based: http://code.google.com/p/pysqlite/source/browse/?r=ctypes |
77 | | - cython based: http://code.google.com/p/pysqlite/source/browse/?r=cython |
78 | | |
79 | | The ctypes based one worked quite fine with Trac at the time I tested it. |
80 | | |
81 | | The reason why this info is commented out is that I don't know if Gerhard really wants those versions to be publicized ;-) |
82 | | }}} |
| 20 | === The SQLite library |
| 21 | |
| 22 | Trac supports '''[http://www.sqlite.org/version3.html SQLite 3.x]'''. |
| 23 | |
| 24 | Pay attention to [http://www.sqlite.org/formatchng.html database format changes] when upgrading the SQLite library, see below for [wiki:PySqlite#UpgradingSQLite details]. Note that SQLite v2.x is no longer supported in Trac 0.12.x, use it at your own risk with earlier versions of Trac. |
| 25 | |
| 26 | The [http://www.sqlite.org/wal.html WAL mode] introduced with SQLite 3.7.0 also works great with Trac. |
| 27 | |
| 28 | ==== Downloading SQLite |
| 29 | |
| 30 | The latest stable version of SQLite can be obtained on the [http://www.sqlite.org/download.html SQLite download] page. However, if you install the pysqlite bindings from a package, it will usually come with the SQLite code bundled. |
| 31 | |
| 32 | If you build the pysqlite bindings by yourself, you also have the possibility to use the `build_static` mode, which will download the latest "amalgamation" code of SQLite and build it together with the extension code. |
| 33 | |
| 34 | ==== Building SQLite yourself |
| 35 | |
| 36 | '''Note:''' If you want to use Trac in a multi-threaded setup by using either TracModPython or TracStandalone, build a '''thread-safe version of SQLite''' by using the `--enable-threadsafe` configuration switch. |
| 37 | |
| 38 | If you use a non thread-safe library, which is unfortunately what you get by default on non-Windows platforms, you face the risk of persistent database locks, see #2170. |
| 39 | |
| 40 | === The Pysqlite2 bindings |
100 | | * The ''2.3.2'' version is the one which ships with Python 2.5 (available there as the `sqlite3` package. Trac tries to use it if the `pysqlite2` package is installed). |
101 | | * You need ''2.3.3'' if you use Apache and `mod_cache` (see Pysqlite:#174). |
102 | | * The versions 2.5.2 - 2.5.4 have a bug that prevents upgrading from 0.11 to 0.12 (and possibly other breakage), so don't use any of those (see #9434). |
103 | | |
104 | | Note that bug reports involving the obsolete pysqlite 1.0.x (for SQLite v2) |
105 | | will simply not be considered. Starting with 0.11.6, there will be an explicit |
106 | | deprecation warning and in 0.12, support for that version will be removed. |
107 | | |
108 | | ==== Downloading Pysqlite ==== |
109 | | |
110 | | * The source tarballs for the versions listed above are available from the |
111 | | [http://code.google.com/p/pysqlite/downloads/list Downloads] tab. |
| 57 | * The ''2.3.2'' version is the one which ships with Python 2.5, available there as the `sqlite3` package. Trac tries to use it if the `pysqlite2` package is installed. |
| 58 | * You need ''2.3.3'' if you use Apache and `mod_cache`, see Pysqlite:#174. |
| 59 | * The versions 2.5.2 to 2.5.4 have a bug that prevents upgrading from 0.11 to 0.12 (and possibly other breakage), so don't use any of those, see #9434. |
| 60 | |
| 61 | Note that bug reports involving the obsolete pysqlite 1.0.x (for SQLite v2) will simply not be considered. Starting with Trac 0.11.6, there will be an explicit deprecation warning and in Trac 0.12, support for that version will be removed. |
| 62 | |
| 63 | ==== Downloading Pysqlite |
| 64 | |
| 65 | The source tarballs for the versions listed above are available from the [https://pypi.python.org/pypi/pysqlite pysqlite site]. |
126 | | ''Note: Users of Mac OS X please take care; the Apple-supplied SQLite contains additional code to support file locking on network filesystems like AFP or SMB. This is not presently (3.3.6) in the mainline sources, so if you build your own SQLite from source it will not function correctly on such filesystems - typically it gives the error "{{{database is locked}}}". [http://www.alastairs-place.net/2006/07/sqlite_and_mac/ A patch] is available for version 3.3.6, based on Apple's code, otherwise you're probably best off using the Apple supplied version (presently 3.1.3).'' |
127 | | |
128 | | == Upgrading SQLite == |
129 | | === Upgrading SQLite from 2.x to 3.x === |
130 | | |
131 | | It is not advised anymore to use SQLite 2.x. Support for this version and the corresponding PySqlite 1.0.x bindings will be dropped in Trac version [milestone:0.12]. |
132 | | If you happen to use that version, you will need to upgrade. |
133 | | |
134 | | The following information is copied from http://dev.ctor.org/pkcs1/wiki/TracUpgrade |
135 | | |
136 | | The database formats used by SQLite 2.x and sqlite 3.x are incompatible. If you upgrade your SQLite version (this can also happen implicitly if you upgrade from PySQLite 1.0.x to 1.1.x or 2.x), then you must convert your database. |
137 | | |
138 | | To do this, install both SQLite 2.8 and SQLite 3.x (they have different filenames so can coexist in the same directory). Then use the following commands (Windows): |
| 78 | '''Users of Mac OS X please take care''': the Apple-supplied SQLite contains additional code to support file locking on network filesystems like AFP or SMB. This is not presently (3.3.6) in the mainline sources, so if you build your own SQLite from source it will not function correctly on such filesystems - typically it gives the error "{{{database is locked}}}". [http://alastairs-place.net/blog/2006/07/10/sqlite-and-mac/ A patch] used to be available, but you're better off using the Apple supplied version (presently 3.1.3). |
| 79 | |
| 80 | == Upgrading SQLite |
| 81 | === Upgrading SQLite from 2.x to 3.x |
| 82 | |
| 83 | It is not advised to use SQLite 2.x, because support for this version and the corresponding PySqlite 1.0.x bindings will be dropped in Trac version [milestone:0.12]. If you happen to use that version, you will need to upgrade. |
| 84 | |
| 85 | The database formats used by SQLite 2.x and SQLite 3.x are incompatible. If you upgrade your SQLite version - and this can also happen implicitly if you upgrade from PySQLite 1.0.x to 1.1.x or 2.x - then you must convert your database. |
| 86 | |
| 87 | To do this, install both SQLite 2.8 and SQLite 3.x. They have different filenames, so can coexist in the same directory. Then use the following commands (Windows): |
158 | | |
159 | | == Troubleshooting == |
160 | | |
161 | | From time to time, there are reports about problems related to |
162 | | Pysqlite and/or SQLite. This section willl guide you through |
163 | | understanding and fixing those issues, should those problems |
164 | | also happen to you. |
165 | | |
166 | | === Determine actual SQLite and PySqlite version === |
167 | | |
168 | | Troubleshooting is greatly helped by knowing exactly what versions are used for {{{sqlite}}} inside your Python interpreter. Especially on *nix systems, a number of different versions might exist - and the actual versions linked and used might not be what you think. |
| 106 | == Troubleshooting |
| 107 | |
| 108 | From time to time, there are reports about problems related to Pysqlite and/or SQLite, and here are a few of those with resolutions. |
| 109 | |
| 110 | === Determine actual SQLite and PySqlite version |
| 111 | |
| 112 | Troubleshooting is greatly helped by knowing exactly what versions are used for {{{sqlite}}} inside your Python interpreter. Especially on *nix systems different versions might exist and the actual versions linked and used might not be what you think. |
260 | | First, if this only happens occasionally and if the Trac server |
261 | | is still reachable after a second attempt, then it's not really |
262 | | a problem. This simply can happen and indicates that some |
263 | | other user was writing to the database at the same time your |
264 | | request triggerd an attempted to write. |
265 | | There are probably a few things that could be enhanced in the |
266 | | future to handle this situation, like automatic retry, see #3446. |
267 | | |
268 | | Also, is this error is somehow inevitable with SQLite, we should make |
269 | | the error message a bit more "user friendly" (#3503). |
270 | | |
271 | | The lock error is also much more frequent if SQLite is used in a |
272 | | multi-threaded environment (like TracStandalone or TracModPython) |
273 | | but the library was not compiled to be thread-safe. |
274 | | See above, [#BuildingSQLiteyourself building from source], and #2170. |
275 | | |
276 | | The real problem with this occurs when ''all'' requests to Trac |
277 | | end up with this error. This indicates a permanent lock situation, |
278 | | which is not normal. Here are the known possible reasons for this: |
| 205 | First, if this only happens occasionally and if the Trac server is still reachable after a second attempt, then it's not really |
| 206 | a problem. This simply can happen and indicates that some other user was writing to the database at the same time your request triggered an attempted to write. |
| 207 | There are probably a few things that could be enhanced in the future to handle this situation, like automatic retry, see #3446. |
| 208 | |
| 209 | Also, if this error is somehow inevitable with SQLite, we should make the error message a bit more "user friendly" (#3503). |
| 210 | |
| 211 | The lock error is also much more frequent if SQLite is used in a multi-threaded environment, like TracStandalone or TracModPython, but the library was not compiled to be thread-safe. See above, [#BuildingSQLiteyourself building from source], and #2170. |
| 212 | |
| 213 | The real problem with this occurs when ''all'' requests to Trac end up with this error. This indicates a permanent lock situation, which is not normal. Here are the known possible reasons for this: |
296 | | ==== `Warning: You can only execute one statement at a time.` ==== |
297 | | |
298 | | This is typically an error which happens when the pysqlite package |
299 | | you're using is loading at runtime a `sqlite3.so` library which has |
300 | | a different version than the one against which pysqlite was built |
301 | | (e.g. see #2993). |
302 | | |
303 | | ==== `DatabaseError: database is full` ==== |
304 | | |
305 | | Besides the obvious reason (no space left on the partition where |
306 | | the trac db resides), you probably ran out of space in either |
307 | | `/tmp` or `/var/tmp`, where SQLite apparently needs to write too. |
308 | | See #2356, #ps20. |
309 | | |
310 | | ==== `NameError: global name 'sqlite' is not defined` ==== |
311 | | |
312 | | This error usually comes from an invalid PySqlite installation:[[BR]] |
313 | | The Python interpreter is not able to import the sqlite module. |
| 231 | ==== `Warning: You can only execute one statement at a time.` |
| 232 | |
| 233 | This is typically an error which happens when the pysqlite package you're using is loading at runtime a `sqlite3.so` library which has a different version than the one against which pysqlite was built, eg see #2993. |
| 234 | |
| 235 | ==== `DatabaseError: database is full` |
| 236 | |
| 237 | Besides the obvious reason (no space left on the partition where the trac db resides), you probably ran out of space in either |
| 238 | `/tmp` or `/var/tmp`, where SQLite apparently needs to write too. See #2356, #ps20. |
| 239 | |
| 240 | ==== `NameError: global name 'sqlite' is not defined` |
| 241 | |
| 242 | This error usually comes from an invalid PySqlite installation: the Python interpreter is not able to import the sqlite module. |
332 | | Another possible mistake is the pysqlite2 installer forgot to make "_sqlite.so" dll unreadable by anyone other than root. Fix by chmod 775 _sqlite.so inside $PYTHON_PATH/site-packages/pysqlite2/. |
333 | | |
334 | | Also make sure you ld.so.cache (ldconfig --print-cache (if using debian)) finds the correct path for installed libraries that Trac depends on. If can't modify, set set your LD_LIBRARY_PATH to point where sqlite dlls are installed. |
335 | | |
336 | | ==== `_sqlite.so: Undefined symbol "PyGILState_Ensure"` ==== |
337 | | |
338 | | If after the import test you encounter this message it most likely means that your Python was not compiled with threads support. On FreeBSD this is easily verified by doing a {{{grep -i threads /var/db/ports/python24/options}}}. If it comes back with a {{{WITHOUT_THREADS=true}}} you need to configure Python to be compiled with threads by executing {{{make config}}} in the appropriate Python ports directory and subsequently recompile and replace Python. |
339 | | |
340 | | ==== `InterfaceError: Cursor needed to be reset because of commit/rollback and can no longer be fetched from.` ==== |
341 | | |
342 | | When upgrading trac databases from 0.11.x to 0.12, the trac-admin upgrade script will die with this error due to a bug in PySqlite versions 2.5.2-2.5.4. It is advised to use either a newer or older release to successfully upgrade. See ticket #9434 for full details. |
343 | | |
344 | | === Other issues === |
345 | | |
346 | | ==== `Segmentation Fault` ==== |
347 | | |
348 | | Some old Python 2.3 versions (like on SuSE 9.0) have a bug |
349 | | related to gc and weakrefs, which might be triggered by Trac. |
350 | | |
351 | | ([http://projects.edgewall.com/trac/ticket/2170#change_8 details]) |
352 | | |
353 | | Segmentation faults can also happen in multi-threaded servers using a SQLite library which has ''not'' been configured to be multi-thread safe (see [#BuildingSQLiteyourself above]). |
354 | | |
355 | | ==== Summary of Known Issues ==== |
| 261 | Another possibility is the pysqlite2 installer forgot to make "_sqlite.so" dll unreadable by anyone other than root. Fix by `chmod 775 _sqlite.so` inside `$PYTHON_PATH/site-packages/pysqlite2/`. |
| 262 | |
| 263 | Also make sure you ld.so.cache (`ldconfig --print-cache` if using Debian) finds the correct path for installed libraries that Trac depends on. If it cannot be modified, set set your LD_LIBRARY_PATH to point where SQLite dlls are installed. |
| 264 | |
| 265 | ==== `_sqlite.so: Undefined symbol "PyGILState_Ensure"` |
| 266 | |
| 267 | If after the import test you encounter this message, it most likely means that your Python was not compiled with threads support. On FreeBSD this is easily verified by doing a {{{grep -i threads /var/db/ports/python24/options}}}. If it comes back with a {{{WITHOUT_THREADS=true}}}, you need to configure Python to be compiled with threads by executing {{{make config}}} in the appropriate Python ports directory and subsequently recompile and replace Python. |
| 268 | |
| 269 | ==== `InterfaceError: Cursor needed to be reset because of commit/rollback and can no longer be fetched from.` |
| 270 | |
| 271 | When upgrading trac databases from 0.11.x to 0.12, the trac-admin upgrade script will die with this error due to a bug in PySqlite versions 2.5.2-2.5.4. It is advised to use either a newer or older release to successfully upgrade. See ticket #9434 for details. |
| 272 | |
| 273 | === Other issues |
| 274 | |
| 275 | ==== `Segmentation Fault` |
| 276 | |
| 277 | Some old Python 2.3 versions (like on SuSE 9.0) have a bug related to gc and weakrefs, which might be triggered by Trac. |
| 278 | |
| 279 | See [http://projects.edgewall.com/trac/ticket/2170#change_8 details]. |
| 280 | |
| 281 | Segmentation faults can also happen in multi-threaded servers using a SQLite library which has ''not'' been configured to be multi-thread safe, see [#BuildingSQLiteyourself above]. |
| 282 | |
| 283 | ==== Summary of Known Issues |