"Segmentation fault" while doing resync or upgrade with trac-admin

[gsmdib@…]

This is the exact message I get trying to perform actions described above on trac 0.10.3 that comes with Ubuntu Feisty. This instance hes some plugins from trac-hacks installed. Please let me know what additional info should I provide.

[Emmanuel Blot]

Well, you could try to disable all the plugins, at first.

You'll probably need to run trac-admin with gdb to see which call triggers the crash.

[gsmdib@…]

I've tried running trac-admin /path_to/trac resync with apache2 stopped, also removed plugins and eggs cache in /var/www/.python… neither worked. Running gdb $(which python) followed by run /usr/bin/trac-admin /path_to/trac resync returned

(no debugging symbols found)

Program received signal SIGSEGV, Segmentation fault.
---Type <return> to continue, or q <return> to quit---
[Switching to Thread 47407057438448 (LWP 12824)]
0x0000000000418f22 in ?? ()

Is there any more info I could provide?

Thanks.

[Christian Boos]

bt? (in order to get a full backtrace)

See TracTroubleshooting for more GDB advice.

But in the end, it will probably be a mismatch in the libraries used, so also to a info shared within gdb.

[anonymous]

Thanks for such a quick reply. I've tracked down the problem to postgres connection. E.g. trying to create a new environment with a postgres backend would cause the same segmentation fault. The existing trac instances work w/o probs though (e.g. i can create tickets, modify wiki, etc.) I use *python-pgsql* that comes with ubuntu feisty and it didn't cause any troubles in the past. Here is the output of bt:

#0  0x0000000000418f22 in ?? ()
#1  0x0000000000419a10 in PyObject_CallMethod ()
#2  0x00002b5a18c4f7e3 in microprotocol_getquoted () from /usr/lib/python2.5/site-packages/psycopg2/_psycopg.so
#3  0x00002b5a18c5129b in ?? () from /usr/lib/python2.5/site-packages/psycopg2/_psycopg.so
#4  0x00002b5a18c51fa8 in ?? () from /usr/lib/python2.5/site-packages/psycopg2/_psycopg.so
#5  0x00002b5a18c527ff in ?? () from /usr/lib/python2.5/site-packages/psycopg2/_psycopg.so
#6  0x00000000004885fd in PyEval_EvalFrameEx ()
#7  0x00000000004895da in PyEval_EvalCodeEx ()
#8  0x00000000004878e5 in PyEval_EvalFrameEx ()
#9  0x00000000004895da in PyEval_EvalCodeEx ()
#10 0x00000000004878e5 in PyEval_EvalFrameEx ()
#11 0x0000000000488951 in PyEval_EvalFrameEx ()
#12 0x00000000004895da in PyEval_EvalCodeEx ()
#13 0x00000000004d3ba8 in ?? ()
#14 0x0000000000417be3 in PyObject_Call ()
#15 0x000000000041e37d in ?? ()
#16 0x0000000000417be3 in PyObject_Call ()
#17 0x000000000045a018 in ?? ()
#18 0x000000000045cf12 in ?? ()
#19 0x0000000000417be3 in PyObject_Call ()
#20 0x0000000000484360 in PyEval_EvalFrameEx ()
#21 0x0000000000488951 in PyEval_EvalFrameEx ()
#22 0x0000000000488951 in PyEval_EvalFrameEx ()
#23 0x00000000004895da in PyEval_EvalCodeEx ()
#24 0x00000000004878e5 in PyEval_EvalFrameEx ()
#25 0x0000000000488951 in PyEval_EvalFrameEx ()
#26 0x00000000004895da in PyEval_EvalCodeEx ()
#27 0x00000000004d3ba8 in ?? ()
#28 0x0000000000417be3 in PyObject_Call ()
#29 0x000000000041e37d in ?? ()
#30 0x0000000000417be3 in PyObject_Call ()
#31 0x0000000000484360 in PyEval_EvalFrameEx ()
#32 0x0000000000488951 in PyEval_EvalFrameEx ()
#33 0x0000000000488951 in PyEval_EvalFrameEx ()
#34 0x00000000004895da in PyEval_EvalCodeEx ()
#35 0x0000000000489622 in PyEval_EvalCode ()
#36 0x00000000004aad1e in PyRun_FileExFlags ()
#37 0x00000000004aafb0 in PyRun_SimpleFileExFlags ()
#38 0x00000000004146b5 in Py_Main ()
#39 0x00002b5a0ed2b8e4 in __libc_start_main () from /lib/libc.so.6
#40 0x0000000000413bf9 in _start ()

TIA

[gsmdib@…]

(gdb) info shared
From                To                  Syms Read   Shared Object Library
0x00002b97f3e95a60  0x00002b97f3eaaff4  Yes         /lib64/ld-linux-x86-64.so.2
0x00002b97f40b7260  0x00002b97f40c1a18  Yes         /lib/libpthread.so.0
0x00002b97f42cde20  0x00002b97f42ce9e8  Yes         /lib/libdl.so.2
0x00002b97f44d1ea0  0x00002b97f44d2648  Yes         /lib/libutil.so.1
0x00002b97f46d7ed0  0x00002b97f4717578  Yes         /lib/libm.so.6
0x00002b97f49746d0  0x00002b97f4a5b6f0  Yes         /lib/libc.so.6
0x00002b97f4caa9b0  0x00002b97f4cad138  Yes         /usr/lib/python2.5/lib-dynload/strop.so
0x00002b97f4eb17b0  0x00002b97f4eb20c8  Yes         /usr/lib/python2.5/lib-dynload/termios.so
0x00002b97f50b7220  0x00002b97f50b9598  Yes         /usr/lib/python2.5/lib-dynload/collections.so
0x00002b97f52bda10  0x00002b97f52bea78  Yes         /usr/lib/python2.5/lib-dynload/cStringIO.so
0x00002b97f54c2c60  0x00002b97f54c3c28  Yes         /usr/lib/python2.5/lib-dynload/time.so
0x00002b97f56c96e0  0x00002b97f56cf388  Yes         /usr/lib/python2.5/lib-dynload/_socket.so
0x00002b97f58d6df0  0x00002b97f58d7e58  Yes         /usr/lib/python2.5/lib-dynload/_ssl.so
0x00002b97f5af0570  0x00002b97f5b166e8  Yes         /usr/lib/libssl.so.0.9.8
0x00002b97f5d8b630  0x00002b97f5e38838  Yes         /usr/lib/libcrypto.so.0.9.8
0x00002b97f60a3200  0x00002b97f60afa58  Yes         /usr/lib/libz.so.1
0x00002b97f62b8d60  0x00002b97f62ba258  Yes         /usr/lib/python2.5/lib-dynload/_locale.so
0x00002b97f64bfea0  0x00002b97f64c1518  Yes         /usr/lib/python2.5/lib-dynload/operator.so
0x00002b97f66c62e0  0x00002b97f66c6d68  Yes         /usr/lib/python2.5/lib-dynload/_hashlib.so
0x00002b97f68c99a0  0x00002b97f68ca858  Yes         /usr/lib/python2.5/lib-dynload/math.so
0x00002b97f6acd190  0x00002b97f6acebc8  Yes         /usr/lib/python2.5/lib-dynload/binascii.so
0x00002b97f6cd23c0  0x00002b97f6cd3278  Yes         /usr/lib/python2.5/lib-dynload/_random.so
0x00002b97f6ed4dd0  0x00002b97f6ed5d98  Yes         /usr/lib/python2.5/lib-dynload/fcntl.so
0x00002b97f719c4a0  0x00002b97f719f338  Yes         /usr/lib/python2.5/lib-dynload/itertools.so
0x00002b97f73a3be0  0x00002b97f73a41b8  Yes         /usr/lib/python2.5/lib-dynload/_bisect.so
0x00002b97f75a5d80  0x00002b97f75a6c58  Yes         /usr/lib/python2.5/lib-dynload/_heapq.so
0x00002b97f77acdc0  0x00002b97f77b91a8  Yes         /usr/lib/python2.5/lib-dynload/cPickle.so
0x00002b97f79bf450  0x00002b97f79c13c8  Yes         /usr/lib/python2.5/lib-dynload/unicodedata.so
0x00002b97f7c355b0  0x00002b97f7c37cd8  Yes         /usr/lib/python2.5/lib-dynload/_struct.so
0x00002b97f7e3c590  0x00002b97f7e3e3a8  Yes         /usr/lib/python2.5/lib-dynload/zlib.so
0x00002b97f8043ef0  0x00002b97f804b308  Yes         /usr/lib/python2.5/site-packages/_ldap.so
0x00002b97f8261630  0x00002b97f8283b48  Yes         /usr/lib/libldap_r.so.2
0x00002b97f84926a0  0x00002b97f8499b28  Yes         /usr/lib/liblber.so.2
0x00002b97f86a1cd0  0x00002b97f86b1028  Yes         /usr/lib/libsasl2.so.2
0x00002b97f88b9310  0x00002b97f88c48a8  Yes         /lib/libresolv.so.2
0x00002b97f8acb9b0  0x00002b97f8ace5a8  Yes         /lib/libcrypt.so.1
0x00002b97f8d1efe0  0x00002b97f8d63b48  Yes         /usr/lib/libgnutls.so.13
0x00002b97f8f7fa40  0x00002b97f8f908d8  Yes         /usr/lib/libtasn1.so.3
0x00002b97f919a600  0x00002b97f91cb598  Yes         /usr/lib/libgcrypt.so.11
0x00002b97f93e1880  0x00002b97f93e1d18  Yes         /usr/lib/libgpg-error.so.0
0x00002b97f95e7f90  0x00002b97f95f3a08  Yes         /lib/libnsl.so.1
0x00002b97f97fdc60  0x00002b97f9806f68  Yes         /usr/lib/python2.5/lib-dynload/parser.so
0x00002b97f9a0eae0  0x00002b97f9a18878  Yes         /usr/lib/python2.5/lib-dynload/datetime.so
0x00002b97f9c1f9a0  0x00002b97f9c1fc48  Yes         /usr/lib/python2.5/lib-dynload/_weakref.so
0x00002b97f9e28270  0x00002b97f9e2dfc8  Yes         /usr/lib/python2.5/site-packages/pysqlite2/_sqlite.so
0x00002b97fa0485a0  0x00002b97fa087b98  Yes         /usr/lib/libsqlite3.so.0
0x00002b97fa2a2500  0x00002b97fa2b6fc8  Yes         /var/lib/python-support/python2.5/libsvn/_fs.so
0x00002b97fa4c9120  0x00002b97fa4ce998  Yes         /usr/lib/libsvn_swig_py-1.so.1
0x00002b97fa6d3960  0x00002b97fa6d4fa8  Yes         /usr/lib/libsvn_fs-1.so.1
0x00002b97fa8e3ec0  0x00002b97fa8fd0e8  Yes         /usr/lib/libsvn_subr-1.so.1
0x00002b97fab14650  0x00002b97fab2fcc8  Yes         /usr/lib/libsvn_client-1.so.1
0x00002b97fad38140  0x00002b97fad3dc18  Yes         /usr/lib/libsvn_delta-1.so.1
0x00002b97faf49010  0x00002b97faf6ba38  Yes         /usr/lib/libsvn_wc-1.so.1
0x00002b97fb17ed10  0x00002b97fb1936d8  Yes         /usr/lib/libapr-1.so.0
0x00002b97fb3a1d70  0x00002b97fb3b2278  Yes         /usr/lib/libsvn_fs_fs-1.so.1
0x00002b97fb5c1380  0x00002b97fb5d9608  Yes         /usr/lib/libsvn_fs_base-1.so.1
0x00002b97fb7ead80  0x00002b97fb7fa688  Yes         /usr/lib/libaprutil-1.so.0
0x00002b97fba019d0  0x00002b97fba027e8  Yes         /usr/lib/libsvn_ra-1.so.1
0x00002b97fbc05ca0  0x00002b97fbc09b88  Yes         /usr/lib/libsvn_diff-1.so.1
0x00002b97fbe0bf70  0x00002b97fbe0cb38  Yes         /lib/libuuid.so.1
0x00002b97fc010280  0x00002b97fc013f98  Yes         /lib/librt.so.1
0x00002b97fc23b250  0x00002b97fc2fb2c8  Yes         /usr/lib/libdb-4.4.so
0x00002b97fc529620  0x00002b97fc53af88  Yes         /usr/lib/libpq.so.5
0x00002b97fc746d70  0x00002b97fc75c828  Yes         /usr/lib/libexpat.so.1
0x00002b97fc9688e0  0x00002b97fc96ae88  Yes         /usr/lib/libsvn_ra_local-1.so.1
0x00002b97fcb732e0  0x00002b97fcb83708  Yes         /usr/lib/libsvn_repos-1.so.1
0x00002b97fcd91fd0  0x00002b97fcd9d7b8  Yes         /usr/lib/libsvn_ra_svn-1.so.1
0x00002b97fcfa9b80  0x00002b97fcfb8468  Yes         /usr/lib/libsvn_ra_dav-1.so.1
0x00002b97fd1d6da0  0x00002b97fd230808  Yes         /usr/lib/libkrb5.so.3
0x00002b97fd445bf0  0x00002b97fd4462a8  Yes         /lib/libcom_err.so.2
0x00002b97fd650ae0  0x00002b97fd661078  Yes         /usr/lib/libneon.so.26
0x00002b97fd86e6a0  0x00002b97fd8816b8  Yes         /usr/lib/libk5crypto.so.3
0x00002b97fda92e60  0x00002b97fdaa5548  Yes         /usr/lib/libgssapi_krb5.so.2
0x00002b97fdcaad40  0x00002b97fdcac6a8  Yes         /usr/lib/libkrb5support.so.0
0x00002b97fdedbed0  0x00002b97fdfacde8  Yes         /usr/lib/libxml2.so.2
0x00002b97fe1fd540  0x00002b97fe223718  Yes         /var/lib/python-support/python2.5/libsvn/_core.so
0x00002b97fe440890  0x00002b97fe44e398  Yes         /var/lib/python-support/python2.5/libsvn/_delta.so
0x00002b97fe662210  0x00002b97fe67ceb8  Yes         /var/lib/python-support/python2.5/libsvn/_repos.so
0x00002b97fe893af0  0x00002b97fe89e358  Yes         /usr/lib/python2.5/site-packages/psycopg2/_psycopg.so
0x00002b97feaaa8a0  0x00002b97feaafe78  Yes         /usr/lib/python2.5/site-packages/mx/DateTime/mxDateTime/mxDateTime.so
0x00002b97fecbb050  0x00002b97fecc16b8  Yes         /lib/libnss_files.so.2
0x00002b97feec5380  0x00002b97feeca2e8  Yes         /lib/libnss_compat.so.2
0x00002b97ff0cf030  0x00002b97ff0d4cb8  Yes         /lib/libnss_nis.so.2

[Christian Boos]

Well, apparently psycopg2 is also installed, and this gets used first (/usr/lib/python2.5/site-packages/psycopg2/_psycopg.so). If possible, disable temporarily psycopg2 and try again, it should then pick up pgsql.

Another possibility would be to upgrade psycopg2. Maybe the one you're using was built against the wrong version of postgresql.

[gsmdib@…]

Thanks, I purged python-psycopg2 and now trac-admin works like charm. Could you please advice whether python-psycopg2 or python-pgsql would suit Trac best? Many thanks for helping sort the issue out!

[Christian Boos]

Well normally both should work. I think jonas is using psycopg2 here on t.e.o, and for my testings on Windows (admittedly superficials), I use both of them and never saw a difference. Perhaps you can find more details in DatabaseBackend.

[Christian Boos]

This was actually a bug with psycopg 2.0.5 and python 2.5 on 64-bit systems, see ​https://bugs.launchpad.net/ubuntu/+source/psycopg2/+bug/108067 and psycopg:#167.

Worked for me with psycopg 2.0.6.