_libpq_pathname in postgres_backend.py should be a str instance

[Jun Omae]

Raised from ​https://github.com/edgewall/trac/pull/24.

ctypes does not handle bytes, but strings. Currently, opening the library just fails with cause ctypes immediately calls libname.endwith("…") on it, which fails if libname isn't a string.

trac/db/postgres_backend.py

@@ -62 +62 @@
                         '''.encode('utf-8'),
                         _f.read(), re.VERBOSE)
                     if _match:
-                        _libpq_pathname = _match.group(1)
+                        _libpq_pathname = _match.group(1).decode('utf-8')
                 else:
                     if re.search(r'\0libpq\.dll\0'.encode('utf-8'), _f.read(),
                                  re.IGNORECASE):

[Jun Omae]

Hmm, the issue is not occurred in my environment for PostgreSQL.

Python 3.11.10 (main, Sep  7 2024, 18:35:42) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> from trac.db.postgres_backend import _libpq_pathname
>>> _libpq_pathname
b'libpq.so.5'
>>> lib = ctypes.CDLL(_libpq_pathname)
>>> lib.PQlibVersion()
120020

According to ​https://github.com/psycopg/psycopg2/blob/2.9.9/NEWS#L398-L399, psycopg2.__libpq_version__ and psycopg2.extensions.libpq_version() have been added to retrieve the version of libpq in psycopg2 2.7. I'm trying to check the libpq_version() method first.

>>> import psycopg2
>>> psycopg2
<module 'psycopg2' from '/home/jun66j5/venv/py311/lib/python3.11/site-packages/psycopg2/__init__.py'>
>>> psycopg2.libpq_version
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'psycopg2' has no attribute 'libpq_version'. Did you mean: '__libpq_version__'?
>>> psycopg2.__libpq_version__
120017
>>> import psycopg2.extensions
>>> psycopg2.extensions.libpq_version()
120020

[BtbN]

This is only an issue with Python 3.13 and up, since the versions before did not have this check: ​https://github.com/python/cpython/commit/408e127159e54d87bb3464fd8bd60219dc527fac

But yeah, the whole version check is a bit dubious for sure. Why does it regexp a libpq.so.5 string out of the library that's bundled with psycopg2-binary, to then try and dlopen() that, which in turn just fails again with a File Not Found error.

Shouldn't it instead just pass psycopg._psycopg.__file__ to CDLL directly? A quick try-out of that seems to work fine at least.

[Jun Omae]

Replying to BtbN:

This is only an issue with Python 3.13 and up, since the versions before did not have this check: ​https://github.com/python/cpython/commit/408e127159e54d87bb3464fd8bd60219dc527fac

Python 3.13.0 (main, Oct  8 2024, 08:51:27) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from trac.db.postgres_backend import _libpq_pathname
>>> _libpq_pathname
b'libpq.so.5'
>>> import ctypes
>>> lib = ctypes.CDLL(_libpq_pathname)
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    lib = ctypes.CDLL(_libpq_pathname)
  File "/usr/lib/python3.13/ctypes/__init__.py", line 353, in __init__
    if name.endswith(".fwork"):
       ~~~~~~~~~~~~~^^^^^^^^^^
TypeError: endswith first arg must be bytes or a tuple of bytes, not str
>>>

I think the issue when passing a bytes instance to ctypes.CDLL is an issue of Python 3.13.0.

But yeah, the whole version check is a bit dubious for sure. Why does it regexp a libpq.so.5 string out of the library that's bundled with psycopg2-binary, to then try and dlopen() that, which in turn just fails again with a File Not Found error.

Shouldn't it instead just pass psycopg._psycopg.__file__ to CDLL directly? A quick try-out of that seems to work fine at least.

psycopg._psycopg.__file__ is _psycopg.so, not libpq. The code is added in [15257] (#12622), in order to detect _psycopg.so do dynamic linked or static linked with libpq module.

[Jun Omae]

Committed BtbN's patch in [17861]. Thanks, BtbN!

Also, used psycopg2.extensions.libpq_version() in [17862] and merged two changes to trunk in [17863].