Add support for pymysql library

[Ryan J Ollos]

Changes from #12130 propose to add support for the pymysql library. Tests are passing for me, so I'd suggest we add the changes soon. I'll post proposed changes shortly, cherry-picked from #12130.

[Ryan J Ollos]

Proposed changes in [4bacb9a74/rjollos.git].

DONE Add Travis CI and AppVeyor build configurations that use pymysql.

[Ryan J Ollos]

I omitted something important. Tests pass with the following change:

setup.py

@@ -95 +95 @@
     extras_require = {
         'genshi': ['Genshi>=0.6'],
         'babel': ['Babel>=0.9.5'],
-        'mysql': ['MySQL-python >= 1.2.2'],
+        'mysql': ['PyMySQL'],
         'postgresql': ['psycopg2 >= 2.0'],
         'pygments': ['Pygments>=1.0'],
         'rest': ['docutils>=0.3.9'],

I haven't looked closely if or how we can specify "MySQL-python or PyMySQL" for the installation requirement in setup.py.

Also, looks like signature of _show_warnings changed in ​PyMySQL 0.7.6, and latest package has a strange version string:

>>> import pymysql as MySQLdb
>>> MySQLdb.__version__
'0.7.11.None'

Additional changes in [42f422aeb/rjollos.git].

[Peter Suter]

how we can specify "MySQL-python or PyMySQL" for the installation requirement in setup.py.

According to ​this (​and this) it seems something like this would be needed:

setup.py

@@ -41 +41 @@
     print("Jinja2 is needed by Trac setup, pre-installing")
     # give some context to the warnings we might get when installing Jinja2
 
+try:
+    import pymysql 
+    mysql = 'PyMySQL'
+except ImportError:
+    mysql = 'MySQL-python >= 1.2.2'
+
 
 setup(
     name = 'Trac',
@@ -95 +101 @@
     extras_require = {
         'genshi': ['Genshi>=0.6'],
         'babel': ['Babel>=0.9.5'],
-        'mysql': ['MySQL-python >= 1.2.2'],
+        'mysql': [mysql],
         'postgresql': ['psycopg2 >= 2.0'],
         'pygments': ['Pygments>=1.0'],
         'rest': ['docutils>=0.3.9'],

[Jun Omae]

It might be good to support mysql+mysqldb://... for MySQL-python and mysql+pymysql://... for PyMySQL, like ​SQLAlchemy.

Another consideration, how about switching to ​PyMySQL from ​MySQL-python? The original MySQL-python doesn't support Python 3 but ​https://github.com/PyMySQL/mysqlclient-python supports Python 3. No new commits since 2014 on ​https://github.com/farcepest/MySQLdb1.

[Ryan J Ollos]

I like the idea of supporting just one MySQL library.

I started to update the compatible distros matrix. pymysql is available on Ubuntu 16.04. I don't see it in Debian 8.8, and looks like it may still be in ​unstable. Does anyone think that matters since a user can just pip install it?

[Jun Omae]

Proposed additional changes in [4f56ad5b2/jomae.git]. I'd like to add charset parameter to database connection string for MySQL, to avoid reconnecting when database charset is not matched with client charset.

Yeah, no matters. Even ​Trac 1.2 on Debian is still sid.

[Ryan J Ollos]

Replying to Ryan J Ollos:

Also, looks like signature of _show_warnings changed in ​PyMySQL 0.7.6, and latest package has a strange version string:

Created a ​pull request for the issue.

[Ryan J Ollos]

Latest changes, including comment:6 changes, in log:rjollos.git:t12821_pymysql.1.

Tests pass on OSX with the following versions of pymysql:

• 0.7.11
• 0.6.7
• 0.6

• DONE Update 1.3/TracInstall (1.3/TracInstall@7)
• DONE Update 1.3/TracUpgrade (1.3/TracUpgrade@15)

[Ryan J Ollos]

Replying to Jun Omae:

Proposed additional changes in [4f56ad5b2/jomae.git]. I'd like to add charset parameter to database connection string for MySQL, to avoid reconnecting when database charset is not matched with client charset.

I might have overlooked an import change. Should the AppVeyor config be modified?:

.appveyor.yml

@@ -21 +21 @@
     TRAC_TEST_DB_URI: sqlite:test.db
   - SVN_BRANCH: trunk
     PYTHONHOME: C:\Miniconda
-    TRAC_TEST_DB_URI: mysql://tracuser:password@localhost/trac
+    TRAC_TEST_DB_URI: mysql://tracuser:password@localhost/trac?charset=utf8mb4
   - SVN_BRANCH: trunk
     PYTHONHOME: C:\Miniconda-x64
     TRAC_TEST_DB_URI: postgres://tracuser:password@localhost/trac?schema=tractest

[Ryan J Ollos]

Committed to trunk in r15965.

• Edited TracDev/ApiChanges/1.3@27
• Edited MySqlDb@76

[Ryan J Ollos]

Added charset to MySQL database connection string in r15966.

[Jun Omae]

I noticed Django recommends ​mysqlclient on Python 3 at ​https://docs.djangoproject.com/en/1.11/ref/databases/#mysql-db-api-drivers.

[Ryan J Ollos]

Replying to Jun Omae:

I noticed Django recommends ​mysqlclient on Python 3 at ​https://docs.djangoproject.com/en/1.11/ref/databases/#mysql-db-api-drivers.

It would be interesting to see if mysqlclient serves as a drop-in replacement for MySQLdb, as stated on the Django page. The issue has been raised in ​gdiscussion:trac-users:ET632_nYAc0.

[Ryan J Ollos]

Replying to Ryan J Ollos:

It would be interesting to see if mysqlclient serves as a drop-in replacement for MySQLdb, as stated on the Django page. The issue has been raised in ​gdiscussion:trac-users:ET632_nYAc0.

⇒ #12952.