Unit tests failing when psycopg2 2.8.x due to raising `UniqueViolation` instead of `IntegrityError`

[Jun Omae]

​https://travis-ci.org/edgewall/trac/jobs/521140057

======================================================================
FAIL: test_component_add_error_already_exists (trac.admin.tests.console.TracadminTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/edgewall/trac/trac/admin/tests/console.py", line 541, in test_component_add_error_already_exists
    self.assertExpectedResult(output)
  File "/home/travis/build/edgewall/trac/trac/admin/tests/console.py", line 172, in assertExpectedResult
    self.assertEqual(expected_result, output)
  File "/home/travis/build/edgewall/trac/trac/admin/tests/console.py", line 194, in assertEqual
    output, diff()))
AssertionError: u'IntegrityError: [...]\n' != u'UniqueViolation: duplicate key value violates unique constraint "component_pkey"\nDETAIL:  Key (name)=(component1) already exists.\n\n'
--- expected
+++ actual
@@ -1,2 +1,4 @@
-IntegrityError: [...]
+UniqueViolation: duplicate key value violates unique constraint "component_pkey"
+DETAIL:  Key (name)=(component1) already exists.
 
+
...

----------------------------------------------------------------------
Ran 1958 tests in 132.317s

FAILED (failures=10)
make: *** [unit-test] Error 1

[Jun Omae]

The UniqueViolation exception is inherited from IntegrityError and not exported.

>>> import psycopg2
>>> psycopg2.__version__
'2.8.2 (dt dec pq3 ext lo64)'
>>> from psycopg2 import UniqueViolation
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name UniqueViolation
>>> from psycopg2._psycopg import sqlstate_errors
>>> sqlstate_errors['23505']
<class 'psycopg2.errors.UniqueViolation'>
>>> sqlstate_errors['23505'].__bases__
(<class 'psycopg2.IntegrityError'>,)
>>> issubclass(sqlstate_errors['23505'], psycopg2.IntegrityError)
True

[Jun Omae]

Ad hoc patch is pinning psycopg2<2.8.

.travis.yml

@@ -103 +103 @@
       2.6) requires="$requires Babel<2.6.0 lxml<4.3.0" ;;
       *)   requires="$requires Babel lxml" ;;
     esac
-    if [ "$tracdb" = postgres ]; then requires="$requires psycopg2"; fi
+    if [ "$tracdb" = postgres ]; then requires="$requires psycopg2<2.8"; fi
     if [ "$tracdb" = mysql ]; then requires="$requires MySQL-python"; fi
     pip install $requires
     python -c 'import sys, pkg_resources as p; p.require(sys.argv[1:])' $requires

contrib/appveyor.ps1

@@ -254 +254 @@
         # "postgres://tracuser:password@localhost/trac?schema=tractest"
         #
 
-        & pip install psycopg2
+        & pip install psycopg2<2.8
 
         Add-AppveyorMessage -Message "1.1. psycopg2 package installed" `
           -Category Information

[Ryan J Ollos]

The workaround looks good to get our build passing again.

Once that is committed I'll do some testing on a branch with Ubuntu Xenial, which is being ​rolled out.

[Jun Omae]

I noticed the unit tests on trunk pass with psycopg2 2.8.2 at ​https://travis-ci.org/edgewall/trac/jobs/521143761#L708. The tests which try to insert non-unique ticket property have been removed in r15380….

[Jun Omae]

Replying to Jun Omae:

I noticed the unit tests on trunk pass with psycopg2 2.8.2 at ​https://travis-ci.org/edgewall/trac/jobs/521143761#L708. The tests which try to insert non-unique ticket property have been removed in r15380….

ResourceExistsError is raised on trunk in the case and the tests exist in trac/ticket/tests/console-tests.txt.

[Ryan J Ollos]

I committed the comment:2 workaround in r16906, r16907, record-only merge in r16908.

[Ryan J Ollos]

We could backport [15774-15775] from #11419 and raise ResourceExistsError. I'd prefer to not spend time on that, and it will result in an API change on 1.0-stable and 1.2-stable.

UniqueViolation inherits from IntegrityError, so any plugins trapping IntegrityError should continue to work fine provided we don't make an API change.

>>> import psycopg2
>>> psycopg2.__version__
'2.8.3 (dt dec pq3 ext lo64)'
>>> psycopg2.errors.UniqueViolation.__bases__
(<class 'psycopg2.IntegrityError'>,)

Alternatively we could just put a workaround in place so that the tests pass.

trac/admin/tests/console.py

@@ -184 +184 @@
             return ''.join(difflib.unified_diff(expected_lines, output_lines,
                                                 'expected', 'actual'))
 
+        output = re.sub('^(UniqueViolation:)', 'IntegrityError:', output)
         if '[...]' in expected_results:
             m = re.match('.*'.join(map(re.escape,
                                        expected_results.split('[...]'))) +

Thoughts?

[Jun Omae]

Alternatively we could just put a workaround in place so that the tests pass.

+1. I think that is the second best solution, but enough to work around it.

[Ryan J Ollos]

Thanks for reviewing. Committed to 1.0-stable in r17076, merged in r17077. Record-only merge on trunk in r17078.