Unit test execution is slow with real SQLite database

[Ryan J Ollos]

First mentioned in comment:20:ticket:11512, here is what I get on 1.0-stable

$ make unit-testPython version: Python 2.7.4
figleaf: 
coverage: 
PYTHONPATH=.:
TRAC_TEST_DB_URI=
server-options= -p 8000 -a '*,~/tracenvs/htdigest.realm,realm' -r -e ~/tracenvs
python setup.py egg_info
running egg_info
writing requirements to Trac.egg-info/requires.txt
writing Trac.egg-info/PKG-INFO
writing top-level names to Trac.egg-info/top_level.txt
writing dependency_links to Trac.egg-info/dependency_links.txt
writing entry points to Trac.egg-info/entry_points.txt
reading manifest file 'Trac.egg-info/SOURCES.txt'
writing manifest file 'Trac.egg-info/SOURCES.txt'
python ./trac/test.py --skip-functional-tests 
...
Ran 1545 tests in 37.987s

OK
$ make db=sqlite unit-test
Python version: Python 2.7.4
figleaf: 
coverage: 
PYTHONPATH=.:
TRAC_TEST_DB_URI=sqlite:test.db
server-options= -p 8000 -a '*,~/tracenvs/htdigest.realm,realm' -r -e ~/tracenvs
python setup.py egg_info
running egg_info
writing requirements to Trac.egg-info/requires.txt
writing Trac.egg-info/PKG-INFO
writing top-level names to Trac.egg-info/top_level.txt
writing dependency_links to Trac.egg-info/dependency_links.txt
writing entry points to Trac.egg-info/entry_points.txt
reading manifest file 'Trac.egg-info/SOURCES.txt'
writing manifest file 'Trac.egg-info/SOURCES.txt'
python ./trac/test.py --skip-functional-tests 
...
Ran 1545 tests in 82.705s

OK

The results seem to be the same on 0.12-stable with and without db=sqlite.

[Ryan J Ollos]

On my system I consistently see 27 seconds for in-memory SQLite, 60 seconds with SQLite before change, < 30 seconds with SQLite after change. The execution times in comment:description are a little high since my system was doing other things at the same time.

I guess the setting is not persistent across database connections since the following change did not affect the execution time:

trac/test.py

@@ -367 +367 @@
 
         if not tables:
             self.global_databasemanager.init_db()
+            if scheme == 'sqlite':
+                # Speed-up tests with SQLite database
+                self.db_transaction("PRAGMA synchronous = OFF")
             # we need to make sure the next get_db_cnx() will re-create
             # a new connection aware of the new data model - see #8518.
             if self.dburi != 'sqlite::memory:':

I'll commit the changes in log:rjollos.git:t11592 to 1.0-stable if there is no additional feedback.

[Jun Omae]

Results of timing on my system.

SQLite database	Elapse
in-memory	21.978s
file on tmpfs	20.745s
file on ext4 with the changes	22.692s
file on ext4 without the changes	800.592s

[Ryan J Ollos]

Committed to 1.0-stable in [12691], merged to trunk in [12692].

[Ryan J Ollos]

Changes from [12691] revised in #11967 ([13818]).