AttributeError: 'Pool' object has no attribute '_pool'

[fbrettschneider@…]

The following happens every then and now. CPU load of this process is 100% for about a minute then. It's very annoying if Trac "hangs" in this situation. The hardware is quite fast.

How to Reproduce

While doing a GET operation on /query, Trac issued an internal error.

(please provide additional details here)

Request parameters:

{'col': [u'id',
         u'summary',
         u'owner',
         u'type',
         u'priority',
         u'totalhours',
         u'remaininghours',
         u'time',
         u'changetime',
         u'severity'],
 'desc': u'1',
 'group': u'resolution',
 'milestone': u'XX XXXXXXX - X2',
 'order': u'time',
 'status': u'closed'}

User agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0

System Information

Trac	1.0.1
Trac	1.0.1
Babel	0.9.6
Bitten	0.7dev-r1008
FullBlog	0.1.1-r12111
Genshi	0.6 (without speedups)
GIT	1.7.11.msysgit.1
GIT	1.7.11.msysgit.1
mod_wsgi	3.3 (WSGIProcessGroup WSGIApplicationGroup %{GLOBAL})
pysqlite	2.6.3
Python	2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)]
Python	2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)]
RPC	1.1.0
setuptools	0.6c11
setuptools	0.6c11
SQLite	3.7.6.2
Subversion	1.7.5 (r1336830)
jQuery	1.7.2

Enabled Plugins

advancedticketworkflowplugin	0.11dev
autocompleteusers	0.4.2dev-r11757
bitten	0.7dev-r1008
ColorMacro	Rev
componenthierarchy	0.0.1
estimationtools	0.4.5
graphviz	0.7.5
groupticketfields	0.0.1
pdfimg	0.0.3
simplemultiproject	0.0.4dev
table-sorter-plugin	1.0
ticketimport	0.8.3
ticketvalidator.admin	N/A
ticketvalidator.core	N/A
timingandestimationplugin	1.3.7b
trac-jsgantt	0.10
tracacronyms	0.2dev
tracannouncer	1.0dev
tracforms	0.5dev-r11562
tracfullblogplugin	0.1.1-r12111
tracincludemacro	3.0.0dev-r12030
tracmastertickets	3.0.5dev
tracmath	0.5
tracmenusplugin	0.1.1-r12147
tracnumberedheadlinesplugin	0.4
tracpoll	0.3.0dev-r11670
tracsubticketsplugin	0.2.0.dev-20130613
tracticketstatusgraph	1.0
tractopmacro	0.11.1
tracusermanagerplugin	0.4
tracvisualization	0.0.1
tracwikigoodies	0.11.0.3dev
tracwysiwyg	0.12.0.4-r11158
tracxmlrpc	1.1.0
translatedpagesmacro	0.3
upcomingmilestoneschartmacro	N/A
wikitablemacro	0.1-r10997

Python Traceback

Traceback (most recent call last):
  File "build\bdist.win32\egg\trac\web\main.py", line 497, in _dispatch_request
    dispatcher.dispatch(req)
  File "build\bdist.win32\egg\trac\web\main.py", line 179, in dispatch
    chosen_handler)
  File "build\bdist.win32\egg\trac\web\main.py", line 325, in _pre_process_request
    chosen_handler = filter_.pre_process_request(req, chosen_handler)
  File "build\bdist.win32\egg\trac\versioncontrol\api.py", line 343, in pre_process_request
    repo = self.get_repository(reponame)
  File "build\bdist.win32\egg\trac\versioncontrol\api.py", line 551, in get_repository
    connector = self._get_connector(rtype)
  File "build\bdist.win32\egg\trac\versioncontrol\api.py", line 695, in _get_connector
    for connector in self.connectors:
  File "build\bdist.win32\egg\trac\core.py", line 78, in extensions
    components = [component.compmgr[cls] for cls in classes]
  File "build\bdist.win32\egg\trac\core.py", line 199, in __getitem__
    component = cls(self)
  File "build\bdist.win32\egg\trac\core.py", line 138, in __call__
    self.__init__()
  File "build\bdist.win32\egg\tracopt\versioncontrol\svn\svn_fs.py", line 285, in __init__
    Pool()
  File "build\bdist.win32\egg\tracopt\versioncontrol\svn\svn_fs.py", line 157, in __init__
    self._pool = core.svn_pool_create(self._parent_pool())
  File "build\bdist.win32\egg\tracopt\versioncontrol\svn\svn_fs.py", line 169, in __call__
    return self._pool
AttributeError: 'Pool' object has no attribute '_pool'

[Jun Omae]

The following happens every then and now. CPU load of this process is 100% for about a minute then. It's very annoying if Trac "hangs" in this situation. The hardware is quite fast.

That is a InstallationIssue. If your repository is large, the response would be slow. So, I recommend explicit synchronization instead of per-request synchronization. See TracRepositoryAdmin#ExplicitSync.

AttributeError: 'Pool' object has no attribute '_pool'

It seems that svn_fs module has race condition on multithread. It can be reproduced with the following script.

import thread, time, sys
from threading import Thread, Event
from tracopt.versioncontrol.svn import svn_fs
from tracopt.versioncontrol.svn.svn_fs import Pool

svn_fs._import_svn()
pools = []
event = Event()

def new_pool():
    try:
        event.wait()
        pools.append(Pool())
    except Exception, e:
        print e

threads = []
for idx in xrange(100 if len(sys.argv) == 1 else len(sys.argv[1])):
    t = Thread(target=new_pool)
    t.start()
    threads.append(t)

time.sleep(1)
event.set()
for t in threads:
    t.join()

$ PYTHONPATH=. ~/venv/py25/bin/python ./svn-pool.py 20
'Pool' object has no attribute '_pool'

The following patch would be fixed.

tracopt/versioncontrol/svn/svn_fs.py

@@ -165 +165 @@
             # then initialize APR and set this pool
             # to be the application-level pool
             core.apr_initialize()
+            self._pool = core.svn_pool_create(None)
             application_pool = self
 
-            self._pool = core.svn_pool_create(None)
         self._mark_valid()
 
     def __call__(self):

[fbrettschneider@…]

I do use the trac-admin command in the post-commit hook. But I need to check if I cleared the [trac] repository_sync_per_request option… if not, would that cause such race condition? Also some Bitten slaves periodically may check if there's something to do on SVN commits.

[fbrettschneider@…]

Replying to jomae:

The following patch would be fixed.

+            self._pool = core.svn_pool_create(None)
             application_pool = self

-            self._pool = core.svn_pool_create(None)

Sounds great! (y)

[Jun Omae]

Replying to fbrettschneider@…:

I do use the trac-admin command in the post-commit hook. But I need to check if I cleared the [trac] repository_sync_per_request option…

Yeah, please confirm the option is empty. Your stacktrace indicates the option is not empty.

  File "build\bdist.win32\egg\trac\versioncontrol\api.py", line 343, in pre_process_request
    repo = self.get_repository(reponame)

And also tags/trac-1.0.1/trac/versioncontrol/api.py@:338,343#L335​.

[fbrettschneider@…]

no, there's no such entry in my [trac] section.

[anonymous]

is the default value not empty?

[Jun Omae]

no, there's no such entry in my [trac] section.

The default of repository_sync_per_request option is (default). To disable per-request sync need to add explicitly empty entry like this. You can check current settings using About Trac page and TracIni about default values.

[trac]
repository_sync_per_request =

[anonymous]

can you the long hang technically explain, please?

[fbrettschneider@…]

Replying to jomae:

It seems that svn_fs module has race condition on multithread. It can be reproduced with the following script.

…

}}} The following patch would be fixed.

tracopt/versioncontrol/svn/svn_fs.py

@@ -165 +165 @@
             # then initialize APR and set this pool
             # to be the application-level pool
             core.apr_initialize()
+            self._pool = core.svn_pool_create(None)
             application_pool = self
 
-            self._pool = core.svn_pool_create(None)
         self._mark_valid()
 
     def __call__(self):

Thanks. Please, add it to the Trac-1.0.2 release.

[Jun Omae]

I've revised the patch in comment:1. Proposed changes can be found in log:jomae.git:t11411_0.12.6dev and log:jomae.git:t11411 for 1.0-stable.

We should use a lock object to get/set the application_pool variable to prevent the race-conditions.

[anonymous]

I changed my settings and explicitely set an empty entry, and this has helped. I don't see that error anymore in the logging. — falkb

[Jun Omae]

Committed to 0.12-stable in [12436] and merged to 1.0-stable and trunk in [12437,12438].