Edgewall Software
Modify

Opened 5 years ago

Closed 5 years ago

#11411 closed defect (fixed)

AttributeError: 'Pool' object has no attribute '_pool'

Reported by: fbrettschneider@… Owned by: Jun Omae
Priority: normal Milestone: 0.12.6
Component: version control Version: 1.0.1
Severity: normal Keywords: svn
Cc:
Release Notes:

Prevent 'Pool' object has no attribute '_pool' caused by race-conditions during creating application-level pool.

API Changes:

Description

The following happens every then and now. CPU load of this process is 100% for about a minute then. It's very annoying if Trac "hangs" in this situation. The hardware is quite fast.

How to Reproduce

While doing a GET operation on /query, Trac issued an internal error.

(please provide additional details here)

Request parameters:

{'col': [u'id',
         u'summary',
         u'owner',
         u'type',
         u'priority',
         u'totalhours',
         u'remaininghours',
         u'time',
         u'changetime',
         u'severity'],
 'desc': u'1',
 'group': u'resolution',
 'milestone': u'XX XXXXXXX - X2',
 'order': u'time',
 'status': u'closed'}

User agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0

System Information

Trac 1.0.1
Trac 1.0.1
Babel 0.9.6
Bitten 0.7dev-r1008
FullBlog 0.1.1-r12111
Genshi 0.6 (without speedups)
GIT 1.7.11.msysgit.1
GIT 1.7.11.msysgit.1
mod_wsgi 3.3 (WSGIProcessGroup WSGIApplicationGroup %{GLOBAL})
pysqlite 2.6.3
Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)]
Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)]
RPC 1.1.0
setuptools 0.6c11
setuptools 0.6c11
SQLite 3.7.6.2
Subversion 1.7.5 (r1336830)
jQuery 1.7.2

Enabled Plugins

advancedticketworkflowplugin 0.11dev
autocompleteusers 0.4.2dev-r11757
bitten 0.7dev-r1008
ColorMacro Rev
componenthierarchy 0.0.1
estimationtools 0.4.5
graphviz 0.7.5
groupticketfields 0.0.1
pdfimg 0.0.3
simplemultiproject 0.0.4dev
table-sorter-plugin 1.0
ticketimport 0.8.3
ticketvalidator.admin N/A
ticketvalidator.core N/A
timingandestimationplugin 1.3.7b
trac-jsgantt 0.10
tracacronyms 0.2dev
tracannouncer 1.0dev
tracforms 0.5dev-r11562
tracfullblogplugin 0.1.1-r12111
tracincludemacro 3.0.0dev-r12030
tracmastertickets 3.0.5dev
tracmath 0.5
tracmenusplugin 0.1.1-r12147
tracnumberedheadlinesplugin 0.4
tracpoll 0.3.0dev-r11670
tracsubticketsplugin 0.2.0.dev-20130613
tracticketstatusgraph 1.0
tractopmacro 0.11.1
tracusermanagerplugin 0.4
tracvisualization 0.0.1
tracwikigoodies 0.11.0.3dev
tracwysiwyg 0.12.0.4-r11158
tracxmlrpc 1.1.0
translatedpagesmacro 0.3
upcomingmilestoneschartmacro N/A
wikitablemacro 0.1-r10997

Python Traceback

Traceback (most recent call last):
  File "build\bdist.win32\egg\trac\web\main.py", line 497, in _dispatch_request
    dispatcher.dispatch(req)
  File "build\bdist.win32\egg\trac\web\main.py", line 179, in dispatch
    chosen_handler)
  File "build\bdist.win32\egg\trac\web\main.py", line 325, in _pre_process_request
    chosen_handler = filter_.pre_process_request(req, chosen_handler)
  File "build\bdist.win32\egg\trac\versioncontrol\api.py", line 343, in pre_process_request
    repo = self.get_repository(reponame)
  File "build\bdist.win32\egg\trac\versioncontrol\api.py", line 551, in get_repository
    connector = self._get_connector(rtype)
  File "build\bdist.win32\egg\trac\versioncontrol\api.py", line 695, in _get_connector
    for connector in self.connectors:
  File "build\bdist.win32\egg\trac\core.py", line 78, in extensions
    components = [component.compmgr[cls] for cls in classes]
  File "build\bdist.win32\egg\trac\core.py", line 199, in __getitem__
    component = cls(self)
  File "build\bdist.win32\egg\trac\core.py", line 138, in __call__
    self.__init__()
  File "build\bdist.win32\egg\tracopt\versioncontrol\svn\svn_fs.py", line 285, in __init__
    Pool()
  File "build\bdist.win32\egg\tracopt\versioncontrol\svn\svn_fs.py", line 157, in __init__
    self._pool = core.svn_pool_create(self._parent_pool())
  File "build\bdist.win32\egg\tracopt\versioncontrol\svn\svn_fs.py", line 169, in __call__
    return self._pool
AttributeError: 'Pool' object has no attribute '_pool'

Attachments (0)

Change History (12)

comment:1 in reply to:  description ; Changed 5 years ago by Jun Omae

Component: generalversion control
Keywords: svn added
Milestone: next-stable-1.0.x

The following happens every then and now. CPU load of this process is 100% for about a minute then. It's very annoying if Trac "hangs" in this situation. The hardware is quite fast.

That is a InstallationIssue. If your repository is large, the response would be slow. So, I recommend explicit synchronization instead of per-request synchronization. See TracRepositoryAdmin#ExplicitSync.

AttributeError: 'Pool' object has no attribute '_pool'

It seems that svn_fs module has race condition on multithread. It can be reproduced with the following script.

import thread, time, sys
from threading import Thread, Event
from tracopt.versioncontrol.svn import svn_fs
from tracopt.versioncontrol.svn.svn_fs import Pool

svn_fs._import_svn()
pools = []
event = Event()

def new_pool():
    try:
        event.wait()
        pools.append(Pool())
    except Exception, e:
        print e

threads = []
for idx in xrange(100 if len(sys.argv) == 1 else len(sys.argv[1])):
    t = Thread(target=new_pool)
    t.start()
    threads.append(t)

time.sleep(1)
event.set()
for t in threads:
    t.join()
$ PYTHONPATH=. ~/venv/py25/bin/python ./svn-pool.py 20
'Pool' object has no attribute '_pool'

The following patch would be fixed.

  • tracopt/versioncontrol/svn/svn_fs.py

    diff --git a/tracopt/versioncontrol/svn/svn_fs.py b/tracopt/versioncontrol/svn/svn_fs.py
    index 9a4ae8b..673b244 100644
    a b class Pool(object):  
    165165            # then initialize APR and set this pool
    166166            # to be the application-level pool
    167167            core.apr_initialize()
     168            self._pool = core.svn_pool_create(None)
    168169            application_pool = self
    169170
    170             self._pool = core.svn_pool_create(None)
    171171        self._mark_valid()
    172172
    173173    def __call__(self):
Last edited 5 years ago by Jun Omae (previous) (diff)

comment:2 Changed 5 years ago by fbrettschneider@…

I do use the trac-admin command in the post-commit hook. But I need to check if I cleared the [trac] repository_sync_per_request option… if not, would that cause such race condition? Also some Bitten slaves periodically may check if there's something to do on SVN commits.

comment:3 in reply to:  1 Changed 5 years ago by fbrettschneider@…

Replying to jomae:

The following patch would be fixed.

+            self._pool = core.svn_pool_create(None)
             application_pool = self

-            self._pool = core.svn_pool_create(None)

Sounds great! (y)

Last edited 5 years ago by Jun Omae (previous) (diff)

comment:4 in reply to:  2 Changed 5 years ago by Jun Omae

Replying to fbrettschneider@…:

I do use the trac-admin command in the post-commit hook. But I need to check if I cleared the [trac] repository_sync_per_request option…

Yeah, please confirm the option is empty. Your stacktrace indicates the option is not empty.

  File "build\bdist.win32\egg\trac\versioncontrol\api.py", line 343, in pre_process_request
    repo = self.get_repository(reponame)

And also tags/trac-1.0.1/trac/versioncontrol/api.py@:338,343#L335.

comment:5 Changed 5 years ago by fbrettschneider@…

no, there's no such entry in my [trac] section.

comment:6 Changed 5 years ago by anonymous

is the default value not empty?

comment:7 in reply to:  5 Changed 5 years ago by Jun Omae

no, there's no such entry in my [trac] section.

The default of repository_sync_per_request option is (default). To disable per-request sync need to add explicitly empty entry like this. You can check current settings using About Trac page and TracIni about default values.

[trac]
repository_sync_per_request =

comment:8 Changed 5 years ago by anonymous

can you the long hang technically explain, please?

comment:9 in reply to:  1 Changed 5 years ago by fbrettschneider@…

Replying to jomae:

It seems that svn_fs module has race condition on multithread. It can be reproduced with the following script.

}}} The following patch would be fixed.

  • tracopt/versioncontrol/svn/svn_fs.py

    diff --git a/tracopt/versioncontrol/svn/svn_fs.py b/tracopt/versioncontrol/svn/svn_fs.py
    index 9a4ae8b..673b244 100644
    a b class Pool(object):  
    165165            # then initialize APR and set this pool
    166166            # to be the application-level pool
    167167            core.apr_initialize()
     168            self._pool = core.svn_pool_create(None)
    168169            application_pool = self
    169170
    170             self._pool = core.svn_pool_create(None)
    171171        self._mark_valid()
    172172
    173173    def __call__(self):

Thanks. Please, add it to the Trac-1.0.2 release.

comment:10 Changed 5 years ago by Jun Omae

Milestone: next-stable-1.0.x0.12.6
Owner: set to Jun Omae
Status: newassigned

I've revised the patch in comment:1. Proposed changes can be found in log:jomae.git:t11411_0.12.6dev and log:jomae.git:t11411 for 1.0-stable.

We should use a lock object to get/set the application_pool variable to prevent the race-conditions.

comment:11 Changed 5 years ago by anonymous

I changed my settings and explicitely set an empty entry, and this has helped. I don't see that error anymore in the logging. — falkb

comment:12 Changed 5 years ago by Jun Omae

Release Notes: modified (diff)
Resolution: fixed
Status: assignedclosed

Committed to 0.12-stable in [12436] and merged to 1.0-stable and trunk in [12437,12438].

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jun Omae.
The resolution will be deleted.
to The owner will be changed from Jun Omae to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.