ExtractionError: Can't extract file(s) to egg cache

[boris.krasnoiarov@…]

How to Reproduce

While doing a GET operation on /report, Trac issued an internal error.

This was a reaction to 3 parallel requests hitting Trac at the same time right after it started.

User Agent was: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 (.NET CLR 3.5.30729)

System Information

Trac	0.11.5
Python	2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)]
setuptools	0.6c9
SQLite	3.3.4
pysqlite	2.3.2
Genshi	0.5.1
Pygments	1.0
jQuery:	1.2.6

Python Traceback

Traceback (most recent call last):
  File "c:\python25\lib\site-packages\Trac-0.11.5-py2.5-win32.egg\trac\web\main.py", line 444, in _dispatch_request
    dispatcher.dispatch(req)
  File "c:\python25\lib\site-packages\Trac-0.11.5-py2.5-win32.egg\trac\web\main.py", line 226, in dispatch
    data, content_type)
  File "c:\python25\lib\site-packages\Trac-0.11.5-py2.5-win32.egg\trac\web\chrome.py", line 730, in render_template
    template = self.load_template(filename, method=method)
  File "c:\python25\lib\site-packages\Trac-0.11.5-py2.5-win32.egg\trac\web\chrome.py", line 696, in load_template
    self.templates = TemplateLoader(self.get_all_templates_dirs(),
  File "c:\python25\lib\site-packages\Trac-0.11.5-py2.5-win32.egg\trac\web\chrome.py", line 434, in get_all_templates_dirs
    dirs += provider.get_templates_dirs()
  File "build\bdist.win32\egg\workfloweditor\workfloweditor_admin.py", line 21, in get_templates_dirs
    return [resource_filename(__name__, 'templates')]
  File "C:\Python25\lib\site-packages\pkg_resources.py", line 841, in resource_filename
    self, resource_name
  File "C:\Python25\lib\site-packages\pkg_resources.py", line 1311, in get_resource_filename
    return self._extract_resource(manager, zip_path)
  File "C:\Python25\lib\site-packages\pkg_resources.py", line 1318, in _extract_resource
    manager, os.path.join(zip_path, name)
  File "C:\Python25\lib\site-packages\pkg_resources.py", line 1332, in _extract_resource
    self.egg_name, self._parts(zip_path)
  File "C:\Python25\lib\site-packages\pkg_resources.py", line 921, in get_cache_path
    self.extraction_error()
  File "C:\Python25\lib\site-packages\pkg_resources.py", line 887, in extraction_error
    raise err
ExtractionError: Can't extract file(s) to egg cache

The following error occurred while trying to extract file(s) to the Python egg
cache:

  [Error 183] Cannot create a file when that file already exists: 'C:\\Documents and Settings\\NetworkService\\Application Data\\Python-Eggs\\workfloweditorplugin-1.0.1-py2.5.egg-tmp\\workfloweditor\\templates'

The Python egg cache directory is currently set to:

  C:\Documents and Settings\NetworkService\Application Data\Python-Eggs

Perhaps your account does not have write access to this directory?  You can
change the cache directory by setting the PYTHON_EGG_CACHE environment
variable to point to an accessible directory.

[Christian Boos]

Was that "workfloweditorplugin-1.0.1-py2.5.egg-tmp
workfloweditor
templates" folder extracted at this occasion, or is it there since a longer time? Check the permissions of that folder.

[anonymous]

This is happening on Windows from Administrator account. No weird permissions on the file. Standalone tracd. Changes to PYTHON_EGG_CACHE have no effect. The egg is being extracted every startup. Definitely a threading issue… The workaround is of course to fully extract eggs with —always-unzip flag…

[Thijs Triemstra <lists@…>]

Should be easy to verify..

[Remy Blank]

Please don't change the milestone field arbitrarily, thanks.

[Christian Boos]

As Remy said, but let me add that your help to the triaging is much appreciated ;-)

In order to make "conformant" changes, please have a look at the notes we write in the milestones themselves. The roadmap page gives you an overview of the Note to maintainers: which detail the guidelines currently in use.

[Remy Blank]

Replying to cboos:

As Remy said, but let me add that your help to the triaging is much appreciated ;-)

Yes, same here! I should have mentioned that, indeed.

[Thijs Triemstra <lists@…>]

Thanks guys, I just thought putting it in 0.13 would speed up the process of verifying this ticket, but I should've done that myself before I changed the milestone.

[Thijs Triemstra]

#9256 seems related.

[Christian Boos]

I just noticed a similar occurrence of this variant of ExtractionError ([Errno 17] File exists), on this Trac instance:

[pid 8930 139950199625440] 2011-01-12 23:02:44,851 Trac[env] INFO: -------------------------------- environment startup [Trac 0.13dev-r10406] --------------------------------
[pid 8932 139950199625440] 2011-01-12 23:02:45,741 Trac[env] INFO: -------------------------------- environment startup [Trac 0.13dev-r10406] --------------------------------
[pid 8932 139950199625440] 2011-01-12 23:02:48,409 Trac[env] WARNING: base_url option not set in configuration, generated links may be incorrect
[pid 8930 139950199625440] 2011-01-12 23:02:48,410 Trac[env] WARNING: base_url option not set in configuration, generated links may be incorrect
[pid 8936 139950199625440] 2011-01-12 23:02:49,009 Trac[env] INFO: -------------------------------- environment startup [Trac 0.13dev-r10406] --------------------------------
[pid 8936 139950199625440] 2011-01-12 23:02:50,121 Trac[env] WARNING: base_url option not set in configuration, generated links may be incorrect
[pid 8937 139950199625440] 2011-01-12 23:02:50,637 Trac[env] INFO: -------------------------------- environment startup [Trac 0.13dev-r10406] --------------------------------
[pid 8930 139950199625440] 2011-01-12 23:02:53,164 Trac[main] ERROR: Internal Server Error:
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/trac/web/main.py", line 447, in _dispatch_request
    dispatcher.dispatch(req)
  File "build/bdist.linux-x86_64/egg/trac/web/main.py", line 199, in dispatch
    content_type)
  File "build/bdist.linux-x86_64/egg/trac/web/chrome.py", line 903, in render_template
    template = self.load_template(filename, method=method)
  File "build/bdist.linux-x86_64/egg/trac/web/chrome.py", line 863, in load_template
    self.get_all_templates_dirs(), auto_reload=self.auto_reload,
  File "build/bdist.linux-x86_64/egg/trac/web/chrome.py", line 576, in get_all_templates_dirs
    dirs.extend(provider.get_templates_dirs() or [])
  File "build/bdist.linux-x86_64/egg/trac/web/chrome.py", line 554, in get_templates_dirs
    pkg_resources.resource_filename('trac', 'templates'),
  File "/usr/local/virtualenv/0.13-stable/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg/pkg_resources.py", line 841, in resource_filename
    self, resource_name
  File "/usr/local/virtualenv/0.13-stable/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg/pkg_resources.py", line 1311, in get_resource_filename
    return self._extract_resource(manager, zip_path)
  File "/usr/local/virtualenv/0.13-stable/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg/pkg_resources.py", line 1318, in _extract_resource
    manager, os.path.join(zip_path, name)
  File "/usr/local/virtualenv/0.13-stable/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg/pkg_resources.py", line 1332, in _extract_resource
    self.egg_name, self._parts(zip_path)
  File "/usr/local/virtualenv/0.13-stable/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg/pkg_resources.py", line 921, in get_cache_path
    self.extraction_error()
  File "/usr/local/virtualenv/0.13-stable/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg/pkg_resources.py", line 887, in extraction_error
    raise err
ExtractionError: Can't extract file(s) to egg cache

The following error occurred while trying to extract file(s) to the Python egg
cache:

  [Errno 17] File exists: '/tmp/.uwsgi-egg-cache-0.13-stable/Trac-0.13dev_r10406-py2.5.egg-tmp/trac/templates'

The Python egg cache directory is currently set to:

  /tmp/.uwsgi-egg-cache-0.13-stable

Perhaps your account does not have write access to this directory?  You can
change the cache directory by setting the PYTHON_EGG_CACHE environment
variable to point to an accessible directory.
ExtractionError: Can't extract file(s) to egg cache

The following error occurred while trying to extract file(s) to the Python egg
cache:

  [Errno 17] File exists: '/tmp/.uwsgi-egg-cache-0.13-stable/Trac-0.13dev_r10406-py2.5.egg-tmp/trac/templates'

The Python egg cache directory is currently set to:

  /tmp/.uwsgi-egg-cache-0.13-stable

As one can see, this happens at server startup, when several processes attempt to do the extraction at the same time. Obviously there's a race condition in setuptools, so we should perhaps trap and ignore this variant of extraction error each time we call pkg_resources.resource_filename. It's actually the same issue as #9107 but for a different location, here when looking for the templates, there for the catalogs.

[Christian Boos]

See #9978.