first time easy_install won't compile the catalogs

[Christian Boos]

By doing a few test installations, I noticed a strange effect: for a new Python installation or for a new virtualenv having just setuptools and Babel, easy_install Trac calls the appropriate install_lib step but this fails to trigger a compile_catalog step (r9638 and r9763).

This can be easily reproduced, in a fresh Python environment having only setuptools and Babel installed so far:

$ tar xvfz Trac-0.12.1.tar.gz
$ cd Trac-0.12.1
$ python setup.py -v bdist_egg 
...
running bdist_egg
running egg_info
writing requirements to Trac.egg-info/requires.txt
writing Trac.egg-info/PKG-INFO
writing top-level names to Trac.egg-info/top_level.txt
writing dependency_links to Trac.egg-info/dependency_links.txt
writing entry points to Trac.egg-info/entry_points.txt
reading manifest file 'Trac.egg-info/SOURCES.txt'
writing manifest file 'Trac.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
...

Whereas if the install is actually an upgrade, then it works as expected (e.g. easy_install Trac==0.12; easy_install -U Trac==0.12.1):

...
running install_lib
running compile_catalog_js
compiling catalog 'trac/locale/sl/LC_MESSAGES/messages-js.po' to 'trac/locale/sl/LC_MESSAGES/messages-js.mo'
...

I suspect the from trac.util.dist import get_l10n_js_cmdclass line to fail in the first situation. Debugging this shows the following error: No module named genshi.builder… The compile_catalog_js step indeed requires Genshi so this is not a side effect error we could try to ignore. With setuptools, the bdist_egg step is executed before the Genshi package gets installed. That actually happens at the very end of easy_install Trac:

Installed .../lib/python2.5/site-packages/Trac-0.12.1-py2.5.egg
Processing dependencies for Trac
Searching for Genshi>=0.6
...

So to solve this, we'd need to be able to trigger an install of Genshi at a very early stage.

In the meantime, we can advise to do an explicit:

easy_install Babel Genshi
easy_install Trac

for first time installations (easy_install Babel Genshi Trac won't work either).

[Christian Boos]

typo

[Jun Omae]

Ad hoc solution, it triggers to install genshi before the build stage.

Index: setup.py
===================================================================
--- setup.py    (revision 10435)
+++ setup.py    (working copy)
@@ -85,6 +85,9 @@
     test_suite = 'trac.test.suite',
     zip_safe = True,

+    setup_requires = [
+        'Genshi>=0.6',
+    ],
     install_requires = [
         'setuptools>=0.6b1',
         'Genshi>=0.6',

[Christian Boos]

The problem is that we require Genshi through the early import trac.utils.dist, in order to provide the generate_messages_js Command. So we should also delay importing the genshi dependencies, like:

dist.py

@@ -29 +29 @@
 from distutils.errors import DistutilsOptionError
 from setuptools.command.install_lib import install_lib as _install_lib
 
-from genshi.core import Stream
-from genshi.input import XMLParser
-
 from trac.util.presentation import to_json
 
 try:
@@ -47 +44 @@
         Select <script type="javascript/text"> tags and delegate to
         `extract_javascript`.
         """
+        from genshi.core import Stream
+        from genshi.input import XMLParser
         out = StringIO()
         stream = Stream(XMLParser(fileobj))
         stream.select('//script[@type="text/javascript"]').render(out=out)

But I'm not sure that's enough, there are indirect dependencies IIRC.

Anyway, thanks for the tip about setup_requires!

[Jun Omae]

I just understand the problem now. Thanks for the more detailed explanation. We need to be able to import get_l10n*cmdclass without Genshi in setup.py.

Then, I think that the cmdclasses should be moved to trac.dist from trac.util.dist.

trac.util.dist depends on trac.util and trac.util.html. trac.util.html directly depends on Genshi. trac.dist can minimize the dependencies with the delay importing.

I create a experimental patch t9961-dist.diff​.

[Christian Boos]

Looks pretty good! I'll test this evening.

[Remy Blank]

Replying to jomae:

Then, I think that the cmdclasses should be moved to trac.dist from trac.util.dist.

Minimizing (or even eliminating all) Trac dependencies in the dist module sounds like an excellent idea.

The fact that trac.util imports all sorts of stuff is a dependency nightmare (I understand we do it for backward compatibility), ideally it should be empty and all functionality should be in trac.util.* sub-modules. Maybe we should do some clean-up at some point.

[Christian Boos]

The patch seems to work great!

(babel-only)cboos@lynx:~/tests/0.12-stable$ python setup.py bdist_egg
warning: no previously-included files found matching 'doc/2000ft.graffle'
warning: no previously-included files matching '*' found under directory 'doc/logo.lineform'
zip_safe flag not set; analyzing archive contents...

Installed /home/cboos/tests/0.12-stable/Genshi-0.6-py2.7.egg
running bdist_egg
running egg_info
creating Trac.egg-info
...
running install_lib
running compile_catalog_js
compiling catalog 'trac/locale/sl/LC_MESSAGES/messages-js.po' to ...

However as you can see above, the first thing we get are the warnings emitted by Genshi's install. This is not so nice, maybe we can do:

setup.py

@@ -43 +43 @@
     from trac.dist import get_l10n_js_cmdclass
     extra['cmdclass'] = get_l10n_js_cmdclass()
 
-except ImportError, e:
+except ImportError:
     pass
 
+try:
+    import genshi
+except ImportError:
+    print "Genshi is needed by Trac setup, pre-installing"
+    # give some context to the warnings we might get when installing Genshi
+
+
 setup(
     name = 'Trac',
     version = '0.12.2',

Then we get:

(babel-only)cboos@lynx:~/tests/0.12-stable$ python setup.py bdist_egg
Genshi is needed by Trac setup, pre-installing
warning: no previously-included files found matching 'doc/2000ft.graffle'
warning: no previously-included files matching '*' found under directory 'doc/logo.lineform'
zip_safe flag not set; analyzing archive contents...

Installed /home/cboos/tests/0.12-stable/Genshi-0.6-py2.7.egg
running bdist_egg
...

only if Genshi was actually missing.

[Jun Omae]

Thanks for your testing and reviews! Committed in [10444].

[Christian Boos]

Good! I'll take care of merging, as I'm currently merging 0.12-stable. On trunk I will redo the trac/util/dist.py → trac/dist.py as a copy ;-)