Replace easy_install and eggs with pip and wheels

[anonymous]

The Trac documentation talks a lot about easy_install and eggs. Presumably when it was written these were the best practices. It seems today using pip and wheels would be much easier and generally preferable.

The ​Python docs say:

pip is the preferred installer program. Starting with Python 2.7.9, it is included by default with the Python binary installers.

But from the Trac documentation (to someone who is not a Python packaging expert) it is far from clear if this is recommended or even possible with Trac. Some examples:

• TracInstall lists easy_install before pip, and the pip section is fairly confusing and assumes Linux, Apache, non-SQLite. This page does not mention wheels (the word egg appears six times).
• TracUpgrade does not mention pip or wheels at all. (easy_install is mentioned four times, egg once.)
• TracPlugins lists easy_install before pip, and much more often (12 vs. 5). This page does not mention wheels, while the word egg occurs >40 times.
• TracDev/PluginDevelopment: egg 10, wheel 0. Links to ​th:wiki:EggCookingTutorial as the tutorial to build plugins.

(And there are many more on other wiki pages.)

Please clarify if this is still the preferred way or just a historical relict nobody has bothered to change yet. Are pip and/or wheels supported and/or even preferred nowadays? Should such old documentation be updated to recommend pip first? Can old easy_install cruft even be removed?

[Ryan J Ollos]

Replying to anonymous:

Please clarify if this is still the preferred way or just a historical relict nobody has bothered to change yet. Are pip and/or wheels supported and/or even preferred nowadays?

That is my impression. There are some minor issues with wheels (#12282, #12283), but otherwise installation with wheels works fine.

Should such old documentation be updated to recommend pip first? Can old easy_install cruft even be removed?

I have intended to modify the TracInstall documentation to use pip when we start development on 1.3.1. In 1.3.1 we will drop support for Python < 2.7. Setuptools is still a required dependency, but the information on installing with easy_install could be moved to an addendum.

Additionally I think the TracInstall documentation should assume Linux, TracStandalone and SQLite. TracInstall can point to other pages for using a different database and web servers. We should have detailed pages for each major OS.

There are some related changes I have in mind that I'll raise for discussion on the MailingList in the coming weeks.

[Ryan J Ollos]

We should also consider having TracInstall specify instructions for installing into a virtualenv. The downside is that it can be difficult to get the Subversion bindings installed in the virtualenv. However, it's easier to cleanup and start over, and may avoid filesystem permissions issues. It may be the best option for beginners.

[Christian Boos]

Moving to some later 1.3.x version.

[Christian Boos]

Note that I installed the ​Trac-1.3.1rc1-py2-none-any.whl on Windows with:

pip install Trac-1.3.1rc1-py2-none-any.whl

and didn't notice any issue with it.

The .whl was built on Linux with python setup.py bdist_wheel.

[Ryan J Ollos]

Replying to anonymous:

• TracInstall lists easy_install before pip, and the pip section is fairly confusing and assumes Linux, Apache, non-SQLite. This page does not mention wheels (the word egg appears six times).

1.3/TracInstall contains revised install instructions. easy_install and eggs are no longer mentioned (moved to setuptools). I didn't see a reason to mention the wheel format. In the future I intend to do additional overhaul to simplify the TracInstall page.

Please let me know if you have any suggestions. I'll modify other pages soon.

[Ryan J Ollos]

Additional changes in 1.3/TracUpgrade@4.

I've started drafting a new TracPlugins page. I'd like to recommend installing in the wheel format and remove information about eggs. If all Trac, dependencies and plugins are installed in wheel format, maybe the egg cache is no longer relevant?

[Ryan J Ollos]

Replying to Ryan J Ollos:

I'd like to recommend installing in the wheel format and remove information about eggs. If all Trac, dependencies and plugins are installed in wheel format, maybe the egg cache is no longer relevant?

More comments in ​trac-dev:z6Vg8o0U5XA/r__A5ldbCwAJ.

[Jun Omae]

I've started drafting a new TracPlugins page. I'd like to recommend installing in the wheel format and remove information about eggs. If all Trac, dependencies and plugins are installed in wheel format, maybe the egg cache is no longer relevant?

Yes if no egg file is used. Of course, the egg cache directory also is not needed if all eggs are installed as directory using easy_install --always-unzip.

[Jun Omae]

Replying to Jun Omae:

I've started drafting a new TracPlugins page. I'd like to recommend installing in the wheel format and remove information about eggs. If all Trac, dependencies and plugins are installed in wheel format, maybe the egg cache is no longer relevant?

Yes if no egg file is used. Of course, the egg cache directory also is not needed if all eggs are installed as directory using easy_install --always-unzip.

Ah, I'm not sure about how to install *.whl file to $ENV/plugins and [inherit] plugins_dir.

[Ryan J Ollos]

Replying to Jun Omae:

Ah, I'm not sure about how to install *.whl file to $ENV/plugins and [inherit] plugins_dir.

Good point. I guess we need to look at whether loader.py can be modified to detect and load wheels?

[scatolina@…]

It looks like easy_install is still required when you decide to run trac on its standalone webserver on Windows (with SRVANY).

In fact, pip installation will NOT create the required tracd-script.py file.

See: 1.3/TracStandalone

DO NOT use tracd.exe. Instead register python.exe directly with tracd-script.py as a parameter. If you use tracd.exe, it will spawn the python process without SRVANY's knowledge. This python process will survive a net stop tracd.

[Jun Omae]

Replying to scatolina@…:

It looks like easy_install is still required when you decide to run trac on its standalone webserver on Windows (with SRVANY).

In fact, pip installation will NOT create the required tracd-script.py file.

We could use python.exe -m trac.web.standalone rather than python.exe tracd-script.py.

[Jun Omae]

Replying to Jun Omae:

We could use python.exe -m trac.web.standalone rather than python.exe tracd-script.py.

Also, if wheel package is installed, tracd.exe is executable format with zip archive. tracd.exe can be specified as a parameter of python.exe.

C:\>unzip -l C:\venv\trac-1.0.15\Scripts\tracd.exe
Archive:  C:/venv/trac-1.0.15/Scripts/tracd.exe
warning [C:/venv/trac-1.0.15/Scripts/tracd.exe]:  95288 extra bytes at beginning or within zipfile
  (attempting to process anyway)
  Length     EAs   ACLs    Date   Time    Name
 --------    ---   ----    ----   ----    ----
      201      0      0  17/08/08 18:24   __main__.py
 --------  -----  -----                   -------
      201      0      0                   1 file

C:\>C:\venv\trac-1.0.15\Scripts\python.exe C:\venv\trac-1.0.15\Scripts\tracd.exe --help
Usage: tracd [options] [projenv] ...

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -a DIGESTAUTH, --auth=DIGESTAUTH
                        [projectdir],[htdigest_file],[realm]
  --basic-auth=BASICAUTH
                        [projectdir],[htpasswd_file],[realm]
  -p PORT, --port=PORT  the port number to bind to
  -b HOSTNAME, --hostname=HOSTNAME
                        the host name or IP address to bind to
  --protocol=PROTOCOL   http|scgi|ajp|fcgi
  -q, --unquote         unquote PATH_INFO (may be needed when using ajp)
  --http10              use HTTP/1.0 protocol version instead of HTTP/1.1
  --http11              use HTTP/1.1 protocol version (default)
  -e PARENTDIR, --env-parent-dir=PARENTDIR
                        parent directory of the project environments
  --base-path=BASE_PATH
                        the initial portion of the request URL's "path"
  -r, --auto-reload     restart automatically when sources are modified
  -s, --single-env      only serve a single project without the project list

[scatolina@…]

Replying to Jun Omae:

Replying to Jun Omae:

We could use python.exe -m trac.web.standalone rather than python.exe tracd-script.py.

Also, if wheel package is installed, tracd.exe is executable format with zip archive. tracd.exe can be specified as a parameter of python.exe. […]

Good to know, thanks!

[Ryan J Ollos]

Replying to Ryan J Ollos:

Replying to Jun Omae:

Ah, I'm not sure about how to install *.whl file to $ENV/plugins and [inherit] plugins_dir.

Good point. I guess we need to look at whether loader.py can be modified to detect and load wheels?

I did some investigation of this. It appears that the wheel format ​cannot currently be directly imported the way the egg format can. Here are two issues that appear relevant:

• ​setuptools 376
• ​wheel issue 145

[Jun Omae]

I found work around is to copy the wheel package with .egg suffix to $ENV/plugins to load wheel packages.

$ pip list --format=columns
Package    Version
---------- -------
Genshi     0.7
pip        9.0.1
setuptools 36.2.7
Trac       1.0.15
wheel      0.29.0
$ python setup.py bdist_wheel
running bdist_wheel
running build
running build_py
....
Copying TracDragDrop.egg-info to build/bdist.linux-x86_64/wheel/TracDragDrop-0.12.0.13-py2.7.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/TracDragDrop-0.12.0.13.dist-info/WHEEL
$ cp dist/TracDragDrop-0.12.0.13-py2-none-any.whl /var/tracenv/plugins/TracDragDrop-0.12.0.13-py2-none-any.egg
$ tracd /var/tracenv
Server starting in PID 13112.
Serving on 0.0.0.0:3000 view at http://127.0.0.1:3000/
Using HTTP/1.1 protocol version
09:07:12 PM Trac[env] INFO: -------------------------------- environment startup [Trac 1.0.15] --------------------------------
09:07:12 PM Trac[loader] DEBUG: Adding plugin TracDragDrop 0.12.0.13 from /var/tracenv/plugins/TracDragDrop-0.12.0.13-py2-none-any.egg
09:07:13 PM Trac[loader] DEBUG: Loading trac.about from /venv/lib/python2.7/site-packages
09:07:13 PM Trac[loader] DEBUG: Loading trac.admin.console from /venv/lib/python2.7/site-packages
...
09:07:13 PM Trac[loader] DEBUG: Loading trac.wiki.web_api from /venv/lib/python2.7/site-packages
09:07:13 PM Trac[loader] DEBUG: Loading trac.wiki.web_ui from /venv/lib/python2.7/site-packages
09:07:13 PM Trac[loader] DEBUG: Loading tracdragdrop.web_ui from /var/tracenv/plugins/TracDragDrop-0.12.0.13-py2-none-any.egg
...

However, it would be good to copy as a directory rather than a file.

$ unzip -x dist/TracDragDrop-0.12.0.13-py2-none-any.whl -d /var/tracenv/plugins/TracDragDrop-0.12.0.13-py2-none-any.egg
Archive:  dist/TracDragDrop-0.12.0.13-py2-none-any.whl
  inflating: /dev/shm/wheel-test-tracenv/plugins/TracDragDrop-0.12.0.13-py2-none-any.egg/tracdragdrop/__init__.py
  inflating: /dev/shm/wheel-test-tracenv/plugins/TracDragDrop-0.12.0.13-py2-none-any.egg/tracdragdrop/web_ui.py
  inflating: /dev/shm/wheel-test-tracenv/plugins/TracDragDrop-0.12.0.13-py2-none-any.egg/tracdragdrop/htdocs/loading.gif
  ....

[Ryan J Ollos]

Replying to scatolina@…:

It looks like easy_install is still required when you decide to run trac on its standalone webserver on Windows (with SRVANY).

In fact, pip installation will NOT create the required tracd-script.py file.

See also comment:2:ticket:12757.

>pip install trac
Collecting trac
  Using cached Trac-1.2.2-py2-none-any.whl
Requirement already satisfied: Genshi>=0.6 in c:\users\ryanol~1\pve-pip\lib\site-packages (from trac)
Requirement already satisfied: setuptools>=0.6 in c:\users\ryanol~1\pve-pip\lib\site-packages (from trac)
Installing collected packages: trac
Successfully installed trac-1.2.2

>ls pve-pip\Scripts
activate          deactivate.bat        pip2.7.exe   trac-admin.exe
activate.bat      easy_install-2.7.exe  pip2.exe     tracd.exe
activate.ps1      easy_install.exe      python.exe   wheel.exe
activate_this.py  pip.exe               pythonw.exe
> wheel install-scripts trac
>ls pve-pip\Scripts
activate          easy_install-2.7.exe  python.exe            tracd.exe
activate.bat      easy_install.exe      pythonw.exe           wheel.exe
activate.ps1      pip.exe               trac-admin-script.py
activate_this.py  pip2.7.exe            trac-admin.exe
deactivate.bat    pip2.exe              tracd-script.py

I'm unsure why wheel install-scripts trac is needed on Windows to generated the -script.py files. The scripts are generated on Linux by "pip install". Maybe it's a defect with pip?

[Jun Omae]

I think that behavior is by design. The tracd.exe and trac-admin.exe are launcher with zip archive when it is installed via wheel. The console script is embedded in the __main__.py in the zip archive. Python interpreter allows to execute a zip archive. Therefore, *-script.py is no longer needed on Windows.

C:>call venv\py27-amd64\Scripts\python.exe -m pip install Trac
Collecting Trac
  Downloading Trac-1.2.2-py2-none-any.whl (3.8MB)
    100% |################################| 3.8MB 111kB/s
Requirement already satisfied (use --upgrade to upgrade): Genshi>=0.6 in c:\src\trac\venv\py27-amd64\lib\site-packages (from Trac)
Requirement already satisfied (use --upgrade to upgrade): setuptools>=0.6 in c:\src\trac\venv\py27-amd64\lib\site-packages (from Trac)
Installing collected packages: Trac
Successfully installed Trac-1.2.2

C:>unzip -l venv\py27-amd64\Scripts\tracd.exe
Archive:  venv/py27-amd64/Scripts/tracd.exe
warning [venv/py27-amd64/Scripts/tracd.exe]:  95287 extra bytes at beginning or within zipfile
  (attempting to process anyway)
  Length     EAs   ACLs    Date   Time    Name
 --------    ---   ----    ----   ----    ----
      201      0      0  17/08/23 16:10   __main__.py
 --------  -----  -----                   -------
      201      0      0                   1 file

C:>unzip -p venv\py27-amd64\Scripts\tracd.exe __main__.py
warning [venv/py27-amd64/Scripts/tracd.exe]:  95287 extra bytes at beginning or within zipfile
  (attempting to process anyway)
# -*- coding: utf-8 -*-
import re
import sys

from trac.web.standalone import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

[Ryan J Ollos]

Thanks, I made some untested edits: 1.3/TracStandalone@3.

[Ryan J Ollos]

Edited 1.3/TracPlugins@5.

The docs now use pip and wheels, except when describing how to create plugins for installing in the environment plugins directory.

Please make a note here if there are other places in the documentation that are/should-be edited.