Opened 13 years ago
Last modified 13 months ago
#10433 new enhancement
bootstrap script for Trac development
Reported by: | Owned by: | ||
---|---|---|---|
Priority: | normal | Milestone: | next-stable-1.6.x |
Component: | general | Version: | |
Severity: | normal | Keywords: | documentation |
Cc: | leho@…, Ryan J Ollos, osimons | Branch: | |
Release Notes: | |||
API Changes: | |||
Internal Changes: |
Description
After some experience with AppEngine launcher, I can not stand to re-construct the environment required for Trac each time I do a new clone anymore, so I've compiled a simple script to do the bootstrapping for development
Hope you'll find it useful - tested on 0.12.2
Attachments (2)
Change History (34)
comment:1 by , 13 years ago
follow-up: 3 comment:2 by , 13 years ago
As I have already mentioned elsewhere, a reasonable development environment for Trac should be done with virtualenv, and not with import magic requiring patches to Trac core. It may work for you, but it's certainly not to be recommended in general.
I wouldn't be against providing a "development set-up" script in contrib
, but your approach is far too fragile.
As for Trac on AppEngine, the first thing we would need is a SQL database (AFAIK, the database engine in AppEngine isn't SQL).
follow-up: 4 comment:3 by , 13 years ago
Replying to cboos:
Do you imply that Trac runs on the AppEngine? Interesting.
No. I just think App Engine Launcher is awesome.
Replying to rblank:
As I have already mentioned elsewhere, a reasonable development environment for Trac should be done with virtualenv, and not with import magic requiring patches to Trac core. It may work for you, but it's certainly not to be recommended in general.
Please, be more specific. Importing modules by hand with priority of current checkout path in sys.path
guarantees that you're executing Trac from this specific working copy. Can you guarantee that with setuptools? Patches are required to neutralize setuptools magic and bring back standard Python import behavior.
I wouldn't be against providing a "development set-up" script in
contrib
, but your approach is far too fragile.
Please be more specific instead of throwing vague points like 'not recommended' and 'too fragile'. Provide some arguments.
As for Trac on AppEngine, the first thing we would need is a SQL database (AFAIK, the database engine in AppEngine isn't SQL).
http://googleappengine.blogspot.com/2011/10/google-cloud-sql-your-database-in-cloud.html
follow-up: 5 comment:4 by , 13 years ago
Replying to anatoly techtonik <techtonik@…>:
Please, be more specific. Importing modules by hand with priority of current checkout path in
sys.path
guarantees that you're executing Trac from this specific working copy. Can you guarantee that with setuptools? Patches are required to neutralize setuptools magic and bring back standard Python import behavior.
You are doing it wrong. See #10419.
http://googleappengine.blogspot.com/2011/10/google-cloud-sql-your-database-in-cloud.html
Heh, I wasn't sure anymore if this was already public information. I have to admit that I thought of Trac when I first heard about it ;)
follow-up: 6 comment:5 by , 13 years ago
Replying to rblank:
Replying to anatoly techtonik <techtonik@…>:
Please, be more specific. Importing modules by hand with priority of current checkout path in
sys.path
guarantees that you're executing Trac from this specific working copy. Can you guarantee that with setuptools? Patches are required to neutralize setuptools magic and bring back standard Python import behavior.You are doing it wrong. See #10419.
Yet another meaningless comment. Can I ask you to repeat or copy-paste arguments, because it is really hard to track things you want to say through the links to various comments that were probably used in another context. So, the question is:
Does created virtualenv guarantee that if you're execute Trac server from the current checkout - the sources that are executed are the same sources from the same checkout?
follow-up: 7 comment:6 by , 13 years ago
Replying to anatoly techtonik <techtonik@…>:
Yet another meaningless comment.
It's actually much more relevant that you think. You know, I'm going to stop wasting time on this. I suggested several times to read the documentation of virtualenv, because I know it will solve your problem. I provided a complete script that does exactly what you want, at least as far as I can understand. If you don't want to read the doc and / or try the script, that's fine by me, but there's no need to post insulting comments.
Does created virtualenv guarantee that if you're execute Trac server from the current checkout - the sources that are executed are the same sources from the same checkout?
Yes, if you install into the virtualenv with python setup.py develop
.
comment:7 by , 13 years ago
Replying to rblank:
Replying to anatoly techtonik <techtonik@…>:
Yet another meaningless comment.
It's actually much more relevant that you think. You know, I'm going to stop wasting time on this. I suggested several times to read the documentation of virtualenv, because I know it will solve your problem
Sorry if it was insulting. The point is - virtualenv
doesn't seems to solve my problem. See below.
Does created virtualenv guarantee that if you're execute Trac server from the current checkout - the sources that are executed are the same sources from the same checkout?
Yes, if you install into the virtualenv with
python setup.py develop
.
And how many clones can I run from the same virtualenv
? I am not sure, but I guess only one - the last one installed with python setup.py develop
. I usually maintain multiple clones - one for each patch in process. So if I want to factor out some big patch into a different branch (clone) - I need to recreate this virtualenv
, deactivate, activate, and run server from new copy (and be extra careful about what and where from I execute).
With my script I just execute it from the corresponding clone and doesn't care about which python setup.py develop
was called last.
comment:8 by , 12 years ago
Cc: | added |
---|
comment:9 by , 12 years ago
Cc: | added |
---|
follow-up: 11 comment:10 by , 12 years ago
Resolution: | → wontfix |
---|---|
Status: | new → closed |
I had a look at your bootstrap.py … well, as this pretty much tries to undo a lot of things we do in Trac w.r.t. setuptools, so I don't see how it could make sense to integrate this in Trac itself.
If you really think this approach has some use, then you should continue to maintain this script externally, otherwise there are plenty of other ways to develop with Trac within the existing framework.
For example, I personally don't even use virtualenv and just add PYTHONPATH=. in front of the command line (see also the source:trunk/Makefile). This works well enough for me, you just have to be careful to use a Python installation (or virtualenv) in which no Trac has been installed.
comment:11 by , 12 years ago
Replying to cboos:
I had a look at your bootstrap.py … well, as this pretty much tries to undo a lot of things we do in Trac w.r.t. setuptools, so I don't see how it could make sense to integrate this in Trac itself.
The script is a crude hack - a demonstration that Trac can work without setuptools. No wonder it doesn't make sense to you. However, it solves a very important user story - makes Trac development setup and testing easier. Sophisticate development setup is the primary reason I don't submit patches for Trac anymore.
If you really think this approach has some use, then you should continue to maintain this script externally, otherwise there are plenty of other ways to develop with Trac within the existing framework.
I know only one way - trac setup.py develop
from here - TracDev/DevelopmentEnvironmentSetup
For example, I personally don't even use virtualenv
Shame on you. =)
and just add PYTHONPATH=. in front of the command line (see also the source:trunk/Makefile). This works well enough for me, you just have to be careful to use a Python installation (or virtualenv) in which no Trac has been installed.
you just have to be careful to use a Python installation (or virtualenv) in which //no// Trac has been installed
- you won't believe, but all this stuff had started after several wasted hours (days and maybe even weeks) thanks to damn setuptools magic that was loading wrong version of the code right after the first package instruction. I couldn't trace what setuptools was doing, but I believe that was the time I have to learn about metaclasses and Trac interface metamagic. It didn't help at that time, so wish it could be easier for other people.
comment:12 by , 12 years ago
So, the proposal is to think if setuptools loader can be made optional (or replaceable) for scripts like mine without having to maintain patches for every Trac version.
comment:13 by , 12 years ago
Hmmm I looked at the bootstrap script and it made my head spin a little. :) Then I re-read the thread here and I think comment:7 is the most understandable description of the problem for me. One I can definitely feel is an issue. Separate virtualenvs for different feature clones that each need a complete plugin, python modules, etc setup is definitely bleh in my book.
Whether anatoly's script is the answer is above me right now. Either way I'll keep this ticket in the back of my mind, I have a feeling I'm going to have to work it out for my own scenario sooner or later. I hope anatoly's not feeling his effort has been in vain and he'll keep submitting whatever he thinks is good in the future, whether accepted in core or not. Thought diversity ftw.
comment:14 by , 12 years ago
Just in case I wasn't clear enough (comment:10): using multiple source code checkouts of Trac from a single Python installation (or alternatively from a single virtualenv) is supported right now.
For example, consider the following organization:
~/Trac |_ testenv |_ trunk-feature1 | |_ Trac.egg-info | |_ trac | |_ tracopt |_ trunk-feature2 |_ Trac.egg-info |_ trac |_ tracopt ~/Python |_ bin |_ Lib |_ site-packages (no trac here!)
To work on a 3rd feature …
$ cd ~/Trac $ svn co http://svn.edgewall.org/trac/trunk trunk-feature3 $ cd trunk-feature3 $ ~/Python/bin/python setup.py egg_info
You need to do the above once per checkout. The presence of the Trac.egg-info
directory and its content is all what is required for setuptools to work.
$ PYTHONPATH=. ~/Python/bin/python trac/admin/console.py ../testenv permission list ... $ PYTHONPATH=. ~/Python/bin/python trac/web/standalone.py -p 9999 ../testenv
And it's even simpler if you use the Makefile.
No need to change anything in Trac, really. While having one checkout for each feature is perhaps not the easiest way to organize your development, it's possible to do so and it's quite close to the one checkout per development line model that I use (i.e. one for 0.12-stable, one for trunk).
comment:15 by , 12 years ago
Hmmm, interesting. I think git new-workdir trac.git trac-feature1.git feature1
is going to work very well with that.
follow-up: 18 comment:16 by , 12 years ago
The development environment still depends on system state, and that suxx, because it is easy to miss the fact that Trac is installed and run python setup.py develop
as advised in development documentation. With 95% probability anyone starting to use this PYTHONPATH method will sooner or later run into the conflict with installed version.
This can be avoided with development bootstrap script, where can be cleaned of hacks by implementing support for alternative plugin/component discovery interfaces in Trac.
comment:17 by , 12 years ago
I have a pretty ugly shell script which installs Trac in a virtualenv. I create a new environment for every ticket I work on. It takes about 30 seconds to run, then the following commands and I'm off and running to dev for ticket #10433.
./install_tracdev_env t10433 trac-trunk genshi-0.6.0
cd t10433
. bin/active
. start_tracd
There have been many days when I've been active on trac-hacks and created 15-20 new dev envs per day, pushed the patches and then deleted all of them.
Anyway, I'm hoping to improve the script soon to support multiple DB backends, run Trac from Apache, etc …
by , 12 years ago
Attachment: | install_tracdev_env added |
---|
comment:18 by , 12 years ago
Keywords: | documentation added |
---|---|
Milestone: | → not applicable |
Resolution: | wontfix |
Status: | closed → reopened |
Replying to anatoly techtonik <techtonik@…>:
The development environment still depends on system state, and that suxx, because it is easy to miss the fact that Trac is installed and run
python setup.py develop
as advised in development documentation.
I'll grant you that the current TracDev/DevelopmentEnvironmentSetup is dedicated to the one checkout, one virtualenv approach, which doesn't mix well with the PYTHONPATH=. method described above. That can be fixed, along with hints about the more sophisticated ways to maintain different patch sets in a single checkout (Mercurial's MQ, branches or bookmarks, Git topic branches). TODO
With 95% probability anyone starting to use this PYTHONPATH method will sooner or later run into the conflict with installed version.
If one can find a way to detect such a situation and give an appropriate warning at startup (or even abort), I'm interested. OPEN
This can be avoided with development bootstrap script, where can be cleaned of hacks by implementing support for alternative plugin/component discovery interfaces in Trac.
This is not going to happen.
follow-up: 23 comment:19 by , 12 years ago
I no longer use virtualenv for development either, instead I use make
with a custom additions to Makefile.cfg
that handle my typical needs - most notably:
make testenv
(andmake clean-testenv
) that hooks into functional tests to just create the complete test project with repository infrastructure.make python
to run a python code or start interactive shell.make trac-admin
for interactive admin mode.
With PYTHONPATH
and all other variables defined in Makefile.cfg
, everything runs in a predictable way - both my custom commands, and the make server
and various test commands defined in the main Makefile
.
Using Mercurial and MQ patch queues, I can swtich and change code as needed in-place and just use make
commands to test in any way I need.
I've made simplified Makefile
and Makefile.cfg
for each of my plugins too, with adjustments such as enabling plugins and adding settings as part of the relevant make
command.
comment:20 by , 12 years ago
Cc: | added |
---|
comment:21 by , 12 years ago
If you don't mind sharing one of your Makefile.cfg
files, I'd be really interested to take a look. I've been looking for something more configurable for each plugin than my dumb bash script allows and had previously been considering adding and reading from a config.ini
in the bash script. I suspect your solution is better.
comment:22 by , 12 years ago
I must say, seeing some screencasts of how exactly you guys do however you do what you do with some commentary, would be awesomely helpful. Trying to wrap my head around all these different approaches is making my head spin twice as fast as anatoly's script alone did. A tiny bit of Screencast-o-matic time perhaps, anyone?
follow-up: 25 comment:23 by , 12 years ago
Replying to osimons:
make testenv
(andmake clean-testenv
) that hooks into functional tests to just create the complete test project with repository infrastructure. OPENmake python
to run a python code or start interactive shell. DONEmake trac-admin
for interactive admin mode. DONE
These would be valuable additions. I thought about adding the last two myself ;-)
To people wondering why it can be better to use e.g. make trac-admin
rather than PYTHONPATH=. python trac/admin/console.py
: you can quickly reuse the other settings you made in Makefile.cfg, like specifying which Python version to use (python=25
), which db backend, etc.
by , 12 years ago
Attachment: | Makefile.cfg added |
---|
A simple Makefile.cfg
adding custom settings and commands.
follow-up: 26 comment:24 by , 12 years ago
I've added my attachment:Makefile.cfg as an example. Its sets paths that I can adjust, adds other settings needed, and defines the custom commands I mentioned.
It does not use the advanced python and db configuration possibilities from Makefile.cfg.sample
shipped with Trac, as I really never got my head wrapped around how it is supposed to work… Nor have I spent time making my config cross-plattform. No doubt Christian can transform my file into something that works for all - but understood by few ;-)
comment:25 by , 11 years ago
Milestone: | not applicable → next-minor-0.12.x |
---|---|
Owner: | set to |
Status: | reopened → assigned |
Replying to cboos:
Replying to osimons:
make testenv
(andmake clean-testenv
) that hooks into functional tests to just create the complete test project with repository infrastructure.make python
to run a python code or start interactive shell.make trac-admin
for interactive admin mode.These would be valuable additions. I thought about adding the last two myself ;-)
I added these two in r12167, starting from the 0.12-stable branch.
comment:26 by , 11 years ago
Replying to osimons:
It does not use the advanced python and db configuration possibilities from
Makefile.cfg.sample
shipped with Trac, as I really never got my head wrapped around how it is supposed to work…
Oh, sorry I missed this… Well, it's really simple. (Hopefully) better explanations added in r12168.
Nor have I spent time making my config cross-plattform. No doubt Christian can transform my file into something that works for all - but understood by few ;-)
Hm, nice try but it didn't happen (yet) ;-)
comment:27 by , 11 years ago
Cc: | added; removed |
---|
comment:28 by , 10 years ago
Milestone: | next-minor-0.12.x → next-stable-1.0.x |
---|---|
Owner: | removed |
comment:29 by , 10 years ago
Status: | assigned → new |
---|
comment:30 by , 8 years ago
Milestone: | next-stable-1.0.x → next-stable-1.2.x |
---|
Moved ticket assigned to next-stable-1.0.x since maintenance of 1.0.x is coming to a close. Please move the ticket back if it's critical to fix on 1.0.x.
comment:31 by , 4 years ago
Milestone: | next-stable-1.2.x → next-stable-1.4.x |
---|
Do you imply that Trac runs on the AppEngine? Interesting.