Use Subversion (or the source repository) for Trac's data as well

[twenim@…]

Use Subversion for data repository like subwiki

[cmlenz]

See TighterSubversionIntegration.

[florian.burka@…]

I think this would be great, but there seems to be no movement in this. Is this still planned or not?

[anonymous]

Added a attachment to TighterSubversionIntegration, which enables a svn wiki backend.

Usage:

create a directory in your svn repos, and add a line like this to the trac.ini file:

[wiki]
svn_root = /path/to/wiki/root/from/svn/scope/

dump all wiki files which trac-admin to this directory, and commit it.

Apply the patch and restart trac.

TODO:

• Deletion of pages.
• Add the TimeLineEventProvider? and ISearchProvider.
• Testing the authz support.

[anonymous]

I have read the TighterSubversionIntegration and WhySQLite page regarding the pros and cons of SQLite. FWIW, I'd like to see trac use SVN as its backend, mainly because I wouldn't have to use hot copy to backup the entire trac db. With our SVN repos using FSFS, we can do incremental backups using rsync. I'd like to be able to do this with trac also.

Also, if something like this were to be implemented, I'd rather trac use its own repo inside the trac folder, rather than store in a special place in our source-code repo. I don't like the idea of something else commiting to our repository. It is for our source ocde and I'd like to keep it that way. Possibly being paranoid but also trying to keep the repo 'as clean as possible'.

Thanks

Russell

[Fergus Henderson <fjh-mailbox-38@…>]

My company is currently using Twiki, but we're planning to switch over to Trac if/when this is supported.

[cboos]

I'm going to update the TracObjectModelProposal with an API for object storage an revisioning.

[eblot]

A question about implementing an SVN backend: what about the search feature ?

[cboos]

Here's how I see it (roughly):

• A pluggable storage backend for the Trac objects (content, properties), which will be able to do the versioning. That backend could be the DB as usual, but also any other repository backend. A simple FS backend could be possible too...
• A database cache for the latest values (queries, full text search). Maybe even some kind of history, too (property changes?).
• A filesystem cache for the attachements (when we need to do a send_file) and possibly other generated data.

The data stored in the last two components could always be rebuilt from the authoritative information in the storage backend.

[MBParker@…]

(This is my first post here, and, except for the lack of WYSIWYG, Trac is very cool BTW - love its Tracklinks and diff displays!, among other coolness)

Google search for Trac store wiki pages led me to ticket:1465 which led me here.

Yes, I would very much like to see Trac store its wiki (and eventually all its data) in the source control system (as Subversion).

However, I read on ticket:1465 that on 04/22/05 04:58:06 cboos said "I always thought that it was not a good idea to re-create a (simple) versioning system for versioning Trac's content, rather than using SVN. Now I think it would be really a bad idea to re-create a "light" distributed versioning system, given the complexity of such a thing, and the fact that a good alternative exist. " However, on 01/24/06 07:52:17, I'm pleased to see he'd devising a plan.

I think it should be done. I understand its hard. To avoid conflicting IDs in the database, you probably would have to go to a dual layer ID system: instance_name+sequential_number, not just sequential_number; and you'd have to sync the database. But this is doable. And, as well as making a cleaner server (not using separate storage), as ticket:1465 points out it would also allow you to run the trac distributed with sync. And if for some reason subversion was short a feature or two, it would then be improved as consequence.

This is my take: Subversion looks like a solid thing - so stick with it. Or if some version control system is better (perhaps Monotone ), use that. But don't have the very tool (Trac) designed to manage repository versioning reinvent versioning as well. -Mike Parker, www.Cytex.com

[Russell Hind <rhind@…>]

My 2 reasons for wanting this are:

• We use rsync for backups of our FSFS repositories, it would be a lot easier if we could do the same with trac (especially to remote locations)
• I know nothing about SQL/sqlite or any other database stuff, but I can administer an svn repository (hence we are using it and not something else for our source code) so for me it would just make sense to store it in
  • The source code repo under a user defined path
  • A separate repo (I have no real preference here)

[Russell Hind <rhind@…>]

(Sorry, didn't realise I'd already posted part of this above)

Cheers

Russell

[anonymous]

It came as a complete surprise to us when we realized (before installing) that Trac-generated content (Wiki pages, Wiki history, issue tracking, etc.) is NOT stored in a Subversion system.

1. We'd like to use Trac's Wiki functionality to maintain project documentation, but we need this documentation to be part of the general project source data, under Subversion.

2. We want a simple system with few dependencies and a simple backup procedure for all generated content. A separate database (with its dependencies) for issues and for the Wiki content unfortunately does not meet this need.

Trac looks awesome and we were so close to choosing it - please consider using Subversion for storing all interesting information. Thank you'''

[mgood]

Replying to anonymous:

Trac looks awesome and we were so close to choosing it - please consider using Subversion for storing all interesting information. Thank you'''

Trac will probably support this at some point, but it is not a "high" priority right now.

2. We want a simple system with few dependencies and a simple backup procedure for all generated content. A separate database (with its dependencies) for issues and for the Wiki content unfortunately does not meet this need.

SQLite was chosen as the default database since it only requires the installation of a library. There's no setup involved since it's all handled internally by Trac. Storing the wiki and/or tickets in Subversion (or other VC software supported by Trac) does not eliminate Trac's need for a database, so that will always be a requirement. There are simple backup solutions for Trac. Please ask on the MailingList if you're interested.

[Russell Hind <rhind@…>]

Replying to mgood:

There are simple backup solutions for Trac. Please ask on the MailingList if you're interested.

Yes, even trac-admin hotsync works ok, but for our svn repositories (fsfs backend), we use rsync which provides very efficient incremental backups to remote locations easily. Using a database for the wiki and ticket information doesn't provide this possibility (well not for those of us who have no experience with databases/SQL).

Cheers

Russell

[cboos]

Another long term idea.

[blais@…]

Every time I have to edit wiki pages through a web browser textarea, I wish for this feature to happen, so I can edit the wiki pages using Emacs. I really cannot understand why it was not designed this way in the first place... using SVN as a backend solves a number of problems, and only seems natural, given the nature of trac.

[gunther.seghers@…]

Can someone please provide a stepwise procedure on how to setup TRAC so that it allows to version wiki pages using SVN. The end-result should allow us to see, export all wiki pages belonging to a certain version label together with all other software artefacts

[eblot]

Replying to gunther.seghers@ipc.be:

Can someone please provide a stepwise procedure on how to setup TRAC so that it allows to version wiki pages using SVN. The end-result should allow us to see, export all wiki pages belonging to a certain version label together with all other software artefacts

Wiki pages are not stored in a SVN repository, so I'm afraid this feature is not available right now (and not really planned for being implemented within a short timeframe).

[anonymous]

I'd be very interested in this feature as well.

[anonymous]

I'm also quite astonished, that Subversion is NOT used for storing the changed wiki-pages. Is there any work in progress on that feature in the meantime? If there is a perspective, we might opt on using Trac, if not, we need to go on unfortunately because Postgres/SQLite/MySQL are not databases of our choice as Oracle or Informix are.

[cboos]

Even if someday the Wiki pages are store in a versioncontrol backend, a database will always be needed, as there's much more persistent data in Trac than just the content of wiki pages. And most of these informations need to be readily available, so scanning through a vc backend is not an option.

OTOH, Oracle can eventually be supported one day, I think it's not more broken than MySQL ;-)

[mgood]

Replying to anonymous:

Postgres/SQLite/MySQL are not databases of our choice as Oracle or Informix are.

I wouldn't think of SQLite as a database. The fact that it happens to support SQL is pretty much irrelevant from the end-user perspective. Learning to administer another database server like Postgres or MySQL obviously has some overhead, but SQLite is pretty much transparent. You're free not to use Trac, but I wouldn't place too much weight on the data format.

Replying to cboos:

OTOH, Oracle can eventually be supported one day, I think it's not more broken than MySQL ;-)

It's not broken in the same ways as MySQL, but Oracle has significant barriers to writing cross-db compatible SQL. Most notably it lacks LIMIT and OFFSET and the workarounds are quite ugly. SQLAlchemy will help, but I don't expect it to be easy.

[Harald]

Thanks for your and mgoods comment. I'm currently using Postgres for my Trac evaluation. Depending on the behavior during backup i might give Sqlite another try. Despite that, my main point was how to get the Wiki pages into a Subversion repo. I've read the whole discussion in the meantime and saw also the remarks about transient data being stored in the database as well. That would be ok. Again, the wiki-pages and attachments are my focus My question now: IS there any work in progress maybe basing on the partial implementation of PScholl as advocated in the article 'Advocating a Tighter Subversion Integration'? TIA Harald

Replying to cboos:

Even if someday the Wiki pages are store in a versioncontrol backend, a database will always be needed, as there's much more persistent data in Trac than just the content of wiki pages. And most of these informations need to be readily available, so scanning through a vc backend is not an option. OTOH, Oracle can eventually be supported one day, I think it's not more broken than MySQL ;-)

[blais@…]

Harald: it's not work-in-progress, but a few months ago I figured out everything that had to be done for it, and started to write the code for it, and there is nothing that would prevent this from happening, it's just a matter of turning the crank: 1. isolate the DB access through a Python interface, 2. move the current code into one interface implementation, make sure everything still works (test), 3. implement another version that does the storage via subversion. The source code is already pretty much separated in a single module (apart from a few accesses to that DB table from another file).

I just haven't had the time to do it... this is really annoying me as well, I want to be able to edit the wiki pages from Emacs, grep them, process them with scripts, etc.

[Marc Fargas <telenieko@…>]

CC'ing myself, would be nice to see this working. And you can sum: all data that is, tickets, pages, etc ;)

[MBParker@…]

http://sharpforge.org/p/SharpForge.aspx "is an open source, c#, dot net 2.0, easy to use project hosting web application. It's similar to SourceForge? or CodePlex? but for your own team or organisation."

SharpForge? HAS a Subversion-based Wiki (though the wiki formatting may not be yet as powerful as Trac), plus "Work Item Tracking" like Trac, plus Forums, chat, and more, and HAS WYSIWYG data entry, and, designed recently (2006?), it is coded probably more cleanly in C# and DotNet? 2.0. It is still in Beta, however, it works today - indeed the link above is apparently to the SharpForge? project hosted in SharpForge? itself. Moreover, its leading sponsor of SharpForge? seems to be commercial hosting provider http://www.ProjXpert.com which uses SharpForge? to host projects for folks commercially (and at low prices).

Overall, provided it actually does what it says (I haven't used it yet), and provided one is ok with hosting on DotNOT (which apparently can be done even in Linux via Mono), SharpForge? seems to have more potential than any other web project hosting application I've heard of, no? For this reason, since about April, I've been planning on using SharpForge? and ProjXpert? (instead of Trac, Bugzilla, and the rest). I may be missing something, but SharpForge? seems clearly worth considering - maybe something Trac wants to copy, or users want to choose instead.

[gunnar_thielebein@…]

i don't think this is a must have feature. I've gone through the trac mailinglist and the goal for futuredevelopment is to have a wider support of repositories, databases and so on. to save wiki in svn is not everyones approach.

For coding in java you already store the documentation of code in svn which is ok but how will you sort your code and your documentation if you also have the trac wiki stored there.

And if the trac wikiarticle also are in repo for what kind use? the oss crowed which makes an checkout doesn't need your articles, and don't forget that changesets want to be kept increment by codechange and not wikichanges.

[anonymous]

Replying to gunnar_thielebein@gmx.net:

i don't think this is a must have feature.

Ah, so you're not one of the people who wants it. That doesn't mean it's not important for others/

I've gone through the trac mailinglist and the goal for futuredevelopment is to have a wider support of repositories, databases and so on. to save wiki in svn is not everyones approach. For coding in java you already store the documentation of code in svn which is ok but how will you sort your code and your documentation if you also have the trac wiki stored there. And if the trac wikiarticle also are in repo for what kind use? the oss crowed which makes an checkout doesn't need your articles, and don't forget that changesets want to be kept increment by codechange and not wikichanges.

I for one would love to be able to keep documentation that's maintained through the wiki in proper version control - there are all kinds of reasons for it.

[Ivan G Shevchenko]

Trac data in SVN - very interesting proposal! Good arguments for this are here: TighterSubversionIntegration#Motivation

[nkantrowitz]

Given the direction we are moving in w.r.t. supporting multiple VCSs, this doesn't seem likely to happen at any point without major changes to Trac. With the newhelp system being developed for 0.12, there will be an example module for displaying documention (or other wiki-like content) from Subversion directly. +1 to retarget to 0.12/newhelp.

[cboos]

This ticket has nothing to do with the newhelp proposal (comment:34 and comment:35 side track on the topic of help, but that's not at all the main point of this ticket)..

Like explained in TighterSubversionIntegration, this is about storing all Trac data in a VCS system (wiki pages but also ticket data, milestones etc.).

One of the primary interest for such an approach is the possibility to distribute Trac data, when a DVCS is used for the backend storage (#1465).

Of course, this can't be done without deep changes, that's why it's 2.0 (experimental or alternative even).