Use trac with a remote Subversion repository

[radix at twistedmatrix.com]

Currently Trac needs to have a direct (read-only) access to the Subversion repository. It's not yet possible to simply specify an URL to a remote repository.

Alternatives

Shun-ichi Goto wrote an alternative backend for Subversion which uses the svn_ra layer, see:  http://www.meadowy.org/~gotoh/hg/remote-svn-plugin

Otherwise, it's quite easy to mirror a remote repository using svnsync and have Trac access the local copy.

[mrowe@…]

At present Subversion's Python bindings do not support the more-generic "repository access layer" that would facilitate accessing  file://,  http:// and  svn:// repositories. Adding support for remote repositories to Trac will be difficult until this situation is remedied.

[daniel]

As Mark pointed out, this is indeed the reason for above mentioned stink.

Whenever there's an official way to us the RA layer, Trac will utilize it.

[anonymous]

This is understandably difficult. However, do ant trac'ers have recommendations for how to do do fine grained access control for users coming through trac? I'm currently using subversions apache module to control which users have read and write access to content via AuthzSVNAccessFile. Since trac is using the server bindings and not going through HTTP as a client, it is bypassing my access control layer. Does any one know how to do svn access control at a lower level???

I'll take this up on the svn lists too.

[Marc Sutton]

Subversion 1.2 has just been released and they've flattened the remote access API so SWIG can access it (and python bindings can be produced).

Is that all that's required to get trac to use a remote repository?

[cmlenz]

I've experimented with this, but unfortunately the SWIG bindings for svn_ra didn't seem to be completely usable.

For example, svn_ra_stat (IIRC) should return a dirent object of some kind, but it expects a pointer to a struct. Apparently the mapping from C to python isn't quite worked out yet for that layer.

[Juanma Barranquero]

[...] Subversion 1.2 has just been released [...]

Well,  Subversion 1.2 has not yet been released, unless you count release candidates as full releases. Let's not spread wrong information, please.

[eric@…]

1.2 has been released now... any improvement on this situation?

[anonymous]

Related to fine grained permissions question asked in this topic:

 http://projects.edgewall.com/trac/wiki/FineGrainedPermissions

[jwt]

I have successfully used  http://svk.elixus.org/ for mirroring a remote repository (to a local trac-server). Obviously this is not always desirable.

Any status-update on this issue yet ?

[Tobi Oetiker <oetiker@…>]

When I try using  svn://svn....../repo as a repository path, I end up with a nice error message:

Assertion failed: is_canonical (base, blen), file subversion/libsvn_subr/path.c, line 114

I am using svn 1.2.1 with trac 0.9b2

any progress on remote repository support ?

[jwin1@…]

I really want to promote using trac here with our development, but our 'server-folks' (sadly) just aren't open-minded enough to keep trac and svn on one machine, not speaking of all political difficulties...

This is really an important one for us.

[mgood]

Well, I think that it will save your "server-folks" some work to keep the two on the same box, since it makes it easier to set up things like the authentication that should be the same between them. If the SVN repository uses the FSFS backend (highly recommended), you can run Trac with read-only access to the repository. If that's still not sufficient, you should be able to share the SVN repository read-only via NFS to the Trac server.

[anonymous]

Any experienced IT administrator knows that keeping multiple services on one box does not "save work". This is the single reason why we cannot adopt Trac for any real project in our organization. We have an established SVN repository, processes, and security policy and there is no way we could install Trac on the SVN server. The SVN server is considered security and mission critical.

If there would be a way to somehow relay authorization from the user's desktop to the SVN server through Trac, that would be perfect. This way we could set up Trac without having to let our IT department set it up (which would take months to actually happen). We could update to a new release and apply patches whenever we need to. Currently, we cannot, since setting up Trac requires all-permissive access to the SVN repository and that means it would have to be administered by the IT guys. If Trac could offload all real authentication and access control to SVN, our IT would be happy.

Anyway, just the possibility to access a remote SVN repository is needed for our IT to even consider setting up a Trac server.

[jwin]

regardless of the status of the swig-python bindings, if I want to test the svn_ra access, is

trunk/trac/versioncontrol/tests

the right place to look for modifying to use svn_ra access ?

Has anyone done this recently ?

[jwin]

how about using a Python-extension written in C the C-Api for svn_ra should work

[jw]

svn 1.3rc7

• new public APIs:
  • new transfer progress callback for DAV (r15948)
  • svn_ra_initialize(), svn_client_open_ra_session()
  • svn_fs_closest_copy(), svn_fs_type()
  • several rev'd APIs, see doxygen docs
• SWIG bindings: No more compile-time or runtime SWIG dependencies
  • SWIG/python bindings:
    • automatic memory management: APIs no longer require pool

[Mitch]

Any progress on this? I'd love to use this feature.

[cmlenz]

WTF?

[anonymous]

Whats the eta on this feature? We have been using trac for about 1 year and would like to decouple Trac from the svn repository because we do not wish to give out mass accounts to the machine hosting our repo, for obvious security/stability reasons.

[eblot]

#2828 has been marked as a duplicate.

[caio1982]

Any progress on making it happen? Does anyone above got some err... "hack" that fully works? Mirroring data using SVK or even DRBD (!) is not a real option and NFS is... well, you know.

[anonymous]

 pysvn seems to have support for accessing remote repositories. And these bindings looks a lot more "active" too.

[cboos]

I've noticed the pysvn suggestion when it was made. This, combined with the VcRefactoring#NewRepositoryCache changes, would make it practical to work with remote subversion repositories, as remote requests would only have to be done for doing changeset synchronizations.

[anonymous]

I want to set up Trac for a SourceForge? project, but I can't until this feature is added. I hope it happens soon.

[Barry Scott]

pysvn indeed can access remote repos. Is there anything missing from pysvn API that would prevent you using it to solve this problem? (Apart from the pain of changing from svn bindings to pysvn that is!)

Barry - pysvn author

[caio1982]

I think it's a quite important feature that would make Trac even more flexible so it's not that good change this ticket's priority to "low" instead of "normal". Workarounds or extensions are acceptable but aren't the best solution, IMHO.

[anonymous]

Lack of this feature is the main reason why my new employer's development department doesn't use Trac (a pretty big international company). I've used Trac on the previous job with great success and I'm very sad I can't use it now.

[anonymous]

We are a small team in a big company evaluating Trac. This single issue is the reason why we won't use it companywide.

[mgood]

Feel free to bump up the priority if you're volunteering to implement it. Otherwise you'll need to be patient.

[trac@…]

In terms of implementing this, I was thinking it would be easy enough to write a new version control module (i.e. svn_pysvn) that uses pysvn instead of subersion's python bindings.

Does that make sense ? Are there any pitfalls to doing it this way ?

[cboos]

This was already acknowledged as being a good idea. See comment:38. The point of waiting for the upcoming refactoring would be to have much less work to do in order to write this new backend, as most of the operations will be done on the cache, and basically only the synchronisation of new changes will have to be backend-specific (at least that's the plan).

Nevertheless, I you feel like doing so, it would be perfectly possible to write a pysvn backend right now for milestone:0.10.

[lists01@…]

just a "me too" re: support for remote SVN repositories. we want to use trac but our sysadmin team already has a stand-alone SVN server set up.

the "on-the-same-box" requirement may be fine for small teams (or teams with that kind of independence), but in large organizations where there is a separation of responsibilities this requirement can be a show-stopper.

[cboos]

Among the possible workarounds, there's also svnsync, which is a new tool part of Subversion 1.4. It allows to easily replicate repositories, from URL / to URL, which gives you a great deal of flexibility.

For example, you can set up a post-commit hook on the central repository that will trigger the replication to a slave repository on the server where you run Trac. Alternatively, on that Trac host, you can periodically call svnsync which will pull new revisions from the central repository.

I've tested that tool with 1.4.0rc5, and it works fine. Of course you could have done the above using svk mirror, but svnsync is more straightforward to install and use.

[Shun-ichi Goto <shunichi.goto@…>]

I guess there is a way to access remote repository by using standard python binding of subversion 1.4 like svnsync (written by C) does. It may require smaller change than implementing pysvn backend. Of course, it will have limitation that both trac side and remote server side need svn 1.4 or later.

[chris@…]

Any news on this? I'm wanting to use Trac for a SourceForge? project but cannot install it on the server since I don't have  file:// access to the repository. I could mirror the repository but if I'm going to do that I might as well just move my repository to my VDS which I rather not do :(

We use Trac at work and are extremely happy with it. We found it's made our lives impressively easier and tend to spend a lot of time reloadng the timeline page. I think having remote SVN capabilities would sell this to the masses 100%.

[sik0fewl]

Another "me too." Has there been any recent progress on this ticket? The suggestion by Shun-ichi Goto seems like it will work well, at least for subversion 1.4 and greater.

[Shun-ichi Goto <shunichi.goto@…>]

I'm working for a week to implement this backend using Subversion RA API with standard python binding. Now the code, called svn_ra, is implemented as plugin and tested to have same behaviour and result with svn_fs. It seems to work for some small project but not yet tested with big one. So it is alpha state. There are many things to be implemented and improved, especially speed up of resync.

I'm placed it in public. You can try it if you are interested in.
 http://www.meadowy.org/~gotoh/projects/remote-svn-plugin/wiki
But don't forget that it should not be used for actual project yet. I'll continue to improve it.

BTW, I've also implemented with pysvn before using RA. It is very costfull because it uses client API (not RA) and client API makes session every API call. I found it is intended to replace client program (wc operation) then I gave up using it.

[cboos]

See the mailing list thread starting with  this mail.

We'll see in milestone:0.12 if we will integrate Shun-ichi Goto's svn_ra.py backend in Trac core, or not. In the meantime, please give a try to his remove-svn-plugin and help him improving it.

[anonymous]

removed my email from CC

[Dan <trac@…>]

Replying to cboos:

See the mailing list thread starting with  this mail.

We'll see in milestone:0.12 if we will integrate Shun-ichi Goto's svn_ra.py backend in Trac core, or not. In the meantime, please give a try to his remove-svn-plugin and help him improving it.

I started using it, but I have not yet understood how to configure/update it with user authentication options. This is a stumbling point for me, because the remote repository(s) I want to target are not visible via anonymous access. The standard bindings want to use a callback function for this, but they don't seem to accept a Python function for this purpose. I'm investigating SWIG in general, but so far haven't found a way to implement this capability. (The document I read indicated this kind of callback is not supported.)

I took a look at pysvn, which seems that it would be very simple to add in the authentication callback function(s), although I haven't tried it yet. However, the Trac remote backend using pysvn is apparently a "dead end", as indicated above in comment:72 .

By the way, much thanks to Shun-ichi Goto for getting this feature going! It will be extremely useful IMHO.

[ryanmccue@…]

This is the one and only reason stopping me from using Trac. I've used Trac on previous projects and it rocks, but as my host doesn't support SVN repositories, I'm keeping them on Google Code and SourceForge?.

[anonymous]

Another me too. This feature would help me for some sourceforge projects and also in my company.

[anonymous]

Adding myself to the CC list - we need remote SVN access - I'm going to try the one that's been advertised on this ticket, but I'm worried about the alpha state of it. Luckily, all of our repositories have anonymous access, so we don't need svn auth for trac's interface. I wish you folks would up the priority on this one - I'd love to see it integrated in Milestone 0.11. Thanks for the great tool anyway!

[anonymous]

Replying to anonymous:

Adding myself to the CC list - we need remote SVN access - I'm going to try the one that's been advertised on this ticket, but I'm worried about the alpha state of it. Luckily, all of our repositories have anonymous access, so we don't need svn auth for trac's interface. I wish you folks would up the priority on this one - I'd love to see it integrated in Milestone 0.11. Thanks for the great tool anyway!

One more note: the primary reasons why we need remote access are not the usual IT department political reasons. Our team works on multiple projects that are scattered over a number of open source repositories - everything from eclipse plug-ins (and if you know anything about eclipse, then you know that their motto is, "IBM's way or the highway") to sourceforge projects to our own internal forge here at Novell. Trac is great, but we really need remote svn repositories.

[anonymous]

will you implement this feature request?

[martin.marcher@…]

Replying to anonymous:

will you implement this feature request?

iirc it has been the decision that trac will wait for the python bindings to provide this functionality and then implement it.

Afaik the problem is that pythons svn bindings only provide the functionality trac needs for file:/// URIs and not other protocols...

[cboos]

One additional note for the svnsync solution (see comment:64):

It might be useful to also run a svnsync copy-revprops command, to catch the changes to revision properties, see  the svn book.

The output of that command should be parsed in order to get the list of modified changesets and then you should call trac-admin <env> resync $rev on those (if using Trac 0.10.4 or above), unless you already have a post-revprop-change hook which does this and svnsync copy-revprops calls that hook (I haven't verified that it calls it, but I assume it does).

[madduck]

The spam filter requires me to enter something here or else I can't just remove myself from Cc. Weird.

[radix@…]

You're lucky. I'm the reporter so I can't stop getting these god damned emails.

[cboos]

Replying to radix@twistedmatrix.com:

You're lucky. I'm the reporter so I can't stop getting these god damned emails.

Just ask ;-)

Btw, I started to write a more detailed step-by-step guide about how to setup a svnsync solution, I'll follow-up here when it's done.

[cmlenz]

(very sorry everyone, need to test config change to limit the amount of mails being sent out)

[anonymous]

Another me-too. I need svn+ssh remote access.

[yuangang28@…]

Replying to radix at twistedmatrix.com:

It stinks pretty bad that I have to install trac on my Subversion server AND give it write access to it.

[eblot]

Replying to radix at twistedmatrix.com: It stinks pretty bad that I have to install trac on my Subversion server AND give it write access to it.

Use trac with a remote Subversion repository;I want to have the character too

Please do not alter the ticket summary!

[Joymarquis]

I also have such requirement because we share source from a read-only SVN server that is owned by another department.

[eblot]

Replying to Joymarquis:

I also have such requirement because we share source from a read-only SVN server that is owned by another department.

If you have only a read-only access to the SVN repository you want to connect to Trac, an easy workaround is to create a local mirror of that remote repository and to keep your local repository in sync with a cron script.

[anonymous]

wasnt this scheduled for 0.11 ? its like the most needed feature in trac - thousands of sourceforge developers would turn to trac overnight if this was done... keeping in mind these are all OPEN SOURCE developers meaning quite a few would help out with trac development as well....

[eblot]

Replying to anonymous:

wasnt this scheduled for 0.11 ?

Read the milestone field...

[anonymous]

i wonder why you don't use:  http://pysvn.tigris.org/ doesn't it have all necessary features?

[anonymous]

I've come up with a workaround for our company. Our subversion server already has viewvc installed on it (any web based subversion viewer will work) so I used the  NavAddPlugin to add a link to point to the viewvc instance and then edited mainnav in my trac.ini to put the new menu item in the right place.

[anonymous]

for a plugin that does this with an svnsync done per request, see  http://trac-hacks.org/wiki/SvnsyncPlugin

[richard@…]

Thanks for trac! I really like this project.

[mike@…]

+1 on the interested list.

[post@…]

Maybe you could priorize this a bit more (please). Track 0.13 seems so far away. Thanks

[hieult@…]

The remote SVN can adapt easily on Windows by using UNC path.

I have tried the  http://www.meadowy.org/~gotoh/projects/remote-svn-plugin/wiki but seem it is buggy, I can get it running, but problem with syncing. Maybe because of the revision history is big.

[ilkka.laukkanen@…]

another potential user here.

[trac.edgewall.com@…]

SPAM filter wants something here

[trac.edgewall.com@…]

SPAM filter wants something here

[trac@…]

SPAM filter wants something here

[andyh]

Is is really time to fix this now!

+1 for needed.

[jesteves@…]

Not longer interested.

[joymarquis@…]

Remove self from the cc list. Thanks for the work.

[lists@…]

Adding a comment to try to satisfy the spam filter so I can unsubscribe

[lists@…]

Crap! it didn't work. How do I get off this list?

[trac-trac@…]

Not longer interested.

[dak@…]

Too much spam from this list, attempting unsubscribe.

[anonymous]

Replying to anonymous:

jasonm23@… - remove please.

[anonymous]

Replying to lists@…:

lists@… - remove please

[anonymous]

Too many people are asking for this for an anonymous "wont fix" to be appropriate. You may not fix it, but it is an Open Source project, therefore someone else may fix it. At least leave an explanation of your decision to close it.

I am reopening it.

[anonymous]

+1, really want this feature badly

[ilkka.laukkanen@…]

removing cc

[andy@…]

This would be useful for me to make the fine grained permission sharing between svnserve, the Apache SVN authz module and Trac automatic, and to mean that I can think about restricting direct file access to the SVN repos for security reasons. +1, for what it matters ;)

[bbdakraxor@…]

I can't believe this ticket has been around for 5 years. This feature would be useful for several reasons, most of them are mentioned above in the comments. Python is one of the few languages I'm not really familiar with, otherwise I'd go ahead and do it myself. I don't want to quit using Trac but right now this is causing me serious problems. If I were competent, I'd raise priority and severity. Please consider.

+1 user requesting this feature!

[matthew.block@…]

Remove me from the CC

[bbdakraxor@…]

Forgot to add: can't thank the developers enough for this GREAT project!

[cboos]

There are many possible solutions to this ticket.

• implement a new VersioningSystemBackend
  • using Subversion SWIG Python bindings, the svn_ra layer API. This has been tried by Shun-ichi Goto, it worked somehow, with some weak points due to bugs in the bindings, IIRC. Unfortunately, that plugin doesn't seem to be available anymore.
  • using Subversion ctypes bindings (with svn 1.5).
  • using the PySVN bindings (not a good idea according to comment:72) Any of the above is possible, and if a good working implementation emerges, then I'm sure it can find its way into Trac proper.
• workaround the limitation by replicating the remote svn repository
  • have a post-commit-hook in the main repository which triggers a svnsync to the Trac-dedicated repository (comment:64)
  • trigger a svnsync from Trac ( TracHacks:SvnsyncPlugin, comment:117)
  • mirror the svn repository using git or Mercurial and have Trac use the mirror

So yes, using a remote Subversion repository is not yet as easy as putting an  http://... URL in the repository_dir, but there are solutions.

I think there are enough +1 voices here so that there's no need to add more... Please consider adding to this ticket only if you want to share with the numerous people on the CC: list a new information about this issue (in particular, an update from Shun-Ichi would be more than welcomed).

[Felix Schwarz <felix.schwarz@…>]

Replying to cboos:

• using Subversion SWIG Python bindings, the svn_ra layer API. This has been tried by Shun-ichi Goto, it worked somehow, with some weak points due to bugs in the bindings, IIRC. Unfortunately, that plugin doesn't seem to be available anymore.

I think it still is - new repository seems to be  http://www.meadowy.org/~gotoh/hg/remote-svn-plugin/

[linux-madagascar@…]

Who'se company/employer would be ready to pay some freelance in order to work on this? I estimate the work time 0.5~0.75 month...

[Thijs Triemstra <info@…>]

big +1, especially when the idea has been around for 5 years. Subversion marked the ctypes python bindings integration as  one of their Google Summer of Code projects, so people that are interested in getting paid to fix that should check it out.

[anonymous]

ugh. really? no activity for 7 months? ok. fine. had enough, off to redmine.

[cboos]

Did you even try to use the  http://www.meadowy.org/~gotoh/hg/remote-svn-plugin?

[cboos]

... and speaking about Redmine, I just had a look and they're just using the svn command line, which is, how to say it nicely, perhaps the smartest way to avoid the nightmarish aspect of the bindings, the associated memory leaks and the various limitations including the lack of remote access discussed here (not to mention the numerous installation horror stories...).

So an other alternative would be to just do the same and have a backend based on the svn command line, somewhat like the GitPlugin? does.

[anonymous]

yes that would be great! and you can avoid many things. We had a major network reorganization and since then we are still trying to get the subversion support up an running again (windows domain issues). Could this be done in 0.11.x or 0.12?

[sorin.sbarnea@…]

+1 for this (I can't use the vote feature because I cannot create an account)

[anonymous]

A svn+ssh would be great, because I can't install anything on on the svn server...

[anonymous]

Replying to anonymous:

A svn+ssh would be great, because I can't install anything on on the svn server...

Could you explain more? Currently, I have a SVN server runs in difference Server, I only access via svn protocol( svn://<IP>/). My Trac runs in on others one, I am trying to connect them together, but fail.

[jae+trac@…]

Hmm, weird... there's a "Milestone changed from 0.11 to 0.12" and a "Milestone changed from 0.13 to 2.0", but I don't see a 0.12->0.13?

Oh, and doesn't "2.0" sound a lot like "probably never"?

[rblank]

Replying to jae+trac@…:

Hmm, weird... there's a "Milestone changed from 0.11 to 0.12" and a "Milestone changed from 0.13 to 2.0", but I don't see a 0.12->0.13?

That's #5658.

Oh, and doesn't "2.0" sound a lot like "probably never"?

Quite close, until either:

• a core developer has the need for the functionality, at which point he may choose to implement it, or
• someone champions the functionality and provides a complete patch that integrates cleanly with trunk.

So far, none of these has been the case.

[cboos]

Milestone 2.0 deleted

[cboos]

(according to the nice summary given by Remy in comment:163)