Edgewall Software

Version 28 (modified by Remy Blank, 15 years ago) ( diff )

Comment on repository name limitations, and some updates.

Multiple Repository Support for 0.12

This branch adds support for accessing multiple repositories inside one Trac project.

For now, there is support for the Mercurial (hg) and Subversion (both svn and direct-svnfs types of repositories).

See:

Checkpoints:

  • r6465: initial "release" of the branch, worksforme
  • r6469: a few bug fixes
  • r7590: that version was quite good
  • r7815: that version was also quite good ;-)
  • r7961: initial support for cached repositories
  • r8178: current version - should be "stable" (i.e. you can report bugs against that revision if you find some). Note that there's now support for cached repositories, and as this involved a schema change, an environment upgrade is needed.

The approach is quite simple: beyond repository connectors, there's now repository providers (IRepositoryProvider components) which are responsible for make available a list of named repositories. That name can be arbitrary (in particular, it may contain '/' to denote a pseudo-hierarchy) and will be used as a prefix for any repository path. Actually, a more complete description of the approach can be found in this mail: ​One trac env for X repositories ?.

If the only repository known is the default one (i.e. the one defined in TracIni, section [trac] repository_...), then Trac behaves exactly as before. But IRepositoryProvider components can make other repositories available. In particular, the default RepositoryManager itself provides a list of repositories listed in a new [repositories] section in TracIni. See r6447 for more details. Since r7969, trac-admin gained some commands to manage list of repositories and their aliases, storing them in a dedicated table in the Trac database.

Finally, repositories of type svn are now supported. This means there will be a cache created for each of those repositories. See MultipleRepositorySupport/Cache for details.

Related Tickets

Status: closed (35 matches)

Ticket Summary Component Milestone
#6654 Changeset view is slow when using sub repository path version control 0.12
#7116 fine-grained permissions are not applied to directories when browsing sources version control 0.12-multirepos
#7706 ValueError: too many values to unpack version control 0.12-multirepos
#7717 Multi-repo branch 'trac-admin resync' msg on one svn version control
#7743 Add/document commit-hook plugin/mercurial
#8203 Internal error when using multirepos, in search search system 0.12-multirepos
#8369 ValueError: too many values to unpack timeline 0.12-multirepos
#8758 Timeline filtering in Multirepos timeline 0.12-multirepos
#8808 KeyError: <Option [account-manager] "account_changes_notify_addresses"> general 0.12
#10144 [Patch] Make it easier to switch off commit messages in timeline timeline 1.2
#1781 Link from 'Browser Source' to Subversion Repository URL version control/browser 0.12-multirepos
#2086 better support for multiple repositories version control/browser 0.12-multirepos
#2482 document usage of post-revprop-change script version control
#3368 Repository subset support in commit hooks version control
#5640 SVN checkin comments float up past svn:authz barrier. version control/browser 0.12-multirepos
#5993 Confusing error message when repository_dir has changed. version control 0.12-multirepos
#6016 trac-admin resync doesn't update revision message after editing a svn revision log version control 0.12-multirepos
#6445 post-commit script example in trac-post-commit-hook is missing environment variable "TRAC_ENV" version control 0.12-multirepos
#6826 Excessive calls of filter.pre_process_request for one request version control 0.12-multirepos
#7723 Make repository cache work on multirepos branch version control 0.12-multirepos
#7822 Manage repositories in DB instead of trac.ini version control 0.12-multirepos
#8291 Create an admin panel for managing repositories admin/web 0.12-multirepos
#8465 Subversion authorization for Browser Source version control/browser
#8525 Multirepos Databse Upgrade from 0.11 Fails w/ PostgreSQL version control 0.12-multirepos
#8536 Don't display Default Repository on main Browse page version control 0.12-multirepos
#8572 AttributeError: 'sqlite3.Connection' object has no attribute 'get_db_cnx' version control 0.12-multirepos
#8690 Missing changeset link version control/browser
#8731 experiment with surrogate keys for the repository table version control 0.12-multirepos
#9038 Latest multirepos revision chokes on revision log links wiki system 0.12
#9090 Browser "Download as zip" is not Multi-repos aware version control/browser 0.12
#9112 Strange behavior with empty svn repositories version control/browser 0.12.1
#5189 Feature request: add --no-notification option to trac-post-commit-hook notification 0.12-multirepos
#6041 trac-post-commit-hook doen't work (nor error message) general 0.12-multirepos
#6370 Contrib post-commit for Win has issues with quotes in project name, failing version control
#5710 add link to last change to browse version control/browser 0.12-multirepos

Status: new (3 matches)

Ticket Summary Component Milestone
#2382 WebAdmin: resync from the web interface version control next-major-releases
#4048 Diff between repositories version control next-major-releases
#8700 patch for trac-svn-post-commit-hook.cmd version control next-stable-1.6.x

Installation (with mercurial)

A current installation procedure which works is the following:

  1. Check out the mercurial plugin: 
    svn co http://svn.edgewall.com/repos/trac/sandbox/mercurial-plugin-0.12
  2. build the mercurial plugin. From within the checked-out directory, build the egg file (it will end up in the dist subdirectory): 
    python setup.py bdist_egg
  3. Check out the multirepos branch: 
    svn co http://svn.edgewall.com/repos/trac/sandbox/multirepos
  4. Install the multirepos version of trac (from within the multirepos directory, as root): 
    python setup.py install
  5. Create your trac environment directory
  1. Copy the mercurial plugin egg file to the <trac-env>/plugins directory
  1. make sure the following is somewhere in <trac-env>/conf/trac.ini: 
    [components]
tracext.hg.* = enabled
  2. Add all your repositories in the Repositories admin panel. Alternatively, you can add the repositories to the [repositories] section of trac.ini, but note that this has some restrictions on the repository names (i.e. lowercase, no '.'): 
    [repositories]
my_repo_a.dir = <path_to_a_repository>
my_repo_b.dir = <path_to_another_repository>
my_repo_c.dir = <path_to_yet_another_repository>
my_repo_c.type = svnfs

[trac]
repository_dir = <path_to_default_repository>
repository_type = hg
    Note that when the repository .type is not specified, the value of [trac] repository_type is used.
    In the above example, this means that my_repo_a and my_repo_b are Mercurial (hg) repositories.
  3. Synchronize all your repositories: 
    trac-admin <trac-env> repository resync "(default)"
trac-admin <trac-env> repository resync my_repo_a
trac-admin <trac-env> repository resync my_repo_b
trac-admin <trac-env> repository resync my_repo_c
  4. Install post-commit hooks for all your repositories, and have them call trac-admin <trac-env> changeset added for every new changeset: 
    trac-admin <trac-env> changeset added my_repo_c $REV
  5. Cross your fingers, and hope it works :)
    It really should or it's a bug ;-)

To make links to the non-default repository, just add a slash and the alias name after the revision identifier, for example: "[changeset:cffd9e480266/my_repos_b a changeset]". The default repository can be one of the explicitly named repositories.

TODO

Major

  • split repository names at "/" in the repository browser (i.e. show only the a entry at toplevel when we have repositories a/one and a/two, show one and two when expanding a)
  • currently, a "default" repository (the one defined in the [trac] section) needs to be defined, otherwise the versioncontrol modules are disabled r7972 - repository_dir entry needs simply to be present (can be empty)
  • fix AJAX query in annotate
  • IRepositoryProvider must have a way to signal new content, in order to rebuild the repository name cache - trac-admin ... repository notify <repos> changeset_added <rev>, see #7723. See sandbox/multirepos/contrib/trac-svn-hook.
  • support the .url repository metadata (#1781)
  • small issue: visual glitch in Firefox 2 for the new Rev column, where the changeset icon doesn't align properly to the right (but works fine for every other web browser, including Firefox 3) FF 2 is now dead, isn't it?
  • "normal" feature: it's currently not possible to compare across repositories (the situation is detected, but raises a "not implemented" error). It would be quite useful to be able to do so, and it's doable without much work (only need to compare folders in a backend neutral way, we already do our own diff at the file level)
  • extend to cached repositories, see MultipleRepositorySupport/Cache and #7723
    In a first step, the existing cache could be adapted to simply handle more than one repository.
    Bonus points if that cache is smart enough to handle multiple scoped repositories without data duplication.
    A second (much) more ambitious step would be to generalize the cache so that it can support other (d)vcs.

Minor Points

  • the browser quickjump menu wants to go to the repository youngest (svn), not the youngest cached so far. There might be more places like that, we need to deal consistently with the youngest cached. fixed in r8396
  • browser expansion doesn't work for the repository index. I'm pretty sure it used to work at some point. fixed in r8076.
  • More IRepositoryProvider components, though this is becoming less pertinent now that we manage the repositories in the db and the repository table:
    • HgForestRepositoryProvider, for providing forest repositories and all sub-repositories managed by each forest
      • work-in-progress on ​http://hg.trbs.net (stalled)
      • in trac.conf use: 
        [hg-forest]
jdk7.dir = /path/to/mercurial/root/forest/repository/jdk7
    • HgParentDirRepositoryProvider, for listing all the working copies present in a given folder
      • work-in-progress on ​http://hg.trbs.net (stalled)
      • in trac.conf use: 
        [hg-parents]
bunch.dir = /path/to/parent/directory/somedir
    • SvnParentDirRepositoryProvider, for listing all the Subversion repositories present in a given folder
  • Add an admin panel to manage repositories (#8291). Complete as of [8257].

Other Open Points

taken from #6665:

A few comments / questions

  1. How can I get the name in the Repository Index to keep the case in the name? It is converted to lower case.

ConfigParser limitation (#7378).

  • rblank: Note that this is less of a concern now that repositories can be added from the web interface.
  1. Since you use the init tag as the name to show in the Repository Index, it seems I cannot have a period in the name of the repository as shown.

Also related to the way the the .ini keys are read. It would be possible to look first for the ending .dir part and then get the name.

  • rblank: Again, not an issue anymore with the web interface.
  1. I get a default in the Repository Index, as well as that same repository listed in the previous single repository manner. I think that when the multiple repositories are listed, and one of them is the same as the trac repository_dir, it does not need to be listed twice or three times.

Yes, known issue: (verify)

  • one possibility would be to not list aliases (even aliases for the [trac] repository_dir). I think that's acceptable, but there should be an easy way to learn about the defined aliases, maybe they can be listed at the bottom of the toplevel page.
  • OTOH, aliases might be defined simply because they are more convenient names than the "real" ones, so in the end, more flexibility is needed. Add a .browseable flag, a la Samba?

Other than that, so far this is going to meet my needs quite nicely. Thanks!

Thanks :-)

Note: See TracWiki for help on using the wiki.