= Multiple Repository Support for 0.12 = {{{#!div style="background: #ffd; padding: 0.5em 1em" This page details the development process that added support for accessing multiple repositories inside one Trac project. The main part of the work has been completed and the sandbox branch has been merged into trunk. The corresponding documentation can be found in ["0.12/TracRepositoryAdmin"]. This page still discusses a number of items that can be implemented in future Trac versions. }}} For now, there is support for the Mercurial (`hg`) and Subversion (both `svn` and `direct-svnfs` types of repositories). See: - ~~log:sandbox/multirepos~~ log:trunk?stop_rev=9125 ''FIXME: log:...?..' links are broken..." - ~~log:plugins/multirepos/mercurial-plugin~~ - log:plugins/0.12/mercurial-plugin for 0.12dev after r9125 - TracDarcs 0.8.1 - [http://groups.google.com/group/trac-dev/browse_thread/thread/33a535f760c3f3f1 multirepo support patch for GitPlugin] Checkpoints: - r6465: initial "release" of the branch, ''worksforme'' - r6469: a few bug fixes - r7590: that version was quite good - r7815: that version was also quite good ;-) - r7961: initial support for cached repositories - r8178: current version - should be "stable" (i.e. you can report bugs against that revision if you find some). Note that there's now support for cached repositories, and as this involved a schema change, an '''environment upgrade''' is needed. - r9084: real close to the end of the branch ;-) - r9125: branch landed on trunk; further stabilization work is now being done there. The approach is quite simple: beyond repository connectors, there's now repository providers (IRepositoryProvider components) which are responsible for making available a list of named repositories. That name can be arbitrary (in particular, it may contain '/' to denote a pseudo-hierarchy) and will be used as a prefix for any repository path. Actually, a more complete description of the approach can be found in this mail: [googlegroups:trac-users:14ca95377e4a53b5 One trac env for X repositories ?]. If the only repository known is the default one (i.e. the one defined in TracIni, section `[trac] repository_...`), then Trac behaves exactly as before. But `IRepositoryProvider` components can make other repositories available. In particular, the default `RepositoryManager` itself provides a list of repositories listed in a new `[repositories]` section in TracIni. See r6447 for more details. Since r7969, trac-admin gained some commands to manage list of repositories and their aliases, storing them in a dedicated table in the Trac database. Finally, repositories of type `svn` are now supported. This means there will be a ''cache'' created for each of those repositories. See MultipleRepositorySupport/Cache for details. == Related Tickets == [[TicketQuery(group=status,order=priority,col=summary|component|milestone,format=default,keywords=~multirepos)]] == Installation (with mercurial) == A current installation procedure which works is the following: 1. Check out the mercurial plugin: {{{ svn co http://svn.edgewall.com/repos/trac/plugins/multirepos/mercurial-plugin }}} 2. Build the mercurial plugin. From within the checked-out directory, build the egg file (it will end up in the dist subdirectory): {{{ python setup.py bdist_egg }}} 3. Get Trac 0.12 or later. ~~Check out the multirepos branch: {{{svn co http://svn.edgewall.com/repos/trac/sandbox/multirepos}}}~~ 4. Install Trac; see TracInstall. ~~Install the multirepos version of trac (from within the multirepos directory, as root): {{{python setup.py install}}}~~ 5. Create your trac environment directory. 6. Copy the mercurial plugin egg file to the /plugins directory, or use {{{easy_install }}} for global installation. 7. Make sure the following is somewhere in /conf/trac.ini: {{{ [components] tracext.hg.* = enabled }}} or use the 'Admin' panel in the browser to enable the plugin. 8. Add all your repositories in the ''Repositories'' admin panel. Alternatively, you can add the repositories to the `[repositories]` section of `trac.ini`, but note that this has some restrictions on the repository names (i.e. lowercase, no '.'): {{{ [repositories] my_repo_a.dir = my_repo_b.dir = my_repo_c.dir = my_repo_c.type = svnfs [trac] repository_dir = repository_type = hg }}} Note that when the repository `.type` is not specified, the value of `[trac]` `repository_type` is used (`[versioncontrol]` `default_repository_type` since Trac 1.2). [[br]] In the above example, this means that `my_repo_a` and `my_repo_b` are Mercurial (hg) repositories. [[br]] 9. Synchronize all your repositories, either with a wildcard: {{{ trac-admin repository resync '*' }}} or, if you want, explicitly: {{{ trac-admin repository resync "(default)" trac-admin repository resync my_repo_a trac-admin repository resync my_repo_b trac-admin repository resync my_repo_c }}} 10. Please note that at the time of writing, no initial resynchronization or any hooks are necessary for Mercurial repositories - see #9485 for more information. However, if you wish, you can install post-commit hooks for all your repositories, and have them call `trac-admin changeset added` for every new changeset: {{{trac-admin changeset added my_repo_c $REV}}} However, why you would want to do this in a normal 0.12 installation is beyond me. 11. Cross your fingers, and hope it works :) [[br]] ''It really should or it's a bug ;-)'' [[br]] Mercurial works fine without hooks and resynchronization. To make links to the non-default repository, just add a slash and the alias name after the revision identifier, for example: "![changeset:cffd9e480266/my_repos_b a changeset]". The default repository can be one of the explicitly named repositories. Note that Step 9 & 10 are not needed for Mercurial integration, because TracMercurial will access Mercurial repository structure when any user view the version control system via Trac. == TODO == === Merge === Now with #7116 nearly done, it's time to plan the merge. * What remains to be done is mainly documentation, as when people using `trunk` will get the multirepository changes, they'll need to find out what to do, in ["0.12/TracRepositoryAdmin"]: - mainly what they need to do for adapting their repository hooks - what they can do with their default repository (setting up an alias for default, or better, "migrating" their unnamed default repository to a named repository and add an alias to the empty repo pointing to the new name) - trac-admin (repository and changeset commands) - hidden and url attributes - admin panel intro ''A first version is now [wiki:0.12/TracRepositoryAdmin available], and should hopefully be sufficient for the merge. It could still be improved before releasing 0.12.'' * merge done in r9125 Not strictly needed before the merge but would be nice: - #7743 for mercurial - should add a contrib file there, containing a `hook` function calling the Trac API appropriately, and have an example of a command hook calling trac-admin directly. - A separate script is not strictly necessary (at least on Linux). The following section in `hgrc` does the trick: {{{ [hooks] commit = trac-admin /path/to/env changeset added repo_name $HG_NODE changegroup = hg log -r $HG_NODE: --template "{node} " | xargs trac-admin /path/to/env changeset added repo_name }}} This should be documented in TracMercurial, though. OTOH, for Windows we will need a Python hook anyway, so we could as well create a generic one. - Mercurial Repositories Please note that at the time of writing, no initial resynchronization or any hooks are necessary for Mercurial repositories - see #9485 for more information. === Major === - split repository names at "/" in the repository browser (i.e. show only the ''a'' entry at toplevel when we have repositories `a/one` and `a/two`, show ''one'' and ''two'' when expanding ''a'') - ~~currently, a "default" repository (the one defined in the `[trac]` section) ''needs'' to be defined, otherwise the versioncontrol modules are disabled~~ r7972 - `repository_dir` entry needs simply to be present (can be empty) - ~~fix AJAX query in annotate~~ - ~~IRepositoryProvider must have a way to signal new content, in order to rebuild the repository name cache - `trac-admin ... repository notify changeset_added `, see #7723.~~ See [source:sandbox/multirepos/contrib/trac-svn-hook]. - support the `.url` repository metadata (#1781) - ~~ small issue: visual glitch in Firefox 2 for the new Rev column, where the changeset icon doesn't align properly to the right (but works fine for every other web browser, including Firefox 3) ~~ FF 2 is now dead, isn't it? - "normal" feature: it's currently not possible to compare across repositories (#4048) (the situation is detected, but raises a "not implemented" error). It would be quite useful to be able to do so, and it's doable without much work (only need to compare folders in a backend neutral way, we already do our own diff at the file level) - extend to '''cached repositories''', see MultipleRepositorySupport/Cache and #7723 [[br]] ~~In a first step, the existing cache could be adapted to simply handle more than one repository.~~ [[br]] Bonus points if that cache is smart enough to handle multiple scoped repositories without data duplication. [[br]] A second (much) more ambitious step would be to generalize the cache so that it can support other (d)vcs, but this is somewhat out of scope of the MultiRepos task itself. === Minor Points === - ~~the browser quickjump menu wants to go to the repository youngest (svn), not the youngest cached so far. There might be more places like that, we need to deal consistently with the youngest cached.~~ fixed in r8396 - ~~browser expansion doesn't work for the repository index. I'm pretty sure it used to work at some point.~~ fixed in r8076. - More [changeset:6448 IRepositoryProvider] components, though this is becoming less pertinent now that we manage the repositories in the db and the `repository` table: - !HgParentDirRepositoryProvider, for listing all the working copies present in a given folder - work-in-progress on http://hg.trbs.net ''repo here is not up-to-date, but code is and works on latests multirepos tree, waiting for the opportunity to submit the provider back into mercurial-plugin'' - !HgForestRepositoryProvider, for providing forest repositories and all sub-repositories managed by each forest - work-in-progress on http://hg.trbs.net ''(stalled in favor of waiting for hg 1.3 subrepos work)'' - !SvnParentDirRepositoryProvider, for listing all the Subversion repositories present in a given folder - ~~Add an admin panel to manage repositories (#8291).~~ Complete as of [8257]. === Other Open Points === taken from #6665: > A few comments / questions > > 1. How can I get the name in the ''Repository Index'' to keep the case in the name? It is converted to lower case. !ConfigParser limitation (#7378). * rblank: Note that this is less of a concern now that repositories can be added from the web interface. > 2. Since you use the init tag as the name to show in the ''Repository Index'', it seems I cannot have a period in the name of the repository as shown. Also related to the way the the .ini keys are read. It would be possible to look first for the ending `.dir` part and then get the name. * rblank: Again, not an issue anymore with the web interface. > 3. I get a ''default'' in the ''Repository Index'', as well as that same repository listed in the previous single repository manner. I think that when the multiple repositories are listed, and one of them is the same as the trac repository_dir, it does not need to be listed twice or three times. Yes, known issue: (''verify'') - one possibility would be to not list aliases (even aliases for the `[trac] repository_dir`). I think that's acceptable, but there should be an easy way to learn about the defined aliases, maybe they can be listed at the bottom of the toplevel page. - OTOH, aliases might be defined simply because they are more convenient names than the "real" ones, so in the end, more flexibility is needed. Add a `.browseable` flag, a la Samba? * there's a `.hidden` flag now (and a corresponding `hidden` attribute on Repository that can be set via the admin panel or via: {{{ trac-admin ... repository set hidden true }}} > Other than that, so far this is going to meet my needs quite nicely. Thanks! Thanks :-)