Multi-Project Support for Trac

The current design of Trac associates one project to one environment.

Typically software projects tend to split themselves in separate sub-projects, sharing common libraries and components. In enterprise setups, it is practical to consolidate the development knowledge base in one Wiki, yet to be able manage the different products or family of products independently. Moreover developers are often working on multiple products at once, and would like to be able to have an overview of the projects they're involved in. At other times, the same developers would like to have the ability to fully focus on a single project. Widening the scope and looking at a whole family of products or even at the entire set of projects is also useful when one wants to get an overview or doesn't know where to start searching for information.

Trac has to be adaptable to these different views. The current approach of setting up different Trac environments only allows to get a fixed view on one project. By using InterTrac links, it is possible to relate resources one to another across Tracs, but the interaction and sharing of information couldn't go beyond linking: no common search or ticket queries, no shared timeline, no possibility to conveniently move resources from one project to the next while retaining the full history, etc.

In addition, for some database back-ends, the single project per environment design impose practical limits to the number of projects that can be run concurrently due to connection sharing concerns (#8058).

The next step is therefore to provide the ability to create multiple Trac projects and sub-projects inside a single environment and to offer some convenient ways to form "views" on which projects should be active for a given request.

In our usual tradition of backward compatibility, when an environment only contains the top-level project everything should continue to work like it always did. This will allow alternate implementations of multiple project setups to continue working unmodified. See the various TracMultipleProjects sub-pages, esp. TracMultipleProjects/SingleEnvironment which contains a proposal quite similar to this: adding a project field and a Projects table.

The Project Tree

A very natural way to group projects and sub-projects is to use a hierarchy of projects. The top-level project contains all the other projects, the leaf projects can focus on some appropriate subset of the development activity (e.g. a given library) and the intermediate project nodes in this project tree are used to group related sub-projects together.

This approach accommodates with the simplest case of the top-level project being the only project, supports quite well the situation of having many independent projects by creating all projects as sub-projects of the top-level project, but also scales to more complex development environments, e.g. where there are different families of products and possibly customer specific sub-projects for each of the products.

Here are some typical setups:

Single project

• "" or "/", the top-level project

Several independent projects

• "" or "/", the top-level project
  • "project_a"
  • "project_b"
  • "project_c"

Hierarchy of projects

• "" or "/", the top-level project
  • "Family A"
    • "Family A/product a"
    • "Family A/product b"
  • "Family B"
    • "Family B/product c"
    • "Family B/product d"
  • "Tools"
  • "3rdParty"
    • …

The hierarchy doesn't limit the possible ways of grouping projects. In the example above, it should still be possible to search for tickets in "Family A" and "Family B" but not in "Tools" and "3rdParty" when needed, using a reasonably simple user interface.

Example:

Project Selector

• Top-level
  • Family A
    • product a
    • product b
  • Family B
    • product c
    • product d
  • Tools
  • 3rdParty

Focal project

However the hierarchy can help to intuitively select a meaningful subset of the existing projects.

To complement the project selector user interface shown above, we could also have a simpler interface (a selection) which would allow the user to pick a project which becomes the center of interest. This means selecting not only that project, but also its ancestor projects and sub-projects:

Family A ┗╴product a ┗╴product b Family B ┗╴product c ┗╴product d Tools 3rd Party

For example, if that center of interest is set to "Family B", then the selected projects will be the top-level one, "Family B" itself, "product c" and "product d", but not the siblings of "Family B", i.e. what has been hand-picked in the Project Selector example above.

Such a selection is meaningful because it usually makes sense to work at a given level and be interested in all what's happening in sub-projects but also to be aware of issues happening at higher level which may impact the current project and sub-projects as well.

In practice and for this example, this means that when looking for tickets, one would see all the tickets associated with sub-projects ("products c" and "project d"), but also tickets directly associated with the "Family B" project (e.g. installer issues), and tickets directly associated with the top-level project (e.g. bugs in the web site shared by all projects).

In the end, this is mainly a matter of user interface, and picking up any other subset of projects should be possible as well by just toggling a few checkboxes. Among other things, it should be possible to select only "Family B" without selecting its ancestor projects or sub-projects, to consider only tickets directly associated with this level.

Projects and milestones, versions, …

On a different level, the hierarchy can also be used to organize resource sharing between projects. For example, in the Ticket system we could imagine that milestones belonging to ancestor projects are also visible in their sub-projects. In our hierarchy example above, imagine setting up a milestone for a "B-1" release in the "Family B" project. In sub-projects "product c" and "product d", no redundant milestone setup needs to be created, "product c" issues could be directly targeted to this "B-1" milestone.

In the typical center-of-interest setup, looking at the roadmap when "Family B/product_c" is the center of interest, the "B-1" milestone will show up and the tickets for "Family B/product_c", "Family B" and top-level "/" will be shown. The tickets for "Family B/product_d" on the other hand won't be visible in that view.

Conversely, when "Family B" is the center of interest, the "B-1" milestone will show the tickets for "Family B/product_c" and "Family B/product_d" (all the sub-projects), "Family B" itself and the top-level ones associated to this milestone. One can see that this could involve the possibility of having multiple target milestones for a ticket, for example a top-level ticket needed for milestone "B-1" and "A-1".

In case the project selection is restricted to "Family B" alone (no ancestors, no sub-projects), then the "B-1" milestone will naturally only show the tickets belonging to the "Family B" project.

So we see that the current project selection is always used to filter the set of resources that are taken into account for any kind of data retrieval. The default project selection obtained by selecting a "center of interest" project (focal project), consisting of the project itself, its ancestors and its sub-projects provides a view on all the resources which should typically be relevant for someone who is working on that project.

With the "focal project" model, the users will also have a clear guideline for selecting which project should be associated with their resource (be it a ticket, wiki page, milestone, version, component, etc.): the project level at which the given resource is relevant for that project and all the sub-projects of that project. Or put differently, you could start with a leaf project and go up until the item is no longer relevant to all the sibling projects of the last level considered.

At this early stage of the proposal, this organization seems to be simple and intuitive, yet very powerful.

The Data Model

The main idea is to associate a resource to one or several projects. This can be achieved by using an extra project association table for each resource.

Some resources might be only indirectly related to a project, like the Changeset and Source which belong to a given repository (see MultiRepos). The repository object itself might be related to some projects.

Likewise, some resources might stay "global" for a while, like the Wiki pages, or even the Milestone and Version in the first steps. We'll first focus on the Ticket.

The empty string '' could be used to qualify the top-level project, which is always there. If no other project has been defined, then Trac behaves as usual, one environment is one project. As soon as other projects are defined, the environment becomes a multi-project one.

An additional project table is also needed, and it can take the form of a resource table specialization described in GenericTrac. Projects should have at least a description field, a creator and a status field. Projects can be linked explicitly to shared data. For example, each projects can be linked to its own set of repositories.

Project
field	uid	id	Parent	summary	description	Creator	created	status
DB type	int	text	int	text	text	int	int64	text
subtype	key	plain	references Project	wiki	wiki	references User	usecs	plain?

For SQL queries that may cover more than one project, a helper method can be used to form the include/exclude constraints, given the project pattern list that has to be used.

On the API side, the multi-project nature will be mostly transparent except when instantiating and manipulating resources. Legacy resources and legacy code not taking the project into account will simply fallback to accessing resources at the global level.

Web User Interface Changes

There should a standard way to change the current list of active projects: typically that would be a pull-down menu as described in #Focalproject above, somewhere near the top-left corner, for Mantis refugees. Next to this simple selector should be a (+) link to open a more advanced project selector (#ProjectTree) showing the projects "pre-selected" by the choice of the focal project, but in which additional projects could be opted in or others could be deselected.

For example, there should be a Projects main navigation entry, which corresponds to a page showing some details about the currently selected projects and pointing to the parent and children objects if they're not themselves selected. This would be much similar to the list of Milestones, and likewise selecting a project would go to that project summary page (#1).

An admin module can be used to create new projects (eventually copying settings from another one), renaming projects and deleting them.

Existing modules should progressively be made project aware and will filter their content according to the current selection of projects, perhaps highlighting the resources pertaining to the focal project.

Utilities

Scripts for importing and exporting projects between environments should be provided. This can be useful for archiving a project before deleting it, for merging multiple single environment Tracs into one, etc.

URL space for different projects

Several kinds of mappings can be defined for mapping URLs to projects, before normal request dispatching takes place. The two default ones are:

• host mapping: the name of the (virtual) host used to access the Trac environment is taken to be the name of the project. A given host can be configured to be an alias for the default, top-level project.
• prefix mapping: if the PATH_INFO starts with the name of a project, this will be taken to be the name of a project. If none such project is detected, then the request is targeted at the top-level project.

In addition to a single project specification, there can sometimes also be multi-project specifications in a given URL. The request parameter 'project' can contain more than one project, and if the request processor can take advantage of this, the scope of the request will cover those projects. The special value '*' can be used to cover all the projects and more generally can be used in a glob pattern to cover different projects at once, and a '!' prefix for a project name or pattern can be used to exclude those projects. Typical users of multi-project specifications are the Timeline, the Search, the Custom Query…

Data Sharing

As briefly mentioned above, some data structure will be accessible globally and are shared between project. This will be the case for the Version Control Subsystem, where the repositories and the associated data are global objects that can be accessed indifferently from different projects. Of course, projects can define which repository they want to cover, eventually none or just one. They can also specify which repository will be the default one for them (the reponame to use when no reponame has been explicitly specified in links or queries).

Session and user data will also be shared across different environments, to some extent: preferences at a global level can be overridden by preferences set at the project level.

Permission will also be shared, though the system of fine-grained permission can be used to limit the permissions to a given project.

Workflow

The question came up whether workflow should be per-project, or per-project group and how to reuse them (ticket:130#comment:157 and follow-up).

Having one workflow per project could quickly make the maintenance of workflows quite complex. To balance this complexity, in hierarchical setups, one could define a workflow at the upper level, and have sub-projects use the workflow defined closest in the hierarchy. For example, one could define a workflow on the root project, and every other project uses this. If one exception has to be defined for a given project, that project can define its own.

Alternatively (ticket:130#comment:159), workflows could be defined as independent entities, and a project will have to be associated to a given workflow (a bit like repositories will be associated to projects). Having workflows as separate entities would allow:

• sharing the same workflow between different projects, but still having the possibility to set another workflow for some specific projects;
• (might) allow (the user?) to specify explicitly how to manage moving a ticket from one installed workflow to another installed workflow - whatever the project;
• ease sharing workflows between users: having workflows encapsulated encourages implementation of several generic workflow solutions (that can be easily tweaked for specific use) that could be worked and provided by default or in a file repository - that would even ease to choose the starting workflow of a new Trac user because he could easily choose a base workflow instead of having to hack the default one, simply adding modifications each time it seems required;

Another option is a combination of both alternatives:

• Workflows are independent entities, that can be associated to projects, resource-realms, resource-types, and maybe even specific resources (I'm using "resources" instead of "tickets" in order to strive towards GenericTrac).
• When a workflow is needed for a specific resource, the workflow used is chosen to be the workflow entity that is the closest to the resource in the hierarchy.