Trac Relations

TracRelations is a general module that provides:

• an api to provide and manage 'strong' relations between any resource in Trac
• various visible 'Related content' renderers
• a foundation for personalisation services (eg. bookmarking)

What is it?

The background for the idea can be found in a large number of tickets that deal with relating information in one way or another, and more than just providing a link between resources inside the test. A 'strong' relationship as used in this document means a linkage that can be queried, and using various Trac APIs can also be acted on in various ways.

TracRelations shares much of its vision with other proposals such as TracObjectModelProposal, GenericTrac and TracCrossReferences.

What is different however, is that the implementation idea starts in more or less the opposite end and builds on the strengths of the existing APIs. As such, the implementation requires no real changes to any of the core entities or interfaces - only some API, model and visualisation additions.

The basic premise is not just for use in Trac, but first and foremost the the power and flexibility it will provide plugin developers to extend the Trac experience.

Basic datamodel

Before continuing, it may be useful to sketch out the basic datamodel - not as a set of fields with types, but more of listing the information that could be foreseen stored. The idea is to provide a flexible storage for relations that can be made between any resource, or even on just a resource without a destination allowing you to link a wikipage to a specific ticket 'component' enum for instance. Or a 'bookmark' type relation between a resource and a user.

Pseudo table definition:

resource_main	(realm, id, version)
resource_dest	(realm, id, version)
relation_type	(relation)
user
additional_id
additional_info
timestamp

However, the storage of details in the Trac database itself is not a requirement. The fetching of data for render and use is intended to be distributed, so that for instance a mailling list reader could just store a resource_main as realm=wiki without further details, and that will be enough for it to be called on any use from the wiki system - allowing it to search its index to locate threads that mention this particular page name for instance. Another simple visualisation of this is a wiki page hierachi relation maintainer that basically just read page names and splits them on '/' like the current pseudo-method.

That means it is not the definate source of relations in Trac. That answer can only be found by asking through the API.

The basic database table small enough, and simple enough to also be used for storage for modules with very limited storage needs. An example is storing tags for modules that don't provide their own - like for instance tagging of changesets, or attaching tags to files in the repository. Storing a tag would use the main resource, a relation type of 'tag' and the actual tag in the attribute field. One page tagged with several keywords would use several records to store it. In this scenario a tag may even be a resource onto its own, effectively linking two resources with a 'tag' relation.

More examples

To provide some better understanding of the basic idea, here are some use cases - but do note that it is not all intended as Trac features, more as illustrations of what some plugin may be able to do:

• Wiki rename: A relation type of 'renamed' that stores references of 'from' and 'to'. In the original place we can render info that it has been moved, and in the destination we can render 'previous names' as information.
• Maintaining or inferring branches of wiki pages - like we do for 0.10, 0.11 and need to do for future revisions. A relation_type of 'branch' from one resource to the other is very useful: We can list links to the other versions (branches) of the page - and help the user find the right version. The plugin that implements wiki branching may provide diff and rudimentary merge features that aids the maintenance.
• Translations: A relation type of 'translation' that can make a translation relationship between any two wiki pages (regardless of name), and with an attribute set to the locale of the translation. A translation-renderer will then be able to list out other versions of the same as related content. Some other module may provide this to make intelligent redirecting of requests.
• Wiki page hierarchies using a 'parent' relation. Same goes for ticket hierarchies.
• Ticket dependency relation type. Also various other advanced ticket features are relations - duplicates is a good example of this.
• Trac-Hacks example of having a wiki page correspond to a ticket 'component' - that relationship can then be made persistent. Such a persistent relation may then on the wiki page provide related content by the plugin in the form of 'Open tickets' into the relations renderer.
• Tagging of resources - the Tags plugin provides a good visualisation of this, and the datamodel could provide storage for tags.
• Store structured relationships between a large number of pages, making it easier to maintain for instance documentation setups where one specification use-case is one wiki page. When viewing a 'buried' page, one can immediately see where in the larger context this page belongs.
• Just basically a 'related content' manager that lets one input internal references and external urls as pointers on a given wiki page.
• And, really tons of other tickets and enhancement ideas that have been proposed… (need to search some up and group them).
• Storing back-tracking information between resources, and having 'What links here' rendered as a relation.
• Time-tracking, and the ability to 'tag' time-based events across wiki edits, ticket updates and changesets.

The aforementioned alternative proposals also includes references to a number of tickets that can be solved using this same idea.

How it could work

The usage inside Trac happens at various levels:

1. The storage interface provided by Trac. It is a general, efficient, secure (includes permissions whenever possible), and provides a do-not-shoot-yourself-in-the-leg kind of interface to setting and getting. And, as importantly, it makes sure you don't shoot others in the leg by malformed SQL.
2. The IRelationsProvider interface (new) that provides the ability to declare management of one or more relationship types, and where the interface also has various methods to syndicate the content in raw or renderable variations.
3. Plugins also needs to able to get input changes from users, and for this we need input and action controllers that render to the users. For some, like changesets for instance, this can be provided by a generic input controller that works in all modules unless a) it is turned off for a specific module, or b) a more specific renderer provides an alternative. Such as specific renderer may for instance be one built around a custom fields provider for tickets when that becomes available.
4. The existing APIs and hooks so that a plugin doing ticket dependency would for instance be sure to implement the ITicketManipulator in order to veto a ticket close when depending tickets are still open. And, it would use the ITicketChangeListener in order to establish and maintain relationships in storage.
5. The main layout.html provides the visualisations for all relationships to the user. It provides an always available 'Related Content' section for any page that is tied to some identifiable resource. It really can show any kind of content. Any kind. Using the Trac-Hacks example again, there might be relations to users for a 'Team Members' list renderer.

The wiki as an example may provide two options for rendering related content - the 'classic' floating sidenote typically where all gory details are listed as provided by renderers, or the plugin may decide to render some things as add_warning()/add_notice() style message - for instance that 'this page is renamed, here is the new page with link'. It is all up to the plugin to determine what api to use for any given use case and context.

Why is this a good model?

1. It keeps relations and relations actions outside the main content of the resource so that the content should focus on - the content itself.
2. Following on from that, there would be no need to set up various cross-referencing macros linking to children, parent, siblings, or macros for redirect and so on.
3. It follows a better model for the future of disconnecting wiki markup and rendering from the request and request-related context, and thereby providing better features for flexible rendering styles. For instance, rendering a wiki page as a web page would join all into a single visual experience. However, some xml-rpc interface to the same wiki page may want to see this very differently, and perhaps provide a number of supporting methods to deal with whatever relations the page has. Getting a list of children as a call is countless times more useful than some in-place macro that renders it as part of the main content.
4. It involves very low-impact changes - most things should remain unchanged.
5. It could provide standardized visualization components across the various modules in layout.html/theme.html that gathers and displays relations in a uniform manner.

What else?

Some notes and thoughts yet to be fully grasped.

• May need to revise permissions somewhat as the idea is only to store the relations. However a link between Foo and Bar resource types would still not tell you what actual permission is needed when rendering Foo to decide if Bar link should be listed. One could assume BAR_VIEW, but that assumption will not always hold - perhaps Bar is just an add-on to the wiki providing comments, and the actual permssion may then be WIKI_COMMENT_VIEW. Need a way to make the connection at least, but standardising permission on *_VIEW type permissions would not support the flexibility of relations very well. Perhaps the most meaningful way is building on the ideas of attachments, by using resources and parent structure for relationships and for relations module to delegate responsibilities to each attached resource.

What now?

This specification is still far from being finished.

• A table definition, and an upgrade script to create it in new and existing projects.
• Some model code providing simple interface methods to maintain the relations table - providing 'resource' and 'relation' input types, and creating, updating or deleting as requested. Something like a new relations.py + API definitions
• It supposes a very distributed model, and apart from API documentation there will likely not be all that much 'core' code in this module. Not finished visualising this yet.
• An implementation for the Wiki… 0.11 was workflow and lifting the feature set of tickets. 0.12+ should do much the same for the wiki. That means working with an implementation of basic provider models, edit controllers, renderers and so on.
• Provide some example plugins that further illustrate the possibilities:
  • Wiki rename comes to mind…
  • Other ideas?