Journaling Proposal

The Problem

Trac maintains coherency upon data changes by using various I...Listener extension points.

While this works in many cases, this approach is somewhat flawed or insufficient in situations where there are multiple server processes. And this is quite a common scenario, with the widespread use of the Apache/prefork web front-end (see #5741).

Actually, the main problem comes from cached data. If what the listener does is only modify the database, then the other server process will see the updated data the next time they'll read it. But that won't work if they maintain an in-memory cache (like the InterWiki component does for the InterMapTxt content, or the WikiModule does for the list of wiki pages, etc.)

A solution

The solution is to invalidate those caches at the right time.

• the WikiModule does it by resetting the cache at regular intervals. This will work most of the time, but of course users will occasionally get the wrong content if the content of the cache lags too much behind the reality of the database
• with r6837, every change in InterMapTxt will trigger a complete configuration reload. This will work, but it's a bit heavy handed. As the changes to that page are not supposed to be very frequent, I think that's an acceptable trade-off.

But a better solution would be to invalidate only the caches that need to be invalidated. This could be achieved by using a table (system?) containing a key/value pair for every such cache.

The listener component that changes the cache will also store a new checksum value of the cached data it is responsible for, associated to a key specific to this cache. The corresponding components in other server processes will know that their cache is invalid by comparing their in-memory checksum with the checksum in the database (that'll be only one lookup, we can assume it will be cheap). If there's a mismatch, the cache is invalid an needs to be rebuild, like it would be from a listener change.

See also TracDev/Proposals/Journaling (which is somewhat older and more heavy weight)