Plugin db upgrade infrastructure

[Felix Schwarz <felix.schwarz@…>]

Trac currently has its own infrastructure to manage db upgrades. However for plugin authors there is no mechanism available for that so every plugin author needs to re-invent this again.

Attached is a module which basically uses trac's mechanism but is a bit more generic so this could be used by plugins as well.

[Felix Schwarz <felix.schwarz@…>]

If there is a chance to get this into trac main, we can polish this class (including docs) and make trac use it too

[rblank]

Sounds useful.

[Felix Schwarz <felix.schwarz@…>]

Hmm, I hoped we could have this in 0.12... Do you think this feature could make it?

[rblank]

About milestones, 0.12 means a developer is committed to integrating the patch in trunk. 0.13 means a developer is committed to integrating the patch, but in the next version. Everything beyond that means we would like to do it eventually, but nobody has taken it up yet.

Personally, I don't have enough knowledge about this part of Trac that I could review and integrate your patch. Sorry about that. If you can persuade another dev to take this up, he will move the milestone closer as well, possibly to 0.12.

[cboos]

Milestone 1.0 deleted

[rblank]

I'd like to pick this up. However, rather than having an abstract component from which plugins would inherit, how about creating a new IDBUpgradeParticipant interface that would be implemented in plugins and provide the plugin name, the expected DB version, possibly the package containing the upgrade scripts, as well as a method for the initial DB setup? Then a single component could implement the upgrade logic, and be used by core as well.

[cboos]

I also think we don't need to inherit from an abstract component here.

However, I'm not sure if it's worth to introduce a new interface, we could perhaps do as well with a set of helper methods (factoring out the code already in env.py and mixing it with some parts of the proposed patch).

[cboos]

... and yes, this would not only be useful for plugins, but also for core sub-systems, to get finer grained upgrades (see #9222).

[osimons]

#9222... Well, this is back to the core vs plugins vs future of Trac development...

Personally I don't see the need for new interfaces either. I think the IEnvironmentSetupParticipant interface is perfectly capable of handling all kinds of upgrades. What is missing is, as also mentioned in #9222, moving upgrades into the various components parts of Trac and letting each self-contained part of Trac be responsible for its own upgrades.

This was the original vision of the component code before things got more and more 'tangled' into each other. For a remaining example, see for instance how attachment.py implements the interface for the sole purpose of creating an 'attachments' folder for new projects.

Naturally, the Trac core code can improve its (optional) services/functions/helpers for module and plugin upgrades - just don't invent a whole new infrastructure for this that will never be able to cover all needs anyway, IMHO.

[rblank]

Replying to osimons:

just don't invent a whole new infrastructure for this that will never be able to cover all needs anyway, IMHO.

That's a bit pessimistic, but I think you (and Christian) are right. Let's try to add a few helper functions to more easily allow distributing upgrades into individual modules and plugins.

That's one risk when you have a cool component architecture: you tend to use it too much...

[psuter]

DB upgrade helper functions

[psuter]

Would the helper functions in the patch attached above be a step in the right direction?
They mostly refactor out functionality from Environment and EnvironmentSetup for DB related environment setup (schema setup, initial data insertion, version check and per-version upgrades).

A plugin could quite easily reuse them to manage its DB schema:

from trac.core import *
from trac.env import * 
from trac.db.schema import Table, Column

PLUGIN_NAME = 'ExamplePlugin'
PLUGIN_VERSION = 2
PLUGIN_SCHEMA = [
    Table('table1', key='name')[
        Column('name'),
        Column('value')]]
def get_initial_plugin_data(db):
    return (('table1',
              ('name', 'value'),
                (('name1', 'value1'),
                 ('name2', 'value2'))),)
                 
class ExamplePlugin(Component):

    implements(IEnvironmentSetupParticipant)

    # IEnvironmentSetupParticipant methods

    def environment_created(self):
        init_environment_schema_for_module(self.env, PLUGIN_SCHEMA)
        init_environment_data_for_module(self.env, get_initial_plugin_data)
        init_environment_version_for_module(self.env, PLUGIN_NAME,
                                            PLUGIN_VERSION)

    def environment_needs_upgrade(self, db):
        return environment_needs_upgrade_for_module(self.env, PLUGIN_NAME,
                                                    PLUGIN_NAME,
                                                    PLUGIN_VERSION)

    def upgrade_environment(self, db):
        upgrade_environment_for_module(self.env, PLUGIN_NAME, PLUGIN_NAME, 
                                       PLUGIN_VERSION,
                                       'ExamplePlugin_upgrades')

(It could also freely mix and match these functions with its own code, or ignore them altogether.)

I'm not too sure with the names. Maybe it would be good to drop or replace the for_module suffix, include _db_ somewhere, or even move them to trac.db.util?

What do you think?