Edgewall Software

Changes between Version 1 and Version 2 of TracDev/Proposals/Journaling


Ignore:
Timestamp:
Nov 23, 2006, 4:36:20 PM (17 years ago)
Author:
Christian Boos
Comment:

Introduce the Transaction class, and also update the proposal to use the <resource>_journal table discussed in the related TracDev/Proposals/DataModel.

Legend:

Unmodified
Added
Removed
Modified

  • TracDev/Proposals/Journaling

    v1 v2  
    3636== A solution ==
    3737
    38 Every ''change'' event could be journalled.
    39 A `journal` table could record all transactions and serve as
    40 a basis for redispatching changes happening in one process
     38Every ''change'' event could be journaled.
     39Several `journal` tables (one per resource) could record all the transactions and serve as
     40a basis for dispatching changes happening in one process
    4141to other processes, in a generic way.
    4242
    43 After all, this journaling is already done in some cases.
     43After all, such a kind of journaling is already done for tickets.
     44
     45=== Current Situation ===
    4446For example, all the ticket  changes are journaled, in the `ticket_change` table:
    4547{{{
    … …  
    6062who did what (see #1890 for details).
    6163
    62 This would lead to even more duplication of data than what we have now.
    63 Granted, currently this duplication (of the ticket/time/author values)
    64 are used to group related changes.
     64But adding those to the above table would lead to even more duplication of data than what we currently have.
     65Currently this duplication (of the ticket/time/author values)
     66is even used to group together related changes!
    6567
     68=== Step in the Right Direction ===
    6669A cleaner approach, for #1890, would be:
    6770{{{
    … …  
    8487}}}
    8588
     89
     90=== The `_journal` and `_history` tables ===
    8691Now, with this proposal, this could be extended to:
    8792{{{
    8893#!sql
    89 CREATE TABLE ticket_change (
     94CREATE TABLE ticket_history (
    9095    tid integer,
     96    id int,
    9197    field text,
    92     oldvalue text,
    93     newvalue text,
     98    value text
    9499);
    95100
    96 CREATE TABLE journal (
     101CREATE TABLE ticket_journal (
    97102    tid integer PRIMARY KEY,
    98     type text,
    99     id text,
     103    id int,
    100104    change text,
    101105    time integer,
    … …  
    106110}}}
    107111
    108 And `ticket_change` could even be generalized to `property_change`
    109 and go in the direction of a generalization of properties to
    110 all Trac objects (remember the TracObjectModelProposal?)
     112`ticket_history` could eventually be spread in multiple, type-specialized tables (`ticket_int_property`, ...).
    111113
    112 The `change` column in `journal` could contain some keyword about
     114See also TracDev/Proposals/DataModel#ResourceChangeHistory.
     115
     116The `change` column in `<resource>_journal` could contain some keyword about
    113117the nature of the change: `CREATE`, `DELETE`, `MODIFICATION`, etc.
    114118
     119Each process would write into the `<resource>_journal` table during the same
     120transaction that modifies the object model tables themselves.
     121
     122This could be something along the lines of the following API:
     123{{{
     124#!python
     125    class WikiModule():
     126        def _do_create(self, pagename):
     127            ...
     128            # Getting a new transaction for creating a Wiki page
     129            tnx = Transaction(self.env.get_db_cnx())
     130            tnx.prepare(req, 'CREATE')
     131            tnx.save('wiki', id=pagename, readonly=readonly, content=content)
     132            tnx.commit()     # flush all changes to disk
     133            self.notify(tnx) # dispatch change information to listeners
     134
     135    class TicketModule():
     136        def _do_save(self, ticket):
     137            tnx = Transaction(self.env.get_db_cnx())
     138            tnx.prepare(req, 'MODIFY')
     139            tnx.save('ticket', ticket)
     140            tnx.commit()     # flush all changes to disk
     141            self.notify(tnx) # dispatch change information to listeners
     142}}}
     143The actual `Transaction` object would know how to modify the underlying (generic) data model.
     144
     145
     146=== Notifying changes ===
    115147Now, how to use this information?
    116148
    117 Each process would write into the `journal` table during the same
    118 transaction that modifies the object model tables themselves.
    119 This will mostly be something along the lines of:
    120 {{{
    121 #!python
    122     tid = record_in_journal(req, db, 'wiki', page, 'CREATE')
    123 }}}
    124 and:
    125 {{{
    126 #!python
    127     tid = record_in_journal(req, db, 'ticket', id, 'MODIFY')
    128 }}}
     149Each module's `notify(tnx)` method would have to propagate the appropriate change information to registered listeners (the IWikiChangeListener, ITicketChangeListener, etc. interfaces).
    129150
    130 Each process will also have to keep track of the last `tid` known.
     151Each module would also have to keep track of the last `tid` it has dispatched.
     152In `notify(tnx)`, we would check for all existing `tid` inserted since the last dispatched `tid` (or the one we had at system startup). If there are more than one `tid`, those are coming from changes created by ''other'' processes.
    131153
    132 If this happens to have changed (details to be finalized:
    133 the detection could be done either during `record_in_journal` itself,
    134 or before request dispatching, or ...), there could be a ''replay''
    135 of those events, triggering the appropriate change listeners.
    136 
    137 The change listeners would anyway gain to be refactored in a more
    138 generic way (merging the IWikiChangeListener, ITicketChangeListener
    139 giving IMilestoneChangeListener, IChangesetChangeListener etc. for free,
    140 the usual TracObjectModelProposal blurb ;) ).
    141 
    142 Last but not least, there would be a need to differentiate between
    143 '''primary''' change and '''secondary''' change.
    144  primary change:: the change originated from the same process;
    145   there's only one process which ever sees a change as being a primary change
    146  secondary change:: the change originated from another process.
     154This way, we could easily differentiate between '''primary''' changes and '''secondary''' changes.
     155 primary change:: `tid == tnx.tid`; the change originated from the same process.
     156  There's only one process which ever sees a change as being a primary change, it's the one which just created that change.
     157 secondary change:: `tid != tnx.tid`; the change originated from another process.
    147158
    148159This distinction is quite important w.r.t. to side-effects.
    … …  
    156167for refreshing all sorts of internal caches (the use cases listed
    157168[TracDev/Proposals/Journaling#Someexamples above]).
    158  
     169
     170Finally note that this Transaction class could make good use of the [http://www.sqlalchemy.org/docs/unitofwork.myt Unit Of Work] concept of SQLAlchemy, should we use that in a future version.