NoSuchChangeset error when switching an existing Trac project to a new repository

[sambloomquist]

I had an existing Trac project environment. I wanted to switch it over to use a new SVN repository. I didn't want to just create a new environment altogether because I wanted to keep all of the existing wiki and ticket data. I'm now seeing a NoSuchChangeset error whenever I try to view the timeline or follow a bracketed [] changeset link. Interestingly, the compare changesets feature through the Source Browser GUI seems to work even though looking at any individual changeset via a bracket link doesn't. The NoSuchChangeset error is being thrown from the CachedChangeset object in cache.py:

class CachedChangeset(Changeset):

    def __init__(self, rev, db, authz):
        self.db = db
        self.authz = authz
        cursor = self.db.cursor()
        cursor.execute("SELECT time,author,message FROM revision "
                       "WHERE rev=%s", (rev,))
        row = cursor.fetchone()
        if row:
            date, author, message = row
            Changeset.__init__(self, rev, message, author, int(date))
        else:
            raise NoSuchChangeset(rev)

The NoSuchChangeset error always gets raised even though there is most definitely a record in my revision table for rev. When I try executing the SQL directly using the SQLite Database Browser, the record always comes back. In my instance 'rev' equals 1 at this point.

Here is the log trace when I get the error:

2006-11-17 12:28:00,811 Trac[svn_fs] DEBUG: Opening subversion file-system at D:/Claims/trac/SVN/covver with scope /
2006-11-17 12:28:00,811 Trac[cache] DEBUG: Checking whether sync with repository is needed
2006-11-17 12:28:00,872 Trac[api] DEBUG: Updating wiki page index
2006-11-17 12:28:00,951 Trac[svn_fs] DEBUG: Closing subversion file-system at D:/Claims/trac/SVN/covver
2006-11-17 12:28:01,122 Trac[svn_fs] DEBUG: Opening subversion file-system at D:/Claims/trac/SVN/covver with scope /
2006-11-17 12:28:01,122 Trac[cache] DEBUG: Checking whether sync with repository is needed
2006-11-17 12:28:01,201 Trac[svn_fs] DEBUG: Closing subversion file-system at D:/Claims/trac/SVN/covver
2006-11-17 12:28:02,904 Trac[svn_fs] DEBUG: Opening subversion file-system at D:/Claims/trac/SVN/covver with scope /
2006-11-17 12:28:02,904 Trac[cache] DEBUG: Checking whether sync with repository is needed
2006-11-17 12:28:03,061 Trac[svn_fs] DEBUG: Closing subversion file-system at D:/Claims/trac/SVN/covver
2006-11-17 12:28:03,201 Trac[svn_fs] DEBUG: Opening subversion file-system at D:/Claims/trac/SVN/covver with scope /
2006-11-17 12:28:03,201 Trac[cache] DEBUG: Checking whether sync with repository is needed
2006-11-17 12:28:03,326 Trac[svn_fs] DEBUG: Closing subversion file-system at D:/Claims/trac/SVN/covver
2006-11-17 12:28:04,874 Trac[svn_fs] DEBUG: Opening subversion file-system at D:/Claims/trac/SVN/covver with scope /
2006-11-17 12:28:04,874 Trac[cache] DEBUG: Checking whether sync with repository is needed
2006-11-17 12:28:04,983 Trac[svn_fs] DEBUG: Closing subversion file-system at D:/Claims/trac/SVN/covver
2006-11-17 12:28:04,983 Trac[main] WARNING: 500 Internal Error (Repository checkins  event provider (<tt>ChangesetModule</tt>) failed:<br /><br />NoSuchChangeset: No changeset 1 in the repository<p>You may want to see the other kind of events from the <a href="/projects/covver.db/timeline?milestone=on&amp;ticket=on&amp;wiki=on">Timeline</a></p>)

For background, my steps for setting up the new repository for use with Trac were as follows:

1. Create new repository.
2. Import a bunch of code into the repository under revision 1.
3. Create new trac project instance with trac-admin initenv
4. Copy an existing trac sqlite database into the new instance 'db' folder over the newly-created database.
5. Drop the 'revision' and 'node_change' data from the sqlite database.
6. Perform a trac-admin 'resync' operation which completes successfully. At this point the 'revision' and 'node_change' tables match the new subversion repository perfectly.
7. Click the Timeline tab in a browser — get the NoSuchChangeset error.

[Christian Boos]

(Repository checkins  event provider (<tt>ChangesetModule</tt>) failed:...

indicates that the error is coming from the Changeset ITimelineEventProvider.

This in turn means that what's failing in your case is the CachedRepository.get_changeset() call from within CachedRepository.get_changesets().

So, a few additional questions:

1. I assume you're using 0.10.3dev, right?
2. Are you using an authz_file?
3. What database are you using (with version details, including for the bindings used)
4. Can you try the following patch?

   Index: cache.py
   ===================================================================
   --- cache.py	(revision 4303)
   +++ cache.py	(working copy)
   @@ -45,7 +45,7 @@
            cursor.execute("SELECT rev FROM revision "
                           "WHERE time >= %s AND time < %s "
                           "ORDER BY time", (start, stop))
   -        for rev, in cursor:
   +        for rev, in list(cursor):
                if self.authz.has_permission_for_changeset(rev):
                    yield self.get_changeset(rev)
    
   

[sambloomquist]

Replying to cboos:

(Repository checkins  event provider (<tt>!ChangesetModule</tt>) failed:...

indicates that the error is coming from the Changeset ITimelineEventProvider.

This in turn means that what's failing in your case is the CachedRepository.get_changeset() call from within CachedRepository.get_changesets().

Could be, but I do know that the NoSuchChangeset error is getting raised in CachedChangeset.__init__(). I tested to make sure by changing the 'raise' statement to the following:

raise NoSuchChangeset(rev + 1)

When I did that, the error message became "NoSuchChangeset: No changeset 2 in the repository" instead of "NoSuchChangeset: No changeset 1 in the repository"

So, a few additional questions:

1. I assume you're using 0.10.3dev, right?
2. Are you using an authz_file?
3. What database are you using (with version details, including for the bindings used)
4. Can you try the following patch?

Answers:

1. Yes.
2. No.
3. sqlite 3.3.8 for windows with pysqlite-1.1.6.win32-py2.3.exe bindings
4. Patch didn't help.

Thanks.

[Christian Boos]

Replying to sambloomquist:

3. sqlite 3.3.8 for windows with pysqlite-1.1.6.win32-py2.3.exe bindings

I'm not sure why you mention 3.3.8, but pysqlite-1.1.6 is certainly not using that itself, most likely some early 3.1.x or 3.2.x release of SQLite. So maybe you hit some kind of incompatibility. In particular, in your point 5., when you said 'drop', did you really mean drop table revision; drop table node_change? How did you recreate those tables, then?

In any case, you should upgrade to a more recent PySqlite release, i.e. 2.3.2 (once initd.org is back online…)

[sambloomquist]

Replying to cboos:

Replying to sambloomquist:

3. sqlite 3.3.8 for windows with pysqlite-1.1.6.win32-py2.3.exe bindings

I'm not sure why you mention 3.3.8, but pysqlite-1.1.6 is certainly not using that itself, most likely some early 3.1.x or 3.2.x release of SQLite. So maybe you hit some kind of incompatibility.

Wasn't thinking when I mentioned 3.3.8 — I just have that version installed for interacting with the db from the command line. I used psqlite1.1.6 because that's what the TracOnWindows page suggested. I will definitely try a newer version once I can download one, but I doubt that's the issue since everything worked fine before I switched to a new repository (and installed 0.10.2).

In particular, in your point 5., when you said 'drop', did you really mean drop table revision; drop table node_change? How did you recreate those tables, then?

Sort of. What I did was…

1. export the tables to comma-separated-value files
   
2. open the csv files and delete all the data except the column names
   
3. drop the tables from the database
   
4. re-import the csv files
   

Then, doing a trac-admin resync populated all of the new repository data back into the revision and node_change tables.

On a side note, is there a better way to switch over to a new repository? Are there any other tables than revision and node_change that reference the repository? Perhaps something else didn't get cleaned up when I did the resync.

[anonymous]

i have the same issue…

[Christian Boos]

Replying to sambloomquist:

What I did was…

1. export the tables to comma-separated-value files
   
2. open the csv files and delete all the data except the column names
   
3. drop the tables from the database
   
4. re-import the csv files
   

Then, doing a trac-admin resync populated all of the new repository data back into the revision and node_change tables.

OK, I tried to reproduce what I think you did:

sqlite> .mode csv
sqlite> .headers on
sqlite> .output revision.csv
sqlite> select * from revision;
sqlite> .output node_change.csv
sqlite> select * from node_change;
sqlite> .output stdout
sqlite> drop table revision;
sqlite> drop table node_change;

(edited the .csv files so that only the headers were left)

sqlite> .import revision.csv revision;
Error: no such table: revision

So obviously you did something else…

Sorry to not "get" it, but I really need to be able to reproduce the issue myself before I can think of a fix. So please give me "dummy-like" directions ;)

[Scott MacVicar <scott@…>]

My boss managed to produce this yesterday by copying the trac directory and then changing the trac.ini entry to point to another repository, though I resolved this by showing him how to use trac-admin. I can look at tracking this down later when I have some free time.

Versions: Python 2.3.4 SQLite 3.3.8 PySQLite 2.3.2 mod_python 3.2.10 Apache 2.2.3 Trac 0.10.3

[Christian Boos]

Replying to Scott MacVicar <scott@vbulletin.com>:

My boss managed to produce this yesterday by copying the trac directory and then changing the trac.ini entry to point to another repository,

Wait, that could possibly have been triggered by #4204. Try again with revision ≥ r4338,

[sambloomquist]

Replying to cboos:

Sorry to not "get" it, but I really need to be able to reproduce the issue myself before I can think of a fix. So please give me "dummy-like" directions ;)

Sorry I wasn't very clear. Actually, I was using the SQLite Database Browser gui tool (​http://sqlitebrowser.sourceforge.net/) version 1.3 for windows. It has import/export abilities.

I imagine that it's doing something pretty similar to what you did above behind the scenes, but there's probably a table create statement in between the drop tables and the imports. It might be easiest to reproduce by just downloading the gui tool, but I can try to work out a reproducible scenario from the command line if you like.

[Christian Boos]

Replying to sambloomquist:

Sorry I wasn't very clear. Actually, I was using the SQLite Database Browser gui tool (​http://sqlitebrowser.sourceforge.net/) version 1.3 for windows. It has import/export abilities.

Ok, now that's clear ;) I was able to reproduce the issue.

[Christian Boos]

Ok, so here's what happened. After the import, the tables were recreated, but without any type annotation. That's not a big deal for SQLite, which can operate fine with typeless columns. But then, there's some kind of auto-casting mechanism taking place, and in the get_changesets() call, we go in with an unicode rev, but receive an int rev from the cursor (here​).

Then, looking for an int rev in CacheChangeset.__init__ fails.

So the problem comes from the way you did the export/import. You can fix that specific issue by doing this little fix:

cache.py

@@ -149 +149 @@
         self.authz = authz
         cursor = self.db.cursor()
         cursor.execute("SELECT time,author,message FROM revision "
-                       "WHERE rev=%s", (rev,))
+                       "WHERE rev=%s", (str(rev),))
         row = cursor.fetchone()
         if row:
             _date, author, message = row

but I can't guarantee that there won't be another problem afterwards.

So the best thing in your case would be to recreate the tables the normal way (using the sqlite3 command line):

DROP TABLE revision;
CREATE TABLE revision (
    rev text PRIMARY KEY,
    time integer,
    author text,
    message text
);
CREATE INDEX revision_time_idx ON revision (time);

DROP TABLE node_change;
CREATE TABLE node_change (
    rev text,
    path text,
    node_type text,
    change_type text,
    base_path text,
    base_rev text,
    UNIQUE (rev,path,change_type)
);
CREATE INDEX node_change_rev_idx ON node_change (rev);

then do a resync.

Next time you have something like that to do, don't bother deleting those tables, as the resync clears them and refills the cache from scratch ;)

[sambloomquist]

Replying to cboos:

Next time you have something like that to do, don't bother deleting those tables, as the resync clears them and refills the cache from scratch ;)

Duly noted. Thanks a lot for the fix — itw worked just fine.