trac-admin $env upgrade fails (missing table)

[daniel@…]

trac-admin $env upgrade fails when upgrading from db_version 19 to 26.

It seems as if it tries to load the environment, finds it, needs upgrade, and then proceeds to load an environment that requires new-style repository with own tables that don't exist yet.

Environment:

• Linux clapton 2.6.18-6-xen-686 #1 SMP Sat Feb 20 01:44:14 UTC 2010 i686 GNU/Linux
• Debian Etch
• Python 2.4.4-2
• trac 0.12dev r9505

$env was created by trac 0.10.3 (from Debian repo), using sqlite3 as data backend.

2010-04-21 15:57:13,215 Trac[__init__] WARNING: Component <trac.env.EnvironmentSetup object at 0xb73c57ac> requires environment upgrade
2010-04-21 15:57:13,216 Trac[__init__] INFO: Trac database schema version is 19, should be 26
2010-04-21 15:57:13,225 Trac[__init__] ERROR: Exception in trac-admin command: 
Traceback (most recent call last):
  File "build/bdist.linux-i686/egg/trac/admin/console.py", line 107, in onecmd
  File "cmd.py", line 218, in onecmd
    return self.default(line)
  File "build/bdist.linux-i686/egg/trac/admin/console.py", line 257, in default
  File "build/bdist.linux-i686/egg/trac/admin/api.py", line 118, in execute_command
  File "build/bdist.linux-i686/egg/trac/env.py", line 791, in _do_upgrade
  File "build/bdist.linux-i686/egg/trac/env.py", line 521, in upgrade
  File "build/bdist.linux-i686/egg/trac/db/util.py", line 60, in transaction_wrapper
  File "build/bdist.linux-i686/egg/trac/env.py", line 533, in do_upgrade
  File "build/bdist.linux-i686/egg/trac/env.py", line 603, in upgrade_environment
  File "build/bdist.linux-i686/egg/trac/upgrades/db20.py", line 13, in do_upgrade
  File "build/bdist.linux-i686/egg/trac/env.py", line 369, in get_repository
  File "build/bdist.linux-i686/egg/trac/versioncontrol/api.py", line 471, in get_repository
  File "build/bdist.linux-i686/egg/trac/versioncontrol/api.py", line 539, in get_all_repositories
  File "build/bdist.linux-i686/egg/trac/versioncontrol/api.py", line 114, in get_repositories
  File "build/bdist.linux-i686/egg/trac/db/util.py", line 123, in execute
  File "build/bdist.linux-i686/egg/trac/db/sqlite_backend.py", line 77, in execute
  File "build/bdist.linux-i686/egg/trac/db/sqlite_backend.py", line 56, in execute
  File "build/bdist.linux-i686/egg/trac/db/sqlite_backend.py", line 48, in _rollback_on_error
OperationalError: no such table: repository

[Christian Boos]

source:trunk/trac/upgrades/db20.py​

Fix could be pretty simple, at the cost of a resync later (but I wonder if there wouldn't be a need for a resync anyway).

db20.py

@@ -15 @@
-            except TracError: # no repository available
+            except: # no repository available

Could you test this?

[Remy Blank]

It seems like I have broken the db20.py upgrade, because using env.get_repository() requires the repository table to be present, which is not the case at DB version 19. Ideally, we should only use SQL in the upgrade scripts, and this is actually the case for all scripts except db20.py.

(… collision again …)

I volunteer to rewrite db20.py with SQL, which should simple enough.

[Remy Blank]

To Daniel: could you please keep a copy of your 0.10.3 environment around to test the "proper" fix? Thanks!

[daniel@…]

The patch from cboos worked:

--- trac/upgrades/db20.py       (Revision 9508)
+++ trac/upgrades/db20.py       (Arbeitskopie)
@@ -12,7 +12,7 @@
     try:
         repos = env.get_repository()
         youngest = repos.get_youngest_rev_in_cache(db) or ''
-    except TracError: # no repository available
+    except: # no repository available
         youngest = ''
     # deleting first, for the 0.11dev and 0.10.4dev users
     cursor.execute("DELETE FROM system WHERE name=%s",

I'll keep the 0.10.3 project for further tests.

[Remy Blank]

Fix db20.py so that it doesn't use a Repository.

[Remy Blank]

Daniel, could you please test an upgrade of your 0.10.3 environment to 0.12dev with the patch above applied?

[daniel@…]

The patch ​9252-db20-upgrade-r9512.patch works for me when upgrading from db_version 19 (trac 0.10.3) to 26.

[Remy Blank]

More correct fix.

[Remy Blank]

Oops, I forgot to take inheritance into account. Could you please test attachment:9252-db20-upgrade-2-r9513.patch​ as well? This time, it should be correct for SVN repositories, and suggest a resync for other backends.

What is your repository type? Do you get the following message when upgrading?

Please perform a "repository resync" after this upgrade.

[Remy Blank]

Even better fix.

[Remy Blank]

Scratch that. Could you please test attachment:9252-db20-upgrade-3-r9513.patch​ instead? If you use SVN, it should just work, without needing a resync. Otherwise, it should display a message asking you to resync.

[daniel@…]

Replying to rblank:

Scratch that. Could you please test attachment:9252-db20-upgrade-3-r9513.patch​ instead? If you use SVN, it should just work, without needing a resync. Otherwise, it should display a message asking you to resync.

Yes, works fine here with 0.10.3 $env.

[Remy Blank]

Thanks for testing. Patch applied in [9517].

[anonymous]

I use Trac 0.12dev-r9663 with Python 2.5.2 and I get the "OperationalError: no such table: repository" error, when I try to do a "trac-admin $env upgrade".

The trac environment was created with debian Trac 0.11.5 and im also using sqlite3 as data backend.

2010-05-10 09:42:34,911 Trac[env] WARNING: Component <trac.env.EnvironmentSetup object at 0x1285250> requires environment upgrade
2010-05-10 09:42:34,912 Trac[env] INFO: Trac database schema version is 21, should be 26
2010-05-10 09:42:34,918 Trac[console] ERROR: Exception in trac-admin command:
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/trac/admin/console.py", line 107, in onecmd
    rv = cmd.Cmd.onecmd(self, line) or 0
  File "/usr/lib/python2.5/cmd.py", line 218, in onecmd
    return self.default(line)
  File "build/bdist.linux-x86_64/egg/trac/admin/console.py", line 257, in default
    return cmd_mgr.execute_command(*args)
  File "build/bdist.linux-x86_64/egg/trac/admin/api.py", line 118, in execute_command
    return f(*fargs)
  File "build/bdist.linux-x86_64/egg/trac/env.py", line 791, in _do_upgrade
    self.env.upgrade(backup=no_backup is None)
  File "build/bdist.linux-x86_64/egg/trac/env.py", line 523, in upgrade
    if participant.environment_needs_upgrade(db):
  File "build/bdist.linux-x86_64/egg/codetags/__init__.py", line 24, in environment_needs_upgrade
  File "build/bdist.linux-x86_64/egg/trac/env.py", line 373, in get_repository
    return RepositoryManager(self).get_repository(reponame)
  File "build/bdist.linux-x86_64/egg/trac/versioncontrol/api.py", line 473, in get_repository
    repoinfo = self.get_all_repositories().get(reponame, {})
  File "build/bdist.linux-x86_64/egg/trac/versioncontrol/api.py", line 541, in get_all_repositories
    for reponame, info in provider.get_repositories():
  File "build/bdist.linux-x86_64/egg/trac/versioncontrol/api.py", line 116, in get_repositories
    "'%s'" % each for each in self.repository_attrs))
  File "build/bdist.linux-x86_64/egg/trac/db/util.py", line 123, in execute
    return self.cursor.execute(sql)
  File "build/bdist.linux-x86_64/egg/trac/db/sqlite_backend.py", line 78, in execute
    result = PyFormatCursor.execute(self, *args)
  File "build/bdist.linux-x86_64/egg/trac/db/sqlite_backend.py", line 56, in execute
    args or [])
  File "build/bdist.linux-x86_64/egg/trac/db/sqlite_backend.py", line 48, in _rollback_on_error
    return function(self, *args, **kwargs)
OperationalError: no such table: repository

[Christian Boos]

Replying to anonymous:

2010-05-10 09:42:34,911 Trac[env] WARNING: Component <trac.env.EnvironmentSetup object at 0x1285250> requires environment upgrade
2010-05-10 09:42:34,912 Trac[env] INFO: Trac database schema version is 21, should be 26
2010-05-10 09:42:34,918 Trac[console] ERROR: Exception in trac-admin command:
Traceback (most recent call last):
  ...
  File "build/bdist.linux-x86_64/egg/codetags/__init__.py", line 24, in environment_needs_upgrade

Two remarks:

• it's the codetags plugin's upgrade which fails; the workaround is easy: disable the plugin, upgrade Trac itself, re-enable the plugin, upgrade the plugin
• I thought the logging was made improved and would explain more clearly which component was being upgraded. Apparently we can still improve this…