Exception error during hotcopy

[anonymous]

I received an exception error during a hotcopy using trac-admin. I have Trac installed on Windows 2000 Server, Service Pack 4.

The error was as follows: Hotcopying d:\source\folder to d:\target\folder … [('d:
source
folder
db
trac.db-journal', 'd:
target
folder
db
trac.db-journal', <exceptions.IOError instance at 0x008BF3C8>)]

The 'attachments', 'conf' and 'db' folders were copied but it didn't get to the 'log', 'templates' and 'wiki-macros' folders. The 'trac.db' file appears to have been successfully copied across (at least, the size of the source and destination 'trac.db' files are the same).

[rhind@…]

I'm just seeing this also. I can't backup the database at the moment because of it.

[rhind@…]

Changed milestone to 0.9 to try and get some visibility on this as no-one seems to have commented on it for a while, but it is still happening for me

WinXP SP2 trac-0.8.1 python-2.3.4

Thanks

Russell

[johan.appelgren@…]

I believe the error is because sqlite has opened trac.db-journal exclusively so that the copy fails. Don't know if the sqlite db will survive being copied with a transaction active and the journal file missing. But if it can survive that, excluding the journal file(es) in the hotcopy should fix this.

[rhind@…]

I've stopped apache running on my system while trying to hotcopy, so there shouldn't be any active transactions should there?

[johan.appelgren@…]

The hotcopy code in trac-admin creates a transaction.

[m.v.kalmthout_1@…]

I had the same problem. Due to small error in TracOnWindows I installed pysqlite-1.0.1.win32-py2.3.exe in stead of pysqlite-1.1.6.win32-py2.3.exe.

After upgrading to pysqlite-1.1.6 this problem was gone.

[rhind@…]

Ok, I had installed pysqlite-1.0.1.win32-py2.3.exe I uninstalled it from control panel, then installed pysqlite-1.1.6.win32-py2.3.exe as suggested. I now can't access the trac database at all. I get the following apache errors in my server log

[Wed Mar 23 20:28:10 2005] [error] [client 10.0.127.18] PythonHandler trac.ModPythonHandler: Traceback (most recent call last): [Wed Mar 23 20:28:10 2005] [error] [client 10.0.127.18] PythonHandler trac.ModPythonHandler: File "C:
Python23
Lib
site-packages
mod_python
apache.py", line 299, in HandlerDispatch\n result = object(req) [Wed Mar 23 20:28:10 2005] [error] [client 10.0.127.18] PythonHandler trac.ModPythonHandler: File "C:
Python23
Lib
site-packages
trac
ModPythonHandler.py", line 186, in handler\n env = get_environment(req, mpr) [Wed Mar 23 20:28:10 2005] [error] [client 10.0.127.18] PythonHandler trac.ModPythonHandler: File "C:
Python23
Lib
site-packages
trac
ModPythonHandler.py", line 176, in get_environment\n env_cache[env_path] = open_environment(env_path, mpr) [Wed Mar 23 20:28:10 2005] [error] [client 10.0.127.18] PythonHandler trac.ModPythonHandler: File "C:
Python23
Lib
site-packages
trac
ModPythonHandler.py", line 137, in open_environment\n version = env.get_version() [Wed Mar 23 20:28:10 2005] [error] [client 10.0.127.18] PythonHandler trac.ModPythonHandler: File "C:
Python23
Lib
site-packages
trac
Environment.py", line 162, in get_version\n cursor.execute("SELECT value FROM system WHERE name='database_version'") [Wed Mar 23 20:28:10 2005] [error] [client 10.0.127.18] PythonHandler trac.ModPythonHandler: File "C:
Python23
Lib
site-packages
sqlite
main.py", line 244, in execute\n self.rs = self.con.db.execute(SQL) [Wed Mar 23 20:28:10 2005] [error] [client 10.0.127.18] PythonHandler trac.ModPythonHandler: DatabaseError: file is encrypted or is not a database

I've uninstalled 1.1.6 and gone back to 1.0.1 and everything works fine. Do I need to upgrade the trac-db somehow? I've tried a trac-admin upgrade but get the same error about the db being encrypted or is not a database.

Any help would be greatly appreciated.

Cheers

Russell

[Mark Rowe]

You need the correct version of PySQLite for the version of SQLite you are using. If you are using SQLite 2.x, you need PySQLite 1.0.x. If you are using SQLite 3, you need PySQLite 1.1.x. As I understand it, the SQLite 2.x and 3.x file formats are incompatible.

[rhind@…]

I don't believe I manually downloaded sqlite at all when I started, just pysqlite-1.0.1 and trac-0.7.1.exe (then upgraded to trac-0.8). I have now downloaded the sqlite3.exe. Is there a way I can upgrade the trac database so I can use the later pysqlite?

Thanks

Russell

[anonymous]

Ok, I've just moved the database on to another server and looked again at upgrading from sqlite2 to sqlite3 (pysqlite-1.1.6). You need to upgrade db/trac.db sqlite database file in you trac directory. The bottom of ​this page details how to do this. I can now hotcopy trac!

I guess this ticket can be closed as worksforme?

[Jonas Borgström]

I can't reproduce this either. Please reopen if this problem still exists.

[anonymous]

I am reopening this ticket because I am seeing this problem on my new installation:

• Trac 0.9 trunk revision [2288]
• pysqlite 1.1.6
• running on a windows 2003 domain server

Is it possible that we need pysqlite 2.x now? The problem might have been introduced in [2263], because you changed something in the transaction logic there (see ticket #2016)

[Matthew Good]

No, [2263] just made sure that only the standard DB-API methods were used instead of pysqlite 1.x-specific methods. The hotcopy command works fine with pysqlite 1.1 on Linux, but this appears to be Windows file-locking issue (bah, Windows shouldn't tell me I can't copy a file).

I guess I can do some testing of hotcopy with different pysqlite versions on Windows, but I don't think we really need the journal file anyways. If that's the only thing that's interfering with the copying on Windows it would be possible to create a modified version of the ​shutil.copytree method that would accept a list of files to ignore.

[Alesque]

I upgraded trac from 0.8x to 0.9b2 on win2k3 today. The hotcopy command doesn't work anymore.

Hotcopying C:\Projects\Trac\XXXX to help ...Command failed: [('C:\\Projects\\Tra
c\\XXXX\\db\\trac.db-journal', 'help\\db\\trac.db-journal', <exceptions.IOError
instance at 0x00B38B20>)]

[gabriel.blum@…]

Does't work here either. Win2k3 server domain controller + 0.9b2 + pysqlite 1.1.6; I agree this looks like a file locking issue

A simple copy of the env dir would work as a backup?

[Christian Boos]

Please try once again with recent trunk (r2355 might have solved your locking issue).

[Alesque]

I try a hot copy with r2357 and the same error occurs:

• Windows 2003 domain server
• pysqlite 2.0.4

[Christian Boos]

I can reproduce this. I'll look into it.

[Christian Boos]

Implemented mgood's suggestion: on Windows, the trac.db-journal file can't be copied, and the shutil.copytree can't cope with that, therefore we need our own version, slightly adapted to cope with anticipated failures.

[Christian Boos]

There's now a patch to test: attachment:robust_hotcopy.patch​

For me at least, it appeared that the only problem was that the trac-db.journal can't be copied due to the way SQLite uses it to perform to lock of the database. So the copy will always fail for this file, we just have to cope with it. Also, it will not be a problem to not copy the journal file, as it was the hotcopy method itself which triggered the (dummy) transaction.

Therefore, I followed mgood's suggestion of skipping the copy of that file and added a variant of shutil.copytree to trac.util.

[Christian Boos]

Second version. copytree is now directly in admin.py, and the ignore list is now a "skip" list (we don't even attempt to copy the file)

[Christopher Lenz]

Second patch looks good.

[Christian Boos]

Second patch applied in r2410.

[goran.wallgren@…]

This problem has reappeared in 0.11b1 on Windows with pysqlite-2.4.0, see new ticket #6627.