Attachments not migrated on Trac upgrade

[Ryan J Ollos]

We've had a number of reports that attachments are not migrated when upgrading Trac. I thought it was time to finally make a list of these and open a ticket for investigation:

• ​Missing attachments after upgrading
• ​Missing attachments after migration from .11 to 1.0.1
• ​Attachment Links broken after upgrade
• ​Attachments not moved to new structure after upgrade to 1.0
• ​attachments not working after migration to new version
• ​th:#11151

It appears that the db28.py​ step is somehow failing.

[cbidwell@…]

I'm having issues with this. I've upgraded to 1.0.1 but still can't get my attachments to migrate over. Am I missing something?

[Ryan J Ollos]

Replying to cbidwell@…:

I'm having issues with this. I've upgraded to 1.0.1 but still can't get my attachments to migrate over. Am I missing something?

You are not missing anything. We've seen the issue several times, but haven't diagnosed the cause. Could you provide more details to help us?:

• Trac version upgrading from
• Platform information

Is there anything in the trac.log file with regard to the failed upgrade step?

[cbidwell@…]

Sure. I upgraded from 0.12.2 → 1.0. Then from 1.0 → 1.0.1 using easy_install for both. Running on RHEL 6.5 x86_64. I did not see anything relevant in trac.log.

[cbidwell@…]

Is there a way to run db28.py manually to get this to work?

[Ryan J Ollos]

Is there a files directory in the environment directory? Do you still have the console output available so that you could check for messages starting with Unable to move attachment? It looks like the messages may be printed but not logged: tags/trac-1.0.1/trac/upgrades/db28.py@:87-89#L75​.

The other possibility that Jun mentioned in ​th:#11151 is that the step is being skipped entirely due to the early return: tags/trac-1.0.1/trac/upgrades/db28.py@:26​. We could at least log something there to try and narrow down the problem.

See ​th:comment:2:ticket:11151 for information on running the upgrade step manually.

[Ryan J Ollos]

Someone added a nice FAQ entry for this issue. Thanks anonymous! :)

[Ryan J Ollos]

Moved ticket assigned to next-stable-1.0.x since maintenance of 1.0.x is coming to a close. Please move the ticket back if it's critical to fix on 1.0.x.

[Ryan J Ollos]

We can reopen the ticket if anyone discovers new information about the source of the problem.

[bobbysmith007@…]

I just migrated from 1.0 to 1.2 (and from Ubuntu 14.04 to Ubuntu 16.04). When I did all my attachements are misindexed.

Trac is searching for: /var/trac/TRACPROJECT/files/attachments/ticket/81ctfc/0ec09ef9836da03f1add21e3ef607627e687e790/fbc9d5453b5338e3dd0e7c7eae458e18a388a921.pdf

But my attachemnets look like:

/var/trac/TRACPROJECT/files/attachments/tickets/81c/81c69212880f2e985e1dedf869c2483ece723d68/1c9c14fcc53550bcd27fa8a0add2ce9befb78117.pdf

(I doubt that these represent the same file, but just as an example).

These trac envs have been repeated upgraded from trac.08-9 to the present and have been in continuous use. They have been been moved multiple times. It seems that a database update reindexed the attachments in the database, but didnt move them.

---

Unfortunately we didnt find this till after the logs had been rotated, so we dont have a log of what occurred there

[Ryan J Ollos]

Replying to bobbysmith007@…:

I just migrated from 1.0 to 1.2 (and from Ubuntu 14.04 to Ubuntu 16.04). When I did all my attachements are misindexed.

Did you change from 32-bit to 64-bit Python or 32-bit to 64-bit OS?

Trac is searching for: /var/trac/TRACPROJECT/files/attachments/ticket/81ctfc/0ec09ef9836da03f1add21e3ef607627e687e790/fbc9d5453b5338e3dd0e7c7eae458e18a388a921.pdf

But my attachemnets look like:

/var/trac/TRACPROJECT/files/attachments/tickets/81c/81c69212880f2e985e1dedf869c2483ece723d68/1c9c14fcc53550bcd27fa8a0add2ce9befb78117.pdf

(I doubt that these represent the same file, but just as an example).

The difference in hash lengths might be a clue. The first 3 characters of the hash should be used to generate the parent id: tags/trac-1.2.2/trac/attachment.py@:741#L730​.

Where do you see Trac searching for a path with a 6 character hash for the parent id?

[bobbysmith007@…]

Replying to Ryan J Ollos:

Replying to bobbysmith007@…: The difference in hash lengths might be a clue. The first 3 characters of the hash should be used to generate the parent id: tags/trac-1.2.2/trac/attachment.py@:741#L730​.

Where do you see Trac searching for a path with a 6 character hash for the parent id?

I added some more logging and now its looking at the right place. Not sure if I had some sort of cached binary in the migration (that doesn't make too much sense to me as its a new computer with new binaries installed, but maybe I installed an old version on accident and reinstalled or something).

Essentially I guess just ignore the previous comment as everything seems to have resolved itself, with a mild prodding — sorry.

[Ryan J Ollos]

Replying to bobbysmith007@…:

Replying to Ryan J Ollos: I added some more logging and now its looking at the right place. Not sure if I had some sort of cached binary in the migration (that doesn't make too much sense to me as its a new computer with new binaries installed, but maybe I installed an old version on accident and reinstalled or something).

Any additional information you can provide about the steps you took may be useful in tracking down the issue.

• Did you run upgrade before or after moving servers, or before/after upgrading your OS?
• Did you change Python versions or Python architecture (32-bit vs 64-bit)?
• Did the architecture of your server change (32-bit vs 64-bit)?

[Jun Omae]

Replying to bobbysmith007@…:

I just migrated from 1.0 to 1.2 (and from Ubuntu 14.04 to Ubuntu 16.04). When I did all my attachements are misindexed.

db30.py through db41.py are used in upgrade from 1.0 to 1.2 but the attachments directory is not touched in the upgrade modules. Only db28.py changes the attachments directory, which is used in upgrade from 0.12 to 1.0.

Could you please share trac.log in your Trac environment?