Edgewall Software
Modify

Opened 17 years ago

Closed 14 years ago

Last modified 10 years ago

#5585 closed defect (fixed)

bugzilla2trac does not work with bugzilla 3.0

Reported by: dekimsey@… Owned by: Jeff Moreland <hou5e@…>
Priority: normal Milestone: 0.11.7
Component: ticket system Version:
Severity: normal Keywords: consider bugzilla
Cc: Branch:
Release Notes:
API Changes:
Internal Changes:

Description

The bugzilla2trac.py script does nto seem to work with bugzilla 3.0. Using the default version number nor a version of '3' will function. However, it should be noted that using the default bugzilla version in the script, actually gets farther along in the import before failing.

Using Bugzilla v3 schema.
Bugzilla MySQL('*':'*':'*':'*******'): connecting...
Trac SQLite('/home/trac/projects/signin'): connecting...

Cleaning all tickets...
All tickets cleaned...

0. Filtering products...
Traceback (most recent call last):
  File "bugzilla2trac.py", line 894, in <module>
    main()
  File "bugzilla2trac.py", line 891, in main
    convert(BZ_DB, BZ_HOST, BZ_USER, BZ_PASSWORD, TRAC_ENV, TRAC_CLEAN)
  File "bugzilla2trac.py", line 505, in convert
    mysql_cur.execute("SELECT product AS name FROM products")
  File "/usr/lib/python2.5/site-packages/MySQLdb/cursors.py", line 164, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.5/site-packages/MySQLdb/connections.py", line 35, in defaulterrorhandler
    raise errorclass, errorvalue
_mysql_exceptions.OperationalError: (1054, "Unknown column 'product' in 'field list'")
Using Bugzilla v2180 schema.
Bugzilla MySQL('*':'*':'*':'*******'): connecting...
Trac SQLite('/home/trac/projects/signin'): connecting...

Cleaning all tickets...
All tickets cleaned...

0. Filtering products...
  Using products Signin

1. Import severities...
  inserting severity 'blocker' - '1'
  inserting severity 'critical' - '2'
  inserting severity 'major' - '3'
  inserting severity 'normal' - '4'
  inserting severity 'minor' - '5'
  inserting severity 'trivial' - '6'

2. Import components...
  inserting component 'General', owner 'bellaire'
  inserting component 'Scheduling, Preference Forms', owner 'bellaire'
  inserting component 'Sign-in, Timesheets', owner 'bellaire'

3. Import priorities...
  inserting priority 'highest' - '1'
  inserting priority 'high' - '2'
  inserting priority 'normal' - '3'
  inserting priority 'low' - '4'
  inserting priority 'lowest' - '5'

4. Import versions...
  inserting version '.Production'
  inserting version 'Development'
  inserting version 'Release'
  inserting version '2.0'
  inserting version 'unspecified'

5. Import milestones...

6. Retrieving bugs...

7. Import bugs and bug activity...
Traceback (most recent call last):
  File "bugzilla2trac.py", line 894, in <module>
    main()
  File "bugzilla2trac.py", line 891, in main
    convert(BZ_DB, BZ_HOST, BZ_USER, BZ_PASSWORD, TRAC_ENV, TRAC_CLEAN)
  File "bugzilla2trac.py", line 681, in convert
    field_name = trac.getFieldName(mysql_cur, activity['fieldid']).lower()
  File "bugzilla2trac.py", line 431, in getFieldName
    (fieldid))
  File "/usr/lib/python2.5/site-packages/MySQLdb/cursors.py", line 164, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.5/site-packages/MySQLdb/connections.py", line 35, in defaulterrorhandler
    raise errorclass, errorvalue
_mysql_exceptions.OperationalError: (1054, "Unknown column 'fieldid' in 'where clause'")

Attachments (4)

bugzilla_3.0_compatibility.patch (1.3 KB ) - added by dekimsey@… 17 years ago.
in bugzilla 3.0 the pkey of the fielddefs changed. this patch should fix that. additionally, removes a newline from an sql statement that was preventing compile.
bugzilla2trac_debian_lenny.py (32.4 KB ) - added by maxence@… 15 years ago.
Modified bugzilla2trac.py which works under Debian 5.0 (Lenny) with standard Trac and Bugzilla packages.
bugzilla2trac_debian_lenny_DeletedBugAttachment_mod.py (32.5 KB ) - added by Jeff 14 years ago.
Minor modification to the 'debian_lenny' script to skip adding attachments when they were deleted from Bugzilla.
bugzilla2trac-postgresql.py (36.1 KB ) - added by Jeff 14 years ago.
Works with Trac 11.6 using PostGreSQL 8.4 and Bugzilla 3.04.4

Download all attachments as: .zip

Change History (22)

by dekimsey@…, 17 years ago

in bugzilla 3.0 the pkey of the fielddefs changed. this patch should fix that. additionally, removes a newline from an sql statement that was preventing compile.

comment:1 by sid, 16 years ago

Keywords: helpwanted added

Can someone test this out and give feedback on the patch?

comment:2 by dave@…, 16 years ago

Patch fails for me

#patch < patch
(Stripping trailing CRs from patch.)
patching file bugzilla2trac.py
Hunk #2 FAILED at 816.
1 out of 2 hunks FAILED -- saving rejects to file bugzilla2trac.py.rej
#cat bugzilla2trac.py.rej
***************
*** 812,819 ****
          ticketid = trac.addTicket(**ticket)

          if BZ_VERSION >= 2180:
-             mysql_cur.execute("SELECT attachments.*, attach_data.thedata FROM attachments, attach_data
-                     WHERE attachments.bug_id = %s AND attachments.attach_id = attach_data.id" % bugid)
          else:
              mysql_cur.execute("SELECT * FROM attachments WHERE bug_id = %s" % bugid)
          attachments = mysql_cur.fetchall()
--- 816,822 ----
          ticketid = trac.addTicket(**ticket)

          if BZ_VERSION >= 2180:
+             mysql_cur.execute("SELECT attachments.*, attach_data.thedata FROM attachments, attach_data WHERE attachments.bug_id = %s AND attachments.attach_id = attach_data.id" % bugid)
          else:
              mysql_cur.execute("SELECT * FROM attachments WHERE bug_id = %s" % bugid)
          attachments = mysql_cur.fetchall()

Running Version: 0.10.3-1etch2

Last edited 10 years ago by Ryan J Ollos (previous) (diff)

in reply to:  2 comment:3 by dave@…, 16 years ago

redownloaded original script and patch from here - can apply patch but get the following erro:

  1. Import bugs and bug activity…
    Traceback (most recent call last):
      File "./bugzilla2trac.py", line 898, in ?
        main()
      File "./bugzilla2trac.py", line 895, in main
        convert(BZ_DB, BZ_HOST, BZ_USER, BZ_PASSWORD, TRAC_ENV, TRAC_CLEAN)
      File "./bugzilla2trac.py", line 685, in convert
        field_name = trac.getFieldName(mysql_cur, activity['fieldid']).lower()
      File "./bugzilla2trac.py", line 435, in getFieldName
        (fieldid))
      File "/usr/lib/python2.4/site-packages/MySQLdb/cursors.py", line 163, in execute
        self.errorhandler(self, exc, value)
      File "/usr/lib/python2.4/site-packages/MySQLdb/connections.py", line 35, in defaulterrorhandler
        raise errorclass, errorvalue
    _mysql_exceptions.OperationalError: (1054, "Unknown column 'fieldid' in 'where clause'")
    MONITOR_SERVER#                                                                                   }}}
    
Last edited 10 years ago by Ryan J Ollos (previous) (diff)

comment:4 by Jeroen Ruigrok van der Werven, 16 years ago

Owner: changed from Jonas Borgström to Jeroen Ruigrok van der Werven
Status: newassigned

comment:5 by Piotr Kuczynski <piotr.kuczynski@…>, 16 years ago

Keywords: bugzilla added

comment:6 by Piotr Kuczynski <piotr.kuczynski@…>, 16 years ago

Component: generalticket system
Milestone: 0.12

comment:7 by Larry Silverman, 16 years ago

I'm running Bugzilla 3.0, also hitting this issue. I think you might need to remove the tostring() on the filedata = StringIO.StringIO line.

comment:8 by maxence@…, 15 years ago

I've also run into problems when trying to migrate from Bugzilla 3.0 under Debian 5.0 (Lenny). Note that I'm running standard APT packages for both Trac (0.11.1-2.1) and Bugzilla (3.0.4.1-2+lenny1).

I'm attaching a modified version of bugzilla2trac.py that worked flawlessy for me, hoping that it will save time to other people.

Besides the 'fieldid' column modification, I had to rename a SQL statement that spelled the 'ticket' table 'Ticket', causing a MySQL query to fail.

With only minor modifications, bugzilla2trac.py could work out-of-the-box, which would make the migration experience much smoother.

by maxence@…, 15 years ago

Modified bugzilla2trac.py which works under Debian 5.0 (Lenny) with standard Trac and Bugzilla packages.

comment:9 by anonymous, 15 years ago

Wow! It works for me with bugzilla 3.0.4, after changing BZ_VERSION to 2233

comment:10 by Christian Boos, 15 years ago

Milestone: 0.13not applicable

comment:11 by NickLeverton, 15 years ago

I can confirm that the bugzilla2trac contained in the Trac 0.11.5 tarball fails on Bugzilla 3.0 (also installed from tarball), with the fieldid error dump given here.

I can also confirm that the script in the bugzilla2trac_debian_lenny.py works for my own non-Debian installation of Trac 0.11.5 and BZ3.0. BZ 3.0 was released some years ago now.

I think, therefore, that this fix needs to go into Trac's contrib scripts to make bugzilla2trac.py usable with modern Bugzillas.

comment:12 by Michael Akinde, 14 years ago

The debian_lenny script fix unfortunately did not work for me (Bugzilla 3.4.2 + Ubuntu 8.04 standard setup from debian packages, i.e., Trac 0.10.4).

It required significant changes and workarounds (a good deal of them hardcoded, so I will not be posting it here) in the script to get it to work, and I had to forget about moving attachments. Not having a working migrating script is a major pain, so IMO this really ought to be prioritized a bit higher.

comment:13 by Jeroen Ruigrok van der Werven, 14 years ago

Owner: Jeroen Ruigrok van der Werven removed
Status: assignednew

comment:14 by Jeff, 14 years ago

The 'debian_lenny' script fix had 2 problems for my setup (Bugzilla 3.4.4 + Fedora 12 gnome, Trac 0.11.6 using PostgreSQL).

  1. Has an error after importing the first ticket into PostgreSQL. My workaround was to settle for using the default SQLite database, and it would work after fixing the other problem below. Error msg:
    ...
    6. Retrieving bugs...
    
    7. Import bugs and bug activity...
      inserting ticket 1 -- the database has no bugs!
    Traceback (most recent call last):
      File "bugzilla2trac.py", line 907, in <module>
        main()
      File "bugzilla2trac.py", line 904, in main
        convert(BZ_DB, BZ_HOST, BZ_USER, BZ_PASSWORD, TRAC_ENV, TRAC_CLEAN)
      File "bugzilla2trac.py", line 820, in convert
        ticketid = trac.addTicket(**ticket)
      File "bugzilla2trac.py", line 349, in addTicket
        return self.db().get_last_id(c, 'ticket')
      File "/usr/lib/python2.6/site-packages/trac/db/postgres_backend.py", line 220, in get_last_id
        cursor.execute("SELECT CURRVAL('%s_%s_seq')" % (table, column))
      File "/usr/lib/python2.6/site-packages/trac/db/util.py", line 65, in execute
        return self.cursor.execute(sql)
      File "/usr/lib/python2.6/site-packages/trac/db/util.py", line 65, in execute
        return self.cursor.execute(sql)
    psycopg2.OperationalError: currval of sequence "ticket_id_seq" is not yet defined in this session
    
  1. The script would fail when an attachment was deleted in Bugzilla. I'll attach my minor fix that worked for me. Hopefully it works well enough for others. This was the error I was getting without my minor modification to skip adding attachments:
    6. Retrieving bugs...
    
    7. Import bugs and bug activity...
      inserting ticket 1 -- ...
      ...
      inserting ticket 38 -- Provider Tab: Deleting acct causes probs
        ->inserting attachment '' for ticket 38 -- video showing steps.
    Traceback (most recent call last):
      File "bugzilla2trac.py", line 906, in <module>
        main()
      File "bugzilla2trac.py", line 903, in main
        convert(BZ_DB, BZ_HOST, BZ_USER, BZ_PASSWORD, TRAC_ENV, TRAC_CLEAN)
      File "bugzilla2trac.py", line 829, in convert
        trac.addAttachment(author, a)
      File "bugzilla2trac.py", line 407, in addAttachment
        attachment.insert(filename, filedata, filesize, datetime2epoch(time))
      File "/usr/lib/python2.6/site-packages/trac/attachment.py", line 222, in insert
        filename))
      File "/usr/lib/python2.6/site-packages/trac/util/__init__.py", line 227, in create_unique_file
        return path, os.fdopen(os.open(path, flags, 0666), 'w')
    OSError: [Errno 21] Is a directory: '/var/trac/test1/attachments/ticket/38/'
    

Please make this process less painful… This script is almost there for converting from Bugzilla v3. It's makes getting away from Bugzilla a much harder sell if this script doesn't work easily. Thanks!

by Jeff, 14 years ago

Minor modification to the 'debian_lenny' script to skip adding attachments when they were deleted from Bugzilla.

comment:15 by Christian Boos, 14 years ago

Keywords: consider added; helpwanted removed
Milestone: not applicable0.11.7

That change can be integrated, yes, but before that please put your contact information as the last author in the script.

Also, IIUC, your script still won't work with PostgreSQL but only with SQLite as a backend, right? If that's the case, please add a comment at the beginning of the script making this clear (it only talks about MySQL for now).

In the long run, it would be nice to import from exported Mozilla data instead of looking directly in their database… (see also #8383).

comment:16 by Jeff, 14 years ago

A co-worker came up with the fix to my problem 1.) above for my setup (Bugzilla 3.4.4 + Fedora 12 gnome, Trac 0.11.6 using Postgresql-8.4.2-1.fc12.x86_64). He used the current head revision of the bugzilla2trac.py script, but it was still missing a critical fix in comment 12 of #2061 that is necessary for import into PostgreSQL. This fix, skipping over deleted attachments, and some others from other Trac tickets, are in the attached version of the script (bugzilla2trac-postgresql.py). I tested this and it worked for my setup on both PostgreSQL and SQLite. I didn't test this script with MySQL.

by Jeff, 14 years ago

Attachment: bugzilla2trac-postgresql.py added

Works with Trac 11.6 using PostGreSQL 8.4 and Bugzilla 3.04.4

in reply to:  16 comment:17 by Christian Boos, 14 years ago

Resolution: fixed
Status: newclosed

Replying to Jeff:

This fix, skipping over deleted attachments, and some others from other Trac tickets, are in the attached version of the script (bugzilla2trac-postgresql.py). I tested this and it worked for my setup on both PostgreSQL and SQLite. I didn't test this script with MySQL.

Great! Thanks for contributing this new version. Committed as [9168].

comment:18 by Christian Boos, 14 years ago

Owner: set to Jeff Moreland <hou5e@…>

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Jeff Moreland <hou5e@…>.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from Jeff Moreland <hou5e@…> to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.