Whitespace normalization of milestone/component names breaks on upgrade

[andres@…]

The whitespace normalization "fixes" for #7704 in r8399 unfortunately break upgrades.

If in an earlier version a milestone was saved with a name like 'foo bar' (i.e. two whitespaces) the search now normalizes the whitespace and doesnt find the milestone.

Thus *no* ticket can be edited anymore - I guess because the full list of milestones is built there, so even if only one such unnormalized component/milestone breaks all tickets.

8399     rblank         name = simplify_whitespace(name)
  1831     cmlenz         if name:
  1831     cmlenz             if not db:
  1831     cmlenz                 db = self.env.get_db_cnx()
  1831     cmlenz             cursor = db.cursor()
  1831     cmlenz             cursor.execute("SELECT value FROM enum WHERE type=%s AND name=%s",
  1831     cmlenz                            (self.type, name))

It doesnt seem to be a good idea to do the normalization uppon searching because that will fail with earlier created milestones. Whats the point anyway? Imho that should only be done while saving.

[andres <andres@…>]

If wanted I can writeup a patch removing all the normalization uppon retrieval - its a trivial one though, so I am not sure its worth it.

[Remy Blank]

Could you please describe the symptoms you get? "No ticket can be edited anymore" is not very precise. Do you get an error? Can you provide the backtrace? Can you provide a procedure to reproduce the issue?

Replying to andres <andres@…>:

If wanted I can writeup a patch removing all the normalization uppon retrieval - its a trivial one though, so I am not sure its worth it.

Yes, please do.

Assigning to 0.12 as this is a regression.

[Remy Blank]

Just to clarify: I believe you are right that normalization should be done only when saving, and not when retrieving. But I am interested to know how this issue manifests itself on your installation.

[andres <andres@…>]

No Backtraces. The error is the (not unobvious)

Error: Invalid milestone name

either when opening "admin/ticket/milestones/<something with two whitespaces>" or when editing a ticket when one of the possible milestones has two whitespaces. Its the exception from

       cursor = db.cursor()
        cursor.execute("""                                                                                           
            SELECT name,due,completed,description                                                                    
            FROM milestone WHERE name=%s                                                                             
            """, (name,))
        row = cursor.fetchone()
        if not row:
            raise ResourceNotFound(_('Milestone %(name)s does not exist.',
                                   name=name), _('Invalid milestone name'))

both of the time.

For reproduction just manually insert a record with two whitespaces (or use an older version, or one without the normalization to insert it).

[andres <andres@…>]

Btw, the "not unsurprisingly" shouldnt sound aggressive or whatnot :P

[Remy Blank]

Thanks for the additional info. Your patch, while fixing the retrieval, unfortunately removes whitespace normalization altogether. The correct behavior, as you mentioned, is probably to normalize when creating a new milestone / component / …

[andres <andres@…>]

Huh? insert() and update() do normalize again already?

[Remy Blank]

Sorry, my bad. I read the patch out of context.

You're absolutely right, we shouldn't normalize in the constructor at all, and my adding that in [8399] was a mistake, triggered by enums and components already having that normalization in the constructor.

Your patch was applied in [9585], thanks!