diffing between utf-8<>utf-16 / utf-16<>utf-16 files does not show inline changes.

[verm@…]

When diffing between a utf-8≠utf-16 file or utf-16≠utf-16 file it does not highlight the inline changes correctly. For instance between two utf-8 files if a single character has changed it will show that single character difference. However if any utf-16 file comes into play it shows the change as spanning across the entire line for both (in 'dark' red/green) versus the single character.

I've tried to track this down myself with little luck i'll keep trying. If I do figure it out i'll update this ticket however I'm hoping one of the developers here will have a quick idea as to what it could be. If you require any further information from me or testing of patches i'll be glad to do so.

[verm@…]

I did a bit more digging, and found out another quirk which probably points to the error. Looking at the HTML source for the generated files, i see:

<del>C<del>o<del>m<del>m<del>e<del>n<del>t<del>:<del> <del>0<del>,<del>0<del>

or

<ins>n<ins>s<ins>e<ins>k<ins>i<ins> <ins>(<ins>€n-^{hQ"o)<ins>:<ins>
(note the funny characters are supposed to be kanji but they're getting mangled, they view fine if i view the file directly)}

Anyways — This should be <del>…change…</del> and <ins>…change…</ins> if i'm not mistaken, so hopefully it narrows down where this bug is exactly.

I'll try looking more into it later this evening unless someone else gets to it first.

[verm@…]

"poor mans patch" to fix the problem.

[anonymous]

I've attached changeset-utf16.diff which is just an example to a fix. Everything works as it should now!. The patch is purely intended to point someone with a better idea of what the best solution is. I'm not familiar with the trac source. (yet)

The problem is the files arn't being detected as utf-16-le/be, so they arn't being converted to utf-8 before diffing occurs.

[cmlenz]

This should be fixed in [2376]. Please verify the fix if you have a chance!