Edgewall Software

Opened 13 years ago

Closed 10 years ago

Last modified 10 years ago

#6656 closed defect (fixed)

CSV export of tickets has wrong encoding — at Version 10

Reported by: jho Owned by: Jun Omae
Priority: normal Milestone: 1.0
Component: report system Version: 0.10.4
Severity: normal Keywords:
Cc: lists@…, jun66j5@… Branch:
Release Notes:

CSV data exports uses UTF-8 BOM for better interoperability

API Changes:
Internal Changes:

Description

When I download a CSV version of a ticket report, all the Umlaut characters (ö,ü,ä, etc.) are displayed incorrectly (i.e.: instead of ä I get ä)

Change History (11)

comment:1 by Christian Boos, 13 years ago

Keywords: consider added
Milestone: 0.10.50.12

The export is currently done using UTF-8 and it's true that Excel for one doesn't like that…

Given that Excel expects UTF16-BE (according to some sources, like http://www.herongyang.com/unicode/utf.html, not yet verified), it might be worth changing if other tools like OpenOffice also understand that encoding.

comment:2 by Thijs Triemstra <lists@…>, 10 years ago

Cc: lists@… added

Doesn't look like OpenOffice (3.2.1) supports utf16-be..

in reply to:  1 comment:3 by Thijs Triemstra <lists@…>, 10 years ago

Owner: changed from Matthew Good to Thijs Triemstra <lists@…>

Replying to cboos:

The export is currently done using UTF-8 and it's true that Excel for one doesn't like that…

Given that Excel expects UTF16-BE (according to some sources, like http://www.herongyang.com/unicode/utf.html, not yet verified), it might be worth changing if other tools like OpenOffice also understand that encoding.

Unless I'm reading that article incorrectly, but it states that "MS Word and Excel don't support this [UTF-16BE] encoding". That same guy wrote this article that states "This section provides a tutorial example to prove that Excel can not open a UTF-16BE text file. Its Text Import Wizard only supports UTF-7 and UTF-8 encodings."

OpenOffice also only support UTF-7 and 8 (and the other encodings that excel also supports) so it looks like this should be fixed, or can be easily fixed.

comment:4 by Thijs Triemstra <lists@…>, 10 years ago

Resolution: worksforme
Status: newclosed

I tested this with OpenOffice and the .csv export below and everything works as expected. Since this was reported against 0.10.4 I expect that this issue has been fixed in the meantime so I'm closing it as worksforme. Please re-open if you can still reproduce it (or if I'm not allowed to close tickets hehe).

__color__,ticket,summary,component,version,milestone,type,owner,status,created,_changetime,_description,_reporter
3,1,Title ticket 1,component1,,,defect,somebody,new,2010-09-29T00:10:30+02:00,2010-10-04T00:43:13+02:00,"[[TracAdminHelp(upgrade)]]

2,sdsadsa
dsadsdsadasdas
",admin
3,2,Title ticket 2,component1,,,defect,somebody,new,2010-09-29T00:10:36+02:00,2010-10-04T00:27:34+02:00,Description ticket 2.,admin
3,3,Title ticküät 3,component1,,,defect,somebody,new,2010-09-29T00:10:43+02:00,2010-10-04T19:37:02+02:00,Descriptiön ticket 3.,admin

comment:5 by Remy Blank, 10 years ago

Wasn't the question rather whether Excel can read UTF-8 or not?

by Jun Omae, 10 years ago

Attachment: t6656-utf8-bom-r10225.diff added

csv export with utf-8 bom patch (r10225)

in reply to:  5 comment:6 by Jun Omae, 10 years ago

I have the same problem and CJK users also. Microsoft Excel read csv files with Shift_JIS in the Japanese version of Windows.

Replying to rblank:

Wasn't the question rather whether Excel can read UTF-8 or not?

Excel cannot read UTF-8 however it can read UTF-8 BOM. I tested with Excel 2003 and 2007 for Japanese version.

t6656-utf8-bom-r10225.diff - CSV export with UTF-8 BOM

comment:7 by Jun Omae, 10 years ago

Cc: jun66j5@… added

comment:8 by Christian Boos, 10 years ago

How do other tools react with this patch? In particular OpenOffice?

comment:9 by Jun Omae, 10 years ago

OpenOffice 3.2 Calc can read UTF-8 w/o BOM and UTF-8 w/ BOM. It works fine.

comment:10 by Christian Boos, 10 years ago

Milestone: next-major-0.1X0.13
Owner: changed from Thijs Triemstra <lists@…> to Jun Omae
Release Notes: modified (diff)

Perfect! I'll apply the patch, then.

Note: See TracTickets for help on using tickets.