Edgewall Software
Modify

Opened 10 years ago

Last modified 5 years ago

#8165 new defect

[PATCH] csv download doesn't process 'description' field

Reported by: pellatt@… Owned by:
Priority: normal Milestone: next-major-releases
Component: report system Version: 0.12dev
Severity: minor Keywords: csv, patch
Cc: Thijs Triemstra Branch:
Release Notes:
API Changes:

Description (last modified by Remy Blank)

In a report that includes the ticket 'description' field, the text is processed by the wiki processor, but when I choose to download the report by clicking on 'comma-delimited-text', the 'description' field in the csv file is *not* processed. E.g. I have a macro which displays its output correctly in the onscreen report, but just shows "[[MyMacro()]]" in the csv file.

Attachments (2)

csv.patch (1.4 KB ) - added by Thijs Triemstra <lists@…> 9 years ago.
csv2.patch (3.1 KB ) - added by Thijs Triemstra <lists@…> 9 years ago.
render description as plain text

Download all attachments as: .zip

Change History (19)

comment:1 by Remy Blank, 10 years ago

Description: modified (diff)

I'm sure I saw a duplicate of this, but I can't find it ATM.

comment:2 by Remy Blank, 10 years ago

Milestone: 1.0

I still can't find it, so yeah, good idea ;-)

comment:3 by Thijs Triemstra <lists@…>, 9 years ago

Cc: lists@… added

Here's a patch that adds the description column, when selected in the ui:

  • trac/ticket/query.py

     
    10991099    def export_csv(self, req, query, sep=',', mimetype='text/plain'):
    11001100        content = StringIO()
    11011101        cols = query.get_columns()
     1102
     1103        if 'description' in query.rows:
     1104            query.cols.insert(2, 'description')
     1105
    11021106        writer = csv.writer(content, delimiter=sep, quoting=csv.QUOTE_MINIMAL)
    11031107        writer.writerow([unicode(c).encode('utf-8') for c in cols])
    11041108

Column's row content still needs to be wiki-fied though.

by Thijs Triemstra <lists@…>, 9 years ago

Attachment: csv.patch added

comment:4 by Thijs Triemstra <lists@…>, 9 years ago

Milestone: unscheduled0.13

Here's a patch that renders the description column as html, with rendered macro output. It would be nice to have this in plain text but I couldn't find a trac api for getting a plain-text output of wiki content. Output:

id,summary,description,status,type,priority,milestone,component
1,sdsad,"<p>
<pre class=""wiki"">upgrade 

    Upgrade database to current version

</pre>
</p>
",new,defect,major,,component1
2,sdsadsa,"<p>
dsadsdsadasdas
</p>
",new,defect,major,,component1
3,sadsadsdasdas,"<p>
sadsadasd
</p>
",new,defect,major,,component1

by Thijs Triemstra <lists@…>, 9 years ago

Attachment: csv2.patch added

render description as plain text

comment:5 by Thijs Triemstra <lists@…>, 9 years ago

Milestone: 0.130.12.2

I attached a new patch that renders the description as plain text instead of html. It also updates the csv export in the report section.

I wasn't sure how to render wiki content as plaintext so I added a utility method that does that. If there's a better way to render markup as text, let me know and I'll update the patch.

comment:6 by Thijs Triemstra <lists@…>, 9 years ago

Keywords: review added
Summary: csv download doesn't process 'description' field[PATCH] csv download doesn't process 'description' field

comment:7 by Remy Blank, 9 years ago

Milestone: 0.12.20.13

I don't think expanded HTML is better than unexpanded wiki text. Expanded plain text would be great, but as you have found out, we don't have a wiki_to_text() (yet). And just stripping the HTML tags would probably have such a bad formatting as to be unusable. More brainstorming needed, I guess.

While this is a defect, it's a big enough change that I'd prefer doing it on trunk. Changing the milestone back to 0.13.

in reply to:  7 comment:8 by Thijs Triemstra, 9 years ago

Cc: Thijs Triemstra added; lists@… removed
Keywords: patch added; review removed

Replying to rblank:

…. Expanded plain text would be great, but as you have found out, we don't have a wiki_to_text() (yet). …

Is there a ticket for this? It would be my nr 1 trac enhancement request.

comment:9 by Remy Blank, 9 years ago

Let's try searching for wiki_to_text… No, there's not ticket explicitly for that.

in reply to:  9 comment:10 by Thijs Triemstra, 9 years ago

Replying to rblank:

Let's try searching for wiki_to_text… No, there's not ticket explicitly for that.

I created #9879 for wiki_to_text().

comment:11 by philipp.meier@…, 9 years ago

Just some more information:


Using a custom query with activated field 'Show under each result: description' produces the expected result in the web browser.

Using the same query but then clicking on 'Download in other formats:' the exported file does not contain the description field or its contents (irrespective of exact format, i.e. comma-delimited or tab-delimited text). This is not the expected result and should be corrected.

comment:12 by Christian Boos, 9 years ago

Milestone: 0.13next-major-0.1X

comment:13 by Christian Boos, 9 years ago

Milestone: next-major-0.1X0.14

comment:14 by AdrianFritz, 9 years ago

  • If manually added &col=description to custom query, you'll get the desired result (when export CSV/TSV you'll have a column with formated text)
    • Example: query?owner=~cboos&status=assigned&col=id&col=summary&col=description results this query

comment:15 by AdrianFritz, 8 years ago

One more though: I think the original idea is to simply download description field contents as originally written (even if it has wiki markup). In this way, proposed patch fulfills the original request. As a use case, we need (very often) wiki marked text to download, mass process contents in a spreadsheet and then import data back to Trac.

in reply to:  15 comment:16 by Christian Boos, 8 years ago

Replying to AdrianFritz:

One more though: I think the original idea is to simply download description field contents as originally written (even if it has wiki markup). In this way, proposed patch fulfills the original request. As a use case, we need (very often) wiki marked text to download, mass process contents in a spreadsheet and then import data back to Trac.

Sounds good. In later versions, we'll be able to "wiki format" using trac-admin, so if some fields of the exported file need further processing, this could be done in a script. Well, doing this processing on one column of a CSV file doesn't seem to be trivial given the lack of utilities for working on CSV files the "Unix way", but the use case you mentioned seems at least as useful as having the ability to export formatted description. Ideally both use cases should be easy to support, let's start with the easiest to implement ;-)

comment:17 by Ryan J Ollos, 5 years ago

Milestone: next-dev-1.1.xnext-major-releases

Retargetting tickets to narrow focus for milestone:1.2. Please move the ticket back to milestone:next-dev-1.1.x if you intend to resolve it by milestone:1.2.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The ticket will remain with no owner.
The ticket will be disowned. Next status will be 'new'.
as The resolution will be set. Next status will be 'closed'.
The owner will be changed from (none) to anonymous. Next status will be 'assigned'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.