[PATCH] csv download doesn't process 'description' field

[pellatt@…]

In a report that includes the ticket 'description' field, the text is processed by the wiki processor, but when I choose to download the report by clicking on 'comma-delimited-text', the 'description' field in the csv file is *not* processed. E.g. I have a macro which displays its output correctly in the onscreen report, but just shows "[[MyMacro()]]" in the csv file.

[Remy Blank]

I'm sure I saw a duplicate of this, but I can't find it ATM.

[Remy Blank]

I still can't find it, so yeah, good idea ;-)

[Thijs Triemstra <lists@…>]

Here's a patch that adds the description column, when selected in the ui:

trac/ticket/query.py

@@ -1099 +1099 @@
     def export_csv(self, req, query, sep=',', mimetype='text/plain'):
         content = StringIO()
         cols = query.get_columns()
+
+        if 'description' in query.rows:
+            query.cols.insert(2, 'description')
+
         writer = csv.writer(content, delimiter=sep, quoting=csv.QUOTE_MINIMAL)
         writer.writerow([unicode(c).encode('utf-8') for c in cols])
 

Column's row content still needs to be wiki-fied though.

[Thijs Triemstra <lists@…>]

Here's a patch that renders the description column as html, with rendered macro output. It would be nice to have this in plain text but I couldn't find a trac api for getting a plain-text output of wiki content. Output:

id,summary,description,status,type,priority,milestone,component
1,sdsad,"<p>
<pre class=""wiki"">upgrade 

    Upgrade database to current version

</pre>
</p>
",new,defect,major,,component1
2,sdsadsa,"<p>
dsadsdsadasdas
</p>
",new,defect,major,,component1
3,sadsadsdasdas,"<p>
sadsadasd
</p>
",new,defect,major,,component1

[Thijs Triemstra <lists@…>]

render description as plain text

[Thijs Triemstra <lists@…>]

I attached a new patch that renders the description as plain text instead of html. It also updates the csv export in the report section.

I wasn't sure how to render wiki content as plaintext so I added a utility method that does that. If there's a better way to render markup as text, let me know and I'll update the patch.

[Remy Blank]

I don't think expanded HTML is better than unexpanded wiki text. Expanded plain text would be great, but as you have found out, we don't have a wiki_to_text() (yet). And just stripping the HTML tags would probably have such a bad formatting as to be unusable. More brainstorming needed, I guess.

While this is a defect, it's a big enough change that I'd prefer doing it on trunk. Changing the milestone back to 0.13.

[Thijs Triemstra]

Replying to rblank:

…. Expanded plain text would be great, but as you have found out, we don't have a wiki_to_text() (yet). …

Is there a ticket for this? It would be my nr 1 trac enhancement request.

[Remy Blank]

Let's try searching for wiki_to_text… No, there's not ticket explicitly for that.

[Thijs Triemstra]

Replying to rblank:

Let's try searching for wiki_to_text… No, there's not ticket explicitly for that.

I created #9879 for wiki_to_text().

[philipp.meier@…]

Just some more information:

---

Using a custom query with activated field 'Show under each result: description' produces the expected result in the web browser.

Using the same query but then clicking on 'Download in other formats:' the exported file does not contain the description field or its contents (irrespective of exact format, i.e. comma-delimited or tab-delimited text). This is not the expected result and should be corrected.

[AdrianFritz]

• If manually added &col=description to custom query, you'll get the desired result (when export CSV/TSV you'll have a column with formated text)
  • Example: query?owner=~cboos&status=assigned&col=id&col=summary&col=description results this query

[AdrianFritz]

One more though: I think the original idea is to simply download description field contents as originally written (even if it has wiki markup). In this way, proposed patch fulfills the original request. As a use case, we need (very often) wiki marked text to download, mass process contents in a spreadsheet and then import data back to Trac.

[Christian Boos]

Replying to AdrianFritz:

One more though: I think the original idea is to simply download description field contents as originally written (even if it has wiki markup). In this way, proposed patch fulfills the original request. As a use case, we need (very often) wiki marked text to download, mass process contents in a spreadsheet and then import data back to Trac.

Sounds good. In later versions, we'll be able to "wiki format" using trac-admin, so if some fields of the exported file need further processing, this could be done in a script. Well, doing this processing on one column of a CSV file doesn't seem to be trivial given the lack of utilities for working on CSV files the "Unix way", but the use case you mentioned seems at least as useful as having the ability to export formatted description. Ideally both use cases should be easy to support, let's start with the easiest to implement ;-)

[Ryan J Ollos]

Retargetting tickets to narrow focus for milestone:1.2. Please move the ticket back to milestone:next-dev-1.1.x if you intend to resolve it by milestone:1.2.