gzip attachments are not downloadable

[jkeating@…]

We have a number of projects creating release attachments that are tar.gz files. However clicking on these files leads to a blank page rather than a preview attempt and offer to download as one would expect.

EG ​https://hosted.fedoraproject.org/projects/snake/wiki/SnakeReleases attempt to click on the .tar.gz version and get a blank page. The .tar and .tar.bz2 work.

[Alec Thomas]

Check your Trac logs, and web server logs. This is likely to be a misconfiguration.

[ThurnerRupert]

trying the link on fedoraproject, the download is named "tar.tar", not "tar.gz" as one would expect. browser: ie6.

with firefox it works, the mime-type gzip, and it is translated to gz extension.

[ThurnerRupert]

just wanted to add that it is like that on our solaris installation as well.

[ThurnerRupert]

could sb verify this for 0.11?

[Alec Thomas]

Verify what exactly? I'm still unable to replicate this and still have no info from the poster.

[Emmanuel Blot]

So I tried with both tracd server and a Apache 2.2 + mod_python server, with the latest trunk:
The attachment data of a .tar.gz file is successfully sent and received in both cases.

However, the MIME type sent by Trac appears to be Content-Type: application/x-tar which is notoriously known to cause troubles with many browsers: a .tgz file is not a .tar file, and using a content encoding header may be even more confusing for some browsers. For example Safari appends a .tar to the received file, because of the application/x-tar MIME type.

It seems there is no consensus about how to declare the MIME type of a .tar.gz file:

• Content-Type: application/x-tar (as with Trac)
• Content-Type: application/octet-stream
• Content-Type: application/x-compressed
• Content-Type: application/x-compress, which collides with .Z compression format.
• Content-Type: application/x-tar and
  Content-Encoding: gzip
• Content-Type: application/x-gzip
• …

I think application/octet-stream is probably the safer way, however there's a major drawback: it does not carry any information about the file type.

application/x-tar is probably not a good idea, as the file type is not a tar file anyway.

See ​Metadata for .tar.gz

Apache declares such a file as application/x-gzip. I would favor this MIME type as many browsers are used to deal with the Apache way, MIME type would be coherent - whatever the actual server of a file: Apache or Trac - and the MIME type matches the actual file type, that is a gzipped file whatever the embedded container (tar or anything else).

This could be easily implemented as:

trunk/trac/mimeview/api.py

@@ -229 +229 @@
     'application/pdf':        ['pdf'],
     'application/postscript': ['ps'],
     'application/rtf':        ['rtf'],
+    'application/x-gzip':     ['gz', 'tgz'],
     'application/x-sh':       ['sh'],
     'application/x-csh':      ['csh'],
     'application/x-troff':    ['nroff', 'roff', 'troff'],

[jkeating@…]

I tried making the change in the last comment, but this doesn't seem to help our site. The Trac log just shows:

2007-12-02 11:58:09,069 Trac[attachment] DEBUG: Trying to open attachment at /srv/web/trac/projects/snake/attachments/wiki/SnakeReleases/snake-0.9.tar.gz 2007-12-02 11:58:09,071 Trac[attachment] DEBUG: Rendering preview of file snake-0.9.tar.gz with mime-type application/x-gzip; charset=iso-8859-15 2007-12-02 11:58:09,071 Trac[api] DEBUG: Trying to render HTML preview using PlainTextRenderer 2007-12-02 11:58:09,077 Trac[api] DEBUG: Binary data; no preview available

Compared to the .bzip2 file which works:

2007-12-02 11:57:44,349 Trac[attachment] DEBUG: Trying to open attachment at /srv/web/trac/projects/snake/attachments/wiki/SnakeReleases/snake-0.9.tar.bz2 2007-12-02 11:57:44,388 Trac[attachment] DEBUG: Rendering preview of file snake-0.9.tar.bz2 with mime-type application/x-bzip2; charset=iso-8859-15 2007-12-02 11:57:44,390 Trac[api] DEBUG: Trying to render HTML preview using PlainTextRenderer 2007-12-02 11:57:44,398 Trac[api] DEBUG: Binary data; no preview available

There is little difference.

There is nothing in the http error_log, and this is the entry in access_log:

10.8.32.56 - - [02/Dec/2007:12:00:19 -0700] "GET /projects/snake/attachment/wiki/SnakeReleases/snake-0.9.tar.gz HTTP/1.1" 200 3993 "​https://hosted.fedoraproject.org/projects/snake/wiki/SnakeReleases" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.10) Gecko/20071126 Fedora/2.0.0.10-1.fc8 Firefox/2.0.0.10"

[Emmanuel Blot]

Replying to jkeating@fedoraproject.org:

I tried making the change in the last comment, but this doesn't seem to help our site.

Yeah sorry, the above comment was a reply to the second comment wrote from ThurnerRupert - that is about the filename alteration. The MIME type should not be related to the initial issue. There's probably an invalid setting on your server.

As I wrote, I've been unable to reproduce the initial issue neither with Apache nor tracd with the trunk. I'll try with 0.10.4 to be sure.

Try to disable any installed Trac plugin, and reduce the number of Apache modules (maybe mod_deflate ?)

[anonymous]

Hrm, this is our production instance, so I can't be messing with it too much. I'll try to set up a test site to play with, it may take a while to get the set up to match, and likely it won't reproduce the issue :/

[Noah Kantrowitz]

OLPC is seeing this too. ​http://dev.laptop.org/attachment/ticket/6073/.

The first two both lead to blank pages in Fx and IE, and get downloaded in Safari. The second two are both the same file, the .bz2 shows the preview just fine, the .gz shows the blank page behavior (the file is actually bzip encoded). This seems to be a browser issue, not a server one.

[Christian Boos]

Did you change anything there in the last 5 minutes?

Now it works for me… but when I tried a bit earlier (15 minutes ago or so, it didn't).

[Christian Boos]

Hm, sorry, ignore the above comment, only the .bz2 were working fine with Firefox or IE7. Everything works fine with wget and konqueror.

[Noah Kantrowitz]

A quick fix:

# Created by  on 2008-01-18.
# Copyright (c) 2008 Noah Kantrowitz. All rights reserved.

from genshi.filters.transform import Transformer

from trac.core import *
from trac.web.api import ITemplateStreamFilter
from trac.resource import get_resource_url

class TarGzHackFilter(Component):
    """A hack to fix gzip'd attachments."""

    implements(ITemplateStreamFilter)

    # ITemplateStreamFilter methods
    def filter_stream(self, req, method, filename, stream, data):
        if filename == 'ticket.html':
            for attachment in data['attachments']['attachments']:
                if attachment.filename.endswith('gz'):
                    old_href = get_resource_url(self.env, 
                                  attachment.resource, req.href)
                    new_href = get_resource_url(self.env, 
                                  attachment.resource, req.href, format='raw')
            
                    stream |= Transformer('//a[@title="View attachment" and '\
                                '@href="%s"]'%old_href).attr('href', new_href)
        return stream

[Christian Boos]

This works now in all browsers I've tested. The problem was probably fixed by r6935, together with #6798.

[Christian Boos]

My bad, it probably hasn't anything to do with r6935 and #6798, as the problem was with the links pointing to the preview page (I was mislead by the "gzip attachments are not downloadable" summary).

Anyway, that problem seems to be gone as well with recent browsers (Firefox 3 and IE7).

[walty]

I use Trac 0.11.1, my google chrome got the similar problem, and here is my quick fix in attachment.py:

669             # Eventually send the file directly
670             format = req.args.get('format')
671             if format in ('raw', 'txt'):
672                 if not self.render_unsafe_content:

⇒

669             # Eventually send the file directly
670             format = req.args.get('format')
671             if format in ('raw', 'txt', 'gz'):  #updated by walty
672                 if not self.render_unsafe_content: