Edgewall Software
Modify

Opened 18 years ago

Closed 17 years ago

Last modified 9 years ago

#2553 closed defect (fixed)

tracd reverts to http:// after a POST even if stunnel is being used via a https:// url.

Reported by: anonymous Owned by:
Priority: high Milestone: 0.10.5
Component: web frontend/tracd Version: devel
Severity: major Keywords: tracd https stunnel mod_proxy patch apache mod_python
Cc: jorge.vargas@… Branch:
Release Notes:
API Changes:
Internal Changes:

Description (last modified by Christian Boos)

Running tracd through stunnel, I access trac via a https://url/trac. Everything works fine until I change something (confirm a wiki edit, add a new ticket, login, etc.) and the url forwards to http://url/trac instead of https.

To fix this, I changed the self.scheme = 'http' in TracHTTPRequest in standalone.py to self.scheme='https'. But I imagine this should autodetect in some way.

Status for this issue

This is a problem affecting Trac up to and including 0.10.4.

In current 0.10.5dev and 0.11dev (b1, b2, rc1), there's a new setting in the TracIni#trac-section, use_base_url_for_redirect: when set to true, Trac will use the value set for base_url in the same .ini section as the actual target for the redirects, so you can set it to https://... if you really have no other choices. See also #6013.

Attachments (2)

base_url.patch (660 bytes ) - added by Christopher Lenz 18 years ago.
Use base_url when computing absolute URL of redirection
https_post_redirect2.patch (1.0 KB ) - added by pkoelle@… 17 years ago.
honor base_url in Request.redirect()

Download all attachments as: .zip

Change History (47)

comment:1 by Christian Boos, 18 years ago

Description: modified (diff)
Milestone: 0.9.4
Owner: changed from Jonas Borgström to Christian Boos
Status: newassigned

One could use the Referer: header, if present.

Index: standalone.py
===================================================================
--- standalone.py       (revision 2721)
+++ standalone.py       (working copy)
@@ -325,21 +325,22 @@
         Request.__init__(self)
         self.__handler = handler
         self.__status_sent = False
-
-        self.scheme = 'http'
+        headers = handler.headers
+        self.scheme = 'http' # as default, but use Referer: header if
present
+        if headers.has_key('Referer'):
+            self.scheme = headers['Referer'].split(':')[0]
         self.method = self.__handler.command
         self.remote_addr = str(self.__handler.client_address[0])
         self.server_name = self.__handler.server.server_name
         self.server_port = self.__handler.server.server_port
-        if self.__handler.headers.has_key('Cookie'):
-            self.incookie.load(self.__handler.headers['Cookie'])
+        if headers.has_key('Cookie'):
+            self.incookie.load(headers['Cookie'])
         self.cgi_location = '/' + project_name
         self.idx_location = '/'
         environ = {'REQUEST_METHOD': self.method,
                    'QUERY_STRING': query_string}
-        headers = self.__handler.headers
         if self.method in ('GET', 'HEAD'):
             headers = None
         self.args = TracFieldStorage(self.__handler.rfile,
environ=environ,

Thoughts?

comment:2 by anonymous, 18 years ago

That fix works for my situation. Thanks!

comment:3 by Christopher Lenz, 18 years ago

This is not that great a solution. The primary problem is that the referer could be a totally different site… for example, some site running over HTTPS that is linking a Trac site somewhere else. So at least, we should be checking whether the URL in the referer is pointing to the same host and base path before setting the scheme to https. But even then, I feel that this change is too much of a hack. Can you please try whether setting the base_url option in [wiki:TracIni trac.ini] also fixes the problem (without the patch above)?

[trac]
base_url = https://example.org/trac/

comment:4 by anonymous, 18 years ago

no, that fix does not work— I reverted standalone.py to the 0.9.2 version and added your line in my trac_env/conf/trac.ini file — doesn't work, same problem. I don't see the problem with the referrer approach- when would a non-trac referrer make a POST to update something within trac?

comment:5 by Christopher Lenz, 18 years ago

Can you please try the attached base_url.patch?

comment:6 by anonymous, 18 years ago

With that base_url.patch + adding base_url to trac.ini and the original standalone.py from 0.9.2, it works. Thanks

by Christopher Lenz, 18 years ago

Attachment: base_url.patch added

Use base_url when computing absolute URL of redirection

comment:7 by brian whitman <trac_bwhitman@…>, 18 years ago

Relatedly, I put a tutorial here on how to set up stunnel with trac with a link back to this ticket for the necessary patch.

comment:8 by anonymous, 18 years ago

I'm running trac with apache2 and I also have this probleem. This apache only listens to port 443. When I post something (like a ticket, wiki edit, …) then I get redirect to the non https url. This results in 404. I used the patch attachted to this ticket and it works like a charm now. My config file has now two times the same config value. One time 'project.url' and one time the new 'trac.base_url'. Is this really needed? I run trac 0.9.2 from debian unstable.

comment:9 by Christian Boos, 18 years ago

cmlenz: should I commit your attachment:base_url.patch?

comment:10 by Christian Boos, 18 years ago

I guess yes… did so in r2820.

Should I port this to stable?

comment:11 by Christopher Lenz, 18 years ago

sure

comment:12 by Christian Boos, 18 years ago

Resolution: fixed
Status: assignedclosed

Ported in r2830.

comment:13 by anonymous, 18 years ago

Keywords: multirepository added; stunnel removed
Resolution: fixed
Status: closedreopened

I'm using Trac 0.9.4 on Debian etch, apache2, mod_python, https://URL/. I'm using multiple repositories. When I select repository trac redirect me to http://. Then I have to rewrite http:// to https:// from this point work well. I have also problem with login, but I'm not sure if this same problem.

comment:14 by coderanger, 18 years ago

Resolution: fixed
Status: reopenedclosed

Please open a new ticket, as that problem is unrelated to this one.

comment:15 by JayJ, 18 years ago

That fix works for me.. http://actgeoeng.mod.net.au

comment:16 by irefuse [at] gmail [dot] com, 18 years ago

Keywords: stunnel added; multirepository removed
Milestone: 0.9.40.10
Priority: normalhigh
Resolution: fixed
Status: closedreopened
Version: 0.9.2devel

im running 0.10 and have my base_url set to the https: address of my trac yet when i make any kind of post it still redirects to http: and i dont understand why…this fix was supposed to have been commited as of 0.9.2 correct?

comment:17 by irefuse [at] gmail [dot] com, 18 years ago

i was looking at the source for stable and it looks like that section of the code still has the fix for this yet on trunk, the code is completely different and there looks like there isnt even a check for base_url anywhere in it. i dont really know python but it really doesnt look like there is any check for it.

comment:18 by irefuse [at] gmail [dot] com, 18 years ago

Severity: normalmajor

well i reread the original post of this ticket and noticed that the person who submitted this changed the self.scheme to use https… so i went looking through wsgi and found a similar set of lines of code:

environ['wsgi.url_scheme'] = 'http'
if environ.get('HTTPS', '').lower() in ('yes', 'on', '1'):
    environ['wsgi.url_scheme'] = 'https'

now, it looks to me like this should check to see if the environment is using https and set the scheme to https…but it's not doing a very good job because if i have my environment using https (with the base_url set to my https address) it doesnt use https when POSTing changes such as wiki modifications and such…i change the line:

environ['wsgi.url_scheme'] = 'http'

to

environ['wsgi.url_scheme'] = 'https'

and now everything works fine and dandy..i think there's something wrong with that wsgi check for https there…

comment:19 by Christopher Lenz, 18 years ago

If you submit a form from an HTTPS address, Trac POSTs to an HTTPS address. Only after the POST we do a redirect (and only if the submission was completed successfully).

If Trac doesn't POST to an HTTPS address for you, something's really weird.

comment:20 by Christopher Lenz, 18 years ago

… because the POST uses a server-relative URL (not including the protocol), but the redirect uses an absolute URL (including the protocol).

comment:21 by Christopher Lenz, 18 years ago

See also #3208.

comment:22 by anonymous, 18 years ago

so where exactly would i put something similar to "SetEnv HTTPS 1" if im not using apache, but tracd with stunnel?

comment:23 by Christian Boos, 18 years ago

Milestone: 0.10
Status: reopenednew

I have not really the time to investigate on that one, so if someone wants to take over, he's more than welcome…

comment:24 by Christian Boos, 18 years ago

Owner: Christian Boos removed

comment:25 by Christian Boos, 17 years ago

Milestone: 0.10.4

See also #3381.

comment:26 by pkoelle@…, 17 years ago

Keywords: mod_proxy added

Got the same problem with tracd running begind apache with mod_proxy. From the outside, all projects are https:// but communication between apache and tracd is plain http.

My reasoning would be: If base_url is set, the user knows what she's doing. So Request.redirect() should check base_url for the scheme and host part and fall back to wsgi.url_scheme and server_name when building url from a relative URL…

DISCLAIMER: I'm not very familiar with trac internals ;)

--- trac/web/api.py	2006-12-12 19:44:08.000000000 +0100
+++ /usr/lib/python2.4/site-packages/trac/web/api.py	2006-12-21 05:30:15.000000000 +0100
@@ -325,10 +325,14 @@
 
         self.send_response(status)
         if not url.startswith('http://') and not url.startswith('https://'):
-            # Make sure the URL is absolute
-            url = urlparse.urlunparse((self.scheme,
-                                       urlparse.urlparse(self.base_url)[1],
-                                       url, None, None, None))
+            # Make sure the URL is absolute, honor base_url for
+            # scheme and host if present
+            scheme, host, path, params, query, fragment = urlparse.urlparse(
+                                                                self.base_url)
+            if not scheme: scheme = self.scheme
+            if not host: host = self.server_name
+            url = urlparse.urlunparse((scheme, host, url, None, None, None))                           
+
         self.send_header('Location', url)
         self.send_header('Content-Type', 'text/plain')
         self.send_header('Pragma', 'no-cache')

comment:27 by Christian Boos, 17 years ago

Keywords: patch added

comment:28 by pkoelle@…, 17 years ago

I have further investigated the situation (0.10.2):

  • client (https) → apache with mod_proxy (http) → tracd(localhost)
  • redirects after POST is fine out of the box for both http and https if base_url is NOT set.
  • If base_url is not set, X-Trac-Ticket-URL header for email notifications points to http://localhost:8000/ticket/xxx. It could (partially) fixed by setting "ProxyPreserveHost On" from apache but this breaks https redirects on POST again (why on earth?) and the scheme is still wrong.
  • If base_url is set, X-Trac-Ticket-URL is correct but this breaks POST redirects, a restart of tracd is needed after deleting base_url.

I thought there should be a less intrusive patch, i.e fixing this in the TicketNotifyEmail class. However, the reason X-Trac-Ticket-URL is wrong if base_url is not set is in Request.init() when Request.abs_href is calculated from base_url OR Request._reconstruct_url(). There is no way to get the correct scheme and host in _reconstuct_url() so base_url is needed and I think my patch makes Request.redirect() behave like Request.init wrt using base_url.

A slightly simpler patch (against 0.10.3):

--- trac/web/api.py	2006-12-12 19:44:08.000000000 +0100
+++ /usr/lib/python2.4/site-packages/trac/web/api.py	2006-12-21 07:07:33.000000000 +0100
@@ -325,10 +325,12 @@
 
         self.send_response(status)
         if not url.startswith('http://') and not url.startswith('https://'):
-            # Make sure the URL is absolute
-            url = urlparse.urlunparse((self.scheme,
-                                       urlparse.urlparse(self.base_url)[1],
-                                       url, None, None, None))
+            # Make sure the URL is absolute, honor base_url for
+            # scheme and host if present
+            scheme, host, path, params, query, fragment = urlparse.urlparse(
+                                                              self.abs_href())
+            url = urlparse.urlunparse((scheme, host, url, None, None, None))                           
+
         self.send_header('Location', url)
         self.send_header('Content-Type', 'text/plain')
         self.send_header('Pragma', 'no-cache')

by pkoelle@…, 17 years ago

Attachment: https_post_redirect2.patch added

honor base_url in Request.redirect()

comment:29 by jorge.vargas@…, 17 years ago

Cc: jorge.vargas@… added
Keywords: apache mod_python added

Hi I got the following setup

2 apache instances on a shared machine, one is system the second is on my home running mod_python, and I had the same issue described below in the ticket it "sometimes" reverted back to http.

I have applied pkoelle's patch and it seems to work fine.

original post here http://groups.google.com/group/trac-users/browse_thread/thread/4ebb6cd6555ec2e?hl=en

comment:30 by Blindauer Emmanuel, 17 years ago

I confirm that the patch https_post_redirect2.patch works fine for 10.3

comment:31 by Christian Boos, 17 years ago

#3207 and #3208 closed as duplicates. The latter explains why using HTTPS environment variable is not reliable. That would explain why the patch above works.

comment:32 by Christian Boos, 17 years ago

#4532 closed as duplicate. See also #5064, which explains why we shouldn't always use the configured data to reconstruct the URL.

comment:33 by Christian Boos, 17 years ago

#5454 was closed as duplicate.

comment:34 by Christian Boos, 17 years ago

Resolution: fixed
Status: newclosed

pkoelle's patch applied in r5642 (trunk) and r5644 (0.10-stable).

Thanks for the patch and thanks to those who provided testing feedback.

comment:35 by egbonme, 17 years ago

Resolution: fixed
Status: closedreopened

Hi, I may be missing something but it does not work for me. I just checked 0.10-stable out, which contains the above patches (right?), and a New Ticket submission reverts to a http:// URL instead of a https.

I'm using trac-0.10-stable (r6009) with stunnel 3.26 on a Ubuntu dapper Linux.

comment:36 by Christian Boos, 17 years ago

What's your [trac] base_url setting in TracIni?

comment:37 by egbonme, 17 years ago

empty; is it a problem?

comment:38 by Christian Boos, 17 years ago

Resolution: fixed
Status: reopenedclosed

Currently it is problematic, yes. In trunk, that setting is mandatory in general, and in 0.10-stable it's mandatory in your situation (set the base_url to https:// something). But we're still looking into improving the situation, see #5064.

I'm re-closing this as fixed, as at least the specific use case described here works well with the current code and setting.

in reply to:  38 comment:39 by egbonme, 17 years ago

Replying to cboos:

Currently it is problematic, yes. In trunk, that setting is mandatory in general, and in 0.10-stable it's mandatory in your situation

Thanks. BTW I think it would be useful to also mention the base_url setting in STunnelTracd explicitly.

comment:40 by Christian Boos, 16 years ago

Description: modified (diff)

Summarizing current status of the issue, after marking #7307 as duplicate.

comment:41 by sbwoodside@…, 16 years ago

Thanks cboos. I just upgraded to 0.11rc1 (see #7307) and my https redirects broke. Yes, I use tracd running behind a proxy, I would think this is a common configuration, considering that virtually all Ruby on Rails projects use this configuration.

Can you make a note of this in the docs for 0.11rc1 or 2 or whatever please, it broke when I upgraded and googling did not help me find this fix.

comment:42 by mail@…, 16 years ago

It seems the problem I'm currently facing is related to this bug too: I have the following Apache configuration:

<Location /project/trac>
        SetHandler mod_python
        PythonInterpreter main_interpreter
        PythonHandler trac.web.modpython_frontend
        PythonOption TracEnv /var/lib/trac/project
        PythonOption TracUriRoot /project/trac
        PythonOption Https 1
</Location>
<Location /project>
        RewriteEngine On
        RewriteCond %{HTTPS} off
        RewriteRule (.*) https://mydomain.com%{REQUEST_URI}
</Location>

This means, all requests going to http://mydomain.com/project or http://mydomain.com/project/trac should be redirected to https:// ….

As long as I don't have the SSL rewrite configuration enabled, trac works fine, but as soon as I enable it, the requests go to the htdocs dir, where nothing is found, instead of being done by mod_python.

I've set base_url in trac.ini to https://mydomain.com/project/trac, but it didn't change anything at all.

The version I'm using is 0.11 from Gentoo, built with USE="cgi mysql sqlite subversion vhosts -fastcgi -postgres".

comment:43 by anonymous, 14 years ago

Hi all,

would like to comment on this but akismet apparently doesnt want that. So I'm trying to add text maybe it helps…

I'm running trac 0.11.6 and still have what seems to be this problem.

after form submit, url changes to http://… especially creepy during login not knowing if the password was sent in cleartext or not!

Thanks

comment:44 by anonymous, 10 years ago

I have this problam even i use a new version and have the use_base_url_for_redirect = true in trac.ini

Version 0, edited 10 years ago by anonymous (next)

comment:45 by sd, 9 years ago

Same problem here.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The ticket will remain with no owner.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from (none) to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.