Edgewall Software
Modify

Opened 12 years ago

Closed 10 years ago

Last modified 3 years ago

#2553 closed defect (fixed)

tracd reverts to http:// after a POST even if stunnel is being used via a https:// url.

Reported by: anonymous Owned by:
Priority: high Milestone: 0.10.5
Component: web frontend/tracd Version: devel
Severity: major Keywords: tracd https stunnel mod_proxy patch apache mod_python
Cc: jorge.vargas@…
Release Notes:
API Changes:

Description (last modified by Christian Boos)

Running tracd through stunnel, I access trac via a https://url/trac. Everything works fine until I change something (confirm a wiki edit, add a new ticket, login, etc.) and the url forwards to http://url/trac instead of https.

To fix this, I changed the self.scheme = 'http' in TracHTTPRequest in standalone.py to self.scheme='https'. But I imagine this should autodetect in some way.

Status for this issue

This is a problem affecting Trac up to and including 0.10.4.

In current 0.10.5dev and 0.11dev (b1, b2, rc1), there's a new setting in the TracIni#trac-section, use_base_url_for_redirect: when set to true, Trac will use the value set for base_url in the same .ini section as the actual target for the redirects, so you can set it to https://... if you really have no other choices. See also #6013.

Attachments (2)

base_url.patch (660 bytes ) - added by Christopher Lenz 12 years ago.
Use base_url when computing absolute URL of redirection
https_post_redirect2.patch (1.0 KB ) - added by pkoelle@… 11 years ago.
honor base_url in Request.redirect()

Download all attachments as: .zip

Change History (47)

comment:1 Changed 12 years ago by Christian Boos

Description: modified (diff)
Milestone: 0.9.4
Owner: changed from Jonas Borgström to Christian Boos
Status: newassigned

One could use the Referer: header, if present.

Index: standalone.py
===================================================================
--- standalone.py       (revision 2721)
+++ standalone.py       (working copy)
@@ -325,21 +325,22 @@
         Request.__init__(self)
         self.__handler = handler
         self.__status_sent = False
-
-        self.scheme = 'http'
+        headers = handler.headers
+        self.scheme = 'http' # as default, but use Referer: header if
present
+        if headers.has_key('Referer'):
+            self.scheme = headers['Referer'].split(':')[0]
         self.method = self.__handler.command
         self.remote_addr = str(self.__handler.client_address[0])
         self.server_name = self.__handler.server.server_name
         self.server_port = self.__handler.server.server_port
-        if self.__handler.headers.has_key('Cookie'):
-            self.incookie.load(self.__handler.headers['Cookie'])
+        if headers.has_key('Cookie'):
+            self.incookie.load(headers['Cookie'])
         self.cgi_location = '/' + project_name
         self.idx_location = '/'
         environ = {'REQUEST_METHOD': self.method,
                    'QUERY_STRING': query_string}
-        headers = self.__handler.headers
         if self.method in ('GET', 'HEAD'):
             headers = None
         self.args = TracFieldStorage(self.__handler.rfile,
environ=environ,

Thoughts?

comment:2 Changed 12 years ago by anonymous

That fix works for my situation. Thanks!

comment:3 Changed 12 years ago by Christopher Lenz

This is not that great a solution. The primary problem is that the referer could be a totally different site… for example, some site running over HTTPS that is linking a Trac site somewhere else. So at least, we should be checking whether the URL in the referer is pointing to the same host and base path before setting the scheme to https. But even then, I feel that this change is too much of a hack. Can you please try whether setting the base_url option in [wiki:TracIni trac.ini] also fixes the problem (without the patch above)?

[trac]
base_url = https://example.org/trac/

comment:4 Changed 12 years ago by anonymous

no, that fix does not work— I reverted standalone.py to the 0.9.2 version and added your line in my trac_env/conf/trac.ini file — doesn't work, same problem. I don't see the problem with the referrer approach- when would a non-trac referrer make a POST to update something within trac?

comment:5 Changed 12 years ago by Christopher Lenz

Can you please try the attached base_url.patch?

comment:6 Changed 12 years ago by anonymous

With that base_url.patch + adding base_url to trac.ini and the original standalone.py from 0.9.2, it works. Thanks

Changed 12 years ago by Christopher Lenz

Attachment: base_url.patch added

Use base_url when computing absolute URL of redirection

comment:7 Changed 12 years ago by brian whitman <trac_bwhitman@…>

Relatedly, I put a tutorial here on how to set up stunnel with trac with a link back to this ticket for the necessary patch.

comment:8 Changed 12 years ago by anonymous

I'm running trac with apache2 and I also have this probleem. This apache only listens to port 443. When I post something (like a ticket, wiki edit, …) then I get redirect to the non https url. This results in 404. I used the patch attachted to this ticket and it works like a charm now. My config file has now two times the same config value. One time 'project.url' and one time the new 'trac.base_url'. Is this really needed? I run trac 0.9.2 from debian unstable.

comment:9 Changed 12 years ago by Christian Boos

cmlenz: should I commit your attachment:base_url.patch?

comment:10 Changed 12 years ago by Christian Boos

I guess yes… did so in r2820.

Should I port this to stable?

comment:11 Changed 12 years ago by Christopher Lenz

sure

comment:12 Changed 12 years ago by Christian Boos

Resolution: fixed
Status: assignedclosed

Ported in r2830.

comment:13 Changed 12 years ago by anonymous

Keywords: multirepository added; stunnel removed
Resolution: fixed
Status: closedreopened

I'm using Trac 0.9.4 on Debian etch, apache2, mod_python, https://URL/. I'm using multiple repositories. When I select repository trac redirect me to http://. Then I have to rewrite http:// to https:// from this point work well. I have also problem with login, but I'm not sure if this same problem.

comment:14 Changed 12 years ago by coderanger

Resolution: fixed
Status: reopenedclosed

Please open a new ticket, as that problem is unrelated to this one.

comment:15 Changed 11 years ago by JayJ

That fix works for me.. http://actgeoeng.mod.net.au

comment:16 Changed 11 years ago by irefuse [at] gmail [dot] com

Keywords: stunnel added; multirepository removed
Milestone: 0.9.40.10
Priority: normalhigh
Resolution: fixed
Status: closedreopened
Version: 0.9.2devel

im running 0.10 and have my base_url set to the https: address of my trac yet when i make any kind of post it still redirects to http: and i dont understand why…this fix was supposed to have been commited as of 0.9.2 correct?

comment:17 Changed 11 years ago by irefuse [at] gmail [dot] com

i was looking at the source for stable and it looks like that section of the code still has the fix for this yet on trunk, the code is completely different and there looks like there isnt even a check for base_url anywhere in it. i dont really know python but it really doesnt look like there is any check for it.

comment:18 Changed 11 years ago by irefuse [at] gmail [dot] com

Severity: normalmajor

well i reread the original post of this ticket and noticed that the person who submitted this changed the self.scheme to use https… so i went looking through wsgi and found a similar set of lines of code:

environ['wsgi.url_scheme'] = 'http'
if environ.get('HTTPS', '').lower() in ('yes', 'on', '1'):
    environ['wsgi.url_scheme'] = 'https'

now, it looks to me like this should check to see if the environment is using https and set the scheme to https…but it's not doing a very good job because if i have my environment using https (with the base_url set to my https address) it doesnt use https when POSTing changes such as wiki modifications and such…i change the line:

environ['wsgi.url_scheme'] = 'http'

to

environ['wsgi.url_scheme'] = 'https'

and now everything works fine and dandy..i think there's something wrong with that wsgi check for https there…

comment:19 Changed 11 years ago by Christopher Lenz

If you submit a form from an HTTPS address, Trac POSTs to an HTTPS address. Only after the POST we do a redirect (and only if the submission was completed successfully).

If Trac doesn't POST to an HTTPS address for you, something's really weird.

comment:20 Changed 11 years ago by Christopher Lenz

… because the POST uses a server-relative URL (not including the protocol), but the redirect uses an absolute URL (including the protocol).

comment:21 Changed 11 years ago by Christopher Lenz

See also #3208.

comment:22 Changed 11 years ago by anonymous

so where exactly would i put something similar to "SetEnv HTTPS 1" if im not using apache, but tracd with stunnel?

comment:23 Changed 11 years ago by Christian Boos

Milestone: 0.10
Status: reopenednew

I have not really the time to investigate on that one, so if someone wants to take over, he's more than welcome…

comment:24 Changed 11 years ago by Christian Boos

Owner: Christian Boos deleted

comment:25 Changed 11 years ago by Christian Boos

Milestone: 0.10.4

See also #3381.

comment:26 Changed 11 years ago by pkoelle@…

Keywords: mod_proxy added

Got the same problem with tracd running begind apache with mod_proxy. From the outside, all projects are https:// but communication between apache and tracd is plain http.

My reasoning would be: If base_url is set, the user knows what she's doing. So Request.redirect() should check base_url for the scheme and host part and fall back to wsgi.url_scheme and server_name when building url from a relative URL…

DISCLAIMER: I'm not very familiar with trac internals ;)

--- trac/web/api.py	2006-12-12 19:44:08.000000000 +0100
+++ /usr/lib/python2.4/site-packages/trac/web/api.py	2006-12-21 05:30:15.000000000 +0100
@@ -325,10 +325,14 @@
 
         self.send_response(status)
         if not url.startswith('http://') and not url.startswith('https://'):
-            # Make sure the URL is absolute
-            url = urlparse.urlunparse((self.scheme,
-                                       urlparse.urlparse(self.base_url)[1],
-                                       url, None, None, None))
+            # Make sure the URL is absolute, honor base_url for
+            # scheme and host if present
+            scheme, host, path, params, query, fragment = urlparse.urlparse(
+                                                                self.base_url)
+            if not scheme: scheme = self.scheme
+            if not host: host = self.server_name
+            url = urlparse.urlunparse((scheme, host, url, None, None, None))                           
+
         self.send_header('Location', url)
         self.send_header('Content-Type', 'text/plain')
         self.send_header('Pragma', 'no-cache')

comment:27 Changed 11 years ago by Christian Boos

Keywords: patch added

comment:28 Changed 11 years ago by pkoelle@…

I have further investigated the situation (0.10.2):

  • client (https) → apache with mod_proxy (http) → tracd(localhost)
  • redirects after POST is fine out of the box for both http and https if base_url is NOT set.
  • If base_url is not set, X-Trac-Ticket-URL header for email notifications points to http://localhost:8000/ticket/xxx. It could (partially) fixed by setting "ProxyPreserveHost On" from apache but this breaks https redirects on POST again (why on earth?) and the scheme is still wrong.
  • If base_url is set, X-Trac-Ticket-URL is correct but this breaks POST redirects, a restart of tracd is needed after deleting base_url.

I thought there should be a less intrusive patch, i.e fixing this in the TicketNotifyEmail class. However, the reason X-Trac-Ticket-URL is wrong if base_url is not set is in Request.init() when Request.abs_href is calculated from base_url OR Request._reconstruct_url(). There is no way to get the correct scheme and host in _reconstuct_url() so base_url is needed and I think my patch makes Request.redirect() behave like Request.init wrt using base_url.

A slightly simpler patch (against 0.10.3):

--- trac/web/api.py	2006-12-12 19:44:08.000000000 +0100
+++ /usr/lib/python2.4/site-packages/trac/web/api.py	2006-12-21 07:07:33.000000000 +0100
@@ -325,10 +325,12 @@
 
         self.send_response(status)
         if not url.startswith('http://') and not url.startswith('https://'):
-            # Make sure the URL is absolute
-            url = urlparse.urlunparse((self.scheme,
-                                       urlparse.urlparse(self.base_url)[1],
-                                       url, None, None, None))
+            # Make sure the URL is absolute, honor base_url for
+            # scheme and host if present
+            scheme, host, path, params, query, fragment = urlparse.urlparse(
+                                                              self.abs_href())
+            url = urlparse.urlunparse((scheme, host, url, None, None, None))                           
+
         self.send_header('Location', url)
         self.send_header('Content-Type', 'text/plain')
         self.send_header('Pragma', 'no-cache')

Changed 11 years ago by pkoelle@…

Attachment: https_post_redirect2.patch added

honor base_url in Request.redirect()

comment:29 Changed 11 years ago by jorge.vargas@…

Cc: jorge.vargas@… added
Keywords: apache mod_python added

Hi I got the following setup

2 apache instances on a shared machine, one is system the second is on my home running mod_python, and I had the same issue described below in the ticket it "sometimes" reverted back to http.

I have applied pkoelle's patch and it seems to work fine.

original post here http://groups.google.com/group/trac-users/browse_thread/thread/4ebb6cd6555ec2e?hl=en

comment:30 Changed 11 years ago by Blindauer Emmanuel

I confirm that the patch https_post_redirect2.patch works fine for 10.3

comment:31 Changed 11 years ago by Christian Boos

#3207 and #3208 closed as duplicates. The latter explains why using HTTPS environment variable is not reliable. That would explain why the patch above works.

comment:32 Changed 11 years ago by Christian Boos

#4532 closed as duplicate. See also #5064, which explains why we shouldn't always use the configured data to reconstruct the URL.

comment:33 Changed 10 years ago by Christian Boos

#5454 was closed as duplicate.

comment:34 Changed 10 years ago by Christian Boos

Resolution: fixed
Status: newclosed

pkoelle's patch applied in r5642 (trunk) and r5644 (0.10-stable).

Thanks for the patch and thanks to those who provided testing feedback.

comment:35 Changed 10 years ago by egbonme

Resolution: fixed
Status: closedreopened

Hi, I may be missing something but it does not work for me. I just checked 0.10-stable out, which contains the above patches (right?), and a New Ticket submission reverts to a http:// URL instead of a https.

I'm using trac-0.10-stable (r6009) with stunnel 3.26 on a Ubuntu dapper Linux.

comment:36 Changed 10 years ago by Christian Boos

What's your [trac] base_url setting in TracIni?

comment:37 Changed 10 years ago by egbonme

empty; is it a problem?

comment:38 Changed 10 years ago by Christian Boos

Resolution: fixed
Status: reopenedclosed

Currently it is problematic, yes. In trunk, that setting is mandatory in general, and in 0.10-stable it's mandatory in your situation (set the base_url to https:// something). But we're still looking into improving the situation, see #5064.

I'm re-closing this as fixed, as at least the specific use case described here works well with the current code and setting.

comment:39 in reply to:  38 Changed 10 years ago by egbonme

Replying to cboos:

Currently it is problematic, yes. In trunk, that setting is mandatory in general, and in 0.10-stable it's mandatory in your situation

Thanks. BTW I think it would be useful to also mention the base_url setting in STunnelTracd explicitly.

comment:40 Changed 9 years ago by Christian Boos

Description: modified (diff)

Summarizing current status of the issue, after marking #7307 as duplicate.

comment:41 Changed 9 years ago by sbwoodside@…

Thanks cboos. I just upgraded to 0.11rc1 (see #7307) and my https redirects broke. Yes, I use tracd running behind a proxy, I would think this is a common configuration, considering that virtually all Ruby on Rails projects use this configuration.

Can you make a note of this in the docs for 0.11rc1 or 2 or whatever please, it broke when I upgraded and googling did not help me find this fix.

comment:42 Changed 9 years ago by mail@…

It seems the problem I'm currently facing is related to this bug too: I have the following Apache configuration:

<Location /project/trac>
        SetHandler mod_python
        PythonInterpreter main_interpreter
        PythonHandler trac.web.modpython_frontend
        PythonOption TracEnv /var/lib/trac/project
        PythonOption TracUriRoot /project/trac
        PythonOption Https 1
</Location>
<Location /project>
        RewriteEngine On
        RewriteCond %{HTTPS} off
        RewriteRule (.*) https://mydomain.com%{REQUEST_URI}
</Location>

This means, all requests going to http://mydomain.com/project or http://mydomain.com/project/trac should be redirected to https:// ….

As long as I don't have the SSL rewrite configuration enabled, trac works fine, but as soon as I enable it, the requests go to the htdocs dir, where nothing is found, instead of being done by mod_python.

I've set base_url in trac.ini to https://mydomain.com/project/trac, but it didn't change anything at all.

The version I'm using is 0.11 from Gentoo, built with USE="cgi mysql sqlite subversion vhosts -fastcgi -postgres".

comment:43 Changed 8 years ago by anonymous

Hi all,

would like to comment on this but akismet apparently doesnt want that. So I'm trying to add text maybe it helps…

I'm running trac 0.11.6 and still have what seems to be this problem.

after form submit, url changes to http://… especially creepy during login not knowing if the password was sent in cleartext or not!

Thanks

comment:44 Changed 4 years ago by anonymous

I have this problam even i use a new version and have the use_base_url_for_redirect = true in trac.ini.

Last edited 3 years ago by Ryan J Ollos (previous) (diff)

comment:45 Changed 3 years ago by sd

Same problem here.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The ticket will remain with no owner.
The resolution will be deleted.
to The owner will be changed from (none) to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .

 
Note: See TracTickets for help on using tickets.