Edgewall Software
Modify

Opened 11 years ago

Closed 11 years ago

Last modified 8 years ago

#9990 closed task (fixed)

Release Trac 0.12.2

Reported by: Christian Boos Owned by:
Priority: normal Milestone: 0.12.2
Component: general Version: 0.12dev
Severity: normal Keywords: release
Cc: Thijs Triemstra Branch:
Release Notes:
API Changes:
Internal Changes:

Description (last modified by Christian Boos)

This task ticket is used to coordinate the finalization and testing of the next stable version of Trac, 0.12.2.

We're entering a one week period of testing for the rc1, tagged in [10465]. Packages can be downloaded from TracDownload#LatestReleaseCandidate.

If everything goes well, the release of 0.12.2 will happen the 29th of January, 2011.

Testing

If you'd like to contribute, please fill an entry in the table in TracDev/ReleaseTesting, detailing the configuration(s) you will test, and report the status of your testing in this ticket.

But you can also report glitches found with the rc1 directly here. If the problem is serious enough, you can also directly create a new ticket against 0.12.2.

Attachments (1)

t9990-tracd-shutdown.diff (4.3 KB ) - added by Jun Omae 11 years ago.

Download all attachments as: .zip

Change History (43)

comment:1 by Christian Boos, 11 years ago

Type: defecttask

comment:2 by Christian Boos, 11 years ago

0.12.2rc1 deployed here for demo-0.12 and [Babel:]. I'm also testing locally and if all seems fine, I'll tag and release the rc1 a bit later today. In the meantime, feedback on r10464 is welcome.

comment:3 by Christian Boos, 11 years ago

Description: modified (diff)

0.12.2rc1 version tagged and corresponding packages uploaded.

comment:4 by Christian Boos, 11 years ago

I noticed an issue on Windows with tracd.exe installed using Trac-0.12.2rc1.win32.exe into either Python 2.4.4 or Python 2.5.4: if you interrupt it with CTRL+C, the tracd.exe process stops and you can interact with the console again, but its child process python.exe continues to run in the background and serves requests.

This can end up being quite confusing, especially coupled with the fact that on Windows you can start again the same tracd.exe on the same port without getting a port in use error. For example:

  • start tracd.exe with an invalid environment path → no projects available
  • interrupt → tracd.exe dies, you're back on the console
  • restart tracd.exe with corrected path → still no projects available!

This is because the python.exe child process started by the first tracd.exe was still running and producing logging output on the console like it would the second server…

This doesn't happen with either Python 2.6.5 or Python 2.7.

comment:5 by Remy Blank, 11 years ago

All tests pass here on a number of combinations of OS (Linux, OS X, Windows XP, Windows Vista, Windows 7), Python versions (2.4, 2.5, 2.6, 2.7) and database backends (SQLite, PostgreSQL, MySQL).

Release package installed on two production servers, and everything looks fine.

Created new environments on Linux/2.4/SQLite, OS X/2.6/SQLite and Windows 7/2.7/SQLite and played with them. Nothing to report.

in reply to:  4 ; comment:6 by Jun Omae, 11 years ago

Replying to cboos:

I noticed an issue on Windows with tracd.exe installed using Trac-0.12.2rc1.win32.exe into either Python 2.4.4 or Python 2.5.4: if you interrupt it with CTRL+C, the tracd.exe process stops and you can interact with the console again, but its child process python.exe continues to run in the background and serves requests.

I have the same issue on Windows XP using Python 2.5.4. It seems that tracd.exe stops with CTRL-C, however, it still accepts requests.

It seems that CTRL-C cannot raise KeyboardInterrupt during calling socket.accept() on Windows.

This doesn't happen with either Python 2.6.5 or Python 2.7.

SocketServer.BaseServer.serve_forever() has changed from Python 2.6+. The method polls using select.select(). tracd.exe can stop with the polling on Windows.

The following patch needs more testing. I tested on Windows XP with Python 2.5.4 and Python 2.6.6.

  • trac/web/standalone.py

     
    2121
    2222import pkg_resources
    2323import os
     24import select
    2425import sys
    2526from SocketServer import ThreadingMixIn
    2627
     
    9293class TracHTTPServer(ThreadingMixIn, WSGIServer):
    9394    daemon_threads = True
    9495
     96    poll_interval = 0.5
     97
    9598    def __init__(self, server_address, application, env_parent_dir, env_paths,
    9699                 use_http_11=False):
    97100        request_handlers = (TracHTTPRequestHandler, TracHTTP11RequestHandler)
    98101        WSGIServer.__init__(self, server_address, application,
    99102                            request_handler=request_handlers[bool(use_http_11)])
    100103
     104    def serve(self):
     105        while 1:
     106            r, w, e = select.select([self], [], [], self.poll_interval)
     107            if self in r:
     108                self.handle_request()
    101109
     110
    102111class TracHTTPRequestHandler(WSGIRequestHandler):
    103112
    104113    server_version = 'tracd/' + VERSION
     
    255264                print 'Serving on http://%s:%s/%s' % (addr, port, base_path)
    256265            if options.http11:
    257266                print 'Using HTTP/1.1 protocol version'
    258             httpd.serve_forever()
     267            httpd.serve()
    259268    elif options.protocol in ('scgi', 'ajp', 'fcgi'):
    260269        def serve():
    261270            server_cls = __import__('flup.server.%s' % options.protocol,

in reply to:  6 ; comment:7 by Christian Boos, 11 years ago

Replying to jomae:

… The following patch needs more testing. I tested on Windows XP with Python 2.5.4 and Python 2.6.6.

I've tested it heavily with Python 2.7 yesterday (without tracd.exe), and a bit today with Python 2.4.4 and tracd.exe. No real issues yesterday (sometimes strange backtraces, but nothing we haven't seen before), and with 2.4 the problem described in comment:4 is gone.

If we can test this on Linux as well, I'm OK to let it go in 0.12-stable before the 0.12.2 release.

Other than that, I think we're ready for the release.

comment:8 by Thijs Triemstra, 11 years ago

Cc: Thijs Triemstra added

Not sure if this one's expected but one test fails on ubuntu 10.10, python 2.7.1:

$ python -m trac.test --skip-functional-tests
SKIP: versioncontrol/tests/svn_fs.py (no svn bindings)
....................................................................................................................................................................................................................................................F..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
======================================================================
FAIL: test_from_suffix_using_mimetypes (trac.mimeview.tests.api.GetMimeTypeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "trac/mimeview/tests/api.py", line 39, in test_from_suffix_using_mimetypes
    self.assertEqual(image_png, get_mimetype('doc/trac_logo.png', None))
AssertionError: 'image/x-png' != 'image/png'

----------------------------------------------------------------------
Ran 979 tests in 18.001s

FAILED (failures=1)

in reply to:  7 ; comment:9 by Remy Blank, 11 years ago

Replying to cboos:

If we can test this on Linux as well, I'm OK to let it go in 0.12-stable before the 0.12.2 release.

The SocketServer module has changed from 2.5 to 2.6, in the way Jun suggests in the patch. I would prefer keeping the new implementation in 2.6, and only "patch" 2.5 and lower, by overriding serve_forever() conditionally. This makes it clear that it's a workaround that can be removed once we drop support for 2.6.

  • trac/web/standalone.py

    diff --git a/trac/web/standalone.py b/trac/web/standalone.py
    a b  
    2121
    2222import pkg_resources
    2323import os
     24import select
    2425import sys
    2526from SocketServer import ThreadingMixIn
    2627
    class TracHTTPServer(ThreadingMixIn, WSG  
    9899        WSGIServer.__init__(self, server_address, application,
    99100                            request_handler=request_handlers[bool(use_http_11)])
    100101
     102    if sys.version_info < (2, 6):
     103        def serve_forever(self, poll_interval=0.5):
     104            while True:
     105                r, w, e = select.select([self], [], [], poll_interval)
     106                if self in r:
     107                    self.handle_request()
     108
    101109
    102110class TracHTTPRequestHandler(WSGIRequestHandler):
    103111

All unit and functional tests pass with this patch on Linux with 2.4.6, 2.5.4 and 2.6.6. With 2.7.1, I get a few failures (including the one reported by Thijs), which I'm currently investigating.

in reply to:  8 ; comment:10 by Christian Boos, 11 years ago

Replying to thijstriemstra:

Not sure if this one's expected but one test fails on ubuntu 10.10, python 2.7.1:

FAIL: test_from_suffix_using_mimetypes (trac.mimeview.tests.api.GetMimeTypeTestCase)
...
AssertionError: 'image/x-png' != 'image/png'

Yeah, not expected, but not surprising either, see #3858. At that time, I we picked image/png as I we figured that would be more "stable". Tough luck.

(Edit: that was actually Simon, not me… could have been me ;-) )

Last edited 11 years ago by Christian Boos (previous) (diff)

in reply to:  10 ; comment:11 by Remy Blank, 11 years ago

Replying to cboos:

Yeah, not expected, but not surprising either, see #3858. At that time, I picked image/png as I figured that would be more "stable". Tough luck.

It's even more complicated: on Linux, the mimetypes module returns "image/png" for the tested filename with 2.6 and 2.7, whereas on Windows 2.7 returns "image/x-png". Can we just accept both?

  • trac/mimeview/tests/api.py

    diff --git a/trac/mimeview/tests/api.py b/trac/mimeview/tests/api.py
    a b class GetMimeTypeTestCase(unittest.TestC  
    3333        self.assertEqual('text/plain', get_mimetype('README.txt', None))
    3434       
    3535    def test_from_suffix_using_mimetypes(self):
    36         image_png = 'image/png'
    37         if sys.version_info >= (2, 7):
    38             image_png = 'image/x-png'
    39         self.assertEqual(image_png, get_mimetype('doc/trac_logo.png', None))
     36        accepted = ('image/png', 'image/x-png')
     37        self.assert_(get_mimetype('doc/trac_logo.png', None) in accepted)
    4038       
    4139    def test_from_content_using_CONTENT_RE(self):
    4240        self.assertEqual('text/x-python',

comment:12 by Christian Boos, 11 years ago

Sigh. Is there nothing that is set in stone in this area? .jpg -> image/jpg perhaps? Oh, well. Perhaps just use assertTrue instead of assert_ then.

in reply to:  11 comment:13 by Remy Blank, 11 years ago

Replying to rblank:

It's even more complicated: on Linux, the mimetypes module returns "image/png" for the tested filename with 2.6 and 2.7, whereas on Windows 2.7 returns "image/x-png".

Since 2.7, the mimetypes module now adds MIME types read from the Windows registry, so that's where the "image/x-png" comes from. Patch applied in [10503].

comment:14 by Remy Blank, 11 years ago

As for the remaining 6 functional test failures with 2.7, they seem to be due to twill. More precisely, it seems that redirects to URLs with a fragment (e.g. http://127.0.0.1/ticket/40#comment:2) loose the fragment.

comment:15 by Christian Boos, 11 years ago

All the functional tests pass for me, with Python 2.7 and twill 0.9 on Windows 7. Did you change your version of twill since comment:5?

in reply to:  15 comment:16 by Remy Blank, 11 years ago

Replying to cboos:

All the functional tests pass for me, with Python 2.7 and twill 0.9 on Windows 7. Did you change your version of twill since comment:5?

No, 2.7 on Linux wasn't one of the combinations I tested. It was 2.7 on OS X Windows. I'm still trying to figure out what happens, but I'm not getting anywhere.

Last edited 11 years ago by Remy Blank (previous) (diff)

comment:17 by Remy Blank, 11 years ago

Ok, so it's definitely an issue with twill on 2.7:

Python 2.6.6 (r266:84292, Dec 10 2010, 11:03:53) 
[GCC 4.4.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import twill.commands as tc
>>> tc.go("http://trac.edgewall.org/#frag")
==> at http://trac.edgewall.org/#frag
'http://trac.edgewall.org/#frag'

but:

Python 2.7.1 (r271:86832, Dec 10 2010, 11:19:19) 
[GCC 4.4.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import twill.commands as tc
>>> tc.go("http://trac.edgewall.org/#frag")
==> at http://trac.edgewall.org/
'http://trac.edgewall.org/'

Note the missing fragment on 2.7.1. With 2.7 on Windows, it works:

Python 2.7 (r27:82525, Jul  4 2010, 09:01:59) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import twill.commands as tc
>>> tc.go("http://trac.edgewall.org/#frag")
==> at http://trac.edgewall.org/#frag
'http://trac.edgewall.org/#frag'

But not with 2.7.1 on Windows:

Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import twill.commands as tc
>>> tc.go("http://trac.edgewall.org/#frag")
==> at http://trac.edgewall.org/
'http://trac.edgewall.org/'

So it's an issue introduced with 2.7.1. Stay tuned…

comment:18 by Remy Blank, 11 years ago

Looks very much related to this entry in the changelog of 2.7.1:

Fix Issue8280 - urllib2's Request method will remove fragements in the url. This is how it is supposed to work, wget and curl do the same. Previous behavior was wrong.

It seems like we should remove the fragment when we verify the current URL:

  • trac/tests/functional/tester.py

    diff --git a/trac/tests/functional/tester.py b/trac/tests/functional/tester.py
    a b class FunctionalTester(object):  
    147147        tc.formvalue('propertyform', 'comment', comment)
    148148        tc.submit("submit")
    149149        # Verify we're where we're supposed to be.
    150         tc.url(self.url + '/ticket/%s#comment:.*' % ticketid)
     150        # The fragment is present up to 2.7, see:
     151        # http://trac.edgewall.org/ticket/9990#comment:18
     152        tc.url(self.url + '/ticket/%s(?:#comment:.*)?$' % ticketid)
    151153        return comment
    152154
    153155    def attach_file_to_ticket(self, ticketid, data=None, tempfilename=None,

With this fix, all tests pass on 2.7.1 as well.

Last edited 11 years ago by Remy Blank (previous) (diff)

in reply to:  9 ; comment:19 by Jun Omae, 11 years ago

Replying to rblank:

The SocketServer module has changed from 2.5 to 2.6, in the way Jun suggests in the patch. I would prefer keeping the new implementation in 2.6, and only "patch" 2.5 and lower, by overriding serve_forever() conditionally. This makes it clear that it's a workaround that can be removed once we drop support for 2.6.

OK. I think that your patch is better than my patch. I applied your patch in r10504. I verified all unit and functional tests on Windows with 2.4.4, 2.5.4, 2.6.6 and 2.7.1, on Linux with 2.4.3, 2.5.2, 2.6.4 and 2.7.0.

in reply to:  19 comment:20 by Christian Boos, 11 years ago

Replying to jomae:

I verified all unit and functional tests on Windows with 2.4.4, 2.5.4, 2.6.6 and 2.7.1, on Linux with 2.4.3, 2.5.2, 2.6.4 and 2.7.0.

Incidentally I just tested r10504 + comment:18 patch on Linux with 2.7.1, all OK as well.

comment:21 by Remy Blank, 11 years ago

Great, thanks for testing. Patch applied in [10505].

comment:22 by Christian Boos, 11 years ago

Ok, so branches/0.12-stable@10506 looks final to me. Do you think it's worth testing a bit more or could I proceed and package it this evening?

comment:23 by Remy Blank, 11 years ago

Nothing else from my side, so I vote for packaging. Except if you want to wait a bit more for translations.

Speaking of translations, I wonder if it wouldn't be better to move translations to a separate package, so people could install Trac from an official release, and install translations from SVN, or more frequent releases? Requiring all translators to update their translation during the one-week string freeze doesn't seem to work too well…

comment:24 by Christian Boos, 11 years ago

Well, in this specific case as there were very little changes between 0.12.1 and 0.12.2, I thought that a week would be enough (using the rc1 test period for the freeze).

But the idea is worth digging.

comment:25 by Thijs Triemstra, 11 years ago

Tests run fine now. I also suggest to add .twill-history to svn:ignore.

in reply to:  25 ; comment:26 by Christian Boos, 11 years ago

Replying to thijstriemstra:

Tests run fine now. I also suggest to add .twill-history to svn:ignore.

… or disable writing it in the first place:

  • trac/tests/functional/better_twill.py

     
    4141    # #5497).  Therefore we turn it off here.
    4242    twill.commands.config('use_tidy', '0')
    4343
     44    # We don't want a .twill-history file
     45    twill.shell.readline = None
     46
    4447    # We use a transparent proxy to access the global browser object through
    4548    # twill.get_browser(), as the browser can be destroyed by browser_reset()
    4649    # (see #7472).

(untested)

in reply to:  26 comment:27 by Thijs Triemstra, 11 years ago

Replying to cboos:

… or disable writing it in the first place:

That patch worked here, file's gone.

in reply to:  26 ; comment:28 by Remy Blank, 11 years ago

Replying to cboos:

… or disable writing it in the first place:

Weird, I can't find a trace of that file anywhere. When is it created?

in reply to:  28 ; comment:29 by Christian Boos, 11 years ago

Replying to rblank:

Weird, I can't find a trace of that file anywhere. When is it created?

Same for me, I've never seen that file, but apparently if you have readline it can get created (see twill/shell.py).

in reply to:  29 ; comment:30 by Christian Boos, 11 years ago

Replying to cboos:

Replying to rblank:

Weird, I can't find a trace of that file anywhere. When is it created?

… as for "when", never seems to be the answer. I suppose Thijs experimented with twill in interactive mode? No need for that patch then, of course.

in reply to:  30 ; comment:31 by Thijs Triemstra, 11 years ago

Replying to cboos:

Replying to cboos:

Replying to rblank:

Weird, I can't find a trace of that file anywhere. When is it created?

… as for "when", never seems to be the answer. I suppose Thijs experimented with twill in interactive mode? No need for that patch then, of course.

Twill was installed into stock python 2.6.6 on ubuntu 10.10 using easy_install, then I simply ran:

PYTHONPATH=. python2.6 ./trac/tests/functional/__init__.py

and svn status showed the file.

in reply to:  31 comment:32 by Christian Boos, 11 years ago

Hm, weird. Can you please try to patch your twill/shell.py like that:

  • ./twill/shell.py

    old new  
    237237    def do_EOF(self, *args):
    238238        "Exit on CTRL-D"
    239239        if readline:
     240            import traceback
     241            debug = open("/tmp/debug-twill.txt", "w")
     242            traceback.print_stack(None, None, debug)
     243            debug.close()
    240244            readline.write_history_file('.twill-history')
    241245
    242246        raise SystemExit()
Last edited 11 years ago by Christian Boos (previous) (diff)

comment:33 by Christian Boos, 11 years ago

Resolution: fixed
Status: newclosed

The Trac 0.12.2 packages are available, enjoy!

Thanks to all for having tested.

(Thijs, despite my closing of the ticket, I'm interested to continue to troubleshoot the problem you had with .twill-history, please redo the functional tests once you've patched twill, the /tmp/debug-twill.txt should tell us how you got to print that file)

comment:34 by Thijs Triemstra, 11 years ago

I'll check it out cboos.

I received this error with 0.12-stable on python 2.6.6, probably needs a new ticket?

Detected modification of /home/thijs/workspaces/opensource/trac-stable-0.12/trac/versioncontrol/web_ui/changeset.py, restarting.
Unhandled exception in thread started by <function serve at 0x18acaa0>
Traceback (most recent call last):
  File "/home/thijs/workspaces/opensource/trac-stable-0.12/trac/web/standalone.py", line 266, in serve
    httpd.serve_forever()
  File "/usr/lib/python2.6/SocketServer.py", line 224, in serve_forever
    r, w, e = select.select([self], [], [], poll_interval)
AttributeError: 'NoneType' object has no attribute 'select'
Server starting in PID 11448.
Serving on http://localhost:8000/
Using HTTP/1.1 protocol version

in reply to:  34 ; comment:35 by Christian Boos, 11 years ago

Replying to thijstriemstra:

I'll check it out cboos.

Ok, thanks!

I received this error with 0.12-stable on python 2.6.6, probably needs a new ticket?

  File "/usr/lib/python2.6/SocketServer.py", line 224, in serve_forever
    r, w, e = select.select([self], [], [], poll_interval)
AttributeError: 'NoneType' object has no attribute 'select'

This was one of the weird errors I referred to in comment:7 ;-) You can find other references to this error here and there (#3957). It's something we never really understood, my guess it's that some code gets executed once the builtin select module has already been destroyed.

Some googling confirms this:

When Python begins to shutdown it takes each module and sets each variable in the global namespace to None. If a thread has not terminated before the interpreter terminates then the thread tries to use a global variable which has been set to None. (http://bugs.python.org/issue1722344#msg62078)

And I just see you were already aware of this issue ;-) I'll look at this later in more depth, but I'm not sure we can do anything about this problem. Also, it's harmless, the process is exiting anyway at this point, so for now I would say there's no need to create a dedicated ticket.

in reply to:  35 ; comment:36 by Christian Boos, 11 years ago

Incidentally when browsing the code for tempfile (#9925), I saw the following in http://svn.python.org/view/python/trunk/Lib/tempfile.py?revision=73744:

class _TemporaryFileWrapper
#...
        # Cache the unlinker so we don't get spurious errors at
        # shutdown when the module-level "os" is None'd out.  Note
        # that this must be referenced as self.unlink, because the
        # name TemporaryFileWrapper may also get None'd out before
        # __del__ is called.
        unlink = _os.unlink

So we could just do something similar for select.select.

in reply to:  36 ; comment:37 by Remy Blank, 11 years ago

Replying to cboos:

So we could just do something similar for select.select.

Too bad BaseServer.serve_forever() is outside of our control, at least for Python ≥2.6. Maybe report it to the Python folks with a reference to _TemporaryFileWrapper?

in reply to:  37 ; comment:38 by Jun Omae, 11 years ago

Replying to rblank:

Replying to cboos:

So we could just do something similar for select.select.

Too bad BaseServer.serve_forever() is outside of our control, at least for Python ≥2.6. Maybe report it to the Python folks with a reference to _TemporaryFileWrapper?

I think that we should wait for the TracHTTPServer.serve_forever() termination before the python interpreter shutdowns.

At Python 2.6+, we can wait using BaseServer.shutdown(). At Python ⇐ 2.5, the method doesn't exist, however we can define TracHTTPServer.shutdown() like TracHTTPServer.serve_forever().

t9990-tracd-shutdown.diff.

by Jun Omae, 11 years ago

Attachment: t9990-tracd-shutdown.diff added

in reply to:  38 comment:39 by Christian Boos, 11 years ago

Replying to jomae:

At Python 2.6+, we can wait using BaseServer.shutdown(). At Python ⇐ 2.5, the method doesn't exist, however we can define TracHTTPServer.shutdown() like TracHTTPServer.serve_forever().

t9990-tracd-shutdown.diff.

Looks great: I'm testing & trying to make sure I understand, this can take a while ;-)

comment:40 by Christian Boos, 11 years ago

Btw, looks like we forgot to send the release announcement, last week… so I just did it now. Any taker for the last 2 other steps in TracDev/ReleaseChecklist (Pypi + Freshmeat)?

comment:41 by Remy Blank, 11 years ago

Isn't the Pypi update a simple:

python setup.py upload

that requires the release packages to be present? Or do we add the new entry by hand? In the latter case, I can do the update. I'm not sure I have access to the Freshmeat account, though.

in reply to:  33 comment:42 by Thijs Triemstra, 11 years ago

Replying to cboos:

(Thijs, despite my closing of the ticket, I'm interested to continue to troubleshoot the problem you had with .twill-history, please redo the functional tests once you've patched twill, the /tmp/debug-twill.txt should tell us how you got to print that file)

I tried it last night with exactly the same setup afaik but I couldn't reproduce it :-/ weird, but oh well.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The ticket will remain with no owner.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from (none) to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.