Trac often fails to load stylesheets

[alex@…]

Hi people!

I have a trac installed here: ​http://trac.php-programs.net/ This has nothing special. TocMacro, AccountManager, AutoCompleteUsers, NewsFlash…

I've simply modified the templates/site.html and added some styles inline to the html. No other files were modified.

When browsing to my trac, it very often fails to load all stylesheets or only parts of some stylesheets. The layout then is mostly broken or just looks somehow weird.

You can test it by yourself, by just reloading my trac page several times.

I thought this may a problem of the browser, but many people are reporting this to me and also tests with different browsers shows the same behaviour.

[Remy Blank]

The site seems to be working fine from here (Firefox 3.6.15 and Chromium on Linux).

Note that some stylesheets are applied by using JavaScript, so any issues in JavaScript code can prevent stylesheets from being applied. You may want to check your JavaScript console when loading fails and see if there are any errors.

[Christian Boos]

I also had a look, I thought that maybe you were also including secondary .css files and therefore you were subject to #9938, but apparently it's not the case. Maybe next time this happens, try to remember exactly what you changed, see what you actually get, and report both here. Maybe it's an issue with Genshi template caching.

Btw, as I was looking at the html of your site, I saw that near the end you have a strange looking <img>:

...><img src="http://stats.mieland.net/piwik.php?idsite=4" style="border:0" alt="Project-Id-Version: Trac 0.12
Report-Msgid-Bugs-To: trac-dev@googlegroups.com
POT-Creation-Date: 2008-01-30 09:20+0100
...

If it's not an error, I'd be curious to know the intent of this.

[Oleg Frantsuzov <franoleg@…>]

Screenshot of how the problem looks like in my case

[Oleg Frantsuzov <franoleg@…>]

I have a similar problem. Trac 0.12.2 is deployed on Apache via mod_wsgi. Sometimes I get this​ instead of a properly rendered page (I haven't modified the default style altogether). Sorry but I can't tell the exact steps to reproduce it; this seems to be an intermittent problem. Most often this happens when I open Trac the first time during a session, but not always so. When this happens, pressing F5 (reload) doesn't help; but force reloading (as in Ctrl+F5) several times does help.

The Trac instance I'm talking about is not public. The browser is Chrome 10 (Windows). There are no errors in JavaScript console when the broken page is loaded.

As a matter of the fact, I wasn't able to “break” the styles on this ticket's reporter Trac instance even though I reloaded a dozen times.

[Christian Boos]

Replying to Oleg Frantsuzov <franoleg@…>:

… Sorry but I can't tell the exact steps to reproduce it; this seems to be an intermittent problem. Most often this happens when I open Trac the first time during a session, but not always so. When this happens, pressing F5 (reload) doesn't help; but force reloading (as in Ctrl+F5) several times does help.

It's a different problem: in your case the CSS files apparently failed to load (verify the status in the "Network" tab, in the developer tools). You'd have to search in the Apache error log files for more hints.

[Christian Boos]

Replying to cboos:

Replying to Oleg Frantsuzov <franoleg@…>: … for more hints.

for example, 401 for your .css files as well ;-)

[Oleg Frantsuzov <franoleg@…>]

Replying to cboos:

…(verify the status in the "Network" tab, in the developer tools). You'd have to search in the Apache error log files for more hints.

Thanks for help. Unfortunately, I don't seem to see any status codes apart from 200 OK on my ​Network tab except for the page itself (which is OK). This is the same Trac installation, only a different environment. The Apache log has no relevant errors for Trac URLs as well.

Should I go to the mailing list with this problem?

[anonymous]

Hi People…

much thanks for your answers! I've now tried some few things. See atached some examples on how my trac looks like after some reloads.

image "20110331-1600x900-001.png" - This is how the page SHOULD look like (except the 500 error in network console) image "20110331-1600x900-002.png" - here is suddenly some margin missing image "20110331-1600x900-003.png" - This database lock also happens very often image "20110331-1600x900-004.png" - And this is the most extremly version after a reload

I'm very curious about the 500 internal server errors in network console of firebug. But there is nothing in the apache error.log (except the one error below).

I also had the mod_mime bug and so I modified my magic.mime file to prevent these errors (commenting out the regex stuff).

But I still get these error in the error.log:

[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185] mod_wsgi (pid=3286): Exception occurred processing WSGI script '/srv/trac/cgi-bin/trac.wsgi'., referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185] Traceback (most recent call last):, referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185]   File "/srv/trac/cgi-bin/trac.wsgi", line 31, in application, referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185]     return dispatch_request(environ, start_request), referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185]   File "build/bdist.linux-x86_64/egg/trac/web/main.py", line 479, in dispatch_request, referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185]     return _dispatch_request(req, env, env_error), referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185]   File "build/bdist.linux-x86_64/egg/trac/web/main.py", line 555, in _dispatch_request, referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185]     send_internal_error(env, req, sys.exc_info()), referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185]   File "build/bdist.linux-x86_64/egg/trac/web/main.py", line 648, in send_internal_error, referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185]     req.send_error(exc_info, status=500, env=env, data=data), referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185]   File "build/bdist.linux-x86_64/egg/trac/web/api.py", line 465, in send_error, referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185]     self.write(data), referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185]   File "build/bdist.linux-x86_64/egg/trac/web/api.py", line 536, in write, referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185]     self._write(data), referer: http://trac.php-programs.net/chrome/common/css/trac.css
[Thu Mar 31 07:11:44 2011] [error] [client 93.135.229.185] IOError: failed to write data, referer: http://trac.php-programs.net/chrome/common/css/trac.css

[alex@…]

Replying to cboos:

I also had a look, I thought that maybe you were also including secondary .css files and therefore you were subject to #9938, but apparently it's not the case. Maybe next time this happens, try to remember exactly what you changed, see what you actually get, and report both here. Maybe it's an issue with Genshi template caching.

Btw, as I was looking at the html of your site, I saw that near the end you have a strange looking <img>:

...><img src="http://stats.mieland.net/piwik.php?idsite=4" style="border:0" alt="Project-Id-Version: Trac 0.12
Report-Msgid-Bugs-To: trac-dev@googlegroups.com
POT-Creation-Date: 2008-01-30 09:20+0100
...

If it's not an error, I'd be curious to know the intent of this.

Btw… this image is okay as it is. It gets automatically inserted by the Piwik statistics tool.

[Remy Blank]

So the real issue here is the "Database is locked" error. If it happens when loading a stylesheet, you will indeed miss some styles. Other symptoms may include missing JavaScript functionality.

As a first measure, you should have your web server serve static files, instead of going through Trac. This will ensure your .css and .js files are always sent correctly.

After that, you should really find out why your database locks so much. Do you have very high traffic? If yes, you may be better served with PostgreSQL (I assume you use SQLite). You should find plenty of duplicates here about "Database is locked".

Replying to alex@…:

Btw… this image is okay as it is. It gets automatically inserted by the Piwik statistics tool.

The question was rather, how come a .pot file (an internal Trac file containing string translations) ends up in the alt= attribute of an <img> tag?

[Christian Boos]

This starts to look a bit like #8708…

[anonymous]

Replying to rblank:

So the real issue here is the "Database is locked" error. If it happens when loading a stylesheet, you will indeed miss some styles. Other symptoms may include missing JavaScript functionality.

As a first measure, you should have your web server serve static files, instead of going through Trac. This will ensure your .css and .js files are always sent correctly.

Thanks for this hint, this seems very senseful to me. I'll take a close look on it in the european evening.

After that, you should really find out why your database locks so much. Do you have very high traffic? If yes, you may be better served with PostgreSQL (I assume you use SQLite). You should find plenty of duplicates here about "Database is locked".

You're right, I'm using sqlite here as this was the first suggested, default installation method. It would maybe be good to change the installation documentations to use mysql or postgresql as a default. I'll try to find a possibility to migrate that all to MySQL. I hopefully will not have to re-create all the pages by hand.

Replying to alex@…:

Btw… this image is okay as it is. It gets automatically inserted by the Piwik statistics tool.

The question was rather, how come a .pot file (an internal Trac file containing string translations) ends up in the alt= attribute of an <img> tag?

I don't know why Piwik does this. And I don't care. ;-) I only know that getting and generating the statistics works fine for this trac. Maybe this is needed to submit some special file-informations or similar, to the piwik scripts….

[Christian Boos]

Replying to anonymous:

Replying to rblank:

After that, you should really find out why your database locks so much. Do you have very high traffic? If yes, you may be better served with PostgreSQL (I assume you use SQLite). You should find plenty of duplicates here about "Database is locked".

You're right, I'm using sqlite here as this was the first suggested, default installation method. It would maybe be good to change the installation documentations to use mysql or postgresql as a default.

It's been quite some time now I haven't been able to trigger any lock to happen when using the SQLite backend, even when running about 50 simultaneous requests of various complexity targeting 2 tracd processes serving the same environment. So I'd say unless you have a rather extreme usage pattern, seeing "database is locked" with 0.12.2 or above is really not normal (see e.g. ticket:10077#comment:2 for the kind of feedback that would be useful for me).

[alex@…]

Well, except it's not supposed to occur anymore with 0.12.2 ;-)

So I'd like to get a few more details from the OP:

• which SQLite version? which PySqlite version?

[14:05:02] root@h1660449:~ $ uname -a
Linux h1660449 2.6.24-25-server #1 SMP Tue Oct 20 07:20:02 UTC 2009 x86_64 GNU/Linux
[14:05:30] root@h1660449:~ $ python --version
Python 2.5.2
[14:05:53] root@h1660449:~ $ sqlite -version
2.8.17
[14:05:56] root@h1660449:~ $ sqlite3 -version
3.4.2

So the PySqlite should be working and uptodate, not? (​http://trac.edgewall.org/wiki/PySqlite) (Don't know how to get the exact version of the bundled pysqlite)

• what web front end (mod_wsgi? version?)

[14:12:30] root@h1660449:~ $ apache2 -v
Server version: Apache/2.2.8 (Ubuntu)
Server built:   Nov 18 2010 21:21:21
[14:12:50] root@h1660449:~ $ find /var/cache/apt -type f -iname "*wsgi*"
/var/cache/apt/archives/libapache2-mod-wsgi_2.0-1~hardy1_amd64.deb

• how often does the problem happen? any usage patterns?

As you can see here ​http://trac.php-programs.net with firebug … this happens with every click on the page. At least two or three files are causing the 500 Internal Server error. When going into Firebug and opening the network tab and looking into these 500 errors, the database block appears.

• does the problem also happen without plugins? (we see account_mgr in the stack trace but maybe this is unrelated)

When deactivating ALL plugins, the problem disappears! Then after reactivating each plugin one after one I found out, that this entry causes the 500 Internal Server error in the [component] section:

trac.web.auth.loginmodule = disabled

But I don't want the http authentification. I want a normal registration form and a login form.

• what is the size of the session and session_attribute tables?

How would I get those sizes when using sqlite? db/trac.db is 7,9 MB in size.


So it seems to me, that deactivating the http auth method causes these problems. What else can I do to use a webbased login form instead of the http auth??

[Christian Boos]

I'll get back in a second to the main issue, but first a few notes about the other issue with your <img> …

Replying to anonymous:

Replying to rblank:

The question was rather, how come a .pot file (an internal Trac file containing string translations) ends up in the alt= attribute of an <img> tag?

I don't know why Piwik does this. And I don't care. ;-)

Ok, so from this answer I gather it's not intentional. A larger excerpt of the source shows:

        <!-- Piwik -->
        <script type="text/javascript">
            var pkBaseURL = (("https:" == document.location.protocol) ? "https://stats.mieland.net/" : "http://stats.mieland.net/");
            document.write(unescape("%3Cscript src='" + pkBaseURL + "piwik.js' type='text/javascript'%3E%3C/script%3E"));
            </script><script type="text/javascript">
            try {
                var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", 4);
                piwikTracker.trackPageView();
                piwikTracker.enableLinkTracking();
            } catch( err ) {}
        </script><noscript><p><img src="http://stats.mieland.net/piwik.php?idsite=4" style="border:0" alt="Project-Id-Version: Trac 0.12
Report-Msgid-Bugs-To: trac-dev@googlegroups.com
POT-Creation-Date: 2008-01-30 09:20+0100
PO-Revision-Date: 2011-02-19 17:04+0100
Last-Translator: Jeroen Ruigrok van der Werven &lt;asmodai@in-nomine.org&gt;
Language-Team: en_US &lt;trac-dev@googlegroups.com&gt;
Plural-Forms: nplurals=2; plural=(n != 1)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Generated-By: Babel 0.9.5
" /></p></noscript>
        <!-- End Piwik Tracking Code -->

Care to show us the part in your site.html template which generates this? Or any py:match that could be related (alt attribute, or img element)?

And the resulting "image" is, according to the info shown by the network tab:

piwik.php
Dimensions  1 × 1
File size  43B
MIME type  image/gif

[alex@…]

this is the original code at the end of site.html to include the piwik thing:

		<!-- Piwik -->
		<script type="text/javascript">
			var pkBaseURL = (("https:" == document.location.protocol) ? "https://stats.mieland.net/" : "http://stats.mieland.net/");
			document.write(unescape("%3Cscript src='" + pkBaseURL + "piwik.js' type='text/javascript'%3E%3C/script%3E"));
			</script><script type="text/javascript">
			try {
				var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", 4);
				piwikTracker.trackPageView();
				piwikTracker.enableLinkTracking();
			} catch( err ) {}
		</script><noscript><p><img src="http://stats.mieland.net/piwik.php?idsite=4" style="border:0" alt="" /></p></noscript>
		<!-- End Piwik Tracking Code -->	

[anonymous]

btw. ​http://stats.mieland.net is also one of my own domains. Here the main piwik application runs.

[Christian Boos]

About the screenshots: it seems to me that 20110331-1600x900-002.png​ is the "correct" one, i.e. what gets rendered when all the .css are loaded. 20110331-1600x900-001.png​ OTOH is what you get when the newsflash.css is missing (try it by yourself, disabling the style elements you'll find there, in particular the "margin: 0em;" for the div.newsflash).

Indeed the rendering is OK for anybody who visit your site as anonymous, but not for you when you're logged in, which is consistent with your findings relative to disabling the possibility to log in.

Replying to alex@…:

Well, except it's not supposed to occur anymore with 0.12.2 ;-)

So I'd like to get a few more details from the OP:

• which SQLite version? which PySqlite version?

[14:05:02] root@h1660449:~ $ uname -a
Linux h1660449 2.6.24-25-server #1 SMP Tue Oct 20 07:20:02 UTC 2009 x86_64 GNU/Linux
[14:05:30] root@h1660449:~ $ python --version
Python 2.5.2
[14:05:53] root@h1660449:~ $ sqlite -version
2.8.17
[14:05:56] root@h1660449:~ $ sqlite3 -version
3.4.2

Should be OK. Note that the version of the sqlite3 program itself is irrelevant, as you're probably using the Python builtin version (see PySqlite#Troubleshooting to figure out the exact version if you're interested).

• what web front end (mod_wsgi? version?)

[14:12:30] root@h1660449:~ $ apache2 -v
Server version: Apache/2.2.8 (Ubuntu)
Server built:   Nov 18 2010 21:21:21
[14:12:50] root@h1660449:~ $ find /var/cache/apt -type f -iname "*wsgi*"
/var/cache/apt/archives/libapache2-mod-wsgi_2.0-1~hardy1_amd64.deb

Ouch, mod_wsgi 2.0 is quite old and probably has a few issues. We should probably recommend a minimal version (not sure which one yet, need to do some research).

• how often does the problem happen? any usage patterns?

(snip)

trac.web.auth.loginmodule = disabled

Well, here you disable login completely. You should keep it activated as it's a builtin trac module. Have you tried with the above not disabled and with the account manager plugin disabled? Does the problem still happen? Do you have other auth related plugins?

• what is the size of the session and session_attribute tables?

How would I get those sizes when using sqlite?

select count(*) from session;

db/trac.db is 7,9 MB in size.

But OK, shouldn't be big.

So it seems to me, that deactivating the http auth method causes these problems. What else can I do to use a webbased login form instead of the http auth?

Well, we need to figure out where's the locking bug and fix it.

Once again, just to be sure we understand each other:

• with trac.web.auth.loginmodule = disabled you have no login possibility at all, and then it works but it's not a solution of course
• with trac.web.auth.loginmodule = enabled and the ​TH:AccountManagerPlugin, you have a "web based login form", you can log in, but then you also experience the lock
• with trac.web.auth.loginmodule = enabled and the ​TH:AccountManagerPlugin disabled, you have the default HTTP auth login, which is not what you want … but does it work? (i.e. no locks?)

[alex@…]

okay… lets check this:

• ​TH:AccountManagerPlugin enabled AND trac.web.auth.loginmodule enabled
  • The 500 error doesn't appear at all - Except for the loginpage itself (​http://trac.php-programs.net/login)
  • Logins not possible at all (trac error: Authentication information not available.)

• ​TH:AccountManagerPlugin enabled AND trac.web.auth.loginmodule disabled
  • The 500 error doesn't appear for guest at all
  • Login through webform possible
  • The 500 error appears again when logged in

• ​TH:AccountManagerPlugin disabled AND trac.web.auth.loginmodule disabled
  • The 500 error doesn't appear for guest at all
  • Login through webform possible (huu?? all auth mplugins disabled!)
  • The 500 error appears again when logged in

• ​TH:AccountManagerPlugin disabled AND trac.web.auth.loginmodule enabled
  • The 500 error doesn't appear for guest at all
  • Logins not possible at all (trac error: Authentication information not available.)

[anonymous]

wait…. when just commenting an entry from the [components] section… will it stay activated?

[Christian Boos]

Replying to alex@…:

this is the original code at the end of site.html to include the piwik thing:

<img src="http://stats.mieland.net/piwik.php?idsite=4" style="border:0" alt="" />

See? alt="". There must be some weird py:match somewhere that inserts the content you end up with and that I quoted in comment:15, but I don't see how that particular content gets in. Furthermore it seems it's corresponding to branches/0.12-stable//trac/locale/en_US/LC_MESSAGES/messages.po​, but with a different PO-Revision-Date (2011-02-19 vs. 2010-07-19).

[OT] Remy, we badly need #7145 for this kind of interleaved conversation ;-)

[Christian Boos]

Replying to anonymous:

wait…. when just commenting an entry from the [components] section… will it stay activated?

It depends: trac.* modules are enabled by default, the others are disabled by default.

To be sure, check the Admin / Plugins panel (well, obviously you need to be logged in as admin to do that), or look in the trac.log for what gets activate at server startup.

[Christian Boos]

Replying to Oleg Frantsuzov <franoleg@…>:

Replying to cboos:

…(verify the status in the "Network" tab, in the developer tools). You'd have to search in the Apache error log files for more hints.

Thanks for help. Unfortunately, I don't seem to see any status codes apart from 200 OK …

Well, that's very strange: you can't have a page look like being rendered without CSS yet have the trac.css loading fine (that one is enough to render correctly a Trac error page). Check that page's HTML source, try with another browser, … there must be something.

[anonymous]

Replying to cboos:

You have just registered in our trac… if you need more rights to test something, tell me.

[Christian Boos]

Replying to anonymous:

Replying to cboos:

You have just registered in our trac… if you need more rights to test something, tell me.

Well, CONFIG_VIEW would be enough, thanks!

[anonymous]

added

[Christian Boos]

Well, for now I don't see anything special. The strange thing is that even logged in, I never got a single 500… Can you please try to log using another (new) account? And if it also works for you with that other account, figure out the difference between that new user and your regular "alex" account?

[alex@…]

very strange… okay, thank you very much for now! I'll test this with another account after work. :P

[alex@…]

really strange things are going on here… ;)

These 500 internal server errors only appears, when logged in as TRAC_ADMIN. But then for every TRAC_ADMIN, no matter if it is a new account or not.

[Carsten Klein <carsten.klein@…>]

Some pointers:

when you get a 500 Error on one of the static files served by your server, then open up Firebug and get the actual server response for that file. It should be available directly from Firebug. Feel free to post your findings here including also the error page rendered by trac which should be available in the response body of the 500 response.

As for your last comment, that every TRAC_ADMIN will see this behaviour. Did you fiddle with the original code base or the code base of one of the plugins you are currently using?

Perhaps you could include the full source for your site.html as an attachment here, perhaps we can find the issue there?

Have you tried configuring apache, mod_python and trac so that they will log at the debug level? If not, try it out, so you can see more information in the logs…

[anonymous]

FWIW, we're experiencing this issue as well, but we've tracked it down to the content type of the CSS files sometimes inexplicably being set to completely random values. Some browsers tolerate this more than others, so our chrome users have the most trouble.

[Christian Boos]

Replying to anonymous:

FWIW, we're experiencing this issue as well, but we've tracked it down to the content type of the CSS files sometimes inexplicably being set to completely random values.

That must have been ​mod-wsgi-issue:255.

As for the original report, well that was for mod_wsgi 2.0, which is quite old. I suppose they have upgraded since.

[wayne@…]

I've more information on this issue. It still happens on 0.12.4. I have wsgi 3.3 on Apache 2.2 on Windows 2008. Firefox seems to be fine, but Chrome (v23) seems to get strange Content-Type headers: Content-Type:n_sequence_fields Content-Type:acct_mgr\locale\fr Content-Type:\htdocs Content-Type:dependency_links.txt

[Jun Omae]

Replying to wayne@…:

I've more information on this issue. It still happens on 0.12.4. I have wsgi 3.3 on Apache 2.2 on Windows 2008.

The issue is a ​mod_wsgi issue and was fixed in mod_wsgi 3.4, ​mod-wsgi-issue:255#c16. Please upgrade to mod_wsgi 3.4 or later.

[Wayne <wayne@…>]

Replying to jomae:

Replying to wayne@…:

I've more information on this issue. It still happens on 0.12.4. I have wsgi 3.3 on Apache 2.2 on Windows 2008.

The issue is a ​mod_wsgi issue and was fixed in mod_wsgi 3.4, ​mod-wsgi-issue:255#c16. Please upgrade to mod_wsgi 3.4 or later.

Thanks, that fixed it. FYI current win32 binaries can be found at ​http://www.lfd.uci.edu/~gohlke/pythonlibs/#mod_wsgi