Edgewall Software
Modify

Opened 4 months ago

Last modified 4 months ago

#13414 new enhancement

Support PyPy

Reported by: kallisti5 Owned by:
Priority: low Milestone: undecided
Component: general Version:
Severity: normal Keywords: pypy
Cc: Branch:
Release Notes:
API Changes:
Internal Changes:

Description (last modified by Jun Omae)

I was able to get trac running under PyPy in https://foss.heptapod.net/pypy/pypy/-/issues/3528

Originally it was crashing on ticket queries, but switching psycopg2 to psycopg2cffi via this patch below resolved the crashing:

  • trac/db/postgres_backend.py

    diff --git a/trac/db/postgres_backend.py b/trac/db/postgres_backend.py
    index f9f910630..8670f83ac 100644
    a b from trac.util.html import Markup  
    3131from trac.util.text import empty, exception_to_unicode, to_unicode
    3232from trac.util.translation import _
    3333
     34# Attempt to use psycopg2cffi in compatibility mode if available
     35# this enables usage of psycopg2cffi under pypy
     36try:
     37    from psycopg2cffi import compat
     38    compat.register()
     39except ImportError:
     40    pass
     41
    3442try:
    3543    import psycopg2 as psycopg
    3644    import psycopg2.extensions

Attachments (0)

Change History (6)

comment:1 by Jun Omae, 4 months ago

Description: modified (diff)
Milestone: undecided

Trac depends on __del__ behavior of CPython. Many failures should exist in unit and functional tests on PyPy, even if applying the changes.

See Differences between PyPy and CPython — PyPy documentation.

Last edited 4 months ago by Jun Omae (previous) (diff)

comment:2 by kallisti5, 4 months ago

Hm. We haven't seen any issues and haven't had any reports of issues. I was looking to trac for some guidance, but the PyPy wiki page just explains what PyPy is.

Trac 1.4.x Performance under python2 has been pretty lackluster lately, the PyPy change gave us a badly needed boost.

As a side note, the trac wiki pages are getting filled with spam. https://trac.edgewall.org/demo-1.4

comment:3 by Jun Omae, 4 months ago

Keywords: pypy added
Priority: normallow

The existence of PyPy wiki page doesn't mean Trac supporting pypy.

Trac doesn't currently support pypy due to depending __del__ behavior.

unit-test in trunk with pypy3.7

Python: /home/jun66j5/venv/pypy3/bin/python

  Package        Version
  ----------------------------------------------------------------------------
  Python       : 3.7.10 (7.3.5+dfsg-1~ppa1~ubuntu16.04, May 23 2021, 14:57:28)
               : [PyPy 7.3.5 with GCC 5.4.0 20160609]
  Setuptools   : 42.0.2
  Pip          : 19.3.1
  Wheel        : 0.33.6
  Jinja2       : 3.0.1
  Babel        : 2.9.1
  sqlite3      : 2.6.0 (3.11.0)
  PySqlite3    : not installed
  PyMySQL      : 1.0.2
  Psycopg2     : not installed
  SVN bindings : not installed
  Mercurial    : not installed
  Pygments     : 2.9.0
  Textile      : 4.0.1
  Pytz         : 2021.1
  Docutils     : 0.17.1
  Selenium     : 3.141.0
  PyTidyLib    : 0.3.2 (5.6.0 libtidy.so.5deb1)
  LXML         : not installed
  coverage     : 5.5

Variables:
  PATH=/home/jun66j5/venv/pypy3/bin:/home/jun66j5/bin:/home/jun66j5/.gem/ruby/2.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
  PYTHONPATH=.
  TRAC_TEST_DB_URI=
  server-options= -p 3000 -a '*,/home/jun66j5/src/trac-htdigest.txt,auth'  -e

External dependencies:
  Git version: git version 2.29.0
  Subversion version: 1.9.3

python  setup.py egg_info
running egg_info
writing Trac.egg-info/PKG-INFO
writing dependency_links to Trac.egg-info/dependency_links.txt
writing entry points to Trac.egg-info/entry_points.txt
writing requirements to Trac.egg-info/requires.txt
writing top-level names to Trac.egg-info/top_level.txt
reading manifest file 'Trac.egg-info/SOURCES.txt'
writing manifest file 'Trac.egg-info/SOURCES.txt'
grep -q '^min_python = (2, [0-5])' setup.py || ( \
    ! grep -r --include='*.py' '\<import *with_statement\>' trac tracopt contrib && \
    ! grep -r --include='*.py' '\<except *.*, *[A-Za-z0-9_]\+:' trac tracopt contrib \
)
SKIP_FUNCTIONAL_TESTS=1 python  -m unittest  trac.test.test_suite
SKIP: tracopt/versioncontrol/svn/tests/svn_fs.py (no svn bindings)
..........................................................................................................................................................................................................................................................................Exception ignored in: <bound method PooledConnection.__del__ of <trac.db.pool.PooledConnection object at 0x0000000004b0fe50>>
Traceback (most recent call last):
  File "/home/jun66j5/src/tracdev/git/trac/db/pool.py", line 52, in __del__
    self.close()
  File "/home/jun66j5/src/tracdev/git/trac/db/pool.py", line 49, in close
    self._pool._return_cnx(cnx, self._key, self._tid)
  File "/home/jun66j5/src/tracdev/git/trac/db/pool.py", line 162, in _return_cnx
    assert (tid, key) in self._active
AssertionError
Exception ignored in: <bound method PooledConnection.__del__ of <trac.db.pool.PooledConnection object at 0x0000000004b0fef8>>
Traceback (most recent call last):
  File "/home/jun66j5/src/tracdev/git/trac/db/pool.py", line 52, in __del__
    self.close()
  File "/home/jun66j5/src/tracdev/git/trac/db/pool.py", line 49, in close
    self._pool._return_cnx(cnx, self._key, self._tid)
  File "/home/jun66j5/src/tracdev/git/trac/db/pool.py", line 162, in _return_cnx
    assert (tid, key) in self._active
AssertionError
<snip>
----------------------------------------------------------------------
Ran 2542 tests in 84.334s

FAILED (failures=4, skipped=3)
Makefile:477: recipe for target 'unit-test' failed
make: *** [unit-test] Error 1

unit-test in 1.4-stable with pypy2.7

$ make python=pypy unit-test

Python: /home/jun66j5/venv/pypy/bin/python

  Package        Version
  ----------------------------------------------------------------------------
  Python       : 2.7.18 (7.3.5+dfsg-1~ppa1~ubuntu16.04, May 23 2021, 14:34:53)
               : [PyPy 7.3.5 with GCC 5.4.0 20160609]
  Setuptools   : 42.0.2
  Pip          : 19.3.1
  Wheel        : 0.33.6
  Jinja2       : 2.11.3
  Genshi       : 0.7.5 (without speedups)
  Babel        : 2.9.1
  sqlite3      : 2.6.0 (3.11.0)
  PySqlite     : not installed
  PyMySQL      : 0.10.1
  Psycopg2     : not installed
  SVN bindings : not installed
  Mercurial    : not installed
  Pygments     : 2.5.2
  Textile      : 3.0.4
  Pytz         : 2021.1
  Docutils     : 0.17.1
  Twill        : not installed
  LXML         : not installed
  coverage     : 5.5

Variables:
  PATH=/home/jun66j5/venv/pypy/bin:/home/jun66j5/bin:/home/jun66j5/.gem/ruby/2.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
  PYTHONPATH=.
  TRAC_TEST_DB_URI=
  server-options= -p 3000 -a '*,/home/jun66j5/src/trac-htdigest.txt,auth'  -e

External dependencies:
  Git version: git version 2.29.0
  Subversion version: 1.9.3

python  setup.py egg_info
running egg_info
writing Trac.egg-info/PKG-INFO
writing dependency_links to Trac.egg-info/dependency_links.txt
writing entry points to Trac.egg-info/entry_points.txt
writing requirements to Trac.egg-info/requires.txt
writing top-level names to Trac.egg-info/top_level.txt
reading manifest file 'Trac.egg-info/SOURCES.txt'
writing manifest file 'Trac.egg-info/SOURCES.txt'
grep -q '^min_python = (2, [0-5])' setup.py || ( \
    ! grep -r --include='*.py' '\<import *with_statement\>' trac tracopt contrib && \
    ! grep -r --include='*.py' '\<except *.*, *[A-Za-z0-9_]\+:' trac tracopt contrib \
)
SKIP_FUNCTIONAL_TESTS=1 python  -m unittest  trac.test.test_suite
SKIP: tracopt/versioncontrol/svn/tests/svn_fs.py (no svn bindings)
......................................................................................trac/config.py:184: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  if name not in self._sections:
trac/config.py:185: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  self._sections[name] = Section(self, name)
trac/config.py:186: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  return self._sections[name]
................................................................................................................Exception AssertionError: AssertionError() in method __del__ of <trac.db.pool.PooledConnection object at 0x000000000647a7c8> ignored
Exception AssertionError: AssertionError() in method __del__ of <trac.db.pool.PooledConnection object at 0x000000000647a838> ignored
Exception AssertionError: AssertionError() in method __del__ of <trac.db.pool.PooledConnection object at 0x000000000439c058> ignored
Exception AssertionError: AssertionError() in method __del__ of <trac.db.pool.PooledConnection object at 0x000000000439c0c8> ignored
Exception AssertionError: AssertionError() in method __del__ of <trac.db.pool.PooledConnection object at 0x00000000032c7398> ignored
Exception AssertionError: AssertionError() in method __del__ of <trac.db.pool.PooledConnection object at 0x00000000032c7408> ignored
Exception AssertionError: AssertionError() in method __del__ of <trac.db.pool.PooledConnection object at 0x0000000005559210> ignored
Exception AssertionError: AssertionError() in method __del__ of <trac.db.pool.PooledConnection object at 0x0000000005559280> ignored
....................................................F.............F.F........................................................................................................................................................................................................E.............................................................................................................................................................................................................................................................................................................................F.........................................................FFFFFFF......................F.FF..FF....FF.............................................FFF.F.F.F................................................................................F..................................................................................................................................F.F..............................................................................................................................................................................................................FF...............................................................F............FF...........................................................F..................F.......................................................................................................................................F.....F.F......F..F...........................F...........................................................................................................................................................F....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................F..................................F............F.....................................

<snip>

Ran 2539 tests in 86.631s

FAILED (failures=44, errors=1)
Makefile:477: recipe for target 'unit-test' failed
make: *** [unit-test] Error 1

in reply to:  2 comment:4 by figaro, 4 months ago

Replying to kallisti5:

Trac 1.4.x Performance under python2 has been pretty lackluster lately, the PyPy change gave us a badly needed boost.

If Trac performance is your chief concern, then please have a look at TracPerformance. It lists a few options to look into. Also optimizing your web server for compressing traffic and caching files is recommended.

comment:5 by anonymous, 4 months ago

If Trac performance is your chief concern, then please have a look at TracPerformance. It lists a few options to look into. Also optimizing your web server for compressing traffic and caching files is recommended.

Our most common issue is the persistent 100% cpu issue. It occurs daily on 1.4.x (even with minimal to zero load per the http request logs)

After switching from python2 to pypy, the "tracd stuck at 100% cpu" issue has not reoccurred.

in reply to:  5 comment:6 by Jun Omae, 4 months ago

If Trac performance is your chief concern, then please have a look at TracPerformance. It lists a few options to look into. Also optimizing your web server for compressing traffic and caching files is recommended.

Our most common issue is the persistent 100% cpu issue. It occurs daily on 1.4.x (even with minimal to zero load per the http request logs)

After switching from python2 to pypy, the "tracd stuck at 100% cpu" issue has not reoccurred.

I think that is probably an issue HTTPServer library and/or CPython. Please create new ticket with the details of the hang (how to reproduce it if possible). At least, I haven't get the same issue with tracd on Linux.

Also, we don't recommend using tracd in production environment. Instead, try mod-wsgi.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The ticket will remain with no owner.
The ticket will be disowned.
as The resolution will be set. Next status will be 'closed'.
The owner will be changed from (none) to anonymous. Next status will be 'assigned'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.