Edgewall Software
Modify

Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#12880 closed enhancement (fixed)

TravisCI test failures running on Ubuntu Trusty

Reported by: Ryan J Ollos Owned by:
Priority: normal Milestone: 1.0.17
Component: general Version:
Severity: normal Keywords: travis-ci
Cc: Branch:
Release Notes:

Fix TravisCI test failures on Ubuntu trusty.

API Changes:
Internal Changes:

Description

There was a build failure yesterday when our tests ran on Trusty rather than precise.

Copied from 12833#comment:7:

  • python-subversion is not yet available in packages, although the logs show that it gets installed. SVN tests are skipped with (no svn bindings).
  • There's one failure:
    ERROR: runTest (trac.db.tests.functional.DatabaseBackupTestCase)
    
    Testing backup
    
    Traceback (most recent call last):
      File "/home/travis/build/rjollos/trac/trac/db/tests/functional.py", line 26, in runTest
        backup_file = env.backup()
      File "/home/travis/build/rjollos/trac/trac/env.py", line 716, in backup
        return DatabaseManager(self).backup(dest)
      File "/home/travis/build/rjollos/trac/trac/db/api.py", line 572, in backup
        return connector.backup(dest)
      File "/home/travis/build/rjollos/trac/trac/db/mysql_backend.py", line 280, in backup
        msg=to_unicode(errmsg.strip())))
    TracError: mysqldump failed: mysqldump: Got error: 1045: Access denied for user 'tracuser'@'localhost' (using password: NO) when trying to connect
    ----------------------------------------------------------------------
    Ran 205 tests in 357.456s
    FAILED (errors=1)
    make: *** [functional-test] Error 1
    

MySQL tests on Precise:

  Package        Version
  ------------------------------------------------------
  Python       : 2.7.3 (default, Dec 18 2014, 19:10:20) 
               : [GCC 4.6.3]
  Setuptools   : 12.0.5
  Jinja2       : 2.9.6
  Genshi       : 0.7 (with speedups)
  Babel        : 2.4.0
  sqlite3      : 2.6.0 (3.7.9)
  PySqlite     : not installed
  PyMySQL      : 0.7.11.None
  Psycopg2     : not installed
  SVN bindings : 1.6.17 (r1128011)
  Mercurial    : 2.0.2
  Pygments     : 2.2.0
  Textile      : 2.3.12
  Pytz         : 2017.2
  Docutils     : 0.13.1
  Twill        : 0.9
  LXML         : 3.8.0
  coverage     : not installed
  figleaf      : not installed

MySQL tests on Trusty:

  Package        Version
  ------------------------------------------------------
  Python       : 2.7.6 (default, Oct 26 2016, 20:30:19) 
               : [GCC 4.8.4]
  Setuptools   : 36.0.1
  Jinja2       : 2.9.6
  Genshi       : 0.7 (with speedups)
  Babel        : 2.4.0
  sqlite3      : 2.6.0 (3.8.2)
  PySqlite     : not installed
  PyMySQL      : 0.7.11.None
  Psycopg2     : not installed
  SVN bindings : not installed
  Mercurial    : not installed
  Pygments     : 2.2.0
  Textile      : 2.3.12
  Pytz         : 2017.2
  Docutils     : 0.13.1
  Twill        : 0.9
  LXML         : 3.8.0
  coverage     : not installed
  figleaf      : not installed

Subversion 1.8.8 is available on Precise.

Attachments (0)

Change History (15)

comment:1 by Ryan J Ollos, 3 years ago

Trusty has:

MySQL: 5.6.33-0ubuntu0.14.04.1

Precise has:

MySQL: 5.5.41-0ubuntu0.12.04.1-log

I would guess the problem is somehow with the MYSQL_PWD environment variable.

comment:2 by Jun Omae, 3 years ago

I'm using Ubuntu 14.04 with MySQL 5.5. I try to fix it after switching to MySQL 5.6.

comment:3 by Jun Omae, 3 years ago

Hmm. Cannot reproduce it. All tests pass with MySQL 5.6. It seems mysqldump with MYSQL_PWD works.

$ mysqldump --version
mysqldump  Ver 10.13 Distrib 5.6.33, for debian-linux-gnu (x86_64)
$ MYSQL_PWD=password mysqldump -h localhost -u tracuser trac_utf8mb4 | head -3
-- MySQL dump 10.13  Distrib 5.6.33, for debian-linux-gnu (x86_64)
--
-- Host: localhost    Database: trac_utf8mb4
...

comment:4 by Jun Omae, 3 years ago

If [client] password is configured in ~/.my.cnf, MYSQL_PWD environment variable is unused. According to MySQL 5.7 Reference Manual, the variable is the default password when connecting to mysqld.

$ cat ~/.my.cnf
[client]
password =
$ MYSQL_PWD=password mysqldump -h localhost -u tracuser trac_utf8mb4
mysqldump: Got error: 1045: Access denied for user 'tracuser'@'localhost' (using password: NO) when trying to connect

Work around is to add --no-defaults option, [7753c0b86/jomae.git]:

$ MYSQL_PWD=password mysqldump --no-defaults -h localhost -u tracuser trac_utf8mb4
-- MySQL dump 10.13  Distrib 5.6.33, for debian-linux-gnu (x86_64)
--
-- Host: localhost    Database: trac_utf8mb4
...

comment:5 by Ryan J Ollos, 3 years ago

That makes sense. Looks like it may have been previously reported in #12698.

comment:6 by Jun Omae, 3 years ago

Committed in [16199] and merged in [16200-16201].

in reply to:  description comment:7 by Jun Omae, 3 years ago

Replying to Ryan J Ollos:

Copied from 12833#comment:7:

  • python-subversion is not yet available in packages, although the logs show that it gets installed. SVN tests are skipped with (no svn bindings).

I think this issue is caused by difference about system-site-packages of python2.7_with_system_site_packages virtualenv between trusty and precise.

Trusty
python2.7_with_system_site_packages on trusty is created without --system-site-packages. no-global-site-packages.txt file exists in lib directory:
https://travis-ci.org/jun66j5/trac/builds/262106620#L466
Precise
python2.7_with_system_site_packages on precise is created with --system-site-packages. no-global-site-packages.txt file doesn't exist in lib directory:
https://travis-ci.org/jun66j5/trac/builds/262106471#L261

comment:8 by Jun Omae, 3 years ago

Work around is to delete no-global-site-packages.txt file when python version is 2.7_with_system_site_packages. See https://travis-ci.org/jun66j5/trac/jobs/262116580#L657.

  • .travis.yml

    diff --git a/.travis.yml b/.travis.yml
    index 083143136..01efbbd11 100644
    a b before_install:  
    5152      pip --version
    5253    fi
    5354install:
     55  - |
     56    if [ "$TRAVIS_PYTHON_VERSION" = "2.7_with_system_site_packages" ]; then
     57      no_global="$HOME/virtualenv/python2.7_with_system_site_packages/lib/python2.7/no-global-site-packages.txt"
     58      if [ -f "$no_global" ]; then rm -v "$no_global"; fi
     59      python -c 'from svn import core'
     60    fi
    5461  - test -d "$HOME/.pip" || mkdir "$HOME/.pip"
    5562  - ( echo "[global]"; echo "cache-dir = $HOME/.cache/pip" ) >"$HOME/.pip/pip.conf"
    5663  - pip install --upgrade pip

comment:9 by Ryan J Ollos, 3 years ago

On Linux and OSX I've used symlinks to avoid using --system-site-packages. On OSX my script is:

svn_python=$(locate -l 1 svn-python) > /dev/null
site_packages=pve/lib/python2.7/site-packages
[ ! -h $site_packages/svn ] && ln -s $svn_python/svn $site_packages/
[ ! -h $site_packages/libsvn ] && ln -s $svn_python/libsvn $site_packages/
[ ! -f $site_packages/svn.pth ] &&  echo $svn_python/libsvn > $site_packages/svn.pth

I don't know if that would also work for this case.

comment:10 by Jun Omae, 3 years ago

Yes. Using symlinks also works well. See https://travis-ci.org/jun66j5/trac/builds/262514664#L644.

  • .travis.yml

    diff --git a/.travis.yml b/.travis.yml
    index 083143136..a8cdfe3bb 100644
    a b before_install:  
    5151      pip --version
    5252    fi
    5353install:
     54  - |
     55    print_version='import sys; print("%d.%d" % sys.version_info[:2])'
     56    if [ "$(/usr/bin/python -c "$print_version")" = "$(python -c "$print_version")" ]; then
     57      print_sitelib='from distutils.sysconfig import get_python_lib; print(get_python_lib())'
     58      sitelib_global="$(/usr/bin/python -c "$print_sitelib")"
     59      sitelib_venv="$(python -c "$print_sitelib")"
     60      ln -s "$sitelib_global/svn" "$sitelib_venv/svn"
     61      ln -s "$sitelib_global/libsvn" "$sitelib_venv/libsvn"
     62      python -c 'from svn import core'
     63    fi
    5464  - test -d "$HOME/.pip" || mkdir "$HOME/.pip"
    5565  - ( echo "[global]"; echo "cache-dir = $HOME/.cache/pip" ) >"$HOME/.pip/pip.conf"
    5666  - pip install --upgrade pip
    jobs:  
    8999      python: "2.6"
    90100      env: tracdb=mysql
    91101    - os: linux
    92       python: "2.7_with_system_site_packages"
     102      python: "2.7"
    93103      env: tracdb=
    94104    - os: linux
    95       python: "2.7_with_system_site_packages"
     105      python: "2.7"
    96106      env: tracdb=sqlite
    97107    - os: linux
    98       python: "2.7_with_system_site_packages"
     108      python: "2.7"
    99109      env: tracdb=postgres
    100110    - os: linux
    101       python: "2.7_with_system_site_packages"
     111      python: "2.7"
    102112      env: tracdb=mysql
    103113    - os: osx
    104114      language: generic

comment:11 by Ryan J Ollos, 3 years ago

Thanks. Interesting that it works without the svn.pth. For some reason I have always found it necessary to create that file.

I imagine you have already considered this for the final yml configuration - we might want to conditionally add the svn lib bindings to the environment only when "$build" != minimum .

comment:13 by Ryan J Ollos, 3 years ago

Looks good to me.

comment:14 by Jun Omae, 3 years ago

Release Notes: modified (diff)
Resolution: fixed
Status: newclosed

Committed in [16203] and merged in [16204-16205].

comment:15 by Ryan J Ollos, 3 years ago

Milestone: 1.0.161.0.17

Milestone renamed

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The ticket will remain with no owner.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from (none) to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.