Mercurial plugin tests

[Peter Suter]

Could tests be added for the Mercurial plugin? It would be nice if they can be run on the automated build systems and help find incompatibilities with new Mercurial releases.

[Peter Suter]

The attached patch​ just copies some of these Git tests​. I omitted tests for bare repos and tests using "fast import data", and crudely modified the others until they passed for me, but I don't understand the the Trac version control system very well, so maybe this is misguided. What do you think?

[Jun Omae]

#12160 was closed as duplicate.

[Peter Suter]

Oops I forgot to check, thanks. Also I forgot a previous Git tests -> HG tests attempt​ for the HG command server approach.

[Peter Suter]

Pushed in [54/mercurial-plugin] for now.

---

It would be nice if they can be run on the automated build systems

Does anyone know if this is possible?

[Jun Omae]

python setup.py test doesn't work.

setup.py

@@ -55 +55 @@
           '': ['COPYING', 'README'],
           'tracext.hg': ['locale/*.*', 'locale/*/LC_MESSAGES/*.*'],
           },
+      test_suite='tracext.hg.tests.test_suite',
       entry_points={'trac.plugins': 'hg = tracext.hg.backend'},
       **extra)

The unit tests run only with trunk.

tracext/hg/tests/backend.py

@@ -15 +15 @@
 import unittest
 from datetime import datetime, timedelta
 
-from trac.test import EnvironmentStub, MockRequest, mkdtemp, rmtree
+from trac.test import EnvironmentStub, MockRequest, rmtree
 from trac.util import create_file
 from trac.util.datefmt import utc
 from trac.versioncontrol.api import Changeset, DbRepositoryProvider, \
@@ -25 +25 @@
 from trac.versioncontrol.web_ui.browser import BrowserModule
 from trac.versioncontrol.web_ui.log import LogModule
 
+try:
+    from trac.test import mkdtemp
+except ImportError:
+    import tempfile
+    def mkdtemp():
+        return os.path.realpath(tempfile.mkdtemp(prefix='trac-testdir-'))
+
 from tracext.hg.backend import MercurialConnector, MercurialRepository
 
 import mercurial.commands

1 failure with Trac 1.0.15:

======================================================================
ERROR: test_get_system_info (tracext.hg.tests.backend.MercurialConnectorTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/mercurial-plugin/tracext/hg/tests/backend.py", line 345, in test_get_system_info
    self.assertIsNotNone(self._hg_version_from_system_info())
  File "/home/jun66j5/src/mercurial-plugin/tracext/hg/tests/backend.py", line 339, in _hg_version_from_system_info
    for name, version in self.env.system_info:
AttributeError: 'EnvironmentStub' object has no attribute 'system_info'

[Jun Omae]

It would be nice if they can be run on the automated build systems

Does anyone know if this is possible?

I think we could use BitBucket piplelines after creating mirror of mercurial-plugin in ​https://bitbucket.org/edgewall/.

[Ryan J Ollos]

@psuter: I can give you Maintainer access for publishing the new release to ​pypi:TracMercurial. Do you have a PyPI account?

[Jun Omae]

1 failure with Trac 1.0.15:

======================================================================
ERROR: test_get_system_info (tracext.hg.tests.backend.MercurialConnectorTestCase)
----------------------------------------------------------------------
...
AttributeError: 'EnvironmentStub' object has no attribute 'system_info'

The system_info property is introduced in Trac 1.3.1dev.

Another issue, mercurial.commands.commit accepts iso8601 date string as date parameter since Mercurial 3.9.x. Prior to 3.9.0, it cannot.

Successfully installed Mercurial-3.8.4
running test
running egg_info
writing requirements to TracMercurial.egg-info/requires.txt
writing TracMercurial.egg-info/PKG-INFO
writing namespace_packages to TracMercurial.egg-info/namespace_packages.txt
writing top-level names to TracMercurial.egg-info/top_level.txt
writing dependency_links to TracMercurial.egg-info/dependency_links.txt
writing entry points to TracMercurial.egg-info/entry_points.txt
reading manifest file 'TracMercurial.egg-info/SOURCES.txt'
writing manifest file 'TracMercurial.egg-info/SOURCES.txt'
running build_ext
EEE.EEE.EEE.EEEE
======================================================================
ERROR: test_duplicate_repo (tracext.hg.tests.backend.SanityCheckingTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/mercurial-plugin/tracext/hg/tests/backend.py", line 103, in test_duplicate_repo
    self._hg_init()
  File "/home/jun66j5/src/mercurial-plugin/tracext/hg/tests/backend.py", line 84, in _hg_init
    user='Joe', date=ts.isoformat())
  File "/dev/shm/hgtest/local/lib/python2.7/site-packages/mercurial/commands.py", line 1682, in commit
    return _docommit(ui, repo, *pats, **opts)
  File "/dev/shm/hgtest/local/lib/python2.7/site-packages/mercurial/commands.py", line 1771, in _docommit
    node = cmdutil.commit(ui, repo, commitfunc, pats, opts)
  File "/dev/shm/hgtest/local/lib/python2.7/site-packages/mercurial/cmdutil.py", line 2610, in commit
    opts['date'] = util.parsedate(date)
  File "/dev/shm/hgtest/local/lib/python2.7/site-packages/mercurial/util.py", line 1724, in parsedate
    raise Abort(_('invalid date: %r') % date)
Abort: invalid date: '2001-01-29T16:39:56'

All versions of Mercurial accept rfc822 date string.

tracext/hg/tests/backend.py

@@ -15 +15 @@
 import unittest
 from datetime import datetime, timedelta
 
-from trac.test import EnvironmentStub, MockRequest, mkdtemp, rmtree
+from trac.test import EnvironmentStub, MockRequest, rmtree
 from trac.util import create_file
-from trac.util.datefmt import utc
+from trac.util.datefmt import http_date, utc
 from trac.versioncontrol.api import Changeset, DbRepositoryProvider, \
                                     InvalidRepository, Node, \
                                     NoSuchChangeset, NoSuchNode, \
@@ -25 +25 @@
 from trac.versioncontrol.web_ui.browser import BrowserModule
 from trac.versioncontrol.web_ui.log import LogModule
 
+try:
+    from trac.test import mkdtemp
+except ImportError:
+    import tempfile
+    def mkdtemp():
+        return os.path.realpath(tempfile.mkdtemp(prefix='trac-testdir-'))
+
 from tracext.hg.backend import MercurialConnector, MercurialRepository
 
 import mercurial.commands
@@ -71 +78 @@
         if data:
             hgignore = os.path.join(self.repos_path, '.hgignore')
             create_file(hgignore)
-            ts = datetime(2001, 1, 29, 16, 39, 56)
+            ts = datetime(2001, 1, 29, 16, 39, 56, 0, utc)
             mercurial.commands.commit(self.hgui, self.hgrepo, hgignore,
                                       message="test", addremove=True,
-                                      user='Joe', date=ts.isoformat())
+                                      user='Joe', date=http_date(ts))
 
     def _create_commit(self, message, filename):
         fullname = os.path.join(self.repos_path, filename)
@@ -111 +118 @@
             create_file(filename, 'commit-%d.txt' % idx)
             mercurial.commands.commit(self.hgui, self.hgrepo, filename,
                                       message='%d' % idx, addremove=True,
-                                      user='Joe', date=ts.isoformat())
+                                      user='Joe', date=http_date(ts))
         self._add_repository()
         repos = self._repomgr.get_repository('hgrepos')
         repos.repo.invalidate()
@@ -327 +334 @@
 
 class MercurialConnectorTestCase(BaseTestCase):
 
+    if hasattr(EnvironmentStub, 'system_info'):
+        def _get_system_info(self):
+            return self.env.system_info
+    else:
+        def _get_system_info(self):
+            return self.env.get_systeminfo()
+
     def _hg_version_from_system_info(self):
         hg_version = None
-        for name, version in self.env.system_info:
+        for name, version in self._get_system_info():
             if name == 'Mercurial':
                 hg_version = version
         return hg_version

With Mercurial 3.7.x - 4.5.x, the following failures are occur. I'm not sure how to fix.

======================================================================
ERROR: test_duplicate_repo (tracext.hg.tests.backend.SanityCheckingTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/mercurial-plugin/tracext/hg/tests/backend.py", line 103, in test_duplicate_repo
    self._hg_init()
  File "/home/jun66j5/src/mercurial-plugin/tracext/hg/tests/backend.py", line 84, in _hg_init
    user='Joe', date=ts.isoformat())
  File "/dev/shm/hgtest/local/lib/python2.7/site-packages/mercurial/commands.py", line 1525, in commit
    return _docommit(ui, repo, *pats, **opts)
  File "/dev/shm/hgtest/local/lib/python2.7/site-packages/mercurial/commands.py", line 1602, in _docommit
    node = cmdutil.commit(ui, repo, commitfunc, pats, opts)
  File "/dev/shm/hgtest/local/lib/python2.7/site-packages/mercurial/cmdutil.py", line 3129, in commit
    dsguard = dirstateguard.dirstateguard(repo, 'commit')
  File "/dev/shm/hgtest/local/lib/python2.7/site-packages/mercurial/dirstateguard.py", line 36, in __init__
    repo.dirstate.savebackup(repo.currenttransaction(), self._backupname)
  File "/dev/shm/hgtest/local/lib/python2.7/site-packages/mercurial/dirstate.py", line 1135, in savebackup
    checkambig=True))
  File "/dev/shm/hgtest/local/lib/python2.7/site-packages/mercurial/dirstate.py", line 650, in _writedirstate
    now = util.fstat(st).st_mtime & _rangemask
TypeError: unsupported operand type(s) for &: 'float' and 'int'

[Peter Suter]

Thanks, the proposed changes sound good to me. Looks like running the tests automatically for old and new versions of Mercurial and Trac would be helpful. Maybe ​Bitbucket Scheduled Pipelines can be used?

Do you have a PyPI account?

I created ​PyPI account psuter.

[Peter Suter]

With Mercurial 3.7.x - 4.5.x, the following failures are occur. I'm not sure how to fix.

    ...
    now = util.fstat(st).st_mtime & _rangemask
TypeError: unsupported operand type(s) for &: 'float' and 'int'

Is this related to your findings in comment:10:ticket:10411?

[Jun Omae]

Replying to Peter Suter:

With Mercurial 3.7.x - 4.5.x, the following failures are occur. I'm not sure how to fix.

    ...
    now = util.fstat(st).st_mtime & _rangemask
TypeError: unsupported operand type(s) for &: 'float' and 'int'

Is this related to your findings in comment:10:ticket:10411?

Yeah, the errors go away after os.stat_float_times(False) is manually added to tracext/hg/tests/backend.py. However, I think the behavior is weird. The os.stat_float_times(False) is automaticaly invoked by import mercurial.util.

Anyway, 1 failure occurs with Mercurial 3.7.x - 4.5.x:

======================================================================
ERROR: test_annotate (tracext.hg.tests.backend.RegressionsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/mercurial-plugin/tracext/hg/tests/backend.py", line 381, in test_annotate
    self.assertEqual([1], node.get_annotations())
  File "/home/jun66j5/src/mercurial-plugin/tracext/hg/backend.py", line 1252, in get_annotations
    annotations.append(fc.rev() or '0')
AttributeError: 'annotateline' object has no attribute 'rev'

tracext/hg/backend.py

@@ -1246 +1246 @@
                     fc = annotateline[0]
                     if isinstance(fc, tuple):
                         fc = fc[0]
+                    elif not isinstance(fc, filectx):
+                        fc = fc.fctx
                 else:
                     fc = annotateline.fctx
                 annotations.append(fc.rev() or '0')

[Peter Suter]

All versions of Mercurial accept rfc822 date string.

Using http_date in Mercurial 4.6.1 I get:

Traceback (most recent call last):
  File "C:\src\mercurial-plugin\tracext\hg\tests\backend.py", line 103, in test_duplicate_repo
    self._hg_init()
  File "C:\src\mercurial-plugin\tracext\hg\tests\backend.py", line 84, in _hg_init
    date=http_date(ts))
  File "C:\venv_py27\lib\site-packages\mercurial\commands.py", line 1543, in commit
    return _docommit(ui, repo, *pats, **opts)
  File "C:\venv_py27\lib\site-packages\mercurial\commands.py", line 1620, in _docommit
    node = cmdutil.commit(ui, repo, commitfunc, pats, opts)
  File "C:\venv_py27\lib\site-packages\mercurial\cmdutil.py", line 2357, in commit
    opts['date'] = dateutil.parsedate(date)
  File "C:\venv_py27\lib\site-packages\mercurial\utils\dateutil.py", line 247, in parsedate
    _('invalid date: %r') % pycompat.bytestr(date))
ParseError: invalid date: 'Mon, 29 Jan 2001 16:39:56 GMT'

[Ryan J Ollos]

Replying to Peter Suter:

Do you have a PyPI account?

I created ​PyPI account psuter.

I've added you a project owner. To create and upload a release for PyPI:

$ python setup.py sdist bdist_wheel
$ twine upload dist/*.tar.gz dist/*.whl

[Jun Omae]

All versions of Mercurial accept rfc822 date string.

Using http_date in Mercurial 4.6.1 I get:

...
ParseError: invalid date: 'Mon, 29 Jan 2001 16:39:56 GMT'

Oh, thanks. Fixed in [058df3c9e351/mercurial-plugin].

[Jun Omae]

Replying to Jun Omae:

Replying to Peter Suter:

With Mercurial 3.7.x - 4.5.x, the following failures are occur. I'm not sure how to fix.

    ...
    now = util.fstat(st).st_mtime & _rangemask
TypeError: unsupported operand type(s) for &: 'float' and 'int'

Is this related to your findings in comment:10:ticket:10411?

Yeah, the errors go away after os.stat_float_times(False) is manually added to tracext/hg/tests/backend.py. However, I think the behavior is weird. The os.stat_float_times(False) is automaticaly invoked by import mercurial.util.

This behavior is because tracext/hg/backend.py restores os.stat_float_times()'s value at mercurial-plugin/tracext/hg/backend.py@e5f80d909b3f#L133​.

Also, I found work around to use os.stat_float_times(False) while writing repository and working copy. Proposed changes in ​https://bitbucket.org/jun66j5/mercurial-plugin/commits/a236958adc560f5f0536cd5a6770f9f19d9a646a.

Anyway, 1 failure occurs with Mercurial 3.7.x - 4.5.x:

======================================================================
ERROR: test_annotate (tracext.hg.tests.backend.RegressionsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun66j5/src/mercurial-plugin/tracext/hg/tests/backend.py", line 381, in test_annotate
    self.assertEqual([1], node.get_annotations())
  File "/home/jun66j5/src/mercurial-plugin/tracext/hg/backend.py", line 1252, in get_annotations
    annotations.append(fc.rev() or '0')
AttributeError: 'annotateline' object has no attribute 'rev'

tracext/hg/backend.py

@@ -1246 +1246 @@
                     fc = annotateline[0]
                     if isinstance(fc, tuple):
                         fc = fc[0]
+                    elif not isinstance(fc, filectx):
+                        fc = fc.fctx
                 else:
                     fc = annotateline.fctx
                 annotations.append(fc.rev() or '0')

Proposed changes in ​https://bitbucket.org/jun66j5/mercurial-plugin/commits/923f81dadc2f5685dd683c810b5149b6edfac983. The fix and revised unit tests are included.

[Peter Suter]

The proposed changes look very good to me.

[Jun Omae]

The proposed changes look very good to me.

Thanks. Pushed the two commits in mercurial-plugin@a236958adc56:923f81dadc2f.

[Ryan J Ollos]

Replying to Peter Suter:

Thanks, the proposed changes sound good to me. Looks like running the tests automatically for old and new versions of Mercurial and Trac would be helpful. Maybe ​Bitbucket Scheduled Pipelines can be used?

You might be able to do the same on Travis CI and AppVeyor. We have quite a bit of effort invested in running tests and deploying (#12833) on those platforms.

[Jun Omae]

Replying to Ryan J Ollos:

Replying to Peter Suter:

Thanks, the proposed changes sound good to me. Looks like running the tests automatically for old and new versions of Mercurial and Trac would be helpful. Maybe ​Bitbucket Scheduled Pipelines can be used?

Yes. The scheduled pipelines is helpful to run unit tests with latest marcurial.

Minimum configuration: ​https://bitbucket.org/jun66j5/mercurial-plugin/commits/096b7acf79659b0aca190ffcad234f6b136081be

However, BitBucket pipelines seems to be missing build matrix feature. It is not easy to test with all combination of the versions (e.g. Mercurial (3.6, 3.7, 4.5, 4.6) x Trac (1.0, 1.2, trunk)).

You might be able to do the same on Travis CI and AppVeyor. We have quite a bit of effort invested in running tests and deploying (#12833) on those platforms.

The mercurial-plugin is in mercurial repository. I think it needs to convert mercurial repository to git to use Travis CI and AppVeyor.