LookupError, <file> not found in manifest

[mike_mp@…]

Traceback:

Traceback (most recent call last):
  File "/Users/classic/Desktop/tmp/Trac-0.12.2/trac/web/main.py", line 511, in _dispatch_request
    dispatcher.dispatch(req)
  File "/Users/classic/Desktop/tmp/Trac-0.12.2/trac/web/main.py", line 237, in dispatch
    resp = chosen_handler.process_request(req)
  File "/Users/classic/Desktop/tmp/Trac-0.12.2/trac/versioncontrol/web_ui/browser.py", line 390, in process_request
    dir_data = self._render_dir(req, repos, node, rev, order, desc)
  File "/Users/classic/Desktop/tmp/Trac-0.12.2/trac/versioncontrol/web_ui/browser.py", line 549, in _render_dir
    entries = [entry(n) for n in node.get_entries()
  File "/Users/classic/Desktop/tmp/mercurial-plugin/tracext/hg/backend.py", line 1026, in get_entries
    str_entries)
  File "/Users/classic/Desktop/tmp/mercurial-plugin/tracext/hg/backend.py", line 956, in find_dirctx
    dr = max(dr, max_ctx.filectx(f).linkrev())
  File "/Users/classic/Desktop/tmp/lib/python2.7/site-packages/mercurial/context.py", line 177, in filectx
    fileid = self.filenode(path)
  File "/Users/classic/Desktop/tmp/lib/python2.7/site-packages/mercurial/context.py", line 166, in filenode
    return self._fileinfo(path)[0]
  File "/Users/classic/Desktop/tmp/lib/python2.7/site-packages/mercurial/context.py", line 161, in _fileinfo
    _('not found in manifest'))
LookupError: lib/sqlalchemy/databases/postgresql.py@58937c3f4abe: not found in manifest

Error appears in Trac 0.12 with Mercurial plugin r10879.

The error is be observed by linking Trac to the SQLAlchemy repository at ​http://hg.sqlalchemy.org/sqlalchemy/ , however I have a suspicion that the issue may be related to the latest revision being against a named branch that's not the "default". The error occurs when navigating to "lib/sqlalchemy/" as well as "test/perf", and probably others.

The only similar error I could find was #9611, but I don't have the same thing here as my Mercurial is reporting the version correctly:

classics-MacBook-Pro:mercurial-plugin classic$ ../bin/python
Python 2.7.2 (v2.7.2:8527427914a2, Jun 11 2011, 15:22:34) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from mercurial import __version__
>>> __version__.version
'2.0.1'

A patch which seems to resolve the issue, though is probably not the right way to go as it's just silently squashing the error:

tracext/hg/backend.py

@@ -953 +953 @@
                     str_files = str_entries[str_dir]
                     dr = str_dirctxs.get(str_dir, 0)
                     for f in str_files[:k]:
-                        dr = max(dr, max_ctx.filectx(f).linkrev())
+                        try:
+                            dr = max(dr, max_ctx.filectx(f).linkrev())
+                        except LookupError:
+                            pass
                     str_files = str_files[k:]
                     if str_files:
                         str_entries[str_dir] = str_files

This error did not occur in Trac .11 with the older version of the plugin. I tried working backwards through TracMercurial revs but the earliest rev that even works with 0.12 seems to be r10698 which also has the issue.

[Remy Blank]

Thanks for the very detailed bug report!

[Christian Boos]

Problem reproduced with Hg 2.0+20111103 but also with 1.8.4 so it's probably our fault ;-)

---

[OT] we did some simultaneous changes again ;-) ⇒ #7145

[Christian Boos]

Ok, so at @7871, when it switches to the "filelog" strategy, the files it is looking for are simply not there… which can happen of course if @7871 is on a different branch than from where we started from.

So yes, the proper way is to simply ignore the error and move on. The patch indeed works fine, thanks!

[Christian Boos]

Applied in r10898 for the 0.12 version and r10898 for the 0.13 version.

Whether the current approach of looking at all the past revisions one by one is adequate or not is another story… If we were to "stay" on a branch and only looking at the previous revision, we would perhaps have a more coherent view (such a bug would have been avoided), the browsing would certainly be even faster, but maybe we would have a slightly less useful view, as you wouldn't see in one place the "recent" activity that happened across several branches. Currently, if you browse Dir/ on branch A, you could see things like Dir/A modified "two days ago" on branch A, Dir/B modified "three days ago" on branch B. If we stayed on A, Dir/B would possibly show "modified ten months ago"…

So it's a trade-off, and maybe a browsing option would be useful here.

[Christian Boos]

Replying to cboos:

Applied in r10898 for the 0.12 version and r10898 for the 0.13 version.

Whether the current approach of looking at all the past revisions one by one is adequate or not is another story… If we were to "stay" on a branch and only looking at the previous revision, we would perhaps have a more coherent view (such a bug would have been avoided), the browsing would certainly be even faster, but maybe we would have a slightly less useful view, as you wouldn't see in one place the "recent" activity that happened across several branches. Currently, if you browse Dir/ on branch A, you could see things like Dir/A modified "two days ago" on branch A, Dir/B modified "three days ago" on branch B. If we stayed on A, Dir/B would possibly show "modified ten months ago"…

So it's a trade-off, and maybe a browsing option would be useful here.

Well, I think the current approach is simply wrong. We shouldn't even try to provide a "flattened" view of all the changes happening across all the branches.

So we need to start from the current changeset and follow the ancestors only.

[Christian Boos]

Fixed in [8af21bda2b3e/mercurial-plugin].

[christian at innoflair dot com]

Hi,

I fear the problem still exists (sort of) in Trac 1.0.1. In particular, the browser crashes when it tries to display a directory that exists only in one of two branches that have been just merged.

How to Reproduce

We're starting of with a directory lunch, containing only 'menu'

Changeset 2:338ba70acc51 in mercurial

Timestamp:
    2014-09-11T16:43:32+02:00 (less than one hour ago) 
Author:
    christian
Branch:
    default
Children:
    3:2cacdab72ed6, 4:00b54988c818
Message:

    tracking trac problem
File:

        1 added


        lunch/menu 

Now we are branching, one author is adding lasagne to the menu, the other fish & chips, i.e. both authors create a clone of the repo.

The lasagna cook adds pasta to the meal, commits and pushes his changes to the repo:

Changeset 3:2cacdab72ed6 in mercurial

Timestamp:
    2014-09-11T16:45:45+02:00 (less than one hour ago) 
Author:
    lasagna
Branch:
    default
Message:

    tracking trac problem
File:

        1 added


        lunch/lasagna/pasta 

Now the fish cook adds the chips, and commits:

Changeset 4:00b54988c818 in mercurial

Timestamp:
    2014-09-11T16:46:56+02:00 (less than one hour ago) 
Author:
    fish
Branch:
    default
Message:

    tracking trac problem
File:

        1 added


        lunch/fish/chips 

When he tries to push he has to perform a merge first:

Changeset 5:d82d1145ae7c in mercurial

Timestamp:
    2014-09-11T16:48:28+02:00 (less than one hour ago) 
Author:
    fish
Branch:
    default
Parents:
    4:00b54988c818 (diff), 3:2cacdab72ed6 (diff)
    Note: this is a merge changeset, the changes displayed below correspond to the merge itself.
    Use the (diff) links above to see all the changes relative to each parent.
Tags:
    tip
Message:

    merging trac problem
(No files)

So far, so good. Rather ordinary mercurial scenario. The problem arises, when you know use the source browser and try to access the parent directory (i.e. lunch) of the newly created directories. It tries to find the lasagna in changeset 4:00b54988c818 (fish&chips branch) where it does not exist.

When I add another sibling to the directories, the problem vanishes and the source browser is usable again:

Changeset 6:18842521b261 in mercurial

Timestamp:
    2014-09-11T17:15:11+02:00 (less than one hour ago) 
Author:
    christian
Branch:
    default
Tags:
    tip
Message:

    temporary problem vanishes
File:

        1 added


        lunch/desert/fruit 

I think this is the minimal set to produce the error.

While doing a GET operation on /browser/mercurial/lunch, Trac issued an internal error.

(please provide additional details here)

Request parameters:

{'path': u'/mercurial/lunch'}

User agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0

System Information

Trac	1.0.1
Babel	0.9.6
Docutils	0.8.1
Genshi	0.6 (with speedups)
Mercurial	2.8.2
mod_wsgi	3.3 (WSGIProcessGroup testproject WSGIApplicationGroup %{GLOBAL})
Pygments	1.6
pysqlite	2.6.0
Python	2.7.3 (default, Mar 13 2014, 11:26:58) [GCC 4.7.2]
pytz	2012c
setuptools	0.6
SQLite	3.7.13
Subversion	1.8.9 (r1591380)
jQuery	1.7.2

Enabled Plugins

graphviz	0.7.5
TracMasterTickets	3.0.2
TracMercurial	1.0.0.3dev
TracWysiwyg	0.12.0.3

Python Traceback

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/trac/web/main.py", line 497, in _dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib/python2.7/dist-packages/trac/web/main.py", line 214, in dispatch
    resp = chosen_handler.process_request(req)
  File "/usr/lib/python2.7/dist-packages/trac/versioncontrol/web_ui/browser.py", line 395, in process_request
    dir_data = self._render_dir(req, repos, node, rev, order, desc)
  File "/usr/lib/python2.7/dist-packages/trac/versioncontrol/web_ui/browser.py", line 563, in _render_dir
    entries = [entry(n) for n in node.get_entries()
  File "/usr/lib/python2.7/dist-packages/tracext/hg/backend.py", line 1077, in get_entries
    str_entries)
  File "/usr/lib/python2.7/dist-packages/tracext/hg/backend.py", line 1003, in _find_dirctx
    dr = max(dr, max_ctx.filectx(f).linkrev())
  File "/usr/lib/python2.7/dist-packages/mercurial/context.py", line 375, in filectx
    fileid = self.filenode(path)
  File "/usr/lib/python2.7/dist-packages/mercurial/context.py", line 161, in filenode
    return self._fileinfo(path)[0]
  File "/usr/lib/python2.7/dist-packages/mercurial/context.py", line 156, in _fileinfo
    _('not found in manifest'))
ManifestLookupError: lunch/lasagna/pasta@00b54988c818: not found in manifest

[Christian Boos]

Replying to christian at innoflair dot com:

Hi,

I fear the problem still exists (sort of) in Trac 1.0.1. In particular, the browser crashes when it tries to display a directory that exists only in one of two branches that have been just merged. […]

   [...]
 File "/usr/lib/python2.7/dist-packages/tracext/hg/backend.py", line 1003, in _find_dirctx
    dr = max(dr, max_ctx.filectx(f).linkrev())
   [...]
ManifestLookupError: lunch/lasagna/pasta@00b54988c818: not found in manifest

Good catch, and nice explanation, thanks!

[Johannes Weißl <jargon@…>]

We experience the same bug with Trac 1.0.2:

Trac detected an internal error:

ManifestLookupError: <file>@<rev>: not found in manifest

Python Traceback:

    File "/usr/lib/python2.7/dist-packages/trac/web/main.py", line 512, in _dispatch_request
    File "/usr/lib/python2.7/dist-packages/trac/web/main.py", line 221, in dispatch
    File "/usr/lib/python2.7/dist-packages/trac/versioncontrol/web_ui/browser.py", line 407, in process_request
    File "/usr/lib/python2.7/dist-packages/trac/versioncontrol/web_ui/browser.py", line 582, in _render_dir
    File "/usr/lib/python2.7/dist-packages/tracext/hg/backend.py", line 1121, in get_entries
    File "/usr/lib/python2.7/dist-packages/tracext/hg/backend.py", line 1047, in _find_dirctx
    File "/usr/lib/python2.7/dist-packages/mercurial/context.py", line 537, in filectx
    File "/usr/lib/python2.7/dist-packages/mercurial/context.py", line 246, in filenode
    File "/usr/lib/python2.7/dist-packages/mercurial/context.py", line 241, in _fileinfo 

[Peter Suter]

Fixed with test in [58/mercurial-plugin].