Edgewall Software
Modify

Opened 6 years ago

Last modified 16 months ago

#10515 assigned defect

LookupError, <file> not found in manifest

Reported by: mike_mp@… Owned by: Christian Boos
Priority: high Milestone: plugin - mercurial
Component: plugin/mercurial Version: 1.0
Severity: normal Keywords: hg
Cc: jargon@…
Release Notes:

Fixed retrieval of last change on directories in the TracBrowser.

API Changes:

Description

Traceback:

Traceback (most recent call last):
  File "/Users/classic/Desktop/tmp/Trac-0.12.2/trac/web/main.py", line 511, in _dispatch_request
    dispatcher.dispatch(req)
  File "/Users/classic/Desktop/tmp/Trac-0.12.2/trac/web/main.py", line 237, in dispatch
    resp = chosen_handler.process_request(req)
  File "/Users/classic/Desktop/tmp/Trac-0.12.2/trac/versioncontrol/web_ui/browser.py", line 390, in process_request
    dir_data = self._render_dir(req, repos, node, rev, order, desc)
  File "/Users/classic/Desktop/tmp/Trac-0.12.2/trac/versioncontrol/web_ui/browser.py", line 549, in _render_dir
    entries = [entry(n) for n in node.get_entries()
  File "/Users/classic/Desktop/tmp/mercurial-plugin/tracext/hg/backend.py", line 1026, in get_entries
    str_entries)
  File "/Users/classic/Desktop/tmp/mercurial-plugin/tracext/hg/backend.py", line 956, in find_dirctx
    dr = max(dr, max_ctx.filectx(f).linkrev())
  File "/Users/classic/Desktop/tmp/lib/python2.7/site-packages/mercurial/context.py", line 177, in filectx
    fileid = self.filenode(path)
  File "/Users/classic/Desktop/tmp/lib/python2.7/site-packages/mercurial/context.py", line 166, in filenode
    return self._fileinfo(path)[0]
  File "/Users/classic/Desktop/tmp/lib/python2.7/site-packages/mercurial/context.py", line 161, in _fileinfo
    _('not found in manifest'))
LookupError: lib/sqlalchemy/databases/postgresql.py@58937c3f4abe: not found in manifest

Error appears in Trac 0.12 with Mercurial plugin r10879.

The error is be observed by linking Trac to the SQLAlchemy repository at http://hg.sqlalchemy.org/sqlalchemy/ , however I have a suspicion that the issue may be related to the latest revision being against a named branch that's not the "default". The error occurs when navigating to "lib/sqlalchemy/" as well as "test/perf", and probably others.

The only similar error I could find was #9611, but I don't have the same thing here as my Mercurial is reporting the version correctly:

classics-MacBook-Pro:mercurial-plugin classic$ ../bin/python
Python 2.7.2 (v2.7.2:8527427914a2, Jun 11 2011, 15:22:34) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from mercurial import __version__
>>> __version__.version
'2.0.1'

A patch which seems to resolve the issue, though is probably not the right way to go as it's just silently squashing the error:

  • tracext/hg/backend.py

    classics-MacBook-Pro:mercurial-plugin classic$ svn diff
     
    953953                    str_files = str_entries[str_dir]
    954954                    dr = str_dirctxs.get(str_dir, 0)
    955955                    for f in str_files[:k]:
    956                         dr = max(dr, max_ctx.filectx(f).linkrev())
     956                        try:
     957                            dr = max(dr, max_ctx.filectx(f).linkrev())
     958                        except LookupError:
     959                            pass
    957960                    str_files = str_files[k:]
    958961                    if str_files:
    959962                        str_entries[str_dir] = str_files

This error did not occur in Trac .11 with the older version of the plugin. I tried working backwards through TracMercurial revs but the earliest rev that even works with 0.12 seems to be r10698 which also has the issue.

Attachments (0)

Change History (10)

comment:1 Changed 6 years ago by Remy Blank

Milestone: plugin - mercurial
Reporter: changed from anonymous to mike_mp@…
Version: 0.12.2

Thanks for the very detailed bug report!

comment:2 Changed 6 years ago by Christian Boos

Keywords: hg added
Priority: normalhigh
Version: 0.12.20.12dev

Problem reproduced with Hg 2.0+20111103 but also with 1.8.4 so it's probably our fault ;-)


[OT] we did some simultaneous changes again ;-) ⇒ #7145

comment:3 Changed 6 years ago by Christian Boos

Cc: mike_mp@… removed
Owner: set to mike_mp@…

Ok, so at @7871, when it switches to the "filelog" strategy, the files it is looking for are simply not there… which can happen of course if @7871 is on a different branch than from where we started from.

So yes, the proper way is to simply ignore the error and move on. The patch indeed works fine, thanks!

comment:4 Changed 6 years ago by Christian Boos

Release Notes: modified (diff)

Applied in r10898 for the 0.12 version and r10898 for the 0.13 version.

Whether the current approach of looking at all the past revisions one by one is adequate or not is another story… If we were to "stay" on a branch and only looking at the previous revision, we would perhaps have a more coherent view (such a bug would have been avoided), the browsing would certainly be even faster, but maybe we would have a slightly less useful view, as you wouldn't see in one place the "recent" activity that happened across several branches. Currently, if you browse Dir/ on branch A, you could see things like Dir/A modified "two days ago" on branch A, Dir/B modified "three days ago" on branch B. If we stayed on A, Dir/B would possibly show "modified ten months ago"…

So it's a trade-off, and maybe a browsing option would be useful here.

comment:5 Changed 6 years ago by Christian Boos

Resolution: fixed
Status: newclosed

comment:6 in reply to:  4 Changed 5 years ago by Christian Boos

Resolution: fixed
Status: closedreopened

Replying to cboos:

Applied in r10898 for the 0.12 version and r10898 for the 0.13 version.

Whether the current approach of looking at all the past revisions one by one is adequate or not is another story… If we were to "stay" on a branch and only looking at the previous revision, we would perhaps have a more coherent view (such a bug would have been avoided), the browsing would certainly be even faster, but maybe we would have a slightly less useful view, as you wouldn't see in one place the "recent" activity that happened across several branches. Currently, if you browse Dir/ on branch A, you could see things like Dir/A modified "two days ago" on branch A, Dir/B modified "three days ago" on branch B. If we stayed on A, Dir/B would possibly show "modified ten months ago"…

So it's a trade-off, and maybe a browsing option would be useful here.

Well, I think the current approach is simply wrong. We shouldn't even try to provide a "flattened" view of all the changes happening across all the branches.

So we need to start from the current changeset and follow the ancestors only.

comment:7 Changed 5 years ago by Christian Boos

Resolution: fixed
Status: reopenedclosed

comment:8 Changed 3 years ago by christian at innoflair dot com

Resolution: fixed
Status: closedreopened
Version: 0.12dev1.0

Hi,

I fear the problem still exists (sort of) in Trac 1.0.1. In particular, the browser crashes when it tries to display a directory that exists only in one of two branches that have been just merged.

How to Reproduce

We're starting of with a directory lunch, containing only 'menu'

Changeset 2:338ba70acc51 in mercurial

Timestamp:
    2014-09-11T16:43:32+02:00 (less than one hour ago) 
Author:
    christian
Branch:
    default
Children:
    3:2cacdab72ed6, 4:00b54988c818
Message:

    tracking trac problem
File:

        1 added


        lunch/menu 

Now we are branching, one author is adding lasagne to the menu, the other fish & chips, i.e. both authors create a clone of the repo.

The lasagna cook adds pasta to the meal, commits and pushes his changes to the repo:

Changeset 3:2cacdab72ed6 in mercurial

Timestamp:
    2014-09-11T16:45:45+02:00 (less than one hour ago) 
Author:
    lasagna
Branch:
    default
Message:

    tracking trac problem
File:

        1 added


        lunch/lasagna/pasta 

Now the fish cook adds the chips, and commits:

Changeset 4:00b54988c818 in mercurial

Timestamp:
    2014-09-11T16:46:56+02:00 (less than one hour ago) 
Author:
    fish
Branch:
    default
Message:

    tracking trac problem
File:

        1 added


        lunch/fish/chips 

When he tries to push he has to perform a merge first:

Changeset 5:d82d1145ae7c in mercurial

Timestamp:
    2014-09-11T16:48:28+02:00 (less than one hour ago) 
Author:
    fish
Branch:
    default
Parents:
    4:00b54988c818 (diff), 3:2cacdab72ed6 (diff)
    Note: this is a merge changeset, the changes displayed below correspond to the merge itself.
    Use the (diff) links above to see all the changes relative to each parent.
Tags:
    tip
Message:

    merging trac problem
(No files)

So far, so good. Rather ordinary mercurial scenario. The problem arises, when you know use the source browser and try to access the parent directory (i.e. lunch) of the newly created directories. It tries to find the lasagna in changeset 4:00b54988c818 (fish&chips branch) where it does not exist.

When I add another sibling to the directories, the problem vanishes and the source browser is usable again:

Changeset 6:18842521b261 in mercurial

Timestamp:
    2014-09-11T17:15:11+02:00 (less than one hour ago) 
Author:
    christian
Branch:
    default
Tags:
    tip
Message:

    temporary problem vanishes
File:

        1 added


        lunch/desert/fruit 

I think this is the minimal set to produce the error.

While doing a GET operation on /browser/mercurial/lunch, Trac issued an internal error.

(please provide additional details here)

Request parameters:

{'path': u'/mercurial/lunch'}

User agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0

System Information

Trac 1.0.1
Babel 0.9.6
Docutils 0.8.1
Genshi 0.6 (with speedups)
Mercurial 2.8.2
mod_wsgi 3.3 (WSGIProcessGroup testproject WSGIApplicationGroup %{GLOBAL})
Pygments 1.6
pysqlite 2.6.0
Python 2.7.3 (default, Mar 13 2014, 11:26:58)
[GCC 4.7.2]
pytz 2012c
setuptools 0.6
SQLite 3.7.13
Subversion 1.8.9 (r1591380)
jQuery 1.7.2

Enabled Plugins

graphviz 0.7.5
TracMasterTickets 3.0.2
TracMercurial 1.0.0.3dev
TracWysiwyg 0.12.0.3

Python Traceback

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/trac/web/main.py", line 497, in _dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib/python2.7/dist-packages/trac/web/main.py", line 214, in dispatch
    resp = chosen_handler.process_request(req)
  File "/usr/lib/python2.7/dist-packages/trac/versioncontrol/web_ui/browser.py", line 395, in process_request
    dir_data = self._render_dir(req, repos, node, rev, order, desc)
  File "/usr/lib/python2.7/dist-packages/trac/versioncontrol/web_ui/browser.py", line 563, in _render_dir
    entries = [entry(n) for n in node.get_entries()
  File "/usr/lib/python2.7/dist-packages/tracext/hg/backend.py", line 1077, in get_entries
    str_entries)
  File "/usr/lib/python2.7/dist-packages/tracext/hg/backend.py", line 1003, in _find_dirctx
    dr = max(dr, max_ctx.filectx(f).linkrev())
  File "/usr/lib/python2.7/dist-packages/mercurial/context.py", line 375, in filectx
    fileid = self.filenode(path)
  File "/usr/lib/python2.7/dist-packages/mercurial/context.py", line 161, in filenode
    return self._fileinfo(path)[0]
  File "/usr/lib/python2.7/dist-packages/mercurial/context.py", line 156, in _fileinfo
    _('not found in manifest'))
ManifestLookupError: lunch/lasagna/pasta@00b54988c818: not found in manifest

comment:9 in reply to:  8 Changed 3 years ago by Christian Boos

Owner: changed from mike_mp@… to Christian Boos
Status: reopenedassigned

Replying to christian at innoflair dot com:

Hi,

I fear the problem still exists (sort of) in Trac 1.0.1. In particular, the browser crashes when it tries to display a directory that exists only in one of two branches that have been just merged. […]

   [...]
 File "/usr/lib/python2.7/dist-packages/tracext/hg/backend.py", line 1003, in _find_dirctx
    dr = max(dr, max_ctx.filectx(f).linkrev())
   [...]
ManifestLookupError: lunch/lasagna/pasta@00b54988c818: not found in manifest

Good catch, and nice explanation, thanks!

comment:10 Changed 16 months ago by Johannes Weißl <jargon@…>

Cc: jargon@… added

We experience the same bug with Trac 1.0.2:

Trac detected an internal error:

ManifestLookupError: <file>@<rev>: not found in manifest

Python Traceback:

    File "/usr/lib/python2.7/dist-packages/trac/web/main.py", line 512, in _dispatch_request
    File "/usr/lib/python2.7/dist-packages/trac/web/main.py", line 221, in dispatch
    File "/usr/lib/python2.7/dist-packages/trac/versioncontrol/web_ui/browser.py", line 407, in process_request
    File "/usr/lib/python2.7/dist-packages/trac/versioncontrol/web_ui/browser.py", line 582, in _render_dir
    File "/usr/lib/python2.7/dist-packages/tracext/hg/backend.py", line 1121, in get_entries
    File "/usr/lib/python2.7/dist-packages/tracext/hg/backend.py", line 1047, in _find_dirctx
    File "/usr/lib/python2.7/dist-packages/mercurial/context.py", line 537, in filectx
    File "/usr/lib/python2.7/dist-packages/mercurial/context.py", line 246, in filenode
    File "/usr/lib/python2.7/dist-packages/mercurial/context.py", line 241, in _fileinfo 

Modify Ticket

Change Properties
Set your email in Preferences
Action
as assigned The owner will remain Christian Boos.
The ticket will be disowned.
as The resolution will be set.
to The owner will be changed from Christian Boos to the specified user.

Add Comment


E-mail address and name can be saved in the Preferences .

 
Note: See TracTickets for help on using tickets.