Opened 4 weeks ago
Closed 3 weeks ago
#13804 closed defect (fixed)
convert_db fails if both spamfilter and tags plugin are enabled
Reported by: | Owned by: | Dirk Stöcker | |
---|---|---|---|
Priority: | normal | Milestone: | plugin - spam-filter |
Component: | plugin/spamfilter | Version: | 1.6 |
Severity: | normal | Keywords: | |
Cc: | btbn@… | Branch: | |
Release Notes: | |||
API Changes: | |||
Internal Changes: |
Description
This is a rather specific case, but I think I have it tracked down to some degree. What is happening is this:
During the migration process, trac calls the "upgrade" admin command on the new environment, after initially creating it empty, so that plugins can create their databases.
At some point during this process, this function ends up called: https://trac.edgewall.org/browser/plugins/trunk/spam-filter/tracspamfilter/filters/trapfield.py#L116
However, it fails with:
17:36:10 Trac[env] ERROR: Component <Component tracspamfilter.filters.trapfield.TrapFieldFilterStrategy> failed with Traceback (most recent call last): File "/usr/local/lib/python3.13/site-packages/trac/env.py", line 382, in component_guard yield File "/usr/local/lib/python3.13/site-packages/trac/env.py", line 750, in needs_upgrade if participant.environment_needs_upgrade(): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/usr/local/lib/python3.13/site-packages/tracspamfilter/filters/trapfield.py", line 119, in environment_needs_upgrade chrome.jenv.add_extension(self._trap_field_ext) ^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'add_extension' Error: Unable to check for upgrade of tracspamfilter.filters.trapfield.TrapFieldFilterStrategy: AttributeError: 'NoneType' object has no attribute 'add_extension'
Upon further digging, why jenv is None, I removed the exception handlers around it, and got the following exception:
17:43:29 Trac[env] ERROR: Component <Component tracspamfilter.filters.trapfield.TrapFieldFilterStrategy> failed with Traceback (most recent call last): File "/usr/local/lib/python3.13/site-packages/trac/env.py", line 382, in component_guard yield File "/usr/local/lib/python3.13/site-packages/trac/env.py", line 750, in needs_upgrade if participant.environment_needs_upgrade(): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/usr/local/lib/python3.13/site-packages/tracspamfilter/filters/trapfield.py", line 116, in environment_needs_upgrade chrome.load_template(os.devnull) # to set chrome.jenv property ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^ File "/usr/local/lib/python3.13/site-packages/trac/web/chrome.py", line 1316, in load_template jinja2_dirs = self.get_all_templates_dirs() File "/usr/local/lib/python3.13/site-packages/trac/web/chrome.py", line 772, in get_all_templates_dirs for provider in self.template_providers: ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.13/site-packages/trac/core.py", line 96, in extensions components = [component.compmgr[cls] for cls in classes] ~~~~~~~~~~~~~~~~~^^^^^ File "/usr/local/lib/python3.13/site-packages/trac/core.py", line 238, in __getitem__ component = cls(self) File "/usr/local/lib/python3.13/site-packages/trac/core.py", line 158, in __call__ self.__init__() ~~~~~~~~~~~~~^^ File "/usr/local/lib/python3.13/site-packages/tractags/wiki.py", line 101, in __init__ self.tag_system = TagSystem(self.env) ~~~~~~~~~^^^^^^^^^^ File "/usr/local/lib/python3.13/site-packages/trac/core.py", line 158, in __call__ self.__init__() ~~~~~~~~~~~~~^^ File "/usr/local/lib/python3.13/site-packages/tractags/api.py", line 249, in __init__ self._populate_provider_map() ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/usr/local/lib/python3.13/site-packages/tractags/api.py", line 465, in _populate_provider_map for provider in self.tag_providers) ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.13/site-packages/trac/core.py", line 96, in extensions components = [component.compmgr[cls] for cls in classes] ~~~~~~~~~~~~~~~~~^^^^^ File "/usr/local/lib/python3.13/site-packages/trac/core.py", line 238, in __getitem__ component = cls(self) File "/usr/local/lib/python3.13/site-packages/trac/core.py", line 158, in __call__ self.__init__() ~~~~~~~~~~~~~^^ File "/usr/local/lib/python3.13/site-packages/tractags/ticket.py", line 53, in __init__ self._fetch_tkt_tags() ~~~~~~~~~~~~~~~~~~~~^^ File "/usr/local/lib/python3.13/site-packages/tractags/ticket.py", line 215, in _fetch_tkt_tags rw_cursor.execute(""" ~~~~~~~~~~~~~~~~~^^^^ DELETE FROM tags ^^^^^^^^^^^^^^^^ ...<3 lines>... """ % (db.cast('tags.name', 'int'), ignore), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (self.realm,)) ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.13/site-packages/trac/db/util.py", line 73, in execute return self.cursor.execute(sql_escape_percent(sql), args) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ psycopg2.errors.UndefinedTable: relation "tags" does not exist LINE 2: DELETE FROM tags ^ UndefinedTable: relation "tags" does not exist LINE 2: DELETE FROM tags ^ Error: Unable to check for upgrade of tracspamfilter.filters.trapfield.TrapFieldFilterStrategy: UndefinedTable: relation "tags" does not exist LINE 2: DELETE FROM tags ^
So basically, upgrading/initializing the spamfilter plugin requires the tags plugin to be already upgraded/initialized, but since this is happening during migration/initialization, it is not.
Not sure how to properly fix this. I guess one way would be to ensure the tags plugin is initialized first on its own, before any other plugins are? Or maybe even generally initialize the plugins one by one during migration, and call upgrade after each one?
Attachments (0)
Change History (7)
comment:1 by , 4 weeks ago
Cc: | added |
---|
follow-up: 4 comment:2 by , 4 weeks ago
The root cause is that TicketTagProvider
in th:TagsPlugin fetches tags
records before creating tags
table.
comment:3 by , 4 weeks ago
Component: | admin/console → plugin/spamfilter |
---|---|
Milestone: | → plugin - spam-filter |
Owner: | set to |
I think we could use IRequestFilter.post_process_request
instead of IEnvironmentSetupParticipant.environment_needs_upgrade()
, in order to avoid the exception being raised from component initialization in other plugins.
-
tracspamfilter/filters/trapfield.py
14 14 import os 15 15 16 16 import jinja2.ext 17 import jinja2.exceptions 17 18 18 from trac.api import IEnvironmentSetupParticipant19 19 from trac.config import IntOption, Option 20 20 from trac.core import Component, implements 21 21 from trac.util import lazy 22 22 from trac.util.text import javascript_quote, shorten_line 23 from trac.web.api import IRequestFilter 23 24 from trac.web.chrome import Chrome 24 25 25 26 from tracspamfilter.api import IFilterStrategy, N_ … … 28 29 class TrapFieldFilterStrategy(Component): 29 30 """Spam filter using a hidden trap field. 30 31 """ 31 implements(IFilterStrategy, I EnvironmentSetupParticipant)32 implements(IFilterStrategy, IRequestFilter) 32 33 33 34 karma_points = IntOption('spam-filter', 'trap_karma', '10', 34 35 """By how many points a trap reject impacts the overall karma of … … 104 105 '<input type="text" name="%s" value="" />' % (self.name) + \ 105 106 '<input type="hidden" name="%s" value="" />' % (self.name_hidden) + \ 106 107 '</div>' 107 # IEnvironmentSetupParticipant interface108 108 109 def environment_created(self): 110 pass 109 # IRequestFilter interface 111 110 112 def environment_needs_upgrade(self): 113 if self.karma_points > 0: 114 chrome = Chrome(self.env) 115 try: 116 chrome.load_template(os.devnull) # to set chrome.jenv property 117 except: 118 pass 119 chrome.jenv.add_extension(self._trap_field_ext) 120 return False 111 def pre_process_request(self, req, handler): 112 return handler 121 113 122 def upgrade_environment(self): 123 pass 114 def post_process_request(self, req, template, data, metadata): 115 if template in self.anchors: 116 # Add trapfield extension before rendering the template 117 self._setup_trap_field_ext() 118 return template, data, metadata 124 119 125 120 # Internal methods 126 121 122 def _setup_trap_field_ext(self): 123 chrome = Chrome(self.env) 124 if chrome.jenv is None: 125 try: 126 # to initialize chrome.jenv attribute 127 chrome.load_template(os.devnull) 128 except jinja2.exceptions.TemplateNotFound: 129 pass 130 except: 131 self.log.warning('Exception caught while letting Chrome.jenv ' 132 'attribute initialize', exc_info=True) 133 return 134 chrome.jenv.add_extension(self._trap_field_ext) 135 127 136 @lazy 128 137 def _trap_field_ext(self): 129 138
follow-up: 6 comment:4 by , 4 weeks ago
Replying to Jun Omae:
The root cause is that
TicketTagProvider
in th:TagsPlugin fetchestags
records before creatingtags
table.
I've filed it at th:#14360.
comment:5 by , 4 weeks ago
At least, it might be good to log the exception rather than TemplateNotFound
and check the jenv
attribute being not None
.
-
tracspamfilter/filters/trapfield.py
14 14 import os 15 15 16 16 import jinja2.ext 17 import jinja2.exceptions 17 18 18 19 from trac.api import IEnvironmentSetupParticipant 19 20 from trac.config import IntOption, Option … … 114 115 chrome = Chrome(self.env) 115 116 try: 116 117 chrome.load_template(os.devnull) # to set chrome.jenv property 118 except jinja2.exceptions.TemplateNotFound: 119 pass 117 120 except: 118 pass 119 chrome.jenv.add_extension(self._trap_field_ext) 121 self.log.warning('Exception caught while letting Chrome.jenv ' 122 'attribute initialize', exc_info=True) 123 if chrome.jenv is not None: 124 chrome.jenv.add_extension(self._trap_field_ext) 120 125 return False 121 126 122 127 def upgrade_environment(self):
comment:6 by , 4 weeks ago
Replying to Jun Omae:
Replying to Jun Omae:
The root cause is that
TicketTagProvider
in th:TagsPlugin fetchestags
records before creatingtags
table.I've filed it at th:#14360.
I just committed the workaround of th:TagsPlugin in th:r18665.
Found a workaround to successfully migrate: Simply added a "catch Exception: pass" clause to this try-Block: https://trac-hacks.org/browser/tagsplugin/trunk/tractags/ticket.py#L52
With that in place, the migration completes successfully, and the hack can be removed again.