Edgewall Software
Modify

Opened 4 years ago

Last modified 17 months ago

#12156 new enhancement

Ticket query renderer extension point

Reported by: Peter Suter Owned by:
Priority: normal Milestone:
Component: query system Version:
Severity: normal Keywords:
Cc: Branch:
Release Notes:
API Changes:

Description

I propose adding a new extension point ITicketQueryRenderer.

class ITicketQueryRenderer(Interface):
    """Extension point interface for components that add HTML renderers of
    ticket query results."""

    def get_ticket_query_formats(self):
        """Return a list of `(format, format_label)` tuples."""

    def render_ticket_query(self, context, query, query_string, tickets, format):
        """Render the ticket query in the specified format."""

The query page could then allow selecting one of the available implementations from a dropdown and display the query results in various different ways.

The existing tables view could become the default implementation. The existing progress bars from the roadmap could provide a second implementation. New third-party plugins could provide further alternate views like pie charts, burndown charts, Kanban boards etc.

The roadmap could perhaps also reuse these renderers to display the milestones in the various ways.

The TicketQuery plugin could also use these renderers, selected by the existing format parameter (which already allows format=table and format=progress).

(Third-party visualization plugins already exist of course, but this extension point would allow them to be instantly deeply integrated with both the ticket query system, and the roadmap, make them usable in any wiki content via macro for free.)


Further considerations:

Possibly the batch editing could be separated out.

The existing ITicketGroupStatsProvider extension point could perhaps eventually be deprecated and removed. In its current form it seems to enable only very limited possibilities, potentially also provided by this new extension point.

Attachments (9)

ITicketQueryRenderer.patch (10.9 KB ) - added by Peter Suter 4 years ago.
ITicketQueryRenderer-SelectFormatOnQueryPage.patch (3.6 KB ) - added by Peter Suter 4 years ago.
ITicketQueryRenderer.2.patch (10.8 KB ) - added by Peter Suter 4 years ago.
ITicketQueryRenderer.3.patch (10.7 KB ) - added by Peter Suter 19 months ago.
ITicketQueryRenderer-SelectFormatOnQueryPage.3.patch (3.5 KB ) - added by Peter Suter 19 months ago.
ITicketQueryRenderer-Plotly.3.patch (3.4 KB ) - added by Peter Suter 19 months ago.
ITicketQueryRenderer.PlotlyPie.png (75.6 KB ) - added by Peter Suter 19 months ago.
ITicketQueryRenderer.PlotlyBar.png (76.9 KB ) - added by Peter Suter 19 months ago.
ITicketQueryRenderer-TimelineFormatNewTicketEvents.patch (6.1 KB ) - added by Peter Suter 17 months ago.

Download all attachments as: .zip

Change History (26)

by Peter Suter, 4 years ago

Attachment: ITicketQueryRenderer.patch added

comment:1 by Peter Suter, 4 years ago

Sorry, these quite old patches don't apply to trunk anymore, but I wanted to share them anyway just in case someone likes the idea and wants to finish it. I still really like the basic idea. If I remember correctly I was mainly dissatisfied with:

  • Having to pass both the query and the query_string parameter to the extension point. (Maybe this can be refactored first, but working with the Query class always seems quite difficult to me.)
  • Uncertainty about how some existing features would fit into this, e.g. the batch editing and maybe the milestone statistics progress bars.

comment:2 by lkraav <leho@…>, 4 years ago

This sounds good. I'm currently exploring JSON-based http://datatables.net (especially inline editing) for ticket management UI upgrades. Feels like the ticket is applicable here.

by Peter Suter, 4 years ago

comment:3 by Peter Suter, 4 years ago

Attached refreshed ITicketQueryRenderer.2.patch, applies to trunk@r14227. ITicketQueryRenderer-SelectFormatOnQueryPage.patch still applies on top of that.

comment:4 by Peter Suter, 4 years ago

Prepared a wiki:TracDev/Proposals/ITicketQueryRenderer page that could be copied to TracDev/PluginDevelopment/ExtensionPoints/trac.ticket.api.ITicketQueryRenderer as documentation.

comment:5 by Ryan J Ollos, 3 years ago

comment:4:ticket:9098 proposes rendering a timeline view for ticket query results. We should investigate a timeline format for the query view in this ticket.

Last edited 2 years ago by Ryan J Ollos (previous) (diff)

by Peter Suter, 19 months ago

by Peter Suter, 19 months ago

by Peter Suter, 19 months ago

by Peter Suter, 19 months ago

comment:6 by Peter Suter, 19 months ago

The refreshed patches ITicketQueryRenderer.3.patch and ITicketQueryRenderer-SelectFormatOnQueryPage.3.patch apply to trunk@r16600.

ITicketQueryRenderer-Plotly.3.patch is an additional proof-of-concept example using plotly.js — omitting the plotly-basic.min.js script itself to avoid attachment size limits — to draw pie charts, bar charts etc.

comment:7 by Ryan J Ollos, 17 months ago

There's one unittest failure with the patches. On investigating, the issue is associated with Query.template_data being called twice while rendering a table:

On the second call, when mutating the milestones field dict, the field['options'] has already been transformed to field['optgroups'] and field['options'] is empty, so the milestone query filter ends up empty: rjollos.git/trac/ticket/query.py@daa75c74:772-773,777-780#L759.

The following change fixes the failure:

  • trac/ticket/query.py

    diff --git a/trac/ticket/query.py b/trac/ticket/query.py
    index aba697389..b82fd2ca7 100644
    a b class Query(object):  
    769769                field['options'].insert(0, {'name': '$USER',
    770770                                            'value': '$USER'})
    771771            if name == 'milestone' and not field.get('custom'):
     772                field = field.copy()
    772773                milestones = [Milestone(self.env, opt)
    773774                              for opt in field['options']]
    774775                milestones = [m for m in milestones

However, it looks like we need to refactoring Query.template_data. Isn't the Query class more like a model? It seems like it shouldn't be involved in preparing data for the view, rather that should be the responsibility of QueryModule.

Last edited 17 months ago by Ryan J Ollos (previous) (diff)

comment:9 by Jun Omae, 17 months ago

I think get_ticket_query_formats() should have priority attribute, like IDatabaseConnector.get_supported_schemes(). Otherwise, it should be able to configure a component to render for each format.

in reply to:  9 ; comment:10 by Ryan J Ollos, 17 months ago

Replying to Jun Omae:

Otherwise, it should be able to configure a component to render for each format.

Do you mean, move the ITicketQueryRenderer implementation from QueryModule to it's own component so that it could be replaced by disabling and activating another component providing the table format with different behavior? I think that makes sense, and we should discuss the trade-off of using a priority attribute vs component implementing only ITicketQueryRenderer.

There are a few plugins on trac-hacks.org using Query.template_data, so we should probably retain it for backwards compatibility, and review the use-cases in the plugins.

$ grep -R "query.template_data" . --exclude-dir=".svn"
./trachoursplugin/branches/refactoring2017/trachours/hours.py:        ticket_data = query.template_data(context, tickets, orig_list,
./trachoursplugin/branches/0.11/trachours/hours.py:        ticket_data = query.template_data(context, tickets, orig_list, orig_time, req)
./trachoursplugin/trunk/trachours/hours.py:        ticket_data = query.template_data(context, tickets, orig_list,
./tracstorypointsplugin/0.11/tags/0.1.1/tracstorypoints/macros.py:        data = query.template_data(formatter.context, tickets)
./tracstorypointsplugin/0.11/trunk/tracstorypoints/macros.py:        data = query.template_data(formatter.context, tickets)
./exceldownloadplugin/0.12/tracexceldownload/ticket.py:        data = query.template_data(context, tickets)
./ticketcalendarplugin/0.12/ticketcalendar/web_ui.py:        return query.template_data(context, tickets, None,

I squashes some changes in a new branch: log:rjollos.git:t12156_query_renderer_extension_point.3, based on assumption that [a01c6837/rjollos.git] is acceptable, but let me know.

comment:11 by Peter Suter, 17 months ago

We should investigate a timeline format for the query view in this ticket.

I'm not 100% sure what you have in mind. We could try this:

  • Extract this main part of timeline.html into a new template timeline_events.html. Presumably the rest wouldn't be needed for a query timeline format, right?
  • Combine some code from TimelineModule and some from from TicketModule. I'm not sure if the code can be easily refactored to be reused, or if it would just have to be partially copied.
  • Use the query result to filter / select the tickets events. I'm not sure if we could get all the event data from the query result or if we still would have to use additional SQL queries to get ticket change event data.

ITicketQueryRenderer-TimelineFormatNewTicketEvents.patch implements a proof-of-concept of this, but only for the simple case of "new ticket" events. It seems simply adding the required columns to the query is enough for that?

Is this going in the intended direction or am I way off?

Last edited 17 months ago by Peter Suter (previous) (diff)

in reply to:  10 comment:12 by Peter Suter, 17 months ago

Replying to Ryan J Ollos:

Replying to Jun Omae:

Otherwise, it should be able to configure a component to render for each format.

Do you mean, move the ITicketQueryRenderer implementation from QueryModule to it's own component so that it could be replaced by disabling and activating another component providing the table format with different behavior? I think that makes sense, and we should discuss the trade-off of using a priority attribute vs component implementing only ITicketQueryRenderer.

Either way sounds OK for me.

log:rjollos.git:t12156_query_renderer_extension_point.3, based on assumption that [a01c6837/rjollos.git] is acceptable, but let me know.

Looks good to me, thanks.

in reply to:  11 ; comment:13 by Ryan J Ollos, 17 months ago

Replying to Peter Suter:

ITicketQueryRenderer-TimelineFormatNewTicketEvents.patch implements a proof-of-concept of this, but only for the simple case of "new ticket" events. It seems simply adding the required columns to the query is enough for that?

What is the lineage of that patch? I tried applying on my branch and on a branch containing your previous 3 patches, both failed to apply cleanly.

Is this going in the intended direction or am I way off?

Yeah, I think that's the intended direction. I was hoping it could be done with significant code reuse. Maybe it won't be clean or easy though.

in reply to:  13 comment:14 by Ryan J Ollos, 17 months ago

Replying to Ryan J Ollos:

What is the lineage of that patch? I tried applying on my branch and on a branch containing your previous 3 patches, both failed to apply cleanly.

Oh, nm. It applies cleanly on top of:

comment:15 by Ryan J Ollos, 17 months ago

I'm giving more thought to the timeline format and will post more on that soon.

Additional changes:

I added method render_ticket_query_options after seeing that many of the display options (columns, group results by, max items per page, …) were not relevant to the progress bar format. I'm requesting feedback on this proof-of-concept change.

comment:16 by Peter Suter, 17 months ago

I added method render_ticket_query_options after seeing that many of the display options (columns, group results by, max items per page, …) were not relevant to the progress bar format

Good idea. But "group results by" is actually relevant for the progress bar format? Maybe it can be added in a query_progressbar_format_options.html?

(For pie- and bar-charts group-by might even be required. For timeline it is irrelevant; but timeline prefs like date-range and event-kind-filters could be useful.)

How batch editing fits into this is also still unclear to me; it probably only works with table format.

in reply to:  16 comment:17 by Ryan J Ollos, 17 months ago

Replying to Peter Suter:

Latest changes in log:rjollos.git:t12156_query_renderer_extension_point.5.

Good idea. But "group results by" is actually relevant for the progress bar format? Maybe it can be added in a query_progressbar_format_options.html?

In [676fd012e/rjollos.git], I tried to address this by allowing the controls in ticket_query_options.html to be individually enabled. This allows us to reuse the template for the progressbar.

Other ways I considered:

  • Duplicating the code in query_progressbar_format_options.html
  • Having a template of macros that could be reused across query_table_format_options.html, query_progressbar_format_options.html and others.
  • Generating the formatting options in Python code.

We might need an interface method for handling the request arguments that are added by render_ticket_query_options. Then QueryModule.process_request would need go be refactored to pull out the arguments added by query_format_options.html. This might be related to #10983. I'm continuing to work on this part of the code.

(For pie- and bar-charts group-by might even be required. For timeline it is irrelevant; but timeline prefs like date-range and event-kind-filters could be useful.)

I'll try modifying some plugins to use the interface, like TracTicketStatsPlugin.

How batch editing fits into this is also still unclear to me; it probably only works with table format.

I see the page being roughly divided into section:

| Filters                                          |
| Result formatting options                        |
| Results                                          |
| Result modification controls (like batch modify) |

We could have an extension point for adding controls that act on the query results, one example being batch modify. The key question is whether there is likely to be another plugin that would use the interface (maybe something like GridModifyPlugin, but would need to investigate).

In the latest changes a template, data is returned by the interface methods ([ae7814d39/rjollos.git]). This is more consistent with other interfaces in Trac, but wouldn't allow generating HTML in Python code. Maybe we should support both a one-arg return value (rendered template) and two arg return value (template and data to be rendered).

Last edited 17 months ago by Ryan J Ollos (previous) (diff)

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The ticket will remain with no owner.
The ticket will be disowned. Next status will be 'new'.
as The resolution will be set. Next status will be 'closed'.
The owner will be changed from (none) to anonymous. Next status will be 'assigned'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.