Batch Modification Functionality

[anonymous]

The following is a request for enhancement.

I recommend that Trac implement batch modification functionality. This would allow users to modify several fields of several tickets that match a certain criteria in a single process.

Bugzilla 2.17.7 currently implements such functionality in the following way (see ​http://landfill.bugzilla.org/bugzilla-tip/):

• first, the user performs a query
• on the resulting bug list page, there is a link at the bottom of the page: "Change several bugs at once"
• on following that link, the same bug list reappears with a checkbox in front of each bug (this allows the user to hand pick a subset of the bugs to modify). This page also contains all the fields in a bug at the bottom of the page. Each of the fields contains a, "—do_not_change—" value by default.
• the user selects the bugs to batch modify
• the user modifies the fields needed to be changed in the fields at the bottom of the page
• the user clicks "commit"

Why would such functionality be useful? Here are some example scenarios:

• There's a milestone titled "Next Mainstream Loadbuild". Thus, when a designer closes an issue in the main stream / trunk, they don't need to know the number of the next loadbuild, they simply use the "Next Mainstream Loadbuild". Then, upon the next successful loadbuild, the loadbuild guy can do a batch modification, changing all issues that are closed and have a milestone of "Next Mainstream Loadbuild" to the actual build number. VERY USEFUL. The designers don't need to be intimate with every single possible build number / milestone.
• Another scenario: management has decided that all unresolved issues that were targeted to be addressed in load x are now to be targeted for load y. A batch modification would make this job a single task, as opposed to n tasks.

[daniel]

Thanks for the references, and presenting the idea. Seems cool.

[brianhv]

I think a more convenient UI would be to have a drop-down in every row for columns that the user wants to affect. For example, a priority-editing query would have drop-downs in the priority column for each row. That way, one ticket's priority could be lowered at the same time another's is raised.

[Christopher Lenz]

In my opinion, batch updates of tickets would be a good fit for the query module. How exactly the functionality is made available in the UI is up for discussion. I personally like the "edit each row via <select> or <input type=text>" better than the bugzilla approach.

[Christian Boos]

#2623 has been closed as a duplicate of this one.

[anonymous]

Let's get on this one guys. This one is a big deal.

[Christopher Lenz]

#3388 has been marked as duplicate of this ticket.

[Christian Boos]

Screenshot added on behalf of Steven Martins (illustrate how the Mantis user interface for this feature looks like)

[Christian Boos]

Screenshot added on behalf of Steven Martins (illustrate how the Mantis user interface for this feature looks like)

[Christian Boos]

#3426 was also marked as a duplicate for this one. That ticket suggested to do it the "Mantis" way, by having a checkbox next to each ticket in a list (custom query?), enabling you to manipulate at once all the selected tickets.

See attachment:Mantis01.png​ and attachment:Mantis02.png​.

[Russell Hind <rhind@…>]

IIRC, Bugzilla does it this way too. When you edit multiple, you get a check box by each ticket in the list so you can select the ones to edit, and the standard properties at the bottom so if you change a property such as component or milestone, it applies to all selected tickets. (Can't provide screen shot as don't have a bugzilla installation up and running any more since moving to trac)

[Markus Tacker <m@…>]

I've made a mockup of a possible batch edit screen in trac. Unfortunately I cannot add attachements at the moment as I get an UnicodeDecodeError so you can see it here: ​http://m.tacker.org/blog/wp-content/uploads/2006/07/batchedit.png

[pkou at ua.fm]

It looks like I have expected to see/implement it! Thank you!

The only things that are missing:

• Comment field where it is possible to enter a common comment for several tickets;
• Action fields where it is possible to see the ticket actions that are common for the selected tickets.

[track+dasspunk at gmail dot com]

I desperately need this and like the mockup but I would also suggest adding a text box that would append it's contents to the bug.

[adam.creeger]

This looks great. May I also suggest that the "Batch edit tickets" box is collapsed initially, with the option to open it if necessary?

[anonymous]

Here is something I am working on…

​http://trac-hacks.org/wiki/BatchModifyPlugin

-Ashwin

[anonymous]

I'd like to offer a slight variation on this theme. I wrote a Quick Change module for a task tracking intranet (attached). I first thought about going down the road described above, but the more I used the system, the more I decided that what I wanted was field-by-field changes that may vary row-to-row. Meaning: I didn't want to force users to make the same change for every item checked. I might want to change 10 items to a new version, but change their priorities one-by-one within that version. So, I just allowed the user to select a number of items to quick change and then the next page (or the same page if you love Javascript) gives comboboxes and text fields allowing you to make the changes in a spreadsheet-like style. Much more powerful, useful and user-friendly.

To illustrate, have a look at the attached screenshot. I selected three items to quick change. The first 3 columns are editable on an individual basis. In the trac case, almost every column should be editable.

​ftp://ftp.ict.usc.edu/dbw/temp/Capture_3.jpg

Thoughts?

[conny@…]

I believe that there are two slightly different and separate usecases:

• Multi-item, multi-property edit (itc.usc.edu)
  • User checkboxes several tickets from a query-results list.
  • User follows a link to a separate view where all/multiple properties can be quickly for all the selected tickets.
  • User clicks submits them off all at once.
• Few batch-updatable properties (Mantis)
  • A new section in the trac configuration decides what properties should be available at the bottom of query result pages.
  • User checkboxes several tickets from a query-results list.
  • User selects new values for the property/-ies directly in a form on the page, and clicks Batch Update.

[anonymous]

Well said. I think both are useful. My issue with #2 is that batch modify touches every item in the result set. That's fantastic if you only want to change ALL the items in the set. My experience, however, is that more often than not, I want to modify multiple items, but rarely ALL of them. In those cases, it's really hard to get a SELECT statement to pick out exactly the ones you want. For example, maybe I have 30 tickets left for milestone 1.0, but I know there is now only time for 15 or less. So, I want to move half the tickets to milestone 1.5, and I want to modify the priorities of the 15 tickets that are left to make sure they get done in the proper order. I really can't use batch modify to do this, because I can't easily (a) select only 15 and (b) modify the particulars of the other 15. Changing 30 of them by hand is super painful. So, my suggestion would be to combine (a) and (b) into one piece of functionality.

Have the top row be GLOBAL CHANGE and then have pull-downs, etc. for that row just like you do for the individual rows, and then you have the row-by-row capability as outlined above. So, if you select the GLOBAL CHANGE milestone to 2.0, it will change that column for every row (solves your (a) case above), but if you leave milestone in the GLOBAL CHANGE row as <unchanged>, then you can change individual lines. That is the combination of (a) and (b) above.

If you really want this to work well, you add the checkboxes on the left side (as described earlier above) with a (select all) checkbox on the global change line. Now you have amazingly fluid functionality. You can do a global change that only affects n items in the selection set (rather than setting the dropdown for n lines). So, you get the best of both worlds.

Does that make sense? If not, I can mock something up.

[anonymous]

#4727 marked as duplicate.

[chad@…]

​FogBugz has a stellar implementation. The rows of a query results page (QRP?) all have checkboxes, and there is a "select all" knob. The cool thing, though, is that the QRP implements the shift- and ctrl-click semantics familiar from spreadsheeting and other apps. I.e., click anywhere on a row to select that row. Then shift-click on another row, and it selects the row range from the first-clicked to the second-clicked.

In FB, you then click an "Edit" button (they also give you keyboard shortcuts for all this), and you get a ticket mod screen that lists all of the tickets in scope instead of a single ticket description.

[henko@…]

A (more advanced?) alternative here is what ThoughtWorks' Mingle does. There you have a grid view, where you can group stories (tickets) in "swimlanes" on any properly you like, for example milestone. You can then just drag a story from one such swimlane to another to change the value for that property. For example change from one milestone to another. It's insanely convenient. AJAX at its best.

Their ​release planning screencast does a very good job at describing it.

[sid]

#6257 was marked as duplicate of this ticket.

[Emmanuel Blot]

#6715 was marked as a duplicate of this ticket

[anonymous]

Lack of bulk-edit capabilities is the BANE of any project manager currently using trac. We don't need no stinkin' AJAX madness, the mock up shown here is perfect and follows the rest of the patterns in TRAC to a tee:

​http://m.tacker.org/blog/wp-content/uploads/2006/07/batchedit.png

This ticket is from 2004 and people clearly need it, who is going to step up?

[Markus Tacker <m@…>]

Replying to anonymous:

Lack of bulk-edit capabilities is the BANE of any project manager currently using trac. We don't need no stinkin' AJAX madness, the mock up shown here is perfect and follows the rest of the patterns in TRAC to a tee:

​http://m.tacker.org/blog/wp-content/uploads/2006/07/batchedit.png

This ticket is from 2004 and people clearly need it, who is going to step up?

Well, at least you could give this plugin a try (alos mentioned above): ​http://trac-hacks.org/wiki/BatchModifyPlugin I does the job very well.

nkantrowitz, please add the link to the description of this ticket.

[dbw]

The BatchModifyPlugin is close, but no cigar. I've used it quite a bit. It doesn't cut it. You have to perform a query that perfectly returns a set of results, and then you can batch modify that entire set. That's not always possible or efficient. As has been covered above, what we really need are two kinds of functionality: (1) an ability to update all items denoted with a check (see the mock-up above). That one tweak to BatchModifyPlugin would be great. but I asked repeatedly for that, and it appears that the developer fell off the face of the earth. (2) also, the ability to modify each individual line item's elements without having to open each up individually is even more important. you want to be able to change the priorities etc. of multiple items simultaneously, and they shouldn't all have to be set to the same value.

[anonymous]

​http://m.tacker.org/blog/wp-content/uploads/2006/07/batchedit.png This ticket is from 2004 and people clearly need it, who is going to step up?

The ticket's from 2004, the mockup's from 2006, it would be brilliant to just build it!

[anonymous]

bump.. New trac user, old bugzilla user. Would be great to have this. I'll give BatchModifyPlugin a try.

[anonymous]

Our company just started using Trac and this is the first thing that I would like Trac to fix. Only if my Python skill is good enough… I would have done it myself.

[agorilla@…]

I would also like to see this.

I haven't noticed any mention of a 'delete' option, but I'll add that to the request. It would be nice to be able to do a query, and select a group of tickets to delete - this could be handy in many situations: dead project, project relocated, spam, etc.

[anonymous]

I vote for this one too.

[kscharpf@…]

I just downloaded and installed ​http://trac-hacks.org/wiki/BatchModifyPlugin last week, and it works great. It allowed me to change the milestones for a whole bunch of tickets in one go, and it also has the ability to only apply the change to some of the items selected. It was a real time saver!

[ischenko@…]

Opened for 4 years…Trac's development is disappointing at times. Either reject it or implement it. pls pls.

[anonymous]

I had installed the batch modify plugin, but still i cant able to see any check box for the tickets in custom query. Do i need to update anything ?

[Sivaprakash]

can u pls tell me a solution for this

[sivaprakash@…]

need to fix this in a day

[anonymous]

Come on 4 years and still nothing!

[anonymous]

Replying to anonymous:

Come on 4 years and still nothing!

If you need it so badly you can do it. Take a look at the ticket-ninja branch if you're interested.

[Remy Blank]

Closed #7583 as a duplicate.

[anonymous]

A good suggestion and very useful for project leaders. If there is a hack for it why not implement the hack as a core module?

[anonymous]

+1 for this. Been using this feature for years in an in-house system. No longer work there and would love this. I can try the plug in, but would be nice if it was core.

Otherwise this system is has some pretty nice sweet spots.

[natethelen]

I would love to see this, too.

[Peter Hartlén]

I really need this too, isn't it possible to add the Batch-plugin to the standard build?

[Christian Boos]

The ​TracHacks:BatchModifyPlugin plugin seems reasonably small and clean.

Once the "pluggability" part of the code gets stripped out and the batch editing functionality gets integrated in the custom query module itself, the change should really be small for to the added value it would bring.

So I think it's indeed a good idea for the next major version. Patch welcomed ;-)

[CuriousCurmudgeon <meeker.brian@…>]

I started maintaining the ​TracHacks:BatchModifyPlugin a couple of months ago. The code is indeed small. A large chunk of it is related to plugging into the Custom Query page.

Some improvements are needed before integrating it into the next major version though. The big one is that it does not integrate with email notifications and the timeline. These are needed to make it seamless with the rest of Trac. If you know of other enhancements that would be needed before including it in Trac please add them ​here.

[Christian Boos]

Thanks Brian for showing interest in this!

Instead of discussing the integration of this feature on TracHacks, I'd prefer to see a TracDev/Proposals/BatchModification page here, discussing the approaches, and then eventually a sandbox branch when the patches start to look promising ;-)

The integration with e-mail notifications could be done the "normal" way via the ITicketChangeListener once #7758 is done (or the planned new IResourceChangeListener (#8834) and the Announcer support). That would be a first step.

But I wonder if a specific interface (or extra method to the IResourceChangeListener interface) wouldn't be more appropriate here, like it would be for other cases of batch modifications, for things like #4582 and #5658, or the batch WikiRename feature, see ticket:4412#comment:8.

That way, this could ideally result in one entry in the timeline for a batch change, and to one notification. Not sure if this can be done easily with the current data model, but the GenericTrac model already takes this into account (one changeid per change, with one set of metadata for the change, and then all resources modified during the batch change share that changeid).

[CuriousCurmudgeon <meeker.brian@…>]

Sorry for dropping off the planet after initially posting. I created a proposal page: TracDev/Proposals/BatchModification. I'm not very familiar with the Trac codebase outside of what I've needed for the current plugin, so it will take some time for me to get familiar with things.

[Christian Boos]

As I just noted in the above Wiki page, I think you have now a good starting point to try to integrate the functionality in Trac core.

[Brian Meeker <meeker.brian@…>]

This is looking good for 0.13 now. It is implemented in my ​branch on github.

[Christian Boos]

Replying to cboos:

… this could ideally result in one entry in the timeline for a batch change,

As this would also be useful for consolidating the ticket change events after a milestone delete + retarget (see e.g. timeline:2010-09-10), we could simply do this at the level of the ticket timeline provider, merging in one single "batch change" event all the change events having the same (date, author, message). In practice, now with usec timestamps, checking the date should be enough. The link for such an event would be a "ticket list" (i.e. #7639,​8699,​9018… the affected tickets).

[andres.riancho@…]

+1 on this! Its a must have feature!

[Remy Blank]

Anyone wants to take this up and review the branch mentioned in comment:61? Jun? Otherwise, we should move this to 0.14.

[Jun Omae]

Replying to rblank:

Anyone wants to take this up and review the branch mentioned in comment:61? Jun? Otherwise, we should move this to 0.14.

I pulled the branch to my repository, ​https://github.com/jun66j5/trac/compare/t525-batch-modify, and reviewed with TracDev/Proposals/BatchModification.

I think that we should move it to 0.14.

• When many tickets is modified by batch-modify;
  • It will send ticket notification mail each modified ticket. A summarizing mail is needed.
  • Many ticket-change events will be shown on timeline page. A summarizing event is needed.
• DB Api:
  • @with_transaction(env) is used. Change to with env.db_transaction as db:.
• Permission:
  • A user with TICKET_ADMIN cannot use batch-modify. If TICKET_BATCH_MODIFY is explicitly granted, a user can use it.
  • TICKET_ADMIN should be enough, not TICKET_BATCH_MODIFY.
• Others:
  • Should clean up the indents in trac/htdocs/js/query.js and trac/ticket/templates/batch_modify.html.

[Oleg Frantsuzov <franoleg@…>]

Even though I'll be really happy to see batch modification integrated into Trac, I must say it's too early for this code to be integrated. I believe the problem described in ​#TH7709 and in the following quote from TracDev/Proposals/BatchModification is not addressed yet.

The plugin as currently written allows for tickets to be put into an inconsistent state, such as with a resolution of 'closed', but with a status of 'accepted'. This needs to be addressed.

Seems to be addressed now (_check_for_resolution / _remove_resolution_if_not_closed)

We have custom workflow in our Trac which includes several states that have resolutions (committed, resolved and closed). We had to patch BatchModifyPlugin because it does not respect custom workflows and forcibly sets ticket state to closed whenever a resolution is set, and removes the resolution when the ticket state is set to something else than closed. This is obviously plain wrong with our workflow (and similar ones). While this is not often the case, I believe that there should be no discrepancy in how workflows are treated in Trac.

As I can see in the integration branch (​batch.py), the offending implementations of _check_for_resolution / _remove_resolution_if_not_closed are still there. This ought to be addressed in a more generic manner.

[psuter <petsuter@…>]

I updated this a bit:

• Merged trunk (specifically [10153])
• Cleaned up (all?) indents etc.
• Fixed TICKET_ADMIN
• Used env.db_transaction as db
• Used action controllers for status changes

You can see the changes on ​bitbucket (Sorry, I'm not too familiar with git and may have messed it up. I hope you all don't mind too much.)

[shiva <shiva@…>]

Hi,

Having championed for tract for some its most appealing and obvious benefits to our engg. team, it is of concern that I have to defend the missing batch update functionality to them.

I find it a bit surprising that in 7 years we have not felt the need to integrate just a basic functionality into main product. This is a functionality that found by default in almost any ticketing system that i have come across ( Clarify, Eventum etc. ). I sincerely request that the priority of this issue be raised.

I am going to try the batch update plugin in whatever state is in and definitely not going to resort to changing the DB even if i have to allocate resources solely to update the 1000+ tickets that require the change. In a organisation with auditing concerns, I am left with no other choice.

Thanks for all your work, It has been pleasurable experience to move to Trac from a host tools previously used to accomplish what this does for us.

Regards, Shiva

[psuter <petsuter@…>]

As far as I can tell the remaining open issue here is the missing support for batch notifications and batch timeline events.

I'm trying to summarize the options:

Batch ticket modify timeline events could either be detected in the ticket timeline provider somehow, or a new batch change listener interface method could be introduced on the planned IResourceChangeListener or the existing ITicketChangeListener.

For batch email notifications there are even more possibilities:

• Move the TicketNotifyEmail calls to a ITicketChangeListener (#7758), call these change listeners on batch modify, using the normal ticket_changed method and detect batch modifications in that listener somehow.
• Use a new batching ITicketChangeListener method instead.
• Use the planned IResourceChangeListener instead.
• Simply directly call TicketNotifyEmail from the batch modification code.
• Wait for the integration of the announcer plugin.

In any case the TicketNotifyEmail (or Announcer) needs:

• a new email template / formatter for such batch notifications
• logic to find the recipients / subscribers from all modified tickets

(Or actually, it seems to me creating a new BatchTicketNotifyEmail subclass of NotifyEmail would work better.)

---

Since IResourceChangeListener, Announcer and #7758 all seem to be stalled themselves, it might make sense to move forward using batch timeline event detection and a direct call of a new BatchTicketNotifyEmail here. This is admittedly not a very pretty or forward thinking solution.

Alternatively this could be integrated for now without batching up timeline events and without sending any batch notification mails at all. (Precedent for this behavior would be milestone delete and retarget tickets.)

Any opinions on this? Are there other open problems or solutions I missed? (And is anyone else still planning to work on this?)

[anonymous]

I would be grateful if this was implemented in the next release, I am not too concerned with the mass e-mail / timeline issue and that could follow in the future, I do however have projects that need to do large updates (for example assign lots of tickets to a milestone) which would be very useful. The precendent with deleting and retarget with the milestone completion is acceptable to me.

[anonymous]

Can I please vote for this feature also.

[Peter Suter]

Replying to psuter <petsuter@…>:

a direct call of a new BatchTicketNotifyEmail here. This is admittedly not a very pretty or forward thinking solution.

[504ac3db9fe7/psuter] is an experiment in this direction.

[Peter Suter]

And [7c593219b924/psuter] batches up the timeline events.

We now seem to have all the missing pieces.

[moak]

Vote to include this feature

[Peter Suter]

I now also updated the unit tests and documentation. (full diff)

The only thing I'm wondering now is about the configuration options:

    [batchmod]
    
    # Which text fields are treated as lists. This means that new values will
    # be merged into or removed from the field appropriately. For example,
    # given a keywords field with the values 'report', adding the value
    # 'component' will result in 'report,component'. You can also remove
    # values by prepending them with '-'. In the example above using '-report'
    # would have removed 'report' from the list.
    fields_as_list = keywords
    
    # The separator for values passed to the list. By default the separator is
    # a comma or any whitespace character.
    list_separator_regex = [,\s]+
    
    # The string used to connect list values after they are merged together.  
    list_connector_string = ,

It seems equivalent hard-coded values appear already quite a few times in other parts of Trac for the keywords and cc fields, e.g. for list_separator_regex:

trac.notification.NotifyEmail.addrsep_re	re.compile(r'[;\s,]+')
trac.ticket.web_ui.TicketModule._query_link_words()	re.split(r'([;,\s]+)'
trac.web.chrome.Chrome.cc_list()	re.split(r'[;,]'
trac.ticket.model._fixup_cc_list()	re.split(r'[;,\s]+'
…	…

For list_connector_string:

trac.ticket.notification.TicketNotifyEmail._notify()	', '.join
trac.ticket.web_ui.TicketModule._populate()	', '.join
trac.web.chrome.Chrome.format_emails()	sep=', '
trac.ticket.model._fixup_cc_list()	', '.join
…	…

Should we just hard-code the values here, too? Or does anyone know if these are especially important for batch modify for some reason?

[Remy Blank]

Replying to psuter:

Should we just hard-code the values here, too? Or does anyone know if these are especially important for batch modify for some reason?

I would tend to say yes. I would hardcode all of these parameters, and set both keywords and cc as list fields. If you allow configuring the fields to be treated as lists, you may want different separators for different fields, so you would have to configure them separately, etc.

It may make sense to allow adding or removing text from a field using the "+…" and "-…" syntax, even for non-list fields.

[Peter Suter]

Replying to rblank:

I would hardcode all of these parameters, and set both keywords and cc as list fields.

OK!

It may make sense to allow adding or removing text from a field using the "+…" and "-…" syntax, even for non-list fields.

OK, I now handle '+' and '-' in any field, but only keywords and cc fields get terms added (instead of replacing the entire field value) without such a prefix.

I'm not aware of any other remaining issues so, unless anything new comes up, my plan is to commit this later this weekend.

[Remy Blank]

Replying to psuter:

OK, I now handle '+' and '-' in any field, but only keywords and cc fields get terms added (instead of replacing the entire field value) without such a prefix.

Are you sure about that? If we have "+…" and "-…" for all fields, you could also allow replacing the content of list fields. The only difference with normal fields would be that you could write "-joe@spam.com,-bar@example.com" and have both removed from the field. Or even "-joe@spam.com,+bar@example.com" and have one entry removed and one entry added to each ticket.

[Peter Suter]

Replying to rblank:

Are you sure about that? If we have "+…" and "-…" for all fields, you could also allow replacing the content of list fields.

No, I'm not 100% sure what the difference between list fields and normal fields should be, so I'm open to suggestions.

The only difference with normal fields would be that you could write "-joe@spam.com,-bar@example.com" and have both removed from the field. Or even "-joe@spam.com,+bar@example.com" and have one entry removed and one entry added to each ticket.

This was already the idea and works the same for both list and normal fields at the moment:

Old value	Batch change	New value (normal field)	New value (list field)
	foo	foo	foo
foo	bar	bar	foo, bar
foo	+bar	foo, bar	foo, bar
foo, bar	-bar	foo	foo
foo, bar	+baz, +qux	foo, bar, baz, qux	foo, bar, baz, qux
foo	bar, +baz	bar, +baz	foo, bar, baz
foo	bar, -foo	bar, -foo	bar
foo	-foo, +bar	bar	bar
foo, bar, baz	-foo, -bar	baz	baz
11:00 +02:00	11:00 +03:00	11:00 +03:00	11:00, +02:00, 03:00
11:00 +02:00	+11:00 +03:00	11:00, +00:20, 03:00	11:00, +02:00, 03:00

(If the first character of the entire batch change string is is a '+' or '-', a normal field gets treated like a list field for this change. This is probably quite confusing, but I'm not sure what a better approach would be.)

If I understand you correctly, you propose the same for the "New value (list field)" column, but something else for the "New value (normal field)" column? I'm not sure how your table would look.

Actually, now I think non-list fields should not support special '+' and '-' behavior to avoid confusion.

[Remy Blank]

Let me try:

Old value	Batch change	New value (normal field)	New value (list field)
	foo	foo	foo
foo	bar	bar	bar
foo	+bar	foobar	foo, bar
foo, bar	-bar	foo,	foo
foo, bar	+baz, +qux	foo, barbaz, +qux	foo, bar, baz, qux
foo	bar, +baz	bar, +baz	bar, baz
foo	bar, -foo	bar, -foo	bar
foo	-foo, +bar	foo	bar
foo, bar, baz	-foo, -bar	foo, bar, baz	baz
11:00 +02:00	11:00 +03:00	11:00 +03:00	11:00, 03:00
11:00 +02:00	+11:00 +03:00	11:00 +02:0011:00 +03:00	11:00, +02:00, 03:00

The rules being:

• For normal fields, "+…" simple concatenates the current value and "…", and "-…" deletes all occurrences of "…" in the current value. The any "+" or "-" characters that are not at the beginning of the batch change are not considered as special, and neither is ",".
• For list fields, the batch change is split on the separator, then each item is checked:
  • If an item is "+…", then "…" is added to the current list.
  • If an item is "-…", then all list items equal to "…" are removed from the current list.
  • Otherwise the list is cleared, and the item becomes the only item in the list.

Looking at the table, I realize that there's not really a valid use case for "+…" and "-…" for non-list fields. So yeah, simply drop the rule for those (and simply replace the old value with the batch change). I think the rules for list fields still make sense, as they allow adding and removing individual items, as well as replacing the whole content.

[Peter Suter]

Replying to rblank:

Let me try:

Thanks :)

Old value	Batch change	New value (normal field)	New value (list field)
foo	bar, +baz	bar, +baz	bar, baz

• Otherwise the list is cleared, and the item becomes the only item in the list.

Do you define this with "operational semantics", where first bar clears the list, then baz is added?

So switching the operations (here: first baz is added, and then bar clears the list) can give a different result?

Old value	Batch change	New value (list field)
foo	+baz, bar	bar

Edit 1: Hm, but that would mean you can't replace a list field with two new items. Here first baz clears the list, then bar clears the list again:

Old value	Batch change	New value (list field)
foo	baz, bar	bar

Is it better to replace the whole field only once, before any "+…" and "-…" processing:

1. If any item is not a "+…" or "-…", clear the whole field.
2. For each item, if it's a "+…" or "…" item add it. If it's a "-…" item, remove all occurrences.

Edit 2: More useful and more intuitive would be to carry over the operator to the next item if it does not have an operator of its own.

Old value	Batch change	New value (list field)	Summary
foo	+bar, baz	foo, bar, baz	baz inherits +
foo, bar, baz	-bar, baz	foo	baz inherits -
foo, bar	-foo, bar, +baz, qux	baz, qux	bar inherits -, qux inherits +
foo	bar, +baz, qux	bar, baz, qux	bar replaces whole field, qux inherits + (Not really useful.)
foo, baz, qux	bar, -baz, qux	bar	bar replaces whole field, qux inherits - (Not really useful.)

Looking at the table, I realize that there's not really a valid use case for "+…" and "-…" for non-list fields. So yeah, simply drop the rule for those (and simply replace the old value with the batch change).

OK, so the new value (normal field) column is always identical with the batch change column.

I think the rules for list fields still make sense, as they allow adding and removing individual items, as well as replacing the whole content.

This is clear and simple, but maybe error prone in the sense that "+…" would probably be the most common operation, but one could easily forget the +. And replacing is an irreversible, destructive operation. How about making "…" equivalent to "+…" and require "=…" to replace the whole content?

Or maybe that just gives a false sense of safety, since it would only be "safe" with keywords and cc.

[Remy Blank]

Replying to psuter:

Do you define this with "operational semantics", where first bar clears the list, then baz is added?

Yes, that was the idea.

So switching the operations (here: first baz is added, and then bar clears the list) can give a different result?

Yes, indeed. That may be counter-intuitive.

Edit 1: Hm, but that would mean you can't replace a list field with two new items. Here first baz clears the list, then bar clears the list again:

To replace the list field with two new items, you would write "baz, +bar".

Is it better to replace the whole field only once, before any "+…" and "-…" processing:

1. If any item is not a "+…" or "-…", clear the whole field.
2. For each item, if it's a "+…" or "…" item add it. If it's a "-…" item, remove all occurrences.

Edit 2: More useful and more intuitive would be to carry over the operator to the next item if it does not have an operator of its own.

Yes, I thought about that, too. It might work, but is more difficult to explain.

And replacing is an irreversible, destructive operation.

That's the critical issue here. If there's no preview operation, the rules must be as simple as possible, otherwise people will shoot themselves in the foot.

How about making "…" equivalent to "+…" and require "=…" to replace the whole content?

That's a nice idea. I think I like that one best.

[Peter Suter]

A preview would definitely be best. And list fields should maybe have a hint mentioning the '+' and '-'. And there should probably be a Note: See TracBatchModify for help on using batch modify.

(OT: Did svn.edgewall.org's host fingerprint change today?)

[Peter Suter]

Proposed UI with operation dropdown

[Peter Suter]

Maybe instead of trying to find the best textual command syntax and adding hints and worrying about how to explain it, it would make sense to take a somewhat different approach. The screenshot above has a drop-down control, similar to the query filters.

old approach	new approach
+foo	[add |▾] [foo ]
-foo	[remove |▾] [foo ]
=foo	[assign |▾] [foo ]
+foo, -bar	[add/remove|▾] [foo ] [bar ]

[Remy Blank]

Nice! If you have "add/remove", then "add" and "remove" aren't really needed anymore, so you could just keep "assign" and "add/remove". I would maybe add explicit "add:" and "remove:" labels before the text boxes in the second case. Or set the value of the drop-down to "add" and show a label "and remove", which makes a nice sentence (but translators are going to hate me for this).

[Peter Suter]

Replying to rblank:

Nice! If you have "add/remove", then "add" and "remove" aren't really needed anymore, so you could just keep "assign" and "add/remove".

Right, I also wondered about the best number of options and the wording, but no perfect solution presented itself.

I left the basic "add" and "remove" cases because they are so clear and simple…

I would maybe add explicit "add:" and "remove:" labels before the text boxes in the second case. Or set the value of the drop-down to "add" and show a label "and remove", which makes a nice sentence (but translators are going to hate me for this).

… while this one is a bit tricky. If the drop-down only offers "add" and "assign" people might miss that "remove" is even a possibility.

I see the value in making the whole construct sentence-like, but it comes with some downsides. ("Keywords add foo and remove _" is still not really a sentence. "remove" is hidden in drop-down. Translators are going to hate you. :)

Adding redundant "add:" and "remove:" labels is redundant, but maybe still the best solution.

[Remy Blank]

I don't have strong feelings for either solution, so just do what you think is best.

[Peter Suter]

I went with the labels.

Applied in [11015].

[framay <franz.mayer@…>]

Replying to psuter:

I went with the labels.

Applied in [11015].

Great work - just quickly tested on our Trac installation. Works fine.

I noticed just a few things:

• when a query is grouped, a checkbox next to each grouped table is printed; when clicking on it, all tickets are selected - I would expect that only ticket of that group would be selected
  • some layout-glitch: the colspan of the group header must be one more (since the first checkbox column is apparently not counted)
• what happens with custom text-area fields? Field "description" is missing as well. I haven't seen any modification when selecting them; so if there is no functionality they should be removed from list; but it would be better with following actions:
  • "append": appends text to field, adding text at the end of field
  • "prepend" (maybe): prepends text to field, means inserting text to the beginning field
  • "remove": removes all text from field
  • "assign": assign text to field
• I like the drop-down box for "add", "remove", "assign", etc.; but at first I didn't know what's the difference between "add" and "assign" - maybe a descriptive tooltip or text next to the input field would help improve user experience

[Christian Boos]

Replying to framay <franz.mayer@…>:

Replying to psuter:

I went with the labels.

Applied in [11015].

Great work - just quickly tested on our Trac installation. Works fine.

Seconded, I did even less testing yet, but great work anyway ;-)

• "assign": assign text to field

• I like the drop-down box for "add", "remove", "assign", etc.; but at first I didn't know what's the difference between "add" and "assign" - maybe a descriptive tooltip or text next to the input field would help improve user experience

I'd suggest s/assign/set to/. set to is also what we use for reporting the initial change to a field value. Furthemore, it seems the ​assign verb has various other meanings than the one we are familiar with in programming language (assignment of a value to a variable).

[Peter Suter]

Thanks for testing!

Replying to framay <franz.mayer@…>:

I noticed just a few things:

• when a query is grouped, a checkbox next to each grouped table is printed; when clicking on it, all tickets are selected - I would expect that only ticket of that group would be selected

Agree, that would make more sense.

• some layout-glitch: the colspan of the group header must be one more (since the first checkbox column is apparently not counted)

Strange.

• what happens with custom text-area fields? Field "description" is missing as well. I haven't seen any modification when selecting them; so if there is no functionality they should be removed from list; but it would be better with following actions:
  • "append": appends text to field, adding text at the end of field
  • "prepend" (maybe): prepends text to field, means inserting text to the beginning field
  • "remove": removes all text from field
  • "assign": assign text to field

I didn't think it would be useful to batch edit such fields. Maybe create a separate enhancement request?

For now I think they should be removed from the list, same as "description". (I thought they already were, but apparently not.)

Replying to cboos:

I'd suggest s/assign/set to/. set to is also what we use for reporting the initial change to a field value. Furthemore, it seems the ​assign verb has various other meanings than the one we are familiar with in programming language (assignment of a value to a variable).

Sounds good.

[Peter Suter]

I think that patch fixes the grouping glitches. (I'm not a jQuery expert though, so maybe there's a better way?) It also filters out custom text-area fields, and renames assign to set to. (And I noticed some obsolete code handling owner changes, that was superseded by the action controllers.)

Anything I missed?

BTW, after comment:5:ticket:10643 I wonder: Was adding the new TracBatchModify page (and updating the TracGuide page) in [11015] a mistake? Should these go to wiki:0.13 too / instead?

[Remy Blank]

Replying to psuter:

BTW, after comment:5:ticket:10643 I wonder: Was adding the new TracBatchModify page (and updating the TracGuide page) in [11015] a mistake? Should these go to wiki:0.13 too / instead?

Yes, they should. You should also update trunk/contrib/checkwiki.py​ and add the new page to the wiki_pages list.

[Peter Suter]

Replying to rblank:

You should also update trunk/contrib/checkwiki.py​ and add the new page to the wiki_pages list.

Thanks for the pointer. Done in [11019].

And fixed the glitches etc. in [11020].

[Mikael Relbe]

I think there is a couple of minor glitches:

1. In the timeline, a batch edit shows the ticket ids in one row, in database time order (I guess). But I think they should be sorted on id.
2. I am using a modified workflow. In the report view, the Batch Modify section shows blank action options.
3. In the same view, I think "Add Field" should be above "Comment".

(Comment: This is a great feature that will be welcomed by many Trac users. Thanks!)

[Peter Suter]

Replying to mrelbe:

1. In the timeline, a batch edit shows the ticket ids in one row, in database time order (I guess). But I think they should be sorted on id.

Fixed in [11024].

3. In the same view, I think "Add Field" should be above "Comment".

Sounds reasonable.

I could also see "Comment" at the top; that aligns a bit nicer with the fields.

2. I am using a modified workflow. In the report view, the Batch Modify section shows blank action options.

Oops, I see. There's a problem in BatchModifyModule._get_action_controls. The action section shows all actions available in any ticket from the query result. So far these were all rendered in the context of the last ticket. In [11023] I change the actions to be rendered in the context of the first ticket where the action is available. (This is still not perfect, e.g. the hint might mention details only true for that one ticket, like its old ticket owner.)

[Peter Suter]

Fixed another small glitch in the ordering of the fields in [11025] (and moved them below "Comment").

Further issues should maybe be raised in new tickets.

[Mikael Relbe]

(Why not continue in this ticket?)

Almost perfect!

But, the button "Change tickets" is not translatable…

Now, it would be perfect if:

• The visual impression of the Action section is the same as for tickets (line_height 2em, for instance, and as fieldset with perhaps a border).
• Available actions are the common set of allowed actions for the selected tickets.

[Ethan Jucovy <ethan.jucovy@…>]

I'm seeing seven functional test failures that seem related to these changes. They're in various places but all the failures have approximately the same stack trace and captured output; here's a representative sample:

======================================================================
FAIL: Test for regression of http://trac.edgewall.org/ticket/7821 var
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/tree/trac/trac/ticket/tests/functional.py", line 1402, in runTest
    self._tester.go_to_query()
  File "/tmp/tree/trac/trac/tests/functional/tester.py", line 128, in go_to_query
    tc.follow('Custom Query')
  File "/tmp/tree/lib/python2.6/site-packages/twill/commands.py", line 199, in follow
    browser.follow_link(link)
  File "/tmp/tree/lib/python2.6/site-packages/twill/browser.py", line 207, in follow_link
    self._journey('follow_link', link)
  File "/tmp/tree/lib/python2.6/site-packages/twill/browser.py", line 549, in _journey
    callable(func_name, *args, **kwargs)
  File "/tmp/tree/trac/trac/tests/functional/better_twill.py", line 112, in _validate_xhtml
    _format_error_log(page, e.error_log))
TwillAssertionError: 
# No declaration for attribute value of element textarea
# URL: http://127.0.0.1:8097/query
# Line 334, column 50

    <tr id="batchmod_comment">
      <th colspan="2">
        <label for="batchmod_value_comment">Comment:</label>
      </th>
      <td class="fullrow"><textarea id="batchmod_value_comment" name="batchmod_value_comment" value="" cols="70" rows="5"></textarea>
      </td>
    </tr>
    <tr id="add_batchmod_field_row">
      <td colspan="3">
        <label class="batchmod_label" for="add_batchmod_field">Add Field:</label>
        <select id="add_batchmod_field">


# Element legend is not declared in td list of possible children
# URL: http://127.0.0.1:8097/query
# Line 349, column 9

    </tr>
    <tr>
      <td colspan="3">
        <legend>Action</legend>
      </td>
    </tr>
  </table>
  <div>
    <input type="hidden" name="selected_tickets" value="" />
    <input type="hidden" name="query_href" value="/query?status=!closed&amp;cc=~admin&amp;owner=admin&amp;order=priority" />
    <input type="submit" id="batchmod_submit" name="batchmod_submit" value="Change tickets" />

[Ethan Jucovy <ethan.jucovy@…>]

The test failures look like they're the result of invalid HTML in the new batch modify template. I've attached a patch that causes the tests to pass with the following changes:

• <textarea> should not have a value="" attribute — textareas don't have value.
• <legend> must be the child of a <fieldset> element. (It's currently the child of a <td> so I wrapped the "actions" section and legend in a nested fieldset, which seems semantically correct anyway, and provides a UI more consistent with the familiar "single ticket modify" screen.

​http://trac.edgewall.org/attachment/ticket/525/html-validation-fix-twill-failures.diff

[Peter Suter]

Replying to Ethan Jucovy <ethan.jucovy@…>:

I've attached a patch that causes the tests to pass

Thanks! Applied in [11027], including also…

Replying to mrelbe:

But, the button "Change tickets" is not translatable…

• The visual impression of the Action section is the same as for tickets (line_height 2em, for instance, and as fieldset with perhaps a border).

…some slight tweaks that should hopefully address these points.

(Why not continue in this ticket?)

In my experience long tickets get unwieldy. Who wants to read over 100 comments to see if anything has been missed?

[Thijs Triemstra]

Also, <legend>Action</legend> was added but cannot be translated.

[Christian Boos]

Replying to thijstriemstra:

Also, <legend>Action</legend> was added but cannot be translated.

Why? It will extracted from the template by the Genshi extraction filter, like for the other <legend> elements.

[Mikael Relbe]

r11027 looks really nice!

But it is somewhat confusing that one can select an action that does not cause a change to a ticket, due to workflow definition…

[Thijs Triemstra]

Replying to cboos:

Replying to thijstriemstra:

Also, <legend>Action</legend> was added but cannot be translated.

Why? It will extracted from the template by the Genshi extraction filter, like for the other <legend> elements.

I thought it wouldn't be extracted because it isn't wrapped like this: ${_('Change tickets')}. But apparently it will get extracted without, so sorry for the noise..

[Christian Boos]

Slightly [OT]:

Replying to psuter:

(Why not continue in this ticket?)

In my experience long tickets get unwieldy. Who wants to read over 100 comments to see if anything has been missed?

We had the same experience with other long running tickets (#6614 or #7723) where intermediate tasks were introduced along the way. It's not always easy to know beforehand if such a subtopic will evolve into a full-fledged issue of its own, or if it's just a quick reminder for a little thing to do. So ideally, we should be able to identify such a "todo" item, mark it as done in place if it's a small enough task, but eventually be able to create a ticket (subticket?) from it… A refined implementation of these ideas is probably for Trac 2.0, but this could already be approximated today by using the ​TH:WikiExtrasPlugin's WikiPhrases, say with a FIXME or TODO later edited into FIXED / DONE.

[bobbysmith007@…]

Just for reference, there was a compatibility patch to the ​th:BatchModifyPlugin that made it work with ​th:TimingAndEstimationPlugin. The bug occurs because T&E adds a tfoot section with hour totals to the query screen and the checkbox gets added to the tfoot td elements (where it doesnt belong).

I will try to adjust ​th:TimingAndEstimationPlugin, but for posterity and cross linking purposes I figured I would go ahead an put this comment on this ticket. There is a patch on that trac-hacks ticket that fixes this bug (in the old plugin).

​http://trac-hacks.org/ticket/9855

[Ryan J Ollos <ryan.j.ollos@…>]

Was there any consideration given to positioning the batch modification section about the ticket report table, rather than below?

[lkraav <leho@…>]

are there significant barriers to having batch modify work with sql reports? looks like query module wont be able to do some things sql can.