Percentage complete

[cerel]

Im using Trac in my projet to track bugs, but also to maintaint a list of tasks to do. One thing that would be great is the possibility to have a "percentage complete" field. This way the task can be updated to reflect the percentage complete. Right now the task can only be "0% complete" or "100% complete". I mean a task is either finished or unfinished, which is not always the case.

[vittorio]

you could add a custom property field (see TracTicketsCustomFields) to set percentage complete.

[ph@…]

Custom fields are the way to go on this. This is basically time-tracking / 100.

See #1005, #710, #560 and #440.

[Christian Boos]

See how this feature looks like in ​flyspray (first 2 screenshots: view task and edit task).

… btw, in general flyspray looks pretty close to what will be in Trac 0.9 and beyond (ticket types, multiple projects, ticket dependencies, comment edit/delete…)

[Christian Boos]

Well, custom fields currently can't provide you with a nice progress bar overview, like here: ​http://bugs.flyspray.org/

I think that'd be a nice feature for 2.0 ;-)

[Christian Boos]

#4916 was closed as duplicate.

[Remy Blank]

#8476 has been closed as a duplicate.

[Christian Boos]

Milestone 2.0 deleted

[Christian Boos]

A favorite of mine, it's a MUST have … one day.

[Christian Boos]

There are interesting variations that can be grafted on that concept.

For example, we could look at the evolution of the % complete value, and the bar could be colored to give a hint about the velocity of the work on that ticket:

• % of work regularly increasing
• % of work stationary or oscillating
• % of work regularly decreasing

If we couple the % of completion with a recording of the time spent, we could additionally show an estimated remaining time, and according to the velocity, an estimated completion date (related to #710).

The above should be implemented as a plugin, but there are most certainly some FieldRefactoring changes that have to be done in Trac core first.

[Remy Blank]

Note that many coders tend to always ​estimate their ​tasks to be ​90% ​complete (bottom of the page)… A more objective measure of completeness would probably be more useful.

Or as the 90:90 rule says: the first 90% of a project takes 90% of the work, and the remaining 10% takes the other 90%.

[Christian Boos]

Sure, people (me included) tend to be overly optimistic about the percent complete… that's why I think it's probably more interesting to focus on the variations of that percentage.

If it steadily increases, chances are that things progress as expected (green). If at any point (including at the dreaded "90% complete" level!) the progress stops, this is likely indicative that the estimate was too optimistic (colors turns to grey). And that is informative, I think. You can identify those stalled tickets, and reassess the % to a lower value, matching the new perceived amount of remaining work (dragging the color to red).

This is a "self-correcting" approach, as after some time the developer will realize that to keep a progress bar "green", they need to make conservative estimates, and tickets following the 90:90 rule can be quickly spotted as such ;-)

Some more refinement of the idea is needed, for example can the above information be inferred simply from the evolutions of the % complete value over time, or does this need to be coupled with a "time spent" field? In the latter, it's easy to detect a staled progress (hours keep ticking, % complete is not modified). In the former, maybe heuristics based on time elapsed since creation and since previous changes can also work…

[Remy Blank]

Note that I forgot a smiley in comment:11. I have nothing against percentages, I just won't use them myself.

Replying to cboos:

This is a "self-correcting" approach, as after some time the developer will realize that to keep a progress bar "green", they need to make conservative estimates, and tickets following the 90:90 rule can be quickly spotted as such ;-)

This contradicts most of the research in project estimation, as well as my own observations. The software sector has been found to be notoriously bad at self-correcting (or learning from its own mistakes), and developers don't correct their estimates based on past experience, they just repeat the same mistakes over and over again. That's why I believe ​evidence-based scheduling has a bright future, as it integrates past experience automatically.

[osimons]

I'm a big fan of keeping things simple - and certainly when it comes to time-tracking a whole new can of worms opens up… Most organisations and developers I know need to register their time for billing or other internal book-keeping, and we can be quite certain there will also be some other system than Trac in the picture.

So, what remains is the planning and follow-up aspect of scheduled work and work in progress. Personally I'm not a fan of "percentage complete", both due to the argument Remy raises, but also to the sorry fact that you cannot compare two tasks at 80% and infer anything about what effort is needed to complete them. One can still need 14 days of work, the other 1 hour.

I'm a big fan of the concept used in ​th:EstimationToolsPlugin. One field to rule them all called "Effort Remaining" where effort can be hours or days or whatever used by the project for counting. Then the change in this number is tracked over time, allowing both to see current status and its progress over time. No major interest in the actual time spent, all that is of interest is what remains before something is done. And, tickets can be summed across other dimensions like milestone to indicate how much effort remains in total. Or how much effort is required by the various people that have tickets assigned to them.

Everything else related to the book-keeping of hours, billing or tracking of otherwise tracking and reporting hours is way outside the scope of Trac core (ie. for plugins to do).

I'd like for Trac to provide the foundation for some simple visualizations that relates to effort.

[ricky.clarkson@…]

Effort remaining and percentage [in]complete sound like the same thing to me.

[Christian Boos]

Replying to ricky.clarkson@…:

Effort remaining and percentage [in]complete sound like the same thing to me.

If you assign some quantity ("effort") to what 100% means for a ticket, then yes.

[Christian Boos]

Replying to rblank:

Replying to cboos:

This is a "self-correcting" approach, as after some time the developer will realize that to keep a progress bar "green", they need to make conservative estimates, and tickets following the 90:90 rule can be quickly spotted as such ;-)

This contradicts most of the research in project estimation, as well as my own observations.

/me humbly trying to improve the state of the art ;-)

The software sector has been found to be notoriously bad at self-correcting (or learning from its own mistakes), and developers don't correct their estimates based on past experience, they just repeat the same mistakes over and over again. That's why I believe ​evidence-based scheduling has a bright future, as it integrates past experience automatically.

That sounds overly pessimistic. Even in the article linked above, you can find some "hope":

As estimators gain more experience, their estimating skills improve. So throw away any velocities older than, say, six months.

And my take is that green/red colors can be effective to help this learning process, much like the green/red builds encourage people to keep the committed code working at all times. Perhaps the color scheme I proposed is not optimal: the red/grey should be permuted, so that the "bad" state (red) is the inadequate estimation that needs to be readjusted, the "not ideal" state (grey) is the readjusted (increase of estimated effort).

But even if they don't improve, the main point of my proposal is to have a way to identify what's taking more time than expected.

What I find not so great about EBS is that you only have a clear picture of what's really going for a task after the fact, once the ticket is closed. Before that point, all you have is the initial estimation, the time spent so far and the predicted duration based on the developer's velocity. As long as the time spent is between the initial estimation and the predicted duration, you can't really tell if a task is slipping or not. That's not a big deal when you have lots of small tasks, as you're advised to do. But what if you have several "big" tickets (as we have here for example), and you try to communicate early that despite the fact the "effort spent" gets to the value of the "estimated effort", you already know you're only still half-way through, or even one quarter.

So the feature I'd like to have is to be able to look at an overview of the opened tickets (in a custom query say), and spot those which are "on track" and those which side-track… I'm not sure how you'd achieve that with EBS (well, you can spot those for which the time spent is even above the predicted time, but that's really late).

This could be achieved by offering the possibility to modify the "estimated effort" continuously, something which is also possible in the ​TH:EstimationToolsPlugin presented by osimons above, as I suppose you can also increase the amount of remaining work. And when a single value is involved I have to agree with him that this concept is more powerful than an isolated "percent complete", as you can't use the latter for any kind of aggregation. However, as soon as you also want to keep track of the "effort spent", this becomes very similar to the approach I had in mind, as the % of completion is the simple ratio of "effort spent" / ("effort spent" + "remaining effort" == "estimated effort"), and conversely, if you change the "effort spent" and "percent complete", you also change the "estimated effort", so this is just a matter of user interface.

Note that I'm not a fan at all of timesheets and other similar barbaric tools, but as osimons said, these are unfortunately of very common use… I'm looking at ways this time tracking (the mechanism of which is belonging to #710) can eventually be put to some interesting use (the colored progress bar reflecting the current velocity on a given task).

[scottj@…]

What about just some visualization fields added to trac? I mean, we can all use the fields the way we want, but it seems like a progress bar of some sort (or just a box/bar field, useful for anything) could be added to the trac core. Is there any possibility of doing this?

[Christian Boos]

To summarize this long discussion, I think a sufficiently flexible approach would be to have the possibility to define a "progress bar" field as being the ratio of two other (custom) fields (e.g. "effort_spent"/"estimated_effort", or "worked"/"storypoints", or "time_spent"/"time_estimated"). Whether the denominator is fixed or not would be a matter of convention (if fixed, the % should likely be allowed to go over 100%, to be indicative of underestimations).

And of course, having the possibility to have numeric TracTicketsCustomFields would be a good first step ;-)