Bug control: which tracking software is best?

Checking these reports will always reveal many bugs being “folded in” since they merely duplicate existing ones, and in some cases not even bugs for that particular piece of software. An important part of any bug reporting system is the way that it manages the lifetime of each bug, which will typically fall into three phases: an initial “opening phase”, a “fixing/killing phase” and a “closing/burial phase”.

First phase

The opening phase is when a bug is first introduced to the system, and will be followed by various types of triage whereby the bug is verified and accepted.

Different systems support the opening of bugs in different ways, but the key decision is whether or not users are permitted to report bugs directly into the bug control system. This is an initially appealing notion, but as I’ve mentioned already, most users aren’t very good at reporting bugs.

Choosing this route will typically lead to a number of bug reports being closed at this point because they can’t be verified – either by the person checking or even reporting the bug – or else they can’t be accepted (as in “We no longer support Internet Explorer 4 on any platform, and never did on the Mac”).

As part of this process, evidence of the bug will need to be captured and assigned to the bug report, and a severity level allocated to it, which in some cases may be decided by a vote (“Do we really need to fix this?”).

Opened bug

Once a bug is opened, it will need to be allocated to a person or group who’s going to fix it. The fixing process can vary greatly, but it will have to work with the tools that we already use: development environments, source code control systems and test suites.

Let’s not forget that testing forms an essential part of fixing

Let’s not forget that testing forms an essential part of fixing, and as such we’ll need some way to hand this bug report to a tester to ensure it’s been eliminated. At some point the bug will have been killed and then there will need to be a closure, in which the changes are consolidated and applied as necessary to any other related systems.

So what was our criteria in choosing our own bug reporting system? For various reasons, we weren’t interested in allowing users to enter bugs directly into our system – we’re going to use a different route.

Next, we needed the bug reporting system to work with our existing source code control system, which is Subversion. Finally, it would be good if the bug reporting system could integrate to some extent with the Eclipse IDE (integrated development environment) we use most of the time.

Eclipse has a module called Mylyn that’s designed to perform precisely this integration, so Mylyn support would be useful. Mylyn has been part of Eclipse for a couple of releases now, and has been improved in the current release (3.6 or “Helios”) as discussed in the boxout below.

Unfortunately, not all our programming is carried out within Eclipse, and Apple’s Xcode environment doesn’t seem to support any bug control system (Apple software never has bugs, you just have to hold it properly), so we needed a good separate user interface too.