Difference between revisions of "Bug Reporting"
Revision as of 20:08, 15 April 2009
- 1 Introduction
- 2 Importance of Reporting Bugs
- 3 What Bugs Should Be Reported
- 4 What Data Should Be In The Report
- 5 Attaching Files to the Bug Report
- 6 Submitting a Patch
- 7 Additional Bug Report Fields
- 8 Following Bug Progress
- 9 References
In oder to focus the efforts of a software project, it is essential to have a clear understanding of the current status of all defects.
Importance of Reporting Bugs
Software defects are commonly tracked using specialized databases. There are several reasons to track defects strictly:
- If a bug is not tracked, it may be overlooked and never fixed
- If a bug is not tracked, key members of the development team may not know it exists
- Tracking bugs provides a way to determine the functional status of the project
- The bug list provides development team members a place to look for things to contribute
- The bug list provides users a place to formally report issues
- The bug tracking system provides a way to divide up work among the development team
- The bug tracking system provides a way for people to check the status of particular issues and present workarounds or fixes in an organized and manageable way
What Bugs Should Be Reported
Follow these steps:
Step 1: Determine if the issue is worth reporting
If the answer to any of these questions is "yes", then this issue probably does not warrant a bug report:
- Can the issue be phrased as a support request? (for examples: "How do I...", "Is it possible to...", "How do I configure...", "Where is documentation on..."
- Is the issue related to future development? (for examples: "Add a feature for...", "Restructure system so...")
- Is the issue caused by an unrelated software package that doesn't directly affect our project? (for example, say we are making a phone system based in Linux that has a Gnome desktop. Our issue is: "I cannot open text documents in gedit because the Gnome project has provided a broken package". This would not be a bug for our phone system at all, as it doesn't affect our project - though it may be an annoyance to people using our applications)
If the answer to any of the following questions is "yes", then it may be time to consider reporting a bug:
- Is a feature completely unusable?
- Is a feature that used to work not working as expected?
- Is an important feature missing that is hindering the usefulness of another feature?
Step 2: Determine if the bug is actually expected behavior
In many cases, software is designed to do things that the user doesn't expect. To avoid reporting bugs when no bug exists, a thorough check of the functionality of the affected component is required. Make sure you have answered the following questions before reporting a bug:
- Do I have a complete understanding of what is happening to produce the bug? (consider checking documentation)
- Have other people experienced this issue and been shown that it is by design? (consider checking forums, IRC channels)
- Is this component under heavy development right now? Perhaps there has been a change in how it works.
Step 3: Determine if the bug has already been reported
Reporting duplicate bugs wastes everyone's time. The reporter takes time to compile their thoughts, the reviewer takes time to mark it as a duplicate, future bug investigators take time to redirect to the proper bug, and the list goes on. To maintain everyone's efficiency, duplicate bugs should be avoided. Make sure you have followed the previous two steps and done significant searching in the bug database before reporting a new bug.
If a bug ticket already exists for the issue you are trying to report, make sure there is enough information in the ticket. Add any new information you have. Useful information is discussed in the next section
Step 4: Report the bug
Assuming all of the previous steps have suggested the issue is worth reporting, it's time to make a bug report. The details that go into the report are discussed in the next section.
What Data Should Be In The Report
An uninformative bug report is a waste of time. To make a good bug report, the following items are essential:
- A clear title/subject - describe what is broken and in what way
- Good example: Unable to adjust screen brightness using hotkeys - works otherwise
- Bad example: Brightness doesn't work
- Clear description of the issue - describe all symptoms of the issue in the body of the report without trailing off into other features. Be sure to explain exactly what you expect to happen, exactly what is happening, and exactly how to make it happen.
- Good example: When the hotkeys associated with screen brightness are pressed, the screen brightness should change, but the screen brightness does not change. The expected brightness notification appears in the top right of the screen, but it displays the wrong brightness level.
- Bad example: I push the hotkey and nothing happens.
- If there is more than one way to perform a task, state exactly which way you did it, and whether the other way works
- Good example: (taking a break from the brightness examples) Select File --> Load to load the file. Loading the file by double clicking in the file explorer works.
- Bad example: Load the file.
- When was the last time it worked the way you expected? - Is this something that broke recently? When, precisely, did it stop working?
- Good example: This worked out of the box in versions 7.04 through 8.10. Using the most recent development packages, it does not work
- Bad example: It used to work, now it doesn't.
- Step by step instructions to reproduce the issue - the more detailed, the better. Do not leave steps out, do not assume any prior knowledge of the issue, even if it is widely known
- Good example: While logged into Gnome in an X session, press Fn+F5 or Fn+F6 several times. Notice that the brightness doesn't change, but the brightness notification appears
- Note any configuration that may impact this issue - If you have configured something differently than expected, be sure to note this.
- Good example: I am using the out-of-the box configuration.
- Good example: I have changed xorg.conf to try to increase my screen resolution. The lines I changed were...
- Hardware description - If hardware could affect the issue, be sure to include all relevant information
- Good example: I am using a Sony VGN-S480 with nVidia 6200Go graphics card
- Workarounds - If you've found a way around the issue, add that in to help people who are suffering the same issue and to provide a better picture of what is actually broken
- Good example: It is possible to adjust screen brightness using the smartdimmer command
- MOST IMPORTANTLY: RE-READ THE REPORT BEFORE YOU SEND IT - Re-reading gives you a chance to collect your thoughts, make sure you haven't missed anything, correct typographical errors that make reading difficult, and clarifying your ideas. Be sure your writing is clear and concise. Read the report as if you know nothing about the issue. Look for places where you may have assumed certain knowledge or a certain process.
When writing your report, be mindful of the following:
- Avoid the use of general terms like "it" or "the window". Instead, use specifics like "the brightness notification dialog at the top right of the screen"
- Give any information that could be related. It is generally acceptable to provide too much information, but leaving something out will make the issue harder to address
- This can't be stressed enough: RE-READ THE REPORT BEFORE YOU SEND IT for all the reasons stated above.
Attaching Files to the Bug Report
- Log files are essential to understanding issues. Attach any log files that may be interesting.
- The output of certain programs may demonstrate the issue well. Attach any interesting output in text files if more than a few lines to avoid filling the bug report with complex text.
Submitting a Patch
If you're attaching a patch, be sure to mention it in the report. Attach it as a separate file - not inline.
- Explain how the patch addresses the problem and any other components that the changes might affect.
- Explain the procedure you went through to test the functionality of the patch
- Explain what you've done to make sure the patch does not break other functionalities.
Additional Bug Report Fields
Be sure to fill out as many of the additional fields on the report as you can as accurately as you can. Fields like "version" should be pretty easy. Fields like "component" or "package" might be more difficult - use your judgment. Be sure to categorize the bug properly - is this a feature request, a defect, a security risk...? If you have a patch to fix the bug, be sure to indicate that in the report and in any field that allows it.
Following Bug Progress
If you are the original reporter of the bug you have a responsibility to make sure it is addressed. Check the bug ticket regularly. If more information is requested, you should provide it in a timely manner.
It is generally not acceptable to post additional comments on a bug report asking for the bug to be fixed faster, so do not do that. Only post additional comments if you have come up with new information that will help resolve the bug.