Simply put, the more effectively you report an error, the more likely a Data Manager will actually fix it.
These guidelines are a general tutorial to teach novice and intermediate error reporters how to compose effective error reports. Not every sentence may precisely apply to your report or project.
Reading this report will involve much more time than it will to report most errors. Most error reports can be completed in just a few minutes while complicated reports may take just a few minutes more. Identifying an error in a dataset or analysis program is the hardest part. Reporting the error is simple and provides invaluable information to Data Managers and System Developers. The value to you is that, in time, data and data product systems will just keep getting better and better.
Useful error reports are ones that get errors fixed. A useful error report normally has two qualities:
Reproducible. If a Data Manager can't see the error herself to prove that it is valid, she'll probably stamp your error report "WORKSFORME" or "INVALID" and move on to the next error. Every detail you can provide helps.
Specific. The quicker the Data Manager can isolate the error to a specific area, the more likely she'll expediently fix it. (If the an error is difficult to understand or decipher they may spend more time cursing the submitter than solving the problem.)
Let's say the Data-Product of a particular Source System you're testing provided erroneous results and you decide to write up an error report:
BAD: "This product didn't work right. I think I was on www.foo.com. I play golf with Tom Karl, so you better fix this problem, or I'll report the error to him. By the way, this isn't the first time I've seen errors in your products. Thx 4 UR help."
GOOD: "I think there is an error in the value for total precipitation of this product. I listed out the values that make up the 'total' and it did not match the calculated total. It looks like the routine that calculates the total precipitation is only summing up the individual reports to the nearest 1/10". I was using a Microsft Explorer web browser on a Windows 2000 computer. I do not think that the error is a browser issue. Other than this single error, the product appears to work quite well.”
Before you enter your error, use Datzilla's search page to determine whether the defect you've discovered is a known, already-reported error. If your error is the 37th duplicate of a known issue, you're more likely to annoy the Data Manager. (Annoyed Data Managers fix fewer errors.)
Next, be sure that the error is reproducible and has not already been reported. Data Managers tend to be most interested in problems that are new and do not want to revisit old problems that have already been fixed.
If you've discovered a new error, report it directly within Datzilla:
From your Datzilla main page, choose "Enter a new error".
Select the Source System and Data-Product that you've found a error in.
Enter your e-mail address, password, and press the "Login" button. (If you don't yet have a password, leave the password field empty, and press the "E-mail me a password" button instead. You'll quickly receive an e-mail message with your password.)
Now, fill out the form. Here's what it all means:
Source System: In which Source System did you find the error? You just specified this on the last page, so you can't edit it here.
Data-Product: In which Data-Product does the error exist? Datzilla requires that you select a Data-Product to enter a error. (Not sure which to choose? Click on the Data-Product link. You'll see a description of each Data-Product, to help you make the best choice.)
Problem Area: In which Problem Area did you find the error? (If applicable)
Browser and OS: On which Browser and Operating System (OS) did you find this error?
The Browser, especially, is an important component of most reports. Data-Products can be produced correctly by the software system, but displayed incorrectly by a Browser.
Severity: How damaging is the error?
This item defaults to 'normal'. If you're not sure what severity yourerror deserves, click on the Severity link. You'll see a description of each severity rating. If you are not sure what severity to report, accept the default value. It may be also changed by the Data Manager.
Assigned To: Which Data Manager should be responsible for fixing this error?
Datzilla will automatically assign the error to a default Data Manager upon submitting an error report. (To see the list of default Data Managers for each Data-Product, click on the Data-Product link.)
Cc: Who else should receive e-mail updates on changes to this error?
List the full e-mail addresses of other individuals who should receive an e-mail update upon every change to the error report. You can enter as many e-mail addresses as you'd like, separated by spaces or commas, as long as those people have Datzilla accounts. Try to restrain these 'cc' addresses. The reports can always be viewed on-line by others so 'cc' reports are not always necessary.
Summary: How would you describe the error in approximately 60 or fewer characters? A good summary should quickly and uniquely identify an error report. Otherwise, a Data Manager cannot meaningfully identify your error by its summary, and will often fail to pay attention to your error report when skimming through a 10 page error list.
A useful summary might be "Rainfall total does not match the data record at the WSFO".
"Bad value." or "Product does not work" would be examples of a bad summary.
Description: Please provide a detailed problem report in this field. Your error's recipients will most likely expect the following information:
Overview Description: More detailed expansion of summary. Give as much detail in the description as possible.
Steps to Reproduce: Minimized, easy-to-follow steps that will trigger the error. Include specific steps that allow the Data Manager to reproduce the exact path to where and how you identified the error.
Actual Results: What the application did after performing the above steps or what the reported data value was.
Expected Results: What the application should have done, or what the data value should have been (if known) were the error not present.
Browser & Operating System: The Browser (an if known, its version) and the OS you were using when viewing a product or retrieving data when you first encountered the error.
Additional Browsers and Operating Systems: Whether or not the error takes place on other Browsers or Operating Systems (if known or applicable).
- Also occurs on Mozilla on Windows NT 4.0
- Doesn't occur on Firefox on Red Hat Linux
Additional Information: Any other information that helps describe or clarify the problem. Information that helps to discover systematic errors are especially valuable. Identifying a systematic error in a reporting practice, analysis routine, or quality control routine could save hundreds of error reports in the future.
After double-checking your entries for any possible errors, press the "Commit" button, and your error report will now be in the Datzilla database.
You will be able to monitor the status of your error report by performing a Datzilla query. This will tell you who is working on the error, if it is being resolved, or if it is being ignored.
You may be asked questions by the Data Manager to clarify your report or to provide additional information hat the Data Manager may not have about a valid data value. All of these requests, as well as your responses, will be tracked by the system.
General Tips for a Useful Error Report
Use an explicit structure, so your error reports are easy to skim. Error report users often need immediate access to specific sections of your error. If your Datzilla installation supports the Datzilla Helper, use it.
Avoid cuteness if it costs clarity. Nobody will be laughing at your funny error title at 3:00 AM when they can't remember how to find your error.
Enter one error per report. Completely different people typically fix, verify, and prioritize different errors. If you mix a handful of errors into a single report, the right people probably won't discover your errors in a timely fashion, or at all. Certain errors are also more important than others. It's impossible to prioritize an error report when it contains four different issues, all of differing importance.
No error is too trivial to report.
How and Why to Write Good Error Summaries
You want to make a good first impression on the Data Manager. Just like a New York Times headline guides readers towards a relevant article from dozens of choices, your error summary should suggest that your error report is worth reading and will receive attention?
Your error report will often be searched by its summary. Just as you'd find web pages with Google by searching keywords through intuition, so will other people locate your errors. Descriptive error summaries are naturally keyword-rich, and easier to find. Ask yourself, "Would someone understand my error from just this summary?" If so, you've written a fine summary.