Nine Tips for an Effective Bug Reporting
Thursday, December 25th, 2008
yogindernath
It is important that a software tester reports everything what he feels. A software tester role is that of a catalyst in any team. He makes up the team on one hand and breaks up the application on the other. It is important to clearly understand all the issues in the application may be big or small with due course of understanding in terms of business and the application. Therefore a strong Bug Report serves as a definite proof in Software Development Life Cycle, provided the status of bugs is updated at all stages. Sole objective of your reporting a bug is that your intention is to get the bug fixed.
1. Clarity of Bug Description - Bug description refers to a small statement, which briefly points towards the exact problem. May be the problem requires a couple of steps to reproduce it, however this small statement of bug description must be able to communicate the exactly nature of problem. For instance in case of a problem concerning an error from the server, the bug description could clearly elaborate the meaning by stating that the server error takes place while performing such an such operation.
2. Don’t Pass your Verdict - even if you are brimming with confidence about authenticity of the bug detected by you, avoid writing a bug report which would reflect as if you are trying to pass your verdict on the genuinity of the bug. In every probability this could initiate a controversy which would reflect your superiority complex as a tester. Your main aim should be to keep your bug report conclusive supporting your bug plus the sole motive must be to get the bug closed ultimately. Try to use diplomacy in bug report, instead of using authoritative statements in favor of your bug thereby making your bug report unpleasant, best way is to be suggestive. Such an approach shall always be taken in good spirit.
3. Steps to Reproduce - How to reach the precise point of bug, with due explanation of the set of conditions so as to reproduce it must be clearly defined in the bug report. For instance, for graphical software, tester needs to communicate to the developers, as to what he had done before getting the bug. The details must be elaborated like which buttons were pressed and in what order. For a program executed by keying in a command across the prompt, the details of the command typed before getting the bug needs to be indicated precisely.
4. Use of Simple Language - People do not like to read long paragraphs containing complex jargons & tongue twisters. A good bug report contains small bulleted like containing small but clear sentences. It should describe the observations relevant to the concerned bug only. Do not make the bug report unnecessarily complex and lengthy by writing too many facts. Avoid narrating superfluous details, which may not be of any help in reproducing the bug. Do not write things, which are commonly known, to everyone.
5. Quote-Relevant Examples - In majority of situations, we find that for reproducing a particular bug, some particular group of inputs is essential. Thus instead of making vague statements like feed invalid name of a person in the contact list, & save; it can be said that feed an invalid input like 035bbb@$% in the name field & click save. In order to get the bug fixed quickly tester must try to help the programmers by providing all the relevant information / to-the-point information.
6. Provide Back References - In case a particular bug happens to contradict the specification document or any other document related to the project; the bug report must provide adequate reference to the particular chapter or clause number of the concerned document being contradicted.
7. Allocate Bug Priority & Severity - Bug reporting is not complete without earmarking the level of Bug Severity & Bug Priority.
Bug Severity: refers to the quantum of danger as to how badly the bug can harm the system. It describes as to how bad the bug is. Severity is a feature of constant nature associated with the bug.
There are three levels of Bug Severity, which are described as under:
Severity level - Critical: is the most dangerous level, which does not permit continuance of the testing effort beyond a particular point. Critical situation can arise due to popping up of some error message or crashing of the system leading to forced full closure or semi closure of the application. Criticality of the situation can be judged by the fact that any type of workaround is not feasible. A bug can fall into “Critical” category in case of some menu option being absent or needing special security permissions to gain access to the desired function being tested.
Severity level - High: is a level of major defect under which the product fails to behave according to the desired expectations or it can lead to malfunctioning of some other functions thereby causing failure to meet the customer requirements. Bugs under this category can be tackled through some sort of workaround. Examples of bugs of this type can be mistake in formulas for calculations or incorrect format of fields in the database causing failure in updating of records. Likewise there can be many instances.
Severity level - Medium: defects falling under this category of medium or average severity do not have performance effect on the application. But these defects are certainly not acceptable due to non-conformance to the standards or companies vide conventions. Medium level bugs are comparatively easier to tackle since simple workarounds are possible to achieve desired objectives for performance. Examples of bugs of this type can be mismatch between some visible link compared with its corresponding text link.
Severity level - Low: defects falling under low priority or minor defect category are the ones, which do not have effect on the functionality of the product. Low severity failures generally do not happen during normal usage of the application and have very less effect on the business. Such types of bugs are generally related to looks & feel of the user interface & are mainly cosmetic in nature.
Bug Priority: refers to the need as to how urgently bug is required to be fixed. It describes the importance of the bug. Bug priority may change according to the schedule of testing. There are three levels of Bug Priority, which are described as under:
High Priority: Such bugs if not fixed immediately, are going to affect the normal functioning at customer end. Hence such bugs are given immediate or topmost priority for fixing.
Medium Priority: is allocated to the bugs with major defects having great effect on the functioning of the customer. Such bugs are allocated great priority so those related problems are resolved prior to release of the present software version. If due to some time constraint it is not possible to resolve this issue, some sort of patch or service pack must be released
Low Priority: is generally allocated to bugs, which do not have significant effect on the performance of the software at the customer end. Effort is made to resolve such bugs prior to release of the present version, if this is not feasible due to constraint of time such fixation can wait till the release of next version.
8. Explain by Screenshots - As per an old saying “A picture is worth more than 100 words”. When we encounter an error, it is best to capture the screenshot of the particular moment. If an error message is seen, its screenshot will help the developer in having a precise understanding of the problem. This is the stage the developer does not try to fix the problem, rather focuses his attention to firstly understand the problem clearly.
Such screenshot should be an appendix of the bug report as evidence. This way tester is able to communicate & explain his bug in a better way & with more clarity to the developer.
9. Standby your Genuine Bugs: Most interesting portion of bug reporting is when the software tester needs to be standby the bugs found by him & needs to defend that bugs covered under his report are genuine bugs needing fixation since it is going to affect the performance of the application. During this the course, the software-testing engineer must be ready to face situations with the programmers like the few ones mentioned below.
Case 1: Developers usually hit back saying that particular bug can’t be reproduced. The best way of reporting the bug is practically showing it to the developer. This can be done by asking them to have a look at the scenario on your computer system, load the application & provide live demonstration of the problematic event. This would provide them actual look & feel of the situation as to how you had fired the application, how you had been interacting with the application & how the software reacts to the inputs provided. As a best practice avoid reporting of non-reproducible bugs in an enthusiasm to report maximum number of bugs in shortest possible time.
Case 2: Sometimes the software tester comes across funny circumstances with some application having inconsistent pattern of failure. Such situations can arise when the tester encounters pressure of deadlines & the application under test fails, but he faces embarrassing moments while demonstrating the same to the developers, since the application behaves normally at that moment. Thus a good tester needs to be patient & always build a defense mechanism in the form of preserving test data & screenshots etc. to justify his statements.
Case 3: If the tester provides the developers a big list of various actions & inputs etc. but the program fails to show anything wrong when executed on the system of the developer. This means that the tester has not provided sufficient information. It is quite possible the systems of the developer & the tester differ in some configuration, thereby causing the fault not to appear on computer system of the developer. It is quite possible that the tester had misunderstood the expectations of the program, while the tester & the developer are witnessing the same display, but with a difference in viewpoint. It might be possible that what appears, as an error to the tester might be correct from the point of view of the developer. Thus to come out of such situations, it is preferred to say as to what you had expected & what you had seen exactly & what had happened.
It is important that a software tester reports everything what he feels. A software tester role is that of a catalyst in any team. He makes up the team on one hand and breaks up the application on the other. It is important to clearly understand all the issues in the application may be big or small with due course of understanding in terms of business and the application. Therefore a strong Bug Report serves as a definite proof in Software Development Life Cycle, provided the status of bugs is updated at all stages. Sole objective of your reporting a bug is that your intention is to get the bug fixed.
1. Clarity of Bug Description - Bug description refers to a small statement, which briefly points towards the exact problem. May be the problem requires a couple of steps to reproduce it, however this small statement of bug description must be able to communicate the exactly nature of problem. For instance in case of a problem concerning an error from the server, the bug description could clearly elaborate the meaning by stating that the server error takes place while performing such an such operation.
2. Don’t Pass your Verdict - even if you are brimming with confidence about authenticity of the bug detected by you, avoid writing a bug report which would reflect as if you are trying to pass your verdict on the genuinity of the bug. In every probability this could initiate a controversy which would reflect your superiority complex as a tester. Your main aim should be to keep your bug report conclusive supporting your bug plus the sole motive must be to get the bug closed ultimately. Try to use diplomacy in bug report, instead of using authoritative statements in favor of your bug thereby making your bug report unpleasant, best way is to be suggestive. Such an approach shall always be taken in good spirit.
3. Steps to Reproduce - How to reach the precise point of bug, with due explanation of the set of conditions so as to reproduce it must be clearly defined in the bug report. For instance, for graphical software, tester needs to communicate to the developers, as to what he had done before getting the bug. The details must be elaborated like which buttons were pressed and in what order. For a program executed by keying in a command across the prompt, the details of the command typed before getting the bug needs to be indicated precisely.
4. Use of Simple Language - People do not like to read long paragraphs containing complex jargons & tongue twisters. A good bug report contains small bulleted like containing small but clear sentences. It should describe the observations relevant to the concerned bug only. Do not make the bug report unnecessarily complex and lengthy by writing too many facts. Avoid narrating superfluous details, which may not be of any help in reproducing the bug. Do not write things, which are commonly known, to everyone.
5. Quote-Relevant Examples - In majority of situations, we find that for reproducing a particular bug, some particular group of inputs is essential. Thus instead of making vague statements like feed invalid name of a person in the contact list, & save; it can be said that feed an invalid input like 035bbb@$% in the name field & click save. In order to get the bug fixed quickly tester must try to help the programmers by providing all the relevant information / to-the-point information.
6. Provide Back References - In case a particular bug happens to contradict the specification document or any other document related to the project; the bug report must provide adequate reference to the particular chapter or clause number of the concerned document being contradicted.
7. Allocate Bug Priority & Severity - Bug reporting is not complete without earmarking the level of Bug Severity & Bug Priority.
Bug Severity: refers to the quantum of danger as to how badly the bug can harm the system. It describes as to how bad the bug is. Severity is a feature of constant nature associated with the bug.
There are three levels of Bug Severity, which are described as under:
Severity level - Critical: is the most dangerous level, which does not permit continuance of the testing effort beyond a particular point. Critical situation can arise due to popping up of some error message or crashing of the system leading to forced full closure or semi closure of the application. Criticality of the situation can be judged by the fact that any type of workaround is not feasible. A bug can fall into “Critical” category in case of some menu option being absent or needing special security permissions to gain access to the desired function being tested.
Severity level - High: is a level of major defect under which the product fails to behave according to the desired expectations or it can lead to malfunctioning of some other functions thereby causing failure to meet the customer requirements. Bugs under this category can be tackled through some sort of workaround. Examples of bugs of this type can be mistake in formulas for calculations or incorrect format of fields in the database causing failure in updating of records. Likewise there can be many instances.
Severity level - Medium: defects falling under this category of medium or average severity do not have performance effect on the application. But these defects are certainly not acceptable due to non-conformance to the standards or companies vide conventions. Medium level bugs are comparatively easier to tackle since simple workarounds are possible to achieve desired objectives for performance. Examples of bugs of this type can be mismatch between some visible link compared with its corresponding text link.
Severity level - Low: defects falling under low priority or minor defect category are the ones, which do not have effect on the functionality of the product. Low severity failures generally do not happen during normal usage of the application and have very less effect on the business. Such types of bugs are generally related to looks & feel of the user interface & are mainly cosmetic in nature.
Bug Priority: refers to the need as to how urgently bug is required to be fixed. It describes the importance of the bug. Bug priority may change according to the schedule of testing. There are three levels of Bug Priority, which are described as under:
High Priority: Such bugs if not fixed immediately, are going to affect the normal functioning at customer end. Hence such bugs are given immediate or topmost priority for fixing.
Medium Priority: is allocated to the bugs with major defects having great effect on the functioning of the customer. Such bugs are allocated great priority so those related problems are resolved prior to release of the present software version. If due to some time constraint it is not possible to resolve this issue, some sort of patch or service pack must be released
Low Priority: is generally allocated to bugs, which do not have significant effect on the performance of the software at the customer end. Effort is made to resolve such bugs prior to release of the present version, if this is not feasible due to constraint of time such fixation can wait till the release of next version.
8. Explain by Screenshots - As per an old saying “A picture is worth more than 100 words”. When we encounter an error, it is best to capture the screenshot of the particular moment. If an error message is seen, its screenshot will help the developer in having a precise understanding of the problem. This is the stage the developer does not try to fix the problem, rather focuses his attention to firstly understand the problem clearly.
Such screenshot should be an appendix of the bug report as evidence. This way tester is able to communicate & explain his bug in a better way & with more clarity to the developer.
9. Standby your Genuine Bugs: Most interesting portion of bug reporting is when the software tester needs to be standby the bugs found by him & needs to defend that bugs covered under his report are genuine bugs needing fixation since it is going to affect the performance of the application. During this the course, the software-testing engineer must be ready to face situations with the programmers like the few ones mentioned below.
Case 1: Developers usually hit back saying that particular bug can’t be reproduced. The best way of reporting the bug is practically showing it to the developer. This can be done by asking them to have a look at the scenario on your computer system, load the application & provide live demonstration of the problematic event. This would provide them actual look & feel of the situation as to how you had fired the application, how you had been interacting with the application & how the software reacts to the inputs provided. As a best practice avoid reporting of non-reproducible bugs in an enthusiasm to report maximum number of bugs in shortest possible time.
Case 2: Sometimes the software tester comes across funny circumstances with some application having inconsistent pattern of failure. Such situations can arise when the tester encounters pressure of deadlines & the application under test fails, but he faces embarrassing moments while demonstrating the same to the developers, since the application behaves normally at that moment. Thus a good tester needs to be patient & always build a defense mechanism in the form of preserving test data & screenshots etc. to justify his statements.
Case 3: If the tester provides the developers a big list of various actions & inputs etc. but the program fails to show anything wrong when executed on the system of the developer. This means that the tester has not provided sufficient information. It is quite possible the systems of the developer & the tester differ in some configuration, thereby causing the fault not to appear on computer system of the developer. It is quite possible that the tester had misunderstood the expectations of the program, while the tester & the developer are witnessing the same display, but with a difference in viewpoint. It might be possible that what appears, as an error to the tester might be correct from the point of view of the developer. Thus to come out of such situations, it is preferred to say as to what you had expected & what you had seen exactly & what had happened.
