Incident: Any event occurring that requires investigation.
Defect: Any flaw in a component/system that can cause a component/system to fail to perform its required functionality.
How is a Test incident report different than a Normal Defect/Bug report?
Please provide a sample incident report.
An incident is just something that did not happen as forecasted. It can be because :
Before even thinking about reporting, the tester must investigate as far as he can(how far depends greatly on the shop), and at least chek the error does not come from himself(forgot a prerequisite, didn't do things in order, had wrong expectations...). It's a classical to have misunderstood the specs, and written a wrong test scenario. Better detect it before raisong a bug...
Once he's sure the origin of the incident is not from his side, then he can begin to think about reporting.
Both reports can be very similar
And don't forget the context in the report. Example : "Choose any foreign currency. Choose any stock whose value is below 1 in this currency. Bug does not happen when stock value is at 1 or above, of for our national currency."
Basically an incident can be a defect or can be a feature. During testing you log suspicious behaviour which you cannot state for sure as a bug and only after further investigation can be titled as defect. In the defect report will be links to features/design specifications/discussion which provide enough evidence on deviation from it.
Since the difference between "Incident" and "Defect" treads on a very thin line. They are often confused to be same. In nutshell, all defects are incidents but all incidents are NOT defects. The TIR contains the details of actual vs expected results and any other investigation points(why the test failed,impact of the failure etc) According to IEEE, it is called an Incident report and not a bug report because the discrepancy between Actual and expected results may occur for reasons other than a fault in the application(defect). An incident may occur due to reasons like(but not limited to) hardware mis-configuration,wrong test data, inconsistent design. The following link contains a sample test incident report template according to the IEEE standard: http://www.ufjf.br/eduardo_barrere/files/2011/06/SQETestIncidentReportTemplate.pdf
Test Incident Report: detailing, for any test that failed, the actual versus expected result, and other information intended to throw light on why a test has failed. This document is deliberately named as an incident report, and not a fault report. The reason is that a discrepancy between expected and actual results can occur for a number of reasons other than a fault in the system. These include the expected results being wrong, the test being run wrongly, or inconsistency in the requirements meaning that more than one interpretation could be made. The report consists of all details of the incident such as actual and expected results, when it failed, and any supporting evidence that will help in its resolution. The report will also include, if possible, an assessment of the impact upon testing of an incident.
Regarding bug-reports: In my opinion, the best possible bug report (assuming that the person that sends the error report does not know the causes of the observed failure) is a bug report with reproducibility information, so that the developers are capable of reproducing the problem.
In several bug tracking sites, for example MySQL, we find literally thousands of bug reports labeled as "can't reproduce" or "can't repeat". Either because the user does not know how a bug should be reported or because the user does not know how to reproduce the error, the issue is the same: reports are sent with little or no historical information on how the error was reached (e.g. a report containing solely the stack trace and/or a memory snapshot and/or a textual description of the event that is both superficial and cumbersome). Sometimes little information is sufficient, but many times it is not, especially in errors that are induced by very specific conditions that take the execution through a very specific path.
Therefore, in short (and in my opinion), a good bug report is one that has sufficient information for the developers to reproduce the observed problem. This is often hard to provide, especially if no logging was performed during the failing execution.
There has been a lot of research in this area in the past decades, but unfortunately there is no solution that solves all problems. In my case, I believe that record&replay systems (a.k.a. fault-replication systems) is more promising, as they automatically log sources of non-determinism during the user's execution and if/when a failure occurs they create bug-reports capable of deterministic replay. Still, they have problems regarding performance overhead, log size and privacy.
Regarding what makes a good bug report, you may find the following research papers to be interesting:
["What makes a good bug report?", IEEE Transactions on Software Engineering, T. Zimmermann, R. Premraj, N. Bettenburg, J. Sascha, A. Schroter, and C. Weiss]
Presents: i) a survey on how bug reports are used among 2,226 developers and reporters, out of which 466 responded; ii) empirical evidence for a mismatch between what developers expect and what reporters provide; iii) the CUEZILLA tool that measures the quality of bug reports and suggests how reporters could enhance their reports, so that their problems get fixed sooner. Publicly available at: http://thomas-zimmermann.com/publications/files/bettenburg-fse-2008.pdf
["Debugging in the (very) large: ten years of implementation and experience", in Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles, K. Glerum, K. Kinshumann, S. Greenberg, G. Aul, V. Orgovan, G. Nichols, D. Grant, G. Loihle, and G. Hunt]
This is a paper from Microsoft regarding ten years of bug report analysis of the Windows Error Reporting System. You can find it publicly available at: http://research.microsoft.com/pubs/81176/sosp153-glerum-web.pdf
["Information needs in bug reports: improving cooperation between developers and users", in Proceedings of the 2010 ACM conference on Computer supported cooperative work, S. Breu, R. Premraj, J. Sillito, and T. Zimmermann]
A paper that analyses the questions asked in a sample of 600 bug reports from the Mozilla and Eclipse projects and also provides some suggestions to improve bug trackers. Link: http://people.ucalgary.ca/~sillito/work/cscw2010.pdf
["Who tested my software? testing as an organizationally cross-cutting activity", 2011 Software Quality Journal , M. Mantyl a, J. Iivonen, and J. Itkonen]
Examines testing activities in different industrial case companies, conducted not only by the specialized testers but also by multiple stakeholders. Link: http://lib.tkk.fi/Diss/2011/isbn9789526043395/article6.pdf
["Survey Reproduction of Defect Reporting in Industrial Software Development", 2011 International Symposium on Empirical Software Engineering and Measurement, Eero I. Laukkanen, Mika V. Mantyla]
Presents a survey of six industrial software development organizations about the defect report information (Seventy-four developers out of 142 completed the survey), from three viewpoints: concerning quality, usefulness and automation possibilities of the information. Link: https://courses.cs.ut.ee/MTAT.03.159/2015_spring/uploads/Main/SWT_bugrep2.pdf
