Comparison of Seven Bug Report Types: A Case-Study of ... - IIIT

or reporter (depending on the application) belonging to therespective bug-type. For the variable p, p i ≥ 0 and i=n∑ p i = 1.i=1The entropy value can vary from 0 (minimal) to 1 (maximum).If the probability of all components or reporters is thesame (same distribution) then the entropy is maximal. This isbecause the p i value is the same for all n. On the other extremeend, if there is only one component or reporter associated to aparticular bug type then the entropy becomes minimal (valueof 0). The interpretation is that when entropy is low for aspecific bug type then it means a small set of components orreporters are associated with the specific bug type.Table III displays the component and reporter entropy andthe frequency values for all seven types of bug reports. Wenotice that not every bug report mentions the component name.We were not able to extract the component and reporter namefor all the bug reports in the experimental dataset and TableIII shows results for the bug reports for which we were ableto extract the component and reporter names (frequency ismentioned in brackets). The component entropy for crash andsecurity bug is 0.552 and 0.577 respectively which is relativelylower than the component entropy for performance (0.777)and usability (0.764) bugs. We notice that for the securitybug type, 242 out of 330 reports belong to only 2 out of 13components. There are 13 unique components assigned to 330security bug reports and 73.3% of the bugs (bug reports) areoriginating from only 2 out of 13 components. For the crashbug report type, we notice that 2617 (89.5%) out of 2922bug reports belong to 3 out of 14 components. Performancebug reports are distributed across multiple components: Internals(68/155), WebKit (36/155), UI (17/155), BrowserUI(15/155), Build (14/155), Misc (4/155) and Feature (1/155).The reporter entropy (0.781) for security bugs is relativelylower than the other types of bugs because we observe three reporters:security...@gtempaccount.com, cev...@chromium.organd aba...@chromium.org reported 48, 61 and 22 (out of 365bug reports for which we are able to extract the reporter name)bugs respectively.Khomh et al. [6] define the entopy graph for triaging crashbugs, we apply same concept in our study to better understandthe similarities and difference between the characteristics ofvarious types of bug reports. Figure 1 presents the componentand reporter entropy graph derived from Table III. The x-axis represents the frequency (or the number) bracket andy-axis represents the entropy bracket. As shown in Figure1, performance and usability bugs fall into the categoryof relatively high entropy and relatively low number ofincidents. We observe that regression and crash bugs havehigh frequency but fall in the middle bracket of reporterentropy. We believe that the entropy graph presented in Figure1 can be used by the project team to understand that impactof each type of bug report across both the dimensions offrequency and coverage (distribution across components anddistribution across reporters).B. Bug Fixing Process Quality and Performance Characteristics1) Bug Opening and Closing Trend and Continuity: Francalanciet al. [9] present an analysis of the performancecharacteristics (such as continuity and efficiency) of the bugfixing process. They identify performance indicators (bugopening and closing trend) reflecting the characteristics andquality of bug fixing process. We apply the concepts presentedby Francalanci et al. [9] in our study. They define bug openingtrend as the cumulated number of opened and verified bugsover time. In their paper, closing trend is defined as thecumulated number of bugs that are resolved and closed overtime. We plot the opening and closing trend for varioustypes of bugs on a graph and investigate the similarities anddifferences in their characteristics. Figure 2, 3 and 4 displaysthe opening and closing trend for crash, cleanup and securitybug reports and Figure 5, 6 and 7 displays the trends forperformance, polish and regression bug reports. At any instantof time, the difference between the two curves (interval) canbe computed to identify the number of bugs which are openat that instant of time. We notice that the debugging processis of high quality as there is no uncontrolled growth ofunresolved bugs (the curve for the closing trend grows nearlyas fast or has the same slope as the curve for the opening trend)across all bug types [9] [10].All opening and closing trend graphs are plotted on the samescale and hence the differences between their characteristicsare visible. We see a noticeable and visible difference betweenthe trends for crash and regression bug reports in contrast toother types. The slope for the crash and regression curves isrelatively steep in comparison to other curves. It is interestingto note that for security bugs the opening and closing trendcurves almost overlap and this shows that in general thenumber of security bugs open at any instance of time isvery small. We notice that for regression bugs, the curveis steep (demonstrating a relatively large number of incomingand closed bugs) and yet the interval between the openingand closing curves is less (both the curves are near to eachother). The graph for usability bug is not plotted due to limitedspace in the paper but we observe the same characteristics forusability bugs as exhibited by performance and polish bugs.2) Mean Time to Repair and Release Date: We studymean time to repair (MTTR) the bug to understand qualityof bug fixing process among different categories. MTTR iscomputed as the amount of time required to close a bug(difference between the bug reporting timestamp and bugclosing timestamp). The metric MTTR is measured in terms ofnumber of hours. Results in Table IV indicates that amongstthe seven bug types, in general MTTR of crash bug reportsis lowest and that of performance and usability bugs is veryhigh. Even though the number of crash and regression bugreports are relatively higher in comparison to other typesof bug reports (refer to Table II), the close time of crashand regression bug reports is relatively less (refer to TableIV). There is a noticeable difference and clear separation of520