Comparison of Seven Bug Report Types: A Case-Study of ... - IIIT

More documents

Recommendations

Info

Fig. 8. Bug Type: Crash Fig. 9. Bug Type: Cleanup Fig. 10. Bug Type: SecurityFig. 11. Bug Type: Performance Fig. 12. Bug Type: Usability Fig. 13. Bug Type: RegressionTABLE VEXAMPLES OF BUG TYPES FROM GOOGLE CHROMIUM BROWSER PROJECT (CRASH, PERFORMANCE, USABILITY)Type Issue ID Text Snippet from Bug ReportCrash 24200 It is currently ranked #1 (based on the relative number of reports in the release). There have been68 reports from 50 clients.Crash 25823 the last step crashed on 4 timesCrash 36951 is overflowing with the below crash. There are about 60 crashes almost all of whichCrash 40472 Chrome crashed for more than 5 times with same stack traceCrash 101544 It crashes every time in less than a minutePerformance 104 Slow scrollingPerformance 3066 CPU maxes out, scrolling is laggyPerformance 11341 tends to load some pages slightly slower thanPerformance 32040 The script takes much longer to load and parse when there is more HTML on the page before itUsability 25767 If other people are also having trouble seeing this button, maybe it should be renamed or highlightedsomehow?Usability 83939 in many common Zoom use cases, the interface remains difficult to see and/or and interact withUsability 92027 feature must be introduced in Chromium (and Chrome Browser & OS) to improve multilingualismUsability 114402 we need to make some decisions about the use of this fontreport. Tables V and VI displays text snippets from some ofthe bug reports belonging to various categories. Some of theterms indicating the bug type (for example terms like slow,lag, longer for performance bugs and risk, malicious, exploitand corruption for security bugs) are marked as bold (referto Tables V and VI). We conduct experiments to identifyterms that are frequent or unique (linguistic indicators) toeach of the seven bug category. We extract all unique terms(corpus vocabulary) from titles and description of all bugreports present in our experimental dataset. We calculateprobability of each unique term corresponding to each bugcategory, using Naive Bayes classifier 10 . Table VII shows Top20 terms for each bug category sorted by their probability ofoccurrence in the respective category. Manual inspection of10 http://alias-i.com/lingpipe/docs/api/com/aliasi/classify/NaiveBayesClassifier.htmlTable VII clearly shows correlation between certain terms andbug category. For example, for crash, regression and securitybugs: “crash”, “regress”, and “secur” are highest probabilityterms respectively. Performance bug reports have terms such“perf”, “slow”, “time” and “cpu” as frequently occurring (system’sperformance indicator). Similarly, usability bug reportshave most frequent terms such as “window”, “user”, “zoom”,“menu” and “click” which captures the notion of user interfaceissues. Our experiment shows that words present in bugreport title and description are not random and we seeassociation between terms and bug types.We calculate the description length (number of characterspresent in the description) for each bug report in the experimentaldataset. Table VIII, displays the five-number summarystatistics for description length across seven different types ofbug reports. The data in Table VIII reveals that the descriptionlength for crash and security bug is the highest whereas522
TABLE VIEXAMPLES OF BUG TYPES FROM GOOGLE CHROMIUM BROWSER PROJECT (POLISH, SECURITY, CLEANUP, REGRESSION)Type Issue ID Text Snippet from Bug ReportPolish 94114 Several infobars are lacking punctuationPolish 51429 Bookmarks bubble (and extensions installed bubble) should have white backgroundPolish 89591 Misaligned text in HTML file input controlSecurity 1208 puts users at more risk from websites trying to spoof a file’s typeSecurity 23693 if the user is duped into clicking ”Create application shortcut” on a malicious pageSecurity 23979 We consider some file extensions to be maliciousSecurity 24733 because a given page can deterministically take down the whole browserSecurity 74665 This can lead to memory corruption and is probably exploitable too if you are clevercleanup 88098 Use javascript objects to make the ui code cleaner and easier to modifycleanup 30151 Toolstrip is obsolete, and these tests should be removed when toolstrip is removedcleanup 68882 src/gfx uses wstring all over. It should use string16Regression 115321 Tab crashes when coming out of the Full screen for the Sublime videosRegression 115934 Info bar offers to save a wrong passwordTABLE VIIFREQUENT TERMS[CHROME-BROWSER]Top K CRASH REGR. SECUR. CLEAN POLISH PERF. USAB.1 crash regress secur should should regress page2 chrome crash crash remov bookmark page window3 report page chrome test button mac user4 stack tab us us window perf chrome5 signatur chrome memori code text time should6 webcor window corrupt move tab sync zoom7 browser bookmark file chrome menu test open8 const doesnt audit webkit bar releas doe9 Tab browser browser api dialog cycler manag10 Int open webcor need mac startup menu11 render work page browser page slow bookmark12 std render access content chrome tab provid13 mac broken window add need linux url14 page new webkit base icon new line15 ... text free clean drag veri dialog16 googl bar sandbox extens us chrome select17 intern show render refactor manag load command18 open button url view open us click19 char click bypass file select theme us20 unsign std open renam close cpu pluginTABLE VIIIDESCRIPTION LENGTHCRASH REGR. SECUR. CLEAN POLISH PERF. USAB.Min 7 5 22 3 11 54 64Q1 556 350 328 117.5 234.75 304.25 404Median 1021 550 743 225 396.5 537.5 593Q3 2909 859 1436 422.5 603 899.5 876Max 20057 19875 16233 9551 1837 3719 2886the description length of cleanup bug reports is the lowest.Bug reports belonging to crash types mostly consists of systemgenerated crash reports and hence size of description is largeas compare to other bug types where users have to provideentire content manually.D. Correlation Between Regression and other Bug TypesA regression bug (one of the seven types of bug reportsin the evaluation dataset) can also belong to other categories(such as crash, polish and performance). We conduct experimentsto identify relation between regression bugs and othertypes of bugs. The motivation behind the experiment is toidentify bug-types which are more prone to cause regressionbugs. We extract all bug-type labels present in regression bugreports and then compute correlations. Table IX shows resultsof our experiment. Table IX reveals relatively high correlationbetween regression bugs and crash bugs (nearly 7% of all theregression bugs are crash bugs), however manual observationof 100 bugs (fixed/verified) shows different correlation. Wediscover that out of 100 regression bugs only 16 bugs belongto crash type and 43 bugs belong to the usability type. Furthermore,there are 31 bugs which can be categorized to anyof usability, polish, or cleanup (based on our interpretation ofbug description/discussion). We believe that a large number523
Page 1 and 2: 2012 19th Asia-Pacific Software Eng
Page 3 and 4: TABLE IIICOMPONENT AND REPORTER ENT
Page 8 and 9: TABLE IXPERCENTAGE OF BUGS IN EACH
Page 10: version cycles. As shown in Table X

Comparison of Seven Bug Report Types: A Case-Study of ... - IIIT

Create successful ePaper yourself

Delete template?

Save as template?