Download - The Bastards Book of Regular Expressions

Data Cleaning with the StarsAnyone can do smart analyses when the data is nicely organized and delivered in a spreadsheet.But when it’s just straight text, that’s where regexes separate the resourceful from the poor chumpswho try to clean up the text by hand, or give up all together.This chapter contains a few examples of dirty data and how regexes can get them ready for thespreadsheet.Normalized alphabetical titlesEven alphabetical-sorting can be a pain, if you’re dealing with titles. The definite and indefinitearticles – “the” and “a” or “an”, respectively – will give us a disproportionate number of titles in the“A” and “T” part of the alphabet:AA BEAUTIFUL MINDA FISTFUL OF DOLLARSAN AMERICAN IN PARISSTAR TREK II: THE WRATH OF KHANTHE APARTMENTTHE BIG LEBOWSKITHE GODFATHERTHE GOOD, THE BAD, AND THE UGLYTHE GREAT GATSBYTHE KING AND ITHE LORD OF THE RINGS: THE RETURN OF THE KINGTHE SHAWSHANK REDEMPTIONTHE WRESTLERIn order for us to sort these titles alphabetically, we need to remove definite and indefinite articles²⁵from the titles, e.g. “THE” and “A/AN”.A simple find-and-delete for the word THE won’t work:LORD OF RINGS: RETURN OF KINGWe need to remove the articles only if they appear at the beginning of the title. And, moreover, weneed to append them to the end of the title, with a comma:²⁵http://en.wikipedia.org/wiki/Article_(grammar)129