business intelligence and analytics: from big data to big impact

Chen et al./Introduction: Business Intelligence Researchadvances such as neural networks for classification/predictionand clustering and genetic algorithms for optimization andmachine learning have all contributed to the success of datamining in different applications.Two other data analytics approaches commonly taught inbusiness school are also critical for BI&A. Grounded instatistical theories and models, multivariate statistical analysiscovers analytical techniques such as regression, factor analysis,clustering, and discriminant analysis that have been usedsuccessfully in various business applications. Developed inthe management science community, optimization techniquesand heuristic search are also suitable for selected BI&A problemssuch as database feature selection and web crawling/spidering. Most of these techniques can be found in businessschool curricula.Due to the success achieved collectively by the data miningand statistical analysis community, data analytics continues tobe an active area of research. Statistical machine learning,often based on well-grounded mathematical models andpowerful algorithms, techniques such as Bayesian networks,Hidden Markov models, support vector machine, reinforcementlearning, and ensemble models, have been applied todata, text, and web analytics applications. Other new dataanalytics techniques explore and leverage unique data characteristics,from sequential/temporal mining and spatial mining,to data mining for high-speed data streams and sensor data.Increased privacy concerns in various e-commerce, e-government, and healthcare applications have caused privacypreservingdata mining to become an emerging area ofresearch. Many of these methods are data-driven, relying onvarious anonymization techniques, while others are processdriven,defining how data can be accessed and used (Gelfand2011/ 2012). Over the past decade, process mining has alsoemerged as a new research field that focuses on the analysisof processes using event data. Process mining has becomepossible due to the availability of event logs in variousindustries (e.g., healthcare, supply chains) and new processdiscovery and conformance checking techniques (van derAalst 2012). Furthermore, network data and web content havehelped generate exciting research in network analytics andweb analytics, which are presented below.In addition to active academic research on data analytics,industry research and development has also generated muchexcitement, especially with respect to big data analytics forsemi-structured content. Unlike the structured data that canbe handled repeatedly through a RDBMS, semi-structureddata may call for ad hoc and one-time extraction, parsing,processing, indexing, and analytics in a scalable and distributedMapReduce or Hadoop environment. MapReducehas been hailed as a revolutionary new platform for largescale,massively parallel data access (Patterson 2008).Inspired in part by MapReduce, Hadoop provides a Javabasedsoftware framework for distributed processing of dataintensivetransformation and analytics. The top three commercialdatabase suppliers—Oracle, IBM, and Microsoft—have all adopted Hadoop, some within a cloud infrastructure.The open source Apache Hadoop has also gained significanttraction for business analytics, including Chukwa for datacollection, HBase for distributed data storage, Hive for datasummarization and ad hoc querying, and Mahout for datamining (Henschen 2011). In their perspective paper, Stonebrakeret al. (2010) compared MapReduce with the parallelDBMS. The commercial parallel DBMS showed clear advantagesin efficient query processing and high-level querylanguage and interface, whereas MapReduce excelled in ETLand analytics for “read only” semi-structured data sets. NewHadoop- and MapReduce-based systems have becomeanother viable option for big data analytics in addition to thecommercial systems developed for RDBMS, column-basedDBMS, in-memory DBMS, and parallel DBMS (Chaudhuriet al. 2011).Text AnalyticsA significant portion of the unstructured content collected byan organization is in textual format, from e-mail communicationand corporate documents to web pages and socialmedia content. Text analytics has its academic roots ininformation retrieval and computational linguistics. In informationretrieval, document representation and query processingare the foundations for developing the vector-spacemodel, Boolean retrieval model, and probabilistic retrievalmodel, which in turn, became the basis for the modern digitallibraries, search engines, and enterprise search systems(Salton 1989). In computational linguistics, statistical naturallanguage processing (NLP) techniques for lexical acquisition,word sense disambiguation, part-of-speech-tagging (POST),and probabilistic context-free grammars have also becomeimportant for representing text (Manning and Schütze 1999).In addition to document and query representations, usermodels and relevance feedback are also important inenhancing search performance.Since the early 1990s, search engines have evolved intomature commercial systems, consisting of fast, distributedcrawling; efficient inverted indexing; inlink-based pageranking; and search logs analytics. Many of these foundationaltext processing and indexing techniques have beendeployed in text-based enterprise search and documentmanagement systems in BI&A 1.0.MIS Quarterly Vol. 36 No. 4/December 2012 11