Qualitative methods for analyzing process data How do we research ...

Qualitative methods foranalyzing process dataAnn Langley, HEC MontréalAOM PDW, HonoluluSaturday August 6th 2005How do we researchprocesses?Simple answer: we go out and look• Observations in vivo(meetings; conversations; events; shadowing; etc.)• Collecting memories and interpretations(interviews; diaries; focus groups; questionnaires)• Collecting temporally-embedded artifacts(minutes of meetings; plans; reports; etc.)Emphasis on qualitative data1

But then what?…Process data (after Mintzberg, 1979)Greatest challenge of processresearch• "Making sense" of the data• while respecting the complexity ("accuracy")• as simply as possible ("parsimony")• with wide applicability ("generality")(Criteria for good theory suggested by Weick,1979)2

Seven strategies for theorizingfrom process data…1. Narrative strategy2. QuantificationKevin's presentation3. Alternate templates4. Grounded theory5. Visual mapping6. Temporal decomposition7. Comparative caseLangley, AMR, 1999Strategy 1: Narrative• Tell the story ("Just write it up…")• Examples: Chandler (history); Geertz(anthropology – "thick description")• Good narrative creates a sense of "déjà-vu"• Priority on accuracy above parsimony andgenerality• "To be determinate, we must be indeterminate"(Van Maanen, 1996)3

Laurel Richardson inHandbook of Qualitative Reseach• I have a confession to make. For 30years, I have yawned my way throughnumerous supposedly exemplaryqualitative studies… (…) [this is]qualitative research's own dirty littlesecret. Our empire is (partially)unclothed.Narrative approach…Have we made any progress?4

My experience….• Description as a first step• Writing as a stimulus for theorizing• The construction of a "narrative"incorporating data and theory as a laststep:• possible only when one has identifiedthe plot (the theoretical angle)More exciting ways to usenarrative approaches• Investigating the narratives producedby organization members – narrative asa unit of analysis• Presenting narratives that draw ondifferent theoretical lenses (see strategy3….)• Literary approaches5

Example: Analyzing identitynarratives during mergers• Alberta / Quebec health care mergers• Narratives of different groups as they livethrough identity threats• Generating summary narratives by group• Identifying key components of narrative:• Change in context• Perceived identity threat• Coping mechanisms (ours, others)• Evolving identity patterns• Comparing narratives, identifying typicalsequences and their driversSeven strategies for theorizingfrom process data…1. Narrative strategy2. Quantification3. Alternate templates4. Grounded theory5. Visual mapping6. Temporal decomposition7. Comparative caseLangley, AMR, 19996

Strategy 3: Alternate templates• Fit different a priori theoretical framesto data• Classic example: Allison, 1971 TheEssence of Decision (Cuban MissileCrisis)• Accuracy, parsimony and generalitycreated through the cumulation ofperspectivesExample: Markus (1983)• Why do people resist informationsystems?• People theory: it’s a question of individuals’cognitive styles• Technological theory: people resistbecause systems are not user-friendly• Political theory: people resist because theyfear loss of power and control7

Alternate templates strategyCompleteness through complementary modelsParsimony by keeping them separate!My experience• Very attractive strategy for study of asingle case• Harder than it looks to keep templatesseparate• The problem of integrating the variousmodels…8

Seven strategies for theorizingfrom process data…1. Narrative strategy2. Quantification3. Alternate templates4. Grounded theory5. Visual mapping6. Temporal decomposition7. Comparative caseLangley, AMR, 1999Strategy 4: Grounded theory• Build theory bottom-up (no a priori frame)• Glaser & Strauss (1967); Corbin & Strauss(1990)• Exemplar paper:• Gioia, Thomas, Clark and Chittipeddi: Symbolismand strategic change in academia, OrganizationScience, 1994• ATLAS/ti or N'Vivo as modern coding tools• Stresses accuracy above parsimony andgenerality9

Doing a qualitative doctoralthesis with ATLAS/ti (example)• P 2: InterviewJFH.txt - 2:2 (523:525) (Super)• Codes: [solitude]• I see my entourage as helpful because they didn't hurt. You are very alone in a processlike that, people don't know that and they don't understand.• P 3: InterviewCL.txt - 3:2 (1154:1159) (Super)• Codes: [solitude]• I do qualitative work now. But I do it with a team of 7 or 8people with me. But what I did for my thesis was not like that. You're all alone. Doingqualitative research is very hard. Miles and Huberman – they were in a team.• P 4: InterviewMEBB.txt - 4:3 (1145:1151) (Super)• Codes: [solitude]• And probably, that's when I came closest to dumpingeverything into the garbage. I felt really alone! OK, my colleague here helped a bit, but infact, you are all alone… you are all alone! The transition from the field to after the field wasvery very difficult.10

FromGioia etal.(detail)Grounded theory strategyDescription built up from the trees, not down from forest11

My experience• Requires patient and laborious micro-analysisbut potentially powerful…• Software (ATLAS/ti, N'VIVO) helps, butincreases the temptation to overcode• Tends to produce a hierarchical andsomewhat similar theoretical frames• Code a bit to explore the data at first; getinsights; complete coding to verify afterwardsSeven strategies for theorizingfrom process data…1. Narrative strategy2. Quantification3. Alternate templates4. Grounded theory5. Visual mapping6. Temporal decomposition7. Comparative caseLangley, AMR, 199912

Strategy 5: Visual mapping• A picture is worth a thousand words (?)• Graphs, drawings, diagrams• Temporal flow charts• Relationships between people• Tables and matrices• Methods gurus: Miles and Huberman• A favorite thinking tool13

Progression inleadershipteamsDenis et al.,2001, AMJMy experience• A parsimonious way to communicate alot of information• …sometimes clearer for the creatorthan the reader!• Need to get beyond the surface"patterns" to understand explanatorymechanisms –otherwise it's just anotherkind of description14

Visual mapping strategy…A metaphor with limitations!Seven strategies for theorizingfrom process data…1. Narrative strategy2. Quantification3. Alternate templates4. Grounded theory5. Visual mapping6. Temporal decomposition7. Comparative caseLangley, AMR, 199915

Strategy 6: Temporaldecomposition• Decompose a process into "phases"• Continuous episodes separated by discontinuities• Phases may or may not have conceptualsignificance…• … but become units of analysis forcomparison across time• Example: Barley (1986) "Technology as anoccasion for structuring," ASQ• …Another favoriteExample: Leadership andchange under ambiguityStartingpointPhase 1Phase 2Phase 3Phase 4Phase 5LeadershipteamcohesionSymbolicchangeSubstantivechangePoliticalchangecredibilityof leadersCEO 1MED 1CEO 2OP MGRMED 216

Iterative model of leadership andstrategic change under ambiguityPhase 1, 2, 3…Leadershipteam•Cohesion•Internal support•External supportTactics•Transparency•Internal vs.externalorientationEffects•Substantive•Symbolic•PoliticalDenis, Lamothe, Langley, AMJ, 2001Barley: Evolution of interactionsassociated with adoption of CT scanner• Hospital 1: coding of conversations betweenradiologists and technicians during examinations• Phase 1: mutual exploration• Unsought validation• Anticipatory questions• Preference stating• Phase 2: technician takeover• Clandestine teaching• Role reversal• Blaming the technologist17

Barley’s analysisTemporal decompositionstrategyNot all processes are easily decomposed18

My experience• A priori, single cases are risky withoutrepetitive elements!• Temporal decomposition allows theorizingfrom a single case• The "phase" becomes an element ofcomparison• This requires a context that can easily bedecomposed…• …and an iterative theoretical structureSeven strategies for theorizingfrom process data…1. Narrative strategy2. Quantification3. Alternate templates4. Grounded theory5. Visual mapping6. Temporal decomposition7. Comparative caseLangley, AMR, 199919

Strategy 7:Comparative case strategy• Identify process characteristicsassociated with extreme outcomes(success, failure)• Exemplar: Eisenhardt (AMJ, 1989):"Making fast decisions in high velocityenvironments" – 8 cases of decisions incomputer firms• Emphasis on parsimony and generalityComparative strategy"Success""Failure"How are the processes different?(Compress processes into variables)20

Table from Eisenhardt (1989)Eisenhardt’s model21

My experience• Powerful approach• Not as easy as it looks…• Successes and failures are not alwaysso easily distinguised• Loss of temporal elements (produces avariance rather than a process model)Seven sense-making strategies incombinationGroundedTheoryNarrativeTemporalbracketingQuantificationAlternateTemplatesVisualMappingSyntheticstrategyGrounding strategies Organizing strategies Replicating strategies22

General observations: Sensemakingfrom process data (1)Type of understanding obtained varies withsense-making strategy…• Interpretations: "sense" of participants• Narrative strategy; grounded theory• Predictions: "sense" of causal laws• Comparative strategy; quantitative strategy• Patterns: "sense" of surface structure• Visual mapping• Mechanisms: "sense" of driving forces• Narrative strategy, alternate templates, temporaldecompositionGeneral observations: Sensemakingwith process data (2)• PREPARE to COMPARE…• Compare with a priori theories• Alternate templates, quantification• Compare with cases or sub-cases chosen for theirdifferences/ similarities• Comparative, visual mapping strategies• Compare different time periods within cases• Temporal decomposition• Compare micro-incidents within a case• Grounded theory, narrative, quantification23

Barley 1990General observations: Sensemakingwith process data (3)Creative inspiration plays a key role• But creativity favours the prepared mindSo…• Collect good data• Analyse data from multiple angles• Build on prior "tacit" knowledgeAnd remember: Rigorous data analysis isneeded to verify emerging ideas24

Other useful notes onqualitative analysisReviewer comments…… the problem of "credibility"• "I was troubled throughout themanuscript by the question of whetherto "believe" the account. I do believethe author, but the point is that the waythe results are presented leaves theauthor open to this criticism…."25

Issues of "credibility" duringqualitative data collection+Access tosituationsRapport withpeople+Proximity tophenomenon+Risk ofcontamination(effect ofthe researcheron thephenomenon)To attenuate the risks• Multiple sources• Multiple researchers• Insider/ outsider / peer review• Make biases explicitRisk of"going native"(becomingsocialized tothe point ofnot seeingthings)Risk ofpoliticalalignment(loss ofneutrality)+-Qualityof dataIssues of "credibility" during dataanalysis & interpretationRichnessand depth ofqualitativedata+Saturation andauthenticity+Credibilityofinterpretation+Ambiguity ininterpretation-+-To attenuate the concernsExhaustive documentation of analysisMember check: validation with informantsMultiple researchers (double coding)Triangulate with quantitative dataField sites named?26

Different responses to the issue ofgeneralizability in qualitative research• Transferability: The important thing is to providesufficient in-depth contextual information so that thereader can judge transferability to another situation• Accent on analytic generalizability (to theory) ratherthan empirical generalizability (to population). Eachcase provides a complete test of a theory (much likean experiment)• Replication on polar cases – if thesamephenomenonoccurs in very different situations, it is more likely tobe general• Generalizability is not relevant – qualitative researchis interested in the particular, not the general27