References - 9Lange, Douglas S., Phillip Verbancsics, Robert S. Gutzwiller, John Reeder, and CullenSarles. "Command and Control of Teams of Autonomous <strong>Systems</strong>." In <strong>Large</strong>-<strong>Scale</strong>Complex IT <strong>Systems</strong>. Development, Operation and Management, pp. 81-93. SpringerBerlin Heidelberg, 2012.Lesbegueries, Julien, Amira Ben Hamida, Nicolas Salatgé, Sarah Zribi, and Jean-PierreLorré. "Multilevel event-based monitoring framework for <strong>the</strong> petals enterprise servicebus: industry article." In Proceedings of <strong>the</strong> 6th ACM International Conference onDistributed Event-Based <strong>Systems</strong>, pp. 48-57. ACM, 2012.Liang, Peng, Paris Avgeriou, Keqing He, and Lai Xu. "From collective knowledge tointelligence: pre-requirements analysis of large and complex systems." In Proceedingsof <strong>the</strong> 1st Workshop on Web 2.0 for Software Engineering, pp. 26-30. ACM, 2010.Lim, Soo Ling, and Anthony Finkelstein. "StakeRare: using social networks andcollaborative filtering for large-scale requirements elicitation." Software Engineering,IEEE Transactions on 38, no. 3 (2012): 707-735.Litoiu, Marin, Murray Woodside, Johnny Wong, Joanna Ng, and Gabriel Iszlai. "A businessdriven cloud optimization architecture." In Proceedings of <strong>the</strong> 2010 ACM Symposiumon Applied Computing, pp. 380-385. ACM, 2010.Does <strong>Scale</strong> Really Matter?: ULS <strong>Systems</strong> <strong>Seven</strong> <strong>Years</strong> LaterLinda Northrop: May 24, <strong>2013</strong>© <strong>2013</strong> Carnegie Mellon University94
References - 10Lucrédio, Daniel, Ethan K. Jackson, and Wolfram Schulte. "Playing with fire: Harnessing<strong>the</strong> hottest technologies for ultra-large-scale systems." In 15th Monterey Workshop-Foundations of Computer Software, Future Trends and Techniques for Development.2008.McGowan, Julie J., Caitlin M. Cusack, and Meryl Bloomrosen. "The future of health ITinnovation and informatics: a report from AMIA's 2010 policy meeting." Journal of <strong>the</strong>American Medical Informatics Association 19, no. 3 (2012): 460-467.Mirakhorli, Mehdi, Amir Azim Sharifloo, and Fereidoon Shams. "Architectural Challengesof <strong>Ultra</strong> <strong>Large</strong> <strong>Scale</strong> <strong>Systems</strong>." In Proceedings of <strong>the</strong> 2nd International Workshop on<strong>Ultra</strong>-large-scale Software-intensive <strong>Systems</strong>, pp. 45-48. ACM, 2008.Müller, Hausi, Mauro Pezzè, and Mary Shaw. "Visibility of control in adaptive systems." InSecond International Workshop on <strong>Ultra</strong>-<strong>Large</strong>-<strong>Scale</strong> Software-Intensive <strong>Systems</strong>(ULSSIS 2008), <strong>ICSE</strong> 2008 Workshop. 2008.Musil, Juergen, Angelika Musil, Dietmar Winkler, and Stefan Biffl. "A first account onstigmergic information systems and <strong>the</strong>ir impact on platform development." InProceedings of <strong>the</strong> WICSA/ECSA 2012 Companion Volume, pp. 69-73. ACM, 2012.Does <strong>Scale</strong> Really Matter?: ULS <strong>Systems</strong> <strong>Seven</strong> <strong>Years</strong> LaterLinda Northrop: May 24, <strong>2013</strong>© <strong>2013</strong> Carnegie Mellon University95
- Page 2 and 3:
Software Engineering Institute (SEI
- Page 4 and 5:
Ultra-Large-Scale (ULS)SystemsDoes
- Page 6 and 7:
Beginning of theULS System JourneyD
- Page 8 and 9:
Societal ProblemsClimate change and
- Page 10 and 11:
Trend Toward Increasing Scale-1•
- Page 12 and 13:
Increasing Scale In Military System
- Page 14 and 15:
Expert PanelGregory AbowdGeorgia In
- Page 16 and 17:
Inspiration: Open Source and Cooper
- Page 18 and 19:
ULS Systems Research Study Reportht
- Page 20 and 21:
What Is an Ultra-Large-Scale (ULS)
- Page 22 and 23:
Approaches to Software DevelopmentT
- Page 24 and 25:
Analogies are UsefulDoes Scale Real
- Page 26 and 27:
Think EcosystemDiverse users with c
- Page 28 and 29:
ChallengesULS systems will present
- Page 30 and 31:
What We LearnedThere is an unstoppa
- Page 32 and 33:
Early Post-Study Observations• We
- Page 34 and 35:
Since ThenDemonstrators in Cairo's
- Page 36 and 37:
Software is Ubiquitous and Often Tr
- Page 38 and 39:
More Fuel for ScaleDoes Scale Reall
- Page 40 and 41:
The “Crowd”DARPA BAA 11-64: Soc
- Page 42 and 43:
Some ULS Systems Buzz~21,266 downlo
- Page 44: Upon ReflectionSTE(Social-Technical
- Page 47 and 48: Selected Experiences withSystems at
- Page 49 and 50: Smart Grid - A ULS SystemDiagrams c
- Page 51 and 52: Broader ULS System ImpactThe expans
- Page 53 and 54: XSEDE’s innovative, open standard
- Page 55 and 56: Healthcare AnalyticsProblem: Define
- Page 57 and 58: Adding AnalyticsBusiness andMission
- Page 59 and 60: Best in Class Healthcare AnalyticsO
- Page 61 and 62: Sample Published WorkContextual Des
- Page 63 and 64: Selected SEI ResearchTargeted at UL
- Page 65 and 66: Architecture in ULS System ContextU
- Page 67 and 68: SEI and Broader Carnegie Mellon Col
- Page 69 and 70: So, Where Are We?The report has bee
- Page 71 and 72: Opportunities and ThreatsOpportunit
- Page 73 and 74: Putting Technology to Work: a Few T
- Page 75 and 76: What I SeeMachine LearningMulticore
- Page 77 and 78: Food for Thought• Is our research
- Page 79 and 80: Thanks To The Entire ULS System Stu
- Page 81 and 82: A Special Thank You and TributeDoes
- Page 83 and 84: InteractionDoes Scale Really Matter
- Page 85 and 86: References for Slides 65-66Does Sca
- Page 87 and 88: References - 2Bryant, Barrett R., J
- Page 89 and 90: References - 4Clements, Paul, and M
- Page 91 and 92: References - 6Froihofer, Lorenz, Ge
- Page 93: References - 8Hill, James H., Jules
- Page 97 and 98: References - 12Salehie, Mazeiar, an
- Page 99 and 100: References - 14Valerdi, Ricardo, El
- Page 101 and 102: References - 16Zhu, Liming, and Yan