References - 13Shams, Fereidoon, Amir Azim Sharifloo, Mehdi Mirakhorli, and Mostafa Emaeli. "A ServiceDriven Development Process (sddp) Model for <strong>Ultra</strong>-<strong>Large</strong>-<strong>Scale</strong> <strong>Systems</strong>." InProceedings of <strong>the</strong> 2nd international workshop on <strong>Ultra</strong>-large-scale software-intensivesystems, pp. 37-40. ACM, 2008.Shaw, Mary. "Continuing Prospects for an Engineering Discipline of Software." Software,IEEE 26, no. 6 (2009): 64-67.Sim, Susan Elliott, Medha Umarji, Sukanya Ratanotayanon, and Cristina V. Lopes. "Howwell do search engines support code retrieval on <strong>the</strong> web?." ACM Transactions onSoftware Engineering and Methodology (TOSEM) 21, no. 1 (2011): 4.Sommerville, Ian, Dave Cliff, Radu Calinescu, Justin Keen, Tim Kelly, Marta Kwiatkowska,John Mcdermid, and Richard Paige. "<strong>Large</strong>-scale Complex IT <strong>Systems</strong>."Communications of <strong>the</strong> ACM 55, no. 7 (2012): 71-77.Stamkopoulos, Konstantinos, Evaggelia Pitoura, Panos Vassiliadis, and Apostolos Zarras."Accelerating Web service workflow execution via intelligent allocation of services toservers." Journal of Database Management (JDM) 21, no. 4 (2010): 60-90.Sullivan, Kevin, William Knaus, and Richard Marks. "An <strong>Ultra</strong>-large-scale <strong>Systems</strong>Approach to National-scale Health Information <strong>Systems</strong>." In Proceedings of <strong>the</strong>FSE/SDP Workshop on Future of Software Engineering Research, pp. 365-368. ACM,2010.Does <strong>Scale</strong> Really Matter?: ULS <strong>Systems</strong> <strong>Seven</strong> <strong>Years</strong> LaterLinda Northrop: May 24, <strong>2013</strong>© <strong>2013</strong> Carnegie Mellon University98
References - 14Valerdi, Ricardo, Elliot Axelband, Thomas Baehren, Barry Boehm, Dave Dorenbos, ScottJackson, Azad Madni, Gerald Nadler, Paul Robitaille, and Stan Settles. "A ResearchAgenda for <strong>Systems</strong> of <strong>Systems</strong> Architecting." International Journal of System of<strong>Systems</strong> Engineering 1, no. 1 (2008): 171-188.von Hanxleden, Reinhard, Edward A. Lee, Christian Motika, and Hauke Fuhrmann. "MultiviewModeling and Pragmatics in 2020." In <strong>Large</strong>-<strong>Scale</strong> Complex IT <strong>Systems</strong>.Development, Operation and Management, pp. 209-223. Springer Berlin Heidelberg,2012.Wang, Hongchun, Keqing He, Bing Li, and Jinhu Lü. "On some recent advances incomplex software networks: Modeling, Analysis, Evolution and Applications."International Journal of Bifurcation and Chaos 22, no. 02 (2012).Wang, Jian, Keqing He, Ping Gong, Chong Wang, Rong Peng, and Bing Li. "RGPS: AUnified Requirements Meta-modeling Frame for Networked Software." In Proceedingsof <strong>the</strong> 3rd International Workshop on Applications and Advances of Problem Frames,pp. 29-35. ACM, 2008.Ward, Jonathan Stuart, and Adam Barker. "Semantic based data collection for large scalecloud systems." In Proceedings of <strong>the</strong> Fifth International Workshop on Data-IntensiveDistributed Computing Date, pp. 13-22. ACM, 2012.Does <strong>Scale</strong> Really Matter?: ULS <strong>Systems</strong> <strong>Seven</strong> <strong>Years</strong> LaterLinda Northrop: May 24, <strong>2013</strong>© <strong>2013</strong> Carnegie Mellon University99
- Page 2 and 3:
Software Engineering Institute (SEI
- Page 4 and 5:
Ultra-Large-Scale (ULS)SystemsDoes
- Page 6 and 7:
Beginning of theULS System JourneyD
- Page 8 and 9:
Societal ProblemsClimate change and
- Page 10 and 11:
Trend Toward Increasing Scale-1•
- Page 12 and 13:
Increasing Scale In Military System
- Page 14 and 15:
Expert PanelGregory AbowdGeorgia In
- Page 16 and 17:
Inspiration: Open Source and Cooper
- Page 18 and 19:
ULS Systems Research Study Reportht
- Page 20 and 21:
What Is an Ultra-Large-Scale (ULS)
- Page 22 and 23:
Approaches to Software DevelopmentT
- Page 24 and 25:
Analogies are UsefulDoes Scale Real
- Page 26 and 27:
Think EcosystemDiverse users with c
- Page 28 and 29:
ChallengesULS systems will present
- Page 30 and 31:
What We LearnedThere is an unstoppa
- Page 32 and 33:
Early Post-Study Observations• We
- Page 34 and 35:
Since ThenDemonstrators in Cairo's
- Page 36 and 37:
Software is Ubiquitous and Often Tr
- Page 38 and 39:
More Fuel for ScaleDoes Scale Reall
- Page 40 and 41:
The “Crowd”DARPA BAA 11-64: Soc
- Page 42 and 43:
Some ULS Systems Buzz~21,266 downlo
- Page 44:
Upon ReflectionSTE(Social-Technical
- Page 47 and 48: Selected Experiences withSystems at
- Page 49 and 50: Smart Grid - A ULS SystemDiagrams c
- Page 51 and 52: Broader ULS System ImpactThe expans
- Page 53 and 54: XSEDE’s innovative, open standard
- Page 55 and 56: Healthcare AnalyticsProblem: Define
- Page 57 and 58: Adding AnalyticsBusiness andMission
- Page 59 and 60: Best in Class Healthcare AnalyticsO
- Page 61 and 62: Sample Published WorkContextual Des
- Page 63 and 64: Selected SEI ResearchTargeted at UL
- Page 65 and 66: Architecture in ULS System ContextU
- Page 67 and 68: SEI and Broader Carnegie Mellon Col
- Page 69 and 70: So, Where Are We?The report has bee
- Page 71 and 72: Opportunities and ThreatsOpportunit
- Page 73 and 74: Putting Technology to Work: a Few T
- Page 75 and 76: What I SeeMachine LearningMulticore
- Page 77 and 78: Food for Thought• Is our research
- Page 79 and 80: Thanks To The Entire ULS System Stu
- Page 81 and 82: A Special Thank You and TributeDoes
- Page 83 and 84: InteractionDoes Scale Really Matter
- Page 85 and 86: References for Slides 65-66Does Sca
- Page 87 and 88: References - 2Bryant, Barrett R., J
- Page 89 and 90: References - 4Clements, Paul, and M
- Page 91 and 92: References - 6Froihofer, Lorenz, Ge
- Page 93 and 94: References - 8Hill, James H., Jules
- Page 95 and 96: References - 10Lucrédio, Daniel, E
- Page 97: References - 12Salehie, Mazeiar, an
- Page 101 and 102: References - 16Zhu, Liming, and Yan