Visual Programming: Cooking the Spaghetti - GOTO Conferences

Visual Programming:
Cooking the Spaghetti
GOTO Chicago '13
Rick Lane
rick.lane@tradingtechnologies.com
@R1ck_L4n3 @trading_tech @GOTOChgo

First things first
● Visual Programming < Regular Programming (usually)
● But, there's a time and a place
● How to make it work, when it makes sense to work

Who I am
● Chief technology officer at Trading Technologies
○ TT makes software for professional traders around
the world
● Background in wargaming: terrorist
network modeling and analysis, agentbased
modeling, etc.
● Many similarities between these
industries: a knowledge gap between the
idea generators and the technologists

Who I am not
Lakin, Fred, Visual Grammars for Visual Languages, Proceedings of the American Association for Artificial Intelligence,
Seattle, Washington, July 1987.
Keywords and Phrases: visual grammar notation, declarative programming, visual rules, formal visual languages, informal
conversational graphics, visual lisp, visual emacs, vmacs, VPL-IV.C: Parsing, VPL-VI.A: Formal definition of VPLs, VPL-II.
A.7: Logic languages
Pandey, Rajeev and Margaret Burnett, Is It Easier to Write Matrix Manipulation Programs Visually or Textually? An
Empirical Study, 1993 IEEE Symposium on Visual Languages, Bergen, Norway, August 24-27, 1993.
Keywords and Phrases: Forms/3, VPL-II.A.4: Form-based and spreadsheet-based languages, VPL-VI.C.1: Cognitive and
user-interface design issues
Burnett, Margaret M. and Allen L. Ambler, Generalizing Event Detection and Response in Visual Programming Languages,
1992 Proceedings of Advanced Visual Interfaces International Workshop, Rome, Italy, May 27-29, 1992.
Keywords and Phrases: Forms/3, VPL-II.A.4: Form-based and spreadsheet-based languages, VPL-III.A: Abstraction, VPL-
III.E: Event handling
● I am not a visual programming academic
● There is a sea of research on the various forms,
families, perils and merits of visual programming
○ You won’t find any of that here
Meyer, Bernd, Beyond Icons: Towards New Metaphors for Visual Query Languages for Spatial Information Systems,
Proceedings of the International Workshop on Interfaces to Database Systems, Glasgow, 113-135, July 1992. (published
by Springer, London, 1993).
Keywords and Phrases: VPL-II.A.7: Logic languages, VPL-II.B.1: Diagrammatic languages, VPL-V.B: Database languages
Yang, S. and M. Burnett, From Concrete Forms to Generalized Abstractions through Perspective-Oriented Analysis of
Logical Relationships, 1994 IEEE Symposium on Visual Languages, St. Louis, MO, 6-14, October 4-7, 1994.
Keywords and Phrases: Forms/3, VPL-II.A.4: Form-based and spreadsheet-based languages
Hays, Judith G. and Margaret M. Burnett, A Guided Tour of Forms/3, Oregon State University, Dept. of Computer Science,
TR 95-60-6, June 1995.
Keywords and Phrases: Forms/3, VPL-II.A.4: Form-based and spreadsheet-based languages
Burnett, Margaret M. and Allen L. Ambler, A Declarative Approach to Event-Handling in Visual Programming Languages,
1992 IEEE Workshop on Visual Languages, Seattle, WA, 34-40, Sept. 15-18, 1992.
Keywords and Phrases: Forms/3, VPL-II.A.4: Form-based and spreadsheet-based languages, VPL-III.A: Abstraction, VPL-
III.E: Event handling
Lakin, Fred, Computing with Text-Graphic Forms, Proceedings of the LISP Conference, Stanford University, August 1980.
Keywords and Phrases: computing with text-graphic forms, visual pattern manipulation language, visual lisp, VPL-II.A.5:
Functional languages, VPL-V.A: General-purpose languages, VPL-IV.D: Translators (interpreters and compilers)
Cordy, James R. and T. C. Nicholas Graham, GVL: Visual Specification of Graphical Output, Journal of Visual Languages
and Computing, 3, 1992.
Keywords and Phrases: VPL-II.A.5: Functional languages, VPL-V.E: User-interface generation languages
Graham, T. C. Nicholas and Tore Urnes, Relational Views as a Model for Automatic Distributed Implementation of Multi-
User Applications, Proceedings of the Fourth Conference on Computer-Supported Cooperative Work, Toronto, October
1992.
Keywords and Phrases: VPL-II.A.5: Functional languages, VPL-V.E: User-interface generation languages
Harvey, N. and J. Morris, NL: A General Purpose Visual Dataflow Language, Technical Report, University of Tasmania,
October 1993.
Keywords and Phrases: VPL-II.A.3: Data-flow languages, VPL-V.A: General-purpose languages
Koelma, D., R. van Balen, and A. Smeulders, SCIL-VP: a multi-purpose visual programming environment, Proceedings of
the 1992 ACM/SIGAPP Symposium on Applied Computing, 1188-1198, 1992.
Keywords and Phrases: VPL-II.A.3: Data-flow languages, VPL-III.A.2: Procedural abstraction, VPL-VI.C.4: Scope
M. Paltrinieri, A Visual Environment for Constraint Programming (Poster Paper), 11th IEEE International Symposium on
Visual Languages, Darmstadt, Germany, September 5-9, 1995.
Keywords and Phrases: VPL-II.A.2: Constraint-based languages
Boursier, P. and M. Mainguenaud, Spatial Query Languages : Extended SQL vs. Visual Languages vs. Hypermaps, 5th
International Symposium on Spatial Data Handling, Charleston, USA, August 3-7, 1992.
Keywords and Phrases: VPL-II: Language Classifications
Standing, C. and G. Roy, A Functional, Visual Programming Interface to Geographical Information Systems, Proceedings of
OZCHI, Melbourne, Australia, 209-214, Nov. 28 - Dec. 1, 1994.
Keywords and Phrases: VPL-II.A.5: Functional languages
Schuerr, Andreas, Rapid Programming with Graph Rewrite Rules, USENIX Symposium Proc. on Very High Level
Languages (VHLL), USENIX Association, 83-100, 1994.
Keywords and Phrases: VPL-II.A.6: Imperative languages, VPL-II.A.8: Multi-paradigm languages, VPL-II.A.11: Rule-based
languages, VPL-II.B.1: Diagrammatic languages
Schuerr, Andreas, Introduction to PROGRES, an Attribute Graph Grammar Based Specification Language, Proc. WG'89
Workshop on Graphtheoretic Concepts in Computer Science, LNCS 411, (Nagl M., ed.), Springer Verlag, 151-165, 1989.
Keywords and Phrases: VPL-II.A.6: Imperative languages, VPL-II.A.8: Multi-paradigm languages, VPL-II.A.11: Rule-based
languages, VPL-II.B.1: Diagrammatic languages
Glinert, Ephraim P. and Charles D. Norton, Novis: A Visual Laboratory for Exploring the Design of Processor Arrays,
Journal of Visual Languages and Computing, Vol. 3, Number 2, 135-159, June 1992.
Keywords and Phrases: Novis, VPL-II.A.1: Concurrent languages, VPL-II.B.2: Iconic languages, VPL-IV.A: Computational
approaches, VPL-VI.C.6: Visual representation issues
Burnett, Margaret M. and Allen L. Ambler, Interactive Visual Data Abstraction in a Declarative Visual Programming
Language, Journal of Visual Languages and Computing, 29-60, March 1994.
Keywords and Phrases: Forms/3, VPL-II.A.4: Form-based and spreadsheet-based languages, VPL-III.A.1: Data
abstraction, VPL-III.E: Event handling
Schuerr, Andreas, A. Zuendorf, Nondeterministic Control Structures for Graph Rewriting Systems, Proc. WG'91 Workshop
in Graph-theoretic Concepts in Computer Science, LNCS 570, Springer Verlag, 48-62, 1991. also: Technical Report AIB
91-17, RWTH Aachen, Germany, 1991.
Keywords and Phrases: VPL-II.A.6: Imperative languages, VPL-II.A.8: Multi-paradigm languages, VPL-II.A.11: Rule-based
languages, VPL-II.B.1: Diagrammatic languages
Schuerr, Andreas, PROGRES: A VHL-Language Based on Graph Grammars, Proc. 4th Int. Workshop on Graph-Grammars
and Their Application to Computer Science, LNCS 532, Springer Verlag, 641-659, 1991. also: Technical Report AIB 90-16,
RWTH Aachen, Germany, 1990.
Keywords and Phrases: VPL-II.A.6: Imperative languages, VPL-II.A.8: Multi-paradigm languages, VPL-II.A.11: Rule-based
languages, VPL-II.B.1: Diagrammatic languages
Norton, Charles D. and Ephraim P. Glinert, A Visual Environment for Designing and Simulating Execution of Processor
Arrays, 1990 IEEE Workshop on Visual Languages, Skokie, Illinois, 227-232, October 1990.
Keywords and Phrases: Novis, VPL-II.A.1: Concurrent languages, VPL-II.B.2: Iconic languages, VPL-IV.A: Computational
approaches, VPL-VI.C.6: Visual representation issues
Carlson, Paul and Margaret Burnett, Integrating Algorithm Animation into a Declarative Visual Programming Language
(poster paper), 1995 IEEE Symposium on Visual Languages, Darmstadt, Germany, September 1995.
Keywords and Phrases: Forms/3, VPL-II.A.4: Form-based and spreadsheet-based languages
Ambler, Allen L. and Margaret M. Burnett, Visual Languages and the Conflict Between Single Assignment and Iteration,
1989 IEEE Workshop on Visual Languages, Rome, Italy, 138-143, Oct. 1989.
Keywords and Phrases: Forms/2, VPL-II.A.4: Form-based and spreadsheet-based languages, VPL-III.B: Control flow
Ambler, Allen L. and Margaret M. Burnett, Visual Forms of Iteration that Preserve Single Assignment, Journal of Visual
Languages and Computing, 1(2), Academic Press, June 1990.
Keywords and Phrases: Forms/2, VPL-II.A.4: Form-based and spreadsheet-based languages, VPL-III.B: Control flow

Low-latency trading & VPL
Communication Barriers
Safety

Why I decided to build a VPL
● I didn't! - I stumbled upon it
● I was still learning the business
● I had insanely aggressive deadlines (see: minutes)
for writing and deploying production code
It was built out of necessity

I got tired of these guys yelling at me

Modules made my life easier
Synthetic
Stop
int qty = currOrderQty - _officialQty;
if (qty > 0)
{
OrderProfileBase orderProfile = null;
AutospreaderSyntheticOrderProfile aop = null;
if (currInstrument.IsASEInstrument)
{
//aop = new AutospreaderSyntheticOrderProfile(currInstrument.OrderFeed,
aop = TopMostAlgorithm.CreateAutospreaderOrderProfile(currInstrument, r
orderProfile = aop;
}
else
{
orderProfile = TopMostAlgorithm.CreateAlgoExchangeChildOrderProfile(cur
}
Order
Quoter
Timed
Hedge
2-Legged
Spread
//orderProfile = new AlgoExchangeChildOrderProfile(currInstrument.Order
AlgoExchangeChildOrderProfile algoExchangeChildOrderProfile = (AlgoExch
algoExchangeChildOrderProfile.LeaveWhenPaused = leaveOnPause;
algoExchangeChildOrderProfile.LeaveWhenCanceled = leaveOnCancel;
if (IsFlagColorSet)
algoExchangeChildOrderProfile.Color = flagColor;
Market
Maker
Arb
Routine
orderProfile.Action = OrderAction.Add;
orderProfile.QuantityToWork = Quantity.FromInt(m_CurrInstrument.Instrument,
orderProfile.BuySell = m_Buy ? BuySell.Buy : BuySell.Sell;
orderProfile.OrderTag = m_Flag;
orderProfile.OrderType = m_OrderType;

But modules weren't enough
● Bugs could still spell disaster
● I wanted the traders to be in control
○ Just piecing them together took work and
programming know-how
○ This "work" wasn't going to add any business
value

The VPL was born
● I built a GUI that let them visually link these modules
together
● I didn't know I was building a VPL
● Algo Design Lab (ADL)

The "Deutsch Limit"
“The problem with visual programming languages is
that you can't have more than 50 visual primitives on
the screen at the same time.”
-Peter Deutsch

The "Deutsch Limit"
● The limit of 50 primitives is arbitrary
● We rely on our IDE to help us solve this problem
○ "Find References/Usages"
○ "Go to Definition/Declaration"
○ "Find in Files"
○ Refactoring
● Mental Map

The solution is threefold
1. Domain specificity
2. Hiding the "dirty stuff"
3. Mitigating the "Deutsch Limit"

1. Domain specificity
"The ability for the general public to understand a VPL is
inversely proportional to the likelihood that it will actually
do something useful."
-me
● This is really no different from a well-made API
○ Abstracts away unnecessary granularity
● Unspecific: Scratch, AppInventor, etc.
● Specific: Lego Mindstorms

2. Hiding the "dirty stuff"
● All VPLs hide some of the generic dirty stuff
● Goes hand-in-hand with domain specificity
○ If not domain-specific, it's hard to identify the
dirty stuff
● Examples of dirty stuff in automated trading
○ In-flight Orders
○ Fairness
○ "It just works"

3. Mitigating the "Deutsch Limit"
● Browser-based navigation
● Show Up/Downstream Blocks
● Block searching
● Componentization
● Breakpoints
● "Jump Blocks"

We want our users to eat fruit
Kathy Sierra - http://goo.gl/hKd95
● Cake vs. Fruit
● We want them to focus on idea generation, not
implementation
● It's also about safety

Let's do an experiment
● Build a simple trading strategy:
○ Place an order to buy 100 shares of stock (e.g.
GOOG) at the last-traded-price (LTP)
○ Whenever the LTP of GOOG changes, we should
change our order's price to match
○ If the LTP of GOOG *ever* falls below $770, we
should cancel our order
● Scratch-->Domain Specific Language-->ADL

stock=GOOG;volume=5433728;LTP=782.03;

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Domain Specific

Algo Design Lab (ADL)
(that's it.)

Domain Specific

Domain Specific

Wrap-up*
● VPLs can work for a very narrow set of
industries and applications
● Domain specificity
● Hiding the dirty stuff
● Mitigating the "Deutsch Limit"
● Are our users fruit or cake eaters?
* Please exit through the back for processing

Questions?

Appendix

Flowcharts are evil
“The flowchart is a very poor abstraction of software
structure.”
-Fred Brooks

Flowcharts are evil
● Poor representation of the actual effort needed to
realize the intended goal
● Flowcharts tend to be written after you write code,
not before
● Too many assumptions; too abstract