2006 Scheme and Functional Programming Papers, University of

2. Explaining Macros
Macro expansion takes place during parsing. As the parser traverses
the concrete syntax, 1 it creates abstract syntax nodes for primitive
syntactic forms, but stops when it recognizes the use of a macro.
At that point, it hands over the (sub)phrases to the macro, which,
roughly speaking, acts as a rewriting rule.
In Lisp and in some Scheme implementations, a macro is expressed
as a plain function; in R 5 RS Scheme [19], macros are
expressed in a sub-language of rewriting rules based on patterns.
Also in Lisp, concrete syntax are just S-expressions; Lisp macro
programming is thus typically first-order functional programming 2
over pairs and symbols. The most widely used Scheme implementations,
however, represent concrete syntax with structures that
carry additional information: lexical binding information, original
source location, code security annotations, and others. Scheme
macro programming is therefore functional programming with a
rich algebraic datatype.
Given appropriate inputs, a Lisp macro can go wrong in two
ways. First, the macro transformer itself may raise a run-time exception.
This case is naturally in the domain of run-time debuggers;
after all, it is just a matter of traditional functional programming.
Second, a Lisp macro may create a new term that misuses a syntactic
form, which might be a primitive form or another macro. This
kind of error is not detected when the macro is executing, but only
afterwards when the parser-expander reaches the misused term.
Modern Scheme macros might go wrong in yet another way.
The additional information in a syntax object interacts with other
macros and primitive special forms. For example, macro-introduced
identifiers carry a mark that identifies the point of their introduction
and binding forms interpret identifiers with different marks
as distinct names. Scheme macros must not only compute a correct
replacement tree but also equip it with the proper additional
properties.
Even in Lisp, which has supported macros for almost 50
years now, macros have always had impoverished debugging environments.
A typical Lisp environment supports just two procedures/tools
for this purpose: expand and expand-once (or
macroexpand and macroexpand-1 [28]). All Scheme implementations
with macros have adapted these procedures.
When applied to a term, expand completely parses and expands
it; in particular, it does not show the intermediate steps of the rewriting
process. As a result, expand distracts the programmer with too
many irrelevant details. For example, Scheme has three conditional
expressions: if, cond, and case. Most Scheme implementations
implement only if as a primitive form and define cond and case
as macros. Whether or not a special form is a primitive form or a
macro is irrelevant to a programmer except that macro expansion
reveals the difference. It is thus impossible to study the effects of
a single macro or a group of related macros in an expansion, because
expand processes all macros and displays the entire abstract
syntax tree.
The task of showing individual expansion steps is left to the second
tool: expand-once. It consumes a macro application, applies
the matching macro transformer, and returns the result. In particular,
when an error shows up due to complex macro interactions,
it becomes difficult to use expand-once easily because the offending
or interesting pieces are often hidden under a large pile of
syntax. Worse, iterated calls to expand-once lose information between
expansion steps, because lexical scope and other information
depends on the context of the expansion call. This problem renders
expand-once unfit for serious macro debugging.
1 We consider the result of (read) as syntax.
2 Both Lisp and Scheme macro programmers occasionally use side-effects
but aside from gensym it is rare.
Implementing a better set of debugging tools than expand and
expand-once is surprisingly difficult. It is apparently impossible
to adapt the techniques known from run-time debugging. For example,
any attempt to pre-process the syntax and attach debugging information
or insert debugging statements fails for two reasons: first,
until parsing and macro expansion happens, the syntactic structure
of the tree is unknown; second, because macros inspect their arguments,
annotations or modifications are likely to change the result
of the expansion process [31].
While these reasons explain the dearth of macro debugging
tools and steppers, they don’t reduce the need for them. What we
present in this paper is a mechanism for instrumenting the macro
expander and for displaying the expansion events and intermediate
stages in a useful manner. Eventually we also hope to derive a wellfounded
model of macros from this work.
3. The Macro Debugger at Work
The core of our macro debugging tool is a stepper for macro
expansion in PLT Scheme. Our macro debugger shows the macro
expansion process as a reduction sequence, where the redexes are
macro applications and the contexts are primitive syntactic forms,
i.e., nodes in the final abstract syntax tree. The debugger also
includes a syntax display and browser that helps programmers
visualize properties of syntax objects.
The macro stepper is parameterized over a set of “opaque”
syntactic forms. Typically this set includes those macros imported
from libraries or other modules. The macro programmers are in
charge, however, and may designate macros as opaque as needed.
When the debugger encounters an opaque macro, it deals with the
macro as if it were a primitive syntactic form. That is, it creates an
abstract syntax node that hides the actual expansion of the macro.
Naturally, it does show the expansion of the subexpressions of the
macro form. The parameterization of primitive forms thus allows
programmers to work at the abstraction level of their choice. We
have found this feature of the debugger critical for dealing with
any nontrivial programs.
The rest of this section is a brief illustrative demonstration of the
debugger. We have picked three problems with macros from recent
discussions on PLT Scheme mailing lists, though we have distilled
them into a shape that is suitably simple for a technical paper.
3.1 Plain Macros
For our first example we consider a debugging scenario where the
macro writer gets the form of the result wrong. Here are three different
versions of a sample macro that consumes a list of identifiers
and produces a list of trivial definitions for these identifiers:
1. in Lisp, the macro writer uses plain list-processing functions to
create the result term:
(define-macro (def-false . names)
(map (lambda (a) ‘(define ,a #f)) names))
2. in R 5 RS the same macro is expressed with a rewriting rule
notation like this:
(define-syntax def-false
(syntax-rules ()
[(def-false a ...) ((define a #f) ...)]))
3. in major alternative Scheme macro systems, the rule specification
is slightly different:
(define-syntax (def-false stx)
(syntax-case stx ()
[(_ a ...) (syntax ((define a #f) ...))]))
16 Scheme and Functional Programming, 2006