What are Lisp macros good for, anyway?

Ask a Lisp’er why anyone of sound mind would choose such an odd programming language and they’ll likely start talking about macros. As in, Lisp has macros and your language doesn’t. Maybe you say “C has macros and they’re mostly just dangerous”, and they say “Lisp macros aren’t like that, they’re different”. So what can you do with macros in a language like Common Lisp or Clojure that you can’t do (sanely) in other programming languages? Let’s look at an example of how we might do something in Java, Python, and Clojure (a Lisp).

Aside: macros in C/C++

The example below could likely be done in C or C++ with a macro. I think most people would agree that these kind of text-replacement macros have been problematic. Additionally, they do not have the full language available for their use. If someone would like to comment about C/C++ macros or show how to do this example with them, go for it.

Timing an operation

Say we have written a function named foo, which takes one argument and returns a number. We want to know roughly how long this function takes on some typical parameters.

Python

One option in python is the built-in timeit library. Usage might look like this:

Using timeit

Note you pass a string containing the code you want to time the execution of. Note also that you can’t use the return value of the call to avg afterwards. The setup argument is a bit cumbersome, maybe that’s a way around that.

It seems a more common approach by Pythonista’s is to just use time :

Using time manually

This can be somewhat error-prone but generally works OK. Since it takes time before and after a line, if the line being timed is more complicated it’s not easy to time part of the line (e.g. some sub-expression of it).

We could also write our own timing function that takes a function of no arguments:

Custom timing function

Then usage would look like this:

Using time_call

Note that this allows the result of the call to foo to be captured and used. This approach is more amenable to adding timing of something inside a bigger piece of code then removing it shortly afterward.

Java

I don’t know of anything built into Java that helps with this. We can write some fairly generic timing functions:

Custom timing functions for Java

The signature taking a Supplier is for something that will return a value, and the one taking a Runnable is for something that will not. Usage looks like:

Using the custom timing functions in Java

This is pretty similar to the Python case, although more verbose due to the type system. Note however that in both Python and Java, timing a piece of code may require wrapping it in another expression so that it’s some kind of function.

Clojure’s turn

Clojure has a built-in operation called time which we can use like so:

Using Clojure time

time also returns its argument, so if the call to foo is within the context of other code (as is typical), it can be easily inserted and removed at will:

Before you say “but I have to balance the parens when removing or adding the call to time!”, good editors for Lisp’s like Clojure make that painless. Removing (time automatically removes the corresponding right parens for that expression.

How does time work without requiring its argument to implement some interface or contract? In Python, the argument could be a string as it was when using timeit , but timeit doesn’t return the result of evaluating the code; additionally, it’s not as easy to insert a timeit call since it requires making the code into a string. In both Python and Java, you can time something by accepting a function, but this requires the caller to wrap the code to be timed in a lambda expression.

The answer of course, is that Clojure time is an example of a Lisp macro rather than a function. Let’s look at the implementation of time (documentation omitted for brevity):

Implementation of time from clojure.core

time is a macro taking one argument namedexpr . The magic happens in the expression ret# ~expr , which executes the provided expression such as (foo 5) and assigns it to the variable ret#. The ~ operator in a macro evaluates an expression. # as in ret# is a way of ensuring a unique variable name in the scope of this macro to avoid using a variable in an enclosing scope (i.e. one defined elsewhere in the program) by mistake.

So, Clojure’s version of time takes an arbitrary code snippet — which in a Lisp-like Clojure is represented as a List surrounded by () — evaluates it, prints how long that took, and returns the result.

We can see how this macro works by using macroexpand:

macroexpand gives some illustration of what Lisp macros are. They are functions from source code to source code; that is, a function from a list to a list. In Clojure you register such functions using defmacro, and then your macros are called during compilation to ‘expand’ the expression. In the example above, the list (time (foo 5)) is expanded into one that times the execution of (foo 5) and then returns its result. The special power that Lisp macros have is that they can control evaluation (as seen by evaluating the input expression via ~expr and do arbitrary source-to-source transformations with the full power of the language available.

More macro examples

Part of the reason for discussing the time macro was because it’s possible (though not as convenient) to implement something similar in most languages. This might make you wonder if macros are worthwhile. Here are some more examples of macros in Clojure; consider how you could do this functionality in your language, or whether it’s feasible to do it at all.

• Convenient control flow constructs like when, when-not, if-not.
• cond, which is similar to switch expressions in Java 12 but more general. Note that cond is a macro because it doesn’t evaluate expressions for branches that won’t be executed, which is why most languages without macros would need built-in support for this.
• List comprehensions like for.
• Threading macros like -> can be used to make nested expressions easier to read, similar in appearance to the pipeline operator in Javascript.
• Macros like doto and .. make interop with ‘host’ languages (Java/Javascript) easier in Clojure.
• Testing libraries often use macros to make tests pleasant to read. For example, Midje makes tests look like examples in Clojure books.
• comment
• Macros can support the more familiar infix notation for math such as (1 + (2 * 3)) by rewriting it into the prefix notation (+ 1 (* 2 3)) used by Lisps. An example is the infix library.

An important idea in Lisp communities is that you can write these kinds of things yourself (or get them from a library) instead of petitioning a language committee or Benevolent Dictator for Life to support them, keeping the core language small and coherent. The downside of this extensibility is the curse of lisp.

Conclusion

Lisp macros provide a level of power not easily replicated in other languages. The key to their power is that Lisps are homoiconic, which means that the source code in Lisp’s can be represented in a data structure the language itself can manipulate (i.e. a list), making these kinds of custom source code transformations sane and useful.

You might question whether more power in a language is always a good thing. Isn’t this just another way to make a mess of things? Powerful features can be a double-edged sword, but I really like how Glenn Vanderburg put it:

Weak developers will move heaven and earth to do the wrong thing. You can’t limit the damage they do by locking up the sharp tools. They’ll just swing the blunt tools harder.

I certainly don’t mean to say developers who don’t write Lisp macros are ‘weak’. But we shouldn’t limit our capabilities for fear of what others might do with it.

Addendum

Many thanks to Reddit user lispm for showing another reason why time could be a macro: if time is a macro it can include the original source code in the timing output. This is awesome and I wish the Clojure version of time did this.