Skip to main content

Clojure: Macro-Writing Macros

  • Author:
  • Updated date:

If you're writing a function and it gets big enough to be clumsy, I'm sure you know to break it down into multiple, smaller functions, each of which is comprehensible. Of course, this is good practice in any language at all, but it's especially simple and powerful in functional languages, where you don't need to worry about side effects: if there's a chunk of code that you want to split up, you can painlessly move that code verbatim into a new function, taking as parameters whatever locals it needs to operate on. The hardest part of the whole process is coming up with a meaningful name for the new helper function!

Macros can develop similar problems if they grow large enough, so naturally you want to split them up too, right? Just slice out a chunk of the too-large macro, paste it into a new macro, and you're all set.

(defmacro with-magic
  [magic-fn log-string & body]
  (let [magic-sym (gensym "magic-")]
    `(let [~magic-sym (fn [arg#]
                        (println ~log-string arg#)
                        (~magic-fn arg#))]
       ~@(postwalk-replace {magic-fn magic-sym} body))))

(macroexpand '(with-magic ! "OMG doing magic with" 
                (let [x 1] (! "test"))))

(let* [magic-2968 (clojure.core/fn
                     (clojure.core/println "OMG doing magic with" arg__2958__auto__) 
                     (! arg__2958__auto__))] 
   (let [x 1] 
     (magic-2968 "test")))

Maybe a silly macro, but not a totally trivial one to write. It takes a symbol to search for, and replaces all instances of that symbol with a specially-constructed function that prints something before doing the real work. So let's say we decide this macro is getting complicated enough we'd like to split it up—the logical way to do that seems to be creating a separate macro that creates these "magic-xxx" functions.

(defmacro make-magic-fn [magic-fn log-string]
  `(fn [arg#]
     (println ~log-string arg#)
     (~magic-fn arg#)))

(defmacro with-magic
  [magic-fn log-string & body]
  (let [magic-sym (gensym "magic-")]
    `(let [~magic-sym ~(make-magic-fn magic-fn log-string)]
       ~@(postwalk-replace {magic-fn magic-sym} body))))

(macroexpand '(with-magic ! "OMG doing magic with" 
                (let [x 1] (! "test"))))

(let* [magic-3340 #<user$with_magic$fn__3327 user$with_magic$fn__3327@15bc46>] 
   (let [x 1] (magic-3340 "test")))

That doesn't look right at all! Instead of the nice (fn...) form we got from the original version, we get this gross #<user$with_magic$fn__3327> thing, which definitely won't be legal code if you try to use the macro (rather than just expanding it as I've done here). But what went wrong? make-magic-fn is just a snippet of the original with-magic macro, so shouldn't the results be identical?

Macros Are Just Functions

The issue is that really, macros are just functions with two special properties:

  1. Their arguments are not evaluated
  2. Their return values are expanded in-place and treated as code

That's it! The backtick provides a useful shortcut for the most common use-case, which is basically a code template into which you stitch the user's arguments at the appropriate places; but you should remember what is actually going on behind the scenes. Your macro is receiving as arguments some number of symbols and lists, and is returning a list.

Here, we want the arguments to our helper function to be evaluated: we don't want the function returned to contain the symbols "magic-fn" or "log-string", we want it to contain the values of those bindings. And we don't want the result of (make-magic-fn) to be expanded in place and interpreted as code within our macro, we just want to take the list returned from make-magic-fn and place it inside the list that with-magic will eventually return.

So what we're actually looking for is something like a macro, but without properties 1 and 2 above. Guess what? That's just a function! What we're really trying to write is a function that makes it easy to generate lists that look like '(fn [...] (...)), and use that function from within our macro. Because macros have the full power of the language available to them at compile time, we can simply create this as a function that we call at compile time.

And in this case, that's all it takes! If you replace (defmacro make-magic-fn) with (defn make-magic-fn), it all just works like a charm. You can keep using the ` shortcut from within the function-ized version of make-magic-fn: ` is not a magical tool that only works inside macros, it's a convenient shorthand for generating lists that have only some of their contents quoted.

Don't Volunteer for a Peg Leg

Nested macros are like amputations: they're very painful, and there's usually a better solution, but on rare occasions you have no choice. So while the above hint about writing helper functions instead of helper macros will usually suffice to keep you out of trouble, the tricks of writing nested macros are still a good thing to know.

For example, let's say you want to write a bunch of very similar macros, like (with-explosions), (with-special-effects), and so on. Maybe they all look like this but with different names and different interposed forms.

(defmacro with-explosions [& body]
     ~@(interpose `(println "BOOM") body)))

Instead of writing this macro hundreds of times, you sensibly decide to write a macro that takes a series of (name, interpose) pairs and writes a macro for each of them. The trick will be getting the nested quotes, unquotes, and gensym# forms right, so I'll just show the solution here and then discuss why it looks the way it does.

(defmacro build-movie-set [& scenes]
  (let [name-vals (partition 2 scenes)]
       ~@(for [[name val] name-vals]
           `(defmacro ~(symbol (str "with-" name "s"))
              ([~'& body#]
                    ~@(interpose `(println ~~val)

Can I Quote You on That?

That definitely looks gross. I count four backtick forms, which are variously unquoted throughout the metamacro. The most interesting things you'll see here are ~~val and [~'& body#].

The first of those is because the context in which val is used is two backtick-levels away from the context in which it has a value, so we have to unquote it twice.

The second exists because, as you know, the backtick operator namespace-qualifies all symbols it sees...and & is a legal symbol! So backtick will try to replace it with user/& or movies/& or something; then when defmacro sees that, it won't realize you wanted & body. The solution is to exit the syntax quote (with ~) and enter a real, literal quote (with '). That will leave your & unmolested.

There are similar cases where you might need access to a gensym'd symbol from a less-deep macro layer: you can get at those with ~foo# - that is, "don't quote foo# in this nested backtick-form, I want the actual value of foo#". You might also need to quote something in the expanded context, but not the expansion context: that's '~foo. You can pile these sorts of things on top of each other as long as you want; here's an example you can use to give better IDE support for your autogenerated macros:

(defmacro whatever [name & args]
  `(defn ^{:arglists '~'([data])} ~name
       (do something with data# and ~args))))

With Great Power...

Clojure's macros make it possible for you to automate writing code, in addition to writing code that automates things. It's a powerful feature, but it has a lot of sharp edges you need to be careful of. In particular, think twice before trying to nest macros: it's usually the wrong answer. But when it's right, hopefully the hints above will make the process easier for you.

This article is accurate and true to the best of the author’s knowledge. Content is for informational or entertainment purposes only and does not substitute for personal counsel or professional advice in business, financial, legal, or technical matters.

© 2011 amalloy


Matthew Molloy on October 29, 2014:

Thanks amalloy,

A good reference for further reading is the wikibooks page on reader macros

Alex Coventry on August 02, 2013:

Your last example doesn't quite work, because "^" is a reader macro.

octopusgrabbus on January 13, 2012:

It seems like needing to count the arguments passed into a macro would be a case where an embedded function would solve the problem. Is that a fair assumption?

Nicolas Buduroi on April 20, 2011:

Great post, saved me a big headache! Also if you need to use the inner macro &form var you'll need to unquote it two time:

(defmacro crazy [name & args]

(let [name* (symbol (str name \*))]


(defn ~name* [self# & params#]

(prn self#))

(defmacro ~name [~'& args#]

`(~~name* (quote ~~'&form) ~@args#)))))