Clojure: Macro-Writing Macros

Updated on February 12, 2018

If you're writing a function and it gets big enough to be clumsy, I'm sure you know to break it down into multiple, smaller functions, each of which is comprehensible. Of course this is good practice in any language at all, but it's especially simple and powerful in functional languages, where you don't need to worry about side effects: if there's a chunk of code that you want to split up, you can painlessly move that code verbatim into a new function, taking as parameters whatever locals it needs to operate on. The hardest part of the whole process is coming up with a meaningful name for the new helper function!

Macros can develop similar problems if they grow large enough, so naturally you want to split them up too, right? Just slice out a chunk of the too-large macro, paste it into a new macro, and you're all set.

(defmacro with-magic
  [magic-fn log-string & body]
  (let [magic-sym (gensym "magic-")]
    `(let [~magic-sym (fn [arg#]
                        (println ~log-string arg#)
                        (~magic-fn arg#))]
       ~@(postwalk-replace {magic-fn magic-sym} body))))

(macroexpand '(with-magic ! "OMG doing magic with" 
                (let [x 1] (! "test"))))

(let* [magic-2968 (clojure.core/fn
                     [arg__2958__auto__]
                     (clojure.core/println "OMG doing magic with" arg__2958__auto__) 
                     (! arg__2958__auto__))] 
   (let [x 1] 
     (magic-2968 "test")))

Maybe a silly macro, but not a totally trivial one to write. It takes a symbol to search for, and replaces all instances of that symbol with a specially-constructed function that prints something before doing the real work. So let's say we decide this macro is getting complicated enough we'd like to split it up - the logical way to do that seems to be creating a separate macro that creates these "magic-xxx" functions.

(defmacro make-magic-fn [magic-fn log-string]
  `(fn [arg#]
     (println ~log-string arg#)
     (~magic-fn arg#)))

(defmacro with-magic
  [magic-fn log-string & body]
  (let [magic-sym (gensym "magic-")]
    `(let [~magic-sym ~(make-magic-fn magic-fn log-string)]
       ~@(postwalk-replace {magic-fn magic-sym} body))))

(macroexpand '(with-magic ! "OMG doing magic with" 
                (let [x 1] (! "test"))))

(let* [magic-3340 #<user$with_magic$fn__3327 user$with_magic$fn__3327@15bc46>] 
   (let [x 1] (magic-3340 "test")))

That doesn't look right at all! Instead of the nice (fn...) form we got from the original version, we get this gross #<user$with_magic$fn__3327> thing, which definitely won't be legal code if you try to use the macro (rather than just expanding it as I've done here). But what went wrong? make-magic-fn is just a snippet of the original with-magic macro, so shouldn't the results be identical?

Macros are just functions

The issue is that really, macros are just functions with two special properties:

  1. Their arguments are not evaluated
  2. Their return values are expanded in-place and treated as code

That's it! The backtick provides a useful shortcut for the most common use-case, which is basically a code template into which you stitch the user's arguments at the appropriate places; but you should remember what is actually going on behind the scenes. Your macro is receiving as arguments some number of symbols and lists, and is returning a list.

Here, we want the arguments to our helper function to be evaluated: we don't want the function returned to contain the symbols "magic-fn" or "log-string", we want it to contain the values of those bindings. And we don't want the result of (make-magic-fn) to be expanded in place and interpreted as code within our macro, we just want to take the list returned from make-magic-fn and place it inside the list that with-magic will eventually return.

So what we're actually looking for is something like a macro, but without properties 1 and 2 above. Guess what? That's just a function! What we're really trying to write is a function that makes it easy to generate lists that look like '(fn [...] (...)), and use that function from within our macro. Because macros have the full power of the language available to them at compile time, we can simply create this as a function that we call at compile time.

And in this case, that's all it takes! If you replace (defmacro make-magic-fn) with (defn make-magic-fn), it all just works like a charm. You can keep using the ` shortcut from within the function-ized version of make-magic-fn: ` is not a magical tool that only works inside macros, it's a convenient shorthand for generating lists that have only some of their contents quoted.

Don't volunteer for a peg leg

Nested macros are like amputations: they're very painful, and there's usually a better solution, but on rare occasions you have no choice. So while the above hint about writing helper functions instead of helper macros will usually suffice to keep you out of trouble, the tricks of writing nested macros are still a good thing to know.

For example, let's say you want to write a bunch of very similar macros, like (with-explosions), (with-special-effects), and so on. Maybe they all look like this but with different names and different interposed forms.

(defmacro with-explosions [& body]
  `(do
     ~@(interpose `(println "BOOM") body)))

Instead of writing this macro hundreds of times, you sensibly decide to write a macro that takes a series of (name, interpose) pairs and writes a macro for each of them. The trick will be getting the nested quotes, unquotes, and gensym# forms right, so I'll just show the solution here and then discuss why it looks the way it does.

(defmacro build-movie-set [& scenes]
  (let [name-vals (partition 2 scenes)]
    `(do
       ~@(for [[name val] name-vals]
           `(defmacro ~(symbol (str "with-" name "s"))
              ([~'& body#]
                 `(do
                    ~@(interpose `(println ~~val)
                                 body#))))))))

Can I quote you on that?

That definitely looks gross. I count four backtick forms, which are variously unquoted throughout the metamacro. The most interesting things you'll see here are ~~val and [~'& body#].

The first of those is because the context in which val is used is two backtick-levels away from the context in which it has a value, so we have to unquote it twice.

The second exists because, as you know, the backtick operator namespace-qualifies all symbols it sees...and & is a legal symbol! So backtick will try to replace it with user/& or movies/& or something; then when defmacro sees that, it won't realize you wanted & body. The solution is to exit the syntax quote (with ~) and enter a real, literal quote (with '). That will leave your & unmolested.

There are similar cases where you might need access to a gensym'd symbol from a less-deep macro layer: you can get at those with ~foo# - that is, "don't quote foo# in this nested backtick-form, I want the actual value of foo#". You might also need to quote something in the expanded context, but not the expansion context: that's '~foo. You can pile these sorts of things on top of each other as long as you want; here's an example you can use to give better IDE support for your autogenerated macros:

(defmacro whatever [name & args]
  `(defn ^{:arglists '~'([data])} ~name
     ([data#]
       (do something with data# and ~args))))

With great power...

Clojure's macros make it possible for you to automate writing code, in addition to writing code that automates things. It's a powerful feature, but it has a lot of sharp edges you need to be careful of. In particular, think twice before trying to nest macros: it's usually the wrong answer. But when it's right, hopefully the hints above will make the process easier for you.

Comments

    0 of 8192 characters used
    Post Comment

    • profile image

      Matthew Molloy 

      3 years ago

      Thanks amalloy,

      A good reference for further reading is the wikibooks page on reader macros http://en.wikibooks.org/wiki/Learning_Clojure/Read...

    • profile image

      Alex Coventry 

      4 years ago

      Your last example doesn't quite work, because "^" is a reader macro.

      http://stackoverflow.com/questions/7754429/clojure...

    • profile image

      octopusgrabbus 

      6 years ago

      It seems like needing to count the arguments passed into a macro would be a case where an embedded function would solve the problem. Is that a fair assumption?

    • profile image

      Nicolas Buduroi 

      7 years ago

      Great post, saved me a big headache! Also if you need to use the inner macro &form var you'll need to unquote it two time:

      (defmacro crazy [name & args]

      (let [name* (symbol (str name \*))]

      `(do

      (defn ~name* [self# & params#]

      (prn self#))

      (defmacro ~name [~'& args#]

      `(~~name* (quote ~~'&form) ~@args#)))))

    working

    This website uses cookies

    As a user in the EEA, your approval is needed on a few things. To provide a better website experience, turbofuture.com uses cookies (and other similar technologies) and may collect, process, and share personal data. Please choose which areas of our service you consent to our doing so.

    For more information on managing or withdrawing consents and how we handle data, visit our Privacy Policy at: https://turbofuture.com/privacy-policy#gdpr

    Show Details
    Necessary
    HubPages Device IDThis is used to identify particular browsers or devices when the access the service, and is used for security reasons.
    LoginThis is necessary to sign in to the HubPages Service.
    Google RecaptchaThis is used to prevent bots and spam. (Privacy Policy)
    AkismetThis is used to detect comment spam. (Privacy Policy)
    HubPages Google AnalyticsThis is used to provide data on traffic to our website, all personally identifyable data is anonymized. (Privacy Policy)
    HubPages Traffic PixelThis is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.
    Amazon Web ServicesThis is a cloud services platform that we used to host our service. (Privacy Policy)
    CloudflareThis is a cloud CDN service that we use to efficiently deliver files required for our service to operate such as javascript, cascading style sheets, images, and videos. (Privacy Policy)
    Google Hosted LibrariesJavascript software libraries such as jQuery are loaded at endpoints on the googleapis.com or gstatic.com domains, for performance and efficiency reasons. (Privacy Policy)
    Features
    Google Custom SearchThis is feature allows you to search the site. (Privacy Policy)
    Google MapsSome articles have Google Maps embedded in them. (Privacy Policy)
    Google ChartsThis is used to display charts and graphs on articles and the author center. (Privacy Policy)
    Google AdSense Host APIThis service allows you to sign up for or associate a Google AdSense account with HubPages, so that you can earn money from ads on your articles. No data is shared unless you engage with this feature. (Privacy Policy)
    Google YouTubeSome articles have YouTube videos embedded in them. (Privacy Policy)
    VimeoSome articles have Vimeo videos embedded in them. (Privacy Policy)
    PaypalThis is used for a registered author who enrolls in the HubPages Earnings program and requests to be paid via PayPal. No data is shared with Paypal unless you engage with this feature. (Privacy Policy)
    Facebook LoginYou can use this to streamline signing up for, or signing in to your Hubpages account. No data is shared with Facebook unless you engage with this feature. (Privacy Policy)
    MavenThis supports the Maven widget and search functionality. (Privacy Policy)
    Marketing
    Google AdSenseThis is an ad network. (Privacy Policy)
    Google DoubleClickGoogle provides ad serving technology and runs an ad network. (Privacy Policy)
    Index ExchangeThis is an ad network. (Privacy Policy)
    SovrnThis is an ad network. (Privacy Policy)
    Facebook AdsThis is an ad network. (Privacy Policy)
    Amazon Unified Ad MarketplaceThis is an ad network. (Privacy Policy)
    AppNexusThis is an ad network. (Privacy Policy)
    OpenxThis is an ad network. (Privacy Policy)
    Rubicon ProjectThis is an ad network. (Privacy Policy)
    TripleLiftThis is an ad network. (Privacy Policy)
    Say MediaWe partner with Say Media to deliver ad campaigns on our sites. (Privacy Policy)
    Remarketing PixelsWe may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.
    Conversion Tracking PixelsWe may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.
    Statistics
    Author Google AnalyticsThis is used to provide traffic data and reports to the authors of articles on the HubPages Service. (Privacy Policy)
    ComscoreComScore is a media measurement and analytics company providing marketing data and analytics to enterprises, media and advertising agencies, and publishers. Non-consent will result in ComScore only processing obfuscated personal data. (Privacy Policy)
    Amazon Tracking PixelSome articles display amazon products as part of the Amazon Affiliate program, this pixel provides traffic statistics for those products (Privacy Policy)