Elixir Macros Demystified, part 2: understanding quote and unquote

Phoenix on Rails is a comprehensive guide to Elixir, Phoenix and LiveView for developers who already know Ruby on Rails. If you’re a Rails developer who wants to learn Phoenix fast, click here to learn more!

This is Part 2 in a series on Elixir macros - what they are, why they matter, and how to use them.

In Part 1 we got the basic overview of what macros are: they’re code that writes code. Roughly speaking, a macro takes an Elixir expression as it’s written in your codebase, then transforms it into a different piece of code at compile time.

For example, as we saw in Part 1, if assert/1 is a macro then the following line of test code…

assert 2 + 2 == 5

… can effectively be transformed at compile time into something that prints useful information if the test fails, like ExUnit would do:

# This is ridiculously simplified... the real ExUnit
# is far more sophisticated than this!
if 2 + 2 == 5 do
  IO.puts("Assertion passed")
else
  IO.puts("""
  Assertion with == failed
  code: assert 2 + 2 == 5
  left: 4
  right: 5
  """)
end

The macro receives its argument 2 + 2 == 5 as a piece of code, rather than as its evaluated result false. Another way of saying this is that it treats 2 + 2 == 5 as an expression, not a value.

Expressions and quoted expressions

Some languages distinguish between “expressions” - code that returns a result when evaluated - and “statements” - units of execution that have no return value.

In Elixir, however, everything is an expression. Every syntactically valid piece of Elixir code can be evaluated to return a result - for example, 2 + 2 is an expression that returns 4. The result can then, for example, be assigned to a variable or returned from a function.

Some simple expressions are their own return values. For example, the return value of the expression 2 is just 2; there’s nothing more to evaluate. More complex expressions are constructed from simpler expressions in a nested manner. An Elixir codebase is fundamentally just one big expression built from a tree of nested sub-expressions.

Macros are just Elixir, so to write them we need a way to represent chunks of code (i.e. expressions) as Elixir data structures. We do this using quoted expressions, otherwise known as abstract syntax trees (ASTs).

For example, when passing the expression 2 + 2 == 5 to a macro, Elixir first transforms it into this tuple:

{:==, […]], [{:+, […]], [2, 2]}, 5]}

I’ve omitted some of the details for brevity. The important point is that this tuple, a quoted expression, is how Elixir represents “2 + 2 == 5” in a way that can be read, manipulated and passed around using Elixir’s normal semantics.

quote

To convert an Elixir expression into a quoted expression, use quote:

iex> quote do: 2 + 2 == 5
{:==, [context: Elixir, imports: [{2, Kernel}]],
 [{:+, [context: Elixir, imports: [{1, Kernel}, {2, Kernel}]], [2, 2]}, 5]}

As with other constructs like def, you can use do and end instead of do:, which is useful when your quoted code spans multiple lines.

iex> quote do
...>   %User{}
...>   |> User.changeset(attrs)
...>   |> Repo.insert()
...> end
{:|>, […], […]}

This is the point where the official docs start going deeper into the inner structure of quoted expressions and what the different parts of that tuple mean. I say skip it; it’s more detail than a beginner needs.

All you need to know for now is that most quoted expressions take the form of a three-element tuple representing a function call:

iex> quote do: add(2, 3)
{:add, [], [2, 3]}

You can see how more complex quoted expressions are built recursively out of smaller ones:

iex> quote do: add(1, add(2, 3))
{:add, [], [1, {:add, [], [2, 3]}]}

Some simple expressions are equivalent to their own quoted form:

iex> quote do: false
false
iex> quote do: 1
1
iex> quote do: "string"
"string"
iex> quote do: :atom
:atom

We’ll leave the other details until later in this series.

Printing the source code

Macro.to_string/1 converts a quoted expression into a string of human-readable Elixir code:

iex> quote(do: 2 + 2 == 5) |> Macro.to_string()
"2 + 2 == 5"

iex> quote do
...>   %User{}
...>   |> User.changeset(attrs)
...>   |> Repo.insert()
...> end
...> |> Macro.to_string()
"%User{} |> User.changeset(attrs) |> Repo.insert()"

Macro.to_string/1 discards the original code’s formatting. For example, you can see above how it condenses the second expression into a single line, ignoring the newlines that were present when we quoted it.

Evaluating a quoted expression

One way to evaluate a quoted expression is with Code.eval_quoted/1:

iex> Code.eval_quoted(quote do: 2 + 2)
{4, []}

This returns a tuple whose first element is the result - 4 here, because we evaluated 2 + 2. The second element ([] here) is something called a binding, which we’ll come back to later.

Code.eval_quoted/1 is useful for educational purposes, or when playing around with macros in the IEx console, but it’s rarely something you’ll use in production code. There are better ways to evaluate a quoted expression, as we’ll see.

Merely quoting an expression doesn’t evaluate it. You can quote an expression even if running it would raise an error:

# This doesn't raise an error:
iex> expression = quote do: raise "error!"
{:raise, [context: Elixir, imports: [{1, Kernel}, {2, Kernel}]], ["error!"]}

# This does:
iex> Code.eval_quoted(expression)
** (RuntimeError) error!

This means that a quoted expression can contain references to functions or variables that don’t exist:

iex> x_plus_3 = quote do: x + 3
{:+, …, …}
iex> Code.eval_quoted(x_plus_3)
# error: undefined variable "x" (context Elixir)

In the above example we say that x is a free variable - a variable whose value is undefined.

By default, quote ignores any variable bindings in the context that it’s called from. So the following won’t make x_plus_3 work:

# This makes no difference:
iex> x = 2
2

# This creates the exact same quoted expression as it would
# if x didn't have a value above:
iex> x_plus_3 = quote do: x + 3
{:+, …, …}

# So this still doesn't work:
iex> Code.eval_quoted(x_plus_3)
# error: undefined variable "x" (context Elixir)

You can set variable values in a quoted expression using :bind_quoted:

iex> binded_x_plus_3 =
...>   quote bind_quoted: [x: 2] do
...>     x + 3
...>   end
{:__block__, …, …}

# Notice how the quoted code now sets x to 2:
iex> binded_x_plus_3 |> Macro.to_string() |> IO.puts()
# x = 2
# x + 3

iex> Code.eval_quoted(binded_x_plus_3)
{5, [{{:x, Elixir}, 2}]}

In this modified version, Code.eval_quoted/2 returns the result 5, and it also returns a binding denoting that the variable x has the value 2.

Importantly, the returned binding tells us the variables’ values at the end of execution, not the beginning:

iex> quote bind_quoted: [x: 2] do
...>   x = 10
...> end
...> |> Code.eval_quoted()
{10, [{{:x, Elixir}, 10}]}

We initially bound x to 2, but while evaluating the expression this was overridden with a new value 10. This new binding is what’s returned at the end.

unquote

Sometimes you want to inject a value directly into your quoted expression without quoting it. You can do this with unquote:

iex> x = 2
2
iex> unquoted_x_plus_3 = quote do: unquote(x) + 3
{:+, …, …}

To see how it works, compare these two quoted expressions:

iex> x = 2
2
iex> Macro.to_string(quote do: x + 3)
"x + 3"
iex> Macro.to_string(quote do: unquote(x) + 3)
"2 + 3"

You can think of unquote as being analogous to string interpolation using #{}. Just as "#{x}" interpolates the value of x into the string (rather than treating it like the string "x"), you can use unquote(x) to interpolate the value of x into a quoted expression.

unquote can only be called inside a quote block. It wouldn’t make sense to use it anywhere else, just like it wouldn’t make sense to use #{} outside of a string.

There’s much more that can be said about quote and unquote, but I think it’s time we finally started writing some macros. In the next lesson (coming soon) we’ll use quote and unquote to build a better assert/1 command for our homegrown testing library.