What’s in a namespace?
Every well-behaved clojure source file starts with a namespace declaration.
The ns
macro, as we all know, is responsible for declaring the namespace
to which the definitions in the rest of the file belong, and generally also
includes some requirements and imports and whatnot. But today (at
Clojure/West, shoutout!) Stuart Sierra made a passing reference to
the internals of ns
during his talk that got me interested.
But what is a namespace really?
Some things in Clojure are implemented as Java classes of terrifying scope and, for reasons known and important only to the Man himself, using a perplexing and obscure formatting style. But, namespace is not one of those things, which is nice because otherwise the rest of this post would be Java source code and nobody wants to see that. (I’m perfectly comfortable with normal Java code, by the way, it’s just that clojure’s, for probably reasonable reasons, is not that).
No, you came to see some down and dirty Clojure internals, and that’s
what you’re going to get. It turns out that ns
is just a regular old
macro that expends to a bunch of regular old functions that just happen
to be how Clojure happens to load code. In other words, we can use our
good friend macroexpand
to peer (partway) down this particular
rabbit hole, which as you might have guessed is what is about to happen.
Luckily, clojure.core
‘s code is about as transparent as the underlying
Java is opaque, so everything went better than expected for the purposes of this post.
The result of (macroexpand '(ns namespaces.test))
is the following:
(do
(clojure.core/in-ns (quote namespaces.test))
(clojure.core/with-loading-context (clojure.core/refer (quote clojure.core)))
(if (.equals
(quote namespaces.test)
(quote clojure.core))
nil
(do (clojure.core/dosync
(clojure.core/commute
(clojure.core/deref
(var clojure.core/*loaded-libs*))
clojure.core/conj
(quote namespaces.test))) nil)))
Hey, that’s not so bad! Inside that do, there are 3 things happening:
in-ns
is called, setting the current value of the ns var in theclojure.core
namespace. Stateful! Shame! But, there’s a rich tradition of lisps in general working this way behind it, and it saves adding an extra indentation level to every source file by including it in somewith-ns
macro, so I think we can let this slide. (Also, judging by the Java code, someone has a vendetta against excess indentation).- Inside the
with-loading-context
macro, werefer
the clojure.core namespace. This is good, because we always want the functions inclojure.core
to be handy. Also,refer
turns out to be the root of all code-loading in clojure; more on this later. - Finally, if the namespace in question is not
clojure.core
, we append its name (as a symbol) to the*loaded-libs*
ref, usingcommute
in adosync
transaction as one does when working with refs.
“I hope he’s not about to go into excruciating detail about each of those steps,” I hear you psychically mutter. Too bad for you!
From the top: in-ns
The in-ns
is one of those things that Clojure defines in Java.
If I wasn’t clear before, I have no intention of delving beyond that barrier, but
from context it’s clear that in-ns
does what ns
would do if ns
wasn’t
dedicated to encapsulating the half-dozen different things involved in initializing
said namespace.
In other words, in-ns
is what you would be using if you were also using the require
and
use
functions in your code, instead of the :require
and :use
(p.s. don’t use use
or :use
)
in ns
. In code form, this…
(in-ns 'namespaces.test)
(require 'clojure.string)
… is about equivalent to:
(ns namespaces.test
(:require clojure.string))
Well, except for all the other stuff that ns
does.
Incidentally, while sternly recommending against it, the clojure.core
internals
use in-ns
extensively, sometimes spreading a namespace across files. But
who are we to judge?
refer
madness
The next thing that happens is that the clojure.core
namespace is
refer
’d into the context of the namespace that was just declared the
current namespace in in-ns
above. This is important because we probably
want access to the functions in clojure.core
. Ultimately, refer
ends up
calling the reference
method on the *ns*
(current namespace) var, which is
an instance of clojure.lang.Namespace
, which does a voodoo dance that results
in the relevant symbols being made available to us.
I didn’t explore too
deeply into what the undocumented with-loading-context
macro does,
accomplishes, because Java, but perhaps it has something to do with
how we’re able to use functions defined in clojure.core
to load clojure.core
.
Or perhaps not?
Getting loaded
Finally, the rigamarole surrounding the *loaded-libs*
refs is just there
so that load-lib
doesn’t reload a loaded lib (er, namespace) unless specifically
asked to. More about load-lib
right now:
What about require
and use
?
Popping a :require
(or :use
or :import
) into your ns
declaration just
adds a matching call to require
(or use
or import
) to the with-loading-context
part of the namespace macro expansion.
Require and use turn out to be different flavors of the same thing:
(defn require "...docs..." [& args] (apply load-libs :require args))
(defn use "...docs..." [& args] (apply load-libs :require :use args))
load-libs
is a collection of checks and whatnot wrapping a bunch of
calls to load-lib
, which we met above. In turn, load-lib
translates
the namespace into a path and does some other stuff and then calls load
on the path, while load
invokes the Java methd clojure.lang.RT/load
on it.
The eldritch magic contained therein in turn grabs the
file, compiles it, and puts all the top-level symbols it found into the current
namespace.
The import
macro is a bit different, deferring to import*
, another magic
clojure symbol that’s only defined in Java. The details aren’t clear to me,
but I think it’s safe to assume that ultimately this just uses Java reflection
to load the class.
Conclusion
So what can we take away from this mutually enlightening experience?
- The
ns
call is actually not so special, until you get to the special parts. - We could manually implement just about everything with
in-ns
andrequire
directly, but we shouldn’t. - I should talk to someone about my underlying Java phobia.
That’s all for tonight!