Building Projects: tools.build and the Clojure CLI

tools.build and the Clojure CLI

tools.build is a library for building artifacts in Clojure projects, which are typically library .jar files for deployment to repositories like Clojars for others to use or application .jar files to run on servers on in containers.

tools.build provides functions to copy files and directories, to run arbitrary commands and capture their output (with special support for java commands), to easily run git commands, to create pom.xml files, to compile Clojure (and Java) code, and to build both .jar and .zip files.

This cookbook will offer examples that go beyond the basics in the official guide, based on real-world projects.

Executing functions with the Clojure CLI

The Clojure CLI was introduced by the core Clojure team in 2018 and focused on starting a REPL and running code, and managing dependencies using a deps.edn file.

Unlike Leiningen, which was more of a "batteries-included" approach, the CLI assumed that you would declare additional tooling through "aliases" in deps.edn that added extra dependencies and evolved, over time, to support both traditional command-line invocation -- a sequence of string arguments passed to a -main function -- and direct invocation of Clojure functions, passing a hash map of options from the command-line:

clojure -X my-proj.api/foo '{:bar 42}'

This will attempt to load the my-proj.api namespace and call the foo function, passing in the hash map {:bar 42}. If you have the following code:

;; src/my_proj/api.clj
(ns my-proj.api)

(defn foo [opts]
  (println (get opts :bar "No :bar passed!")))

Then it will print 42. You can also specify the hash map as individual key/value pairs on the command-line:

clojure -X my-proj.api/foo :bar 42

You can shorten that in two ways:

Add an alias to your deps.edn file that includes the default namespace you want to use:

;; deps.edn
{
 :aliases
 {
  :api {:ns-default my-proj.api}
 }}

Now you can omit the namespace from the command-line:

clojure -X:api foo :bar 42

Or you can specify a default function to run in an alias:

;; deps.edn
{
 :aliases
 {
  :api {:ns-default my-proj.api}
  :foo {:exec-fn my-proj.api/foo}
 }}

Now -X on its own will run that foo function:

clojure -X :bar 42

The -X option to the Clojure CLI stands for "eXecute function" and it uses the same default context as your project, so your source code and its dependencies are all available. This is useful for running tests, for example, using the Cognitect Labs' test-runner project:

;; deps.edn
{
 :aliases
 {
  ;; add this to :aliases in deps.edn:
  :test {:extra-paths ["test"]
         :extra-deps {io.github.cognitect-labs/test-runner
                      {:git/tag "v0.5.1" :git/sha "dfb30dd"}}}
 }}

and now you can run your tests with:

clojure -X:test cognitect.test-runner.api/test

which you can shorten by specifying the function you want to execute by default directly in the alias:

;; deps.edn
{
 :aliases
 {
  ;; add this to :aliases in deps.edn:
  :test {:extra-paths ["test"]
         :extra-deps {io.github.cognitect-labs/test-runner
                      {:git/tag "v0.5.1" :git/sha "dfb30dd"}}
         :exec-fn cognitect.test-runner.api/test}
 }}

Now you can run your tests with:

clojure -X:test

However, sometimes you want to run some tooling without the context of your project and the -T option is provided for that -- "execute Tooling": it omits the dependencies and paths from your project, using only those declared in the aliases you specify with -T (if any).

The functions in tools.build are intended to be used with -T and you typically declare a :build alias in deps.edn for this:

;; deps.edn
{
 :aliases
 {
  ;; add this to :aliases in deps.edn:
  :build {:deps {io.github.clojure/tools.build
                 {:git/tag "v0.9.4" :git/sha "76b78fe"}}
          :ns-default build}
 }}

The -T option implicitly sets :paths ["."] (as opposed to :paths ["src"] which is the default for -M and -X).

The code for the build processes would typically be in a build.clj file in the root of your project -- so its namespace would be build (since the file is relative to "." -- the project root). As shown above, the :ns-default key then allows you to omit the namespace portion when you invoke functions in build.clj:

(ns build
  (:require [clojure.tools.build.api :as b]))

(defn hello [opts]
  (println (str "Hello, " (:name opts "World") "!")))

Try this out by running that hello function:

clojure -T:build hello

clojure -T:build hello :name '"Build"'

The extra quotes in that second example are necessary to pass a Clojure string (with double quotes) through the shell as a literal value (with single quotes). You can do the same thing with:

clojure -T:build hello '{:name "Build"}'

The tools.build Library

The official guide provides three examples, and talks briefly about passing parameters into build task functions:

Those examples are a good starting point for simple projects but there is so much you can do with build.clj to automate all manner of things in larger projects:

  • Parameterizing builds using aliases in deps.edn
  • Multi-version testing
  • Continuous Integration pipelines
  • Automated deployments
  • Using a "build REPL"
  • Coordinating build tasks across multiple subprojects

For reference, here's the official documentation:

Before we start on more complex tasks, let's first look as a task to run an arbitrary process based on aliases.

Running Tasks based on Aliases

tools.build provides functions to construct a Java-based command-line and then run it as a subprocess, using a "basis" to control what classpath is passed to the java command.

Given the deps.edn above (containing the :build alias) and the build.clj above (containing the hello function), we're going to start out by adding a run function that will run a specific Java-based command-line. Then we'll parameterize it using aliases in `deps.edn:

(defn run [opts]
  (let [cmd (b/java-command {:basis     (b/create-basis)
                             :main      'clojure.main
                             :main-args ["-e" "(clojure-version)"]})]
    (b/process cmd)))

We can run this with:

clojure -T:build run

and we'll see the version of Clojure we're running: "1.11.1".

Since we will generally want the build to fail if the command exits with a non-zero status, we'll check the return value of b/process and throw an exception if the exit status is non-zero:

    (when-not (zero? (:exit (b/process cmd)))
      (throw (ex-info (str "run failed for " aliases) opts)))

We want to parameterize this so we can run any command-line we want, so we will pass :aliases in the opts and use that to construct the basis and also to retrieve both the :main class to run and the :main-args we want to use with it.

We will need to use tools.deps to process the aliases, so that we can retrieve data from those aliases in deps.edn:

(ns build
  (:require [clojure.tools.build.api :as b]
            ;; add this:
            [clojure.tools.deps :as t]))

;; change run to this:
(defn run [{:keys [aliases] :as opts}]
  (let [basis      (b/create-basis {:aliases aliases})
        alias-data (t/combine-aliases basis aliases)
        cmd        (b/java-command
                    {:basis     basis
                     :main      (get alias-data :main/class 'clojure.main)
                     :main-args (get alias-data :main/args
                                     ["-e" "(clojure-version)"])})]
    (when-not (zero? (:exit (b/process cmd)))
      (throw (ex-info (str "run failed for " aliases) opts)))))

We need the :aliases in create-basis so paths and dependencies from those aliases are taken into account for building the classpath. We've added the call to combine-aliases so that we can get the raw data from those aliases in deps.edn -- we'll get back a hash map which is the merge of the values identified by those aliases.

Note: we're using qualified keys to avoid conflicts with the keys that the CLI recognizes, and make it clear this is data we are providing. It's likely that we'll use the clojure.main default for all of our tasks but :main/class allows us to override that if we do need it at some point.

Next we're going to add :main/args to the :test alias in deps.edn:

  :test {:extra-paths ["test"]
         :extra-deps {io.github.cognitect-labs/test-runner
                      {:git/tag "v0.5.1" :git/sha "dfb30dd"}}
         :exec-fn cognitect.test-runner.api/test
         ;; add this:
         :main/args ["-m" "cognitect.test-runner"]}

If we pass the :test alias to our run task like this:

clojure -T:build run :aliases '[:test]'

we'll see the test runner output (assuming you don't have any tests yet):

Running tests in #{"test"}

Testing user

Ran 0 tests containing 0 assertions.
0 failures, 0 errors.

Let's add a test function to build.clj to make this easier to run:

(defn test [opts]
  (run (update opts :aliases conj :test)))

Since test is also a function in clojure.core, we'll suppress the warning that would cause by excluding test from being referred in:

(ns build
  ;; add this:
  (:refer-clojure :exclude [test])
  (:require [clojure.tools.build.api :as b]
            [clojure.tools.deps :as t]))

Now we can run the tests with:

clojure -T:build test

Multi-Version Testing

With the above run and test functions in place, we can automatically run our tests for multiple versions of Clojure. We'll add aliases to deps.edn that specify versions of Clojure to test against, and then use those in a new test-multi function in build.clj.

Add these aliases to deps.edn:

  :1.9  {:override-deps {org.clojure/clojure {:mvn/version "1.9.0"}}}
  :1.10 {:override-deps {org.clojure/clojure {:mvn/version "1.10.3"}}}
  :1.11 {:override-deps {org.clojure/clojure {:mvn/version "1.11.1"}}}

When these aliases are used in combination with other aliases, the default version of Clojure will be overridden with the specified version. We can see this by running clojure -T:build run :aliases '[:1.9]' and seeing "1.9.0" for example.

Here's our test-multi function:

(defn test-multi [opts]
  (doseq [v [:1.9 :1.10 :1.11]]
    (println "\nTest with Clojure" v)
    (test (update opts :aliases conj v))))