Caching Clojure deps on Gitlab

The Problem

I recently tried to get some Babashka to run some of the deployment tasks on our CI.

Gitlab doesn’t support caching directories outside of your project local diretory, which adds some pain when trying to cache dependencies that live outside the project directory.

The solution is to change the settings to store dependencies at the same directory as the project local files.

When I looked online I found tutorials suggesting to use -Sdeps flag on Clojure to do that.

But I had problems with this solution:

1. Babashka doesn’t support -Sdeps, which means we can’t use that for the :deps from bb.edn
2. Sometimes a task may spawn a sub-process in Clojure, and this sub-process may not respect the -Sdeps from the parent process

Because of those, I was having difficulty caching the dependencies properly.

The Solution

The solution I found was to change the actual files deps.edn and bb.edn to use a local directory. But if we do it strait in the files, the developers now will have the deps in the project, instead of shared in the system. To handle both cases, first, I add a commented line in deps.edn and bb.edn that has the desired setting for CI:

deps.edn

{;;CI :mvn/local-repo ".m2/repository" ;; DONT REMOVE THIS LINE
 :deps {...}}

bb.edn

{;;CI :mvn/local-repo ".m2/repository" ;; DONT REMOVE THIS LINE
 :tasks {...}}

We will remove the ;;CI comments on CI, so these configs get applied.

I made an extension called .clj-cache that I can use on different tasks to run all the required operations for the cache to work:

.gitlab-ci.yml

.clj-cache:
  variables:
    GITLIBS: ".gitlibs"
    DEPS_CLJ_TOOLS_DIR: ".deps-clj"
  before_script:
    # uncomment the ;;CI lines on both deps.edn and bb.edn
    - sed -i 's,;;CI,,g' deps.edn bb.edn
  cache:
    key:
      files:
        - deps.edn
        - bb.edn
    paths:
      - ./.m2/repository
      - ./.gitlibs
      - ./.deps-clj

test:
  extends: .clj-cache
  script:
    - bb test

You can see the test task using the extension and re-use it on any task you want to enable Clojure cache.

Follow closer

If you like to know in more details about my projects check my open Roam database where you can see development details almost daily.

Support my work

I'm currently an independent developer and I spent quite a lot of my personal time doing open-source work. If my work is valuable for you or your company, please consider supporting my work though Patreon, this way you can help me have more available time to keep doing this work. Thanks!

Current supporters

And here I like to give a thanks to my current supporters: