An alternative for clj->js

The Clojure lang was designed to make easy interop with the host language. That’s true for Clojure, and its true for Clojurescript; when we need to call some host function, I use the . notation, and everything is great!

Except that the data structures are different, JS uses JSON everywhere. In contrast, ClojureScript uses immutable data structures, and when those worlds need to integrate, the clj->js is the most common bridge.

Using clj->js is fine, but is it the only option?

JS Proxy

So another day that I found myself reading the documentation of JS Proxy from Mozilla. I find the idea quite interesting because that’s a way to trap the standard JS object accesses, making it possible to implement a transparent interface for custom data structures.

So what if instead of making a full deep data structure conversion (which is what happens in clj->js), we create a proxy and delegate the reads to the CLJS data structure directly?

The first thing about this is that generating a Proxy would have constant time (instead of depending on the data structure size), but the reads on CLJS data structure are more expensive than reads in plain JSON.

So it’s time to make a proxy and compare it to clj->js.

Proxy for Persistent Maps

Let’s start with a simple proxy implementation:

(defn map-proxy-get
  [t k]
  (get t (keyword k)))

; list keys
(defn map-proxy-own-keys [t _]
  (to-array (map #(subs (pr-str %) 1) (keys t))))

(defn map-proxy-has [t k]
  (contains? t (keyword k)))

(defn map-proxy-property-descriptor [t k]
  (if-let [x (find t (keyword k))]
    #js {:value        (val x)
         :writable     true
         :enumerable   true
         :configurable  true}
    js/undefined))

; proxy factory
(defn map-proxy [m]
  (js/Proxy. m
    #js {:get
         map-proxy-get

         :ownKeys
         map-proxy-own-keys

         :has
         map-proxy-has

         :getOwnPropertyDescriptor
         map-proxy-property-descriptor}))

Using this we can start to play:

(let [proxy (map-proxy {:name "Alex"})]
  (.-name proxy)         ; => "Alex"
  (js/Object.keys proxy) ; => #js ["name"]
  (js-in "name" proxy)   ; => true
  (js-in "other" proxy)  ; => false
  (js/Object.assign #js {:other "data"} proxy)
  ; => #js {:other "data", :name "Alex"}
  )

Nested data

This is fun, but the current implementation only goes one level deep, so if we try to do:

(let [proxy (map-proxy {:name   "Alex"
                        :mother {:name "Francisca"}})]
  (-> proxy .-mother .-name) ; => nil, because we are trying to read name on immutable map
  )

To fix this, we can make our get function wraps the value it’s reading; this adds a recursive property to this system making all children also a proxy, lazily:

(declare map-proxy)

; first lets creates function to do the wrapping
(defn js-proxy [x]
  (if (associative? x)
    (map-proxy x)

    x))

; change the get function to wrap the get with js-proxy
(defn map-proxy-get
  [t k]
  (js-proxy (get t (keyword k))))

; and with those updates, this works as expected:
(let [proxy (map-proxy {:name   "Alex"
                        :mother {:name "Francisca"}})]
  (-> proxy .-mother .-name) ; => "Francisca"
  )

Sequences

To finish the coverage of the basic structures, we also need to deal with sequences. I want to be able to do things like this:

(let [proxy (map-proxy {:name    "Alex"
                        :friends [{:name "Luke"}
                                  {:name "Laura"}]})]
  (-> proxy .-friends (.map #(.-name %))))

To make this work, we can extend the js-proxy to also handle sequences, it goes like this:

(defn array-push
  ([res] res)
  ([res x] (doto res (.push x))))

(defn into-js-array [xform from]
  (transduce xform array-push (array) from))

(defn js-proxy [x]
  (cond
    ; when sequential, convert to an array making each item a proxy as well
    (sequential? x)
    (into-js-array (map js-proxy) x)

    (associative? x)
    (map-proxy x)

    :else
    x))

Now the previous code runs as:

(let [proxy (map-proxy {:name    "Alex"
                        :friends [{:name "Luke"}
                                  {:name "Laura"}]})]
  ; note we are using the JS .map, not the Clojure one
  (-> proxy .-friends (.map #(.-name %)))
  ; => #js ["Luke" "Laura"]
  )

This covers all the standard access patterns!

The code for this is released in a library; you can check the full sources at https://github.com/wilkerlucio/js-data-interop/blob/master/src/main/com/wsscode/js_interop/js_proxy.cljs

Measure Time

Time to start benchmarking and see how it performs compared to clj->js.

To help with the measurements, I’ll use these helper functions:

(defn bench [times f]
  (let [start (system-time)]
    (dotimes [_ times]
      (f))
    (/ (- (system-time) start) times)))

So, note that all times you will see from now on are in milliseconds.

First, I want to test the lookup speed.

Before the numbers, here are some assumptions I have about each option:

• clj->js - This operation will do a scan in the whole data structure and build a new one, therefore, I expect the conversion time to increase with the size of the source data. After conversion, this looking up in the JSON is as fast as it gets in JS.
• map-proxy - The map proxy creates a proxy instance but doesn’t read anything from the data structure at the start; therefore, it should have a constant creation time. But given each lookup triggers a lookup in the CLJS data structure (which is about 10x slower than the JSON lookup), so as the user makes more lookups, the cost of using map-proxy increases when compared to clj->js.

To make this comparison, I’ll plot some charts that tell how long it takes to do N lookups in a structure of size S. So, for instance, how long clj->js takes to convert and do one lookup, vs how long a proxy takes to created plus one lookup.

The chart below is computing (here in your browser) the times and plotting it; the X-axis is how many reads (ranging from 0 to 30) are done. The Y-axis is how long it takes for the full operation. You can use the slider to modify the structure size (the number of keys in the map) and see how it varies. Play with:

This is running each test 100 times and getting the average.

Use the Re-run button a couple of times to stabilize the data, and for such micro measurement. Let us ignore the outliers.

As expected, as the structure size increases, the number of reads needed for proxy to cost the same processing time as clj->js increases.

On my machine, I see that at one entry size, the line takes 30 reads to cross, and increases with the size.

This means that, if you have large maps and only a few accesses on parts of it, map-proxy can get considerable faster than clj->js.

Here are the important code fragments related to this benchmark:

; generate a map of a specific size
(defn gen-map-sized [size]
  (zipmap
    (map #(str "entry-" %) (range size))
    (repeatedly #(rand-int size))))

; run the benchmark once for a range of lookup numbers
(defn bench-reads [{::keys [data f]}]
  (let [reads-range (range 31)
        keys        (conj (keys data) nil) ; add nil for also include a lookup miss
        keys-count  (count keys)]
    (into []
          (map-indexed
            (fn [n reads]
              {:x n
               :y (bench/bench 100
                    ; call the wrapper, which is clj->js or map-proxy to construct x
                    #(let [x (f data)]
                       (dotimes [k reads]
                         ; lookup by rotating across the data keys
                         (aget x (aget keys (mod k keys-count))))))}))
          reads-range)))

; chart plot code
(charts/line-chart
  {:data [{:id   "clj->js"
           :data (bench-reads
                   {::data data
                    ::f    clj->js})}
          {:id   "map-proxy"
           :data (bench-reads
                   {::data data
                    ::f    map-proxy})}]})

Spreading maps

In the JS world, because of the mutable state world, it’s widespread to copy a full JSON object into another, Object.assign is a core function to do it.

Now I’m going to compare how fast it is to spread proxy vs clj->js.

This time the X-axis is the map size (given the numbers of reads is equals the size in the spread operation).

The results here seem pretty close with map-proxy getting slower on larger sizes, but not by much.

Code for this benchmark:

(defn bench-spread [{::keys [f]}]
  (let [size-range (range 31)
        times      100]
    (into []
          (map
            (fn [n]
              (let [data (gen-map-sized n)]
                {:x n
                 :y (bench/bench times
                      #(let [x (f data)]
                         (js/Object.assign (js-obj) x)))})))
          size-range)))

(charts/line-chart
  {:data [{:id   "clj->js"
           :data (bench-spread
                   {::f clj->js})}
          {:id   "map-proxy"
           :data (bench-spread
                   {::f jsp/map-proxy})}]})

Sequences

For the next test, I want to see how the performance compares when processing sequences of things.

As a reminder, when I see a sequence (or a set) in my proxy implementation, I make a loop on the sequence and create proxies for every item (if the item is another sequence, do it over again). It breaks the constant time creation property, but the loops are only doing the quick proxy instantiation.

I see no difference in doing this, just a multiplication factor of what I saw before.

Code for this benchmark:

(defn bench-seq-reads [{::keys [data f]}]
  (let [reads-range (range 31)
        times       100
        keys        (conj (keys data) nil)
        keys-count  (count keys)]
    (into []
          (map-indexed
            (fn [n reads]
              {:x n
               :y (bench/bench times
                    #(let [x (f data)]
                       (.forEach x
                         (fn []
                           (dotimes [k reads]
                             (gobj/get x (aget keys (mod k keys-count))))))))}))
          reads-range)))

(charts/line-chart
  {:data [{:id   "clj->js"
           :data (bench-reads
                   {::data data
                    ::f    clj->js})}
          {:id   "proxy"
           :data (bench-reads
                   {::data data
                    ::f    jsp/jsp})}]})

Conclusion on benchmarks

To be honest, I think the results of these benchmarks is inconclusive; running the same tests on my machine generated different results (and I’m not sure why it changes; same code, the same machine).

Maybe I’m doing something wrong; if you know better, please reach me out.

I guess that there are some crazy V8 optimizations that sometimes fire, sometimes not, so in general, the performance differences between clj->js and map-proxy seems negligible most of the time.

If that were all, the project would be a failure, IMO, but that’s not all.

Proxy on Custom Map Types

The proxy can work with map-like data structures from Clojurescript. If you saw my last post here, I talked about the Pathom 3 Smart Maps.

If you didn’t have seen it, check it first and come back, I’ll wait :)

Ok, a proxy combined with Smart Maps means that we can get a JSON structure that triggers the Pathom property evaluation when accessed!

It looks like this:

; creating the classic full-name resolver
(pco/defresolver full-name [{:keys [first-name last-name]}]
  :full-name (str first-name " " last-name))

; generate the index
(def indexes
  (pci/register [full-name]))

(def person-base {:first-name "Andressa" :last-name "Matos"})

; create smart map
(def person (psm/smart-map indexes person-base))

; create proxy
(def person-proxy (map-proxy person))

; trigger resolver engine
(gobj/get person-proxy "full-name") ; => "Andressa Matos"

Final thoughts

In this post, I skipped the modification parts of the proxy entirely (set, delete, etc), I did some experimenting around it, but making it right is quite tricky, if you like to know more about this part, let me know, and I may do another write up around this in the future.

One real usage example of map-proxy you just saw here, to plot the data, I used the nivo library, which requires data in JS format, so I did the conversion using map-proxy instead of clj->js.

You can find the full source for the benchmarks here.

If you want to play with the proxy yourself, check https://github.com/wilkerlucio/js-data-interop.

I had a lot of fun playing with it. I think this deserves more exploration in real world situations, if you think you see a useful application, get in touch.

That’s all for today, cya.

Follow closer

If you like to know in more details about my projects check my open Roam database where you can see development details almost daily.

Support my work

I'm currently an independent developer and I spent quite a lot of my personal time doing open-source work. If my work is valuable for you or your company, please consider supporting my work though Patreon, this way you can help me have more available time to keep doing this work. Thanks!

Current supporters

And here I like to give a thanks to my current supporters: