hoplon/javelin

Spreadsheet-like dataflow programming in ClojureScript.

Hoplon

Hoplon is a ClojureScript library that unify some of the web platform's idiosyncrasies and present a fun way to design and build single-page web applications.

Hoplon tightly integrates with Javelin to reactively bind DOM elements to the underlying Javelin cell graph.

Quickstart

Install deps-new if you haven't already:

clojure -Ttools install-latest :lib io.github.seancorfield/deps-new :as new

And then generate a starter project with:

clojure -Sdeps '{:deps {io.github.hoplon/project-template {:git/tag "v1.0.0" :git/sha "14361f1"}}}' -Tnew create :template hoplon/hoplon :name your/app-name

Example

A small bit of Hoplon:

(ns view.index
  (:require
    [hoplon.core  :as h]
    [hoplon.dom]
    [javelin.core :as j]))

(defn my-list [& items]
  (h/div :class "my-list"
    (apply h/ul (map #(h/li (h/div :class "my-list-item" %)) items))))

(def clicks (j/cell 0))

(defn hello []
  (h/div
    (h/h1 "Hello, Hoplon")
    (my-list
      (h/span "first thing")
      (h/span "second thing"))
    (h/p (h/text "You've clicked ~{clicks} times, so far."))
    (h/button :click #(swap! clicks inc) "click me")))

Browser Support

Hoplon has been thoroughly tested on desktop and mobile devices against the following browsers:

Documentation

• The Wiki
• Design Document (early version)

Demos

• Hoplon demos and demos source code

Developing Hoplon itself

# build and install locally
clojure -T:build ci :snapshot true
clojure -T:build install :snapshot true

Testing

This setup will run tests using chrome-webdriver.

Setup

npm install
npm install -g karma-cli

Run

; Run tests in simple compilation
clojure -T:build test

; Run tests in advanced compilation
clojure -T:build advanced-test

Contributors

This project exists thanks to all the people who contribute.

License

Copyright (c) Alan Dipert and Micha Niskin. All rights reserved.

The use and distribution terms for this software are covered by the Eclipse
Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php) which can
be found in the file epl-v10.html at the root of this distribution. By using
this software in any fashion, you are agreeing to be bound by the terms of
this license. You must not remove this notice, or any other, from this software.

javelin's Issues

advanced compilation breaking cell= macro

Not sure exactly what is happening here:

Only breaks with advanced compilation enabled and not reliably.

Using formula-of to explicitly scope email seems to workaround the issue.

Warnings with latest versions of libraries

With the following entries in my project file:

  :dependencies [[org.clojure/clojure "1.6.0"]
                 [org.clojure/clojurescript "0.0-2311"]
                 [tailrecursion/javelin "3.6.0"]
                 [org.clojure/core.async "0.1.319.0-6b1aca-alpha"]
                 [prismatic/dommy "0.1.3"]]

  :plugins [[lein-cljsbuild "1.0.3"]]

I am receiving a bunch of warnings like the following:

WARNING: Symbol ILookup is not a protocol at line 7 file:/C:/Users/mark/.m2/repository/tailrecursion/cljs-priority-map/1.0.3/cljs-priority-map-1.0.3.jar!/tailrecursion/priority_map.cljs

It appears that the formula cells are not working properly, due to these warnings.

Any idea what is causing these warnings, or can you suggest a combination of relatively recent dependencies known to work with javelin?

cell macro should handle keywords in operation position

Keywords in operation position aren't lifted or transformed into an equivalent lifted get form.

As a result, keywords can't be used as operations inside the cell macro, which is unintuitive.

We should support keywords as functions.

Cycle detection

It's possible to set cells in formulas, which makes it possible to inadvertently introduce cycles that cause propagation to run forever. This could happen when a formula function sets one of its dependencies, directly or indirectly, in the midst of propagation.

Currently, when this happens, the browser locks up and the user is left to find the cycle manually 😂

Ken Tilton tells us that in his Cells library, he dynamically maintains a set of all active formulas. When a formula is invoked that is already present in the set, an error is signaled and the relevant dependency chain is printed.

I think we could probably easily do the same.

In addition, for cells created via macro, or that can otherwise carry source/line/column information, in our error message we could even potentially point to relevant places in code, to expedite debugging.

Cell graphs

I loved the idea expressed by @alandipert on Slack about making the cell graph first-class. It would be great if we could explore the space and maybe narrow down the design in this issue.

While working on the JVM Javelin support I've implemented a very basic reified graph support which allows me to have separate concurrency-safe graphs of cells on this branch.

The other benefits I see:

• Custom error propagation strategy per graph. E.g. you can set the graph to behave like Excel and propagate #ERR on formula errors instead of the default throwing behaviour.
• Graph destruction. Useful when doing live-reload of a web-app, otherwise the formula cells keep accumulating on every reload.
• Cleanly separating independent components without having them inadvertently interfere with cells in other parts of the application.

Javelin and Wisp

Hi! Is there a way to use Javelin with Wisp?
I am using Meteor.js and there is meteor-wisp package.
I want to try Javelin with Meteor (I like the spreadsheet like data bindings), will it compile seamlessly with Wisp?

basic error with javelin-clj

Just trying javelin-clj out.

This works

user=> (def a (cell 0))
#'user/a
user=> (def b (cell= (inc a)))
#'user/b
user=> 

user=> [a b]
[#<Atom@4699fe0a: 0> #<Atom@5bb166dd: 1>]
user=> 

user=> (swap! a inc)
1
user=> 

user=> [a b]
[#<Atom@4699fe0a: 1> #<Atom@5bb166dd: 2>]
user=> 

This fails:

user=> (defn start []
  #_=>   (let [a (cell 0)              
  #_=>         b (cell= (inc a))  ]     
  #_=> [a b]))
#'user/start
user=>   

user=> (start)

user=> ClassCastException clojure.lang.Atom cannot be cast to java.lang.Number  clojure.lang.Numbers.inc (Numbers.java:110)

Using

REPL-y 0.3.2, nREPL 0.2.3
Clojure 1.6.0
OpenJDK 64-Bit Server VM 1.7.0_65-b32

[tailrecursion/javelin "3.6.3"]

Application slow on Ipad

I use to write applications Javelin and found a problem: dom element moves pretty slow, jerky. Code:

(defc mouse-x 0)
(defc mouse-y 0)

;;...

(.addEventListener (by-id "my-dom-element") "touchmove"
                     (fn [event]
                       (.preventDefault event)
                       (let [touches (get (js->clj (.-targetTouches event)) "0")]
                         (reset! mouse-x (get touches "clientX"))
                         (reset! mouse-y (get touches "clientY")))
                       (update-new-my-dom-element-position!))
                       false)

Rewriting the code for no-frp-style (using atoms) - work was quickly.
Then I did a benchmark:

(for [x 1000]
    (reset! cell-instance x))

time: ~300msec

(for [x 1000]
    (reset! atom-instance x))

time: ~8msec

What do you think, for this task is not suitable Javelin or i do something wrong?

Importing from MS Excel

Do you have any recommendations for importing data from MS Excel spreadsheets into a matrix of Javelin cells?

custom cell propagation logic

currently javelin checks if a value is = before propagating it and this is hardcoded.

the rationale is that we can save more CPU "downstream" in the graph than we spend calculating the equality of old/new values at each step of the propagation "upstream".

this is a fair and valuable assumption most of the time, but is not always true, for example i ran into an edge case with a datascript db in a cell. in this case, every new transact! call against the cell triggers an equality check where:

• the equality check can be very expensive as every datom in the db needs to be checked individually
• the equality check is almost guaranteed to return false as transact! almost always changes something in the db

in my personal experience, this unnecessary equality check for a modestly sized (< 500 datoms) datascript db was adding up to 13ms to UI updates in a hoplon app, which makes it hard to avoid jank in some situations.

datascript offers other challenges too, as entities are equal as long as their entity ids are equal, regardless of the values of their attributes... this means that entities cannot usefully be the values of cells, as regardless of their value in the db, they never propagate under a straight = check.

even ignoring datascript, when we look at hoplon, the main consumer/implementation of javelin we run into even more problems such as hoplon/hoplon#194 when we need to trigger callbacks with side effects in response to upstream cells changing their value, but using the downstream cell's value.

examples of this:

• triggering a :focus event whenever an upstream cell's predicate is satisfied using the true value of a downstream cell (workarounds include juggling uuids or cycling artificially between true/false - but false triggers a focusout which can be undesirable)
• trying to sort a list of form fields based on user input (see the above issue)
• adding an add-watch to a cell to respond to updates with a callback (add-watch also seems to only trigger callbacks after an equality check, whereas native atoms unconditionally trigger the callback)

i found the last example odd as the update callback for a lens is always triggered, even if sequential reset! calls against a lens "set" the lens to an equal value, but reset! calls against a normal cell don't trigger any callbacks if subsequent values are equal.

ultimately though, my understanding is that the inclusion of the = check in cell propagation is purely a performance tweak, effectively working as a network of "mini caches", and has nothing to do with the "correctness" of javelin (if anything it can undermine the correctness). most cache systems need a mechanism to deal with or explicitly avoid side effects and unusual invalidation logic eventually...

Destructuring namespaced keyword keys with cell-let

In ClojureScript 1.9.227 & Javelin 3.8.4

(cell-let [{:keys [user/email]} {:user/email "[email protected]"}]
        (js/console.log email))

Fails with

No such namespace: user, could not locate user.cljs, user.cljc, or Closure namespace "" at line 42 src/page/index.cljs
Use of undeclared Var user/email at line 42 src/page/index.cljs

(cell-let [{:keys [:user/email]} {:user/email "[email protected]"}]
        (js/console.log email))

Fails with

Use of undeclared Var page.index/email at line 43 src/page/index.cljs

While

(cell-let [{email :user/email} {:user/email "[email protected]"}]
        (js/console.log email))

or

(let [{:keys [user/email]} {:user/email "[email protected]"}]
        (js/console.log email))

Works as expected.

Lazy evaluation

Hi,

So basically there is no need to propagate updates to cells which have no listeners.

Imagine you have this cell structure and you are building app which tracks some other cell i. So, when a updates there is no need to compute cells e, f, g and h and we can defer that computation to deref call.

Does it make sense?
Thanks.

`set-cell!=` causes `Use of undeclared Var javelin.core/c`

I think https://github.com/hoplon/javelin/blob/master/src/javelin/core.clj#L204 is missing a unquote on c on the right side of the binding.

ClojureScript 0.0-1853 won't compile javelin

Not a javelin bug, just an FYI for anyone that hits the same problem: ClojureScript 0.0-1853 (and any later release until CLJS-571 is fixed) will not compile javelin.

The workaround would be to avoid the %& variadic literal fn arg notation in specials.cljs, but I presume the reader/cljs bug will be considered significant.

Programmatic cycle detection

Background

Currently a cycle in the Cell graph trashes the browser and no additional information is provided about its cause.

The current behavior is tenable because:

1. Browsers handle infinite JavaScript loops gracefully, and the browser's reload button be clicked no matter what.
2. Cycles are usually not hard to find in a mostly static dataflow scenario, as cells tend to have a syntactic representation. Source bisection can be used to discover which edit introduced the cycle.

The Enhancement

In more dynamic scenarios the cause of a cycle can be more difficult to determine. These are scenarios in which the shape of the dataflow graph is changing while the program is running, instead of it having a fixed shape determined in source code.

I think a worthy enhancement would be to detect cycles at runtime and notify the developer of their presence, programatically, with as much information as possible without introducing significant overhead.

This would ease development of dynamic dataflow applications and would also amene dynamic Javelin graphs to generative testing.

Implementation Idea

The Flapjax paper mentions the cycle detection approach taken in Garnet and Amulet:

...they do support cyclic constraint networks, without the need for explicit time delays as Flapjax requires. Instead, they resolve cycles with a simple depth-first "once-around" algorithm, which stops propagation when it returns toa node that has already been update.

I think we can do the same thing very easily. In propagate!, we store the cells we've propagated so far in a set. When we encounter a cell we've already processed a second time, we throw an exception.

The Garnet/Amulet "Lessons Learned" paper elaborates slightly:

Support for Cycles: A constraint is evaluated at most once if it is in a cycle.
If the constraint is asked to evaluate itself a second time, it simply returns its
original value.

Uncaught Error: No protocol method ISwap.-swap! defined for type tailrecursion.javelin/Cell: [object Object]

This is because of the new IAtom stuff in cljs version 0.0-2173. The workaround for this for the moment is to use 0.0-2156. A back-compatible fix must be devised.

set! in cell body has no effect

I would expect this interaction to show obj.foo being updated:

(def obj (js-obj))
; #<[object Object]>
;= nil
(def a (cell 0))
; #<[object Object]>
;= nil
(def b (cell (do (println a)
                 (set! (.-foo obj) a))))
; 0
; #<[object Object]>
;= nil
(js->clj obj)
;= {}
(reset! a 12)
; 12
;= 12
(js->clj obj)
;= {}

If I get the set! out of the body of cell, then it works as expected:

(def obj (js-obj))
; #<[object Object]>
;= nil
(def a (cell 0))
; #<[object Object]>
;= nil
(def b (let [set-foo #(set! (.-foo obj) %)]
         (cell (do (println a)
                   (set-foo a)))))
; 0
; #<[object Object]>
;= nil
(js->clj obj)
;= {"foo" 0}
(reset! a 12)
; 12
;= 12
(js->clj obj)
;= {"foo" 12}

Clojure support?

Are there any plans or interest in using Javelin from Clojure? I was thinking earlier today that it would be nice to have cells on the backend too.

Yeah, core.async works for most of the same problems as Javelin, but the cell abstraction is often easier and simpler to use than channels.

Cells created with literal data structures do not propagate updates

(def root (cell {:a :b}))
; #<[object Object]>
;= nil
(def a (cell (:a root)))
; #<[object Object]>
;= nil
@a
;= :b
(swap! root assoc :a :c)
;= {:a :c}
@a
;= :b

Same interaction, but with the initial value taken "by reference":

(def start {:a :b})
;= {:a :b}
(def root (cell start))
; #<[object Object]>
;= nil
(def a  (cell (:a root)))
; #<[object Object]>
;= nil
@a
;= :b
(swap! root assoc :a :c)
;= {:a :c}
@root
;= {:a :c}
@a
;= :c

priority-map 0.0.7

Javelin uses priority-map version 0.0.2, which is really old.

Notify uses priority-map version 0.0.7, but restricting the requires to just priority-map-keyfn, which is not in version 0.0.2, seems to do the trick.

Even so, Javelin should probably be updated to use the newer version.