redplanetlabs/specter

Clojure(Script)'s missing piece

specter's Issues

Does Specter support Equi-joins of 2 collections?

Hey Nathan, Cool project! More of a question rather than an issue, and I guess native clojure joins aren't too too crazy, but does Specter support joins, e.g. equi-joins based on 2 or more keys?
e.g. better than

(def test-1
[{:k1 "a" :k2 "a" :metric 10}
 {:k1 "a" :k2 "b" :metric 20}
 {:k1 "a" :k2 "c" :metric 30}
 {:k1 "b" :k2 "a" :metric 40}
 {:k1 "b" :k2 "b" :metric 50}
 {:k1 "b" :k2 "c" :metric 60}
 {:k1 "c" :k2 "a" :metric 70}
 {:k1 "c" :k2 "b" :metric 80}
 {:k1 "c" :k2 "c" :metric 90}
 ])

(def test-2
[{:k1 "c", :k2 "c", :metric-2 9} 
 {:k1 "b", :k2 "b", :metric-2 5} 
 {:k1 "a", :k2 "b", :metric-2 2} 
 {:k1 "c", :k2 "b", :metric-2 8} 
 {:k1 "b", :k2 "c", :metric-2 6} 
 {:k1 "a", :k2 "a", :metric-2 1} 
 {:k1 "b", :k2 "a", :metric-2 4} 
 {:k1 "c", :k2 "a", :metric-2 7} 
 {:k1 "a", :k2 "c", :metric-2 3}])

(pprint (sort-by (juxt :k1 :k2) (clojure.set/join test-1 test-2)))

({:metric-2 1, :k1 "a", :k2 "a", :metric 10}
 {:metric-2 2, :k1 "a", :k2 "b", :metric 20}
 {:metric-2 3, :k1 "a", :k2 "c", :metric 30}
 {:metric-2 4, :k1 "b", :k2 "a", :metric 40}
 {:metric-2 5, :k1 "b", :k2 "b", :metric 50}
 {:metric-2 6, :k1 "b", :k2 "c", :metric 60}
 {:metric-2 7, :k1 "c", :k2 "a", :metric 70}
 {:metric-2 8, :k1 "c", :k2 "b", :metric 80}
 {:metric-2 9, :k1 "c", :k2 "c", :metric 90})
nil

?

Thanks,
Avram

No longer works with Clojure 1.6

That seems premature?

Adding ^:const to precompiled path var causes exception when using it

    (def ^:const y (spec/comp-paths [:a :b]))
    => #'dove.core.repl/y
    (spec/compiled-select-first y {:a {:b 1}})
    CompilerException java.lang.IllegalArgumentException: No matching field found: select-executor for      class com.rpl.specter.impl.ExecutorFunctions, compiling:... 

Without ^:const it will work properly.

Is this intentional or a bug?

Parameterized selectors

Right now, selectors that require parameterization (such as keypath) cannot be precompiled, since the parameters need to be known first. Precompilation is important as it enables very high performance. Having explicitly parameterized selectors that can be composed together would allow pre-compilation of any selector path. This would then make Specter an ideal interface for data structure manipulation in almost all scenarios as you're no longer taking a performance hit from using Specter in situations that require parameterization.

In terms of design, the basic idea is this: if selector1 takes one param and selector2 takes two params, then (comp-paths selector1 selector2) takes in three params in the order in which the selectors were given. Then, using that selector looks something like this:

(def combined (comp-paths selector1 selector2))

(compiled-select (combined 1 2 3) structure)

Returning sub-map with select, not value

Hello! Been a while.

I'm writing some ETL code which basically filters a map by another map. Specter seems right for this. For example:

{:a 
  {:b {:c 4 :d 5} 
   :e {:f 6 :g 7} 
   :h {:i 8 :j 9} }}

filter by [ [:a :b :c] [:a :e :f] ]

 ---> {:a 
        {:b {:c 4}}
         :e {:f 6}}

I think the main thing I'm not understanding is getting select to return a sub-map, instead of the value. For instance (select [ALL :a :b :c]) returns "4" instead of {:a {:b {:c 4}}.

What concept am I missing?

Add declarepath functionality

You can build recursive selectors with protocol paths, but sometimes you want recursive selectors that don't involve protocol paths. The best way to accomplish this would be to declare the path before defining it so it can be used like a normal path (including in its own definition). This design is superior to something like having a special "THIS" selector because it would allow for mutual recursion as well.

Redo documentation

Right now the "docs" are spread out over a blog post, youtube video, and examples on the README. They are also incomplete. The wiki for this repository should consolidate existing information and be extended with more docs. Two kinds of documentation are needed most:

• Overview of features: navigation, selection, transformation, replace-in, protocol to make new navigators, higher-order navigators, precompilation, late-bound parameterization, recursive navigation, protocol paths
• Reference documentation for built-in navigators: Should describe what they do and provide examples of usage.

defpath should error if function names are incorrectly specified

Current behavior seems to interpret missing select_/transform_ as identity navigation.

How can I transform the value at a specified key in a nested map?

The key may occur at any depth in the nested map structure. I can select the key with
(s/walker #(= % my-key))
but how can I access its value to transform that, not the key?

Special case to avoid late-bound parameterized code path

When a late-bound parameterized selector is given params directly (and not composed with anything else), Specter should use an alternate code path which puts the parameters in the lexical closure immediately instead of sneaking it in later with the params/params-idx logic. So (keypath "a") should return a StructurePath with no params rather than a RichPath with params attached to it.

Although late-bound parameterization is fast, it does incur some additional overhead – reading from the parameter array and lifting a StructurePath to a RichPath. Implementing this issue will remove that overhead for these special (though not that uncommon) cases.

Because late-bound parameterized selectors are defined using macros, it should be fairly easy to capture an alternate function to use in this case.

Clean up the DSL

You might be blind to this, as you've been using specter for so long, but I found the specter DSL somewhat aesthetically lacking. Here are my major gripes, in no particular order:

• Mixing of lower-case and upper-case selectors.
• Duplicating existing names, e.g. srange for range and filterer for filter.

It seems to me that both of these could be easily fixed by just turning functions like transform and select into macros and then having that macro call a transformer on the navigation spec, which would transform first to FIRST, range to srange etc before delegating to the existing implementation.

How do you compose select vectors?

Hey @nathanmarz - really loving specter, and I'm trying to learn it to get the most power out of it. In particular, I'm wondering if you could help me understand this bit of code I wrote. This function walks into a Clojure map m, collects all of the keywords, then filters for a particular key k, and counts all instances of k:

(defn cardinality [m k]
  (first (select [(filterer (partial = k)) (view count)]
                 (select (walker keyword?) m))))

This works, but I have to call first and make two calls to select. When I try to compose the select vectors like this, I get back IllegalArgumentException:

(defn cardinality-2 [m k]
  (select [(walker keyword?)
           (filterer (partial = k))
           (view count)]
          m))

Ultimately, after experimentation, I refactored to:

(defn cardinality-3 [m k] (count (select (walker (partial = k) m))))

But I still don't understand why cardinality-2 didn't work. I'm thinking that walker maybe doesn't compose like filterer and the other functions? The blog post and docs didn't really give me a good intuition about this in particular, so I was hoping you could help out when you have a chance.

More vector examples?

There are plenty of examples about working with map keys and vector indices (and ranges), but not many about working with values within vectors. My current issue is taking this:

   [:declare-statement
    [:type
     [:function-type
      "ƒ"
      [:function-parameter-type [:type [:identifier "string"]]]
      [:function-return-type [:type [:identifier "string"]]]]]]

and turning it into (by running each [:type ...] sequence through (get-in item [1 1])):

   [:declare-statement
      "ƒ"
      [:function-parameter-type "string"]
      [:function-return-type "string"]]

How would this be done with specter? I think there should be more vector-based examples, including:

1. An obvious replacement for (get-in foo [0 1 1 ...])
2. As described above, a way to walk and update vectors based on a given nth value

Support Number and String keypaths by default

Would you accept a pull request to extend StructurePath to java.lang.Number and java.lang.String by default?

It's quite common to use these as map keys, and I find wrapping them in (keypath) tends to clutter up code.

Dynamically dispatch compiled paths for eg transforms using just one function

So won't have to switch between using compiled-transform and the regular transform function, letting us just change the type of path we send (compiled or not) depending on eg debug/production builds.

Or maybe I'm missing something obvious :)

Select rest after match

Hey 😄

I'm trying to use specter to test hiccup structures. I would like to know if there's a way to select the rest of the expression after finding a match.

Example:

(def hiccup [:div [:a "content"]])

(select-first [(walker vector?) (filterer #(= :a (first %))) some-selector] hiccup)
;; "content"

This is useful to test my reagent components without rendering them.

Non-parameterized protocol paths fail to compose when extended with a path that captures params internally

Test case:

(defprotocolpath Foo [])


(extend-protocolpath Foo
  Object nil
  clojure.lang.PersistentHashMap [(keypath :a) Foo]
  clojure.lang.PersistentArrayMap [(keypath :a) Foo]
  )

(select Foo {:a 1})

select/update on empty selector produce error

(select [] {:a 1 :b 2})
(update [] identity {:a 1 :b 2})

select should return the data structure in a vector, while update should just return the data structure.

Make a utility to generate paths for record fields that use optimized access

e.g.

(defrecord Foo [a b])
(define-record-field-paths Foo)

This would generate paths Foo-a-path and Foo-b-path. The select code path would use direct field access rather than do a keyword lookup to achieve a performance improvement.

selecting/filtering on vector indices

Hi,

I don't know if there is a better place to have a conversation about using specter so I am posting here.

I need to select and transform my data by filtering on vector indices. I don't know if it is already supported with the current api. As I could not find out how to do it, I have added filterer-indexed which takes two parameters similar to map-indexed (ix and item), I can submit them as I patch if this is desired, though the implementation simply copies existing filterer and filter+ancestry.

I have tried a few approaches. I have gotten furthest by by using a 3 step process

1. transforming data (using transformed and/or view) into [index item] vectors
2. filtering appropriate items
3. transforming data back to original shape (loosing index / second)

but this does not work correctly for transforms mainly because the transform back doesn't get applied for non selected items. So if for example i wan't to for example i want to up-case the second item in a vector ["a" "b" "c"] I get a result of [[0 "a"] "B" [2 "c"]]...

Making paths available to callers of `select`.

https://github.com/marick/structural-typing/ uses Specter to allow a sort of type checking, where types can be described like this:

(type! :Example {[:x ALL :y ALL] [integer?]})   ; there's a non-side-effecting version too

When producing a type error, I display the path (as above). I'd like to display it like so:

[:x ALL :y ALL][2,3] should be `integer?`; it is `"2"`

... which would identify which particular element of the nested structure is wrong. As it is, the best I can do is to display [:x ALL :y ALL][6] and let the user figure out which is the 6th leaf in the structure being checked.

I'd like to do some magic lifting-style stuff where the path gets passed along, state-monad-like, for the delectation of clients who want more than just the leaf value. But I'm not smart enough to see it, if indeed it's there to be seen.

If I'm pointed to a clever twist, I'll do the scut work of implementing and documenting it.

update with FIRST on an empty collection has odd behavior

From @marick:

 (update [FIRST ALL] #(do (prn :called %) "never used") []) => [nil])

A simpler case is below:

(setval FIRST :a []) => [:a]

The following throws an error, which is inconsistent with the previous case:

(setval LAST :a [])

Either FIRST and LAST should have the same behavior on empty collections, or they should throw an error when doing selects/updates on empty collections.

Non-LazySeq SetVal method?

Hi @nathanmarz et al!

Playing around a little with this library, it appears that transform and setval will return a LazySeq and select will return a PersistentVector. Does Specter provide any way of controlling the return collection type?

Thanks for your time and for your work!
-Stephen

Add more late-bound parameterized paths

There should be late-bound parameterized paths for:

• filtering (a parameterized version of using a function directly)
• view
• walker

Operation with zippers

It seems that specter should would well with zippers.
That is zippers are good at relative movement and updates and specter does well with (absolute) paths.
My questions are:
Does specter interoperate with zippers?
Do you have any examples?
Are there any pitfalls?

My present case involves walking the zipper and running a multi-function on each node.
The default function is to leave the node unchanged but if it meets certain criteria
then I want to update the zipper. The zipper is implemented as a PersistentVector pair.
The first element is the child tree and it seems like any specter path into that tree should be fine.
What do you think?

Replace object-array with something faster for JVM implementation

Did some profiling and found that Clojure's object-array is unacceptably slow for binding late-bound params, mostly because of the "intValue" invocation and the boxing/unboxing that happens. This should be replaced with a direct invocation of creating an object array. This will require some Java code.

Value collection does not work with multi-path + transform

Example:

(transform (multi-path VAL) (fn [a b] b) :a)
;; => Exception thrown

The solution is to change variable-pathed-path and fixed-pathed-path to generate rich handlers rather than the lean ones they're currently generating.

Clojurescript support

Make protocol paths work in cljs

The protocol path implementation currently only works in Clojure because it relies on extend and reified protocol objects, both of which are unavailable in ClojureScript

Make new code path for select-first

Using selectors like ALL is inefficient when doing a select-first, since it will continue traversing beyond the first element found. For ALL, a lazy version would help with this but then adds the overhead of lazy sequences – and you would still have the overhead of wrapping the result in a sequence. Having a special code path for select-first would allow this use case to be handled more efficiently.

This issue would also make selected? much more efficient.

This should be done as a new protocol, with a default implementation over select* provided if the protocol impl is missing.

Keys cannot be strings

(select ["name"] {"name" 1}

Maybe I'm using this wrong or have the nesting wrong, but it seems like strings cannot be keys in maps.

Speed up ClojureScript implementation

I haven't benchmarked it yet, but there are two glaring parts of the code likely to cause performance problems:

1. Field access using "aget": One way to solve this would be to have two separate "field" macros in two separate files and switch in the namespace which one to refer to depending on target platform.
2. structure-path-impl/etc. functions in CLJS just make a protocol invocation instead of looking up the function a priori. While I was able to find the appropriate function by digging into the prototype, I was unable to execute it in ClojureScript land. I'm not sure why and that's why it relies on doing protocol invocation at the moment.

Precompile automagically

Pre-compilation is basically always wanted, and the only reason not to do so is because there's a cost to doing it now (more typing for new code, refactoring of old code, or lack of knowledge on the user's part) .

By turning transform and select into macros, it looks to me like the pre-compilation of navigation specs can be done automatically.

This would have several benefits:

• Easier to use, by not requiring the user to write more, or transform existing, code to get pre-compilation
• Easier to learn, by turning pre-compilation into an implementation detail and not something the user has to think about.

Trying to understand difference between `[ALL :a even?]` and `[ALL :a :b]`

I've been writing some executable examples and have come across behavior I don't understand:

  ;; The following can be read as "select all the even values of :a":
  (select [ALL :a even?] [{:a 1} {:a 2}]) => [2] 
  ;; Notice that the first map does not contribute to the output.

  ;; And then the following might be read as "select all the :b values of :a". 
  ;; But it is not.
  (select [ALL :a :b] [{:a "a"}
                       {:b "b"}
                       {:a {:b "ab"}}
                       {:a {:c "ac"}}]) =not=> ["ab"]
  ;; In this case, maps with an :a but without a :b DO contribute:
  (select [ALL :a :b] [{:a "a"}
                       {:b "b"}
                       {:a {:b "ab"}}
                       {:a {:c "ac"}}]) => [nil nil "ab" nil]

  ;; Why is that? Note that it's NOT that the last component is returning `nil` rather than `false`.
  (select [ALL :a #(if (even? %) true nil)] [{:a 1} {:a 2}]) => [2])

What am I missing?

Add "locate" functionality

This is an idea for major new functionality for Specter. Not all the details are worked out yet.

The idea is to have a new top-level function called "locate" that provides brand-new functionality on paths. This function would be as significant as "select" or "transform" and would add a new dimension to the library. "locate" returns paths to the individual elements found, and then those paths can be used to do further selection/manipulation. Here's a speculative example:

(def locations (locate [ALL :a ALL] structure))
(if (>= (count locations) 2)
  (->> structure
         (transform (first locations) inc)
         (transform (second locations) dec)))

The locations could also be introspected to know – in this case – which indices in the two sequences the locations correspond to.

A "location" works like any other path but it can navigate back to itself in the data structure faster than using the general path that found it in the first place (for example, because it knows the specific indices in the sequence to jump to rather than do a scan). It is also specific. So you could have workflows that locate something of interest deep in the data structure, do further queries on that location along with retrieving relevant information from other sources, and then finally do a transformation on that location when all the info has been put together.

Implementation-wise, any Specter path statically implies what its location path will be. [ALL :a ALL]'s location path would be [keypath :a keypath] (minus working out some details about dealing with non-vector sequences). So the specific location is just a parameter list for those parameterized selectors.

Could not locate com.rpl.specter

Edit: My mistake, this is a duplicate of #16 - I mistook which branch I was on in our code.

Running on Clojure 1.6, I get the following error with specter v0.7.0 and v0.7.1:

user> (require 'com.rpl.specter)
FileNotFoundException Could not locate com/rpl/specter__init.class or com/rpl/specter.clj on classpath:   clojure.lang.RT.load (RT.java:443)

Some parts of specter are definitely available, though:

user> (require 'com.rpl.specter.defhelpers)
;; => nil

Make new ALL implementation for ClojureScript

ALL currently relies on core.reducers which much of ClojureScript appears to be incompatible with (e.g. #50). A different implementation specific to ClojureScript that still preserves collection types should be developed

Switch to cljx

While using .cljc for combined Clojure and ClojureScript compatibility is easy, it forces users to use Clojure 1.7. cljx should be used instead so that Specter is compatible with Clojure 1.6.

Exception when transforming maps where size > 8 in ClojureScript

I'm getting the following on version 0.9.1 in cljs in the browser (no exception for clj on the JVM), and the problem seems to manifest when the size of the provided map is greater than 8:

(prn
  (s/transform
    [s/ALL s/LAST]
    identity
    {"0" 0
     "1" 1
     "2" 2
     "3" 3
     "4" 4
     "5" 5
     "6" 6
     "7" 7
     "8" 8}))

;;=>

Uncaught Error: No protocol method IReduce.-reduce defined for type cljs.core/PersistentHashMap: {"3" 3, "4" 4, "8" 8, "7" 7, "5" 5, "6" 6, "1" 1, "0" 0, "2" 2}

cljs$core$missing_protocol @ core.cljs:261
cljs.core._reduce.cljs$core$IFn$_invoke$arity$3 @ core.cljs:557
cljs$core$_reduce @ core.cljs:557
clojure.core.reducers.folder.clojure.core.reducers.t_clojure$core$reducers35621.cljs$core$IReduce$_reduce$arity$3 @ reducers.cljs:90
cljs.core.reduce.cljs$core$IFn$_invoke$arity$3 @ core.cljs:2182
cljs.core.into.cljs$core$IFn$_invoke$arity$2 @ core.cljs:4468
cljs$core$into @ core.cljs:4462
com.rpl.specter.impl.AllStructurePath.com$rpl$specter$protocols$StructurePath$transform_STAR_$arity$3 @ impl.cljs:513
com$rpl$specter$protocols$transform_STAR_ @ protocols.cljs:5
com.rpl.specter.impl.structure_path_impl @ impl.cljs:170
(anonymous function) @ impl.cljs:201
com.rpl.specter.impl.combine_same_types.combiner @ impl.cljs:281
(anonymous function) @ impl.cljs:76
com$rpl$specter$impl$compiled_transform_STAR_ @ impl.cljs:458
com$rpl$specter$transform @ specter.cljs:89

Running on a map of size 8 works fine:

(prn
  (s/transform
    [s/ALL s/LAST]
    identity
    {"0" 0
     "1" 1
     "2" 2
     "3" 3
     "4" 4
     "5" 5
     "6" 6
     "7" 7}))

;;=>  {"0" 0, "1" 1, "2" 2, "3" 3, "4" 4, "5" 5, "6" 6, "7" 7}

Anybody else running into this?

Protocol path extensions should verify correct number of parameters

If number of parameters of extended path does not match protocol path declaration, an error should be thrown

Howto: cause short-circuiting selects to return collected (`putval`) values

Summary: how can an implementation of select* get access to the collection collect-val builds?

---

ONLY is like ALL, except it is to complain whenever it descends into a non-singleton collection. Currently, it complains by throwing. Instead, I want it to return a special value, as that will allow a better error message (etc).
Here's an implementation that doesn't work:

(extend-type OnlyVariantType
  sp/StructurePath
  (select* [this structure next-fn]
    (cond (not (coll? structure))
          (boom! "%s is not a collection" structure)

          (not= 1 (count structure)) ; TODO: make this work for infinite collections
          :return-some-value-that-indicates-truncation
          ; (->Truncation explain/err:only-wrong-count structure))

          :else
          (next-fn (first structure))))
  (transform* [& _] (no-transform!)))

This works as you'd expect for a simple select:

user=> (select [:a ONLY] {:a [3]})
[3]
user=> (select [:a ONLY] {:a []})
:return-some-value-that-indicates-truncation

... and for a successful traversal with a select that uses putval:

user=> (select [(putval "HI!") :a ONLY] {:a [3]})
[["HI!" 3]]

However, I was initially surprised when it didn't work for a short-circuited traversal:

user=> (select [(putval "HI!") :a ONLY] {:a []})
:return-some-value-that-indicates-truncation

The reason is pretty clear: whatever is the last step in a traversal has to conj the leaf/return value onto the collected values. But for the life of me, I can't find an accessor that a select* function can call. (It would also be convenient, though not necessary, for the select* to be able to "pass" a modified collection to the rest of the selector path.)

Filtering using functions

Not sure if this is the right channel for general questions regarding the usage of specter, so please point me in the right direction if not. I have the following hashmap that I'd like to navigate:

(def store {"foo" {[1 2] {:value 1} [2 3] {:value 2} [3 4] {:value 3}}})

Specifically, I'd like to get all values where the key includes a specific number, and saw that functions can also be used as filters:

(defn my-selector [number record]
  (some #{number} (flatten (keys record))))

However, the results are not as I expected:

(select ["foo" (partial my-selector 2) :value] store)
;; => [nil]
(select ["foo" (partial my-selector 2)] store)
;; => [{[2 3] {:value 2}, [3 4] {:value 3}, [1 2] {:value 1}}]

Any pointers as to where I'm going wrong would be much appreciated.

Creating a group-by path

EDIT: I'd like to label this as a "question" but I don't know how to add labels to issues since I don't have the gear button on the right next to "Labels". Sorry about that.

Hi,

I'm just starting to play with Specter (I'm using ClojureScript), and I am impressed at how simple the API is, and how easy it is to think in Specter.

I have data that I sometimes need to group-by multiple times:

[{:type "person" :id "1"},
 {:type "person" :id "2"}]

grouped by :type and :id becomes

{"person" {"1" {:type "person" :id "1"}
           "2" {:type "person" :id "2"}}

It seemed like I could write a GROUP-BY path for specter. Here is my first pass:

(defpath GROUP-BY [f]
  (select* [this structure next-fn]
           (let [grouped (group-by f structure)
                 result (for [k (keys grouped)]
                          (let [v (get grouped k)
                                nv (next-fn v)]
                            [k nv]))]
             (into {} result)))
  (transform* [this structure next-fn]))

(def data {:entities [{:id "1" :type "person" :name "Joe"}
                      {:id "2" :type "person" :name "Bob"}
                      {:id "1" :type "robot" :name "Marvin"}
                      {:id "2" :type "robot" :name "Bender"}]})

(do
  (println "******************")
  (select [(GROUP-BY :type) (GROUP-BY :id) FIRST] (:entities data)))

(I use FIRST because group-by gives a map of keys to vectors, and i know that my groups will only have one element because I am grouping by a unique id)

It's almost working, but I am getting back vectors in the nested map:

{"person" {"1" [{:type "person" :id "1"}]
           "2" [{:type "person" :id "2"}]}

If I don't use FIRST, then I get back a vector inside of another vector:

{"person" {"1" [[{:type "person" :id "1"}]]
           "2" [[{:type "person" :id "2"}]]}

Using multiple FIRSTs at the end of the path doesn't get rid of the final vector. Actually it returns the first map entry inside of a vector (e.g. [[:type "person"]])

Is there a good way to do this? Or is what I'm going just plain wrong? :)

Implement protocol paths

Sometimes its useful to have polymorphism in how a path navigates through a structure. For example, if you had:

(defrecord Account [money])
(defrecord Person [account])
(defrecord Family [accounts])

And your structure was a list of either Person or Family, then navigating to all the accounts requires polymorphism on the type of record you're looking at. A solution to this would be protocol paths, like so:

(defprotocolpath AccountPath [])

(extend-protocolpath AccountPath
  Person :account
  Family [:accounts ALL]
 )

(select [ALL AccountPath :money] list-of-persons-and-families)

Protocol paths are defined with the amount of parameters they are expected to have for integration with parameterized precompilation. Protocol paths will automatically precompile the result paths for each type.

Protocol Issue when working with Specter and Om

I'm getting the following error when trying to use specter to query my nested dictionary No protocol method IReduce.-reduce defined for type om.core/IndexedCursor: [object Object]

Pic of Stack Trace

Query
(s/select [:events s/ALL] the-comp)

Full Code Sample

(defn score-entry-widget [data owner]
  (om/component
    (print data)
    (let [the-comp (:competition data)
          event-id (:current-event data)
          measures (s/select [:events s/ALL] the-comp)]
      (html [:div {:class "score-entry"}
              [:div {:class "input"}
                (for [m measures]
                  [:label 
                    [:input {:type "text"}] 
                    [:span (:name m)]])]
              [:button "Submit"]]))))

Sample Payload of the-comp

{:name "Fittest Games",
 :competitors
 [{:id 1,
   :name "Jeff Vanlandingham",
   :gym "CrossFit Jaakarhu",
   :division-id 1,
   :scores {1 {:value 20, :uom :sec}, 2 {:value 20, :uom :reps}}}
  {:id 2,
   :name "Reid Reagan",
   :gym "CrossFit Jaakarhu",
   :division-id 1,
   :scores {1 {:value 21, :uom :sec}, 2 {:value 21, :uom :reps}}}
  {:id 3,
   :name "Albert Leyva",
   :gym "CrossFit Dallas Central",
   :division-id 1,
   :scores {1 {:value 22, :uom :sec}, 2 {:value 22, :uom :reps}}}
  {:id 4,
   :name "Jeremy Kampen",
   :gym "CrossFit Jaakarhu",
   :division-id 1,
   :scores {1 {:value 23, :uom :sec}, 2 {:value 23, :uom :reps}}}],
 :divisions [{:id 1, :name "Mens Rx", :description ""}],
 :events
 [{:id 1,
   :name "Fran",
   :division-id 1,
   :priority :task,
   :measures
   [{:id 1,
     :name "Reps",
     :description "Total Reps",
     :sort :task,
     :conversion-factor 1}]}
  {:id 2,
   :name "Cindy",
   :division-id 1,
   :priority :time,
   :measures
   [{:id 1,
     :name "Reps",
     :description "Total Reps",
     :sort :task,
     :conversion-factor 1}]}]}

java.io.FileNotFoundException

I switched my project from 0.7.1 to 0.8.0 and I get:

Figwheel: focusing on build-ids (dev)
Compiling "resources/public/js/compiled/treedatabase.js" from ["src"]...
Compiling "resources/public/js/compiled/treedatabase.js" failed.
java.io.FileNotFoundException: The file resources/public/js/compiled/out/com/rpl/specter.cljs does not exist.
 at cljs.compiler$compile_file$fn__3138.invoke (compiler.cljc:1345)
cljs.compiler$compile_file.invoke (compiler.cljc:1306)
cljs.closure$compile_file.invoke (closure.clj:426)
cljs.closure$compile_from_jar.invoke (closure.clj:466)
cljs.closure$eval3586$fn__3587.invoke (closure.clj:485)
cljs.closure$eval3534$fn__3535$G__3525__3542.invoke (closure.clj:383)
cljs.closure$get_compiled_cljs.invoke (closure.clj:548)
cljs.closure$cljs_dependencies.invoke (closure.clj:619)
cljs.closure$add_dependencies.doInvoke (closure.clj:642)
clojure.lang.RestFn.applyTo (RestFn.java:139)
clojure.core$apply.invoke (core.clj:632)
cljs.closure$build.invoke (closure.clj:1679)
clojurescript_build.core$build_source_paths_STAR_.invoke (core.clj:160)
figwheel_sidecar.auto_builder$default_build_options$fn__15773.invoke (auto_builder.clj:61)
figwheel_sidecar.auto_builder$insert_figwheel_connect_script$fn__15819.invoke (auto_builder.clj:201)
figwheel_sidecar.auto_builder$warning$fn__15770.invoke (auto_builder.clj:57)
clojurescript_build.auto$time_build$fn__15573.invoke (auto.clj:56)
clojurescript_build.auto$after$fn__15579.invoke (auto.clj:65)
clojurescript_build.auto$after$fn__15579.invoke (auto.clj:65)
clojurescript_build.auto$error$fn__15582.invoke (auto.clj:72)
clojurescript_build.auto$before$fn__15576.invoke (auto.clj:61)
clojurescript_build.auto$make_conditional_builder$fn__15594.invoke (auto.clj:104)
clojure.core$mapv$fn__6727.invoke (core.clj:6616)
clojure.lang.PersistentVector.reduce (PersistentVector.java:333)
clojure.core$reduce.invoke (core.clj:6518)
clojure.core$mapv.invoke (core.clj:6616)
clojurescript_build.auto$autobuild_STAR_$fn__15680$state_machine__9517__auto____15681$fn__15686.invoke (auto.clj:187)
clojurescript_build.auto$autobuild_STAR_$fn__15680$state_machine__9517__auto____15681.invoke (auto.clj:178)
clojure.core.async.impl.ioc_macros$run_state_machine.invoke (ioc_macros.clj:940)
clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke (ioc_macros.clj:944)

Thoughts?

How to use keypath?

I have a question concerning keypath, I see how you created a param-compiled, but how do I pass it a list of params? The example shows (param-compiled :a :b :c), so how would I pass it a list (:a :b :c)? I have a "path" (i.e."/Shops/shop0001/cust0001") that I can split into (Shops shop0001 cust0001), but how do I pass it to select/transform? I have tried map and apply but everything gives me errors back.

Lein uberjar classcast exception

I'm seeing a problem with building uberjars. I'm testing with a minimal app made with lein new app. lein run works fine, but lein uberjar fails with this exception:

Caused by: java.lang.ClassCastException: com.rpl.specter.impl.TransformFunctions cannot be cast to com.rpl.specter.impl.TransformFunctions

No one else seems to be having this issue so it's probably something I'm doing wrong.

The test machine details:
CentOS 6.5 with openJDK 1.8.0_45
Leiningen 2.5.3 on Java 1.8.0_45 OpenJDK 64-Bit Server VM

project.clj:

(defproject spectertest "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.6.0"]
                 [com.rpl/specter  "0.8.0"]]
  :main ^:skip-aot spectertest.core
  :target-path "target/%s"
  :profiles {:uberjar {:aot :all}})

core.clj:

(ns spectertest.core
  (:gen-class)
  (:require  [com.rpl.specter :refer :all] ))

(defn -main
  [& args]
  (prn
    (transform [(collect-one :hb) :status :service-status]
               +
               {:hb 1 :status { :service-status 3 }})))

The full exception stacktrace is:

Compiling spectertest.core                                                                                                                                                                                  [15/1015]
java.lang.ClassCastException: com.rpl.specter.impl.TransformFunctions cannot be cast to com.rpl.specter.impl.TransformFunctions, compiling:(specter.clj:138:10)
Exception in thread "main" java.lang.ClassCastException: com.rpl.specter.impl.TransformFunctions cannot be cast to com.rpl.specter.impl.TransformFunctions, compiling:(specter.clj:138:10)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3558)
        at clojure.lang.Compiler$DefExpr.eval(Compiler.java:417)
        at clojure.lang.Compiler.compile1(Compiler.java:7226)
        at clojure.lang.Compiler.compile(Compiler.java:7292)
        at clojure.lang.RT.compile(RT.java:398)
        at clojure.lang.RT.load(RT.java:438)
        at clojure.lang.RT.load(RT.java:411)
        at clojure.core$load$fn__5066.invoke(core.clj:5641)
        at clojure.core$load.doInvoke(core.clj:5640)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invoke(core.clj:5446)
        at clojure.core$load_lib$fn__5015.invoke(core.clj:5486)
        at clojure.core$load_lib.doInvoke(core.clj:5485)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invoke(core.clj:626)
        at clojure.core$load_libs.doInvoke(core.clj:5524)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invoke(core.clj:626)
        at clojure.core$require.doInvoke(core.clj:5607)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at spectertest.core$loading__4958__auto__.invoke(core.clj:1)
        at clojure.lang.AFn.applyToHelper(AFn.java:152)
        at clojure.lang.AFn.applyTo(AFn.java:144)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3553)
        at clojure.lang.Compiler.compile1(Compiler.java:7226)
        at clojure.lang.Compiler.compile1(Compiler.java:7216)
        at clojure.lang.Compiler.compile(Compiler.java:7292)
        at clojure.lang.RT.compile(RT.java:398)
        at clojure.lang.RT.load(RT.java:438)
        at clojure.lang.RT.load(RT.java:411)
        at clojure.core$load$fn__5066.invoke(core.clj:5641)
        at clojure.core$load.doInvoke(core.clj:5640)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invoke(core.clj:5446)
        at clojure.core$compile$fn__5071.invoke(core.clj:5652)
        at clojure.core$compile.invoke(core.clj:5651)
        at user$eval9$fn__18.invoke(form-init1749168943273755311.clj:1)
        at user$eval9.invoke(form-init1749168943273755311.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:6703)
        at clojure.lang.Compiler.eval(Compiler.java:6693)
        at clojure.lang.Compiler.load(Compiler.java:7130)
        at clojure.lang.Compiler.loadFile(Compiler.java:7086)
        at clojure.main$load_script.invoke(main.clj:274)
        at clojure.main$init_opt.invoke(main.clj:279)
        at clojure.main$initialize.invoke(main.clj:307)
        at clojure.main$null_opt.invoke(main.clj:342)
        at clojure.main$main.doInvoke(main.clj:420)
        at clojure.lang.RestFn.invoke(RestFn.java:421)
        at clojure.lang.Var.invoke(Var.java:383)
        at clojure.lang.AFn.applyToHelper(AFn.java:156)
        at clojure.lang.Var.applyTo(Var.java:700)
        at clojure.main.main(main.java:37)
Caused by: java.lang.ClassCastException: com.rpl.specter.impl.TransformFunctions cannot be cast to com.rpl.specter.impl.TransformFunctions
        at com.rpl.specter.impl$extype.invoke(impl.clj:238)
        at clojure.core$partition_by$fn__6373.invoke(core.clj:6613)
        at clojure.lang.LazySeq.sval(LazySeq.java:40)
        at clojure.lang.LazySeq.seq(LazySeq.java:49)
        at clojure.lang.RT.seq(RT.java:484)
        at clojure.core$seq.invoke(core.clj:133)
        at clojure.core$map$fn__4245.invoke(core.clj:2551)
        at clojure.lang.LazySeq.sval(LazySeq.java:40)
        at clojure.lang.LazySeq.seq(LazySeq.java:49)
        at clojure.lang.RT.seq(RT.java:484)
        at clojure.lang.RT.countFrom(RT.java:537)
        at clojure.lang.RT.count(RT.java:530)
        at com.rpl.specter.impl$fn__692.invoke(impl.clj:324)
        at com.rpl.specter.impl$fn__284$G__279__289.invoke(impl.clj:21)
        at com.rpl.specter$comp_paths.doInvoke(specter.clj:29)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3553)
        ... 51 more
Compilation failed: Subprocess failed

Get transform to only receive collect values

I have a data structure like this:

    {:index 0
     :run-id run-id
     :individuals (repeatedly n (fn [] {:genome (->genome)
                                        :parents-ids []
                                        :id (utils/id)
                                        :traits {}}))}

And i have a function that generations the :traits from the :genome.

I have it working like this:

    (sp/transform [:individuals sp/ALL (sp/collect-one :genome) :traits]
      (fn [genome _] (genome->traits genome))
      generation)))

But I would like to remove that anonymous function that just discards the second argument (the old traits) and passes the genome on. Can I use tell transform somehow to not pass in the last values, just the collected ones? Or should I not be using transform here?

Find an element's parent?

I suspect the answer may resemble that in #57 . If so, I may be understanding that one a bit!