juxt/yada

A powerful Clojure web library, full HTTP, full async - see https://juxt.pro/yada/index.html

yada's Issues

Is there any documentation, usage examples or sample code?

From reading the readme yada sounds really cool and something I feel I'd be very interested in, but I'm having a hard time figuring it out from the tests and source. I see there is a bunch of code in /dev/src/yada/dev and I'm slowly piecing it together, but as its quite dense with little commenting and few docstrings its quite a bit of detective work to figure out whats what and why.

Is there any other sample/example code or other documentation available anywhere?

Any and all help appreciated!

HTTP status 418 requires BREW method

RFC 2324 states for the status code 418:

Any attempt to brew coffee with a teapot should result in the error
code "418 I'm a teapot". The resulting entity body MAY be short and
stout.

http://tools.ietf.org/html/rfc2324#section-2.3.2

To enable proper HTCPCP for yada the BREW method must be supported.

Additionally, the mandatory media type for a response to the BREW method is message/coffeepot

http://tools.ietf.org/html/rfc2324#section-4

By having a correct implementation for the HTCPCP yada could show it's full power and flexibility.

Swagger bidi match problem

Hi,

Is this a bug or am I doing something wrong?

(def route ["" {"/api" (y/swaggered
{:info {:title "test"
:version "0.1"
:description "Bug"}
:basePath "/api"}
["/v1" (yada.yada/yada "a")])}])

(bidi.bidi/match-route route "/api") -> nil

(bidi.bidi/match-route route "/api/") -> {:handler #object[yada.swagger.Swaggered$reify__41225 0x35792c1 "yada.swagger.Swaggered$reify__41225@35792c1"]}

Regards,

Timur

Need newer version of ring-swagger to support collections in query and form parameters

Yada will need at least 0.21.0-SNAPSHOT to generate the correct swagger json for collections in query and form parameters.

Can be generated with:
{:query {(schema/optional-key :some-key) [schema/Str]}}

The important part is the schema inside a vector:
[schema/Str]

Nested arity exception

Problem described in comment in yada.methods/apply-response-fn

Needs a failing test.

500: Internal Server Error

There was an internal server error.

clojure.lang.ExceptionInfo: Error on GET {:response #yada.context.Response{:headers {}, :vary #{}, :produces {:media-type #yada.media_type.MediaTypeMap{:name "text/html", :type "text", :subtype "html", :parameters {}, :quality 1.0}}}, :resource yada.resource.Resource, :error #error {
 :cause "Wrong number of args (2) passed to: template/new-page-resource/responsefn--30958"
 :via
 [{:type clojure.lang.ArityException
   :message "Wrong number of args (2) passed to: template/new-page-resource/responsefn--30958"
   :at [clojure.lang.AFn throwArity "AFn.java" 429]}]
 :trace
 [[clojure.lang.AFn throwArity "AFn.java" 429]
  [clojure.lang.AFn invoke "AFn.java" 36]
  [clojure.lang.AFn applyToHelper "AFn.java" 156]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invoke "core.clj" 632]
  [yada.methods$apply_response_fn invoke "methods.clj" 43]
  [yada.methods$eval23870$fn__23877$fn__23882 invoke "methods.clj" 186]

Swagger seems to fail with the new version

With yada:
[yada "1.1.0-20160210.093249-22"]

The following example from the tutorial results in an exception:
["/api"
(swaggered
{:info {:title "Hello World!"
:version "1.0"
:description "A greetings service"}
:basePath "/api"}
["/greetings"
[
["/hello" (yada "Hello World!\n")]
["/goodbye" (yada "Goodbye!\n")]
]
])]

Exception:

               RT.java:  947  clojure.lang.RT/nthFrom
               RT.java:  897  clojure.lang.RT/nth
             bidi.cljc:  387  bidi.bidi$route_seq/invokeStatic
             bidi.cljc:  387  bidi.bidi$route_seq/invoke
             bidi.cljc:  391  bidi.bidi$route_seq/invokeStatic
             bidi.cljc:  387  bidi.bidi$route_seq/invoke
           swagger.clj:   89  yada.swagger/swagger-spec
           swagger.clj:   87  yada.swagger/swagger-spec
           RestFn.java:  425  clojure.lang.RestFn/invoke
           swagger.clj:  172  yada.swagger/swaggered
           swagger.clj:  171  yada.swagger/swaggered
           RestFn.java:  423  clojure.lang.RestFn/invoke

basic-auth following user-manual.html not working

Just trying to use example doc (user-manual.html) without success, at last i had to use straightforward remvee/ring-basic-authentication example but i lost swagger definitions

curl .... api/v1/swagger.json

"paths":{"/masterdata/":{},"/masterdata/{uuid}":{},"/invoices/":{},"/invoices/{uuid}":{},"/external/transaction/{uuid}":{},"/external/masterdata/{uuid}":{}},

Could not locate clojure/core/cache

Yada cannot be included in a project with the following dependencies:

                 [[org.clojure/clojure "1.8.0"]
                 [org.clojure/clojurescript "1.7.170"]
                 [yada "1.1.0-20160202.093502-16"]
                 [aleph "0.4.1-beta3"]
                 [hiccup "1.0.5"]
                 [reagent "0.5.1"]
                 [re-frame "0.5.0"]
                 [bidi "1.25.0"]
                 [kibu/pushy "0.3.6"]
                 [cljsjs/react-bootstrap "0.27.3-0"]
                 [cljsjs/jquery "2.1.4-0"]
                 [cljsjs/lodash "3.10.1-0"]
                 [puppetlabs/trapperkeeper "1.2.0"]]

Exception is:

Could not locate clojure/core/cache__init.class or
clojure/core/cache.clj on classpath.

Current solution: Include [org.clojure/core.cache "0.6.4"] in dependencies.

Regards,

Timur

error handler

I would like to be able to define what happens to internal server errors. Currently they return a 500 with the full stack trace, I want to not show the stack trace for example.

cannot treat nil as http response

I have this error when trying out yada and aleph

(ns components.yada
  (:require
   [clojure.pprint :refer [pprint]]
   [bidi.ring :refer [make-handler] :as bidi]
   [yada.yada :refer [yada] :as yada]
   [aleph.http :refer [start-server]]))

(def api
  ["/" {"hello" (yada "hello world" {:id :hello})
        "hello-atom" (yada (atom "hello world"))}])

(def server
  (start-server
   (bidi/make-handler api)
   {:port 3000}))
#_(.close server)

when I visit `http://localhost:3000/

java.lang.IllegalArgumentException: cannot treat nil as HTTP response for request to ''
    at aleph.http.server$invalid_value_response.invoke(server.clj:131)
    at aleph.http.server$handle_request$fn__23992$fn__23993.invoke(server.clj:184)
    at manifold.deferred$eval5912$chain_SINGLEQUOTE____5933.invoke(deferred.clj:750)
    at manifold.deferred$eval5912$subscribe__5913$fn__5918.invoke(deferred.clj:716)
    at manifold.deferred.Listener.onSuccess(deferred.clj:219)
    at manifold.deferred.Deferred$fn__5755.invoke(deferred.clj:396)
    at manifold.deferred.Deferred.success(deferred.clj:396)
    at manifold.deferred$success_BANG_.invoke(deferred.clj:243)
    at manifold.deferred$catch_SINGLEQUOTE_$fn__5996.invoke(deferred.clj:959)
    at manifold.deferred.Listener.onSuccess(deferred.clj:219)
    at manifold.deferred.Deferred$fn__5755.invoke(deferred.clj:396)
    at manifold.deferred.Deferred.success(deferred.clj:396)
    at manifold.deferred$success_BANG_.invoke(deferred.clj:243)
    at manifold.deferred$eval5912$chain_SINGLEQUOTE____5933.invoke(deferred.clj:737)
    at manifold.deferred$eval5912$subscribe__5913$fn__5914.invoke(deferred.clj:710)
    at manifold.deferred.Listener.onSuccess(deferred.clj:219)
    at manifold.deferred.Deferred$fn__5755.invoke(deferred.clj:396)
    at manifold.deferred.Deferred.success(deferred.clj:396)
    at manifold.deferred$success_BANG_.invoke(deferred.clj:243)
    at aleph.http.server$handle_request$fn__23977$f__5358__auto____23978.invoke(server.clj:152)
    at clojure.lang.AFn.run(AFn.java:22)
    at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62)
    at manifold.executor$thread_factory$reify__5240$f__5241.invoke(executor.clj:36)
    at clojure.lang.AFn.run(AFn.java:22)
    at java.lang.Thread.run(Thread.java:745)

I'm very new to yada, so I'm sure I might miss something, but according to the manual this should work.

(if an :authorized fn returns a Deferred it's treated as true rather than a deferred value)

Testing function `response-for` doesn't accept map responses

The testing function response-for (as described in the manual) chokes on maps:

IllegalArgumentException Don't know how to convert class clojure.lang.PersistentArrayMap into class java.lang.String  byte-streams/convert (byte_streams.clj:196)

Don't pass core.async channels through manifold chains

As per Stijn Opheide's post in yada-discuss.

500 Status Error using yada.resources.misc/just-methods

Hi yada guys!
all was working fine when I was using a yada/yada with a fn to have :get by default

(ns dev-system
  "Dev Components and their dependency reationships"
  (:require
...
   [yada.yada :as yada]
   [yada.resources.misc :refer (just-methods)]
   [bidi.bidi :refer [RouteProvider]]
   [bidi.ring :refer (make-handler)]
...
  ))

(reify
    RouteProvider
    (routes [_]
     ["/api/v1"
      (yada/swaggered
       {:info {:title "api backend"
               :version "0.0"
               :description "having good times with clojure and rest"}
        :basePath "/api/v1"}
       ["/" {"masterdata/" {[:uuid] (yada/yada
                                     (fn [ctx] (str "UUID: " (get-in ctx [:parameters :uuid])))
                                     {:parameters {:get {:path {:uuid String}}}})}}])]))

But i get 500 status error when i use just-methods to specify the same definition

(reify
    RouteProvider
    (routes [_]
     ["/api/v1"
      (yada/swaggered
       {:info {:title "api backend"
               :version "0.0"
               :description "having good times with clojure and rest"}
        :basePath "/api/v1"}
       ["/" {"masterdata/" {[:uuid] (yada/yada
                                     (just-methods
                                      :get {:response (fn [ctx]
                                                        (str "UUID: " (get-in ctx [:parameters :uuid])))
                                            :parameters {:path {:uuid String}}}))}}])]))

Any ideas about, maybe I'm missing something??
Thanks in advance!
Juan

Problems with CORS

CORS seems to have a few issues right now. Going by this comment in the source[1], I understand this is still a work in progress, but here are the two issues I've encountered so far anyway in case its helpful:

1. I get this error when doing a cross-origin request from Chrome: Request header field Content-Type is not allowed by Access-Control-Allow-Headers. I was using cljs-ajax to perform the requests and it automatically sends Content-Type, I believe. Maybe the headers list used in https://github.com/juxt/yada/blob/master/src/yada/core.clj#L129 should be pulled from the context, if provided?
2. Preflight requests trigger the authorization function (which then fails for me because the requests do not contain the credentials my auth function looks for). Perhaps OPTION requests should bypass the :authorization step?

[1] CORS support: build this in, make allow-origin first-class, which headers should be allowed should be a hook (with default)

Invalid json gives 500

sending {"x":"1} to an endpoint that accepts json will result in a 500 where cheshire is unable to parse it, this should be a 400.

Demo site doesn't compile

The repository has a dependency not met on clojars, namely juxt.modular:co-dependency:jar:0.3.0.

Still if a revert it to [juxt.modular/co-dependency "0.2.2"] (or revert to a few commits backs), issuing lein run in the repository fails with:

Exception in thread "main" java.lang.IllegalArgumentException: Can't define method not in interfaces:
walker, compiling:(modular/component/co_dependency/schema.clj:7:1) 

NPE handling large multipart upload

An NPE will occur in yada.multipart/finish-up if you upload a large file to a Yada-based server.

To repro on OS X, clone the yada repo and start up lein repl in its top level directory. Then

(require '[aleph.http :refer [start-server]]
         '[bidi.ring :refer [make-handler]]
         '[yada.yada :refer [resource]])

and

(start-server
  (make-handler
    ["/upload"
     (resource
       {:methods
        {:post
         {:consumes "multipart/form-data"
          :response (constantly nil)}}})])
  {:port 3000})

Then create a large file:

dd if=/dev/zero of=file.txt count=1024 bs=1024

and use cURL to upload the file to the server:

curl -F [email protected] http://localhost:3000/upload

You should get an error:

20:02:42.021 [manifold-pool-2-7] ERROR yada.multipart - Failed: null
java.lang.NullPointerException: null
    at clojure.lang.Numbers.ops(Numbers.java:1013)
    at clojure.lang.Numbers.add(Numbers.java:128)
    at yada.multipart$finish_up.invoke(multipart.clj:313)
    at yada.multipart$process_chunk.invoke(multipart.clj:334)
    at yada.multipart$parse_multipart$this__2646__auto____24204$fn__24205$fn__24206.invoke(multipart.clj:420)
    at manifold.deferred$eval2555$chain___2576.invoke(deferred.clj:860)
    at yada.multipart$parse_multipart$this__2646__auto____24204$fn__24205.invoke(multipart.clj:414)
    at yada.multipart$parse_multipart$this__2646__auto____24204.invoke(multipart.clj:402)
    at clojure.lang.AFn.applyToHelper(AFn.java:156)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.core$apply.invoke(core.clj:632)
    at yada.multipart$parse_multipart$this__2646__auto____24204$fn__24232.invoke(multipart.clj:402)
    at manifold.deferred.Listener.onSuccess(deferred.clj:219)
    at manifold.deferred.Deferred$fn__2357$fn__2358.invoke(deferred.clj:396)
    at clojure.lang.AFn.run(AFn.java:22)
    at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62)
    at manifold.executor$thread_factory$reify__1758$f__1759.invoke(executor.clj:36)
    at clojure.lang.AFn.run(AFn.java:22)
    at java.lang.Thread.run(Thread.java:745)

Confused about compositional relationships & protocols

Very cool project!

This is a documentation issue. I've spent about a day reading all the docs, source and example code. I'm still pretty confused about the fundamentals.

The main issue is that there are a lot of protocols, and multiple layers of abstraction. Bidi, Yada, and their Modular companions. There are several things that seem to abstract Ring for example. There are multiple things that get coerced into other things. Some docs explain what can go directly into what would be helpful.

Simple example of confusion:

I see RouteProvider used a lot. I thought I could just drop something implementing RouteProvider into a bidi route tree. Doesn't seem to work.

I'm also pretty confused about all the protocols implemented by Handler, versus things like WebRequest and the various flavors of ring wrapping.

In general the execution model is unclear (when all these protocols would be invoked during a) creating the master ring handler, and b) during request processing.

Support for other Web Servers

Hi,

Are you planning to support any other web servers in the near future?

The library looks quite promising but usage with only Aleph limits its adoption. (I have nothing against Aleph but probably its not the most commonly used one)

Regards,

Timur

Parsing request body

I can find stuff that e.g. serializes return values to JSON, but there doesn't appear to be anything that puts stuff in the ctx the same way that stuff from the path or query parameters gets put in.

Similarly, headers; but I could see how that's a separate ticket.

Yada-discuss should be open

You must be a member of this group to view and participate in it.

I think users should be able to at least see what's being discussed... if only to increase the number of potential posters.

Headers map is not ring compliant - may contains Integers

The "content-length" element is an Integer. But it would be nice if the :headers element was compatible with the ring schema because other libraries depend on this. Quote from the ring wiki:

:headers A Clojure map of HTTP header names to header values. These values may either be strings, in which case one name/value header will be sent in the HTTP response, or a collection of strings, in which case a name/value header will be sent for each value.

yada throws exception when media-type is not parsable

When a client sends an invalid Accept header e.g. 'Accept: application/json, text/plain, /', yada.media-type/string->media-type will throw an exception, resulting in a 500 response.

This should be 400 Bad Request.

401/403 responses should be handled by yada, not in the user's authorization validation function

The logic for determining whether to send a 401 or 403 should be in yada.security/authorize instead of yada.authorization/validate. Otherwise each user of the library needs to implement this logic when overriding the default validation for RBAC.

See slack discussion for details: http://clojurians-log.mantike.pro/yada/2016-02-15.html

Direct memory exhaustion large multipart upload

If you do a large multipart upload with your own Partial and PartConsumer implementations (which could stream to storage, but instead simply discard the uploaded data), then the system will exhaust direct memory within a few minutes.

To reproduce, first do lein new foo to create a blank project and add the following to the :dependencies vector in project.clj:

[aleph "0.4.1-beta5"]
[bidi "2.0.1"]
[yada "1.1.2"]

then, start a REPL in that project and issue the following forms in the REPL:

(require '[aleph.http :refer [start-server]]
         '[yada.yada :refer [resource]]
         '[yada.aleph :refer [listener]]
         '[yada.request-body :as request-body])

(import (yada.multipart Partial PartConsumer))

(defrecord MyPartial []
  Partial
  (continue [this _] (prn (java.util.Date.)) this)
  (complete [_ state _] state))

(defrecord MyPartConsumer []
  PartConsumer
  (consume-part [_ _ _])
  (start-partial [_ _] (->MyPartial))
  (part-coercion-matcher [_] {}))

;; Hook in our own part consumer
(let [original-impl (get-method request-body/process-request-body "multipart/form-data")]
  (defmethod request-body/process-request-body "multipart/form-data"
    [ctx body-stream media-type & args]
    (let [new-ctx (assoc-in ctx [:options :part-consumer] (->MyPartConsumer))]
      (apply original-impl new-ctx body-stream media-type args))))

(listener
  ["/upload"
   (resource
     {:methods
      {:post
       {:consumes "multipart/form-data"
        :response (constantly nil)}}})]
  {:port 3000})

(Note that the process-request-body multimethod is modified in lieu of :options setting.)

Now, make a large file:

dd if=/dev/zero of=file.txt count=10240000 bs=1024

and upload it to the server started in the REPL:

curl -F [email protected] http://localhost:3000/upload

Timestamps will be displayed in the REPL as chunks come in:

#inst "2016-03-18T02:03:40.141-00:00"
#inst "2016-03-18T02:03:40.188-00:00"

after a few minutes things will fail. On Linux, the exception will look like the following:

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 3653632 bytes for committing reserved memory.
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000ff815000, 3653632, 0) failed; error='Cannot allocate memory' (errno=12)
# An error report file with more information is saved as:
# /home/vagrant/yada-dm/hs_err_pid3621.log
Exception in thread "Thread-3" clojure.lang.ExceptionInfo: Subprocess failed {:exit-code 1}
    at clojure.core$ex_info.invokeStatic(core.clj:4617)
    at clojure.core$ex_info.invoke(core.clj:4617)
    at leiningen.core.eval$fn__5625.invokeStatic(eval.clj:271)
    at leiningen.core.eval$fn__5625.invoke(eval.clj:267)
    at clojure.lang.MultiFn.invoke(MultiFn.java:233)
    at leiningen.core.eval$eval_in_project.invokeStatic(eval.clj:373)
    at leiningen.core.eval$eval_in_project.invoke(eval.clj:363)
    at leiningen.repl$server$fn__11767.invoke(repl.clj:243)
    at clojure.lang.AFn.applyToHelper(AFn.java:152)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.core$apply.invokeStatic(core.clj:646)
    at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1881)
    at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1881)
    at clojure.lang.RestFn.invoke(RestFn.java:425)
    at clojure.lang.AFn.applyToHelper(AFn.java:156)
    at clojure.lang.RestFn.applyTo(RestFn.java:132)
    at clojure.core$apply.invokeStatic(core.clj:650)
    at clojure.core$bound_fn_STAR_$fn__4671.doInvoke(core.clj:1911)
    at clojure.lang.RestFn.invoke(RestFn.java:397)
    at clojure.lang.AFn.run(AFn.java:22)
    at java.lang.Thread.run(Thread.java:745)

SocketException The transport's socket appears to have lost its connection to the nREPL server
    clojure.tools.nrepl.transport/bencode/fn--10128/fn--10129 (transport.clj:95)
    clojure.tools.nrepl.transport/bencode/fn--10128 (transport.clj:95)
    clojure.tools.nrepl.transport/fn-transport/fn--10100 (transport.clj:42)
    clojure.core/binding-conveyor-fn/fn--4676 (core.clj:1938)
    java.util.concurrent.FutureTask.run (FutureTask.java:266)
    java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1142)
    java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:617)
    java.lang.Thread.run (Thread.java:745)

And on OS X you will see:

Mar 17, 2016 6:46:51 PM clojure.tools.logging$eval420$fn__424 invoke
WARNING: error in HTTP server
java.lang.OutOfMemoryError: Direct buffer memory
    at java.nio.Bits.reserveMemory(Bits.java:693)
    at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
    at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
    at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:645)
    at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:229)
    at io.netty.buffer.PoolArena.allocate(PoolArena.java:213)
    at io.netty.buffer.PoolArena.allocate(PoolArena.java:133)
    at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:262)
    at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
    at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:170)
    at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:131)
    at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:73)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:111)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:510)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:467)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:381)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:353)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    at java.lang.Thread.run(Thread.java:745)

Make examples linkable

DELETE should return 404 when resource does not exist

DELETE always returns 204 regardless of the value of :exists?.

According to the spec:

A successful response SHOULD be 200 (OK) if the response includes an entity describing the status, 202 (Accepted) if the action has not yet been enacted, or 204 (No Content) if the action has been enacted but the response does not include an entity.

However, deleting a non-existent resource, I believe should not be considered 'successful'.

juxt/iota 0.2.2 not yet in clojars

[juxt/iota "0.2.2"] is not yet in clojars, so it's impossible to run the test suite.

modular template not found for yada

On yada user guide there's a command to generate a project:

lein new modular yada-demo yada

But it gives error: No such template: yada. Is this template not ready yet?

[juxt/iota "0.2.1"] not in clojars

Hi Malcolm again!
I tried to start the repl with a fresh cloned yada repo but I got this error
Caused by: org.sonatype.aether.resolution.DependencyResolutionException: Could not find artifact juxt:iota:jar:0.2.1 in central (https://repo1.maven.org/maven2/)
and after checking in clojars last version is still 0.2.0 ....
could you deploy last required version?

thanks in advance!
Juan

PS: anyway i'll downgrade iota version and i'll try again hopefully it's not a breaking change

Dependencies are not available

Hi yada guys!
I'm trying to get a try to this code but this is the result of cider-jack-inor lein repl

...
error in process sentinel: Could not start nREPL server: Retrieving prismatic/schema/0.3.5/schema-0.3.5.pom from clojars

...
Could not find artifact juxt.modular:aleph:jar:0.0.3 in central (https://repo1.maven.org/maven2/)
Could not find artifact juxt.modular:aleph:jar:0.0.3 in clojars (https://clojars.org/repo/)

cheers!
Juan

:consumes should be available everywhere :produces is

By what I can generate for swagger-ui, :consumes is only accepted in the top level option map for swaggered:

(swaggered {:info {:title "Some API"
                   :version "1.0.0"
                   :description "Bla bla bla"}
            :consumes #{....}  ;;  <-----------------
            :basePath "/api/v1"} ...

(A quick scan of the Yada source doesn't reveal anything related to :consumes, as opposed to :produces)

camel-snake-kebap dependency version upgrade

camel-snake-kebab [1] is outdated, the most recent version is [camel-snake-kebab "0.3.2"]. Are you planning to update your project dependency?

[1] https://github.com/qerub/camel-snake

OPTIONS request fails when giving a fn to yada

When you create a resource with yada and pas it a function (to only have a GET) operation

(yada/yada 
  (fn [ctx]
    (when-let [...] ...))
    {:parameters {:get {:query {:q String}}}})

when sending an OPTIONS request, it throws:

java.lang.IllegalArgumentException: No implementation of method: :interpret-options-result of protocol: #'yada.methods/OptionsResult found for class: nil

support schema Any

When using prismatic/schema Any yada.coerce throws an error :

For instance :
:parameters {:query { (sch/optional-key sch/Any) sch/Any}} will throw :

java.lang.IllegalArgumentException: No implementation of method: :to-key of protocol: #'yada.coerce/ParameterKey found for class: schema.core.AnythingSchema

Bidi's ->TaggedMatch doesn't work and breaks swagger

A path with ->TaggedMatch will not behave like vanilla bidi if 'swaggerized'.

path-for will stop working AND the resource will disappear from the swagger UI.

yada/swaggered seems to think this regex is a part of the resource path

I have a resource where an email address is part of the path like "/foo/by-email/[email protected]"

A bidi route accepting this with :email as a route param would look like:

[["/foo/" [#".+\@.+\..+" :email]] :foo]

Note the regex format for the :email param which is required by bidi to parse the email as a route param. This is explained in juxt/bidi#98 (comment)

When I created a yada/swaggered api on top of such a routing I got an error saying Pattern does not implement the encode method of yada.swagger/SwaggerPath which makes me think yada/swaggered thinks the regex in the above example is a separate path element from :email. For now I fixed it in my project with:

(extend-protocol yada.swagger/SwaggerPath
  java.util.regex.Pattern (encode [_] nil))

But I doubt this is the right thing to do in the long term.

error loading yada: assoc-query-params is not public

with: [yada "1.0.0-20150903.093751-9"]

(require '[yada.yada :refer (yada)])
CompilerException java.lang.IllegalAccessError: assoc-query-params is not public, compiling:(yada/core.clj:1:1)

websocket supported

Do we have websocket supported within the lib ? Or is there any other way we can get websocket working with the lib ( using aleph websocket maybe ) ?

Thanks for creating a beautiful lib.

Produces doesn't result in charset in content type

Hello, I can't say I'm certain about this, but I'm wondering why, when I specify a charset in produces for a resource, the response doesn't include the charset in the content type of the response? My example is this (probably not the best place for it, but it shows the problem):

I changed the hello-world example to have a new resource:

...
(defn say-hello-json [ctx]
  {:say "hello" :p (get-in ctx [:parameters :query :p])})
....
(defn hello-json []
  (yada
   (resource
    {:methods
     {:get
      {:parameters {:query {:p String}}
       :produces [{:media-type "application/json"
                   :charset "utf-8"}]
       :response say-hello-json}}}))) 
...
(routes
   ...
   ["hello-json" (tag (hello-json) ::hello-json)]
   ...
)

and I created a new test in helloworld_test.clj like this:

(deftest json-test
  (let [resource (hello/hello-json)
        response @(resource (request :get "/?p=bob"))
        headers (:headers response)]
    (is (= 200 (:status response)))
    (is (= (to-string (:body response)) "{\"say\":\"hello\",\"p\":\"bob\"}"))
    (is (= (get-in response [:headers "content-type"]) "application/json;charset=utf-8"))))

My thinking is that, as I specified the resource to have :produces with :charset "utf-8, the response should have a content-type of application/json;charset-utf-8. However the response only has the content-type application/json.

And my question is: is this a bug? Or not?

Also, another point, I wonder why the actual body has a trailing \n?

If you look at my branch https://github.com/mdaley/yada you'll be able to see my check-in.

If this is a problem, if you give me some pointers, I'd be more than willing to be involved in fixing it.

Disallowed POST validation example doesn't specify the validation

It just says {:body "Hello World!"}:

Swagger tags

In an older version of Yada, it was possible to give tags using the 'swagger' metadata.

In 1.1.0, it doesn't seem to have an effect.

If I try to it the :tags into the resource definition, it results in an error "Cannot turn resource-model into resource, because it doesn't conform to a resource-model schema..."

Is there another way to provide tags for swagger?

Typo in project.clj

On this line: https://github.com/juxt/yada/blob/master/project.clj#L23
It looks like :exclusions is miss-typed.

Cheers

Availability of parsed parameters in yada.protocols/Properties

We already discussed this on slack, but maybe I should add it here too.

When calculating the properties on request (properties [_ ctx]) arity, it would be nice to have parameters parsed already. This would mean that the get-properties interceptor would be executed after the malformed? interceptor.

As far as I can see,

method-allowed?
process-request-body
malformed?

do not need anything that is processed in get-properties, at least not the properties on request version. But I might be wrong on that one.

On the other hand if we have the output of malformed? already in the ctx, get-properties would be much easier to implement, especially if you need to query e.g. a database / filesystem to answer :exists?, :representations, :last-modified, ...

Document dependency on Clojure 1.7?

Might be worth mentioning somewhere that this requires Clojure 1.7 (at least until 1.7 is out of beta) as it throws an exception on require otherwise. Took a little while to realise and might save others some time.

yada.bidi also crashes with a rather obscure exception if a version of bidi < 1.19.0 is being used (I was accidentally pulling in 1.18.9 through juxt.modular/bidi). Again, might be worth mentioning minimum version requirements.

text/plain Content-Type should specify charset when rendering a string

It seems to be UTF-8, but I don't know if something is specifying this explicitly somewhere or if that's just an accidental default somewhere.

Example of this going awry using httpie and a UTF-8 term:

➜  ~ http GET http://localhost:8090/user-guide/examples/ResourceStateWithBody/17382343
HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 24
Content-Type: text/plain
Date: Thu, 14 May 2015 03:37:25 GMT
Server: Aleph/0.4.0

Your balance is ฿1300

Content Security problems with the yada website

Both Chromium and Firefox refuses to load a bunch of resources because of this.

Example error message from the console:

Refused to load the stylesheet 'http://yada.juxt.pro/bootstrap/css/bootstrap.min.css' because it violates the following Content Security Policy directive: "default-src https: data: 'unsafe-inline' 'unsafe-eval'". Note that 'style-src' was not explicitly set, so 'default-src' is used as a fallback.

Return body from PUT or POST

Currently it's impossible to return a body that contains e.g. a map of data from a PUT or POST method.

SEVERE: error sending body of type  clojure.lang.PersistentArrayMap
java.lang.IllegalArgumentException: Don't know how to convert class clojure.lang.PersistentArrayMap into (stream-of io.netty.buffer.ByteBuf)
    at byte_streams$convert.invoke(byte_streams.clj:187)
    at aleph.netty$to_byte_buf_stream.invoke(netty.clj:170)
    at aleph.http.core$send_streaming_body.invoke(core.clj:277)
    at aleph.http.core$eval24414$send_message__24421$fn__24422.invoke(core.clj:353)
    at aleph.http.core$eval24414$send_message__24421.invoke(core.clj:352)
    at aleph.http.server$eval24451$send_response__24456$f__24005__auto____24460.invoke(server.clj:124)
    at aleph.http.server$eval24451$send_response__24456$fn__24463.invoke(server.clj:117)
    at clojure.lang.AFn.run(AFn.java:22)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:339)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:356)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    at java.lang.Thread.run(Thread.java:745)

The reason is that there is no body conversion in PUT/POST as it is done in GetMethod/request.

Is this something yada intends to support? The HTTP spec allows for either referencing the newly created/updated entity or actually specifying it.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
Section 10.2.1 200 OK
POST an entity describing or containing the result of the action;

There's no mention about PUT 200 or 201 other than The newly created resource can be referenced by the URI(s) returned in the entity of the response, with the most specific URI for the resource given by a Location header field. The response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate.

As this impacts all methods that can return a body, I'm not sure how you'd like to pull that out of the GetMethod.

Dependency on unreleased iota 0.2.3

yada's project.clj cites iota 0.2.3 (latest generally available 0.2.2) and co-dependency 0.3.0 (latest 0.2.2).

(Thanks for a fab-looking library, by the way.)