daviddenton/fintrospect

Implement fast, type-safe HTTP webservices for Finagle

I'm a London-based Engineering Lead; passionate about building simple, Test-Driven products and the teams to deliver them. Oh, and also about building the simple, Test-Driven, Open Source tools to power them. 🙃

My latest coding passion is Kotlin, in which I co-created http4k, the Functional toolkit for building Kotlin HTTP applications. http4k, was featured in the ThoughtWorks TechRadar, in particular being included for:

"Apart from its elegance and simplicity, we also like its emphasis on testability — given that the entities in the libraries are immutable and the routes in the app, as well as the app itself, are just functions, they're super easy to test." - TW TechRadar

I was pretty happy with that. 😊

I'm also active as speaker at various international technology conferences and have also enjoyed developing and delivered Coding Dojos and full Training Courses, both in the UK and around the world. At the end of 2020, I was awarded a place in the Google Expert Developer program.

Below is the history of my introduction to various languages; I've been on a typical polyglot journey and worked in many industries including Investment, Internet and Mobile Banking, Academic Publishing, Internet Provision and Government:

Some more detailed stats on my GitHub career in the last 11 years:

Language Breakdown:

• 24584 pushed commits.
• 2775 stars on my projects.
• 618 submitted pull requests.
• 139 issues opened.
• 85 personal repos.
• 20 public repos contributed to.

fintrospect's Issues

Generated ETags are not RFC 7232 compliant

According to the RFC 7232, ETags should be wrapped in double quotes.
ETags generated by fintrospect cause the following warnings to be raised by our Akka http client:
Illegal header: Illegal 'etag' header: Invalid input 'd', expected 'W' or opaque-tag (line 1, column 1): d41d8cd98f00b204e9800998ecf8427e

Form body decoding throws java.lang.ArrayIndexOutOfBoundsException when handling completely empty form

On invalid request, AutoFilters blow up instead of producing BadRequest response

Add support for Composite Swagger modules

Suggested by @ncreep:

Something along these lines http://azimi.me/2015/07/16/split-swagger-into-smaller-files.html it should be simple to add, since we just need to get the description base-paths from each of the other modules and use them to generate the composite JSON doc.

I'm envisaging that you've got a few modules sitting at various contexts, then a fallback root module with the composite renderer. Is that what you had in mind? So along the lines of:

/context1/swagger.json
/context2/swagger.json
/swagger.json <-- composite with $refs
The trickiest part is getting the API to look nice. Easiest option would probably be to have a CompositeModule function which takes a bunch of other modules or creates them in a builder like way (see below):

CompositeModule(Root, CompositeRenderer)
.withModules(m1, m2, m3)
However, since the Module is not typed by the Renderer implementation, it would currently not be possible to have the compiler guarantee that all of the submodules were in fact Swagger modules, so an alternative is to have the "super" module create the submodules and pass them to a function for configuration:

val m: Module = SwaggerCompositeModule(Root)
.withModule(_ / "context", {
module =>
module.withRoute(r1).withRoute(r2)
})

Routes are added in the incorrect (reverse) order

When defining routes in a FintrospectModule, the overall order in which the paths are matched is inverted (ie. the last route added will be checked first).

This makes it confusing to know in which order to add the routes in the case where the paths might overlap.

Extractables incorrectly shortcircuiting in case of all Optional parameters

Allow empty model objects in response example definitions

Hi,

I'm trying to render an example value with the Swagger renderer. My example value contains an empty list and thus JsonToJsonSchema throws:

new IllegalSchemaException("Cannot use an empty list to generate a schema!")

In my case, it would still be useful to actually render the example value, despite not knowing the exact type of the array (I don't care if it's considered to be Object or something).

Could it be possible to not throw the exception and have some fallback rendering of the example value without a schema?

Thanks

Form fields not encoded correctly in body

Client implementation does not bind parameters properly.

Multipart form deserialisation allows for empty filenames

Because of the way in which browsers send multipart form file fields, a multipart entry is sent to the server with no filename, even if no file is chosen in the browser. This means that required multipart file fields are passed through even though they are missing.

Fix is to filter out files with no filename in the multipart form during the extraction process.

Json4S object builder obj() method is broken

... returns an array instead of an object.

Configuring the Swagger output for Swagger-UI

Hi,

I'm trying to use the Swagger output from a route-module that I've defined in Swagger-UI and I've stumble upon a number of issues.

With Swagger-UI 3.0.x I'm getting:

Errors
Parser error 
duplicated mapping key

Not sure what's at fault here, but after downgrading to 2.2.x, I did manage to parse the JSON and render it in the UI.

Now, to be able to hit the "Try it out!" button in the UI I need to specify the host and schemes keys in the generated Swagger JSON.
I've quickly hacked-up a modified Swagger2dot0Json class to allow these modifications, see this gist, and it seems to work.

For a proper change I'm guessing that passing the relevant info in the some dedicated class would be more appropriate.

If you find any of this useful, I can open a pull request.

Thanks.

Netty warning on startup with Scala 2.12

There is a bug in the version of finagle-http that fintrospect uses with scala 2.12.X:

twitter/finagle#581

Unknown if this causes any further effects, but recommend NOT using 2.12.X version of Scala until above is fixed.

Form extraction fails on empty fields

...does not deserialise properly. Blows up with IllegalStateException exception

Auto should not use implicit conversion functions

Hi,

Currently io.fintrospect.formats.Auto uses implicit functions in various methods. E.g.:

def Out[OUT](svc: Service[Request, OUT], successStatus: Status = Status.Ok)
            (implicit transform: OUT => R): Service[Request, Response]

This can be problematic, for example in cases where one is trying to serialize raw JSON (e.g., Play's JsValue). Since in this case OUT =:= R , and Fintrospect's built-in implicit conversion competes with the standard library's <:< (which extends Function1).

The workaround for this particular problem is to pass the implicit argument explicitly. But a more general solution would be to introduce a dedicated type for the conversions that take place in Auto. This will avoid polluting the implicit scope with a common type and thus won't compete with the standard library (or anything else for that matter).

StaticModule doesn't resolve / to index.html

This should be the default behaviour - since we don't serve file listings.

API does not allow binding of custom body to an outbound request

Circe displays ints as decimals

Checklist for Scala 2.12.X cross-build

Dependencies that need to have 2.12.X version:

Core:

• finagle-http
• scala-xml
• scalatest

Optional modules:

• argonaut
• circe
• json4s
• handlebars-scala-fork
• msgpack4s
• play-json
• spray-json

Examples:

• finagle-oauth2

Missing parameter type on Path.apply leads to requiring reflective imports flag

Caching filters apply cache headers even when the request code is an error

They should only apply when response status < 400

Static resources in subdirs not found when StaticModule is added at the Root context

Returns a 404 instead.

Support for custom Swagger tags

Hi,

Currently, when rendering with the Swagger2dot0Json each path is tagged with the basePath of the module being rendered. As a result all routes in a given module have the same tag and are grouped in the UI accordingly.
Since I don't see how it would be possible to render a number of modules in the same Swagger file. Would it be possible to add support for adding custom tags to RouteSpec so that the Swagger renderer will be able to group by them?

Thanks

Path parameter values not being encoded at all in clients

Path parts need to be encoded correctly, with the exception of reserved characters.

See here for details of the reality of this issue: http://blog.lunatech.com/2009/02/03/what-every-web-developer-must-know-about-url-encoding

Body models not being outputted in Swagger 2 JSON schema.

They are ignored in the definitions section of the JSON, but appear correctly in the parameter section of the route description.

Extractable incorrectly short-circuiting if the last clause in the for comprehension resolves to NotProvided

Example test case:

      val ex = Extractable.mk {
        request: Request => for {
          req <- Query.required.int("req") <--? request
          opt <- Query.optional.int("optional") <--? request
        } yield Some((req, opt))
      }

      ex <--? Request("/?req=123") shouldBe Extracted((Some(123), None))

      Fails with:  NotProvided was not equal to Extracted((Some(123),None)) 

If a body or parameter is not declared in the RouteSpec, extracting it for the first time fails

v14.12.0

fintrospect-handlebars module brings in sl4j-simple as transitive depedency

This is caused by the handlebars implementation bringing it in. Need to exclude the sl4j-simple dependency from our module.

Empty Forms can be successfully retrieved from requests to be valid regardless of having missing required fields

v12.14.0

This only matters if they are not pre-validated by using a body() call on the RouteSpec().