twisted/klein

werkzeug + twisted.web

Twisted

_ _ _ _

For information on changes in this release, see the NEWS file.

What is this?

Twisted is an event-based framework for internet applications, supporting Python 3.6+. It includes modules for many different purposes, including the following:

• twisted.web: HTTP clients and servers, HTML templating, and a WSGI server
• twisted.conch: SSHv2 and Telnet clients and servers and terminal emulators
• twisted.words: Clients and servers for IRC, XMPP, and other IM protocols
• twisted.mail: IMAPv4, POP3, SMTP clients and servers
• twisted.positioning: Tools for communicating with NMEA-compatible GPS receivers
• twisted.names: DNS client and tools for making your own DNS servers
• twisted.trial: A unit testing framework that integrates well with Twisted-based code.

Twisted supports all major system event loops -- select (all platforms), poll (most POSIX platforms), epoll (Linux), kqueue (FreeBSD, macOS), IOCP (Windows), and various GUI event loops (GTK+2/3, Qt, wxWidgets). Third-party reactors can plug into Twisted, and provide support for additional event loops.

Installing

To install the latest version of Twisted using pip:

$ pip install twisted

Additional instructions for installing this software are in the installation instructions.

Documentation and Support

Twisted's documentation is available from the Twisted Matrix website. This documentation contains how-tos, code examples, and an API reference.

Help is also available on the Twisted mailing list.

There is also an IRC channel, #twisted, on the Libera.Chat network. A web client is available at web.libera.chat.

Unit Tests

Twisted has a comprehensive test suite, which can be run by tox:

$ tox -l                       # to view all test environments
$ tox -e nocov                 # to run all the tests without coverage
$ tox -e withcov               # to run all the tests with coverage
$ tox -e alldeps-withcov-posix # install all dependencies, run tests with coverage on POSIX platform

You can test running the test suite under the different reactors with the TWISTED_REACTOR environment variable:

$ env TWISTED_REACTOR=epoll tox -e alldeps-withcov-posix

Some of these tests may fail if you:

• don't have the dependencies required for a particular subsystem installed,
• have a firewall blocking some ports (or things like Multicast, which Linux NAT has shown itself to do), or
• run them as root.

Static Code Checkers

You can ensure that code complies to Twisted coding standards:

$ tox -e lint   # run pre-commit to check coding stanards
$ tox -e mypy   # run MyPy static type checker to check for type errors

Or, for speed, use pre-commit directly:

$ pipx run pre-commit run

Copyright

All of the code in this distribution is Copyright (c) 2001-2024 Twisted Matrix Laboratories.

Twisted is made available under the MIT license. The included LICENSE file describes this in detail.

Warranty

THIS SOFTWARE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE USE OF THIS SOFTWARE IS WITH YOU.

IN NO EVENT WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE LIBRARY, BE LIABLE TO YOU FOR ANY DAMAGES, EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Again, see the included LICENSE file for specific legal details.

klein's Issues

Python 2.6. should probably be removed from supported versions

According to PyPi Twisted no longer supports Python 2.6 https://pypi.python.org/pypi/Twisted. This means that there's probably no point supporting Python 2.6 in Klein.

Users should be able to define functions for handling errors.

This should encompass both errors caused by routing (404s and 405s, etc)… but also errors that occur during the dispatch to a user defined handler function.

The obvious implementation of this is to move routing into the same deferred chain as endpoint execution and add an errback which dispatches to the user defined handler function.

The user defined handler should be invoked with request and failure as arguments. And it should useful as a decorator.

class MyAwesomeThing(object):
    app = Klein()

    @app.route('/{userid}')
    def users_stuff(self, request, userid):
        pass

    @app.handle_errors
    def my_error_handler(self, request, failure):
        pass

It's unclear to me what should happen in the case of resource traversal, and if the handler for HTTPErrors from routing should be remain in tact. I suspect they should, so that an error handler which does not trap them will still get the default rendering of werkzeug errors.

Perhaps Klein.handle_errors should also support *args of exception types but I'm not sold on the necessity or desirability of that, and either way catch all exceptions should be support by the above syntax proposal (without necessarily invoking the decorator with Exception)

allow for "overlay" routing

Sometimes I have a structure like this:

@route("/a/", branch=True)
def some_tree(request):
    return static.File("...")

@route("/a/foo/bar/special_thing.txt")
def some_overlaid_resource(request):
    return EventSource(...)

I'd really like that second route to show up if that specific route is requested, but the first to take over for everything else within that hierarchy.

Probably this is a bug in Werkzeug, but I am experiencing it with Klein so I'm starting here.

There are problems with registerAdapter(KleinRequest, Request, IKleinRequest)

app.py registers KleinRequest as an adapter from twisted.web.server.Request to IKleinRequest.

It looks like the purpose of this is to create an extra object where some extra Klein-specific state can can live. The state is per-request so Klein wants to associate it with a twisted.web.server.Request instance and be able to retrieve it from that instance later.

It so happens that with this adapter registered IKleinRequest(someRequest) is IKleinRequest(someRequest) - providing the desired persistent state storage and access. However, this is only because twisted.web.server.Request subclasses Componentized.

It's somewhat questionable whether "is a Componentized subclass" is really meant to be part of the public interface of the Twisted Web request object (though public or not it clearly is currently part of that interface). In practice, the fact that zero Twisted Web unit tests fail if twisted.web.server.Request is made to not be a Componentized subclass makes relying on this property at least risky.

Apart from that, this dependency seems to make unit testing Klein-using applications more difficult.

To exercise the request dispatch logic it becomes necessary to manufacture a suitably test-friendly IRequest implementation which also has the same persistent adapter behavior of Componentized (probably by subclassing Componentized) and then register KleinRequest as an adapter for that class as well.

I suspect this is a deeper reliance on some internals of Klein than the Klein developers would have aimed for in their testing story.

Notice that Klein itself probably suffers from some testing issues related to this. DummyRequest in test_app.py is not a Componentized so the behavior presented when adapting it to IKleinRequest does not accurately reflect the behavior that real Klein code will get in non-test usage.

I think a good change to make to Klein would be to explicitly wrap a KleinRequest around the underlying IRequest provider and then pass it around wherever it is needed.

Perhaps this means KleinRequest should also be a proxy for IRequest to the wrapped object (yay proxyForInterface) so that only one request object actually needs to be passed everywhere.

A way to structure klein apps into (reusable) modules

nb. the pattern I'm showing here is basically flask's blueprints

It seems that currently Klein doesn't have any builtin way to help factoring out view functions to separate modules.

For example, if you had the following app:

app = Klein()

@app.route('/users')
def list_all_users(request):
    return 'all users'

then you could divide it into two modules like this:

# main.py
app = Klein()

# users.py - version 1
from main import app

@app.route('/users')
# ...

This is bad because it leads to several problems, the most immediate being circular import issues, but also this is only a separate module technically, not logically: you can't really re-use that module with another app, or test it in isolation.

An easy way to help that would be to use dependency injection:

# main.py
from users import add_users_routes

app = Klein()
add_users_routes(app)

# users.py - version 2

def list_all_users(request): 
    return 'all_users'

def add_users_routes(app):
    app.route('/users')(list_all_users)

Now users is a separate logical module, but it's a bit awkward, with the central add_users_routes function. We could use Klein's .resource() to help that:

# users.py - version 3
users = Klein()

@users.route('/')
def list_all_users(request):
    return 'all users'

def add_users_routes(app):
    @app.route('/users', branch=True)
    def branch_to_users(request):
        return users.resource()

This is already pretty nice, could maybe use some helpers so that you wouldn't need to implement that branch function, but is reusable and possible to test in isolation. The problem however is, routing to a Klein resource returns control to twisted.web, which then calls .render() on the nested resource, which causes a werkzeug map bind, etc - it's a performance hit. A simple hello-world-grade benchmark shown that a root Klein can serve ~2200 requests per second, routing one branch deep: ~1700, two branches deep: ~1400 (experience with flask's blueprints shows that 2 levels of nesting are enough in practice)

I'm aware Klein is a web framework, and web applications aren't typically constrained by strict real-time requirements, and they're often easy to scale, but I think Klein needs some guideline on structuring applications, and while we're at it, might as well make it fast :)

Basically I think we want the syntax of users.py version 2, with the performance of version 3. Here's how flask does it (translated to Klein):

# main.py
from users import users_blueprint
app = Klein()
users_blueprint.register(app, url_prefix='/users')

# users.py
users_blueprint = Blueprint()

@users_blueprint.route('/')
# ...

The Blueprint is simply a container that records all @route() calls it was used for, and does them on the app when .register is called. This makes it only a setup-time thing, with no runtime (performance) effects.

I've put together a proof-of-concept implementation for reference, see https://github.com/tehasdf/klein/tree/blueprints (here's an example.rst in the repo)

☂ general awesomeness for klein.Plating

Here are some things that should be added to klein.Plating to make it really complete, which were left out of the initial implementation (#118) to keep it compact:

• Support for different types of templates
  • the ones from twisted; XMLFile #121
  • and XMLString as implied by #89's basic prototype #122
  • HTML5Lib #123
• Deferred support #131
• including some kind of solution for asynchronous JSON serialization #124
• More documentation, particularly for how to use widgets #125
• Something on the input side, for conveniently parsing JSON post bodies and handling form data submission - something that doesn't involve handling state in memory, by the way, since that doesn't scale. (Is there even a way in werkzeug's routing syntax to extract a query arg as a python parameter, or doe sit only accept path segments?) #126
• attr.s support - specifically, automatically invoking asdict so applications can work entirely in terms of nice data structures and not manually decompose things into dicts and lists #127
• pluggable renderer support. the example even has a great use-case: latitude and longitude are conventionally presented as "east" and "west", "north" and "south", not positive and negative numbers. We may want to present an individual serializable value in a different way. I think that the way to do this might be @plating.renderer("blub") def blubrender(plated_data, request, tag): "..." to stuff things into a specific renderer's element. Still not sure how to do composition / renderer libraries nicely, but at the very least you could write a function and then do add_utility_renderers(my_plating) #128 and #130 - closely related, possibly resolvable with the same implementation
• datetime serialization (what even is the best practice for this in JSON) #129
• real content-type negotiation #132
• support for methods #133 (oops)
• allow for nesting within :list renderer #161

documentation for widgets in klein.Plating

Template from file fails

OS: Mac OS X 10.6.8
Python: 2.7.3
Twisted: 12.0.0
Klein: latest master, from git

When I try to load a template from a file, e.g.:

loader = XMLFile(FilePath('templates/index.xml'))

I get a traceback. However, when I instead load that file into an XMLString, like this:

loader = XMLString(FilePath('templates/index.xml').getContent())

it succeeds.

This is the first time that I've used non-Nevow templates, so I'm not sure if this is a known issue, if it's Klein-only, or if it also shows up in Twisted.

Here's the full traceback I get:

2012-06-19 00:53:53-0400 [HTTPChannel,0,127.0.0.1] Unhandled Error
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 551, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/klein/resource.py", line 73, in process
    return flattenString(request, r).addCallback(process)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/web/_flatten.py", line 318, in flattenString
    d = flatten(request, root, io.write)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/web/_flatten.py", line 301, in flatten
    _writeFlattenedData(state, write, result)
--- <exception caught here> ---
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/web/_flatten.py", line 260, in _writeFlattenedData
    element = state.next()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/web/_flatten.py", line 227, in _flattenTree
    raise FlattenerError(e, roots, extract_tb(exc_info()[2]))
twisted.web.error.FlattenerError: Exception while flattening:
  <tharsk.routes.DumbTemplate object at 0x1019cc650>
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/web/_flatten.py", line 218, in _flattenTree
    element = stack[-1].next()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/web/_flatten.py", line 189, in _flattenElement
    result = root.render(request)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/web/_element.py", line 184, in render
    return loader.load()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/web/template.py", line 397, in load
    self._loadedTemplate = self._loadDoc()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/web/template.py", line 388, in <lambda>
    self._loadDoc = lambda: _flatsaxParse(fobj)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/web/template.py", line 335, in _flatsaxParse
    parser.parse(fl)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/sax/expatreader.py", line 102, in parse
    source = saxutils.prepare_input_source(source)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/sax/saxutils.py", line 295, in prepare_input_source
    if source.getByteStream() is None:
AttributeError: FilePath instance has no attribute 'getByteStream'

Conflict with klein.resource

Mac OS X, mountain lion, Python 2.7

Looking at __init__.py, looks like klein.resource was intended to be klein.app._globalKleinApp.resource, but there is also a resource.py.

>>> from klein import resource, route
>>> print resource
<module 'klein.resource' from '/Users/cyli/.virtualenvs/klein/lib/python2.7/site-packages/klein-0.0.1-py2.7.egg/klein/resource.pyc'>
>>> print route
<bound method Klein.route of <klein.app.Klein object at 0x10cf2f090>>

If that was not the intention, perhaps klein.app._globalKleinApp.resource should be imported (and exported) as get_resource?

Unable to serve index.html at the route "/"

The following doesn't work, returning a file not found error.

@app.route("/")
def root(request):
   return File("index.html")

The following works

@app.route("/index.html")
def root(request):
   return File("index.html")

And this also works

@app.route("/")
def root(request):
   return "Hello"

This is a failing test that I think illustrates the problem

    def test_staticRoot_noBranch(self):
        """
        The following should return the contents of the file index.html

        @app.route("/")
        def root(request):
            return File("index.html")

        """
        app = self.app
        request = requestMock("/")

        @app.route("/")
        def root(request):
            return File(__file__)

        d = _render(self.kr, request)

        def _cb(result):
            self.assertEqual(request.getWrittenData(), 
                open(__file__).read())
            self.assertEqual(request.finishCount, 1)

        d.addCallback(_cb)
        return d

Document best practices for integration/functional tests

How do I test my klein apps, without using urllib or similar?

Currently I'm considering adapting these Twisted web tests or these ones.

Traceback when browsing static dir

OS: Mac OS X 10.6.8
Python: 2.7.3
Twisted: 12.0.0
Klein: latest master, from git

I have a static dir declared in the following manner:

@route("/assets/")
def assets(request):
    return File("./assets")

When I go to /assets/css or /assets/img, etc., I get the expected twisted.web directory listing. However, visiting /assets/ directly renders the following traceback:

2012-06-19 00:26:03-0400 [HTTPChannel,0,127.0.0.1] Unhandled Error
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/web/server.py", line 167, in render
    body = resrc.render(self)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/klein/resource.py", line 83, in render
    d.addCallback(process)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 301, in addCallback
    callbackKeywords=kw)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 290, in addCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 551, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/klein/resource.py", line 67, in process
    request.postpath != request._klein_postpath_):
exceptions.AttributeError: Request instance has no attribute '_klein_postpath_'

errors aren't being handled on branched resources

Branched routes don't handle errors of the "base" Resource. Hopefully an example to start things off will be beneficial.

from klein import Klein
from werkzeug.exceptions import NotFound

app1 = Klein()
app2 = Klein()

#----- Catch Errors -----#
@app1.route('/<path:allpaths>')
def catchall(request, allpaths):
    request.setResponseCode(404)
    return 'Catch-All: paths'

@app1.handle_errors(ValueError)
def handle(request, failure):
    request.setResponseCode(400)
    return 'Catch: ValueError()'

@app1.handle_errors(NotFound)
def huh(request, failure):
    request.setResponseCode(413)
    return 'Catch-All: NotFound()'

#----- Routes -----#
@app1.route('/one')
def one(request):
    raise ValueError('raised a ValueError')

@app1.route('/two', branch=True)
def two(request):
    return app2.resource()

@app2.route('/blue')
def blue(request):
    raise ValueError('blue')        # this won't be handled by app1

if __name__ == '__main__':
    app1.run('localhost', 9000)

If a route isn't present, then either the catchall() or huh() function will execute. Similarly, if a ValueError is raised, then the handle() function is executed.

> curl localhost:9000/
# Catch-All: NotFound()

> curl localhost:9000/foo/baz/bar
# Catch-All: paths

> curl localhost:9000/one
# Catch: ValueError()

However, if an invalid route from a branched resource is used, then the errors are left unhandled. The following will result in a traceback and the default Twisted 404 page:

> curl localhost:9000/two/blue
> curl localhost:9000/two/red

This all makes it very tedious and very error prone when creating expansive web or RESTful applications because devs must then ensure each each Resource has all the appropriate error handling. Though there are ways around this, there should be a dedicated "Klein" way to map branched routes to the base set of routes.

Exceptions raised in @app.route-decorated functions are passed to log.err twice

The "processing_failed" function calls log.err on any exception:

https://github.com/twisted/klein/blob/master/klein/resource.py#L125

...but it also calls t.web Request.processingFailed which also calls log.err:

https://github.com/twisted/twisted/blob/trunk/twisted/web/server.py#L314

But the failure is also passed to log.err when the Deferred is garbage-collected.

This means errors/tracebacks are logged twice - not useful.

New version of Werkzeug throws exception on HTTP errors

On a 404 or other HTTP error werkzeug displays "Request did not return bytes" and returns a 500 error instead of the appropriate error in version 0.9.3 of Werkzeug. Reverting to v0.8.3 fixes the problem.

Looks like klein returns a unicode string instead of expected byte type.

Klein's documentation examples should all be test-driven

The documentation examples should be test-driven, with an example test, and then code to pass the test.

Another Example Route: catch all

It would be helpful to have an example of a catch all route, one that uses the Werkzeug's PathConverter. The information can be found if dug deep enough, but having examples makes everyone's life easier.

asynchronous JSON serialization for klein.Plating

You can put a Deferred into a template so you kinda need to be able to put one into the JSON.

here's the implementation, basically: https://gist.github.com/glyph/f1e7dc3143e35ff78d29395021d24f72

Test package is not installed

The setup.py file only specifies klein as a package. klein.test is not installed. I'm not sure if this is really a bug, but rackerlabs/otter relies on klein.test_resource existing. It is now called klein.test.test_resource, but we can't import that, because that package isn't included.

If this is a bug (and I'm tentatively tagging it as such), I'd be happy to file the PR to fix it.

Upstream issue: rackerlabs/otter#889.

TypeError: Class advice impossible in Python3. Use the @implementer class decorator instead.

Hi,
I see to run into an error trying to create a Klein process. When using python 3.5.1, twisted 16.2.0 and Klein 0.2.3 (or 0.2.1) I receive a TypeError: Class advice impossible in Python3. Use the @implementer class decorator instead. on a rather basic bit of code:

from klein import Klein
from twisted import inlineCallbacks, returnValue

app = Klein()

@app.route("/")
@inlineCallbacks
def hello(request):
    returnValue("hello")

app.run("0.0.0.0", 8080)

I also import a separate module using umongo with txmongo, but I don't beleive those should collide.

This is the stack trace I receive:

raceback (most recent call last):
  File "/opt/src/auth.py", line 3, in <module>
    from klein import Klein
  File "/usr/local/lib/python3.5/site-packages/klein/__init__.py", line 1, in <module>
    from klein.app import Klein, run, route, resource
  File "/usr/local/lib/python3.5/site-packages/klein/app.py", line 32, in <module>
    class KleinRequest(object):
  File "/usr/local/lib/python3.5/site-packages/klein/app.py", line 33, in KleinRequest
    implements(IKleinRequest)
  File "/usr/local/lib/python3.5/site-packages/zope/interface/declarations.py", line 412, in implements
    raise TypeError(_ADVICE_ERROR % 'implementer')
TypeError: Class advice impossible in Python3.  Use the @implementer class decorator instead.

Port to Python 3

Time marches relentlessly onwards.

Possible to add a WSGI instance?

How about

   k = Klein()
   app = k.wsgi()

So that it can be used with gunicorn?

Return HTTP status code 204 for empty response

Would it be sensible to change the status code from 200 OK to 204 No Content without having to explicitly call request.setResponseCode(204) when the response body is empty?

Error handlers aren't working on apps

Here's my simple TODO app with basic error handling using your handle_errors decorator: https://gist.github.com/SamuelMarks/9d7c796b8a336bb556f3

Unfortunately it is always throwing 500s. Is there some other way I'm meant to be passing errors? E.g.: in Node.js most of my functions have the signature (err, res).

Thanks for all assistance

PS: The functions from line 103 to 110 was another attempt of mine to get exceptions handled

MIME, someday, maybe*

In a properly REST application, the server should engage in content negotiation to deliver the client's preferred representation of a resource.

I would love to make it possible, within klein, to separate the reification of the resource (i.e. instantiation of a model object) from the serialization of that resource (i.e. invocation of a view).

For example: let's say we have an API, /foo, which is a JSON API that returns {"foo": "bar"}. However, the user might hit it in a web browser, where it might be more helpful to present this information via an HTML template (rendered, perhaps, via twisted.web.template).

I'd love to do something like this:

@something("/foo")
def foo(request, values):
    return {"foo": "bar"}

@foo.serializer("text/html")
def as_html(request, model):
    return someTemplate.fillSlots(**model)

@foo.serializer("application/json")
def as_json(request, model):
   return json.dumps(model)

Of course, serializing as JSON and HTML are probably two pretty common ones, so maybe there should be a less verbose way to set this up, simply a way to specify a template. I am not sure how this should look, I'd just like it to be easy. Also, this should be possible on inputs as well: you might want to accept parameters as either JSON or urlencoded key/value pairs.

*: With apologies to Marc Andreessen.

support XMLFile loader for klein.Plating

The original prototype in #89 referred to multiple different loader types, and the initial implementation in #118 .

Producer not being unregistered when wrapping static.File

(Using Klein master and Twisted 13.2.0)

Hi, just started using Klein for a small project, and I noticed that whenever I return static.File from a route-decorated function or method, I will intermittently see "Producer was not unregistered" when accessing files in the directory being served by static.File.

Plain old twisted.web works, this error only shows up when using Klein.

Simple example, run with "twistd --nodaemon web --class=testing.resource":

testing.py

from klein import resource, route

from twisted.web import static


@route('/files/', branch=True)
def files(request):
    return static.File('/Some/Directory/')

And lastly, the traceback from the logfile:

2013-11-13 10:47:26-0600 [HTTPChannel,0,127.0.0.1] 127.0.0.1 - - [13/Nov/2013:16:47:25 +0000] "GET /files/robots.txt HTTP/1.1" 200 27 "http://127.0.0.1:8080/files/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.71 (KHTML, like Gecko) Version/6.1 Safari/537.71"
2013-11-13 10:47:26-0600 [HTTPChannel,0,127.0.0.1] Unhandled Error
Traceback (most recent call last):
Failure: exceptions.RuntimeError: Producer was not unregistered for /files/robots.txt

Incorrect body/headers returned when werkzeug.router raises a redirect

As of v10.0, werkzeug.routing raises a RequestRedirect (subclass of HTTPException) when you pass an URL without a trailing slash, trying to redirect it to the corresponding canonical URL with the slash.

However, what appears to me to be a bug in Flask's error handler causes the returned redirect response to have mismatching body and headers:

• werkzeug.routing raises RequestRedirect (wkz/routing.py:1421)
• RequestRedirect is subclass of HTTPException but overridden to return a customized Response, as created by the redirect helper function. (wkz/utils.py:338)
• The exception bubbles up through Klein's Deferred, and eventually ends up in the failure handler processing_failed. (klein/resource.py:220)
• The handler checks for a HTTPException and then starts to propagate it up to response. (klein/resource.py:234)
• However, instead of setting everything using the response object obtained by calling he.getResponse(), it takes only the headers from it. The error code and body are taken directly from the exception with he.get_body().
• The HTTPException.get_body method is programmed to return a fixed string depending on the error code represented by the class. It ignores the overridden getResponse from RequestRedirect. (wkz/exceptions.py:113)
• The headers set in klein/resource.py:238 and the body returned in klein/resource.py:241 mismatch.

Notably, the headers of the RequestRedirect response include Content-Length of the original message. With the request actually returning a different body, with a different actual length, this causes problems. For example, curl will hang forever, trying to read all the bytes which the header says the response has, but will never arrive.

support XMLString loader for klein.Plating

#120

klein can not listen on unix sockets

Example script (broken.py)

from klein import route, resource

@route("/")
def index(request):
    return "It works!"

Run with twistd web -p unix:web.sock --class=broken.resource, the following exception occurs.

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/twisted/protocols/basic.py", line 581, in dataReceived
    why = self.lineReceived(line)
  File "/usr/local/lib/python2.7/dist-packages/twisted/web/http.py", line 1611, in lineReceived
    self.allContentReceived()
  File "/usr/local/lib/python2.7/dist-packages/twisted/web/http.py", line 1686, in allContentReceived
    req.requestReceived(command, path, version)
  File "/usr/local/lib/python2.7/dist-packages/twisted/web/http.py", line 790, in requestReceived
    self.process()
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/dist-packages/twisted/web/server.py", line 192, in process
    self.render(resrc)
  File "/usr/local/lib/python2.7/dist-packages/twisted/web/server.py", line 241, in render
    body = resrc.render(self)
  File "/usr/local/lib/python2.7/dist-packages/klein/resource.py", line 29, in render
    server_port = request.getHost().port
exceptions.AttributeError: 'UNIXAddress' object has no attribute 'port'

html5lib loader for klein.Plating

Twisted only supports loading formally-correct XML in twisted.web.template, but browser-ish tag soup is the order of the day. We should use https://github.com/html5lib/html5lib-python for this.

Disabling tracebacks in response should be much easier

By default Klein displays tracebacks in 500 response if unhandled error occurs. This is ok and useful for debugging but I think there should be some optional argument for run() disabling tracebacks and there should be information in documentation about this.

Figuring out how to disable tracebacks in production is currently not obvious and requires you to go into Klein code find out it uses Site() object under the hood, find out this object has displayTraceback attribute and set this attribute to False on class level.

So something like this:

from klein import Klein
from twisted.web.server import Site

app = Klein()
Site.displayTracebacks = False

@app.route('/')
def home(request):
    raise
    return 'Hello, world!'

app.run("localhost", 8080)

it would be better to simply have optional debug=True/False argument for Klein().run() that would set this for you.

Make `Klein` introspectable.

I'm working on generating some sphinx documentation from a klein application (loosely based on the autoflask directive from sphinxcontrib-httpdomain) but I need to access _url_map and _endpoints to get the relevant information.

form handling

possibly with integration with klein.Plating

Basically this should be a decorator that explains how to get inputs from either a form post or a JSON post body.

"plating" decorator for quickly constructing simple sites

Right now it's a bit tedious to construct a Klein site with the usual accoutrements of a simple website; some consistent navigation and page chrome, different things in the content area of each page, et cetera.

I propose that we add a decorator that assists in the construction of such sites, making it easy to quickly type some templates in-line, then gradually move them to external XML templates as they become more complex.

The decorator I propose decorates functions which return dictionaries of slot-filling data, so the decoratee's job is simply to populate the correct model data and the decorator's job is to fill out the Element with the appropriate twisted.web.template stuff to make it render HTML.

This is related to #78, in that ideally, the model data returned by the function could be returned as serialized JSON if the client asks for JSON, and only converted into HTML with styling if the client is a browser asking for HTML.

Prototype will be attached below.

How to host a klein based service?

Hey,

Suppose I have a klein based application which I'd like to deploy on a web server. Of course, the application shouldn't run as root and it should be automatically launched whenever the server is started (or at least when it's requested). Optionally, it would be great to run the app inside a virtualenv. What's the generic way to accomplish this?

I'd guess there is something you could do with twistd but in a quick search I couldn't find anything sufficiently easy for my level of expertise (=zero).

I think a few sentences about this in the documentation would do neatly. Apologies if I somehow overlooked the corresponding section.

Best regards,
Thomas

200 OK / HTML content type / empty response body when returning None from a branch route

In the following Klein application:

from klein import route, run

@route("/something", branch=True, strict_slashes=False)
def something(request):
    return None

run("localhost", 8080)

Then GET /something/else will result in a 200 OK with no content.

In fact, GET /something/ and GET /something have the same behavior, unless you set strict_slashes=True, in which case only GET /something/ will return a 404 (GET /something/else and GET /something still return 200s even with strict_slashes=True).

This seems off to me; a None ought to be 404s all around.

Don’t list mock as a runtime dependency

I’m pretty sure klein doesn’t need mock at runtime so I would prefer if it wouldn’t get installed along of it. :)

When rendering renderables, klein ought to emit a DOCTYPE.

The documentation for twisted.web.template has a brief note on DOCTYPE declarations:

http://twistedmatrix.com/documents/current/web/howto/twisted-templates.html#auto9

Whoever renders some stuff with HTML in it, it's their job to insert an appropriate DOCTYPE and set an appropriate MIME type. Maybe in your decorator you ought to be able to specify it somehow, but I think <!DOCTYPE html> is a good default.

Klein's codebase should meet the Twisted code standard

Rather than be a mismatched PEP8-Twisted code standard mess, it should be moved to meet Twisted's standard, with the old ones throwing deprecation warnings if possible.

Klein should implement Twisted's release concepts

Incremental and towncrier are the new hotness(tm) for this task and will bring Klein in line with Twisted release concepts.

It'll also be a good test run to prove their robustness for Twisted.

twistd klein

Thus spoke @glyph, in March of 2012:

"I want to take the example at the top of the readme, clip off the 'run' function at the bottom, then do

twistd klein myklein.py --port tcp:8080

so as to get all the usual goodness of listening on an endpoint.

Although what I really want is the ability to do

twistd klein --personal

just like I can do

twistd web --personal ...

So I can run awesome klein apps on twistedmatrix.com."

Non-ASCII chars in the path part of an URL raise an UnicodeDecodeError

ISTM that currently klein will always crash if a non-ASCII URL is called (eg. localhost/föö):

from klein import run, route

@route('/')
def hello(request):
    return "Hello"

run("localhost", 8080)

If you run curl 0.0.0.0:8080/fööö or curl 0.0.0.0:8080/f%C3%B6%C3%B6 you’ll get:

* Hostname was NOT found in DNS cache
*   Trying 0.0.0.0...
* Connected to 0.0.0.0 (127.0.0.1) port 8080 (#0)
> HEAD /fööö HTTP/1.1
> User-Agent: curl/7.37.1
> Host: 0.0.0.0:8080
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
HTTP/1.1 500 Internal Server Error
< Date: Fri, 21 Nov 2014 14:21:20 GMT
Date: Fri, 21 Nov 2014 14:21:20 GMT
< Content-Length: 6534
Content-Length: 6534
< Content-Type: text/html
Content-Type: text/html
* Server TwistedWeb/14.0.2 is not blacklisted
< Server: TwistedWeb/14.0.2

(in the second case it’s > HEAD /f%C3%B6%C3%B6 HTTP/1.1 of course)

and this traceback:

2014-11-21 15:26:49+0100 [HTTPChannel,0,127.0.0.1] Unhandled Error
    Traceback (most recent call last):
      File "/Users/hynek/.virtualenvs/71a275356bff3bc9/lib/python2.7/site-packages/twisted/protocols/basic.py", line 571, in dataReceived
        why = self.lineReceived(line)
      File "/Users/hynek/.virtualenvs/71a275356bff3bc9/lib/python2.7/site-packages/twisted/web/http.py", line 1656, in lineReceived
        self.allContentReceived()
      File "/Users/hynek/.virtualenvs/71a275356bff3bc9/lib/python2.7/site-packages/twisted/web/http.py", line 1731, in allContentReceived
        req.requestReceived(command, path, version)
      File "/Users/hynek/.virtualenvs/71a275356bff3bc9/lib/python2.7/site-packages/twisted/web/http.py", line 827, in requestReceived
        self.process()
    --- <exception caught here> ---
      File "/Users/hynek/.virtualenvs/71a275356bff3bc9/lib/python2.7/site-packages/twisted/web/server.py", line 189, in process
        self.render(resrc)
      File "/Users/hynek/.virtualenvs/71a275356bff3bc9/lib/python2.7/site-packages/twisted/web/server.py", line 238, in render
        body = resrc.render(self)
      File "/Users/hynek/.virtualenvs/71a275356bff3bc9/lib/python2.7/site-packages/klein/resource.py", line 73, in render
        default_method=request.method, url_scheme=url_scheme)
      File "/Users/hynek/.virtualenvs/71a275356bff3bc9/lib/python2.7/site-packages/werkzeug/routing.py", line 1124, in bind
        url_scheme, path_info, default_method, query_args)
      File "/Users/hynek/.virtualenvs/71a275356bff3bc9/lib/python2.7/site-packages/werkzeug/routing.py", line 1235, in __init__
        self.path_info = to_unicode(path_info)
      File "/Users/hynek/.virtualenvs/71a275356bff3bc9/lib/python2.7/site-packages/werkzeug/_compat.py", line 202, in to_unicode
        return x.decode(charset, errors)
    exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)

N.B. Query parameters like localhost/?föö=bär work fine.

---

I’ve checked the werkzeug functions and it’s apparent that the only way to fix this, is to fix it in Klein and pass only Unicode strings down to Werkzeug.

Maybe add an option to force certain charset and alternatively use chardet?

Debian / Ubuntu packaging

I was looking at the maturity of the project to evaluate its use in https://github.com/globaleaks/GlobaLeaks as replacement for cyclone and i found strange that the librrary regardless of the good results obtained is still not packaged in Debian / Ubuntu.

This ticket is to keep track of such activity.

I see that the process of packagin klein has been started in 2014 (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=738156) but i think no progress has been done.

@hawkowl can you clarify on the status so that eventually i will try to help on this front?

Can't serve encoded (e.g. gzip) resources

There's this code in server.Request.process:

        try:
            resrc = self.site.getResourceFor(self)
            if resource._IEncodingResource.providedBy(resrc):
                encoder = resrc.getEncoder(self)
                if encoder is not None:
                    self._encoder = encoder
            self.render(resrc)
        except:
            self.processingFailed(failure.Failure())

And since Klein resources don't provide _IEncodingResource the resource is never asked for an encoder. So things like this serve but aren't gzipped:

class GzipFile(File):        
    def getChild(self, path, request):
        child = File.getChild(self, path, request)
        return EncodingResourceWrapper(child, [GzipEncoderFactory()])

class MainApp(object):

    app = Klein()

    @app.route('/static', branch=True)
    def static(self, request):
        return GzipFile('/tmp')

I'm not sure where the code goes that would allow serving gzipped data. I can easily duplicate what the GzipEncoder does in my own code, but it seems like a useful thing for klein to provide (or at least facilitate).

I'm happy to do the work if someone can point me toward where the code should go.

TLS/SSL and twistd?

I want to run klein with https, and I've read the doc here: http://klein.readthedocs.org/en/latest/introduction/2-twistdtap.html

Unless I'm mistaken, this is using twistd to start a klein app by passing it to --class:

      --class=            Create a Resource subclass with a zero-argument
                          constructor.

But what if I want to pass arguments to my klein app? How would that work with twistd?

Request helper functions

Right now, I have a module that contain helper functions I use to minimize the number lines of code I'm forced to write. Most of the functions are similar to those that Tornado uses to get arguments or form values from a request. I use the following functions frequently (they're similar to Tornado's get_arguments, get_argument, decode_argument functions):

def getter(args, key, default=None):
    try:
        key = key.encode('UTF-8')
    except:
        raise Exception('ERROR: Cannot encode key (%s) using UTF-8' % (str(key)))

    return args.get(key, default)

def getOne(args, key, default=None):
    return getter(args, key, [default])[0]

def getOneCast(args, key, default=None, cast=None, decode=None):
    result = getOne(args, key, default)
    if decode:
        try:
            result = result.decode(decode)
        except Exception as e:
            raise Exception(ERROR: Unable to decode {0} using {1}.\nTRACEBACK: {2}'.format(result, decode, e))
    try:
        return cast(result)
    except Exception as e:
        raise Exception('ERROR: Unable to cast {0}.\nTRACEBACK: {1}'.format(result, e))

I wanted to take a "cleaner" and "simpler" approach and integrate the Request object with these functions, so that I can make a quick call like request.getOne('key'). But I cannot seem to find how to do that. I tried adding the functions to IKleinRequest and KleinRequest for a quick test, but that didn't work. The Request obj that gets passed into the route function is a twisted.web.server.Request obj, so I'm guessing the is getting passed somewhere in the the Twisted code. Is it possible to overload the Request object that gets passed into the route functions using Klein, or will I have to do more diving into Twisted?

Ideally I'd like to be able to only modify Request.args to be able to use a dict like object, but that's a pipe dream at the moment :)

Links

• Tornado API - Input

Support transports without ports

Currently, Klein calls request.getHost().port (see https://github.com/twisted/klein/blob/master/src/klein/resource.py#L77-L81 ) to get the port number to add to the server name. Unfortunately, if the IAddress returned by request.getHost() doesn't have a port attribute (if it's from a loopback transport or a unix socket, for example) it breaks.

A possible solution is to check for a port attribute or catch an AttributeError instead of just assuming it's there. :-)

middleware support

I'm thinking about adding a way to add middleware to a klein app.

I suppose this would hook into KleinResource.render? or is there some way to do this higher up in twisted already?

In middleware it would be nice to be able to add methods and instance variables to the request: Example log bound with request UUID prefix to be able to piece together all log messages that belong to a particular request (like done with thread local globals in other frameworks)

Another use is to perform generic authoriziation and return an early auth error.

This can of be done today by adding decorators but it would be nice not to have to add decorators to all routes.

Thoughts?

Tag a release / pop some champagne

Klein hasn't seen a release in a bit, and specifically, it'd be great to have one for #17.

CC @hawkowl

TIA!