Giter Site home page Giter Site logo

django-jackfrost's Issues

Support for Medusa renderers

If the class has get_paths and generate, it's a medusa class. Wrap it so that a class implementing __call__ calls self.renderer.paths internally.

Trailing slashes handling

Looking at the output of ./demo_project.py collectstaticsite, it doesn't look like I'm handling URLs without trailing slashes correctly.
Not sure what to do about them, if they don't have a file extension that a server could use to hint at the content-type to return, but they certainly shouldn't end up with a /index.html

Support for Sitemap classes

If a class has a __get attribute (and maybe get_urls?) assume it's a Sitemap class. Wrap it with another class which has __call__ and enumerates over the paginator and calls get_urls per-page.

Split out building of error pages

If I can get #6 done, I won't want to be writing the 401/403/404/500 templates (if they exist) to whatever storage backend is chosen multiple times (once per process).

Splitting the URLBuilder into a URLBuilder and an Errorbuilder, and calling the Errorbuilder once as part of the build process would circumvent that, at the expense of more complexity.

Add a test project

Having all these tests is all well and good, but I need to shove in an example project to demonstrate for myself, if nothing else, that it works.

Implement multi-processing

Invoking the management command with --processes=N or whatever (look to medusa, haystack etc to see if there's a convention) ought to split the URLCollector's results over N URLBuilders. This might wreak havoc with self.stdout though, if my recollection of haystack is anything to go by.

Add celery support

probably 2 tasks:
one task which collects all URLs from renderers
one task which, given a single URL, will build the page for that (probably calling utilsbuild_page_for_obj)
the collector task can spin out a single URL task for each thing it finds ...

Could potentially add this as a CLI option for the management command?

Split URLBuilder into URLReader and URLWriter?

URLReader should fetch all the responses, and shove them into an OrderedDict where the key is the target output filename and the value is the response content itself.

URLWriter should take the data from URLReader and write it to the storage backend defined.

This'll end up taking up more memory, especially if files are returned (eg PDFs, streaming responses etc) but does mean it'd be easier to hash the file contents and keep a store of that, and avoid writing unchanged files to the backend at all. This would be most useful in non-local-filesystem backends, I suspect, where the latency wouldn't be worth worrying about.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.