natebrune/creep

A webservice for displaying nodeinfo information.

I am Nate Brune, bio is wip.

creep's Issues

User Agent

I know you said you'd get a User Agent but you haven't done it yet so I'm filing a bug

Incorrect "Host" http header

I've noticed that "Host" http header seem to be received here in a bit mangled form, which causes e.g. Django to return errors instead of replies for potentially-valid URLs.

Invalid HTTP_HOST header: '[[fc7b:d79a:4eb5:28f6:5686:637d:9f75:5db6]]'. The domain name provided is not valid according to RFC 1034/1035.
...
'HTTP_HOST': '[[fc7b:d79a:4eb5:28f6:5686:637d:9f75:5db6]]',
'HTTP_USER_AGENT': "NAT's Creep nodeinfo.json scanner +https://github.com/NateBrune/Creep",
'PATH_INFO': u'/nodeinfo.json',
...

Given that request gets handled by nginx here, then passed to uwsgi and only then to django, I'm not 100% sure if it's not one of these mangling it, but since there's no issue with other clients, it's likely not the case.
Plus should be trivial to test, I guess.

Keep track of which requests fail

Maybe have some sort of list of known nodes that have httpds that don't respond with 200 for /nodeinfo.json, maybe include some other shit like response headers to better figure out who runs them so we can yell at them.

Use a templating engine to generate the output file

This would make it more configurable and easier to change the layout, etc

Python 3

jesus christ it's time to move on from python 2. You suggested on IRC that it was an issue with urllib3, which is fine, use requests, it's way the fuck better than urllib anyway

Use celery to manage workers

Much better than managing the threads yourself, also allows threads to be distributed

Heading sizes, nesting

Contact is a child of Node, it should have a smaller heading (currently they're both <h4>s)

Output to somewhere configurable

maybe a command line flag, or output to stdout and leave redirection to the shell