arlolra/check

forked from https://git.torproject.org/check.git

check's Issues

Project requires exit-data or retrieval information

check won't run unless there is data/exit-policies available
exitips won't run unless there public/exit-addresses is available so it won't generate a new exit-policy listing

I suggest either including these files back into the repo for testing or adding instructions to the readme on how to get either one of these files easily.

Translations

Transifex seems to have many more check translations than are listed on https://check2.torproject.org/

It would be good to get that updated and generate a new .pot file that contains the missing strings (ex. "This page is also available in the following languages").

What to do about false positives that are found?

Perhaps add a wiki page to list false positives at specific times to see if we can track down causes?

I just exited from 199.48.147.41 @ 00.34 GMT+1 27/09/2013 and got a 'You're not connected to Tor' message but I've verified I can connect to onions and such, so I'm pretty sure it's wrong. This is likely due to missing an ip from TorDNSEL but I can't check it right this moment so I'm leaving it here to investigate later! Which reminds me, we should probably add investigation instructions to the docs?

I guess this is really a false negative. Regular check.tpo gets it right.

Saw multiple response.WriteHeader calls

2013/10/02 03:04:50 http: multiple response.WriteHeader calls

Investigate.

Extending B) from https://trac.torproject.org/projects/tor/ticket/9204

(Just creating an issue to track this here)

"For users who want to consider all relays in the past n hours, this script should be
able to handle reading multiple consensus files (ideally just by concatenating them
before they go in)."

Tordnsel disregards things not seen in 48hours according to the exit-list format spec. If the user wants to check n > 48, then do we also need to handle that?

Thinking about this, I've noticed that quite a few addresses are not listed in the exitlists file, if you edit exitips.py@line25 you can see also.

if router.fingerprint in exits:
    if r.Address == exits[router.fingerprint]:
        print "Already have correct IP %s" % r.Address
    else:
        print "OK %s" % r.Address
    r.Address = exits[router.fingerprint]
else:
    print "Not found in exits: %s" % r.Address

I'm not sure exactly what this means for check, the consensus is meant to be relays thought to be running at a given time? Is Tordnsel slacking?

rsync

Karsten is shutting down the metrics server's rsync service
https://lists.torproject.org/pipermail/tor-dev/2014-July/007101.html

Link to Atlas

If Tor.

See around, #14 (comment)

Documentation

Not enough said.

False negatives

Needs investigation.

check2
https://lists.torproject.org/pipermail/tor-talk/2013-October/030791.html

check
https://lists.torproject.org/pipermail/tor-talk/2013-October/030797.html

gzip vs no gzip

What kind of infrastructure will check run on? If the server has a weak CPU, gzip should be disabled. If there's free capacity though, I'll send a pull request for the code.

Here's some initial stats for a linode 1024 at a concurrency of 400 (as required), without making any attempts to improve performance. The key thing here is the document length from the end-user point of view.

Testing was done over a 12MB/s link at a latency of 100ms.

Without gzip:(cpu: 350%, mem: 120MB on an 8 core VM)
Document Path: /
Document Length: 2739 bytes

Concurrency Level: 400
Time taken for tests: 6.633 seconds
Total transferred: 45664000 bytes
HTML transferred: 43824000 bytes
Requests per second: 2412.24 #/sec
Time per request: 165.821 ms
Transfer rate: 6723.18 [Kbytes/sec] received

With gzip: (cpu: 450%, mem: 100MB on an 8 core VM)
Document Path: /
Document Length: 1027 bytes

Concurrency Level: 400
Time taken for tests: 8.415 seconds
Total transferred: 19008000 bytes
HTML transferred: 16432000 bytes
Requests per second: 1901.29 #/sec
Time per request: 210.383 ms
Time per request: 0.526 [ms](mean, across all concurrent requests)
Transfer rate: 2205.80 [Kbytes/sec] received

Tests

Should be possible to add tests against the datastore and exitips script.

Init.d script

For running in production on Debian.

"This browser"

Use the wording "This browser" instead of "Your browser". Mentioned in #tor-dev.

velope: please, please, can we at least get one basic wording fix done -- "This browser"
velope: yes. this point was made quite a while ago, based on observing new users
velope: "your browser" is the one they already have and know how to use, and running TBB does not make that one safe

make descriptors is downloading too much data

really only want an hours worth

More stuff

As suggested in https://lists.torproject.org/pipermail/tor-talk/2013-October/030264.html

I started changing the design a bit in https://github.com/arlolra/check/tree/design but meh ...

Bulk should have a port option in the form

subject says it all

API

I started working on an API,

https://check.torproject.org/api/bulk?ip=38.229.72.19&port=443

This needs to be further fleshed out and documented.