Giter Site home page Giter Site logo

peers's Introduction

Build Status

A geographically sorted list of public peering credentials for joining Hyperboria.

Hyperboria uses cjdns to construct an end-to-end-encrypted ipv6 mesh network. Connections between nodes are established manually, and traffic is restricted to the resulting social graph.

This repository exists for those who don't already know somebody on Hyperboria.

Using credentials

First, set up a cjdns node.

To connect your node to one of these public peers, follow the steps in the cjdns README.

Adding your public node's credentials

If you've created a public node, and would like to have it listed here, fork the repo, add a keyfile, run ./tests.py (make sure your file passes the test), and submit a PR.

We won't merge your credentials until our tests are passing.

Filepath conventions

Credentials are sorted geographically, by continent code. Nodes may be classified further, at the discretion of the node operator, and the administrators of the repository.

The suggested format is /continent/country/region/municipality. For example, /na/ca/ontario/toronto/.

Region and municipality codes are based on self identification, not any ISO standard. An operator might prefer to list their node in Cascadia instead of Washington state. For simplicity's sake, we'd prefer that new credentials conform to existing structures.

JSON formatting

We have tried to standardize the structure of the actual credential files, as such, they have the strictest requirements of anything in this repository.

  • Your credentials must be valid JSON.
  • They must contain the necessary fields:
    • ip/port
    • password
    • publicKey
    • contact (a means of contacting the operator)
  • The following fields are not yet required, but are recommended:
    • gpg, listing your 16 character pgp fingerprint (all caps, no spaces)
    • peerName, a human-readable name for the node
  • credentials should be formatted such that:
    • indentation uses four spaces
    • the file ends with a newline character.
  • credentials must use IP:port strings for keys
    • credentials using hostnames will not be accepted
  • If you are hosting it on a major server provider, please provide the name and shorthand for server.
    • On digitalocean, That may be digitalocean sfo2

    • On linode, that may be linode uswest or linode tokyo2

    • On AWS, use the region, followed by the city. Usable Server names are listed can be found on found on the AWS WEBSITE

      EX: useastnorthernvirginia,useastohio,southamericasaopaulo

{
    "192.168.1.5:10326":{
        "contact":"[email protected]",
        "gpg":"FC00FC00FC00FC00",
        "login":"default-login",
        "password":"nq1uhmf06k8c5594jqmpgy26813b81s",
        "peerName":"your-name-goes-here",
        "publicKey":"ssxlh80x0bqjfrnbkm1801xsxyd8zd45jkwn1zhlnccqj4hdqun0.k"
    }
}

Naming your entry

Credential files must end with .k. Otherwise, you can name your file whatever you want, but for simplicity's sake, avoid characters which will need to be escaped at the command line (or within the javascript api).

Javascript API

Peering credentials in this repository can be accessed via a simple Javascript API (using Nodejs).

It's available as a module on npm:

npm install hyperboria-peers

Usage

var Peers = require("hyperboria-peers");

/*  return a list of public peers located in North America */
Peers.filter(function (creds, path) {
    return path.indexOf('NA') !== -1;
});

/*  return a list of public keys */
Peers.map(function (creds, path) {
    return creds[Object.keys(creds)[0]].publicKey;
});

/*  the underlying data is exposed in a nested json structure */
console.log(Peers.peers);

console.log(Peers.peers.na.us.california);

peers's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

peers's Issues

uniaikratom down

@kyrias your peer seems to be down, please look into that. :)

The comment says it's going to expire during november 2016 (which was quite a while ago).

create json linter

jsonlint.com isn't quite what we need, since credentials would all need to be wrapped in curly braces to be considered valid.

Putting this on a github.io page with a textarea field and some buttons would be sufficient.

try{
    JSON.parse("{"+theirJSON+"}");
}catch(err){
    /* you messed up */
}

peers UNRESPONSIVE

hi.
several peers are no longer online and appear UNRESPONSIVE:

v0.0000.0000.0000.0015.kw0vfw3tmb6u6p21z5jmmymdlumwknlg3x8muk5mcw66tdpqlw30.k UNRESPONSIVE in 0kb/s out 0kb/s "Magik6k-sbg1"
v0.0000.0000.0000.001b.yrgb0xwfr9pz8swvnv6m9by8zw7v7uxxhl07qz318cjuvfgs1fc0.k UNRESPONSIVE in 0kb/s out 0kb/s "jazzanet"
v0.0000.0000.0000.0019.8268mn1bvz66nbb74tqw7ynjkcjrtruv8pgjf9kr34zv5d60p3r0.k UNRESPONSIVE in 0kb/s out 0kb/s "llygaid.cwningen.cymru"
v0.0000.0000.0000.0017.7ktfb2n336bguhfx81ts15qgjb8mjj4rz3vrnvj2dw89wsmmtpg0.k UNRESPONSIVE in 0kb/s out 0kb/s "penfar.cwningen.cymru"
v0.0000.0000.0000.0015.1nctdb89gtfrlnu71zyq97n14frl1r4z0ylwzc8vn7kpvrzu4yl0.k UNRESPONSIVE in 0kb/s out 0kb/s "igel-amersfoort"

the peers above are used, in my specific environment, only over ipv4.

are there any plans on bringing them back on?

thanks

accept ipv6 peers?

if the clearnet goes down.... we can't resolve ipv4. and since we cant use domains and our ipv4 addresses keep changing, can we use ipv6 instead? why can't we ditch ipv4 peering? that way we can still peer with each other if the clearnet goes down.

Test only changed things

Currently we test the entire repo, but it might be beneficial to have a way to detect what has changed for eg a pull request and only test those files. This way if a peer that fails the tests somehow get's committed, future pull requests don't fail. Would also be good for warnings, which some things in the repo are expected to have.

Script to add all peers to cjdroute.conf

Is there a script that imports all peers in the format cjdroute.conf expects? I'm doing it manually because the shape of the JSON returned by Peers.peers doesn't seem to match the example ones a in a newly-created cjdroute.conf. Thanks for putting this together, btw! Super useful for those who are just starting out like me :)

DNS in credentials

DNS records in peering credentials are only resolved at launch time.

While they make it possible to provide a public node even if you have a dynamic IP address, this can make it difficult to debug peering problems when users encounter problems stemming from 'UNRESPONSIVE' peers.

Each time the IP being pointed to changes, nodes which are using those credentials will need to restart (or otherwise dynamically reconfigure their peering credentials). This is problematic when considering that the purpose of this repo was initially to provide a simplified means of gaining access to the network.

Thoughts?

tests should actually try to connect to the peer

Right now we don't do much beyond validate the JSON structure. Tests should actually run cjdns, insert the peer, then verify that the connection succeeded via peerStats. It could also attempt to reach known nodes on Hyperboria to verify that the peer is connected to Hyperboria

further nesting for credentials

The js api recurses over the directory structure, finding keys, and exposing everything as a searchable JSON structure.

Now that this api is being used by a queryable http api, it's apparent that this repo could benefit from a further nested structure.

/<continent>/<country>/<region>/<city>/<neighbourhood>/credential.k would expose more search terms that users could employ to find nearby nodes.

Determine exactly which fields are and aren't required, and which are recommended

Currently test.py will fail if these fields are not present in any of the peering credentials:

  • publicKey
  • password
  • contact
    And throw warnings (which don't show up very easily, see #20) if these fields aren't present:
  • gpg
    Would people like to see these changed? I just kinda made these up while I was writing the tests script.

Other things to note:

  • Currently almost none of the peers listed have a GPG key (I believe EU/se/stockholm/leeloo.kyriasis.com.k is the only one)
  • The tests.py file has these fields defined in variables at the very top so it's trivial to change. Maybe eventually we'll even break it out into a proper config file.

The example code isn't working

I just run the example in README.md after installing hyperboria-peers successfully.
I get the following error.
console.log(Peers.peers.NA.us.california);
^

TypeError: Cannot read property 'us' of undefined
at Object. (/home/kaotisk/hype-peers/new.js:12:27)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
at Module.runMain (module.js:604:10)
at run (bootstrap_node.js:394:7)
at startup (bootstrap_node.js:149:9)
at bootstrap_node.js:509:3

Communist sympathizers

Lets move this interesting thread over here so the community can see what you're all about.

hyperboria/logos#3

If hyperboria network is a rats nest of Communists & Antifa we're going to need to figure out how to transition to a less destructive ideology or non-ideology.

Lets hash this out @ansuz @wfleurant

IPs vs Hostnames?

Do we prefer IPs or DNS hostnames? or do we care? If we do, we should update tests.py to reflect that.

support 'open peers' listings

Where 'open peers' are defined as an endpoint supporting a programmatic way to request and receive valid cjdns peering credentials with a unique password, as opposed to many peers sharing one published password.

four peers unresponsive.

v0.0000.0000.0000.001b.mh9m0411cfcg7xhdc8n6ckls1tjgnvvbdfzdgqf5196tfkw96rr0.k UNRESPONSIVE in 0kb/s out 0kb/s "igel-losangeles"
v0.0000.0000.0000.0019.6d2kt2hbcp7v0pw9q6f1u2s039kfnt4m4123rjxg26hsgrc12v80.k UNRESPONSIVE in 0kb/s out 0kb/s "salesforce-tower"
v0.0000.0000.0000.0017.ppj4n90uw68nvtmz7hz4g1ksubsxylgbhqg9p9x6f37vp4s599c0.k UNRESPONSIVE in 0kb/s out 0kb/s "kylers-awesome-website"
v0.0000.0000.0000.0013.q5nhj9kg6ddmk571jb259mct2ljd10z9xvyf05y3hu66wk4qnk20.k UNRESPONSIVE in 0kb/s out 0kb/s "outer" (weuxel)

Add GeoJSON for Public Peer Location

Adding more metadata on where peers are would assist in third-party mapping of the CJDNS network using openstreetmap mashups.

For example, fc00.org could add a map background and root known nodes before calculating force layout, representing known nodes with "pins" and location-unknown nodes as "clouds".

For this data, would have to insist that the geo-mapping match the actual location of the server and not the server operator, because someone hosting a VPS in a datacentre might accidentally add their location as operator.

It would also be helpful to correlate this map with a public database of significant clearnet infrastructure. For example, in Cork, Ireland I'm thinking of approaching a local datacentre to suggest a Cork CJDNS node on which to build a local network, which would then be adjacent to several inbound oceanic cables/fibres. It'd be cool for the map to present this data so that international peering negotiations between nodes could have more useful visual data attached. If you ask to peer with me and I observe that you're connected through some fat pipes that'll improve the network, that's a big argument in favour of agreeing.

EU/ru/moscow/h.bunjlabs.com.k

Now (UTC 2016-09-18 20:30:00) cjdns peer EU/ru/moscow/h.bunjlabs.com.k is offline. Please recheck it and, if approved, delete.

re: decentralization

A common convention for organizing public peering credentials is a step towards the goal of making cjdns easier to configure for the first time.

That being said, keeping the credentials in this repository where a set of gatekeepers can add or remove files centralizes an important aspect of cjdns (who connects with whom).

I'm considering creating a reference implementation that can scrape this repository, or another repository with the same structure. The conventions that make that possible should be agreed upon (indexing via html? keeping a json index?).

@kpcyrd, I expect you'll be interested in this because of yrd and nightfall.

@clehner Maybe we can tie this in with nij, generating a nodeinfo.json file in accordance with commonly agreed on location data, and then read nodeinfo.json when looking for nearby peers (programatically)?

Better warnings in tests

Right now tests.py can pass with warnings, but those warnings are only visible in the Travis CI web interface or by manually running tests.py. I'd like to investigate better ways to showing these warnings. Options that come to mind:

  • Expanding tests.py to post the warnings on GitHub. eg. if the warning is thrown when running a test for a pull request, comment directly in pull request thread like landscape-bot does, and if it was committed directly to the repo open an issue or something.
  • Build our own test server that listens for requests from a GitHub webhook to be notified of new pulls or commits, then uses the GitHub Status API to communicate the warning (eg, state=success and description=Not all recommended fields present or something, possibly with a target_url that could better explain which fields are missing and what needs to be fixed)
  • Build a server that listens for webhooks for pull requests like above, but use GitHub's tagging system to tag pull requests with warnings.

We should make sure to deal with #22 first so that this doesn't go off for everything if an existing peer causes a warning.

Thoughts?

PR #126 may have excessive entrys, or revealed a "known" issue in Configurator.c

see PR #126

during start

1524071046 DEBUG AdminClient.c:333 Connecting to [127.0.0.1:11234]
1524071046 INFO Configurator.c:135 Checking authorized password 0.
1524071046 INFO Configurator.c:159 Adding authorized password #[0] for user [default-login].
1524071051 CRITICAL Configurator.c:97 Failed to make function call [Timed out waiting for a response], error: [UDPInterface_beginConnection]
1524071051 CRITICAL Configurator.c:57 enable Log_LEVEL=KEYS to see message content.
1524071051 CRITICAL Configurator.c:71 Aborting.

Add "Supported Layers"?

It'd be nice if peers could announce somehow an ordered / conditional list of preferential layers for peering.

I'm thinking:

  • Local-only peers could use this to announce that they're available as public peers, but only within geographical boundaries.
  • Conditional public peers could announce their existence for members of, for example, a hackerspace.
  • Public-public peers could announce themselves but indicate that they only accept peering over Tor, or Wifi, or IP-over-carrier-pigeon.
  • Public peers without preferences could simply point out that if you're living somewhere local, it'd be better to use a direct peering system than to tunnel over the clearnet: a peer might have an ordered list of preferences starting with "ethernet/cat6", then "wifi/band:channel", then "clearnet/ipv4", or something.

This would need RFCifying or standardisation, and because each interface might present differently, it might even require multiple peer entries, with a new key indicating applicability, geographical boundaries, or special conditions.

This might seem at first glance like a way of adding conditions to the public peering list that make it less useful, but I think conditions like these would actually make it easier to add a public peer without worrying about becoming an inappropriate global hub.

Of course, that would rely on people actually respecting the instructions given therein, so perhaps a way to announce a peer without actually providing peering credentials, and instead indicating an email address to ask, might also be needed to make it work in practice..

Redo README.md

I wrote it off the top of my head, and would like some input.

add requirements for json to README

  1. Creds should be valid json (no comments).
  2. Creds should not be so long that they overflow (we need a linter)
  3. The json should have required fields. Obviously publicKey and password, but also:
    • location
    • contact
  4. Path must adhere to the repository layout. That means keeping a consistent hierarchy for folders.
  5. ???

peer unresponsive

it would be neat if the repo were able to serve up a magic online/offline image -- table of peers with status image?

New node

/eu/de/bandura.k

{
	"5.252.226.123:36433": {
		"login": "default-login",
		"password": "b0rvv0npd34c8pshl9j3dz4tr5sgdgw",
		"publicKey": "7nrlms2xb7tc0xqmf1vzkbcnt86kdusvyx354d5hx4f10fg9uh50.k",
		"peerName": "banduras-primary-server",
		"contact": "[email protected]",
		"gpg": "9AA28159FCEB3CD83BC212017E869146699108C7",
		"netcup": "ANX84"
	},
	"[2a03:4000:40:ad:14c8:36ff:fefa:2ef2]:36433": {
		"login": "default-login",
		"password": "b0rvv0npd34c8pshl9j3dz4tr5sgdgw",
		"publicKey": "7nrlms2xb7tc0xqmf1vzkbcnt86kdusvyx354d5hx4f10fg9uh50.k",
		"peerName": "banduras-primary-server",
		"contact": "[email protected]",
		"gpg": "9AA28159FCEB3CD83BC212017E869146699108C7",
		"netcup": "ANX84"
	}
}

Travis checker for peering info

Iterate all files, load them as JSON, bencode them and check size. Just to have correct peering info. Also we could require essential and contact info in the peer file.

Attempting to build on Raspberry Pi 3

I manually ran the pimesh install instructions here for all the dependencies but am getting this error.

After a lot of CryptoAuth test output:

memory/Allocator.c:237 TODO(cjd): make this optional.
net/Benchmark.c:193 TODO(cjd): this will fail with a different encoding scheme
benc/serialization/json/JsonBencSerializer.c:331 FIXME(gerard): silently skipping anything we don't understand
admin/angel/Core.c:69 TODO(cjd): we need to begin detecting MTU and informing the OS properly!
client/Configurator.c:594 * FIXME(sdg): moving noforks after setuser might make nproc <- 0,0 work
./interface/tuntap/test/TUNInterface_ipv4_root_test.c:59 TODO(cjd): fix TUNConfigurator_addIp4Address() for Illumos, BSD.
dht/dhtcore/NodeStore.h:156 * FIXME(arceliar): Documentation is out of date
./tunnel/test/IpTunnel_test.c:391 Allocator_free(alloc); //TODO(cjd): This is caused by an allocator bug.
switch/SwitchCore.c:219 TODO(cjd): hmm should we return an error packet?
net/InterfaceController.c:838 TODO(cjd): eps are created in 3 places, there should be a factory function.
interface/tuntap/NDPServer.c:128 TODO(cjdns, Kubuxu): Filtering basing on cjdns network and tunnels.
dht/DHTModuleRegistry.c:64 TODO(cjd): Call a debugger with all unhandlable messages?
dht/dhtcore/NodeStore.c:646 TODO(cjd): This does not depend on nodeStore or alter the link, consider moving to Node.c
dht/dhtcore/NodeStore.c:877 FIXME(arceliar,cjd): linking every node with 0 link cost, this can't be right.
dht/dhtcore/NodeStore.c:1195 TODO(arceliar): protect keyspace, evict the worst bestParent instead?
dht/dhtcore/NodeStore.c:1275 TODO(arceliar) the 1023 here is pretty arbitrary...
dht/dhtcore/NodeStore.c:1277 TODO(arceliar): is this safe?
dht/dhtcore/NodeStore.c:1493 TODO(cjd): this could return ~0
dht/dhtcore/NodeStore.c:1513 TODO(cjd): this has been the source of assert failures and we might be sweeping
dht/dhtcore/NodeStore.c:1618 TODO(cjd): Schlameil the painter
dht/dhtcore/NodeStore.c:2007 FIXME(arceliar): calcNextCost is guessing what the cost would stabilize to
dht/dhtcore/NodeStore.c:2023 TODO(cjd): What we really should be doing here is storing this link in a
memory/BufferAllocator.c:25 * TODO(cjd): addOnFreeJob adds a job which is only run when the root allocator is freed
dht/dhtcore/Janitor.c:239 FIXME(arceliar): Probably need stronger requirements than this.
dht/dhtcore/Janitor.c:398 FIXME(arceliar): This target probably isn't optimal.
dht/dhtcore/RouterModule.c:85 * TODO(cjd): ---
Failed to build cjdns.
Total build time: 40405ms.

improve readme structure

15:12 <@mk-fg> Also, I find structure of that document utterly confusing
...
15:15 <@mk-fg> Still, saying something like "this repo is a list of public peer credentials, which can be added here (in Nodeinfo.json format) by anyone via github PR" is like one sentence totally tying it all together

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.