For what are the use of the POST urls curl -X POST https://backstroke.us/_dfasdf1asd1f3dsf32df...?

[backstrokeapp]

For what are the use of the POST urls curl -X POST https://backstroke.us/_dfasdf1asd1f3dsf32df...?

[1egoman]

@evandrocoan That url is the url that is called behind the scenes by Backstroke (ie, the github webhook). As explained on the dashboard, running curl -X POST https://backstroke.us/_XXX... kicks off Backstroke at any arbitrary time if you'd like to run the check external to Backstroke for some reason.

[evandrocoan]

But if I understand correctly, backstroke is kicking off automatically. Why would I need the URL?

If some one take my URL on the git repository they can start kicking it like ddos, is there a problem? Should the URL be a secret or they can be public?

[1egoman]

@evandrocoan In 99% of cases, the url is not required. I've run into cases (and others have as well) where they've wanted to integrate backstroke into a different system and wanted to kick off Backstroke on their own from a system they've written (100% not required, an edge case). That's the only reason why the url is user facing (it's hidden in the dashboard to try and emphasize this too).

As fas as any denial of service concerns: the url can be public. Since nothing will happen if the repo is already up-to-date, the worst thing that could happen is that your repository is checked more often for changes. Also, backstroke.us is behind cloudflare, which should take care of any abuse.

[evandrocoan]

the url can be public

I got this response from GitHub while running my pull algorithm:

API rate limit exceeded for evandrocoan.

How does it does my name? Does it know my name from the consecutive calls to the backstroke URL https://backstroke.us/_dfasdf1asd1f3dsf32df...?

If so, could someone be using my URL to kick off GitHub, then when I actually need to kick some repository, I cannot because they exhausted/kick off all my daily GitHub API quota?

...
[DEBUG] 13:14:24:617565  -5069 Index: 28/54, submodule "Packages/Indent and braces"
b'{"status":"ok","output":{"status":"ok","pullRequest":{"msg":"There\'s already a pull request for this repo, no need to create another."},"isEnabled":true,"many":false,"forkCount":1}}'
[DEBUG] 13:14:25:753405 135840 Index: 29/55, submodule "Packages/Invert Selection"
b'{"status":"ok","output":{"status":"ok","pullRequest":{"msg":"There\'s already a pull request for this repo, no need to create another."},"isEnabled":true,"many":false,"forkCount":1}}'
[DEBUG] 13:14:26:907257 153852 Index: 30/56, submodule "Packages/LaTeXTools"

ERROR!  b'{"error":{"code":403,"status":"Forbidden","message":"{\\"message\\":\\"API rate limit exceeded for evandrocoan.\\",\\"documentation_url\\":\\"https://developer.github.com/v3/#rate-limiting\\"}"}}'

[1egoman]

@evandrocoan Backstroke uses your Github access token to make requests, and Github rate limits users with tokens to 6000 requests per hour. At worst, someone would prevent Backstroke from working on your repositories for an hour, and then once the rate limit expired, Backstroke would work again. This is a Github limitation and something that cannot be easily circumvented.

[evandrocoan]

On the example above, I did not hit the 6000 request limit, I only had make about 50 sequential requests to the URL. GitHub seems to block repeated requests, on the 6000 hit limit.

So, by the fact that someone has my URLs they can be popping the GitHub API under my name, therefore run dry all my 6000 requests?

[1egoman]

Backstroke makes many calls to Github for a given request. If someone were to hit a link's webhook url repeatedly over an over, yes, they would "run dry" all of your requests. At worst, this means that your repository would be out of date for an hour until the Github rate limit reset.

[evandrocoan]

Thanks for the patience. Seeing now how it works, I would like to ask if it is possible to block repeated requests to the same URL, as the GitHub does their rate limit.

So as my URLs are public, anyone can create a simple script which start pinging repeatedly the same URL, over and over again for ever. This would cause GitHub to exhaust all my 6000 requests just for on repository URL, and can keep me blocking for ever from ping other repositories, as they are kicking off all my requests limit with the same repository.

Basically limiting the kicking off of a URL as only one request is acceptable for 6000 seconds. So it would block any one from exhausting my GitHub rate limit using one or just a few URLs from my account.

This should not affect the system, as it is only a tool to help integrate changes from the upstream, when they are available, which means, I do not need to see them immediately, so it should delay a few hours.

[1egoman]

Currently, this doesn't exist. If you'd like to try to implement it, here's the most up-to-date branch. I can review any submitted pull requests.

[evandrocoan]

Thanks, for now I cannot engage on it, but I added it on my issue tracker. So the simplest solution today is to be removing all the URLs from public. Perhaps could you place a warning on the page stating to not publish the URL.

But now I we can discuss the algorithm to accomplish so, I can think about one, do you approve it for implementation?

1. Implement a hash table with the repositories as the key, so when performing check to the URL https://backstroke.us/_dfasdf1asd1f3dsf32df, the repository table key would be dfasdf1asd1f3dsf32df, which is accessed and compared against the current time in EPOCH as 1501635118.

   If there is the minimum time of 6000 seconds passed since the last input, the call is approved and the time updated on the hashtable. Otherwise the call is just rejected. But when there is no entry, a entry is added and the call is performed.

   This algorithm can be improved, so the hashtable is restarted every 24h, to clean/free up some memory, and also when its size passes some limit to avoid staving the server memory.

---

Update

I think it is just better not publish the URL anywhere, so you do not have to care about anything. When I got time I can look into implement it, but probably I will end up not to.