communication stops on 'Unauthenticated'

[cazala]

communication stops on 'Unauthenticated'

[cazala]

nope, I'm about to release v2

[procloud]

https://github.com/clintar/cryptonote-universal-pool/blob/master/lib/pool.js

cryptonote-universal-pool supports keep alive

[sunk818]

Duplicate of #30

I use pkill coin-hive-stratum and re-run coin-hive-stratum

[procloud]

also appears data just stops for no outputed reason after a random time...

[Oct 30th 02:30] message sent to miner: {"type":"job","params":{"blob":"0606e29cdacf054aeddeab2efa4f9cc2d4b6c80639c06161ad0759645e4b3427b833622e678e98000000001966adc86557676acc604a6e7a8251f9f3135e5d1de475678662a16dbdc8b03004","job_id":"P3fjMZHZRjZVwaW6uIFZGo5V6+4L","target":"9bc42000","id":"6b905dcb-ebab-41da-8d4e-b9fd8056fc50"}}

[Oct 30th 02:31] message from pool to miner: {"jsonrpc":"2.0","method":"job","params":{"blob":"0606e29cdacf054aeddeab2efa4f9cc2d4b6c80639c06161ad0759645e4b3427b833622e678e98000000007ae70a1f9cc70c4d14af9ba3be43592c39d81b945ea9b82e453c8db4d3b4ac2c04","job_id":"1e162YNPHz8rjgyWL1dSEHZzw/w8","target":"9bc42000","id":"6b905dcb-ebab-41da-8d4e-b9fd8056fc50"}}

[Oct 30th 02:31] message sent to miner: {"type":"job","params":{"blob":"0606e29cdacf054aeddeab2efa4f9cc2d4b6c80639c06161ad0759645e4b3427b833622e678e98000000007ae70a1f9cc70c4d14af9ba3be43592c39d81b945ea9b82e453c8db4d3b4ac2c04","job_id":"1e162YNPHz8rjgyWL1dSEHZzw/w8","target":"9bc42000","id":"6b905dcb-ebab-41da-8d4e-b9fd8056fc50"}}

[Oct 30th 02:32] message from pool to miner: {"jsonrpc":"2.0","method":"job","params":{"blob":"0606af9edacf05735d8be0dd372c9c73074e8c204dc696e02c26f884152d6aff81c333288f67120000000083477fc091faaf02ef868a4903213360966fcda6a55ed18e5f298dee6cec2edf01","job_id":"e1QJbNXRCdquk/ANgY5P/7bdq+6s","target":"9bc42000","id":"6b905dcb-ebab-41da-8d4e-b9fd8056fc50"}}

[Oct 30th 02:32] message sent to miner: {"type":"job","params":{"blob":"0606af9edacf05735d8be0dd372c9c73074e8c204dc696e02c26f884152d6aff81c333288f67120000000083477fc091faaf02ef868a4903213360966fcda6a55ed18e5f298dee6cec2edf01","job_id":"e1QJbNXRCdquk/ANgY5P/7bdq+6s","target":"9bc42000","id":"6b905dcb-ebab-41da-8d4e-b9fd8056fc50"}}

i am testing with only one web miner...

ok... an unauthed message just appeared in the logs 5mins laters...

[Oct 30th 02:37] message from miner to pool: {"type":"submit","params":{"job_id":"e1QJbNXRCdquk/ANgY5P/7bdq+6s","nonce":"1a5299ea","result":"d209d67cab711467b0c6f72e3ddd27c78ab0c99bcd2f3e4a6bfd9c1a84550100"}}

[Oct 30th 02:37] message sent to pool: {"id":4,"method":"submit","params":{"id":"6b905dcb-ebab-41da-8d4e-b9fd8056fc50","job_id":"e1QJbNXRCdquk/ANgY5P/7bdq+6s","nonce":"1a5299ea","result":"d209d67cab711467b0c6f72e3ddd27c78ab0c99bcd2f3e4a6bfd9c1a84550100"}}

[Oct 30th 02:37] message from pool to miner: {"id":4,"jsonrpc":"2.0","error":{"code":-1,"message":"Unauthenticated"}}

i am using supportxmr

[sunk818]

I'll bite since you ignored my duplicate post.

What is the keep alive for the pool? Did you submit an acceptable hash in that five minutes?

[procloud]

keep alive may be related to #45

[procloud]

the miner only replies when it solves a hash. forcing the unauthed response far later. since a new job will occur at block intervals at a minimum of block time. what is the block time for monero? unauths are common and must be handled gracefully by the code.

[cazala]

do you guys keep seeing this with v1.4.1? Does this happen due to stale connections or because of miners not sending shares? In v1.4.1 the tcp connections are shared by monero address and the sockets set keepAlive to true so there shouldn't be idle connections, but i don't know if this is something else,

[sunk818]

You can see latest blocks. It is every few minutes.

because of miners not sending shares

Maybe the keep alive if pool supports it will help.

I think low hash rate and high pool difficulty means some miners can not send acceptable share. Pool times out on miner, expects re-auth, but since miner has not done this, sees the share submission as Unauthenticated. My theory. Need to test more.

[cazala]

I could kill the miner connection when an Unauthorized arrives.. then the miner will try to reconnect and reauth automatically. Would that help? At least so the miner doesn't keep submitting shares forever without success.

[sunk818]

Yes, I think that would help. With the recent changes (e.g. keep-alive), maybe Unauthenticated will not occur. But if pool doesn't support keep alive, or it still occurs, it is a nice backup. Thanks for making all these changes. You really need a LTC address too! ;)

[cazala]

I just added it to the README (;

[cazala]

Okay I just added this, now when an Unauthorized occurs it will kill the miner connection. v1.4.3

[procloud]

its good to know reauths are bing handled now. i will let you know if i still see idle connects, otherwise the miner will sit there until it reauths.

[procloud]

there are still large time gaps in communication. i am using one low hash web miner at 4h/s. pool is supportxmr.

[Oct 30th 11:55] message from pool (FOGG): {"jsonrpc":"2.0","method":"job","params":{"blob":"0606a4f7decf05cea2e44b8e131a0c6f4205ac0c01f3fc24e181626e4ff37764730587521168aa00000000f2601f864e710e2834b8000d82ab546940ecaa4d5975a4116179771a3782ce1904","job_id":"A1XncZUf6t/t9MFHyDNzfNhJw1qk","target":"9bc42000","id":"7af85f00-3192-4183-a1f4-b985e88b8ccc"}}

...time gap no logging

[Oct 31st 12:04] message from miner (7af85f00-3192-4183-a1f4-b985e88b8ccc) {"type":"submit","params":{"job_id":"Mc1DdZdkZ2VtxF0jYO8mETS//AXw","nonce":"da3ea0d3","result":"02846d2a0f5c1356de4eaacefacfeb414cf8a16e70eeee324404b42032150e00"}}
[Oct 31st 12:04] message sent to pool (46WNbmwXpYxiBpkbHjAgjC65cyzAxtaaBQjcGpAZquhBKw2r8NtPQniEgMJcwFMCZzSBrEJtmPsTR54MoGBDbjTi2W1XmgM): {"id":4,"method":"submit","params":{"id":"1f3200ca-09ce-4c1c-bfea-9d188ea65342","job_id":"Mc1DdZdkZ2VtxF0jYO8mETS//AXw","nonce":"da3ea0d3","result":"02846d2a0f5c1356de4eaacefacfeb414cf8a16e70eeee324404b42032150e00"}}
[Oct 31st 12:04] message sent to miner (7af85f00-3192-4183-a1f4-b985e88b8ccc): {"type":"hash_accepted","params":{"hashes":4}}
[Oct 31st 12:04] message from pool (46WNbmwXpYxiBpkbHjAgjC65cyzAxtaaBQjcGpAZquhBKw2r8NtPQniEgMJcwFMCZzSBrEJtmPsTR54MoGBDbjTi2W1XmgM): {"id":4,"jsonrpc":"2.0","error":{"code":-1,"message":"Unauthenticated"}}
[Oct 31st 12:06] message from miner (7af85f00-3192-4183-a1f4-b985e88b8ccc) {"type":"submit","params":{"job_id":"Mc1DdZdkZ2VtxF0jYO8mETS//AXw","nonce":"a880ac76","result":"d8530b641b37614f170fcd1b2437fe4bb24dc25f5bcf0e15aaab08b606940300"}}

... time gap no logging for 4 minutes now

does not appear as though re-auth occurred... no connection destroy. miner has remained connected on same reverse port throughout

[procloud]

miner submitted job

[Oct 31st 12:12] message from miner (7af85f00-3192-4183-a1f4-b985e88b8ccc) {"type":"submit","params":{"job_id":"Mc1DdZdkZ2VtxF0jYO8mETS//AXw","nonce":"f444fea6","result":"a51bf9964347857ddd4d497a30a038b20713928825bc69bf293bfd304b691100"}}

no response from pool

[cazala]

that job Mc1DdZdkZ2VtxF0jYO8mETS//AXw was already submitted, that's why it was not sent to the pool (see [Oct 31st 12:04] message sent to pool ...). Once a job_id is sent, if the miner sends more jobs with that id they are not sent to the pool (otherwise the pool responds with invalid_job_id). In that case the Unauthenticated user was the donation one (from address 46WNbmwXpYxiBpkbHjAgjC65cyzAxtaaBQjcGpAZquhBKw2r8NtPQniEgMJcwFMCZzSBrEJtmPsTR54MoGBDbjTi2W1XmgM), that Unauthenticated is not being handled properly, yea. I''ll look into that.

[cazala]

I deleted some conditions that were causing jobs from miner not being sent to the pool and jobs from pool not being sent to the miners. Also I deleted some references to undefined variables (old code) that was throwing in certain corner cases. I published this under version v1.4.7.

[procloud]

still getting periods with no comm

[Oct 31st 07:05] message from pool (FOGG): {"jsonrpc":"2.0","method":"job","params":{"blob":"0606a591e3cf0577a88e3a3b83a0f4865f53915e7def5f045e28c1ec0bc845d744a3982b26b259000000004986336a391ef5320da0ae24723431a51013da6177893fbbea1db36e11f3762e10","job_id":"flMvKs9O3yyY50LU0nKvRD5u1r9f","target":"9bc42000","id":"d5416d8c-8773-4f36-90fc-c4accca4397b"}}
[Oct 31st 07:05] message sent to miner (d5416d8c-8773-4f36-90fc-c4accca4397b): {"type":"job","params":{"blob":"0606a591e3cf0577a88e3a3b83a0f4865f53915e7def5f045e28c1ec0bc845d744a3982b26b259000000004986336a391ef5320da0ae24723431a51013da6177893fbbea1db36e11f3762e10","job_id":"flMvKs9O3yyY50LU0nKvRD5u1r9f","target":"9bc42000","id":"d5416d8c-8773-4f36-90fc-c4accca4397b"}}

... idle no comm

[Oct 31st 07:08] message from miner (d5416d8c-8773-4f36-90fc-c4accca4397b) {"type":"submit","params":{"job_id":"flMvKs9O3yyY50LU0nKvRD5u1r9f","nonce":"b87a888e","result":"a7d520413fccee53cf7b8580f539c5ec14a3bb1010d8175ee4345e23e18a0700"}}
[Oct 31st 07:08] message sent to pool (FOGG): {"id":4,"method":"submit","params":{"id":"d5416d8c-8773-4f36-90fc-c4accca4397b","job_id":"flMvKs9O3yyY50LU0nKvRD5u1r9f","nonce":"b87a888e","result":"a7d520413fccee53cf7b8580f539c5ec14a3bb1010d8175ee4345e23e18a0700"}}
[Oct 31st 07:08] message from pool (FOGG): {"id":4,"jsonrpc":"2.0","error":{"code":-1,"message":"Unauthenticated"}}
[Oct 31st 07:08] miner conection destroyed (d5416d8c-8773-4f36-90fc-c4accca4397b)
......

this time the miner does auth without issue after the connect is destroyed, so that is working now. but miner/pool comm is still missing something. not sure what, causing delays.

im still using the same setup as before. one low hash miner ~4h/s and supportxmr pool

[cazala]

mm, it looks like you've been logged out because of inactivity. I mean, looking at those logs, the job was successfully sent to the miner, and this miner took 3 minutes to submit it. The submitted job was successfully sent to the pool, which responded with Unauthenticated because it doesn't know who the miner is (session expired). Therefore, the miner connection is destroyed and the miner reauthenticates.

Was this working for you with the previous versions? because I don't see why this wasn't happening before when there were individual connections.

[sunk818]

You get unauthenticated error because your session timed out but pool didn't drop your connection. This is a bug in cryptonote-universal-pool. Ask the pool operator to reduce difficulty (so you can submit acceptable hash sooner) or raise time out (so you have longer to submit an acceptable hash). If your hash rate is 4 H/s, even doubling the timeout to 20 minutes may not help. That's really low hash rate and you shouldn't bother using your smartphone.

Might look into using coin-hive-stratum -> xmr-node-proxy -> pool . xmr-node-proxy may be able to insulate from the timeout issue somewhat. Or if you use more miners, one of them should be able to submit an acceptable hash before timing out.

I think this is an edge case because of how @procloud is testing. Does not reflect real-world use scenario especially if @cazala will aggregate same monero address to one connection. Easy test is to use another hasher with higher hash rate (at least 20 H/s) and I bet the Unauthenticated does not occur or may take many hours.

@cazala

1. when miner connection is destroyed, what happens to your last message? I'm thinking you can't resubmit because your hash is unique to your job_id and new connection means new jobid. Old hash re-submit wouldn't be accepted...

2. Can you log what difficulty the pool is currently announcing?

3. Is coin-hive-stratum able to detect if session was dropped by the pool before client submits hash?

[cazala]

1. yes, if the miner submits a job after re-authenticating, the pool will respond with invalid_job_id

2. If there's a way to get that data out of the pool, yea, but it's not in any of the messages i've seen being exchanged between pool and miners.

3. No, but I should read more about the stratum protocol to know if that's possible.

[cazala]

Something that I could try to mitigate this is authenticating only once per address. All the miners that come afterwards could reuse the workerId of that address instead of sending another login message. So for the pool there would be only one miner. If I do so then I guess I should implement some logic to distribute the work among workers, so they don't get the same jobs (for this I'll need to ask the pool for jobs using the getjob method). Just an idea.

[procloud]

im not sure if it was occurring before... as i was using forever and the script was more likely to restart often in the earlier versions. also, i had multiple miner connects making it harder to debug. from what i understand timeout is fairly high, about 15mins on supportxmr, but that may be incorrect. i also notice mining clients like ccminer have a time-out setting for stratum long polling. also, this logic may be built into coinhives protocol somehow, see #45 to mitigate. perhaps they are doing a bit of what cazala just suggested in the last post here. i noticed solid blues very often when connected to coinhive (accepted hashes) even at very low hash rates. considering this entire approach is to be able to use even 1h/s miners in aggregation... low h/s miners must be given important attention for perf overall. i have considered aggregating to a proxy. i am already using xmrig proxy with amazing results, but its only nicehash, and i only use it for my xmrig clients.

[sunk818]

1. This is getting complicated. :) Since pool gives new job to each connection, you want to ask for additional jobs (for each miner). You are hoping between all the miners hashing, one should be able to submit an acceptable hash before the time out. This way, each miner could get updates on miner.getAcceptedHashes() in the JavaScript as well. That sounds like a good idea to me!

2. I read xmr-node-proxy is able to lower the share difficulty so low hashing miners can submit shares too! I wonder if you can implement that code/algorithm into coin-hive-stratum?

3. What happens to CoinHive.User instead of 'CoinHive.Anonymous'. Will CoinHive.User get their connections? Or can your idea apply to CoinHive.User as well as long as Monero address is the same?

[sunk818]

i noticed solid blues very often when connected to coinhive (accepted hashes) even at very low hash rates.

CoinHive's share difficulty is 256. Quite a low bar when the lowest share difficulty I've seen for a Monero pool is 3000. You multiply H/s * 60 to get the share difficulty. 256 is right for 4 H/s. For share difficulty of 3000, you should be 50 H/s or more. Why don't you test with a another miner with 50 H/s or more and see if Unauthenticated ever occurs. Then you will know if the error is due to timeout or not.

[procloud]

ahh, i see. ok. yes, im on the lowest i could find... supportxmr has a 1000.

[procloud]

even still, i notice in the logs, that the message from pool is a duplicate blob every time before the unauth timeout

[Nov 1st 12:37] message from pool (FOGG): {"jsonrpc":"2.0","method":"job","params":{"blob":"0606b4aee4cf0588266e7c82e6a057568ce833a1775517cd7c6e43f1e87a5e3e3491ccd
862c12300000000fe5ed72d301e83ce1a2cdc97f309edb4f5f6fe7d1d1e88613699c82eb98fbbca01","job_id":"q4X7h34c3ekMKFg7XlZzvoC1d4XT","target":"9bc42000","id":"608d069b-6f27-
471e-9648-b3f6eae78088"}}
[Nov 1st 12:37] message sent to miner (608d069b-6f27-471e-9648-b3f6eae78088): {"type":"job","params":{"blob":"0606b4aee4cf0588266e7c82e6a057568ce833a1775517cd7c6e4
3f1e87a5e3e3491ccd862c12300000000fe5ed72d301e83ce1a2cdc97f309edb4f5f6fe7d1d1e88613699c82eb98fbbca01","job_id":"q4X7h34c3ekMKFg7XlZzvoC1d4XT","target":"9bc42000","i
d":"608d069b-6f27-471e-9648-b3f6eae78088"}}
[Nov 1st 12:37] message from pool (FOGG): {"jsonrpc":"2.0","method":"job","params":{"blob":"0606b4aee4cf0588266e7c82e6a057568ce833a1775517cd7c6e43f1e87a5e3e3491ccd
862c12300000000c446ea69bf8a3fe96678b4abe52e44742300b77286f09c89db2d5d4f1709b7fe01","job_id":"8HUFcgr4u3kUhdkvS0IUtXDOp8qe","target":"9bc42000","id":"608d069b-6f27-
471e-9648-b3f6eae78088"}}
[Nov 1st 12:37] message sent to miner (608d069b-6f27-471e-9648-b3f6eae78088): {"type":"job","params":{"blob":"0606b4aee4cf0588266e7c82e6a057568ce833a1775517cd7c6e4
3f1e87a5e3e3491ccd862c12300000000c446ea69bf8a3fe96678b4abe52e44742300b77286f09c89db2d5d4f1709b7fe01","job_id":"8HUFcgr4u3kUhdkvS0IUtXDOp8qe","target":"9bc42000","i
d":"608d069b-6f27-471e-9648-b3f6eae78088"}}

.. period of no communication

[Nov 1st 12:44] message from miner (608d069b-6f27-471e-9648-b3f6eae78088) {"type":"submit","params":{"job_id":"8HUFcgr4u3kUhdkvS0IUtXDOp8qe","nonce":"fc8bdece","re
sult":"ef2a51d3608d4fb3a8da8304975ad89a717185c3de351287823990cf0eec1300"}}
[Nov 1st 12:44] message sent to pool (FOGG): {"id":143,"method":"submit","params":{"id":"608d069b-6f27-471e-9648-b3f6eae78088","job_id":"8HUFcgr4u3kUhdkvS0IUtXDOp8
qe","nonce":"fc8bdece","result":"ef2a51d3608d4fb3a8da8304975ad89a717185c3de351287823990cf0eec1300"}}
[Nov 1st 12:44] message from pool (FOGG): {"id":143,"jsonrpc":"2.0","error":{"code":-1,"message":"Unauthenticated"}}

[procloud]

or perhaps im just running circles here...

maybe the solution is a low diff mining pool that needs to be started collectively for this application... or at least some kind of proxy architecture to aggregate the low hashers so they dont timeout

[sunk818]

The duplicate blob is expected. Pool sends to proxy. Proxy sends to miner reformatted. Vice versa.

Aggregate low hashers

It takes time and money to run a pool. You can make one yourself if you think there is a need. No point really because it's not worth anyone's time to deal with low hash rate miners. Stop using your smart phone to mine!

[procloud]

there does appear to be a keep-alive method for pools

perl5577/cpuminer-multi@0c8aedb

[sunk818]

keep-alive method for pool

cryptonote-universal-pool does not support keep alive .

[sunk818]

https://github.com/clintar/cryptonote-universal-pool/blob/master/lib/pool.js
cryptonote-universal-pool supports keep alive

I'm not qualified enough to know by reviewing the code, but xmrig said keep alive is not supported. He writers the xmrig miner software, so I assume he really knows.

[cazala]

mm so yea the pool does support it: https://github.com/clintar/cryptonote-universal-pool/blob/master/lib/pool.js#L1245-L1249

and it looks like the timeout is 30 seconds: https://github.com/clintar/cryptonote-universal-pool/blob/master/lib/pool.js#L276-L299

so I should send a keepalived message every, I dunno, 25 seconds at least to make sure miners are kept online?

EDIT: 30 seconds is not the timeout, that's the interval on which timed out miners are deleted. It looks like the default timeout is 20 minutes: https://github.com/clintar/cryptonote-universal-pool/blob/master/config.json#L25

I ll add this later and push a new version, hopefully that will help

EDIT 2: it looks like 30 seconds is the right interval for the heartbeat: https://github.com/Snipa22/xmr-node-proxy/blob/master/proxy.js#L672

[procloud]

i would decide on a default, but also make it a switch, so i can play with it here in my dev env.

i would also consider adding a ping method to the coinhive client at the same time, perhaps making that disabled by default with a switch, and i can test that as well.

it does look like on my side, when grepping the logs, low hash clients time out on ~20 mins intervals.

[sunk818]

Hopefully keep alive from proxy will be enough to prevent unauthenticated error on slow miners. I thought @cazala already implemented this. The implementation is for TCP and now you're working on the application layer?

Curious why xmr miner author would say keep alive is not supported?

[procloud]

if the app layer keep alive does not work, the only other option is to set a default timeout, where job timestamps can be compared, if the miner has not been sent a new job within say 3minutes, it destroys the connection so it can reauth. this would at least cut the idle miners down to 3 minutes vs as much as an hour that i have seen in my tests.

[sunk818]

I think you're confusing some terminology in your last post. Miner only submits hashes of acceptable difficulty. The pool will either accept or reject the hash.

[procloud]

has development stopped on this project?

[procloud]

great news! i look forward to rolling it out