iamgroute/nptpy

nptpy's Issues

CI: add Python 3.6

According to https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json, Python 3.6 requires Ubuntu 20.04 not 22.04 which is currently ubuntu-latest.
Maybe specify matrix for distros?

Short looping after crash

If an exception occurs (at least at the stage of authenticating a Link portal-side), and is not handled before it reaches portal.main.py, main() short loops after recovery (after time.sleep(10)).
Perhaps there is a reference cycle that prevents gc from closing some sockets ?

`requestCloseChannel()` and `CancelledError`

When a request is pending and link drops:

1604085298.3 [00000073] DataEndpoint:    Finishing        ()
1604085298.3 [00000000] Loop:            Enqueue          (8,)
1604085298.3 [00000000] Loop:            Running          (8, 'requestCloseChannel ControlEndpoint:129')
1604085298.3 [00000000] Loop:            NewFuture        ()
1604085298.3 [00000000] Loop:            Paused           (8, 'requestCloseChannel ControlEndpoint:136')
.
1604085298.4 [00000066] Connector:       Error            (ConnectionResetError(104, 'Connection reset by peer'),)
1604085298.403 [Portal.Link    ]    Connection lost, reason: Closed by other end
1604085298.404 [Portal.Link    ]    Reconnecting
1604085298.4 [00000000] Loop:            Enqueue          (10,)
1604085298.4 [00000000] Loop:            Running          (10, 'reconnect Link:163')
1604085298.4 [0000002A] Link:            Disconnect       ()
1604085298.4 [00000066] Connector:       [Log deleted]
1604085298.4 [00000000] Loop:            Enqueue          (8,)
1604085298.4 [00000073] DataEndpoint:    [Log deleted]
1604085298.4 [00000076] Connector:       [Log deleted]
1604085298.4 [0000002A] Link:            Connect          (None,)
1604085298.4 [00000000] Loop:            NewFuture        ()
1604085298.4 [00000000] Loop:            Paused           (10, 'reconnect Link:165, __call__ Event:26, _connect Link:62, _connect_p1 Link:74, requestRelay Portal:241')
1604085298.4 [00000000] Loop:            Running          (8, 'requestCloseChannel ControlEndpoint:136')
1604085298.4 [00000000] Loop:            RunError         (CancelledError(),)
.

'Endpoint' object has no attribute 'readable'

After some stress testing:

...................
1604087477.8 [00000031] Listener:        NewConnection    ()
1604087477.8 [00000000] Loop:            Enqueue          (22,)
1604087477.8 [00000000] Loop:            Running          (22, 'main Listener:49')
1604087477.8 [000000FB] Endpoint:        [Log created]
1604087477.8 [000000FB] Endpoint:        Inited           (7, -1)
1604087477.8 [00000000] Loop:            NewFuture        ()
1604087477.8 [00000000] Loop:            Paused           (22, 'main Listener:50, requestChannel Link:221, requestChannel Channels:100, requestNewChannel ControlEndpoint:67')
.
1604087477.9 [000000EA] Connector:       Error            (ConnectionResetError(104, 'Connection reset by peer'),)
1604087477.942 [Portal.Link    ]	Connection lost, reason: Closed by other end
1604087477.943 [Portal.Link    ]	Reconnecting
1604087477.9 [00000000] Loop:            Enqueue          (28,)
1604087477.9 [00000000] Loop:            Running          (28, 'reconnect Link:165')
1604087477.9 [0000002A] Link:            Disconnect       ()
1604087477.9 [00000000] Loop:            RunError         (AttributeError("'Endpoint' object has no attribute 'readable'"),)
..
1604087483.0 [00000000] Loop:            Enqueue          (23,)
1604087483.0 [00000000] Loop:            Running          (23, 'requestCloseChannel ControlEndpoint:136')
1604087483.0 [00000000] Loop:            RunError         (CancelledError(),)
.
1604087485.7 [00000000] Loop:            Enqueue          (26,)
1604087485.7 [00000000] Loop:            Running          (26, 'requestKA Portal:244')
1604087485.7 [00000000] Loop:            Finished         (26, None)
1604087485.8 [00000000] Loop:            Enqueue          (1,)
1604087485.8 [00000000] Loop:            Running          (1, 'rtask Portal:167, recvPacketAsync AsyncConnectorPacketized:30, bufferAsync AsyncConnectorPacketized:19, tryRecvAsync AsyncConnector:93, recvAsync AsyncConnector:84, wait AsyncConnector:16')
1604087485.8 [00000000] Loop:            NewFuture        ()
1604087485.8 [00000000] Loop:            Paused           (1, 'rtask Portal:167, recvPacketAsync AsyncConnectorPacketized:30, bufferAsync AsyncConnectorPacketized:19, tryRecvAsync AsyncConnector:93, recvAsync AsyncConnector:84, wait AsyncConnector:16')
..
1604087490.8 [00000000] Loop:            Enqueue          (13,)
1604087490.8 [00000000] Loop:            Running          (13, 'requestCloseChannel ControlEndpoint:136')
1604087490.8 [00000000] Loop:            RunError         (CancelledError(),)
.1604087495.781 [root]	'NoneType' object has no attribute 'socket'
Traceback (most recent call last):
  File "portal.main.py", line 38, in <module>
    p.main()
  File "/home/pi/git/NPTpy/NPTpy/Portal/Portal.py", line 56, in main
    Globals.runReminders()
  File "/home/pi/git/NPTpy/NPTpy/Globals.py", line 12, in runReminders
    kaReminderTX.run()
  File "/home/pi/git/NPTpy/NPTpy/Common/TimedReminder.py", line 13, in run
    Reminder.run(self)
  File "/home/pi/git/NPTpy/NPTpy/Common/Reminder.py", line 24, in run
    delegate.run()
  File "/home/pi/git/NPTpy/NPTpy/Common/Reminder.py", line 56, in run
    if f is not None: f(owner)
  File "/home/pi/git/NPTpy/NPTpy/Portal/Link.py", line 195, in handleRemindTX
    self.channels.sendKA()
  File "/home/pi/git/NPTpy/NPTpy/Portal/Channels.py", line 55, in sendKA
    self.epControl.sendKA()
  File "/home/pi/git/NPTpy/NPTpy/Portal/ControlEndpoint.py", line 54, in sendKA
    self.send(b'.', True)
  File "/home/pi/git/NPTpy/NPTpy/Portal/ControlEndpoint.py", line 23, in send
    self.parent.send(self.formMessage(data), untracked)
  File "/home/pi/git/NPTpy/NPTpy/Portal/Channels.py", line 52, in send
    self.myLink.send(data, untracked)
  File "/home/pi/git/NPTpy/NPTpy/Portal/Link.py", line 269, in send
    self.conRT.socket.settimeout(5)
AttributeError: 'NoneType' object has no attribute 'socket'

Caused by e89c4b7 ?

Link: attempts to do TLS handshake with socket timeout = 0 (non-blocking)

1566313813.999 [root]   do_handshake_on_connect should not be specified for non-blocking sockets
Traceback (most recent call last):
  File "NPTpy/NPTpy/Portal/Portal.py", line 53, in main
    s.task()
  File "NPTpy/NPTpy/Portal/Link.py", line 159, in task
    elif self.state == self.States.WaitReady:    self.taskReady()
  File "NPTpy/NPTpy/Portal/Link.py", line 170, in taskReady
    if   data == b'Ready !\n': self.secureForward()
  File "NPTpy/NPTpy/Portal/Link.py", line 84, in secureForward
    self.conRT.secureServer(certFilename='portal.cer', keyFilename='portal.key')
  File "NPTpy/NPTpy/Common/Connector.py", line 176, in secureServer
    self.socket = self.sslContext.wrap_socket(self.socket, server_side=True)
  File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/usr/lib/python3.6/ssl.py", line 816, in __init__
    raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
ValueError: do_handshake_on_connect should not be specified for non-blocking sockets

TCP keep-alive is not reliable for link status detection

In some cases it takes way too much time (> 2 minutes) to detect a loss of communication.
It is also possible for the detection to be missed entirely if other data is sent (but not received) within this period, as the keep-alive counters are reset.

RuntimeError: coroutine reuse when terminated

Exception ignored in: <function Promise.__del__ at 0x7616d738>
Traceback (most recent call last):
  File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 19, in __del__
  File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 57, in fire
  File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 55, in fire
  File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 119, in <lambda>
  File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 126, in _resolve
  File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 33, in then
  File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 29, in attach
  File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 55, in fire
  File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 126, in <lambda>
  File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 132, in _cont
  File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 141, in run
RuntimeError: cannot reuse already awaited coroutine

Support for asymmetric channel closing

TCP allows for one endpoint to send FIN and terminate the communication in one direction, while still allowing for data to be sent the other direction. Some applications use this to signal EOF.
We could implement this by calling shutdown(read=False, write=True) when the control endpoint receives a channel deletion request.
However, we need to avoid causing a short-loop on the portal next to the device that sends the first FIN.

Support for permanent links maintained by both ends

By separating the 'client' and 'permanent' conditions, it's possible to have links that are configured and maintained (re-established after disconnect) on both ends.

DataEndpoint's `sendall` can block indefinitely

NPTpy/NPTpy/Portal/DataEndpoint.py

Lines 26 to 32 in 68f068f

	def acceptMessage(self, data):
	try:
	self.con.socket.settimeout(None)
	self.con.sendall(data)
	self.con.socket.settimeout(0)
	except OSError:
	self.remove()

Setting to blocking mode to send was temporary and has to be remove.
If the device (to which the data is being sent) stops receiving, send will block indefinitely.
To remove the blocking send we need per-channel flow control.

Non-selectable passed to select()

Trace:

... (many keepalives)
1568793504.496 [Portal.Link    ]        Sent                  5   Bytes
1568793504.497 [Portal.Link    ]            content: 0100000000
1568793504.498 [Portal.Link    ]        Sent                  5   Bytes
1568793504.499 [Portal.Link    ]            content: 0100000000
1568793511.769 [Portal.Link    ]        Received              5   Bytes
1568793511.770 [Portal.Link    ]            content: 0100000000
1568793513.801 [Portal.Link    ]        Received              5   Bytes
1568793513.801 [Portal.Link    ]            content: 0100000000
1568793514.887 [Portal.Link    ]        Sent                  5   Bytes
1568793514.888 [Portal.Link    ]            content: 0100000000
1568793514.889 [Portal.Link    ]        Sent                  5   Bytes
1568793514.890 [Portal.Link    ]            content: 0100000000
1568793525.391 [Portal.Link    ]        Sent                  5   Bytes
1568793525.392 [Portal.Link    ]            content: 0100000000
1568793525.393 [Portal.Link    ]        Sent                  5   Bytes
1568793525.394 [Portal.Link    ]            content: 0100000000
1568793535.892 [Portal.Link    ]        Sent                  5   Bytes
1568793535.893 [Portal.Link    ]            content: 0100000000
1568793535.895 [Portal.Link    ]        Sent                  5   Bytes
1568793535.895 [Portal.Link    ]            content: 0100000000
1568793546.356 [Portal.Link    ]        Sent                  5   Bytes
1568793546.357 [Portal.Link    ]            content: 0100000000
1568793546.358 [Portal.Link    ]        Sent                  5   Bytes
1568793546.359 [Portal.Link    ]            content: 0100000000
1568793556.831 [Portal.Link    ]        Sent                  5   Bytes
1568793556.832 [Portal.Link    ]            content: 0100000000
1568793556.833 [Portal.Link    ]        Sent                  5   Bytes
1568793556.834 [Portal.Link    ]            content: 0100000000
1568793561.024 [Portal.Link    ]        RX keepalive timeout
1568793561.028 [Portal.Link    ]            reconnecting
1568793561.028 [Portal.Link    ]        Stopping
1568793561.030 [Portal.Link    ]        Stopped
1568793561.030 [Portal.Link    ]        Stopping
1568793561.031 [Portal.Link    ]        Stopped
1568793561.035 [Portal.Portal  ]        Sent                 64   Bytes
1568793561.036 [Portal.Portal  ]            content: 01000000765432103030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030
1568793561.036 [Portal.Link    ]        RX keepalive timeout
1568793561.039 [Portal.Link    ]            disconnecting
1568793561.040 [Portal.Link    ]        Stopping
1568793561.041 [Portal.Link    ]        Stopped
1568793561.042 [Portal.Link    ]        Stopping
1568793561.042 [Portal.Link    ]        Stopped
1568793562.391 [Portal.Portal  ]        Received             39   Bytes
1568793562.391 [Portal.Portal  ]            content: 76302E316F70696F206C81D767869F38BB0172302E736572766572732E6E6574706F72742E696F
1568793562.396 [Portal.Link    ]        Starting
1568793562.397 [Portal.Link    ]        Started on         [0.0.0.0]:40361
1568793562.397 [Portal.Link    ]        Connecting to      [r0.servers.netport.io]:443
1568793562.456 [Portal.Link    ]            connected
1568793562.457 [Portal.Link    ]        Sent                 64   Bytes
1568793562.458 [Portal.Link    ]            content: 206C81D767869F383030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030
1568793562.501 [root]   argument must be an int, or have a fileno() method.
Traceback (most recent call last):
  File "portal.main.py", line 31, in <module>
    p.main()
  File "/NPTpy/NPTpy/Portal/Portal.py", line 69, in main
    readables, _, _         = select.select(rlist, [], [], 1) # Temporary
TypeError: argument must be an int, or have a fileno() method.