snstac/aiscot

AIS to Cursor-on-Target (CoT) Gateway

aiscot's Issues

ERROR:asyncio:Task exception was never retrieved

INFO - Complete: <Task finished name='Task-3' coro=<RXWorker.run() done, defined at /home/x/.local/lib/python3.9/site-packages/pytak/classes.py:226> exception=DecodeError('Error parsing message')>
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-3' coro=<RXWorker.run() done, defined at /home/x/.local/lib/python3.9/site-packages/pytak/classes.py:226> exception=DecodeError('Error parsing message')>
Traceback (most recent call last):
File "/home/x/.local/lib/python3.9/site-packages/pytak/classes.py", line 233, in run
data: bytes = await self.readcot()
File "/home/x/.local/lib/python3.9/site-packages/pytak/classes.py", line 219, in readcot
tak_v1 = takproto.parse_proto(cot)
File "/usr/local/lib/python3.9/dist-packages/takproto/functions.py", line 50, in parse_proto
parsed = parse_mesh(msg)
File "/usr/local/lib/python3.9/dist-packages/takproto/functions.py", line 60, in parse_mesh
protobuf.ParseFromString(bytes(msg))
google.protobuf.message.DecodeError: Error parsing message

running on current RaspbianOS on RaspberryPi3b+

aiscot -c /usr/local/bin/config.ini
2024-01-25 14:43:22,184 pytak INFO - COT_URL: tcp://192.168.1.245:8087
2024-01-25 14:43:22,185 aiscot INFO - Run: <class 'pytak.classes.CLITool'>
2024-01-25 14:43:22,189 aiscot INFO - Run: <class 'pytak.classes.TXWorker'>
2024-01-25 14:43:22,189 aiscot INFO - Sending to: tcp://192.168.1.245:8087
2024-01-25 14:43:22,190 aiscot INFO - Listening for AIS on 0.0.0.0:5050
2024-01-25 14:43:22,191 aiscot INFO - Run: <class 'pytak.classes.RXWorker'>
2024-01-25 14:43:22,191 aiscot INFO - Connection from peer (no peername available).

Create rtfd.io site aiscot

Need some docs.

Running on Windows fails with `ValueError: can not get address information`

PS C:\Users\gba> C:\Users\gba\AppData\Roaming\Python\Python39\Scripts\aiscot.exe
c:\python39\lib\site-packages\pytak\crypto_functions.py:40: UserWarning: Python cryptography module not installed. Install with:  python3 -m pip install cryptography
  warnings.warn(INSTALL_MSG)
Traceback (most recent call last):
  File "c:\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\gba\AppData\Roaming\Python\Python39\Scripts\aiscot.exe\__main__.py", line 7, in <module>
  File "C:\Users\gba\AppData\Roaming\Python\Python39\site-packages\aiscot\commands.py", line 28, in main
    pytak.cli(__name__.split('.', maxsplit=1)[0])
  File "c:\python39\lib\site-packages\pytak\client_functions.py", line 438, in cli
    asyncio.run(main(app_name, config, full_config), debug=debug)
  File "c:\python39\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "c:\python39\lib\asyncio\base_events.py", line 642, in run_until_complete
    return future.result()
  File "c:\python39\lib\site-packages\pytak\client_functions.py", line 324, in main
    await clitool.create_workers(config)
  File "c:\python39\lib\site-packages\pytak\classes.py", line 320, in create_workers
    reader, writer = await pytak.protocol_factory(i_config)
  File "c:\python39\lib\site-packages\pytak\client_functions.py", line 262, in protocol_factory
    reader, writer = await pytak.create_udp_client(cot_url, iface, local_addr)
  File "c:\python39\lib\site-packages\pytak\client_functions.py", line 96, in create_udp_client
    writer: DatagramClient = await dgconnect((host, port), local_addr=local_addr)
  File "c:\python39\lib\site-packages\pytak\asyncio_dgram\aio.py", line 270, in connect
    transport, protocol = await loop.create_datagram_endpoint(
  File "c:\python39\lib\asyncio\base_events.py", line 1315, in create_datagram_endpoint
    raise ValueError('can not get address information')
ValueError: can not get address information

AttributeError: 'NoneType' object has no attribute 'replace'

 aiscot_aishub    File "/usr/lib/python3/dist-packages/pytak/client_functions.py", line 264, in main
 aiscot_aishub      await clitool.run()
 aiscot_aishub    File "/usr/lib/python3/dist-packages/pytak/classes.py", line 275, in run
 aiscot_aishub      self.run_tasks()
 aiscot_aishub    File "/usr/lib/python3/dist-packages/pytak/classes.py", line 268, in run_tasks
 aiscot_aishub      self.run_task(task)
 aiscot_aishub    File "/usr/lib/python3/dist-packages/pytak/classes.py", line 262, in run_task
 aiscot_aishub      self.running_tasks.add(asyncio.ensure_future(task.run()))
 aiscot_aishub  Traceback (most recent call last):
 aiscot_aishub    File "/usr/lib/python3/dist-packages/aiscot/classes.py", line 204, in run
 aiscot_aishub      await self.get_aishub_feed(session, aishub_url)
 aiscot_aishub    File "/usr/lib/python3/dist-packages/aiscot/classes.py", line 182, in get_aishub_feed
 aiscot_aishub      await self.handle_data(ships)
 aiscot_aishub    File "/usr/lib/python3/dist-packages/aiscot/classes.py", line 164, in handle_data
 aiscot_aishub      event: str = aiscot.ais_to_cot(msg, self.config, known_craft=known_craft)
 aiscot_aishub    File "/usr/lib/python3/dist-packages/aiscot/functions.py", line 219, in ais_to_cot
 aiscot_aishub      cot: ET.ElementTree = ais_to_cot_xml(craft, config, known_craft)
 aiscot_aishub    File "/usr/lib/python3/dist-packages/aiscot/functions.py", line 101, in ais_to_cot_xml
 aiscot_aishub      ais_name: str = craft.get("name", craft.get("NAME", "")).replace("@", "").strip()
 aiscot_aishub  AttributeError: 'NoneType' object has no attribute 'replace'

aiscot on Windows cannot bind to port 5050/udp

Starting aiscot on Windows using the default UDP listen port of 5050 will return the error:

OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted

This is because svchost.exe is listening to 5050/udp (WHY?!):

PS C:\Users\gba> netstat -ano|findstr 5050
  UDP    0.0.0.0:5050           *:*                                    7208

And this process (7208) cannot be killed :(

PS C:\Users\gba> ps|findstr 7208
    383      23     5188      18736              7208   0 svchost
PS C:\Users\gba> taskkill /F /PID 7208
ERROR: The process with PID 7208 could not be terminated.
Reason: Access is denied.

pip install fails on Windows

PS C:\Users\gba> pip install aiscot
Collecting aiscot
  Downloading aiscot-5.2.1-py3-none-any.whl (773 kB)
     ---------------------------------------- 773.3/773.3 kB 16.2 MB/s eta 0:00:00
Collecting pytak>=5.4.0 (from aiscot)
  Obtaining dependency information for pytak>=5.4.0 from https://files.pythonhosted.org/packages/1d/4f/afbcd684d21e4a8bb244a0d810bcb73dbb4eb9806633bbd4ecae947a423b/pytak-6.0.0-py3-none-any.whl.metadata
  Downloading pytak-6.0.0-py3-none-any.whl.metadata (3.4 kB)
Collecting aiohttp (from aiscot)
  Obtaining dependency information for aiohttp from https://files.pythonhosted.org/packages/c2/7c/66aff492b444f0d089bd74ffcb7346ebc3521ba68c77ac5479a2b947091c/aiohttp-3.8.5-cp39-cp39-win_amd64.whl.metadata
  Downloading aiohttp-3.8.5-cp39-cp39-win_amd64.whl.metadata (8.0 kB)
Collecting attrs>=17.3.0 (from aiohttp->aiscot)
  Downloading attrs-23.1.0-py3-none-any.whl (61 kB)
     ---------------------------------------- 61.2/61.2 kB 3.4 MB/s eta 0:00:00
Requirement already satisfied: charset-normalizer<4.0,>=2.0 in c:\python39\lib\site-packages (from aiohttp->aiscot) (3.2.0)
Collecting multidict<7.0,>=4.5 (from aiohttp->aiscot)
  Downloading multidict-6.0.4-cp39-cp39-win_amd64.whl (28 kB)
Collecting async-timeout<5.0,>=4.0.0a3 (from aiohttp->aiscot)
  Obtaining dependency information for async-timeout<5.0,>=4.0.0a3 from https://files.pythonhosted.org/packages/a7/fa/e01228c2938de91d47b307831c62ab9e4001e747789d0b05baf779a6488c/async_timeout-4.0.3-py3-none-any.whl.metadata
  Downloading async_timeout-4.0.3-py3-none-any.whl.metadata (4.2 kB)
Collecting yarl<2.0,>=1.0 (from aiohttp->aiscot)
  Downloading yarl-1.9.2-cp39-cp39-win_amd64.whl (61 kB)
     ---------------------------------------- 61.7/61.7 kB ? eta 0:00:00
Collecting frozenlist>=1.1.1 (from aiohttp->aiscot)
  Obtaining dependency information for frozenlist>=1.1.1 from https://files.pythonhosted.org/packages/f9/16/ef36f5b20ee10dba86d4b5223d55b416e97dfa2dbf5546f0c6d9aa8a26ba/frozenlist-1.4.0-cp39-cp39-win_amd64.whl.metadata
  Downloading frozenlist-1.4.0-cp39-cp39-win_amd64.whl.metadata (5.3 kB)
Collecting aiosignal>=1.1.2 (from aiohttp->aiscot)
  Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Requirement already satisfied: idna>=2.0 in c:\python39\lib\site-packages (from yarl<2.0,>=1.0->aiohttp->aiscot) (3.4)
Downloading pytak-6.0.0-py3-none-any.whl (22 kB)
Downloading aiohttp-3.8.5-cp39-cp39-win_amd64.whl (327 kB)
   ---------------------------------------- 327.1/327.1 kB 21.1 MB/s eta 0:00:00
Downloading async_timeout-4.0.3-py3-none-any.whl (5.7 kB)
Downloading frozenlist-1.4.0-cp39-cp39-win_amd64.whl (44 kB)
   ---------------------------------------- 44.7/44.7 kB ? eta 0:00:00
Installing collected packages: pytak, multidict, frozenlist, attrs, async-timeout, yarl, aiosignal, aiohttp, aiscot
  WARNING: Failed to write executable - trying to use .deleteme logic
ERROR: Could not install packages due to an OSError: [WinError 2] The system cannot find the file specified: 'c:\\python39\\Scripts\\aiscot.exe' -> 'c:\\python39\\Scripts\\aiscot.exe.deleteme'

Filter config

Could you provide an example of a filter config?
I'm looking for a way to filter based on ship type, if possible only keep vessels that belong to two types of ships

ISO-8859 CSV 'known craft' Files are not supported

[admin@takserver ~]$ file *.csv
Airports.csv:    ASCII text, with CRLF line terminators
known_craft.csv: ISO-8859 text, with CRLF line terminators
known_mil.csv:   UTF-8 Unicode (with BOM) text, with CRLF line terminators

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/adsbxcot/classes.py", line 165, in run
    self.known_craft_db = aircot.read_known_craft(self.known_craft)
  File "/usr/local/lib/python3.6/site-packages/aircot/functions.py", line 297, in read_known_craft
    for row in reader:
  File "/usr/lib64/python3.6/csv.py", line 112, in __next__
    row = next(self.reader)
  File "/usr/lib64/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 1816: invalid start byte