cn-uofbasel/picn

PiCN: Python ICN and NFN by University of Basel

picn's Issues

decoding of name crashes with non-ascii characters

def components_to_string in Name.py uses decode(ascii).
Decoding non-ascii chars leads to a UnicodeDecodeError which isn't caught.

Failed Test: Test a simple forwarding scenario with one additional node forwarding the data

Occurs rarely:

======================================================================
1) FAIL: Test a simple forwarding scenario with one additional node forwarding the data
----------------------------------------------------------------------
   Traceback (most recent call last):
    /home/claudio/software/PiCN/PiCN/ProgramLibs/NFNForwarder/test/test_NFNForwarder.py line 164 in test_NFNForwarder_simple_compute_two_nodes
      self.assertEqual(len(self.forwarder1.pit.container), 0)
   AssertionError: 1 != 0

Management command 'shutdown' causes exception

Sending 'shutdown' is not handled properly (sent with Mgmt.py):

2018-01-31 16:36:17,998 - MgmtSys 	 INFO: 	 Shutdown
Process Process-6:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/claudio/software/PiCN/PiCN/Mgmt/Mgmt.py", line 157, in _run
    self.mgmt(mgmt_sock)
  File "/home/claudio/software/PiCN/PiCN/Mgmt/Mgmt.py", line 79, in mgmt
    self.shutdown()
  File "/home/claudio/software/PiCN/PiCN/ProgramLibs/ICNForwarder/ICNForwarder.py", line 86, in stop_forwarder
    self.mgmt.stop_process()
  File "/home/claudio/software/PiCN/PiCN/Mgmt/Mgmt.py", line 165, in stop_process
    self.process.terminate()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 116, in terminate
    self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'

6 Unit Tests in the BasicPacketEncodingLayer rely on Simple Packet format

6 Unit Tests in the BasicPacketEncodingLayer rely on Simple Packet Format.

Three Options:

• Skip them
• Add specific tests for the packetformat (i think better use encoder tests in this case)
• Remove those tests or Remove NDNTLV tests from BasicPacketEncodingLayer.

Software Tests: "Address already in use"

Occasionally, a software test fails due to OS Error "Address already in use":

======================================================================
1) ERROR: Test adding content
----------------------------------------------------------------------
   Traceback (most recent call last):
    /home/claudio/software/PiCN/PiCN/Mgmt/test/test_Mgmt.py line 24 in setUp
      self.linklayer = UDP4LinkLayer(9000+self.portoffset)
    /home/claudio/software/PiCN/PiCN/Layers/LinkLayer/UDP4LinkLayer.py line 25 in __init__
      self.sock.bind(("0.0.0.0", self._port))
   OSError: [Errno 98] Address already in use

Unable to run first example : syntax error

Hello guys,

I'm new in python and using PiCN too.
I follow the steps in your guthub to test the PiCN example but I have an error that I don't understand.
While running this command picn-repo --format ndntlv /tmp/repo /the/prefix 10000 &
I have the following error : Traceback (most recent call last): File "/home/ubuntu/PiCN/PiCN/Executable/ICNDataRepository.py", line 6, in <module> from PiCN.ProgramLibs.ICNDataRepository import ICNDataRepository File "/home/ubuntu/PiCN/PiCN/ProgramLibs/ICNDataRepository/__init__.py", line 3, in <module> from .ICNDataRepository import ICNDataRepository File "/home/ubuntu/PiCN/PiCN/ProgramLibs/ICNDataRepository/ICNDataRepository.py", line 59 self.repo: BaseRepository = SimpleMemoryRepository(prefix, manager, logger) ^ SyntaxError: invalid syntax
I use python version 3.7 in ubuntu 16.04.

Your help will be appreciated.

invalid escape sequence

Inside the NFN Parser, the Escapes seem to be deprecated

Remove NoseTests, use buildin Unit Tests

Seems, tests must be in a separate Tree to switch to python-buildin tests.

Remove @unittest.skip("No Nack Crafting for NDNTLV yet")

Remove @unittest.skip("No Nack Crafting for NDNTLV yet") from Tests as soon as NACK Crafting is available for NDNTLV.

This temporally disables tests for for NDNTLV where a nack is required.

Tests for the encoders?

Do we need tests for the packet encoder or do we assume, they are working correctly?

NACK reason "No PIT Entry found"

NACK reason "No PIT Entry found" looks wrong and should be reviewed. Seen twice:

Layers/NFNLayer/test/test_BasicNFNLayer.py:242:

nack = Nack(name, reason="No PIT Entry found", interest=interest)

Layers/NFNLayer/test/test_BasicNFNLayer.py:267:

nack = Nack(name, reason="No PIT Entry found", interest=interest)

Tests: "dictionary changed size during iteration"

It occurs very rarely that a test fails with to the following output.
Looks like a race condition?

> python3 setup.py nosetests
running nosetests
running egg_info
writing PiCN.egg-info/PKG-INFO
writing dependency_links to PiCN.egg-info/dependency_links.txt
writing top-level names to PiCN.egg-info/top_level.txt
reading manifest file 'PiCN.egg-info/SOURCES.txt'
writing manifest file 'PiCN.egg-info/SOURCES.txt'
Searching for nose2
Best match: nose2 0.7.3
Processing nose2-0.7.3-py3.6.egg

Using /home/claudio/software/PiCN/.eggs/nose2-0.7.3-py3.6.egg
Searching for coverage>=4.4.1
Best match: coverage 4.4.2
Processing coverage-4.4.2-py3.6-linux-x86_64.egg

Using /home/claudio/software/PiCN/.eggs/coverage-4.4.2-py3.6-linux-x86_64.egg
.............................................................................
.............................................................................
...........................Exception in thread Thread-88:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/claudio/software/PiCN/PiCN/Layers/NFNLayer/BasicNFNLayer.py", line 172, in handle_computation_queue
    for cid in running_computations.keys():
RuntimeError: dictionary changed size during iteration

PIT entry update crash

I encountered the following stacktraces (on separate occasions), crashing the BasicICNLayer process:

Process Process-98:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File ".../picn/PiCN/Processes/LayerProcess.py", line 144, in _run
    self._run_poll(from_lower, from_higher, to_lower, to_higher)
  File ".../picn/PiCN/Processes/LayerProcess.py", line 85, in _run_poll
    self.data_from_lower(to_lower, to_higher, from_lower.get())
  File ".../picn/PiCN/Layers/ICNLayer/BasicICNLayer.py", line 83, in data_from_lower
    self.handle_interest(face_id, packet, to_lower, to_higher, False)
  File ".../picn/PiCN/Layers/ICNLayer/BasicICNLayer.py", line 103, in handle_interest
    self.update_timestamp_in_pit(pit_entry)
  File ".../picn/PiCN/Layers/ICNLayer/BasicICNLayer.py", line 282, in update_timestamp_in_pit
    pit.update_timestamp(pit_entry)
  File ".../picn/PiCN/Layers/ICNLayer/PendingInterestTable/PendingInterestTableMemoryExact.py", line 40, in update_timestamp
    self._container.remove(pit_entry)
ValueError: list.remove(x): x not in list

Process Process-58:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File ".../picn/PiCN/Processes/LayerProcess.py", line 144, in _run
    self._run_poll(from_lower, from_higher, to_lower, to_higher)
  File ".../picn/PiCN/Processes/LayerProcess.py", line 85, in _run_poll
    self.data_from_lower(to_lower, to_higher, from_lower.get())
  File ".../picn/PiCN/Layers/ICNLayer/BasicICNLayer.py", line 83, in data_from_lower
    self.handle_interest(face_id, packet, to_lower, to_higher, False)
  File ".../picn/PiCN/Layers/ICNLayer/BasicICNLayer.py", line 115, in handle_interest
    self.add_used_fib_entry_to_pit(interest.name, new_face_id)
  File ".../picn/PiCN/Layers/ICNLayer/BasicICNLayer.py", line 266, in add_used_fib_entry_to_pit
    pit.add_used_fib_entry(name, used_fib_entry)
  File ".../picn/PiCN/Layers/ICNLayer/PendingInterestTable/PendingInterestTableMemoryExact.py", line 47, in add_used_fib_entry
    self._container.remove(pit_entry)
ValueError: list.remove(x): x not in list

Cache Content from Higher even if no PIT entry?

Content from local may be meaningful even if there is no PIT entry. Cache it?

Routing Solution

No routing solution was found in the code.

ContentStoreMemoryPrefix is not Iterable....

.... this is in one or the other way required for ageing!

Peek Tool / NdnTlvPrinter: Escaping special characters

Enhance escaping of special characters for ASCII interpretation of blobs (e.g. newline, backspace)
Only a problem in some terminals (e.g urxvt) but not in others (pycharm built-in).

Format "simple" causes logging issues

claudio@notebook ~> picn-relay --format simple
2018-12-19 11:42:44,328 - ICNForwarder 	 INFO: 	 Starting a CCN Forwarder...
2018-12-19 11:42:44,329 - ICNForwarder 	 INFO: 	 UDP Port:       9000
2018-12-19 11:42:44,329 - ICNForwarder 	 INFO: 	 Log Level:      info
2018-12-19 11:42:44,329 - ICNForwarder 	 INFO: 	 Packet Format:  simple
Traceback (most recent call last):
  File "/home/claudio/software/PiCN/PiCN/Executable/ICNForwarder.py", line 89, in <module>
    main(args)
  File "/home/claudio/software/PiCN/PiCN/Executable/ICNForwarder.py", line 76, in main
    forwarder = PiCN.ProgramLibs.ICNForwarder.ICNForwarder(args.port, log_level, encoder, autoconfig=args.autoconfig)
  File "/home/claudio/software/PiCN/PiCN/ProgramLibs/ICNForwarder/ICNForwarder.py", line 42, in __init__
    encoder.set_log_level(log_level)
TypeError: set_log_level() missing 1 required positional argument: 'log_level'

Works fine with "ndntlv" packet format:

claudio@notebook ~> picn-relay --format ndntlv
2018-12-19 11:44:58,652 - ICNForwarder 	 INFO: 	 Starting a CCN Forwarder...
2018-12-19 11:44:58,652 - ICNForwarder 	 INFO: 	 UDP Port:       9000
2018-12-19 11:44:58,652 - ICNForwarder 	 INFO: 	 Log Level:      info
2018-12-19 11:44:58,652 - ICNForwarder 	 INFO: 	 Packet Format:  ndntlv

picn-setup enhancements

Potential Enhancements for picn-setup:

• Write output to files (as complement or replacement for tmux output)
• Read network configurations from external (user-defined) bash scripts
• DSL to define network configurations
• ... needs?