opennetworkinglab/fabric-p4test

PTF-based data plane tests for ONOS fabric.p4

fabric-p4test's Issues

Remove use of p4config_pb2 in ptf_runner.py

Pull Request Deprecate temporary P4DeviceConfig message in PI removed the requirement on message P4DeviceConfig.

This change was brought to Stratum through PR #194. Removing this dependency facilitates running PTF tests with Stratum.

Reference to p4config_pb2 should be removed from ptf_runner.py since it is no longer needed with recent versions of PI.

Running PTF tests on Tofino

The current readme instructions to run PTF tests on Tofino say:

sudo ./ptf_runner.py --device tofino \ 
    --p4info p4info.txt \
    --tofino-bin tofino.bin \
    --tofino-ctx-json context.json \
    --ptf-dir fabric.ptf --port-map port_map.veth.json \
    all ^spgw

However, I don't think this can realistically work. Updating the P4 program on Tofino will remove the dataplane ports. Maybe we need a --skip-tests flag so that the runner can be invoked twice:

./ptf_runner.py --skip-tests ....`
// add ports
./ptf_runner.py --skip-config ...`

or some other solution...

Maybe I'm missing something @ccascone? How do you run the tests?

FabricIntSourceTest fails on onos-1.14

FAIL: test.FabricIntSourceTest
----------------------------------------------------------------------
Traceback (most recent call last):
 File “fabric.ptf/test.py”, line 469, in runTest
   tagged1=tagged[0], tagged2=tagged[1])
 File “/home/sdn/fabric-p4test/tests/ptf/base_test.py”, line 665, in handle
   return f(*args, **kwargs)
 File “fabric.ptf/test.py”, line 452, in doRunTest
   self.runIntSourceTest(**kwargs)
 File “/home/sdn/fabric-p4test/tests/ptf/fabric_test.py”, line 765, in runIntSourceTest
   tagged1=tagged1, tagged2=tagged2)
 File “/home/sdn/fabric-p4test/tests/ptf/fabric_test.py”, line 483, in runIPv4UnicastTest
   testutils.verify_each_packet_on_each_port(self, exp_pkts, exp_ports)
 File “/usr/local/lib/python2.7/dist-packages/ptf-0.9.1-py2.7.egg/ptf/testutils.py”, line 2455, in verify_each_packet_on_each_port
   % (port, device_number, result.format()))
AssertionError: Did not receive expected packets on port 2 for device 0.

Perform PTF tests for BMv2 on latest ONOS master via Travis CI

This might require using Docker support in Travis since Ubuntu 16.04 is not offered as an environment target. At a high level we need a script that checks out ONOS master, installs all dependencies (using ONOS p4vm scripts https://github.com/opennetworkinglab/onos/tree/master/tools/dev/p4vm), runs PTF tests on BMv2. This Travis build should be executed for each PR, as well periodically (e.g. every day) to make sure changes to fabric.p4 in ONOS are passing tests.

Consider using p4runtime-shell for P4Runtime operations in PTF tests

We currently use a custom library to read/write P4Runtime entities such as table entries. We might consider dropping that in favor of p4runtime-shell, as explained here:
https://github.com/p4lang/p4runtime-shell#using-p4runtime-shell-in-scripts

Rename parameter --external-hercules-urls for --external-stratum-urls for Stratum usage.

Parameter --external-hercules-urls was recently renamed for --external-stratum-urls in the Stratum code.

Function get_stratum_cmd should be updated accordingly.

Tests are broken because stratum_bmv2 crashes when processing packets via action profiles

The issue can be reproduced by running ./docker_run.sh bug from commit d4b8f1f in the fix-travis branch.

stratum_bmv2.log
write-reqs.log

From stratum_bmv2.log:

[21:07:10.163] [bmv2] [D] [thread 83] [52.0] [cxt 0] Looking up key:
* next_id             : 00000064

[21:07:10.163] [bmv2] [D] [thread 83] [52.0] [cxt 0] Choosing member 1 from group 0
stratum_bmv2: action_profile.cpp:182: const bm::ActionEntry& bm::ActionProfile::lookup(const bm::Packet&, const bm::ActionProfile::IndirectIndex&) const: Assertion `is_valid_mbr(mbr)' failed.
*** Aborted at 1590095230 (unix time) try "date -d @1590095230" if you are using GNU date ***
PC: @                0x0 (unknown)
*** SIGABRT (@0x4f) received by PID 79 (TID 0x7f3ce7f10700) from PID 79; stack trace: ***
    @     0x7f3ceb0c20e0 (unknown)
    @     0x7f3ceab2dfff gsignal
    @     0x7f3ceab2f42a abort
    @     0x7f3ceab26e67 (unknown)
    @     0x7f3ceab26f12 __assert_fail
    @     0x7f3cebb9ff4f bm::ActionProfile::lookup()
    @     0x7f3cebc397c4 bm::MatchTableIndirect::lookup()
    @     0x7f3cebc3ac93 bm::MatchTableIndirectWS::lookup()
    @     0x7f3cebc36b5c bm::MatchTableAbstract::apply_action()
    @     0x7f3cebd0ed7e bm::MatchActionTable::operator()()
    @     0x7f3cebce0b77 bm::Pipeline::apply()
    @     0x7f3cebb047f4 SimpleSwitch::ingress_thread()
    @     0x7f3cebb213ae std::__invoke_impl<>()
    @     0x7f3cebb212b3 std::__invoke<>()
    @     0x7f3cebb211f0 _ZNKSt12_Mem_fn_baseIM12SimpleSwitchFvvELb1EEclIJPS0_EEEDTcl8__invokedtdefpT6_M_pmfspcl7forwardIT_Efp_EEEDpOS6_
    @     0x7f3cebb2116f _ZNSt12_Bind_simpleIFSt7_Mem_fnIM12SimpleSwitchFvvEEPS1_EE9_M_invokeIJLm0EEEEvSt12_Index_tupleIJXspT_EEE
    @     0x7f3cebb20ef3 std::_Bind_simple<>::operator()()
    @     0x7f3cebb20dc0 std::thread::_State_impl<>::_M_run()
    @     0x7f3cec494e6f (unknown)
    @     0x7f3ceb0b84a4 start_thread
    @     0x7f3ceabe3d0f clone
    @                0x0 (unknown)