bisdn/rofl-common

OpenFlow protocol endpoint written in C++

What it is

The Revised OpenFlow Library (ROFL) helps you adding OpenFlow support to your software to build control applications, controller frameworks and/or datapath elements.

ROFL-common is a C++ library that helps on writing OpenFlow endpoints (OF agents), for controllers, proxy controllers and datapaths.

Requirements

• A modern GNU build-system (autoconf, automake, libtool, ...)
• pkg-config
• GNU/Linux and libc development headers (linux-libc-dev and libc6-dev package in Debian/Ubuntu systems)
• GLOG (libgoogle-glog-dev in Debian/Ubuntu systems)
• [optional] if you want to run automatic tests (make check), libcunit and libcppunit are required (in libcunit1-dev and libcppunit-dev packages in Debian-like systems).
• [optional] Doxygen to generate the documentation.

How to build

Install the dependencies and run:

sh# ./autogen.sh  
sh# cd build  
sh# ../configure  
sh# make  
sh# make install  

Optionally you can 'make check' for consistency checks.

Optional ../configure parameters

--enable-debug: Compile with debug symbols (-g) and debug output (warning, it may affect performance)  
--enable-verbose: Increase the level of debug (useless without --enable-debug)  
--disable-silent-rules: Enable verbose compilation mode (AM_SILENT_RULES disabled)

Doxygen documentation

Documentation can be compiled via make doc. Documentation is generated in build/doc/ folder

Known issues

• Certain GCC 4.8.X family versions are buggy and don't properly compile ROFL with -O3 optimizations. Please use GCC 4.8.3 or higher, or another version family.

FAQ

Some distributions, in particular Red Hat based, do not include by default /usr/local/lib on the search path of ldconfig, not allowing to link against -lrofl and -lrofl_pipeline. In this case, you might have to add /usr/local/lib/ into /etc/ld.so.conf and re-run ldconfig.

More information, documentation and support

You can have a look on:

• https://bisdn.github.io/rofl-core/rofl-common/index.html
• https://bisdn.github.io/rofl-core/rofl-common/modules.html
• http://www.roflibs.org/
• http://www.roflibs.org/doc/

Documentation can be compiled via make doc. Documentation is generated in build/doc/ folder

For getting support or participate in the discussions, join the mailing list [email protected] at https://lists.roflibs.org/mailman

Authorship

(c) Copyright BISDN GmbH 2013

Andreas Koepsel<andreas.koepsel (at) bisdn.de>
Tobias Jungel<tobias.jungel (at) bisdn.de>
Marc Sune<marc.sune (at) bisdn.de>
Victor Alvarez<victor.alvarez (at) bisdn.de>

And others.

rofl-common's Issues

Strange output in DEBUG level 7. Missing output too.

The OF agent throws strange outputs when launched with debug level 7:

[rofl-common][cioloop][add_writefd] fd:27, tid: 0x7fffb0dfd700
[rofl-common][cioloop][drop_writefd] fd:27, tid: 0x7fffb0dfd700
[rofl-common][crofconn] received Echo.reply: -Echo-Reply- OFP:{version: 1.0, type: 3, len: 8, xid: 0xd938ddbd}  
[rofl-common][crofconn] event-echo-rcvd: OFP transport connection is good. state: -ESTABLISHED- 
[rofl-common][crofconn] event-need-life-check
[rofl-common][crofconn] sending Echo.request: -Echo-Request- OFP:{version: 1.0, type: 2, len: 8, xid: 0xd938ddbe}  
[rofl-common][cioloop][add_writefd] fd:27, tid: 0x7fffb0dfd700
[rofl-common][cioloop][drop_writefd] fd:27, tid: 0x7fffb0dfd700

Add/drop fds should be (at least) DEBUG as well as event-need-life-check.

There is no INFO comprehensible message on when the connection with the controller is lost, or is recovered. Also the levels seems strange, ON/OFF, basically between -d7 and -d8.

In -d 7 there is way few information (seems to have changed lately v0.6RC1 or so), like message reception (e.g. PKT_OUT, PKT_IN, FLOWMOD etc..) without cmemory dumping that probably belongs to DEBUG level.

crofchantest: valgrind results

==30722==
==30722== HEAP SUMMARY:
==30722== in use at exit: 123,776 bytes in 85 blocks
==30722== total heap usage: 16,219 allocs, 16,134 frees, 6,682,390 bytes allocated
==30722==
==30722== Searching for pointers to 85 not-freed blocks
==30722== Checked 704,364,264 bytes
==30722==
==30722== LEAK SUMMARY:
==30722== definitely lost: 0 bytes in 0 blocks
==30722== indirectly lost: 0 bytes in 0 blocks
==30722== possibly lost: 51,072 bytes in 84 blocks
==30722== still reachable: 72,704 bytes in 1 blocks
==30722== suppressed: 0 bytes in 0 blocks
==30722== Rerun with --leak-check=full to see details of leaked memory
==30722==
==30722== ERROR SUMMARY: 4 errors from 1 contexts (suppressed: 0 from 0)
==30722==
==30722== 4 errors in context 1 of 1:
==30722== Invalid write of size 4
==30722== at 0x499DF9: rofl::cthread::run_loop() (cthread.cpp:368)
==30722== by 0x49B1D4: rofl::cthread::start_loop(void_) (cthread.hpp:299)
==30722== by 0x5044554: start_thread (in /usr/lib64/libpthread-2.21.so)
==30722== by 0x6494B9C: clone (in /usr/lib64/libc-2.21.so)
==30722== Address 0x2b74c64c is 60 bytes inside a block of size 552 free'd
==30722== at 0x4C2A184: operator delete(void_) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30722== by 0x41A091: rofl::crofchan::~crofchan() (crofchan.h:186)
==30722== by 0x407EF7: crofchantest::tearDown() (crofchantest.cpp:37)
==30722== by 0x40B779: CppUnit::TestCaller::tearDown() (TestCaller.h:182)
==30722== by 0x40B82B: non-virtual thunk to CppUnit::TestCaller::tearDown() (TestCaller.h:183)
==30722== by 0x527C3A1: CppUnit::TestCaseMethodFunctor::operator()() const (in /usr/lib64/libcppunit-1.12.so.1.0.0)
==30722== by 0x5271F72: CppUnit::DefaultProtector::protect(CppUnit::Functor const&, CppUnit::ProtectorContext const&) (in /usr/lib64/libcppunit-1.12.so.1.0.0)
==30722== by 0x527938C: CppUnit::ProtectorChain::protect(CppUnit::Functor const&, CppUnit::ProtectorContext const&) (in /usr/lib64/libcppunit-1.12.so.1.0.0)
==30722== by 0x5282458: CppUnit::TestResult::protect(CppUnit::Functor const&, CppUnit::Test_, std::string const&) (in /usr/lib64/libcppunit-1.12.so.1.0.0)
==30722== by 0x527C10F: CppUnit::TestCase::run(CppUnit::TestResult_) (in /usr/lib64/libcppunit-1.12.so.1.0.0)
==30722== by 0x527C742: CppUnit::TestComposite::doRunChildTests(CppUnit::TestResult_) (in /usr/lib64/libcppunit-1.12.so.1.0.0)
==30722== by 0x527C65D: CppUnit::TestComposite::run(CppUnit::TestResult_) (in /usr/lib64/libcppunit-1.12.so.1.0.0)
==30722==
==30722== ERROR SUMMARY: 4 errors from 1 contexts (suppressed: 0 from 0)

cofgroupmod::set_type

in cofgroupmod.cc

cofgroupmod::set_type(uint8_t type)
sould be renamed to
cofgroupmod::set_group_type(uint8_t type)
and
cofgroupmod::get_type() const
sould be renamed to
cofgroupmod::get_group_type() const

to be consistent with e.g.
cofmsg_group_mod::get_group_type()

Running programs with -d x

Scenario: xdpd connected with a rofl-based controller, everything works fine when I run both without -d or with the same like -d 5

Issue: when it differs (e.g. ./xdpd -d 8 and ./rofl-controller -d 5) the connection is terminated and the controller throws

[rofl-common][crofdpt] transaction expired, xid: <transaction xid:a601d418 msg-type:18 msg-subtype:12 > <expires: > <cclock sec:aa7c4ae6 nsec:2ad03466 >

SSL 1.1 issues on Debian Stretch

I have these ssl libraries installed on my system:

# ls -al /usr/lib/x86_64-linux-gnu/ | grep ssl
-rw-r--r-- 1 root root   737466 Feb 16 13:57 libssl.a
lrwxrwxrwx 1 root root       13 Feb 16 13:57 libssl.so -> libssl.so.1.1
-rw-r--r-- 1 root root   431232 Jan 26 11:39 libssl.so.1.0.2
-rw-r--r-- 1 root root   442920 Feb 16 13:57 libssl.so.1.1
drwxr-xr-x 1 root root       14 Dec  7 14:20 openssl-1.0.2
# ls -al /usr/lib/x86_64-linux-gnu/ | grep crypto
-rw-r--r-- 1 root root  4799548 Feb 16 13:57 libcrypto.a
lrwxrwxrwx 1 root root       16 Feb 16 13:57 libcrypto.so -> libcrypto.so.1.1
-rw-r--r-- 1 root root  2492192 Jan 26 11:39 libcrypto.so.1.0.2
-rw-r--r-- 1 root root  2686448 Feb 16 13:57 libcrypto.so.1.1
lrwxrwxrwx 1 root root       18 Dec  4 16:37 libk5crypto.so.3 -> libk5crypto.so.3.1
-rw-r--r-- 1 root root   203656 Dec  4 16:37 libk5crypto.so.3.1

I need to run the following to ensure ssl libraries are included, I see the -lcrypto is applied properly (configure seems to miss somehow):

LIBS=-lssl ../configure

In addition, with SSL 1.1, I run the following before I do the autogen.sh:

sed -i 's/TLSv1_2_method()/TLS_method()/' src/rofl/common/crofsock.c

Some relevant information:

https://www.openssl.org/docs/man1.1.0/ssl/SSL_CTX_new.html

output of group_id

ROFL-integration-0.7:

in cofgroupmod.h (https://github.com/bisdn/rofl-common/blob/integration-0.7/src/rofl/common/openflow/cofgroupmod.h#L195) output of group_id: can we please have this output in hex-format to be consistent with / comparable to the output of a group_id in cofflowmod.h (https://github.com/bisdn/rofl-common/blob/integration-0.7/src/rofl/common/openflow/cofflowmod.h#L312)

get_group_mod()

Can we please have a get_group_mod() function, similar to get_flowmod().
That is used quite often in my controller.

Example:
rofl::openflow::cofmsg_flow_mod& msg
rofl::openflow::cofflowmod fm_new = msg.get_flowmod();

and btw, get_flowmod/set_flowmod should be renamed to get_flow_mod/set_flow_mod to be consistent

output of coxmatch.h

in

rofl-common/src/rofl/common/openflow/coxmatch.h

Line 555 in 0adb372

os << indent(2) << "<in-port: " << std::hex << (int)oxm.get_u32value() << std::dec << " >" << std::endl;

"<in-port: " << std::hex should be "<in-port:" << std::dec

same on line 581 with in-phy-port

ethswctld runtime error with openvswitch 2.7.90 on debian stretch

I was hoping I could build and run the example and see some results. But, sigh, no.

Being totally new to the code, I am unfamiliar with the threads and messages.

Might there be someone experienced be able to quickly provide an indication of the issue?

(gdb) info source
Current source file is ../../../../src/rofl/common/crofconn.cc
Compilation directory is /home/vagrant/rofl-common/build/src/rofl/common
Located in /home/vagrant/rofl-common/src/rofl/common/crofconn.cc
Contains 1973 lines.
Source language is c++.
Producer is GNU C++11 6.3.0 20170425 -mtune=generic -march=x86-64 -g -g -O2 -O0 -std=c++11 -fno-inline -fPIC.
Compiled with DWARF 2 debugging format.

gdb) r
Starting program: /home/vagrant/rofl-common/build/examples/ethswctld/ethswctld
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff5bde700 (LWP 27149)]
[New Thread 0x7ffff53dd700 (LWP 27150)]
[New Thread 0x7ffff4bdc700 (LWP 27151)]
[New Thread 0x7fffeffff700 (LWP 27152)]
[New Thread 0x7fffef7fe700 (LWP 27153)]
[New Thread 0x7fffeeffd700 (LWP 27155)]
[cetherswitch] datapath attached, dptid: 0
[cetherswitch] caught some exception, use debugger for getting more info
<cofmsg version: 4 type: 10 length: 84(wire: 84) xid: 0x0 >
<cofmsg_packet_in >
<buffer-id:-1 >
<total-len:42 >
<reason:no match  >
<table-id: 0 >
<cookie: 0x0 >
<cofmatch ofp-version:4 >
<type: 1 >
<coxmatches #matches:1 >
<id: 0x80000000 >
<coxmatch oxm_id: 0x80000004 class: 0x8000 field: 0x0 hasmask: 0 length: 4>
<in-port: 0x1 >
<cpacket data:0x7fffe8001160 datalen:42 head:64 tail:256>
0000: ff ff ff ff ff ff 7e 7a   dd f9 85 02 08 06 00 01   08 00 06 04 00 01 7e 7a   dd f9 85 02 0a 00 00 01
0001: 00 00 00 00 00 00 0a 00   00 02
ethswctld: ../../../examples/ethswctld/cetherswitch.cc:285: virtual void rofl::examples::ethswctld::cetherswitch::handle_packet_in(rofl::crofdpt&, const rofl::cauxid&, rofl::openflow::cofmsg_packet_in&): Assertion `0' failed.

Thread 6 "crofconn" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffef7fe700 (LWP 27153)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.

(gdb) info stack
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff647e3fa in __GI_abort () at abort.c:89
#2  0x00007ffff6475e37 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x55555575f194 "0",
    file=file@entry=0x55555575f168 "../../../examples/ethswctld/cetherswitch.cc", line=line@entry=285,
    function=function@entry=0x55555575f2a0 <rofl::examples::ethswctld::cetherswitch::handle_packet_in(rofl::crofdpt&, rofl::cauxid const&, rofl::openflow::cofmsg_packet_in&)::__PRETTY_FUNCTION__> "virtual void rofl::examples::ethswctld::cetherswitch::handle_packet_in(rofl::crofdpt&, const rofl::cauxid&, rofl::openflow::cofmsg_packet_in&)") at assert.c:92
#3  0x00007ffff6475ee2 in __GI___assert_fail (assertion=0x55555575f194 "0", file=0x55555575f168 "../../../examples/ethswctld/cetherswitch.cc", line=285,
    function=0x55555575f2a0 <rofl::examples::ethswctld::cetherswitch::handle_packet_in(rofl::crofdpt&, rofl::cauxid const&, rofl::openflow::cofmsg_packet_in&)::__PRETTY_FUNCTION__> "virtual void rofl::examples::ethswctld::cetherswitch::handle_packet_in(rofl::crofdpt&, const rofl::cauxid&, rofl::openflow::cofmsg_packet_in&)")
    at assert.c:101
#4  0x00005555555813ec in rofl::examples::ethswctld::cetherswitch::handle_packet_in (this=0x7fffffffdba0, dpt=..., auxid=..., msg=...)
    at ../../../examples/ethswctld/cetherswitch.cc:285
#5  0x00005555556964f8 in rofl::crofdpt::packet_in_rcvd (this=0x7fffe8000ad0, auxid=..., msg=0x7fffe80014e0) at ../../../../src/rofl/common/crofdpt.cc:690
#6  0x00005555556944a9 in rofl::crofdpt::handle_recv (this=0x7fffe8000ad0, chan=..., conn=..., msg=0x7fffe80014e0) at ../../../../src/rofl/common/crofdpt.cc:121
#7  0x0000555555690a9c in rofl::crofchan::handle_recv (this=0x7fffe8000ae8, conn=..., msg=0x7fffe80014e0)
    at /home/vagrant/rofl-common/build/../src/rofl/common/crofchan.h:525
#8  0x0000555555627cd3 in rofl::crofconn::handle_rx_messages (this=0x7ffff0000970) at ../../../../src/rofl/common/crofconn.cc:1227
#9  0x00005555556277f0 in rofl::crofconn::handle_wakeup (this=0x7ffff0000970, thread=...) at ../../../../src/rofl/common/crofconn.cc:1168
#10 0x00005555556c7cb9 in rofl::cthread::run_loop (this=0x7ffff0000988) at ../../../../src/rofl/common/cthread.cpp:459
#11 0x00005555556c84c6 in rofl::cthread::start_loop (arg=0x7ffff0000988) at ../../../../src/rofl/common/cthread.hpp:210
#12 0x00007ffff79bb494 in start_thread (arg=0x7fffef7fe700) at pthread_create.c:333
#13 0x00007ffff653293f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

ciosrv: stucks at high loads

Thanks for having a look at this!

Unfortunately, although it got better, it seems it hasn't solved the problem completely. I tried l2flows.py with :

@set_ev_cls(ofp_event.EventOFPSwitchFeatures, CONFIG_DISPATCHER)
def switch_features_handler(self, ev):
    self.logger.info("Hello")
    datapath = ev.msg.datapath
    ofproto = datapath.ofproto
    parser = datapath.ofproto_parser

    start_time = time.time()

    for i in range(1,100000):
            mac = self.getNextMAC(i)
            match = parser.OFPMatch(eth_dst=mac)
            actions = [parser.OFPActionOutput(2)]
            self.add_flow(datapath,1,match,actions,0)

    self.logger.info("Flows added.")
    print("--- %s seconds ---" % (time.time() - start_time))

---

root@dev:/home/marc/git/other/ryu# ryu-manager l2flows.py
loading app l2flows.py
loading app ryu.controller.ofp_handler
instantiating app l2flows.py of SimpleSwitch13
instantiating app ryu.controller.ofp_handler of OFPHandler
Hello
Flows added.
--- 29.5651071072 seconds ---

And I got installed around 22K flows:

xcli(local)>show lsi dp0
{
    "name" : "dp0",
    "dpid" : 1,
    "of_version" : "1.3",
    "num-of-tables" : 8,
    "miss-send-len" : 128,
    "num-of-buffers" : 512,
    "attached-ports" : {
        "1" : "veth0",
        "2" : "veth2",
        "3" : "veth4",
        "4" : "veth6"
    },
    "tables" : {
        "0" : {
            "number" : 0,
            "matching-algorithm" : 0,
            "num-of-entries" : 22195,
            "max-entries" : 4294967295,
            "table-miss" : "drop",
            "statistics" : {
                "pkts-looked-up" : 8,
                "pkts-matched" : 0
            }
        },

I can confirm that is not a pipeline issue, because during the unit test of the new matching algorithm (trie) I was able to install 100K flows with the linear algorithm without a problem (well it took forever, but it finally did).

There is another minor issue; commit:

454b1a4

was merged in 0.6 (although the original PR was issues against integration-0.7, because it changes the API). Now it cannot be easily remove/reverted from 0.6, because commits on top conflict.

Ofc, I can adapt xdpd uses of the logger to the new API (as I did to test the patch proposed). But just in case you did not realise, the controllers will be broken with rofl-common v0.6.2 once released.

missing ctspaddress.h under /usr/include/rofl/common/

after running make install, /usr/include/rofl/common/ctspaddress.h is missing

ctimespec::operator- () with invalid overflow handling

also occuring in ctimespec::operator-= ()

-Wstringop-truncation and -Wstringop-overflow

https://copr-be.cloud.fedoraproject.org/results/bisdn/rofl/fedora-rawhide-x86_64/00717574-rofl-common/build.log.gz

../../../../../src/rofl/common/openflow/cofport.cc: In member function 'void rofl::openflow::cofport::pack(uint8_t*, size_t)':
../../../../../src/rofl/common/openflow/cofport.cc: In member function 'void rofl::openflow::cofport::pack(uint8_t*, size_t)':
../../../../../src/rofl/common/openflow/cofport.cc:52:12: error: 'char* strncpy(char*, const char*, size_t)' specified bound 16 equals destination size [-Werror=stringop-truncation]
../../../../../src/rofl/common/openflow/cofport.cc:52:12: error: 'char* strncpy(char*, const char*, size_t)' specified bound 16 equals destination size [-Werror=stringop-truncation]
     strncpy(hdr->name, name.c_str(), OFP_MAX_PORT_NAME_LEN);
     strncpy(hdr->name, name.c_str(), OFP_MAX_PORT_NAME_LEN);
     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../../../../src/rofl/common/openflow/cofport.cc:70:12: error: 'char* strncpy(char*, const char*, size_t)' specified bound 16 equals destination size [-Werror=stringop-truncation]
../../../../../src/rofl/common/openflow/cofport.cc:70:12: error: 'char* strncpy(char*, const char*, size_t)' specified bound 16 equals destination size [-Werror=stringop-truncation]
     strncpy(hdr->name, name.c_str(), OFP_MAX_PORT_NAME_LEN);
     strncpy(hdr->name, name.c_str(), OFP_MAX_PORT_NAME_LEN);
     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../../../../src/rofl/common/openflow/cofport.cc:35:12: error: 'char* strncpy(char*, const char*, size_t)' writing 16 bytes into a region of size 6 overflows the destination [-Werror=stringop-overflow=]
../../../../../src/rofl/common/openflow/cofport.cc:35:12: error: 'char* strncpy(char*, const char*, size_t)' writing 16 bytes into a region of size 6 overflows the destination [-Werror=stringop-overflow=]
     strncpy(hdr->name, name.c_str(), OFP_MAX_PORT_NAME_LEN);
     strncpy(hdr->name, name.c_str(), OFP_MAX_PORT_NAME_LEN);
     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~