alexbredo/ipfix-receiver

IPFIX Receiver and Conversation-Aggregator

ipfix-receiver's Issues

Error: Ignoring flow, because not Layer 3 (IP).

Message: INFO@OpenSocketAggregator: Ignoring flow, because not Layer 3 (IP).

Currently Layer 2 flows (MAC-Src + Dst) can't be aggregated, because there is no aggregator available. So this L2-flows are ignored.

TODO:
--> Rewrite Aggregator, so that it handles L2, L3 and probably physical :-D

Idea:

• Superclass FlowAggregator
• L3Aggregator inherits from FlowAggregator
• L2Aggregator inherits from FlowAggregator

Bottleneck Conversation Aggregator

ERROR: Overall Queue-Size is extremly long ((Queue-Size) Flows: 0. Corrector: 916. Stats: 25. Conversation: 88949. Security: 0. Postprocessing: 0. Finisher: 0.)

Conversation-Aggregator is currently a bottleneck. Rewrite of Aggregator is necessary, as already written in #4.

Unsupported length of field (6). Packet can't be parsed.

Non-Standard Field-Size

https://docs.python.org/3/library/struct.html#format-characters

Solution: "6s" ? (to be checked)

Log gets very big.

Implement Logrotation, because Log gets very big.

Change Setup elasticsearch (newer es releases)

Method old: POST, new: PUT:

Content:
PUT /ipfix
{
"mappings": {
"conversation": {
"properties": {
"sourceIPv6Address": {
"type": "ip"
},
"sourceIPv4Address": {
"type": "ip"
},
"destinationIPv6Address": {
"type": "ip"
},
"destinationIPv4Address": {
"type": "ip"
},
"@timestamp": {
"type": "date",
"format": "epoch_millis"
},
"sourceHostname" : {"type" : "string", "index" : "not_analyzed"},
"destinationHostname" : {"type" : "string", "index" : "not_analyzed"},
"destinationTransportPortName" : {"type" : "string", "index" : "not_analyzed"},
"sourceTransportPortName" : {"type" : "string", "index" : "not_analyzed"},
"protocolIdentifierName" : {"type" : "string", "index" : "not_analyzed"},
"networkLocation" : {"type" : "string", "index" : "not_analyzed"},
"securityReason" : {"type" : "string", "index" : "not_analyzed"}
}
}
}
}

Error: Length of Template and Data not equal

If multiple IPFIX-Senders send messages with same the Template-ID, but different template data, there is an error while parsing: "Length of Template and Data not equal". So packets can't be parsed, because they will use a wrong template.

Solution (todo): Implement a datastructure, which holds and finds templates by source ip of ipfx-sender.

Create data/flows directory at start