openixia/ixvision-zta

Zero-Touch Automation for Ixia Vision Network Packet Brokers

ixvision-zta's Issues

lldptag on a device w/o LLDP support

`/tmp/tmp.6mgiacviP3/tempScript6486070844546450885
10.36.237.160 ; echo $
Discovered a Vision Edge 10S device with a management IP address 10.36.237.160 - initializing ZTP
Starting port status discovery for 10.36.237.160
Enabled port 10.36.237.160:P03
Enabled port 10.36.237.160:P04
Enabled port 10.36.237.160:P05
Enabled port 10.36.237.160:P06
Enabled port 10.36.237.160:P07
Enabled port 10.36.237.160:P08
Enabled port 10.36.237.160:P09
Enabled port 10.36.237.160:P10
Enabled port 10.36.237.160:P11
Enabled port 10.36.237.160:P12
Enabled port 10.36.237.160:P13
Enabled port 10.36.237.160:P14
Enabled port 10.36.237.160:P15
Enabled port 10.36.237.160:P16
Enabled port 10.36.237.160:P19
Enabled port 10.36.237.160:P20
Enabled port 10.36.237.160:P21
Enabled port 10.36.237.160:P22
Enabled port 10.36.237.160:P23
Enabled port 10.36.237.160:P24
Enabled port 10.36.237.160:P25
Enabled port 10.36.237.160:P26
Enabled port 10.36.237.160:P27
Enabled port 10.36.237.160:P28
Enabled port 10.36.237.160:P29
Enabled port 10.36.237.160:P30
Enabled port 10.36.237.160:P31
Enabled port 10.36.237.160:P32
Enabled port 10.36.237.160:P33
Enabled port 10.36.237.160:P34
Enabled port 10.36.237.160:P35
Enabled port 10.36.237.160:P36
Enabled port 10.36.237.160:P37
Enabled port 10.36.237.160:P38
Enabled port 10.36.237.160:P39
Enabled port 10.36.237.160:P40
Enabled port 10.36.237.160:P41
Enabled port 10.36.237.160:P42
Enabled port 10.36.237.160:P43
Enabled port 10.36.237.160:P44
Enabled port 10.36.237.160:P45
Enabled port 10.36.237.160:P46
Enabled port 10.36.237.160:P47
Enabled port 10.36.237.160:P48

Collected port 10.36.237.160:P03 status: DOWN
Collected port 10.36.237.160:P04 status: DOWN
Collected port 10.36.237.160:P05 status: DOWN
Collected port 10.36.237.160:P06 status: DOWN
Collected port 10.36.237.160:P07 status: DOWN
Collected port 10.36.237.160:P08 status: DOWN
Collected port 10.36.237.160:P09 status: DOWN
Collected port 10.36.237.160:P10 status: DOWN
Collected port 10.36.237.160:P11 status: DOWN
Collected port 10.36.237.160:P12 status: DOWN
Collected port 10.36.237.160:P13 status: DOWN
Collected port 10.36.237.160:P14 status: DOWN
Collected port 10.36.237.160:P15 status: DOWN
Collected port 10.36.237.160:P16 status: DOWN
Collected port 10.36.237.160:P19 status: DOWN
Collected port 10.36.237.160:P20 status: DOWN
Collected port 10.36.237.160:P21 status: DOWN
Collected port 10.36.237.160:P22 status: DOWN
Collected port 10.36.237.160:P23 status: DOWN
Collected port 10.36.237.160:P24 status: DOWN
Collected port 10.36.237.160:P25 status: DOWN
Collected port 10.36.237.160:P26 status: DOWN
Collected port 10.36.237.160:P27 status: DOWN
Collected port 10.36.237.160:P28 status: DOWN
Collected port 10.36.237.160:P29 status: DOWN
Collected port 10.36.237.160:P30 status: DOWN
Collected port 10.36.237.160:P31 status: DOWN
Collected port 10.36.237.160:P32 status: DOWN
Collected port 10.36.237.160:P33 status: DOWN
Collected port 10.36.237.160:P34 status: DOWN
Collected port 10.36.237.160:P35 status: DOWN
Collected port 10.36.237.160:P36 status: DOWN
Collected port 10.36.237.160:P37 status: DOWN
Collected port 10.36.237.160:P38 status: DOWN
Collected port 10.36.237.160:P39 status: DOWN
Collected port 10.36.237.160:P40 status: DOWN
Collected port 10.36.237.160:P41 status: DOWN
Collected port 10.36.237.160:P42 status: DOWN
Collected port 10.36.237.160:P43 status: DOWN
Collected port 10.36.237.160:P44 status: DOWN
Collected port 10.36.237.160:P45 status: DOWN
Collected port 10.36.237.160:P46 status: DOWN
Collected port 10.36.237.160:P47 status: DOWN
Collected port 10.36.237.160:P48 status: DOWN
Converted port 10.36.237.160:P03 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P04 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P05 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P06 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P07 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P08 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P09 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P10 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P11 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P12 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P13 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P14 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P15 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P16 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P19 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P20 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P21 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P22 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P23 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P24 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P25 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P26 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P27 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P28 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P29 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P30 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P31 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P32 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P33 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P34 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P35 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P36 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P37 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P38 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P39 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P40 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P41 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P42 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P43 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P44 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P45 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P46 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P47 to 1G/Auto, NETWORK
Converted port 10.36.237.160:P48 to 1G/Auto, NETWORK

Collected port 10.36.237.160:P03 status: DOWN
Collected port 10.36.237.160:P04 status: DOWN
Collected port 10.36.237.160:P05 status: DOWN
Collected port 10.36.237.160:P06 status: DOWN
Collected port 10.36.237.160:P07 status: DOWN
Collected port 10.36.237.160:P08 status: DOWN
Collected port 10.36.237.160:P09 status: DOWN
Collected port 10.36.237.160:P10 status: DOWN
Collected port 10.36.237.160:P11 status: DOWN
Collected port 10.36.237.160:P12 status: DOWN
Collected port 10.36.237.160:P13 status: DOWN
Collected port 10.36.237.160:P14 status: DOWN
Collected port 10.36.237.160:P15 status: DOWN
Collected port 10.36.237.160:P16 status: DOWN
Collected port 10.36.237.160:P19 status: DOWN
Collected port 10.36.237.160:P20 status: DOWN
Collected port 10.36.237.160:P21 status: DOWN
Collected port 10.36.237.160:P22 status: DOWN
Collected port 10.36.237.160:P23 status: DOWN
Collected port 10.36.237.160:P24 status: DOWN
Collected port 10.36.237.160:P25 status: DOWN
Collected port 10.36.237.160:P26 status: DOWN
Collected port 10.36.237.160:P27 status: DOWN
Collected port 10.36.237.160:P28 status: DOWN
Collected port 10.36.237.160:P29 status: DOWN
Collected port 10.36.237.160:P30 status: DOWN
Collected port 10.36.237.160:P31 status: DOWN
Collected port 10.36.237.160:P32 status: DOWN
Collected port 10.36.237.160:P33 status: DOWN
Collected port 10.36.237.160:P34 status: DOWN
Collected port 10.36.237.160:P35 status: DOWN
Collected port 10.36.237.160:P36 status: DOWN
Collected port 10.36.237.160:P37 status: DOWN
Collected port 10.36.237.160:P38 status: DOWN
Collected port 10.36.237.160:P39 status: DOWN
Collected port 10.36.237.160:P40 status: DOWN
Collected port 10.36.237.160:P41 status: DOWN
Collected port 10.36.237.160:P42 status: DOWN
Collected port 10.36.237.160:P43 status: DOWN
Collected port 10.36.237.160:P44 status: DOWN
Collected port 10.36.237.160:P45 status: DOWN
Collected port 10.36.237.160:P46 status: DOWN
Collected port 10.36.237.160:P47 status: DOWN
Collected port 10.36.237.160:P48 status: DOWN
Converted port 10.36.237.160:P03 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P04 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P05 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P06 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P07 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P08 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P09 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P10 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P11 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P12 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P13 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P14 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P15 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P16 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P19 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P20 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P21 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P22 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P23 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P24 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P25 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P26 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P27 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P28 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P29 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P30 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P31 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P32 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P33 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P34 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P35 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P36 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P37 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P38 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P39 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P40 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P41 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P42 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P43 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P44 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P45 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P46 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P47 to 10G, NETWORK and DISABLED
Converted port 10.36.237.160:P48 to 10G, NETWORK and DISABLED
Pausing for 30 seconds for LLDP neighbors to be learned
Run port discovery via LLDP for the following keywords: TAP,SPAN,bro-monitor,moloch-capture
Starting lldp-based port tagging for 10.36.237.160
Traceback (most recent call last):
File "/home/abortok/ixvision/ixvision-ztp/ixvztp", line 125, in
tag_ports(host, port, username, password, tags)
File "/home/abortok/ixvision/ixvision-ztp/ixvision_ztp_lldp_tag.py", line 36, in tag_ports
neighbor_list = nto.getAllNeighbors()
File "/home/abortok/ixvision/ixvision-ztp/ixia_nto.py", line 1927, in getAllNeighbors
return self._callServer('POST', '/api/actions/get_neighbors', {'port_id_list': port_id_list})['message']
File "/home/abortok/ixvision/ixvision-ztp/ixia_nto.py", line 97, in _callServer
data = json.loads(data.decode('ascii'))
File "/usr/lib/python2.7/json/init.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Convert ports, tagged as TAP or SPAN, to network mode, if needed
Starting port mode update for 10.36.237.160
No mode update requied for ports with keywords TAP, SPAN
Create/update load-balance tool port group BRO with ports tagged as monitor
Starting port group formation for 10.36.237.160
No group found, created a new one with id 65
No matching ports found
Convert ports, tagged as MOLOCH-CAPTURE, to tool mode, if needed
Starting port mode update for 10.36.237.160
No mode update requied for ports with keywords moloch-capture
Create IoC_Detection filter for traffic from TAPs
Starting dynamic filter formation for 10.36.237.160
No existing DF found, created a new one with id 66
No matching ports found with keywords TAP
No matching ports found with keywords BRO
Starting dynamic filter formation for 10.36.237.160
Found an existing DF F2 in PASS_ALL mode
Connect IoC_Detection filter to SPANs as well
Starting dynamic filter formation for 10.36.237.160
Found an existing DF F2 in PASS_ALL mode
No matching ports found with keywords SPAN
No matching ports found with keywords BRO
Create IPv4_Capture filter for traffic from TAPs
Starting dynamic filter formation for 10.36.237.160
No existing DF found, created a new one with id 67
No matching ports found with keywords TAP
No matching ports found with keywords MOLOCH-CAPTURE
Connect IPv4_Capture filter to SPANs as well
Starting dynamic filter formation for 10.36.237.160
Found an existing DF F3 in PASS_BY_CRITERIA mode
No matching ports found with keywords SPAN
No matching ports found with keywords MOLOCH-CAPTURE
0
abortok@CloudDC-Ansible-01:~$ `

Feature Request Needed

Need to add a feature to identify the port numbers on the ingress side and the tool side connections.

(port_group_type) is an undefined property for Port Group

Starting with 5.0.0.9, Port Group property port_group_type is no longer valid. See BUG1494490.

Logs when running 'pgform' action:
Sending a message to the server with parameters:
HTTPMethod=POST
URL=/api/port_groups
argsAPI={'keywords': ['_ZTP_LLDP', 'PROBE'], 'mode': 'TOOL', 'name': 'PROBES', 'port_group_type': 'LOAD_BALANCE'}
Response:
Status=500
Reason=Server Error
Headers=HTTPHeaderDict({'Content-Length': '58', 'X-XSS-Protection': '1; mode=block', 'X-Content-Type-Options': 'nosniff', 'Content-Security-Policy': "default-src 'self' 'unsafe-eval' 'unsafe-inline'; frame-ancestors 'self'; img-src http: https: data:", 'Accept-Ranges': 'bytes', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'Server': 'Restlet-Framework/2.3.7', 'Pragma': 'no-cache', 'Cache-Control': 's-maxage=100, must-revalidate, no-cache, no-store, no-transform, private', 'Date': 'Tue, 12 Feb 2019 16:40:28 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Type': 'application/xml; charset=UTF-8', 'Public-Key-Pins': 'pin-sha256="91eaiBlpaQ2uZ95MhUIbgk5bNM5sh+hDN9mNa/2+704="; max-age=5184000; includeSubDomains'})
Data=(port_group_type) is an undefined property for Port Group.
decode=True

FR: set disconnected ports to default settings

sysinfo fails while pulling get_login_info

Platform AS5812, version "trunk-213356-20190130-131158"

(ixvision) ubuntu@ixvzta:~/ixvision$ "$PYENV_DIR/ixvision-zta/ixvztp" -u $WEB_API_USERNAME -p $WEB_API_PASSWORD -d $DEVICE_IP sysinfo
Starting system information inquiry for 172.20.100.32
Traceback (most recent call last):
File "/home/ubuntu/ixvision/ixvision-zta/ixvztp", line 113, in
nto_get_sysinfo(host, port, username, password)
File "/home/ubuntu/ixvision/ixvision-zta/ixvision_ztp_sysinfo.py", line 37, in nto_get_sysinfo
nto_hardware_info = nto.getLoginInfo()['hardware_info']
File "/home/ubuntu/ixvision/ixvision-zta/ixia_nto.py", line 324, in getLoginInfo
return self._callServer('POST', '/api/actions/get_login_info', None)
File "/home/ubuntu/ixvision/ixvision-zta/ixia_nto.py", line 97, in _callServer
data = json.loads(data.decode('ascii'))
File "/usr/lib/python2.7/json/init.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Debug

Sending a message to the server with parameters:
HTTPMethod=POST
URL=/api/actions/get_login_info
argsAPI=None
Response:
Status=400
Reason=Bad Request
Headers=HTTPHeaderDict({'Content-Length': '76', 'X-XSS-Protection': '1; mode=block', 'X-Content-Type-Options': 'nosniff', 'Content-Security-Policy': "default-src 'self' 'unsafe-eval' 'unsafe-inline'; frame-ancestors 'self'; img-src http: https: data:", 'Accept-Ranges': 'bytes', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'Server': 'Restlet-Framework/2.3.7', 'Pragma': 'no-cache', 'Cache-Control': 's-maxage=100, must-revalidate, no-cache, no-store, no-transform, private', 'Date': 'Tue, 12 Feb 2019 16:10:14 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Type': 'text/plain; charset=UTF-8', 'Public-Key-Pins': 'pin-sha256="91eaiBlpaQ2uZ95MhUIbgk5bNM5sh+hDN9mNa/2+704="; max-age=5184000; includeSubDomains'})
Data=org.json.JSONException: A JSONObject text must begin with '{' at character 1
decode=True

pgform: member ports are ordered in reverse

When an LB port group is created, member ports order is reversed - higher port numbers come first. Not clear if that matters load balancing algorithm, but ascending ordering shall be enforced for consistency reasons

README: update repo clone syntax

Change from
git clone [email protected]:bortok/ixvision-zta.git
to
git clone https://github.com/bortok/ixvision-zta.git

Handle connection errors

Starting port status discovery for 172.20.100.33
Traceback (most recent call last):
File "/Users/albortok/Documents/00_Projects/ix/ixvision/ixvision-zta/ixvztp", line 134, in
discover_ports(host, port, username, password, keyword)
File "/Users/albortok/Documents/00_Projects/ix/ixvision/ixvision-zta/ixvision_ztp_port_discovery.py", line 24, in discover_ports
nto = NtoApiClient(host=host_ip, username=username, password=password, port=port, debug=True, logFile="ixvision_ztp_debug.log")
File "/Users/albortok/Documents/00_Projects/ix/ixvision/ixvision-zta/ixia_nto.py", line 49, in init
response = self.connection.urlopen('GET', '/api/auth', headers=self.password_headers)
File "/Users/albortok/Documents/00_Projects/ix/ixvision/lib/python2.7/site-packages/urllib3/connectionpool.py", line 667, in urlopen
**response_kw)
File "/Users/albortok/Documents/00_Projects/ix/ixvision/lib/python2.7/site-packages/urllib3/connectionpool.py", line 667, in urlopen
**response_kw)
File "/Users/albortok/Documents/00_Projects/ix/ixvision/lib/python2.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/Users/albortok/Documents/00_Projects/ix/ixvision/lib/python2.7/site-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='172.20.100.33', port='8000'): Max retries exceeded with url: /api/auth (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x105dd6750>: Failed to establish a new connection: [Errno 61] Connection refused',))

.Python symlink

.Python -> /usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/Python

Feature Request - Add PacketStack

Need a feature that will allow PacketStack features such as MPLS stripping and Deduplication added to the zta.