job / aclhound Goto Github PK

View Code? Open in Web Editor NEW

19.0 19.0 9.0 4.49 MB

ACLHound

License: BSD 2-Clause "Simplified" License

Makefile 0.24% Python 98.44% Shell 0.73% Roff 0.59%

aclhound's People

Contributors

Stargazers

Watchers

Forkers

ebayclassifiedsgroup mgmoerman rmrf-run djinh favoretti glebpoljakov tsuvanto2 dmeulen

aclhound's Issues

please add icmp as a valid protocol

only tcp, udp and any are allowed at this moment

Adding subnetmask to ip's in object files seem to fail

It seems there are 2 issues here:

adding subnetmask in the mask notation seems to use the mask as the actual host during translation towards device specific ACL syntax
adding subnetmask in CIDR notation is not getting translated to mask notation (required for device specific ACL syntax)

Example for CIDR notation:

mmoerman@aclhound001:~/aclhound$ aclhound deploy all
INFO: deploying devices/10.32.10.26
{'generic_policy-v6': [{'dir': 'in', 'int': 'inside'}],
'nw-management-v4': [{'dir': 'in', 'int': 'towebservers'}],
'nw-management-v6': [{'dir': 'out', 'int': 'towebservers'}]}
INFO: uploading name: nw-management-v6, afi: 6
configure terminal
clear configure ipv6 access-list LOCKSTEP-nw-management-v6
ipv6 access-list LOCKSTEP-nw-management-v6 permit tcp any any eq 80
ipv6 access-list LOCKSTEP-nw-management-v6 deny ip any any
end
INFO: lockstepping policy nw-management-v6 afi 6
configure terminal
access-group LOCKSTEP-nw-management-v6 out interface towebservers
end
INFO: uploading name: nw-management-v6, afi: 6
configure terminal
clear configure ipv6 access-list nw-management-v6
ipv6 access-list nw-management-v6 permit tcp any any eq 80
ipv6 access-list nw-management-v6 deny ip any any
end
INFO: lockstepping policy nw-management-v6 afi 6
configure terminal
access-group nw-management-v6 out interface towebservers
end
configure terminal
clear configure ipv6 access-list LOCKSTEP-nw-management-v6
end
INFO: uploading name: nw-management-v4, afi: 4
configure terminal
clear configure access-list LOCKSTEP-nw-management-v4
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 10.32.12.36 eq 465
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 10.10.10.10 eq 465
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 1.2.3.4 eq 465
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 2.3.4.5 eq 465
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 6.6.6.6 eq 465
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 6.7.8.8 eq 465
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 9.9.9.9 eq 465
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 10.32.12.36 eq 25
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 10.10.10.10 eq 25
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 1.2.3.4 eq 25
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 2.3.4.5 eq 25
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 6.6.6.6 eq 25
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 6.7.8.8 eq 25
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 9.9.9.9 eq 25
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 10.32.12.36 eq 110
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 10.10.10.10 eq 110
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 1.2.3.4 eq 110
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 2.3.4.5 eq 110
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 6.6.6.6 eq 110
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 6.7.8.8 eq 110
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 9.9.9.9 eq 110
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 10.32.12.36 eq 993
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 10.10.10.10 eq 993
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 1.2.3.4 eq 993
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 2.3.4.5 eq 993
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 6.6.6.6 eq 993
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 6.7.8.8 eq 993
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 9.9.9.9 eq 993
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 10.32.12.36 eq 143
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 10.10.10.10 eq 143
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 1.2.3.4 eq 143
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 2.3.4.5 eq 143
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 6.6.6.6 eq 143
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 6.7.8.8 eq 143
access-list LOCKSTEP-nw-management-v4 extended permit tcp any host 9.9.9.9 eq 143
access-list LOCKSTEP-nw-management-v4 extended permit tcp any any eq 80
access-list LOCKSTEP-nw-management-v4 extended permit tcp host 10.32.11.5 any eq 22
access-list LOCKSTEP-nw-management-v4 extended deny ip any any
access-list LOCKSTEP-nw-management-v4 extended permit tcp 10.32.1.0/24 10.32.2.0/24 eq 22
Traceback (most recent call last):
File "/usr/local/bin/aclhound", line 9, in
load_entry_point('aclhound==1.2', 'console_scripts', 'aclhound')()
File "build/bdist.linux-x86_64/egg/aclhound/cli.py", line 598, in main
File "build/bdist.linux-x86_64/egg/aclhound/cli.py", line 485, in deploy
File "build/bdist.linux-x86_64/egg/aclhound/cli.py", line 482, in do_deploy
File "build/bdist.linux-x86_64/egg/aclhound/deploy.py", line 45, in deploy
File "build/bdist.linux-x86_64/egg/aclhound/deploy.py", line 53, in deploy_asa
File "build/bdist.linux-x86_64/egg/aclhound/targets/deploy_asa.py", line 140, in deploy
File "build/bdist.linux-x86_64/egg/aclhound/targets/deploy_asa.py", line 116, in lock_step
File "build/bdist.linux-x86_64/egg/aclhound/targets/deploy_asa.py", line 67, in s
File "/usr/local/lib/python2.7/dist-packages/Exscript/protocols/Protocol.py", line 888, in execute
return self.expect_prompt()
File "/usr/local/lib/python2.7/dist-packages/Exscript/protocols/Protocol.py", line 998, in expect_prompt
raise InvalidCommandException('Device said:\n' + self.response)
Exscript.protocols.Exception.InvalidCommandException: Device said:
access-list LOCKSTEP-nw-management-v4 extended permit tcp 10$

access-list LOCKSTEP-nw-management-v4 extended permit tcp 10.32.1.0/24 10.32.2.0 ^/24 eq 22

ERROR: % Invalid input detected at '^' marker.
test-asa(config)#
mmoerman@aclhound001:~/aclhound$

Add Cisco ASA rendering support

Cisco ASA 9 support (unified v4 + v6 policy)

Cisco ASA 9 and onwards have policies in which IPv4 and IPv6 are combined, the 'any' statement can be replaced with 'any4' and 'any6' as well. We can deal with this by concatting the IPv4 and IPv6 policy and writing a new asa9 deployer

@mylex666 will provide a public accessible ASA9 device for testing

Can we move tcp/udp/any into the object, instead of the policy?

It is nicer to define the object as what it actually is, instead of having to define multiple policies. For example, DNS is both TCP & UDP, it requires then 2 policy entries in a policy file, while you could better define the object as both TCP & UDP

Splitting on port ranges

allow tcp src 10.10.10.1 port 32766-32768 dst 10.10.11.0/24 stateful

Returns 2 acls

   permit tcp host 10.10.10.1 eq 32768 10.10.11.0 0.0.0.255 established
   permit tcp host 10.10.10.1 range 32766 32767 10.10.11.0 0.0.0.255 established

Is split it into 2 rules, because the set() on the port range 32766-32768 returns 2 items in https://github.com/gdelaney/aclhound/blob/master/aclhound/aclsemantics.py#L233

#51

'make test' currently failing locally

I don't know if this is an issue with my setup or not ...

After fixing my local test runs in #55, I'm getting failing tests. eg, snip of console output:

IOS

  Seed policy name: s2-internet-in
     IPv4:
     permit ip 37.77.58.0 0.0.0.63 any
     permit ip host 94.142.241.49 any
     permit ip host 94.142.241.204 any
     permit ip host 94.142.241.51 any
     permit ip host 94.142.241.52 any
     permit ip host 94.142.241.54 any
-    permit tcp host 94.142.241.49 any eq [(0, 1024)]
?                                       ^^^^ -     --
+    permit tcp host 94.142.241.49 any range 0 1024
?                                      ++++ ^
-    permit tcp host 94.142.241.204 any eq [(0, 1024)]
?                                        ^^^^ -     --
+    permit tcp host 94.142.241.204 any range 0 1024
?                                       ++++ ^
-    permit tcp host 94.142.241.51 any eq [(0, 1024)]
?                                       ^^^^ -     --
+    permit tcp host 94.142.241.51 any range 0 1024
?                                      ++++ ^

ASA:

  Seed policy name: s2-internet-in
     IPv4:
     access-list s2-internet-in-v4 extended permit ip 37.77.58.0 255.255.255.192 any
     access-list s2-internet-in-v4 extended permit ip host 94.142.241.49 any
     access-list s2-internet-in-v4 extended permit ip host 94.142.241.204 any
     access-list s2-internet-in-v4 extended permit ip host 94.142.241.51 any
     access-list s2-internet-in-v4 extended permit ip host 94.142.241.52 any
     access-list s2-internet-in-v4 extended permit ip host 94.142.241.54 any
-    access-list s2-internet-in-v4 extended permit tcp host 94.142.241.49 any eq [(0, 1024)]
?                                                                              ^^^^ -     --
+    access-list s2-internet-in-v4 extended permit tcp host 94.142.241.49 any range 0 1024
?                                                                             ++++ ^
-    access-list s2-internet-in-v4 extended permit tcp host 94.142.241.204 any eq [(0, 1024)]
?                                                                               ^^^^ -     --
+    access-list s2-internet-in-v4 extended permit tcp host 94.142.241.204 any range 0 1024
?                                                                              ++++ ^
-    access-list s2-internet-in-v4 extended permit tcp host 94.142.241.51 any eq [(0, 1024)]
?                                                                              ^^^^ -     --
+    access-list s2-internet-in-v4 extended permit tcp host 94.142.241.51 any range 0 1024
?                                                                             ++++ ^

Are the test files currently broken due to recent changes, or is there something wrong with my system?

Figure out syntax for ICMP type & code

Probably need an update to the EBNF
Use words instead of numbers? (To help with IPv4 / IPv6)

create 'aclhound test connectivity' feature

aclhound test connectivity or a similar command should check reachability & vendor type for all devices in the devices directory. Design it so it can run from cron

can't build/deploy with specific device only "all"

INFO: building configuration for 10.32.10.68
Traceback (most recent call last):
File "/usr/local/bin/aclhound", line 9, in
load_entry_point('aclhound==1.5', 'console_scripts', 'aclhound')()
File "build/bdist.linux-x86_64/egg/aclhound/cli.py", line 609, in main
File "build/bdist.linux-x86_64/egg/aclhound/cli.py", line 437, in build
File "build/bdist.linux-x86_64/egg/aclhound/cli.py", line 410, in go_build
IOError: [Errno 2] No such file or directory: '10.32.10.68'

Tests don't run out of the box due to missing /etc/aclhound/aclhound.conf

Setting up ACLHound locally following the config documentation - some deviations from the instructions actually, but should be ok:

$ git clone [email protected]:job/aclhound.git
$ cd aclhound
$ virtualenv venv
$ source venv/bin/activate
$ pip install -r requirements.txt
$ make test

snip some outputs, and get failures:

======================================================================
ERROR: test_01__build_ios (tests.test_regression.TestAclhound)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jzohrab/Documents/Projects/aclhound/tests/test_regression.py", line 68, in test_01__build_ios
    u'debug': False, u'jenkins': True})
  File "/Users/jzohrab/Documents/Projects/aclhound/aclhound/cli.py", line 215, in __init__
    self._settings = Settings()
  File "/Users/jzohrab/Documents/Projects/aclhound/aclhound/cli.py", line 82, in __init__
    sys.exit(2)
SystemExit: 2

======================================================================
ERROR: test_02__build_asa (tests.test_regression.TestAclhound)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jzohrab/Documents/Projects/aclhound/tests/test_regression.py", line 87, in test_02__build_asa
    u'debug': False, u'jenkins': True})
  File "/Users/jzohrab/Documents/Projects/aclhound/aclhound/cli.py", line 215, in __init__
    self._settings = Settings()
  File "/Users/jzohrab/Documents/Projects/aclhound/aclhound/cli.py", line 82, in __init__
    sys.exit(2)
SystemExit: 2

The failure is due to cli.py's lines:

        if not os.path.exists('/etc/aclhound/aclhound.conf'):
            print("ERROR: Could not open /etc/aclhound/aclhound.conf")
            print("Has ACLHound been properly installed? Contact your admin")
            sys.exit(2)

The fresh install creates /etc/aclhound/aclhound.conf.dist, but not /etc/aclhound/aclhound.conf.

If you give some notes on how this should be resolved here, I can fork and PR back to master. Should I just copy the .dist file to .conf?

can you implement scp to speedup the config upload process?

At this moment we have pretty big acls which takes a hour+ to upload them to all devices, is it possible to do this with scp instead of the ssh command line?

Thx!
-Ivo

Add rendering support for Cisco IOS

dynamic objects in a policy

Is it possible to have dynamic objects in a policy? (see example)
https://www.dropbox.com/s/oxb1xxlp5hgct73/Screenshot%202014-12-03%2009.53.54.png?dl=0

Not sure about ACLHound workflow

From the guide, it appears that I can use ACLHound to actually deploy ACLs, but I've seen notes around that imply that the changes are getting committed to a git repo or similar for code review before getting pushed out to the actual devices. That would be a very nice feature, allowing for a code review prior to configurations getting pushed out to devices.

Is there a doc, that I've missed, explaining such a workflow? The TODO has "Subcommands to be created - aclhound diff" etc, so maybe this is work-in-progress somewhere.

This may be quicker to discuss via skype or similar, if you are available and interested. I'm doing a comparison of a few ACL automation tools, and have assessed this and Google's Capirca.

Thanks, jz

Please make log statement on default deny configurable

Currently it's not configurable to have the last "deny any any" statement which is automatically included by the compiler to have it log. Can we fix this?

ACLhound always seems to execute the lockstep process

even if the ACL is not bound to any interface...

Flow Spec support

Hello, folks!

You have done nice work!

I have similar project but with another approach: https://github.com/pavel-odintsov/FlowACL I have tried to use unified flow spec for ACL management on switched and routers without native BGP Flow Spec support.

log statement in policies not working properly

according to job it's not working properly yet.

Please allow more options for port specifcations

Currently it seems the following is not supported:

greater then
less then
range

More then 1 policy include in the device file seems to fail

It looks like the secondary include line (and maybe 3rd/4th etc as well) is not working.

filename markplaats-ops-to-web referenced in markplaats-ops-to-web does not exist
HINT: ensure you are in your ACLHound data directory
mmoerman@aclhound001:/aclhound$ cd policy/
mmoerman@aclhound001:/aclhound/policy$ cat marktplaats-ops-to-web
allow tcp src @marktplaats-ops port any dst @marktplaats-web port 22
allow tcp src @marktplaats-ops port any dst @marktplaats-web port 80
allow tcp src @marktplaats-ops port any dst @marktplaats-web port 443
mmoerman@aclhound001:~/aclhound/policy$

remove git related commands

Users of ACLHound are expected to have in-depth knowledge of how git works. ACLHound cannot be a good git-wrapper so makes more sense to remove the task related commands

src port is missing the "eq" with the build

policy:
allow tcp src 10.32.102.100 port 25 dst 10.40.2.20 port 25

aclhound build:
permit tcp host 10.32.102.100 25 host 10.40.2.20 eq 25

Missing final deny any on ASA

A final deny any any for both IPv4 / IPv6 should be added when generating ASA output

save_config flag not working correctly

If the save_config flag is not specified, it doesn't default, and gives back an error

please add support for "established" rules

I added a rule:

dns

allow tcp src @src_servers port any dst @dst_dns port @dns_services stateful

with aclhound build I was expecting:
permit tcp host 1.1.1.1 host 2.2.2.2 eq 53 established

current output:
permit tcp host 1.1.1.1 host 2.2.2.2 eq 53

improve aclhound build performance

figure out what is going on here:

job@irime:~/aclhound$ strace -e open -f /usr/lib/pypy/../../local/bin/aclhound build devices/10.32.10.26 2>&1 | grep drac_services
open("objects/drac_services.ports", O_RDONLY) = 24
open("objects/drac_services.ports", O_RDONLY) = 36
open("objects/drac_services.ports", O_RDONLY) = 42
open("objects/drac_services.ports", O_RDONLY) = 34
open("objects/drac_services.ports", O_RDONLY) = 54
open("objects/drac_services.ports", O_RDONLY) = 30
open("objects/drac_services.ports", O_RDONLY) = 45
open("objects/drac_services.ports", O_RDONLY) = 41
open("objects/drac_services.ports", O_RDONLY) = 124
^C
job@irime:~/aclhound$

Allow different data directory when building/deploying

Currently , when using jenkins, data directory is /var/lib/jenkins/workspace/{$job-to-be-done}

There could be multiple aclhound jobs running, but they do need to run from the proper directory, so we'll need a parameter to manually override the data/work directory

Please allow to specify transport method

right now, it's not clear what is used as transport with a device to login.

Would be nice to have an extra line saying transport {telnet|ssh}

Detection of duplicate statements.

We were in a situation where duplicate statements/lines were configured in a policy, with a small policy this is easy to catch but by time policies are growing and therefore less evident to spot duplicates.

Can this be checked somewhere in the verification process ? Of course comments should be excluded from this. Not high priority.

expire is broken in current build

hanna:aclhound job$ aclhound build devices/91.211.74.75 | grep 1.2.34
Traceback (most recent call last):
  File "/usr/local/bin/aclhound", line 9, in <module>
    load_entry_point('aclhound==1.3', 'console_scripts', 'aclhound')()
  File "build/bdist.macosx-10.9-x86_64/egg/aclhound/cli.py", line 598, in main
  File "build/bdist.macosx-10.9-x86_64/egg/aclhound/cli.py", line 432, in build
  File "build/bdist.macosx-10.9-x86_64/egg/aclhound/cli.py", line 422, in go_build
  File "build/bdist.macosx-10.9-x86_64/egg/aclhound/generate.py", line 75, in generate_policy
  File "/usr/local/lib/python2.7/site-packages/grako/contexts.py", line 172, in parse
    result = rule()
  File "/usr/local/lib/python2.7/site-packages/grako/contexts.py", line 48, in wrapper
    return self._call(rule, name, params, kwparams)
  File "/usr/local/lib/python2.7/site-packages/grako/contexts.py", line 384, in _call
    node, newpos, newstate = self._invoke_rule(rule, name, params, kwparams)
  File "/usr/local/lib/python2.7/site-packages/grako/contexts.py", line 417, in _invoke_rule
    rule(self)
  File "build/bdist.macosx-10.9-x86_64/egg/aclhound/parser.py", line 43, in _start_
  File "/usr/local/lib/python2.7/site-packages/grako/contexts.py", line 547, in _check_eof
    self._error('Expecting end of text.')
  File "/usr/local/lib/python2.7/site-packages/grako/contexts.py", line 364, in _error
    item
grako.exceptions.FailedParse: (1:42) Expecting end of text. :
deny tcp src any dst 1.2.3.4 port 45 log expire 20131131
                                         ^
start

Add Arista support

Can we add arista support for ACLhound? Syntax is similar to Cisco.

Details for a virtual arista will be provided

Feature request: Threaded deploying

The process to upload new ACLs to devices takes quite some time. It might be handy to have deploying done through multiple threads, as it's not the deploying host that's taking up time, but it's the SSH/Telnet process taking up the time.

src port is missing the "eq" with the build for ASA

same as for IOS devices a few issues ago.

Add configuration flag to save config after deploy

Currently, configuration is not saved after a deploy. Can we make this a configurable parameter which I think need to default (when not specified) to automatically save the configuration once done with deploying?

not converting wrong ipv6 host ip into an ipv6 network

example:
ipv6 access-list test-in-v6 permit tcp 2001:630:440:400::1/64 any

asa output:
ipv6 access-list test-in-v6 line 1 permit tcp 2001:630:440:400::/64 any (hitcnt=0) 0x6fccbeda

Add Juniper rendering support

openstack support

CC @mgmoerman

Figure out how to conceptually translate openstack's security model to a filesystem hierarchy under devices

Should availability zones be represented as directories, with VMs as files containing policy includes?
Should neutron controllers be a directory with stuff underneath it?
....?

Feature request: Please add the possibility to add a timeout when save_config flag is set

This is a feature request. Right now, if you run aclhound and the save_config is set to true, when saving the config back to nvram on a cisco, and the config is big and it takes quite some time. Deploy will return not successfull, as it doesn't wait until the config is actually written. (Especially annoying if you run it from jenkins)

Cover most of the semantics

port ranges
invalid/valid IP addresses
etc