mpenning/ciscoconfparse

Parse, Audit, Query, Build, and Modify Cisco IOS-style configurations.

Hello 👋, I'm Mike Pennington a Network Engineer from Austin, TX.

• 💬 These are some things I'm interested in:
  • My Projects
    • CiscoConfParse2: parse / query Cisco (and other vendor) text configurations
  • Misc Tools I Like
    • pmlib
    • Taskwarrior: (also see Usage examples, and my taskwarrior notes)
    • rst2pdf_http: A tool to render and serve restructured text as pdf using a go http server
    • Graphing / Charting
      • matplotlib / rrdtool: the industry-standard time-series db and graphing library
      • go-charts and plot (also see plot examples)
    • pingtracer (Windows) / mtr (*nix) / nethogs (*nix) / gping (*nix) / sniffer (cross-platform)
    • mitmproxy / go-mitmproxy: Client-side socket logging and diagnostics
    • flent / crusader / goben / iperf3 / Cisco TRex
    • nsd / coredns / unbound / haproxy / frp: a reverse-proxy to expose services across a NAT
    • containerlab: Build networking labs with Go

ciscoconfparse's Issues

Cannot build a universal wheel

Because of the conditional in setup.py for IPADDR and DNSPYTHON it is not possible to build a universal wheel for this package.

ccp_abc.re_match_typed() returns "None" (as a string) if there is no match group assigned

re_match_typed() returns a "None" string when there is no match-group assigned; CiscoConfParse verson 1.1.12 and earlier are exposed to this bug.

from ciscoconfparse import CiscoConfParse
def test_failure():
    config = ['hostname TESTME']
    parse = CiscoConfParse(config, factory=True)
    obj = parse.find_objects('hostname')[0]
    return obj.re_match_typed(r'hostname\s(\d+)*', default='', group=1, result_type=str)

This happens because the match group had a * after it.

JunOS Config Comment Lines

After some testing it appears that the regex and line_level function break when you have a line in a JunOS config file. The first line of a config file will always start with, for example:

Last commit: 2015-05-18 15:07:43 BST by jbloggs

I have fixed the regex and added a check for the comment character - this appears to work fine:

I'll submit the changes in a minute if you are happy with them

Python wheel packages

Is it possible for you to compile down some python wheel packages for pypi?

Appending / modifying a parsed configuration is very slow; make it faster

A script like this, which adds interfaces to a pre-parsed configuration is very slow; this takes over a second to run with ciscoconfparse version 1.2.13 on Intel i5-2450P / python 2.7.3...

from ciscoconfparse import CiscoConfParse

parse = CiscoConfParse(['!'], factory=True)
for ii in range(0,100):
    parse.append_line('interface FastEthernet{0}'.format(ii))
    parse.append_line(' switchport')
    parse.append_line(' switchport mode access')
parse.commit()

for line in parse.ioscfg:
    print line

Running that script as do_it.py...

(py27_test)[mpenning@tsunami ~]$ time python do_it.py
....
interface FastEthernet99
 switchport
 switchport mode access

real    0m1.372s    <-------
user    0m1.344s
sys     0m0.020s
(py27_test)[mpenning@tsunami ~]$ 

ciscoconfparse with factory=True doesn't do what you want (yet)

It seems that the factory=True code in models_cisco.py and models_asa.py is becoming somewhat popular. These days, I am getting bugs filed against it as much as regular ciscoconfparse code.

Sadly, there is a lot of work to do on the factory=True methods, and I am not convinced that the factory model for configuration lines is the right approach yet; this doesn't mean I will rip it out, but it certainly does mean I'm not done...

Please heed the warning at the top of the source code, until I work out the right direction to take this effort. As such, the code in models_cisco.py and models_asa.py might change... significantly.

### HUGE UGLY WARNING:
###   Anything in models_cisco.py could change at any time, until I remove this
###   warning.  I have good reason to believe that these methods are stable and 
###   function correctly, but I've been wrong before.  There are no unit tests
###   for this functionality yet, so I consider all this code alpha quality. 
###
###   Use models_cisco.py at your own risk.  You have been warned :-)

All that said, I don't mind bugs filed against factory=True; I definitely want to know what to prioritize.

IPv6 Static Routes are Broken

After some head scratching I have found that when using factory=True the "_RE_IP_ROUTE" regex does not match IPv6. I get:

AttributeError: 'IOSRouteLine' object has no attribute 'route_info'

Because an (assumed) existing configuration file is being parsed then IOS should only allow valid entries and so I match both IPv4 and IPv6 myself with no validation.

Would this be the best way to fix this? If so I can modify the regex to cater for this.

save_as() adds multiple carriage returns (under windows)

save_as() adds multiple carriage returns (under windows)

banner motd breaks parsing

I may be doing something wrong, but from what I can tell, the following line is breaking the parse command: banner motd

It gives this error:

Traceback (most recent call last):
  File "parse.py", line 3, in <module>
    parse = CiscoConfParse('xxx')
  File "C:\Python27\lib\site-packages\ciscoconfparse\ciscoconfparse.py", line 164, in __init__
    CiscoConfParse=self)
  File "C:\Python27\lib\site-packages\ciscoconfparse\ciscoconfparse.py", line 2116, in __init__
    self._list = self._bootstrap_obj_init(data)
  File "C:\Python27\lib\site-packages\ciscoconfparse\ciscoconfparse.py", line 2401, in _bootstrap_obj_init
    self._banner_mark_regex(BANNER_RE)
  File "C:\Python27\lib\site-packages\ciscoconfparse\ciscoconfparse.py", line 2300, in _banner_mark_regex
    if bannerdelimit in obj.text.strip():
TypeError: 'in <string>' requires string as left operand, not NoneType

Junos vs IOS codebase compatibility

The codebase for Junos and IOS don't quite stay in sync (this is 100% understandable). A quick diff shows the missing parts.

sed -e 's/junos/cisco/g' -e 's/Junos/IOS/g' < ciscoconfparse/models_junos.py | diff ciscoconfparse/models_cisco.py - | more

Maybe some NotImplementedError additions could help point this out and mark the points where code needs to be written in order to support both platforms.

One other thing noted includes the self._address_family = "ipv6" variable being different in the two files.

sync_diff() includes double negatives in some cases

In CiscoConfParse 1.2.36 and lower, the following script will remove no ip redirects from the BASELINE config by including no no ip redirects in the diff.

from operator import attrgetter

from ciscoconfparse import CiscoConfParse


BASELINE = """!
interface GigabitEthernet0/1
 ip address 10.0.0.1 255.255.255.0
 no ip redirects
!""".splitlines()

REQUIRED_CONFIG = """!
interface GigabitEthernet0/1
 ip address 10.0.0.1 255.255.255.0
 no ip proxy-arp
 no ip unreachables
 arp timeout 240
!""".splitlines()

base = CiscoConfParse(BASELINE)
reqd = CiscoConfParse(REQUIRED_CONFIG)
base_parents = base.find_objects('^interface')
reqd_parents = reqd.find_objects('^interface')

for bobj in base_parents:
    for robj in reqd_parents:
        ## Simplistic example, which assumes the base and reqd lists match...
        if bobj.text==robj.text:
            bconf = map(attrgetter('text'), bobj.lineage)
            rconf = map(attrgetter('text'), robj.lineage)
            bparse = CiscoConfParse(bconf)
            diff = bparse.sync_diff(rconf, '')
            print "OBJ", bobj.text, diff

This is wrong; the diff should remove any no no case.

Fail to install ciscoconfparse on Windows 10

C:\Python27>python.exe Lib\site-packages\ciscoconfparse\setup.py
Traceback (most recent call last):
File "Lib\site-packages\ciscoconfparse\setup.py", line 14, in
from version_info.version import version as ccpversion
File "C:\Python27\Lib\site-packages\ciscoconfparse\version_info__init__.py", line 1, in
from .version import *
File "C:\Python27\Lib\site-packages\ciscoconfparse\version_info\version.py", line 1
../ciscoconfparse/version.py
^
SyntaxError: invalid syntax

Import issues on 3.4.3 with CiscoConfParse > 1.2.14.

CiscoConfParse 1.2.15 and greater appear to have issues importing correctly in Python 3.4.3 under Windows 7. Rolling back, 1.2.14 is the last version that appears to work as expected. Example below using 1.2.17.

Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:43:06) [MSC v.1600 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.

>>> from ciscoconfparse import CiscoConfParse
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    from ciscoconfparse import CiscoConfParse
ImportError: cannot import name 'CiscoConfParse'

>>> from ciscoconfparse import ciscoconfparse
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    from ciscoconfparse import ciscoconfparse
  File "C:\Users\user\virtuals\test_env\lib\site-packages\ciscoconfparse\ciscoconfparse.py", line 8, in <module>
    from models_cisco import IOSHostnameLine, IOSRouteLine, IOSIntfLine
ImportError: No module named 'models_cisco'

CiscoConfParse.append('config_line_here') is a null operation

Somehow I managed to break the append() method; the problem was discovered in version 1.2.1, but it certainly existed beforehand.

This bug tracks resolution of the problem.

Recursive ipaddr dependencies in ciscoconfparse breaks pip (version 1.2.11)

Installation fails due to a failure to find ipaddr:

vagrant@controller:~$ uname -a
Linux controller 3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
vagrant@controller:~$ pip --version
pip 6.0.8 from /usr/local/lib/python2.7/dist-packages (python 2.7)
vagrant@controller:~$ python --version
Python 2.7.6
vagrant@controller:~$ sudo pip install ciscoconfparse
The directory '/home/vagrant/.cache/pip/log' or its parent directory is not owned by the current user and the debug log has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want the -H flag.
The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want the -H flag.
The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want the -H flag.
Collecting ciscoconfparse
  Downloading ciscoconfparse-1.2.11.tar.gz (225kB)
    100% |################################| 225kB 457kB/s
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "/tmp/pip-build-QXc5qg/ciscoconfparse/setup.py", line 12, in <module>
        from ciscoconfparse.ciscoconfparse import __version__ as __ccpversion__
      File "ciscoconfparse/__init__.py", line 1, in <module>
        from ciscoconfparse import *
      File "ciscoconfparse/ciscoconfparse.py", line 8, in <module>
        from models_cisco import IOSHostnameLine, IOSRouteLine, IOSIntfLine
      File "ciscoconfparse/models_cisco.py", line 5, in <module>
        from ccp_abc import BaseCfgLine
      File "ciscoconfparse/ccp_abc.py", line 7, in <module>
        from ccp_util import IPv4Obj
      File "ciscoconfparse/ccp_util.py", line 14, in <module>
        from ipaddr import IPv4Network, IPv6Network, IPv4Address, IPv6Address
    ImportError: No module named ipaddr
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

      File "<string>", line 20, in <module>

      File "/tmp/pip-build-QXc5qg/ciscoconfparse/setup.py", line 12, in <module>

        from ciscoconfparse.ciscoconfparse import __version__ as __ccpversion__

      File "ciscoconfparse/__init__.py", line 1, in <module>

        from ciscoconfparse import *

      File "ciscoconfparse/ciscoconfparse.py", line 8, in <module>

        from models_cisco import IOSHostnameLine, IOSRouteLine, IOSIntfLine

      File "ciscoconfparse/models_cisco.py", line 5, in <module>

        from ccp_abc import BaseCfgLine

      File "ciscoconfparse/ccp_abc.py", line 7, in <module>

        from ccp_util import IPv4Obj

      File "ciscoconfparse/ccp_util.py", line 14, in <module>

        from ipaddr import IPv4Network, IPv6Network, IPv4Address, IPv6Address

    ImportError: No module named ipaddr

    ----------------------------------------
    Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-QXc5qg/ciscoconfparse

Looks to be associated with this commit:

bd0a8b1

Banners not being removed completed

I am using CiscoConfParse for parse a full config to a basic file (removing management lines). Sruggling for removing the banners. Lines are not being completed removed. I´m attaching the files and results for a better view. Wondering if that is a bug or if I am using that incorrect.

as it accepts only picts for attachments, have uploaded the code and txt files into:
https://github.com/fabiano-lima/ciscoconfparse_banner

Thank you !

Cannot use variable inside linespec

Hello,

I have tried to use this interesting libraries in conjunction with a string-variable inside the "linespec" argument, but it doesn't work.

For example:
1)
if_temp = '9/2'
linea = parse.find_children(r'._if_temp$') is NOT working, i.e. returns nothing inside variable linea
2)
linea = parse.find_children(r'._9/2$') is working, it returns what i requested.

That's quite a pity because I was trying to automate a long configuration migration task, but without this cannot move forward. I read all documentation but couldn't find anything like that.

Any workaround I could use or is it possible to add it to the enhancement requests?

Thank you.
Giulio.

CiscoConfParse does not accept unicode paths

Unicode paths are not accepted; this needs to be fixed.

(py27_test)[mpenning@tsunami fast]$ python
Python 2.7.3 (default, Jan  2 2013, 13:56:14) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ciscoconfparse import CiscoConfParse
>>> config = u'configs/sample_01.ios'
>>> parse = CiscoConfParse(config)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "ciscoconfparse/ciscoconfparse.py", line 199, in __init__
    " an invalid argument\n")
RuntimeError: [FATAL] CiscoConfParse() received an invalid argument

>>>

Need a way to find interface objects by their Cisco IOS abbreviation

We need something like this...

find_interface_objects('Se 1/0'), which should return the interface configuration object for "interface Serial 1/0". This would be very useful when finding interface objects that correspond to show ip int brief or snmpwalk output.

Banners with comment characters in them are not parsed correctly

Example, if an IOS banner has a ! in it, the current algorithm breaks. This problem can be seen when parsing the banner in sample_01.iosxr, among several problems line 27's parent is line 26 (another banner line):

<IOSCfgLine # 19 'banner login #'>
<IOSCfgLine # 20 '_____________________________________________________________________' (parent is # 19)>
<IOSCfgLine # 21 '                      -= FROG PARK SERVICES =-' (parent is # 19)>
<IOSCfgLine # 22 '               ||        ||         ' (parent is # 19)>
<IOSCfgLine # 23 '               ||        ||         ' (parent is # 19)>
<IOSCfgLine # 24 '              ||||      ||||       ' (parent is # 19)>
<IOSCfgLine # 25 '          ..:||||||:..:||||||:..   ' (parent is # 19)>
<IOSCfgLine # 26 '         C I S C O  S Y S T E M S    ' (parent is # 19)>
<IOSCfgLine # 27 '                   ! UNAUTHORISED ACCESS PROHIBITED !' (parent is # 26)>
<IOSCfgLine # 28 '     *************************************************************' (parent is # 20)>
<IOSCfgLine # 29 '     * This system is the property of Frog Park Services.    *' (parent is # 20)>
<IOSCfgLine # 30 '     * Disconnect IMMEDIATELY as you are not an authorized user! *' (parent is # 20)>
<IOSCfgLine # 31 '     *************************************************************' (parent is # 20)>
<IOSCfgLine # 32 '______________________________________________________________________'>

The fix properly parses this same banner...

<IOSCfgLine # 19 'banner login #'>
<IOSCfgLine # 20 '_____________________________________________________________________' (parent is # 19)>
<IOSCfgLine # 21 '                      -= FROG PARK SERVICES =-' (parent is # 19)>
<IOSCfgLine # 22 '               ||        ||         ' (parent is # 19)>
<IOSCfgLine # 23 '               ||        ||         ' (parent is # 19)>
<IOSCfgLine # 24 '              ||||      ||||       ' (parent is # 19)>
<IOSCfgLine # 25 '          ..:||||||:..:||||||:..   ' (parent is # 19)>
<IOSCfgLine # 26 '         C I S C O  S Y S T E M S    ' (parent is # 19)>
<IOSCfgLine # 27 '                   ! UNAUTHORISED ACCESS PROHIBITED !' (parent is # 19)>
<IOSCfgLine # 28 '     *************************************************************' (parent is # 19)>
<IOSCfgLine # 29 '     * This system is the property of Frog Park Services.    *' (parent is # 19)>
<IOSCfgLine # 30 '     * Disconnect IMMEDIATELY as you are not an authorized user! *' (parent is # 19)>
<IOSCfgLine # 31 '     *************************************************************' (parent is # 19)>
<IOSCfgLine # 32 '______________________________________________________________________' (parent is # 19)>
<IOSCfgLine # 33 '#' (parent is # 19)>
<IOSCfgLine # 34 '! LOGGING'>

Please add models_asa.py support for standby addresses

for example:
ip address 198.101.172.106 255.255.255.128 standby 198.101.172.107

i just changed that in my repo...
thanks for the good work!

Config parsing breaks with `ignore_blank_lines=True` (object parent assignments are missed)

I was struggling with an issue for a good while today why the first line of every section of my config was being truncated. Turns out it's really a fatal flaw in many of the routines. In many places there is for idx, obj in enumerate(self._list): where it is expected that idx == obj.linenum. Especially during __init__. I say this because many functions call iter_with_comments or iter_no_comments and pass in obj.linenum as the the begin_index parameter. Thus creating the relationship between list index and the config line number.

I'm submitting this as an issue instead of a pull request directly since I'm not sure of the intended use of the iter_*_comments functions (they are public api). So it is a conundrum on fixing this issue. Wasn't sure the best way to go about fixing it, but am willing to do so with some guidance on the matter.

Issue when parsing IOS-XR configs with more than 3 levels of config

Hey, I am having some issues parsing through some config to pull route targets out of IOS-XR config, after working with it for a while it seems like any config with 4 indents doesn't get evaluated.

Config:

Script:

Output:
node$ python vrf.py
['vrf test', ' address-family ipv4 unicast', ' import route-policy bgp-pref', ' import route-target', ' export route-target']

Let me know what you think, everything else I have done has worked great.
Thanks

Please improve Cisco IOS static route parsing (with factory=True)

Not sure if you are still improving the different line types, but if you are:

Below is a regexp that will match all the pieces of a static route in IOS, based on the command reference options:

^ip\s+route(?:\s+(?:vrf\s+(\S+)))?\s+(\d+\.\d+\.\d+\.\d+)\s+(\d+\.\d+\.\d+\.\d+)\s+(?:(\w\S+)(?:\s+(\d+\.\d+\.\d+\.\d+))?|(\d+\.\d+\.\d+\.\d+))(?:\s+(dhcp))?(?:\s+(global))?(?:\s+(\d+))?(?:\s+(multicast))?(?:\s+name\s+(\S+))?(?:\s+(?:(permanent)|(?:track\s+(\d+))))?(?:\s+tag(\d+))?

The match groups (if there is no match, returns None):
0: The whole line
1: Name of the VRF
2. Destination network
3. Destination netmask
4/5: Destination interface and optional nexthop
6: Destination nexthop if no destination interface
7: dhcp
8: global
9: Administrative distance
10: multicast
11: Route name
12: permanent
13. Tracking item
14: tag #

Python 2.6.6 (r266:84292, Jan 22 2014, 09:42:36)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>> p = re.compile(r'^ip\s+route(?:\s+(?:vrf\s+(\S+)))?\s+(\d+\.\d+\.\d+\.\d+)\s+(\d+\.\d+\.\d+\.\d+)\s+(?:(\w\S+)(?:\s+(\d+\.\d+\.\d+\.\d+))?|(\d+\.\d+\.\d+\.\d+))(?:\s+(dhcp))?(?:\s+(global))?(?:\s+(\d+))?(?:\s+(multicast))?(?:\s+name\s+(\S+))?(?:\s+(?:(permanent)|(?:track\s+(\d+))))?(?:\s+tag\s+(\d+))?')
>>> m = p.match("ip route vrf TESTING 1.1.1.1 255.255.255.255 TenGigabitEthernet1/1 4.4.4.4 dhcp global 120 multicast name testing track 12 tag 1")
>>> m.groups()
('TESTING', '1.1.1.1', '255.255.255.255', 'TenGigabitEthernet1/1', '4.4.4.4', None, 'dhcp', 'global', '120', 'multicast', 'testing', None, '12', '1')
>>> m = p.match("ip route 1.1.1.1 255.255.255.255 4.4.4.4 global 150 permanent tag 12")
>>> m.groups()
(None, '1.1.1.1', '255.255.255.255', '4.4.4.4', None, None, None, 'global', '150', None, None, 'permanent', None, '12')
>>>

find_blocks() performance degradation

the find_blocks() function is very slow in 0.9.24, probably since commit 8bb6dfb. It spends most of its time in deepcopy. The below code illustrates this problem:

import ciscoconfparse

lines = ['parent foo_{}'.format(i/5) if (i%5)==0
         else '  child bar_{}_{}'.format((i/5)%200, i%5)
         for i in range(0,5000)]

conf = ciscoconfparse.CiscoConfParse(lines)

for x in range(1,100):
    conf.find_blocks('bar_{}_3'.format(x))

WIth 0.9.24, profiling output (python -m cProfile -s cumulative) looks like:

         65371636 function calls (51110623 primitive calls) in 30.843 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.003    0.003   30.843   30.843 block.py:3(<module>)
       99    0.438    0.004   30.154    0.305 ciscoconfparse.py:347(find_blocks)
12280455/99    9.584    0.000   28.905    0.292 copy.py:145(deepcopy)
495297/99    1.932    0.000   28.903    0.292 copy.py:306(_reconstruct)
495297/99    3.874    0.000   28.898    0.292 copy.py:234(_deepcopy_tuple)
495198/99    0.735    0.000   28.897    0.292 copy.py:226(_deepcopy_list)
495198/99    3.147    0.000   28.889    0.292 copy.py:253(_deepcopy_dict)
 15783955    2.108    0.000    2.108    0.000 {method 'get' of 'dict' objects}
   495297    1.991    0.000    1.991    0.000 {method '__reduce_ex__' of 'object' objects}
  2973069    1.441    0.000    1.984    0.000 copy.py:267(_keep_alive)
 17234613    1.250    0.000    1.250    0.000 {id}
       99    0.001    0.000    0.744    0.008 ciscoconfparse.py:1162(_find_line_OBJ)
       99    0.125    0.001    0.731    0.007 {filter}

With an older version (0.9.5), I get:

         2345239 function calls (2345173 primitive calls) in 1.005 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.003    0.003    1.005    1.005 block.py:3(<module>)
       99    0.004    0.000    0.833    0.008 ciscoconfparse.py:549(find_blocks)
       99    0.222    0.002    0.819    0.008 ciscoconfparse.py:1163(_find_line_OBJ)
   547000    0.231    0.000    0.674    0.000 re.py:139(search)
   547000    0.229    0.000    0.313    0.000 re.py:226(_compile)
        1    0.000    0.000    0.167    0.167 ciscoconfparse.py:70(__init__)
        1    0.017    0.017    0.167    0.167 ciscoconfparse.py:102(_parse)
   547000    0.130    0.000    0.130    0.000 {method 'search' of '_sre.SRE_Pattern' objects}
        7    0.026    0.004    0.077    0.011 ciscoconfparse.py:184(_mark_banner)
   547036    0.072    0.000    0.072    0.000 {method 'get' of 'dict' objects}

So in this case it is a factor of 30.

Traceback while using ciscoconfparse/local_py/ipaddr.py

Some users have reported problems when parsing with CiscoConfParse('a-config-here.conf', factory=True)... they receive the following traceback...

NOTE: This problem only happens when parsing with factory=True...

  File "/opt/virtual_env/py27_default/local/lib/python2.7/site-packages/ciscoconfparse/ciscoconfparse.py", line 2126, in ConfigLineFactory
    comment_delimiter=comment_delimiter) # instance of the proper subclass
  File "/opt/virtual_env/py27_default/local/lib/python2.7/site-packages/ciscoconfparse/models_cisco.py", line 766, in __init__
    super(IOSIntfLine, self).__init__(*args, **kwargs)
  File "/opt/virtual_env/py27_default/local/lib/python2.7/site-packages/ciscoconfparse/models_cisco.py", line 60, in __init__
    self.default_ipv4_addr_object = IPv4Network('127.0.0.1/32')
  File "/opt/virtual_env/py27_default/local/lib/python2.7/site-packages/ciscoconfparse/local_py/ipaddr.py", line 1357, in __init__
    self.netmask = IPv4Address(self._ip_int_from_prefix(self._prefixlen))
  File "/opt/virtual_env/py27_default/local/lib/python2.7/site-packages/ciscoconfparse/local_py/ipaddr.py", line 1266, in __init__
    self._ip = self._ip_int_from_string(addr_str)
  File "/opt/virtual_env/py27_default/local/lib/python2.7/site-packages/ciscoconfparse/local_py/ipaddr.py", line 1096, in _ip_int_from_string
    raise AddressValueError(ip_str)
ipaddr.AddressValueError: 4294967295
(py27_default)mpenning@Mudslide:~/tmp/$

banner delimiter issue

banner configs generated by IOS for show differ from those generated by copy

show generates a "^" followed by a "C" as the delimiter and inserts a comment before the next non-banner lines.

copy generates an actual ^C (\x03) as the delimiter and does not insert a comment after the end of the banner

0.9.30 handles show-style output but does not assign banner lines to their parent on copy-style output

See below to reproduce. Possible patch

$ cat tmp-confg.py
#!/usr/bin/env python
import sys
sys.path[0] = "/tmp/ciscoconfparse/ciscoconfparse"
import ciscoconfparse
v = ciscoconfparse.__version__
p = ciscoconfparse.CiscoConfParse(sys.argv[1],linesplit_rgx="[\r\n]+", ignore_blank_lines=False)
for o in p.ConfigObjs: print "("+v+") ", o.parent.linenum, ": ", o

$ jobs
[1]+  Stopped                 ssh 169.254.0.2

$ fg %1
router#sh run | redirect tftp://169.254.0.1/rtr.show
!
router#~^Z [suspend ssh]

$ ####### process config generated with "show run" with 0.9.30
$ ####### banner lines assigned to parent
$ python ./tmp-confg.py rtr.show | sed -n '/banner /,/line con/p'
(0.9.30)  380 :  <IOSCfgLine # 380 'banner login ^C'>
(0.9.30)  380 :  <IOSCfgLine # 381 'Github, Incorporated           Unauthorized Access Prohibited' (parent is # 380)>
(0.9.30)  380 :  <IOSCfgLine # 382 '=============================================================' (parent is # 380)>
(0.9.30)  380 :  <IOSCfgLine # 383 '^C' (parent is # 380)>
(0.9.30)  384 :  <IOSCfgLine # 384 '!'>
(0.9.30)  385 :  <IOSCfgLine # 385 'line con 0'>

$ fg %1
router#
router#copy run tftp://169.254.0.1/rtr.copy
Address or name of remote host [169.254.0.1]?
Destination filename [rtr.copy]?
!!
11937 bytes copied in 1.952 secs (6115 bytes/sec)
router#~^Z [suspend ssh]

$ ####### process config generated with "copy run tftp" with 0.9.30
$ ####### banner lines not assigned to parent
$ python ./tmp-confg.py rtr.copy | sed -n '/banner /,/line con/p'
(0.9.30)  379 :  <IOSCfgLine # 379 'banner login '>
(0.9.30)  380 :  <IOSCfgLine # 380 'Github, Incorporated           Unauthorized Access Prohibited'>
(0.9.30)  381 :  <IOSCfgLine # 381 '============================================================='>
(0.9.30)  382 :  <IOSCfgLine # 382 ''>
(0.9.30)  383 :  <IOSCfgLine # 383 '!'>
(0.9.30)  384 :  <IOSCfgLine # 384 'line con 0'>

VLAN parsing/diffing of "blobs" of config

(As asked via mail, here's the issue)

Love your python package, but I'm struggling to work out how I can achieve something.

Put simply, I want to sync VLAN configs across my devices. I want to use one device as the seed or master, and generate diffs to the rest.

Lines such as:

    vlan 51
     name SOME-VLAN
    vlan 52
     name BLAH

I've tried using req_cfgspec_all_diff, but this goes line by line. I need those lines above I guess to be treated as a two line object instead, because a device may have the vlan 51 line, but be missing the name... or extraneous vlan definitions may exist at a destination that I want to remove.

In essence, rsync for vlans ;)

models_asa is broken in Python3.2

For now, I am removing Python3.2 from the Travis CI config...

(py32_test)[mpenning@tsunami fast]$ make test
cd ciscoconfparse; python ciscoconfparse.py; ./runtests.sh
................................
----------------------------------------------------------------------
Ran 32 tests in 0.061s

OK
.....FFF
======================================================================
FAIL: testL4Object_asa_lt01 (__main__.knownValues)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_Ccp_Util.py", line 56, in testL4Object_asa_lt01
    self.assertEqual(pp.port_list, range(1, 7))
AssertionError: range(1, 7) != range(1, 7)

======================================================================
FAIL: testL4Object_asa_lt02 (__main__.knownValues)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_Ccp_Util.py", line 61, in testL4Object_asa_lt02
    self.assertEqual(pp.port_list, range(1, 7))
AssertionError: range(1, 7) != range(1, 7)

======================================================================
FAIL: testL4Object_asa_range01 (__main__.knownValues)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_Ccp_Util.py", line 51, in testL4Object_asa_range01
    self.assertEqual(pp.port_list, range(25, 33))
AssertionError: range(25, 33) != range(25, 33)

----------------------------------------------------------------------
Ran 8 tests in 0.002s

FAILED (failures=3)
..........................................
----------------------------------------------------------------------
Ran 42 tests in 0.209s

OK
...F.
======================================================================
FAIL: testVal_object_group_service_02 (__main__.knownValues)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_Models_Asa.py", line 172, in testVal_object_group_service_02
    self.assertEqual(obj.ports, result_correct)
AssertionError: Lists differ: [<L4Object tcp [25]>, <L4Objec... != [<L4Object tcp [25]>, <L4Objec...

First differing element 2:
<L4Object tcp range(8080, 8082)>
<L4Object tcp range(8080, 8082)>

  [<L4Object tcp [25]>, <L4Object tcp [443]>, <L4Object tcp range(8080, 8082)>]

----------------------------------------------------------------------
Ran 5 tests in 0.008s

FAILED (failures=1)
make: *** [test] Error 1
(py32_test)[mpenning@tsunami fast]$

Possible to check if startup and running are the same

Hi, Is it possible to check if the startup and running config are the same? If possible how would you go about doing it?

Incorrect results on find_children & find_all_children

Hi,
I am parsing a config file and I can see that the results are varying and not consistent.

To begin with, my test config file is

interface VlanAlpha3

interface VlanAlpha2

interface VlanAlpha1

interface VlanAlpha
  vrf member alpha01
  some config1 
  ip address 100.100.102.0/24
  some config2
  blah blah blah

interface VlanAlpha32

interface VlanAlpha22

interface VlanAlpha12

interface VlanAlpha2
  vrf member alpha012
  some config1 
  ip address 100.100.101.0/24
  some config2
  blah blah blah

interface VlanAlpha31

interface VlanAlpha21

interface VlanAlpha11

interface VlanAlpha1
  vrf member alpha011
  no shutdown
  ip flow monitor netflow-monitor input  
    no ip redirects
    ip address 100.100.100.0/24
    ip ospf passive-interface
    ip router ospf 1 area 0.0.0.3
  ip pim sparse-mode
  hsrp version 2
  hsrp 224 

With the following code, I get an output, which seems incorrect.

from ciscoconfparse import CiscoConfParse
parseObject = CiscoConfParse("testconfig1")
allChildren = parseObject.find_all_children('interface VlanAlpha1')
print allChildren

 ['interface VlanAlpha1', 'interface VlanAlpha12', 'interface VlanAlpha11', 'interface ?VlanAlpha1', '  vrf member alpha011']

Shouldnt this list all the child nodes of Interface VlanAlpha1
I am using Python v 2.7.5 and CiscoConfParse v 1.1.1

ImportError: No module named 'version_info.version' Installing into Pycharm

I'm receiving an error attempting to install ciscoconfparse 1.2.37 into Pycharm. Details:
Pycharm Porject Interpreter 3.4.3
pycharm-->project-->settings-->my project-->project interpreter-->green + to install-->select ciscoconfparse-->click install package to dir c:\users\username\AppData\Roaming\Python-->receive ImportError: No module named 'version_info.version' error

this is on WIN7 64-bit SP1

thanks

markyjackson5/Cisco is a Copyright and GPL violation of mpenning/ciscoconfparse

In May 2014, I filed a bug against a github repository named markyjackson/Cisco as a GPL and Copyright violation of my repositories:

• bitbucket.org/mpenning/ciscoconfparse
• github.com/mpenning/ciscoconfparse
• sourceforge/ciscoconfparse

To be exact, markyjackson duplicated the commit for the comment delimiter feature on 2010-01-18, and claimed himself as the sole author of ciscoconfparse (see bottom of this github issue for a screenshot). I am speculating, but perhaps Mr Jackson cherry-picked this commit because it was one where I commited to ciscoconfparse as the "django" username, instead of my username (which shows up as myself in the repo).

A mobile screenshot of github/markyjackson, taken May 5th, 2014:

markyjackson is also known as Mark Raymond Jackson (Twitter: @mrayjackson5):

My bug pointed out that there were obvious problems with Mr Jackson claiming sole authorship of my previous commits; however, I had no problem with him forking my repository as long as he gave me proper credit for the source code. This bug was never acknowledged by Mark Jackson on github; however, on June 27th, 2014, I received the following anonymous email, stating that this was a "mistake" while trying to contribute to my repository. The github/markyjackson account was subsequently deleted.

However, Mark Jackson created another identity as github/markyjackson5 on August 18, 2014; markyjackson5's repositories include the same content as the original "github/markyjackson":

To cite a specific example, when markyjackson5 when uploaded Cisco/cisco_parse_test.py, he neglected to change his hyperlinks in other parts of that repo, which pointed to the original markyjackson github account. Search for "markyjackson" in this file: markyjackson5/Cisco/rtrdef-master/README.md.

---

Other Copyright violations in markyjackson5/Cisco

As a side note:

1. markyjackson5/Cisco/nxconfig is a Copyright violation of tonhe/nxconfig-script/nxconfig
2. markyjackson5/Cisco/Cisco-Nexus-Configurator-master is a copyright violation of colinmcnamara/Cisco-Nexus-Configurator.
3. markyjackson5/Cisco/rtrdef-master mentioned above could be another copyright violation of pello/routerdefense, see pello/routerdefense/README.md. routerdefense is licensed Apache 2.0 on google code.

---

This is a screenshot of markyjackson5's infringing ciscoconfparse material:

Failed install of 1.2.40 with Python 3.5.2

I've tried to install this package with both 3.5.2 and 2.7 without success using PIP. I've also tried to load an older version, 1.2.38 but it also returns the same error.
Python is new for me, so I'm sure I am probably missing something. This is running on a window7 box with cygwin.

This is what I get:

$ pip3 install --upgrade ciscoconfparse
Collecting ciscoconfparse
Using cached ciscoconfparse-1.2.40.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "C:\cygwin64\tmp\pip-build-q270j8sh\ciscoconfparse\setup.py", line 14, in
from version_info.version import version as ccpversion
File "c:\cygwin64\tmp\pip-build-q270j8sh\ciscoconfparse\version_info__init__.py", line 1, in
from .version import *
ImportError: No module named 'version_info.version'

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in C:\cygwin64\tmp\pip-build-q270j8sh\ciscoconfparse\

Can anyone offer any help in resolving this install problem?

Thank you,
Paul

Cisco IOS Banner issue when run CiscoConfParse

when running CiscoConfParse for IOS configuration which included banner settings. The CiscoConfParse will show error. Attached testing config and error output. Thank you!

=== Python 2.7.5 ===
from ciscoconfparse import CiscoConfParse

p = CiscoConfParse("config.txt")

=== Tested config ===
!
spanning-tree mode pvst
spanning-tree extend system-id
no spanning-tree vlan 450-451
!
!
banner motd ^CCCC

---

** ** UNAUTHORIZED ACCESS PROHIBITED ** **

---

** ** UNAUTHORIZED USE PROHIBITED ** **

---

---

---

^C
!
!
end

======= Error =======
Traceback (most recent call last):
File "C:/My Projects/test1/t1.py", line 4, in
p = CiscoConfParse("config.txt")
File "C:\Python27\lib\site-packages\ciscoconfparse\ciscoconfparse.py", line 125, in init
ignore_blank_lines=ignore_blank_lines)
File "C:\Python27\lib\site-packages\ciscoconfparse\ciscoconfparse.py", line 1278, in init
self._bootstrap_obj_init(data)
File "C:\Python27\lib\site-packages\ciscoconfparse\ciscoconfparse.py", line 1419, in _bootstrap_obj_init
self.maintain_obj_sanity()
File "C:\Python27\lib\site-packages\ciscoconfparse\ciscoconfparse.py", line 1432, in maintain_obj_sanity
self._mark_banner("motd", "ios")
File "C:\Python27\lib\site-packages\ciscoconfparse\ciscoconfparse.py", line 1547, in _mark_banner
if (DBGFLAG is True):
NameError: global name 'DBGFLAG' is not defined
Process finished with exit code 1

Relative module import name conflict

I'd love to be able to use this module, but I have an unresolvable name conflict with an existing dns module. ciscoconfparse uses relative imports, and since my app is higher on the PYTHONPATH this means my dns module is being loaded in preference to the ciscoconfparse internal one.

>>> print "\n".join(sys.path)
/home/ben/projects/hdb/hdb
/usr/lib/entropy/lib
/home/ben/.virtualenvs/hdb/lib64/python27.zip
/home/ben/.virtualenvs/hdb/lib64/python2.7
/home/ben/.virtualenvs/hdb/lib64/python2.7/plat-linux2
/home/ben/.virtualenvs/hdb/lib64/python2.7/lib-tk
/home/ben/.virtualenvs/hdb/lib64/python2.7/lib-old
/home/ben/.virtualenvs/hdb/lib64/python2.7/lib-dynload
/usr/lib64/python2.7
/usr/lib/python2.7/plat-linux2
/usr/lib64/python2.7/lib-tk
/usr/lib/python2.7/lib-tk
/home/ben/.virtualenvs/hdb/lib/python2.7/site-packages

>>> from ciscoconfparse import CiscoConfParse
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/ben/.virtualenvs/hdb/lib/python2.7/site-packages/ciscoconfparse/__init__.py", line 1, in <module>
    from ciscoconfparse import *
  File "/home/ben/.virtualenvs/hdb/lib/python2.7/site-packages/ciscoconfparse/ciscoconfparse.py", line 11, in <module>
    from models_cisco import IOSHostnameLine, IOSRouteLine, IOSIntfLine
  File "/home/ben/.virtualenvs/hdb/lib/python2.7/site-packages/ciscoconfparse/models_cisco.py", line 5, in <module>
    from ccp_util import _IPV6_REGEX_STR_COMPRESSED1, _IPV6_REGEX_STR_COMPRESSED2
  File "/home/ben/.virtualenvs/hdb/lib/python2.7/site-packages/ciscoconfparse/ccp_util.py", line 8, in <module>
    from dns.exception import DNSException
ImportError: No module named exception

If ciscoconfparse used absolute imports instead, I think this problem could be avoided. Short of renaming the local module in my application (not simple), I don't think there's any way I can work around it without code changes in ciscoconfparse.

Empty requirements file

Hi !

Can you fill out the requirements file please ? :)
For this time, I'm going to install all dependencies one by one, but it would be nice to freeze your virtualenv for the next contributor.
I can push mine then, but I think it would be more convenient you push yours.

Thanks !

Make Junos-style configs searchable

Some people want ciscoconfparse to search Junos configurations... this bug tracks implementation of the feature request.

decrypt() fails for long type 7 password hashes

In all versions of ciscoconfparse, calling the decrypt() function does not correctly return the password when the input hash is long. Ruchir Jain (rucjain) of Cisco Systems emailed me and pointed out the correct way to decrypt() long type 7 password hashes.

does not install properly for version 1.1.1 with python 3.4

Using Linux Mint 17, the command sudo python3 setup.py install initially gives an error with print diff in excldiff_test.py. This was fixed by wrapping diff in parenthesis.

After that, it installed the package in /usr/local/lib/python3.4/dist-packages/ciscoconfparse-1.1.1-py3.4.egg/ciscoconfparse rather than /usr/local/lib/python3.4/dist-packages/ciscoconfparse. This made the need for from ciscoconfparse import CiscoConfParse as show in tutorials to become from ciscoconfparse.ciscoconfparse import CiscoConfParse. I don't know if this is a bug or intended behavior and the documentation hasn't been updated. I temprarily fixed this by moving the ciscoconfparse directory up a level, and making a change to easy-install.pth. I'm pretty new to python, but I installed the way that is shown in the tutorial. I also get the same results using pip3.

Error in parsing F5 configuration. (junos parser)

Hi,

I think you have a issue when open and close curly brackets are at the same line.
When I parse

ltm virtual virtual1 {
    profiles {
        test1 { }
    }
}
ltm virtual virtual2 {
    profiles2 {
        test2 { }
    }
}

profiles2 don't have any parent

IOSConfigLine.delete() can delete too much

As of CiscoConfParse version 1.2.18, calling delete() on an IOSConfigLine() can delete too many config lines... this is an example....

from ciscoconfparse import CiscoConfParse
CONFIG = ['!',
    'interface GigabitEthernet0/0',
    ' ip address 10.0.0.1 255.255.255.0',
    'banner motd ^', '    trivial banner1 here ^',   ## Deleting this banner line fails
    'interface GigabitEthernet0/1',
    ' ip address 192.0.2.1 255.255.255.0',
    'banner exec ^', '    trivial banner2 here ^',
    'end'] 

parse = CiscoConfParse(CONFIG)
banner = parse.find_objects('banner motd')[0]
banner.delete()
parse.atomic()

The code above fails with:

Traceback (most recent call last):
  File "banner.py", line 18, in <module>
    banner.delete()
  File "/home/mpenning/fail/ciscoconfparse/ccp_abc.py", line 198, in delete
    child.delete()
  File "/home/mpenning/fail/ciscoconfparse/ccp_abc.py", line 199, in delete
    del self.confobj._list[self.linenum]
IndexError: list assignment index out of range

don't use mutable types in definitions

def _bootstrap_obj_init(self, text_list=[]): should be def _bootstrap_obj_init(self, test_list)

http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments

Also, there doesn't seem to be a need to default text_list to None since it's expecting that data anyway.

Cisco IOS banner endpoint detection is broken

Assume we have a banner like this...

banner login ^C
Go away, we don't like you.
Log off now while you still can type. I break the fingers
of all tresspassers.
^C
alias exec showthang show ip route vrf THANG

As of ciscoconfparse version 1.2.10 (and earlier versions), the alias command is included in the banner's children; that's not correct.

Feature Request: Return boolean on successful save

While it may be a trivial thing and possible to check locally, it would be helpful if the CiscoConfParse.save_as method were to confirm that the file save was successful and return a boolean.

    def save_as(self, filepath):
        """Save a text copy of the configuration at ``filepath``; this
        method uses the OperatingSystem's native line separators (such as
        ``\\r\\n`` in Windows).

        Returns a boolean if writing to file was successful or not"""
        try:
            with open(filepath, 'w') as newconf:
                for line in self.ioscfg:
                    newconf.write(line+'\n')
            return True
        except IOError:
            return False

Perhaps returning the exception could be an option for debugging, so if disk space were full or insufficient permissions occurred, the user could troubleshoot and correct the issue?

Add CiscoConfParse support for Python generators

Based on discussions with users, yielding from a generator as input to CiscoConfParse would be useful in case you need to dynamically modify the configuration as it is input to CiscoConfParse... simple example follows...

import re

def fix_brocade_bug(config_file):
    with open('/var/local/RANCID/configs/CORE.02.DEN.conf') as fh:
        for line in fh.read().splitlines():
            #  Fix brocade configuration indent bug
            line = re.sub('^\s{3,}(address-family.+)', '  \<g>1', line)
            yield line

config_generator = fix_brocade_bug('/var/local/RANCID/configs/CORE.02.DEN.conf')
parse = CiscoConfParse(config_generator)

Currently generators are not supported; a minor change would enable generator input.

CiscoConfParse slows by roughly O(n**2), where n is number of config lines

Attempt to improve algorithm to scale more gracefully as n increases

Unable to install ciscoconfparse 1.2.23 into Python 3.4.3 Windows

Hi,

Not sure did anyone have this issue...I was not able to install 1.2.23 onto my Windows Python 3.4.3...It give any error 'ImportError: No module named 'version_info.version'....

Can I know what is causes it?

C:>python
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:43:06) [MSC v.1600 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

^Z

C:>pip install --upgrade ciscoconfparse
Collecting ciscoconfparse
Using cached ciscoconfparse-1.2.23.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 20, in
File "D:\temp\pip-build-k6qnoy2c\ciscoconfparse\setup.py", line 14, in
from version_info.version import version as ccpversion
File "D:\temp\pip-build-k6qnoy2c\ciscoconfparse\version_info__init__.py", line 1, in
from .version import *
ImportError: No module named 'version_info.version'

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in D:\temp\pip-build-k6qnoy2c\ciscoconfparse

ValueError: Could not parse 'ipv6 router ospf 6'

Hi,

ciscoconfparse-1.2.37-py2.7.egg

can't parse

ipv6 router ospf 6

in IOS configuration.

ciscoconfparse 1.2.11 & 1.2.12 won't install in Python3

(py34_test)[mpenning@tsunami this]$ pip3 install --upgrade ciscoconfparse
Downloading/unpacking ciscoconfparse from https://pypi.python.org/packages/source/c/ciscoconfparse/ciscoconfparse-1.2.12.tar.gz#md5=be0298e2d01471921b23c5860640fba4
  Downloading ciscoconfparse-1.2.12.tar.gz (225kB): 225kB downloaded
  Running setup.py (path:/opt/virtual_env/py34_test/build/ciscoconfparse/setup.py) egg_info for package ciscoconfparse
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/opt/virtual_env/py34_test/build/ciscoconfparse/setup.py", line 14, in <module>
        from version_info.version import __version__ as __ccpversion__
      File "/opt/virtual_env/py34_test/build/ciscoconfparse/version_info/__init__.py", line 1, in <module>
        from version import *
    ImportError: No module named 'version'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/opt/virtual_env/py34_test/build/ciscoconfparse/setup.py", line 14, in <module>

    from version_info.version import __version__ as __ccpversion__

  File "/opt/virtual_env/py34_test/build/ciscoconfparse/version_info/__init__.py", line 1, in <module>

    from version import *

ImportError: No module named 'version'

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /opt/virtual_env/py34_test/build/ciscoconfparse
Storing debug log for failure in /home/mpenning/.pip/pip.log
(py34_test)[mpenning@tsunami this]$