spotify / pyfg Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
Hello guys,
I recently tried to use pyFG to config Fortigate (5.4.x firmware). It is good for the "config" and "edit" cmd.
However, when I trying to use "move" cmd under "config firewall policy" to re-order a policy, It is found that the compare_config() method cannot return the changed config.
As a result the diff cannot be committed.
Below are the cmds that I would like to execute, to move the policy id 611 after id 999.
'move_policy.txt'
config firewall policy
move 611 after 999
end
This is the script that I modify from "https://github.com/spotify/pyfg/blob/master/examples/example5.py" to execute the above move_policy.txt to the 'vpn' vdom
#!/usr/bin/env python
# Gets router bgp config from the device, then do some changes to the BGP parameters, deletes a neighbor,
# creates a new one, modifies another and computes the difference
from pyFG import FortiOS
import logging
host = '1.1.1.1'
vdom = 'vpn'
cmdfile = 'move_policy.txt'
user = 'admin'
passwd = 'password'
if __name__ == '__main__':
f = open(cmdfile, 'r')
candidate = f.read()
f.close()
print "*** This is the candidate configs:"
print candidate
print "\n"
d = FortiOS(hostname=host, vdom=vdom, username=user, password=passwd)
d.open()
d.load_config(config_text=candidate, in_candidate=True)
print "*** This is the diff of the conigs: (compare_config(text=True))"
print (d.compare_config(text=True))
print "\n"
print "*** This is how to reach the desired state: (compare_config())"
config_changes = d.compare_config()
print config_changes
print "*** Result of applying the changes: (d.commit)"
print d.commit(config_changes, force=True)
d.close()
This is the output of the result,
the "move" cmd cannot be compared such that nothing can be committed.
The move 611 after 999
cmd cannot found in diff
*** This is the candidate configs:
config firewall policy
move 611 after 999
end
*** This is the diff of the configs: (compare_config(text=True))
+ config firewall policy
+ end
*** This is how to reach the desired state: (compare_config())
*** Result of applying the changes: (d.commit)
None
Process finished with exit code 0
Are are any suggestion to execute "move" command in firewall policy using PyFG?
thank you.
I am trying to install pyfg
using pipenv
(python 3.9) but the command fails with the following error:
>pipenv install pyfg
Installing pyfg…
Error: An error occurred while installing pyfg!
Error text: Collecting pyfg
Using cached pyfg-0.50.tar.gz (9.1 kB)
ERROR: Command errored out with exit status 1:
command: /Users/vikrant.tyagi/.virtualenvs/fortios-automation-uy-maohS/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/kh/m16w2sgn10jd06jfxd58wl_c_2h4vh/T/pip-install-_8rwn0my/pyfg/setup.py'"'"'; __file__='"'"'/private/var/folders/kh/m16w2sgn10jd06jfxd58wl_c_2h4vh/T/pip-install-_8rwn0my/pyfg/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/kh/m16w2sgn10jd06jfxd58wl_c_2h4vh/T/pip-pip-egg-info-c6kf6k4d
cwd: /private/var/folders/kh/m16w2sgn10jd06jfxd58wl_c_2h4vh/T/pip-install-_8rwn0my/pyfg/
Complete output (5 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/kh/m16w2sgn10jd06jfxd58wl_c_2h4vh/T/pip-install-_8rwn0my/pyfg/setup.py", line 4, in <module>
from pip.req import parse_requirements
ModuleNotFoundError: No module named 'pip.req'
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
This is likely caused by a bug in pyfg. Report this to its maintainers.
✘ Installation Failed
If the configuration contains the character '#' the script will fail with the following traceback:
Traceback (most recent call last):
File "./pyFG-test.py", line 16, in <module>
d.load_config(sys.argv[3])
File "build/bdist.linux-x86_64/egg/pyFG/fortios.py", line 215, in load_config
File "build/bdist.linux-x86_64/egg/pyFG/forticonfig.py", line 377, in parse_config_output
AttributeError: 'NoneType' object has no attribute 'get_block_names'
It seems that the character '#' is considered harmful for most parts of the configuration, usually trying to use it fails in the following way:
# config firewall address
(address) # edit "test#"
The string contains XSS vulnerability characters
value parse error before 'test#'
Command fail. Return code -173
At least vpn ssl web user-bookmark
allows the character:
# config vpn ssl web user-bookmark
(user-bookmark) # edit "test#"
new entry 'test#' added
(test#) # end
show vpn ssl web user-bookmark
config vpn ssl web user-bookmark
edit "test#"
next
end
I actually see no reason why that character is part of the configuration and I will most likely remove it. There also is a chance of this being a FortiOS issue in case this character should not be allowed at all in any part of the configuration. Maybe you can and want to fix this, otherwise it is at least documented.
Hi there!
First of all, thank you so much for this awesome API you guys nailed.
I've got a question. Is there any way this API could be used for configuration files validation? I have this project in which I need to make sure no invalid FortiGate configuration file is uploaded (that is, sections without an 'end' or 'next', weird characters...).
Lib is not working with python 2.6 & centos 6:
from pyFG import py23_compat
File "/usr/lib/python2.6/site-packages/pyFG/py23_compat.py", line 7
PY2 = sys.version_info.major == 2
AttributeError: 'tuple' object has no attribute 'major'
This is a common bug, PR is comming
It looks like FortiOS.open() has no option to specify a port parameter yet paramiko.SSHClient has port parameter enabled.
Hi
We have problem load full-config in one vdom.
get system status
pip show pyfg
[chris@automata pyfg]$ python fg_backup.py fnet01 vdom01
Traceback (most recent call last):
File "fg_backup.py", line 16, in
d.load_config('user adgrp')
File "/usr/local/lib/python2.7/site-packages/pyFG/fortios.py", line 220, in load_config
self.running_config.parse_config_output(config_text)
File "/usr/local/lib/python2.7/site-packages/pyFG/forticonfig.py", line 383, in parse_config_output
current_block.set_param(parameter, ' '.join(data))
AttributeError: 'NoneType' object has no attribute 'set_param'
[chris@automata01 pyfg]$ more fg_backup.py
from pyFG import FortiOS
import sys
if name == 'main':
hostname = sys.argv[1]
vdom = sys.argv[2]
d = FortiOS(hostname, vdom=vdom, username='admin', password='password')
d.open()
d.load_config('user adgrp')
#d.load_config('full-configuration')
print d.running_config.to_text()
d.close()
[chris@automata pyfg]$
Hello,
it seems that this project is abandoned. Maybe you could give another person write-access so it is possible to merge the PRs and upload the latest version to PyPI, please.
Hi
I tried to connect to a FG with plan basic Paramiko and struggled. I then tested with your module and basically got the same error:
from pyFG import FortiOS
d = FortiOS('10.8.20.102')
d.open()
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Library/Python/2.7/site-packages/pyFG/fortios.py", line 94, in open
self.ssh.connect(**cfg)
File "build/bdist.macosx-10.11-intel/egg/paramiko/client.py", line 325, in connect
t.start_client()
File "build/bdist.macosx-10.11-intel/egg/paramiko/transport.py", line 492, in start_client
raise e
EOFError
Any idea what this could be? It looks like it's only happening with FortiOS Devices, I can connect to normal Linux hosts without any issues.
Install error:
Collecting pyFG
Downloading pyfg-0.50.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-GOjqC9/pyFG/setup.py", line 4, in <module>
from pip.req import parse_requirements
ImportError: No module named req
From upstream, you should not be depending on this pip internal: pypa/pip#3240
I recently tried to use pyFG's FortiConfig on a stock 100D (5.4.x firmware) Fortigate config, and found that it would truncate multi-line configurations, such as the certificate lines (which meant the resulting configs would not apply back cleanly).
From the original config:
config vpn certificate local
edit "Fortinet_CA_SSLProxy"
set certificate "-----BEGIN CERTIFICATE-----
MIID3jCCAsagAwIBAgIILmc2LyCLCs4wDQYJKoZIhvcNAQEFBQAwgaUxCzAJBgNV
...
OaKXO5cGgRFs5sSQMWunVOlATHhkMe+UQeZstT78oP4s6A==
-----END CERTIFICATE-----"
next
end
Which after adding to a FortiConfig and calling to_text() produced:
config vpn certificate local
edit Fortinet_CA_SSLProxy
set certificate "-----BEGIN CERTIFICATE-----
next
end
I'm happy to provide further details, but I think the issue is clear enough. The problem is that the parser seems to be line based, and so would need to build up a line memory if it ever doesn't encounter a closing quote (which will add complexity). At the moment it appears that quotes are removed completely, which may cause separate problems with named sections that contain spaces. So there's a fair bit of complexity surrounding handling this correctly. I'm happy to test any potential fixes for this if it would help though?
I found a problem while reading fortios config. With large config, the read takes more than 15 minutes to complete.
Hello there, I am facing an issue of timeout using pyfg in fortios driver for napalm integrated in salt. But the problem does seem to come from here.
When trying to discard_config on a fortios device, the channel raise a socket timeout and the operation fails.
I achieved to make this working perfectly just by changing the timeout value
<...>/python2.7/dist-package/pyFG/fortios.py, line 386:
--chan.settimeout(5)
++chan.settimeout(15)
I dont have tried other timeout values but '5 seconds' failed 30 tries over 30 whereas '15 seconds' worked each 5 times I tried.
I seem to be the only having ever faced this issue, so I suppose it could be a problem on my side.
But I don't see for now what it is, so, question is :
do you see from where on my side the problem could come
would it be interesting to do a pr to change the value, or make it a param.
or do i just go away and change it locally
Thanks
2018-06-12 15:20:03,453 [/usr/lib/python2.7/dist-packages/salt/utils/napalm.pyc :229 ][ERROR ][12137] Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/utils/napalm.py", line 167, in call
out = getattr(napalm_device.get('DRIVER'), method)(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/napalm_fortios/fortios.py", line 142, in discard_config
self.device.load_config(in_candidate=True)
File "/usr/local/lib/python2.7/dist-packages/pyFG/fortios.py", line 212, in load_config
config_text = self.execute_command(command)
File "/usr/local/lib/python2.7/dist-packages/pyFG/fortios.py", line 151, in execute_command
for e in error_chan.read():
File "/usr/local/lib/python2.7/dist-packages/paramiko/file.py", line 193, in read
new_data = self._read(self._DEFAULT_BUFSIZE)
File "/usr/local/lib/python2.7/dist-packages/paramiko/channel.py", line 1344, in _read
def _read(self, size):
File "/usr/local/lib/python2.7/dist-packages/paramiko/channel.py", line 733, in recv_stderr
raise socket.timeout()
timeout
pyfg: 0.50
paramiko: 2.4.1
Version: FortiGate-60C-POE v5.2.13,build0762,171212 (GA)
Virus-DB: 16.00560(2012-10-19 08:31)
Extended DB: 1.00000(2012-10-17 15:46)
IPS-DB: 5.00555(2014-10-07 01:21)
IPS-ETDB: 0.00000(2001-01-01 00:00)
Serial-Number: FG60CP3W13000001
Botnet DB: 1.00000(2012-05-28 22:51)
BIOS version: 04000003
System Part-Number: P12195-01
Log hard disk: Not available
Internal Switch mode: switch
Hostname: fg60cp3w13000001
Operation Mode: NAT
Current virtual domain: root
Max number of virtual domains: 10
Virtual domains status: 1 in NAT mode, 0 in TP mode
Virtual domain configuration: disable
FIPS-CC mode: disable
Current HA mode: standalone
Branch point: 762
Release Version Information: GA
System time: Tue Jun 12 08:38:03 2018
` def to_dict(self, root=None, tree_dict=None):
"""
This method convert FortiConfig to dict.
Args:
- **root** (class:`~pyFG.forticonfig.FortiConfig` object):
* If ``None`` use self as tree root.
- **tree_dict** (dict): This value is for output.
* If ``None`` use a blank dict.
"""
if root is None:
root = self
if tree_dict is None:
tree_dict = {}
if not hasattr(root, 'get_block_names') or not root.get_block_names():
return
for child in root.get_block_names():
if not tree_dict.get(child):
if root[child].get_parameter_names():
tree_dict[child] = {k: root[child].get_param(k).replace('"', '')
for k in root[child].get_parameter_names()}
else:
tree_dict[child] = {}
self.to_dict(root[child], tree_dict[child])
return tree_dict
`
This would be useful for when configuring devices that still only have self-signed certs. A crude workaround was documented on the fortinet developers network page.
https://fndn.fortinet.net/index.php?/forums/topic/1896-examples-of-ansible-28-and-new-fortinet-modules-fortiosapi-in-action/&do=findComment&comment=7294
Using the latest version of the pyFG, I'm trying to merge the running config on a fortigate of mine with a new snippet of code. I'm running with VDOMs enabled.
In [1]: import pyFG
In [2]: device = pyFG.FortiOS("fw1", vdom="FW", username="admin", password="password")
In [3]: device.open()
In [4]: device.candidate_config = pyFG.FortiConfig("candidate")
In [5]: device.running_config = pyFG.FortiConfig("running")
In [6]: configuration = open("/Users/tim/code/napalm/generated/fw1.txt").read()
In [7]: device.load_config(in_candidate=True, config_text=configuration)
In [8]: for block in device.candidate_config.get_block_names():
...: device.load_config(path=block, empty_candidate=True)
...:
In [12]: print device.compare_config()
config firewall address
edit Marist
set subnet 148.100.0.0 255.255.0.0
next
end
The load config looks mostly correct, but is missing the commands to move this into the correct VDOM. So when I try to commit if get an error
In [13]: device.commit()
---------------------------------------------------------------------------
FailedCommit Traceback (most recent call last)
<ipython-input-13-764a21633a04> in <module>()
----> 1 device.commit()
/Library/Python/2.7/site-packages/pyFG/fortios.pyc in commit(self, config_text, force)
277 * :class:`~pyFG.exceptions.ForcedCommit` -- Something failed but we avoided any rollback
278 """
--> 279 self._commit(config_text, force)
280
281 def _commit(self, config_text=None, force=False, reload_original_config=True):
/Library/Python/2.7/site-packages/pyFG/fortios.pyc in _commit(self, config_text, force, reload_original_config)
332
333 if exit_code < 0 :
--> 334 raise exceptions.FailedCommit(wrong_commands)
335
336 def rollback(self):
FailedCommit: [('-1', 'edit Marist'), ('-1', 'set subnet 148.100.0.0 255.255.0.0'), ('-1', 'next'), ('-1', 'end')]
Now what is interesting is that if I run compare_config again, I get the correct config snippet it, VDOM and all. I'm also able to commit this just fine.
In [14]: print device.compare_config()
conf vdom
edit FW
config firewall address
edit Marist
set subnet 148.100.0.0 255.255.0.0
next
end
end
In [15]: device.commit()
In [16]:
I noticed this issue when trying to use napalm and this thread started there
napalm-automation-community/napalm-fortios#13
Hi
I had some some error while backuping configuration on Fortigate 3000.
I have the message "Error reading running config" and a raise socket.timeout() exception.
My investigating found that it was the error channel of the ssh connection which raise this timeout and stop the reading for the standard output.
I do this modification on the plugin python module:
pyfg/pyFG/fortios.py
from line 149 replace this
error = '' output = '' for e in error_chan.read(): error = error + self._read_wrapper(e) for o in output_chan.read(): output = output + self._read_wrapper(o)
by
error = '' output = '' try: for e in error_chan.read(): error = error + self._read_wrapper(e) except: pass for o in output_chan.read(): output = output + self._read_wrapper(o)
And with this modification, i have no more error.
As diagnosis, i suspect that the Fortigate 3000 don't send any caracter to the stream
Hello there,
I've faced this issue and waste a bit of time to debug it but I've finally hunt it !
I use this bit of code to add new firewall rules :
d.load_config('firewall policy')
newRule05 = FortiConfig(config_type='edit', name='0')
newRule05.set_param('srcintf', subnetDescription)
newRule05.set_param('dstintf', firewallIntercoZone[0])
newRule05.set_param('srcaddr', 'all')
newRule05.set_param('dstaddr', fwObjectGrpSrvTransverses)
newRule05.set_param('action', 'accept')
newRule05.set_param('status', 'enable')
newRule05.set_param('service', fwObjectGrpServicesTransverses)
newRule05.set_param('schedule', 'always')
d.candidate_config['firewall policy']['0'] = newRule05
d.commit()
The commit failed intermittently with this generic error :
raise exceptions.FailedCommit(wrong_commands)
pyFG.exceptions.FailedCommit: [('-3', 'set dstaddr grp-srv-transverses')]
Sometimes it works without error and before this bit of code, the script commits dozens of other rules. The problem is :
edit 0
set action accept
set schedule always
set status enable
set srcaddr all
set service grp-services-transverses
set srcintf PP-SUMIT-BD
set dstintf Interco-TechRH-HProd
set dstaddr grp-srv-transverses
next
AND :
edit 0
set srcaddr all
set status enable
set service grp-services-transverses
set dstaddr grp-srv-transverses
set action accept
set schedule always
set srcintf PP-SUMIT-BD
set dstintf Interco-TechRH-HProd
next
The generation of the rule set should be in a sequential order to avoid this bug.
Regards.
Hardware: FortiGate 40C
This code is returning a timeout since it take a long time to retrieve all the configuration :
>>> from pyFG import FortiOS, FortiConfig
>>> d = FortiOS(hostname="10.76.0.252", username="admin", password="***********")
>>> d.open()
>>> d.execute_command('show full-configuration')
Executing commands:
show full-configuration
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/pyFG/fortios.py", line 152, in execute_command
for e in error_chan.read():
File "/usr/local/lib/python2.7/dist-packages/paramiko/file.py", line 192, in read
new_data = self._read(self._DEFAULT_BUFSIZE)
File "/usr/local/lib/python2.7/dist-packages/paramiko/channel.py", line 1305, in _read
return self.channel.recv_stderr(size)
File "/usr/local/lib/python2.7/dist-packages/paramiko/channel.py", line 715, in recv_stderr
raise socket.timeout()
socket.timeout
In my tests (local network) it takes about :
show full-configuration
command result,show
command result.Hi
Tried to move Interfaces from vdom to another, when static router still on the Interface, no all nessessay steps are build.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.