The Preupgrade Assistant performs an assessment of the system from the "upgradeability" point of view. Disclaimer: It was fun. R.I.P. :-)
License: GNU General Public License v3.0
decode/encode operations are sometimes needless and waste much of time. Mainly
inside function run_subprocess in preup/utils.py. It should be used wisely.
Preupgrade-assistant should be able to generate two new reports:
Report should be like reports-admin.html and report-user.html
By default CLI will show only admin contents.
user contents will be show on the request.
Each content should define in INI file 'report_part' where could be 'admin' or 'user'
Function get_system is duplicate in code.
All tests should be improved for Fedora.
ElementTree.fromString method can't parse unicode strings which contain non-ascii characters. That's known upstream issue and will not be fixed in python 2.x (fixed since python 3.3).
All strings which could contain non-ascii characters must be encoded before call of ElementTree.fromString to utf-8 - our variable 'settings.defenc' can't be used for it.
post_scan function is deprecated and have to be checked If it is true.
Remove the functionality if it is really deprecated.
$ preupg-xccdf-compose ./RHEL6_7
********** ERROR **********
The file /etc/fstab is not executable
Traceback (most recent call last):
File "/usr/bin/preupg-xccdf-compose", line 24, in <module>
main()
File "/usr/bin/preupg-xccdf-compose", line 21, in main
xccdf_compose.generate_xml()
File "/usr/lib/python2.7/site-packages/preuputils/compose.py", line 53, in generate_xml
target_tree = ComposeXML.run_compose(target_tree, self.dir_name)
File "/usr/lib/python2.7/site-packages/preuputils/compose.py", line 300, in run_compose
group_xmls = cls.collect_group_xmls(dir_name, content=content, level=0)
File "/usr/lib/python2.7/site-packages/preuputils/compose.py", line 96, in collect_group_xmls
cls.collect_group_xmls(new_dir, level=level + 1))
File "/usr/lib/python2.7/site-packages/preuputils/compose.py", line 86, in collect_group_xmls
oscap_group.write_xml()
File "/usr/lib/python2.7/site-packages/preuputils/oscap_group_xml.py", line 86, in write_xml
self.rule = xml_utils.prepare_sections()
File "/usr/lib/python2.7/site-packages/preuputils/xml_utils.py", line 222, in prepare_sections
self.create_xml_from_ini(main)
File "/usr/lib/python2.7/site-packages/preuputils/xml_utils.py", line 342, in create_xml_from_ini
function(key, k)
File "/usr/lib/python2.7/site-packages/preuputils/xml_utils.py", line 249, in fnc_check_script
self.check_script_modification(key, name)
File "/usr/lib/python2.7/site-packages/preuputils/xml_utils.py", line 195, in check_script_modification
self.update_values_list(self.rule, "{scap_name}", key[k].split('.')[:-1][0])
IndexError: list index out of range
ini
[preupgrade]
content_title: Plugable authentication modules (PAM) +ľš[34m~Mťť
author: Ondrej Vasik <[email protected]>
content_description: Content checks for no-longer supported pluggable authentication modules
solution: pam.txt
check_script: /etc/fstab
applies_to: pam
bugzilla: 1056003
On Fedora 22 oscap returns this error. It seems like rule isn't complete (look at dash on the end of rule ID). May this exists since user/admin separation. I didn't remember this message before.
['/usr/bin/oscap',
'xccdf',
'generate',
'report',
'--output',
'/root/preupgrade/result-admin.html',
'/root/preupgrade/result-admin.xml']
'element tr: validity error : ID rule-overview-leaf- already defined\n'When more sets of contents exists in directory, preupg prints wrong error message:
"There were no contents found in directory /usr/share/preupgrade. If you would like to use this tool, you have to install some."
It's due to wrong condition
On Fedora we should generate a content before assessment.
This is needed for Fedora 22 release.
Plugable authentication modules (PAM) +ľš[34m~Mťť
Traceback (most recent call last):
File "/usr/bin/preupg", line 34, in <module>
sys.exit(main())
File "/usr/bin/preupg", line 21, in main
ret = app.run()
File "/usr/lib/python2.7/site-packages/preup/application.py", line 638, in run
tarball_path = self.scan_system()
File "/usr/lib/python2.7/site-packages/preup/application.py", line 481, in scan_system
self.run_scan_process()
File "/usr/lib/python2.7/site-packages/preup/application.py", line 252, in run_scan_process
self.run_scan(function=self.scanning_progress.show_progress)
File "/usr/lib/python2.7/site-packages/preup/application.py", line 269, in run_scan
return run_subprocess(cmd, print_output=False, function=function)
File "/usr/lib/python2.7/site-packages/preup/utils.py", line 124, in run_subprocess
function(stdout_data)
File "/usr/lib/python2.7/site-packages/preup/scanning.py", line 97, in show_progress
stdout_data.strip()))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 39-40: ordinal not in range(128)
Check if risk check API has a test. If not then create at least one and should contain all messages.
Preupgrade assistant should be able to add user to kickstart. Content should generate a file which will contain all information.
Only local users are required at the moment
We should add functionality for partition layout to the kickstart.
$ preupg-xccdf-compose ./RHEL6_7
Traceback (most recent call last):
File "/usr/bin/preupg-xccdf-compose", line 24, in <module>
main()
File "/usr/bin/preupg-xccdf-compose", line 21, in main
xccdf_compose.generate_xml()
File "/usr/lib/python2.7/site-packages/preuputils/compose.py", line 53, in generate_xml
target_tree = ComposeXML.run_compose(target_tree, self.dir_name)
File "/usr/lib/python2.7/site-packages/preuputils/compose.py", line 300, in run_compose
group_xmls = cls.collect_group_xmls(dir_name, content=content, level=0)
File "/usr/lib/python2.7/site-packages/preuputils/compose.py", line 96, in collect_group_xmls
cls.collect_group_xmls(new_dir, level=level + 1))
File "/usr/lib/python2.7/site-packages/preuputils/compose.py", line 86, in collect_group_xmls
oscap_group.write_xml()
File "/usr/lib/python2.7/site-packages/preuputils/oscap_group_xml.py", line 86, in write_xml
self.rule = xml_utils.prepare_sections()
File "/usr/lib/python2.7/site-packages/preuputils/xml_utils.py", line 222, in prepare_sections
self.create_xml_from_ini(main)
File "/usr/lib/python2.7/site-packages/preuputils/xml_utils.py", line 342, in create_xml_from_ini
function(key, k)
File "/usr/lib/python2.7/site-packages/preuputils/xml_utils.py", line 249, in fnc_check_script
self.check_script_modification(key, name)
File "/usr/lib/python2.7/site-packages/preuputils/xml_utils.py", line 194, in check_script_modification
check_func=check_func[check])
File "/usr/lib/python2.7/site-packages/preuputils/script_utils.py", line 176, in check_inplace_risk
lines = get_file_content(get_full_path(dir_name, script_name), "r")
File "/usr/lib/python2.7/site-packages/preup/utils.py", line 195, in get_file_content
f = open(path, perms)
IOError: [Errno 21] Is a directory: u'/'
We want to run tests on rhel-6 too, however out tests needs some methods which are available only under python 2.7+. like assertIsNotNone().. we can replace it by alternative assert methods which are available under python 2.6 too. Then we will not need to download newer python version through scl and can tests PA inside default environment
It would be good to separate admin content from user contents.
For this aim we need to define a variable like result_view: admin or user
If variable is not specified then admin will use.
On RHEL6.6:
$ rpm -ivh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
$ yum install -y git python-six pykickstart python-setuptools
$ git clone https://github.com/phracek/preupgrade-assistant.git
$ cd preupgrade-assistant/
$ ./setup install
$ python tests/__init__
I get this
Traceback (most recent call last):
File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
exec code in run_globals
File "/root/preupgrade-assistant/tests/__init__.py", line 2, in <module>
from tests import test_preup
File "tests/__init__.py", line 2, in <module>
from tests import test_preup
File "tests/test_preup.py", line 6, in <module>
from preup.application import Application
File "preup/application.py", line 24, in <module>
from preup.kickstart import KickstartGenerator
File "preup/kickstart.py", line 122, in <module>
class RepoData(commands.repo.F21_RepoData):
AttributeError: 'module' object has no attribute 'F21_RepoData'On the base of settings PA should be able to work with Fedora and RHEL
For Fedora 22 returns "one)" - should be modified for correct output on both systems or be sure that will not be called on Fedora systems.
May could be more sufficient rename function to get_variant() only, because should return variant of system: Server, Client, etc...
preupg unexpectedly generate contents from .ini files again although this should be done only by other utilities (preupg-xccd-compose,...)
print function for python 2 doesn't know flush argument. Should be removed and checked for master and rhel-5 branch too
Traceback (most recent call last):
File "/usr/bin/preupg", line 34, in <module>
sys.exit(main())
File "/usr/bin/preupg", line 21, in main
ret = app.run()
File "/usr/lib/python2.7/site-packages/preup/application.py", line 600, in run
tarball_path = self.scan_system()
File "/usr/lib/python2.7/site-packages/preup/application.py", line 443, in scan_system
self.prepare_xml_for_html()
File "/usr/lib/python2.7/site-packages/preup/application.py", line 315, in prepare_xml_for_html
self.prepare_for_generation()
File "/usr/lib/python2.7/site-packages/preup/application.py", line 301, in prepare_for_generation
self.run_generate(report, report.replace('.xml', '.html'))
File "/usr/lib/python2.7/site-packages/preup/application.py", line 248, in run_generate
return run_subprocess(cmd, print_output=True)
File "/usr/lib/python2.7/site-packages/preup/utils.py", line 136, in run_subprocess
print (stdout_data, end="", flush=True)
TypeError: 'flush' is an invalid keyword argument for this functionTemplate.xml file used for generation and solution scripts should have proper platform tag.
find_solution_txt functionality should be enhanced so that it will use report_parser.get_solution_files.
New openscap (in Fedora 22) generate different *.html output from given xslt and therefore our previous searched substring isn't available in html file - tag <p> isn't around *_SOLUTION_MSG_TEXT now.
In addition on old systems it looks that we generate wrong HTML due this substitution, when we remove open tag <p> but </p> is kept.
It should be resolved for fedora and rhel systems universally and it would be nice to have tests for it.
.gitignore is inside tarball, but .git is not included. Probably this file shouldn't be part of tarball too. What do you think?
Preupgrade-Assistent has to be able work also with Unicode characters.
PA doesn't generate *.html output for admin & user xml reports
Currently now it is not planned to ship UI on Fedora. Only RHEL is supported
Contents should be copied before an assessment to /root/preupgrade directory
Traceback (most recent call last):
File "/usr/bin/preupg-xccdf-compose", line 24, in <module>
main()
File "/usr/bin/preupg-xccdf-compose", line 21, in main
xccdf_compose.generate_xml()
File "/usr/lib/python2.7/site-packages/preuputils/compose.py", line 53, in generate_xml
target_tree = ComposeXML.run_compose(target_tree, self.dir_name)
File "/usr/lib/python2.7/site-packages/preuputils/compose.py", line 300, in run_compose
group_xmls = cls.collect_group_xmls(dir_name, content=content, level=0)
File "/usr/lib/python2.7/site-packages/preuputils/compose.py", line 96, in collect_group_xmls
cls.collect_group_xmls(new_dir, level=level + 1))
File "/usr/lib/python2.7/site-packages/preuputils/compose.py", line 86, in collect_group_xmls
oscap_group.write_xml()
File "/usr/lib/python2.7/site-packages/preuputils/oscap_group_xml.py", line 84, in write_xml
self.find_all_ini()
File "/usr/lib/python2.7/site-packages/preuputils/oscap_group_xml.py", line 53, in find_all_ini
config.readfp(open(file_name))
File "/usr/lib64/python2.7/ConfigParser.py", line 324, in readfp
self._read(fp, filename)
File "/usr/lib64/python2.7/ConfigParser.py", line 546, in _read
raise e
ConfigParser.ParsingError: File contains parsing errors: ./RHEL6_7-results/system/pam/pam.ini
[line 9]: '[]\n'
ini
[preupgrade]
content_title: Plugable authentication modules (PAM)íážíá
author: Ondrej Vasik <[email protected]>
content_description: Content checks for no-longer supported pluggable authentication modules
solution: pam.txt
check_script: pam.sh
applies_to: pam
bugzilla: 1056003o
[]
a=b
We could print info about unknown tags inside INI files during parsing. Now we ignore it without any additional info.
Apply functionality is deprecated and can be removed.
Only assessment and kickstart is going to be supported.
--help option and man page should be updated so that they will contain the same information.
Gathering logs used by preupgrade assistant:
All installed packages : 01/5 ...runningTraceback (most recent call last):
File "/usr/bin/preupg", line 34, in
sys.exit(main())
File "/usr/bin/preupg", line 21, in main
ret = app.run()
File "/usr/lib/python2.7/site-packages/preup/application.py", line 650, in run
if not self.common.common_results():
File "/usr/lib/python2.7/site-packages/preup/common.py", line 78, in common_results
run_subprocess(cmd, output=common_file_path, shell=True)
File "/usr/lib/python2.7/site-packages/preup/utils.py", line 95, in run_subprocess
stdout += stdout_data.decode()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 37: ordinal not in range(128)
Those two binaries should not finish with traceback
if there isn't argument.
actual devel-version of PA crash when get_file_content read and decode binary data (data are read due to calculation of hash by function get_hash_file()).
Can be fixed by adding two next arguments to get_file_content
content = get_file_content(filename, "r", False, False)however get_file_content should maybe test if variable data is None and in this case raise exception or read raw data again without decoding.
According to description of test_amp_expand, this test is probably wrong. I expects double call of html_escape or tests of & and other html entities directly - I expect that this is original idea of this test. Am I right Peter?
function mimetypes.guess_type returns type None type, when can't guess MIME type of file. However dict file_types contains key 'None'.
In the case of simple filename without suffix, we can't get any other information about MIME type of file. May we could check shebang in the case of unknown MIME type.
command
preupg -u http://127.0.0.1:8099/submit/ -r preupg_results-*.tar.gz
fails with error ValueError: invalid literal for int() with base 10: 'http://127.0.0.1:8099/1/detail/'
Please create tests for unicode contents with ASCII contents. Like
author Petr Hráček
It would be good to write a function which will remove row or rows from kickstart.
Like key --skip in anaconda.cfg on RHEL-5 is not needed on RHEL-7.
we still forget set version inside setup.py, so this resolve at last or we will use in devel new version immadiately after new release.
(version for 0.11.11 is wrong too for original commit, it was set later)
Test suite should be able to test kickstart generation.
$ python3 ./setup.py test
running test
running egg_info
writing top-level names to preupgrade_assistant.egg-info/top_level.txt
writing preupgrade_assistant.egg-info/PKG-INFO
writing dependency_links to preupgrade_assistant.egg-info/dependency_links.txt
reading manifest file 'preupgrade_assistant.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'Makefile'
warning: no files found matching '*' under directory 'preup_ui/lib'
writing manifest file 'preupgrade_assistant.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
File "./setup.py", line 68, in <module>
test_suite = 'tests.suite',
File "/usr/lib64/python3.4/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib64/python3.4/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3.4/site-packages/setuptools/command/test.py", line 138, in run
self.with_project_on_sys_path(self.run_tests)
File "/usr/lib/python3.4/site-packages/setuptools/command/test.py", line 118, in with_project_on_sys_path
func()
File "/usr/lib/python3.4/site-packages/setuptools/command/test.py", line 164, in run_tests
testLoader = cks
File "/usr/lib64/python3.4/unittest/main.py", line 92, in __init__
self.parseArgs(argv)
File "/usr/lib64/python3.4/unittest/main.py", line 139, in parseArgs
self.createTests()
File "/usr/lib64/python3.4/unittest/main.py", line 146, in createTests
self.module)
File "/usr/lib64/python3.4/unittest/loader.py", line 146, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/lib64/python3.4/unittest/loader.py", line 146, in <listcomp>
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/lib64/python3.4/unittest/loader.py", line 105, in loadTestsFromName
module = __import__('.'.join(parts_copy))
File "/home/pstodulk/special/preupgrade-assistant/tests/__init__.py", line 1, in <module>
import test_preup
ImportError: No module named 'test_preup'Logout button could be nice when user is logged in as possibility to logout from session.
Unfortunately unicode again - it's like evergreen since I have done painful decision rewrite PA to use unicode.
So for next version of PA we need use unicode solution, which is compatible for python 2 and python 3 too. It's important do it for Fedora 23, where is default python 3. So here is lot's of time for it, but it could be nice have it inside v0.12.0
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.