sassoftware/viya4-ark

SAS Viya Administration Resource Kit for Containerized Deployments. Tools and utilities to help SAS Customers with SAS Viya running in containers.

SAS Viya 4 Administration Resource Kit

The SAS Viya Administration Resource Kit (SAS Viya ARK) provides tools and utilities to help SAS customers prepare for and gather information about a SAS Viya platform deployment.

Introduction

SAS Viya ARK is a collection of resources that automate and streamline tasks that prepare an environment for a SAS Viya platform deployment. It also gathers information and generates reports about a deployment.

The main branch supports the latest release of the SAS Viya platform. Visit the releases page for specific information about SAS Viya ARK and related SAS Viya platform product releases.

SAS Viya ARK provides the following types of assistance:

• Pre-deployment assessment and optional configuration
• Post-deployment automation and utilities

Prerequisites for SAS Viya ARK

Obtain the latest version of SAS Viya ARK whenever you pull down new deployment assets or a new manifest.

Each tool that is included in the resource kit provides a readme that describes its specific prerequisites and functionality.

Required Python Packages

SAS Viya ARK tools require third-party packages be installed before use. All required packages can be installed using the provided requirements.txt:

python3 -m pip install -r requirements.txt

The Python packages are only required on the host where SAS Viya ARK tools are executed.

Index of Tools

Tool support for the latest release of the SAS Viya platform:

• SAS Viya Platform Deployment Report
• SAS Viya Platform Log Download Tool
• Pre-installation of SAS Viya Platform System Requirements
• SAS Viya LDAP Platform Validator

Contributing

We welcome your contributions! Please read CONTRIBUTING.md for details on how to submit contributions to these projects.

License

This project is licensed under the Apache 2.0 License.

viya4-ark's Issues

Setup on AWS

Hi,

We are running this and receiving this error on EKS.

[root@ip-10-215-189-101 viya4-ark]# python3 viya-ark.py pre-install-report -i nginx -H a6441ea4329ec545ebbda92e78bc9a78-cfbc33c162a90ab2.xb.us-east-1.amazonaws.com -p 80
Traceback (most recent call last):
File "viya-ark.py", line 139, in
main(sys.argv[1:])
File "viya-ark.py", line 77, in main
command.run(argv[1:])
File "/root/viya-poc/viya4-ark/viya4-ark/pre_install_report/pre_install_report.py", line 98, in run
main(argv)
File "/root/viya-poc/viya4-ark/viya4-ark/pre_install_report/pre_install_report.py", line 221, in main
kubectl = Kubectl(namespace=name_space)
File "/root/viya-poc/viya4-ark/viya4-ark/viya_ark_library/k8s/sas_kubectl.py", line 57, in init
self.version()
File "/root/viya-poc/viya4-ark/viya4-ark/viya_ark_library/k8s/sas_kubectl.py", line 386, in version
return json.loads(version_json.decode())
File "/usr/lib64/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 342, in decode
raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 25 column 1 (char 625)
[root@ip-10-215-189-101 viya4-ark]#

We have set the KUBECONFIG and exported the variables to make sure that we are getting the correct values but still are getting the json error. Is there something we are missing?

Refactor the CPU, Memory, Workers and Storage Class tables and make them easier to read

Used Fixed column widths and eliminate repeating column names

Deployment Report: Orphaned pods can cause null dictionary keys in the `sas_components` dictionary yielding sort errors

Currently, if a pod is orphaned (say, a job is deleted without a cascading deletion and its dependent pods are left around), a null key is written into the sas_components dictionary collected for the deployment report. This null key will cause issues when sorting the dictionary as standard sort comparisons (e.g., <, >) don't support NoneType operands.

The fix is trivial. There is an if / elif statement determining when the component_name is determined for a SAS component, and that value is ultimately used as the key when it's inserted into the sas_components dictionary. The if / elif is too restrictive and, in scenarios where the pod owner has been removed, can cause the component_name to be null. Adjusting the logic to if / else fixes this corner case and provides the same functionality as is currently implemented. However, the pod will set a component name and owning objects can update that value if necessary.

Issue executing Deployment Report on develop branch using --include-pod-log-snips

This traceback was reported executing on develop branch.

Traceback (most recent call last):
  File "viya-ark.py", line 139, in <module>
    main(sys.argv[1:])
  File "viya-ark.py", line 77, in main
    command.run(argv[1:])
  File "...\viya4-ark\deployment_report\deployment_report.py", line 48, in run
    main(argv)
  File "...\viya4-ark\deployment_report\deployment_report.py", line 134, in main
    include_pod_log_snips=args.include_pod_log_snips)
  File "...\viya4-ark\deployment_report\model\viya_deployment_report.py", line 236, in gather_details
    log_snip: List = kubectl.logs(pod_name)
TypeError: logs() missing 1 required positional argument: 'container_name'```

This was run against deployment to AWS.

Check that the kubeconfig file specified with KUBECONFIG environment variable exists.

Hardening code to add additional check. It throws an environments in the QA container environment.

Pre-Install-Check: Update for System Requirements based on Documentation

Consult the latest documentation to update the report generated by the pre-install check tool.

Could download_pod_logs grow an option for --format=json, please.

Its a shame to have the logs start out as structured logs, already easily consumable by other tooling, and have download_pod_logs transform them to sugared logs that might be easier on the human eye, but harder on the tool chain

would y'all consider an option that says "leave them as json please"

Deployment Report: request to add cadence version to the report

Viya 4 introduced the concept of Cadence based releases.
All cadences represent software that has gone through the quality gates and promotion stages prior to being made available to customers.
This is to request adding cadence version to the deployment report.

Suggestion: pre-installation check should be designed to be periodically executable

Hello,

following some discussions with colleagues, I thought it would be good to open an issue on this topic.

Although we have a tendency to think of pre-reqs as a one-time thing you check before first deployment, I believe that is not the optimal design.

For various reasons, pre-reqs that were done before a Viya deployment, could be later un-done. For example, someone might mistakenly delete components that are critical to Viya. (Ingress Controller, Cert-Manager), etc..
Or, someone might replace 10 very powerful nodes with 3 much less powerful nodes.

In those situations, the Viya Admin might be un-aware in the changes happening at those levels. All that would be visible would be a suddenly mis-behaving Viya environment. The admin would look to Viya as the culprit when really the fault lies somewhere else.

Instead, it would be much nicer is some sort of alert was raised as soon as some pre-reqs are suddenly "missing".

Therefore, I'd propose that the pre-installation report:

1. Have a clear "pass/fail" output, like 0 failures is a pass, and anything greater than 1 is a fail.
2. Be designed in a way that it can fairly easily be executed over and over, either within kubernetes itself, or from some sort of third-party external tool. (Jenkins, cron, etc...)
3. Take into account that not all the pre-reqs are exactly the same once Viya is actually deployed. If you needed X spare capacity before Viya was running in the cluster, you clearly don't need as much once Viya is deployed.
4. Allow for the evolution of the software and its pre-reqs. It's likely that as time goes by, those pre-reqs will change. It would be nice to be able to run the same report with different Viya versions, so that I could easily see that although I have all the pre-reqs for Viya LTS-2020.1, I am missing 2 pre-reqs for Viya LTS-2021.1. This is important not just for new deployments, but also for updates from one version to the next.

I hope this makes sense, and can help with the future design.

Check Permissions for azure Persistent Volume Claims

Check Permissions to allocate storage on Azure. If available, create the persistent volume claims for the following:
azurefile                         kubernetes.io/azure-file 
azurefile-premium      kubernetes.io/azure-file  
managed-premium     kubernetes.io/azure-disk  

Provide guidance to determine Ingress Host and port

Currently, there is no guidance on determining the Ingress Host and Port values on Azure with nginx. Adding some hints and examples to assist the user.

Adding it to README, but may need to be moved to a different section or separate document.

Potential security vulnerability in dependencies: jinja2

GitHub's dependabot sent notification of potential vulnerability from CVE-2020-28493. The suggestion to update requirements.txt jinja2 version to be equal to or greater than 2.11.3 is the course of action.

Update sas_kubectl to handle renamed "APIVERSION" column in `kubectl api-resources` response at v1.20.0

Hi viya4-ark team, receiving a ValueError when attempting to execute the pre-install tool but the log doesn't show any errors. Any chance someone could point to what exactly is missing or where I need to look based on the following output?

Gathering facts...................................DONE

Traceback (most recent call last):
  File "viya-ark.py", line 139, in <module>
    main(sys.argv[1:])
  File "viya-ark.py", line 77, in main
    command.run(argv[1:])
  File "/usr/csuser/clouddrive/viya4-ark/pre_install_report/pre_install_report.py", line 98, in run
    main(argv)
  File "/usr/csuser/clouddrive/viya4-ark/pre_install_report/pre_install_report.py", line 246, in main
    sas_pre_check_report.check_details(kubectl, ingress_port, ingress_host, ingress_controller, output_dir)
  File "/usr/csuser/clouddrive/viya4-ark/pre_install_report/library/pre_install_check.py", line 137, in check_details
    self._check_permissions(permissions_check)
  File "/usr/csuser/clouddrive/viya4-ark/pre_install_report/library/pre_install_check.py", line 409, in _check_permissions
    permissions_check.check_rbac_role()
  File "/usr/csuser/clouddrive/viya4-ark/pre_install_report/library/pre_install_check_permissions.py", line 450, in check_rbac_role
    found = self.utils.get_rbac_group_cmd()
  File "/usr/csuser/clouddrive/viya4-ark/pre_install_report/library/pre_install_utils.py", line 97, in get_rbac_group_cmd
    data: KubernetesApiResources = self._kubectl.api_resources(False)
  File "/usr/csuser/clouddrive/viya4-ark/viya_ark_library/k8s/sas_kubectl.py", line 143, in api_resources
    apigroup_index: int = api_resource_headers.index(_HEADER_APIGROUP_)
ValueError: substring not found

Pre-Install-Check: Update tool to correctly parse modified json from storage class

The following traceback is captured running tool against latest cluster.
$ python viya-ark.py pre-install-report -i "nginx" -n deployment -H dkdkdk -p 27373
Gathering facts...................................DONE

Traceback (most recent call last):
File "viya-ark.py", line 139, in
main(sys.argv[1:])
File "viya-ark.py", line 77, in main
command.run(argv[1:])
File "C:\Public\viya401\github\develop_4_6\viya4-ark\pre_install_report\pre_install_report.py", line 98, in run
main(argv)
File "C:\Public\viya401\github\develop_4_6\viya4-ark\pre_install_report\pre_install_report.py", line 271, in main
sas_pre_check_report.check_details(kubectl, ingress_port, ingress_host, ingress_controller, output_dir)
File "C:\Public\viya401\github\develop_4_6\viya4-ark\pre_install_report\library\pre_install_check.py", line 123, in check_details
storage_data = self._get_storage_classes(storage_json)
File "C:\Public\viya401\github\develop_4_6\viya4-ark\pre_install_report\library\pre_install_check.py", line 331, in _get_storage_classes
node_data['selfLink'] = node['metadata']['selfLink']

Modify Information for Determining Ingress Port

The comands for extracting Ingree port need to be changed from "port" to "nodePort'

{{$ export INGRESS_HTTP_PORT=$(kubectl -n get service -o jsonpath='
{.spec.ports[?(@.name=="http")].nodePort}')}}

{{$ export INGRESS_HTTPS_PORT=$(kubectl -n get service -o sonpath='
{.spec.ports[?(@.name=="https")].nodePort}')}}

Deployment Report: Error generating report against new Ingress definitions

(Per @jkmin96)

An error is raised when generating a Deployment Report against a deployment containing new Ingress definitions (version networking.k8s.io/v1), causing the following traceback:

Generating deployment report......................DONE
Traceback (most recent call last):
File "viya-ark.py", line 139, in <module>
  main(sys.argv[1:])
File "viya-ark.py", line 77, in main
  command.run(argv[1:])
File "/local/install/viya4-ark/deployment_report/deployment_report.py", line 48, in run
  main(argv)
File "/local/install/viya4-ark/deployment_report/deployment_report.py", line 134, in main
  include_pod_log_snips=args.include_pod_log_snips)
File "/local/install/viya4-ark/deployment_report/model/viya_deployment_report.py", line 202, in gather_details
  gathered_resources[k8s_kinds.INGRESS])
File "/local/install/viya4-ark/deployment_report/model/utils/viya_deployment_report_utils.py", line 167, in define_service_to_ingress_relationships
  http_path[KubernetesResource.Keys.BACKEND][KubernetesResource.Keys.SERVICE_NAME]
KeyError: 'serviceName'

The issue stems from a change in the service name key between the new Ingress version (networking.k8s.io/v1) and the old, soon-to-be-deprecated version (extensions/v1beta1).

The new definition defines the value under:

spec.rules[].http.paths[].backend.service.name

the old definition defines the value under

spec.rules[].http.paths[].backend.serviceName

This change causes the KeyError that is raised.

Proposed Fix

Use the version of the Ingress object that is returned to determine the path for access to the service name key.

Check disk size in pre-installation check

The pre-installation check should check the disk size of the machines to make sure it meets the minimum documented in the Virtual machine requirements.

Pre_install_check: Migrate hello-ingress manifest to use the networking.k8s.io/v1 API version, available since v1.19.

The extensions/v1beta1 and networking.k8s.io/v1beta1 API versions of Ingress will no longer be served in v1.22.

Migrate manifests and API clients to use the networking.k8s.io/v1 API version, available since v1.19.

Deployment report: requst to add promotion model of cadence information

For Stable the promotion model can be Release Prod or Release Shipped
For Fast 2020 the promotion models include those for Stable plus Release Testready and Release Verified.
It would be very helpful to have the promotion model included with the cadence information in the deployment Report.

Download Pod Logs: Error raised when log line is parsed as JSON but is not structured as a dictionary

When pod logs are downloaded and parsed, an error can be raised if the log line contains a value that can be parsed as valid JSON but not returned as a Python-native dictionary (i.e. numbers, raw strings, boolean values, lists). The code expects a dictionary to be returned and checks for the presence of certain required keys. However, unless a JSON object is parsed, the Python-native value may not be a dictionary. To fix this, the parsing class needs to be updated to check if the value returned is a dictionary and, if not, return the line as a string without attempting to parse.

Add support for command installation via pip installation

The setuptools support recently added to this project also supports installing a script as a command-line during pip installation. This would provide a secondary method of installation while maintaining all the current functionality, as well as the current installation method of cloning the project locally.

The README should also be updated to reflect this new option, with examples of how to use when installed this way.

Show the Calculated aggregate “Allocatable Memory” in “GB” on the Summary line

Also round the calculated aggregate Allocatable Memory across all nodes to 2 decimal places.

download-pod-logs `TypeError: 'NoneType' object is not iterable` exception

The following error can occur because the container_statuses list could be returned as a None type object which generates an exception if the function get_status_value returns None due to missing keys

Downloading pod logs..............................DONE
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/install/viya4-ark/download_pod_logs/model.py", line 179, in _write_log
for container_status_dict in container_statuses:
TypeError: 'NoneType' object is not iterable
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "viya-ark.py", line 139, in <module>
main(sys.argv[1:])
File "viya-ark.py", line 77, in main
command.run(argv[1:])
File "/install/viya4-ark/download_pod_logs/download_pod_logs.py", line 47, in run
main(argv)
File "/install/viya4-ark/download_pod_logs/download_pod_logs.py", line 142, in main
selected_components=args.selected_components, tail=args.tail)
File "/install/viya4-ark/download_pod_logs/model.py", line 148, in download_logs
err_info: Optional[Tuple[Text, Text]] = process.get_process().get(timeout=self._wait)
File "/usr/lib64/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
TypeError: 'NoneType' object is not iterable

Modify README for determining Ingress HTTP and HTTPS Ports

change "nodeport" back to "port"

Add setup files to make viya4-ark pip installable via Git

Add the requisite files to make the viya4-ark project pip installable via the git protocol:

https://setuptools.readthedocs.io/en/latest/userguide/quickstart.html

This allows users to use the model classes supporting the CLI for their own purposes.

Suggestion: Include version number for ingress type

The Deployment Report currently notes the Ingress Type. Would it be possible to also include the version of the Ingress type?

LDAP Validator

The tool does not allow/perform an anonymous bind. It uses bind User DN and password to authenticate. If anonymousBind is included in the yaml file, the value must be set to false. If the the user's sitedefault yaml file includes 'anonymousBind: true' then display error message and exit. Modify and also update README

Deployment Report: Update supported Ingress definitions

The deployment report should be updated to support more ingress controller definitions.

TypeError: argument of type 'int' is not iterable

Hello,
Trying "download-pod-logs", git cloned it "now" and getting the below issue

[cloud-user@rext03-0009 viya4-ark]$ python3 viya-ark.py download-pod-logs --namespace lab --tail 1000

Downloading pod logs..............................DONE
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/home/cloud-user/viya4-ark/download_pod_logs/model.py", line 233, in _write_log
file_content = SASStructuredLoggingParser.parse_log(log)
File "/home/cloud-user/viya4-ark/viya_ark_library/structured_logging/parser.py", line 40, in parse_log
unstructured_log.append(SASStructuredLoggingParser.parse_log_entry(log_entry))
File "/home/cloud-user/viya4-ark/viya_ark_library/structured_logging/parser.py", line 65, in parse_log_entry
if required_key not in structured_log_entry:
TypeError: argument of type 'int' is not iterable
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "viya-ark.py", line 139, in
main(sys.argv[1:])
File "viya-ark.py", line 77, in main
command.run(argv[1:])
File "/home/cloud-user/viya4-ark/download_pod_logs/download_pod_logs.py", line 47, in run
main(argv)
File "/home/cloud-user/viya4-ark/download_pod_logs/download_pod_logs.py", line 146, in main
selected_components=args.selected_components, tail=args.tail, noparse=args.noparse)
File "/home/cloud-user/viya4-ark/download_pod_logs/model.py", line 150, in download_logs
err_info: List[Tuple[Optional[Text], Text]] = process.get_process().get(timeout=self._wait)
File "/usr/lib64/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
TypeError: argument of type 'int' is not iterable

Am I missing something ?
Thanks !
Younes

istio not supported for 2020.0.6 - adjust checker tool

As of a decision made on 15Oct, Istio will not be supported for 2020.0.6. You can see the updated System Requirements:

Kubernetes Cluster Requirements
An Ingress controller,such as NGINX, that is supported by Kubernetes
Istio is not supported.

It would be helpful if the pre-install check report warns folks if it detects istio and not NGINX in the cluster.

Deployment Report: Traceback using Kubernetes Server & Client (kubectl) version v1.19+

(Per @StuartAAustin)

An error is raised when generating a Deployment Report with Kubernetes Client (kubectl) version v1.19+ causing the following traceback:

Traceback (most recent call last):
  File "viya-ark.py", line 139, in <module>
    main(sys.argv[1:])
  File "viya-ark.py", line 77, in main
    command.run(argv[1:])
  File "/install/viya-ark/viya4-ark/deployment_report/deployment_report.py", line 48, in run
    main(argv)
  File "/install/viya-ark/viya4-ark/deployment_report/deployment_report.py", line 134, in main
    include_pod_log_snips=args.include_pod_log_snips)
  File "/install/viya-ark/viya4-ark/deployment_report/model/viya_deployment_report.py", line 169, in gather_details
    networking_kind)
  File "/install/viya-ark/viya4-ark/deployment_report/model/utils/viya_deployment_report_utils.py", line 59, in gather_resource_details
    api_resources.get_name(resource_kind))
  File "/install/viya-ark/viya4-ark/viya_ark_library/k8s/sas_kubectl.py", line 294, in get_resources
    resources_list: List = json.loads(resource_json.decode()).get(KubernetesResource.Keys.ITEMS)
  File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The issue stems from a new warning that is printed when kubectl get ingresses is run at v1.19+:

Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress

This warning causes a parsing error when the output from the command is parsed as JSON.

Work-around

Revert the kubectl installation to a version less than v1.19+.

Add LDAP Validator

Equivalent functionality of the LDAP Validator from Viya ARK (Viya 3.x) in a Kubernetes environment.

Deployment Report: Refactor to use argparse instead of getopts

The Python argparse module provides almost exactly the same feature as getopts but with much less technical overhead and a lot more built in validation and handling of arguments. The download_pod_logs command has been updated to use argparse. The deployment report should also be updated.

Pre_install_Check: Add additional messages to report when All or Some Node(s) are not in READY state

Add additional messages to report when All or Some Node(s) are not in READY state and sample application deployment fails.

1. If the tool can query the state of the Nodes(s) (kubeconfig has cluster level permissions) add the Status of the Nodes to the report. New Node Status column added to Worker Node Services Tab.  Node Status of True or Unknown will be displayed.

2. If check of sample deployment status fails, the output from the kubectl command will be added to the report.
   Deployment -   Insufficient. Sample Deployment Check failed! Ensure Node(s) Status is Ready. Check Permissions in specified namespace. b'Waiting for deployment "hello-world" rollout to finish: 0 of 2 updated replicas are available...\n'
   

3. Add a timeout to the kubectl rollout status command of 3min.

Request to add storage information for internal/external database to the deployment report

This is to request to add storage information for database to the deployment report.
There are two types: internal/external.
With external database, host/port/user/dbname informations will be listed in the report.

Deployment-Report: Short-lived DaemonSets/Pods can cause intermittent TypeError issue

Short-lived DaemonSets/Pods can cause intermittent TypeErrors when sorting the data dictionary used for generating the deployment report, which yields the following traceback:

Traceback (most recent call last):
  File "/var/lib/jenkins/workspace/Deployment/DailyAutomatedDeployments/Viya4Deploy/autodeploy-viya4/viya-arkcd/viya-ark.py", line 139, in <module>
    main(sys.argv[1:])
  File "/var/lib/jenkins/workspace/Deployment/DailyAutomatedDeployments/Viya4Deploy/autodeploy-viya4/viya-arkcd/viya-ark.py", line 77, in main
    command.run(argv[1:])
  File "/var/lib/jenkins/workspace/Deployment/DailyAutomatedDeployments/Viya4Deploy/autodeploy-viya4/viya-arkcd/deployment_report/deployment_report.py", line 48, in run
    main(argv)
  File "/var/lib/jenkins/workspace/Deployment/DailyAutomatedDeployments/Viya4Deploy/autodeploy-viya4/viya-arkcd/deployment_report/deployment_report.py", line 146, in main
    data_file, html_file = sas_deployment_report.write_report(
  File "/var/lib/jenkins/workspace/Deployment/DailyAutomatedDeployments/Viya4Deploy/autodeploy-viya4/viya-arkcd/deployment_report/model/viya_deployment_report.py", line 549, in write_report
    data_json = json.dumps(self._report_data, cls=KubernetesObjectJSONEncoder, indent=4, sort_keys=True)
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/json/__init__.py", line 234, in dumps
    return cls(
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/json/encoder.py", line 201, in encode
    chunks = list(chunks)
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/json/encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/json/encoder.py", line 353, in _iterencode_dict
    items = sorted(dct.items())
TypeError: '<' not supported between instances of 'NoneType' and 'str'

In diagnosing the problem, it appears the Pods are only maintained for about 15-30 seconds and the spawning DaemonSet is alive for even less time. If the deployment-report is run at the right time, the Pods will be gathered but the DaemonSet will have already been removed, causing the error above.

Workaround

Waiting a few seconds and retrying the deployment-report will allow for a successful execution. There are extended periods where neither the problematic Pods or DaemonSet exists; if run during this timeframe, an error will not be raised.

Deployment Report: Refactor to use LRPIndicator while task runs

In a recent push, the LRPIndicator class was added to present a visual indicator on the command line while a long-running process is executing. The deployment-report should be refactor to use this class for visual consistency and because execution time may be extended if logs or resource definitions are included.

Namespace

Can you add a step showing that namespace needs to be set before running the pre-reqs? Thanks!

Indicate Permissions of Insufficient for Azure Persistent Volume Claims if Storage Classes cannot be read

If kubeconfig file does not permit listing of storage classes mark the azure storage PVCs Insufficient or Not Applicable

CHANGELOG updates

The Changelog of the most recent version (viya-ark 1.12 does not seem to have the latest updates.
Could they be added?

request: Add all container logs to download-pod-logs tool for pods in non-ready/non-terminated state

Request to add extra useful information from initContainers to logs for a pod which is not in a running state. This information may be helpful for debugging.

Potential security vulnerability in dependencies: PyYAML

GitHub's dependabot sent notification of potential vulnerability from the version of PyYAML we are currently using. The suggestion remediation is to update requirements.txt to use PyYAML>=5.4.

Remove Control Plane Information

Status field has extra and/or bad information.
The following will be removed from the report:

Implement Pre-Install Check permissions for AWS Elastic Block Storage

The Pre-install Check tool now reports on Permissions for storage type gp2 (default) kubernetes.io/aws-ebs.
When pre-install-check tool is run against an AWS cluster, the Permissions for AWS EBS Storage will be reported.

1. Status of Adequate - indicates that the Persistent Volume Claim applied for gp2 was successful and the status is Bound
   

2. Status of Insufficient - indicates that the Persistent Volume Claim applied for gp2 does not have a status of Bound. The logs can provide more details on why the status is not Bound.

Add feature to download logs for previously deleted pods

Please add new feature to download pod logs for deleted pods using "kubectl kubectl logs --previous" command. This will be useful for debugging the issue as logs gets lost when pod gets re-created.
Thanks a lot!

Remove support for Istio from Viya 4 ARK tools

Update requirements.txt versions so packages are no longer pinned to a specific version

We should move away from pinning requirements to a specific version in requirements.txt. This will allow packages containing a fix to be installed without having to update this project to adjust the accepted version.

deployment report issue

Generating deployment report......................DONE
Traceback (most recent call last):
File "viya-ark.py", line 139, in
main(sys.argv[1:])
File "viya-ark.py", line 77, in main
command.run(argv[1:])
File "/sasinstall/viya4-ark-master/deployment_report/deployment_report.py", line 48, in run
main(argv)
File "/sasinstall/viya4-ark-master/deployment_report/deployment_report.py", line 149, in main
include_resource_definitions=args.include_resource_definitions)
File "/sasinstall/viya4-ark-master/deployment_report/model/viya_deployment_report.py", line 532, in write_report
data_json = json.dumps(self._report_data, cls=KubernetesObjectJSONEncoder, indent=4, sort_keys=True)
File "/usr/bin/python3.7/lib/python3.7/json/init.py", line 238, in dumps
**kw).encode(obj)
File "/usr/bin/python3.7/lib/python3.7/json/encoder.py", line 201, in encode
chunks = list(chunks)
File "/usr/bin/python3.7/lib/python3.7/json/encoder.py", line 431, in _iterencode
yield from _iterencode_dict(o, _current_indent_level)
File "/usr/bin/python3.7/lib/python3.7/json/encoder.py", line 405, in _iterencode_dict
yield from chunks
File "/usr/bin/python3.7/lib/python3.7/json/encoder.py", line 353, in _iterencode_dict
items = sorted(dct.items(), key=lambda kv: kv[0])
TypeError: '<' not supported between instances of 'NoneType' and 'str'

Pre-Install throws exception if Nodes are in a "NotReady" state

Fix the exception/crash in pre_install_report/library/pre_install_utils.py when NodeStatus= NOT READY. Crash occurred during logging because error code not cast to string.
New section in README for Pre-Install:

• All Nodes in cluster be in READY state before running the pre-install check tool.
• It is also preferable to run the Pre-Install tool with kubeconfig that has Admin level Access to obtain a full report.
• If the Nodes are NOT in Ready state, the tool will be unable to clean up the pods and the replicaset created by the tool and must be manually deleted.

Minimum Kubernetes version update needed

The pre_install_report/viya_cluster_settings.properties VIYA_KUBELET_VERSION_MIN needs to be updated to 1.18.