Giter Site home page Giter Site logo

turbot / steampipe-plugin-kubernetes Goto Github PK

View Code? Open in Web Editor NEW
44.0 12.0 16.0 979 KB

Use SQL to instantly query Kubernetes API resources. Open source CLI. No DB required.

Home Page: https://hub.steampipe.io/plugins/turbot/kubernetes

License: Apache License 2.0

Makefile 0.05% PLSQL 0.96% Go 89.65% JavaScript 4.88% HCL 4.46%
sql postgresql postgresql-fdw kubernetes kubernetes-api steampipe steampipe-plugin k8s hacktoberfest backup

steampipe-plugin-kubernetes's Introduction

image

Kubernetes Plugin for Steampipe

Use SQL to query Kubernetes components.

Apart from querying the deployed resources, the plugin also supports scanning the Kubernetes manifest files from different sources, parsing the configured Helm charts and scanning all the templates to get the list of Kubernetes resources.

Quick start

Install

Download and install the latest Kubernetes plugin:

steampipe plugin install kubernetes

Installing the latest Kubernetes plugin will create a config file (~/.steampipe/config/kubernetes.spc) with a single connection named kubernetes:

connection "kubernetes" {
  plugin = "kubernetes"

  # By default, the plugin will use credentials in "~/.kube/config" with the current context.
  # OpenID Connect (OIDC) authentication is supported without any extra configuration.
  # The kubeconfig path and context can also be specified with the following config arguments:

  # Specify the file path to the kubeconfig.
  # Can also be set with the "KUBECONFIG" or "KUBE_CONFIG_PATH" environment variables. Plugin will prioritize KUBECONFIG if both are available.
  # config_path = "~/.kube/config"

  # Specify a context other than the current one.
  # config_context = "minikube"

  # List of custom resources that will be created as dynamic tables.
  # No dynamic tables will be created if this arg is empty or not set.
  # Wildcard based searches are supported.

  # For example:
  #  - "*" matches all custom resources available
  #  - "*.storage.k8s.io" matches all custom resources in the storage.k8s.io group
  #  - "certificates.cert-manager.io" matches a specific custom resource "certificates.cert-manager.io"
  #  - "backendconfig" matches the singular name "backendconfig" in any group

  # Defaults to all custom resources
  custom_resource_tables = ["*"]

  # If no kubeconfig file can be found, the plugin will attempt to use the service account Kubernetes gives to pods.
  # This authentication method is intended for clients that expect to be running inside a pod running on Kubernetes.

  # Specify the source(s) of the resource(s). Possible values: `deployed`, `helm` and `manifest`.
  # Defaults to all possible values. Set the argument to override the default value.
  # If `deployed` is contained in the value, tables will show all the deployed resources.
  # If `helm` is contained in the value, tables will show resources from the configured helm charts.
  # If `manifest` is contained in the value, tables will show all the resources from the kubernetes manifest. Make sure that the `manifest_file_paths` arg is set.
  # source_types = ["deployed", "helm", "manifest"]

  # Manifest File Configuration

  # Manifest file paths is a list of locations to search for Kubernetes manifest files
  # Manifest file paths can be configured with a local directory, a remote Git repository URL, or an S3 bucket URL
  # Refer https://hub.steampipe.io/plugins/turbot/kubernetes#supported-path-formats for more information
  # Wildcard based searches are supported, including recursive searches
  # Local paths are resolved relative to the current working directory (CWD)

  # For example:
  #  - "*.yml" or "*.yaml" or "*.json" matches all Kubernetes manifest files in the CWD
  #  - "**/*.yml" or "**/*.yaml" or "**/*.json" matches all Kubernetes manifest files in the CWD and all sub-directories
  #  - "../*.yml" or "../*.yaml" or "../*.json" matches all Kubernetes manifest files in the CWD's parent directory
  #  - "steampipe*.yml" or "steampipe*.yaml" or "steampipe*.json" matches all Kubernetes manifest files starting with "steampipe" in the CWD
  #  - "/path/to/dir/*.yml" or "/path/to/dir/*.yaml" or "/path/to/dir/*.json" matches all Kubernetes manifest files in a specific directory
  #  - "/path/to/dir/main.yml" or "/path/to/dir/main.yaml" or "/path/to/dir/main.json" matches a specific file

  # If the given paths includes "*", all files (including non-kubernetes manifest files) in
  # the CWD will be matched, which may cause errors if incompatible file types exist

  # Defaults to CWD
  # manifest_file_paths = [ "*.yml", "*.yaml", "*.json" ]

  # Helm configuration

  # A map for Helm charts along with the path to the chart directory and the paths of the value override files (if any).
  # Every map should have chart_path defined, and the values_file_paths is optional.
  # You can define multiple charts in the config.
  # helm_rendered_charts = {
  #   "chart_name" = {
  #     chart_path        = "/path/to/chart/dir"
  #     values_file_paths = ["/path/to/value/override/files.yaml"]
  #   }
  # }
}

Run steampipe:

steampipe query

List all pods:

select
  name,
  namespace,
  phase,
  creation_timestamp,
  pod_ip
from
  kubernetes_pod;
+-----------------------------------------+-------------+-----------+---------------------+-----------+
| name                                    | namespace   | phase     | creation_timestamp  | pod_ip    |
+-----------------------------------------+-------------+-----------+---------------------+-----------+
| metrics-server-86cbb8457f-bf8dm         | kube-system | Running   | 2021-06-11 14:21:48 | 10.42.0.5 |
| coredns-7448499f4d-klb8l                | kube-system | Running   | 2021-06-11 14:21:48 | 10.42.0.6 |
| helm-install-traefik-crd-hb87d          | kube-system | Succeeded | 2021-06-11 14:21:48 | 10.42.0.3 |
| local-path-provisioner-5ff76fc89d-c9hnm | kube-system | Running   | 2021-06-11 14:21:48 | 10.42.0.2 |
+-----------------------------------------+-------------+-----------+---------------------+-----------+

Engines

This plugin is available for the following engines:

Engine Description
Steampipe The Steampipe CLI exposes APIs and services as a high-performance relational database, giving you the ability to write SQL-based queries to explore dynamic data. Mods extend Steampipe's capabilities with dashboards, reports, and controls built with simple HCL. The Steampipe CLI is a turnkey solution that includes its own Postgres database, plugin management, and mod support.
Postgres FDW Steampipe Postgres FDWs are native Postgres Foreign Data Wrappers that translate APIs to foreign tables. Unlike Steampipe CLI, which ships with its own Postgres server instance, the Steampipe Postgres FDWs can be installed in any supported Postgres database version.
SQLite Extension Steampipe SQLite Extensions provide SQLite virtual tables that translate your queries into API calls, transparently fetching information from your API or service as you request it.
Export Steampipe Plugin Exporters provide a flexible mechanism for exporting information from cloud services and APIs. Each exporter is a stand-alone binary that allows you to extract data using Steampipe plugins without a database.
Turbot Pipes Turbot Pipes is the only intelligence, automation & security platform built specifically for DevOps. Pipes provide hosted Steampipe database instances, shared dashboards, snapshots, and more.

Developing

Prerequisites:

Clone:

git clone https://github.com/turbot/steampipe-plugin-kubernetes.git
cd steampipe-plugin-kubernetes

Build, which automatically installs the new version to your ~/.steampipe/plugins directory:

make

Configure the plugin:

cp config/* ~/.steampipe/config
vi ~/.steampipe/config/kubernetes.spc

Try it!

steampipe query
> .inspect kubernetes

Further reading:

Open Source & Contributing

This repository is published under the Apache 2.0 (source code) and CC BY-NC-ND (docs) licenses. Please see our code of conduct. We look forward to collaborating with you!

Steampipe is a product produced from this open source software, exclusively by Turbot HQ, Inc. It is distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our Open Source FAQ.

Get Involved

Join #steampipe on Slack →

Want to help but don't know where to start? Pick up one of the help wanted issues:

steampipe-plugin-kubernetes's People

Contributors

aminvielledebatatbedrock avatar bigdatasourav avatar bob-bot avatar burahimu avatar cbruno10 avatar charlklein avatar dependabot[bot] avatar dongho-jung avatar e-gineer avatar graza-io avatar hileef avatar jacksgt avatar johnsmyth avatar khushboo9024 avatar lalitlab avatar madhushreeray30 avatar mafrosis avatar misraved avatar nmische avatar parthai avatar pdecat avatar rajlearner17 avatar subhajit97 avatar svend avatar whume avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

steampipe-plugin-kubernetes's Issues

Add table kubernetes_version

Hi,
thanks for this super cool plugin!

I think it would be useful to add the Kubernetes client and server version as a table, like this:

> select * from kubernetes_version;
+-----------+------------+-------------+
| component | version    | revision    |
+-----------+------------+-------------+
| server    | v1.25.6    | 9ec5f337f9f |
| client    | v1.26.1    | deadbeeeeef |
+-----------+------------+-------------+

The Gitlab plugin does something similar.

The plugin not support Auth Provider OIDC

Describe the bug
To connect me to my k8s clusters using kubectl use or OIDC authentication provider, but it looks like the plugin doesn't support it. When trying to run as queries I just got an error message [2021-08-18 17:43:19] [HV000] ERROR: rpc error: code = Unknown desc = no authentication provider found for name "oidc".

Steampipe version (steampipe -v)
v0.7.2

Plugin version (steampipe plugin list)
v0.0.2

To reproduce
Need to have OIDC authentication provider configured on your k8s clusters. After that just try running any queries.

Expected behavior
Normal execution of queries and use of auth providers.

Additional context
N/A

Dynamic CRD table creation fails when using an aggregator connection

Describe the bug
Dynamic CRD table creation fails when using an aggregator connection

Steampipe version (steampipe -v)
v0.18.0-dev.10

Plugin version (steampipe plugin list)
v0.13.0

To reproduce
Create 2 connections in kubernetes.spc, and then create 1 aggregator connection using both of the aforementioned connections.

Expected behavior
Aggregator connections should work correctly.

Additional context
Add any other context about the problem here.

Add table kubernetes_deployment_config

References
Add any related links that will help us understand the resource, including vendor documentation, related Kubernetes issues, and Go SDK documentation.

Add integration tests

Add integration test for below tables:

  • kubernetes_cluster_role
  • kubernetes_cluster_role_binding
  • kubernetes_config_map
  • kubernetes_daemonset
  • kubernetes_deployment
  • kubernetes_replicaset
  • kubernetes_role
  • kubernetes_role_binding
  • kubernetes_service_account
  • kubernetes_ingress

Improve error messaging on connection error or timeout

Is your feature request related to a problem? Please describe.
I manage several dozens connections for the kubernetes plugin. Occasionally one of the clusters behind these connections goes stale (the cluster gets destroyed or rebuilt). When I try to run queries when this happens the steampipe cli returns:

Error: failed to start plugin 'hub.steampipe.io/plugins/turbot/kubernetes@latest': timed out waiting for hub.steampipe.io/plugins/turbot/kubernetes@latest to startup after 5 seconds (SQLSTATE HV000)

This error gives no information about which connection may have had an error or timedout.

Describe the solution you'd like
The above error gives information about connections that failed.

Describe alternatives you've considered
I've tried running steampipe query with trace logging enabled, but, as far as I can tell, that doesn't seem to report the issue.

Additional context
Add any other context or screenshots about the feature request here.

SQLSTATE HV000 error when selecting kubernetes_cronjob

Describe the bug
I'm especially interested in the kubernetes_cronjob table. Unfortunately I've never managed to get a SELECT upon this table to work since I first tried, 3 or 4 months ago.

Steampipe version (steampipe -v)
Steampipe v0.20.0-alpha.4

Plugin version (steampipe plugin list)
Example: v0.18.1

To reproduce

steampipe query 'select * from kubernetes_cronjob;'

This produces

Error: the server could not find the requested resource (SQLSTATE HV000)

followed by an empty table.

Add table kubernetes_pod_template

References
Add any related links that will help us understand the resource, including vendor documentation, related Kubernetes issues, and Go SDK documentation.

Add column selector_query to kubernetes_* tables

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Add support to read resources from kubernetes manifests

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Using manifest path comes up with multple folders (single file is fine) or YAML comes up with an ERROR: failed to start plugin 'hub.steampipe.io/plugins/turbot/kubernetes@latest': timed out waiting for hub.steampipe.io/plugins/turbot/kubernetes@latest to s

Describe the bug
A clear and concise description of what the bug is.

Steampipe version (steampipe -v)
Example: v0.3.0

Plugin version (steampipe plugin list)
Example: v0.5.0

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Getting error `Error: the server could not find the requested resource (SQLSTATE HV000)`

Describe the bug
Running the select * from .. query on some tables fails with an error: Error: the server could not find the requested resource (SQLSTATE HV000).

The following tables are impacted by the above error:

  • kubernetes_endpoint_slice
  • kubernetes_horizontal_pod_autoscaler
  • kubernetes_pod_disruption_budget
  • kubernetes_pod_security_policy

Steampipe version (steampipe -v)
Example: v0.19.3

Plugin version (steampipe plugin list)
Example: v0.18.0

To reproduce
Run any select * queries on the above mentioned tables.

For example:

select * from kubernetes_pod_security_policy

Expected behavior
If there is no resource of specific kind, the table should return zero result without any error.

Additional context
Add any other context about the problem here.

Add column available_replicas to kubernetes_stateful_set table.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Update kubernetes doc

References
Update doc to indicate Kubernetes plugin support for OIDC Auth Provider.

Table `kubernetes_replication_controller` fails with an error `Error: failed to populate column 'selector_query': rpc error: code = Internal desc = transform labelSelectorToString failed with panic interface conversion: interface {} is map[string]string, not *v1.LabelSelector (SQLSTATE HV000)`

Describe the bug
Running any query against the table kubernetes_replication_controller gives the following error:

Error: failed to populate column 'selector_query': rpc error: code = Internal desc = transform labelSelectorToString failed with panic interface conversion: interface {} is map[string]string, not *v1.LabelSelector (SQLSTATE HV000)

Steampipe version (steampipe -v)
Example: v0.19.3

Plugin version (steampipe plugin list)
Example: v0.18.0

To reproduce
Run any query against the table kubernetes_replication_controller.

Expected behavior
The query should return the results without any error.

Additional context
Add any other context about the problem here.

Want to hard code credentials directly into "kubernetes plugin spc file" instead of giving absolute path to the kubeconfig file inside the spc file

I am using steampipe tool for kubernetes inventory and compliance. The below is the "kubernetes.spc file" which will be used by steampipe. However instead of giving the path to the "kubeconfig file" under the field "config_path" in the "kubernetes.spc file" , I just simply want to hardcode the credentials(like certificate-authority-data, client-certificate-data, client-key-data) of kubeconfig file in the below spc file. How can i achieve this? What would be the exact format of my spc file. please help

connection "kubernetes" {
  plugin = "kubernetes"
   By default, the plugin will use credentials in "~/.kube/config" with the current context.
   OpenID Connect (OIDC) authentication is supported without any extra configuration.
   The kubeconfig path and context can also be specified with the following config arguments:
   Specify the file path to the kubeconfig.
   Can also be set with the "KUBE_CONFIG_PATHS" or "KUBERNETES_MASTER" environment variables.
   config_path = "~/.kube/config"
   Specify a context other than the current one.
   config_context = "minikube"
   If no kubeconfig file can be found, the plugin will attempt to use the service account Kubernetes gives to pods.
   This authentication method is intended for clients that expect to be running inside a pod running on Kubernetes.
}

I want to hard code creds directly in spc file from kubeconfig file. Let say i dont have kubeconfig file exist in my system. like in cases of aws we directly mention creds like aws_secret_key, access_key in spc file. I want to achieve same with kubernetes spc file like the below pattern but it is not working.

connection "kubernetes" {
  plugin = "kubernetes"
  config = {
    # Hardcoded credentials for Kubernetes API server
    host = "https://your-kubernetes-host/"
    certificate-authority-data = "<certificate-authority-data>"
    client-certificate-data = "<client-certificate-data>"
    client-key-data = "<client-key-data>"
  }
}

Add config arg to allow filtering of which CRDs to create tables for

Is your feature request related to a problem? Please describe.
I want to be able to choose which CRDs tables are created for per connection.

Describe the solution you'd like
A config arg that allows me to specify names/patterns of CRDs. Possible names:

  • custom_resources
  • custom_resource_definitions
  • crds

Describe alternatives you've considered
N/A

Additional context
Add any other context or screenshots about the feature request here.

Add template column in kubernetes_stateful_set table.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Support InClusterConfig for Kubernetes

Is your feature request related to a problem? Please describe.
We run steampipe dashboards inside a container. We would like to not have to configure a KubeConfig file and rather really on "/var/run/secrets/kubernetes.io/serviceaccount/token". In KubeCtl the method is called InClusterConfig.

Describe the solution you'd like
Add InClusterConfig support.

Describe alternatives you've considered
Attempted to create Kube config from existing information dynamically.

Additional context
Link to InClusterConfig method
https://github.com/kubernetes/client-go/blob/fcdf37233b57445e4a6a4038ec580e9aaac931de/rest/config.go#L274

Plugin times out on startup

Describe the bug
Plugin times out on startup when there are several connections with dynamic CRD tables.

Warning: failed to start plugin 'hub.steampipe.io/plugins/turbot/kubernetes@latest': timed out waiting for hub.steampipe.io/plugins/turbot/kubernetes@latest to startup after 5 seconds

Steampipe version (steampipe -v)

steampipe version 0.18.0

Plugin version (steampipe plugin list)

hub.steampipe.io/plugins/turbot/aws@latest 0.95.0
hub.steampipe.io/plugins/turbot/csv@latest 0.5.0
hub.steampipe.io/plugins/turbot/github@latest 0.25.1
hub.steampipe.io/plugins/turbot/kubernetes@latest 0.17.0

To reproduce

  • Configure multiple connections to kubernetes clusters containing several CRDs.
    • In my case I see this when trying to connect to more than 6 clusters running Istio.
  • Run a steampipe command: steampipe plugin list or steampipe query

Expected behavior
All kubernetes connections load.

Additional context
Kubernetes clusters in my case are EKS running Istio.

Update tables for better use of API filters, context cancellation in list calls and page limiting for limit clause in query

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Add support to parse Helm charts and templates

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Remove deprecated config arg and env vars from the plugin

Is your feature request related to a problem? Please describe.
The below parameters are deprecated and will be removed after next month.
config_paths (please use config_path instead)
KUBE_CONFIG_PATHS (please use KUBECONFIG or KUBE_CONFIG_PATH instead)
KUBERNETES_MASTER (please use KUBECONFIG or KUBE_CONFIG_PATH instead)

Add table kubernetes_statefulset

References
Need for Kubernetes compliances
For reference - clientset.AppsV1().StatefulSets("").List(ctx, metav1.ListOptions{})

Dynamic CRD table creation fails when a dynamic column conflicts with common column

Describe the bug
Dynamic CRD table creation fails when a dynamic column conflicts with a common column, which are defined in

func k8sCRDResourceCommonColumns(columns []*plugin.Column) []*plugin.Column {
allColumns := []*plugin.Column{
{Name: "name", Type: proto.ColumnType_STRING, Description: "Name of resource."},
{Name: "uid", Type: proto.ColumnType_STRING, Description: "UID is the unique in time and space value for this object.", Transform: transform.FromField("UID")},
{Name: "kind", Type: proto.ColumnType_STRING, Description: "Type of resource."},
{Name: "api_version", Type: proto.ColumnType_STRING, Description: "The API version of the resource.", Transform: transform.FromField("APIVersion")},
{Name: "namespace", Type: proto.ColumnType_STRING, Description: "Namespace defines the space within which each name must be unique."},
{Name: "creation_timestamp", Type: proto.ColumnType_TIMESTAMP, Transform: transform.FromGo().Transform(v1TimeToRFC3339), Description: "CreationTimestamp is a timestamp representing the server time when this object was created."},
{Name: "labels", Type: proto.ColumnType_JSON, Description: "Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services."},
}
allColumns = append(allColumns, columns...)
return allColumns
}

Steampipe version (steampipe -v)
v0.18.0-dev.10

Plugin version (steampipe plugin list)
v0.13.0

To reproduce
Create a custom resource definition and object with the property labels and then start Steampipe.

Expected behavior
All dynamic columns should be created alongside the common columns.

Additional context
Add any other context about the problem here.

Add optional qual support in kubernetes_pod for label_selector column

Is your feature request related to a problem? Please describe.
Add optional qual support in kubernetes_pod for label_selector

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

EKS works with static credentials, but not MFA/token credentials

Describe the bug
steampipe query 'select name from kube_dev.kubernetes_node'
Works with simple (aws_access_key_id / aws_secret_access_key) in ~/.aws/credentials
Fails like this with MFA token (aws_access_key_id / aws_secret_access_key / aws_session_token) in ~/.aws/credentials (while kubectl --context=dev get nodes works):

Warning: executeQueries: query 1 of 1 failed: ERROR: rpc error: code = Unknown desc = Unauthorized (SQLSTATE HV000)

Steampipe version (steampipe -v)
0.15.4

Plugin version (steampipe plugin list)
hub.steampipe.io/plugins/turbot/kubernetes@latest 0.10.0

To reproduce
Use MacOS app "Leapp" to populate/update MFA in ~/.aws/credentials files

Expected behavior
Expected support for aws_session_token

Additional context

2022-08-17 16:07:57.558 UTC [ERROR] steampipe-plugin-kubernetes.plugin: [ERROR] 1660752476690: streamRows error chan select: Unauthorized
2022-08-17 16:07:57.559 UTC [WARN]  steampipe-plugin-kubernetes.plugin: [WARN]  1660752476690: Execute call failed err: Unauthorized cancelled: <nil> - cancelling pending item in cache

For some reason I'm not seeing the logs in utils.go

Honor ClientConfigLoadingRules

Describe the bug
To be fair I don't know if it's working as intended or a bug, but here we go.

The plugin doesn't seem to use the ClientConfigLoadingRules provided by the k8s client-go sdk. So for example setting the KUBECONFIG env doesn't lead to the plugin using the path provided in the value of the env to build the client config. That's unfortunate because all other Kubernetes CLIs that we use work with the KUBECONFIG env.

I guess that because here https://github.com/turbot/steampipe-plugin-kubernetes/blob/main/kubernetes/utils.go#L184 a default config path is set and here https://github.com/turbot/steampipe-plugin-kubernetes/blob/main/kubernetes/utils.go#L209-L213 if configPaths has one or multiple entries they are configured with precedence or explicit. In the default case it's set explicit which leads to the plugin not trying any other location for the Kubernetes config at all.

I know there is KUBE_CONFIG_PATHS variable but why introduce a new one when there might be already an env set with the desired value (KUBECONFIG).

I didn't test it but removing the default config path from the code may resolve all the above mentioned problems.

Steampipe version (steampipe -v)
Example: v0.16.4

Plugin version (steampipe plugin list)
Example: v0.12.0

To reproduce
Set KUBECONFIG env to something else than the default value (~/.kube/config)

Expected behavior
The envs which are used across almost all Kubernetes CLIs should be usable in the plugin.

Private certificates from an "unknown authority" on the k8s API server are not supported

Describe the bug
When using private certificates for the Kubernetes API server - the CA root being referenced in the kubeconfig file, I get the following error:

> select * from kubernetes_namespace

Error: Get "https://api.k8s.example.com/api/v1/namespaces?limit=500": tls: failed to verify certificate: x509: certificate signed by unknown authority (SQLSTATE HV000)

Steampipe version (steampipe -v)
Steampipe v0.20.12

Plugin version (steampipe plugin list)
hub.steampipe.io/plugins/turbot/kubernetes@latest | 0.23.0 | kubernetes

To reproduce
Use a cluster with a self-signed private certificate on the api server.

Expected behavior
A way to configure the plugin to ignore unknown authorities.

Empty returned `data` field in configmap and secret tables

Describe the bug

If I understood the behaviour correctly, the data field ( as in : kubernetes_config_map.data and kubernetes_secret.data ) is incorrectly overriden, and therefore returned as nil due to a bug introduced in #139

In particular, the issue is that the Data field from parsedContent

overrides the Data field from the kubernetes API object struct with a nil value,
which I believe to be unintended behvaiour, in lines such as this one :

I would personally feel more confortable with an implementation that does not mix fields from internal struct with fields from the kubernetes API object structs, but given my limited knowledge of this codebase, for now I will open a hotfix PR that changes the Data field's name : see #150


Steampipe version
Steampipe v0.20.10

Plugin version (steampipe plugin list)
Found bug when testing versions :

But is NOT present in version : hub.steampipe.io/plugins/turbot/[email protected]

To reproduce

Run any query attempting to make use of the data field, such as the default one from the documentation :

select
  name,
  namespace,
  data.key,
  data.value,
  age(current_timestamp, creation_timestamp)
from
  kubernetes_config_map,
  jsonb_each(data) as data
order by
  namespace,
  name;

The data field is always null.

Expected behavior

If the configmap's (or secret) actual data within etcd is not empty, and therefore the API response from kubernetes's data field is not empty, then this plugin should be correctly returning it.

Additional context
Hopefully this description contains enough information 🙂

When `source_type = "all"` (default or explicitly set), if I only have helm or manifest config args, query fails with invalid configuration

Describe the bug
When source_type = "all" (default or explicitly set) in my kubernetes.spc, if I run a query, I receive an error since Kube is looking for a cluster to connect to:

connection "kubernetes" {
  plugin = "kubernetes"
  
  manifest_file_paths =  ["/Users/cbruno/kube/deployments.yaml"]
  # Defaults to "all"
  #source_type = "all"
}
> select * from kubernetes.kubernetes_deployment

Error: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable (SQLSTATE HV000)

+------+-----------+-----+----------+----------------+----------+----------+----------+-------------------+------------------------+--------+---------------------------+---------------------+-----------------+---
| name | namespace | uid | replicas | selector_query | selector | template | strategy | min_ready_seconds | revision_history_limit | paused | progress_deadline_seconds | observed_generation | status_replicas | up
+------+-----------+-----+----------+----------------+----------+----------+----------+-------------------+------------------------+--------+---------------------------+---------------------+-----------------+---
+------+-----------+-----+----------+----------------+----------+----------+----------+-------------------+------------------------+--------+---------------------------+---------------------+-----------------+---

Time: 29ms.

If I then change source_type = "manifest", then the query works:

> select * from kubernetes.kubernetes_deployment
+-----------------------+-----------+--------+----------+---------------------------+-------------------------------------------------+-----------------------------------------------------------------------------
| name                  | namespace | uid    | replicas | selector_query            | selector                                        | template
+-----------------------+-----------+--------+----------+---------------------------+-------------------------------------------------+-----------------------------------------------------------------------------
| redis-cart            |           | <null> | <null>   | app=redis-cart            | {"matchLabels":{"app":"redis-cart"}}            | {"metadata":{"creationTimestamp":null,"labels":{"app":"redis-cart"}},"spec":
...

Time: 126ms. Rows fetched: 12. Hydrate calls: 12.

Steampipe version (steampipe -v)
v0.20.9

Plugin version (steampipe plugin list)
v0.22.1

To reproduce
See above

Expected behavior
If source_type = "all", even if I don't have a Kube cluster configured, the helm and manifest information should still be loaded

Additional context
Add any other context about the problem here.

Add kubernetes networking and volume tables

Tables:

  • kubernetes_endpoint_slice
  • kubernetes_endpoints
  • kubernetes_ingress
  • kubernetes_job
  • kubernetes_network_policy
  • kubernetes_persistent_volum
  • kubernetes_persistent_volume_claim
  • kubernetes_replication_controller

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.