reactive-tech/kubegres

Kubegres is a Kubernetes operator allowing to deploy one or many clusters of PostgreSql instances and manage databases replication, failover and backup.

Kubegres is a Kubernetes operator allowing to deploy one or many clusters of PostgreSql pods with data replication and failover enabled out-of-the box. It brings simplicity when using PostgreSql considering how complex managing stateful-set's life-cycle and data replication could be with Kubernetes.

Features

• It can manage one or many clusters of Postgres instances. Each cluster of Postgres instances is created using a YAML of "kind: Kubegres". Each cluster is self-contained and is identified by its unique name and namespace.

• It creates a cluster of PostgreSql servers with Streaming Replication enabled: it creates a Primary PostgreSql pod and a number of Replica PostgreSql pods and replicates primary's database in real-time to Replica pods.

• It manages fail-over: if a Primary PostgreSql crashes, it automatically promotes a Replica PostgreSql as a Primary.

• It has a data backup option allowing to dump PostgreSql data regularly in a given volume.

• It provides a very simple YAML with properties specialised for PostgreSql.

• It is resilient, has over 93 automatized tests cases and has been running in production.

How does Kubegres differentiate itself?

Kubegres is fully integrated with Kubernetes' lifecycle as it runs as an operator written in Go.
It is minimalist in terms of codebase compared to other open-source Postgres operators. It has the minimal and yet robust required features to manage a cluster of PostgreSql on Kubernetes. We aim keeping this project small and simple.

Among many reasons, there are 5 main ones why we recommend Kubegres.

Getting started

If you would like to install Kubegres, please read the page Getting started.

Sponsor

Kubegres is sponsored by Etikbee which is using Kubegres in production with over 25 clusters of Postgres. Etikbee is a UK based marketplace which promotes reuse by allowing merchants to list their products for rent, for sale and advertise services such as product repair.

Contribute

If you would like to contribute to Kubegres, please read the page How to contribute.

More details about the project

Kubegres was developed by Reactive Tech Limited and Alex Arica as the lead developer. Reactive Tech offers support services for Kubegres, Kubernetes and PostgreSql.

It was developed with the framework Kubebuilder version 3, an SDK for building Kubernetes APIs using CRDs. Kubebuilder is maintained by the official Kubernetes API Machinery Special Interest Group (SIG).

Support

Reactive Tech Limited offers support for organisations using Kubegres. And we prioritise new features requested by organisations paying supports as long the new features would benefit the Open Source community. We start working on the implementation of new features within 24h of the request from organisations paying supports. More details in the support page.

Interesting links

• A webinar about Kubegres was hosted by PostgresConf on 25 May 2021. Watch the recorded video.
• The availability of Kubegres was published on PostgreSql's official website.
• Google talked about Kubegres in their Kubernetes Podcast #146.

kubegres's Issues

Kubegres manager fails to start on local minikube

Hi! I am trying to run Kubegres on my Minikube instance (applying the default resource yaml) but it is failing to start with:

2021-10-05T19:45:11.477Z ERROR setup unable to start manager {"error": "Get \"https://10.96.0.1:443/api?timeout=32s\": dial tcp 10.96.0.1:443: i/o timeout"}

My kubernetes service:

[~] $ kubectl get services
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   58m

Is there something else I need to configure that might be missing?

Create helm chart

It would be great to have a helm chart! At the very least breakout the CRDs or create kustomize. This helps greatly in a GitOps model.

Change service type

How to change service type to load balance or node port?

Kubegres only works on Postgres 12 and newer

The Kubegres documentation states:

It works with the PostgreSql containers created by the Docker Official Images team. Kubegres uses the standard PostgreSql libraries to manage replication and failover and it does not require a custom Docker image with a set of third party libraries.

However, the Kubegres Replica/Standby containers are spun up with the promote_trigger_file argument:

https://github.com/reactive-tech/kubegres/search?q=promote_trigger_file

Which was introduced in Postgres 12:

https://www.postgresql.org/docs/12/runtime-config-replication.html

I've tried using Kubegres with the official Postgres 9.6.23, 10.18 and 11.13 images, but the Replica/Standby instances in these cases error out with:

$ kubectl logs pods/atscale-postgres-2-0 -f

FATAL:  unrecognized configuration parameter "promote_trigger_file"

Postgres 12.8 and newer are fine, with the Replica/Standby instances coming up as expected:

$ kubectl logs pods/atscale-postgres-2-0 -f

2021-10-12 21:17:55.041 GMT [27] LOG:  entering standby mode
2021-10-12 21:17:55.044 GMT [27] LOG:  redo starts at 0/2000028
2021-10-12 21:17:55.045 GMT [27] LOG:  consistent recovery state reached at 0/2000100
2021-10-12 21:17:55.045 GMT [1] LOG:  database system is ready to accept read only connections
2021-10-12 21:17:55.050 GMT [31] LOG:  started streaming WAL from primary at 0/3000000 on timeline 1

Bug in expanding database storage.

I am following the getting started example, using an expandable K8s StorageClass on AWS:

 % cat ~/gp2-expandable-storage-class.yaml                                                                                                          
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: gp2-expandable
  annotations:
    storageclass.beta.kubernetes.io/is-default-class: "true"
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2
reclaimPolicy: Delete
allowVolumeExpansion: true
mountOptions:
  - debug

± % k get sc gp2-expandable                                                                                                                          !10397
NAME                       PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
gp2-expandable (default)   kubernetes.io/aws-ebs   Delete          Immediate           true                   93m

I created the Kubegres cluster with the following:

apiVersion: kubegres.reactive-tech.io/v1
kind: Kubegres
metadata:
  name: mypostgres
  namespace: default

spec:

   replicas: 3
   image: postgres:13.2

   database:
      storageClassName: gp2-expandable
      size: 200Mi

   env:
      - name: POSTGRES_PASSWORD
        valueFrom:
           secretKeyRef:
              name: mypostgres-secret
              key: superUserPassword

      - name: POSTGRES_REPLICATION_PASSWORD
        valueFrom:
           secretKeyRef:
              name: mypostgres-secret
              key: replicationUserPassword

I then created a database, table, and inserted some rows into it.

Then I edited the Kubegres CR, changing spec.database.size from 200Mi to 10Gi.

The Kubegres operator then resized the PVC for postgres-db-mypostgres-3-0 to be 10Gi:

± % k get pvc                                                                                                                                        !10390
NAME                         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS     AGE
postgres-db-mypostgres-1-0   Bound    pvc-7913a8da-3407-45d8-a3a2-b0cf0759a640   1Gi        RWO            gp2-expandable   17m
postgres-db-mypostgres-2-0   Bound    pvc-7442165d-c31c-4136-a598-03da51c9ad7f   1Gi        RWO            gp2-expandable   17m
postgres-db-mypostgres-3-0   Bound    pvc-22d3c475-a6aa-4a98-9ae5-780895092af9   10Gi       RWO            gp2-expandable   16m

And then it failed with a timeout error:

reactive-tech.io/kubegres/controllers/ctx/log.(*LogWrapper).ErrorEvent
	/workspace/controllers/ctx/log/LogWrapper.go:62
reactive-tech.io/kubegres/controllers/spec/enforcer/statefulset_spec.(*AllStatefulSetsSpecEnforcer).logSpecEnforcementTimedOut
	/workspace/controllers/spec/enforcer/statefulset_spec/AllStatefulSetsSpecEnforcer.go:169
reactive-tech.io/kubegres/controllers/spec/enforcer/statefulset_spec.(*AllStatefulSetsSpecEnforcer).EnforceSpec
	/workspace/controllers/spec/enforcer/statefulset_spec/AllStatefulSetsSpecEnforcer.go:109
reactive-tech.io/kubegres/controllers.(*KubegresReconciler).enforceAllStatefulSetsSpec
	/workspace/controllers/kubegres_controller.go:146
reactive-tech.io/kubegres/controllers.(*KubegresReconciler).enforceSpec
	/workspace/controllers/kubegres_controller.go:138
reactive-tech.io/kubegres/controllers.(*KubegresReconciler).Reconcile
	/workspace/controllers/kubegres_controller.go:100
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.1
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:198
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99
2021-09-06T16:20:22.504Z	DEBUG	controller-runtime.manager.events	Warning	{"object": {"kind":"Kubegres","namespace":"default","name":"mypostgres","uid":"74432c5f-c21f-4e23-967d-7f0ca4441dbc","apiVersion":"kubegres.reactive-tech.io/v1","resourceVersion":"106964060"}, "reason": "StatefulSetSpecEnforcementTimedOutErr", "message": "Last Spec enforcement attempt has timed-out for a StatefulSet. You must apply different spec changes to your Kubegres resource since the previous spec changes did not work. Until you apply it, most of the features of Kubegres are disabled for safety reason.  'StatefulSet's name': mypostgres-3, 'One or many of the following specs failed: ': StorageClassSize: 10Gi - Spec enforcement timed-out

The status of the Kubegres CR shows:

    "status": {
        "blockingOperation": {
            "operationId": "Enforcing StatefulSet's Spec",
            "statefulSetOperation": {
                "instanceIndex": 3,
                "name": "mypostgres-3"
            },
            "statefulSetSpecUpdateOperation": {
                "specDifferences": "StorageClassSize: 10Gi"
            },
            "stepId": "StatefulSet's spec is updating",
            "timeOutEpocInSeconds": 1630945221
        },
        "enforcedReplicas": 3,
        "lastCreatedInstanceIndex": 3,
        "previousBlockingOperation": {
            "operationId": "Replica DB count spec enforcement",
            "statefulSetOperation": {
                "instanceIndex": 3,
                "name": "mypostgres-3"
            },
            "statefulSetSpecUpdateOperation": {},
            "stepId": "Replica DB is deploying",
            "timeOutEpocInSeconds": 1630944608
        }
    }

How do I recover from this scenario?

How do I delete the postgres instances?

Hi, sorry for the dumb question, but after I initialize a cluster of PostgreSql instances, how do I remove it from my k8s cluster? I deleted the statefulset, pvc, pod, etc, but there seems to be something still in my system that knows about the clsuter I'm trying to delete, because when I reaply the example YAML it says:
kubegres.kubegres.reactive-tech.io/db unchanged

Thanks in advance

How to check which pod is in "Primary state"

Hi,
How to check with kubectl get pod ... display information about master and replicas pods?

Postgres args

Currently there is no way to set args for creating the database, e.g:

# enable logical decoding
args: ["-cwal_level=logical", "-cmax_wal_senders=1", "-cmax_replication_slots=1"]

If there are not sufficient nodes in a cluster, Kubegres to deploy multiple PostgreSql instances on same node

Hi,
1st replica of postgres stay in pending: mypostgres-2-0 in Kubernetes dashboard with this message:
0/1 nodes are available: 1 node(s) didn't match pod affinity/anti-affinity, 1 node(s) didn't match pod anti-affinity rules.

Can kubegress use custom PV as a main storage?

Hi, I'd like to use GlusterFs or NFS Volume as a main storage for the database. Is it possible?

Thanks in advance

Persistent Volumes not reused/Stateful Sets not working properly

Hi, couple of days ago I deployed a two-node cluster with the help of the Kubegres operator. I followed the getting started instructions and ended up with two Stateful Sets (mypostgres-1 as primary and mypostgres-2). So far so good. I wanted to enable statistics so I updated the base-kubegres-config map with the following lines in the postgres.conf section:

shared_preload_libraries = 'pg_stat_statements' # (change requires restart)
pg_stat_statements.max = 10000
pg_stat_statements.track = all

I could not figure out how to restart the pods, the change of the config map was not picked up by the kubegres operator ( I am not sure if it is supposed to) and also kubectl upgrade would not work with the kubegres kind. As there was no documentation about how to do it properly I decided to kill the pods expecting that they would be recreated according to the standard stateful set policy. I started with the slave pod mypostgres-2-0 and it was recreated properly and picked up the changes. When I killed the master (mypostgres-1-0), mypostgres-2 was promoted to be a master, which worked as expected, but mypostgres-1was not recreated. After a while, and quite a bit of confusion on the Kubernetes part, a new pod mypostgres-3 was brought up.

Now, I had three PVCs, postgres-db-mypostgres-1-0, postgres-db-mypostgres-2-0, postgres-db-mypostgres-3-0 ... and the first one was bound but not used.

I expected that either pod 1 would be recreated according to the kubernetes stateful set rules, or if that is not possible for some reason, at least the disk would be reused.

Just to be sure, I killed the new master (#2) and after a while I ended up with #3 as master and brand new pod #4 as slave (and four allocated disks)

Regardless of me doing it the right or wrong way, it was a good test because it showed what would happen if a pod dies.

Am I missing something or this is how it is supposed to work? If the latter is the case, it does not look right.

thanks,

could kubegres made upgrade major posgresql version without downtime?

Hello guys, thanks for your great work ;) kubegres look like simple operator which cover most of my use cases, but
could kubegres made upgrade major posgresql version without downtime?

will you run pg_upgrade if I will change PostgreSQL image in custom resource to v14?

Wrong apiVersion in getting started examples

When I apply the example resource as is I get the following error:

error: unable to recognize "resource.yaml": no matches for kind "Kubegres" in version "io.reactive-tech/v1"

When I modify the apiVersion to kubegres.reactive-tech.io/v1, it works.

I also think the namespace requires an indent, so it will be inside metadata.

Trying getting started get error when trying to create cluster on minikube

I'm testing Kubegres on minikube v1.19.0. When I'm running installation no problem:

 2021-04-16 10:58:08 ⌚  paulem-latitude-5590 in ~/projects
○ → kubectl get all -n kubegres-system
NAME                                              READY   STATUS    RESTARTS   AGE
pod/kubegres-controller-manager-999786dd6-m5shd   2/2     Running   0          11m

NAME                                                  TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/kubegres-controller-manager-metrics-service   ClusterIP   10.98.110.218   <none>        8443/TCP   11m

NAME                                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kubegres-controller-manager   1/1     1            1           11m

NAME                                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/kubegres-controller-manager-999786dd6   1         1         1       11m

 2021-04-16 10:58:14 ⌚  paulem-latitude-5590 in ~/projects

I create the secret: no problem
When I try to create the cluster I get this error:

○ → kubectl apply -f my-postgres.yaml
error: unable to recognize "my-postgres.yaml": no matches for kind "Kubegres" in version "io.reactive-tech/v1"

my-postgres.yaml.gz

Expand storage of an existing cluster of Postgres

Currently, Kubernetes does not allow the support for volume expansion through StatefulSets. And when a PVC is manually updated, it is unable to automatically update a StatefulSet.

To allow to expand the storage of an existing cluster of Postgres, we have to wait on the release of the Kubernetes enhancement kubernetes/enhancements#2842 which aims in addressing this issue by allowing support for volume expansion through StatefulSets.

Once that feature is available in Kubernetes, this feature can be available in Kubegres via this ticket.

wal_level

How can I set wal_level to logical?

PVC not getting bound

Hi,

I have deployed the latest version and was trying to create a cluster but it is getting stuck. What is missing ?

PVC created is as follows

kubectl describe pvc postgres-db-myssodb-1-0
Name: postgres-db-myssodb-1-0
Namespace: aaa
StorageClass: manual
Status: Pending
Volume:
Labels: app=myssodb
index=1
Annotations:
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Filesystem
Used By: myssodb-1-0
Events:
Type Reason Age From Message

---

Warning ProvisioningFailed 14m (x122 over 44m) persistentvolume-controller no volume plugin matched name: kubernetes.io/no-provisioner
Warning ProvisioningFailed 6m25s (x21 over 11m) persistentvolume-controller no volume plugin matched name: kubernetes.io/no-provisioner
Warning ProvisioningFailed 1s (x23 over 5m31s) persistentvolume-controller no volume plugin matched name: kubernetes.io/no-provisioner

PV is as follows
kubectl describe pv myssodb-nfs-pv
Name: myssodb-nfs-pv
Labels: app=myssodb
Annotations:
Finalizers: [kubernetes.io/pv-protection]
StorageClass: manual
Status: Available
Claim:
Reclaim Policy: Retain
Access Modes: RWX
VolumeMode: Filesystem
Capacity: 15Gi
Node Affinity:
Message:
Source:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: [server]
Path: [path]
ReadOnly: false
Events:

And my kubegre yaml is as follows

apiVersion: kubegres.reactive-tech.io/v1
kind: Kubegres
metadata:
name: myssodb
namespace: aaa

spec:

replicas: 2
image: postgres:13.2
port: 5432

database:
size: 15Gi
storageClassName: manual
volumeMount: /var/lib/postgresql/data

Could Kubegres supports additional volumes with shared PVC?

We have a service that adds cert files to a mount path, how can we mount the same path to Postgres pod using Kubegres.

Backup failed

We enabled the backup function as documented here and get the following error.

28/08/2021 00:00:01 - Starting DB backup of Kubegres resource postgres into file: /var/lib/backup/postgres-backup-28_08_2021_00_00_01.gz
28/08/2021 00:00:01 - Running: pg_dumpall -h postgres-replica -U postgres -c | gzip > /var/lib/backup/postgres-backup-28_08_2021_00_00_01.gz
pg_dump: error: Dumping the contents of table "table" failed: PQgetResult() failed.
pg_dump: error: Error message from server: ERROR:  canceling statement due to conflict with recovery
DETAIL:  User query might have needed to see row versions that must be removed.
pg_dump: error: The command was: COPY public.table(column1, column2) TO stdout;
pg_dumpall: error: pg_dump failed on database "db", exiting
28/08/2021 00:00:01 - DB backup completed for Kubegres resource postgres into file: /var/lib/backup/postgres-backup-28_08_2021_00_00_01.gz

In this Stackoverflow post they recommend that we should activate hot_standby_feedback or increase max_standby.
Is there a suggested solution from your side to overcome this issue?

Restore database from pv backup

Hi - first of all, fantastic repository. Great to see all the hard work you've put into this.

I have in mind a scenario and would like to know to what extent this is supported already or would need to be a feature request:

After a catastrophic event, the entire cluster is deleted. However, the administrator specified a reclaim policy of retain on the storage class that backed the original database volumes, and the cloud provider has retained these volumes, which are immediately re-mounted to a new cluster as persistent volumes. Since the old cluster has been destroyed, the administrator would like to quickly restore the database by referencing the new volume he just created in a new Kubegres deployment.

Something like this comes to mind:

apiVersion: kubegres.reactive-tech.io/v1
kind: Kubegres
...
restoreFromVolume: [pv-name]

Thanks!

PVC are not deleted

I'm not sure if it's a bug or feature, but other operators like the ECK are deleting the PVC's when I delete the stateful set. So it would be great if Kubegres can support deleting the PVC's, too.

version 1.9 creates cluster IP services with "None". The Documentation doesn't address this

on the getting started page it says:

In this example, Kubegres created 2 Kubernetes clusterIP services using the name defined in YAML (e.g. "mypostgres"):

a Kubernetes service "mypostgres" allowing to access to the Primary PostgreSql instances
a Kubernetes service "mypostgres-replica" allowing to access to the Replica PostgreSql instances

Consequently, a client app running inside a Kubernetes cluster, would use the hostname "mypostgres" to connect to the Primary PostgreSql for read and write requests, and optionally it can also use the hostname "mypostgres-replica" to connect to any of the available Replica PostgreSql for read requests.

---

Which is incorrect. The Services created are unusable, apparently by design. They have no Cluster IP address. See issue #1.
The documentation needs to cover manually creating the services.

How to create creating a custom user and a database by overriding primary_init_script.sh

I'm trying to override primary_init_script.sh in order to create creating a custom user and a database.

Based on the tutorial, it works file (3 Pods are created and "postgres" database are accessible by "postgres" user).

However if I modified yaml files for creating the user "user1" with password "user1Psw" and database "testdb", it does not affect. Database "postgres" and user "postgres" exist, but "user1" and "testdb" are not created.

Procedure:

1. Recreate Kube cluster
2. $ kubectl apply -f https://raw.githubusercontent.com/reactive-tech/kubegres/v1.1/kubegres.yaml
3. $ kubectl apply -f my-postgres-secret.yaml
4. $ kubectl apply -f my-postgres-config.yaml
5. $ kubectl apply -f my-postgres.yaml
6. $ kubectl exec -it mypostgres-1-0 -- /bin/bash
7. # psql -l -U postgres -> return 3 databases "postgres", "template0" and "template1"

I attached yaml files.
The modification of yaml files is based on the site:

• postgres-secret.yaml
  Append stringData.myDbUserPassword: user1Psw.
• my-postgres-config.yaml
  Newly created using Example. Then changed followings:
  namespace: default
customDatabaseName="testdb"
customUserName="user1"
• my-postgres.yaml
  Append spec.customConfig: mypostgres-conf.
  Append

     - name: POSTGRES_MY_DB_PASSWORD
       valueFrom:
          secretKeyRef:
             name: mypostgres-secret
             key: myDbUserPassword
  

Could you kindly help me to resolve this issue?

yamlfiles.zip

Passing custom annotations to backup pods

This is a ticket based on #14
I would like to pass custom annotations to backup pods so that third party apps can access backup pods like they do with the master and replica pods.

Missing Docs on Volumes

Great product!

I'd like to know more on where and how the postgres data is stored. I understand this has to do with a PVC, but more in the docs would be great. I'd be happy to issue a PR for that.

I see mentions of "pvc"s and "persistent volume claim"s in the kubegres.yaml, but how does this work? For example if I am running locally on minikube, where is the data stored? E.g.:

$ kubectl get sc              

NAME                 PROVISIONER                RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
standard (default)   k8s.io/minikube-hostpath   Delete          Immediate           false                  12d

What if in production I wanted to store the data on, say, a GCE bucket? How would I go about doing this?

initdb: error: directory "/var/lib/postgresql/data" exists but is not empty

I followed the getting-started tutorial and got the following yaml:

apiVersion: v1
kind: Secret
metadata:
  name: mypostgres-secret
type: Opaque
stringData:
  superUserPassword: postgresSuperUserPsw
  replicationUserPassword: postgresReplicaPsw
---
apiVersion: kubegres.reactive-tech.io/v1
kind: Kubegres
metadata:
  name: mypostgres
spec:
   replicas: 2
   image: postgres:13.2

   database:
      size: 200Mi
      storageClassName: csi-cinder-high-speed

   env:
      - name: POSTGRES_PASSWORD
        valueFrom:
           secretKeyRef:
              name: mypostgres-secret
              key: superUserPassword

      - name: POSTGRES_REPLICATION_PASSWORD
        valueFrom:
           secretKeyRef:
              name: mypostgres-secret
              key: replicationUserPassword

But I'm getting the following error message:

The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

initdb: error: directory "/var/lib/postgresql/data" exists but is not empty
It contains a lost+found directory, perhaps due to it being a mount point.
Using a mount point directly as the data directory is not recommended.
Create a subdirectory under the mount point.

Any idea how I can fix this issue?

master pod failure

When the master pod(1) is crashed, the second pod starts again as the master and a new pod(4) is created because the first pod is crashed. While the second pod and the fourth pod work synchronously, the third pod works alone.

Additional Documentation

Hi, this is a really great project, happy to see someone doing this from gitops principles. Are there any near term plans for additional documentation, specifically around typical use cases that might be needed in production like restoring from backup and expanding storage?

Allow disabling automatic failover and allow to manually promote a Pod as Primary

We would like the option to disable the automatic failover feature for a cluster of Postgres so that when manually maintaining a node (e.g. restarting it, patching it, ...) the Kubegres operator does not automatically failover a Primary or Replica instances during that maintenance.

Additionally, we would like to be able to manually promote a Pod as a Primary Pod by setting the name of a Replica Pod name in the YAML.

Allow kubegres cluster to run on secure Kubernetes environments (Pod security policies)

Hi,

First thanks for your work on a Postgres cluster Kubernetes Operator.

We are deploying Kubernetes clusters in a secure by design manner using Rancher's RKE2 (aka RKE Government v1.20.11+rke2r2)

This creates a cluster with a hardened Pod Security policy which forbids, among others pods, from running as root.

This implies the workloads must have security context defined with at least these to settings (1001 is the postgres image running user):

securityContext:
  - runAsNonRoot: true
  - runAsUser: 1001

Looking at the baseConfigMap I also see you do some chown postgres:postgres when copying data from primary to replicas. I guess these would fail with these settings.

In "enterprise" setups this is actually not needed as CSI provisioned Pas do belong to the pod running user AFAIK.

While I understand this level of security is not needed by everyone and some users want to be able to run things in smaller clusters where security is not mandatory, I was wondering if it was possible to have some boolean flag in the CR yaml (ie: hardened : true/false) that would allow the workload to run in hardened PSP clusters.

This flag would basically use an alternate baseConfigMap with no chown commands and the added securityContext to the statefulSet.

FYI, right now we run a single node not replicated postgres server with the above securityContext with no issues whatsoever.

Let me know what you think about this.

many thanks,

Eric

In Kubegres yaml, add the field "imagePullSecrets"

In Kubegres yaml, we would like the possibility to specify the field "imagePullSecrets" so that we can pull Postgres images from a private repo.

This feature was kindly suggested by the Reddit user hardwaresofton, as follows:
" Wanting a customized image is likely to be fairly common, postgres has a lot of extensions that are nice -- if you're going to let people specify custom images you might want to let them specify imagePullSecrets to use. "

Kubegres 1.1 on minikube running but service mypostgres and mypostgres-replica don't assing external IP

Hi,
I deployed successfully on minikube version: v1.19.0

kubectl get all
NAME                 READY   STATUS    RESTARTS   AGE
pod/mypostgres-1-0   1/1     Running   0          24m
pod/mypostgres-2-0   1/1     Running   0          23m
pod/mypostgres-3-0   1/1     Running   0          22m

NAME                         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
service/kubernetes           ClusterIP   10.96.0.1    <none>        443/TCP    36m
service/mypostgres           ClusterIP   None         <none>        5432/TCP   23m
service/mypostgres-replica   ClusterIP   None         <none>        5432/TCP   22m

NAME                            READY   AGE
statefulset.apps/mypostgres-1   1/1     24m
statefulset.apps/mypostgres-2   1/1     23m
statefulset.apps/mypostgres-3   1/1     22m

But I can't access Postgres db:

○ → minikube service mypostgres
|-----------|------------|-------------|--------------|
| NAMESPACE |    NAME    | TARGET PORT |     URL      |
|-----------|------------|-------------|--------------|
| default   | mypostgres |             | No node port |
|-----------|------------|-------------|--------------|
😿  service default/mypostgres has no node port

kubectl get services 
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
kubernetes           ClusterIP   10.96.0.1    <none>        443/TCP    46m
mypostgres           ClusterIP   None         <none>        5432/TCP   33m
mypostgres-replica   ClusterIP   None         <none>        5432/TCP   33m

Regards,
PaulemPG

Add support to manage volumes from Kubegres YAML

By default, the docker image of official PostgreSQL only configure 64MB for /dev/shm (check the Caveats section in Docker hub. It could cause problem for larger database.

# This is taken from the pod generated by Kubegres
Filesystem                  Size  Used Avail Use% Mounted on
shm                          64M   64K   64M   1% /dev/shm

There is one possible method, it is to increase the /dev/shm inside the container to be the same size as the host OS, default to 50% of RAM.
It is mentioned in Stackoverflow.

But currently, Kubegres Kind definition only allowed "volumeMount:". It would not recognize "volumes:" and "volumeMounts:", so it seems there is no way to increase the value for the shared memory.

Adding custom annotations?

I want to use Linkerd and Bank Vault with Kubegres. But Kubegres doesn't seem to support custom annotations. Would you consider this feature?

if adding custom annotations is supported, I'd use Vault to inject secrets into the environment without relying on creating a secret. But right now it looks like replica pods would fail to start without valueFrom.secretKeyRef. It keeps asking Password in the logs.

spec:
  env:
    - name: POSTGRES_PASSWORD
      value: gffdggfdgfdf (Would use #vault:secret/data/path#DB_PASSWORD if custom annotations can be used)

    - name: POSTGRES_REPLICATION_PASSWORD
      value: fgdgfdgdffd

Database not getting created on NFS persistent volume

Hi Alex,

This operator works fine when configured to use the node pv storage (hostpath), but as soon as I try to configure with nfs pv it does not work. The default database postgres is also not getting created. I am sure my nfs server and storage have no issues as I have tried the same with postgres image (without the operator) and it works fine. All required databases were properly created in my nfs pv.
Any idea, why it won't work with operator ? And yeah, there are absolutely no errors in the logs (controller logs, pod logs etc.) The primary pod logs show that the database is ready to accept connections, and replication pod init fails as there is nothing to replicate. Does the operator support NFS storage ?

Service Type in CR

Hi,

It would be great to have a way to change primary and replica service types from default ClusterIP to LoadBalancer and NodePort. Right now I'm using nginx ingress TCP loadbalancing but it adds complexity and dependency to ingress pod.

Something like below would be great;

apiVersion: kubegres.reactive-tech.io/v1
kind: Kubegres
metadata:
  name: mypostgres
  namespace: default

spec:

   replicas: 3
   image: postgres:13.2

  service:
    primary:
      type: LoadBalancer
    replica:
      type: NodePort
      nodePort: 30543

   database:
      size: 200Mi

   env:
      - name: POSTGRES_PASSWORD
        valueFrom:
           secretKeyRef:
              name: mypostgres-secret
              key: superUserPassword

      - name: POSTGRES_REPLICATION_PASSWORD
        valueFrom:
           secretKeyRef:
              name: mypostgres-secret

If method at #44 wouldn't require UID of original service I could create a helm chart that can create external facing service with pre-install hook.

When the field "spec.database.storageClassName" is omitted in Kubegres YAML, Kubegres operator should assign the default storageClass to the deployed PostgreSql cluster

Hi, I found that kubeges's controller segfaults and stays down when I omit the storageClassName property. It recovers once the offending Kubegres object is removed.

Expected behavior: Kubegres would request PVCs with the default storage class, i.e. simply omit the storageClassName field when generating its PVCs.

Steps to reproduce: Apply the following yaml:

apiVersion: v1
kind: Secret
metadata:
  name: postgres
type: Opaque
stringData:
  rootpasswd: foo
  replpasswd: bar
  userpasswd: baz
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: postgres-conf
data:
  primary_init_script.sh: |
    #!/bin/bash
    set -e
    psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
    CREATE DATABASE $MY_POSTGRES_USER;
    CREATE USER $MY_POSTGRES_USER WITH PASSWORD '$MY_POSTGRES_PASSWORD';
    GRANT ALL PRIVILEGES ON DATABASE $MY_POSTGRES_USER to $MY_POSTGRES_USER;
    EOSQL
---
apiVersion: kubegres.reactive-tech.io/v1
kind: Kubegres
metadata:
  name: postgres
spec:
  image: postgres:13.2
  port: 5432
  replicas: 1
  database:
    size: 10Gi
  env:
    - name: POSTGRES_PASSWORD
      valueFrom:
        secretKeyRef:
          name: postgres
          key: rootpasswd
    - name: POSTGRES_REPLICATION_PASSWORD
      valueFrom:
        secretKeyRef:
          name: postgres
          key: replpasswd
    - name: MY_POSTGRES_USER
      value: notisvc
    - name: MY_POSTGRES_PASSWORD
      valueFrom:
        secretKeyRef:
          name: postgres
          key: userpasswd
  customConfig: postgres-conf

And observe how the manager container of kubegres-controller-manager panics immediately:

manager
2021-06-07T17:14:53.192Z    INFO controllers.Kubegres =======================================================
manager
2021-06-07T17:14:53.192Z    INFO controllers.Kubegres =======================================================
manager
2021-06-07T17:14:54.192Z    INFO controllers.Kubegres KUBEGRES {"name": "postgres", "Status": {"blockingOperation":{"statefulSetOperation":{},"statefulSetSpecUpdateOperation":{}},"previousBlockingOperation":{"statefulSetOperation":{},"statefulSetSpecUpdateOperation":{}}}}
manager
2021-06-07T17:14:54.192Z    INFO controllers.Kubegres Corrected an undefined value in Spec. {"spec.database.volumeMount": "New value: /var/lib/postgresql/data"}
manager
2021-06-07T17:14:54.192Z    INFO controllers.Kubegres Updating Kubegres Spec {"name": "postgres"}
manager
2021-06-07T17:14:54.192Z    DEBUG controller-runtime.manager.events Normal {"object": {"kind":"Kubegres","namespace":"kubegres-crash","name":"postgres","uid":"c4fcc9d9-e21c-4af3-9e33-18c1f980654e","apiVersion":"kubegres.reactive-tech.io/v1","resourceVersion":"62606471"}, "reason": "SpecCheckCorrection", "message": "Corrected an undefined value in Spec. 'spec.database.volumeMount': New value: /var/lib/postgresql/data"}
manager
E0607 17:14:54.207006 1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
manager
goroutine 403 [running]:
manager
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x15f20e0, 0x22f5700)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:74 +0xa6
manager
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:48 +0x89
manager
panic(0x15f20e0, 0x22f5700)
manager
    /usr/local/go/src/runtime/panic.go:969 +0x1b9
manager
reactive-tech.io/kubegres/controllers/states.(*DbStorageClassStates).getSpecStorageClassName(...)
manager
    /workspace/controllers/states/DbStorageClassStates.go:80
manager
reactive-tech.io/kubegres/controllers/states.(*DbStorageClassStates).GetStorageClass(0xc000ae8b50, 0x8, 0xc000ea7be0, 0x19)
manager
    /workspace/controllers/states/DbStorageClassStates.go:62 +0x47
manager
reactive-tech.io/kubegres/controllers/states.(*DbStorageClassStates).loadStates(0xc000ae8b50, 0x8, 0xc000ea7be0)
manager
    /workspace/controllers/states/DbStorageClassStates.go:46 +0x2f
manager
reactive-tech.io/kubegres/controllers/states.loadDbStorageClass(...)
manager
    /workspace/controllers/states/DbStorageClassStates.go:39
manager
reactive-tech.io/kubegres/controllers/states.(*ResourcesStates).loadDbStorageClassStates(0xc000ae9370, 0x14d91f5, 0x8)
manager
    /workspace/controllers/states/ResourcesStates.go:75 +0xe5
manager
reactive-tech.io/kubegres/controllers/states.(*ResourcesStates).loadStates(0xc000ae9370, 0xc000843200, 0x0)
manager
    /workspace/controllers/states/ResourcesStates.go:46 +0x45
manager
reactive-tech.io/kubegres/controllers/states.LoadResourcesStates(...)
manager
    /workspace/controllers/states/ResourcesStates.go:40
manager
reactive-tech.io/kubegres/controllers/ctx/resources.CreateResourcesContext(0xc0005d22c0, 0x19a3880, 0xc000eaee70, 0x19ac260, 0xc0001ba2e0, 0x19b6f60, 0xc00044bae0, 0x19a06c0, 0xc00054a2c0, 0x0, ...)
manager
    /workspace/controllers/ctx/resources/ResourcesContext.go:105 +0x6c5
manager
reactive-tech.io/kubegres/controllers.(*KubegresReconciler).Reconcile(0xc00054a300, 0x19a3880, 0xc000eaee70, 0xc0006bf2c0, 0xe, 0xc0006bf2a0, 0x8, 0xc000eaee70, 0x40a1ff, 0xc000030000, ...)
manager
    /workspace/controllers/kubegres_controller.go:74 +0x17f
manager
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0005448c0, 0x19a37c0, 0xc00029a000, 0x16482e0, 0xc001000460)
manager
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263 +0x317
manager
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0005448c0, 0x19a37c0, 0xc00029a000, 0x203000)
manager
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235 +0x205
manager
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.1(0x19a37c0, 0xc00029a000)
manager
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:198 +0x4a
manager
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1()
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185 +0x37
manager
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0004b0f50)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
manager
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000afbf50, 0x196dee0, 0xc000991620, 0xc00029a001, 0xc0003621e0)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0xad
manager
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0004b0f50, 0x3b9aca00, 0x0, 0xc00029a001, 0xc0003621e0)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x98
manager
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext(0x19a37c0, 0xc00029a000, 0xc0001547c0, 0x3b9aca00, 0x0, 0xc000704501)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185 +0xa6
manager
k8s.io/apimachinery/pkg/util/wait.UntilWithContext(0x19a37c0, 0xc00029a000, 0xc0001547c0, 0x3b9aca00)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99 +0x57
manager
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
manager
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:195 +0x4e7
manager
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
manager
    panic: runtime error: invalid memory address or nil pointer dereference
manager
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x14aafe7]
manager
goroutine 403 [running]:
manager
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:55 +0x10c
manager
panic(0x15f20e0, 0x22f5700)
manager
    /usr/local/go/src/runtime/panic.go:969 +0x1b9
manager
reactive-tech.io/kubegres/controllers/states.(*DbStorageClassStates).getSpecStorageClassName(...)
manager
    /workspace/controllers/states/DbStorageClassStates.go:80
manager
reactive-tech.io/kubegres/controllers/states.(*DbStorageClassStates).GetStorageClass(0xc000ae8b50, 0x8, 0xc000ea7be0, 0x19)
manager
    /workspace/controllers/states/DbStorageClassStates.go:62 +0x47
manager
reactive-tech.io/kubegres/controllers/states.(*DbStorageClassStates).loadStates(0xc000ae8b50, 0x8, 0xc000ea7be0)
manager
    /workspace/controllers/states/DbStorageClassStates.go:46 +0x2f
manager
reactive-tech.io/kubegres/controllers/states.loadDbStorageClass(...)
manager
    /workspace/controllers/states/DbStorageClassStates.go:39
manager
reactive-tech.io/kubegres/controllers/states.(*ResourcesStates).loadDbStorageClassStates(0xc000ae9370, 0x14d91f5, 0x8)
manager
    /workspace/controllers/states/ResourcesStates.go:75 +0xe5
manager
reactive-tech.io/kubegres/controllers/states.(*ResourcesStates).loadStates(0xc000ae9370, 0xc000843200, 0x0)
manager
    /workspace/controllers/states/ResourcesStates.go:46 +0x45
manager
reactive-tech.io/kubegres/controllers/states.LoadResourcesStates(...)
manager
    /workspace/controllers/states/ResourcesStates.go:40
manager
reactive-tech.io/kubegres/controllers/ctx/resources.CreateResourcesContext(0xc0005d22c0, 0x19a3880, 0xc000eaee70, 0x19ac260, 0xc0001ba2e0, 0x19b6f60, 0xc00044bae0, 0x19a06c0, 0xc00054a2c0, 0x0, ...)
manager
    /workspace/controllers/ctx/resources/ResourcesContext.go:105 +0x6c5
manager
reactive-tech.io/kubegres/controllers.(*KubegresReconciler).Reconcile(0xc00054a300, 0x19a3880, 0xc000eaee70, 0xc0006bf2c0, 0xe, 0xc0006bf2a0, 0x8, 0xc000eaee70, 0x40a1ff, 0xc000030000, ...)
manager
    /workspace/controllers/kubegres_controller.go:74 +0x17f
manager
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0005448c0, 0x19a37c0, 0xc00029a000, 0x16482e0, 0xc001000460)
manager
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263 +0x317
manager
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0005448c0, 0x19a37c0, 0xc00029a000, 0x203000)
manager
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235 +0x205
manager
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.1(0x19a37c0, 0xc00029a000)
manager
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:198 +0x4a
manager
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1()
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185 +0x37
manager
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0004b0f50)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
manager
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000afbf50, 0x196dee0, 0xc000991620, 0xc00029a001, 0xc0003621e0)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0xad
manager
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0004b0f50, 0x3b9aca00, 0x0, 0xc00029a001, 0xc0003621e0)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x98
manager
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext(0x19a37c0, 0xc00029a000, 0xc0001547c0, 0x3b9aca00, 0x0, 0xc000704501)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185 +0xa6
manager
k8s.io/apimachinery/pkg/util/wait.UntilWithContext(0x19a37c0, 0xc00029a000, 0xc0001547c0, 0x3b9aca00)
manager
    /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99 +0x57
manager
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
manager
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:195 +0x4e7

Node scheduling

Is it possible to specify node scheduling parameters? For example, if I want to ensure the pods schedule on on-demand nodes (instead of spot nodes) or if I want to add a toleration so it's able to schedule on a node with a taint.

Exposing kubegres as NodePort

Hi,

I am trying to expose kubegres to traffic external to the cluster by following the guidance provided here and have created a service before creating the resource postgres of kind kubegres.

I have set the service to be the below:

apiVersion: 1
kind: Service
metadata:
  name: postgres
  namespace: kubegres
spec:
  type: NodePort
  selector:
    app: postgres
    replicationRole: primary
  ports:
  - port: 5432
    targetPort: 5432
    nodePort: 30432
    protocol: TCP

I can't seem to connect to the pod via <NODEIP>:30432 however. I can see in the pod logs of the primary that the database system is ready to accept connections. If I use kubectl port forwarding I can connect an SQL client and run queries against it like create db and see some relevant output when tailing the pod logs, so I know its up and responding.

Am I missing something? How do I expose this db to external traffic?

NB: If it matters, I have replicas set to 1 for testing

failover - Skipping PostgreSql is already a Primary

Hi, I'm trying this operator, it starts and works but as soon as the primary pod goes down, problems start:
the new primary pods run the init container, but the log is always like this:
15/07/2021 11:31:45 - Attempting to promote a Replica PostgreSql to Primary... 15/07/2021 11:31:45 - Skipping as this PostgreSql is already a Primary since the file '/var/lib/postgresql/data/pgdata/standby.signal' does not exist.

So the replication user is not created and replicas stop working.

How can I solve it?

Thanks in advance

How is postgres service Created?

I don't think it is entirely clear from the docs how the postgres service is created. Is it from the metadata.name field in the kubegres yaml? In the docs this is set to mypostgres.

I changed this to "postgres," took down the pods (using skaffold) and checked there were no postgres services. I brought the app back up and the mypostgres service was created a-new.

I installed kubegres, by downloading the yaml from https://raw.githubusercontent.com/reactive-tech/kubegres/v1.10/kubegres.yaml and listing it for skaffold.

I grepped my repo for "mypostgres" and there are no instance of it left, so I'm not sure how a service with this name is being created.

How to recover from a full disk

In the real world, there is always a risk of exceeding available storage. I ran the getting started configuration, and then ran the following test case to exceed the default storage of 200Mi. The PODs kept crashing. I tired to increase the storage space in the CR, but it didn't work. What is the proper procedure to recover this real work scenario?

CREATE DATABASE tgates_test_db1;

\c tgates_test_db1

CREATE TABLE scale_data (
section NUMERIC NOT NULL,
id1     NUMERIC NOT NULL,
id2     NUMERIC NOT NULL
);

INSERT INTO scale_data
SELECT sections.*, gen.*, CEIL(RANDOM()*100)
FROM GENERATE_SERIES(1, 300)     sections,
     GENERATE_SERIES(1, 900000) gen
WHERE gen <= sections * 3000;

The backup file name does not contain the kubegres resource name

When a backup happens, the gun zipped file does not contain the kubegres YAML resource name which identifies the PostgreSql cluster's name.

[SPEC] Add Support for resource requests / limits

Currently, the deployed pods don't have any resource requests/limits assigned to them and currently there is no way to add the same via specs.

A possible way to do the same in the following format:

spec.primary.resources.limits.cpu
spec.primary.resources.limits.memory
spec.primary.resources.requests.cpu
spec.primary.resources.requests.memory

spec.replica.resources.limits.cpu
spec.replica.resources.limits.memory
spec.replica.resources.requests.cpu
spec.replica.resources.requests.memory

Ref: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/

Where does init_script.sh log to?

I have added this to my Kubegres CRD (in spec.customConfig) to override the init_script.sh . I'd like to know if it's working. If I echo in the overridden script, where can I find the output? I tried looking at the pods' logs, and the CRD Kubegres doesn't have logs:

apiVersion: kubegres.reactive-tech.io/v1
kind: Kubegres
metadata:
  name: mypostgres
  namespace: default
spec:
   replicas: 3
   image: postgres:13.2
   database:
      size: 200Mi
   customConfig: mypostgres-conf

   env:
      - name: POSTGRES_PASSWORD
        valueFrom:
           secretKeyRef:
              name: mypostgres-secret
              key: superUserPassword

      - name: POSTGRES_REPLICATION_PASSWORD
        valueFrom:
           secretKeyRef:
              name: mypostgres-secret
              key: replicationUserPassword

In the ConfigMap mypostgres-conf.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mypostgres-conf
  namespace: default
data:
  primary_init_script.sh: |
    #!/bin/bash
    set -e
    
    # This script assumes that the env-var $POSTGRES_MY_DB_PASSWORD contains the password of the custom user to create.
    # You can add any env-var in your Kubegres resource config YAML.

    dt=$(date '+%d/%m/%Y %H:%M:%S');
    echo "$dt - Running init script the 1st time Primary PostgreSql container is created...";

    # customDatabaseName="my_app_db"
    # customUserName="my_username"

    # echo "$dt - Running: psql -v ON_ERROR_STOP=1 --username $POSTGRES_USER --dbname $POSTGRES_DB ...";

    # psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
    # CREATE DATABASE $customDatabaseName;
    # CREATE USER $customUserName WITH PASSWORD '$POSTGRES_MY_DB_PASSWORD';
    # GRANT ALL PRIVILEGES ON DATABASE "$customDatabaseName" to $customUserName;
    # EOSQL

    # echo "$dt - Init script is completed";

The update of an existing Kubegres resource fails if the field 'resources' contains a value with a decimal point

Thank you for maintaining this repo.

I am looking for steps/recommendations for upgrading between minor versions and major versions.

I am guessing that upgrading between minor versions is as simple as changing the container image, i.e. postgres:13.2 -> postgres:13.4.

Now that the official image for Postgres 14 is available, are there any steps that need to be followed to go from postgres:13.2 -> postgres:14.0 ?

Cheers.

Disable expand Storage

I just started using the operator and I think its got a really good potential. How do you expand the storage class for new Pods in the same Statefulset without affecting existing pods?

Select Nodes for statefulsets

Hi, Thank you for your efforts and time to develop and manage this operator. I had a specific requirement of selecting specific nodes while creating the db cluster pods, Can I specify the nodeSelector and then key: value in the YAML of "kind :Kubegres" ? I have created labels for nodes and would like to use those for scheduling the pods. Thanks !

kubegres-controller-manager keeps restarting but there's no error in container log

I'm new to kubegres, so this might probably be a miss-configuration or sth.

After I setup a cluster following the getting-started tutorial, the kubegres-controller-manager pod keeps restarting, while the postgres instance pods seems running ok:

I checked the log of two containers running inside kubegres-controller-manager pod, neither of them contains error:

manager container:

kube-rbac-proxy container:

But the postgres cluster and service are all running well, I can access the database with no problem. (I port-forward the postgres service to access the db)

I'm not sure what's going on, please help me get the kubegres-controller-manager working

Add one or many sidecar container options in Kubegres YAML

Prometheus is very popular with Kuberentes users and it'll be helpful to be able to deploy the postgres_exporter along ith postgres databaes instances so that the metrics can be collected by Proemtheus.