Comments (7)
WanzenBug
commented on July 17, 2024
1
All seems to be in order in the kernel logs. Perhaps it really is the old utils version in the linstor-csi image.
You could try building an updated version of the image from here: https://github.com/piraeusdatastore/linstor-csi
Replace debian:bullseye-slim with debian:bookworm-slim in the Dockerfile and run make upload REGISTRY=<private-registry>
from piraeus-operator.
WanzenBug
commented on July 17, 2024
1
We've updated the necessary base images, so this will be fixed by next release.
from piraeus-operator.
WanzenBug
commented on July 17, 2024
Never seen that error before. The only thing I can think of is that the version of resize2fs we have in the linstor-csi container is somehow too old. That could be cause we use a Debian 11 base image. On the other hand, the LINSTOR Satellite (that first creates the FS) is also running in a Debian 11 container, so there should be no difference there.
from piraeus-operator.
jonathon2nd
commented on July 17, 2024
Any additional information or logs that would be helpful? I have really been stumped by this one.
from piraeus-operator.
WanzenBug
commented on July 17, 2024
Perhaps the kernel logs have some additional output?
from piraeus-operator.
jonathon2nd
commented on July 17, 2024
New attempt
0s Warning FailedMount Pod liqo-test-5cf6bb89df-mnbn5 kubelet, k8s-test-worker01.example.com MountVolume.SetUp failed for volume "pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05" : rpc error: code = Internal desc = NodePublishVolume failed for pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: unable to resize volume: resize of device /dev/drbd1024 failed: exit status 1. resize2fs output: resize2fs 1.46.2 (28-Feb-2021) resize2fs: Filesystem has unsupported feature(s) (/dev/drbd1024) 2m12s 9 liqo-test-5cf6bb89df-mnbn5.17bcf90782fdb9fd default
0s Warning FailedMount Pod liqo-test-5cf6bb89df-mnbn5 kubelet, k8s-test-worker01.example.com Unable to attach or mount volumes: unmounted volumes=[vol-lceme], unattached volumes=[], failed to process volumes=[]: timed out waiting for the condition 0s 1 liqo-test-5cf6bb89df-mnbn5.17bcf921b446f9e5 default
0s Normal SuccessfulAttachVolume Pod liqo-test-5cf6bb89df-mnbn5 attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05" 0s 1 liqo-test-5cf6bb89df-mnbn5.17bcf9059caa55d0 default
0s Normal ProvisioningSucceeded PersistentVolumeClaim liqo-test linstor.csi.linbit.com_linstor-csi-controller-5fd545c648-59d8h_13054df0-d091-4983-9f49-a973322de465 Successfully provisioned volume pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 0s 1 liqo-test.17bcf904e18bca83 default
0s Normal SuccessfulCreate ReplicaSet liqo-test-5cf6bb89df replicaset-controller Created pod: liqo-test-5cf6bb89df-mnbn5 0s 1 liqo-test-5cf6bb89df.17bcf903a91e319e default
0s Normal ScalingReplicaSet Deployment liqo-test deployment-controller Scaled up replica set liqo-test-5cf6bb89df to 1 from 0 0s 1 liqo-test.17bcf903a707d687 default
0s Normal ExternalProvisioning PersistentVolumeClaim liqo-test persistentvolume-controller waiting for a volume to be created, either by external provisioner "linstor.csi.linbit.com" or manually created by system administrator 0s 1 liqo-test.17bcf903aa28e47e default
0s Normal Provisioning PersistentVolumeClaim liqo-test linstor.csi.linbit.com_linstor-csi-controller-5fd545c648-59d8h_13054df0-d091-4983-9f49-a973322de465 External provisioner is provisioning volume for claim "default/liqo-test" 0s 1 liqo-test.17bcf903aa321818 default
1s Normal WaitForFirstConsumer PersistentVolumeClaim liqo-test persistentvolume-controller waiting for first consumer to be created before binding 15s 2 liqo-test.17bcf8ffff2aef51 default
0s Normal Scheduled Pod liqo-test-5cf6bb89df-mnbn5 default-scheduler, default-scheduler-k8s-test-master03.example.com Successfully assigned default/liqo-test-5cf6bb89df-mnbn5 to k8s-test-worker01.example.com 0s 1 liqo-test-5cf6bb89df-mnbn5.17bcf9050faae31b default
Mar 15 08:15:58 k8s-test-worker01.example.com systemd[1]: Created slice libcontainer container kubepods-besteffort-poda812177b_1f86_441c_8405_36b0d6dc29c3.slice.
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Starting worker thread (from drbdsetup [886374])
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen10: Starting sender thread (from drbdsetup [886379])
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen12: Starting sender thread (from drbdsetup [886381])
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen13: Starting sender thread (from drbdsetup [886385])
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: susp-io( no -> quorum ) [connect]
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen10: conn( StandAlone -> Unconnected ) [connect]
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen10: Starting receiver thread (from drbd_w_pvc-5ae8 [886375])
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen10: conn( Unconnected -> Connecting ) [connecting]
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen12: conn( StandAlone -> Unconnected ) [connect]
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen12: Starting receiver thread (from drbd_w_pvc-5ae8 [886375])
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen12: conn( Unconnected -> Connecting ) [connecting]
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen13: conn( StandAlone -> Unconnected ) [connect]
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen13: Starting receiver thread (from drbd_w_pvc-5ae8 [886375])
Mar 15 08:15:58 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen13: conn( Unconnected -> Connecting ) [connecting]
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen13: Handshake to peer 0 successful: Agreed network protocol version 122
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen13: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen13: Peer authenticated using 20 bytes HMAC
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen13: Preparing remote state change 620929781
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen10: Handshake to peer 1 successful: Agreed network protocol version 122
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen10: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen10: Peer authenticated using 20 bytes HMAC
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen12: Handshake to peer 2 successful: Agreed network protocol version 122
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen12: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen12: Peer authenticated using 20 bytes HMAC
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd1024: detected capacity change from 0 to 104859696
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024: size = 50 GB (52429848 KB)
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen13: my exposed UUID: 0000000000000000
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen13: peer 94A5272DF179A3B8:FFFFFFFFFFFFFFFF:0000000000000000:0000000000000000 bits:0 flags:1020
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024: Setting exposed data uuid: 94A5272DF179A3B8
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen10: Rejecting concurrent remote state change 4294503611 because of state change 620929781
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen13: Committing remote state change 620929781 (primary_nodes=0)
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: susp-io( quorum -> no ) [remote]
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen13: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [remote]
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024: quorum( no -> yes ) [remote]
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen13: pdsk( DUnknown -> UpToDate ) repl( Off -> Established ) [remote]
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen12: Preparing remote state change 733498211
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen10: my exposed UUID: 94A5272DF179A3B8
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen10: peer 94A5272DF179A3B8:FFFFFFFFFFFFFFFF:0000000000000000:0000000000000000 bits:0 flags:1020
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen12: my exposed UUID: 94A5272DF179A3B8
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen12: peer 94A5272DF179A3B8:FFFFFFFFFFFFFFFF:0000000000000000:0000000000000000 bits:0 flags:1020
Mar 15 08:15:59 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen12: Aborting remote state change 733498211
Mar 15 08:16:00 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen12: Preparing remote state change 2657871747
Mar 15 08:16:00 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen12: my exposed UUID: 94A5272DF179A3B8
Mar 15 08:16:00 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen12: peer 94A5272DF179A3B8:FD1611AE36F8967C:0000000000000000:0000000000000000 bits:0 flags:1020
Mar 15 08:16:00 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen12: Committing remote state change 2657871747 (primary_nodes=0)
Mar 15 08:16:00 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen12: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [remote]
Mar 15 08:16:00 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen12: pdsk( DUnknown -> UpToDate ) repl( Off -> Established ) [remote]
Mar 15 08:16:00 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen10: Preparing remote state change 3035596731
Mar 15 08:16:00 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen10: my exposed UUID: 94A5272DF179A3B8
Mar 15 08:16:00 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen10: peer 94A5272DF179A3B8:FD1611AE36F8967C:0000000000000000:0000000000000000 bits:0 flags:1020
Mar 15 08:16:00 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen10: Committing remote state change 3035596731 (primary_nodes=0)
Mar 15 08:16:00 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05 ovbh-pprod-xen10: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [remote]
Mar 15 08:16:00 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024 ovbh-pprod-xen10: pdsk( DUnknown -> UpToDate ) repl( Off -> Established ) [remote]
Mar 15 08:16:08 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 2429726810 (3->-1 3/1)
Mar 15 08:16:08 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 2429726810: primary_nodes=8, weak_nodes=FFFFFFFFFFFFFFF0
Mar 15 08:16:08 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 2429726810 (1ms)
Mar 15 08:16:08 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Secondary -> Primary ) [mount:886448 auto-promote]
Mar 15 08:16:08 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024: sending new current UUID: BE8023A227E3DE11
Mar 15 08:16:09 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): mounted filesystem with ordered data mode. Quota mode: none.
Mar 15 08:16:09 k8s-test-worker01.example.com systemd[1]: var-lib-kubelet-pods-a812177b\x2d1f86\x2d441c\x2d8405\x2d36b0d6dc29c3-volumes-kubernetes.io\x7ecsi-pvc\x2d5ae83a90\x2d9684\x2d4985\x2d85c9\x2d2f10c3a08a05-mount.mount: Deactivated successfully.
Mar 15 08:16:09 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): unmounting filesystem.
Mar 15 08:16:09 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 3438004721 (3->-1 3/2)
Mar 15 08:16:09 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 3438004721: primary_nodes=0, weak_nodes=0
Mar 15 08:16:09 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 3438004721 (1ms)
Mar 15 08:16:09 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Primary -> Secondary ) [umount:886512 auto-demote]
Mar 15 08:16:09 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 564971754 (3->-1 3/1)
Mar 15 08:16:09 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 564971754: primary_nodes=8, weak_nodes=FFFFFFFFFFFFFFF0
Mar 15 08:16:09 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 564971754 (1ms)
Mar 15 08:16:09 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Secondary -> Primary ) [mount:886514 auto-promote]
Mar 15 08:16:09 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024: sending new current UUID: 17C0F67131D57FDD
Mar 15 08:16:10 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): mounted filesystem with ordered data mode. Quota mode: none.
Mar 15 08:16:10 k8s-test-worker01.example.com systemd[1]: var-lib-kubelet-pods-a812177b\x2d1f86\x2d441c\x2d8405\x2d36b0d6dc29c3-volumes-kubernetes.io\x7ecsi-pvc\x2d5ae83a90\x2d9684\x2d4985\x2d85c9\x2d2f10c3a08a05-mount.mount: Deactivated successfully.
Mar 15 08:16:10 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): unmounting filesystem.
Mar 15 08:16:10 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 3625810449 (3->-1 3/2)
Mar 15 08:16:10 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 3625810449: primary_nodes=0, weak_nodes=0
Mar 15 08:16:10 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 3625810449 (1ms)
Mar 15 08:16:10 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Primary -> Secondary ) [umount:886526 auto-demote]
Mar 15 08:16:11 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 1416541914 (3->-1 3/1)
Mar 15 08:16:11 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 1416541914: primary_nodes=8, weak_nodes=FFFFFFFFFFFFFFF0
Mar 15 08:16:11 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 1416541914 (0ms)
Mar 15 08:16:11 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Secondary -> Primary ) [mount:886528 auto-promote]
Mar 15 08:16:11 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024: sending new current UUID: AACB55844BDC5DBF
Mar 15 08:16:11 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): mounted filesystem with ordered data mode. Quota mode: none.
Mar 15 08:16:11 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): unmounting filesystem.
Mar 15 08:16:11 k8s-test-worker01.example.com systemd[1]: var-lib-kubelet-pods-a812177b\x2d1f86\x2d441c\x2d8405\x2d36b0d6dc29c3-volumes-kubernetes.io\x7ecsi-pvc\x2d5ae83a90\x2d9684\x2d4985\x2d85c9\x2d2f10c3a08a05-mount.mount: Deactivated successfully.
Mar 15 08:16:11 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 2039387320 (3->-1 3/2)
Mar 15 08:16:11 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 2039387320: primary_nodes=0, weak_nodes=0
Mar 15 08:16:11 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 2039387320 (1ms)
Mar 15 08:16:11 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Primary -> Secondary ) [umount:886537 auto-demote]
Mar 15 08:16:14 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 2755293188 (3->-1 3/1)
Mar 15 08:16:14 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 2755293188: primary_nodes=8, weak_nodes=FFFFFFFFFFFFFFF0
Mar 15 08:16:14 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 2755293188 (1ms)
Mar 15 08:16:14 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Secondary -> Primary ) [mount:886540 auto-promote]
Mar 15 08:16:14 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024: sending new current UUID: DED937CD6065D835
Mar 15 08:16:14 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): mounted filesystem with ordered data mode. Quota mode: none.
Mar 15 08:16:14 k8s-test-worker01.example.com systemd[1]: var-lib-kubelet-pods-a812177b\x2d1f86\x2d441c\x2d8405\x2d36b0d6dc29c3-volumes-kubernetes.io\x7ecsi-pvc\x2d5ae83a90\x2d9684\x2d4985\x2d85c9\x2d2f10c3a08a05-mount.mount: Deactivated successfully.
Mar 15 08:16:14 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): unmounting filesystem.
Mar 15 08:16:14 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 689545723 (3->-1 3/2)
Mar 15 08:16:14 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 689545723: primary_nodes=0, weak_nodes=0
Mar 15 08:16:14 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 689545723 (1ms)
Mar 15 08:16:14 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Primary -> Secondary ) [umount:886550 auto-demote]
Mar 15 08:16:18 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 1339146762 (3->-1 3/1)
Mar 15 08:16:18 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 1339146762: primary_nodes=8, weak_nodes=FFFFFFFFFFFFFFF0
Mar 15 08:16:18 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 1339146762 (1ms)
Mar 15 08:16:18 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Secondary -> Primary ) [mount:886556 auto-promote]
Mar 15 08:16:18 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024: sending new current UUID: F393E75D6734E0C1
Mar 15 08:16:19 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): mounted filesystem with ordered data mode. Quota mode: none.
Mar 15 08:16:19 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): unmounting filesystem.
Mar 15 08:16:19 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 1578303405 (3->-1 3/2)
Mar 15 08:16:19 k8s-test-worker01.example.com systemd[1]: var-lib-kubelet-pods-a812177b\x2d1f86\x2d441c\x2d8405\x2d36b0d6dc29c3-volumes-kubernetes.io\x7ecsi-pvc\x2d5ae83a90\x2d9684\x2d4985\x2d85c9\x2d2f10c3a08a05-mount.mount: Deactivated successfully.
Mar 15 08:16:19 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 1578303405: primary_nodes=0, weak_nodes=0
Mar 15 08:16:19 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 1578303405 (1ms)
Mar 15 08:16:19 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Primary -> Secondary ) [umount:886620 auto-demote]
Mar 15 08:16:27 k8s-test-worker01.example.com sshd[886625]: Accepted publickey for fpcuser from 10.1.3.3 port 60386 ssh2: RSA SHA256:ZTx55Zs7eg8QJJNdR76q00YFhivwNUk4tjKGyCcdN64
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[1]: Created slice User Slice of UID 1000.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[1]: Starting User Runtime Directory /run/user/1000...
Mar 15 08:16:27 k8s-test-worker01.example.com systemd-logind[835]: New session 3 of user fpcuser.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[1]: Finished User Runtime Directory /run/user/1000.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[1]: Starting User Manager for UID 1000...
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: pam_unix(systemd-user:session): session opened for user fpcuser(uid=1000) by (uid=0)
Mar 15 08:16:27 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 3528629516 (3->-1 3/1)
Mar 15 08:16:27 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 3528629516: primary_nodes=8, weak_nodes=FFFFFFFFFFFFFFF0
Mar 15 08:16:27 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 3528629516 (1ms)
Mar 15 08:16:27 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Secondary -> Primary ) [mount:886637 auto-promote]
Mar 15 08:16:27 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024: sending new current UUID: 9202D9B67D97AA83
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Queued start job for default target Main User Target.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Created slice User Application Slice.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Started Mark boot as successful after the user session has run 2 minutes.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Started Daily Cleanup of User's Temporary Directories.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Reached target Paths.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Reached target Timers.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Starting D-Bus User Message Bus Socket...
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Starting Create User's Volatile Files and Directories...
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Finished Create User's Volatile Files and Directories.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Listening on D-Bus User Message Bus Socket.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Reached target Sockets.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Reached target Basic System.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Reached target Main User Target.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[886633]: Startup finished in 185ms.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[1]: Started User Manager for UID 1000.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[1]: Started Session 3 of User fpcuser.
Mar 15 08:16:27 k8s-test-worker01.example.com sshd[886625]: pam_unix(sshd:session): session opened for user fpcuser(uid=1000) by (uid=0)
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[1]: Starting Hostname Service...
Mar 15 08:16:27 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): mounted filesystem with ordered data mode. Quota mode: none.
Mar 15 08:16:27 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): unmounting filesystem.
Mar 15 08:16:27 k8s-test-worker01.example.com systemd[1]: var-lib-kubelet-pods-a812177b\x2d1f86\x2d441c\x2d8405\x2d36b0d6dc29c3-volumes-kubernetes.io\x7ecsi-pvc\x2d5ae83a90\x2d9684\x2d4985\x2d85c9\x2d2f10c3a08a05-mount.mount: Deactivated successfully.
Mar 15 08:16:28 k8s-test-worker01.example.com systemd[1]: Started Hostname Service.
Mar 15 08:16:28 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 3670254188 (3->-1 3/2)
Mar 15 08:16:28 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 3670254188: primary_nodes=0, weak_nodes=0
Mar 15 08:16:28 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 3670254188 (1ms)
Mar 15 08:16:28 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Primary -> Secondary ) [umount:886661 auto-demote]
Mar 15 08:16:33 k8s-test-worker01.example.com sudo[886681]: fpcuser : TTY=pts/0 ; PWD=/home/fpcuser ; USER=root ; COMMAND=/bin/su
Mar 15 08:16:33 k8s-test-worker01.example.com sudo[886681]: pam_unix(sudo:session): session opened for user root(uid=0) by fpcuser(uid=1000)
Mar 15 08:16:33 k8s-test-worker01.example.com su[886689]: (to root) root on pts/0
Mar 15 08:16:33 k8s-test-worker01.example.com su[886689]: pam_unix(su:session): session opened for user root(uid=0) by fpcuser(uid=0)
Mar 15 08:16:44 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 3811126735 (3->-1 3/1)
Mar 15 08:16:44 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 3811126735: primary_nodes=8, weak_nodes=FFFFFFFFFFFFFFF0
Mar 15 08:16:44 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 3811126735 (1ms)
Mar 15 08:16:44 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Secondary -> Primary ) [mount:886810 auto-promote]
Mar 15 08:16:44 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05/0 drbd1024: sending new current UUID: ED591F77C577CD59
Mar 15 08:16:44 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): mounted filesystem with ordered data mode. Quota mode: none.
Mar 15 08:16:44 k8s-test-worker01.example.com systemd[1]: var-lib-kubelet-pods-a812177b\x2d1f86\x2d441c\x2d8405\x2d36b0d6dc29c3-volumes-kubernetes.io\x7ecsi-pvc\x2d5ae83a90\x2d9684\x2d4985\x2d85c9\x2d2f10c3a08a05-mount.mount: Deactivated successfully.
Mar 15 08:16:44 k8s-test-worker01.example.com kernel: EXT4-fs (drbd1024): unmounting filesystem.
Mar 15 08:16:44 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Preparing cluster-wide state change 79709970 (3->-1 3/2)
Mar 15 08:16:44 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: State change 79709970: primary_nodes=0, weak_nodes=0
Mar 15 08:16:44 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: Committing cluster-wide state change 79709970 (1ms)
Mar 15 08:16:44 k8s-test-worker01.example.com kernel: drbd pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05: role( Primary -> Secondary ) [umount:886820 auto-demote]
Mar 15 08:16:58 k8s-test-worker01.example.com systemd[1]: systemd-hostnamed.service: Deactivated successfully.
from piraeus-operator.
jonathon2nd
commented on July 17, 2024
Yes, that change fix it and it is now working!
Is this an update that can be done?
0s Normal Created Pod liqo-test-5cf6bb89df-9jqf7 spec.containers{container-0} kubelet, k8s-test-worker01.example.com Created container container-0 0s 1 liqo-test-5cf6bb89df-9jqf7.17bdf626a4a285f8
0s Normal Pulled Pod liqo-test-5cf6bb89df-9jqf7 spec.containers{container-0} kubelet, k8s-test-worker01.example.com Successfully pulled image "phusion/baseimage:jammy-1.0.2" in 7.621340313s (7.621387364s including waiting) 0s 1 liqo-test-5cf6bb89df-9jqf7.17bdf626a32cbf1f
0s Normal Started Pod liqo-test-5cf6bb89df-9jqf7 spec.containers{container-0} kubelet, k8s-test-worker01.example.com Started container container-0 0s 1 liqo-test-5cf6bb89df-9jqf7.17bdf626adc560c9
0s Normal Pulling Pod liqo-test-5cf6bb89df-9jqf7 spec.containers{container-0} kubelet, k8s-test-worker01.example.com Pulling image "phusion/baseimage:jammy-1.0.2" 0s 1 liqo-test-5cf6bb89df-9jqf7.17bdf624dce70781
5s Normal SuccessfulAttachVolume Pod liqo-test-5cf6bb89df-9jqf7 attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-5ae83a90-9684-4985-85c9-2f10c3a08a05" 5s 1 liqo-test-5cf6bb89df-9jqf7.17bdf622b7404eaa
7s Normal SuccessfulCreate ReplicaSet liqo-test-5cf6bb89df replicaset-controller Created pod: liqo-test-5cf6bb89df-9jqf7 7s 1 liqo-test-5cf6bb89df.17bdf6222acccb45
7s Normal ScalingReplicaSet Deployment liqo-test deployment-controller Scaled up replica set liqo-test-5cf6bb89df to 1 from 0 3d5h 2 liqo-test.17bcf903a707d687
7s Normal Scheduled Pod liqo-test-5cf6bb89df-9jqf7 default-scheduler, default-scheduler-k8s-test-master03.example.com Successfully assigned default/liqo-test-5cf6bb89df-9jqf7 to k8s-test-worker01.example.com 7s 1 liqo-test-5cf6bb89df-9jqf7.17bdf6222ae7509c
from piraeus-operator.
Related Issues (20)
- Linstor looking for next version of DRBD on evacuate HOT 10
- Fixing errors in the filesystem HOT 2
- Linstor-Satellite pod zfs create results in invalid arg. HOT 3
- Deployment on k8s failed due to drbd-module-loader container HOT 1
- StorageException: Failed to pvcreate on device: /dev/sdb HOT 13
- Clarify the meaning of the CRDs .status.conditions HOT 1
- StorageException: Failed to mkfs /dev/drbd1002 HOT 9
- etcd-operator adoption HOT 1
- Linstor, installed via Piraeus operator in Kubernetes cluster, disables LVM monitoring HOT 5
- ImplementationError: Layer 'DRBD did not delete the volume 0 of resource ... properly HOT 4
- Resizing of LVM after Host-Reboot not working HOT 9
- A potential risk in piraeus-operator that could lead to takeover of the cluster HOT 1
- ZFS: change mounting location (for Talos Linux) HOT 4
- failed to fail-over resource HOT 1
- make master node only for DISKLESS as TieBreaker HOT 3
- [bug] The priorityClass of the pods is not set. Cascade of failure ensues. HOT 3
- `Satellite not online` for only one node. No errors, just hangs. HOT 2
- Importing/Mounting pre-existing volumes in linstor/DRBD HOT 2
- setup fails for minikube HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from piraeus-operator.