buehler/dotnet-operator-sdk

KubeOps is a kubernetes operator sdk in dotnet. Strongly inspired by kubebuilder.

KubeOps

This is the repository of "KubeOps" - The dotnet Kubernetes Operator SDK.

The documentation is provided in the code itself (description of the methods and classes) and each package contains a README with further information/documentation. For a more detailed documentation, head to the GitHub Pages.

Packages

All packages support .NET6.0 and higher. The reason is that modern C# features are used and client libraries are still possible for .NET 6.0 and up. Also, the KubernetesClient package follows the same strategy regarding the older framework versions. The following packages exist:

Package	Description	Latest Version
KubeOps.Abstractions	Contains abstractions, attributes, etc. for the SDK
KubeOps.Cli	CLI Dotnet Tool to generate stuff
KubeOps.Generator	Source Generator for the SDK
KubeOps.KubernetesClient	Extended client to communicate with the Kubernetes API
KubeOps.Operator	Main SDK entrypoint to create an operator
KubeOps.Operator.Web	Web part of the operator (for webhooks)
KubeOps.Transpiler	Transpilation helpers for CRDs and RBAC elements

Contribution

If you want to contribute, feel free to open a pull request or write issues :-) Read more about contribution (especially for setting up your local environment) in the CONTRIBUTING file.

In short:

• Check out the code
• Develop on KubeOps
• Use some Kubernetes to run the test operator against
• Create tests
• Build the whole solution (lint warnings will result in an error)
• Open PR

Motivation

The motivation was to learn more about the quirks of kubernetes itself and provide an alternative to kubebuilder and operator sdk which are both written in GoLang.

dotnet-operator-sdk's Issues

Provide propper documentation

Remove MVC Testing from KubeOps

Microsoft.AspNetCore.Mvc.Testing package is still referenced in KubeOps package.
this is not needed anymore, since the KubeOps.Testing package provides the reference.

CrdGenerator: Array types

Config generated during build is put in current working directory

I have a solution in a root folder, with an operator project in src/operator-project/.

If I run dotnet build on a solution in the root folder, then the config folder is generated in config/. If I run dotnet build in src/operator-project/, then the config folder is generated in src/operator-project/config.

Given that generated config files are generated at build-time, and belong to the src/operator-project/, I would not expect them in the root folder (or actually, the current working directory), no matter how you compile it.

feat: allow complex status objects

I'm not sure if this is working as intended, but the following code thus results in an infinite loop:

protected override async Task<TimeSpan?> Updated(MyEntity resource)
{
    resource.Status.Progressing = true;
    Client.UpdateStatus(resource);

    //Even if I do nothing here, just switch the states around, it will still re-trigger the Updated event asynchronously.
    await base.Updated(resource);

    resource.Status.Progressing = false;
    Client.UpdateStatus(resource);

    //The second UpdateStatus can yield in a 409 conflict response. Pretend that's fixed by Client.Get()'ing the up to date resource and applying the state change to that again, before re-calling Client.UpdateStatus.
}

I understand that there's also a StatusModified event callback, of which the docs state:
// "StatusModified" (i.e. when only the status was updated)

Is Updated meant to be called when we only update the status object of the entity/resource?

Refactoring of Operator

In collaboration with @ocdi;
Refactoring of the logic how the operator is used.

It should be more like a "normal" addon to the generic host of asp.net core and introduce it's complexity there.
Therefore we introduce some breaking changes and make a major version bump.

Topics to tackle:

• Refactor the operator startup logic to be an extension method of the generic host
• Return a IOperatorBuilder from services.AddKubernetesOperator that can perform more finetuning
• Add proper code linting
• Add proper (more or less) testing tools (maybe a WebApplicationFactory class for the operator? in context of xUnit)
• Finetune the IOperatorBuilder methods (add resource controller, add finalizer, add ...)
• Refactor finalizer appending -> it would be cool to have a "finalizer register" when one can add a finalizer to a resource instead of having to fetch the finalizer from DI and add a myriad of generic information to the method
• Refactor / overlook all the generics. some types cannot be infered and it would be nice to have less generic type arguments (like the add resource controller method)

anything else? :-)

feat: provide controller instantiation logic like asp.net

It would be preferable to have some magic like in "normal" asp.net api controllers.
so when an entity event (with the watcher) arrives, a scope is created and the controller is called with the entity.
this way, the "resource controller base" can be removed and the implementation is simpler.

thoughts:

• rxjs instead of events?
• lock yes/no?

CRD schema type for x-kubernetes-embedded-resource should be object

Properties marked as x-kubernetes-embedded-resource must have a schema type of object.

https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/schema/validation.go#L113

The CRD generator is not including the type property with a value of object resulting in failures when submitting CRDs with embedded resources.

Make Queue size configurable

the event queue size should be configurable

Clear file when regenerate crds and other yamls

Sometimes, there are race conditions when outputting files.
-> clear file and output it then

Remove emtpy "required" fields from V1BetaCRD

Errors thrown from k8s API when no CRD instances are present

[14.05.2020 - 11:20:41] fail: KubeOps.Operator.Watcher.ResourceWatcher[0]
      There was an error while watching the resource "HostedDatabaseOperator.Entities.HostedDatabase".
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an applicatio
n request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.Security.SslStream.<FillBufferAsync>g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask`1 task, Int32 min, Int32 initial)
   at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory`1 buffer)
   at System.Net.Http.HttpConnection.FillAsync()
   at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.ReadAsyncCore(Memory`1 buffer, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.ReadAsyncCore(Memory`1 buffer, CancellationToken cancellationToken)
   at k8s.WatcherDelegatingHandler.CancelableStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at System.IO.StreamReader.ReadBufferAsync(CancellationToken cancellationToken)
   at System.IO.StreamReader.ReadLineAsyncInternal()
   at k8s.WatcherDelegatingHandler.PeekableStreamReader.PeekLineAsync()
   at k8s.WatcherDelegatingHandler.LineSeparatedHttpContent.SerializeToStreamAsync(Stream stream, TransportContext context)
   at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at k8s.Kubernetes.ListClusterCustomObjectWithHttpMessagesAsync(String group, String version, String plural, String continueParameter, String fieldSelector, Stri
ng labelSelector, Nullable`1 limit, String resourceVersion, Nullable`1 timeoutSeconds, Nullable`1 watch, String pretty, Dictionary`2 customHeaders, CancellationTok
en cancellationToken)
   at k8s.WatcherExt.<>c__DisplayClass0_0`2.<<Watch>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at k8s.Watcher`1.WatcherLoop(CancellationToken cancellationToken)

CrdGenerator: fetch descriptions from documentation - xml file if able

When the csproj file defines that documentation files are generated, the crd generator should try to fetch them to gather descriptions for the crd fields

CrdGenerator: add the validation attributes

Until now, only the Display attribute is used.

It would be nice to have the whole set of DataAnnotations attributes availabe (for the validator schema)

Support leader election

Add a mechanism to elect a leader when multiple instances of the operator are running.

Building operator fails because of an invalid ~/.kube/config

I'm using version 3.0.0 of the dotnet-operator-sdk. I have a small project that I cannot compile due to an error in my ~/.kube/config.

When running dotnet build, it fails because the generated crds command fails. When I manually run that command, I get the stack trace below.

I understand it fails, because it is unable to parse my ~/.kube/config file because it doesn't understand the provideClusterInfoProperty property in one of my contexts (I don't know why it is there). Removing this property makes it work. What I don't understand: why is the code generation step loading my ~/.kube/config? That shouldn't be necessary to generate CRD's, right?

>>> dotnet bin/Debug/netcoreapp5.0/Operator.dll generator crds --out /src/Operator/config/crds --format Yaml

Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  Operator -> /src/Operator/bin/Debug/netcoreapp5.0/Operator.dll
  Generating Dockerfile
  Dockerfile already exists. Don't overwrite.
  Generating CRDs
  Unhandled exception. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
   ---> YamlDotNet.Core.YamlException: (Line: 19, Col: 7, Idx: 1985) - (Line: 19, Col: 7, Idx: 1985): Exception during deserialization
   ---> System.Runtime.Serialization.SerializationException: Property 'provideClusterInfoProperty' not found on type 'k8s.KubeConfigModels.ExternalExecution'.
     at YamlDotNet.Serialization.TypeInspectors.TypeInspectorSkeleton.GetProperty(Type type, Object container, String name, Boolean ignoreUnmatched)
     at k8s.Yaml.AutoRestTypeInspector.GetProperty(Type type, Object container, String name, Boolean ignoreUnmatched)
     at YamlDotNet.Serialization.NodeDeserializers.ObjectNodeDeserializer.YamlDotNet.Serialization.INodeDeserializer.Deserialize(IParser parser, Type expectedType, Func`3 nestedObjectDeserializer, Object& value)
     at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     --- End of inner exception stack trace ---
     at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     at YamlDotNet.Serialization.ValueDeserializers.AliasValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.<>c__DisplayClass3_0.<DeserializeValue>b__0(IParser r, Type t)
     at YamlDotNet.Serialization.NodeDeserializers.ObjectNodeDeserializer.YamlDotNet.Serialization.INodeDeserializer.Deserialize(IParser parser, Type expectedType, Func`3 nestedObjectDeserializer, Object& value)
     at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     at YamlDotNet.Serialization.ValueDeserializers.AliasValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.<>c__DisplayClass3_0.<DeserializeValue>b__0(IParser r, Type t)
     at YamlDotNet.Serialization.NodeDeserializers.ObjectNodeDeserializer.YamlDotNet.Serialization.INodeDeserializer.Deserialize(IParser parser, Type expectedType, Func`3 nestedObjectDeserializer, Object& value)
     at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     at YamlDotNet.Serialization.ValueDeserializers.AliasValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.<>c__DisplayClass3_0.<DeserializeValue>b__0(IParser r, Type t)
     at YamlDotNet.Serialization.NodeDeserializers.CollectionNodeDeserializer.DeserializeHelper(Type tItem, IParser parser, Func`3 nestedObjectDeserializer, IList result, Boolean canUpdate)
     at YamlDotNet.Serialization.NodeDeserializers.CollectionNodeDeserializer.YamlDotNet.Serialization.INodeDeserializer.Deserialize(IParser parser, Type expectedType, Func`3 nestedObjectDeserializer, Object& value)
     at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     at YamlDotNet.Serialization.ValueDeserializers.AliasValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.<>c__DisplayClass3_0.<DeserializeValue>b__0(IParser r, Type t)
     at YamlDotNet.Serialization.NodeDeserializers.EnumerableNodeDeserializer.YamlDotNet.Serialization.INodeDeserializer.Deserialize(IParser parser, Type expectedType, Func`3 nestedObjectDeserializer, Object& value)
     at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     at YamlDotNet.Serialization.ValueDeserializers.AliasValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.<>c__DisplayClass3_0.<DeserializeValue>b__0(IParser r, Type t)
     at YamlDotNet.Serialization.NodeDeserializers.ObjectNodeDeserializer.YamlDotNet.Serialization.INodeDeserializer.Deserialize(IParser parser, Type expectedType, Func`3 nestedObjectDeserializer, Object& value)
     at YamlDotNet.Serialization.ValueDeserializers.NodeValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     at YamlDotNet.Serialization.ValueDeserializers.AliasValueDeserializer.DeserializeValue(IParser parser, Type expectedType, SerializerState state, IValueDeserializer nestedObjectDeserializer)
     at YamlDotNet.Serialization.Deserializer.Deserialize(IParser parser, Type type)
     at YamlDotNet.Serialization.Deserializer.Deserialize[T](IParser parser)
     at YamlDotNet.Serialization.Deserializer.Deserialize[T](TextReader input)
     at YamlDotNet.Serialization.Deserializer.Deserialize[T](String input)
     at k8s.Yaml.LoadFromString[T](String content)
     at k8s.Yaml.LoadFromStreamAsync[T](Stream stream)
     at k8s.KubernetesClientConfiguration.LoadKubeConfigAsync(FileInfo kubeconfig, Boolean useRelativePaths)
     at k8s.KubernetesClientConfiguration.BuildConfigFromConfigFileAsync(FileInfo kubeconfig, String currentContext, String masterUrl, Boolean useRelativePaths)
     at k8s.KubernetesClientConfiguration.BuildConfigFromConfigFile(FileInfo kubeconfig, String currentContext, String masterUrl, Boolean useRelativePaths)
     at k8s.KubernetesClientConfiguration.BuildConfigFromConfigFile(String kubeconfigPath, String currentContext, String masterUrl, Boolean useRelativePaths)
     at k8s.KubernetesClientConfiguration.BuildDefaultConfig()
     at DotnetKubernetesClient.KubernetesClient..ctor()
     at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
     at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
     at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
     at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
     at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitDisposeCache(ServiceCallSite transientCallSite, RuntimeResolverContext context)
     at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
     at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
     at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
     at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
     at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
     at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ServiceProvider.GetService(Type serviceType)
     at McMaster.Extensions.CommandLineUtils.CommandLineApplication.System.IServiceProvider.GetService(Type serviceType)
     at McMaster.Extensions.CommandLineUtils.Conventions.ConstructorInjectionConvention.FindMatchedConstructor[TModel](ConstructorInfo[] constructors, IServiceProvider services, Boolean throwIfNoParameterTypeRegistered)
     at McMaster.Extensions.CommandLineUtils.Conventions.ConstructorInjectionConvention.ApplyImpl[TModel](ConventionContext context)
     --- End of inner exception stack trace ---
     at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
     at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
     at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
     at McMaster.Extensions.CommandLineUtils.Conventions.ConstructorInjectionConvention.Apply(ConventionContext context)
     at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Builder.McMaster.Extensions.CommandLineUtils.Conventions.IConventionBuilder.AddConvention(IConvention convention)
     at McMaster.Extensions.CommandLineUtils.ConventionBuilderExtensions.UseConstructorInjection(IConventionBuilder builder, IServiceProvider additionalServices)
     at KubeOps.Operator.HostExtensions.RunOperatorAsync(IHost host, String[] args)
     at Operator.Program.Main(String[] args) in /src/Operator/Program.cs:line 20
     at Operator.Program.<Main>(String[] args)
  Aborted (core dumped)
/home/basilfx/.nuget/packages/kubeops/3.0.0/build/KubeOps.targets(59,9): error MSB3073: The command "dotnet bin/Debug/netcoreapp5.0/Operator.dll generator crds --out /src/Operator/config/crds --format Yaml" exited with code 134. [/src/Operator/Operator.csproj]

Build FAILED.

/home/basilfx/.nuget/packages/kubeops/3.0.0/build/KubeOps.targets(59,9): error MSB3073: The command "dotnet bin/Debug/netcoreapp5.0/Operator.dll generator crds --out /src/Operator/config/crds --format Yaml" exited with code 134. [/src/Operator/Operator.csproj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:02.88

Support multiple versions of a CRD

When multiple versions of a crd are present

[Entity...]
class V1Foobar{}
[Entity...]
class V2Foobar{}

It should be possible to reflect this matter in the CRDs that are generated.

feat: Add validation webhook possibility

Like "kubebuilder", the operator should be able to support webhooks to validate the crds when uploaded to kubernetes

Add Role Binding in namespace to manager role

CRD schema generator does not consider PropertyName value in JsonPropertyAttribute

The CRD generator uses the property name from the PropertyInfo even when a PropertyName value has been set in a JsonPropertyAttribute. Because the schema and serialized JSON payload differ for such properties the Kubernetes API ignores them.

Unhandled exception. System.NotSupportedException and then crash and exit

Starting from the sample operator provider in this repo I have slightly modified the entity and create a few instances of my CRD.
cluster is kind (running k8s v1.19.1)
the operator project is run locally .. when a crd instance is created I get the notification correctly .. however , if I wait a while I get the message
dbug: KubeOps.Operator.Watcher.ResourceWatcher[0]
The server closed the connection. Trying to reconnect.

then when the connection is re established i start getting notification again ?! .. and then boom due to the excpetion below Note that the application crash and exit

Unhandled exception. trceUnhandled exception. : KubeOps.Operator.Watcher.ResourceWatcher[0]
Received watch event "Added" for "productinfoentity/1".
Unhandled exception. System.NotSupportedException: Dynamic Objects Are Not Supported as TryGetMember requires Microsoft.CSharp which requires .NET Framework 4.5 and higher. ExpandoObjects are supported.
See https://github.com/GregFinzer/Compare-Net-Objects/issues/103

Full stack trace below

info: KubeOps.Operator.Leadership.LeaderElector[0]
Startup Leader Elector for operator "kubeops-warehouse".
trce: KubeOps.Operator.Leadership.LeaderElector[0]
Fetching namespace for leader election.
trce: KubeOps.Operator.Leadership.LeaderElector[0]
Fetch V1Lease object for operator "kubeops-warehouse".
dbug: KubeOps.Operator.Leadership.LeaderElector[0]
The instance "DESKTOP-LORENZO" is still the leader for operator "kubeops-warehouse".
dbug: KubeOps.Operator.Leadership.LeaderElection[0]
Leadership state changed to: Leader.
info: KubeOps.Operator.Controller.ResourceControllerBase[0]
Startup CRD Controller for "KubeOps.WareHouse.Entities.ProductInfoEntity".
info: KubeOps.Operator.Controller.ResourceControllerBase[0]
This instance was elected as leader, starting event queue.
dbug: KubeOps.Operator.Queue.ResourceEventQueue[0]
Event queue startup for type "KubeOps.WareHouse.Entities.ProductInfoEntity".
dbug: KubeOps.Operator.Watcher.ResourceWatcher[0]
Resource Watcher startup for type "KubeOps.WareHouse.Entities.ProductInfoEntity".
trce: KubeOps.Operator.Queue.ResourceEventQueue[0]
Start queue reader for type "KubeOps.WareHouse.Entities.ProductInfoEntity".
trce: KubeOps.Operator.Watcher.ResourceWatcher[0]
Received watch event "Added" for "productinfoentity/p1".
trce: KubeOps.Operator.Queue.ResourceEventQueue[0]
Resource "productinfoentity/p1" comparison result "New".
trce: KubeOps.Operator.Queue.ResourceEventQueue[0]
Enqueue event "Created" for resource "productinfoentity/p1".
trce: KubeOps.Operator.Watcher.ResourceWatcher[0]
Received watch event "Added" for "productinfoentity/2".
trce: KubeOps.Operator.Queue.ResourceEventQueue[0]
Read event "Created" for resource "KubeOps.WareHouse.Entities.ProductInfoEntity".
trce: KubeOps.Operator.Queue.ResourceEventQueue[0]
Resource "productinfoentity/2" comparison result "New".
trce: KubeOps.Operator.Queue.ResourceEventQueue[0]
Enqueue event "Created" for resource "productinfoentity/2".
trce: KubeOps.Operator.Watcher.ResourceWatcher[0]
Received watch event "Added" for "productinfoentity/1".
trce: KubeOps.Operator.Queue.ResourceEventQueue[0]
Resource "productinfoentity/1" comparison result "New".
info: Microsoft.Hosting.Lifetime[0]
Now listening on: http://localhost:5000
trce: KubeOps.Operator.Queue.ResourceEventQueue[0]
Enqueue event "Created" for resource "productinfoentity/1".
info: Microsoft.Hosting.Lifetime[0]
Now listening on: https://localhost:5001
info: Microsoft.Hosting.Lifetime[0]
Application started. Press Ctrl+C to shut down.
dbug: KubeOps.Operator.Controller.ResourceControllerBase[0]
Execute event "Created" on resource "productinfoentity/p1".
info: Microsoft.Hosting.Lifetime[0]
Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
Content root path: E:_sabba-dev\k8sWareHouse\KubeOps.WareHouseOperator\bin\Debug\netcoreapp3.1
dbug: KubeOps.WareHouse.TestManager.ProductInfoManager[0]
Created
fail: KubeOps.WareHouse.Controller.ProductController[0]
error in ProductInfoEntityCreated
Microsoft.Rest.HttpOperationException: Operation returned an invalid status code 'NotFound'
at k8s.Kubernetes.CreateNamespacedCustomObjectWithHttpMessagesAsync(Object body, String group, String version, String namespaceParameter, String plural, String pretty, Dictionary2 customHeaders, CancellationToken cancellationToken) at k8s.KubernetesExtensions.CreateNamespacedCustomObjectAsync(IKubernetes operations, Object body, String group, String version, String namespaceParameter, String plural, String pretty, CancellationToken cancellationToken) at k8s.KubernetesExtensions.CreateNamespacedCustomObject(IKubernetes operations, Object body, String group, String version, String namespaceParameter, String plural, String pretty) at KubeOps.WareHouse.TestManager.ProductInfoManager.Created(ProductInfoEntity entity) in E:\_sabba-dev\k8sWareHouse\KubeOps.WareHouseOperator\TestManager\ProductInfoManager.cs:line 42 at KubeOps.WareHouse.Controller.ProductController.Created(ProductInfoEntity resource) in E:\_sabba-dev\k8sWareHouse\KubeOps.WareHouseOperator\Controller\ProductController.cs:line 32 info: KubeOps.Operator.Controller.ResourceControllerBase[0] Event type "Created" on resource "productinfoentity/p1" successfully reconciled. Requeue not requested. trce: KubeOps.Operator.Queue.ResourceEventQueue[0] Read event "Created" for resource "KubeOps.WareHouse.Entities.ProductInfoEntity". dbug: KubeOps.Operator.Controller.ResourceControllerBase[0] Execute event "Created" on resource "productinfoentity/2". dbug: KubeOps.WareHouse.TestManager.ProductInfoManager[0] Created fail: KubeOps.WareHouse.Controller.ProductController[0] error in ProductInfoEntityCreated Microsoft.Rest.HttpOperationException: Operation returned an invalid status code 'NotFound' at k8s.Kubernetes.CreateNamespacedCustomObjectWithHttpMessagesAsync(Object body, String group, String version, String namespaceParameter, String plural, String pretty, Dictionary2 customHeaders, CancellationToken cancellationToken)
at k8s.KubernetesExtensions.CreateNamespacedCustomObjectAsync(IKubernetes operations, Object body, String group, String version, String namespaceParameter, String plural, String pretty, CancellationToken cancellationToken)
at k8s.KubernetesExtensions.CreateNamespacedCustomObject(IKubernetes operations, Object body, String group, String version, String namespaceParameter, String plural, String pretty)
at KubeOps.WareHouse.TestManager.ProductInfoManager.Created(ProductInfoEntity entity) in E:_sabba-dev\k8sWareHouse\KubeOps.WareHouseOperator\TestManager\ProductInfoManager.cs:line 42
at KubeOps.WareHouse.Controller.ProductController.Created(ProductInfoEntity resource) in E:_sabba-dev\k8sWareHouse\KubeOps.WareHouseOperator\Controller\ProductController.cs:line 32
info: KubeOps.Operator.Controller.ResourceControllerBase[0]
Event type "Created" on resource "productinfoentity/2" successfully reconciled. Requeue not requested.
trce: KubeOps.Operator.Queue.ResourceEventQueue[0]
Read event "Created" for resource "KubeOps.WareHouse.Entities.ProductInfoEntity".
dbug: KubeOps.Operator.Controller.ResourceControllerBase[0]
Execute event "Created" on resource "productinfoentity/1".
dbug: KubeOps.WareHouse.TestManager.ProductInfoManager[0]
Created
fail: KubeOps.WareHouse.Controller.ProductController[0]
error in ProductInfoEntityCreated
Microsoft.Rest.HttpOperationException: Operation returned an invalid status code 'NotFound'
at k8s.Kubernetes.CreateNamespacedCustomObjectWithHttpMessagesAsync(Object body, String group, String version, String namespaceParameter, String plural, String pretty, Dictionary2 customHeaders, CancellationToken cancellationToken) at k8s.KubernetesExtensions.CreateNamespacedCustomObjectAsync(IKubernetes operations, Object body, String group, String version, String namespaceParameter, String plural, String pretty, CancellationToken cancellationToken) at k8s.KubernetesExtensions.CreateNamespacedCustomObject(IKubernetes operations, Object body, String group, String version, String namespaceParameter, String plural, String pretty) at KubeOps.WareHouse.TestManager.ProductInfoManager.Created(ProductInfoEntity entity) in E:\_sabba-dev\k8sWareHouse\KubeOps.WareHouseOperator\TestManager\ProductInfoManager.cs:line 42 at KubeOps.WareHouse.Controller.ProductController.Created(ProductInfoEntity resource) in E:\_sabba-dev\k8sWareHouse\KubeOps.WareHouseOperator\Controller\ProductController.cs:line 32 info: KubeOps.Operator.Controller.ResourceControllerBase[0] Event type "Created" on resource "productinfoentity/1" successfully reconciled. Requeue not requested. trce: KubeOps.Operator.Leadership.LeaderElector[0] Fetch V1Lease object for operator "kubeops-warehouse". dbug: KubeOps.Operator.Leadership.LeaderElector[0] The instance "DESKTOP-LORENZO" is still the leader for operator "kubeops-warehouse". trce: KubeOps.Operator.Leadership.LeaderElector[0] Fetch V1Lease object for operator "kubeops-warehouse". dbug: KubeOps.Operator.Leadership.LeaderElector[0] The instance "DESKTOP-LORENZO" is still the leader for operator "kubeops-warehouse". trce: KubeOps.Operator.Leadership.LeaderElector[0] Fetch V1Lease object for operator "kubeops-warehouse". dbug: KubeOps.Operator.Leadership.LeaderElector[0] The instance "DESKTOP-LORENZO" is still the leader for operator "kubeops-warehouse". trce: KubeOps.Operator.Leadership.LeaderElector[0] Fetch V1Lease object for operator "kubeops-warehouse". dbug: KubeOps.Operator.Leadership.LeaderElector[0] The instance "DESKTOP-LORENZO" is still the leader for operator "kubeops-warehouse". dbug: KubeOps.Operator.Watcher.ResourceWatcher[0] The server closed the connection. Trying to reconnect. trce: KubeOps.Operator.Watcher.ResourceWatcher[0] Restarting resource watcher for type "KubeOps.WareHouse.Entities.ProductInfoEntity". trce: KubeOps.Operator.Watcher.ResourceWatcher[0] Received watch event "Added" for "productinfoentity/p1". trce: KubeOps.Operator.Watcher.ResourceWatcher[0] Received watch event "Added" for "productinfoentity/2". Unhandled exception. trceUnhandled exception. : KubeOps.Operator.Watcher.ResourceWatcher[0] Received watch event "Added" for "productinfoentity/1". Unhandled exception. System.NotSupportedException: Dynamic Objects Are Not Supported as TryGetMember requires Microsoft.CSharp which requires .NET Framework 4.5 and higher. ExpandoObjects are supported. See https://github.com/GregFinzer/Compare-Net-Objects/issues/103 at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.HandleDynamicObject(ComparisonConfig config, Object objectValue, Type objectType) at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.GetCurrentProperties(CompareParms parms, Object objectValue, Type objectType) at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.PerformCompareProperties(CompareParms parms, Boolean ignoreBaseList) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareProperties(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms) at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms) at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms) at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms) at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms) at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms) at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms) at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.CompareProperty(CompareParms parms, PropertyEntity info, List1 object2Properties)
at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.PerformCompareProperties(CompareParms parms, Boolean ignoreBaseList)
at KellermanSoftware.CompareNetObjects.TypeComparers.ClassComparer.CompareType(CompareParms parms)
at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms)
at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.CompareProperty(CompareParms parms, PropertyEntity info, List1 object2Properties) at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.PerformCompareProperties(CompareParms parms, Boolean ignoreBaseList) at KellermanSoftware.CompareNetObjects.TypeComparers.ClassComparer.CompareType(CompareParms parms) at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.CompareProperty(CompareParms parms, PropertyEntity info, List1 object2Properties)
at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.PerformCompareProperties(CompareParms parms, Boolean ignoreBaseList)
at KellermanSoftware.CompareNetObjects.TypeComparers.ClassComparer.CompareType(CompareParms parms)
at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms)
at KellermanSoftware.CompareNetObjects.CompareLogic.Compare(Object expectedObject, Object actualObject)
at KubeOps.Operator.Caching.ResourceCache1.CompareCache(TEntity resource) at KubeOps.Operator.Caching.ResourceCache1.Upsert(TEntity resource, CacheComparisonResult& result)
at KubeOps.Operator.Queue.ResourceEventQueue1.Enqueue(TEntity resource, Nullable1 enqueueDelay)
at KubeOps.Operator.Queue.ResourceEventQueue1.OnWatcherEvent(Object _, ValueTuple2 args)
at System.Threading.Tasks.Task.<>c.b__139_1(Object state)
at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()
System.NotSupportedException: Dynamic Objects Are Not Supported as TryGetMember requires Microsoft.CSharp which requires .NET Framework 4.5 and higher. ExpandoObjects are supported.
See https://github.com/GregFinzer/Compare-Net-Objects/issues/103
at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.HandleDynamicObject(ComparisonConfig config, Object objectValue, Type objectType)
at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.GetCurrentProperties(CompareParms parms, Object objectValue, Type objectType)
at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.PerformCompareProperties(CompareParms parms, Boolean ignoreBaseList)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareProperties(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms)
at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms)
at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms)
at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms)
at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms)
at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms)
at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms)
at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.CompareProperty(CompareParms parms, PropertyEntity info, List1 object2Properties) at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.PerformCompareProperties(CompareParms parms, Boolean ignoreBaseList) at KellermanSoftware.CompareNetObjects.TypeComparers.ClassComparer.CompareType(CompareParms parms) at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareItems(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.ListComparer.CompareType(CompareParms parms) at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms) at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.CompareProperty(CompareParms parms, PropertyEntity info, List1 object2Properties)
at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.PerformCompareProperties(CompareParms parms, Boolean ignoreBaseList)
at KellermanSoftware.CompareNetObjects.TypeComparers.ClassComparer.CompareType(CompareParms parms)
at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms)
at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.CompareProperty(CompareParms parms, PropertyEntity info, List1 object2Properties) at KellermanSoftware.CompareNetObjects.TypeComparers.PropertyComparer.PerformCompareProperties(CompareParms parms, Boolean ignoreBaseList) at KellermanSoftware.CompareNetObjects.TypeComparers.ClassComparer.CompareType(CompareParms parms) at KellermanSoftware.CompareNetObjects.RootComparer.Compare(CompareParms parms) at KellermanSoftware.CompareNetObjects.CompareLogic.Compare(Object expectedObject, Object actualObject) at KubeOps.Operator.Caching.ResourceCache1.CompareCache(TEntity resource)
at KubeOps.Operator.Caching.ResourceCache1.Upsert(TEntity resource, CacheComparisonResult& result) at KubeOps.Operator.Queue.ResourceEventQueue1.Enqueue(TEntity resource, Nullable1 enqueueDelay) at KubeOps.Operator.Queue.ResourceEventQueue1.OnWatcherEvent(Object _, ValueTuple`2 args)
at System.Threading.Tasks.Task.<>c.b__139_1(Object state)
at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()
E:_sabba-dev\k8sWareHouse\KubeOps.WareHouseOperator\bin\Debug\netcoreapp3.1\KubeOps.WareHouse.exe (process 17960) exited with code 0.
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
Press any key to close this window . . .

feat: throw an error if a crd has no proper type

Regarding #75
If a parsed type in the CRD has no parsed type (like object or int), the generator should throw an error so a developer is notified that the CRD is not properly structured.

Add prometheus metrics

Like "kubebuilder" some default prometheus metrics should be collected and exposed if one wishes to observe the operator.

Use correct api group for Deployment

feat: allow forcing of event method for delayed resource

If I re-enqueue an object in the Updated event (returning a non null TimeSpan), "My" expectation is to receive the re-enqueued object in the original callback (Updated method) when the timespan expires.
Currently the object is re-pushed in the NotModified method

feat: Kubernetes Version Gate

This is a rather hard problem. Newer versions of kubernetes client as well as the kubernetes server serve different versions of the entities (models).

As an example: Corev1Event vs Eventv1Event.

It would be nice to have some version gate that manages the compatibility.

maybe this should better be implemented in https://github.com/buehler/dotnet-kubernetes-client

Allow kube-proxy flag for commands

The kubernetes client config for .net does not yet support the gcloud authenticator.
Therefore, the support for "kube-proxy" would be nice to use the install / uninstall / run commands

Let the docs specify that the output type should be 'Exe'.

While it may seem obvious, the docs (at least as far as I can tell) do not currently mention that the output type of the generated .NET Core application should be set to Console Application (as opposed to Class Library). This will cause error messages on build such as:

The command "dotnet path/to/dll generator crds --out path/to/config/crds --format Yaml" exited with code -2147450749
or
The command "dotnet path/to/dll generator docker --out path/to/Dockerfile --dotnet-tag latest --solution-dir path/to/sln --target-file DllName.dll --project-path path/to/csproj" exited with code -2147450749.

Just leaving this here, as it took me about twenty minutes figuring out why stuff wasn't building, even though I had followed the Getting Started section of the docs.

The fix is, ofcourse, to simply add <OutputType>Exe</OutputType> to your .csproj..

Other than that, thanks a lot for the SDK!

Proper testing

add tests to the kube ops solution :)

Namespaced controllers / operator

Add the ability to run a controller or the whole operator in "namespaced" mode. This mode should then only watch resources of certain namespaces

Changing the settings via the msbuild extensions

How do you exactly change the default location for KubeOpsConfigRoot? I added the following to the csproj

	<PropertyGroup>
		<KubeOpsConfigRoot>$(SolutionDir)KubeOps</KubeOpsConfigRoot>
	</PropertyGroup>

But unfortunately I still see it create the files at config folder.

Support AdditionalPrinterColumns for crds

the AdditionalPrinterColumns are shown when using kubectl get ...

feat: Add migration webhooks

https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#configure-customresourcedefinition-to-use-conversion-webhooks

Support (nullable) unsigned primitive numbers (or throw during CRD generation).

Currently it seems that the CRD generation process does not support (nullable) unsiged primitive numbers, in the sense that it omits the type property entirely, resulting in error messages such as:

CustomResourceDefinition.apiextensions.k8s.io "mycrd.com" is invalid: [spec.validation.openAPIV3Schema.properties[spec].properties[deleteRetentionMs].type: Required value: must not be empty for specified object fields

I.e. the OAS3 spec is generated as follows:

        openAPIV3Schema:
          spec:
            properties:
              deleteRetentionMs:
                nullable: true

For an entity such as:

    public class MyEntity
    {
        public ulong? DeleteRetentionMs { get; set; }
    }

Provide KubeOps.Testing package

provide a testing package that enables developers to use kube ops mocks to test their own implemented operator.

chore: add preview-builds

with the magic of semantic release; add a "next" branch that should create -prerelease.XX packages that are deployed as prereleases.

Those may contain breaking changes as long as no new version is released.

Maybe the master branch could be used as the "next versioN" branch and a custom "release" branch is there for effective releases.

Build hangs on generate CRD task (kubeops 2.2.0, dotnetcore 3.1)

any suggestion about what i should check ?
I actually just dont need these steps in the project I am working on .. is there a way to skip them ?
thank you

Task "CallTarget"
1> Target GenerateCrds:
1> Task "Message"
1> Generating CRDs
1> Done executing task "Message".
1> Task "Message"
1> Configuration path: E:_sabba-dev\WareHouse\warehouseapi\config\crds
1> Done executing task "Message".
1> Using "Exec" task from assembly "Microsoft.Build.Tasks.Core, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a".
1> Task "Exec"
1> dotnet bin\Debug\netcoreapp3.1\warehouseapi.dll generator crds --out E:_sabba-dev\WareHouse\warehouseapi\config\crds --format Yaml

Support leader takeover

Regarding #22: It would be nice to "force" a leader to be set when using local development, so one can develop an operator on the same kubernetes instance that the deployed operator runs on.

Deployment yaml of operator is missing liveness / readiness probe

since those healthchecks are now supported, the generated yaml for the operator deployment should have the liveness and readiness probes configured.

Collection properties are generated as object instead of array in JSON schema

The CRD code generator does not cater for a wide range of enumerable types. In the generated JSON Schema these unsupported collection properties have a type of object instead of array which breaks validation when posting resources. It would be ideal for the following additional types to be supported:

• List<T>, IList<T> and IReadyOnlyList<T>
• Collection<T>, ICollection<T> and IReadyOnlyCollection<T>
• HashSet<T>, ISet<T> and IReadOnlySet<T>
• Types derived from List<T> and Collection<T>

These could all be detected by targeting IEnumerable<T> based interfaces on the type.

feat: Add mutation webhooks

feat: Provide .net new template

I typically don't like to ask these kind of things but are you planning on providing a sample project, or even a .NET template?

The documentation is good on how to get started, but always great if you have a reference project or scaffold option.

Remove "UniqueItems" since it is not allowed

Forbidden: uniqueItems cannot be set to true since the runtime complexity becomes quadratic

Support entity classes located in other dll, not only in main exe

I have a project which will consist at list 3 controllers and 3 CRD. Sometime 1 controller will watch 1 type of CRD and update the other. So i think it will a good point to move entity classes to single shared dll. I tried to use your project and it looks really cool. But generation operations doesn't work for me. Because GenerateCrds looking for a KubernetesEntityAttribute only in main assembly.

What do you think about this ? I can implement and make PR if you advise how to solve the problem.

Add "Ignore" Attribute for entities

nice to have: one could use classes for external entities but ignore them during crd generation (example: ambassador mapping)

Preload the resource cache on watcher start

Currently, the watcher is started up and all elements that are found in the namespaces are fired with a "created" event. This may be problematic for certain operators.

A better way would be to fetch a list of already existing objects in kubernetes with the client and preload the cache with the existing objects. So when the watcher starts, all elements that prematurely existed in kubernetes would fire a "not modified" event.

Provide documentation for .NET template

Provide documentation for .NET template on https://buehler.github.io/dotnet-operator-sdk/index.html.

Relates to #133

Provide possibility to have structured logs on production

structured logs instead of the default .net logs would be better to catch for loki or a similar log gatherer

feat: Add easy mechanism for "events" (kubernetes object events)

The kubernetes sub-resource "Events" should be supported on crds

Production Logging is always enabled