Datasafe - ListAlertPolicies

[oracle]

Datasafe - ListAlertPolicies

[jbcintra]

Effectively the same fault with ListAuditEvents too.

[github-anurag]

@jbcintra
The same code works for me without issues. The code I am using:-

using System;
using System.Linq;
using Oci.DatasafeService;
using Oci.DatasafeService.Requests;
using Oci.Common;
using Oci.Common.Auth;

namespace Oci.Examples
{
    public class DatasafeExample
    {
        private static NLog.Logger logger = NLog.LogManager.GetCurrentClassLogger();

        public static void Main()
        {
            logger.Info("Starting example");
            DataSafeClient client = null;
            try
            {
                // Assumption: the compartment id has been set in environment variable.
                var compartmentId = Environment.GetEnvironmentVariable("OCI_COMPARTMENT_ID");
                logger.Info($"Compartment ID: {compartmentId}");

                var provider = new ConfigFileAuthenticationDetailsProvider("DEFAULT");

                using (client = new DataSafeClient(provider, new ClientConfiguration()))
                {
                    var responseEnum = client.Paginators.ListAlertPoliciesResponseEnumerator(new ListAlertPoliciesRequest
                    {
                        CompartmentId = compartmentId
                    });
                    var policyList = responseEnum.SelectMany(r => r.AlertPolicyCollection.Items).ToList();
                    logger.Info($"eventList size:{policyList.Count}");
                    foreach (var p in policyList)
                    {
                        logger.Info($"{p.Id}");
                    }
                }
            }
            catch (Exception e)
            {
                logger.Error($"Failed Datasafe example: {e.Message}");
            }
        }
    }
}

[github-anurag]

Also when there is an empty response in ListAuditEvents, the code is working fine:-

using System;
using System.Linq;
using Oci.DatasafeService;
using Oci.DatasafeService.Requests;
using Oci.Common;
using Oci.Common.Auth;

namespace Oci.Examples
{
    public class DatasafeExample
    {
        private static NLog.Logger logger = NLog.LogManager.GetCurrentClassLogger();

        public static void Main()
        {
            logger.Info("Starting example");
            DataSafeClient client = null;
            try
            {
                // Assumption: the compartment id has been set in environment variable.
                var compartmentId = Environment.GetEnvironmentVariable("OCI_COMPARTMENT_ID");
                logger.Info($"Compartment ID: {compartmentId}");

                var provider = new ConfigFileAuthenticationDetailsProvider("DEFAULT");

                using (client = new DataSafeClient(provider, new ClientConfiguration()))
                {
                    var responseEnum = client.Paginators.ListAuditEventsResponseEnumerator(new ListAuditEventsRequest
                    {
                        CompartmentId = compartmentId
                    });
                    var eventList = responseEnum.SelectMany(r => r.AuditEventCollection.Items).ToList();
                    logger.Info($"eventList size:{eventList.Count}");
                    foreach (var e in eventList)
                    {
                        logger.Info($"{e.Id}");
                    }
                }
            }
            catch (Exception e)
            {
                logger.Error($"Failed Datasafe example: {e.Message}");
            }
        }
    }
}

The output:-

2022-12-07 12:01:03.8629|INFO|Oci.Examples.DatasafeExample|Starting example
2022-12-07 12:01:04.1347|INFO|Oci.Examples.DatasafeExample|Compartment ID: ocid1.tenancy.oc1........
2022-12-07 12:01:04.1622|INFO|Oci.Common.ConfigFileReader|Loading config file from: ...........
2022-12-07 12:01:04.1889|INFO|Oci.Common.Auth.ConfigFileAuthenticationDetailsProvider|Choosing Configuration authentication details provider
2022-12-07 12:01:04.2498|INFO|Oci.Common.ClientBase|Setting endpoint to ..........
2022-12-07 12:01:04.3064|INFO|Oci.Common.Utils.RetryUtils|Environment Variable OCI_SDK_DEFAULT_RETRY_ENABLED not found or is not a bool, setting Global Default retry enabled as: False
2022-12-07 12:01:06.6433|INFO|Oci.Examples.DatasafeExample|eventList size:0
2022-12-07 12:01:06.6441|INFO|Oci.Common.ClientBase|Disposing rest client.

[jbcintra]

Thanks Anurag, I'll dig into why this is happening at my end... only a few calls are raising the issue?

[github-anurag]

Hi @jbcintra
Hmm, perhaps it is happening for certain calls. However, I can't be sure as I am unable to reproduce the error. The two APIs provided me to test both the cases, namely: when no data is returned by the API and when some data is returned by the API.

Not sure if it helps, but can you try updating the oci-dotnet-sdk to the latest version if you are not on that currently? There may have been some changes in the service that might have gotten introduced between the version you are using v/s the latest one.

[jbcintra]

Thanks, I'm running yesterday's release, so all current.

Now I've just been digging through it again, across a number of customers. In some customers it is working, in some, it is not returning empty/or 4xx, rather it's blowing up with the above message. Perhaps then that is the actual problem?

[github-anurag]

@jbcintra
Thanks for this info! I will relay this information to the Datasafe team and check if they can figure this out.

[github-anurag]

@jbcintra
Can you enable Trace level logs for oci-dotnet-sdk and check if we can get the Service Response in the logs? Also, if we can get the opc-request-id for the failed API calls that would help the Data Safe team to narrow down the issue.

[KartikShrikantHegde]

Hi @jbcintra , our engineer tried to reproduce the issue, but they couldn't.

Here are the codes they used for ListAlertPolicies and ListAuditEvents, and both APIs seem to work fine. version for oci-dotnet-sdk is 54.0.0. Can you please verify again from your end?

ListAlertPolicies:

using System;
using System.Linq;
using Oci.DatasafeService;
using Oci.DatasafeService.Requests;
using Oci.Common;
using Oci.Common.Auth;
namespace Oci.Examples{
    public class DatasafeExample1
    {
        private static NLog.Logger logger = NLog.LogManager.GetCurrentClassLogger();
        public static void Main()
        {
            logger.Info("Starting example1");
            Console.WriteLine("Starting example1");
            DataSafeClient client = null;
            try
            {
                var compartmentId = "<value>";
                logger.Info($"Compartment ID: {compartmentId}");
                Console.WriteLine($"Compartment ID: {compartmentId}");
                var provider = new ConfigFileAuthenticationDetailsProvider("DEFAULT");
                using (client = new DataSafeClient(provider, new ClientConfiguration()))
                {
                    var request = new ListAlertPoliciesRequest() { CompartmentId = compartmentId };
                    var responseEnum = client.Paginators.ListAlertPoliciesResponseEnumerator(request);
                    var policyList = responseEnum.SelectMany(r => r.AlertPolicyCollection.Items).ToList();
                    logger.Info($"eventList size:{policyList.Count}");
                    Console.WriteLine($"policyList size:{policyList.Count}");
                    foreach (var p in policyList)
                    {
                        logger.Info($"{p.Id} - {p.DisplayName}");
                        Console.WriteLine($"{p.Id} - {p.DisplayName}");
                    }
                }
            }
            catch (Exception e)
            {
                logger.Error($"Failed Datasafe example: {e.Message}");
                Console.WriteLine($"Failed Datasafe example: {e.Message}");
            }
        }
    }
} 

ListAuditEvents:

using System;
using System.Linq;
using Oci.DatasafeService;
using Oci.DatasafeService.Requests;
using Oci.Common;
using Oci.Common.Auth;

namespace Oci.Examples{
    public class DatasafeExample2
    {
        private static NLog.Logger logger = NLog.LogManager.GetCurrentClassLogger();
        public static void Main()
        {
            logger.Info("Starting example2");
            Console.WriteLine("Starting example2");
            DataSafeClient client = null;
            try
            {
                var compartmentId =  "<value>";
                logger.Info($"Compartment ID: {compartmentId}");
                Console.WriteLine($"Compartment ID: {compartmentId}");
                var provider = new ConfigFileAuthenticationDetailsProvider("DEFAULT");                    
                using (client = new DataSafeClient(provider, new ClientConfiguration()))
                {
                    var request = new ListAuditEventsRequest() { CompartmentId = compartmentId };
                    var responseEnum = client.Paginators.ListAuditEventsResponseEnumerator(request);
                    var eventList = responseEnum.SelectMany(r => r.AuditEventCollection.Items).ToList();
                    logger.Info($"eventList size:{eventList.Count}");
                    Console.WriteLine($"eventList size:{eventList.Count}");
                    foreach (var e in eventList)
                    {
                        logger.Info($"{e.Id} - {e.EventName}");
                        Console.WriteLine($"{e.Id} - {e.EventName}");
                    }
                }
            }
            catch (Exception e)
            {
                logger.Error($"Failed Datasafe example: {e.Message}");
                Console.WriteLine($"Failed Datasafe example: {e.Message}");
            }
        }
    }
} 

Different CompartmentId used for non-zero result:

using System;
using System.Linq;
using Oci.DatasafeService;
using Oci.DatasafeService.Requests;
using Oci.Common;
using Oci.Common.Auth;

namespace Oci.Examples{
    public class DatasafeExample3
    {
        private static NLog.Logger logger = NLog.LogManager.GetCurrentClassLogger();
        public static void Main()
        {
            logger.Info("Starting example3");
            Console.WriteLine("Starting example3");
            DataSafeClient client = null;
            try
            {
                var compartmentId =  "<value>";
                logger.Info($"Compartment ID: {compartmentId}");
                Console.WriteLine($"Compartment ID: {compartmentId}");
                var provider = new ConfigFileAuthenticationDetailsProvider("DEFAULT");                    
                using (client = new DataSafeClient(provider, new ClientConfiguration()))
                {
                    var request = new ListAuditEventsRequest() { CompartmentId = compartmentId };
                    var responseEnum = client.Paginators.ListAuditEventsResponseEnumerator(request);
                    var eventList = responseEnum.SelectMany(r => r.AuditEventCollection.Items).ToList();
                    logger.Info($"eventList size:{eventList.Count}");
                    Console.WriteLine($"eventList size:{eventList.Count}");
                    foreach (var e in eventList)
                    {
                        logger.Info($"{e.Id} - {e.EventName}");
                        Console.WriteLine($"{e.Id} - {e.EventName}");
                    }
                }
            }
            catch (Exception e)
            {
                logger.Error($"Failed Datasafe example: {e.Message}");
                Console.WriteLine($"Failed Datasafe example: {e.Message}");
            }
        }
    }
}

[jbcintra]

Hi, sorry I've a number of projects in parallel, so just getting to look back at this and it's now working in the CLI and SDK for a tenancy with Data Safe in place.
For a client without, I'm getting "{"data":{"items":null}}

just checked on a non data-safe client, and still errors in the dotnet logs:

System.AggregateException: One or more errors occurred. (Exception has been thrown by the target of an invocation.)
---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> Newtonsoft.Json.JsonSerializationException: Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'Oci.DatasafeService.Models.AlertCollection' because the type requires a JSON object (e.g. {"name":"value"}) to deserialize correctly.
To fix this error either change the JSON to a JSON object (e.g. {"name":"value"}) or change the deserialized type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List that can be deserialized from a JSON array. JsonArrayAttribute can also be added to the type to force it to deserialize from a JSON array.
Path '', line 1, position 1.
at Oci.Common.Http.Internal.ResponseHelper.ReadEntity[T](HttpResponseMessage response)
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Span1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at Oci.Common.GenericHelper.InvokeGenericMethod(GenericInvokerParameters parameters) at Oci.Common.Converter.FromHttpResponseMessage[T](HttpResponseMessage responseMessage) at Oci.DatasafeService.DataSafeClient.ListAlerts(ListAlertsRequest request, RetryConfiguration retryConfiguration, CancellationToken cancellationToken, HttpCompletionOption completionOption) --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification)
at cintra_oci_extract_sdk.Oci.OciDataSafe.ListAlerts(String compartmentId) in *****\Oci\OciDataSafe.cs:line 53
at cintra_oci_extract_sdk.Stage2.ProcessDatasafe(Compartment compartment) in *****\Stage2.cs:line 3332
2023-03-14 05:43:17.993 +00:00 [ERR] Inner exception
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> Newtonsoft.Json.JsonSerializationException: Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'Oci.DatasafeService.Models.AlertCollection' because the type requires a JSON object (e.g. {"name":"value"}) to deserialize correctly.
To fix this error either change the JSON to a JSON object (e.g. {"name":"value"}) or change the deserialized type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List that can be deserialized from a JSON array. JsonArrayAttribute can also be added to the type to force it to deserialize from a JSON array.
Path '', line 1, position 1.
at Oci.Common.Http.Internal.ResponseHelper.ReadEntity[T](HttpResponseMessage response)
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Span`1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Oci.Common.GenericHelper.InvokeGenericMethod(GenericInvokerParameters parameters)
at Oci.Common.Converter.FromHttpResponseMessage[T](HttpResponseMessage responseMessage)
at Oci.DatasafeService.DataSafeClient.ListAlerts(ListAlertsRequest request, RetryConfiguration retryConfiguration, CancellationToken cancellationToken, HttpCompletionOption completionOption)
2023-03-14 05:43:18.132 +00:00 [INF] capitaregservices 404 NotFound, Authorization failed or requested resource not found

I'll have a dig in again with the code you gave and see what outcome I get on different client types.

[jbcintra]

I've just removed the "return null;" statements to stop the code blowing up and the following is returned from a tenancy in which datasafe has never been enabled. specifically on the ManagedCompartmentForPaas.

2023-03-15 12:03:25.558 +00:00 [ERR] Unknown exception
System.AggregateException: One or more errors occurred. (Exception has been thrown by the target of an invocation.)
---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> Newtonsoft.Json.JsonSerializationException: Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'Oci.DatasafeService.Models.AlertPolicyCollection' because the type requires a JSON object (e.g. {"name":"value"}) to deserialize correctly.
To fix this error either change the JSON to a JSON object (e.g. {"name":"value"}) or change the deserialized type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List that can be deserialized from a JSON array. JsonArrayAttribute can also be added to the type to force it to deserialize from a JSON array.
Path '', line 1, position 1.
at Oci.Common.Http.Internal.ResponseHelper.ReadEntity[T](HttpResponseMessage response)
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Span1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at Oci.Common.GenericHelper.InvokeGenericMethod(GenericInvokerParameters parameters) at Oci.Common.Converter.FromHttpResponseMessage[T](HttpResponseMessage responseMessage) at Oci.DatasafeService.DataSafeClient.ListAlertPolicies(ListAlertPoliciesRequest request, RetryConfiguration retryConfiguration, CancellationToken cancellationToken, HttpCompletionOption completionOption) at Oci.Common.Utils.ResponseEnumerable2.GetNextPageResponseAsync(String nextPageToken)
at Oci.Common.Utils.ResponseEnumerable2.<>c__DisplayClass4_0.<<GetEnumerator>b__0>d.MoveNext() --- End of inner exception stack trace --- at Oci.Common.Utils.ResponseEnumerable2.GetEnumerator()+MoveNext()
at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.ToList()
at cintra_oci_extract_sdk.Oci.OciDataSafe.ListAlertPolicies(String compartmentId) in ****\Oci\OciDataSafe.cs:line 36
at cintra_oci_extract_sdk.Stage2.ProcessDatasafe(Compartment compartment) in ****\Stage2.cs:line 3308

So a standard 4xx error is not being returned, rather some value that is causing the error.

The function code is very simple:

public IEnumerable ListAlertPolicies(string compartmentId)
{
var request = new ListAlertPoliciesRequest() { CompartmentId = compartmentId };
var response = _client.Paginators.ListAlertPoliciesResponseEnumerator(request);
return response.SelectMany(r => r.AlertPolicyCollection.Items).ToList();
//error logged with Oracle 2022-12-06, return null till resolved
//return null;
}

I'll pass the tenancy details privately to @KartikShrikantHegde

[KartikShrikantHegde]

Hi @jbcintra , thanks for sending the info privately. And as we discussed the fix has been rolled out in the latest version of the .NET SDK. Please give it a try and let us know if you are still facing the issue.

Thanks.

[KartikShrikantHegde]

Hi @jbcintra , I am closing this ticket for now. Please feel free to reach out to us if you are still facing the issue. thanks.