microsoft/web-insights-service-archive

Web Insights Service

The repository is archived and is no longer maintained

Web Insights Service

Web Insights Service is a service that can be used to scan websites for various compliance issues on a periodic basis. It is Node.js/TypeScript project with shell scripts for install and update scenarios.

Contributing

All contributions are welcome! Please visit our Contributing page.

Contact us

Please file a Github Issue.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

Reporting security vulnerabilities

Please visit our Security page for more information about how to report security vulnerabilities.

web-insights-service-archive's Issues

[Bug] Intermittent 5xx responses from the storage API

Describe the bug

The storage API function app occasionally times out during a request, with error logs showing

Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.post-website-func
 ---> Microsoft.Azure.WebJobs.Script.Workers.WorkerProcessExitException: node exited with code 134
  2: 0xa1aa4c node::OnFatalError(char const*, char const*) [node], 3: 0xb9a62e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node], 6: 0xd68578 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]

and/or

fail: Host.Function.Console[0]
      FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
info: Host.Function.Console[0]
       1: 0xa1a640 node::Abort() [node]
fail: Host.Function.Console[0]
       2: 0xa1aa4c node::OnFatalError(char const*, char const*) [node]
fail: Host.Function.Console[0]
       3: 0xb9a62e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
info: Host.Function.Console[0]
       4: 0xb9a9a9 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
info: Host.Function.Console[0]
       5: 0xd57c25  [node]
fail: Host.Function.Console[0]
       6: 0xd68578 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]

The issue causes the function host to restart, leaving the service unavailable for several minutes. This seems to be an issue with the function host memory allocation, not memory allocation to the pod, because the pod's memory use does not approach its limit when this error occurs.

Area

• Install script
• Update script
• Service

To Reproduce

1. Either deploy a new dev resource group of the Web Insights Service and leave it running for an hour or more, or find an existing resource group which has been running for some time
2. Observe that the dashboard shows intermittent E2E test failures, availability drops, and low reliability
3. Open the logs (either with Application Insights or kubectl logs) and find the most recent exception/error logs from the storage API

Expected behavior

The service API should be reliable, and should not send frequent 5xx responses

Log files

See the description for log error messages

Additional context

Potential causes could be:

• Lack of scalability in the service making it unable to handle the request load from E2E tests (Increasing the number of replicas for the API pod had no effect on the frequency of 5xx responses)
• Too many functions in one function app (all functions share one heap)

[Bug] Investigate migrating to RBAC Key Vault

SDL TASK 12041

[Bug] Migrate app from ADAL to MSAL

Migrate applications to the Microsoft Authentication Library (MSAL)
How to migrate a Node.js app from ADAL to MSAL

[Bug] Investigate setting up vnet and private link for resources so that they are not externally accessible

SDL TASK 12040