Function hangs and doesn't even timeout unless terminated

[shyamal890]

Function hangs and doesn't even timeout unless terminated

[shyamal890]

Recent hung orchestration instance id - 983c33044cfa4df6b0669502b56bfea9

Attaching get status output, check how even with functionTimeout of 03:00:00 it takes more than 8 hours.

{
    "name": "durableScraping",
    "instanceId": "983c33044cfa4df6b0669502b56bfea9",
    "runtimeStatus": "Failed",
    "input": "{\"domain\":\"hubspot.com\",\"max_pages\":\"2000\",\"scrape_with\":\"auto\"}",
    "customStatus": null,
    "output": "Orchestrator function 'durableScraping' failed: Activity function 'durableActScrapeWebsite' failed: Timeout value of 03:00:00 was exceeded by function: Functions.durableActScrapeWebsite \n {\"$type\":\"Microsoft.Azure.WebJobs.Host.FunctionTimeoutException, Microsoft.Azure.WebJobs.Host\",\"Timeout\":\"03:00:00\",\"InstanceId\":\"8ecce615-99a4-4107-8326-4fd065458147\",\"MethodName\":\"Functions.durableActScrapeWebsite\",\"Handled\":false,\"ClassName\":\"Microsoft.Azure.WebJobs.Host.FunctionTimeoutException\",\"Message\":\"Timeout value of 03:00:00 was exceeded by function: Functions.durableActScrapeWebsite\",\"Data\":null,\"InnerException\":null,\"HelpURL\":null,\"StackTraceString\":\"   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryHandleTimeoutAsync(Task invokeTask, CancellationToken shutdownToken, Boolean throwOnTimeout, CancellationToken timeoutToken, TimeSpan timeoutInterval, IFunctionInstance instance, Action onTimeout) in D:\\\\a\\\\_work\\\\1\\\\s\\\\src\\\\Microsoft.Azure.WebJobs.Host\\\\Executors\\\\FunctionExecutor.cs:line 663\\n   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker invoker, ParameterHelper parameterHelper, CancellationTokenSource timeoutTokenSource, CancellationTokenSource functionCancellationTokenSource, Boolean throwOnTimeout, TimeSpan timerInterval, IFunctionInstance instance) in D:\\\\a\\\\_work\\\\1\\\\s\\\\src\\\\Microsoft.Azure.WebJobs.Host\\\\Executors\\\\FunctionExecutor.cs:line 571\\n   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstanceEx instance, ParameterHelper parameterHelper, ILogger logger, CancellationTokenSource functionCancellationTokenSource) in D:\\\\a\\\\_work\\\\1\\\\s\\\\src\\\\Microsoft.Azure.WebJobs.Host\\\\Executors\\\\FunctionExecutor.cs:line 527\\n   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in D:\\\\a\\\\_work\\\\1\\\\s\\\\src\\\\Microsoft.Azure.WebJobs.Host\\\\Executors\\\\FunctionExecutor.cs:line 306\\n   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in D:\\\\a\\\\_work\\\\1\\\\s\\\\src\\\\Microsoft.Azure.WebJobs.Host\\\\Executors\\\\FunctionExecutor.cs:line 352\\n   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance, CancellationToken cancellationToken) in D:\\\\a\\\\_work\\\\1\\\\s\\\\src\\\\Microsoft.Azure.WebJobs.Host\\\\Executors\\\\FunctionExecutor.cs:line 108\",\"RemoteStackTraceString\":null,\"RemoteStackIndex\":0,\"ExceptionMethod\":null,\"HResult\":-2146233088,\"Source\":\"Microsoft.Azure.WebJobs.Host\",\"WatsonBuckets\":null}",
    "createdTime": "2023-09-27T09:05:00Z",
    "lastUpdatedTime": "2023-09-27T17:11:08Z"
}

[nytian]

As mentioned in the issue #544, these two issues are from the same reason so I would suggest close this one and keep the discussions in #544.