Timeout during `/improve` command

[zmeir]

Timeout during `/improve` command

[okotek]

#181
@mrT23 can you review?

[krrishdholakia]

Hey @okotek were y'all able to get this - it's the force_timeout param

Happy to make a PR if required.

[zmeir]

Was addressed in #181 as far as I can tell, although even with 180 seconds I'm still seeings timeouts with large PRs. I'll try creating a reproducible example.

[krrishdholakia]

Hmm.. maybe we can remove timeouts from lite / make that an optional parameter, this should prevent these scenarios from occurring.

This was primarily added to deal with Azure / Replicate related issues where the model calls would hang for an unusual amount of time.

[krrishdholakia]

Ok, tracking this on our end - BerriAI/litellm#73

Will aim to have an updated version tested + pushed out by EOD. I'll make a PR for the change, which can then run through the normal testing y'all have for PR-Agent as well.

[zmeir]

This was primarily added to deal with Azure / Replicate related issues where the model calls would hang for an unusual amount of time.

It's possible that this is the issue I'm facing since I'm using Azure. I've only ever noticed it happen with /improve though, never with /review, /describe, etc.

[krrishdholakia]

I'm guessing pre-lite it didn't timeout, so did it hang or always complete successfully?

[zmeir]

From what I can tell it always completed successfully, but then again this is all pretty new stuff so it's possible I just didn't run into a large enough PR when using /improve in the pre-lite agent.

When I get the timeout again I'll try to run the pre-lite agent on the same PR and see if it works.

[krrishdholakia]

sounds great! i'll put set_timeouts as an optional param, which should remove any lite-related blockers between you-your azure calls.

[krrishdholakia]

update: default timeouts are now set to 600 seconds (~10 minutes)

[zmeir]

update: tested /improve again today on a fairly large PR with the latest main version and it didn't time out. I'll close this issue for new, and if it resurfaces I'll reopen it / open a new one with more details.

[mrT23]

/similar_issue

[github-actions]

Similar Issues:

1. [GITLAB] Installation Guide for GitLab please?

2. Failure in /improve command

3. Running Github Action: openai.error.RateLimitError