Investigate/document how to use with OpenRouter

[ErikBjare]

Investigate/document how to use with OpenRouter

[zigabrencic]

@vibor you can assign me to this one. Will start looking into it at towards the end of the week.

[TheoMcCabe]

Yeah i saw the same thing with mynt. The only things that could handle the logic required to contruct the requests properly were anthropic or open ai models. Even google gemini sucked!

There are other benefits to using open router other than just getting access to additional models which work well with gpte... like being visible on the open router app rankings. And having access to new models early (which may in future work well with gpte)

I'd say the models not working very well shouldnt be a blocker on us completing this integration

[viborc]

Done, thanks, @zigabrencic and @ErikBjare!

[TheoMcCabe]

I wrote an article on this

https://medium.com/@tedisaacs/from-openai-to-opensource-in-2-lines-of-code-b4b8d2cf2541

[zigabrencic]

@TheoMcCabe that's great. Thanks for sharing the post.

The issue I came across with gpte was that specific open model {sizes, types} result into rather poor performance of gpte. Running on the local machine has the same issue as with OpenRouter.

In other words. Making the API calls to open router via langchain is the easy part. Making gpte work with open models, is well harder. Hopefully running new gpte benchmarks with Open Models gives us some clarity here.

I'm waiting for llama3 to come out this week to see if it's any better. If it is I suspect llama3 to be also available on OpenRouter as well.

I'm not against adding a few lines from your post to the docs right away. We just need to warn the users as Erik mentioned above about experimentally of the feature.