Comments (25)
I just pushed a PR to solve that issue: #210.
from finary_uapi.
My first thoughts:
- Could be as simple as a referer or user-agent check ? (I doubt but it's easy to try...)
- Could because there is no delay between the call with username and password and the call with the 2FA code... We could add a delay but it could fail because of the validity period of the 2FA code. Or we change our API, and we ask for the code interactively...
Any other ideas ?
from finary_uapi.
Actually, the error occurs right away on the first code, we don't even get the chance to submit the 2FA code.
from finary_uapi.
The following works, not sure why it doesn't work with requests yet.
curl 'https://clerk.finary.com/v1/client/sign_ins' \
--http1.1 \
-H 'User-Agent: finary_uapi 0.2.0' \
-H 'Connection: keep-alive' \
-H 'Origin: https://app.finary.com' \
-H 'Referer: https://app.finary.com/' \
-H 'Accept-Encoding: identity' \
--data-raw 'password=PASSWORD&identifier=EMAIL' \
--verbose
Translating this request exactly to requests
causes the bot_detected error.
The same call using httpx works. There is something special about requests
and clerk or finary detects it.
from finary_uapi.
@nmathey @MadeInPierre
A possible solution is to switch to httpx. There must be a subtle difference between requests and curl or httpx for requests to be detected as a bot with the configuration by default. I can't find this difference... Using httpx, most of the things seems to work for now... Is it a problem for your respective projects to use httpx instead of requests ?
from finary_uapi.
Note that even with httpx running from github actions doesn't work... clerk detects a bot.
Locally all the tests but 2 are passing.
from finary_uapi.
See https://github.com/lasconic/finary_uapi/tree/switch_httpx
from finary_uapi.
Moving away from requests shouldn't be a problem for me.
So from GitHub it still doesn't but you were able to get it working from your wrapper?
from finary_uapi.
Yes, the tests don't work from Github Actions server but they run fine from my home computer. You can try to signin with the main branch (and should get the bot detected error) and with this branch and you should get success.
from finary_uapi.
Got same error
INFO:httpx:HTTP Request: POST https://clerk.finary.com/v1/client/sign_ins "HTTP/1.1 403 Forbidden"
{
"errors": [
{
"message": "Bot traffic was detected. Please try again or contact support.",
"code": "bot_detected"
}
]
}
Did your branch includes all headers additions you tested with curl?
--http1.1 \
-H 'User-Agent: finary_uapi 0.2.0' \
-H 'Connection: keep-alive' \
-H 'Origin: https://app.finary.com' \
-H 'Referer: https://app.finary.com/' \
-H 'Accept-Encoding: identity' \
from finary_uapi.
See https://community.finary.com/t/api-finasync-extension-open-source/10104/24 for the current official status (by Finary CTO).
The behavior described in this bug is "expected". For me, the HTTPX branch works. I tried several headers but of course, I made sure to use exactly the same one while testing. I had the same header in the main branch (using requests), with curl and with HTTPX (including forcing HTTP 1.1 when possible). Clerk does smart things for their bot detection for sure.
For now, I would like to get more data. Does the main branch work for you ? Does the HTTPX branch work ?
from finary_uapi.
Sorry I'm bit confuse, and might be a dummy question, but can you confirm headers you successfully tested are hardcoded in your Httpx branch?
Or should I configured them myself somewhere?
Because when I tested your branch, I just did as is.
from finary_uapi.
@nmathey they aren't hardcoded but the HTTPX branch works for my laptop as it, without configuring anything.
from finary_uapi.
Hey!
I've been developing my own client with reqwest to automatically update balances of manual accounts, and I faced the same problem while developing:
- I tried to put the same headers than the browser but I was detected as a bot
- I tried CURL and it works
I was using this repo as a reference for implementation (thanks for the awesome work btw), and I naively tried to set my user-agent to: finary_uapi 0.2.0
, and it worked. FYI, I'm also setting the following headers:
- Origin:
https://app.finary.com
- Referer:
https://app.finary.com
- Connection:
keep-alive
- Accept:
*/*
I haven't tried anything else since then, but my guess is that Clerk detects http clients that try to behave like browsers as they're probably bots most of the times, and allows clients that are transparent and saying that they're an API.
So basically, I don't think it's an issue related to the http client, but it's probably linked to headers. Hope those information can help you!
from finary_uapi.
Thank you for the new look. I checked every possible combinations 6 months ago but your headers seems to just work ! It can't hurt to merge the PR.
from finary_uapi.
@all-contributors please add @clemlesne for code
from finary_uapi.
I've put up a pull request to add @clemlesne! 🎉
from finary_uapi.
@all-contributors please add @clemlesne for code
from finary_uapi.
@clemlesne already contributed before to code
from finary_uapi.
Still getting same message with finary-uapi 0.2.1 b9a9a3f
{ "errors": [ { "message": "Bot traffic was detected. Please try again or contact support.", "code": "bot_detected" } ] }
Is that 100% time working on your side ?
from finary_uapi.
Don't you think Finary is actively monitoring our solutions here to update their bot detection with any found solution? I mean if they want to close their api it's not a big work to track this issue and ban any further solution proposed... :(
from finary_uapi.
@nmathey Locally it works for me, 100% of the time. I can run the tests without problems, and they do quite a lot of calls. You don't do any funny things ? like overriding the default headers ? Do you get this message on sign-in ? or after multiple calls ?
@OxyFlax I believe they have better things to do than monitoring our conversation... Also they did provide a demo account to test the code. I'm under they impression Finary just checked a box in Clerk admin UI for bot detection and Clerk is changing their bot detection algorithm continuously
from finary_uapi.
Let me add that the tests running on Github actions are not working and we still get a "bot traffic detected". See https://github.com/lasconic/finary_uapi/actions/runs/10584142604/job/29327710214#step:8:1424
Maybe Clerk uses an IP blacklist ? No idea.
from finary_uapi.
Yes, they provide testing tokens for development instances : https://clerk.com/docs/testing/overview#testing-tokens so they definitely do something against github actions runners.
from finary_uapi.
I'm using finary_uapi out of the box, and getting the error straight from on sign-in.
I'm using the same IP for both finary_uapi and browsing finary app/portal, so I don't think my IP is blacklisting (?)
from finary_uapi.
Related Issues (20)
- Import/update real estate data? HOT 14
- 3rd party platform integration: via importer or separate project integrating finary_uapi? HOT 7
- Fetch checking account transactions HOT 5
- Fetch investment account transactions HOT 1
- Fetch credit card transactions ? HOT 7
- implement a function to call `users/me/organizations`
- Add support for Organisation
- Handling currencies while adding/updating real estates? HOT 1
- Minor finary_uapi versioning for bug fix update? HOT 2
- Add crowdlending projects HOT 17
- `/auth/signin` will soon be retired HOT 7
- Remove retired endpoints
- Access to shared portfolio
- Add tests for real_estates
- saving_accounts disappear HOT 1
- Add "contructionYear" in real_estates HOT 5
- Can't signing anymore ? HOT 11
- START-UPS / PME / PRIVATE EQUITY HOT 1
- Add Nexo support HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from finary_uapi.