Login fails due to Recaptcha

[ondrej1024]

Login fails due to Recaptcha

[ondrej1024]

Here is the plan:

The initial login procedure has to be done in the browser manually. If this was successful, e cookie containing an auth token is provided on the web page. The cookie can be easily saved via a browser addon (I tried Cookie Quick Manager). Then we can pass the file containing the cookie to carelink_client_cli or carelink_client_proxy as parameter which will get things going. After that the token is refreshed each time it is about to expire.

I have done some modifications to the carelink client and it is working here. Still need to do some cleanup before releasing the new version.

This way the client should continue to get data from Carelink as long as token refresh is successful. If it fails a new manual login has to be done.

To improve the token handling, the carelink_client_proxy could provide a simple web page where the new token obtained with the manual login can be pasted. Eventually it should be possible to get a new token automatically even after the old one has expired. There must be some kind of special "refresh token" which can be used for that. Apparently the Carelink Connect app uses this procedure because it asked for manual login only the very first time. But at the moment there are not sufficient information available to implement this.

[ondrej1024]

I released a workaround in the recapcha_workaround branch. Can you please try this and report back if it works for you?

[ondrej1024]

Happy to hear it's working also for you. But be aware that this is really a quick'n'dirty hack. It will break when the recapcha token verification is implemented properly on the Carelink server. Hopefully we will also have a solution for this when it happens.

[ondrej1024]

Here my tokens are still refreshing successfully. This is probably because my carelink_python_client hasn't made a complete login for two 2 days.

If login fails for you it means that capcha response validation is now implemented on the Carelink server and we cannot do a successful login anymore with the current version of the carelink_python_client.

xDrip is implementing the capcha validation via the manual login procedure on the Carelink web page. While this is feasible for a smarthone app, it is no option for a command line tool like carelink_python_client. So we need to find ways around that.

[ondrej1024]

Thanks for the feedback. Good to hear that things also work for others. Here the carelink_client_proxy runs for about a week before requesting a new login. I still have to fix some small issues and add a simple web interface to pass a valid auth token to the proxy without the need to restart it. And documentation needs to be updated. I hope to have an official release by the end of this week.

[hhuitema]

Just did a quick try, and it works perfectly to download the 'recent_data'...

[Lintzman]

Just did a quick try also and it works. Thanks

[tloczekt]

Hi Ondrej1024, should I set my own response or leave like this : ""g-recaptcha-response":"abc" ?

[hhuitema]

This is probably a first attempt of Medtronic to limit the amount of calls to their systems by unauthorized sources. Looking at the code and making a test myself it seems that they simply added a new mandatory variable to the API call. But they are not yet checking the actual validity.

[ondrej1024]

The value you put there doesn't seem to matter. I guess it is not verified at all on the Carelink server ATM. It seems just to check that it is not empty. When proper validation will be done we need to provide a valid token there.

[tloczekt]

The value you put there doesn't seem to matter. I guess it is not verified at all on the Carelink server ATM. It seems just to check that it is not empty. When proper validation will be done we need to provide a valid token there.

Thank you. I have guessed already that value doesn't matter nowadays.

[ondrej1024]

I have published a new version of the carelink_client on the refresh_token branch

This implements the periodic auth token refresh instead of logging in with user credentials each time the token expires. I have had it running all night here and I can see that the token provided by the Carelink server is valid for 40min. Before expiration it has been refreshed successfully each time.

So when requesting data from Carelink periodically (with carelink_client_proxy, for example) user login has to be done just once at the beginning or when the token refresh fails for any reason. We'll see later if we can do the login automatically (as it is done today) even when the recapcha response validation is implemented on the Carelink server.

[tloczekt]

Hi everybody,
Only me or you also have a problem with carelink_client.py today (since around 4am)?
Both recaptcha branch and refresh_token_branch doesn't work for me. Error code 400 is only response.

[Lintzman]

Yes, I stopped getting data about 5 hours ago with the same error.

[hhuitema]

Same here.. tried both branches.

[2023-10-06 10:25:16:INFO] Performing login [2023-10-06 10:25:17:INFO] Trying to refresh token [2023-10-06 10:25:17:INFO] No token to refresh [2023-10-06 10:25:17:INFO] Trying to refresh token [2023-10-06 10:25:17:INFO] No token to refresh [2023-10-06 10:25:17:INFO] Trying to refresh token [2023-10-06 10:25:17:INFO] No token to refresh [2023-10-06 10:25:17:INFO] Trying to refresh token [2023-10-06 10:25:17:INFO] No token to refresh Traceback (most recent call last): File "C:\my stuff\Dev\Python\carelink-python-client-refresh_token\carelink_client_cli.py", line 56, in <module> if client.login(): File "C:\my stuff\Dev\Python\carelink-python-client-refresh_token\carelink_client.py", line 437, in login printdbg("auth_token_validto = " + self.__httpClient.cookies.get(CARELINK_TOKEN_VALIDTO_COOKIE_NAME)) TypeError: can only concatenate str (not "NoneType") to str

[hhuitema]

it might be the care that Medtronic added a validation check on the content provided in the Captcha parameter.

The XDrip project the approach was to add features to 'go with the flow' on the captcha and providing correct information.
My java skills are a bit rusty so i have some difficulty following what they are exactly doing.

For reference : benceszasz/xDripCareLinkFollower#37
So far no complaints on this topic in recent hours.

[hhuitema]

This would indeed work nicely on a mobile device, machine to machine makes live difficult in this case.

I did find this piece of code that can provide the Captcha ID based on the anchor url in the login page.. I hope this might help.
https://github.com/Hartman5/recaptchaV3-Bypass/blob/main/index.js

[greencoder]

I guess we always assumed Medtronic would shit us out, but boy does this really suck. I've been able to keep my A1C at 5.2 by paying close attention to the analysis and reports I generated with my data until now.

Knowing our options could be very limited in the future, what is the viability of doing something with an interactive login? Yes, it sucks for a server-based client, but at this point I'm not sure what choice there is apart from trying to play cat-and-mouse against captcha tech.

If I had to set something up on a Raspberry Pi in my house it would be better than nothing, especially if the token can be refreshed without an interactive session. (The carelink connect app hasn't asked me to re-log in for as long as I've had it)

I've got experience with headless Chrome and pypetteer, I'm happy to help if there is any way I can be useful. (I used to scrape the carelink web app until I discovered this project)

[ondrej1024]

I have released a beta version of the carelink client on the initial_token branch.

Both carelink_client_cli and carelink_client_proxy can be launched without any parameters now, but they look for a json file containing a valid auth token in the file in /tmp/cookies.json. You can also specify a different file name with the -t parameter.

The file can be obtained with the following procedure. You need to have the Cookie Quick Manager plugin installed.

• Perform login on the Carelink Web page
• With the page still active, open Cookie Quick Manger from the extensions menu
• Select option "Search Cookies: carelink.minimed.eu"

• From the new page select "Save domain to file" from the "Export/Import" icon

• This will save a file called cookies.json to your download folder.
• Now you should close the Carlink login page to avoid automatic logout after some time

I recommend to use a dedicated Carelink follower account for this to avoid invalidating a token which carelink client is using when logging into the Carelink account from the web page.

Please give it a try and let me know if it works for you.

[greencoder]

Unfortunately, it didn't work for me:

Scotts-iMac:carelink-python-client snewman$ python carelink_client_cli.py --tokenfile cookies.json --verbose
Client created!
Client login error! Response code: None Error message: None

[ondrej1024]

@greencoder Please check your cookie file. It shoud look like this:
cookies.json

You need to have the auth_tmp_token and application_country cookies in it.

[tloczekt]

Edit: my fault. I checked the carelink_client.py. not cli or proxy

To me also doesn't work. I have a proper structure of cookies.json.
Return message for :
print(client.getLastErrorMessage())
print(client.getLastResponseCode())

is both NONE

[ondrej1024]

I have pushed an update which should resolve the problem you encountered. Please test again and let me know.

[tloczekt]

The answer now is :
Malform initial token
Client login error! Response code: None Error message: None

[ondrej1024]

@tloczekt Could you send me the token which generates this error? It will be expired by now, so there is no security issue. You can send it to my email (see the comment header in the code). Thanks

[tloczekt]

It's just sent.

[ondrej1024]

Pushed another fix. Please try again.

[tloczekt]

Pushed another fix. Please try again.

Ok. When I will be on computer I will try it

[MinecraftLPtwo]

Pushed another fix. Please try again.

I will try.
actually i get data and i have add publish mqtt to the Python code for my HomeAssistent.

[greencoder]

I was able to log in:

Scotts-iMac:carelink-python-client snewman$ python carelink_client_cli.py 
[2023-10-15 11:22:32:INFO] Initial token expires in 2365s (Sun Oct 15 18:01:58 UTC 2023)
[2023-10-15 11:22:32:INFO] Performing login
[2023-10-15 11:22:33:INFO] Found patient Scott Newman (scottnewman)
[2023-10-15 11:22:33:INFO] Login successful

I haven't tried using the client in my own program yet but it's looking promising!

[MinecraftLPtwo]

Pushed another fix. Please try again.

I will try. actually i get data and i have add publish mqtt to the Python code for my HomeAssistent.

It works, only the cookie save I missed, but @ondrej1024 you added it as a comment.
I hope this will come soon. I try to add it but failed.

[greencoder]

Is there functionality yet to save the refreshed token? I tried to run it later and got this:

Scotts-iMac:carelink-python-client snewman$ python carelink_client_cli.py 
[2023-10-15 16:48:20:INFO] Initial token has expired 17182s ago
Client login error! Response code: None Error message: None

[Lintzman]

I found that new auth token and expiry time cookies are set once I called the 'reauth' API method so I replace the existing values with the new cookie values and I haven't had to login for the last 3 or 4 days. Hope this helps.

[ondrej1024]

In the current version of carelink_client_cli and carelink_client_proxy the initial token is read from a file and then it is refreshed each time the token is about to expire. The refreshed token is not saved to a file at the moment.

I have been running the carelink_client_proxy for 3 days now and it has been refreshing the token successfully all the time. So I had no need to restart it and get a new token manually.

But I agree, it might be handy to save the refreshed token to a file, so if you restart the application within the expiry period it should just start working again.

[greencoder]

I found that new auth token and expiry time cookies are set once I called the 'reauth' API method so I replace the existing values with the new cookie values and I haven't had to login for the last 3 or 4 days. Hope this helps.

Do you have a code snippet I could try?

client = carelink_client.CareLinkClient(token, country, patient)

if client.login():
    # do some fancy thing to save the new cookie values

[Adirael]

Works perfect for me. Note that it'll fail with the original account (403 on auth) but works great with a follower account. I used this chance to replace a crappy script I used to run for the proxy as a systemd service that I query directly from HomeAssistant.

Thanks!

[ondrej1024]

Hi all,
If anyone wants to test this, I just pushed a new version of the carelink_client_proxy to the initial_token branch.

With this version you can still pass the cookie file with the initial token via command line parameter as before, but now you can also set the initial token or update it via a Web GUI which is provided by the proxy. Just navigate to this URL with your brower:

http://<proxy ip address>:8081

The Web GUI should load and you can paste the token and country code in the input fields. Then hit "Save" and the proxy will try to log into the Carelink server and continue with the periodic data downloads.

[code-with-abe]

Hi, I currently connect to carelink prior to the recaptcha as shown here, i downloaded the cookie.json, how do i modify this to use the cookie file?

[ondrej1024]

@code-with-abe Take a look at carelink_client_cli.py as an example for how this can be done.
Please note that the latest changes are only in the initial_token branch and not yet merged to master.

[greencoder]

@code-with-abe In case it helps, I have changed my approach to pulling data in the short term. Instead of logging in every 15 minutes and fetching data, I'm keeping carelink_proxy.py running on a Raspberry Pi on my desk and writing the data to a JSON file that I consume with another script to store the data and do my reporting.

Eventually I hope that the updated token will get persisted so I can change it back to a cron job that runs periodically instead of maintaining a long-running task.

[code-with-abe]

@greencoder thanks, i like your solution, how do you run the carelink_proxy.py, do you use gunicorn and such. Sorry i'm bad at webserver stuff.

[greencoder]

how do you run the carelink_proxy.py, do you use gunicorn and such. Sorry i'm bad at webserver stuff.

It's WAY more low-tech than that in my setup - I have a raspberry pi on my desk and I run the proxy script and don't log out - I just turn the monitor off.

You could also look at using the linux screen command to start a session that will stay open after you disconnect:
https://www.tecmint.com/keep-remote-ssh-sessions-running-after-disconnection/

[tloczekt]

@code-with-abe Take a look at carelink_client_cli.py as an example for how this can be done. Please note that the latest changes are only in the initial_token branch and not yet merged to master.

carelink_client_cli.py from initial_token was modified last month, so it seems work only 40min when 1st token which is loaded from JSON file is valid. It can't refresh token properly in my case.
I have website site when I used cron to run carelink_client_cli.py, so solution with carelink_client_proxy.py is not solution for me.

[code-with-abe]

I tried proxy didnt work with entering the token in UI as well as placing the /tmp/cookies.json

[greencoder]

@code-with-abe Try this:

python carelink_client_proxy.py --tokenfile /path/to/cookies.json

[code-with-abe]

No luck

[ondrej1024]

I tried proxy didnt work with entering the token in UI as well as placing the /tmp/cookies.json

Did you also provide the country code in the UI?

[ondrej1024]

Also make sure you don't log out from the Carelink web page as this will invalidate your token. You need to close the page after login, otherwise it will logout automatically after some time.

[Akilan1999]

Hi All,
Thanks a lot for the effort for implementing the following fix for Carelink.
When trying to login using a patient account I get the following error.

{"error":{"type":"NoPermission","group":"AUTH"}}

On the follower account the login works as intended.

[ondrej1024]

This Carelink client is intended to be used with a follower account. I never tried to make it work with a patient account. Is there any reason why it should be used with a patient account?

[Akilan1999]

We currently have sessionIDs of a bunch of Carelink patients accounts. The flow follows similar to xDrip to get the session information and we intend to use your library to refresh the tokens and extract the readings.

This flow was working before Carelink introduced Captcha.

Apologies for the late response.

[ondrej1024]

This issue has been resolved so I close it.

I am now working on implementing the new API which is used by the official Carelink Connect app. This API has the advantage that after login it provides a refresh token which is valid for 7 days. So in most cases this should be sufficient to refresh the tokens automatically even if the carelink client is not used for some time or if you have quite a long network outage. Therefore manual login procedure with reCapcha should be used only the very first time.