Get tokens fails because of a captcha

[chouffy]

Get tokens fails because of a captcha

[euro2]

I now tried a fresh docker run to get tokens. Same result. It errors on the captch url. If you paste it in the browser you get an actual captcha.... Maybe too many polls to their server?

[joshuaferrara]

maybe a dumb question, but are you running v5.2.0 of home_assistant_tgtg? i requested new tokens using the underlying Python library @ its latest version (see their README.md), upgraded from v5.1.0 to v5.2.0 in HACS w/o restarting, updated my HA configuration to use the new tokens, and then restarted everything. after that I'm back in action.

[tjorim]

I now tried a fresh docker run to get tokens. Same result. It errors on the captch url. If you paste it in the browser you get an actual captcha.... Maybe too many polls to their server?

Define 'fresh docker run', did you also do an clean build (as it might have cached some layers. Try this: docker build --no-cache https://github.com/Chouffy/home_assistant_tgtg.git#main --tag "homeassistant_tgtg_tokens:latest".

[euro2]

I now tried a fresh docker run to get tokens. Same result. It errors on the captch url. If you paste it in the browser you get an actual captcha.... Maybe too many polls to their server?

Define 'fresh docker run', did you also do an clean build (as it might have cached some layers. Try this: docker build --no-cache https://github.com/Chouffy/home_assistant_tgtg.git#main --tag "homeassistant_tgtg_tokens:latest".

A fresh run is quite vaque isnt it :)...

What i ment is that i installed docker in a new LXC container and issued the commands as described in installation 1b:
docker build https://github.com/Chouffy/home_assistant_tgtg.git#main --tag "homeassistant_tgtg_tokens:latest" docker run --rm -it homeassistant_tgtg_tokens

EDIT: it works now. The captcha thought i was a bot yesterday..... I now have edited my old WORKING config to match the newly obtained one but my sensors are all still unavailable. I have updated the HACS to 5.2.0 a couple of days ago as well.... (+ full reboot)

{ "domain": "tgtg", "name": "TooGoodToGo", "codeowners": ["@chouffy", "@tjorim"], "documentation": "https://github.com/Chouffy/home_assistant_tgtg", "iot_class": "cloud_polling", "issue_tracker": "https://github.com/Chouffy/home_assistant_tgtg/issues", "requirements": ["tgtg==0.16.0"], "version": "5.2.0" }

Home Assistant 2023.5.4 Supervisor 2023.04.1 Operating System 10.2 Frontend-versie: 20230503.3 - latest

So still troubleshooting the unavailable sensors.

[euro2]

New info: the Home assistant log says captcha again:

`Logger: homeassistant.components.sensor
Source: custom_components/tgtg/sensor.py:189
Integration: Sensor (documentation, issues)
First occurred: 10:04:05 (1 occurrences)
Last logged: 10:04:05

Error while setting up tgtg platform for sensor
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 304, in _async_setup_platform
await asyncio.shield(task)
File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/config/custom_components/tgtg/sensor.py", line 76, in setup_platform
add_entities([TGTGSensor(each_item_id)])
File "/config/custom_components/tgtg/sensor.py", line 96, in init
self.update()
File "/config/custom_components/tgtg/sensor.py", line 189, in update
self.tgtg_answer = tgtg_client.get_item(item_id=self.item_id)
File "/usr/local/lib/python3.10/site-packages/tgtg/init.py", line 276, in get_item
self.login()
File "/usr/local/lib/python3.10/site-packages/tgtg/init.py", line 152, in login
self._refresh_token()
File "/usr/local/lib/python3.10/site-packages/tgtg/init.py", line 138, in _refresh_token
raise TgtgAPIError(response.status_code, response.content)
tgtg.exceptions.TgtgAPIError: (403, b'{"url":"https://geo.captcha-delivery.com/captcha/?initialCid=AHrlqAAAAAMAjokilHqJT8cA2WmLtw==&cid=73DGL~8P1WGJMtY3~PkNluG_n8aH7T3hn8JzlPC~DPsg6SGlH8~E4_4Ec9fhe56p7eD6ZZ9T3he0GuJXiMG80dv1_xVtulrutO7riDwH6_5NbE-4dnofL3jIdSs0e8Nr&referer=https%3A%2F%2Fapptoogoodtogo.com%2Fapi%2Fauth%2Fv3%2Ftoken%2Frefresh&hash=1D42C2CA6131C526E09F294FE96F94&t=fe&s=35558&e=1f8a41279d5b89aed8e94e08de3b9b0ba52cb7b17ec8fd7e1699e673dbb10f51"}')
`

So TGTG thinks im spamming? My refresh is 300sec.

Once i started my android app on the same IP as HA, the app gave me the captcha as well. So that should be it. They changed something to stops excessive polling.

[euro2]

nope. keep getting captcha's again with docker run --rm -it homeassistant_tgtg_tokens
And the one time this morning it worked and i got the tokens, i pasted over the tokens to HA, but then i i had captchas in the Homeassistant tgtg error logs. So i think im flagged as spammer or something.

On android i cleared cache and storage of the app and started over. Again a captcha before i could continue with email. After that the app works fine.

I not sure what to do at the moment. Ill wait a few days before trying again. In HA i remove the HACS addon so it wont poll the tgtg api

[Alfagek]

Here now also problems with version 5.2 on HA 2023.5.4.

Did have this error first in HA log

`Logger: homeassistant.helpers.entity
Source: custom_components/tgtg/sensor.py:189
Integration: tgtg (documentation, issues)
First occurred: 13:11:45 (200 occurrences)
Last logged: 16:06:16

Update for sensor.tgtg_xxxxxxxxxxxxxxxxxxxxxx fails
Update for sensor.tgtg_xxxxxxxxxxxxxxxxxxxxxx fails
Update for sensor.tgtg_xxxxxxxxxxxxxxxxxxxxxx fails
Update for sensor.tgtg_xxxxxxxxxxxxxxxxxxxxxx fails
Update for sensor.tgtg_xxxxxxxxxxxxxxxxxxxxxx fails
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 554, in async_update_ha_state
await self.async_device_update()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 781, in async_device_update
await hass.async_add_executor_job(self.update)
File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/config/custom_components/tgtg/sensor.py", line 189, in update
self.tgtg_answer = tgtg_client.get_item(item_id=self.item_id)
File "/usr/local/lib/python3.10/site-packages/tgtg/init.py", line 287, in get_item
raise TgtgAPIError(response.status_code, response.content)
tgtg.exceptions.TgtgAPIError: (403, b'{"url":"https://geo.captcha-delivery.com/captcha/?initialCid=AHrlqAAAAAMAtC-4u-6q-ZsAVPUifw==&cid=5tBB_ZqEGa4qKLQdSZY-aErGCQ7kzOCaftnig7A-WITA9DLKzuIQu_cbODD-B9nirRHYmOFZvVYwT1WLapEk7-Nr1c15d94mhcB5hovNNIMN0t7aPUU8dGPRXFmdJUzB&referer=https%3A%2F%2Fapptoogoodtogo.com%2Fapi%2Fitem%2Fv8%2F53574&hash=1D42C2CA6131C526E09F294FE96F94&t=fe&s=35558&e=f73668f01a34c24a90c7911b0116da56e864aca6dbbd9b332046b85022e4601a"}')`

Then i did try to get new tokens in terminal. Can start the "get tokens", but after entering my mail adres ill get the next error:
➜ tgtg python3 tgtg_get_tokens.py Type your email linked to your TGTG account: <MY MAIL ADRES> Using version 23.5.1 Traceback (most recent call last): File "/config/tgtg/tgtg_get_tokens.py", line 11, in <module> tgtgClient.get_credentials() File "/usr/lib/python3.11/site-packages/tgtg/__init__.py", line 90, in get_credentials self.login() File "/usr/lib/python3.11/site-packages/tgtg/__init__.py", line 181, in login raise TgtgLoginError(response.status_code, response.content) tgtg.exceptions.TgtgLoginError: (403, b'{"url":"https://geo.captcha-delivery.com/captcha/?initialCid=AHrlqAAAAAMAngbzfJ75Gb8AVPUifw==&cid=4Wh6TDs5~PIfjvdQ527wcGACGk~DqRsvsxbFIm-Nb9SZlK6etsWNOVvkVC7yZjdPVfFAvPBlS2sKtlawh8XS7_9xgv4zphorYoP2AVpNVPmu_GdF~aQLbpiepEeZTwNd&referer=https%3A%2F%2Fapptoogoodtogo.com%2Fapi%2Fauth%2Fv3%2FauthByEmail&hash=1D42C2CA6131C526E09F294FE96F94&t=fe&s=35587&e=726300966a9904a2190c56879aacd14e94f93e4e461247581dc219889f763395"}')

Also before i did do the new "get token" command, when i did opened the app on my iPhone, i did get a "captcha" puzzle. Never had that before.

[euro2]

I hope this hurdle doesnt kill the Integration :(

[Alfagek]

It’s working here again. I think it was some kind of Ip block. Only strange thing that the app did work on the same WiFi network.

Did do this afternoon a new “get token” past it in to de yaml. Restart HA and error was gone. And sensors are working again (with boxes)

Edit:

It did work for a few hours. And now I do get errors again. And when i start the app, I did get again the captcha puzzle. Did have a pull for 60 sec. So adjust it to 120sec and not restarted HA. Let’s see what will happen.

[euro2]

please keep us updated on your progress

[Alfagek]

Still no succes. 2 days ago i did get a message from tgtg addon on my phone. And after that it was down again. And again the captcha puzzle in the tgtg app. So didn't do anything with the intergration in ha. It wasn't doing anything (i think) because i did get the error, intergration is not working because of the captcha 403 error.

Today i started the app, all working. So restart HA and error captcha 403. Opening the app on phone capctha puzzle again :-(. So did that, restart HA en and again same error 403.

Today i disabled tgtg in HA. And will wait 24 hours, before i turn it on again. See if that wil do something.

A other strange thing i did mention in HA with TGTG. If i go to terminal and i give the command to get tokens. I do get the next error:
➜ tgtg python3 tgtg_get_tokens.py Traceback (most recent call last): File "/config/tgtg/tgtg_get_tokens.py", line 4, in <module> from tgtg import TgtgClient ModuleNotFoundError: No module named 'tgtg'

When i "reinstall tgtg" in terminal, its working again. I also did had this a few days ago. I can't remeber if this was also before i did have alle the "tgtg errors". Working now with 2023.6.1

[Alfagek]

Well i don't have a clue how to handle this. I'll hope that @tjorim has a few suggestions to try.

Before i did start this afternoon, i started the app on my phone: No captcha puzzle or what so ever.

I did try to get tokens by using "get tokens". After entering mail adres: 403 error.
Then i did see on my phone, a captcha puzzle was there. So did puzzle, closed app en did try again with same result.
Then i removed tgtg install with "pip uninstall tgtg". And used "pip install tgtg", and tgtg 16 was installing again.
Then i did try again to get the token, same 403 error, and no captcha on the app.
I did try to use the "docker" way on a other virtual machine with HA (on Advanced ssh & Web Terminal) (my test system), same result.

So i'm totally lost.
All did on the same network.

[euro2]

i think TGTG noticed the 'api-spam' and introduced captchas.... :(

[Alfagek]

Update:

I disabled last weekend the config in HA for Tgtg. And only did use the Tgtg app twice this week. And no captcha puzzle.

So today i did a new attempt: used “get token” and after entering mail adres, no 403 error 😱

then copie all the thing in to my yaml file. And adjust the pull time to 300 seconds. Made the yaml file active again. And restarted HA, no errors 👌🏻. So that did look good.

After about 15 minutes the first packs did come visible in HA. And that was about 16:00 my time. It’s now 21:00 and it’s still running without a 403 error.

So it’s looks like it’s going the right way!
fingers crossed

[Alfagek]

It was working great with a 5 minute pull.

But changed the pull this afternoon to 1 minute and I do get the 403 back again.

So I’m disable the sensor again and will wait 2 days en place it back with a pull of 5 minutes. Better 5 minutes then a not working addon ;-)

[Strauman]

It was working great with a 5 minute pull.

But changed the pull this afternoon to 1 minute and I do get the 403 back again.

So I’m disable the sensor again and will wait 2 days en place it back with a pull of 5 minutes. Better 5 minutes then a not working addon ;-)

Did it work?

[Alfagek]

It was working great with a 5 minute pull.
But changed the pull this afternoon to 1 minute and I do get the 403 back again.
So I’m disable the sensor again and will wait 2 days en place it back with a pull of 5 minutes. Better 5 minutes then a not working addon ;-)

Did it work?

Sorry for the late reaction (holiday). But it did work. It’s running now for a long period and have it set to 180 seconds.

[Strauman]

Worked for me too. Anything under 300 sec triggers captcha for me.

[Strauman]

A other strange thing i did mention in HA with TGTG. If i go to terminal and i give the command to get tokens. I do get the next error: ➜ tgtg python3 tgtg_get_tokens.py Traceback (most recent call last): File "/config/tgtg/tgtg_get_tokens.py", line 4, in <module> from tgtg import TgtgClient ModuleNotFoundError: No module named 'tgtg'

When i "reinstall tgtg" in terminal, its working again. I also did had this a few days ago. I can't remeber if this was also before i did have alle the "tgtg errors". Working now with 2023.6.1

I suspect that could because the folder name Is the same name as the module. So if you rename /config/tgtg/ to /config/toogood/ it might fix it :)

[lacrima08]

Hi,

Got errors too, ask me for captcha

Last logged: 18:23:56

Update for sensor.tgtg_xxxxxxx fails
Update for sensor.tgtg_xxxxxxx fails
Update for sensor.tgtg_xxxxxxx fails
Update for sensor.tgtg_xxxxxxx fails
Update for sensor.tgtg_xxxxxxx fails
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 699, in async_update_ha_state
await self.async_device_update()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 940, in async_device_update
await hass.async_add_executor_job(self.update)
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/tgtg/sensor.py", line 189, in update
self.tgtg_answer = tgtg_client.get_item(item_id=self.item_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/tgtg/init.py", line 287, in get_item
raise TgtgAPIError(response.status_code, response.content)
tgtg.exceptions.TgtgAPIError: (403, b'{"url":"https://geo.captcha-delivery.com/captcha/?initialCid=AHrlqAAAAAMAA8S4ze7nR8MAUkJApQ==&cid=3WSwSWOHo1arVZ4K2MMPeEx30T6XGSFfkhn2OIRLuSiDZxN3uuT_XjxxrdPlQ4-991rvdBfYujeW2le8ACflyKfK7V5PlSMz0OSsClBGlmWCYiRcCwlI1hpF1EnBzCWg&referer=https%3A%2F%2Fapptoogoodtogo.com%2Fapi%2Fitem%2Fv8%2F50108&hash=1D42C2CA6131C526E09F294FE96F94&t=fe&s=35558&e=339dd2286a20985f6ff651b1be55fe65ce034c76b4b385c98360aaadfec8be14"}')

And the python script on windows got the same :

tgtg_get_favorites_item_id.py
Type your email linked to your TGTG account: [email protected]
Using version 23.9.2
Traceback (most recent call last):
File "C:\Users\USER\Downloads\tgtg_get_favorites_item_id.py", line 6, in
from tgtg_get_tokens import tgtgClient
File "C:\Users\USER\Downloads\tgtg_get_tokens.py", line 13, in
tgtgClient.get_credentials()
File "C:\Users\USER\AppData\Local\Programs\Python\Python312-32\Lib\site-packages\tgtg_init_.py", line 87, in get_credentials
self.login()
File "C:\Users\USER\AppData\Local\Programs\Python\Python312-32\Lib\site-packages\tgtg_init_.py", line 178, in login
raise TgtgLoginError(response.status_code, response.content)
tgtg.exceptions.TgtgLoginError: (403, b'{"url":"https://geo.captcha-delivery.com/captcha/?initialCid=AHrlqAAAAAMAB280dAYzuyIAUkJApQ==&cid=5HUfPzsUXlqS4paE4bcK53HyQjRTZOQCIhIul7vZMcbdfSwjsGUVW713pgMkFoi921cMSYG21I9~C~br7R4slRAOl-29yj3lXm3OB13mekRy6idDoQ25QorVoXVcA3k_&referer=https%3A%2F%2Fapptoogoodtogo.com%2Fapi%2Fauth%2Fv3%2FauthByEmail&hash=1D42C2CA6131C526E09F294FE96F94&t=fe&s=35587&e=c6746ea4e22ebaa54df9da5b07063ef584a71c03b1dd9de6101c3b1cebdf3b0f"}')

[tjorim]

We can't do anything about the captcha but it's probably related to polling too often (scan_interval).
The default we currently recommend is 900 seconds.
I'm also not sure if it is limited by the total calls per day or too many in a short period.

Some users reported similar issues in the upstream library we use: ahivert/tgtg-python#250.
I pushed the change here #84 and published version 5.3.0: https://github.com/Chouffy/home_assistant_tgtg/releases/tag/5.3.0.
I'll close this issue for now as we don't have any control over it.