[BUG] v3.1.17 (3.2?) causes many timeouts for web-requests/url-commands

[schlaubstar]

[BUG] v3.1.17 (3.2?) causes many timeouts for web-requests/url-commands

[fredlcore]

Thanks for this detailed bug report, this really helps. I can reproduce this problem to some extent, but after more than 20 or so calls with a 1 second delay inbetween.
Could you do two things please: Test this with release version 3.1 which was published a few days ago under "Releases" and also have a serial monitor running while you are doing this test. I would like to see what messages appear between the last successful call and the moment it fails.
Thanks!

[fredlcore]

I have somewhat narrowed down the problem, but I can't explain it:
When doing the testing mentioned before, could you change the client.writeline in print2webclient.h's flushToWebClient function to this:

printlnToDebug(PSTR("1"));
    client.write(bigBuff, bigBuffPos);
printlnToDebug(PSTR("2"));

It seems to be that sometimes there is an (unexplicable to me) long delay between the output of 1 and 2, apparently resulting in the timeout you mention. But since this is a general library call, I don't see where I can change that behaviour...

[schlaubstar]

I was just testing so feedback to your initial question:

• Problem is also there for official 3.1. release 3.1.17-20230520012002

noMonitoring_BSBCap.txt

I will modify the lines proposed and will check the results

[fredlcore]

Also, are you using WiFi or Ethernet? If you have a possibility to switch (temporarily) to Ethernet, this might also be a helpful indication.
However, if you say that it did work rock-solid with a previous version, the only solution I can see right now is to go back in steps and install the software based on each previous pull request and see where the error takes effect. I'm willing to do all the changes necessary to fix this, but since I'm not on site right now, I have no means to do this process.

[schlaubstar]

Maybe your "output 1/2 fix" already gives some indications.
Although I did not dig into the code to deep but to me it looks as if the corresponding function sometimes gets called subsequently and "to often"

newlog.txt

The ESP32 has no wired connectivity, so I have to rely on Wifi.

[fredlcore]

No, that the frequent calls are correct because the output is often times too large to fit in the buffer. You can see the contents to be sent if you change the code above to:

printlnToDebug(PSTR("1"));
Serial.write(bigBuff, bigBuffPos);
    client.write(bigBuff, bigBuffPos);
printlnToDebug(PSTR("\r\n2"));

I was a bit suspicious that it could be WiFi-related, but if you say that it worked rock-solid with 3.1.15, then that cannot be it...

[schlaubstar]

It is, for sure, not Wifi related - because I just need to go back to : Version 3.1.15-20230402082920 via OTA without touching anything, and I get 100% reply-rate.

I found out how to retrieve the code for a specific commit, so I am slowly testing the older versions to identify from which to which the issue occurs.

[fredlcore]

You go to this page:
https://github.com/fredlcore/BSB-LAN/commits/master
And then go to the commit you want to test and click on the <>. Then the repository opens with the code at that time and you can download it as usual via the green download button.

But before you do this you might want to try this:
I erased the EEPROM using /NE and then flashed anew using the default BSB_LAN_config.h just modified for the networking parameters. Now I don't have any delays or timeouts anymore. If that is the same with you, it might be easier to isolate the causing factor by re-activating the configuration options from before and see where the problem starts to occur. Of course take note of all your web-configuration settings before erasing the EEPROM :)!

[fredlcore]

I also realized that there is still a bug in the preferred BSSID functionality. If the BSSID parameter is empty, it will lead to a reconnect every 60 seconds. I'll fix this later, but you may want to look out for the serial monitor and see if that is a problem with you, too.

[fredlcore]

And have you disabled Energy Saving for the ESP32 in the web configuration? I just realized that I had it turned on and that also resulted in bad connection.

[fredlcore]

Enery saving seems to be the root cause of this. Try this first before anything else.

[schlaubstar]

ESP32 PowerSave was and is constantly de-activated but I am already on May 16th ;-)

[fredlcore]

Ok, too bad :(. But great that you're making such fast progress :)...

[schlaubstar]

Some insights but no 100% explanation, yet.

• Enabling and using /NE did have a positive effect ... now the timeouts have reduced from ~10% to ~1% and I had to change my python script to wait less and perform more rounds
• I never achieved a timeout until commit: b87ea8d
• I also encounter the frequent "WiFi Re-Scans" you mentioned with Versions having this option for a fixed BSSID when left in default config.
• I will enter the correct BSSID and try out a later version. I can't fight the feeling that the Wifi-Scans might interfere with the URL-command

[schlaubstar]

Final insight: The problem is caused with the introduction of the ability to lock WiFi to a certain BSSID if you do not really use it and provide the MAC address. As you observed this causes frequent "disconnects" where the ESP32 simply seems to loose connectivity for a short period of time, leaving existing requests "unhandled".

After compiling the latest commit (960c326) but providing the MAC Address of my 2.4GHz Wifi ... I got 1000 requests with 1 sec. delay without any timeout or loss of connectivity and the Debug output also does not show the Wifi Scans anymore.

From my view you can close this issue, but I think it would make sense to reasonably deal with a "default BSSID"

Thanks for your support to track this issue down. 👍

[fredlcore]

Ok, thanks, the default BSSID issue is fixed in the most recent master repo.