Comments (17)
There are two limits:
- "single page" limit (number of views of a single page/single OKAPI endpoint calls per second): 150/s
- "entire site" limit: 200 views/s
The first limiter did not work properly and took the default value (5) instead of the one I set.
from opencaching-pl.
It isn't OC PL code related problem, but site/user related.
from opencaching-pl.
@deg-pl I'm so surprised by your answer from above, and also I just read the Facebook discussion with @pawelko-g about this issue. This is so technically incorrect from so many points of view that I'm shocked you're still pushing for this approach.
The rate limits you introduced to the OC server are an undocumented breaking change for OKAPI. @pawelko-g was using c:geo before your changes, but now you blame him for making too much traffic to the OCPL server.
He is just an end-user of the opencaching site, and he sends his requests via the C: Geo app that he downloaded from Google Play. Why are you expecting the end user to stick to those limits just by using the app?
The 'actor' responsible for the violation of those rate limits is not pawelko, but the application (in this ticket it is c:geo, but there are a lot of others app affected by your change now). If you @deg-pl want to still keep these rate limits, you need to at least communicate about them to all OKAPI developers and at least add info about them to OKAPI documentation. However, the effort that needs to be made on the apps developer side to make possible use of opencaching data is so extreamlly huge (moving calling okapi to separate threads, some sleeps to avoid your limits etc), that I believe they need some time before those rate limits will be introduced. For now, you should do the rollback of those changes.
My proposal to mitigate the issue:
- Exclude OKAPI endpoints from those rate-limiting (long term solution)
Short-term solutions if you still want to keep those ridiculous limits:
- Add documentation for OKAPI description of your breaking change.
- Communicate about the changes to developers (send e-mail to all OKAPI consumers)
- Create a new endpoint to OKAPI or add some more info to the existing one (i.e https://opencaching.pl/okapi/services/apisrv/stats.html) to allow developers to get the allowed number of requests. Maybe @wrygiel has an idea?
- Exclude our biggest customers from those limits for now, like c:geo, to give them time to apply needed code changes.
As long as OCPL end users are banned due to those limits, this is a high-priority issue caused by a misconfiguration of the OCPL server.
P.S: my other personal applications that using OKAPI are also broken now :(
P.S2, EDIT: How small those limits are that c:geo is not able even to download 8 geocaches?
from opencaching-pl.
There is reproduction from my side
Now you will take responsibility for your changes?
from opencaching-pl.
A quick test:
I'm using the map to download caches in my area.
I have 200 caches to download offline, in my case it looked like this:
- 48 and failure in c:geo
When downloading, I can select the option in c:geo to do nothing with the downloaded caches and continue downloading more.
- 12 and failure
- 4 and failure
- 22 and failure
When refreshing 200 caches, I donβt have this option, so it always ends in an error.
I believe that at least 200 caches should be downloadable without a limit. When traveling to a new area for events with new geopaths, one typically needs to download around this number of caches.
The last event I attended was on 17.05.2024, and I didn't have any problems with downloading caches at that time.
from opencaching-pl.
@pawelko-g is the only user I know who has such a problem with c:geo.
If the problem is actually related to limiting the number of queries, it is still not a problem related to the OC PL code.
Added later:
The current limit is 200/second and this causes a soft ban of only 2 seconds
The limit is the result of a DOS attacks on the server and has been in effect for many months.
from opencaching-pl.
@pawelko-g is the only user who reported this issue. The other users don't know what to do, and blame c:geo for hanging the update procedure.
from opencaching-pl.
@pawelko-g started a discussion on Facebook in the OC group, but no one else confirmed the problem.
from opencaching-pl.
@mzylowski Please provide steps to reproduce the problem
from opencaching-pl.
- Download GPX for GeoPath (50 caches will be enough for sure) -> https://opencaching.pl/powerTrail.php?ptAction=showSerie&ptrail=912
- Import downloaded GPX to c:geo
- Refresh list of geocaches
'Odwiez wszystkie' menu option
from opencaching-pl.
Thanks @mzylowski.
Now I can reproduce problem. This makes it much easier to find a solution.
Server logs indicate that, in addition to OKAPI queries, images are being downloaded to caches. The logs also indicate that the real limits are lower than those I configured (both the number of queries and the time unit in which it is tracked).
from opencaching-pl.
@pawelko-g @mzylowski @stefopl Could you please check again?
Now the limiter seems to behave in accordance with the assumed configuration.
from opencaching-pl.
@deg-pl
I intentionally chose more. Download completed successfully 264/264.
Thanks
from opencaching-pl.
After your intervention I was able to successfully refresh entire 'Rowerowe Pomorze' geopath (the biggest GP in our service)
Thanks!
How the limits are set currently?
from opencaching-pl.
@deg-pl @mzylowski @stefopl Thank you for your help. It also works fine for me now.
from opencaching-pl.
There are two limits:
* "single page" limit (number of views of a single page/single OKAPI endpoint calls per second): 150/s * "entire site" limit: 200 views/s
The first limiter did not work properly and took the default value (5) instead of the one I set.
Where/how are these limits configurable per OC site?
from opencaching-pl.
@andrixnet These limits are not OC PL code related. We use "evasive" as additional Apache2 module.
from opencaching-pl.
Related Issues (20)
- PHP templating system changes - discussion HOT 18
- Provide up-to-date information about PHP and MySQL versions on official nodes HOT 4
- The regular user sees the option to edit the GeoPath description.
- Dynamic properties will be deprecated in PHP8.2 HOT 1
- Skrzynka moblina - geohotel - geokrety
- COG kopiowanie emaila usera HOT 2
- Add link to geopath in powerTrailCOG.php
- Add trim to GeoPathName - add and edit
- New OC devel VM (v9) HOT 6
- Picure in a log entry can destroy opencaching layout HOT 1
- Some style inconsistencies (?) HOT 1
- Automatic expansion of logs on mobile
- Potencial security vulnerability with cache verification.
- Huge quantity of database queries related to geokrety
- Cache title does not support UTF8 emoji HOT 1
- Geokrety cron does not handle too old lastupdate
- Google chats in geopaths don't work HOT 1
- State Retention Issue for Collapsible Sections in viewprofile.php
- 403 Forbidden Error After Attempting to Log In on Cache and PowerTrail Pages HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opencaching-pl.