Comments (19)
I've experienced these missing tiles myself, but your feedback motivated me to investigate this deeper. I couldn't reproduce it reliably with my setup, but I think I found the reason. It seems to be caused by the connection keep-alive handling of the integerated web server (hyperium/hyper#368). I therefore increased the default number of threads from 4 to 8 and decreased the the keep-alive timeout from 75s to 5s.
Are you able to build the current master branch for testing, or shall I make a binary available?
from t-rex.
Thanks for the response!
Running the server on master yields huge improvements!
OpenLayers:
This is much better :) I don't want to pivot this issue too much, but Mapbox is giving some very strange results:
Mapbox:
(ignore the blue line, I added that manually). As you can see, there are a lot of line artifacts displayed as well. These don't exist in OpenLayers. I know that #18 seems pretty similar, in which you point out invalid geometries. Given that OpenLayers seems to be perfect, and that this is vanilla OSM data, would you have any ideas as to what is causing that?
Thanks for the help!
from t-rex.
I've imported a little osm country file from geofabrik with osm2psql (0.82) but couldn't reproduce these artefacts. Could you point me to your data - e.g. a metro extract?
Another difference could be the PostGIS version in use (SELECT postgis_full_version()
). I'm testing with PostGIS 2.2.2.
from t-rex.
Data is sourced from http://download.geofabrik.de/north-america/canada/ontario.html . I downloaded the PBF, used Osmosis to parse out a subsection with the bounding box tool
osmosis --read-pbf ./ontario-latest.osm.pbf --bounding-box top=43.753963 bottom=43.561912 left=-79.632868 right=-79.194903 --write-xml toronto.osm
then loaded the data into Postgres (fresh Postgres 9.6 with Postgis 2.3.2)
osm2pgsql -s -C 1024 --proj 3857 -d osm2 -j ~/Downloads/toronto.osm.pbf -H localhost
I'm wondering if Postgis 2.3.2 vs 2.2.2 is causing a difference, although I'd be surprised if that was the issue.
from t-rex.
I'm not exactly sure what I did, but I am no longer getting the artifacts. I believe that the issue was around projections, and converting between them though. I can try to pinpoint what the issue is if you would like.
Unfortunately, I'm experiencing the missing/incomplete tile issue which I opened this ticket for initially :( I am running off of master, with 8 threads and a 5sec keep-alive timeout and am still seeing significant loss. I've tried tweaking these values, but to no avail.
Do you have any thoughts/recommendations on how to resolve this?
from t-rex.
It looks like Hyper 0.11.0 was just released today, and includes asynchronous IO. I believe (from reading other issues similar to this) that this may help solve this problem.
Any chance on getting t-rex updated to use Hyper 0.11.0? :) I'd love to help with a PR, but my Rust is lacking.
from t-rex.
Missing tiles:
Will turn off keep-alive for the next release and migrate to Hyper 0.11.0 as soon as Nickel migrates.
from t-rex.
Artifacts in MB viewer:
In the meantime I saw the artifacts twice with my own datasets, but couldn't look deeper into it. At least in one case, clipping was apparently turned off, so big geometries may have caused the problem. I saw a message in the browser console saying this. Could you have an eye on the console in the browser next time you see this?
from t-rex.
Hi guys,
I also have the same issue, rendering with OL. I used osmosis and osm2pgsql to import the dataset. The console displays "references" warning from ol.js, but no hard error. Yesterday, I tried with a manual build of t-rex, but the issue is still there.
I am quite new with all this, but if I can be of any help, let me know.
from t-rex.
Hi Frederic,
The keep-alive fix has been been commited this morning (European time). Can you try again with 9deaf62 or newer?
MB artifact fix is following in a few minutes.
Pirmin
from t-rex.
Hi @pka . I tested with the latest patch, with keep-alive disabled, and confirmed that it was disabled in the logging. I still am experiencing the same issue though.
I've noticed that it's the tiles with the most densely populated geometries, and at a high zoom level that this occurs. Furthermore, the data seems to be fine in the bottom left corner, and then is blank at the top right hand corner of the tile. Could there be some memory allocation issue here? Perhaps the vector or other data structure that the data is being pushed to is filling up and maxing out? Just a thought.
from t-rex.
For fixing MB viewer artifacts see #18 (comment)
Don't forget to clear the browser cache before testing again!
from t-rex.
@kiambogo ok, problem number 3 in this issue... Looks like you are hit by the feature limitation (1000 per tile). You can increase or comment query_limit
in the config file or use the serve command with --simplify=false
which also turns off feature limiting. Don't forget to clear the browser cache before testing again!
from t-rex.
Hi @pka ,
Unfortunately for me, it did not help, even with the --simplify=false. I am starting to think that perhaps it comes from the importation, if you do not have the issue on your end.
from t-rex.
@pka That was my suspicion, but I have tried setting the query_limit
to 10000 and I get the same results. I verified that there is no issue with the config by removing the LIMIT
from the SQL in postgis.rs
as well.
I have disabled browser cache.
from t-rex.
I'm now also testing an osm2pgsql import of Toronto and get similar effects mostly on levels < 12. The import looks complete (tested with QGIS). Using the inspector viewer I see that most tiles with gaps have often nearly 5000 features, even without query_limit. No explanation for that - will take a sleep first...
from t-rex.
I was able to confirm that the issue is due to the limit in the SQL. I exported the SQL statements run against the DB, created them into a view in the db and ran it with QGIS to check if there were issues prior to encoding. It turns out that, with the map data I have been using, there is not much difference between using a limit of 1000 vs a limit of 10,000. When I adjusted the limit to 1mil I was able to get a complete map rendering; similarly, excluding query_limit
yields a complete map.
from t-rex.
It is indeed because of the query_limit
, though now even though I have a decent computer, t-rex takes most of the CPU and has problem serving the maps. Thanks @kiambogo for pointing to the error.
from t-rex.
2f0c053 is now emitting an info message when features are limited by query_limit
. Thanks a lot for your input!
from t-rex.
Related Issues (20)
- startup take too long time with many postgis tables HOT 1
- Pixel size for sampling tolerance in WGS84 HOT 1
- gpkg / gdal parallel performance less than sequential HOT 6
- How to configure t-rex to use a database function with PostGIS ST_AsMVT HOT 2
- corrupted polygons with clipping tile borders
- jammy packaging for 0.14x HOT 9
- --cache command line option ignored HOT 2
- Unable to generate tiles, tile extents appear to be wrong ... HOT 1
- t-rex vs other MVT servers? HOT 1
- gdal 3.5 support HOT 1
- Error in loading vector tiles with standalone mapbox gl js viewer HOT 1
- What default tile size is used by Trex? 256 or 512? HOT 1
- Polygon Shift: Custom Projection HOT 2
- cargo build error for GDAL HOT 6
- Parallel calls performance with geopackage as datasource HOT 1
- t-rex doesn't recognize the CRS/SRS HOT 1
- Debian packages for bookworm HOT 3
- Invalid MBTiles metadata.json HOT 4
- Buffer region not served as part of the vector tile HOT 2
- T-Rex doesn't handle invalid geometry results correctly when transforming between coordinate systems HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from t-rex.