mapserver/mapcache

MapCache source code directory. Please submit pull requests to the 'main' branch.

MapServer

---

Summary

MapServer is a system for developing web-based GIS applications. The basic system consists of a CGI program that can be configured to respond to a variety of spatial requests like making maps, scalebars, and point, area and feature queries. Virtually all aspects of an application, from web interface to map appearance can be developed without any programming. For the more ambitious user, MapServer applications can be enhanced using Python, PHP, Java, JavaScript or many other web technologies. For more information and complete documentation please visit:

https://mapserver.org/

Bug reports and enhancement submissions can be reported in the MapServer issue tracker at the following url. If you do make changes and/or enhancements, please let us know so that they might be incorporated into future releases.

https://github.com/MapServer/MapServer/issues

Join the MapServer user mailing list online at:

https://mapserver.org/community/lists.html

Credits

MapServer was originally written by Stephen Lime. Major funding for development of MapServer has been provided by NASA through cooperative argreements with the University of Minnesota, Department of Forest Resources.

PHP/MapScript developed by DM Solutions Group.

GDAL/OGR support and significant WMS support provided by DM Solutions Group which received funding support from Canadian Government's GeoConnections Program and the Canadian Forest Service.

Raster support developed by Pete Olson of the State of Minnesota, Land Management Information Center, and maintained by Frank Warmerdam (DM Solutions).

PostGIS spatial database support provided by Dave Blasby of Refractions Research.

PDF support developed by Jeff Spielberg and Jamie Wall of Market Insite Group, Inc.

OracleSpatial support developed by Rodrigo Cabral of CTTMAR/UNIVALI, Brazil.

Portions Copyright (c) 1998 State of Minnesota, Land Management Information Center.

Portions derived from Shapelib, Copyright 1995-1999 Frank Warmerdam.

Supporting packages are covered by their own copyrights.

License

See LICENSE.md

Security policy

See SECURITY.md

How to Contribute

See CONTRIBUTING.md

Documentation Repository

Use the separate MapServer-documentation repository for filing documentation tickets and changes.

mapcache's Issues

Allow for fixed date/time expires settings in mapcache

Reporter: rdewit
Date: 2011/10/10 - 08:38
Currently and <auto_expire> are set in seconds after tile creation.

Sometimes it can be useful to set the expires value to a fixed point in time, for example when creating caches of weather forecasts that renew every hour on the hour.

Note: Ideally it would be possible to use a cron-like syntax for automatically updating the and <auto_expire> values, but that is out of scope for this ticket.

Incomplete mapcache seed when not using metatiles

Reporter: mwalker
Date: 2011/11/16 - 10:29
When running mapcache_seed on a tileset without metatiles, only very few tiles are created and only on some zoom levels.

Example configuration (mapcache_seed.xml):

<?xml version="1.0" encoding="UTF-8"?>
<mapcache>
  <cache name="disk" type="disk">
    <base>/tmp/mapcache/tiles_debug</base>
    <symlink_blank/>
  </cache>

  <source name="debug" type="wms">
    <getmap>
        <params>
          <FORMAT>image/png</FORMAT>
          <LAYERS>basic</LAYERS>
          <TRANSPARENT>false</TRANSPARENT>
        </params>
    </getmap>
    <http>
      <url>http://vmap0.tiles.osgeo.org/wms/vmap0</url>
      <connection_timeout>30</connection_timeout>
    </http>
  </source>

  <tileset name="debug_meta">
    <source>debug</source>
    <cache>disk</cache>
    <format>PNG</format>
    <grid>WGS84</grid>
    <metatile>4 4</metatile>
    <metabuffer>0</metabuffer>
  </tileset>

  <tileset name="debug_nometa">
    <source>debug</source>
    <cache>disk</cache>
    <format>PNG</format>
    <grid>WGS84</grid>
  </tileset>

  <service type="wmts" enabled="true"/>
  <service type="tms" enabled="true"/>
  <service type="wms" enabled="true"/>
  <service type="kml" enabled="true"/>
  <service type="gmaps" enabled="true"/>
  <service type="ve" enabled="true"/>

  <default_format>JPEG</default_format>

  <errors>report</errors>

  <lock_dir>/tmp</lock_dir>
</mapcache>

Seeding with metatiles generates all tiles on all levels:

mapcache_seed -c mapcache_seed.xml -t debug_meta -g WGS84 -e 0,40,10,50 -z 0,5

Seeding without metatiles generates only a single tile on the first level ( ''debug_nometa/WGS84/00/000/000/001/000/000/000.png'' ):

mapcache_seed -c mapcache_seed.xml -t debug_nometa -g WGS84 -e 0,40,10,50 -z 0,5

MSVC10 compile failure

I've tried to compile MapCache with Visual Studio 2010 on a Windows 7 x64 box, as described in the package document.

Issued nmake /f Makefile.vc, then I've got the below error:
...........
........
lib\cache_disk.c(584) : error C2143: syntax error : missing ';' before 'type'
lib\cache_disk.c(594) : error C2065: 'retry_count_create_file' : undeclared identifier
lib\cache_disk.c(596) : error C2065: 'retry_count_create_file' : undeclared identifier
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\cl.EXE"' : return code '0x2'
Stop.

After checked the cache_disk.c, line 584. I've moved the line to top of the function, and then compile passes.

MapCache GetFeatureInfo support ???

Has MapCache support for GetFeatureInfo??

I'm getting an OperationNotSupported exception from my mapcache server but documentation (http://mapserver.org/mapcache/index.html) says it is supported (forwarding requests):

responds to WMS/WMTS GetFeatureInfo requests (forwarded to source service)

Thanks

Support OPTIONS requests

Utilising Access-Control-Allow headers to enable cross domain requests is failing because Mapcache sends HTTP_METHOD_NOT_ALLOWED for all requests that aren't GET.

As far as I can see it's just line 262 of mod_mapcache_request_handler() in mod_mapcache.c that needs to change

I'd offer to fix it, but I'm not sure where to start with pull requests and whatnot in Github. Is there a guide?

Ref: https://developer.mozilla.org/en/http_access_control#Preflighted_requests

tbonfort commented:
If I understand correctly, this isn't just a question of mapcache not returning an error in case an OPTION is sent. We'd also have to return a response with the correct Access-Control-Allow-* headers, correct?

KeithMoss commented:
For Access-Control-Allow to actually work with clients, yes, but I'd be expecting devs to be setting the correct headers elsewhere (i.e. in Apache).

Support for "Namespace" in Memcached tilesets?

I have a weather application, and I am generating tiled images on the fly, out of PostGIS 2.0 database rasters. I expect to serve a lot of these tiles, so Memcached is my preferred back end caching solution.

The trouble: It is not always simple to tell when a tile has expired. For instance... sometimes, a forecast for 7 days out will be unchanged, but the forecast for tomorrow is new. It would be silly to expire the images for both forecasts, when only Tomorrow is changed.

My preferred solution would be to implement a Memcache "Namespace" solution.

http://code.google.com/p/memcached/wiki/NewProgrammingTricks#Deleting_By_Namespace

The idea is to store a second key->value pair in Memcache, to use as the "namespace" for each unique tileset.

Example...
Let's say I have a High Temperature forecast which is valid for April 1, 2013. I can reference this in a High Temperature Tileset with the dimension:

<tileset name="HighTemperature">
<dimensions>
<dimension>validtime</dimension> <!-- YYYYMMDDHH -->
</dimensions>
</tileset>

This dimension is part of the Memcache key that contains the map tiles.
/HighTemperature/GoogleMapsCompatible/2013040100/{zoom}/{tileX}/{tileY}/png

I propose creating a new namespace element in the tileset.

<tileset name="HighTemperature">
<dimensions>
<dimension>validtime</dimension> <!-- YYYYMMDDHH -->
</dimensions>

<namespace>
<dimension>validtime</dimension> <!-- YYYYMMDDHH -->
</namespace>
</tileset>

The namespace dimensions would be used to construct a key name that contains the namespace value. The key -> value would be something like...

/{tileset_name}/{namespace_dimensions}/namespace -> 1

example: /HighTemperature/2013040100/namespace -> 1

To fetch the actual tile, we would first have to fetch the namespace value (1), and use that namespace value as part of the tile's key.

/HighTemperature/{namespace_value}/GoogleMapsCompatible/2013040100/4/10/4/png
/HighTemperature/1/GoogleMapsCompatible/2013040100/4/10/4/png

Every time I update the source WMS, I would increment this namespace value directly in Memcache.

example: I update the Apr 1 2013 High Temperature forecast...
/HighTemperature/2013040100/namespace -> 2

The next request for high temperature Apr 1 2013 would search for tiles with Namespace 2.

/HighTemperature/2/GoogleMapsCompatible/2013040100/4/10/4/png

Since tiles for HighTemperature, Apr 1 2013, Namespace 2 do not exist, Mapcache will fetch new tiles out of the source WMS.

Tiles with Namespace 1 would no longer be requested. Eventually, they fall off the end of Memcache's LRU queue.

The effect is, I can tell a Memcached Mapcache exactly when a subset of tiles in a tileset is expired. There is no latency between new data and generating a new tile... and there is no redrawing of maps that have not changed.

Thoughts on this?

libpng / IDAT CRC error when seeding tiles (CAIRO/PNG Driver)

Reporter: theram
Date: 2012/02/16 - 22:43
While using mapcache_seed to seed tiles from Mapserver, using CAIRO/PNG driver, mapcache_seed fails due to possible corrupt image
returned from Mapserver.

Error:

libpng warning: Ignoring bad adaptive filter type
libpng warning: Extra compressed data.
libpng warning: Extra compression data.
libpng error: IDAT: CRC error
failed to setjmp(png_jmpbuf(png_ptr))
failed to load image data from metatile
failed to setjmp(png_jmpbuf(png_ptr))
failed to load image data from metatile
failed to setjmp(png_jmpbuf(png_ptr))
seeding tile 50 30 7

Switching Mapserver OUTPUTFORMAT to use AGG/PNG driver does not encounter the error.

Mapfile & mapcache.xml file attached.

mapcache_seed params that reproduces the error:

mapcache_seed -c /tmp/tb/png_error_mapcache.xml -t 2012-02-02_text_cache_nztm -g portalgridnztm1 -z 6,7

(reproducable at these zoom levels, error occurs at same tile each time: seeding tile 50 30 7)

Fonts & data sent to Thomas directly

Mapcache seed does not delete or overwrite tiles in sqlite cache

Reporter: mwalker
Date: 2011/11/18 - 17:41
The sqlite cache does not change when using ''mapcache_seed'' with the options ''-o now'' to force an update or ''-m delete'' to remove tiles.

The same options work correctly when seeding a disk cache.

Tile creation and symlink creation failure in cache_disk.c on nfs

Reporter: tomhel
Date: 2012/03/02 - 16:16
When seeding with more than one thread we get intermittent errors that seeding has failed because the directory where the tile is to be stored does not exist.

This error only happens when seeding to nfs mounted network storage. We do not get these errors when seeding to local disk.

Numerous mounting parameters have been tried, but no success.

What we did to solve this was to add a check to cache_disk.c, so it checks if a directory exists before writing a tile or symlink. If the directory does not exist, it will try to create it before writing the tile. We have it set to retry this 10 times before aborting. Our tests have shown that one retry might be sufficient.

The following patch adds the following functionality to cache_disk.c:

• if writing a tile fails, it will try to create the destination directory and retry. If the tile still fail to write. we try 10 more times before aborting.
• Same goes for symlink creation.

mapcache_seed.c is missing include of mapcache-util-config.h

Since the include is missing it doesn't enable OGR and GEOS support even though they are found during the cmake run.

Adding the #include "mapcache-util-config.h" fixes the problem.

Return empty images with correct image mimetype

Reporter: jtaranufbs
Date: 2011/11/30 - 16:19
It is possible to configure MapCache to output blank image tiles when handling a request that does not exist in the source cache. However, these image tiles are returned with the mimetype "text/plain" which is incorrect. They should be returned with the appropriate mimetype - if PNG, with mimetype "image/PNG" and if JPEG with mimetype "image/JPEG".

Mapcache fastcgi doesn't return correct Content-Length

Reporter: dmartella
Date: 2012/03/21 - 18:12
Running mapcache as fastcgi (on nginx) don't work. This bug is due to incorrect integer modifier in a printf call

Extent problem with WMS and ArcGIS 10.0

There is a problem to open a WMS in ArcGIS 10.0. It says something like "extent is not consistent with the associated spatial reference information". It seems that ArcGIS can't read any extent information because no value shows up.

http://geopole.org/checks shows some errors if I check such a WMS (for example missing LatLonBoundingBox).

WMS 1.1.1 specification says:

7.1.4.5.6 LatLonBoundingBox
Every Layer shall have exactly one "LatLonBoundingBox" element

mapcache kml superoverlay error

Reporter: tbonfort
Date: 2012/02/20 - 10:41
adapt kml service to return a superoverlay for the whole globe (-180,-180,180,180)

Having trouble building mapcache against nginx

I tried nginx-1.1.19 (from mapserver guide) and mapcache master and I realized that /mapcache/nginx/config doesn't exist, but config.in does.

adding module in /usr/local/src/mapcache/nginx
./configure: error: no /usr/local/src/mapcache/nginx/config was found

Can someone explain why this is done this way? I have to change all instances of files with .in in order for ./configure to recognize the config file. Still fails after I change the file name.

make -f objs/Makefile
make[1]: Entering directory `/usr/local/src/nginx-1.1.19'
gcc -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Wunused-function -Wunused-variable -Wunused-value -Werror -g  @APR_CPPFLAGS@ -I src/core -I src/event -I src/event/modules -I src/os/unix -I @PROJECT_BINARY_DIR@/include -I @APR_INCLUDE_DIR@ -I objs \
        -o objs/src/core/nginx.o \
        src/core/nginx.c
gcc: error: @APR_CPPFLAGS@: No such file or directory
make[1]: *** [objs/src/core/nginx.o] Error 1
make[1]: Leaving directory `/usr/local/src/nginx-1.1.19'
make: *** [build] Error 2

Incorrect HTTP code when empty_img is set in error tag in mapcache.xml

When i set empty_img in mapcache configuration file and when tile is not present in cache, mapcache return an error on request.
Normaly, I think the normal issue is send empty image and HTTP 200 OK ? If this is the normal way core.c must be patched to set code 200 before return tile.
Here is my patch to perform the correct return request for me:

 
--- core.c  2012-08-02 13:19:00.133930898 +0200
+++ /home/pla/Territor/SVN/trunk/mapcache-src/lib/./core.c  2012-06-18 16:51:46.272659544 +0200
@@ -553,6 +553,7 @@
       response->data->avail = response->data->size;
    } else if(ctx->config && ctx->config->reporting == MAPCACHE_REPORT_EMPTY_IMG) {
       response->data = ctx->config->empty_image;
-      response->code = 200;
     apr_table_set(response->headers, "Content-Type", ctx->config->default_image_format->mime_type);
     apr_table_set(response->headers, "X-Mapcache-Error", msg);
  } else if(ctx->config && ctx->config->reporting == MAPCACHE_REPORT_ERROR_IMG) {

Segfault when calling a WMS service with no VERSION parameter

Reporter: homme
Date: 2012/01/20 - 11:44
Calling a mapcache wms service based on the example 'mapcache.xml' file with the following query string results in a segmentation fault:

LAYERS=test&SERVICE=WMS&REQUEST=GetMap

mapcache_seed segmentation fault if cannot create output sqlite file

I had a typo in the dbfile path for an mbtiles cache, and mapcache_seed crashed with a segmentation fault. Would it be possible to provide a message saying that the output sqlite file couldn't be created?

mapcache install problem with libpng

I run on Ubuntu 13.4 (raring ringtail) 64 bit.

I have installed Mapserver 6.2.1 by the source.

I now want to install mapcache. I hvae libpng12, which is the latest version for Ubuntu.

./ configure:

checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @file support... @
checking for strip... strip
checking for ranlib... ranlib
checking for gawk... no
checking for mawk... mawk
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking for a BSD-compatible install... /usr/bin/install -c
checking for ANSI C header files... (cached) yes
checking for symlink... yes
checking for apxs2... /usr/bin/apxs2
Use of uninitialized value $includedir in concatenation (.) or string at (eval 9) line 1.
Use of uninitialized value $includedir in concatenation (.) or string at (eval 9) line 1.
checking for apachectl utility... /usr/sbin/apachectl
Use of uninitialized value $includedir in concatenation (.) or string at (eval 9) line 1.
Use of uninitialized value $includedir in concatenation (.) or string at (eval 9) line 1.
Use of uninitialized value $includedir in concatenation (.) or string at (eval 9) line 1.
Use of uninitialized value $includedir in concatenation (.) or string at (eval 9) line 1.
Use of uninitialized value $includedir in concatenation (.) or string at (eval 9) line 1.
checking whether to enable FASTCGI... not requested. use --with-fastcgi=dir to enable
checking for apr-config in default locations... using /usr/bin/apr-config, use --with-apr-config=/path/to/apr-(1-)config to modify
checking for apu-config in default locations... using /usr/bin/apu-config, use --with-apu-config=/path/to/apu-(1-)config to modify
checking for apr_memcache_hash in -laprutil-1... no
configure: WARNING: supplied apr-util does not contain memcache functions. memcache cache backend disabled
checking sqlite3.h usability... yes
checking sqlite3.h presence... yes
checking for sqlite3.h... yes
checking for sqlite3_open_v2 in -lsqlite3... yes
using libsqlite3 from system libs.
checking for pkg-config... /usr/bin/pkg-config
checking for Pixman... searching default locations
checking for pixman pkg-config entry... found
checking if pixman version is >= 0.13.2... 0.28.2
configure: checking where PNG is installed...
checking for png_init_io in -lpng... yes
checking png.h usability... yes
checking png.h presence... yes
checking for png.h... yes
using libpng from system libs.
checking setjmp.h usability... yes
checking setjmp.h presence... yes
checking for setjmp.h... yes
configure: checking where JPEG is installed...
checking for jpeg_read_header in -ljpeg... yes
checking jpeglib.h usability... yes
checking jpeglib.h presence... yes
checking for jpeglib.h... yes
using libjpeg from system libs.
configure: checking where TIFF is installed...
checking for TIFFReadScanline in -ltiff... no
checking tiffio.h usability... no
checking tiffio.h presence... no
checking for tiffio.h... no
configure: WARNING: cannot find tiff headers in system path
configure: WARNING: TIFF (libtiff) library cannot be found. install or reconfigure with --with-tiff=DIR
checking for pcre library inclusion... in system locations
checking for pcre_compile in -lpcre... yes
checking pcre.h usability... yes
checking pcre.h presence... yes
checking for pcre.h... yes
checking for gdal-config... no
configure: WARNING: geos support disabled as gdal/ogr support was disabled/not found
checking for geos-config... /usr/bin/geos-config
checking for GEOSPreparedIntersects in -lgeos_c... yes
checking for db.h... no
configure: no Berkeley DB version 4 or higher found
checking for curl-config... /usr/bin/curl-config
configure: creating ./config.status
config.status: creating Makefile.inc
config.status: creating nginx/config
config.status: creating lib/Makefile
config.status: creating apache/Makefile
config.status: creating util/Makefile
config.status: creating cgi/Makefile
config.status: executing libtool commands

make is where I encouner the problem. The erroneous part of the result is as follows:

...
libtool: link: gcc -o .libs/mapcache_seed -DUSE_PCRE -DUSE_GEOS -DUSE_SQLITE -DUSE_PIXMAN -D_REENTRANT -D_GNU_SOURCE -pthread -O2 -Wall -DHAVE_SYMLINK -DNDEBUG -I../include -I/usr/include/apr-1.0 -I/usr/include/apr-1.0 -I/usr/include -I/usr/include/pixman-1 -I/usr/include mapcache_seed.c ../lib/.libs/libmapcache.so -L/usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so -lpng -ljpeg /usr/lib/libapr-1.so /usr/lib/libaprutil-1.so -lpcre /usr/lib/x86_64-linux-gnu/libsqlite3.so -lpixman-1 -L/usr/lib -lgeos_c -pthread -Wl,-rpath -Wl,/usr/local/bin
../lib/.libs/libmapcache.so: undefined reference to png_set_longjmp_fn' collect2: error: ld returned 1 exit status make[1]: *** [mapcache_seed] Error 1 make[1]: Leaving directory.../mapcache-1.0.0/util'
make: *** [all] Error 2

Can anybody help?

Add PRAGMA support to mapcache.xml configuration

The sqlite db filesize is limited to 1Tb unless the PRAGMA MAX_PAGE_COUNT in increased from the default (see http://www.sqlite.org/pragma.html#pragma_max_page_count)

It would be good to support this via the cache configuration section of the mapcache xml.

Add mapnik as a mapcache source

It would be a great feature to be able to generate tiles from a built in Mapnik data source rather than having to run through an external WMS. This would involve linking directly against the mapnik library and updating the mapcache.xml format to support something along the lines of <source type="mapnik"> (which would point to a mapnik XML configuration file).

./configure does not work at all, ubuntu 64 12.04

1. git clone https://github.com/mapserver/mapcache
   result: ok
2. cd mapcache;
   result: ok
3. ./configure
   result: bash: ./configure: /bin/sh^M: bad interpreter: No such file or directory
4. run dos2unix or autoconf
   result: ok
5. ./configure
   result: configure: error: cannot run /bin/bash ./config.sub

Of course I have config.sub in work directory (mapcache), what should I do next?

singleTile option or Tile Assembling

I've configured a cached transparent layer in mapcache using a mapserver wms source and in the test link the tiled version is ok but when i use the single tile, mapcache lose the transparency.

It's simple to test juste declare a transparent layer in your config and check your demo at :
http://-your server-/mapcache/demo/wms

mapcache wms service does not limit image size requests

Reporter: tbonfort
Date: 2011/09/29 - 17:27
not limiting image size can cause malicious users to request large images, potentially starving server memory resources.

by default, limit wms image requests to 2048 pixel images, overridable with a tag for the wms service

mapcache_seed gives misleading error message for metatile size problems

mapcache_seed provides the following error message:
metatile size is not set to a power of two, rerun with e.g -m 8,8

according to the list of options provided below the error message,
-m|--mode [value]: mode: seed (default), delete or transfer
-M|--metasize [value]: override metatile size while seeding, eg 8,8

So the error message should be changed to:
metatile size is not set to a power of two, rerun with e.g -M 8,8

Permit forwarded WMS requests to be cached

I'm going to throw this out here and see what people think of the idea.

If MapCache forwards a request to a WMS service for a single tile (256x256 / 272x272) of a layer it would be great to be able to specify that that layer be stored in a cache. I'm envisioning something like this:

   <cache name="sqlite" type="sqlite3">
      <dbname_template>/ms4w/tmp/ms_tmp/mapcache/{layer}/{grid}.db</dbname_template>
      <hitstats>false</hitstats>
   </cache>

   <service type="wms" enabled="true">
       <forwarding_rule name="first rule">
        <cache>sqlite</cache>
        <grid>g</grid>
        <append_pathinfo>true</append_pathinfo>
        <http>
          <url>http://server/mapserver?</url>
        </http>
       </forwarding_rule>
   </service>

I see the grid as either being specified manually, as here, or there being some smarts behind it by examining the SRS provided and working out the grid to use from there.

For expiry it would be great if mapcache could examine the Cache-Control or Expires headers and use that to determine when it needs to bypass the cache and request a new copy.

Thoughts and revisions more than welcome.

GetMap requests to forward WMS return the wrong Content-Type

When a GetMap request is forwarded to a WMS server by mapcache the Content-Type being returned by mapcache is 'application/xml', not the Content-Type as returned by the WMS server. Other than that the response is fine, i.e. a perfectly valid PNG file.

mapcache 0.5-dev

configure mapcache problem

I used http://mapserver.org/mapcache/install.html to install mapcache on my Ubuntu raring ringtail (13.04) 64 bit machine.
I downloaded the tarbal from https://github.com/mapserver/mapcache/zipball/master and also cloned the GIT from git://github.com/mapserver/mapcache.git.
However, when I want to configure it, there is no configure file in the aforementioned sources. Therefore, it is not configured with this error: bash: ./configure: No such file or directory
Can anyone help?

ArcGIS Tiles support

Reporter: aboudreault
Date: 2012/03/14 - 16:01
It would be useful if mapcache could serve tiles generated by arcgis. The files are stored on disk using this template:

L01/R0000001f/C00000032.jpg

The numbers are simply the z/y/x values in hexadecimal with zero padding. Level begins with L, y with R and x with C.

Thomas, Do you agree with me that we could simply add 3 disk template variables for this? ie. {arcgis_z}, {arcgis_y} and {arcgis_x} ?

thread_mutex not created for MPM Prefork apache set up.

The code in mod_mapcache_child_init does a call to ap_mpm_query to see if the MPM supports threads before creating the thread_mutex. In the case of prefork MPM this returns No and so the thread_mutex isn't created. This can break the cache_bdb and cache_sqlite code as the creation of the reslists aren't locked yet threads can still be used inside of mapcache.

One simple solution is to remove the ap_mpm_query call but I was not sure what the reason was behind the use of it. Our current set up is Prefork with APR_THREADS enabled in mapcache.

Serving tiles (TMS) into ArcBrutile

Hi,

I was looking for a solution to consume tiles generated with mapcache (compiled from source last week) into ArcGIS Desktop (version 10).

ArcBruTile seem to be an interesting avenue since the TMS spec is supported:
http://arcbrutile.codeplex.com/

I tried to add the TMS example that come with mapcache but I received an error when I added the service into ArcBruTile (see attachment image)

The get capabilities seem ok: http:///mapcache/tms/1.0.0/
return:

<TileMapService version="1.0.0">
<TileMaps>
<TileMap title="no title set, add some in metadata" srs="EPSG:4326" profile="global-geodetic" href="http://<my server>/mapcache/tms/1.0.0/test@WGS84"/>
<TileMap title="no title set, add some in metadata" srs="EPSG:900913" profile="global-mercator" href="http://<my server>/mapcache/tms/1.0.0/test@g"/>
</TileMaps>
</TileMapService>

Any thoughts?

MartinOuellet

Ability to extract tiles from an existing cache to another

Reporter: aboudreault
Date: 2011/11/08 - 18:11
One of our needs is to generate tiles for a specific extent/region and make a packages or those tiles. This is what I would need that utility to do:

• Generated the tiles list of the region given (taking in account the zoom levels given, ogr intersects options etc.)
• If a tile already exists in the cache of the tileset given, it will be simply copied in the extract cache specified
• If a tile doesn't exists in the cache of the tileset give, render it then copy it in the extract cache specified
• If a tile already exists in the extract cache specified, do nothing.

My first idea was to create a new utilise called mapcache_extract. I then realized that mapcache_seed handle 80% of the needs and that it would be ugly to duplicate and maintain twice that code.

Here's what I propose:

• Add a command line option in mapcache_seed: -x, --extract <cache_name>

By specifying that command line option, mapcache_seed will render all needed tiles (if they don't already exists) and extract those tiles in the specified cache. The cache specified must have been added in the xml config. By implementing this this way, all cache formats will be supported directly.

Example of config and use:

<cache name="beauce-disk" type="disk">
      <template>/tmp/{tileset}/{grid}/{z}/{x}/{inv_y},{ext}</template>
      <symlink_blank/>
   </cache>

-----------------------------

mapcache_seed --config /opt/osm/mapcache/osm.xml --tileset osm -z 0,10 -e -71.34,45.9,-70.25,46.61 -x beauce-disk

Implementation details

The attached patch is a simple proof of concept that test the idea. It's not the final patch that I would commit in svn. In that patch, I hack the tile->tileset->cache to do the appropriate operations. The patch works well for disk cache (DEPTH_MODE only) but do not work with mbtiles. This is due to some initial operations that should be done in the post_configuration function but aren't since my cache isn't associated to any tileset.

In the final patch, If the extract option is specified, I would create a temporary tileset (clone of the specified tileset) with a different cache set (the extract cache) and add it cleanly in mapcache_configuration. So, things would be cleaner (no more hack with tile->tileset->cache) and all cache formats will work as expected.

Before finalizing the patch, tbonfort, what do you think of that feature and the approach. Do you think it's worth to add it in mapcache and agree with to fact to include that in mapcache_seed? Anything else to suggest?

No intersection check in delete or transfer mode

Reporter: tomhel
Date: 2012/03/02 - 15:44
mapcache_seed.c, function examine_tile(...) is missing a case when seeding with an ogr-datasource and mode is delete or transfer. In this case the whole cache will be deleted or transfered.

At the moment this can be worked around by always specifying the option '-o now' when deleteing/transfering tiles with an ogr-datasource.

The attached patch resolves this issue.

mapcache windows build

Reporter: assefa
Date: 2011/12/02 - 18:21
mapcache windows build

mapcache_seed should return a non-zero exit code after failure

Currently mapcache_seed returns zero if seeding was interrupted on error condition (WMS backend errors, failure to create tiles..). It would be beneficial to batch jobs etc, if these cases (i.e. error_detected > 0) were reported with a non-zero return value.

Also, at least in our use case (we use OGR sources to update the map tiles), if the source contains no features is not considered an error, rather than an indication that there's nothing to do:

if((nClippers=OGR_L_GetFeatureCount(layer, TRUE)) == 0) {
return usage(argv[0],"no features in provided ogr parameters, cannot continue");
}

Would it be better if this case was handled as a successful run?

Determine currency of cached WMS tiles based on response headers

This issue relates to 'Permit forwarded WMS requests to be cached' #34 (comment).

When mapcache caches the response of a WMS request it should use either max-age within the Cache-Control header (preferable), or the Expires header, to compute a timestamp to store in its cache. When a second request is made for the same tile the stored timestamp should be consulted first to determine if the request needs to be reissued to the service providing the WMS.

Owing to the need to store the expiry timestamp I am guessing this might have to be limited to the caches that use a database (Sqlite, BerkeleyDB, et al).

Integrate mod-geocache to MapServer

Reporter: tbonfort
Date: 2011/08/13 - 10:42
This Ticket is to keep track of the integration of mod-geocache into the mapserver suite, renamed as MapCache.

http://www.mapserver.org/development/rfc/ms-rfc-71.html

this is a test issue

testing commit integration

segfault when config XML doesn't exist

Reporter: ezequiel
Date: 2012/02/22 - 17:52
Greetings... I've just been trying mapcache from trunk (r13163) and found that, if the config file passed does not exist, it makes the server crash (more troublesome if what crashes is the apache module on a restart!).

with gdb, using environment variable

`````` MAPCACHE_CONFIG_FILE, compiled with CFLAGS="-O2 -ggdb -pipe"```
, debian wheezy, and configure
```./configure --enable-debug --prefix=/opt/qrms/mapcache --disable-module --enable-memcache --enable-tiff-write-support --with-geotiff --with-gdal --with-geos```
:

#0  0x00007ffff72e1b4a in apr_pvsprintf () from /usr/lib/libapr-1.so.0
#1  0x00007ffff7bcaa10 in _mapcache_context_set_error_default (ctx=0x6071d0, code=500, msg=0x402752 "failed to open config file %s")

```
at util.c:176
```
#2  0x0000000000401c74 in load_config (ctx=0x6071d0, filename=0x7fffffffe8f3 "/opt/qrms/wms/mapcache/mapcache.xml") at mapcache.c:160
#3  0x0000000000402125 in main (argc=1, argv=0x7fffffffe688) at mapcache.c:262

From the look of it I'm guessing some string substitution of the filename is not being replaced, which gets to the apr print, which tries to expand it and crashes by doing so without no extra string passed. It would also appear to be dangerous in case of a filename with
%s
or similar patterns such as
%f
.

seeding is not optimal when metatile size is not a power of 2

In drill down seeding mode, many metatiles will be checked more than once if the metatile size is not a power of 2. This happens because a "child" metatile of a given metatile can also be the child of the adjacent metatile.

To make things worse, if seeding has been launched with --force, these duplicate metatiles will be rendered twice which is very unefficient.

To overcome this, the seeder must be modified so that the metatile size can be overriden on the command line. The seeder must also fail with an error message in case the selected metatile size is not a power of two.

tile size of 0 bytes results in stale cache

Reporter: tomhel
Date: 2012/03/02 - 16:06
If a tile has no size mapcache will fail to reseed or delete this tile.

This happens because cache_disk.c, function _mapcache_cache_disk_get(...) will return MAPCACHE_FAILURE if a tile exist but is of zero size. If MAPCACHE_CACHE_MISS is returned instead it will successfully be reseeded.

mapcache_seed.c, function examine_tile(...) will have to modified to handle this case. That is the tile exist but gives a cache miss.

The attached patches contains the following changes:

• In mapcache_seed.c, function examine_tile(...) code has been added to handle the case when a cache miss is returned.
• cache_disk.c, function _mapcache_cache_disk_get(...) will now return MAPCACHE_CACHE_MISS if a tile exists, but has no size.

mapcache_seed aborts but gives no error code indicating that seeding has failed

Reporter: tomhel
Date: 2012/03/02 - 15:59
In its current state mapcache_seed will abort a seeding if a tile failed to be retrived from the wms. It will however give no indication (error code) that seeding failed.

When seeding a cache containing several million of tiles, it might not matter much if few tiles fail. At least it does not in my case. When seeding for several weeks there is bound the be some timeouts. Of cource, i understand that there are cases when it does indeed matter.

What i propose is to add some configurable options:

• ignore failed tiles, yes/no
• threshold of how many tiles is allowed to fail before aborting.

The attached patch adds the following non-configurable functionality to mapcache_seed.c

• ignore tiles that failed to be retrieved from wms as long as the number of failed tiles is less than 1% of what has currently been seeded, otherwise abort.
• mapcache_seed will return an error code indicating if tiles failed to be fetched but was ignored (below 1%), another one if seeding was aborted because of to many failed tiles, a third if seeding was aborted by some other error.

mapcache - WMTS service accessing to a defined style in the WMS source

It would be useful to access to the different defined styles in the wms source.

<source name="layer_source" type="wms">
      <getmap>
         <params>
            <FORMAT>image/png</FORMAT>
            <LAYERS>layer1</LAYERS>
            <map>/var/www/mapserver/mapfile/layer.map</map>
            <STYLE>style1, style2 </STYLE>
        </params>
      </getmap>
     <http>
         <url>http://localhost/cgi-bin/mapserv</url>
      </http>
  </source>

At the moment only is possible to use the default style. To use different styles I would have to define different mapfiles with 1 default style/layer and, if it's possible, it would be cleaner to use style parameter .

Unable to set projection on mapObj

Reporter: hobu
Date: 2012/03/01 - 02:35
http://trac.osgeo.org/mapserver/browser/trunk/mapserver/mapcache/lib/source_mapserver.c#L57 produces a poorly useable error message and doesn't tell me why it can't set the projection on the mapObj. I think the issue is I do not have 900913 in my epsg file (I had been using EPSG:3857 everywhere) for a that is g, but the error message is unable to tell me that. Can we msGetErrorObj() here and return the info in that?

BerkeleyDB Caches - create one database per zoom level

Today for a given bdb cache, only a single database file is created.
It would be useful if mapcache could store each zoom level into a separate database file so
that it is easy to monitor the disk space each zoom level uses and to quickly remove a zoom level with a system level file delete.

libtool build error when --enable-mapserver

Reporter: msmitherdc
Date: 2012/01/12 - 22:13
Building mapcache on redhat 6 x86_64 against Oracle OHS (apache 2.2).

Compiles fine without --enable-mapserver. With --enable-mapserver i get

/bin/sh ../libtool --mode=link --tag CC gcc -rpath /home/gridusr/local/lib ./image_error.lo ./service_wmts.lo ./core.lo ./cache_sqlite.lo ./grid.lo ./configuration_xml.lo ./source.lo ./cache_tiff.lo ./imageio_mixed.lo ./dimension.lo ./http.lo ./ezxml.lo ./services.lo ./axisorder.lo ./source_mapserver.lo ./service_demo.lo ./strptime.lo ./configuration.lo ./service_kml.lo ./lock.lo ./buffer.lo ./service_wms.lo ./tileset.lo ./cache_disk.lo ./image.lo ./cache_memcache.lo ./service_ve.lo ./source_wms.lo ./source_gdal.lo ./imageio_png.lo ./imageio.lo ./imageio_jpeg.lo ./util.lo ./service_tms.lo -lcurl -lpng -ljpeg -L/u01/app/oracle/product/ofm/Oracle_WT1/lib -lapr-1 -L/u01/app/oracle/product/ofm/Oracle_WT1/lib -laprutil-1 -lsqlite3 -lpixman-1 -ltiff ../../libmapserver.la-o libmapcache.la
libtool: link: you must specify an output file
libtool: link: Try libtool --help --mode=link' for more information. make[1]: *** [libmapcache.la] Error 1 make[1]: Leaving directory/home/gridusr/local/src/mapserver/mapcache/lib'
make: *** [all] Error 2

mapcache woes with apache 2.4

Reporter: tbonfort
Date: 2012/02/28 - 20:02

• ap_unixd_setup_child isn't accessible anymore, mapcache cannot fork to run setup steps as the apache user instead of root
• security model seems changed, doc needs updating? apache conf file change:

order allow,deny
allow from all

must be replaced by

Require all granted

• check usage of apr/apr-util shipped with osx lion

Seg Fault after compiling and instaling mapcache with apache2, ubuntu 12.04

I've installed apache2 on my desktop and then followed these instructions https://github.com/mapserver/mapcache/blob/master/INSTALL

Mapcache is cloned to /home/kasheftin/work/mapcache directory, and I have such virtual host config:

<VirtualHost *:80>
    DocumentRoot /home/kasheftin/work/mapcache
    <IfModule mapcache_module>
        <Directory /home/kasheftin/work/mapcache>
            Require all granted
            Satisfy Any
            Order Allow,Deny
            Allow from all
        </Directory>
        MapCacheAlias /mapcache "/home/kasheftin/work/mapcache/mapcache.xml"
    </IfModule>
    ErrorLog /home/kasheftin/work/mapcache/logs/error.log
    CustomLog /home/kasheftin/work/mapcache/logs/access.log combined
</VirtualHost>

After that I restart apache and go to http://localhost/mapcache/demo/, and the demo page works. Then I go, for example, to this page http://localhost/mapcache/demo/wmts

• html is loading successfully, I have +- icons, but map is blank, and resources like http://localhost/mapcache/wmts/?SERVICE=WMTS&REQUEST=GetTile&VERSION=1.0.0&LAYER=test&STYLE=default&TILEMATRIXSET=WGS84&TILEMATRIX=0&TILEROW=0&TILECOL=0&FORMAT=image%2Fpng are loading infinitely.

I have theese rows in /var/log/apache2/error.log:
Apache/2.2.20 (Ubuntu) mod-mapcache/0.5-dev configured -- resuming normal operations
caught SIGTERM, shutting down
Apache/2.2.20 (Ubuntu) mod-mapcache/0.5-dev configured -- resuming normal operations
child pid 9617 exit signal Segmentation fault (11)

What should I do to get more specific description of this error?

BerkeleyDB: problems opening database with dbsql utility

• using master with "Berkeley DB 11g Release 2, library version 11.2.5.3.21"
• would like to use dbsql utility to examine the 'tiles' table
• command: dbsql my-mapcache.db
• no errors
• however the next command causes errors: .tables

  Error: file is encrypted or is not a database

• examining, I find that when dbsql utility is used, it is trying to open another database from that same file (with a name of "my-mapache.db-journal")
• if I look inside that new -journal folder, there is a "sql-errors.txt" file, that contains:

D:\ms4w\tmp\ms_tmp\cache\my-mapache.db:BDB1014 D:\ms4w\tmp\ms_tmp\cache\my-mapache.db-journal\..\berkeley.db: multiple databases specified but not supported by file

• from the BDB docs, it seems we need to make sure our db->open code allows for multiple databases to be created (see the optional "database" parameter at http://docs.oracle.com/cd/E17076_02/html/api_reference/C/dbopen.html)
• I think this is around line#105: https://github.com/mapserver/mapcache/blob/master/lib/cache_bdb.c#L105

Thoughts?

Time dimension returns more than 1 configured entries although time entries don't overlap

@tbonfort When trying to seed I get the following error although the only two time entries don't overlap:

Time dimension TIME=2011-04-10T11:46:53Z/2011-04-10T11:47:24Z returns more than 1 configured entries

The failing seed command is:

mapcache_seed -c mapcache.xml -t LI3_X___3O -g WGS84 -z 0,8 -D TIME=2011-04-10T11:46:53Z/2011-04-10T11:47:24Z

When I try a slightly shorter interval (1sec at end) it works:

mapcache_seed -c mapcache.xml -t LI3_X___3O -g WGS84 -z 0,8 -D TIME=2011-04-10T11:46:53Z/2011-04-10T11:47:23Z

A test case is available: https://github.com/Schpidi/misc/tree/master/mapcache

Thanks for any hints.