Giter Site home page Giter Site logo

gdal2mbtiles's Introduction

gdal2mbtiles

Convert GDAL-readable datasets into an MBTiles file

gdal2mbtiles helps you generate web mapping tiles that can be shown through a browser-based mapping library on your website.

GDAL-readable files are images that are georeference, that means that they are positioned and projected on to the world. In order to display a dynamic map on the web, you don't want to serve the whole image at once, so it must be sliced into tiles that are hosted by a tile server.

The MBTiles file format was developed by MapBox to make tile storage easier. You can upload the final file to their service, or run your own tile server. MapBox provides one called TileStream.

Later versions of GDAL (>= 2) allow generation of mbtiles files via the gdal_translate and gdaladdo commands. However, gdal2mbtiles offers some advantages:

  • allows you to specify an upper resolution/zoom level. GDAL always uses the native resolution of the input raster to determine the highest zoom level of the mbtiles output, whereas gdal2mbtiles can also upsample to create zoom levels at a higher resolution than your original file.
  • the gdal_translate command only converts the geotiff at the native resolution, so the lower resolutions are added to the file via overviews (gdaladdo)
  • gdaladdo can only add overviews down to the zoom level corresponding to the size of the tile/block size (256x256). gdal2mbtiles can always create images down to zoom level 1.
  • performance: gdal2mbtiles uses pyvips for image processing, which is parallel and quick. Compared to the equivalent processing with GDAL, gdal2mbtiles is typically 2-4 times quicker. For example:
    • a resolution 14 file, 13000x11000 pixels, min resolution 0, max resolution 14: ~5 minutes with gdal2mbtiles and ~8 minutes with GDAL commands.
    • a resoluton 11 file, 200,000x200,000, zoom level 11 only: ~30min with gdal2mbtiles and ~133min with GDAL (with GDAL_CACHE_MAX and GDAL_NUM_THREADS options)

Installation

PyPI package page: https://pypi.python.org/pypi/gdal2mbtiles/

Warning

gdal2mbtiles requires Python 2.7 or higher and relies on installing the items from the External Dependencies section below before the python package.

Using pip:

$ pip install gdal2mbtiles

From source:

$ git clone https://github.com/ecometrica/gdal2mbtiles.git
$ cd gdal2mbtiles
$ python setup.py install

External Dependencies

We rely on GDAL to read georeferenced datasets. Under Debian or Ubuntu, you can install the GDAL library & binary via apt.

Default GDAL versions in Ubuntu LTS:

  • Xenial: 1.11
  • Bionic: 2.2
  • Focal: 3.0

Warning

GDAL 2 is the current supported version. GDAL 3 support is in progress - contributions welcome!

We recommend using the UbuntuGIS PPA to get more recent versions of GDAL, if needed, as is the case for Xenial.

sudo add-apt-repository ppa:ubuntugis/ppa && sudo apt-get update
sudo apt-get install gdal-bin libgdal-dev

The ubuntugis PPA also usually includes python-gdal or python3-gdal that will install the python bindings at the system level. Installing that may be enough for you if you aren't planning to use a non-default python or a virtual environment.

Otherwise, you will also need to install the GDAL python bindings package from PyPI. Make sure to install the version that matches the installed GDAL library. You can double-check that version with gdal-config --version.

pip install \
  --global-option=build_ext \
  --global-option=--gdal-config=/usr/bin/gdal-config \
  --global-option=--include-dirs=/usr/include/gdal/ \
  GDAL=="$(gdal-config --version)"

We also rely on VIPS (version 8.2+) to do fast image processing.

Under Debian or Ubuntu, run the following to install it without the GUI nip2:

$ sudo apt-get install --no-install-recommends libvips libvips-dev

You'll also need a few other libraries to deal with large TIFF files and to optimize the resulting PNG tiles.

Under Debian or Ubuntu, run the following to install them:

$ sudo apt-get install libtiff5 optipng pngquant

Command Line Interface

$ gdal2mbtiles --help
usage: gdal2mbtiles [-h] [-v] [--name NAME] [--description DESCRIPTION]
                    [--layer-type {baselayer,overlay}] [--version VERSION]
                    [--format {jpg,png}]
                    [--spatial-reference SPATIAL_REFERENCE]
                    [--resampling {near,bilinear,cubic,cubicspline,lanczos}]
                    [--min-resolution MIN_RESOLUTION]
                    [--max-resolution MAX_RESOLUTION] [--fill-borders]
                    [--no-fill-borders] [--zoom-offset N]
                    [--coloring {gradient,palette,exact}]
                    [--color BAND-VALUE:HTML-COLOR]
                    [--colorize-band COLORIZE-BAND]
                    [--png8 PNG8]
                    [INPUT] [OUTPUT]

Converts a GDAL-readable into an MBTiles file

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         explain what is being done

Positional arguments:
  INPUT                 GDAL-readable file.
  OUTPUT                Output filename. Defaults to INPUT.mbtiles

MBTiles metadata arguments:
  --name NAME           Human-readable name of the tileset. Defaults to INPUT
  --description DESCRIPTION
                        Description of the layer. Defaults to ""
  --layer-type {baselayer,overlay}
                        Type of layer. Defaults to "overlay"
  --version VERSION     Version of the tileset. Defaults to "1.0.0"
  --format {jpg,png}    Tile image format. Defaults to "png"

GDAL warp arguments:
  --spatial-reference SPATIAL_REFERENCE
                        Destination EPSG spatial reference. Defaults to 3857
  --resampling {near,bilinear,cubic,cubicspline,lanczos}
                        Resampling algorithm for warping. Defaults to "near"
                        (nearest-neighbour)

Rendering arguments:
  --min-resolution MIN_RESOLUTION
                        Minimum resolution/zoom level to render and slice.
                        Defaults to None (do not downsample)
  --max-resolution MAX_RESOLUTION
                        Maximum resolution/zoom level to render and slice.
                        Defaults to None (do not upsample)
  --fill-borders        Fill image to whole world with empty tiles. Default.
  --no-fill-borders     Do not add borders to fill image.
  --zoom-offset N       Offset zoom level by N to fit unprojected images to
                        square maps. Defaults to 0.
  --png8                Quantizes 32-bit RGBA to 8-bit RGBA paletted PNGs.
                        value range from 2 to 256. Default to False.

Coloring arguments:
  --coloring {gradient,palette,exact}
                        Coloring algorithm.
  --color BAND-VALUE:HTML-COLOR
                        Examples: --color="0:#ff00ff" --color=255:red
  --colorize-band COLORIZE-BAND
                        Raster band to colorize. Defaults to 1

Contributing

Reporting bugs and submitting patches

Please check our issue tracker for known bugs and feature requests.

We accept pull requests for fixes and new features.

Development and Testing

We use Tox and Pytest to test locally and CircleCI for remote testing.

  1. Clone the repo

  2. Install whichever External Dependencies are suitable for your OS/VM.

  3. Create and activate a virtual environment

  4. Install tox: pip install tox

  5. Set the GDAL_CONFIG env var for tox via the venv activations script.

    If using virtualenv: echo 'export GDAL_VERSION=$(gdal-config --version)' >> $VIRTUAL_ENV/bin/postactivate

    If using venv: echo 'export GDAL_VERSION=$(gdal-config --version)' >> $VIRTUAL_ENV/bin/activate

  6. Run tests to confirm all is working: tox

  7. Do some development:

    • Make some changes
    • Run the tests
    • Fix any errors
    • Run the tests again
    • Update CHANGELOG.rst with a line about the change in the UNRELEASED section
    • Add yourself to AUTHORS.rst if not already there
    • Write a nice commit message
    • Repeat
  8. Make a PR

You don't need to worry initially about testing in every combination of GDAL and Ubuntu, leave that to the remote CI build matrix when you make a PR and let the reviewers figure out if it needs more work from that.

Credits

Maxime Dupuis and Simon Law wrote this program, with the generous support of Ecometrica.

See AUTHORS.rst for the full list of contributors.

gdal2mbtiles's People

Contributors

chewthemonkey avatar damon-rand avatar faibarazi avatar kukkolka avatar mghughes avatar pauricthelodger avatar rebkwok avatar sfllaw avatar skylarmt avatar spatialhast avatar vijaymadhavan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gdal2mbtiles's Issues

TypeError: in method 'SpatialReference_ImportFromEPSG', argument 2 of type 'int'

I'm trying to convert the TIFFs from the FAA's VFR Raster Charts page and I'm getting this error:

Traceback (most recent call last):
  File "/usr/local/bin/gdal2mbtiles", line 11, in <module>
    load_entry_point('gdal2mbtiles==2.1.1', 'console_scripts', 'gdal2mbtiles')()
  File "/home/ryan/.local/lib/python2.7/site-packages/gdal2mbtiles/main.py", line 309, in main
    colors=colors, band=band)
  File "/home/ryan/.local/lib/python2.7/site-packages/gdal2mbtiles/helpers.py", line 194, in warp_mbtiles
    validate_resolutions(resolution=dataset.GetNativeResolution(),
  File "/home/ryan/.local/lib/python2.7/site-packages/gdal2mbtiles/gdal.py", line 497, in GetNativeResolution
    dst_ref = self.GetSpatialReference()
  File "/home/ryan/.local/lib/python2.7/site-packages/gdal2mbtiles/gdal.py", line 463, in GetSpatialReference
    sr = sr.FromEPSG(sr.GetEPSGCode())
  File "/home/ryan/.local/lib/python2.7/site-packages/gdal2mbtiles/gdal.py", line 799, in FromEPSG
    s.ImportFromEPSG(code)
  File "/home/ryan/.local/lib/python2.7/site-packages/osgeo/osr.py", line 1138, in ImportFromEPSG
    return _osr.SpatialReference_ImportFromEPSG(self, *args)
TypeError: in method 'SpatialReference_ImportFromEPSG', argument 2 of type 'int'

Here is the output from gdalinfo -noct test.tif:

Driver: GTiff/GeoTIFF
Files: test.tif
Size is 17921, 12358
Coordinate System is:
PROJCS["Lambert Conformal Conic",
    GEOGCS["NAD83",
        DATUM["North_American_Datum_1983",
            SPHEROID["GRS 1980",6378137,298.2572221010042,
                AUTHORITY["EPSG","7019"]],
            AUTHORITY["EPSG","6269"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4269"]],
    PROJECTION["Lambert_Conformal_Conic_2SP"],
    PARAMETER["standard_parallel_1",46.66666666666666],
    PARAMETER["standard_parallel_2",41.33333333333334],
    PARAMETER["latitude_of_origin",42.3],
    PARAMETER["central_meridian",-97],
    PARAMETER["false_easting",0],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]]]
Origin = (-411171.578339697443880,257040.430107307882281)
Pixel Size = (42.335440573740584,-42.334973725198978)
Metadata:
  AREA_OR_POINT=Area
  TIFFTAG_DATETIME=2019:06:21 12:39:53
  TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
  TIFFTAG_SOFTWARE=Adobe Photoshop CS5.1 Windows
  TIFFTAG_XRESOLUTION=300
  TIFFTAG_YRESOLUTION=300
Image Structure Metadata:
  COMPRESSION=LZW
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  ( -411171.578,  257040.430) (102d10'43.70"W, 44d29'57.95"N)
Lower Left  ( -411171.578, -266135.175) (101d47'45.50"W, 39d47'47.93"N)
Upper Right (  347521.852,  257040.430) ( 92d37'16.44"W, 44d31'57.52"N)
Lower Right (  347521.852, -266135.175) ( 92d56'42.51"W, 39d49'38.48"N)
Center      (  -31824.863,   -4547.373) ( 97d23' 9.33"W, 42d15'30.11"N)
Band 1 Block=17921x1 Type=Byte, ColorInterp=Palette
  Color Table (RGB with 256 entries)

I'm using:

Ubuntu 18.04
Python 2.7.15+
pip 9.0.1
gdal2mbtiles 2.1.1
GDAL 2.4.2

Update the package to match the dependencies.

The package appears to be experiencing challenges with backward compatibility.
There are updated releases for the dependencies, and some of them are not compatible with the current version.

Do not create tiles for areas with no data.

For large corridor projects, the tiling process ends up creating thousands of empty tiles (transparent). If there is a way to ignore areas with no data, it will help create tiles faster and of lower size.

AttributeError: 'Image' object has no attribute '_VImageAdapter__inputref'

Traceback (most recent call last):
  File "/opt/conda/envs/gis-dataprocessing/bin/gdal2mbtiles", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/envs/gis-dataprocessing/lib/python3.7/site-packages/gdal2mbtiles/main.py", line 309, in main
    colors=colors, band=band)
  File "/opt/conda/envs/gis-dataprocessing/lib/python3.7/site-packages/gdal2mbtiles/helpers.py", line 211, in warp_mbtiles
    pngdata=pngdata)
  File "/opt/conda/envs/gis-dataprocessing/lib/python3.7/site-packages/gdal2mbtiles/helpers.py", line 73, in image_mbtiles
    pyramid = preprocessor(**locals())
  File "/opt/conda/envs/gis-dataprocessing/lib/python3.7/site-packages/gdal2mbtiles/helpers.py", line 324, in resample_after_warp
    pyramid.dataset.resample(resolution=resolution)
  File "/opt/conda/envs/gis-dataprocessing/lib/python3.7/site-packages/gdal2mbtiles/vips.py", line 601, in resample
    ratios=self.GetScalingRatios(resolution=resolution, places=5)
  File "/opt/conda/envs/gis-dataprocessing/lib/python3.7/site-packages/gdal2mbtiles/vips.py", line 582, in _resample
    output_size=(dst_width, dst_height)
  File "/opt/conda/envs/gis-dataprocessing/lib/python3.7/site-packages/gdal2mbtiles/vips.py", line 392, in stretch
    interpolate='near'
  File "/opt/conda/envs/gis-dataprocessing/lib/python3.7/site-packages/gdal2mbtiles/vips.py", line 349, in _scale
    interpolate=interpolate)
  File "/opt/conda/envs/gis-dataprocessing/lib/python3.7/site-packages/gdal2mbtiles/vips.py", line 281, in affine
    image.__inputref = self.image
AttributeError: 'Image' object has no attribute '_VImageAdapter__inputref'

Versions

$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"
$ vips --version
vips-8.4.5-Wed Jan 18 12:36:44 UTC 2017
$ gdalinfo --version
GDAL 2.4.2, released 2019/06/28
$ pip freeze | grep gdal
gdal2mbtiles==2.1.1
$ pip freeze | grep proj
pyproj==2.2.1
$ pip freeze | grep rasterio
rasterio==1.0.28

Error "default_rgba.png" not found

I guess "default_rgba.png" missing.

When executing

$ gdal2mbtiles --spatial-reference 4326 test.tiff test.mbtiles

it shows an error

File ".../gdal2mbtiles/storages.py", line 86, in _border_image
    width, height, ink=rgba(r=0, g=0, b=0, a=0)
  File ".../gdal2mbtiles/vips.py", line 166, in new_rgba
    os.path.dirname(os.path.realpath(__file__)), 'default_rgba.png'
  File ".../pyvips/vimage.py", line 210, in new_from_file
    raise Error('unable to load from file {0}'.format(vips_filename))
pyvips.error.Error: unable to load from file .../gdal2mbtiles/default_rgba.png
  VipsForeignLoad: file ".../gdal2mbtiles/default_rgba.png" not found

No 512px option and --resampling not working

Thank you for this project that I'm trying to use in order to generate mbtiles down to zoom level 2 (gdaladdo's minsize is not working for some reason so I can't get overviews smaller than 256 px).

I add to make the following adjustments in order to get an image quality as good as what I get with GDAL:

  • in constants.py, set TILE_SIDE to 512. It would be great to have this as an option.
  • in vips.py, update the shrink_affine & stretch methods to use the bicubic resampling method, and support it in the affine method.

Would you accept pull requests to fix these 2 problems? The second one requires some discussion, as it seems to me that the --resampling option should be used here.

Also, the return c_int.in_dll(self.libvips, 'vips__concurrency').value in vips.py's get_concurrency throws a ValueError: dlsym(0x7ff915fc8f50, vips__concurrency): symbol not found (I'm on macOS). I had to wrap it in a try/exept and return 0 in the exept (so that it then uses cpu_count()).

support for OpenFileGDB format

i'm trying to convert GeoDB files into MBTiles (to ultimately convert to OSMAnd SQLiteDB)

this gets me some of the way there:

diff --git a/gdal2mbtiles/gdal.py b/gdal2mbtiles/gdal.py
index 0eaf77f..157baae 100644
--- a/gdal2mbtiles/gdal.py
+++ b/gdal2mbtiles/gdal.py
@@ -35,7 +35,7 @@ from xml.etree import ElementTree
 
 import numpy
 
-from osgeo import gdal, gdalconst, osr
+from osgeo import gdal, gdalconst, osr, ogr
 from osgeo.gdalconst import (GA_ReadOnly, GRA_Bilinear, GRA_Cubic,
                              GRA_CubicSpline, GRA_Lanczos,
                              GRA_NearestNeighbour)
@@ -419,7 +419,9 @@ class Dataset(gdal.Dataset):
             inputfile = inputfile.encode('utf-8')
         try:
             # Since this is a SWIG object, clone the ``this`` pointer
-            self.this = gdal.Open(inputfile, mode).this
+            #self.this = gdal.Open(inputfile, mode).this
+            driver = ogr.GetDriverByName("OpenFileGDB")
+            self.this = driver.Open(inputfile, mode).this
         except RuntimeError as e:
             raise GdalError(str(e))

however, i'm running into an error which may be related to #45

  File "/src/github.com/ecometrica/gdal2mbtiles/.venv/lib/python3.11/site-packages/osgeo/gdal.py", line 3047, in GetGeoTransform
    return _gdal.Dataset_GetGeoTransform(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: in method 'Dataset_GetGeoTransform', argument 1 of type 'GDALDatasetShadow *'

Usage instructions

Hi this looks like a useful tool, but I don't see any usage instructions in the README. Could you add a quick note about how to use it?

KAP Files Supported?

I'm running this with the following and getting an error. I extracted the BSB/KAP files from here: https://www.charts.noaa.gov/RNCs/IL_RNCs.zip

gdal2mbtiles BSB_ROOT/14927/14927_1.KAP
Traceback (most recent call last):
File "/usr/local/bin/gdal2mbtiles", line 11, in
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/gdal2mbtiles/main.py", line 309, in main
colors=colors, band=band)
File "/usr/local/lib/python2.7/dist-packages/gdal2mbtiles/helpers.py", line 194, in warp_mbtiles
validate_resolutions(resolution=dataset.GetNativeResolution(),
File "/usr/local/lib/python2.7/dist-packages/gdal2mbtiles/gdal.py", line 497, in GetNativeResolution
dst_ref = self.GetSpatialReference()
File "/usr/local/lib/python2.7/dist-packages/gdal2mbtiles/gdal.py", line 463, in GetSpatialReference
sr = sr.FromEPSG(sr.GetEPSGCode())
File "/usr/local/lib/python2.7/dist-packages/gdal2mbtiles/gdal.py", line 799, in FromEPSG
s.ImportFromEPSG(code)
File "/usr/lib/python2.7/dist-packages/osgeo/osr.py", line 1123, in ImportFromEPSG
return _osr.SpatialReference_ImportFromEPSG(self, *args)
TypeError: in method 'SpatialReference_ImportFromEPSG', argument 2 of type 'int'

vrt compatibility

Hi, my input file is a vrt composed of thoundands of tif files with /vsitar/ and /vsizip/ paths. Any chance to have it supported ? alternatevely, do you know any workaround without rendering it to a huge intermediate tiff file ? thanks in advance

GDAL 3.X?

Will this support GDAL 3.X? Would love to use it but can't since its only on GDAL 2.X

Unsupported SRS

I'm getting issues about identifying the right SRS. Whether I let the system detect it or set it manually using --spatial-reference, I get the same error.
I've pasted the error, as well as the output from gdalinfo of my tif.

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/gdal2mbtiles-2.1.1-py3.6.egg/gdal2mbtiles/gdal.py", line 457, in GetSpatialReference
  File "/home/ubuntu/.local/lib/python3.6/site-packages/osgeo/osr.py", line 808, in AutoIdentifyEPSG
    return _osr.SpatialReference_AutoIdentifyEPSG(self, *args)
RuntimeError: OGR Error: Unsupported SRS

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/gdal2mbtiles", line 11, in <module>
    load_entry_point('gdal2mbtiles==2.1.1', 'console_scripts', 'gdal2mbtiles')()
  File "/usr/local/lib/python3.6/dist-packages/gdal2mbtiles-2.1.1-py3.6.egg/gdal2mbtiles/main.py", line 309, in main
  File "/usr/local/lib/python3.6/dist-packages/gdal2mbtiles-2.1.1-py3.6.egg/gdal2mbtiles/helpers.py", line 194, in warp_mbtiles
  File "/usr/local/lib/python3.6/dist-packages/gdal2mbtiles-2.1.1-py3.6.egg/gdal2mbtiles/gdal.py", line 497, in GetNativeResolution
  File "/usr/local/lib/python3.6/dist-packages/gdal2mbtiles-2.1.1-py3.6.egg/gdal2mbtiles/gdal.py", line 463, in GetSpatialReference
  File "/usr/local/lib/python3.6/dist-packages/gdal2mbtiles-2.1.1-py3.6.egg/gdal2mbtiles/gdal.py", line 799, in FromEPSG
  File "/home/ubuntu/.local/lib/python3.6/site-packages/osgeo/osr.py", line 1138, in ImportFromEPSG
    return _osr.SpatialReference_ImportFromEPSG(self, *args)
TypeError: in method 'SpatialReference_ImportFromEPSG', argument 2 of type 'int'

gdalinfo output

Driver: GTiff/GeoTIFF
Files: mmr.tif
Size is 13797, 16140
Coordinate System is:
PROJCS["unnamed",
    GEOGCS["NAD83",
        DATUM["North_American_Datum_1983",
            SPHEROID["GRS 1980",6378137,298.2572221010042,
                AUTHORITY["EPSG","7019"]],
            TOWGS84[0,0,0,0,0,0,0],
            AUTHORITY["EPSG","6269"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4269"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",42.83333333333334],
    PARAMETER["central_meridian",-70.16666666666667],
    PARAMETER["scale_factor",0.9999666666666667],
    PARAMETER["false_easting",2952750],
    PARAMETER["false_northing",0],
    UNIT["US survey foot",0.3048006096012192,
        AUTHORITY["EPSG","9003"]]]
Origin = (2908649.651220000348985,295540.606010000046808)
Pixel Size = (0.059410639269400,-0.059410452912021)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  ( 2908649.651,  295540.606) ( 70d19'59.77"W, 43d38'38.64"N)
Lower Left  ( 2908649.651,  294581.721) ( 70d19'59.75"W, 43d38'29.17"N)
Upper Right ( 2909469.340,  295540.606) ( 70d19'48.62"W, 43d38'38.66"N)
Lower Right ( 2909469.340,  294581.721) ( 70d19'48.60"W, 43d38'29.19"N)
Center      ( 2909059.496,  295061.164) ( 70d19'54.19"W, 43d38'33.92"N)
Band 1 Block=13797x1 Type=Byte, ColorInterp=Red
  Mask Flags: PER_DATASET ALPHA 
Band 2 Block=13797x1 Type=Byte, ColorInterp=Green
  Mask Flags: PER_DATASET ALPHA 
Band 3 Block=13797x1 Type=Byte, ColorInterp=Blue
  Mask Flags: PER_DATASET ALPHA 
Band 4 Block=13797x1 Type=Byte, ColorInterp=Alpha

Set zoom levels

How do you set zoom levels such as --zoom 10-15?
Is this what the resolution arguments are for?

ValueError: yscale 0.11356327132282978 cannot be less than 1.0

Installed gdal2mbtiles in a virtual environment. Tried to use it with this raster from https://www.naturalearthdata.com:

https://www.naturalearthdata.com/downloads/10m-gray-earth/gray-earth-with-shaded-relief-and-water/

Direct link for the raster: https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/raster/GRAY_LR_SR_W.zip

Got this error:

ValueError: yscale 0.11356327132282978 cannot be less than 1.0

Full output:

(my_env) $ gdal2mbtiles --coloring=gradient --color "0:#fff" GRAY_LR_SR_W.tif GRAY_LR_SR_W.mbtiles
Traceback (most recent call last):
  File "/home/pvieira/my_env/bin/gdal2mbtiles", line 8, in <module>
    sys.exit(main())
  File "/home/pvieira/my_env/lib/python3.10/site-packages/gdal2mbtiles/main.py", line 297, in main
    warp_mbtiles(inputfile=inputfile.name, outputfile=outputfile.name,
  File "/home/pvieira/my_env/lib/python3.10/site-packages/gdal2mbtiles/helpers.py", line 203, in warp_mbtiles
    return image_mbtiles(inputfile=warped, outputfile=outputfile,
  File "/home/pvieira/my_env/lib/python3.10/site-packages/gdal2mbtiles/helpers.py", line 73, in image_mbtiles
    pyramid = preprocessor(**locals())
  File "/home/pvieira/my_env/lib/python3.10/site-packages/gdal2mbtiles/helpers.py", line 322, in resample_after_warp
    pyramid.dataset.resample_to_world()
  File "/home/pvieira/my_env/lib/python3.10/site-packages/gdal2mbtiles/vips.py", line 601, in resample_to_world
    result = self._resample(ratios=ratios)
  File "/home/pvieira/my_env/lib/python3.10/site-packages/gdal2mbtiles/vips.py", line 570, in _resample
    self._image = VImageAdapter(self.image).stretch(
  File "/home/pvieira/my_env/lib/python3.10/site-packages/gdal2mbtiles/vips.py", line 377, in stretch
    raise ValueError(
ValueError: yscale 0.11356327132282978 cannot be less than 1.0

I'm using gdal 3.4.1.

Failed to initialize PROJ pipeline

gdal2mbtiles.exceptions.CalledGdalError: Command '['gdalwarp', '-q', '-of', 'VRT', '-t_srs', 'EPSG:3857', '-r', 'bilinear', 'input.tif', '/vsistdout']' returned non-zero exit status 1.: proj_create: Error -38: failed to load datum shift file
ERROR 6: Failed to initialize PROJ pipeline from `+proj=utm +zone=4 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs' to `+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs': failed to load datum shift file

versions

$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"
$ python --version
Python 3.6.7
$ pip freeze | grep gdal
gdal2mbtiles==2.1.1
$ gdalinfo --version
GDAL 2.4.3, released 2019/10/28
$ proj --version
Rel. 6.1.1, July 1st, 2019

TypeError: in method 'SpatialReference_ImportFromEPSG', argument 2 of type 'int'

I note this issue has already been raise at : #22 however in that case it was for a ,png with no projection data/EPSG number embedded.
I am encountering the same error trying to run on a geotiff which gdal info shows has spatial referencing:

my command is:
gdal2mbtiles -v masked.tif masked.mbtiles --min-resolution=7 --max-resolution=8

Traceback (most recent call last):
File "/home/nev/.local/bin/gdal2mbtiles", line 11, in
load_entry_point('gdal2mbtiles==2.1.2', 'console_scripts', 'gdal2mbtiles')()
File "/home/nev/.local/lib/python2.7/site-packages/gdal2mbtiles/main.py", line 309, in main
colors=colors, band=band)
File "/home/nev/.local/lib/python2.7/site-packages/gdal2mbtiles/helpers.py", line 194, in warp_mbtiles
validate_resolutions(resolution=dataset.GetNativeResolution(),
File "/home/nev/.local/lib/python2.7/site-packages/gdal2mbtiles/gdal.py", line 497, in GetNativeResolution
dst_ref = self.GetSpatialReference()
File "/home/nev/.local/lib/python2.7/site-packages/gdal2mbtiles/gdal.py", line 463, in GetSpatialReference
sr = sr.FromEPSG(sr.GetEPSGCode())
File "/home/nev/.local/lib/python2.7/site-packages/gdal2mbtiles/gdal.py", line 799, in FromEPSG
s.ImportFromEPSG(code)
File "/usr/lib/python2.7/dist-packages/osgeo/osr.py", line 797, in ImportFromEPSG
return _osr.SpatialReference_ImportFromEPSG(self, *args)
TypeError: in method 'SpatialReference_ImportFromEPSG', argument 2 of type 'int'

the gdalinfo on the file returns:
Driver: GTiff/GeoTIFF
Files: masked.tif
Size is 3620, 2530
Coordinate System is:
PROJCS["unnamed",
GEOGCS["unnamed ellipse",
DATUM["unknown",
SPHEROID["unnamed",6378137,0]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433]],
PROJECTION["Mercator_2SP"],
PARAMETER["standard_parallel_1",0],
PARAMETER["central_meridian",0],
PARAMETER["false_easting",0],
PARAMETER["false_northing",0],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]]]
Origin = (15660855.904725419357419,-4021344.132019802927971)
Pixel Size = (289.432029319978540,-286.592914214417760)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left (15660855.905,-4021344.132) (140d41' 1.90"E, 33d56'39.98"S)
Lower Left (15660855.905,-4746424.205) (140d41' 1.90"E, 39d10'21.33"S)
Upper Right (16708599.851,-4021344.132) (150d 5'45.26"E, 33d56'39.98"S)
Lower Right (16708599.851,-4746424.205) (150d 5'45.26"E, 39d10'21.33"S)
Center (16184727.878,-4383884.169) (145d23'23.58"E, 36d36'10.04"S)
Band 1 Block=3620x1 Type=Byte, ColorInterp=Red
Band 2 Block=3620x1 Type=Byte, ColorInterp=Green
Band 3 Block=3620x1 Type=Byte, ColorInterp=Blue

ModuleNotFoundError: No module named 'UserDict'

Hi,
I'm trying to run gdal2mbtiles in Ubuntu 16.04.1 LTS terminal (Linux bash shell in Win10) and python 2.7 virtual environment created through Anaconda.

After installation of dependencies and establishing symlink for vipscc, gdal2mbtiles --help runs successfully. However when I try to convert the GeoTiff to MbTiles, I receive the following error about UserDict missing.

This is something that I'm not able to install via pip either. Is there any way to resolve this?

Thank you!

Traceback (most recent call last):
  File "C:\Users\Jess\Anaconda3\Scripts\gdal2mbtiles-script.py", line 11, in <module>
    load_entry_point('gdal2mbtiles==1.3.2', 'console_scripts', 'gdal2mbtiles')()
  File "C:\Users\Jess\Anaconda3\lib\site-packages\pkg_resources\__init__.py", line 570, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "C:\Users\Jess\Anaconda3\lib\site-packages\pkg_resources\__init__.py", line 2751, in load_entry_point
    return ep.load()
  File "C:\Users\Jess\Anaconda3\lib\site-packages\pkg_resources\__init__.py", line 2405, in load
    return self.resolve()
  File "C:\Users\Jess\Anaconda3\lib\site-packages\pkg_resources\__init__.py", line 2411, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "C:\Users\Jess\Anaconda3\lib\site-packages\gdal2mbtiles\main.py", line 52, in <module>
    from .mbtiles import Metadata
  File "C:\Users\Jess\Anaconda3\lib\site-packages\gdal2mbtiles\mbtiles.py", line 28, in <module>
    from UserDict import DictMixin
ModuleNotFoundError: No module named 'UserDict'

ImportError: No module named vipsCC.VError

I got this erro when install it using pip. on CentOS7


Traceback (most recent call last):
  File "/usr/bin/gdal2mbtiles", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/gdal2mbtiles/main.py", line 242, in main
    args = parse_args(args=args)
  File "/usr/lib/python2.7/site-packages/gdal2mbtiles/main.py", line 209, in parse_args
    args.coloring = coloring_arg(args.coloring)
  File "/usr/lib/python2.7/site-packages/gdal2mbtiles/main.py", line 65, in coloring_arg
    from gdal2mbtiles import vips
  File "/usr/lib/python2.7/site-packages/gdal2mbtiles/vips.py", line 36, in <module>
    from vipsCC.VError import VError
ImportError: No module named vipsCC.VError

TypeError LoadLibrary() argument 1 must be str, not None

Python 3.7.6
GDAL 3.0.4, released 2020/01/28
Windows 10, 64 bit

gdal works normally as expected, e.g., gdal_translate

gdal2mbtiles -h shows help items.

Installed gdal2mbtiles with git; installation has no error message.

While running on a test tif image, the following error is thrown. It seems related to LoadLibrary() and being new to this new stack (python, gdal, gdal2mbtiles), I'm confused where the error is coming from and where to look. Is it related or unrelated to gdal2mbtiles. Python seems to be working normally.

gdal2mbtiles test_cl.tif test.mbt --name TEST --description CMCSST --layer-type overlay --version 0.0 --format png --spatial-reference 4326 --resampling near --min-resolution 2 --max-resolution 7 --fill-borders --png8 256 --coloring palette --color="0:#ff00ff"

Error
Traceback (most recent call last):
File "C:\Users\Dash\Anaconda3\Scripts\gdal2mbtiles-script.py", line 11, in
load_entry_point('gdal2mbtiles==2.1.2', 'console_scripts', 'gdal2mbtiles')()
File "C:\Users\Dash\Anaconda3\lib\site-packages\gdal2mbtiles-2.1.2-py3.7.egg\gdal2mbtiles\main.py", line 264, in main
File "C:\Users\Dash\Anaconda3\lib\site-packages\gdal2mbtiles-2.1.2-py3.7.egg\gdal2mbtiles\main.py", line 231, in parse_args
File "C:\Users\Dash\Anaconda3\lib\site-packages\gdal2mbtiles-2.1.2-py3.7.egg\gdal2mbtiles\main.py", line 65, in coloring_arg
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 668, in _load_unlocked
File "", line 638, in load_backward_compatible
File "C:\Users\Dash\Anaconda3\lib\site-packages\gdal2mbtiles-2.1.2-py3.7.egg\gdal2mbtiles\vips.py", line 116, in
File "C:\Users\Dash\Anaconda3\lib\site-packages\gdal2mbtiles-2.1.2-py3.7.egg\gdal2mbtiles\vips.py", line 87, in init
File "C:\Users\Dash\Anaconda3\lib\ctypes_init
.py", line 442, in LoadLibrary
return self.dlltype(name)
File "C:\Users\Dash\Anaconda3\lib\ctypes_init
.py", line 364, in init
self._handle = _dlopen(self._name, mode)
TypeError: LoadLibrary() argument 1 must be str, not None

Can someone put me on the right track where to look for the issue or help me to troubleshoot?
Thank you

JPEG format supported?

I tried to use the --format jpg option. By exploring mbtile file content with sqlite3 utility, I can see that the generated mbtile contains PNG tiles.

Is the JPEG image format supported?

Here is the executed command line:
gdal2mbtiles --format jpg --min-resolution 0 --max-resolution 15 --no-fill-borders input.jp2 output.mbtiles
OS=ubuntu 17.10
gdal2mbtiles=tag version-2.0.0

Thanks for this great tool!

gdalwarp - No target filename specified

When I run gdal2mbtiles I get the following error:

gdal2mbtiles ./Untitled.tif
Traceback (most recent call last):
File "/usr/local/bin/gdal2mbtiles", line 9, in
load_entry_point('gdal2mbtiles==1.3.0', 'console_scripts', 'gdal2mbtiles')()
File "/usr/local/lib/python2.7/dist-packages/gdal2mbtiles/main.py", line 283, in main
colors=colors, band=band)
File "/usr/local/lib/python2.7/dist-packages/gdal2mbtiles/helpers.py", line 189, in warp_mbtiles
resampling=resampling, compress='LZW')
File "/usr/local/lib/python2.7/dist-packages/gdal2mbtiles/gdal.py", line 111, in preprocess
functions=functions, compress=compress, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/gdal2mbtiles/gdal.py", line 129, in pipeline
vrt = f(previous)
File "/usr/local/lib/python2.7/dist-packages/gdal2mbtiles/gdal.py", line 206, in warp
return VRT(check_output_gdal([str(e) for e in warp_cmd]))
File "/usr/local/lib/python2.7/dist-packages/gdal2mbtiles/gdal.py", line 73, in check_output_gdal
error=stderrdata.rstrip('\n'))
gdal2mbtiles.exceptions.CalledGdalError: Command '['gdalwarp', '-q', '-of', 'VRT', '-t_srs', 'EPSG:3857', '-r', 'near', './Untitled.tif', '/dev/stdout']' returned non-zero exit status 1:
FAILURE: No target filename specified.

But running gdalwarp on its own with the above args it appears to work.

TypeError: in method 'SpatialReference_ImportFromEPSG', argument 2 of type 'int' in gdal.py line 799

Hello again~
Error occurs when converting PNG to MBTiles (whenever calling _osr.SpatialReference_ImportFromEPS actually).

$ gdal2mbtiles test.png test.mbtiles
Traceback (most recent call last):
  File ".../bin/gdal2mbtiles", line 11, in <module>
    sys.exit(main())
  File ".../gdal2mbtiles/main.py", line 309, in main
    colors=colors, band=band)
  File ".../gdal2mbtiles/helpers.py", line 194, in warp_mbtiles
    validate_resolutions(resolution=dataset.GetNativeResolution(),
  File ".../gdal2mbtiles/gdal.py", line 497, in GetNativeResolution
    dst_ref = self.GetSpatialReference()
  File ".../gdal2mbtiles/gdal.py", line 463, in GetSpatialReference
    sr = sr.FromEPSG(sr.GetEPSGCode())
  File ".../gdal2mbtiles/gdal.py", line 799, in FromEPSG
    s.ImportFromEPSG(code)
  File ".../osgeo/osr.py", line 1128, in ImportFromEPSG
    return _osr.SpatialReference_ImportFromEPSG(self, *args)
TypeError: in method 'SpatialReference_ImportFromEPSG', argument 2 of type 'int'

I tried to show what "code" was and therefore added print("code: {}, type(code): {}".format(code, type(code))) to line 799 in gdal.py.

$ gdal2mbtiles test.png test.mbtiles
code: 3857, type(code): <type 'int'>
code: 3857, type(code): <type 'int'>
code: None, type(code): <type 'NoneType'>
Traceback (most recent call last):
  File ".../bin/gdal2mbtiles", line 11, in <module>
    sys.exit(main())
  File ".../gdal2mbtiles/main.py", line 309, in main
    colors=colors, band=band)
  File ".../gdal2mbtiles/helpers.py", line 194, in warp_mbtiles
    validate_resolutions(resolution=dataset.GetNativeResolution(),
  File ".../gdal2mbtiles/gdal.py", line 497, in GetNativeResolution
    dst_ref = self.GetSpatialReference()
  File ".../gdal2mbtiles/gdal.py", line 463, in GetSpatialReference
    sr = sr.FromEPSG(sr.GetEPSGCode())
  File ".../gdal2mbtiles/gdal.py", line 800, in FromEPSG
    s.ImportFromEPSG(code)
  File ".../osgeo/osr.py", line 1128, in ImportFromEPSG
    return _osr.SpatialReference_ImportFromEPSG(self, *args)
TypeError: in method 'SpatialReference_ImportFromEPSG', argument 2 of type 'int'

Therefore I tried to give a --spatial-reference 4326, it shows the following.

$ gdal2mbtiles --spatial-reference 4326 test.png test.mbtiles
code: 4326, type(code): <type 'int'>
code: 3857, type(code): <type 'int'>
code: None, type(code): <type 'NoneType'>
Traceback (most recent call last):
  File ".../bin/gdal2mbtiles", line 11, in <module>
    sys.exit(main())
  File ".../gdal2mbtiles/main.py", line 309, in main
    colors=colors, band=band)
  File ".../gdal2mbtiles/helpers.py", line 194, in warp_mbtiles
    validate_resolutions(resolution=dataset.GetNativeResolution(),
  File ".../gdal2mbtiles/gdal.py", line 497, in GetNativeResolution
    dst_ref = self.GetSpatialReference()
  File ".../gdal2mbtiles/gdal.py", line 463, in GetSpatialReference
    sr = sr.FromEPSG(sr.GetEPSGCode())
  File ".../gdal2mbtiles/gdal.py", line 800, in FromEPSG
    s.ImportFromEPSG(code)
  File ".../osgeo/osr.py", line 1128, in ImportFromEPSG
    return _osr.SpatialReference_ImportFromEPSG(self, *args)
TypeError: in method 'SpatialReference_ImportFromEPSG', argument 2 of type 'int'

Seems that it always fails in the third round.
Please help.

Throws error: pyvips.error.Error: unable to call embed linear: vector must have 1 or 3 elements

Script runs for awhile, then throws the following error.

Running Ubuntu 18.04, gdalinfo is 2.2.3. Any ideas?

ubuntu@:~$ sudo gdal2mbtiles --min-resolution 1 --max-resolution 15 test.tif test.mbtiles
Traceback (most recent call last):
  File "/usr/local/bin/gdal2mbtiles", line 11, in <module>
    load_entry_point('gdal2mbtiles==2.1.1', 'console_scripts', 'gdal2mbtiles')()
  File "/home/ubuntu/.local/lib/python2.7/site-packages/gdal2mbtiles/main.py", line 309, in main
    colors=colors, band=band)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/gdal2mbtiles/helpers.py", line 211, in warp_mbtiles
    pngdata=pngdata)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/gdal2mbtiles/helpers.py", line 75, in image_mbtiles
    pyramid.slice(fill_borders=fill_borders)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/gdal2mbtiles/vips.py", line 1081, in slice
    fill_borders=fill_borders)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/gdal2mbtiles/vips.py", line 990, in slice_downsample
    levels=(self.resolution - max_resolution),
  File "/home/ubuntu/.local/lib/python2.7/site-packages/gdal2mbtiles/vips.py", line 835, in downsample
    offset=offset)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/gdal2mbtiles/vips.py", line 428, in tms_align
    x, y, width, height, background=[0, 0, 0, 0]  # Transparent
  File "/home/ubuntu/.local/lib/python2.7/site-packages/pyvips/vimage.py", line 804, in call_function
    return pyvips.Operation.call(name, self, *args, **kwargs)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/pyvips/voperation.py", line 189, in call
    raise Error('unable to call {0}'.format(operation_name))
pyvips.error.Error: unable to call embed linear: vector must have 1 or 3 elements

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.