vincentsarago/lambda-tiler

AWS Lambda + rio-tiler to serve tiles from any web hosted files

lambda-tiler

AWS Lambda + rio-tiler to serve tiles from any web hosted files

lambda-tiler is a simple serverless (AWS Lambda function) application that serves Map Tiles dynamically created from COGs hosted remotely (s3, http, ...)

Deploy

Requirement

• AWS Account
• Docker (+ docker-compose)
• node + npm (serverless)

Create the package

# Build Amazon linux AMI docker container + Install Python modules + create package
$ git clone https://github.com/vincentsarago/lambda-tiler.git
$ cd lambda-tiler/

$ docker-compose build
$ docker-compose run --rm package

# Tests
$ docker-compose run --rm tests

Note: Docker image from https://github.com/RemotePixel/amazonlinux

Deploy to AWS

#configure serverless (https://serverless.com/framework/docs/providers/aws/guide/credentials/)
npm install
sls deploy

API

Viewer

/viewer - GET

A web viewer that allows you to pan & zoom the COG.

Inputs:

• url (required, str): mosaic id
• kwargs (optional): Other querystring parameters will be forwarded to the tile url.

Outputs:

• html (text/html)

$ curl {your-endpoint}/viewer?url=https://any-file.on/the-internet.tif

TileJSON (2.1.0)

/tilejson.json - GET

Inputs:

• url (required): mosaic definition url
• tile_format (optional, str): output tile format (default: "png")
• kwargs (in querytring): Other querystring parameters will be forwarded to the tile url

Outputs:

• tileJSON (application/json)

$ curl https://{endpoint-url}/tilejson.json?url=https://any-file.on/the-internet.tif

{
    "bounds": [...],      
    "center": [lon, lat], 
    "minzoom": 18,        
    "maxzoom": 22,        
    "name": "the-internet.tif",
    "tilejson": "2.1.0",  
    "tiles": [...] ,      
}

Bounds

Inputs:

• url (required): mosaic definition url

Outputs:

• metadata (application/json)

$ curl https://{endpoint-url}/bounds?url=https://any-file.on/the-internet.tif

{
  "url": "https://any-file.on/the-internet.tif", 
  "bounds": [...]
}

Metadata

/metadata - GET

Inputs:

• url (required, str): dataset url
• pmin (optional, str): min percentile (default: 2).
• pmax (optional, str): max percentile (default: 98).
• nodata (optional, str): Custom nodata value if not preset in dataset.
• indexes (optional, str): dataset band indexes
• overview_level (optional, str): Select the overview level to fetch for statistic calculation
• max_size (optional, str): Maximum size of dataset to retrieve for overview level automatic calculation
• histogram_bins (optional, str, default:20): number of equal-width histogram bins
• histogram_range (optional, str): histogram min/max

Outputs:

• metadata (application/json)

$ curl https://{endpoint-url}/metadata?url=s3://url=https://any-file.on/the-internet.tif

{
    "address": "s3://myfile.tif",
    "bbox": [...],
    "band_descriptions": [(1, "red"), (2, "green"), (3, "blue"), (4, "nir")],
    "statistics": {
        "1": {
            "pc": [38, 147],
            "min": 20,
            "max": 180,
            "std": 28.123562304138662,
            "histogram": [
                [...],
                [...]
            ]
        },
        ...
    }
}

Tiles

/tiles/{z}/{x}/{y} - GET

/tiles/{z}/{x}/{y}.{ext} - GET

/tiles/{z}/{x}/{y}@{scale}x - GET

/tiles/{z}/{x}/{y}@{scale}x.{ext} - GET

Inputs:

• z: Mercator tile zoom value
• x: Mercator tile x value
• y: Mercator tile y value
• ext: image format (e.g jpg)
• scale (optional, int): tile scale (default: 1)
• url (required, str): dataset url
• indexes (optional, str): dataset band indexes (default: dataset indexes)
• expr (optional, str): dataset expression
• nodata (optional, str): Custom nodata value if not preset in dataset (default: None)
• rescale (optional, str): min/max for data rescaling (default: None)
• color_formula (optional, str): rio-color formula (default: None)
• color_map (optional, str): rio-tiler colormap (default: None)

Outputs:

• image body (image/jpeg)

$ curl {your-endpoint}/tiles/7/10/10.png?url=https://any-file.on/the-internet.tif

Note:

• expr and indexes cannot be passed used together
• if no ext passed, lambda-tiler will choose the best format (jpg or png) depending on mask (use png for tile with nodata)

Example

A web viewer that allows you to pan & zoom on a sample tiff.

Inputs: None

Outputs:

• html (text/html)

$ curl {your-endpoint}/example

lambda-tiler's Issues

add /point handler

update with https://github.com/RemotePixel/cogeo-layer

Viewer/example can break due to case-sensitivy of host in Lambda event

The example and viewer pages were not working while I was experimenting with this a bit. It turned out the endpoint being passed to the template was empty due to the incoming Lambda event["headers"] having a lowercase host key, whereas the app expects a Host key.

I'm guessing it's a browser thing (e.g. serverless/serverless#2765). Using Firefox here and all the keys in the headers are lowercased, whereas in the multiValueHeaders they are not.

State of this repo

👋 I've decided to fork this repo and continue the development over https://github.com/developmentseed/cogeo-tiler

Discrete color maps

Hi Vincent,

I was just wondering if discrete colormaps will work with lambda-tiler or if it's something that could be useful to add in the future. Especially for things like land use maps and other classified rasters.

Was doing some digging and found that they were added to rio-tiler recently (cogeotiff/rio-tiler#92) but it seems lambda-tiler uses rio-tiler's get_colormap with just the color_map query string parameter.

Not sure what would be the best approach to implement this. Passing a dict in a query string seems a bit messy, maybe another parameter for a discrete color map, or some other format for the color_map parameter to make lambda-tiler realize that it needs to fetch a discrete color map instead.

Thanks!

switch to lambda-proxy v4

vincentsarago/lambda-proxy#10

add mosaic endpoints

either with mosaicJSON support or multiple urls https://medium.com/devseed/cog-talk-part-2-mosaics-bbbf474e66df

add WMTS support

Cannot allocate bytes errror

When deploying with only changing the region and deploymentBucket in serverless.yml, I am receiving the following errors when trying to access a public OAM image (as well as other tiffs I've tried).

https:///production/tiles/0/0/0.jpeg?url=https://oin-hotosm.s3.amazonaws.com/56f9b5a963ebf4bc00074e70/0/56f9c2d42b67227a79b4faec.tif

leads to this in the browser:
{
"errorMessage": "Read or write failed"
}

and this in the logs:
ERROR 2: rasterio.cpp, 1048: cannot allocate 9324026721 bytes

Tiling from private bucket (S3 Signed URLs)

I'd like to dynamically tile some COGs stored in a private S3 bucket (no public access). I'm trying to do so with S3 signed URLs:

https://{bucket}.s3.amazonaws.com/{key}?AWSAccessKeyId={val}&Signature={val}&Expires={val}

This is causing an error when the URL is passed to lambda-proxy because AWSAccessKeyId, Signature and Expires are being treated as arguments rather than part of the URL:

[lambda-tiler] - [ERROR] - bounds_handler() got an unexpected keyword argument 'Expires'

I'm not set on using S3 Signed URLs in particular, so if there is a better way of authenticating a private bucket for use with the tiler which currently works please let me know.