noandrea/distill

URL shortener for events

Distill

Another url shortener

[]

Motivations

Existing url shorteners are not suitable for a private deploy use or are too complex in terms of requirements.

Distill aims to create a easy deployable short url service that can be used for specific events.

Features

• Choose the alphabet set for the generate short id
• Choose the length of the generate short id
• Load existing short id <-> url mappings*
• Overwrite an existing short id with a different target url*
• Set a time to live on short ids (globally or per id)
• Set a expiration date on short ids (globabbly or per id)
• Set a request limit on short ids (globally or per id)
• Set a redirect for the / path
• Set a redirect url for exhausted ids (request limit reached)
• Set a redirect url for expired ids (ttl/end date reached)
• Backup/restore urls in csv or binary format
• Import data via csv
• Get statistics both globally and for short id

* the alphabet and lenght can be enforced

Expiration strategy

There are 3 ways to set an expiration for a short id:

• TTL (seconds)
• Epiration date
• Max requests

The three options can be configured globally or per short id, the value specified for the short id takes always precedence over the global configuration.

For the TTL and the expiration date the actual expiration is selected as max ( creation_date + ttl, expiration_date)

!!! the expiration is set upon short id creation, changing global configuration will not affect the short ids already set !!!

For redirects, the expiration url redirect takes precedence over the exhaustion url redirect.

If no redirects are set for exhausted / expired url then a 404 is returned.

Api Doc

Register a redirect

Miniaml request is:

POST http://localhost:1804/api/short
X-API-KEY: 123123_changeme_changeme
Content-Type: application/json

{
    "url": "https://example.com/target_url"
}

Repsonse:

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: *
Access-Control-Allow-Methods: *
Access-Control-Allow-Origin: *
Content-Type: application/json
Date: Sun, 17 Mar 2019 21:18:16 GMT
Content-Length: 16
Connection: close

{
  "id": "wBNaqx"
}

A request can contain additional fields:

POST http://localhost:1804/api/short
X-API-KEY: 123123_changeme_changeme
Content-Type: application/json

{
    "id": "myid"
    "url": "https://example.com/target_url",
    "max_requests": 20,
    "url_exhausted" : "https://example.com/max_requests_reached_url",
    "ttl": 0,
    "expire_on": "2039-03-17T22:05:28+01:00",
    "url_expired" : "https://example.com/ttl_or_epiration_reached_url"
}

Repsonse:

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: *
Access-Control-Allow-Methods: *
Access-Control-Allow-Origin: *
Content-Type: application/json
Date: Sun, 17 Mar 2019 21:18:16 GMT
Content-Length: 16
Connection: close

{
  "id": "myid"
}

Backup / Restore

Offline backup in csv and binary format

Import data

required fields

url

all fields

url,id,max_requests,url_exhausted,ttl,expires_on,url_expired

the dates are expressed in RFC3339 format

Configuration

TODO

Example API request

TODO

Build targets

default

build (build-dist) clean

docker (docker-build) docker-push docker-run lint test

Development

• to generate the Colfer model run colf -b internal Go api/model.colf from the project root

• to enable coverage badge use ^coverage:\s(\d+(?:\.\d+)?%) as regexp in gilab configuration

Hints

To generate an API Token randomly use the make gen-secret (linux/mac only):

make gen-secret 
WiYS8DauSwVIMeNGIp63ScmY-pgA1ECA7ai7Oce7

Installation

Distill is distributed via different channels

Docker

Distill is available on docker hub, as for the images:

• latest tag is built from the develop git branch and contains the latest changes
• tags in the form x.y.z are built from git tags and are considered stable

to run distill in docker use the following command:

docker run -p 1804:1804 noandrea/distill 

the default configuration for the docker image is available

• here for stable releases
• here for latest releases

Mount points

To override the configuration mount a volume in /settings.docker.yaml path. For the data folder the mount point is /data.

Docker Compose

A docker-compose example is available in the examples/docker

Systemd

Distill can be run via systemd, check the example configuration.

distill's Issues

Allow to set a dataset read only

Is your feature request related to a problem? Please describe.
On a marketing campaign, the leaking of the api-key may cause major embarrassment.
To secure a setup against involuntary leaking of the api key, make it so that a system can be set read only, so the short urls cannot be manipulated.

Describe the solution you'd like
A confguration entry for read only api

Describe alternatives you've considered
Filtering / blacklisting the /api endpoint will also work but require more configuration/knowledge that is against the spirit of the project

Additional context
Add any other context or screenshots about the feature request here.

Improve docker image offering

Is your feature request related to a problem? Please describe.
Docker images seems not be super updated, and it is not clear how they function

Describe the solution you'd like
I would like to see automated docker builds with

• tag latest coming from develop branch
• and proper tags from git tags

Url with anchor not recognized

Describe the bug
redirect urls with anchor (#) are not accepted

To Reproduce
use this json to define a target url for redirect :

{
  "url": "http://example.com#anchor
}

Expected behavior
they are valid urls and should be accepted

Adopt the EUPL license

Is your feature request related to a problem? Please describe.
Apparently the MIT license is not technically valid in EU
the EU compatible license is
https://joinup.ec.europa.eu/collection/eupl/eupl-text-11-12

Use snake case in json reply

currently the json keys are in pascal case, make them in snake case:

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: *
Access-Control-Allow-Methods: *
Access-Control-Allow-Origin: *
Content-Length: 409
Content-Type: application/json
Date: Tue, 09 Apr 2019 08:52:26 GMT
Vary: Accept-Encoding
Connection: close

{
  "ID": "ypxhlx",
  "URL": "xxxxx",
  "Counter": 0,
  "BountAt": "2019-03-29T11:15:02.500478396Z",
  "MaxRequests": 1,
  "ExhaustedURL": "",
  "TTL": 0,
  "ExpireOn": "0001-01-01T00:00:00Z",
  "ExpiredURL": ""
}

Add K8s example setup

Describe the solution you'd like
just a K8s example