pmonks/unfurl

This library implements "URL unfurling" approximately according to how Slack does it.

main
dev

unfurl

This library implements "URL unfurling" approximately according to how Slack does it. See this blog post for more info.

Installation

unfurl is available as a Maven artifact from Clojars. The latest version is:

Trying it Out

Clojure CLI

$ clj -Sdeps '{:deps {com.github.pmonks/unfurl {:mvn/version "#.#.#"}}}'  # Where #.#.# is replaced with an actual version number >= 0.10.0

Leiningen

$ lein try com.github.pmonks/unfurl

Usage

The functionality is provided by the unfurl.api namespace.

Require it in the REPL:

(require '[unfurl.api :as uf] :reload-all)

Require it in your application:

(ns my-app.core
  (:require [unfurl.api :as uf]))

The library provides a single function - unfurl. The API documentation has full details, and the unit tests provide some example usages.

Contributor Information

Contributing Guidelines

Bug Tracker

Code of Conduct

Developer Workflow

The repository has two permanent branches: main and dev. All development must occur either in branch dev, or (preferably) in feature branches off of dev. All PRs must also be submitted against dev; the main branch is only updated from dev via PRs created by the core development team. All other changes submitted to main will be rejected.

This model allows otherwise unrelated changes to be batched up in the dev branch, integration tested there, and then released en masse to the main branch, which will trigger automated generation and deployment of the release (Codox docs to GitHub Pages, JARs to Clojars, etc.).

Why are there so many different groupIds on Clojars for this project?

The project was originally developed under my personal GitHub account. In early 2018 it was transferred to the clj-commons GitHub organisation, but then, as that group refined their scope and mission, it was determined that it no longer belonged there, and the project were transferred back in late 2021. During this time the build tooling for the project also changed from Leiningen to tools.build, which created further groupId churn (tools.build introduced special, useful semantics for com.github.username groupIds that don't exist with Leiningen or Clojars).

License

Copyright © 2016 Peter Monks

Distributed under the Apache License, Version 2.0.

SPDX-License-Identifier: Apache-2.0

unfurl's Issues

Add support for HTTP proxies

The options map should allow HTTP proxy information to be passed in.

Handle 416 HTTP response codes properly

Currently unfurl blindly issues an HTTP range request without checking if the server supports them or not. Although servers are supposed to return the full content with a 200 status code if they do not support HTTP range requests, some return a 416 status code instead, and in these cases unfurl should retry the request without the Range header.

(see also the discussion in issue #6)

Make this library compatible with ClojureScript

Feature Request

Description of Problem:

I'm working on a new feature for status-react. This feature is about adding URL unfurling to its chat. The problem is that it's a ClojureScript project, so it's currently not possible to import this library and use it there.

Potential Solutions:

I've researched about this .cljc extension, which is compatible between Clojure and ClojureScript. I'd like to clone this repo and transform your .clj into a .cljc, and then create a Pull Request if you're interested in this.

@pmonks How hard do you think it would be to do it? Do you think it's worth it, or should I better find another solution for my need? I'm fairly new to Clojure and ClojureScript, so I would really appreciate your input here. Thanks!

Implement oEmbed

Description of Problem:

oEmbed is not yet supported. This request is to add support for it.

Port from midje to clojure.test

Description of Problem:

It's probably past time to port from midje to clojure.test. Despite offering a more pleasant developer experience than clojure.test, midje doesn't seem to be very actively maintained any longer.

Range header is not syntactically valid

As per RFC-2616 §14.35, the value of a "Range" header must include a range unit (e.g. "bytes"). Currently, the unfurl library is not doing this.

Migrate from TravisCI to GitHub Actions

When I push changes to GitHub, I want to see well-integrated results, so I can simplify the systems I need to look at for status updates.

Proposed Solution:

Migrate from TravisCI based CI to GitHub Actions based CI.

Note: we will (hopefully temporarily) lose the ability to test across a matrix of JVM versions

Add request information to exceptions, to facilitate debugging

When an exception is thrown by unfurl, I want to be able to see the details of the HTTP GET request that was issued, so I can more easily debug what's going on, and test with other tools (e.g. curl, wget, httpie, etc.).