kong/httpsnippet

HTTP Request snippet generator for many languages & libraries

HTTPSnippet

HTTP Request snippet generator for many languages & tools including: cURL, HTTPie, JavaScript, Node, C, Java, PHP, Objective-C, Swift, Python, Ruby, C#, Go, OCaml and more!

Relies on the popular HAR format to import data and describe HTTP calls.

See it in action on companion service: APIembed

• HTTPSnippet
  • Quickstart
    • Core Concepts
    • CLI Quickstart
    • TypeScript Library Quickstart
  • CLI Usage
    • CLI Installation
    • Example
  • TypeScript Library Usage
    • Library Installation
    • Types
      • HarRequest
      • HarEntry
      • TargetId
      • ClientId
      • Converter
      • Client
      • ClientInfo
      • Extension
      • TargetInfo
      • Target
    • Library Exports
      • new HTTPSnippet(source: HarRequest | HarEntry)
      • snippet.convert(targetId: string, clientId?: string, options?: T)
      • isTarget
      • addTarget
      • isClient
      • addTargetClient
  • Bugs and feature requests
  • Contributing

Quickstart

Core Concepts

1. HTTPSnippet's input is a JSON object that represents an HTTP request in the HAR Request Object format.
2. HTTPSnippet's output is executable code that sends the input HTTP request, in a wide variety of languages and libraries.
3. You provide HTTPSnippet your desired target, client, and options.
   • a target refers to a group of code generators. Generally, a target is a programming language like Rust, Go, C, or OCaml.
   • client refers to a more specific generator within the parent target. For example, the C# target has two available clients, httpclient and restsharp, each referring to a popular C# library for making requests.
   • options are per client and generally control things like specific indent behaviors or other formatting rules.

CLI Quickstart

httpsnippet har.json \ # the path your input file (must be in HAR format)
  --target shell \ # your desired language
  --client curl \ # your desired language library
  --output ./examples \ # an output directory, otherwise will just output to Stdout
  --options '{ "indent": false }' # any client options as a JSON string

TypeScript Library Quickstart

import { HTTPSnippet } from 'httpsnippet';

const snippet = new HTTPSnippet({
  method: 'GET',
  url: 'http://mockbin.com/request',
});

const options = { indent: '\t' };
const output = snippet.convert('shell', 'curl', options);
console.log(output);

CLI Usage

CLI Installation

NPM	Yarn
npm install --global httpsnippet	yarn global add httpsnippet

httpsnippet [harFilePath]

the default command

Options:
      --help     Show help                                   [boolean]
      --version  Show version number                         [boolean]
  -t, --target   target output                     [string] [required]
  -c, --client   language client                              [string]
  -o, --output   write output to directory                    [string]
  -x, --options  provide extra options for the target/client  [string]

Examples:
  httpsnippet my_har.json --target rust --client actix --output my_src_directory

Example

The input to HTTPSnippet is any valid HAR Request Object, or full HAR log format.

`example.json`

{
  "method": "POST",
  "url": "http://mockbin.com/har?key=value",
  "httpVersion": "HTTP/1.1",
  "queryString": [
    {
      "name": "foo",
      "value": "bar"
    },
    {
      "name": "foo",
      "value": "baz"
    },
    {
      "name": "baz",
      "value": "abc"
    }
  ],
  "headers": [
    {
      "name": "accept",
      "value": "application/json"
    },
    {
      "name": "content-type",
      "value": "application/x-www-form-urlencoded"
    }
  ],
  "cookies": [
    {
      "name": "foo",
      "value": "bar"
    },
    {
      "name": "bar",
      "value": "baz"
    }
  ],
  "postData": {
    "mimeType": "application/x-www-form-urlencoded",
    "params": [
      {
        "name": "foo",
        "value": "bar"
      }
    ]
  }
}

httpsnippet example.json --target shell --client curl --output ./examples

$ tree examples
examples/
└── example.sh

inside examples/example.sh you'll see the generated output:

curl --request POST \
  --url 'http://mockbin.com/har?foo=bar&foo=baz&baz=abc&key=value' \
  --header 'accept: application/json' \
  --header 'content-type: application/x-www-form-urlencoded' \
  --cookie 'foo=bar; bar=baz' \
  --data foo=bar

provide extra options:

httpsnippet example.json --target shell --client curl --output ./examples --options '{ "indent": false }'

and see how the output changes, in this case without indentation

curl --request POST --url 'http://mockbin.com/har?foo=bar&foo=baz&baz=abc&key=value' --header 'accept: application/json' --header 'content-type: application/x-www-form-urlencoded' --cookie 'foo=bar; bar=baz' --data foo=bar

TypeScript Library Usage

Library Installation

NPM	Yarn
npm install --save httpsnippet	yarn add httpsnippet

Types

HarRequest

See https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/har-format for the TypeScript type corresponding to this type

HarEntry

interface Entry {
  request: Partial<HarRequest>;
}

interface HarEntry {
  log: {
    version: string;
    creator: {
      name: string;
      version: string;
    };
    entries: {
      request: Partial<HarRequest>;
    }[];
  };
}

TargetId

type TargetId = string;

ClientId

type ClientId = string;

Converter

type Converter<T extends Record<string, any>> = (
  request: Request,
  options?: Merge<CodeBuilderOptions, T>,
) => string;

Client

interface Client<T extends Record<string, any> = Record<string, any>> {
  info: ClientInfo;
  convert: Converter<T>;
}

ClientInfo

interface ClientInfo {
  key: ClientId;
  title: string;
  link: string;
  description: string;
}

Extension

type Extension = `.${string}` | null;

TargetInfo

interface TargetInfo {
  key: TargetId;
  title: string;
  extname: Extension;
  default: string;
}

Target

interface Target {
  info: TargetInfo;
  clientsById: Record<ClientId, Client>;
}

Library Exports

new HTTPSnippet(source: HarRequest | HarEntry)

Name of conversion target

import { HTTPSnippet } from 'httpsnippet';

const snippet = new HTTPSnippet({
  method: 'GET',
  url: 'http://mockbin.com/request',
});

snippet.convert(targetId: string, clientId?: string, options?: T)

The convert method requires a target ID such as node, shell, go, etc. If no client ID is provided, the default client for that target will be used.

Note: to see the default targets for a given client, see target.info.default. For example shell's target has the default of curl.

Many targets provide specific options. Look at the TypeScript types for the target you are interested in to see what options it provides. For example shell:curl's options correspond to the CurlOptions interface in the shell:curl client file.

import { HTTPSnippet } from 'httpsnippet';

const snippet = new HTTPSnippet({
  method: 'GET',
  url: 'http://mockbin.com/request',
});

// generate Node.js: Native output
console.log(snippet.convert('node'));

// generate Node.js: Native output, indent with tabs
console.log(
  snippet.convert('node', {
    indent: '\t',
  }),
);

isTarget

Useful for validating that a custom target is considered valid by HTTPSnippet.

const isTarget: (target: Target) => target is Target;

import { myCustomTarget } from './my-custom-target';
import { isTarget } from 'httpsnippet';

try {
  console.log(isTarget(myCustomTarget));
} catch (error) {
  console.error(error);
}

addTarget

Use addTarget to add a new custom target that you can then use in your project.

const addTarget: (target: Target) => void;

import { myCustomClient } from './my-custom-client';
import { HAR } from 'my-custom-har';
import { HTTPSnippet, addTargetClient } from 'httpsnippet';

addTargetClient(myCustomClient);

const snippet = new HTTPSnippet(HAR);
const output = snippet.convert('customTargetId');
console.log(output);

isClient

Useful for validating that a custom client is considered valid by HTTPSnippet.

const isClient: (client: Client) => client is Client;

import { myCustomClient } from './my-custom-client';
import { isClient } from 'httpsnippet';

try {
  console.log(isClient(myCustomClient));
} catch (error) {
  console.error(error);
}

addTargetClient

Use addTargetClient to add a custom client to an existing target. See addTarget for how to add a custom target.

const addTargetClient: (targetId: TargetId, client: Client) => void;

import { myCustomClient } from './my-custom-client';
import { HAR } from 'my-custom-har';
import { HTTPSnippet, addTargetClient } from 'httpsnippet';

addTargetClient('customTargetId', myCustomClient);

const snippet = new HTTPSnippet(HAR);
const output = snippet.convert('customTargetId', 'customClientId');
console.log(output);

Bugs and feature requests

Have a bug or a feature request? Please first read the issue guidelines and search for existing and closed issues. If your problem or idea is not addressed yet, please open a new issue.

Contributing

Please read through our contributing guidelines. Included are directions for opening issues, coding standards, and notes on development.

For info on creating new conversion targets, please review this guideline

Moreover, if your pull request contains TypeScript patches or features, you must include relevant unit tests.

Editor preferences are available in the editor config for easy use in common text editors. Read more and download plugins at http://editorconfig.org.

httpsnippet's Issues

Curl and wget should not escape newlines in body

If you provide a body like this:

{\n\t"hello": "world"\n}

you will get a snippet like this:

curl --request POST \
  --url https://insomnia.rest/ \
  --header 'content-type: application/json;' \
  --data '{\n	"hello": "world"\n}'

But the expected output is this:

curl --request POST \
  --url https://insomnia.rest/ \
  --header 'content-type: application/json;' \
  --data '{
	"hello": "world"
}'

The HTTPie target seems to work this way.

Feature Request - Request POJO code generation

I believe it would be highly beneficial for the library to add POJO creation within its Java section.

Many popular libraries in Java development (e.g. Retrofit, GSON) use POJOs in HTTP requests during serialization, so having httpsnippet generate these objects from a request would therefore be quite useful.

An example of what I'd expect this feature to do can be seen using this example

Request (in cURL format)

curl -X POST -H "Content-Type: application/json" -d '{
  "Email": "[email protected]",
  "Password": "password",
  "Settings": {
    "MySetting": true
  }
}' "http://example.com/login"

Generated

public class LoginRequest {
    private String email;
    private String password;
    private Settings settings;

   public String getEmail(){ return email; }
   public void setEmail(String email){ 
      this.email = email;
   }
   // ... more getters and setters 
}

public class Settings {
    public boolean mySetting;
    public void getMySetting(){ return mySetting; }
    public boolean setMySetting(boolean mySetting){
        this.mySetting = mySetting;
    }
}

Is this considered in scope for the library and if so, could it be added potentially?

Windows - Powershell

https://technet.microsoft.com/en-us/library/hh849901.aspx

HTTP Method not capitalized

The HTTP Method is not capitalized which is not correct. According RFC 7231, the HTTP method should be in capitals.
As the methods are in lower cases, it breaks using the snippets for example with curl:

curl --request get --url http://example.com

This example will result in a 400 Bad Request in most cases. The correct call should be:

curl --request GET --url http://example.com

Can you quickly fix this, or should I (when I have time) create a PR?

They should be an explicit error when using an invalid target from command line

I use the command line.

If I write :

httpsnippet -t curl test.har

I get :

✓ test.har > curl [default] :
false

Since curl is not a valid target, I would expect the program to write an error message such as : curl is not a valid target, here is the list of valid targets : ...

Typo in CONTRIBUTING.md

The letter c can be seen before the # of the main heading of CONTRIBUTING.md. I can only assume this is a typo.

form.pipe is not a function

I bundled the module using browserify and used in my application, code is not generated for the mimetype is multipart/form-data and params is filled.
Then i changed the version of form-data in package.json to "^0.2.0" then the error is resolved.

Docs: put a description file in the README

An example of how to describe a endpoint should be included in the README otherwise people will have a hard time understanding how this work

Adding "?" to path

This line of code is adding a question mark to the end of paths even if they have no query string. I changed it but it broke some node tests, is this intentional?

Here's what I would have changed it to, since the path is set right above with just the pathname.

// add search property to path
if (this.source.uriObj.search) {
  this.source.uriObj.path = this.source.uriObj.pathname + '?' + this.source.uriObj.search
}

[objc/native] Improvements

I want to implement a couple more things:

About literals (I like literals because they generate less cluttered code snippets and allow more flexibility to the people pasting them):

• Add a literals or verbose flag (verbose would be more consistent with other targets): If true, will generate literals from parameters/headers like currently, if false, will just use the objects computed by httpsnippet. It will be a nice options if people don't want too much verbose at the cost of flexibility for "pasters".
• Make body parameters literals (not sure about this one: very verbose)
• Make querystring parameters literals (not sure about this one either: very verbose)

About code style (JS):

• Comment what's going on in the generation (native.js)
• Improve the code climate rating (:cry:)

About improvements that could be made but are waiting on general guidelines:

• Parse the response body from NSData. Need to wait on response definition for now.
• Add an option for explanatory comments generation? Depends if other languages will do that too in the future. It could be a new guideline.

Create test fixtures for custom HTTP methods

JavaScript - Fetch API

https://developer.mozilla.org/en/docs/Web/API/Fetch_API

Don't convert header names to lowercase

This is a super minor issue of aesthetics, but since HTTP header names are case insensitive, it would be nice if the header names are preserved as provided in the HAR object. I just think it looks nicer in my documentation to have:

-H 'Authorization: Bearer <Your Token>'

Instead of:

-H 'authorization: Bearer <Your Token>'

Javascript conversion crashes during multipart form data

Hi,

In javascript code generation template you are doing deletion of content type header in case of multi part form data. Code crashes saying "Cannot read property 'indexOf' of undefined" if javascript conversion is called twice and if same snippet is converted to other language content header will be missing.
Ex:
var snippet = new HTTPSnippet(data);
console.log(snippet.convert('javascript'));
console.log(snippet.convert('javascript'));

Regards,
Karthik M L

Leverage promises to reduce code slant

Escape necessary for postfields?

Hai @ahmadnassri

Is escape necessary for CURLOPT_POSTFIELDS in httpsnippet/src/targets/php/curl.js?
Instead we can enclose data in single quotes(' ') because it is effecting the readability for the end-users

Use in the browser?

What sorts of Node dependencies are currently required, and how difficult would you estimate a browser compatible version would be?

HAR Schema Questions & Discussion

I have several questions after implementing objc native and unirest (coming):

• What handles or should handle schema errors? It seems weird having to make checks such as:

if (this.source.postData && (this.source.postData.params || this.source.postData.text)) {

Right now it is possible to have a postData without params or text. This should probably throw an error to the user saying his HAR description is wrong? For this reason (and because the fixtures for unit tests probably don't cover enough cases) Node native is currently wrong, see #8. And it makes us write annoying checks instead of a simple... say switch for form-encoded and others.

• Cannot build a native JSON object when mimeType is 'application/json' because the text property is a string. Having to send JSON as a string where the majority of other examples on the Internet are object literals seems... weird. It also serves well enough its purpose for a copy-pasted code snippet but the programmer cannot add logic on top of it, or intuitively add a parameter.

Example for obj-c sending json like most people do:

// ...
NSDictionary *parameters = @{@"parameter": @"value", @"foo": @"bar"};
NSData *postData = [NSJSONSerialization dataWithJSONObject:parameters options:0 error:&error];
[request setHTTPBody:postData];

But httpsnippet makes us send stringified JSON manually encoded:

NSData *postData = [[NSData alloc] initWithData:[@"{\"foo\": \"bar\"}" dataUsingEncoding:NSUTF8StringEncoding]];
[request setHTTPBody:postData];

• No way to know the response type in advance for sure? For Unirest the class of the response object will vary, because one will deserialize the response if JSON (UNIHTTPJsonResponse) and the other will return it as is (UNIHTTPStringResponse) (same for .NET I think). Sure I can rely on the Accept header but who says it will be acknowledged?

it's not so much about the class of the response because when you think about it, we are missing opportunities to show how to handle JSON responses in every other language (where JSON is the most used response type)

Swift - alamofire

Add support for Python Requests

Ruby - Unirest

add support for JavaScript XHR

CFML

TypeScript - Angular2

https://angular.io/docs/ts/latest/tutorial/toh-pt6.html

Add node > request option

(adding this more so I can get an issue #)

add support for Native Ruby

Add support for C using libcurl

There's a cool trick to generate C code from curl itself by adding the flag --libcurl output.c

Here's an implementation for the full.json request:

#include <curl/curl.h>

int main(int argc, char *argv[])
{
  CURLcode ret;
  CURL *hnd;
  struct curl_slist *headers;

  headers = NULL;
  headers = curl_slist_append(headers, "accept: application/json");
  headers = curl_slist_append(headers, "content-type: application/json");

  hnd = curl_easy_init();
  curl_easy_setopt(hnd, CURLOPT_CUSTOMREQUEST, "POST");
  curl_easy_setopt(hnd, CURLOPT_URL, "http://mockbin.com/request?foo=bar&foo=baz");
  curl_easy_setopt(hnd, CURLOPT_HTTPHEADER, headers);
  curl_easy_setopt(hnd, CURLOPT_COOKIE, "foo=bar; bar=baz");
  curl_easy_setopt(hnd, CURLOPT_POSTFIELDS, "{\"foo\": \"bar\"}");
  curl_easy_setopt(hnd, CURLOPT_NOPROGRESS, 1L);
  curl_easy_setopt(hnd, CURLOPT_POSTFIELDSIZE_LARGE, (curl_off_t)14);
  curl_easy_setopt(hnd, CURLOPT_MAXREDIRS, 50L);
  curl_easy_setopt(hnd, CURLOPT_TCP_KEEPALIVE, 1L);

  ret = curl_easy_perform(hnd);

  curl_easy_cleanup(hnd);
  hnd = NULL;
  curl_slist_free_all(headers);
  headers = NULL;

  return (int)ret;
}

saved as full.c it can be compiled with:

gcc -o full full.c -lcurl

then run with:

./full

JavaScript - Angular ($http)

https://docs.angularjs.org/api/ng/service/$http

automatically run generated code snippet for testing

idea:

• convert and execute full.json output for every target in the command line
• response from mockbin will give a HAR object, that can be used to compare against original source (should be 1:1 match, with some additional transport headers)

challenges:

• some languages might need compiling
• can all languages can output a clean string to stdout?
• forcing snippets generators to have a console.log function for this to work, perhaps in an option flag
  • tests can check for existence of said options flag, but not for actual implementation .. that'll be human eye

add support for jQuery

PHP - Unirest

add support for OkHttp

http://square.github.io/okhttp/

objc - afnetworking

Docs: README examples are outdates/broken

The README says:

httpsnippet my-api-endpoint.json --langauge php --output ./snippets

But typo aside, --language doesn't work, it's --target or -t now. Also I heard @SGrondin mentioning than the out/ option was not working?

Improve target list in README

In the targets section it is getting harder to read as we add targets and sub-targets. Perhaps a table would be better suited? Also it would be cool to have a link to the source.

Homepage example doesn't work

curl --request POST \
  --url "http://httpconsole.com/debug?foo=bar&foo=baz" \
  --cookie "foo=bar; bar=baz" \
  --header "Accept: text/plain" \
  --header "Content-Type: application/json" \
  --data "{\"foo\": \"bar\"}"

Gives:

Cannot POST /debug?foo=bar&foo=baz

objc - unirest

Occasional Bluebird failures?

Every so often snippet generation will fail with blank output. The following gets written to STDERR:

✖ code_sample_322968423736805296.json20150921-8796-1364q6w fail: Catch filter must inherit from Error or be a simple predicate function

    See http://goo.gl/o84o68

(Note that code_sample_322968423736805296.json20150921-8796-1364q6w is just the name of the Tempfile I'm using)

Input JSON looks something like:

{"method":"GET","url":"http://testurl.com/test","httpVersion":"HTTP/1.1","headers":[{"name":"accept","value":"application/json"},{"name":"ApiKey","value":"YOUR API KEY HERE"}],"cookies":[],"queryString":[],"postData":{},"headersSize":-1,"bodySize":-1}

But the error seems intermittent - if I re-run httpsnippet with the same JSON usually it works fine.

Add node > unirest option

(again, adding to get an issue #)

[docs] Full example

I am not sure anybody currently landing on this project understands instantly what it does, as there is no clear example of a generated code snippet, and 4 sections before any example. What do you all think about including a full example, with a smaller file?

---

HTTP Snippet

HTTP requests code snippet generator for many programming languages. (Or httpsnippet generates code snippets to perform HTTP requests in many languages? Something maybe simpler.)

Targets

Installation

Example

From this request description:

<small fixture> (query.json)

Run:

$ httpsnippet query.json -t node

To generate a Node.js code snippet performing the request:

var request = require('request');
// ...

(and then the module example)

Tests: force every language to have tests against every fixture

And fixtures should be more descriptive in their name. Instead of simple.json or http1.json we could have get.json, querystring.json, application-form-encoded.json, application-json.json, multipart.json. Or well, anything more descriptive.

A link "see the examples here" in the README pointing to the fixtures/ folder would be nice too.

Use Ruby's Native JSON conversion rather than outputting a JSON string

i.e instead of:

  body = " { \"a\": 123 }"

Do:

  require 'json'
  body = { a: 123 }.to_json

Clojure - clj-http

Have created an implementation already, will create a PR referencing this issue.

add support for C#

Create test fixtures for urls using the https protocol

Some languages require you to be explicit about which protocol you are using. For example in Ruby you must set the use_ssl flag.

Lua lua-resty-http

Node native sends undefined when missing PostData

With the query.json fixture, Node native outputs:

req.write(undefined)
req.end();

Scala - Native

Python - Unirest

objc:nsurlsession target does not seem to support boolean values in postData payload

Hello,

I set my HAR request as follow:

{
  "headers": [
    {
      "name": "Host",
      "value": "example.com"
    },
    {
      "name": "Accept",
      "value": "application/json"
    },
    {
      "name": "Content-Type",
      "value": "application/json"
    }
  ],
  "bodySize": 160,
  "postData": {
    "text": "{\n  \"active\" : false\n}",
    "mimeType": "application/json"
  },
  "httpVersion": "HTTPS/1.1",
  "method": "POST",
  "url": "https://example.com/sample/"
}

Then I launch the following command line:

httpsnippet /tmp/sources/httpsnippet/test.json --target objc --client nsurlsession

I get the following exception:

/usr/local/lib/node_modules/httpsnippet/src/targets/objc/helpers.js:65
        return '@"' + value.replace(/"/g, '\\"') + '"'
                            ^
TypeError: Object false has no method 'replace'
    at Object.module.exports.literalRepresentation (/usr/local/lib/node_modules/httpsnippet/src/targets/objc/helpers.js:65:29)
    at Object.module.exports.literalRepresentation (/usr/local/lib/node_modules/httpsnippet/src/targets/objc/helpers.js:61:63)
    at Object.module.exports.nsDeclaration (/usr/local/lib/node_modules/httpsnippet/src/targets/objc/helpers.js:37:24)
    at module.exports (/usr/local/lib/node_modules/httpsnippet/src/targets/objc/nsurlsession.js:59:29)
    at /usr/local/lib/node_modules/httpsnippet/src/index.js:200:19
    at Array.map (native)
    at HTTPSnippet.convert (/usr/local/lib/node_modules/httpsnippet/src/index.js:199:33)
    at iterator (/usr/local/lib/node_modules/httpsnippet/bin/httpsnippet:83:24)
    at /usr/local/lib/node_modules/httpsnippet/node_modules/async/lib/async.js:246:17
    at /usr/local/lib/node_modules/httpsnippet/node_modules/async/lib/async.js:122:13

Version of node.js: 0.10.32
Version of httpsnippet: 1.16.2