Giter Site home page Giter Site logo

putsreq's Introduction

Build Status Code Climate Test Coverage

The PutsReq codebase will be private going forward. You can still integrate with the products as you could before and we've updated the support instructions below. Questions? Write us at support [email protected].

PutsReq

PutsReq lets you record HTTP requests and simulate responses like no other tool available. Try it now!

Check this post: Play Rock-paper-scissors with Slack and PutsReq for some other examples.

Getting Started

Response Builder

The Response Builder is the place where you can create your responses using JavaScript V8.

Check the list below with the request attributes you can access to create your own responses:

request

// curl -X POST -H 'X-MyHeader: MyHeaderValue' -d 'name=Pablo' https://putsreq.com/<YOUR-TOKEN>

request.request_method;
// => POST

request.body;
// => name=Pablo

request.params.name;
// => Pablo

request.headers["HTTP_X_MYHEADER"];
// => MyHeaderValue

Parsing a JSON request:

// curl -i -X POST -H 'Content-Type: application/json' -d '{"message":"Hello World"}' https://putsreq.com/<YOUR-TOKEN>

var parsedBody = JSON.parse(request.body);

parsedBody.message;
// => Hello World

response

response.status = 200; // default value
response.headers = {}; // default value
response.body = "ok"; // default value

Returning a JSON response:

response.headers["Content-Type"] = "application/json";

response.body = { message: "Hello World" };

forwardTo

If you only want to log your requests, you can use PutsReq just as a proxy for your requests.

request.forwardTo = "http://example.com/api";

But you can always modify requests before forwarding them.

// add or change a header
request.headers["X-MyNewHeader"] = "MyHeaderValue";

var parsedBody = JSON.parse(request.body);

// add or change a value
parsedBody["my_new_key"] = "my new value";

request.body = parsedBody;

request.forwardTo = "http://example.com/api";

CLI

Do want to test Webhook calls against your localhost? PutsReq makes it easy!

You can think of it, as a kind of ngrok, but instead of creating a tunnel to your localhost, PutsReq polls requests from YOUR-PUTSREQ-TOKEN and forwards to your localhost.

gem install putsreq

putsreq forward --to http://localhost:3000 --token YOUR-TOKEN

Listening requests from YOUR-TOKEN
Forwarding to http://localhost:3000
Press CTRL+c to terminate
2016-12-21 20:49:54 -0200       POST    200

Ajax

PutsReq supports CORS, so you can use it to test your Ajax calls.

<html>
  <head>
    <title>Your Website</title>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js"></script>
    <script>
      // Sample PutsReq Response Builder
      // https://putsreq.com/<YOUR-TOKEN>/inspect
      // response.headers['Content-Type'] = 'application/json';
      // response.body = { 'message': 'Hello World' };

      // Sample Ajax call
      $.get("https://putsreq.com/<YOUR-TOKEN>", function (data) {
        alert(data.message);
        // => 'Hello World'
      });
    </script>
  </head>
  <body></body>
</html>

Sample Integration Tests

https://github.com/phstc/putsreq_integration_sample

Steps to run PutsReq in development

For following the instructions below, you will need to install Docker.

cd ~/workspace

git clone [email protected]:phstc/putsreq.git

docker-compose up -d

open http://localhost:3000

docker-compose logs --follow --tail=100 app

Running tests

docker-compose run app bundle exec rspec

Production

In production (Heroku), PutsReq runs on mLab sandbox, with a storage of 500 MB. For avoiding exceeding the capacity, the requests and responses collections must be converted into capped collections.

db.runCommand({ convertToCapped: "requests", size: 15000000 });
db.runCommand({ convertToCapped: "responses", size: 15000000 });

Production setup through Docker

This walks you through a quick guideline in order to setup PutsReq on your own server through Docker and Docker Compose. Please read this section carefully and know what you are doing in order not to lose any data.

Configuration

The docker-compose.yml file.

version: '3.6'
services:
  db:
    image: mongo:3.6.17
    tty: true
    stdin_open: true
    volumes:
      - data:/data/db
  redis:
    image: redis:alpine
  app:
    image: daqzilla/putsreq
    tty: true
    stdin_open: true
    command: /bin/sh -c "rm -f /app/tmp/pids/server.pid && bundle exec rails server -p 3000 -b '0.0.0.0'"
    ports:
      - '5050:3000'
    env_file:
      - .env.docker
    depends_on:
      - db
      - redis
volumes:
  data:
    external:
      name: putsreq_mongodb

The .env.docker file.

RAILS_ENV=production
MONGOLAB_URI=mongodb://db
REDIS_URL=redis://redis
DEVISE_SECRET_KEY=123
SECRET_TOKEN=123

External Docker volume FTW

The external volume referenced within the docker-compose.yml file has been created by invoking:

docker volume create --name=putsreq_mongodb

The rationale for creating the external volume is https://stackoverflow.com/questions/53870416/data-does-not-persist-to-host-volume-with-docker-compose-yml-for-mongodb. Otherwise, data stored in MongoDB might get lost as already observed by @ddavtian within #51.

Reverse-proxy configuration for Nginx

Last but not least, this Nginx snippet has been used for configuring a HTTP reverse proxy to the PutsReq instance:

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name putsreq.example.org;

    #include snippets/snakeoil.conf;
    include snippets/ssl/putsreq.example.org;

    proxy_buffering off;

    location / {
        proxy_set_header        Host               $http_host;
        proxy_set_header        X-Real-IP          $remote_addr;
        proxy_set_header        X-Forwarded-For    $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto  $scheme;
        proxy_pass              http://localhost:5050/;
    }

}

Ready-made Docker image on Docker Hub

The Docker image on https://hub.docker.com/r/daqzilla/putsreq has been amended using the patch putsreq-production.patch.txt.

It is not the most performant way to compile and serve assets like that on a production instance, precompiling and serving them from a webserver in a static manner should be preferred.

Outlook

Improving this quick & dirty production-configuration would be nice, pull requests are welcome. In order to make that possible, a) the image on Docker Hub should be republished without the amendments but with precompiled assets and b) the Nginx snippet should be adjusted to serve the assets in a static manner.

The Docker images published to https://hub.docker.com/u/daqzilla have been built like that:

# Acquire sources
git clone https://github.com/phstc/putsreq
cd putsreq

# Apply patch
wget https://github.com/phstc/putsreq/files/4554757/putsreq-production.patch.txt
patch -p1 < putsreq-production.patch.txt

# Build
docker build --tag daqzilla/putsreq:latest .

# Upload
docker login
docker push daqzilla/putsreq

License

Please see LICENSE for licensing details.

putsreq's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

putsreq's Issues

undefined method `strip' for nil:NilClass

I'm getting always this error

undefined method `strip' for nil:NilClass

when using request.forwardTo

This is the POST request i'm sending:

curl -i -X POST -d 'name=Pablo' http://putsreq.herokuapp.com/12345...

And this is the content of response

HTTP/1.1 500 Internal Server Error
Server: Cowboy
Connection: close
Date: Mon, 04 Aug 2014 08:14:11 GMT
Status: 500 Internal Server Error
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Ua-Compatible: chrome=1
Content-Type: text/plain
Cache-Control: no-cache
X-Request-Id: 7968c9b6-fb03-4d5c-8598-a0ca9885f282
X-Runtime: 0.041194
Via: 1.1 vegur

undefined method `strip' for nil:NilClass

This is the content on Putsreq

// Build a response
var msg = 'Hello World';

if(request.params.name) {
  msg = 'Hello ' + request.params.name;
}

response.body = msg;

// Forward a request
request.forwardTo = 'http://example.com/api';

If I remove the request.forwardTo everything works fine.

License information?

I am running a lab and would like to use this on my local network. In order to do that I need to fill out paperwork which states the license of the software I am using. Could you please provide a license file? If you don't have a license selected already I would prefer MIT or another academic-style license.

Putsreq down?

Hi,

It seems like Putsreq is down. I'm just seeing the standard Heroku error page.

screen shot 2018-03-28 at 14 45 17

Build failing

Hi Pablo!

The build is failing since 90565a5. Would you be able to fix it ?

Thanks,
Samuel

Add last_header page

Is possible to add a 'last_header' page to return the last header received?

Thank you!

Simple response mechanisms

You might consider providing a set of simple responses that
do not require using responsebuilder. Specifically

  1. allow automatic response http code: e.g. 200 or 404 or whatever.
  2. simple response body text.
    You might also allow http: access.

Issue while making ajax call with javascript

Hi.

I tried all the below code

    <!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Document</title>
  <script src="jquery.min.js"></script>
</head>

<body>

  <script src="jquery.min.js"></script>
  <script>
      const url = 'https://putsreq.com/bWVINoAIwR78cl7ajqhX';

      // Using Jquery

      jQuery.ajax({
        url: url,
        type: 'POST',
        dataType: 'json',
        contentType: 'application/json',
        processData: false,
        data: '{"foo":"bar"}',
        success: function (data) {
          console.error(JSON.stringify(data));
        },
        error: function () {
          console.error("Cannot get data");
        }
      });

      /*
      // Using Vannila JS

      let xhr = new XMLHttpRequest();
      xhr.open("POST", url);

      xhr.setRequestHeader("Content-Type", "application/json");

      xhr.onreadystatechange = function () {
         if (xhr.readyState === 4) {
            console.log(xhr.status);
            console.log(xhr.responseText);
         }};

      let data = '{"login":"my_login","password":"my_password"}';

      xhr.send(data);
      */

      /*
       // Using Fetch

      fetch(url, {
        method: 'POST', // or 'PUT'
        headers: {
          // 'Content-Type': 'application/x-www-form-urlencoded',
          'Content-Type': 'text/plain; charset=utf-8',
          // 'Content-Type': 'application/json',


        },
        body: JSON.stringify(data),
      })
        .then((response) => response.text())
        .then((data) => {
          console.log('Success:', data);
        })
        .catch((error) => {
          console.error('Error:', error);
        });
        */
    
  </script>
</body>

</html>

and i am getting response as

image

don't force SSL and / or update SSLv3 to TLS1.2 at least

The current image send a 301 - permanent moving to https wheh trying to connect with http but the SSL ask for SSLv3 which is no more available.
When used bahind a front like nginx / apache, SSL is useless.
This behavior should be de/activable using parameter (either docker command line or docker-compose env for example.
I tried to locate where to modify but didn't know ruby :(
I didn't found how to specify development instead of production by login into running container either

400 Error when posting invalid json

Hi !

When posting invalid JSON while using application/json content type, the server refuse the request.

Is it intentional ? You should probably accept any payload, and if possible display a warning in the UI that the payload is invalid. This way we would be possible to track what went wrong !

Thanks!

Duplicate buckets

Add a button to duplicate the Bucket copying the Response Builder.

Related to: #9

Create a readonly URL

Sometimes we want to share the PutsReq link to show the Response Builder or the received requests (for example for answers/questions on Stack Overflow), but we don't want other people to make new requests or change the response builder contents.

So the idea is to create a "shareable readonly link"

Assuming the original link is:

The readonly would be:

We can use something similar to:

We can also add a prefix in the readonly links to avoid ifs in the buckets#show action.

Dosent work with current ruversion of ruby 2.4.1

Getting error: Your Ruby version is 2.4.1, but your Gemfile specified 2.3.3
Tried changing ruby version in Gemfile but that led to dependency inconsistencies. I am not familiar with Ruby/Rails so i am not sure if that is to be done.
Any way this error can be solved?

hosted putsreq down?

Seems to go down quite a few times a day, anyway to mitigate the issue? Many thanks again for a such a great service.

๐Ÿ“Œ #45

Option to set delay for API response

I had the idea to add a parameter to the URL (or a setting in the admin) so responses could be delayed a specified amount of time. I'm testing an API right now that takes a few seconds to respond and this would be very useful.

data loss?

Hosting my own version locally and things are running as expected. After sending lots of traffic the web container stopped accepting traffic and to fix the issue I issued the following two commands

docker-compose down followed by docker-compose up

Containers came up but now it seems that the user I had created previously and all the buckets are gone. Is there a way to recover to perhaps a better way to approach this issue in the future as I had created many different buckets, etc.

Thanks

jQuery ($) is not recognized in putsreq.com

It seems like jquery is not loaded or something like that, because it is not identified in the ruby's javascript interpreter. To reproduce, just use the code from the readme.md:

$.get('https://putsreq.com', function(data) {
  alert(data.message);
  // => 'Hello World'
});

It will break when you try to access the bucket with response:

$ is not defined

Permalinks to requests/responses

Hey,

It would be nice if we could have permalinks to Putsreq requests and responses, like so:

https://putsreq.herokuapp.com/:bucket_id/requests/:request_id
https://putsreq.herokuapp.com/:bucket_id/response/:response_id

So we could share them with each other.

I might give it a try later ๐Ÿป

putsreq app not starting

Thank you for a great application. Given the latest issues with the hosted version going up/down frequently throughout the day I decided to run my own. I ran into the following issue, hoping that you can help?

docker-compose up -d

docker logs -f e82bddc68903
bin/rails: Devise.secret_key was not set. Please add the following to your Devise initializer: (RuntimeError)

  config.secret_key = '29f0704f974f5321371f4cc3bce47f68534fa70b5d48db5f1aaf74185c3374d7459274418b8f0594a38d614fc59fe5f9dee72ffe383e906dd06971985a5bf649'

Please ensure you restarted your application after installing Devise or setting the key.
=> Booting Puma
=> Rails 5.1.7 application starting in development
=> Run `rails server -h` for more startup options
Exiting

๐Ÿ“Œ #47

Running PutsReq in production through Docker

Dear Pablo,

first things first: Thank you so much for conceiving and maintaining PutsReq. @thiasB started using it and now we would to ramp up a dedicated instance for our beehive monitoring project, see https://hiveeyes.org/ and https://community.hiveeyes.org/.

We need it to transform HTTP requests originating from TTN to converge properly to our data acquisition backend running Kotori. @thiasB documented this specific telemetry path at TTN/LoRaWAN setup for Terkin, enjoy!

We have been able to get started with it already but humbly ask for further guidance to get things right.

With kind regards,
Andreas.

Create a cache counter for requests count

As the issue #5 delete requests, the "Requests Recorded: 3144" on the bottom of the PutsReq pages isn't accurate anymore. So we need to cache the counter in another collection i.e. PutsReqStats.increment('requests.count'); PutsReqStats.get('requests.count');

Problem using custom headers

When I use the header X-TLRG.force-error it does not appear in the Request Headers list when using putsreq.com

If I run putsreq on my local machine the header does appear in the list. Any idea why there is the difference?

Is it possible to call request.forwardTo twice?

Many thanks, Pablo for developing PutsReq! It helps a lot to re-write e.g. TTN payloads for different APIs! Today we tried to send data that arrives at PutsReq to two different Servers / APIs in one Response Builder. But we struggled! Seems it is not possible to call request.forwardTo twice?

We did this here:

const input   = JSON.parse(request.body);

// -----
// B O B
// -----

const payload_ttn = input.payload_fields;

var output_bob   = {};

[doing cool stuff here]

request.body = output_bob;
request.forwardTo = 'https://example-server-1.org';

// ---------------
// H I V E E Y E S
// ---------------

var output_swarm = {};

[doing other cool stuff here]

const URL = "https://example-server-2.org/api/" + input.dev_id.replace(/-/g, '/') + "/data";

request.body = output_swarm;
request.forwardTo = URL;

Do you have a hint how we can realize to send different re-written data structurs to two different endpoints with PutsReq? Is it not possible by design or do we miss something or our syntax is wrong?

Destroy button?

Hey @rdgr any ideas to make the "destroy" more user-friendly?

Another thing - A friend of mine thought that the Bucket Name field was readonly - only to make easier to copy the token. Can we make it more explicit that users can change it? Maybe it was an isolated case?!

Content-Type unexpected transformation

I'm the developer of a software (Orion Context Broker) that as part of its functionality sends notifications in HTTP/HTTPS as the following one (real stuff, not crafted, i.e. copy&pasted from a local netcat):

POST http://127.0.0.1:9997/notify?entityId=Lamp1
Fiware-Servicepath: /
Ngsiv2-Attrsformat: custom
Content-Length: 24
X-Auth-Token: 8472e87bf49c4bba11f731aed6ab12a2
User-Agent: orion/1.9.0 libcurl/7.38.0
X-Entitytype: Lamp
Host: 127.0.0.1:9997
Accept: application/json
Fiware-Service: formacion
Content-Type: application/xml
Fiware-Correlator: 48305f2a-fae2-11e7-ae1b-000c29173617

<battery>-10.2</battery>

I'm testing with PutsReq but what I get is the following set of headers from PutsReq inspect page:

{
  "VERSION": "HTTP/1.1",
  "CONNECTION": "close",
  "ACCEPT-ENCODING": "gzip",
  "USER-AGENT": "orion/1.9.0 libcurl/7.38.0",
  "FIWARE-SERVICE": "formacion",
  "FIWARE-SERVICEPATH": "/",
  "X-AUTH-TOKEN": "8472e87bf49c4bba11f731aed6ab12a2",
  "ACCEPT": "application/json",
  "CONTENT-TYPE": "text/plain; charset=utf-8,application/xml",
  "FIWARE-CORRELATOR": "a220284e-fae2-11e7-ae1b-000c29173617",
  "NGSIV2-ATTRSFORMAT": "custom",
  "X-ENTITYTYPE": "Lamp",
  "CONTENT-LENGTH": "24"
}

All seems to be ok, except Content-Type, which has changed to the (syntactically wrong): "text/plain; charset=utf-8,application/xml" instead of showing what it has received, i.e. "application/xml".

It seems that PutsReq is not showing Content-Type header its receives. Is this a normal PutsReq behaviour? Is a bug?

(Fiware-Correlator is unique per HTTP request so it is normal it changes)

Add ability to edit the bucket name

It is hard to find the correct Bucket in the history without a name:

An option to edit the bucket name, like GitHub does with repositories would be very useful.

When no name (but without Website):

When name (link to Edit)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.