danwrong/restler

REST client library for node.js

restler's Issues

What is the proper way to specify content type header from my application?

When posting json to remote service I need to set 'Content-Type' to 'application/json'. For example shopify api will decline json requests without 'Content-Type' header set to 'application/json'. So, how to do it properly in my application? :)

Thanks.

Content-Length header in multipart upload

Hi,

It looks like restler is leaving off the main Content-Length header when doing multi-part uploads (even though it is setting Content-Length headers for individual files).

Nginx is rejecting multipart uploads made from restler because the header is missing.

Node.js deprecated process.mixin, breaks restler

http://github.com/ry/node/commit/c1613e6ebb37a74b103e600467c89e3df15d005c

Missing return statement

Hey Dan,

I already mentioned it in comments but in case you missed it:

http://github.com/danwrong/restler/blob/master/lib/restler.js#L112

is missing a return statement.

Thanks,

Jacek

npm

The module is not published.

should list zlib as a dependency or bump required version of Node.

Restler has a requirement on zlib, which is not part of the currently listed minimum version of Node required to use Restler.

Bug in querystring.js

line 46 in querystring.js is
name = name+'[]';
it should be
name = name+'[';
and inside of the for loop should be
var tn = name+i+']';
s.push( QueryString.stringify(obj[i], sep, eq, tn) );

MultiPart does not work, parameters used wrong

From multipartform.js:
189: write: function(stream, data, boundary, callback) {

From restler.js:
146: multipart.write(this.request, this.options.data, function() {

Changes resler.js:
146: multipart.write(this.request, this.options.data, multipart.defaultBoundary, function() {

Without the changes the boundary set in the header will be the defaultBounary by multipart and it will be send with the function used as a boundary like:
--function () { self.request.end(); }--

Wireshark:
0110 73 65 0d 0a 0d 0a 36 38 0d 0a 2d 2d 66 75 6e 63 se....68 ..--func
0120 74 69 6f 6e 20 28 29 20 7b 0a 20 20 20 20 20 20 tion () {.
0130 20 20 73 65 6c 66 2e 72 65 71 75 65 73 74 2e 65 self.r equest.e
0140 6e 64 28 29 3b 0a 20 20 20 20 20 20 7d 0d 0a 43 nd();. }..C
0150 6f 6e 74 65 6e 74 2d 44 69 73 70 6f 73 69 74 69 ontent-D ispositi

support xml2js-expat as well

I came up with some issues when using xml2js that were fixed when switching to xml2js-expat instead and since both libraries support basically the same api why not support them both?

https://github.com/Poetro/node-xml2js-expat

restler should support HTTP HEAD method

Why not?

Ignore and Delete

Issue is from our end, looks like restler API had changed drastically, ignore and close issue

Adding an example with custom parser to README

Hi!

What about that?

BTW, in the README:

See parsers section below. 

probably should read:

See parsers section above.

Regards,

Włodek Bzyl

Any request resulting in the error throws Uncaught exception

Any request resulting in the error code from the server side to client running node.js 0.6.0 and restler results in the following (or similar stack) trace:

node restler_test.js 
The "sys" module is now called "util". It should have a similar interface.

events.js:50
        throw new Error("Uncaught, unspecified 'error' event.");
              ^
Error: Uncaught, unspecified 'error' event.
    at EventEmitter.emit (events.js:50:15)
    at EventEmitter._respond (/Users/artem/.node_libraries/restler/lib/restler.js:127:12)
    at EventEmitter._fireEvents (/Users/artem/.node_libraries/restler/lib/restler.js:131:52)
    at /Users/artem/.node_libraries/restler/lib/restler.js:115:19
    at IncomingMessage.<anonymous> (/Users/artem/.node_libraries/restler/lib/restler.js:208:5)
    at IncomingMessage.<anonymous> (/Users/artem/.node_libraries/restler/lib/restler.js:200:49)
    at IncomingMessage.<anonymous> (/Users/artem/.node_libraries/restler/lib/restler.js:113:32)
    at IncomingMessage.emit (events.js:88:20)
    at HTTPParser.onMessageComplete (http.js:137:23)
    at Socket.ondata (http.js:1125:24)

Investigation showed that the problem is with the event name and is probably similar to (or same as) socketio/socket.io#476

For example changing

  _fireEvents: function(body, response) {
    if (parseInt(response.statusCode) >= 400) this._respond('error', body, response);

to

  _fireEvents: function(body, response) {
    if (parseInt(response.statusCode) >= 400) this._respond('restlererror', body, response);

Makes everything work again, but of course event is incorrect then

Requesting an invalid domain returns some weird data

The following code:

    restler.get('http://invaliddomainname/').addListener('success',
        function(data) {
            console.log(data);
        }
    );

prints this weird shit to the console:

<html>
    <head>
        <title>  </title>
        <script type="text/javascript">
        function bredir(d,u,r,v,c){var w,h,wd,hd,bi;var b=false;var p=false;var s=[[300,250,false],[250,250,false],[240,400,false],[336,280,false],[180,150,false],[468,60,false],[234,60,false],[88,31,false],[120,90,false],[120,60,false],[120,240,false],[125,125,false],[728,90,false],[160,600,false],[120,600,false],[300,600,false],[300,125,false],[530,300,false],[190,200,false],[470,250,false],[720,300,true],[500,350,true],[550,480,true]];if(typeof(window.innerHeight)=='number'){h=window.innerHeight;w=window.innerWidth;}else if(typeof(document.body.offsetHeight)=='number'){h=document.body.offsetHeight;w=document.body.offsetWidth;}for(var i=0;i<s.length;i++){bi=s[i];wd=Math.abs(w-bi[0]);hd=Math.abs(h-bi[1]);if(wd<=2&&hd<=2){b=true;p=bi[2];}}if(b||(w<100&&w!==0)||(h<100&&h!==0)){if(p&&self==parent){self.close();return;}return'/b'+'anner.php?w='+w+'&h='+h+'&d='+d+'&u='+u+'&r='+r+'&view='+v;}else{return c;}}
        </script>
    </head>
    <body onLoad="window.location = bredir('', 'invaliddomainname', '', 'error', '/main?url=invaliddomainname');" style="margin: 0px;">
        <noscript>
            <iframe frameborder="0" src="/main?url=invaliddomainname" width="100%" height="100%"></iframe>
        </noscript>
    </body>
</html>

What the hell is going on?

http digest authentication not supported

Do you have any plans to support this auth mode?

get method causes 400 errors

data {} should be sent via query string on get requests. It seems to be POSTing the data regardless of method? Both rest.service and rest.get currently need to have data set to null to get around this problem.

Restler 2.x hangs on 403

Hi,

If I try a GET request on a url which returns a 403 Forbidden, Restler hangs. All was ok with 0.2.x

Is this due to a change in API? eg.

testUrl = function(url, callback) {
return restler.get(url).on('success', function(data, resp) {
  return callback(null, data);
}).on('error', function(data, resp) {
  return callback(true, data);
});
 };

With 0.2.x the error event is successfully emitted but on 2.x it isnt. Is this due to an API change or is it a bug?

Implement Digest Authorization

Might have a go at it myself, but I'm unsure when I'll have time to do it and there's likely more qualified people around here that might give it a go, so here's a ticket for it ;)

Running restler behind a proxy

It seems that it's not possible to issue requests if nodejs runs behind a (non-transparent) proxy. This is won't play well in corporate environments for instance.
Is there a solution for configuring a proxy server IP/port in Restler?

JSON parser should be used on custom JSON mime types

By convention, a custom mime type ending in +JSON should still use the JSON parser. E.g.

application/vnd.[MY DOC NAME]+JSON

This should not require setting a parser manually.

Getting "Uncaught, unspecified 'error' event."

I'm getting a node error when the host is unavailable. It appears that restler doesn't quite know what to do with an 'error' event?

node.js:134
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: Uncaught, unspecified 'error' event.
at EventEmitter.emit (events.js:47:15)
at EventEmitter._respond (/home/dbrogdon/stadium/node_modules/restler/lib/restler.js:127:12)

connection reset results in no response

Running curl on http://www.westnipissingouest.ca results in the following:

* Recv failure: Connection reset by peer
* Closing connection #0

The following code will simply hang.

var rest = require('./')

var done = false
rest.get('http://www.westnipissingouest.ca').on('complete', function(data, err) {
  done = true
  sys.puts(data)
  sys.puts(err)
})

;(function loopUntilDone() {
  if (!done) process.nextTick(loopUntilDone)
})()

Test Failing

Running the tests gives me:

module:238
    throw new Error("Cannot find module '" + request + "'");
          ^
Error: Cannot find module 'mjsunit'
at loadModule (module:238:15)
at require (module:364:12)
at Object.<anonymous> (/Users/jacob/code/python/appengine/icebreaker/utils/restler/test/test_helper.js:1:74)
at Module._compile (module:384:23)
at Module._loadScriptSync (module:393:8)
at Module.loadSync (module:296:10)
at loadModule (module:241:16)
at require (module:364:12)
at Object.<anonymous> (/Users/jacob/code/python/appengine/icebreaker/utils/restler/test/restler.js:1:76)
at Module._compile (module:384:23)

text/javascript Content-Type

I was briefly puzzled that when getting data from facebook the "auto" parser didn't work.

Turns out that facebook is sending the content-type as 'text/javascript; charset=UTF-8'

A little bit old-school but the 'auto' type might want to consider this to be one of the JSON content-types.

I guess it depends on whether or not it's more common that people fetch JSON data with this (incorrect) content-type or if they would sometimes use restler to download actual javascript code which they do not want to parse to JSON.

Update npm package

Can you update the npm package so I can get the change from require('sys') to require('util')?

Thanks,

Chris

Redirect after POST

A common response to POST messages is to redirect to another page, and the PRG pattern requires the request type to be changed to GET in this case (http://en.wikipedia.org/wiki/Post/Redirect/Get). Could you change this please?

Token authentication

Is there any method to implement token authentication for all http verbs? (GET, POST,PUT, DELETE)

HTTP 303

Hi,

Looks like the code currently defines redirects as being only 301 and 302. But 303 is very commonly used, so ought to be supported as a standard form of redirect.

L.

JSON not being automatically parsed

I recently updated to the latest version of Restler (a3dbabd) and it appears JSON responses are not being deserialized as such.

In my complete handlers, the data object is always coming back as a string, meaning I manually need to call JSON.parse on it.

I was previously using 54cd767, and the JSON was being parsed automatically by Restler.

I will try digging into this, but perhaps you have an idea of what recent change caused this?

when using restler to download binary files it corrupts them.

I'm trying to use restler to download binary files and they get corrupted in the reading process. Setting https://github.com/danwrong/restler/blob/master/lib/restler.js#L111 to 'binary' instead of 'utf8' solves the problem but I agree it has to be implemented in a better way in order to handle different encodings.

would be nice to be able to use no options on a Service

This will break down with 'Object.keys called on non-object':

var someService = new restler.Service({});
someService.get('someurl');

---

Will work:

var someService = new restler.Service({});
someService.get('someurl', {});

---

function mixin(target, source) {
if (!source)
return;

Object.keys(source).forEach(function(key) {
target[key] = source[key];
});

return target;
}

Appears to solve the problem, but a more elegant way could probably be found. Underscore's extend method does not have this problem and could also be an option.

make complete run every time also on error

make complete run every time also on error
to make impossible to loose callback

Only one service at a time?

It seems I can only have one service at a time. I'm trying to make a scraper that pulls from one API and pushes into another, but when I do the second rest.service() call it seems to overwrite the first service I made!

var rest = require('restler')
function noop () {}
var TestA = rest.service(noop, {}, {
  log: function () {
    console.log('first')
  }
})

var TestB = rest.service(noop, {}, {
  log: function () {
    console.log('second')
  }
})

var testa = new TestA
testa.log()

var testb = new TestB
testb.log()

Support Digest Authentication

There already is a library for enabling Digest Auth with http.Client().
It would be great to implement this into restler.

Option to specify timeout

I couldn't find a way to specify a timeout for the requests that's different from node.js's default (60s).

It would be great if 'timeout' were one of the 'options'.

Node exits silently...

Current node.js exits without printing anything when fed this:

// http://github.com/ry/node/commit/a2d809fe902f6c4102dba8f2e3e9551aad137c0f
// http://github.com/danwrong/restler/commit/b3ac694b086761614c6402c32f374aa9898c1494
var sys = require("sys"),
  restler = require('./restler');

sys.puts("Doesn't print it"); 

Default query string parameters

In the readme, there is an example of using rest.Service() to create a wrapper for Twitter. It assigns to this.defaults.username and this.defaults.password.

I need to set a default querystring parameter. There are lots of APIs that take an AppId or ClientId as a querystring parameter on every call.

I tried this:

... rest.service(function(appid) {
  this.defaults.query = { appid : appid };
}, { ...

Which does set the query property, but the mixin function overrides it when I call this.get('/theurl', options) in my wrapper function. where options has a query property.

Ideally, the mixin function would be applied to the query property as well as its parent options property.

error calling get on service

Calling get on a service results in a fatal error. I'm not sure if I misunderstand the purpose of the service constructor, or if this is a bug. Should I be able to call get/post/etc directly?

Failing code slightly adapted from service example:

var sys = require('util'),
    rest = require('restler');
// create a service constructor for very easy API wrappers a la HTTParty...
Twitter = rest.service(function(u, p) {
  this.defaults.username = u;
  this.defaults.password = p;
}, {
  baseURL: 'http://twitter.com'
}, {
  update: function(message) {
    return this.post('/statuses/update.json', { data: { status: message } });
  }
});

var client = new Twitter('danwrong', 'password');
client.get('/').on('complete', function(data) {
  sys.p(data);
});

node twittest.js                              

node.js:201
        throw e; // process.nextTick error, or 'error' event on first tick
              ^
TypeError: Object.keys called on non-object
    at Function.keys (native)
    at mixin (./node_modules/restler/lib/restler.js:10:10)
    at Service._withDefaults (./node_modules/restler/lib/restler.js:320:12)
    at Service.get ./node_modules/restler/lib/restler.js:303:38)
    at Object.<anonymous> (./twittest.js:16:8)
    at Module._compile (module.js:432:26)
    at Object..js (module.js:450:10)
    at Module.load (module.js:351:31)
    at Function._load (module.js:310:12)
    at Array.0 (module.js:470:10)

node 0.6.2
restler 0.2.4

Port number missing in Host header

When the port number is anything other than 80, according to the spec, it must be included in the Host header:

A "host" without any trailing port information implies the default port for the service requested (e.g., "80" for an HTTP URL)

In practice, most clients include the port number regardless.

No way to cancel a request

Sometimes you need to cancel a request and guarantees that the request callback does not get executed.

Here is an implementation (note that in Node 0.5 builds, but not 0.4.x, removeAllListeners with no arguments can be used instead of manually manipulating _events), however it is apparently incorrect as I find the callback still gets executed and I need to manually check the .cancelled key in my callback:

restler.Request.prototype.cancel = function() {
    if (this.options.originalRequest) {
      this.options.originalRequest._events = {};
    }
    this._events = {};
    this.cancelled = true;
    return;
  };

Does RESTler do some kind of internal queueing or something that is preventing this from working?

Typo in example for basic auth

You've got the following in your example:

data: {
  username: 'danwrong',
  password: 'wouldntyouliketoknow',
  'sound[message]': 'hello from restler!',
  'sound[file]': rest.file('doug-e-fresh_the-show.mp3', 'audio/mpeg')
}

which should read:
username: 'danwrong',
password: 'wouldntyouliketoknow',
data: {
'sound[message]': 'hello from restler!',
'sound[file]': rest.file('doug-e-fresh_the-show.mp3', 'audio/mpeg')
}

https context not created correctly

-FYI In order to get https working I had to replace line 61 restler.js:
//this.client.setSecure('X509_PEM');
this.client.setSecure();
This Allows node to create the context. With latest version of both node and restler.
Cheers.
Peter.

Take into account endless redirects

If restler is presented with an endless redirect (301) loop, it will just run forever. It should cap redirects at some limit, and emit an error if it hits that limit.

Emitting xml2js error in console

A call somewhere is causing the following error to be continually emitted in stdout:

{ stack: [Getter/Setter],
  arguments: undefined,
  type: undefined,
  message: 'Cannot find module \'xml2js\'' }

Options for the xml2js parser

Hi Dan,

I'd like to hand over something other than the default options to the xml2js constructor, like this.

instead of:

var parser = new xml2js.Parser();

I'd like to have it do

var parser = new xml2js.Parser({normalize: false});

Would it be possible to somehow give restler the options object so it constructs xml2js with it?

(The normalizer otherwise destroys valueable whitespace, that's why...)

Regards,
Joris

npm

Despite the presence of a package.json file, it seems that the module is not actually published.

HTTPS request ?

Can we send HTTPS request using restler ??

needs to be updated to new api in node 1.29

'error' event on http.Client not caught

When an HTTP request fails, for example with connection refused (ECONNREFUSED), an 'error' event is triggered on the HTTP client object, not the request object.

This means 'error' events on the client should be caught and reported through the 'error' callback.

Reference: nodejs/node-v0.x-archive#136

unspecified error on "complete" - only will return body with "error"

I am getting a strange error if I use addListener("complete") instead of addListener("error") on a post call. I thought that complete catches both errors and successes. Here is the error:

Error: Uncaught, unspecified 'error' event.
at EventEmitter.emit (events:14:15)
at EventEmitter._respond (/usr/local/lib/node/.npm/restler/1.0.0/package/lib/restler.js:101:15)
at IncomingMessage.<anonymous> (/usr/local/lib/node/.npm/restler/1.0.0/package/lib/restler.js:90:56)
at IncomingMessage.emit (events:48:20)
at HTTPParser.onMessageComplete (http:107:23)
at Client.onData [as ondata] (http:854:27)
at IOWatcher.callback (net:494:29)
at node.js:773:9