danwrong / restler Goto Github PK
View Code? Open in Web Editor NEWREST client library for node.js
License: MIT License
REST client library for node.js
License: MIT License
When posting json to remote service I need to set 'Content-Type' to 'application/json'. For example shopify api will decline json requests without 'Content-Type' header set to 'application/json'. So, how to do it properly in my application? :)
Thanks.
The following code:
restler.get('http://invaliddomainname/').addListener('success',
function(data) {
console.log(data);
}
);
prints this weird shit to the console:
<html>
<head>
<title> </title>
<script type="text/javascript">
function bredir(d,u,r,v,c){var w,h,wd,hd,bi;var b=false;var p=false;var s=[[300,250,false],[250,250,false],[240,400,false],[336,280,false],[180,150,false],[468,60,false],[234,60,false],[88,31,false],[120,90,false],[120,60,false],[120,240,false],[125,125,false],[728,90,false],[160,600,false],[120,600,false],[300,600,false],[300,125,false],[530,300,false],[190,200,false],[470,250,false],[720,300,true],[500,350,true],[550,480,true]];if(typeof(window.innerHeight)=='number'){h=window.innerHeight;w=window.innerWidth;}else if(typeof(document.body.offsetHeight)=='number'){h=document.body.offsetHeight;w=document.body.offsetWidth;}for(var i=0;i<s.length;i++){bi=s[i];wd=Math.abs(w-bi[0]);hd=Math.abs(h-bi[1]);if(wd<=2&&hd<=2){b=true;p=bi[2];}}if(b||(w<100&&w!==0)||(h<100&&h!==0)){if(p&&self==parent){self.close();return;}return'/b'+'anner.php?w='+w+'&h='+h+'&d='+d+'&u='+u+'&r='+r+'&view='+v;}else{return c;}}
</script>
</head>
<body onLoad="window.location = bredir('', 'invaliddomainname', '', 'error', '/main?url=invaliddomainname');" style="margin: 0px;">
<noscript>
<iframe frameborder="0" src="/main?url=invaliddomainname" width="100%" height="100%"></iframe>
</noscript>
</body>
</html>
What the hell is going on?
Restler has a requirement on zlib, which is not part of the currently listed minimum version of Node required to use Restler.
Any request resulting in the error code from the server side to client running node.js 0.6.0 and restler results in the following (or similar stack) trace:
node restler_test.js
The "sys" module is now called "util". It should have a similar interface.
events.js:50
throw new Error("Uncaught, unspecified 'error' event.");
^
Error: Uncaught, unspecified 'error' event.
at EventEmitter.emit (events.js:50:15)
at EventEmitter._respond (/Users/artem/.node_libraries/restler/lib/restler.js:127:12)
at EventEmitter._fireEvents (/Users/artem/.node_libraries/restler/lib/restler.js:131:52)
at /Users/artem/.node_libraries/restler/lib/restler.js:115:19
at IncomingMessage.<anonymous> (/Users/artem/.node_libraries/restler/lib/restler.js:208:5)
at IncomingMessage.<anonymous> (/Users/artem/.node_libraries/restler/lib/restler.js:200:49)
at IncomingMessage.<anonymous> (/Users/artem/.node_libraries/restler/lib/restler.js:113:32)
at IncomingMessage.emit (events.js:88:20)
at HTTPParser.onMessageComplete (http.js:137:23)
at Socket.ondata (http.js:1125:24)
Investigation showed that the problem is with the event name and is probably similar to (or same as) socketio/socket.io#476
For example changing
_fireEvents: function(body, response) {
if (parseInt(response.statusCode) >= 400) this._respond('error', body, response);
to
_fireEvents: function(body, response) {
if (parseInt(response.statusCode) >= 400) this._respond('restlererror', body, response);
Makes everything work again, but of course event is incorrect then
By convention, a custom mime type ending in +JSON should still use the JSON parser. E.g.
application/vnd.[MY DOC NAME]+JSON
This should not require setting a parser manually.
Current node.js exits without printing anything when fed this:
// http://github.com/ry/node/commit/a2d809fe902f6c4102dba8f2e3e9551aad137c0f
// http://github.com/danwrong/restler/commit/b3ac694b086761614c6402c32f374aa9898c1494
var sys = require("sys"),
restler = require('./restler');
sys.puts("Doesn't print it");
I'm trying to use restler to download binary files and they get corrupted in the reading process. Setting https://github.com/danwrong/restler/blob/master/lib/restler.js#L111 to 'binary' instead of 'utf8' solves the problem but I agree it has to be implemented in a better way in order to handle different encodings.
From multipartform.js:
189: write: function(stream, data, boundary, callback) {
From restler.js:
146: multipart.write(this.request, this.options.data, function() {
Changes resler.js:
146: multipart.write(this.request, this.options.data, multipart.defaultBoundary, function() {
Without the changes the boundary set in the header will be the defaultBounary by multipart and it will be send with the function used as a boundary like:
--function () { self.request.end(); }--
Wireshark:
0110 73 65 0d 0a 0d 0a 36 38 0d 0a 2d 2d 66 75 6e 63 se....68 ..--func
0120 74 69 6f 6e 20 28 29 20 7b 0a 20 20 20 20 20 20 tion () {.
0130 20 20 73 65 6c 66 2e 72 65 71 75 65 73 74 2e 65 self.r equest.e
0140 6e 64 28 29 3b 0a 20 20 20 20 20 20 7d 0d 0a 43 nd();. }..C
0150 6f 6e 74 65 6e 74 2d 44 69 73 70 6f 73 69 74 69 ontent-D ispositi
Hi,
Looks like the code currently defines redirects as being only 301 and 302. But 303 is very commonly used, so ought to be supported as a standard form of redirect.
L.
The module is not published.
Sometimes you need to cancel a request and guarantees that the request callback does not get executed.
Here is an implementation (note that in Node 0.5 builds, but not 0.4.x, removeAllListeners with no arguments can be used instead of manually manipulating _events), however it is apparently incorrect as I find the callback still gets executed and I need to manually check the .cancelled key in my callback:
restler.Request.prototype.cancel = function() {
if (this.options.originalRequest) {
this.options.originalRequest._events = {};
}
this._events = {};
this.cancelled = true;
return;
};
Does RESTler do some kind of internal queueing or something that is preventing this from working?
A call somewhere is causing the following error to be continually emitted in stdout:
{ stack: [Getter/Setter],
arguments: undefined,
type: undefined,
message: 'Cannot find module \'xml2js\'' }
Hey Dan,
I already mentioned it in comments but in case you missed it:
http://github.com/danwrong/restler/blob/master/lib/restler.js#L112
is missing a return statement.
Thanks,
Jacek
Do you have any plans to support this auth mode?
I came up with some issues when using xml2js that were fixed when switching to xml2js-expat instead and since both libraries support basically the same api why not support them both?
Running the tests gives me:
module:238
throw new Error("Cannot find module '" + request + "'");
^
Error: Cannot find module 'mjsunit'
at loadModule (module:238:15)
at require (module:364:12)
at Object.<anonymous> (/Users/jacob/code/python/appengine/icebreaker/utils/restler/test/test_helper.js:1:74)
at Module._compile (module:384:23)
at Module._loadScriptSync (module:393:8)
at Module.loadSync (module:296:10)
at loadModule (module:241:16)
at require (module:364:12)
at Object.<anonymous> (/Users/jacob/code/python/appengine/icebreaker/utils/restler/test/restler.js:1:76)
at Module._compile (module:384:23)
I recently updated to the latest version of Restler (a3dbabd) and it appears JSON responses are not being deserialized as such.
In my complete handlers, the data
object is always coming back as a string, meaning I manually need to call JSON.parse
on it.
I was previously using 54cd767, and the JSON was being parsed automatically by Restler.
I will try digging into this, but perhaps you have an idea of what recent change caused this?
A common response to POST messages is to redirect to another page, and the PRG pattern requires the request type to be changed to GET in this case (http://en.wikipedia.org/wiki/Post/Redirect/Get). Could you change this please?
Can we send HTTPS request using restler ??
Running curl on http://www.westnipissingouest.ca results in the following:
* Recv failure: Connection reset by peer
* Closing connection #0
The following code will simply hang.
var rest = require('./')
var done = false
rest.get('http://www.westnipissingouest.ca').on('complete', function(data, err) {
done = true
sys.puts(data)
sys.puts(err)
})
;(function loopUntilDone() {
if (!done) process.nextTick(loopUntilDone)
})()
There already is a library for enabling Digest Auth with http.Client().
It would be great to implement this into restler.
When the port number is anything other than 80, according to the spec, it must be included in the Host
header:
A "host" without any trailing port information implies the default port for the service requested (e.g., "80" for an HTTP URL)
In practice, most clients include the port number regardless.
Hi,
If I try a GET request on a url which returns a 403 Forbidden, Restler hangs. All was ok with 0.2.x
Is this due to a change in API? eg.
testUrl = function(url, callback) {
return restler.get(url).on('success', function(data, resp) {
return callback(null, data);
}).on('error', function(data, resp) {
return callback(true, data);
});
};
With 0.2.x the error event is successfully emitted but on 2.x it isnt. Is this due to an API change or is it a bug?
Hi!
What about that?
BTW, in the README:
See parsers section below.
probably should read:
See parsers section above.
Regards,
Włodek Bzyl
Hi Dan,
I'd like to hand over something other than the default options to the xml2js constructor, like this.
instead of:
var parser = new xml2js.Parser();
I'd like to have it do
var parser = new xml2js.Parser({normalize: false});
Would it be possible to somehow give restler the options object so it constructs xml2js with it?
(The normalizer otherwise destroys valueable whitespace, that's why...)
Regards,
Joris
-FYI In order to get https working I had to replace line 61 restler.js:
//this.client.setSecure('X509_PEM');
this.client.setSecure();
This Allows node to create the context. With latest version of both node and restler.
Cheers.
Peter.
make complete run every time also on error
to make impossible to loose callback
I'm getting a node error when the host is unavailable. It appears that restler doesn't quite know what to do with an 'error' event?
node.js:134
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: Uncaught, unspecified 'error' event.
at EventEmitter.emit (events.js:47:15)
at EventEmitter._respond (/home/dbrogdon/stadium/node_modules/restler/lib/restler.js:127:12)
I am getting a strange error if I use addListener("complete") instead of addListener("error") on a post call. I thought that complete catches both errors and successes. Here is the error:
Error: Uncaught, unspecified 'error' event.
at EventEmitter.emit (events:14:15)
at EventEmitter._respond (/usr/local/lib/node/.npm/restler/1.0.0/package/lib/restler.js:101:15)
at IncomingMessage.<anonymous> (/usr/local/lib/node/.npm/restler/1.0.0/package/lib/restler.js:90:56)
at IncomingMessage.emit (events:48:20)
at HTTPParser.onMessageComplete (http:107:23)
at Client.onData [as ondata] (http:854:27)
at IOWatcher.callback (net:494:29)
at node.js:773:9
Can you update the npm package so I can get the change from require('sys') to require('util')?
Thanks,
Chris
line 46 in querystring.js is
name = name+'[]';
it should be
name = name+'[';
and inside of the for loop should be
var tn = name+i+']';
s.push( QueryString.stringify(obj[i], sep, eq, tn) );
You've got the following in your example:
data: {
username: 'danwrong',
password: 'wouldntyouliketoknow',
'sound[message]': 'hello from restler!',
'sound[file]': rest.file('doug-e-fresh_the-show.mp3', 'audio/mpeg')
}
which should read:
username: 'danwrong',
password: 'wouldntyouliketoknow',
data: {
'sound[message]': 'hello from restler!',
'sound[file]': rest.file('doug-e-fresh_the-show.mp3', 'audio/mpeg')
}
Hi,
It looks like restler is leaving off the main Content-Length header when doing multi-part uploads (even though it is setting Content-Length headers for individual files).
Nginx is rejecting multipart uploads made from restler because the header is missing.
This will break down with 'Object.keys called on non-object':
var someService = new restler.Service({});
someService.get('someurl');
Will work:
var someService = new restler.Service({});
someService.get('someurl', {});
function mixin(target, source) {
if (!source)
return;
Object.keys(source).forEach(function(key) {
target[key] = source[key];
});
return target;
}
Appears to solve the problem, but a more elegant way could probably be found. Underscore's extend method does not have this problem and could also be an option.
Calling get on a service results in a fatal error. I'm not sure if I misunderstand the purpose of the service constructor, or if this is a bug. Should I be able to call get/post/etc directly?
Failing code slightly adapted from service example:
var sys = require('util'),
rest = require('restler');
// create a service constructor for very easy API wrappers a la HTTParty...
Twitter = rest.service(function(u, p) {
this.defaults.username = u;
this.defaults.password = p;
}, {
baseURL: 'http://twitter.com'
}, {
update: function(message) {
return this.post('/statuses/update.json', { data: { status: message } });
}
});
var client = new Twitter('danwrong', 'password');
client.get('/').on('complete', function(data) {
sys.p(data);
});
node twittest.js
node.js:201
throw e; // process.nextTick error, or 'error' event on first tick
^
TypeError: Object.keys called on non-object
at Function.keys (native)
at mixin (./node_modules/restler/lib/restler.js:10:10)
at Service._withDefaults (./node_modules/restler/lib/restler.js:320:12)
at Service.get ./node_modules/restler/lib/restler.js:303:38)
at Object.<anonymous> (./twittest.js:16:8)
at Module._compile (module.js:432:26)
at Object..js (module.js:450:10)
at Module.load (module.js:351:31)
at Function._load (module.js:310:12)
at Array.0 (module.js:470:10)
node 0.6.2
restler 0.2.4
If restler is presented with an endless redirect (301) loop, it will just run forever. It should cap redirects at some limit, and emit an error if it hits that limit.
Despite the presence of a package.json file, it seems that the module is not actually published.
I was briefly puzzled that when getting data from facebook the "auto" parser didn't work.
Turns out that facebook is sending the content-type as 'text/javascript; charset=UTF-8'
A little bit old-school but the 'auto' type might want to consider this to be one of the JSON content-types.
I guess it depends on whether or not it's more common that people fetch JSON data with this (incorrect) content-type or if they would sometimes use restler to download actual javascript code which they do not want to parse to JSON.
When an HTTP request fails, for example with connection refused (ECONNREFUSED), an 'error' event is triggered on the HTTP client object, not the request object.
This means 'error' events on the client should be caught and reported through the 'error' callback.
Reference: nodejs/node-v0.x-archive#136
Might have a go at it myself, but I'm unsure when I'll have time to do it and there's likely more qualified people around here that might give it a go, so here's a ticket for it ;)
Issue is from our end, looks like restler API had changed drastically, ignore and close issue
It seems I can only have one service at a time. I'm trying to make a scraper that pulls from one API and pushes into another, but when I do the second rest.service() call it seems to overwrite the first service I made!
var rest = require('restler')
function noop () {}
var TestA = rest.service(noop, {}, {
log: function () {
console.log('first')
}
})
var TestB = rest.service(noop, {}, {
log: function () {
console.log('second')
}
})
var testa = new TestA
testa.log()
var testb = new TestB
testb.log()
It seems that it's not possible to issue requests if nodejs runs behind a (non-transparent) proxy. This is won't play well in corporate environments for instance.
Is there a solution for configuring a proxy server IP/port in Restler?
Why not?
data {} should be sent via query string on get requests. It seems to be POSTing the data regardless of method? Both rest.service and rest.get currently need to have data set to null to get around this problem.
I couldn't find a way to specify a timeout for the requests that's different from node.js's default (60s).
It would be great if 'timeout' were one of the 'options'.
In the readme, there is an example of using rest.Service() to create a wrapper for Twitter. It assigns to this.defaults.username and this.defaults.password.
I need to set a default querystring parameter. There are lots of APIs that take an AppId or ClientId as a querystring parameter on every call.
I tried this:
... rest.service(function(appid) {
this.defaults.query = { appid : appid };
}, { ...
Which does set the query property, but the mixin function overrides it when I call this.get('/theurl', options) in my wrapper function. where options has a query property.
Ideally, the mixin function would be applied to the query property as well as its parent options property.
Is there any method to implement token authentication for all http verbs? (GET, POST,PUT, DELETE)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.