ljharb / qs Goto Github PK
View Code? Open in Web Editor NEWA querystring parser with nesting support
License: BSD 3-Clause "New" or "Revised" License
A querystring parser with nesting support
License: BSD 3-Clause "New" or "Revised" License
I'm trying to parse an IPN message from PayPal, and it has an array of objects in a query string:
transaction%5B0%5D.is_primary_receiver=true&transaction%5B0%5D.id_for_sender_txn=5GN104358V194442M&log_default_shipping_address_in_transaction=false&transaction%5B0%5D.receiver=seller1.test%40mercher.net&action_type=PAY&ipn_notification_url=http%3A//staging.mercherdev.com/ipn&transaction%5B1%5D.paymentType=SERVICE&transaction%5B0%5D.amount=USD+136.25&charset=windows-1252&transaction_type=Adaptive+Payment+PAY&transaction%5B1%5D.id_for_sender_txn=9L346159YT798301J&transaction%5B1%5D.is_primary_receiver=false&transaction%5B0%5D.status=Completed¬ify_version=UNVERSIONED&transaction%5B0%5D.id=3D429284U5773243B&cancel_url=http%3A//staging.mercherdev.com/test&transaction%5B1%5D.status_for_sender_txn=Completed&transaction%5B1%5D.receiver=dmitriy.s.les-facilitator%40gmail.com&verify_sign=AFcWxV21C7fd0v3bYYYRCpSSRl31Aaz5mmivhYQQvF-Zz1csgdIo0LYX&sender_email=buyer1.test%40mercher.net&fees_payer=PRIMARYRECEIVER&transaction%5B0%5D.status_for_sender_txn=Completed&return_url=http%3A//staging.mercherdev.com/test&transaction%5B0%5D.paymentType=GOODS&transaction%5B1%5D.amount=USD+2.50&reverse_all_parallel_payments_on_error=true&tracking_id=32&transaction%5B1%5D.pending_reason=NONE&pay_key=AP-8LF09716FJ765462U&transaction%5B1%5D.id=43253900X5221472M&transaction%5B0%5D.pending_reason=NONE&status=COMPLETED&transaction%5B1%5D.status=Completed&test_ipn=1&payment_request_date=Mon+Oct+06+09%3A16%3A01+PDT+2014
Qs.parse()
gives me this result:
{
"transaction": [ "NONE", "Completed" ],
"log_default_shipping_address_in_transaction": "false",
"action_type": "PAY",
"ipn_notification_url": "http://staging.mercherdev.com/ipn",
"charset": "windows-1252",
"transaction_type": "Adaptive Payment PAY",
"notify_version": "UNVERSIONED",
"cancel_url": "http://staging.mercherdev.com/test",
"verify_sign": "AFcWxV21C7fd0v3bYYYRCpSSRl31Aaz5mmivhYQQvF-Zz1csgdIo0LYX",
"sender_email": "[email protected]",
"fees_payer": "PRIMARYRECEIVER",
"return_url": "http://staging.mercherdev.com/test",
"reverse_all_parallel_payments_on_error": "true",
"tracking_id": "32",
"pay_key": "AP-8LF09716FJ765462U",
"status": "COMPLETED",
"test_ipn": "1",
"payment_request_date": "Mon Oct 06 09:16:01 PDT 2014"
}
while I was expecting for:
{
"transaction": [
{
"is_primary_receiver": "true",
"id_for_sender_txn": "1EN35283R74763703",
"receiver": "[email protected]",
"amount": "USD 107.00",
"status": "Completed",
"id": "90E58532R7615693H",
"status_for_sender_txn": "Completed",
"paymentType": "GOODS",
"pending_reason": "NONE"
},
{
"paymentType": "SERVICE",
"id_for_sender_txn": "7XF42668MS476925V",
"is_primary_receiver": "false",
"status_for_sender_txn": "Completed",
"receiver": "[email protected]",
"amount": "USD 2.00",
"pending_reason": "NONE",
"id": "6XR12273235492049",
"status": "Completed"
}
],
"log_default_shipping_address_in_transaction": "false",
"action_type": "PAY",
"ipn_notification_url": "http://staging.mercherdev.com/ipn",
"charset": "windows-1252",
"transaction_type": "Adaptive Payment PAY",
"notify_version": "UNVERSIONED",
"cancel_url": "http://staging.mercherdev.com/test",
"verify_sign": "AFcWxV21C7fd0v3bYYYRCpSSRl31AW50ZrxhpTVKQjEObfEFfTcdVouU",
"sender_email": "[email protected]",
"fees_payer": "PRIMARYRECEIVER",
"return_url": "http://staging.mercherdev.com/test",
"reverse_all_parallel_payments_on_error": "true",
"tracking_id": "31",
"pay_key": "AP-10U799054T2106106",
"status": "COMPLETED",
"test_ipn": "1",
"payment_request_date": "Mon Oct 06 09:00:28 PDT 2014"
}
Is there any way to achieve this? I'm using v2.2.4
brief reproduction (from expressjs/connect-multiparty#11 (comment)):
$ node -pe 'o=Object.create(null);o.r=4;require("qs").parse({"a": o})'
node_modules\qs\lib\utils.js:47
if (source.hasOwnProperty(i)) {
^
TypeError: Object object has no method 'hasOwnProperty'
at Object.exports.clone (node_modules\qs\lib\utils.js:47:20)
at Object.exports.clone (node_modules\qs\lib\utils.js:48:31)
at Object.module.exports [as parse] (node_modules\qs\lib\parse.js:141:89)
at [eval]:1:43
at Object.<anonymous> ([eval]-wrapper:6:22)
at Module._compile (module.js:456:26)
at evalScript (node.js:536:25)
at startup (node.js:80:7)
at node.js:906:3
Hi, so an issue was brought up in expressjs/connect-multiparty#11 earlier today. Basically it boils down to a need for a real object conversion API here, I think. I would like to be able to pass in an object and have this library expand out just the keys into a rich object and leave the values alone.
Basically I would like to have something that takes
{
"user[name]": {"pop[bob]": 3},
"user[email]": null
}
and gives
{
"user": {
"name": {"pop[bob]": 3},
"email": null
}
}
and it only looks at the first level keys and does not look down into the objects. I hope I'm making sense, I'm in a rush :)
If no =
is given for a flag, it should be handled as boolean (true
).
For example:
Qs.parse('a&b=', {allowBoolean: true});
// {a: true, b: ''}
without the allowBoolean
option, it should not do this conversion (as it does today):
Qs.parse('a&b=');
// {a: '', b: ''}
For hot code paths for is more performant... it may be worth refactoring and using for
I see the license text and it seems okay, I apparently just have to include it if I modify the source and of course add the license.
But is there an official name for the license? It doesn't seem to be MIT or Apache or BSD.
I am not really sure why you are using this limit. There is no option to change this value. Do you have a good reason for this? Because I have JSON structure with list of pictures greater than 20
Hi,
I noticed a change in the behavior of this library compared to the one it replaces (https://github.com/visionmedia/node-querystring)
The previous :
console.log(parse('param[]=')) // log []
This one :
console.log(parse('param[]=')) // log ['']
To me the previous behavior made more sense.
Now even if the param has no value the array returned by the parser has a length of 1
Can you confirm that this behavior is wanted ?
See #20
This is causing a pretty significant performance hit and is completely unnecessary.
Add a parseArrays: false
option to explicitly disable parsing of arrays, rather than setting arrayLimit: -1
What has changed from 2.x to 3?
io.js 1.6.0 landed a perf boost for parse and stringify maybe worth taking a look at? nodejs/node#847
Hi,
a windows client sends me the following sequence:
%f6%e4%fc%e1%20
Javascripts unescape
can handle this sequence,
Qs can't. I think it's ucs2 encoding.
A spec would look like this:
qs = require('qs')
describe 'Qs', ->
describe '#parse', ->
it 'parses ucs2 from windows', (done) ->
expect(unescape('%f6%e4%fc%e1%20')).to.be.like('öäüá ')
expect(qs.parse('a=%f6%e4%fc%e1%20')).to.be.like( {'a': 'öäüá '})
done()
Is this to be considered a Qs-Bug?
Regards
Dieter
So I wanted to disable array parsing by doing arrayLimit: 0
, but apparently it will still create arrays when the index is 0
, which is oddly misleading :)
$ node -pe 'require("qs").parse("a[0]=b", {arrayLimit: 0})'
{ a: [ 'b' ] }
But I figured out I could do arrayLimit: -1
and it would work, so I hope it's an official option :)
$ node -pe 'require("qs").parse("a[0]=b", {arrayLimit: -1})'
{ a: { '0': 'b' } }
But now I'm running into the problem where a person can specify a negative number and turn my objects into arrays anyway, and the only way to get around that is ridiculous:
$ node -pe 'require("qs").parse("a[-1]=b", {arrayLimit: -1})'
{ a: [] }
$ node -pe 'require("qs").parse("a[-1]=b", {arrayLimit: -Infinity})'
{ a: { '-1': 'b' } }
Hi,
Would it be possible to get a number
when possible, instead of a string
?
For this query string: ?a=2
, we currently get { a: '2' }
. This would be awesome to have { a: 2 }
.
Thanks!
The version 2.2.5
breaks the compatibly with 2.2.4, returning TypeError: Object.keys called on non-object
instead of returning the value.
Clean up tests to be in line with outmoded/discuss#24
The following:
qs.parse('a[2]=2&a[999999999]=1')
results in the following
{ a: { '0': undefined, '1': undefined, '2': '2', '999999999': '1' } }
Since 0 and 1 were actually holes in the array (i.e. !(0 in arr)
), they should not have properties created from them in the object. I would expect:
{ a: { '2': '2', '999999999': '1' } }
which is the output if you had them backwards: qs.parse('a[999999999]=1&a[2]=2')
TL;DR I think qs.parse('a[2]=2&a[999999999]=1')
should have the same resulting object (with the same existing properties) as qs.parse('a[999999999]=1&a[2]=2')
.
I would like to have the option to omit array indices from the stringify. That is:
qs.stringify({ a: ['b', 'c'] }, { indices: false }); // should return a=b&a=c
This is important for formatting requests that are compatible with the Google APIs which don't accept array indices in their request query parameters.
It seems there is a technical report on adding in JSON to HTML forms: http://www.w3.org/TR/html-json-forms/
The syntax seems to pretty much be exactly what is being done here, but with slight differences (really, the differences just define the edge cases like how foo=bar&foo[0]=baz
should parse: {"foo":{"":"bar","0":"baz"}}
). It would also mean you have a good bullet against all the arguments that show up now and again for how this module operates :)
https://github.com/hapijs/qs/blob/master/lib/parse.js#L32
When the object is created via Object.create(null)
it does not have object properties. Instead, the following should be used:
Object.prototype.hasOwnProperty.call(obj, key)
Consider adding an option to sort object keys. This could be useful when using query strings for (or as part of) a unique key, ensuring the order of property declaration doesn't affect output.
qs.stringify({ a: 'b', z: 'x' }) === 'a=b&z=x';
qs.stringify({ z: 'x', a: 'b' }) === 'z=x&a=b';
qs.stringify({ z: 'x', a: 'b' }, { sort: true }) === 'a=b&z=x';
I can work on this and provide pull request if okay.
Sorry for the bad title, I have no idea what to call it :) So, take a look at this bug: expressjs/body-parser#36
It looks like "order[0][quantity]=5&order[0][size]=small&order[0][status]=frozen"
is getting parsed to order: [ { status: 'frozen' } ]
.
var str = 'foo&bar=1&baz';
assert(str === qs.stringify(qs.parse(str)));
// AssertionError: false == true
Hi there,
I stumbled upon this in remix-run/react-router#599 - I cannot override properites using lib/utils:merge
:
> merge({a:'a'}, {a:'b'})
{ a: 'a' }
// triggers different code path
// https://github.com/hapijs/qs/blob/9250c4cda5102fcf72441445816e6d311fc6813d/lib/utils.js#L25
> merge({a:'a'}, {a:undefined})
{ a: 'a' }
Is this desired behaviour? If not, any idea for a workaround?
Thanks!
Max
So when using qs.parse
and passing in an object to expand out the top-level keys, if somewhere in the value is a circular reference, it will fail with RangeError: Maximum call stack size exceeded
. Here is a simple example:
var a = {}; a.b = a;
qs.parse({
'foo[bar]' = 'baz',
'foo[baz]' = a
})
I'm really only reporting this because it used to work with the old qs
library. It was reported to me here: expressjs/connect-multiparty#11 (comment)
I'm just opening a new issue so that old one does not get lost. This is a general continuation as a new issue from #73 (comment) and onwards.
Because qs
silently discards properties, it makes this library unsuitable for any real REST API. It only works for "toy" web sites that have a very explicit list of keys in all routes. With this module, one cannot make a route that contains customer-defined properties, as they will evidently choose one that is silently dropped just because traffic happens to flow through Node.js.
The main other parsers doesn't particularly have this issue:
$ node -pe 'JSON.parse('"'"'{"hasOwnProperty":"yes"}'"'"')'
{ hasOwnProperty: 'yes' }
$ node -pe 'require("querystring").parse("hasOwnProperty=yes")'
{ hasOwnProperty: 'yes' }
$ node -pe 'require("qs").parse("hasOwnProperty=yes")'
{}
Hi, I noticed this went from 1.2.2 to 2.1.0 and via semver conventions that would imply a breaking change? Though I don't see any Changelog - any reason there's not one? And can I safely update to the new version seamlessly or do I need to change my code/use of qs appropriately?
Thanks.
So I just noticed this when comparing the options of qs
to querystring
: querystring
accepts 0
or Infinity
to say no limit on the keys (though it by default has no limit). Providing either of those two values to this module results in an empty parse. I'm not sure if you want to support those (the 0
thing seems dumb to indicate "no limit", but whatever), but thought I'd bring it up :)
The following shows that when a key starts as an array, values that match Object.prototype
keys are basically treated as if they are just an empty bracket for some reason. I would not expect the 1
property to exist in the example below:
$ node -pe 'require("qs").parse("a[]=b&a[hasOwnProperty]=c&a[x]=y")'
{ a: { '0': 'b', '1': 'c', x: 'y' } }
$ node -pe 'require("qs/package").version'
2.4.1
The following testcase would fail:
it('parses mix of simple and explicit array', function (done) {
expect(Qs.parse('a=b&a[]=b')).to.deep.equal({ a: ['b', 'b'] });
expect(Qs.parse('a[]=b&a=b')).to.deep.equal({ a: ['b', 'b'] });
done();
});
Not sure if above is valid testcase to be honest.
References to the Buffer
global variable assume the environment is node.js, but it's a shame to limit the use of this package to node environments when it could easily be used in browsers as well.
Can we modify these checks to make qs more browser-friendly?
I would love to see this client-side.
Qs.parse('a%5B12+v%5D=c&a%5Bfoo%5D=bar')
{a: {'12': 'c', foo: 'bar'}}
Qs.parse('a%5B12+v%5D=c');
{a: ['c']}
Is the above behavior expected? If yes, how to handle such keys?
Following my conversation with @blipsofadoug on Twitter
He says that It is an issue when you are using "extended" urlencoded body parsing
Empty values of a POST array disappear after being submitted.
HTML :
<form action="" method="post">
<input type="text" name="data[]" value="snow" />
<input type="text" name="data[]" value="" />
<input type="text" name="data[]" value="hey oh" />
<input type="submit" value="send me the sausage" />
</form>
in php : print_r($_POST['data'])
Array ( [0] => snow [1] => [2] => hey oh )
in nodejs console.log(req.body.data)
[ 'snow', 'hey oh' ]
Since this module can be browserified and used in the browser, would be nice to be available through bower, and have a dist folder with the already browserified package.
I couldn't find it in the readme but does this module support deduping params?
I was expecting if I send a=true&a=false I would get a: 'false'
But currently I get a: [ 'true', 'false']
Is this something qs supports? If not would you be open to supporting it?
https://github.com/hapijs/qs/blob/master/lib/utils.js#L134 contains a call to Buffer.isBuffer
. This means that when [email protected]
is browserified (we're still on 0.6 for the client side...), the whole Buffer module is pulled in - which adds an extra 40kb (~20kb minified) of code in the bundle - and all of it for just this one function, which is only used once.
I'll try to see if there's a workaround on the browserify side of things, but the question still stands...
The workaround in browserify is to create a fake file which exports Buffer.isBuffer = () => false
and expose
that as "buffer"
. It would still be nice to not have to use this workaround...
Heys guys.
When I have a input like:
Qs.parse('obj[0]=value&obj[1]=value')
I get the expected array value
{obj: ['value', 'value']};
But when the input doesn't start with a zero based index, for instance:
Qs.parse('obj[1]=value&obj[2]=value')
I would expect a return value such as:
{obj: {'1': 'value', '2': 'value'}};
But I still get the same result as before:
{obj: ['value', 'value']};
Reference: expressjs/body-parser#40
Hi, I use this awesome library. But there is one wonder point.
var qs = require("qs");
console.dir(qs.parse("x[]=")); // case1
console.dir(qs.parse("a[]=&a[]=b&a[]=c")); // case2
console.dir(qs.parse("a[]=b&a[]=&a[]=c")); // case3
console.dir(qs.parse("a[]=b&a[]=c&a[]=")); // case4
{ x: [ '' ] }
{ a: [ 'b', 'c' ] }
{ a: [ 'b', '', 'c' ] }
{ a: [ 'b', 'c', '' ] }
Is case2 expected value?
It would be nice if the second parameter was just an object to specify the options like the separator, parametersLimit, etc. Or maybe copy the interface of https://github.com/joyent/node/blob/v0.10.24/lib/querystring.js#L160 (though I think just a single second options argument is good enough).
Inspiration: expressjs/body-parser#41
Can we get an option to disable the new "Transform dot notation to bracket notation" functionality? We would like to keep our code base current with the latest version of this module, but we do NOT want to allow users to submit this new syntax to our API. Since this is done just through a .replace
, it seems like it would be trivial to add a switch that would skip that transformation (but it can still be enabled by default, of course).
closed via e2e0a5c
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.