Comments (17)
Update (2018-06-27):
After reaching out to the remote API owner again, he tracked down the CORS issue and the configuration has been fixed! 🎉 That said, he does not expect the User-Agent
to be anything other than the one the browser normally sends when the request is made in a browser environment, so the code changes made here to only set it when running outside the browser can remain.
from hibp.
I am also getting this error when trying in runkit. Maybe add a global option to set a header in axios e.g. const hibp = require('hibp')(my-agent-name);
or as an object: const hibp = require('hibp')({useragent: "my-agent-name"});
or maybe const hibp = require('hibp')({headers: ["User-Agent: my-name"]});
from hibp.
Hmm, something has changed. Investigating.
from hibp.
This is currently causing us issues as well.
from hibp.
I'm reaching out to Troy, something seems to have changed on his end and request are being denied with a message in the response body indicating the requester has been blocked.
from hibp.
Looks to be a recent thing on the breached accounts endpoint.
I could previously visit the example link on the API documentation here and it would return the response, now it returns a message that I've been blocked.
Hopefully this can be sorted soon :)
from hibp.
Oh, good catch! So that pretty much confirms it's nothing on this end. I'll leave this issue open for now, but I doubt there is anything we can do but wait.
from hibp.
I've been in contact with Troy today and he said he's made some tweaks of the back of my messages and everything is looking good on my side now.
Could you test and let me know?
I can curl the endpoint fine right now but I'm still seeing:
Error: Forbidden - no user agent has been specified in the request.
The code is currently run server side as well. I think going forward it would be great to be able to choose our own user-agent when requiring the library.
from hibp.
I'm still being blocked in the browser (visiting the example URL) on basically every network I have access to, but library usage is working again.
His API documentation states 403 Forbidden responses are indicative of failing to include a User-Agent request header, which is why you see that message. This particular problem is not a matter of omitting the User-Agent, but rather being blocked for other reasons (abuse prevention, likely Cloudflare) which also respond with 403 Forbidden. So I think the message should be modified to be more generic (maybe just "Forbidden - access denied" or similar, or possibly go the other way and make it more detailed (perhaps a link to somewhere explaining you might have been blocked by the remote API). I don't see exposing the User-Agent as an option to add much value (not adamantly opposed, but would need a little more convincing as to its usefulness).
from hibp.
I spoke to him and if I made a GET request to the page it failed along with a CURL request.
After he made some tweaks I've since been able to CURL the page with a User-Agent set.
$ curl -H "User-Agent: kieran-hibp-integration" https://haveibeenpwned.com/api/v2/breachedaccount/[email protected]
[{"Name":"000webhost","Title":"000webhost","Domain":"000webhost.com"... <snip>
Otherwise I get:
curl -H "User-Agent: usecure-hibp-integration" https://haveibeenpwned.com/api/v2/breachedaccount/[email protected]
...
<div class="cf-error-details cf-error-1020">
<h1>Access denied</h1>
<p>This website is using a security service to protect itself from online attacks.</p>
<ul class="cferror_details">
<li>Ray ID: 49b01be2db8a365f</li>
<li>Timestamp: 2019-01-18 09:41:58 UTC</li>
<li>Your IP address: <snip></li>
<li class="XXX_no_wrap_overflow_hidden">Requested URL: haveibeenpwned.com/api/v2/breachedaccount/[email protected] </li>
<li>Error reference number: 1020</li>
<li>Server ID: FL_63F12</li>
<li>User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36
</li>
</ul>
</div>
I think adding the ability to set a custom user-agent will do the job for server side. Other than that I'm not entirely sure you'd fix it for client facing applications.
from hibp.
What user-agent would you set that you feel would help? I already set one if you aren't in a browser.
from hibp.
The way I see it is, if I use this library and the default UA gets blocked because somebody else using the UA abuses the service or accidentally does something they shouldn't have then blocks it for everyone.
If I can customise it, it reduces the risk and lets me identify my app to the endpoint where Troy can relax rules for it/help debug issues.
Saves me the headache of having to rewrite this from scratch for myself when there's another problem!
from hibp.
It sounds like you're under the impression the blocking is happening based purely on UA. Is that the case? That means every user in the world using the latest stable version of Chrome on a particular OS is blocked right now if one bad actor used that popular combination to abuse the service? That doesn't seem feasible. 🤔
If that is indeed the case, then yeah, I can see the benefit of exposing a UA option. I guess we could add it regardless, I just want to be careful with exposing too many non-HIBP related options.
from hibp.
Note: a custom UA would not currently work when using hibp in the browser as Troy's API does not include User-Agent
in the Access-Control-Allow-Headers
header field, so attempting to use a custom UA will be blocked. See here for a demonstration of this:
from hibp.
It sounds like you're under the impression the blocking is happening based purely on UA. Is that the case? That means every user in the world using the latest stable version of Chrome on a particular OS is blocked right now if one bad actor used that popular combination to abuse the service? That doesn't seem feasible. 🤔
Yeah, I absolutely have no idea but that's the only way I can seem to replicate the issue. I did a bit more testing on this and it seems to be entirely UA based:
curl -H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" https://haveibeenpwned.com/api/v2/breachedaccount/[email protected]
...
<h1>You have been blocked from accessing this resource on Have I Been Pwned.</h1>
...
And with a custom UA:
curl -H "User-Agent: kieran-usec-hibp" https://haveibeenpwned.com/api/v2/breachedaccount/[email protected]
[{"Name":"000webhost","Title":"000webhost","Domain":"000webhost.com","BreachDate":"2015-03-01", ...
If that is indeed the case, then yeah, I can see the benefit of exposing a UA option. I guess we could add it regardless, I just want to be careful with exposing too many non-HIBP related options.
I think having a custom UA is definitely a HIBP option though considering the documentation tells you to use a custom one that explains what your app is.
from hibp.
It definitely appears to be UA related, but not 100%. For example, here is a user being blocked, but the same command works fine for me. Same UA.
Edit: And again, adding an option to customize the UA is worthless at the moment as that is not currently allowed when running in the browser.
from hibp.
Closing in favor of #60 as this issue was originally for a different problem that has since been resolved but was emitting the same (misleading) error message. We can continue the discussion there.
from hibp.
Related Issues (20)
- Add API key support for paid endpoints (namely, `breachedaccount` and `pasteaccount`) HOT 1
- Update documentation for API v3 HOT 1
- Action Required: Fix Renovate Configuration
- CORS Error HOT 1
- Refactor pwnedPasswordRange to resolve with object? HOT 2
- All Contributors HOT 14
- Dependency Dashboard
- LogoType HOT 1
- Is there timeout on fetch? HOT 2
- Rate limiting `retry-after` HOT 2
- Error - "You may need an appropriate loader to handle this file type" HOT 5
- Not finding realy simple passwords HOT 3
- Errors in web workers or extension background threads HOT 9
- TypeError: Cannot read properties of undefined (reading 'startsWith') HOT 4
- Request blocked or forbidden HOT 8
- Cannot use hibp modules in React Native development mode HOT 3
- Update rate limit handling HOT 1
- Invert `includeUnverified` option on `breachedaccount` endpoint HOT 1
- Invert `truncateResponse` option on `breachedaccount` endpoint HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hibp.