whatwg / fetch Goto Github PK
View Code? Open in Web Editor NEWFetch Standard
Home Page: https://fetch.spec.whatwg.org/
License: Other
Fetch Standard
Home Page: https://fetch.spec.whatwg.org/
License: Other
window.fetch
is a great progress to make asynchronous requests, thanks for that!
I am not sure whether that's the right place for doing so, I also propose that we can parse the response as HTML content.
Something like response.html([context])
where context
is the parent node of where the nodes would be inserted. It would return Promise<DocumentFragment>
.
A polyfill of them would be:
response.html = function(context) {
return this.text().then((text) => {
var range = document.createRange();
range.selectNodeContents(context);
return range.createContextualRange(text);
};
};
Use case: having directly parsed elements to possibly manipulate their content before inserting.
What do you think?
Florent
Currently new Request()
will get a context of "fetch" even though it may not be used with the fetch()
function. Perhaps we should distinguish this case by using a different default context value. Maybe empty string or "synthesized".
This is related to "what should we do with context" in Cache:
... or leave as is. So this is now not a off-topic. Please restore all our comments here.
https://fetch.spec.whatwg.org/#http-fetch
- Let response be the result of performing a CORS preflight fetch using request.
- If response is a network error, return response.
response is already used for a variable introduced in step 1, which I think this variable is supposed to shadow. It would be clearer to use "preflight response", say.
We could avoid teeing the request body with this protocol feature: http://httpwg.github.io/specs/rfc7231.html#header.expect
We would need to decide how much of the protocol is implemented in Fetch and how much we leave up to developers.
I'm implementing Request class in Chromium.
While I'm writing test codes like the following, I noticed a problem.
var baseRequest = new Request('test.html', {headers: {'X-Test': 'test'}});
var request = new Request(baseRequest, {mode: 'no-cors'});
console.log(request.headers.get('X-Test')) // -> 'test'
This request.headers should not have 'X-Test' header, because the mode is no-cors and 'X-Test' is not a simple header.
http://fetch.spec.whatwg.org/#request-class
The Request(input, init) constructor must run these steps:
...
If r's request's mode is no CORS, run these substeps:
- If r's request's method is not a simple method, throw a TypeError.
- Set r's Headers object's guard to request-no-CORS.
I think we should remove non simple headers here.
Is my understanding correct?
The manifest spec currently lacks a "context", as none of the ones listed appear to be suitable. This is a feature request to add a suitable context for manifest.
When initiating a fetch request for a resource, the user agent initializes the fetch with content/context specific settings. For example, when fetching an <img>
, the UA applies the appropriate image-src
CSP checks; may advertise different set of request headers (e.g. advertise support for some image formats via Accept
header, and may emit additional "hint" headers); may set different transport options, such as request priority, dependencies, etc.
How do we enable the same functionality with fetch()
API? Without the above, in the worst case, an image resource fetch initiated via fetch()
may violate CSP, will return an incorrect/suboptimal asset due to missing headers, and may be fetched with incorrect priority. Of course, images are just one example; same logic applies to all other content types. Related discussions:
Intuitively, the difference between using fetch()
vs. say, and <img>
, is that the latter knows its context and is thus able to initialize all the right checks + properties. Perhaps we can address this whole issue by providing "treat this fetch as if it was an 'image'" setting or signal? Handwaving...
fetch(url, {as: 'image', headers: {...}, otherOpt: {}, ...})
as
(or whatever fits best) as an explicit context signal that initializes appropriate defaults?headers
and other fetch options are then applied on top of initialized defaults as overridesThe mental model here is very similar to what @domenic proposed in #37 (comment):
fetchSettings = mergeMultimaps(asDefaults, optSpecifiedSettings, uaDefaultSettings);
Assuming above sounds plausible, we could then also enable this in other parts of the platform:
<img src="photo.jpg" fetch-settings="{...}"
> -- implicit: as == image
, and fetch-settings
can be used to override other UA + fetch defaults.<link rel=preload as="image" href="photo.jpg" fetch-settings="{...}">
-- as
is declared explicitly, and options passed in through same mechanism.WDYT, does it sound crazy?
As I read the Body Mixins section, options.body
can be introspected for certain types, which will cause the Content-Type
header to be set appropriately.
Can we have it check for plain-ol'-JS-objects (or any other stringifyable type), run JSON.stringify(options.body
), and set options.headers["Content-Type"] = "application/json"
?
If you manage all network activity yourself you still need a way to inject the results into existing APIs that consume them. There's two approaches here that I know of:
<img>.srcObject = responseInstance
<img>.src = convertResponseToURL(responseInstance)
The first is more elegant, the second invokes Fetch twice (but the convertResponseToURL()
URL should be a way shorter trip). If we go with the second, which scales better due to the many APIs in existence, we need to avoid the flaws from blob URLs and make sure the URL can be used only once and consumes the underlying stream (you cannot use responseInstance
again, if you wanted to do that you'd have to clone it first).
Browsers have default "busy indicators" that provide feedback to users that the page is loading, eg, the tab icon, status bar, and reload icon. These busy indicators are triggered in some situations (eg, clicking a link), but not others (eg, issuing an XHR). (See http://www.stevesouders.com/blog/2013/06/16/browser-busy-indicators/.)
In the absence of these busy indicators, users are uncertain and thus anxious about the status of pages loading. If the XHR response is slow to arrive and its content is critical to the user experience, the user has no idea that anything is happening. This problem grows as more pages adopt XHRs for core functionality.
A good example is this article from eBay about how their XHR-based single-page-web-app was perceived as slow until they added a progress bar (http://calendar.perfplanet.com/2014/the-power-of-perceived-performance/). That article also mentions Twitter's approach of adding a throbber to the page when XHRs are fetching tweets.
While requiring developers to add custom progress indicators is a workaround to this problem, it's not ideal because 1) it's more work for developers and 2) each progress indicator is a custom implementation and thus there is not a typical experience for the user that they are trained to look for and expect.
Since users are "trained" to look for the typical busy indicators, these indicators should be triggered when an XHR is issued for content the user is waiting on. However, XHRs are also used for background requests where the busy indicators should NOT be triggered. I suggest we add an option to XMLHttpRequest, eg, showBusy. Setting showBusy=true would explicitly tell the browser to trigger the busy indicators. This would allow for the current behavior (no busy indicators) to be preserved, but also allows an easier and more standard way for developers to use XHR while giving progress feedback to users.
According to the CSP spec
https://w3c.github.io/webappsec/specs/content-security-policy/#csp-request-header
If the user agent monitors or enforces a policy that contains a directive that contains a source list, then the user agent MUST set a CSP Request Header when requesting cross-origin resources, as described in §3.4 The CSP HTTP Request Header.
But "CSP" is not a simple header
https://fetch.spec.whatwg.org/#simple-header
A simple header is a header whose name is either one of
Accept
,Accept-Language
, andContent-Language
, or whose name isContent-Type
and value, once parsed, has a MIME type (ignoring parameters) that is one ofapplication/x-www-form-urlencoded
,multipart/form-data
, andtext/plain
.
So when the user agent requests a cross-origin resource which CSP is set, it must send a CORS preflight fetch.
This means when we use CSP, we can't use CDN which doesn't support CORS preflight.
Is this my understanding correct?
When getting a response back from the service worker, need to operate on internal response.
The spec currently has three examples under "5 Fetch API", using ES6 syntax. Since the document is pretty long, it would help a lot to have some usage examples at the start of the spec, for example as part of the "Goals" section, similar to the examples in the Web Audio spec. I also suggest using at most ES5 syntax for those examples.
What would those examples accomplish? A lot of specs, including this one, are mostly written for implementers. Getting early feedback from users is generally desirable, but often hindered by the lack of usage examples. Adding those, or in this case, making them easier to find, would make the spec a lot more accessible.
Provide developers with a method to abort something initiated with fetch()
in a way that is not overly complicated.
We have two contenders. Either fetch()
returns an object that is more than a promise going forward or fetch()
is passed something, either an object or a callback that gets handed an object.
In order to not clash with cancelable promises (if they ever materialize) we should pick a somewhat unique method name for abortion. I think terminate()
would fit that bill.
var f = fetch(url)
f.terminate()
Note: the Twitter-sphere seemed somewhat confused about the capabilities of this method. It would most certainly terminate any ongoing stream activity as well. It's not limited to the "lifetime" of the promise.
The limited discussion on es-discuss https://esdiscuss.org/topic/cancelable-promises seemed to favor a controller. There are two flavors that keep coming back. Upfront construction:
var c = new FetchController
fetch(url, {controller: c})
c.abort()
Revealing constructor pattern:
fetch(url, {controller: c => c.abort()})
When execute fetch() on origin http://a.com, the spec seems to allow us to fetch or redirect to a URL with username/password, if the all URLs in the redirect chain before that URL are on the origin http://a.com.
Particularly, if other conditions are OK (e.g. Access-Control-Allow-Origin is properly set),
fetch('http://user:[email protected]/', {mode: 'cors'}): OK
fetch('http://user:[email protected]/', {mode: 'cors'}): OK
fetch('http://a.com/', {mode: 'cors'}), which is redirected to http://user:[email protected]/: OK
fetch('http://a.com/', {mode: 'cors'}), which is redirected to http://user:[email protected]/: OK
fetch('http://b.com/', {mode: 'cors'}), which is redirected to http://user:[email protected]/: Network Error
fetch('http://b.com/', {mode: 'cors'}), which is redirected to http://user:[email protected]/: Network Error
Is my understanding is correct and is this behavior intentional?
(e.g. the spec doesn't intend to reject all cross-origin requests/redirects to URLs with username/password, right?)
If you do the following:
headers.append("accept", "text/plain")
headers.append("Accept", "application/x-www-form-urlencoded")
I assume all of these should return an array of two items:
headers.getAll("accept");
headers.getAll("Accept");
headers.getAll("ACCEPT");
But if we have to serialize out, does it matter what form of the header case should be used?
I propose allowing the browser to select a single case for output serialization, iteration, etc. Preserving case seems relatively unimportant and would require memory/complexity.
Current list of contexts contains prefetch
but omits prerender
and preload
, which need to be added to the list. I'm thinking...
preload
-> connect-src
(see w3c/preload#17 (comment))prefetch
-> no CSP policy? Is there a reason? Should it be connect-src
as well?prerender
-> intuitively, seems that it should be the same as prefetch
./cc @yoavweiss @mikewest
CC: @slightlyoff
The current specification for an opaque filtered response requires that the status
always be set to 0. Could this be relaxed?
I understand that the restriction is in place to prevent leaking information that wouldn't otherwise be available for non-CORS requests/responses, but exposing either the actual HTTP response code or a boolean success/error would be very useful, and may not(*) leak more information that could already be inferred via the onerror
/onload
handlers for elements like <img>
and <link>
.
As it's currently implemented, when fetch()
ing a non-CORS request from the context of a Service Worker, it's not possible to make an informed decisions about whether the filtered opaque response should be cached or not. Blindly caching a (possibly transient) error response isn't ideal, but the other extreme of not caching any filtered opaque responses runs counter to the utility of Service Workers.
(*It's entirely possible that I'm missing some important privacy implications.)
The spec does not document any way to specify GET query parameters (e.g. search?q=unicorns&limit=10&foobar
) for a request. GET is the most common case but in practice, query parameters may be used with any HTTP method.
As I understand, the way to do this is using URLSearchParams
from the URL spec. It would be good to provide a clearer view of how these two APIs fit together, especially as query strings are often supported directly by the high-level XHR APIs that fetch
is meant to replace (e.g. jQuery.ajax or reqwest), often as a data
option for example.
In general, it feels like it would be really great to have a boilerplate-free way to use only native web APIs to do such common things as passing URI parameters.
See also this Twitter thread.
Should the patch verb be included in the spec? (https://tools.ietf.org/html/rfc5789)
here -
https://fetch.spec.whatwg.org/#concept-method-normalize
and
https://fetch.spec.whatwg.org/#http-network-or-cache-fetch
If request's body is null and request's method is `HEAD`, `POST`, or `PUT`, set contentLengthValue to `0`.
Currently JSON.stringify()
returns simply {}
for Request and Response objects. It might be nice to define serializers for these classes to help developers debug their code.
I'm not qualified to comment on if it is appropriate or not to do something like:
// serviceworker.js
onfetch = function(e) {
e.respondWith(fetch(e.request).then(function(response) {
response.headers.append("Reply-from-SW", "yes");
return response;
})
}
but from talking with @annevk on IRC, it seems this is not ok.
IIUC we can create a GET request with a non-null body:
var req = new Request('https://www.example.com/', {method: 'POST', body: 'hello'});
var req2 = new Request(req, {method: 'GET'});
Is this OK?
Should fetch()
set user-agent
by default? Allow appending bytes? Allow replacing it? Allow it to be omitted?
See w3c/ServiceWorker#348 (comment) for context.
And why is it on the forbidden header list? It has been since forever, but is there strong rationale?
As discussed a bit in mikewest/internetdrafts@d2e4cd8, the First-Party-Only
cookie attribute requires tracking the originator of a request across navigations. That doesn't seem possible at the moment via Fetch's current definition.
I'm proposing to add support for (federated) Token Binding to fetch.
I've written a requirements/problem-description document for a Token Binding DOM API: https://docs.google.com/document/d/1x_3KI_v35NQFEsAbGGjNcyTw3Waoy1mL3YfTaQbyyuM/view?rm=minimal&pli=1
While I have included a strawman proposal in the doc, I'd appreciate help from the fetch editors/implementers to figure out what the right approach is.
It would be much more useful if it did, or if there were an option for it to.
It seems to some extent the new cache mode feature can be emulated using timing information. However, it is unclear whether that is sufficient justification to expose the information with more certainty.
Sharing whether the user has an entry in its HTTP cache does allow for fingerprinting and figuring out what sites a user visits. (Note that CORS doesn't really change this as far as I can tell.)
It could be added since XHR2 support PATCH method
https://fetch.spec.whatwg.org/#methods
After trying to stub fetch()
in some unit tests in this project, I came across that bodyUsed
wasn't what I expected. I thought it was a boolean showing that the response contained a body, instead of a value telling that the body has already been used somewhere in your own code.
As the specs are currently not very explanatory on that particular subject:
The bodyUsed attribute's getter must return true if the used flag is set, and false otherwise.
I'd suggest to rename bodyUsed
to bodyAlreadyUsed
, to make the intent clearer.
What do you all think?
The trailer semantics in RFC 7230 (and also since 2616) provides an ability to send/receive a meta-data out of and after the body part.
HTTP2 also supports it though the chunked TE has been deprecated.
gRPC (https://github.com/grpc/grpc) utilizes it and other applications may also benefit from it.
For XHR2, there was some discussion about trailer support in 2010: https://lists.w3.org/Archives/Public/public-webapps/2010OctDec/thread.html#msg163. @mnot well described that together with observation such as increasing use of the trailers.
Anne declined it at that time https://lists.w3.org/Archives/Public/public-webapps/2010OctDec/0233.html. But maybe it's good to revisit this needs in the context of the Fetch API standardization. Introduction of interfaces to access the trailer may affect streams integration design. So, we should start thinking how it would be like asap if we support it.
dictionary RequestInit {
...
Promise<HeadersInit> trailer;
...
};
interface Request {
...
readonly attribute Promise<Headers> trailer;
...
};
dictionary ResponseInit {
...
Promise<HeadersInit> trailer;
...
};
interface Response {
...
readonly attribute Promise<Headers> trailer;
...
};
We have an internal implementation of this in Gecko, but it would be nice if we could expose that to script.
Research:
Todo:
Tentative plan:
X-Content-Type-Options
header.Open issue:
From @wanderview:
Looking at the gecko code it appears that there are many cases where the content is explicitly ignored regardless of what the server does:
- HEAD method
- CONNECT method
- Responses with status codes 101, 204, 205, 304
See mNoContent in the code here:
https://dxr.mozilla.org/mozilla-central/source/netwerk/protocol/http/nsHttpTransaction.cpp#1523
See also w3c/ServiceWorker#655 and web-platform-tests/wpt#1697
Throughout the document, I have not found anything that clearly states whether the when/then API of fetch() is just defined as such for readability, or if fetch() requests are promises. I was also expecting to see the request class extend some sort of thenable interface, but it looks like it's a standalone object.
Currently all fetches will tee the request body due to these three reasons:
1 can be disabled through redirect: "error"
. 2 is already disabled for fetch()
(handling it automatically and prompting the user that is). 3 cannot be disabled at the moment.
The question is whether it would be acceptable to provide an option to error on 3 as well and thereby opening up the ability to make a client-side decision not to tee request's body.
Since we are rejecting with a TypeError when the Response is type error
(aka network error per spec), what's your suggestion on how to distinguish between unparseable url, broken location header, network timeout, redirect limit reached.
Is checking error message the suggested way?
This might be less of a problem on client side, but when implemented on server side, once I allowed custom timeout and maximum redirect count, that error is no longer solely caused by remote resource, it can also be related to request config. This is a concern for debugging and logging purposes.
for server-side implementation, see node-fetch, also related to #20.
In a discussion on .NET MVC Framework, aspnet/HttpAbstractions#245 there's some talk about what's the future of the old mvc version feature: Request.IsAjaxRequest() Method.
Right now, the only way we developers checks for a request that's initiated via javascript we check the used X-Requested-With
custom header.
I was wondering if there's some plan or intention to have a 'standard' header or realiable mechanism for a server to know if the request originates from a fetch request or we should use custom headers for this kind of scenario.
Thanks and sorry if this has already been addresses, couldn't find it on current issues.
The use case here is to allow a service worker to make a fetch just like <img>
can with equivalent effects.
This requires an opaque response that can only be used by features that can legitimately set the same context as otherwise CSP could be circumvented. E.g. if a page were only allowed to fetch images this could be used to fetch "an image" and instead use the result for something else.
I think do this properly I need @mikewest to finish the CSP rewrite in terms of Fetch. And we need to fix the SW and CSP issue: w3c/webappsec#227
Since XMLHttpRequest.prototype.responseXML
used to exist, I guess response.xml()
should exist too, right?
Florent
Most ajax libraries and xhr itself offer a convenient way to set a timeout for a request. Adding timeout
to the properties accepted in the options object therefore seems like a sensible addition to fetch.
Is there any way I can attach a progress listener to the xhr.upload object used in fetch?
xhr.upload.addEventListener('progress', function(){ ... })
since I use fetch to upload files (with FormData), I would find this very useful.
I initially reported this at JakeChampion/fetch#89, and was suggested to have it reported here
@mayhemer it's not entirely clear to me how to deal with partial content. Does us handling that mean we also pay special attention to a 206 response? What's the logic here, if we notice we have a partial cache entry, we set If-Range
, then if there's a 206 response, we combine some stuff and hand out the combined response as a 200 to content? Does this only happen for media elements basically?
current fetch api spec says
it is currently lacking when it comes to reporting progression.
I think this solves when response will supports stream api of whatwg.
but there is no spec about that, only returns promise and resolve only once.
do you currently have any idea about this ?
or any point of discussion about this ?
We should stipulate in the specification what happens there.
The fetch
spec doesn't define what a user agent should do when the document unloads. I expect the fetch request be aborted. This expectation seems reasonable, since it is similar to what the SW spec states: https://slightlyoff.github.io/ServiceWorker/spec/service_worker/index.html#on-fetch-request-algorithm:
18 . ...
If task is discarded or the script has been aborted by the termination of activeWorker, set handleFetchFailed to true.
...
21 . If handleFetchFailed is true, then:
- Return a network error and run the following substeps in parallel.
22 . Else:
- Return a response represented by response and run the following substeps in parallel.
Chrome's implementation already follows this behavior, Firefox does not (see https://bugzilla.mozilla.org/show_bug.cgi?id=1165237).
In Chrome Canary, if searching for a non-existent pattern via https://domenic.github.io/streams-demo/, memory usage increases from ~65MB to ~155MB (as measured by the Task Manager on OS X). That is, the Chrome process seems to grow by about 90MB for each 1000MB downloaded.
(Perhaps this is because the standard method of processing "infinite" streams via a recursive calls seems to require at least two (?) stack frames per read() chunk. Is there any TCO?)
Over in yutakahirano/fetch-with-streams#37 (comment) @yutakahirano noticed that this code may not do what is expected:
var req = new Request(url, {body: 'hello'});
fetch(req);
fetch(req);
I think we would expect the second fetch() to reject on a drained Request, but it won't. The first fetch() sets the request's used flag to true and its body to null. The second fetch(), however, only looks at the used flag if the body is not null.
What do you think @annevk?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.