Giter Site home page Giter Site logo

picture-element's Introduction

This repo is obsolete

The canonical repository is the WHATWG HTML repository.

Purpose

This repository contains the history for a subset of the HTML Standard covering most of the Embedded content section, including the picture, source and img elements. The history of earlier proposals for the picture element, before it was adopted in the HTML specification, is also in this repository.

Found a bug?

If you found a problem with the document, no matter how small, please file an issue in whatwg/html.

Old issues can be found in this repo as well as in W3C Bugzilla.

Join us!

Join the RI Community Group and help us make the Web a better place ... for images :)

picture-element's People

Contributors

aarongustafson avatar annevk avatar anselmh avatar bmuenzenmeyer avatar gordonbrander avatar grayghostvisuals avatar grigs avatar hixie avatar iandevlin avatar marcoscaceres avatar marcoscaceres-remote avatar mdmcginn avatar nwtn avatar tabatkins avatar wilto avatar wturrell avatar yoavweiss avatar zcorpan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

picture-element's Issues

W3C Publication issue

During publication, Robin said:

it's giving me a warning that you're using "Table of contents" instead of "Table of Contents"
it's also getting lost because of script being included in the SotD

Simpler to implement source selection?

Following the latest feedback from the Blink team RE increased PreloadScanner complexity in our current source selection algorithm, do we want to modify it so that all matching sources are downloaded (even if only the first one is displayed)?
That will force webdevs to specify non-overlapping MQs for sources:

<picture>
    <source src="big.jpg" media="screen and (min-width: 800px)">
    <source src="small.jpg" media="screen and (max-width: 799px)">
</picture>

This is already the case for non-matching external CSS files, so it might be a good practice to teach devs to do that. However, if devs don't markup their media attributes that way, it can result in a significant negative performance impact.

Thoughts?

Indentation in example code not aligned properly

A minor issue for sure, but it aids readability of the specification.

In many of the examples provided in the spec. (e.g. Example 1 and 2) the first source element within a picture element is indented correctly, but subsequent ones are indented further which at first glance makes them look to be child elements of the first source element.

Enabling `src` on source?

Should we add the possibility of adding an src attribute to the <source> element?
It can be useful as a shortcut if only art-direction is needed, or if CH is used to do res switching. (with the RW hint).

Alternatively, we can ignore it, since a single URL with no descriptors is perfectly legal with srcset.

events and states triggered by picture

I haven't seen a discussion of which events should be triggered by picture and source. There are a lot of possibilities: some of the media events might apply, and presumably the same load and error events triggered by img should apply as well.

Also: will these events only be triggered by picture, or by source as well?

In addition to the media events, the other media elements also have various networkStates and readyStates that we might want to consider for picture.

Clarify/Define the accessibility model of the <picture> element

What is the accessibility model of the picture element?

A subquestion of this is: What is the ARIA role of the element. Is it, for instance, img? () If yes, then the answer will probably have implications for the accessibility model, since the fallback of an element of role="img" by default is that the content is not rendered to AT users unless the author uses aria-labelledby/aria-describedby.

Currently, the accessibility model is hinted only in the 'Example of usage' section, where it is evident that the thought is that the fallback will serve as accessible content.

To let the a11y model rely on fallback content has several implications:
(A) The implication of the 'img' role ought to mean that picture’s role cannot (by default) be 'img' since such a think would hide the fallback from being rendered to AT users.
(B) When accessible content is needed, then authors would be obligued to provde fallback content in the form of an element with a non-empty alt attribute - or by providing other fallback elements.

Preloading based on MQs & `@viewport`

<source> resources (or srcset ones) can be preloaded based on MQs that are evaluated against the viewport's width. That is currently working in the prototype.
However, as pointed out by @igrigorik, that will soon change when the @viewport rule (that was recently introduced into WebKit) will be merged into the prototype.
The @viewport rule comes to fix the <meta viewport> mess, but with a major difference. It can come in an external CSS, and as such, may not be available at the time the PreloadScanner kicks in. This may result in fetching some resource in the PreloadScanner, only to find out later that other resources should have been preloaded, and download them as well.

One way of solving it is to tell Web developers (as part of "best practices") that @viewport rules must always be inlined rather than external. That is likely to cause confusion, but I don't see any other way around it at the moment.

Thoughts?

What's the need for the source-set list?

Going over the algorithms, I noticed that the selection algorithm picks consistently the first element from the the source-set list and ignores other source-sets in it entirely.
As far as I can tell, no one else is using that list.

If my understanding is correct, it would be more efficient to simply return the first source set parsed in the update source sets algorithm, and avoid creating the list at all.

Thoughts?

Aspect ratios and default styling

Images and videos both have intrinsic aspect ratios. But given this markup...

<video src="300x150.ogv" />
<img src="300x150.jpg" />

...and this CSS:

video, img { width: 300px; height: 300px; }

<img> and <video> do different things. <img> distorts the JPEG, stretching it vertically to fill its box, while <video> preserves the OGG's native aspect ratio via letterboxing.

There are a number of things you could do here, all possible with the widely-unimplemented object-fit property:

  1. Proportionally resize the resource to contain it within the box, padding the non-fit dimension (what <video> does).*
  2. Proportionally resize the resource to cover the box, cropping the non-fit dimension.
  3. Distort the resource to fill the box without any cropping or padding (what <img> does).
  4. Display the image at its intrinsic resolution and crop or pad both the height and width dimensions (what background-image does).

As far as I can tell, except for using background-image in the "rough CSS equivalent" stylesheet, the <picture> spec dosen't specify a behavior here. Should it?

In my view, the only one of the above that makes sense as a default for content images within a responsive layout is containing -- just like <video>. All of the rest have their uses, but seem wrong as the default.

There are probably some very good arguments for keeping <picture>'s default styling as similar as possible to <img>. But a new element seems like a golden opportunity to set things right.

* You can often size an <img> this way using a max/min-width/height property or two, but that only works if you know that your resource is bigger, smaller, wider, or taller than the box. It is currently impossible to resize a content image to be contained within a box if you don't know the resource's intrinsic dimensions relative to the dimensions of that box (without resorting to Javascript).

[on hold] Need to specify DOM constructor

We should be able to create a <picture> from JS using a DOM NamedConstructor, e.g. var pic = new Picture (cf. var img = new Image or var img = new Image()).

The img constructor is:

[NamedConstructor=Image(optional unsigned long width, optional unsigned long height)]

I don't think we'd need anything different really for picture.

Should picture.currentSrc be writeable?

In the IRC the other day it was suggested that for my pet usecase (non-art-directed images whose markup isn't tied to a particular layout) lazy/defer + Javascript logic would be the preferred solution. I've been wondering how this might be achievable with <picture>.

At its most general, I see <picture> as a semantically meaningful way to group multiple versions of the same image which happen to have different technical characteristics (format, resolution, cropping, whatevs) that make them more or less suitable for a given user. The markup could be picking-logic agnostic.

But practically, per the current spec, it seems impossible to implement a custom picking logic. For starters, picture.currentSrc is read-only.

I have vague intuitions that making it writeable and doing whatever else might be necessary to afford that kind of flexibility might be uh, non-trivial. But I figured I'd bring it up for discussion.

Is picture a media element?

We should decide whether <picture> should be considered a media element as defined in the HTML spec. If it is, this could resolve some of the existing issues (e.g. #26, #33). The problem is that much of the media spec doesn't really apply to images.

If <picture> is not a media element and doesn't inherit that specification, we'll need to start grabbing the bits that apply so we can state them explicitly.

Include section to explain the difference from <object>

The content model and semantics looks like <object> in HTML4, so people might wonder why introducing a new element than extending <object>. There's a section explaining the difference between <img> and <picture> in the current draft. Why not have another one to explain the difference from <object>?

The behavior of picture's `src` attribute is unclear

We need to define the behavior with regard to <picture>'s src attribute:

  • What should be done when src is present and no <source> matches - probably load & display that src if we want to match the <video> tag behavior
  • What should be done when src and a matching source are both present - ???

Querying `src` on `picture`

We should define what happens if we do picture.src on a picture that has no src attribute itself, but is displaying one from source. I personally think this should return the displayed src, but I'm fairly sure this isn't what currently happens with video or audio.

Media query evaluation in browser preloader will hurt performance

I've brought this up across several threads in bits and pieces, but here is a (more) coherent document describing why changing the current preloader behavior will, in fact, hurt performance:

https://docs.google.com/document/d/1HanD5V0ousYmHZNPt21myXtGm5IKxXr6CNGBoWVj8Sk/edit

Conserving bytes is an important consideration, and I hope we can find a way to address this case. But conserving bytes is not the only consideration, there is also battery life, incurred latency, and inconsistent UX.

Related tickets: #7, ResponsiveImagesCG/ri-usecases#16

The root issue is that the current proposal conflates two issues: (a) downloading resources which you may need later, and (b) downloading resources which you won't need at all. The best behavior for (a) is as implemented currently (fetch all on initial load), and (b) is what we need to figure out how to solve.

Relationship to srcset

The relationship to srcset section now doesn't make much sense since the spec defines a modified src attribute instead (that includes some of srcset's syntax)

Drop dependency on Media Element

In response to objection that <picture>'s handling of <source> is painfully complicated:

http://lists.w3.org/Archives/Public/public-html/2013Sep/0137.html

I've reviewed the tests:

http://w3c-test.org/web-platform-tests/submissions/326/html/semantics/embedded-content-0/media-elements/loading-the-media-resource/

and found that significant number of them is testing MediaElement's networkState property.

MediaElement is complicated due to all the states needed to expose loading and buffering states for video. Images don't need that, so this burden could be dropped.

Second contributor to number of tests (edge cases) seems to be <source onerror>. A lot of those cases could be avoided if all error events were fired on <picture>.

[on hold] Borrow interface from Media Elements

Following up on #26, #29, #33, and #37.

The interface bits I propose adapting from Media Elements are:

// error state
readonly attribute MediaError? error;

The MediaError object itself fits our needs fine, except possibly the name. The four different possible values (MEDIA_ERR_ABORTED, MEDIA_ERR_NETWORK, MEDIA_ERR_DECODE, MEDIA_ERR_SRC_NOT_SUPPORTED) are all applicable to picture.

// network state
         attribute DOMString src;
readonly attribute DOMString currentSrc;
         attribute DOMString crossOrigin;
const unsigned short NETWORK_EMPTY = 0;
const unsigned short NETWORK_IDLE = 1;
const unsigned short NETWORK_LOADING = 2;
const unsigned short NETWORK_NO_SOURCE = 3;
readonly attribute unsigned short networkState;
void load();
DOMString canPlayType(DOMString type);

The above are all applicable to picture; buffered is not and hs been removed. canPlayType could be renamed to something like canShowType. I also removed preload, but perhaps it could be repurposed for something like defer (#5)?

// ready state

Edit per discussion below: Not needed, because img already has state, as well as the IDL attribute complete. May be needed in the future, if we want to look at meta data.

// playback state
// media controller
// controls
// tracks

None of these are applicable to picture.

Thoughts?

[on hold] CSS content

With CSS3 content we can replace the src of an img:

img {
  content: url(image.png);
}

This should be possible with picture as well, but it's unclear (to me) how it should affect a source element. If we replace the content of a displayed source, should the page update with the new content?

Imagemaps

img allows the usemap and ismap attributes for imagemaps. We may want to allow these for picture too, although it could lead to some interesting problems, especially in cases of art direction.

Thoughts?

[on hold] Interaction with canvas

I think we should codify in the spec how picture will interact with canvas. If we have something like this:

<script>
  var ctx = document.getElementById('canvas').getContext('2d');
  ctx.drawImage(document.getElementById('picture'), 0, 0);
</script>
<canvas id="canvas"></canvas>
<picture id="picture">
  <source id="source1" ... />
  <source id="source2" ... />
</picture>

Which source gets drawn on the canvas? I assume whichever one is currently shown, but it should probably be spelled out. Also, should we be able to draw from one of the source elements directly, rather than via the parent picture?

  ctx.drawImage(document.getElementById('source1'), 0, 0);

This should probably also be spelled out for srcset.

media.canPlayType() equivalent

With video and audio we have media.canPlayType(), which we can use to determine if a certain MIME type is supported. We should spec a similar function for picture. IMO, it makes sense to just use canPlayType as is, although the verb "play" isn't really right. Alternatively, perhaps something like canShowType.

[on-hold] Explain how <img> fetching should work when it's a child of <picture>

The <img> tag fallback seem to have some inherent issues that result in fetching of the srcattribute value, even though it is not displayed.
I've looked around the WebCore code, and the issue seems to be inherent to <img>'s fetching mechanisms. Basically, fetching starts at the moment that srcis added. This happens before the node is added to DOM, so at that phase, the node is not aware that it will be inside <picture>and should avoid fetching.
I could not find other examples for an <img> fallback that actually works. <video> fetches internal <img>, and I couldn't find any mentions in the <video> spec of an <img> fallback.
If anyone knows of such a fallback use of <img> that works, please comment.
Otherwise, we may need to re-think the fallback <img> mechanism.

Reflecting matched media

We need to define the use cases (if any) for returning media that matched the selection of an image.
That is:

var resimg = $("#myresimg"); 
//which media matches 
if(resimg.media .... ){
  // 
}

We have the option to either return a string or a MediaList. We need to investigate which one we want to use.

var resimg = $("#myresimg"); 
//which media matches 
if(resimg.mediaText === "screen"){
  // 
}

Media lists are cool because they have the following:

[Constructor(DOMString text)]
interface MediaList {
    stringifier attribute DOMString mediaText;
    readonly attribute unsigned long length;
    DOMString item (unsigned long index);
    void appendMedium (DOMString medium);
    void deleteMedium (DOMString medium);
};

add support for width/height on source tag

I just saw a comment in #49 referring to the lack of width and height on the source tag, for the polyfill the used by Drupal we added it to avoid reflow while the page is loading.

Can we add it to source?

Add lowsrc

This bug about img lowsrc caught my eye, and I wondered if we shouldn't spec lowsrc as part of picture. It seems like it could be a good part of a responsive solution. Specifically, I would not integrate lowsrc into the main resource selection algorithms at all, but specify that a browser may display it in a situation that it feels is appropriate, such as when on a slow network connection, or based on a user pref.

Thoughts?

section 5 is missing the type-attribute and its behavior

Section 5 of the spec says " Algorithm for deriving the source image" but it misses the behavior if source has a type attribute given.

I think it should be described as following:

If the type attribute is provided with an unsupported type to the User Agent, source should be ignored. User Agent only fetches sources that are applying the media query and are of a supported mime type.

Why is the type attribute of source ignored?

In the spec under, Algorithm for deriving the source image, it says:
"To avoid complexity, the type attribute is all child source elements is ignored in this context."

If you want to download the smallest possible image that a browser can handle the type attribute can be very usefull.

Please see this example by David Newton in a picturefill fork.

Browser that support it can use the smaller webp. Other browsers can fall back to jpg.
This would also make the picture/source future-proof as it would allow for support of new imagetypes.

Maybe I'm missing something but is this really that complex?

A unified solution for <picture>

A Unified solution to <picture>

Perhaps there's a very simple way to support both pre- and post-layout queries with <picture>, and sacrifice neither functionality or performance.

If sources specify the dimensions of the images (and more than one image matches the media queries), we delay image fetching until CSS is downloaded; otherwise fetching can occur immediately.

We can then apply sizing constraints to further filter the list of images (media queries are still king, but if more than 1 image 'matches', we use size constraints).

I've written up the details here: https://gist.github.com/nathanaeljones/5047077


I also propose the expansion of the Use Cases and Requirements document to include:

11 The solution SHOULD offer an method to leverage breakpoints defined in CSS.

12 The solution SHOULD support a simplified syntax to support primary use case 3.1 (preferably a list of images and their dimensions), in order to reach users of content management systems and those without detailed knowledge of CSS media queries.

This allows complexity to be moved from HTML to CSS, and removes the need for high-volume repetition of breakpoint logic.

Authors who wish to use responsive web design will be able to use a CSS framework or snippet and matching CSS classes on to achieve responsive images - a path much less intimidating than CSS media queries, and much easier for CMSes and authoring tools to support (how would a GUI for media queries be designed)?

I fear for adoption of unless we can make it CMS and 'non-coder' friendly.

"Algorithm for deriving the source image" is unclear

In the Note in the unofficial spec (http://picture.responsiveimages.org/#algorithm-for-deriving-the-source-image), the CSS translation seems to indicate that the source applied for "all" media would be used first and then the others would be considered, but traditionally media queries are parsed solely in document order, they grant no specificity (as this would seem to imply). As such, the Note does not make sense.

Resolution: The Note should be updated to reflect the way media queries are handled or it should be explicitly spelled out that the behavior demonstrated is what should happen (e.g. add more media queries to gain greater source weight).

Just as a side note: I still hold that the first-match parsing algorithm used in the video element makes the most sense.

Solve Media Query verbosity/repetition with MQ variables (new proposal)

Authors often define media query "breakpoints" for small/medium/large layouts (preferrably named), and hate repeating those media queries all over the place, especially in many places in the HTML markup for <source>.

Use of breakpoint-specific media query in <source> is a maintenance work if author decides to change the breakpoints (HTML templating could help a bit, HTML and CSS use different preprocessors, so there's still mix of syntaxes and technologies).

I propose solving this by introducing Media Query Variables. This solves verbosity problem not only for picture, but for breakpoints everywhere in general.

CSS Variables with cascade/inheritance can't be used for MQs, because cascade and inheritance is calculated after media queries. However, a similar, familiar syntax can be used for media query variables.

You define a Media Query Variable (breakpoint name) at the top of a stylesheet with @var-media:

@var-media breakpoint-name: (min-width:etc) and (whatever-you-like:etc);

(note there's no {}, this only defines that a name is equal to a query).

To use the variable you use var(name) in the place where you'd use the full media query:

  @media var(breakpoint-name) { ...CSS goes here as usual... }

For example if you define this in CSS (I recommend in <style> element in <head>, more on that later):

   @var-media small-touchscreen: screen and (max-device-width: 480px) and (pointer: coarse);

Then you can use images specifically for this breakpoint like this:

    <picture>
        <source media="var(small-touchscreen)" src="tap.png">
        <source src="click.png">
    </picture>

and it works just as if it was:

    <picture>
        <source media="screen and (max-device-width: 480px) and (pointer: coarse)" src="tap.png">
        <source src="click.png">
    </picture>

I think @srcset could also be changed from weird w/h syntax to use MQ variables.

Bigger example:

<!DOCTYPE html>
<head>
<style>
    /* @var-media <name of the variable>: <media query>; */

    @var-media smalltouchscreen: screen and (max-device-width: 480px) and (pointer: coarse);
</style>
<style>

/* var(<name of mq variable>) is equivalent to using full MQ in its place */

@media var(smalltouchscreen) {
}

/* thanks to the definition at the top is now same as: */

@media screen and (max-device-width: 480px) and (pointer: coarse) {
}
</style>
<link rel=stylesheet href="…"><!-- can use @media var(smalltouchscreen) too -->
</head>
<body>
    <!-- 
    <img srcset="<file> <1x/2x/etc optional> <media query optional>" 
    -->
    <img srcset="tap-lowres.png var(smalltouchscreen), tap-hires.png 2x var(smalltouchscreen), click-hires.png 2x, click.png"/>
    <img srcset="click.png screen, pencil-and-fax-back.eps print"/>

    <picture>
        <source media="var(smalltouchscreen)" src="tap.png">
        <source src="click.png">
    </picture>
</body>

CSS vs Preload Scanner

Browsers want to start downloading images before external CSS is loaded, and before layout is computed. Therefore solutions that depend on layout or cascaded/inherited CSS properties on elements are disqualified.

Luckily MQ variables don't use inheritance and don't depend on layout. If they're put in <style> in <head> it will be possible to interpret them without waiting for external CSS or layout (authors who can't edit <head> can still use full MQs in <picture>).

I'm sure there will be some grimacing about parsing CSS in the preload scanner, but just extracting @var-media rules from top of the stylesheet is very easy (I'll write the few lines of parser code for browser vendors who complain it's too hard).

Having MQ variables in <head> rather than external file isn't ideal for authors, but it's the best compromise that can be done for maintainability without discarding the preload scanner.

Variables that are not (yet) defined don't match anything. This gives nice feature at zero extra implementation cost: If @var-media definitions are put in an external CSS file and all <source> alternatives use a MQ variable, then <picture> wouldn't load anything until external CSS file is loaded (this is good — it's a bit like lazyload/postpone and avoids double-loading by preload scanner).

This way authors have a choice of having decent maintainability and good performance by inlining <style> with MQ definitions, or putting definitions in an external CSS file only at cost of losing preload scanner (which may be reasonable in a JS app without progressive enhancement, because it can't benefit from preload scanner already).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.