linebender/piet

An abstraction for 2D graphics.

piet's Issues

Reexport kurbo types

Reexporting the types defined in kurbo would help to avoid conflicts when users don't specify the same kurbo version in their crates and the one used by piet (e.g two different Shape trait versions).

Migrate to newer direct2d

Currently we're using direct2d 0.2.0. There are a number of problems with this, including excess debug logging on paths, not being able to set the closed/open status of a path in a single pass when iterating from path elements. I'd like to get us onto the newer 0.3 versions of the direct2d and related crates, but they're still in alpha. I'm wondering what it will take to get those released. It might be worth developing the patch based on alpha so we can shake out any problems.

I've been figuring that we would update the direct2d deps before taking on #60, but if it's going to take a while maybe we should special case Rect in particular, as that's the biggest pain point.

@Connicpu who owns the direct2d crates. I might ask for admin privs on those crates so we can move those forward. Also, this is a great "help wanted" issue for someone who wants to dive in.

A simple color type

Discussion branched from review of #39.

Use of a raw u32 is problematic to represent color. We should have a simple color type.

One model is ColorF from the direct2d crate, which wraps D2D_COLOR_F but has its own conversions. I think having a From conversion from u32 interpreted as rgba (note the ColorF conversion is from rgb) would be good. I'd also be open to a conversion from (u32, f32), interpreted as (rgb, a); this latter conversion would be the same as ColorF. And APIs that directly accept color (solid_brush) is the main thing, should accept impl Into so any of these variants can be used.

I'll note that the concept of color is extremely deep water. There are questions of colorspace (sRGB vs other standard RGB color spaces vs the idea of "device color"), gamut, depth (including HDR), use of color for print (CMYK and other spaces that include spot color), other appearance measurements such as gloss, etc. I don't want to get into any of that right now. The new color type would be, as much as specified, an 8 bit sRGB value with separate alpha. I can imagine adding more complex color types later as optional extensions.

Bikeshedding regarding naming and other similar questions are welcome.

Color ergonomics improvements

This is split off from discussion in #70. The current Color::rgb24(0xrr_gg_bb) is deemed not very rust-like. Let the bikeshedding begin!

I think the most appealing proposal on the table is to let Color::rgb(r: u8, g: u8, b: u8) be the primary constructor. We can rename the u32 constructors from_rgba32 and the like. The existing Color::rgb that takes three float values would have to be renamed. What to? Perhaps we keep that as-is and use Color::rgb8 for the new constructor?

Dealing with the alpha value is tricky. A Color::rgba that takes an additional a: u8 feels consistent but that does mismatch CSS, which uses a float for alpha.

There are other proposals in the thread.

Another consideration is the use of Into<Trait>. This prevents the constructor from being const, so I'd prefer the primary constructors not to use it, but to favor concrete types.

Lastly, we might want to clarify our thinking on whether the RGB values are defined to be interpreted as sRGB. That's more of a documentation issue than one involving constructors though.

Deferred errors?

This is branched off #32, particularly the complaint that too many methods have Result types.

I don't agree with the suggestion of just panicking, as I think that's extremely unfriendly in a GUI context - it seems to me that in most cases it's best to have incomplete display than bring down the whole app.

That said, I am wondering if there might be a better approach. I notice that both Cairo and Direct2D have a style where most drawing ops succeed, but the render target can go into an error state, which in Cairo is retrieved with a status call, and in Direct2D seems mostly to be retrieved at EndDraw. Part of this, I think, is to support pipelined operation, where the error might not occur until later.

I'm wondering whether it would be better to similarly defer errors for simple draw calls. My thinking is that it's still valuable to have them for resource creation (brush, image, text layout). Part of the reason this is coming to mind now is that I'm working out the signature for gradient creation, and thinking it would be best to have the error only on the final build call.

One benefit is less branchy control flow, which at the least will result in smaller code size. The main downside is that when there is an error, it might be a little harder to track down the exact source, but if there is a status check like Cairo, that can be added to isolate faults during debugging.

What do people think?

Gradient::new concept

Now that I've created a few gradients, I believe it would be possible to make them slightly nicer to use with a constructor.

My thinking is something like this:

Gradient::new_linear(shape: impl Shape, stops: Vec<GradientStop>, rotation: f64)

Gradient::new_radial(shape: impl Shape, stops: Vec<GradientStop>)

For linear you could use the top and bottom of the shape's bounding_box for start and end points if rotation is 0. Not sure how to calculate it otherwise, so maybe this won't actually work. But it would be nice! :)

This gives you all the expressivity of CSS afaik while saving the user from needing to calculate the specific positions for the start and end points.

If it can't be done without having to define explicit start and end points (the main convenience I'm hoping to gain) then it might not be worth the hassle.

piet_common API to create an image file?

Playing around with testing, and have wanted to do something that I don't see an obvious API for: basically I would like to use the piet_common types (Device, BitmapTarget etc) to do some drawing, but then I would like to save the result to disk. I won't mind if the format of the image differs per-backend (although I suppose this isn't ideal) but I would like to be able to easily dump the current buffer in a way that lets me go take a look at it with an image viewer. Is this possible somehow?

Global color changes

Hello!

I was wondering, are there plans for global color tinting? Or more specifically things like setting a global alpha value.

For example currently I'm drawing in multiple passes and want to draw several things with transparency, and it would be easier to set it globally than to conditionally change it everywhere.

expose font metrics (ascender / descender etc)

I'm sure this is on some font roadmap I'm too lazy to look for, so feel free to close as a dupe if appropriate:

When drawing text (via RenderCtx::draw_text) we supply a position where the text should draw. This is an origin in "text space", that is with (0,0) at the intersection of the baseline with the left edge.

Unfortunately, there's no way to know where the baseline is relative to the overall height. (There's also no real way of getting the overall height, afaict, although what "height" means is not well defined; I would settle for "the union of the bounding boxes of all glyphs in the font").

Anyway: this is more a practical issue than a font-metaphysics issue. It would be nice if we could, in the general case, draw some text in a box with equal padding on the top and bottom. A reasonable first step for this would be exposing general metrics like ascender/descender.

High Level Docs

We already have auto-generated API docs. I propose two more kinds:

• High-level overview: These are intended for Rust devs with at least some experience in 2D graphics who would like to explore piet's feature set through prose instead of through examples and API docs.
• The 2D graphics book: This is intended for Rust devs with little to no experience in 2D graphics who would like to learn from scratch using piet.

Ownership / borrowing of Shape

As of #2, the main way of passing a Shape through the API is &impl Shape. I think that's good, but here are possible ways it could be improved.

It might be desirable to pass in a shape either through moving or borrowing. From the caller's perspective, it's a tiny ergonomics bump, as you'd be able to elide the & from stroke(&Line::(...)). But perhaps more importantly, an implementer that records into a display list would be able to take ownership, avoiding a clone.

As a related issue, it might also be desirable to add a Clone bound, either here or in kurbo. This is not strictly necessary, as this use case might well special-case the shapes it cares about, and just collect the to_bez_path iterator results in the general case. The Clone bound would in general result in a Box<dyn Shape>, which might be useful. Another question: if we add a Clone bound, should we also add Send so that display lists can be passed across thread boundaries?

One possibility is suggested by this playground gist. Another possibility is the supercow crate.

Use TextMetrics properties for piet-web linebreaking

For linebreaking, piet needs to be able to get the height and baseline of a line.

In piet-web, the best way to do this should be through web_sys::CanvasRenderingContext2d::measure_text, which returns TextMetrics.

However, TextMetrics currently has only width as a property. All other baseline and height related properties are more recent additions to the web api.

There's an issue open, hopefully the bindings will be updated. rustwasm/wasm-bindgen#2069

Until then, the piet-web uses some heuristics to guess at line height and baseline.

Crate metadata does not list repository

Please add to crates:

repository = "https://github.com/linebender/piet"

[wishlist] support Raqote backend

https://github.com/jrmuizel/raqote is a 2D graphics library in pure (but not 100% safe) Rust. It is complete enough to serve as a seemingly fully-featured backend for resvg.

piet-cairo do not hendle new line charater propely

When it ask to render a new line char piet-cairo do generate a proper text layout, but it display a tofu where the new line is instead of displaying nothing.

Unable to `cargo test` on windows when no cairo installed

With #97 adding default members to piet's workspace Cargo.toml, I was unable to run cargo test on windows because cargo attempted to build piet-cairo and I did not have the required cairo libraries installed.

Removing default-members in the Cargo.toml fixed the issue for me for now.

Not sure what the permanent fix is yet, though.

Related to #33

Text abstraction level

Related to #10: what is the right level of abstraction for text handling that should be provided through the piet::RenderContext trait? (And so by every Piet backend.) What should be left to the user, or to separate libraries in the Druid ecosystem that work on top of Piet?

There are three options that I think make sense for enabling high-quality text:

• Low level: glyph runs. In cairo this is cairo_show_glyphs.

  Backends can render a sequence of positioned glyphs, all in the same font face. No text layout is done at all.

  Open question: is it in scope to provide access to glyph metrics and other data from fonts?

  Bonus: for backends whose output have semantics in addition to visuals (e.g. supporting text search (CTRL+F) in, or copy/paste from, a PDF file generated by a Piet backend), a way to associate a Unicode string with each glyph run. In cairo this is cairo_show_text_glyphs.

• Mid level: text spans. Or should we call them segments? This is Skribo. (Maybe also Pango’s low level?)

  Backends can render a Unicode string that shares a single bidi direction, (also Unicode script?), styling (color or brush, size, weight, stretch, italic, letter-spacing, etc), and ordered list of preferred font families. This does text layout including complex shaping and font fallback. (For example an English sentence with emoji will likely use two font faces.)

  There’s a notion of collections of fonts available for a given layout. Ideally with a choice of whatever fonts are provided by the system/environment, specific fonts provided by the user/application, or both.

  Also useful: rendering only a substring, computing its advance “width”, mapping a position in rendering space back to an index in the Unicode string.

• High level: paragraphs. In Pango this is PangoLayout.

  Backends can render some rich text structure where different parts of the text can have different styles or font preferences, with some space constraints like “available width”. In addition to shaping and fallback, bidi and line breaking are taken care of.

Ideally when a higher level is provided for convenience, all lower levels should also be available for users who want more control. However this may be a problem for the web backend in particular: glyph rendering is simply not available, if using the browser’s fonts and text handling at all.

Windows CI

Currently the CI only runs on Linux. We need to also run on macOS and Windows. There are a few small challenges to this:

• Linux and macOS cannot build piet-direct2d. The .travis.yml script needs to ensure that building of piet-direct2d doesn't happen on non-Windows OSs.
• Travis Windows and macOS don't have cairo installed and this is needed to run. macOS returns the following error: Package cairo was not found in the pkg-config search path. Perhaps you should add the directory containing `cairo.pc' to the PKG_CONFIG_PATH environment variable No package 'cairo' found

Scope for font/text handling?

Reading #5, #7, and https://github.com/linebender/piet/tree/f8bae672/piet-cairo#toy-text-api I’d like to ask: how much text handling do you consider in scope for Piet, @raphlinus? How much do you think should be required of each backend v.s. provided by Piet itself on top of lower-level operations v.s. left to other libraries on top of Piet?

(Also CC @rylev who seems interested.)

I suppose the lowest level (giving the most control to the user) might be rendering a single glyph (or a sequence of positioned glyphs) of a given font. Possibly as a Shape, supporting all operations they do.

Beyond this, there’s a lot that seems somewhat outside of the realm of graphics rendering. Line breaking? Bidi? Vertical text? Hit testing? Selection?

Getting from Unicode strings to positioned glyphs is text layout, which is tricky to do correctly for international text. In the open-source world, Pango provides this functionality. http://behdad.org/text/ describes where it fits, and explains that some applications that want a lof of control over text layout may want to do some of it themselves and only use lower-level components such as Harfbuzz. This document is old though, I don’t know how much this is still the case today.

I’ve been thinking about what it would look like to do that in Rust. Some of the pieces are already there:

• https://crates.io/crates/unicode-bidi
• https://crates.io/crates/xi-unicode
• https://crates.io/crates/harfbuzz-sys (it seems that re-implementing Harfbuzz would be a lot more work than the rest of text layout)
• https://crates.io/crates/font-kit

… but I have a hard time figuring out how, on a high level, these pieces fit together. (Sorry if this issue tracker is not quite the place to discuss this.) For example, as far as I understand Harfbuzz operates on one text segment at a time that only uses a single font, direction, and script (writing system). It also maps N Unicode code points to M glyphs in the general case. But how does on split these segments in the first place? If it’s Harfbuzz that maps code points to glyphs but it only deals with a single font at time, where does font selection and fallback happen?

add greyscale color constructors

grey and grey8 would take a f64 and a u8, respectively, and use it for the color channels in the respective rgb and rgb8 fns.

Recommend using wasm-pack instead of running wasm-bindgen directly

Make sure that wasm-bindgen is installed. This needs to be the same version of wasm-bindgen used by piet-web.

FYI, we found this to be quite annoying across projects that expect different wasm-bindgen versions, so wasm-pack will manage this for you, and all you have to do is wasm-pack build and the proper wasm-bindgen CLI is transparently downloaded for you.

Perhaps you ran into the issue of it not supporting workspaces properly? Good news! That is fixed on master and we are cutting a new release today :)

https://github.com/rustwasm/wasm-pack

Cheers!

Unresolved import `direct2d::math`

I am (probably naively) trying to run "cargo build" on a fresh pull of piet and I'm seeing several errors in the build, e.g.:

error[E0432]: unresolved import `direct2d::math`
 --> piet-direct2d/src/conv.rs:3:15
  |
3 | use direct2d::math::{Matrix3x2F, Point2F, RectF};
  |               ^^^^ could not find `math` in `direct2d`

error[E0433]: failed to resolve: could not find `shared` in `winapi`
  --> piet-direct2d/src/lib.rs:11:13
   |
11 | use winapi::shared::basetsd::UINT32;
   |             ^^^^^^ could not find `shared` in `winapi`

error[E0433]: failed to resolve: could not find `um` in `winapi`
  --> piet-direct2d/src/lib.rs:12:13
   |
12 | use winapi::um::dcommon::D2D_SIZE_U;
   |             ^^ could not find `um` in `winapi`

I assume that as a new user of rust I just don't know what I'm doing :)

Error type Send/Sync?

I'm trying to send an error across threads, but there is no Send/Sync trait bound on BackendError, so I can't. Is there a better way to handle this? For now I'm just unwrapping these errors.

Make piet-coregraphics real

#4 has a work in progress CoreGraphics backend. I think there's a good case to be made to bring this up to usable state.

• After a period of fluidity, I think the piet API is more stable. One of the main things that still needs work is richer text, and I think there's use in considering both CoreText and DirectWrite at roughly the same time.

• The Cairo dependency is a papercut for macOS build, test, and deployment.

• The Cairo "toy text API" is the source of degraded capabilities, in particular fallback fonts.

• A skribo-based text module is the future path for the Cairo backend, but this won't be ready for quite some time.

Simplify api and dependencies between piet family of crates

The current structure of piet— traits defined in the piet crate, with lots of associated types, and then implemented in the various backends, and then exposed as aliases in piet-common— is complicated and fragile-feeling.

For a while I've been trying to figure out a better way to do this, based on my experience refactoring druid-shell, and I think I might have a satisfying solution.

structs instead of traits define common interfaces

The basic idea for this is that we use wrapper structs to define common interfaces. Let's take the text module as an example. We basically translate,

pub trait Text {
    type FontBuilder: FontBuilder<Out = Self::Font>;
    type Font: Font;
    type TextLayoutBuilder: TextLayoutBuilder<Out = Self::TextLayout>;
    type TextLayout: TextLayout;

    fn new_font_by_name(&mut self, name: &str, size: f64) -> Self::FontBuilder;
    fn new_text_layout(&mut self, font: &Self::Font, text: &str) -> Self::TextLayoutBuilder;
}

pub trait FontBuilder {
    type Out: Font;
    fn build(self) -> Result<Self::Out, Error>;
}

pub trait Font {}

pub trait TextLayoutBuilder {
    type Out: TextLayout;
    fn build(self) -> Result<Self::Out, Error>;
}

Into,

use crate::platform;

struct Text(platform::Text);
struct Font(platform::Font);
struct FontBuilder(platform::FontBuilder);
struct TextLayout(platform::TextLayout);
struct TextLayoutBuilder(platform::TextLayoutBuilder);

impl Text {
    fn new_font_by_name(&mut self, name: &str, size: f64) -> FontBuilder {
        self.0.new_font_by_name(name, size).into()
    }

    fn new_text_layout(&mut self, font: &Font, text: &str) -> TextLayoutBuilder {
        self.0.new_text_layout(&font.0, text).into()
    }
}

impl FontBuilder {
    fn build(self) -> Result<Font, Error> {
        self.0.build().map(Into::into)
    }
}

//etc

In this world, piet and piet-common are unified. By default, in piet, the 'platform' is the NullRenderer; but we support a "platform" feature, which uses the best backend available for the platform. If the user wants to use a specific backend, they can just use it directly; the API is identical, and this is enforced by the convention of having the methods on the types in piet directly call identical methods on the inner type.

Downsides

The major downside to this strategy is that it doesn't let us have functions that are generic over piet backends. This isn't impossible to overcome, however; one simple (if a bit annoying) approach is to also have traits that cover this API, and which can be used when necessary. A major question to ask is whether or not we actually ever anticipate having more than one piet backend in use at a time, in a way that requires us to be generic. My hunch is that this is a very speculative concern, and one we really don't need to be too concerned about.

Am I missing anything? It's very possible that I am, but from my current understanding this really feels like a solid win: we can combine two or three crates into one (piet, piet-common, piet-test(?)), and we can get rid of the associated types and the way they complicate the API elsewhere (the really hair type aliases like pub type Text<'a> = <piet_common::Piet<'a> as piet_common::RenderContext>::Text) which complicate API.

d2d: Using DeviceContext with Swapchain

Repository: https://github.com/msiglreith/piet_d2d_tests

As mentioned a few days ago I has some issues with integrating piet in my GUI library in the past. Here are my attempts at using it in combination with winit and further initializing the piet context from a DeviceContext + Swapchain. The issue actually seems to be inside the underlying direct2d crate which fails the create a rendertarget from the swapchain backbuffer. Nonetheless it might be interesting as well for this project:

D2D DEBUG ERROR - The bitmap options [0x0] must be a subset of the flags associated with the DXGI surface.

So to be fair not really a piet issue. After some tries to guess the flags I gave up. I investigated a bit more and compared it to my current way of creating the bitmap:

paw:

let surface = backbuffer.cast::<dxgi::IDXGISurface>().unwrap();
unsafe {
    let mut bitmap = ptr::null_mut();
    let _hr = self.CreateBitmapFromDxgiSurface(
        surface.as_raw(),
        ptr::null(),
        &mut bitmap as *mut _,
    );
    Bitmap(BitmapRaw::from_raw(bitmap))
}

direct2d:

const DEFAULT_BITMAP_PROPS: D2D1_BITMAP_PROPERTIES1 = D2D1_BITMAP_PROPERTIES1 {
    pixelFormat: D2D1_PIXEL_FORMAT {
        format: Format::Unknown as u32,
        alphaMode: AlphaMode::Premultiplied as u32,
    },
    dpiX: 96.0,
    dpiY: 96.0,
    bitmapOptions: BitmapOptions::NONE.0,
    colorContext: ptr::null(),
};

BitmapSource::Dxgi(surface, context) => {
    let mut ptr = ptr::null_mut();
    let hr = (*context.get_raw()).CreateBitmapFromDxgiSurface(
        surface.get_raw(),
        &self.properties,
        &mut ptr,
    );

    if SUCCEEDED(hr) {
        Ok(Bitmap::from_raw(ptr as _))
    } else {
        Err(hr.into())
    }
}

drop shadows for rects

this is the one thing I am most frequently missing in druid.

(piet-common) Creating BitmapTarget from an existing context

For resvg, I need an ability to init a common backend from an existing pointer. This is required to implement an ability to render stuff on native widgets. So I have a cairo_t * or QPainter * and I have to render on them.

Is this planned?

Add documentation to Text, FontBuilder, FontLayoutBuilder

These are fairly important public types, and currently undocumented.

Cleaning up the dependency situation (proposal / discussion)

I find the current distinction between the various piet-family crates confusing, which means it is probably confusing to other people as well.

I think that it should be easy to use piet in the common case, which is with a single platform-dependent backend.

I've been thinking about how we could structure things to make this simple, with the goal of making the simple case require only one dependency.

A modest proposal

Goal: I would like to have a single crate called piet that exposes the shared types (in the current piet crate) and also the preferred backend for the current platform.

To do this:

• I would like to rename piet to piet-core.
• I would like to add a new crate, piet, that is a wrapper around piet-core and piet-common, and which just reexports everything.

This means that in the general case, a consumer crate only needs to include the piet crate.

If people want to get fancy, they can use individual backends along with piet-core and do whatever they like.

As a bonus, existing code will still work.

Please let me know if I'm missing anything

Color::as_rgba does not take self param?

Is this an oversight or am I missing something?

/// Convert a color value to four f64 values, each in the range 0.0 to 1.0.
    pub fn as_rgba(rgba: &Color) -> (f64, f64, f64, f64) {
        let rgba = rgba.as_rgba_u32();
        (
            (rgba >> 24) as f64 / 255.0,
            ((rgba >> 16) & 255) as f64 / 255.0,
            ((rgba >> 8) & 255) as f64 / 255.0,
            (rgba & 255) as f64 / 255.0,
        )
    }

This requires you to write Color::as_rgba(&my_color) instead of my_color.as_rbga().

remove `FillRule` argument in `fill`

I think that fill should default to NonZero, and there should be a separate fill_even_odd method exposed. I strongly suspect an overwhelming majority of users want the NonZero behaviour, and those that want EvenOdd know they do.

(I also think that stroke should lose the style argument, and there should be a separate stroke_styled fn, but one thing at a time?)

A bunch of hit test tests fail on Windows 7.

My environment

Windows 7 SP1 (build 7601) x64
rustc 1.39.0 (4560ea788 2019-11-04)
nVidia GTX 1060 (397.93)

The tests fail with both stable-x86_64-pc-windows-msvc and stable-x86_64-pc-windows-gnu.

These two tests fail in piet-direct2d

test test::test_hit_test_point_complex ... FAILED
test test::test_hit_test_text_position_complex_1 ... FAILED

---- test::test_hit_test_point_complex stdout ----
text pos 2: Some(HitTestTextPosition { point: (6.275390625, 0.0), metrics: HitTestMetrics { text_position: 2 } })
text pos 9: Some(HitTestTextPosition { point: (13.365234375, 0.0), metrics: HitTestMetrics { text_position: 9 } })
text pos 10: Some(HitTestTextPosition { point: (19.833984375, 1.546875), metrics: HitTestMetrics { text_position: 10 } })
text pos 14: Some(HitTestTextPosition { point: (28.669921875, 0.0), metrics: HitTestMetrics { text_position: 14 } })
thread 'test::test_hit_test_point_complex' panicked at 'assertion failed: `(left == right)`
  left: `6`,
 right: `2`', piet-direct2d\src\lib.rs:1001:9

---- test::test_hit_test_text_position_complex_1 stdout ----
thread 'test::test_hit_test_text_position_complex_1' panicked at 'assertion failed: x <= max && x >= min', piet-direct2d\src\lib.rs:707:9

piet-web also has a bunch of test failures

Chrome 78.0.3904.108 (64-bit) fails these three

test_hit_test_point_basic_0

piet_web_example.js:506 text pos 4: Some(HitTestTextPosition { point: (17.994140625, 0.0), metrics: HitTestMetrics { text_position: 4 } })
piet_web_example.js:506 text pos 5: Some(HitTestTextPosition { point: (20.994140625, 0.0), metrics: HitTestMetrics { text_position: 5 } })
RuntimeError: unreachable
    at __rust_start_panic (wasm-function[396]:0x1ab88)
    at rust_panic (wasm-function[254]:0x1a1a6)
    at std::panicking::rust_panic_with_hook::h5e7c2dc110ae79d4 (wasm-function[142]:0x172f1)
    at std::panicking::continue_panic_fmt::hb5b3e4b5160fe2ab (wasm-function[175]:0x18978)
    at std::panicking::begin_panic_fmt::h62d8474306d91923 (wasm-function[206]:0x19622)
    at piet_web_example::test::test_hit_test_point_basic_0::h63a9a73ce15a91e1 (wasm-function[66]:0x80af)
    at run (wasm-function[76]:0xd6ac)
    at Module.run (webpack:///./dist/piet_web_example.js?:68:66)
    at eval (webpack:///./index.js?:4:16)

test_hit_test_point_basic_1

RuntimeError: unreachable
    at __rust_start_panic (wasm-function[394]:0x1a723)
    at rust_panic (wasm-function[254]:0x19d6e)
    at std::panicking::rust_panic_with_hook::h5e7c2dc110ae79d4 (wasm-function[142]:0x16eb9)
    at std::panicking::continue_panic_fmt::hb5b3e4b5160fe2ab (wasm-function[175]:0x18540)
    at std::panicking::begin_panic_fmt::h62d8474306d91923 (wasm-function[206]:0x191ea)
    at piet_web_example::test::test_hit_test_point_basic_1::h48971481aa43754a (wasm-function[72]:0xba70)
    at run (wasm-function[76]:0xd29d)
    at Module.run (webpack:///./dist/piet_web_example.js?:68:66)
    at eval (webpack:///./index.js?:4:16)

test_hit_test_point_complex_1

RuntimeError: unreachable
    at __rust_start_panic (wasm-function[394]:0x1a4af)
    at rust_panic (wasm-function[254]:0x19afa)
    at std::panicking::rust_panic_with_hook::h5e7c2dc110ae79d4 (wasm-function[142]:0x16c45)
    at std::panicking::continue_panic_fmt::hb5b3e4b5160fe2ab (wasm-function[175]:0x182cc)
    at std::panicking::begin_panic_fmt::h62d8474306d91923 (wasm-function[206]:0x18f76)
    at piet_web_example::test::test_hit_test_point_complex_1::hd758df858f1cc9ef (wasm-function[75]:0xcc4f)
    at run (wasm-function[76]:0xd07b)
    at Module.run (webpack:///./dist/piet_web_example.js?:68:66)
    at eval (webpack:///./index.js?:4:16)

Firefox 71.0 (64-bit) fails these different three

test_hit_test_point_basic_0

piet_web_example.js:506:13
text pos 4: Some(HitTestTextPosition { point: (18.0, 0.0), metrics: HitTestMetrics { text_position: 4 } }) piet_web_example.js:506:13
text pos 5: Some(HitTestTextPosition { point: (21.0, 0.0), metrics: HitTestMetrics { text_position: 5 } }) piet_web_example.js:506:13
RuntimeError: "unreachable executed"
    run webpack:///./dist/piet_web_example.js?:68
    <anonymous> webpack:///./index.js?:4

test_hit_test_point_complex_0

piet_web_example.js:506:13
text pos 2: Some(HitTestTextPosition { point: (5.333333492279053, 0.0), metrics: HitTestMetrics { text_position: 2 } }) piet_web_example.js:506:13
text pos 9: Some(HitTestTextPosition { point: (11.333333015441895, 0.0), metrics: HitTestMetrics { text_position: 9 } }) piet_web_example.js:506:13
text pos 10: Some(HitTestTextPosition { point: (17.33333396911621, 0.0), metrics: HitTestMetrics { text_position: 10 } }) piet_web_example.js:506:13
text pos 14: Some(HitTestTextPosition { point: (26.16666603088379, 0.0), metrics: HitTestMetrics { text_position: 14 } }) piet_web_example.js:506:13
RuntimeError: "unreachable executed"
    run webpack:///./dist/piet_web_example.js?:68
    <anonymous> webpack:///./index.js?:4

test_hit_test_point_complex_1

piet_web_example.js:506:13
RuntimeError: "unreachable executed"
    run webpack:///./dist/piet_web_example.js?:68
    <anonymous> webpack:///./index.js?:4

Discussion

Does anyone have ideas why the tests might be failing? @hwchen

Provide install instructions for cairo-rs dependency which requires provided cairo dll

Cairo is quite portable, and it is quite feasible to build on other systems. However, the cairo-rs crate seems to expect a library to be provided, rather than building it from sources. On Windows, I've been using prebuilt binary releases from cairo-windows.

We should file an issue with cairo-rs to see if we can make this a bit more friendly.

Until then, we should provide install instructions so that people don't get tripped up on this.

d2d: Use specialized functions for fill/draw

The direct2d backend converts every shape to a path and build a geometry from this.
The API provides specialized functions for most of the shapes which should be prefered.

(TODOs in the code, this issue is mostly for tracking these)

add #[doc(hidden)] to Null-prefixed types

these are for testing and don't need public documentation.

Switch to Result based error handling

Currently the codebase is full of unwraps and only assumes a happy path. We should soon switch to a real error handling using Result.

Text::Coord v.s. RenderContext::Coord?

The Text and RenderContext traits both have Coord and TextLayout associated types. The former also has a type Text: Text<TextLayout = Self::TextLayout>; associated type, with an equality that ensures that the two TextLayout types are the same.

But there is no similar constraint to ensure that the two Coord types are the same. Is this deliberate?

Expose FontWeight please

Currently, all are hard coded to normal.

docs todo for rawd2d

• module level docs for d3d
• crate level doc to direct people to d2drendercontext::new as entry point.

Rendering on a window

I am very new to graphics rendering and stuff so this is probably a very uninformed question. But, could someone kindly point me towards how I can use Piet to actually render graphics on a screen?

What kind of other pieces to I need to put together along with Piet to get a rectangle or something on a window?

migrate to azure pipelines for CI?

this is working out well enough for druid, and travis is taking forever.

Line breaks does not work with Cairo

While trying to write a multi-line label widget for druid I noticed that line breaks work fine on Windows but not at all on Linux.

Behavior on Windows:

• Breaks lines on whitespace as needed
• \n always causes a break

Behavior on Linux:

• No breaks at all

Here is an example as an interactive druid application of this behavior:

// [dependencies.druid]
// git = "https://github.com/xi-editor/druid"

use druid::{
    piet::{FontBuilder, PietTextLayout, RenderContext, Text, TextLayout, TextLayoutBuilder},
    widget::prelude::*,
    AppLauncher, Point, WidgetExt, WindowDesc,
};

// a fudgey way to get an approximate line height from a font size
const LINE_HEIGHT_FACTOR: f64 = 1.2;
// a fudgey way of figuring out where to put the baseline, relative to line height
const BASELINE_GUESS_FACTOR: f64 = 0.8;
// added padding between the edges of the widget and the text.
const LABEL_X_PADDING: f64 = 2.0;

struct WrapLabel {
    layout: Option<PietTextLayout>,
}

impl WrapLabel {
    pub fn new() -> Self {
        WrapLabel { layout: None }
    }
}

impl Widget<String> for WrapLabel {
    fn event(&mut self, _ctx: &mut EventCtx, _event: &Event, _data: &mut String, _env: &Env) {}
    fn lifecycle(
        &mut self,
        _ctx: &mut LifeCycleCtx,
        _event: &LifeCycle,
        _data: &String,
        _env: &Env,
    ) {
    }
    fn update(&mut self, _ctx: &mut UpdateCtx, _old_data: &String, _data: &String, _env: &Env) {}

    fn layout(
        &mut self,
        ctx: &mut LayoutCtx,
        bc: &BoxConstraints,
        data: &String,
        env: &Env,
    ) -> Size {
        let font_name = env.get(druid::theme::FONT_NAME);
        let font_size = env.get(druid::theme::TEXT_SIZE_NORMAL);
        let font = ctx
            .text()
            .new_font_by_name(font_name, font_size)
            .build()
            .unwrap();

        let layout = ctx
            .text()
            .new_text_layout(&font, &data, bc.max().width)
            .build()
            .unwrap();

        let width = layout.width();
        self.layout = Some(layout);

        bc.constrain(Size::new(width, font_size * LINE_HEIGHT_FACTOR))
    }

    fn paint(&mut self, ctx: &mut PaintCtx, _data: &String, env: &Env) {
        let font_size = env.get(druid::theme::TEXT_SIZE_NORMAL);
        let text_layout = self.layout.as_ref().unwrap();
        let line_height = font_size * LINE_HEIGHT_FACTOR;

        let origin = Point::new(LABEL_X_PADDING, line_height * BASELINE_GUESS_FACTOR);
        let color = env.get(druid::theme::LABEL_COLOR);

        ctx.draw_text(&text_layout, origin, &color);
    }
}

fn make_widget() -> impl Widget<String> {
    WrapLabel::new().center()
}

fn main() {
    let window = WindowDesc::new(make_widget).window_size((400.0, 400.0));
    let state =
        "This is some long text which should cause the WrapLabel to display multiple lines.\nThis one particualarly.".into();
    AppLauncher::with_window(window)
        .launch(state)
        .expect("launch failed");
}

And here one using only piet:

// [dependencies]
// piet = { git = "https://github.com/linebender/piet" }
// piet-common = { git = "https://github.com/linebender/piet", package = "piet-common", features = 

["png"]}
use piet::kurbo::Point;
use piet::{Color, RenderContext, Text, FontBuilder, TextLayoutBuilder};
use piet_common::Device;

// a fudgey way to get an approximate line height from a font size
const LINE_HEIGHT_FACTOR: f64 = 1.2;
// a fudgey way of figuring out where to put the baseline, relative to line height
const BASELINE_GUESS_FACTOR: f64 = 0.8;
// added padding between the edges of the widget and the text.
const LABEL_X_PADDING: f64 = 2.0;

/// Feature "png" needed for save_to_file() and it's disabled by default for optional dependencies
/// cargo run --example png --features png
fn main() {
    let text: String =
        "This is some long text which should cause the WrapLabel to display multiple lines.\nThis one particualarly.".into();

    let mut device = Device::new().unwrap();
    let width = 400;
    let height = 400;
    let mut bitmap = device.bitmap_target(width, height, 1.0).unwrap();
    let mut rc = bitmap.render_context();
    rc.clear(Color::WHITE);


    let font_name = "Arial";
    let font_size = 14.0;
    let font = rc
            .text()
            .new_font_by_name(font_name, font_size)
            .build()
            .unwrap();
    let layout = rc
            .text()
            .new_text_layout(&font, &text, 400.0)
            .build()
            .unwrap();
    let line_height = font_size * LINE_HEIGHT_FACTOR;

    let origin = Point::new(LABEL_X_PADDING, line_height * BASELINE_GUESS_FACTOR);
    let color = Color::rgb(0.0, 0.0, 0.0);

    rc.draw_text(&layout, origin, &color);


    rc.finish().unwrap();
    std::mem::drop(rc);

    bitmap
        .save_to_file("temp-image.png")
        .expect("file save error");
}

Make piet capable of being a back-end for resvg

The resvg project has a very helpful list of requirements for back-ends. I think piet is now at a point where it would be possible to start coding such a back-end, and it makes sense to carefully review where we stand.

Imaging model and blending/compositing

Right now, piet effectively implements a painter's imaging model, where each object is alpha-composited on top of pixels on the canvas surface. There is an entire class of operations - group opacity, nonstandard blend modes, using an image as a mask, and of course filters, that do not fit into this imaging model. I'm hesitating slightly at this point, because I want to make sure to expose this functionality in a way that piet back-ends can efficiently implement them, and also doesn't add too much complexity and burden for implementers.

The imaging model of resvg is fundamentally a tree, with nodes representing various blend and mask operations. The way resvg works with, say, the Cairo back-end, is to treat the graphics library as a source for pixel data, and do the blending and filter operations in software. We can do this approach, and it is probably the shortest path to getting a working integration with resvg. Basically it requires exposing an interface for creating an image surface, and for getting RGBA pixels out of the image surface. This is is already done to some extent in the demo apps for writing image data, but (significantly) not in the web back-end, as the demo app has a one-way flow of data, drawing on the screen.

When rendering is done on GPU, getting pixel data back is inefficient. Doing blending operations in software is doubly so, especially for pointwise operations (including alpha groups, mask, and blend modes), all of which can be implemented efficiently in GPU shaders. The Direct2D library has a rich set of effects, all of which seem to be available on Windows 7 with platform update. Not using these is leaving performance on the table.

That said, requiring the complete set of effects for all piet back-ends may be onerous. Further, many of these effects are of fairly limited use in UI contexts. If I were to prioritize, I'd pick group opacity, as that is something that's quite useful for building UI, and is also reasonably easy to implement. I think it can serve as a bit of a proxy for potentially more sophisticated blend modes.

For blend modes, it's a bit confusing what functionality to support. resvg seems to implement a limited set, consisting of Normal, Multiply, Screen, Darken, and Lighten. The SVG Compositing spec has a much larger set. It also specifies complex options such as "enable-background" and "knock-out". The latter is effectively a second alpha channel, and seems to be imported from PDF 1.4 transparency. The complexity of implementing that for the web back-end would be considerable, and doing so performantly is challenging. But I think expanding the set of "comp-op" values is fairly straightforward, as all 3 current back-ends seem to support the same basic set.

What I'm leaning towards is exposing a "get RGBA pixels" interface so resvg can use software techniques for blending, and punting on the complex blends, with the likely exception of group opacity.

Sophisticated text

There's no question piet needs fancier text (partially captured in #8 and #10), and there are number of text things in the resvg requirements list. I would like to punt on this for now. Part of the issue is that the web canvas back-end has only primitive text, as does the current Cario "toy text" implementation. For web canvas, we'll have to return "not implemented" error results for some queries, or fake it somehow. For the others, I want to get a solid text solution, but feel this is orthogonal to most of the rest of the resvg requirements

Gradients

To some extent, gradients are the easiest problem, though of course the devil is always in the details. I propose to subsume gradients under the existing Brush associated type, and simply add more builders. It is likely we'll have to do some workarounds for rendering quirks.

In any case, I'm posting this issue to get a read on whether it makes sense to start implementation work on integrating resvg and piet, and what needs to be done on the piet side. @RazrFalcon is of course invited to provide feedback as well.

Remove double result in text methods

Doing anything with text requires a lot of result unwrapping. Bascially, creating the builder can fail, as can the result of building. This is true for both Font and TextLayout. As part of our ergonomics push, we should make the first infallible. If there's a platform error in constructing the lower level builder object, we could either panic (if it's basically a "totally out of resources" or "configuration is irrecoverably borked" situation), or retain an error indication in the builder which would get reported on the call to the final builder.

Add a piet::utils module for code common to different backends

Not just the stuff below, but also the blurred rect code in coregraphics & piet, and other shared code for writing images. Probably other stuff too.

I'd recommend calling into a helper function; https://github.com/linebender/piet/blob/master/piet-direct2d/src/text.rs#L230 is reasonable though we should have better ways of avoiding cross-crate duplication.

Originally posted by @raphlinus in #176

compilation fails on master with --all-features enabled

This is a collision between #178 and #181, patch shortly.

Get rid of Coord and Point associated types

These are more hassle than they're worth; it would be nice if we could just use Point directly, or at least something like impl Into<Point>.

Which library should replace the Cairo toy text API for text layout and rendering?

For simplicity, the back-end currently uses the toy text API in Cairo. Essentially, this means there is no shaping, so complex scripts won't render correctly at all, and Latin will be missing kerning, ligatures, and other refinements. According to the docs, "Any serious application should avoid them."

Fairly soon, @raphlinus hopes to have some type of higher-level text in place. One possibility is pango. From what @raphlinus can tell, this should work well on Linux, but since it has a non-optional glib dependency, it might be non-trivial to get it building portably. It's also not clear how well this approach handles discovering system fonts.

Another possibility is to use HarfBuzz more directly, using the rust-harfbuzz bindings. This will require more work for font discovery and selection but has the possibility to be considerably more native. A good Rust-native candidate for system font discovery is font-kit.

A third possibility is to adapt libTXT from Flutter. This is a state of the art text layout library, with considerable investment in making it work well on mobile. However, it is in C++ and thus at the very least will need nontrivial work to make good Rust bindings.

Basic Testing

What would a basic testing infrastructure look like. Due to the nature of the code and the need for rapid iteration, unit testing won't get us very far.

Can we set up some sort of testing framework where simple graphics programs built against the latest version of piet and then diffed with known good images. This could work with the direct-2d and cairo backends but is problematic with a web based backend. How robust would this testing be? Would it easily miss real issues or report failures on things that aren't actually failures?