ollix/metalnanovg

The Metal port of NanoVG.

metalnanovg's Issues

Option to present drawable?

Might be that I don't understand metal well enough yet, but I was wondering if there could be an option whether MetalNanoVG calls presentDrawable or not. I have been trying to render ImGui on top of a MetalNanoVG scene, which in its metal example implementation, also calls present drawable. currently in order to avoid flickering I need to comment out the MetalNanoVG presentDrawable

objc_release taking significant cpu time in profile

I'm noticing that objc_release is taking significant cpu time.

Looking at the generated assembly for mtlnvg__convexFill, it seems that a retain/release is due to using a @property for pipelineState.

Crash: Blending enabled for render target 0, pixelformat MTLPixelFormatInvalid

I just got a report from a beta tester of my app that after a few hours leaving the interface on mac, the app crashed with this error. Currently, I can't reproduce this bug on my machine. I guess checking if colorTexture has an invalid pixel format could fix this issue, or something else might be wrong here.

Process:               Youlean Loudness Meter 2 [14160]
Path:                  /Applications/Youlean Loudness Meter 2.app/Contents/MacOS/Youlean Loudness Meter 2
Identifier:            com.Youlean.standalone.Youlean-Loudness-Meter-2
Version:               2.4.1 (2.4.1)
Code Type:             X86-64 (Native)
Parent Process:        ??? [1]
Responsible:           Youlean Loudness Meter 2 [14160]
User ID:               501

Date/Time:             2020-09-02 22:09:47.673 +0100
OS Version:            Mac OS X 10.12.6 (16G2136)
Report Version:        12
Anonymous UUID:        52E837E2-0DED-4E93-D3A1-BBEA57065E3E


Time Awake Since Boot: 26000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
-[MTLRenderPipelineDescriptorInternal validateWithDevice:]:1798: failed assertion `Blending is enabled for render target 0; however, the pixelformat MTLPixelFormatInvalid for this render target is not blendable.'
 

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib         0x00007fffa02e7d42 __pthread_kill + 10
1   libsystem_pthread.dylib       0x00007fffa03d5457 pthread_kill + 90
2   libsystem_c.dylib             0x00007fffa024d420 abort + 129
3   libsystem_c.dylib             0x00007fffa0214893 __assert_rtn + 320
4   com.apple.Metal               0x00007fff8de69e0b MTLReportFailure + 822
5   com.apple.Metal               0x00007fff8de68c26 -[MTLRenderPipelineDescriptorInternal validateWithDevice:] + 988
6   com.apple.Metal               0x00007fff8de5b20b -[MTLCompiler newRenderPipelineStateWithDescriptor:options:reflection:error:completionHandler:] + 70
7   com.apple.Metal               0x00007fff8de430f0 -[_MTLDevice newRenderPipelineStateWithDescriptor:error:] + 65
8   com.Youlean.standalone.Youlean-Loudness-Meter-2 0x0000000100469803 -[MNVGcontext updateRenderPipelineStatesForBlend:pixelFormat:] + 467
9   com.Youlean.standalone.Youlean-Loudness-Meter-2 0x00000001004686b0 -[MNVGcontext renderFlush] + 896
10  com.Youlean.standalone.Youlean-Loudness-Meter-2 0x00000001002b77da nvgEndFrame + 26
11  com.Youlean.standalone.Youlean-Loudness-Meter-2 0x0000000100258d7a ygraphics::yg_end_draw_frame(yg_t*, double, double, double, double) + 362
12  com.Youlean.standalone.Youlean-Loudness-Meter-2 0x000000010033e848 -[IGraphicsCocoa_vYoulean_Loudness_Meter_sa render] + 184
13  com.Youlean.standalone.Youlean-Loudness-Meter-2 0x000000010033d9e8 -[IGraphicsCocoa_vYoulean_Loudness_Meter_sa displayLayer:] + 88
14  com.apple.QuartzCore           0x00007fff9057fa86 CA::Layer::display() + 142
15  com.apple.QuartzCore           0x00007fff90574036 CA::Layer::display_if_needed(CA::Transaction*) + 572
16  com.apple.QuartzCore           0x00007fff90574161 CA::Layer::layout_and_display_if_needed(CA::Transaction*) + 35
17  com.apple.QuartzCore           0x00007fff90569918 CA::Context::commit_transaction(CA::Transaction*) + 280
18  com.apple.QuartzCore           0x00007fff9045e701 CA::Transaction::commit() + 475
19  com.apple.QuartzCore           0x00007fff9045f11c CA::Transaction::observer_callback(__CFRunLoopObserver*, unsigned long, void*) + 108
20  com.apple.CoreFoundation       0x00007fff8a590857 __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23
21  com.apple.CoreFoundation       0x00007fff8a5907c7 __CFRunLoopDoObservers + 391
22  com.apple.CoreFoundation       0x00007fff8a570fd6 CFRunLoopRunSpecific + 454
23  com.apple.HIToolbox           0x00007fff89acfebc RunCurrentEventLoopInMode + 240
24  com.apple.HIToolbox           0x00007fff89acfcf1 ReceiveNextEventCommon + 432
25  com.apple.HIToolbox           0x00007fff89acfb26 _BlockUntilNextEventMatchingListInModeWithFilter + 71
26  com.apple.AppKit               0x00007fff88064a04 _DPSNextEvent + 1120
27  com.apple.AppKit               0x00007fff887e07ee -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2796
28  com.apple.AppKit               0x00007fff8805938b -[NSApplication run] + 926
29  com.apple.AppKit               0x00007fff88023dbe NSApplicationMain + 1237
30  libdyld.dylib                 0x00007fffa01b9235 start + 1

mtlnvg__renderDelete crash (old version)

I'm experiencing an occasional crash when quitting my application and destroying the Nanovg context. There is a message on the console,

BUG IN CLIENT OF LIBDISPATCH: Semaphore object deallocated while in use

which led me to this workaround https://lists.apple.com/archives/cocoa-dev/2014/Apr/msg00484.html

Not yet sure if this completely fixes it
iPlug2/iPlug2@2526e83

I'm currently stuck with an older version of MetalNanoVG, because with the latest I am getting some deadlocks that didn't happen before (iPlug2/iPlug2#152 but maybe due to incorrect use). I need to investigate in a barebones app. However, it looks like the dispatch_semaphore_create call is similar

would be possible to explain the usage of libdispatch in MetalNanoVG, so I can try and better understand the problems?

Flicker on drawableSizeWillChange

I am getting intermittent "chop" in draws that occur immediately after window resizing. By that, I mean that the rendering flickers, with every other frame drawn at an offset, and somewhat random location. My guess is that this occurs when the drawable size changes between nvgBeginFrame and nvgEndFrame. It is fixed by setting MTKView.autoResizeDrawable = false, but then of course the rendering just scales with the window size.

How can you force the drawable size to change only at appropriate times? Any ideas?

static analyzer reports spurious memory leak

The MTLDevice is a singleton (I checked), but it would be good to fix this warning anyway. I think all that is needed is a call to release after assigning to metalLayer->device.

s_framebuffer isn't thread safe

s_framebuffer is a global variable and isn't thread safe.

Furthermore, mnvgDeleteFramebuffer leaves s_framebuffer dangling.

could path tessellation be done using the GPU?

In my profiles, bezier tessellation is consuming considerable CPU time, while the GPU is mostly idle.

I've tried optimizing bezier tessellation on the front-end with little success.

I was wondering about the feasibility of supporting path tessellation (nvg__flattenPaths) on the GPU. This would involve a change to the front-end, but could be easily made compatible with other back-ends.

Bezier tessellation on the GPU could work as follows. The input is a series of Bezier segments to be flattened. A compute kernel is responsible for flattening. Each segment is processed by a thread. If the segment is flat (either by a flag on the segment, or a test if the flag isn't set), then the segment is outputted. If not flat, two segments are outputted according to bezier subdivision. Atomic operations would be used to append to the output buffer (note that this wouldn't preserve segment order, and that might be an issue. If it is, then a prefix-sum could be used to generate output indices).

Here is some pseudocode:

kernel void subdivideBezier(device BezierSegment* input,
                                               device BezierSegment* output,
                                               device atomic_uint& outCount,
                                               uint gid [[ therad_index_in_grid ]]) {

  BezierSegment out[2];
  int n=1;

  BezierSegment s = input[gid];

  if(s.flat) {
     out[0] = s;
  } else if(flattnessTest(s)) {
     s.flat = true;
     out[0] = s;
  } else {
     out[0] = leftChild(s);
     out[1] = rightChild(s);
     n = 2;
   }

   int i = atomic_add(&outCount, n);
   output[i] = out[0];
   if(n == 2) output[I+1] = out[1];
}

This kernel would be run in multiple passes over the curves according to the maximum subdivision level (hardcoded at 10 in the front-end). Buffers would be sized to accept the maximum number of segments (2^10).

The back end would have to handle generating segments for strokes, line-cap, line-end, etc.

The other GPU path rendering approaches I'm aware of involve sophisticated fragment shaders, so this may be easier to implement.

It may also be worth looking at https://github.com/linebender/piet-metal. It currently doesn't seem to handle curves on the GPU (Beziers are subdivided on the CPU), but takes a distance-field approach to path rendering.

See also http://hhoppe.com/ravg.pdf and https://www.microsoft.com/en-us/research/wp-content/uploads/2005/01/p1000-loop.pdf.

Metal shader to bit code

Hey, how do you compile metal shader to bit code?

Possibility for variable-width strokes?

Would it be at all possible to extend the line rendering to support per-point stroke widths? I'd like to be able to draw using the Apple pencil and modulate the thickness of the curves continuously. Currently, nanovg applies a single width per-stroke, with no transitions possible. (The same goes for color.)

How might I be able to write this extension myself, since I realize it's not supported in the original?

MetalNanoVG changes drawable pixel format to MTLPixelFormatBGRA8Unorm

Before drawing, the pixel format is: MTLPixelFormatBGRA10_XR_sRGB, as I set in the storyboard.

After drawing, pixel format changed to: MTLPixelFormatBGRA8Unorm.

Looks like it's getting changed in mtlnvg__renderCreate.

Should MetalNanoVG respect the existing pixel format?

block implicitly retains self

I'm getting this warning:

nanovg_mtl.m:1387:33: warning: block implicitly retains 'self'; explicitly mention 'self' to indicate this is intended behavior [-Wimplicit-retain-self]
      dispatch_semaphore_signal(_semaphore);
                                ^
                                self->

Easy to fix :)

Hangs on High Sierra

nvgEndFrame(); hangs the whole desktop for me for many seconds on High Sierra when using MetalNanoVG, and nothing ends up being rendered.

Memory leaks on MacOS Mojave

I created a simple app with nvgBeginFrame and nvgEndFrame only. And there are memory leaks there. I guess It's somehow related with mtlnvg__renderFlush. When I comment code there it doesn't produce the leaks.

VisionOS support?

Hello,

Do you know what it would take to add visionOS support to MetalNanoVG?

Thanks!

mtlnvg__renderCreate: MTLLibrary leaked

With stencil strokes enabled, strokewidth = 1. is invisible on non-retina screen

This didn't used to be the case. Here is a test project demonstrating the problem:

https://github.com/olilarkin/TestNVGMetal/blob/master/TestNVGMetal/Renderer.m#L13

Why is ARC not supported?

It seems like there would be no problem supporting it. And almost all ObjC code now uses ARC.

Procedural Text without generating new textures?

I need to render text procedurally in every frame of my application’s runtime. It seems that nanovg generates a new atlas every time I have a new text line or zoom. That causes major performance and memory issues. Is there a way to circumvent this?
Also, I did look into the original nvg headers, but it’s unclear when the redundant texture generation happens. I always thought for procedural text you’d just render a quad per character in the correct space, not generate a bitmap and texture per text line.

The possibility of partnership with the Avalonia project

It would be great for the community if a partnership happens, as Avalonia lacks an Accelerated Hardware API (OpenGL).
Mainly for OpenGL targets like Android, IOS, MacOs, Linux, WASM (WebAssembly)

AvaloniaUI/Avalonia#1387 (comment)