auburnsounds/dplug

Audio plugin framework. VST2/VST3/AU/AAX/LV2 for Linux/macOS/Windows.

dplug's Issues

Screen space reflection

Would be cool.
It requires a separate buffer and pass for reflections, the problem is that the reflection pass would have to be full GUI all the time, so very slow.

Audio concept

Concept for buffers and processors, or both at once.

Started as dplug:audio (nothing depends on it for now). It's not easy.

Fix invalidation error

Uncovering the plugin window makes some artifacts.

Invalidation is not optimal on Windows

In particular, complex update regions should be used.

Visual defect around levels

Probably a bug in mipmap updates.

Ensure audio thread cannot be blocked by GC

It seems like it will be attached to the GC for now.

Multithreaded compositing

It is slow.

Overdraw

Easy to reproduce with automation. Background is not redrawn but control is redrawn.

Seen in at least Cubase / FLStudio / Orion

Multi-threaded onDraw for non-overlapping UIElement

Would speed up rendering a bit.

VST: registration race

There could be a race condition if registered threads are used for process callback.

Fix license

Boost + IPlug license should be used in a per-file basis

Do we need a blocking message queue on overflow in VST wrapper?

Currently lock-free, but maybe it should block instead of crashing when overflowing.

White highlights are annoying

VST: FLStudio problems when bridged

These problems are only with bridged plugins:

• connection lost with plugin: leads to the plugin being reopened, but no redraw / dirtying
• redraw does not work well when other window are invalidating an area

UI depth maps

Because shadows are desirable and baking them takes huge space, it would be desirable to have sort of a SSAO effect software renderered. So far a solution with a 8-bit depth map would be winning. This needs multiple render buffers and a deffered pass for SSAO (and another for lights?). All this requires lazy UI updates for elements and z-order.

Race condition on _dirtyRect

All access to _dirtyRect should be protected by a spinlock or mutex.

Color bleeding is not the right hue

Would need using emissive as coverage while mipmapping diffuse :(

Key light shadow casting create white highlights

Which are ugly and illogical

Handle sysex

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Chunks

Useful for versionned parameters

Faster rendering performance

Measurements based on rendering 2 bargraphs

Do nothing => 3.5ms
AO => 11ms
cast shadows => 5ms
secundary light => 1ms
specular => 2.5ms
skybox => 10ms
emissive => 20ms
Total => 58ms

Some things are very expensive, eg: a sqrt() cost 1ms.

Implement cpuRelax for GDC

Is currently a NOP.

Race in text rendering

Now that UIElement render concurrently, it creates problems when sharing something like a Font.

• either make Font thread-safe
• or find something else

Param should have listeners

Building with a good codegen on Windows

This won't be easy.

Windows

• DMD 64b => #35 At least one memory corruption, disappears without optimization. (edit: fixed)
• ~~GDC => hitting a stopper bug http://bugzilla.gdcproject.org/show_bug.cgi?id=189~~(EDIT: giving up with GDC since still at 2.066 and as such won't support asm blocks with nothrow @nogc)
• LDC with 64-bit target => ICE (ldc-developers/ldc#943) Lots of warnings

Assertion failed: !(Pair.first != Ty && Pair.second == Ty) && "mapping to a sour
ce type", file D:\Temp\LDC-BUILD\src\llvm-3.6.0.src\lib\Linker\LinkModules.cpp,
line 265

• LDC with 32-bit target => also an ICE

Error: 0x7536F1AB (0x0028FABC 0x0119070D 0x00000000 0x0028FBF0), tolower() + 0x2
B bytes(s)
0x7536FA44 (0x1AD22330 0x00000080 0x0119070D 0x00000000), _vsnprintf_l() + 0x54
bytes(s)
0x7536F9D8 (0x1AD22330 0x00000080 0x0119070D 0x0028FBF0), _vsnprintf() + 0x18 by
tes(s)
0x0042F599 (0x00000000 0x0195FA20 0x778A9950 0x0028FD70)
0x778A1EAA (0x0028FD0C 0x0028FD22 0x0028FD8C 0x6275642E), RtlFreeHeap() + 0x1BA
bytes(s)
0x7536B197 (0x00000023 0x0195FA20 0x0028FD60 0x778B47C4), malloc() + 0x57 bytes(
s)
0x6FC8A669 (0x1AABF6B0 0x1B15A8E8 0x1AABF6BC 0x6FC89FE7), _ZNSs4_Rep9_S_createEj
jRKSaIcE() + 0x59 bytes(s)
0x778B47C4 (0x1B15A8E8 0x1B15A8FE 0x0028FE1C 0x1B15A8E8), RtlFreeUnicodeString()
 + 0x34 bytes(s)
0x6FC8BAE7 (0x0DF25D68 0x0028FE14 0x0028FE03 0x00000000), _ZNSsC1EPKcRKSaIcE() +
 0x37 bytes(s)
0x01144E0B (0x00032A80 0x00000056 0x00000002 0x004013DE)
0x004013DE (0x7FFDE000 0x75237BE0 0x2B77CE40 0x0028FFDC)
0x75237C04 (0x7FFDE000 0x29D3E292 0x00000000 0x00000000), BaseThreadInitThunk()
+ 0x24 bytes(s)
0x778BAD1F (0xFFFFFFFF 0x778A0240 0x00000000 0x00000000), RtlInitializeException
Chain() + 0x8F bytes(s)
0x778BACEA (0x004014E0 0x7FFDE000 0x00000000 0x78746341), RtlInitializeException
Chain() + 0x5A bytes(s)

• LDC 32-bit with --combined (ldc-developers/ldc#943)

c:\d\ldc-32b\bin/../lib/libdruntime-ldc.a(dll.obj):(.text[__D4core3sys7windows3d
ll18dll_process_attachFPvbZb]+0x12): undefined reference to `_tlsstart'
c:\d\ldc-32b\bin/../lib/libdruntime-ldc.a(dll.obj):(.text[__D4core3sys7windows3d
ll18dll_process_attachFPvbZb]+0x1a): undefined reference to `_tlsend'
c:\d\ldc-32b\bin/../lib/libdruntime-ldc.a(dll.obj):(.text[__D4core3sys7windows3d
ll18dll_process_attachFPvbZb]+0x21): undefined reference to `_tls_callbacks_a'

No redraw while holding a slider in distort example

Fix shadow trail

Windows VST hosts compatibility

Current Status

• Orion: OK
• VSTHost: OK in 32 and 64-bit, crash at the end though (EDIT: only sometimes)
• Reaper: OK in 32 and 64-bit, even when bridged
• Live: looks OK
• Audacity (Windows): OK in 32-bit (does not seem to be available in 64-bit)
• Savihost: OK in 32-bit and 64-bit
• Presonus Studio One: OK in 32-bit and 64-bit
• Bitwig: One issue #33 OK in 32-bit
• Fruity Loops: issues when bridged #43, OK in 32-bit and 64-bit otherwise
• BlueCat Patchwork:
  • VST 32-bit hosting VST 32-bit: OK
  • VST 64-bit hosting VST 64-bit: OK
• Cubase: OK, 32-bit and 64-bit

TODO: Samplitude, Sonar, and other VST hosts first

Separate color bleed and AO

4th component in depth map could be "emissive strenght", to be looked at for color bleed. Allows many lights. there is no reasons low-depth widgets don't color bleed on high-depth things.
Make bleed multiscale and multiply it by the emissive factor.

AO could look more like textbook SSAO and be color-less. Currently high

Are spinlock really worth it?

Critical sections / mutexes usually have a lock-free part if no contention, and that is as fast as a spin-lock. If the lock is taken, then we'd better pause the thread than spin all that time.

Maybe synchronized can help in staying @nogc.

GC races

There are places where a GC cycle could occur in the audio thread given exceptional circumstances.

The only one I think there is : https://github.com/p0nce/dplug/blob/master/vst/dplug/vst/client.d#L675

VST Multiple instances failure with Bitwig

First bargraph not updating in distort example

Problems with dirty state being a single rectangle

The dirty rect of toplevel element expands when a widget is dirty.
The problem is that some of the area isn't redrawn, so some widget may flicker and be erased by toplevel redraw.

Solution:

• maybe make the dirty state a rectangle list
• maybe have every UIElement have an intermediate output buffer (meh)

Support parameter changes

For now the parameters are declared, but there is no way to know if they have changed in the audio callback.

Implement SpinlockedQueue

Presets

• Need a preset class which contain all parameters in float form.
• Number of preset is set upront on plugin creation, strange.
• Multiple output/input format to be expected: better let the presets be in memory as is.

PBR rendering

Remove shininess and implement roughness and metalness as documented on the Internet, especially UE4.
Get interested in PBR and see what can be done.
Rename diffuse into "albedo" which is the same concept in PBR.
Define defaults albedo colors from UE4 documentation.
Same for metals and materials.

Not scheduled because with envmap metals are metal enough.

@nogc dplug:dsp

For use in audio callbacks.
Including initializations and resets.

Support VST programs.

x64 DMD builds has rendering problems.

Text is odd looking and no lights.

Support GUI

But stay separated.

Note sure if the Windows invalidation code is optimal

InvalidateRect might be avoided since we check for dirty parts anyway. Find a way to trigger WM_PAINT without invalidation?

Priority queue for MIDI messages

Depends on #54

Several advantages:

• ensure processMidiMsg will be called with the right messages before processAudio (this is currently ensured by VST hosts but only as a convention)

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Not quite right lighting at widget boundaries

When an UIElement is updated, lighting is not correctly recomputed / displayed for neighbour pixels.

Mouse modifiers

For shift + click and control + click

Example makes no sound.

Distort example has text overdraw

The problem comes from drawing outside the dirty areas:

• either draw background first
• either turn the text into Labels

Avoid overlapping dirty rects

Currently, drawing is supposed to happen only in dirty areas, but those might be overlapping.

Envmap is broken