tobski/simple_vulkan_synchronization

A single-header library with a simplified interface for Vulkan synchronization

Simplified Vulkan Synchronization

In an effort to make Vulkan synchronization more accessible, I created this stb-inspired single-header library in order to somewhat simplify the core synchronization mechanisms in Vulkan - pipeline barriers and events.

Rather than the complex maze of enums and bitflags in Vulkan - many combinations of which are invalid or nonsensical - this library collapses this to a much shorter list of 40 distinct usage types, and a couple of options for handling image layouts.

Use of other synchonization mechanisms such as semaphores, fences and render passes are not addressed in this API at present.

Usage

#define the symbol THSVS_SIMPLER_VULKAN_SYNCHRONIZATION_IMPLEMENTATION in one C/C++ file before the #include of the header; the implementation will be generated in that file.

Version

alpha.9

Alpha.9 adds the thsvsGetAccessInfo function to translate access types into a thsvsVkAccessInfo.

Version History

alpha.8

Alpha.8 adds a host preinitialization state for linear images, as well as a number of new access sets for extensions released since the last update.

alpha.7

Alpha.7 incorporates a number of fixes from @gwihlidal, and fixes handling of pipeline stages in the presence of multiple access types or barriers in light of other recent changes.

alpha.6

Alpha.6 fixes a typo (VK_ACCESS_TYPE_MEMORY_READ|WRITE_BIT should have been VK_ACCESS_MEMORY_READ|WRITE_BIT), and sets the pipeline stage src and dst flag bits to VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT and VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT during initialization, not 0 as per alpha.5

alpha.5

Alpha.5 now correctly zeroes out the pipeline stage flags before trying to incrementally set bits on them... common theme here, whoops.

alpha.4

Alpha.4 now correctly zeroes out the access types before trying to incrementally set bits on them (!)

alpha.3

Alpha.3 changes the following:

Uniform and vertex buffer access in one enum, matching D3D12_RESOURCE_STATE_VERTEX_AND_CONSTANT_BUFFER:

• THSVS_ACCESS_ANY_SHADER_READ_UNIFORM_BUFFER_OR_VERTEX_BUFFER

Color read and write access, matching D3D12_RESOURCE_STATE_RENDER_TARGET:

• THSVS_ACCESS_COLOR_ATTACHMENT_READ_WRITE

Also the "THSVS_ACCESS_*_SHADER_READ_SAMPLED_IMAGE" enums have been renamed to the form "THSVS_ACCESS_*_SHADER_READ_SAMPLED_IMAGE_OR_UNIFORM_TEXEL_BUFFER"

alpha.2

Alpha.2 adds four new resource states for "ANY SHADER ACCESS":

• THSVS_ACCESS_ANY_SHADER_READ_UNIFORM_BUFFER
• THSVS_ACCESS_ANY_SHADER_READ_SAMPLED_IMAGE
• THSVS_ACCESS_ANY_SHADER_READ_OTHER
• THSVS_ACCESS_ANY_SHADER_WRITE

alpha.1

Alpha.1 adds three new resource states:

• THSVS_ACCESS_GENERAL (Any access on the device)
• THSVS_ACCESS_DEPTH_ATTACHMENT_WRITE_STENCIL_READ_ONLY (Write access to only the depth aspect of a depth/stencil attachment)
• THSVS_ACCESS_STENCIL_ATTACHMENT_WRITE_DEPTH_READ_ONLY (Write access to only the stencil aspect of a depth/stencil attachment)

It also fixes a couple of typos, and adds clarification as to when extensions need to be enabled to use a feature.

alpha.0

This is the very first public release of this library; future revisions of this API may change the API in an incompatible manner as feedback is received. Once the version becomes stable, incompatible changes will only be made to major revisions of the API - minor revisions will only contain bugfixes or minor additions.

Memory Allocation

The thsvsCmdPipelineBarrier and thsvsCmdWaitEvents commands allocate temporary storage for the Vulkan barrier equivalents in order to pass them to the respective Vulkan commands.

These use the THSVS_TEMP_ALLOC(size) and THSVS_TEMP_FREE(x) macros, which are by default set to alloca(size) and ((void)(x)), respectively. If you don't want to use stack space or would rather use your own allocation strategy, these can be overridden by defining these macros in before #include-ing the header file with THSVS_SIMPLER_VULKAN_SYNCHRONIZATION_IMPLEMENTATION defined.

I'd rather avoid the need for these allocations in what are likely to be high-traffic commands, but currently just want to ship something - may revisit this at a future date based on feedback.

Expressiveness Compared to Raw Vulkan

Despite the fact that this API is fairly simple, it expresses 99% of what you'd actually ever want to do in practice. Adding the missing expressiveness would result in increased complexity which didn't seem worth the tradeoff - however I would consider adding something for them in future if it becomes an issue.

Here's a list of known things you can't express:

• Execution only dependencies cannot be expressed. These are occasionally useful in conjunction with semaphores, or when trying to be clever with scheduling - but their usage is both limited and fairly tricky to get right anyway.
• Depth/Stencil Input Attachments can be read in a shader using either VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL or VK_IMAGE_LAYOUT_DEPTH_STENCIL_READ_ONLY_OPTIMAL - this library always uses VK_IMAGE_LAYOUT_DEPTH_STENCIL_READ_ONLY_OPTIMAL. It's possible (though highly unlikely) when aliasing images that this results in unnecessary transitions.

Error Checks

By default, as with the Vulkan API, this library does NOT check for errors. However, a number of optional error checks (THSVS_ERROR_CHECK_*) can be enabled by uncommenting the relevant #defines. Currently, error checks simply assert at the point a failure is detected and do not output an error message. I certainly do not claim they capture all possible errors, but they capture what should be some of the more common ones. Use of the Vulkan Validation Layers in tandem with this library is strongly recommended: https://github.com/KhronosGroup/Vulkan-LoaderAndValidationLayers

Issues

This header was clean of warnings using -Wall as of time of publishing on both gcc 4.8.4 and clang 3.5, using the c99 standard.

There's a potential pitfall in thsvsCmdPipelineBarrier and thsvsCmdWaitEvents where alloca is used for temporary allocations. See Memory Allocation for more information.

Testing of this library is so far extremely limited with no immediate plans to add to that - so there's bound to be some amount of bugs. Please raise these issues on the repo issue tracker, or provide a fix via a pull request yourself if you're so inclined.

simple_vulkan_synchronization's Issues

Inefficiency in access masks

Hey,

To be more efficient, you need to change this:

typedef struct ThsvsVkAccessInfo {
    VkPipelineStageFlags    stageMask;
    VkAccessFlags           accessMask;
    VkImageLayout           imageLayout;
} ThsvsVkAccessInfo;

into something like this:

typedef struct ThsvsVkAccessInfo {
    VkPipelineStageFlags    stageMask;
    VkAccessFlags           dstAccessMask;
    VkAccessFlags           srcAccessMask;
    VkImageLayout           imageLayout;
} ThsvsVkAccessInfo;

For example, for THSVS_ACCESS_COLOR_ATTACHMENT_READ_WRITE, you need to use VK_ACCESS_COLOR_ATTACHMENT_READ_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT as the dstAccessMask when transitioning into the state, but when transitioning out, you just need VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT for srcAccessMask, as WAR hazards are satisfied with execution barriers alone.

(In short, srcAccessMask never needs a read access).

interface suggest: thsvsGetAccessInfo

I find the ThsvsAccessType is absolutely a great way to encapsulate synchronizations, but on top of the current design, sometimes we would need a little more fine-grained control, e.g. translate a set of access types to essentially, thsvsVkAccessInfo.

For instance, the current interface becomes inadequate when we try to incorporate subpass dependencies into the system. Say each render pass attachment is given two sets of access types (begin/end), and we'd want to map these to both initial/final layouts and external subpass dependencies.

With something like thsvsGetAccessInfo we can get all the flexibility we'd need for any other synchronization primitives that are not yet fully covered here in the library.

Wrong dstStage

The following test seems to be wrong because the image layout transition is counted as a write operation and therefore it's required to specify VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT in the dstAccessMask.

    image_barrier_test("Graphics fragment read from sampled image, Graphics write to color attachment",
                        THSVS_ACCESS_FRAGMENT_SHADER_READ_SAMPLED_IMAGE_OR_UNIFORM_TEXEL_BUFFER,                        
                        THSVS_ACCESS_COLOR_ATTACHMENT_WRITE,
                        VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT,
                        VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
                        0,
                        0,
                        VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL,
                        VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL);

The problem might be in thsvsGetVulkanImageMemoryBarrier in this block:

        if (pVkBarrier->srcAccessMask != 0)
            pVkBarrier->dstAccessMask |= pNextAccessInfo->accessMask;

I think this code should run after determining the layout to see if a layout transition is specified and then set the dstaccessMask even when the srcAccessMask is 0.

For reference here is this exact synchronization example:

https://github.com/KhronosGroup/Vulkan-Docs/wiki/Synchronization-Examples-(Legacy-synchronization-APIs)#first-draw-samples-a-texture-in-the-fragment-shader-second-draw-writes-to-that-texture-as-a-color-attachment

Enum members can be updated

While porting this library to another language: https://github.com/jayrulez/Bulkan/blob/master/Bulkan.Utilities/src/SimpleVulkanSynchronization.bf

I came across 3 enum names that can be updated to the standard KHR variants:

VK_PIPELINE_STAGE_SHADING_RATE_IMAGE_BIT_NV => VK_PIPELINE_STAGE_FRAGMENT_SHADING_RATE_ATTACHMENT_BIT_KHR
VK_ACCESS_SHADING_RATE_IMAGE_READ_BIT_NV =>	VK_ACCESS_FRAGMENT_SHADING_RATE_ATTACHMENT_READ_BIT_KHR
VK_IMAGE_LAYOUT_SHADING_RATE_OPTIMAL_NV => VK_IMAGE_LAYOUT_FRAGMENT_SHADING_RATE_ATTACHMENT_OPTIMAL_KHR

Build error (undefined *_NVX symbols) with Vulkan SDK 1.2.135.0

VK_PIPELINE_STAGE_COMMAND_PROCESS_BIT_NVX, VK_ACCESS_COMMAND_PROCESS_READ_BIT_NVX, and VK_ACCESS_COMMAND_PROCESS_WRITE_BIT_NVX are no longer defined in this week's Vulkan SDK. I naively assume they've morphed into *_PREPROCESS_*_NV but what do I know, I'm way out of the loop these days :)

Multiple writes clarification

simple_vulkan_synchronization/thsvs_simpler_vulkan_synchronization.h

Lines 936 to 937 in a4c7ede

	// Asserts that the access is a read, else it's a write and it should appear on its own.
	assert(prevAccess < THSVS_END_OF_READ_ACCESS || thBarrier.prevAccessCount == 1);

Currently we assert that pPrevAccesses and pNextAccesses can have at most one write access for each global barrier. Imagine a scenario like this:

1. RayTracing shader writes to buffer A
2. Compute shader writes to buffer B
3. Pipeline barrier
4. Compute shader reads from buffer A
5. RayTracing shader reads from buffer B

This should be a valid case, but the assertion is asking us to make it "appear on its own."
However, we only take one global memory barrier parameter in cmd_pipeline_barrier:

simple_vulkan_synchronization/thsvs_simpler_vulkan_synchronization.h

Line 1197 in a4c7ede

uint32_t memoryBarrierCount = (pGlobalBarrier != NULL) ? 1 : 0;

@Tobski Can you clarify what we could do in this case? Thanks.