wunkolo/vulkanator

An Adobe After Effects sample project with Vulkan GPU acceleration

Vulkanator

Vulkanator is a sample project that demonstrates integrating the Adobe After Effects plugin SDK as Vulkan GPU acceleration in the form of a trivial plugin. The Adobe After Effects plugin SDK provides a sample-project called GLator to demonstrate how to integrate OpenGL into a native After Effects plugin. In the same spirit, Vulkanator demonstrates the same thing, but utilizing Vulkan.

Dependencies

• Cmake 3.7.0+
• Vulkan SDK
• Adobe After Effects plugin SDK

OSX

• MoltenVK

Building

Clone the repository with submodules:

git clone --recursive [email protected]:Wunkolo/Vulkanator.git

Download the Adobe After Effects plugin SDK and extract the required library folders into extern/Adobe After Effects SDK.

See extern/Adobe After Effects SDK for more information.

Windows

Visual Studio

Open CMakeLists.txt using Visual Studio's built-in support for opening CMake projects and build.

The compiled plugin will be found in build/bin/{Debug,Release}/Vulkanator.aex

Visual Studio Code

With the CMake Tools extension and C/C++ extensions, open the top level folder as a cmake-project and build. The compiled plugin will be found in build/bin/Vulkanator.aex

Mac

If you intend to make an ARM+Intel Universal Binary, be sure to install the Vulkan SDK from Lunarg and not the vulkan-loader from brew. Currently(2/25/2013) brew will only provide either an x86_64 or arm64 version of libvulkan and does not provide a universal-binary.

Xcode

An Xcode project may be generated through CMake via the Terminal

mkdir build
cd build
cmake -GXcode ..
open Vulkanator.xcodeproj

Visual Studio Code

With the CMake Tools extension and C/C++ extensions, open the top level folder as a cmake-project and build. The compiled plugin will be found in build/bin/Vulkanator.plugin

vulkanator's Issues

Multi-Frame Rendering

Basalt plugins have a pretty good pattern of MFR rendering on top of vulkan, but I want vulkanator to ultimately use something better.

The easiest way to have MFR implemented is to give each SmartRender context its own: CommandPool, DescriptorPool, DescriptorSet, and DeviceMemory for both the Input and Output and Uniforms. This is likely what "version 1" will have.

The improved pattern will utilize a pool of each type that is recycled and protected using a global VK_KHR_timeline_semaphore(Core in Vulkan 1.2) to indicate when a particular entry in each pool is available for re-use or not and allocating a new element in the case that all entries are in-use. This allows for resource allocation to scale with the amount of data that can be processed and acts as a sort of swapchain of GPU resources, scaling only by as fast as the system can let go of resources. A faster processor and GPU will have wide resource-pools. A slower and more serial processor and GPU will have narrower slots.

MoltenVK supports VK_KHR_timeline_semaphore KhronosGroup/MoltenVK#1124 and can safely utilize this pattern as well.

Conditionally utilize `VK_EXT_external_memory_host` to optimize bandwidth

This is something I've already done before in my other personal plugins, where rather than copying data into a staging ring-buffer to upload/download onto the GPU, VK_EXT_external_memory_host allows arbitrary pointers to be imported directly. This would remove an entire copy from the upload/download path by allowing vulkan to directly write into host-memory.

This would also have to introduce the concept of "trivially importable memory", where minImportedHostPointerAlignment is the same as the operating system's page-size (4096 bytes) allowing practically any pointer to be imported.

MoltenVK support is tracked over at KhronosGroup/MoltenVK#794.

Utilize `PF_State`-based caching for uploads

PF_GetCurrentState from PF_ParamUtilSuite3 can be utilized to get a pretty coherent "hash" of an input layer. With this, the uploaded contents of a "texture" derived from a PF_EffectWorld can be identified to have been uploaded already to the GPU and can be re-used.
A PF_State may be added to the current sequence-cache allowing cases such as a still-image input layer to only have to be uploaded to the GPU once and re-used much quicker.

Vulkanator/include/Vulkanator.hpp

Lines 91 to 116 in 9e38576

	struct SequenceCache
	{
	// If we are only using a smaller subset of the entire buffer
	// Then we should resize the entire buffer to be smaller if it is this
	// percentage smaller than the cached size
	// If an allocation comes in that is %15 smaller than the cache, then
	// the buffer to fit
	static constexpr glm::f32 ShrinkThreshold = 0.15f;
	// Cache for the staging buffer
	std::size_t StagingBufferSize = 0u;
	vk::UniqueBuffer StagingBuffer = {};
	vk::UniqueDeviceMemory StagingBufferMemory = {};
	// We use these structs so that we can easily "==" compare the image in
	// the cache with any new requests coming in
	vk::ImageCreateInfo InputImageInfoCache = {};
	vk::ImageCreateInfo OutputImageInfoCache = {};
	vk::UniqueImage InputImage = {};
	vk::UniqueDeviceMemory InputImageMemory = {};
	vk::UniqueImage OutputImage = {};
	vk::UniqueDeviceMemory OutputImageMemory = {};
	} Cache;

Additional testing may have to be done depending on the coherency of the PF_State such as changes in extents, quality, bit-depth, etc.