khronosgroup/spirv-reflect

SPIRV-Reflect is a lightweight library that provides a C/C++ reflection API for SPIR-V shader bytecode in Vulkan applications.

SPIRV-Reflect

SPIRV-Reflect is a lightweight library that provides a C/C++ reflection API for SPIR-V shader bytecode in Vulkan applications.

SPIRV-Reflect has been tested on Linux and Windows.

NEWS

• 2023-07-04 - We have removed support for Bazel.
• 2023-06-07 - We are planning to remove support for Bazel. If you rely on Bazel for building this code, please let us know in #188.
• 2023-05-03 - The master branch has been renamed to main (see #164 for details). Please update your fork as per instructions in that issue.

Features

• Extract descriptor bindings from SPIR-V bytecode, to assist in the generation of Vulkan descriptor set and pipeline layouts.
• Extract push constant block size from SPIR-V bytecode to assist in the generation of pipeline layouts.
• Extract full layout data for uniform buffer and push constant blocks from SPIR-V bytecode, to assist in application updates of these structures.
• Extract input/output variables from SPIR-V bytecode (including semantic decorations for HLSL shaders), to assist in validation of pipeline input/output settings.
• Remap descriptor bindings at runtime, and update the source SPIR-V bytecode accordingly.
• Log all reflection data as human-readable text.

Integration

SPIRV-Reflect is designed to make integration as easy as possible. The only external dependency is the Vulkan SDK.

To integrate SPIRV-Reflect into a project, simply add spirv_reflect.h and spirv_reflect.c in the project's build, and include spirv_reflect.h from the necessary source files.

If the project wants to use it's own SPIRV-Header path, it can set SPIRV_REFLECT_USE_SYSTEM_SPIRV_H

# CMake example
target_compile_definitions(project_name PUBLIC SPIRV_REFLECT_USE_SYSTEM_SPIRV_H)

Building Samples

This step is only necessary when building/running SPIRV-Reflect's example applications.

SPIRV-Reflect includes a collection of sample programs in the examples/ directory which demonstrate various use cases:

• descriptors: This sample demonstrates the retrieval of descriptor bindings, including the population of VkDescriptorSetLayoutCreateInfo structures from reflection data.
• hlsl_resource_types: This sample shows how various HLSL resource types are represented in SPIR-V.
• io_variables: This sample demonstrates the retrieval of input/output variables, including the population of VkPipelineVertexInputStateCreateInfo structures from reflection data.

To build the included sample applications, use CMake to generate the appropriate project files for your platform, then build them as usual.

Note that you can set VulkanSDK directory as your preference. For example, on Linux:

VULKAN_SDK=$HOME/VulkanSDK/1.1.70.1/x86_64 cmake -G Ninja  ..

Usage

SPIRV-Reflect's core C API should be familiar to Vulkan developers:

#include "spirv_reflect.h"

int SpirvReflectExample(const void* spirv_code, size_t spirv_nbytes)
{
  // Generate reflection data for a shader
  SpvReflectShaderModule module;
  SpvReflectResult result = spvReflectCreateShaderModule(spirv_nbytes, spirv_code, &module);
  assert(result == SPV_REFLECT_RESULT_SUCCESS);

  // Enumerate and extract shader's input variables
  uint32_t var_count = 0;
  result = spvReflectEnumerateInputVariables(&module, &var_count, NULL);
  assert(result == SPV_REFLECT_RESULT_SUCCESS);
  SpvReflectInterfaceVariable** input_vars =
    (SpvReflectInterfaceVariable**)malloc(var_count * sizeof(SpvReflectInterfaceVariable*));
  result = spvReflectEnumerateInputVariables(&module, &var_count, input_vars);
  assert(result == SPV_REFLECT_RESULT_SUCCESS);

  // Output variables, descriptor bindings, descriptor sets, and push constants
  // can be enumerated and extracted using a similar mechanism.

  // Destroy the reflection data when no longer required.
  spvReflectDestroyShaderModule(&module);
}

A C++ wrapper is also provided.

Building Examples

By adding -DSPIRV_REFLECT_EXAMPLES=ON in CMake you can see the use of the API used in action.

The main_explorer.cpp file is designed to allow an user to use their debugger of choice to breakpoint and explorer the contents of the SpvReflectShaderModule object. Just build with -DCMAKE_BUILD_TYPE=Debug and setup a debugger to run ./bin/explorer path/to/shader.spv

Building Self-Test Suite

SPIRV-Reflect uses googletest for self-testing. This component is optional, and generally only of interest to developers modifying SPIRV-Reflect. To run the self-tests:

• git submodule init
• git submodule update
• Enable SPIRV_REFLECT_BUILD_TESTS in CMake
• Build and run the test-spirv-reflect project.

License

Copyright 2017-2018 Google Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

spirv-reflect's Issues

Support thread group size reflection for compute shaders

There doesn't seem to be a way to retrieve the local group-thread count for compute shaders yet. Are there any plans of adding support for that?

spirv-reflect dumping wrong info for $Global block

Input HLSL:

struct VSInput
{
    float2 pos : POSITION;
    float4 color : COLOR0;
    float2 uv : TEXCOORD0;
};

struct VSOutput
{
    float4 pos : SV_POSITION;
    float4 color : COLOR0;
    float2 uv : TEXCOORD0;
};

cbuffer myconstants : register(b0)
{
    float delta;
    float delta2;
};

float himom : register(b1);
float byemom : register(b2);

VSOutput main(VSInput input)
{
    VSOutput o;
    o.pos = float4(input.pos, 0.f, 1.f);
    o.color = input.color + float4(delta, delta2, delta + byemom, delta + himom);
    o.uv = input.uv;

    return o;
}

Compiling with dxc -spirv -fspv-reflect -fspv-target-env=vulkan1.1 and running spirv-reflect on the output, I see the following bit:

  Descriptor bindings: 2
    0:
      binding : 0
      set     : 0
      type    : VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER (CBV)
      name    : myconstants (type.myconstants)
          // offset = 0, abs offset = 0, size = 16, padded size = 16
          struct type.myconstants {
              float delta;          // offset = 0, abs offset = 0, size =  4, padded size =  4
              float delta2;         // offset = 4, abs offset = 4, size =  4, padded size = 12
          } type.myconstants;

    1:
      binding : 1
      set     : 0
      type    : VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER (CBV)
      name    : $Globals (type.myconstants)
          // offset = 0, abs offset = 0, size = 16, padded size = 16
          struct type.myconstants {
              float delta;          // offset = 0, abs offset = 0, size =  4, padded size =  4
              float delta2;         // offset = 4, abs offset = 4, size =  4, padded size = 12
          } type.myconstants;

It looks like it's dumping the myconstants block twice, instead of dumping $Globals the second time around.

$Global constant buffer is not reflected

When building from HLSL SPRIV-Reflect ignores $Global constant buffer.

glslangValidator -V -D -e main test.frag

uniform float4 u_test0;

cbuffer : register(b0)
{
    float4 u_test1;
};

void main(out float4 fragColor : SV_TARGET0)
{
    fragColor = u_test0 * u_test1;
}

spirv-reflect frag.spv

will output only descriptor binding for unnamed constant buffer that contains u_test1.

Add parsing for UAV hidden counter

Add parsing for UAV hidden counter

Support for push constants

I don't seem to find support for push constants in SPIRV-Reflect. Following shader:

#version 450

layout(set=1,binding=2) uniform texture2D mytexture1;
layout(set=1,binding=3) uniform texture2D mytexture2;

layout(push_constant,  binding = 1, std430) uniform PD {
	vec4 section;
} pd;

layout(location=0) out vec4 pepo;

void main() { 

	pepo=pd.section;
}

Compiled to SPIR-V, then running spirv-reflect, outputs:

generator       : Khronos Glslang Reference Front End
entry point     : main
source lang     : GLSL
source lang ver : 450
source file     : 
shader stage    : PS


  Output variables: 1

    0:
      spirv id  : 9
      location  : 0
      type      : float4
      semantic  : 
      name      : pepo
      qualifier : 


  Descriptor bindings: 2

    Binding 1.2
      spirv id : 20
      set      : 1
      binding  : 2
      type     : VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE (SRV)
      count    : 1
      accessed : false
      name     : mytexture1

    Binding 1.3
      spirv id : 21
      set      : 1
      binding  : 3
      type     : VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE (SRV)
      count    : 1
      accessed : false
      name     : mytexture2

With no trace of the push constant. Would appreciate some hint on where this information is obtained, or spirv-reflect updated to also show how to obtain this information.

spirv.h inclusion should be more robust

Since #44 was merged, spirv_reflect.h uses #include <spirv.h> to include the SPIRV header. Its CMake file first looks for spirv.h in a SPIRV-Headers submodule (falling back on the version in Vulkan SDK if the headers submodule is not found), and configures the project's include paths appropriately for both cases. so that #include <spirv.h> works. Unfortunately, this logic must be replicated in projects that embed SPIRV-Reflect, or else the public header's #include <spirv.h> is unlikely to work out of the box. This adds an additional obstacle (albeit a mild one) to the integration process.

The core frustrating issue is that the required include path for the SPIRV-Headers submodule isn't compatible with the way most projects include the Vulkan SDK headers. In my experience, most projects add ${VULKAN_SDK}/include to their include paths, and use #include <vulkan/vulkan.h> to access the public Vulkan API. The corresponding include statement for spirv.h would therefore be #include <vulkan/spirv.h>. However, the "vulkan/" prefix won't work if spirv.h is coming from the SPIRV-Headers repo; the full path to the file is:

${SPIRV-Headers_SOURCE_DIR}/include/spirv/[1.0? 1.1? 1.2? unified1?]/spirv.h

Ideally, including spirv_reflect.h should work out-of-the-box for the common case (while still allowing projects to use custom SPIRV headers if desired). But it seems that's only possible if projects have ${VULKAN_SDK}/include/vulkan in their include paths, instead of (or in addition to) ${VULKAN_SDK}/include.

Associate samplers with images that are used together

Is there a way to use SPIRV-Reflect to identify which samplers are used with which images?
I'm explicitly not interested in combined image-samplers, instead I need to gather all samplers that are used in conjunction with each individual image.

Consider the following fragment shader:

uniform sampler LinearSampler, NearestSampler;
uniform texture2D Tex0, Tex1;

in vec2 TexCoord;
out vec4 OutColor;

void main*() {
  OutColor = (
    texture(sampler2D(Tex0, LinearSampler), TexCoord) +
    texture(sampler2D(Tex0, NearestSampler), TexCoord) +
    texture(sampler2D(Tex1, LinearSampler), TexCoord)
  );
}

The information I need to find then would be the following:

Tex0: LinearSampler, NearestSampler
Tex1: LinearSampler

Is SPIRV-Reflect able to deliver that kind of information?

Add support for compiling without Vulkan SDK

It would be nice to have a mode of compiling SPIRV-Reflect without the need for the Vulkan SDK to be installed (especially if unavailable, like on OS X). It looks like the Vulkan SDK is mainly used for the API that returns descriptor types, etc.. Perhaps in a compilation mode without the Vulkan SDK, the tool would be useful to do general reflection without returning VK specific types.

Currently I get the following cmake error on OS X
-- The C compiler identification is AppleClang 9.0.0.9000039 -- The CXX compiler identification is AppleClang 9.0.0.9000039 -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done CMake Error at CMakeLists.txt:15 (message): VULKAN_DIR not set, use -DVULKAN_DIR=<path> or set VULKAN_SDK environment variable

This is not a huge issue, but definitely impeding some of my workflows.

Thank you,
Graham

Unused variable warnings under /W4

spirv_reflect.cpp(658): warning C4189: 'deco_width': local variable is initialized but not referenced
spirv_reflect.cpp(786): warning C4189: 'execution_model': local variable is initialized but not referenced
spirv_reflect.cpp(787): warning C4189: 'entry_point': local variable is initialized but not referenced

Can they be used/removed?

Wrong push constant block reflection

spvReflectEnumeratePushConstantBlocks in this spir-v shader properly returns that there is one push constant, but when requesting the pointer to the SpvReflectBlockVariable, it fills the 1-element array with a null pointer.

I think the push constant may not be used, but when creating a render pipeline, the validation layer complains that the push constant range has not been properly defined, so this information definitely exists in the spir-v shader (attached below) and should be reflected.

popo.zip

spvReflectGetDescriptorBinding() needs two indices

This function requires both a set index and a binding index, but currently only takes a single index parameter.

Specialization constants cannot be used as an array size

The following shader will currently make SPIRV-Reflect return an error.

#version 450

layout (constant_id = 0) const int CONST = 64;
layout(binding = 0) uniform Args {
    vec4 array[CONST];
};

void main() {
    gl_Position = array[0];
    for (int i = 1; i < CONST; i++) {
        gl_Position += array[i];
    }
}

I belive that the faulty code is in this line.
https://github.com/chaoticbob/SPIRV-Reflect/blob/8fd9b11dbd217f6b02c9d6d4e364a7f0327b219a/spirv_reflect.c#L1432

The reason is that it tries to read the constant as if it's always a literal. When specialization constants are in play, this is no longer true. The array size constant is OpSpecConstant instead of OpConstant.

Relevant bits of spirv-dis:

             %CONST = OpSpecConstant %int 64
%_arr_v4float_CONST = OpTypeArray %v4float %CONST
              %Args = OpTypeStruct %_arr_v4float_CONST

Why the maximum descriptor set number is a fixed value?

Is there any specific reason for that? Just wondering.

Infinite loop in ParseDescriptorBlockVariableUsage when parsing

Hi!
I get an infinite loop in ParseDescriptorBlockVariableUsage when trying to call spvReflectCreateShaderModule() on the code below. It seems that it's the array passed to "FailFunc" that upsets it..

`
struct S_cbPerObjectBones
{
float4 v4Bones[4095];
};

shared cbuffer _cbPerObjectBones : register(b4)
{
S_cbPerObjectBones cbPerObjectBones;
};

struct v2f
{
float4 HPosition: SV_POSITION;
float4 Position: POSITION;
};

struct a2v_StandardWeighted
{
float4 Position: POSITION0;
int4 Indices: BLENDINDICES;
};

float3x4 FailFunc(int4 i, float4 v4Bones[4095])
{
float4 mRow1 = v4Bones[i.x+0];
return float3x4(mRow1, mRow1, mRow1);
}

v2f Test_VS(a2v_StandardWeighted input)
{
v2f OUT=(v2f)0;
float4 inputPosition = float4(input.Position.xyz, 1);
int4 i = input.Indices;
float3x4 mMatrix = FailFunc(i, cbPerObjectBones.v4Bones);
float4 v4Position = float4(mul(mMatrix, inputPosition), 1);
OUT.Position = v4Position;
return OUT;
}
`
Compiled with dxc
Version: dxcompiler.dll: 1.5 - 1.5.0.2428 (HEAD, bd57c441)

dxc.exe -T vs_6_0 -spirv -E Test_VS -Fo reflectbug.spirv reflectbug.hlsl

SPIRV-Reflect breaks on arrays of StructuredBuffer

SPIRV-Reflect does not seem to like arrays of StructuredBuffer. Trying to reflect this declaration:

StructuredBuffer<float3> test[16];

will cause it to fail this assert:
https://github.com/chaoticbob/SPIRV-Reflect/blob/0780aee6ec8882d1f34a0de443a706fdae03e618/spirv_reflect.c#L1778-L1780

which is caused by this code:
https://github.com/chaoticbob/SPIRV-Reflect/blob/0780aee6ec8882d1f34a0de443a706fdae03e618/spirv_reflect.c#L1332-L1336

The decoration_flags are applied only once but not on recurring calls to ParseType(). I fixed this in my build by modifying the code to this:

    if (p_type->id == INVALID_VALUE) {
      p_type->id = p_node->result_id;
      p_type->op = p_node->op;
      p_type->decoration_flags = 0;
    }
    p_type->decoration_flags |= ApplyDecorations(&p_node->decorations);

ShaderReflection constructor fails when using copy = true

I hit a bug when using the C++ ShaderReflection() constructor where setting the bool copy to true:

https://github.com/chaoticbob/SPIRV-Reflect/blob/d6c8ac8d25fdaa5fff532cad9bd977f33a1e372e/spirv_reflect.h#L381

The problem is that m_result is set to SPV_REFLECT_RESULT_NOT_READY on entry to the constructor so then it never actually performs the memcpy so then reflection fails.

Another minor point: my first attempt at using ShaderReflection I was getting back descriptor bindings with corrupt names. I tracked it down to the fact that my spir-v code was out of scope and I had not passed true to the copy parameter in the constructor. I think the behavior that it tries to conserve memory by pointing to the spir-v is probably a good one, but it did take me a bit to track down.

Structures with generic names in global namespace can cause conflicts in client applications

Internal structures like String, Node, Parser, Function are in global namespace and can lead to issues in client applications due to conflicting names in the global namespace.

As an example, we're having a MSVC debugger issue with String in Godot Engine, see the discussion here:
godotengine/godot#50025

What about giving these structures more specific names, like using Spv or Spirv prefix to prevent this kind of issue in applications using the library?

Remove harmless but potentially confusing "Module" references

It looks like the ShaderReflection class was renamed from "Module" at some point, but several references to the old name (class comment, "m_module" and "p_module" variables) still exist.

spirv.h is not available on macOS

Hello, I'm trying to build it through bazel on macOS.

But there's no header file named <vulkan/spirv.h> in the vulkan SDK for macOS, while spirv.hpp is exists.

How can I build it on macOS?

SPIRV-Reflect is not compliant with SPIRV 1.5

C++ bindings don't have copy constructors and it causes crashes

This code works correctly:
spv_reflect::ShaderModule shaderModule = spv_reflect::ShaderModule(vertexShaderData);

This does not:

spv_reflect::ShaderModule shaderModule;
shaderModule = spv_reflect::ShaderModule(vertexShaderData);

The copy constructor and the assignment operator for ShaderModule are not defined, and the default ones simply copy the pointers. After the assignment the old copy of the object gets destroyed, freeing the memory and leaving dangling pointers in the second object.

ParseFormat() error in trivial shader

ParseFormat() returns an error while creating a reflection shader module for the following shader. It looks like it's failing while parsing the format of the built-in gl_PerVertex?

#version 450
#pragma shader_stage(vertex)

layout (location = 0) out vec3 texcoord;

void main() {
  gl_Position = vec4(gl_VertexIndex, 0, 0, 1);
  texcoord = vec3(0,0,0);
}

SpvReflectInterfaceVariable seems to be missing a size property

So the only ways I can see of determing the size of a SpvReflectInterfaceVariable seems to be either:

1. Combining the numeric, array and members members manually which sounds pretty hard to get right and definitely cumbersome.
2. Using the format member and mapping it to a size manually, although I assume that'd disregard the possibility of the variable being an array or possessing members of its own.

This makes it very hard to determine the offset of vertex properties.

I could totally be missing something though, in which case I'm sorry!

GLSL Storage buffers don't seem to get reflected properly

A resource binding with the following syntax doesn't show up properly.

layout(binding = 1, std430, set = 0) buffer TriangleCounts { uint g_triangleCounts[]; };

I think the bug is two-fold; it doesn't show up because of the various (p_node->storage_class != SpvStorageClassUniform) && (p_node->storage_class != SpvStorageClassUniformConstant) checks, but after improving them with a SpvStorageClassStorageBuffer check it's still broken (shows up as a uniform buffer).

Rows and Columns not reflecting properly

I'm having a couple of issues with shader reflection for matrices at the moment.

1. A simple shader with a 3x4 or 4x3 matrix always comes out with the dimensions flipped around. Another example, a 1x4 matrix says it has 4 rows and 1 column, always.

2. Matrices are also not decorated with SPV_REFLECT_DECORATION_ROW_MAJOR or the COLUMN_MAJOR variant, the decoration flags are always 0.

3. I'm also not understanding the stride, but it might be consistent with the rows and columns being flipped. The 3x4 matrix has a stride of 64 whereas the 4x3 has a stride of 48 bytes, whereas I would expect the opposite.

The issues all probably come from the fact that this code doesn't seem to account for them. Would it be possible to confirm whether this is indeed a bug? Thanks!

case SpvOpTypeMatrix: {
        p_type->type_flags |= SPV_REFLECT_TYPE_FLAG_MATRIX;
        uint32_t column_type_id = (uint32_t)INVALID_VALUE;
        IF_READU32(result, p_parser, p_node->word_offset + 2, column_type_id);
        IF_READU32(result, p_parser, p_node->word_offset + 3, p_type->traits.numeric.matrix.column_count);
        SpvReflectPrvNode* p_next_node = FindNode(p_parser, column_type_id);
        if (IsNotNull(p_next_node)) {
          result = ParseType(p_parser, p_next_node, NULL, p_module, p_type);
        }
        else {
          result = SPV_REFLECT_RESULT_ERROR_SPIRV_INVALID_ID_REFERENCE;
          SPV_REFLECT_ASSERT(false);
        }
        p_type->traits.numeric.matrix.row_count = p_type->traits.numeric.vector.component_count;
        p_type->traits.numeric.matrix.stride = p_node->decorations.matrix_stride;
        // NOTE: Matrix stride is decorated using OpMemberDecoreate - not OpDecoreate.
        if (IsNotNull(p_struct_member_decorations)) {
          p_type->traits.numeric.matrix.stride = p_struct_member_decorations->matrix_stride;
        }
      }
      break;

Fix use of strcat_s when parsing SpvOpSourceContinued

When concatenating the source strings while parsing SpvOpSourceContinued, strcat_s is called with a wrong length for the target buffer on Windows. PR #127 is a suggested fix.

How much does this library trust the input SPIR-V module?

The SPIR-V parser is pretty bare-bones. How robust is it intended to be?

I'd like the README and the .h file to be update to state the assumptions about the input.

E.g.: "The SPIR-V module is assumed to be valid for Vulkan 1.0 or 1.1"
Or is it a weaker condition?

Undefined sanitizers bugs

thirdparty/spirv-reflect/spirv_reflect.c:293:37: runtime error: applying zero offset to null pointer
thirdparty/spirv-reflect/spirv_reflect.c:2205:19: runtime error: implicit conversion from type 'int' of value -2 (32-bit, signed) to type 'unsigned int' changed the value to 4294967294 (32-bit, unsigned)

not sure about reproduction steps

SPIRV-Reflect/spirv_reflect.c

Line 293 in 4bfcf15

const uint32_t* arr1_end = p_arr1 + arr1_size;

SPIRV-Reflect/spirv_reflect.c

Line 2205 in 4bfcf15

p_var->flags &= ~SPV_REFLECT_VARIABLE_FLAGS_UNUSED;

RWTexture2D InterlockedMax bug

spv_reflect::ShaderModule::EnumerateEntryPointDescriptorBindings returns 0 as the binding count for following shader. The shader was compiled using the latest release of dxc to spirv. Verified that it works correctly with spirv-cross but spirv-reflect fails to detect rwTexture as a used_uniform. If I insert a dummy instruction like rwTexture[uint2(5000, 5000)] = 1; it works. Also attached the spirv bytecode file

cs_main.zip

HLSL shader to reproduce bug

RWTexture2D<uint> rwTexture : register(u0);

[numthreads(8, 8, 1)]
void cs_main(uint2 coords : SV_DispatchThreadID)
{
        uint2 newCoord = coords + uint2(1, 1);
        uint iDepth = newCoord.x + newCoord.y;
	InterlockedMax( rwTexture[newCoord], iDepth );
	InterlockedMax( rwTexture[coords], iDepth );
}

Support for NVidia RTX extension

Hi,

would it be possible to add support for RTX shader extensions (ray generation, miss, closest hit, intersection and any hit shader types) ?

How hard or easy would that be ?

If I had to do it on my own, do you have any pointers where to start ?

Clarify what "index" parameters are relative to

Does spvReflectGetDescriptorSet(&module, N, &result) return module.descriptor_sets[N]? Or does it return the entry in module.descriptor_sets[] whose "set" field equals N? Or are these by definition the same (i.e. is the descriptor_sets[] array padded with empty sets so that set=N is always the Nth array element)?

(sure, I can look in the source and see that it returns the Nth array entry, but I could see this being a source of confusion/bugs)

Same general question for most of the other spvReflectGet*() functions. I guess the real request here would be "minimal API docs/comments plz!"

[i]samplerbuffer identified as VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER

In the following GLSL shader, both bindings are categorized as combined image samplers instead of uniform texel buffers.

#version 450
#pragma shader_stage(vertex)
layout (set = 0, binding = 1) uniform isamplerBuffer texel_buffer_i;
layout (set = 0, binding = 2) uniform samplerBuffer texel_buffer_f;

void main() {
  int i = texelFetch(texel_buffer_i, gl_InstanceIndex).x;
  float f = texelFetch(texel_buffer_f, gl_InstanceIndex).x;
  gl_Position = vec4(i, f, 0, 1);
}

Quick exploratory debugging revealed that on line 1181 of spirv_reflect.c, p_descriptor->image.dim is SpvDimImage1D, not SpvDimBuffer. The latter would put the descriptor into the texel buffer path; the former makes it a combined image sampler. Root cause left as an exercise for the author.

Add support for multiple entrypoints within a module

Support for AccelerationStructure

Hello,

Can we expect to get the AccelerationStructure types being reflected in the future? I want to migrate from SPIRV-Cross library's reflection as this library is smaller and I only need reflection. Is this library going to be maintained in the future, or is it preferred to use Spirv-Cross?

What about support multiple OpSource in SpvReflectShaderModule?

At the moment, several embedded source files are incorrectly processed, each next one overwrites the previous one, which is not the desired behavior, this is also reflected in the API, GetSourceFile, it is assumed that there is only one include per source file when it is far from the truth. At least from the point of view of correct behavior, this is not permissible, I would like to see a complete list of included files.

From the point of view of the correctness of the API, it should at least look something like this:

  uint32_t                GetSourceFileCount() const;
  const char*           GetSourceFile(uint32_t  index) const;

this would correspond to the API of receiving an entry point:

  uint32_t                      GetEntryPointCount() const;
  const char*                   GetEntryPointName(uint32_t index) const;

Bindings missing from reflection data if passed by parameter

When I compile this shader and run it through spirv_reflect, I get reflection data as I expect

#version 450

layout (set = 0, binding = 1) uniform sampler smp;
layout (set = 1, binding = 3) uniform texture2D tex;

layout (location = 0) out vec4 out_color;

vec4 normal_map(
    texture2D t,
    sampler s,
    vec2 uv
) {
    // *** NOT using the parameters, just accessing tex/smp directly ***
    return texture(sampler2D(tex, smp), uv);
}

void main() {
    out_color = normal_map(tex, smp, vec2(1.0, 1.0));
}

If I use this code instead, the bindings are not in the reflection data.

#version 450

layout (set = 0, binding = 1) uniform sampler smp;
layout (set = 1, binding = 3) uniform texture2D tex;

layout (location = 0) out vec4 out_color;

vec4 normal_map(
    texture2D t,
    sampler s,
    vec2 uv
) {
    // *** Using parameters t and s instead of "global" tex/smp ***
    return texture(sampler2D(t, s), uv);
}

void main() {
    out_color = normal_map(tex, smp, vec2(1.0, 1.0));
}

I'm going through rust bindings, but this is the code I'm calling:

    let enumerated_entry_points = shader_module.enumerate_entry_points()?;
    for entry_point in enumerated_entry_points {
        for descriptor_set in &entry_point.descriptor_sets {
            for binding in &descriptor_set.bindings {
                println!("binding {} {} {}", binding.name, binding.set, binding.binding);
            }
        }
    }

Is this expected behavior? I expected that these shaders would produce identical reflection data.

Boolean Specialization Constants

I am checking SPIRV-Reflect, but I see nowhere where the list of boolean specialization constants can be obtained from a stage. Is this supported?

Reflecting resource usage

To aid in making minimal memory buffers flag choices in Vulkan, it would be great to get more information from reflection about resource usage. I see there is already SPV_REFLECT_DECORATION_NON_WRITABLE. Having a SPV_REFLECT_DECORATION_NON_READABLE could be useful as well.

However even better would be if the 'accessed' member (or similar) could be more specific and say if a resources is written to, read from, or both, within a stage. This would be done based on actual potential usage vs. just the decoration hint. This would allow for more exact choices for src and dst access flags when creating my Vulkan memory barriers for an arbitrary shader.

Is this hidden anywhere right now, or would it be considered to be added?
Thanks for your consideration.

Issue parsing SPIRV with function taking struct parameter.

I've found an unusual issue where the library returns SPV_REFLECT_RESULT_ERROR_SPIRV_INVALID_BLOCK_MEMBER_REFERENCE.
This is caused by accessing past the end of the p_access_chain->indexes[] array.

I've narrowed it down to a descriptor block which has a struct in, and a function is called which takes the struct as a parameter. The glsl for this minimal repro looks like this:

struct B {
	float a;
};
layout (binding = 0) uniform Test {
	B b;
} test;
float dothing(const in B b) {
	return b.a;
}
layout(location = 0) out vec4 outColor;
void main() {
	outColor = vec4(0);
	// this is fine
	outColor.z = test.b.a;
	// this line causes SPV_REFLECT_RESULT_ERROR_SPIRV_INVALID_BLOCK_MEMBER_REFERENCE
	outColor.z = dothing(test.b);
}

I'm using glslc.exe to compile the shader. I've attached the spirv for this version (test-broken.frag.spv) and also a version with the last line commented out (test-works.frag.spv).

I'm sorry, I was not able to understand why it is broken yet, maybe someone else could take a look too?

test-spv.zip

ParseFormat does not handle doubles correctly

I think ParseFormat should be checking the width of the float/vec types in order to determine the format. https://github.com/chaoticbob/SPIRV-Reflect/blob/8394d3eedd4e62aa1bd7229426256151962539a9/spirv_reflect.c#L2118

Formats like SPV_REFLECT_FORMAT_R64_SFLOAT, etc. appear to be missing.

Support task and mesh shaders

Right now descriptor sets don't get enumerated properly because the entry points for task and mesh shaders don't get recognized.

Ability to build SPIRV-Reflect libraries only.

Hi there,

It would be really convenient if it was possible to build SPIRV reflect as a static or shared library so multiple things could link against it at the same time without having to deal with symbol conflicts.

Additionally, it would be useful to be able to configure the CMakeLists to only build said library and optionally turn off the main executable and stripper util.

If this is something you're happy for me to do, it probably shouldn't be a massive hassle.

Many thanks,
Alister

constify ShaderReflection::Get*() methods

GetDescriptor(), GetDescriptorSet(), GetInputVariable(), and GetOutputVariable() aren't tagged as const. Any reason they couldn't be? Also, why do they return "Foo* const" instead of "const Foo*"?

SpvReflectDescriptorBinding.array may need a new struct type

It currently uses SpvReflectArrayTraits, which seems more geared towards arrays of POD types, not arrays of descriptors. For instance, I'm not sure what the "stride" of an array of 4 uniform buffer objects would be.

Rename "size" parameters to "size_in_bytes"

I can never keep track of which APIs mean size = byte_count and which mean size = element_count. This time, I guessed wrong :(

SpvReflectShaderModule fields are used without explicit initialization

This leads to crashes if uninitialized modules are passed to spvReflectGetShaderModule(). Examples:

• p_module->push_constant_count
• p_module->push_constants
• p_module->input_variables

The code currently assumes that the module is initialized to zero by the caller; I think that's a good habit for callers, but the library shouldn't require it; it should reset the object to safe defaults before/during initialization.

Use spirv.h from the Vulkan SDK

If the project's dependency is the Vulkan SDK, why is the spirv.h copy needed?

If it's there to allow people to use spirv-reflect without the Vulkan SDK, it would still be nice to have the option to use the SDK's version through a define.

#include <vulkan.h>

I may be wrong, but isn't the canonical way to include Vulkan "#include <vulkan/vulkan.h>"? Including vulkan.h directly required me to add another include dir to my project -- not the end of the world, but a potentially avoidable integration step.

Array size specialization constant reflection

It's currently not possible to see the specialization constants for a module. Furthermore, it's not possible to get the size of an array when it's dependent on a specialization constant. I would be interested in adding an API for that. Effectively, given the specialization constants for a shader module (e.g. in a vulkan-like VkSpecializationInfo equivalent), it should be possible to retrieve array sizes.

Another (mildly related) issue i've stumbled across is that OpTypeRuntimeArray is completely ignored. The library could mark the dimension of the array with a special value instead, signaling that it's a runtime array.

Below is an API sketch, consider this something like an RFC. Are there any other cases where specialization constants should be considered? Is this out of scope of the library altogether? It would involve evaluating a whole set of specialization constant operations after all.

An implementation problem is that we would need to preserve the nodes from the parser to correctly evaluate a specialization constant operation later on I guess (we could just extract the nodes that are needed for the constant op or build a smaller internal representation of the operations but that adds further complexity). I'd start with int/uint/bool spec constant ops since only those are allowed for shaders but the API could be extended later on for float spec constants ops (for kernels).

// Changes to ArrayTraits:
typedef struct SpvReflectArrayTraits {
  uint32_t                          dims_count;
  // For a variable-sized array (OpTypeRuntimeArray), the value will be 0x0.
  // For a specialization constant size, it will be 0xFFFFFFFF (see spec_const_op_ids)
  uint32_t                          dims[SPV_REFLECT_MAX_ARRAY_DIMS];
  // For dimensions that have a specialization constant operation as size, this
  // array holds the id of the speicalization constant operation.
  // It can be evaluated (given a set of original specialization constants)
  // using spvReflectEvaluateSpecConstOpID
  uint32_t                          spec_const_op_ids[SPV_REFLECT_MAX_ARRAY_DIMS];
  uint32_t                          stride; // Measured in bytes
} SpvReflectArrayTraits;

// Allow to reflect over specialization constants
typedef struct SpvReflectSpecializationConstant {
  const char* name;
  uint32_t spirv_id;
  uint32_t constant_id;
  SpvReflectTypeDescription* type_description;
  uint64_t default_value;
} SpvReflectSpecializationConstant;

// Changes to SpvReflectShaderModule
typedef struct SpvReflectShaderModule {
  // ...
  uint32_t spec_constant_count;
  SpvSpecializationConstant** spec_constants;
} SpvReflectShaderModule;


// Add new SpvReflectSpecializationInfo, SpvReflectSpecializationMapEntry structs mirroring vulkan
// ...


// Like spvReflectCreateShaderModule but stores the parsed spv data inside the
// shader module, allowing future operations at the cost of higher memory consumption.
SpvReflectResult spvReflectCreateShaderModuleWithParser(
  size_t                   size,
  const void*              p_code,
  SpvReflectShaderModule*  p_module
);

// Only valid when the given shader module was created with a parser-preserving function.
// Evaluates the given constant op id using the given specialization info.
SpvReflectResult spvReflectEvaluateSpecConstOpID(
  SpvReflectShaderModule*           p_module,
  uint32_t                          spec_const_op_id,
  SpvReflectSpecializationInfo*     p_spec_info,
  uint32_t*                         p_out_const_op_value);

Building as Release/RelWithDebInfo will throw a warning (warnings are treated as errors)

Building using cmake version 3.16.3 and setting the build type flag as -DCMAKE_BUILD_TYPE=Release will throw a warning.

[ 29%] Building CXX object CMakeFiles/spirv-reflect.dir/common/output_stream.cpp.o
/home/john/Development/SPIRV-Reflect/common/output_stream.cpp: In function ‘void WriteReflection(const spv_reflect::ShaderModule&, bool, std::ostream&)’:
/home/john/Development/SPIRV-Reflect/common/output_stream.cpp:945:20: error: variable ‘result’ set but not used [-Werror=unused-but-set-variable]
  945 |  
cc1plus: all warnings being treated as errors
make[2]: *** [CMakeFiles/spirv-reflect.dir/build.make:115: CMakeFiles/spirv-reflect.dir/common/output_stream.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:121: CMakeFiles/spirv-reflect.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....