microsoft/ifc-spec

IFC format specification

IFC Format Specification

The IFC specification aims to formally define a binary format for describing the semantics of C++ programs (or program fragments) at a high level of abstraction, before lowering to machine code or similar.

This format is designed to offer a persistent form of the in-memory Internal Program Representation (IPR) of C++ programs originally developed by Gabriel Dos Reis and Bjarne Stroustrup. As such, it follows the same principles as the IPR:

• Completeness: Represents the semantics of all Standard C++ constructs
• Generality: Suitable for every kind of application, rather than targeted to a particular application area
• Regularity: Does not mimic C++ language irregularities; general rules are used, rather than long lists of special cases
• Typefulness: Every expression has a type
• Minimality: No redundant values, and traversal involves no redundant indirections
• Compiler neutrality: Not tied to a particular compiler
• Scalability: Able to represent hundreds of thousands of lines of codes on common machines

Downloading

The latest IFC draft specification can be downloaded here.

Official releases can be found on the releases page.

Contribute

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Building

In order to build the PDF rendering of the specification, ensure that you have a modern LaTeX processor installed.

Clone a copy of the repo

git clone https://github.com/microsoft/ifc-spec.git

Change to the LaTeX directory

cd ltx

Issue the command

pdflatex ifc

followed by

bibtex ifc

and possibly two more times

pdflatex ifc

ifc-spec's Issues

Provide ability to query tool producer

The IFC format, at the moment, does not provide ability (for a consumer of an IFC file) to query the producer of said IFC file -- nor is there a standard ability for a tool to store such information.

DirSort::SpecifiersSpread has an incomplete specification

The specifiers field is specified as DeclSpecifierSequence which does not exist; I'm assuming this is SyntaxSort::DeclSpecifierSeq.

There's also a targets field of ProclamatorSequence which does also does not exist, I don't see an immediate parallel.

Remove `Deprecated` from `BasicSpecifiers`

It should be an attribute trait.

DeclSort::SyntaxTree is not specified

IFC 0.43 (and earlier) contain DeclSort::SyntaxTree but do not document its fields.

ExprSort::Generic is not specified

IFC 0.43 (and earlier) contain ExprSort::Generic but do not document its fields.

Clarify trait specification to clarify whether traits are mono or multi-value

IFC traits do not note whether there should be one trait per DeclIndex or (potentially) multiple traits per DeclIndex. This should be clarified (ideally per trait, e.g., function definitions it doesn't make sense to have multiple traits, attributes there is an argument for, though this is redundant with the attribute tuple).

Unclear ExprSort::Cast target type

In the structure of a cast expression we see that target field denotes the type to convert that expression to.

In the document target has type Index, but doesn't mention where to look it up.

Should target be of type TypeIndex or what should this Index reference?

IFC ExprType may be used for a template template argument

Consider the following:

template<template<typename> class Container>
struct HashMap
{
    struct Entry {};
    constexpr HashMap(int val) {}
    using ContainerType = Container<Entry>;
private:
    ContainerType m_entries;
};

template<typename T>
struct Array
{
    T m_data;
};

template<typename T>
struct KeymapBackend : private Array<T>
{
    constexpr KeymapBackend() = default;
private:
    int m_size = 0;
};

When the template class HashMap is supplied with the template template argument KeymapBackend:

constexpr HashMap<KeymapBackend> keymap = { 0 };
                  ^^^^^^^^^^^^^

This is represented in the IFC as an expression ExprType with a denotation of TypeDesignated pointing to a DeclTemplate. This representation is a bit troubling as ExprType implies (in my mind at least) a type template argument.

I don't see an existing IFC node I'd suggest using in place, perhaps a ExprTemplate would be justified(?)

Make `CallingConvention` more abstract

It is a platform/vendor dependent concept. The current definition focuses on MSVC implementation; the specification should abstract over that and provides the MSVC definition in a vendor-specific trait.

Cross link to Microsoft own ifc de-facto standard parser

Personally I think that IFC is great - it tries to abstract and simplify C++ language structure.

However - I as developer would prefer to have a C++ library for parsing and maybe also generating .ifc file format.

Ideally ifc could serve as base for C++ reflection, in similar manner to C#.

There exists library, which is not provided by Microsoft:

https://github.com/AndreyG/ifc-reader

and it can be used for parsing ifc file format.

• I would prefer that Microsoft would provide such library

• As open source code

• Proper unit testing would be good to have

• API is tweaked from developer perspective - for example it should be possible to enumerate classes, functions, function parameters and so on.

See also C# reflection:

https://docs.microsoft.com/en-us/dotnet/framework/reflection-and-codedom/reflection

After we have de-facto standard ifc parsing library - we could just insert link to this git repository.

IFC documentation is useful indeed, but only for IFC parser. I would prefer to use IFC parser than analyzing IFC documentation.
Main idea is to have easy to use API, and not to long to read documentation.

DeclSort::Barren is not specified

IFC 0.43 (and earlier) contain DeclSort::Barren but do not document its fields.

String literal structure (i.e., "const.str") should forbid interior null characters

MSVC produced IFCs currently can contain interior null characters in represented strings.

These null characters can be a source of bugs as implementations may not recognize the need to remove them when reading text from the IFC.

Additionally, their presence cause a repeated performance burden on the consumer as the consumer must always traverse the entire string, and discard null characters. If an implementation requires a copy of the text for its purposes, optimized copies (via memcpy) are made inaccessible via the presence of null characters (assuming the implementation does not handle interior null charters itself). Similarly, interior null characters unnecessarily inflate the size of the IFC file.

It would be ideal if the IFC documented the presence of a terminating null character (i.e., #79) and (both for easy of implementation in consumers, and performance) forbids interior null character.

Sequester MSVC-specific expression sorts out of `ExprSort`

They include:

• UnqualifiedId
• Temporary
• Condition
• SyntaxTree
• FunctionString
• CompoundString
• StringSequence
• This
• TemplateReference
• TypeTraitsIntrinsic
• AssignInitializer

DeclSort::Parameter's initializer field type is out of date

The DeclSort::Parameter declaration should have its field's updated so that initializer is typed as a DefaultIndex (DefautlArgIndex?) as it's an index into the decl.default-arg partition.

Provide representation of constants (and literals) close to input source code

Consider providing representations for literals (and constants in general) close to the input source code, without devolving into token soup. In particular, consider the case of user-defined literals.

Add MSVC specific trait for BlockLocus

Internal ref: https://devdiv.visualstudio.com/DevDiv/_git/msvc/pullrequest/368315

Add MSVC specific trait for mapping statements to a pair of locations.

Trait partition is named ".msvc.trait.block-locus".

Typo in Table for ExprSort::PackedTemplateArguments

ExprSort::PackedTemplateArguments has an arguments "type" of ExpexIndex instead of ExprIndex.

Contradiction in the definition of Operator

In the general description of Operator (sec:ifc-operators) field index is defined as 12-bit value, but further concrete operators (Monadic), (Dyadic, ...) are defined as 13-bit values.

IFC Literals may contain surprising values

MSVC produced IFCs have been observed with the following unusual literal representations. These representations require special cased logic to correctly handle due to their surprising semantics.

Template Parameter References

The IFC sometimes represents references to template parameters via an ExprSort::Literal with a value of 0 and type of TypeSort::Designated. This designated type's associated decl can be a DeclSort::Parameter representing a reference to a template parameter:

  template<typename T>
  using x = typename y<sizeof(T)>;
                              ^

Indirect Template Parameter References

Similarly, the IFC sometimes represents references to template parameters indirectly via an ExprSort::Literal with a value of 0 and a type of TypeSort::Syntactic. This syntactic type's associated expr is an ExprSort::TemplateId .

This manifest from code of the ilk:

  template <class T, size_t = sizeof(remove_reference_t<T>)>
  struct y;

  template <typename T>
  using x = typename y<T>;

Where the IFC "inlines" the default template argument representing the latter given declaration as something like (with the underlined type represented as an IFC ExprSort::Literal):

  template <typename T>
  using x = typename y<T, sizeof(remove_reference_t<T>)>;
                                 ^^^^^^^^^^^^^^^^^^^^^

Decltype

The IFC additionally sometimes wraps decltype types (IFC TypeSort::Decltype) in ExprSort::Literal expressions with a value of 0 of the ilk:

  struct x { /* ... */};

  template <x V>
  using x = typename y<sizeof(decltype(V.data[0]))>;
                              ^^^^^^^^^^^^^^^^^^^

IFC Padding Documentation is Inconsistent

So as @AndreyG points out in #84, the IFC's documentation for padding is currently inconsistently specified. This is further complicated by #78 where interior padding is featured.

I'd suggest either specifying all instances of padding (as is done for TypeSort::Fundamental and Words), or specifying the rules for padding.

The former is a burden on specification, and the latter can be a burden (or easing of) implementation (depending on the design). The latter would (in my mind) require as a prerequisite that the IFC never use interior padding (as in #78), or have very specific rules about when this happens.

Divergence between placeholder-type definition in the specification and .ifc produced by MSVC

According to the definition of placeholder-type-structure (sec:ifc:TypeSort:Placeholder) its sizeof should be equal 8, but entry_size in the partition "type.placeholder" produced by MSVC (IFC-version 0.40) is equal to 12.

Missing initializer of constant defined in template class

Let's consider the following module:

constexpr bool global = true;

struct A {
	static constexpr bool static_field = true;
};

template<typename> struct B {
	static constexpr bool primary_template = true;
};

template<typename T> struct B<T*> {
	static constexpr bool partial_spec = true;
};

template<> struct B<void> {
	static constexpr bool explicit_spec = true;
};

Partition decl.variable contains 5 elements, 3 of them have initializer, but for variables primary_template and partial_spec initializer is null. I understand that it's impossible to evaluate value of variable inside template, because it could be dependent expression, but shouldn't be initializer somehow preserved in IFC?

Define arity as a distinct type, not bitfield

The IFC spec has the notion of arity (to describe the number of parameter of abstractions such as macros, functions, templates, etc.) and describe them as bitfields. That creates an interpretation problem when considering endianness of the IFC.

Suggestion: define Arity and variadicity as distinct types, not bitfields.

Remove `IsMemberOfGlobalModule` from `BasicSpecifiers`

This is an ownership property.

Missing enumerator in enumeration TypeBasis

According to the specification (sec:ifc-fundamental-type-basis), TypeBasis::Auto has value 20 and TypeBasis::DecltypeAuto 21, but in the .ifc produced by MSVC (IFC-version 0.40) those enumerators have values 21 and 22 correspondingly.

Make literal value representation more general and predictable

Currently, literal floating point values are represented only as 64-bit values. Make it more general and predictable. Same for integer values of precision wider than 64-bit.

See also #15 .

Divergence between StringIndex definition in the specification and .ifc produced by MSVC

According to the specification StringIndex has 4-bit tag, but it seems that in IFC produced by MSVC tag occupies 3 bits.

IFC DeclSort::Parameter Padding

The IFC does not document the two bytes of trailing padding for DeclSort::Parameter.

Partition of specialization form (named "form.spec") is not specified

This partition is mentioned twice in DeclSort::PartialSpecialization and DeclSort::Specialization sections, but is not defined anywhere in the specification.

Misleading definition of QualifiedName expression

Definition of QualifiedName says (sec:ifc:ExprSort:QualifiedName)

The elements field designates the sequence of unqualified names (ExprSort::UnqualifiedId)...

but actually part of qualified name could be not only UnqualifiedId, but also TemplateId.

Canonical graph patterns in the IFC format

Ideally, the IFC should indicate expected patterns for each standard notion representable in the IFC file format.

Divergence between base-type definition in the specification and .ifc produced by MSVC

The definition of base-type looks like

struct BaseType {
    TypeIndex type;
    Access access;
    bool shared;
    bool pack_expanded;
};

in the specification right now (sec:ifc:TypeSort:Base).
But shared and pack_exanded occupy exactly one bit each one in .ifc file produced by MSVC. And so it seems, that definition of BaseType should be

struct BaseType {
    TypeIndex type;
    Access access;
    bool shared : 1;
    bool pack_expanded : 1;
};

Divergence between template-decl definition in the specification and .ifc produced by MSVC

According to the definition of template-decl-structure (sec:ifc:DeclSort:Template) its sizeof should be equal 40, but entry_size in the partition "decl.template" produced by MSVC (IFC-version 0.33) is equal to 44. Maybe uint32_t padding-or-reserved; should be added to the end of template-decl-structure?

LaTeX Error: File bytefield.sty not found

FYIs
Following the readme instructions, on my setup (Fedora, texlive)
> pdflatex ifc
terminated with this error and stopped on it's own command line
(unfamiliar, I quit with a couple of CTRL-C's I think).

Solved by installing with sudo dnf install texlive-bytefield

(Might be worth enabling the Wiki, or the Discussions feature if available.)

Sequester MSVC-specific type sorts out of `TypeSort`

Make them vendor extensions. They include

• Unaligned
• SyntaxTree

Divergence between fundamental-type-precision definition in the specification and .ifc produced by MSVC

precision of the type __int64 in .ifc file produced by MSVC is equal to 6, but according to the specification sec:ifc-fundamental-type-precision

enum class TypePrecision : uint8_t {
  Default,
  Short,
  Long,
  Bit16,
  Bit32,
  Bit64,
  Bit128,
};

__int64 should have precision 5 (TypePrecision::Bit64).

Unify `ExceptionSpecification` and `NoexceptSort`

ExceptionSpecification and NoexceptSort are very close in effects and semantics. Unify them.

Representation of aggregate struct initializer

struct X {
    int field;
};

export constexpr X x { 1 };

According to the current state of IFC-spec, initializer of x is ProductTypeValue expression, with .members = ExprSort::SubobjectValue.
Unfortunately, structure SubobjectValue is undefined and marked as "scheduled for removal in future releases".
So, right now it's impossible to read as basic stuff as aggregate initialization from IFC.

Divergence between expr.template-reference definition in the specification and .ifc produced by MSVC

The definition of Expression.TemplateReference looks like

struct {
        SourceLocation locus;
        TypeIndex type;
        NameIndex member_name;
        SourceLocation member_locus;
        TypeIndex scope;
        ExprIndex arguments;
}

in the specification right now (sec:ifc:ExprSort:TemplateReference), so its sizeof should be equal 32, but entry_size in the partition "expr.template-reference" produced by MSVC is equal to 28.

Divergence between syntax.function-declarator definition in the specification and .ifc produced by MSVC

The definition of Syntax.FunctionDeclarator looks like

struct {
    SyntaxIndex parameters;
    SyntaxIndex eh_spec;
    SourceLocation left_paren;
    SourceLocation  right_paren;
}

in the specification right now (sec:ifc:SyntaxSort:FunctionDeclarator), so its sizeof should be equal 24, but entry_size in the partition "syntax.function-declarator" produced by MSVC (IFC-version 0.41) is equal to 44.

DirSort::StructuredBinding has an incomplete specification

The bindings field is specified as DeclSequence which does not exist; I'm not sure what this is.

Similar to #100, there's also a specifiers field with the type DeclSpecifierSequence which I'm assuming is SyntaxSort::DeclSpecifierSeq.

There's also a names field with the unspecified type IdentifierSequence.

Create representation for implicit/explicit object parameters

Consider:

struct S {
  void f(this S);
};

Today, the IFC has no way of representing the explicit object parameter member function S::f. We could fix this by creating a new type to wrap the parameter type of f but this is not generic enough. Rather we should consider a solution which could see MethodType removed in favor of a more generalized function type containing the explicit/implicit object parameter. This was also discussed in an IPR issue.

Augment Module Reference to optionally retain checksum of referenced IFC

An IFC file header has provision for storing the hash of its contents. However, a reference to an IFC (as in Module Reference) does not make provision for storing the hash of the IFC referenced when the referencing IFC was built. This can lead to mismatched or skewed situations that are not readily detected.

Suggestion: Augment the structure for Module References to optionally store the hash of the IFC that was referenced when the referencing IFC was built.

TypeSort::Placeholder has surprising interior padding following "basis"

The IFC spec describes TypeSort::Placeholder as:

|-------------|-----------------------------|
| Name        | Type                        |
|-------------|-----------------------------|
| constraint  | ExprIndex                   |
| basis       | TypeBasis                   |
| elaboration | TypeIndex                   |
|-------------|-----------------------------|

The true representation is:

|-------------|-----------------------------|
| Name        | Type                        |
|-------------|-----------------------------|
| constraint  | ExprIndex                   |
| basis       | TypeBasis                   |
| __pading__  | uint8[3]                    |
| elaboration | TypeIndex                   |
|-------------|-----------------------------|

Ideally, this would be as follows to match other cases of padding in the IFC spec:

|-------------|-----------------------------|
| Name        | Type                        |
|-------------|-----------------------------|
| constraint  | ExprIndex                   |
| basis       | TypeBasis                   |
| elaboration | TypeIndex                   |
| __pading__  | uint8[3]                    |
|-------------|-----------------------------|

At a minimum, this padding should be documented in the spec.

Sequester MSVC-specific declaration sorts out of `DeclSort`

Remove the MSVC-specific declaration sorts from DeclSort, and move them to vendor extensions. They include:

• SyntaxTree
• Intrinsic
• Property
• OutputSegment

DeclSort::UsingDirective

IFC 0.43 (and earlier) contain DeclSort::UsingDirective but do not document its fields.

Type StorageClass not defined

For SyntaxSort::DeclSpecifierSeq, the type StorageClass of the field storage_class is not defined anywhere in the document.

What exactly stored in "const.f64" partition?

The specification says

The value at that entry is a 64-bit floating point value

but entry_size in the partition "const.f64" produced by MSVC (IFC-version 0.40) is equal to 12.

String literal structure (i.e., "const.str") length documentation

The string literal should have its length documentation updated to indicate that the given length includes the terminating null character from the string table.

Missing definition of enum 'PointerDeclaratorSort' in the section sec:ifc:SyntaxSort:PointerDeclarator

https://github.com/microsoft/ifc-spec/blob/main/ltx/parse.tex#L609