Crash Course: runtime reflection system

Introduction
Names and identifiers
Reflection in a nutshell

Introduction

Reflection (or rather, its lack) is a trending topic in the C++ world and, in the specific case of EnTT, a tool that can unlock a lot of other features. I looked for a third-party library that met my needs on the subject, but I always came across some details that I didn't like: macros, being intrusive, too many allocations. In one word: unsatisfactory.
I finally decided to write a built-in, non-intrusive and macro-free runtime reflection system for EnTT. Maybe I didn't do better than others or maybe yes, time will tell me, but at least I can model this tool around the library to which it belongs and not the opposite.

Names and identifiers

The meta system doesn't force users to rely on the tools provided by the library when it comes to working with names and identifiers. It does this by offering an API that works with opaque identifiers that may or may not be generated by means of a hashed string.
This means that users can assign any type of identifier to the meta objects, as long as they are numeric. It doesn't matter if they are generated at runtime, at compile-time or with custom functions.

That being said, the examples in the following sections are all based on the hashed_string class as provided by this library. Therefore, where an identifier is required, it's likely that a user defined literal is used as follows:

auto factory = entt::meta<my_type>().type("reflected_type"_hs);

For what it's worth, this is likely completely equivalent to:

auto factory = entt::meta<my_type>().type(42);

Obviously, human-readable identifiers are more convenient to use and highly recommended.

Reflection in a nutshell

Reflection always starts from real types (users cannot reflect imaginary types and it would not make much sense, we wouldn't be talking about reflection anymore).
To create a meta node, the library provides the meta function that accepts a type to reflect as a template parameter:

auto factory = entt::meta<my_type>();

This isn't enough to export the given type and make it visible though.
The returned value is a factory object to use to continue building the meta type. In order to make the type visible, users can assign it an identifier:

auto factory = entt::meta<my_type>().type("reflected_type"_hs);

Or use the default one, that is, the built-in identifier for the given type:

auto factory = entt::meta<my_type>().type();

Identifiers are important because users can retrieve meta types at runtime by searching for them by name other than by type.
On the other hand, there are cases in which users can be interested in adding features to a reflected type so that the reflection system can use it correctly under the hood, but they don't want to also make the type searchable. In this case, it's sufficient not to invoke type.

A factory is such that all its member functions returns the factory itself or a decorated version of it. This object can be used to add the following:

Constructors. Actual constructors can be assigned to a reflected type by specifying their list of arguments. Free functions (namely, factories) can be used as well, as long as the return type is the expected one. From a client's point of view, nothing changes if a constructor is a free function or an actual constructor.
Use the ctor member function for this purpose:
```
entt::meta<my_type>().ctor<int, char>().ctor<&factory>();
```
Destructors. Free functions can be set as destructors of reflected types. The purpose is to give users the ability to free up resources that require special treatment before an object is actually destroyed.
Use the dtor member function for this purpose:
```
entt::meta<my_type>().dtor<&destroy>();
```

A function should neither delete nor explicitly invoke the destructor of a given instance.

Data members. Both real data members of the underlying type and static and global variables, as well as constants of any kind, can be attached to a meta type. From a client's point of view, all the variables associated with the reflected type will appear as if they were part of the type itself.
Use the data member function for this purpose:
```
entt::meta<my_type>()
  .data<&my_type::static_variable>("static"_hs)
  .data<&my_type::data_member>("member"_hs)
  .data<&global_variable>("global"_hs);
```

This function requires as an argument the identifier to give to the meta data once created. Users can then access meta data at runtime by searching for them by name.
Data members can also be defined by means of a setter and getter. Setters and getters can be either free functions, class members or a mix of them, as long as they respect the required signatures. This approach is also convenient to create a read-only variable from a non-const data member:

  entt::meta<my_type>().data<nullptr, &my_type::data_member>("member"_hs);

Refer to the inline documentation for all the details.

Member functions. Both real member functions of the underlying type and free functions can be attached to a meta type. From a client's point of view, all the functions associated with the reflected type will appear as if they were part of the type itself.
Use the func member function for this purpose:
```
entt::meta<my_type>()
  .func<&my_type::static_function>("static"_hs)
  .func<&my_type::member_function>("member"_hs)
  .func<&free_function>("free"_hs);
```

This function requires as an argument the identifier to give to the meta function once created. Users can then access meta functions at runtime by searching for them by name.

Base classes. A base class is such that the underlying type is actually derived from it. In this case, the reflection system tracks the relationship and allows for implicit casts at runtime when required.
Use the base member function for this purpose:
```
entt::meta<derived_type>().base<base_type>();
```

From now on, wherever a base_type is required, an instance of derived_type will also be accepted.

Conversion functions. Actual types can be converted, this is a fact. Just think of the relationship between a double and an int to see it. Similar to bases, conversion functions allow users to define conversions that will be implicitly performed by the reflection system when required.
Use the conv member function for this purpose:
```
entt::meta<double>().conv<int>();
```

That's all, everything users need to create meta types and enjoy the reflection system. At first glance it may not seem that much, but users usually learn to appreciate it over time.
Also, do not forget what these few lines hide under the hood: a built-in, non-intrusive and macro-free system for reflection in C++. Features that are definitely worth the price, at least for me.

Any as in any type

The reflection system comes with its own meta_any type. It may seem redundant since C++17 introduced std::any, but it is not.
In fact, the type returned by an std::any is a const reference to an std::type_info, an implementation defined class that's not something everyone wants to see in a software. Furthermore, the class std::type_info suffers from some design flaws and there is even no way to convert an std::type_info into a meta type, thus linking the two worlds.

The class meta_any offers an API similar to that of its most famous counterpart and serves the same purpose of being an opaque container for any type of value.
It minimizes the allocations required, which are almost absent thanks to SBO techniques. In fact, unless users deal with fat types and create instances of them through the reflection system, allocations are at zero.

Creating instances of meta_any, whether empty or from existing objects, is trivial:

// a container for an int
entt::meta_any any{0};

// an empty container
entt::meta_any empty{};

The meta_any class takes also the burden of destroying the contained object when required.
Furthermore, an instance of meta_any is not tied to a specific type. Therefore, the wrapper will be reconfigured by assigning it an object of a different type than the one contained, so as to be able to handle the new instance.

A particularly interesting feature of this class is that it can also be used as an opaque container for non-const unmanaged objects:

int value;
entt::meta_any any{std::ref(value)};

In other words, whenever meta_any intercepts a reference_wrapper, it acts as a reference to the original instance rather than making a copy of it. The contained object is never destroyed and users must ensure that its lifetime exceeds that of the container.
Similarly, it's possible to create non-owning copies of meta_any from existing ones:

// aliasing constructor
entt::meta_any ref = any.ref();

In this case, it doesn't matter if the starting container actually holds an object or acts already as a reference for unmanaged elements, the new instance thus created won't create copies and will only serve as a reference for the original item.
It means that, starting from the example above, both ref andany will point to the same object, whether it's initially contained in any or already an unmanaged one. This is particularly useful for passing instances of meta_any belonging to the external context by reference to a function or a constructor rather than making copies of them.

The meta_any class also has a type member function that returns the meta type of the contained value, if any. The member functions try_cast, cast and convert are then used to know if the underlying object has a given type as a base or if it can be converted implicitly to it.

Enjoy the runtime

Once the web of reflected types has been constructed, it's a matter of using it at runtime where required.
All this has the great merit that, unlike the vast majority of the things present in this library and closely linked to the compile-time, the reflection system stands in fact as a non-intrusive tool for the runtime.

To search for a reflected type there are a few options:

// direct access to a reflected type
auto by_type = entt::resolve<my_type>();

// lookup of a reflected type by identifier
auto by_id = entt::resolve_id("reflected_type"_hs);

// lookup of a reflected type by type id
auto by_type_id = entt::resolve_type(entt::type_info<my_type>::id());

There exits also an overload of the resolve function to use to iterate all the reflected types at once as well as a resolve_if function to use to perform more refined searches when needed:

resolve([](auto type) {
    // ...
});

auto by_lookup = resolve_if([](auto type) { return type.is_floating_point(); });

In all cases, the returned value is an instance of meta_type. This kind of objects offer an API to know their runtime identifiers, to iterate all the meta objects associated with them and even to build instances of the underlying type.
Refer to the inline documentation for all the details.

The meta objects that compose a meta type are accessed in the following ways:

Meta constructors. They are accessed by types of arguments:
```
auto ctor = entt::resolve<my_type>().ctor<int, char>();
```

The returned type is meta_ctor and may be invalid if there is no constructor that accepts the supplied arguments or at least some types from which they are derived or to which they can be converted.
A meta constructor offers an API to know the number of its arguments and their expected meta types. Furthermor, it's possible to invoke it and therefore to construct new instances of the underlying type.

Meta data. They are accessed by name:

auto data = entt::resolve<my_type>().data("member"_hs);

The returned type is meta_data and may be invalid if there is no meta data object associated with the given identifier.
A meta data object offers an API to query the underlying type (for example, to know if it's a const or a static one), to get the meta type of the variable and to set or get the contained value.

Meta functions. They are accessed by name:

auto func = entt::resolve<my_type>().func("member"_hs);

The returned type is meta_func and may be invalid if there is no meta function object associated with the given identifier.
A meta function object offers an API to query the underlying type (for example, to know if it's a const or a static function), to know the number of arguments, the meta return type and the meta types of the parameters. In addition, a meta function object can be used to invoke the underlying function and then get the return value in the form of a meta_any object.

Meta bases. They are accessed through the name of the base types:
```
auto base = entt::resolve<derived_type>().base("base"_hs);
```

The returned type is meta_base and may be invalid if there is no meta base object associated with the given identifier.
Meta bases aren't meant to be used directly, even though they are freely accessible. They expose only a few methods to use to know the meta type of the base class and to convert a raw pointer between types.

Meta conversion functions. They are accessed by type:
```
auto conv = entt::resolve<double>().conv<int>();
```

The returned type is meta_conv and may be invalid if there is no meta conversion function associated with the given type.
The meta conversion functions are as thin as the meta bases and with a very similar interface. The sole difference is that they return a newly created instance wrapped in a meta_any object when they convert between different types.

All the objects thus obtained as well as the meta types can be explicitly converted to a boolean value to check if they are valid:

if(auto func = entt::resolve<my_type>().func("member"_hs); func) {
    // ...
}

Furthermore, all meta objects can be iterated through an overload that accepts a callback through which to return them. As an example:

entt::resolve<my_type>().data([](auto data) {
    // ...
});

A meta type can be used to construct actual instances of the underlying type.
In particular, the construct member function accepts a variable number of arguments and searches for a match. It then returns a meta_any object that may or may not be initialized, depending on whether a suitable constructor has been found or not.

There is no object that wraps the destructor of a meta type nor a destroy member function in its API. The reason is quickly explained: destructors are invoked implicitly by meta_any behind the scenes and users have not to deal with them explicitly. Furthermore, they have no name, cannot be searched and wouldn't have member functions to expose anyway.
Therefore, exposing destructors would be pointless and would add nothing to the library itself.

Meta types and meta objects in general contain much more than what is said: a plethora of functions in addition to those listed whose purposes and uses go unfortunately beyond the scope of this document.
I invite anyone interested in the subject to look at the code, experiment and read the inline documentation to get the best out of this powerful tool.

Container support

The meta module supports containers of all types out of the box.
Moreover, containers doesn't necessarily mean those offered by the C++ standard library. In fact, user defined data structures can also work with the meta system in many cases.

To make a container be recognized by the meta module, users are required to provide specializations for either the meta_sequence_container_traits class or the meta_associative_container_traits class, according with the actual type of the container.
EnTT already exports the specializations for some common classes. In particular:

std::vector and std::array are exported as sequence containers.
std::map, std::set and their unordered counterparts are exported as associative containers.

It's important to include the header file container.hpp to make these specializations available to the compiler when needed.
The same file also contains many examples for the users that are interested in making their own containers available to the meta system.

When a specialization of the meta_sequence_container_traits class exists, the meta system treats the wrapped type as a sequence container. In a similar way, a type is treated as an associative container if a specialization of the meta_associative_container_traits class is found for it.
Proxy objects are returned by dedicated members of the meta_any class. The following is a deliberately verbose example of how users can access a proxy object for a sequence container:

std::vector<int> vec{1, 2, 3};
entt::meta_any any{std::ref(vec)};

if(any.type().is_sequence_container()) {
    if(auto view = any.as_sequence_container(); view) {
        // ...
    }
}

The method to use to get a proxy object for associative containers is as_associative_container instead.
It goes without saying that it's not necessary to perform a double check. Instead, it's sufficient to query the meta type or verify that the proxy object is valid. In fact, proxies are contextually convertible to bool to know if they are valid. For example, invalid proxies are returned when the wrapped object isn't a container.
In all cases, users aren't expected to reflect containers explicitly. It's sufficient to assign a container for which a specialization of the traits classes exists to a meta_any object to be able to get its proxy object.

The interface of the meta_sequence_container proxy object is the same for all types of sequence containers, although the available features differ from case to case. In particular:

The value_type member function returns the meta type of the elements.
The size member function returns the number of elements in the container as an unsigned integer value:
```
const auto size = view.size();
```
The resize member function allows to resize the wrapped container and returns true in case of succes:
```
const bool ok = view.resize(3u);
```

For example, it's not possible to resize fixed size containers.

The clear member function allows to clear the wrapped container and returns true in case of success:
```
const bool ok = view.clear();
```

For example, it's not possible to clear fixed size containers.

The begin and end member functions return opaque iterators that can be used to iterate the container directly:
```
for(entt::meta_any element: view) {
  // ...
}
```

In all cases, given an underlying container of type C, the returned element contains an object of type C::value_type which therefore depends on the actual container.
All meta iterators are input iterators and don't offer an indirection operator on purpose.

The insert member function can be used to add elements to the container. It accepts a meta iterator and the element to insert:
```
auto last = view.end();
// appends an integer to the container
view.insert(last.handle(), 42);
```

This function returns a meta iterator pointing to the inserted element and a boolean value to indicate whether the operation was successful or not. Note that a call to insert may silently fail in case of fixed size containers or whether the arguments aren't at least convertible to the required types.
Since the meta iterators are contextually convertible to bool, users can rely on them to know if the operation has failed on the actual container or upstream, for example for an argument conversion problem.

The erase member function can be used to remove elements from the container. It accepts a meta iterator to the element to remove:
```
auto first = view.begin();
// removes the first element from the container
view.erase(first);
```

This function returns a meta iterator following the last removed element and a boolean value to indicate whether the operation was successful or not. Note that a call to erase may silently fail in case of fixed size containers.

The operator[] can be used to access elements in a container. It accepts a single argument, that is the position of the element to return:
```
for(std::size_t pos{}, last = view.size(); pos < last; ++pos) {
  entt::meta_any value = view[pos];
  // ...
}
```

The function returns instances of meta_any that directly refer to the actual elements. Modifying the returned object will then directly modify the element inside the container.

Similarly, also the interface of the meta_associative_container proxy object is the same for all types of associative containers. However, there are some differences in behavior in the case of key-only containers. In particular:

The key_only member function returns true if the wrapped container is a key-only one.
The key_type member function returns the meta type of the keys.
The mapped_type member function returns an invalid meta type for key-only containers and the meta type of the mapped values for all other types of containers.
The value_type member function returns the meta type of the elements.
For example, it returns the meta type of int for std::set<int> while it returns the meta type of std::pair<const int, char> for std::map<int, char>.
The size member function returns the number of elements in the container as an unsigned integer value:
```
const auto size = view.size();
```
The clear member function allows to clear the wrapped container and returns true in case of success:
```
const bool ok = view.clear();
```
The begin and end member functions return opaque iterators that can be used to iterate the container directly:
```
for(entt::meta_any element: view) {
  // ...
}
```

The insert member function can be used to add elements to the container. It accepts two arguments, respectively the key and the value to be inserted:
```
auto last = view.end();
// appends an integer to the container
view.insert(last.handle(), 42, 'c');
```

This function returns a boolean value to indicate whether the operation was successful or not. Note that a call to insert may fail when the arguments aren't at least convertible to the required types.

The erase member function can be used to remove elements from the container. It accepts a single argument, that is the key to be removed:
```
view.erase(42);
```

This function returns a boolean value to indicate whether the operation was successful or not. Note that a call to erase may fail when the argument isn't at least convertible to the required type.

The operator[] can be used to access elements in a container. It accepts a single argument, that is the key of the element to return:
```
entt::meta_any value = view[42];
```

The function returns instances of meta_any that directly refer to the actual elements. Modifying the returned object will then directly modify the element inside the container.

Container support is deliberately minimal but theoretically sufficient to satisfy all needs.

Pointer-like types

As with containers, it's also possible to communicate to the meta system which types to consider pointers. This will allow to dereference instances of meta_any, obtaining light references to the pointed objects that are also correctly associated with their meta types.
To make the meta system recognize a type as pointer-like, users can specialize the is_meta_pointer_like class. EnTT already exports the specializations for some common classes. In particular:

All types of raw pointers.
std::uniqe_ptr and std::shared_ptr.

It's important to include the header file pointer.hpp to make these specializations available to the compiler when needed.
The same file also contains many examples for the users that are interested in making their own containers available to the meta system.

When a type is recognized as a pointer-like one by the meta system, it's possible to dereference the instances of meta_any that contain these objects. The following is a deliberately verbose example to show how to use this feature:

int value = 42;
// meta type equivalent to that of int *
entt::meta_any any{&value};

if(any.type().is_meta_pointer_like()) {
    // meta type equivalent to that of int
    if(entt::meta_any ref = *any; ref) {
        // ...
    }
}

It goes without saying that it's not necessary to perform a double check. Instead, it's sufficient to query the meta type or verify that the returned object is valid. For example, invalid instances are returned when the wrapped object hasn't a pointer-like type.
Note that dereferencing a pointer-like object returns an instance of meta_any which refers to the pointed object and allows users to modify it directly.

Policies: the more, the less

Policies are a kind of compile-time directives that can be used when recording reflection information.
Their purpose is to require slightly different behavior than the default in some specific cases. For example, when reading a given data member, its value is returned wrapped in a meta_any object which, by default, makes a copy of it. For large objects or if the caller wants to access the original instance, this behavior isn't desirable. Policies are there to offer a solution to this and other problems.

There are a few alternatives available at the moment:

The as-is policy, associated with the type entt::as_is_t.
This is the default policy. In general, it should never be used explicitly, since it's implicitly selected if no other policy is specified.
In this case, the return values of the functions as well as the properties exposed as data members are always returned by copy in a dedicated wrapper and therefore associated with their original meta types.
The as-void policy, associated with the type entt::as_void_t.
Its purpose is to discard the return value of a meta object, whatever it is, thus making it appear as if its type were void.
If the use with functions is obvious, it must be said that it's also possible to use this policy with constructors and data members. In the first case, the constructor will be invoked but the returned wrapper will actually be empty. In the second case, instead, the property will not be accessible for reading.

As an example of use:

  entt::meta<my_type>().func<&my_type::member_function, entt::as_void_t>("member"_hs);

The as-ref policy, associated with the type entt::as_ref_t.
It allows to build wrappers that act as references to unmanaged objects. Modifying the object contained in the wrapper for which the reference was requested will make it possible to directly modify the instance used to initialize the wrapper itself.
This policy works with constructors (for example, when objects are taken from an external container rather than created on demand), data members and functions in general (as long as their return types are lvalue references).

As an example of use:

  entt::meta<my_type>().data<&my_type::data_member, entt::as_ref_t>("member"_hs);

Some uses are rather trivial, but it's useful to note that there are some less obvious corner cases that can in turn be solved with the use of policies.

Named constants and enums

A special mention should be made for constant values and enums. It wouldn't be necessary, but it will help distracted readers.

As mentioned, the data member function can be used to reflect constants of any type among the other things.
This allows users to create meta types for enums that will work exactly like any other meta type built from a class. Similarly, arithmetic types can be enriched with constants of special meaning where required.
Personally, I find it very useful not to export what is the difference between enums and classes in C++ directly in the space of the reflected types.

All the values thus exported will appear to users as if they were constant data members of the reflected types.

Exporting constant values or elements from an enum is as simple as ever:

entt::meta<my_enum>()
        .data<my_enum::a_value>("a_value"_hs)
        .data<my_enum::another_value>("another_value"_hs);

entt::meta<int>().data<2048>("max_int"_hs);

It goes without saying that accessing them is trivial as well. It's a matter of doing the following, as with any other data member of a meta type:

auto value = entt::resolve<my_enum>().data("a_value"_hs).get({}).cast<my_enum>();
auto max = entt::resolve<int>().data("max_int"_hs).get({}).cast<int>();

As a side note, remember that all this happens behind the scenes without any allocation because of the small object optimization performed by the meta_any class.

Properties and meta objects

Sometimes (for example, when it comes to creating an editor) it might be useful to attach properties to the meta objects created. Fortunately, this is possible for most of them.
For the meta objects that support properties, the member functions of the factory used for registering them will return a decorated version of the factory itself. The latter can be used to attach properties to the last created meta object.
Apparently, it's more difficult to say than to do:

entt::meta<my_type>().type("reflected_type"_hs).prop("tooltip"_hs, "message");

Properties are always in the key/value form. There are no restrictions on the type of the key or value, as long as they are copy constructible objects.
Multiple formats are supported when it comes to defining a property:

Properties as key/value pairs:

entt::meta<my_type>().type("reflected_type"_hs).prop("tooltip"_hs, "message");

Properties as std::pairs:

entt::meta<my_type>().type("reflected_type"_hs).prop(std::make_pair("tooltip"_hs, "message"));

Key only properties:

entt::meta<my_type>().type("reflected_type"_hs).prop(my_enum::key_only);

Properties as std::tuples:

entt::meta<my_type>().type("reflected_type"_hs).prop(std::make_tuple(std::make_pair("tooltip"_hs, "message"), my_enum::key_only));

A tuple contains one or more properties. All of them are treated individually.

Annotations:

entt::meta<my_type>().type("reflected_type"_hs).prop(&property_generator);

An annotation is an invocable object that returns one or more properties. All of them are treated individually.

It's possible to invoke the prop function several times if needed, one for each property to associate with the last meta object created:

entt::meta<my_type>()
    .type("reflected_type"_hs)
        .prop(entt::hashed_string{"Name"}, "Reflected Type")
    .data<&my_type::data_member>("member"_hs)
        .prop(std::make_pair("tooltip"_hs, "Member"))
        .prop(my_enum::a_value, 42);

Alternatively, the props function is available to associate several properties at a time. However, in this case properties in the key/value form aren't allowed, since they would be interpreted as two different properties rather than a single one.

The meta objects for which properties are supported are currently the meta types, meta constructors, meta data and meta functions. It's not possible to attach properties to other types of meta objects and the factory returned as a result of their construction won't allow such an operation.

These types offer a couple of member functions named prop to iterate all properties at once or to search a specific property by key:

// iterate all properties of a meta type
entt::resolve<my_type>().prop([](auto prop) {
    // ...
});

// search for a given property by name
auto prop = entt::resolve<my_type>().prop("tooltip"_hs);

Meta properties are objects having a fairly poor interface, all in all. They only provide the key and the value member functions to be used to retrieve the key and the value contained in the form of meta_any objects, respectively.

Unregister types

A type registered with the reflection system can also be unregistered. This means unregistering all its data members, member functions, conversion functions and so on. However, the base classes won't be unregistered, since they don't necessarily depend on it. Similarly, implicitly generated types (as an example, the meta types implicitly generated for function parameters when needed) won't be unregistered.
Roughly speaking, unregistering a type means disconnecting all associated meta objects from it and making its identifier no longer visible. The underlying node will remain available though, as if it were implicitly generated:

entt::meta<my_type>().reset();

The type can be re-registered later with a completely different name and form.

meta.md 34 KB

Historie Surový

Crash Course: runtime reflection system

Table of Contents

Introduction

Names and identifiers

Reflection in a nutshell

Any as in any type

Enjoy the runtime

Container support

Pointer-like types

Policies: the more, the less

Named constants and enums

Properties and meta objects

Unregister types

meta.md 34 KB Historie Surový

Crash Course: runtime reflection system

Table of Contents

Introduction

Names and identifiers

Reflection in a nutshell

Any as in any type

Enjoy the runtime

Container support

Pointer-like types

Policies: the more, the less

Named constants and enums

Properties and meta objects

Unregister types

meta.md 34 KB

Historie Surový