Category Archives: Vulkan

Everything about Vulkan API: programming, implementation details, ideas and suggestions. How to take the best from this understimated library.

TextureMind Framework – Progress #14 – Vulkan – Skinned mesh

Skinned mesh rendering is a fundamental part of every modern 3D engine, so I couldn't avoid to implement it. The skinned mesh with weights, indices, bones, skeleton, and animated nodes is imported with AssImp library into my format. I added weights and indices to the vertex attributes while bones matrices are written into a shader storage object. The skinned mesh is computed with the GPU, by the vertex shader.

In the video you can see the final result of the implementation. The model has been imported from Doom 3 format into my format, then animated and rendered by the 3D engine. For now, the keys with quaternions are interpolated with a slerp for every frames. An optimization can be to pre-calculate all the bones into a SSBO at fixed frame rate (like 60 fps) and use it to render a massive amount of meshes.

TextureMind Framework – Progress #13 – Vulkan – Advanced materials

I improved the material system, introducing the lighting stage. I removed the fragment stage and replaced it with color and lighting stage. The output color is calculated as the sum of the color stage and the lighting stage. The color stage has only one material node as input that is used to produce the output color for this stage.

The lighting stage wants more inputs, like ambient, diffuse (or albedo), specular, roughness, metalness, that are mixed in a physical based rendering (PBR). Each input is connected to one material node that can be the result of the operation between more material nodes, so every stage can have it's own textures or the math operation between more textures, uniforms and constants. In the video you can see a model with advanced materials imported to show the benefits of the last optimizations. In this case, the ambient stage is rendered correctly and mixed with the diffuse textures.

TextureMind Framework – Progress #12 – Vulkan – Import materials and normal maps

Now the importer with AssImp library is capable to import model materials and textures into my format. I added also support for normal maps with tangent and bitangent vertex attributes, improving the lighting stage in the fragment shader to render it properly.

In the video you can see nano suit model imported from collada format. As the object rotates, you can see the benefits of bump mapping and specular textures.

TextureMind Framework – Progress #11 – Vulkan – Import 3D model

I decided to use AssImp library to import models from other formats to my 3D mesh format. The video shows a first implementation of the importer.

Vertices and normals are converted along the skeleton structure, while the red material is generated just to render the model on the screen. The next step is to load the materials and the associated textures.

TextureMind Framework – Progress #10 – Vulkan – Materials and 3D rendering

Finally, the very first 3D model rendered by the 3D engine. Even if it looks like a simple torus demo, the main feature this time is the format used for the 3D mesh and the convertion from material nodes to vulkan shader, for the rendering.

The mesh is composed by a polygon hull, a set of vertex attributes and a layout that defines the nature of vertex attributes. The polygon hull represents the geometric structure of the mesh while vertex attributes define the graphics and the physical aspect. A mesh can have virtually any number of vertex attributes, that can be: position, normal, colors, texcoords and other new attributes used by the material.

Materials are composed by expression nodes, then converted to shaders in a second step. Every material has a layout with the number of vertex attributes required for the rendering. The material structure used to render this model is the following:

The layout of the mesh doesn't have to match exactly with the material's one: if the mesh has the required vertex attribute then it's used, otherwise 0 values are used instead. It's for the material to decide how to use the vertex attributes offered by the mesh. In this way, a single material can be used to render any kind of mesh. Of course, a mesh without normals cannot render diffuse or specular, or without texcoords cannot render textures, normal maps and so on.

Uniform buffers can be used by a single mesh to change the material content, like colors or texture coords. For instance, the color of diffuse in this material can be connected to a uniform contained by a 3D mesh, that can be changed on the fly, changing the color of the object. In this way, it's possible to reuse the same materials for multiple objects, even with different aspects, like particles or game characters.

TextureMind Framework – Progress #9 – Vulkan – Materials and textures

I improved the implementation of materials and textures with Vulkan. Now every material is translated into a GLSL shader that is converted into SPIRV code with shaderc library. The shader is generated along the graphics pipeline to match the material settings. For now materials are very simple and used to draw an image texture with alpha blending or a filled color.

As you can see from the video, now the gui has normal appearence instead of rainbow rectangles of before. The next step is to support path rendering and font rendering, for drawing the text. In the future, the same materials system will be used to draw 3D content too.

TextureMind Framework – Progress #8 – Graphics context and 2D GUI with Vulkan

I am happy to announce that Vulkan library has finally been integrated into my framework. For the moment nothing complicated, I limited myself to implement a specialization of the Graphics Context that draws simple colored rectangles instead of the images drawn by Cairo library. It's possible to invoke drawing commands with the same degree of complexity and practically identical management of textures, materials and uniforms, at programming interface level.

Each rectangle is associated with a transformation matrix, which is translated into a uniform buffer. It's also possible to rationalize the rendering into multiple layers, allowing the reuse of command buffers with a minimum programming effort.

As you can see from the above image, the 2D GUI based on the graphics context worked quite well. It's possible to drag the windows and see them move on the screen at a high framerate, which is the main purpose for which it's worth bothering the Vulkan libraries.

For the moment there is an implementation of textures and materials, but I have not yet finished the rendering part at shader level. The difficulty lies in the fact that the framework must resolve the material nodes to extract the proper GLSL shader to be converted into SPIRV, create a suitable graphics pipeline and set it before rendering. The next step is to finish this part and make the 2D GUI identical to Cairo version.

Then I can proceed implementing 3D functionality, with a full material management. The main goal is to implement an importer with assimp library and load 3D models. Then I will proceed refining the 3D functionality with a sophiticated engine optimized for modern real-time computer graphics.

Why I decided to adopt Vulkan

I had to decide the graphics library for the 3D rendering part of my framework, and I decided to adopt Vulkan for a series of reasons. First of all, after years of experience with OpenGL library, I decided that the trend of adding extensions is not good for the maintenance of the software. You may say that there are many wrappers around and you don't have to care about the underlying library, but it's not a good practice at all.

Vulkan (API) - Wikipedia

You are going to add an external dependency on your software, you don't have control over it and as a programmer, you are not going to learn anything (apart the wrapper itself). I programmed several OpenGL applications without using any wrapper and it's the best thing to do. In this way, you learn how the library works and all the issues related to that. One of the worst is the huge quantity of extensions. If you are lucky, the OpenGL version that you are targetting already has all the extensions that you need to use, so you don't need to check them one by one for the functionality that you are supporting, but just the OpenGL version. However, the target version always depends on what you need to do. If you want to support OpenGL 4.5, you need to know if all the GPUs that you want to support have that particular version of OpenGL. One day you could find that OpenGL 4.5 is not supported on a particular hardware and that your engine is not so good from a scalability point of view. Falling back to older OpenGL versions may require an entire refactor of the engine, that is not so nice if you had to release for tomorrow. On the contrary, if you need to support older versions of OpenGL from 3.0 onward, you will experiment on your person how hellish could be to deal with the outstanding amount of extensions that the library has just to support the most basic things. And anyway, this is not even the main problem. Even if you drop the past, you cannot prevent the GPUs vendors and the Khronos group from releasing extensions in the future that will be required to have decent performance on modern hardware, because the entire OpenGL architecture is obsolete.
Continue reading