ScreenLockDetector/docs/VULKAN_INTEGRATION.md

Vulkan Integration Documentation

Overview

This document describes the Vulkan integration in the ScreenLockDetector project. The project now includes a VulkanWidget that uses native window properties to create Vulkan surfaces and renders using the Vulkan API.

Architecture

Components

1. VulkanWidget (src/vulkanwidget.h, src/vulkanwidget.cpp)

   • Qt widget that uses native window handles for Vulkan surface creation
   • Implements Vulkan rendering pipeline with swapchain management
   • Provides frame counting and rendering control

2. Volk Integration (third_party/volk/)

   • Meta-loader for Vulkan that eliminates the need to link directly against vulkan-1.dll/libvulkan.so
   • Loads Vulkan functions dynamically at runtime
   • Version: 1.3.295

Key Features

• Native Window Surface Creation: Uses Qt's native window handles to create platform-specific Vulkan surfaces
• Dynamic Vulkan Loading: Uses volk to load Vulkan functions at runtime
• Cross-Platform Support: Supports Windows (Win32), Linux (Xlib), and macOS (Metal)
• Swapchain Management: Automatic swapchain recreation on window resize
• Frame Synchronization: Uses semaphores and fences for proper frame synchronization
• Simple Rendering: Demonstrates color cycling as a proof-of-concept

Implementation Details

Widget Attributes

The VulkanWidget sets the following Qt attributes:

setAttribute(Qt::WA_NativeWindow);      // Create native window handle
setAttribute(Qt::WA_PaintOnScreen);     // Paint directly to screen
setAttribute(Qt::WA_OpaquePaintEvent);  // Widget is opaque

These attributes ensure that:

• Qt creates a native window handle accessible via windowHandle()
• Qt doesn't interfere with Vulkan rendering
• The widget owns its painting surface

Vulkan Initialization Sequence

1. Initialize Volk: volkInitialize() loads the Vulkan loader
2. Create Instance: Create Vulkan instance with required surface extensions
3. Create Surface: Create platform-specific surface from native window handle
4. Select Physical Device: Find GPU that supports graphics and presentation
5. Create Logical Device: Create device with swapchain extension
6. Create Swapchain: Set up swapchain with appropriate format and present mode
7. Create Command Objects: Allocate command pool and command buffers
8. Create Sync Objects: Create semaphores and fences for frame synchronization

Platform-Specific Surface Creation

Windows (Win32)

VkWin32SurfaceCreateInfoKHR createInfo = {};
createInfo.sType = VK_STRUCTURE_TYPE_WIN32_SURFACE_CREATE_INFO_KHR;
createInfo.hwnd = reinterpret_cast<HWND>(window->winId());
createInfo.hinstance = GetModuleHandle(nullptr);
vkCreateWin32SurfaceKHR(m_instance, &createInfo, nullptr, &m_surface);

Linux (Xlib)

VkXlibSurfaceCreateInfoKHR createInfo = {};
createInfo.sType = VK_STRUCTURE_TYPE_XLIB_SURFACE_CREATE_INFO_KHR;
createInfo.dpy = reinterpret_cast<Display*>(
    qApp->platformNativeInterface()->nativeResourceForWindow("display", window));
createInfo.window = static_cast<Window>(window->winId());
vkCreateXlibSurfaceKHR(m_instance, &createInfo, nullptr, &m_surface);

macOS (Metal)

VkMetalSurfaceCreateInfoEXT createInfo = {};
createInfo.sType = VK_STRUCTURE_TYPE_METAL_SURFACE_CREATE_INFO_EXT;
createInfo.pLayer = nullptr; // Needs proper Metal layer setup
vkCreateMetalSurfaceEXT(m_instance, &createInfo, nullptr, &m_surface);

Rendering Loop

The rendering loop runs at ~60 FPS (16ms intervals) when enabled:

1. Wait for previous frame fence
2. Acquire next swapchain image
3. Reset command buffer and record commands
4. Submit command buffer with semaphore synchronization
5. Present image to screen
6. Handle swapchain recreation if needed

Swapchain Recreation

The swapchain is automatically recreated when:

• Window is resized (resizeEvent)
• Swapchain becomes out of date (VK_ERROR_OUT_OF_DATE_KHR)
• Swapchain becomes suboptimal (VK_SUBOPTIMAL_KHR)

Building

Prerequisites

• Vulkan SDK: Install the LunarG Vulkan SDK

  • Windows: Download from https://vulkan.lunarg.com/
  • Linux: sudo apt install vulkan-tools libvulkan-dev vulkan-validationlayers
  • macOS: Download Vulkan SDK and install MoltenVK

• Qt5: Qt 5.15 or later with Widgets module

• X11 (Linux only): sudo apt install libx11-dev

CMake Configuration

The CMakeLists.txt has been updated to:

1. Find the Vulkan package: find_package(Vulkan REQUIRED)
2. Build volk as a static library from source
3. Link the application with volk and Vulkan libraries
4. Add platform-specific libraries (X11 on Linux, Metal/QuartzCore on macOS)

Build Commands

cd ScreenLockDetector
mkdir -p build
cd build
cmake ..
make

Usage

In the Application

The VulkanWidget is integrated into the main window as a tab:

1. QPainter Widget Tab: Original custom widget using Qt's QPainter
2. Vulkan Widget Tab: New Vulkan-based rendering widget

Controls

• Enable Rendering: Initialize Vulkan (if needed) and start rendering
• Disable Rendering: Stop the rendering loop
• Reset Frame Count: Reset the frame counter to 0

Status Information

The widget displays:

• Vulkan Status: Initialization state and any error messages
• Rendering Status: Whether rendering is active or stopped
• Frame Count: Total number of frames rendered

Troubleshooting

Vulkan Initialization Fails

Problem: "Failed to initialize volk" or "Failed to create Vulkan instance"

Solutions:

• Ensure Vulkan SDK is installed and Vulkan drivers are up to date
• Check that VK_LOADER_DEBUG=all environment variable shows loader activity
• Verify GPU supports Vulkan (run vulkaninfo or vkcube)

Surface Creation Fails

Problem: "Failed to create Vulkan surface"

Linux Solutions:

• Ensure X11 development libraries are installed
• Check that the application is running under X11 (not Wayland)
• Verify DISPLAY environment variable is set

macOS Solutions:

• Ensure MoltenVK is properly installed
• Check that Metal is supported on the device

Black Screen or No Rendering

Problem: Widget shows but nothing renders

Solutions:

• Check the status message for initialization errors
• Verify that "Enable Rendering" has been clicked
• Check console output for Vulkan errors
• Ensure window is visible and has non-zero size

Performance Issues

Problem: Low frame rate or stuttering

Solutions:

• Check if VSync is forcing specific frame rates
• Verify GPU is not thermal throttling
• Check for swapchain recreation messages (indicates excessive resizing)
• Reduce window size for testing

Code Structure

ScreenLockDetector/
├── src/
│   ├── vulkanwidget.h          # VulkanWidget class declaration
│   ├── vulkanwidget.cpp        # VulkanWidget implementation
│   ├── mainwindow.h            # Updated with VulkanWidget integration
│   └── mainwindow.cpp          # Updated with VulkanWidget tab
├── third_party/
│   └── volk/                   # Volk meta-loader source
│       ├── volk.h              # Volk header
│       ├── volk.c              # Volk implementation
│       └── ...
└── CMakeLists.txt              # Updated with Vulkan/volk support

Extending the Renderer

Adding Graphics Pipeline

To add a proper graphics pipeline:

1. Create shader modules (SPIR-V compiled shaders)
2. Define vertex input format
3. Create pipeline layout
4. Create graphics pipeline
5. Create framebuffers for swapchain images
6. Update recordCommandBuffer() to use the pipeline

Adding Geometry

To render actual geometry:

1. Create vertex and index buffers
2. Allocate device memory
3. Upload geometry data
4. Bind buffers in command buffer recording
5. Issue draw calls

Adding Textures

To add texture support:

1. Create VkImage and VkImageView
2. Create VkSampler
3. Update descriptor sets
4. Sample textures in fragment shader

References

• Vulkan Tutorial: https://vulkan-tutorial.com/
• Vulkan Specification: https://www.khronos.org/registry/vulkan/
• Volk Repository: https://github.com/zeux/volk
• Qt Native Interface: https://doc.qt.io/qt-5/qpa.html

License

This Vulkan integration follows the same license as the main project. Volk is licensed under the MIT License.

Future Improvements

• Add validation layers in debug builds
• Implement proper render pass and framebuffers
• Add graphics pipeline with shaders
• Implement geometry rendering
• Add texture support
• Integrate with screen lock detection (pause rendering on lock)
• Add Wayland support for Linux
• Improve macOS Metal layer setup
• Add performance metrics (FPS, frame time)
• Implement multi-threading for command buffer recording