Skip to content

store owning icd_term in loader_device instead of phys_dev_term - #2109

Open
aizu-m wants to merge 1 commit into
KhronosGroup:mainfrom
aizu-m:loader-device-icd-term
Open

aizu-m wants to merge 1 commit into
KhronosGroup:mainfrom
aizu-m:loader-device-icd-term

Conversation

@aizu-m

@aizu-m aizu-m commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

AddressSanitizer, macOS Debug build with LOADER_ENABLE_ADDRESS_SANITIZER=ON, new test run against the unpatched loader:

ERROR: AddressSanitizer: heap-use-after-free on address 0x6030000129d8
READ of size 8 at 0x6030000129d8 thread T0
    #0 in loader_lookup_device_dispatch_table vk_loader_extensions.c:1874
    #1 in vkGetDeviceProcAddr trampoline.c:148
0x6030000129d8 is located 8 bytes inside of 24-byte region [0x6030000129d0,0x6030000129e8)
freed by thread T0 here:
    #2 in loader_instance_heap_free allocation.c:100
    #3 in setup_loader_term_phys_devs loader.c:6447
    #4 in terminator_EnumeratePhysicalDevices loader.c:6613
previously allocated by thread T0 here:
    #3 in check_and_add_to_new_phys_devs loader.c:6180
    #4 in setup_loader_term_phys_devs loader.c:6393

Found while following up the note in #2103. That PR kept the driver loaded when its physical device disappears. The physical device object is still freed.

Investigation:

  • setup_loader_term_phys_devs runs on every vkEnumeratePhysicalDevices. It frees each old loader_physical_device_term that is not in the new list. It does not check for live devices.
  • terminator_CreateDevice stores that object in loader_device::phys_dev_term.
  • loader_lookup_device_dispatch_table reads dev->phys_dev_term->this_icd_term->this_instance before the name switch. Any vkGetDeviceProcAddr for a vk* name the trampoline does not answer itself gets there.
  • The other two readers (init_extension_device_proc_terminator_dispatch, the vkCreateSwapchainKHR log) also only want this_icd_term.

Conclusion: the device only needs its driver. The driver owns the loader_device through logical_device_list, so that pointer cannot dangle. The fix stores the loader_icd_term instead. The generator and generated file are updated together and generate_source.py --verify passes.

The test creates a device on each of two drivers and makes one driver fail enumeration. It then calls vkGetDeviceProcAddr on both devices. Unpatched it gives the trace above. Patched it passes. macOS (Debug, ASAN, BUILD_WERROR=ON): 655/655.

setup_loader_term_phys_devs frees every loader_physical_device_term
that is missing from the latest vkEnumeratePhysicalDevices call. A
VkDevice created from such a physical device kept a pointer to it, and
loader_lookup_device_dispatch_table read it on every vkGetDeviceProcAddr.

All users of the pointer only need the driver, which owns the
loader_device, so store the driver instead.
@ci-tester-lunarg

Copy link
Copy Markdown

User aizu-m not on autobuild list. Waiting for curator authorization before starting CI build.

1 similar comment
@ci-tester-lunarg

Copy link
Copy Markdown

User aizu-m not on autobuild list. Waiting for curator authorization before starting CI build.

@ci-tester-lunarg

Copy link
Copy Markdown

CI Vulkan-Loader build queued with queue ID 201736.

@ci-tester-lunarg

Copy link
Copy Markdown

CI Vulkan-Loader build # 4014 running.

@ci-tester-lunarg

Copy link
Copy Markdown

CI Vulkan-Loader build # 4014 passed.

@charles-lunarg charles-lunarg left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't the right solution. There is an underlying problem when the loader receives VK_ERROR_INITIALIZATION_FAILED from vkEnumeratePhysicalDevices, it frees VkPhysicalDevice handles that the application holds, which is against the spec.

I can confirm this by adding the following code to the end of DriverUnloadingFromZeroPhysDevs.GetDeviceProcAddrAfterPhysicalDeviceRemoved

    {
        DeviceWrapper dev_2{inst};
        dev_2.create_info.add_device_queue(DeviceQueueCreateInfo{}.add_priority(0.0f));
        dev_2.CheckCreate(phys_devs[0]);
        DeviceWrapper dev_3{inst};
        dev_3.create_info.add_device_queue(DeviceQueueCreateInfo{}.add_priority(0.0f));
        dev_3.CheckCreate(phys_devs[1]);
        PFN_vkVoidFunction create_command_pool_0 = inst->vkGetDeviceProcAddr(dev_0, "vkCreateCommandPool");
        ASSERT_NE(nullptr, create_command_pool_0);
        PFN_vkVoidFunction create_command_pool_1 = inst->vkGetDeviceProcAddr(dev_1, "vkCreateCommandPool");
        ASSERT_NE(nullptr, create_command_pool_1);
    }

It crashes with a heap-use-after-free when unwrapping the VkPhysicalDevice during the call to vkCreateDevice.

I can work on this unless you want to take a crack at it. If you don't/can't, I'll create an issue to track it since its a pretty serious bug (even if unlikely to be encountered in the wild).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants