Optimization potential: Texture/model load-in

[Daskard]

Issue Description (Required):

After spending quite some time optimizing the ingame settings (playing in 2160p), I realized that the game takes its sweet time building up the frame rate after loading into the hub or even moving from one end to the other. Same in missions: perfectly stable 80-90 fps (capped at 90) and while progressing, suddenly the fps drop to 50-60ish. After a few seconds its back up at 80-90.

Your file user_settings.config (in Appdata/roaming - see attachment) does not change all the settings defined in your file win32_settings.ini (in gamedir/bundle/application_settings).
Does this mean that settings like [streaming_texture_pool_size = 512] are actually active settings in the game?

Attempted Solutions (Optional):

You already posted that you know about the utilization issue of your game.

My observation (if it’s any help):

1. My GPU is well-utilized in regards of wattage (320-340 with a few drops), but it stays quite cool compared to other games fully using its wattage. Maybe the task-distribution between the card’s segments is unoptimized.
2. Since the frame-drops happen only when moving to other areas, it looks like your texture/model load-in could use some major optimization. Not to mention the shopkeepers taking their sweet time loading in all the (fully rendered?!) gear in the shop menu. And when you scroll back up … THEY LOAD AGAIN, because the game forgot it already loaded them

If I shall provide any additional files or screenshots with performance overlays and settings, I will gladly upload them, if that helps getting the game somewhat optimized. The power draw for higher settings does in no way reflect the minor improvements in visuals they bring.

Edit: While playing, only 6,5/16 GB of VRam and 10/32 GB of system ram are used. There is no reason not to keep more textures in the memory.

PS: Personally I am quite fond of the ~35.000 lines saying that the camera is too near in your console.log

Platform (Required):

PC - Steam

[PC] PC Specifications (Optional):

GPU: Powercolor Liquid Devil RX6950 XT 16 GB
CPU: AMD Ryzen 7 5800X
MB: Gigabyte X570 Aorus Pro (Bios F35d)
32 GB system memory
PSU: beQuiet 1000W
Win 10 64bit

Upload Supporting Evidence (Optional):

user_settings.config (5.03 KB)

[PC] Upload Console Log & darktide_launcher.log (Optional):

darktide_launcher.log (378 KB)
console-2024-10-07-20.32.39-33b9f042-4a88-4fcc-bdaa-7fd6f05f7546.log (5.47 MB)

[grasmann]

@Daskard

Since the frame-drops happen only when moving to other areas, it looks like your texture/model load-in could use some major optimization. Not to mention the shopkeepers taking their sweet time loading in all the (fully rendered?!) gear in the shop menu. And when you scroll back up … THEY LOAD AGAIN, because the game forgot it already loaded them

There is a mod called dark cache that can cache different stuff and hold it in memory.
For example the item and cosmetic icons.
I don’t know why this is not implemented in the game, seems like a no-brainer really.
When the last big update came and I briefly used the cosmetics menus without any mods I was amazed actually how slow it loads the items and keeps loading them.

@Daskard

PS: Personally I am quite fond of the ~35.000 lines saying that the camera is too near in your console.log

Are you using any mods? Perspectives maybe?
If not then this is definitely odd.
I’ve seen these errors as well, but I figured it’s probably a mod issue.

[Daskard]

Are you using any mods? Perspectives maybe?

My game is completely vanilla with only in-Launcher and ingame settings changed.

There is a mod called dark cache that can cache different stuff and hold it in memory.
For example the item and cosmetic icons.

Ah, thank you, I’ll check that out.
Edit: Unfortunately the mod (which should be a dev task) is not working anymore
→ DarkCache at Warhammer 40,000: Darktide Nexus - Mods and community

[grasmann]

I’m using the mod dark cache right now.
Seems to work just fine.
Not sure if everything is working but icon caching definitely works.

[DashHandsome]

If you feel like experimenting, go into the settings_common.ini file in the same applications settings folder as win32_settings. Find mesh_streamer_settings and set disable = true. This will force the game to load all high res assets so loading screens will be longer and you’ll use a few more gb of vram. One of the devs mentioned adding this a year ago but the game still struggled to load assets in time. One side effect of this I’ve noticed is that it makes loading weapon/cosmetic icons VERY slow in menus.

You can also try messing with direct storage in the experimental build. One thing I noticed is that the first setup they have you try is with “disable_bypass_io = true” and “force_file_buffering = true” which the microsoft website for direct storage they link to in the same post says is only really good for HDDs.

I don’t know what the default is for the game currently or if it dynamically checks and sets it for you but if it’s this then that would also explain a lot.

[Axloss]

we check if the game is on a HDD volume and if that’s the case we set the recommended options for HDD and otherwise we use the recommended default options for SSDs.

The big difference is likely the missing file cache for SSDs.

If I recall default options for HDD are

direct_storage = {
    enabled = true
    disable_bypass_io = true
    force_file_buffering = true
    force_mapping_layer = false
}

and for SSDs

direct_storage = {
    enabled = true
    disable_bypass_io = false
    force_file_buffering = false
    force_mapping_layer = false
}

[Axloss]

Can’t answer regarding the texture streamer implementation, but the answer for

1. My GPU is well-utilized in regards of wattage (320-340 with a few drops), but it stays quite cool compared to other games fully using its wattage. Maybe the task-distribution between the card’s segments is unoptimized.

is likely that you are CPU bottlenecked. Which is common for VT and Darktide

[DashHandsome]

Oh that’s good to know. I’ll leave it as it is then. I did check one other option on the direct storage website and set DisableGpuDecompression to true out of curiosity. iirc direct storage 1.0 games didn’t have GPU decompression and still used the CPU so I thought it might help in GPU limited scenarios but it gave me some big hitches so I got rid of it. That’s totally on me though.

Also slight addendum to my above post: leaving the mesh shader off and playing for a bit has lead to things loading much smoother overall (save for a funny moment joining Clandestium Gloriana mid mission and only having teammates, crates, rubble, etc loaded in an empty world for 1 second). Have you thought about adding it in as a launcher setting? Maybe under an “experimental” settings tab with a popup warning about higher VRAM usage and it not being officially recommended or something.

It does make the cosmetic icons load much slower but the recent hitching from looking at cosmetic lists or mousing over a cosmetic is gone and clicking a cosmetic shows the high detail model immediately.

ps: As long as a dev is here, any update on the psykers rainbow shield glitch that happens on AMD GPUs sometimes? Any way I can help? I know there’s some AMD debugging software like Radeon Memory Visualizer to find memory leaks.

[Daskard]

@Axloss

is likely that you are CPU bottlenecked. Which is common for VT and Darktide

My CPU is relaxing at 30-60% usage. And 346 W is my GPU’s max wattage anyway

I was able to improve performance and stability by rolling back the driver from 24.9.1 to 24.7.1 though. On 24.9.1 the game kept crashing after 4-5 runs on borderless mode. In fullscreen it would crash DURING a run, which was very frustrating.
Now with Adrenaline 24.7.1 I had zero crashes.

The performance could still be better, but it improved A LOT by rolling back the drivers. Please reach out to AMD to fix this, since not everyone is willing to experiment with drivers.

[Daskard]

@DashHandsome

ps: As long as a dev is here, any update on the psykers rainbow shield glitch that happens on AMD GPUs sometimes?

Do you mean the shield looking like an oil stain, which you cannot really see through?
I had that, too.
How did I fix it?

1. I deleted the DXshader cache (a checkbox in the Windows drive cleaning tool)
2. I deleted some of the AMD shader caches: In the windows taskbar enter the word run → open the tool → Enter this: %localappdata%\AMD
3. I regenerated the game’s shader cache (which happened, after I ran darktide_shader_cache_builder.exe from the game’s subdirectory shader_cache)

Then the issue was gone.

[DashHandsome]

Weird, I’ve tried full game reinstalls with deleting all game folders and wiping my drivers with the AMD cleanup utility and it eventually pops back up again. Guess I’ll try that, though it’s not as extreme as my previous fix attempts.

Also I agree with your above point about not wanting to experiment with drivers but 24.9.1 is performing great for me. You might want to check out this comment I made in another thread about how I fixed my crashes

[Axloss]

Also slight addendum to my above post: leaving the mesh shader off and playing for a bit has lead to things loading much smoother overall

If you mean to render the whole actual cosmetics? Then the issue is that you are waiting on them to load i guess. It will be faster when not loading stuff. Why it’s not faster than this I don’t know. Note that I’m not aware of all features of the game, I mostly work on the engine integration and have some insight in the direct storage implementation since I’ve been helping out with issues related to it.

The AMD isn’t something I’m aware of. But test if the shader cache is related might be a good idea.

[DashHandsome]

No worries, I know turning off the mesh shader isn’t technically recommended and I accept the drawbacks that comes with it.

I tested the fix @Daskard gave me and it didn’t work so I’ll probably make a thread about it in tech support in a few days for it to be passed on to the relevant people. Restarting fixes it briefly even when I don’t wipe the shader cache but it tends to happen again after a few missions which is why I suspect a memory leak.

I want to get a couple more clips of it first though because sometimes the first time the glitch happens in a mission it only covers part of the psyker shield and seems to follow the pattern of the wavy effect on it. The next mission it’s then fully covering the shield. The partial effect doesn’t happen often though so I don’t have a clip of it yet.

[Daskard]

@DashHandsome

I tested the fix @Daskard gave me and it didn’t work

Sorry to hear that.
I checked your suggestions from your other post regarding BIOS settings etc. - you did quite a lot.
Since it’s stable now, I’ll refrain from testing those right away, but when I NEED to update my driver, I will get back to it. “Never change a running system” ‘n’ stuff
Thank you in advance, though

[DashHandsome]

Yeah, thats where I’m at with mine right now. I don’t want to start changing stuff again to figure out what exactly fixed it when I finally found something that works.

[DashHandsome]

I can’t believe this but I think I figured out the issue with the shield. I turned off anti-lag for darktide in my AMD driver And it hasn’t happened for a day so far.

Also, my crashes are gone after messing with some windows settings and doing a driver wipe. I’ll be making a post about it soon.

[TeaPiss]

question
If you are on a full AMD setup.
Why is Nvidias AI agent constantly triggering in the log?
This isnt a repeat of their last AMD botch job is it?

Also do you have ray tracing on because the config is showing reflections set to high and GI set to medium if not it would be nice to see this actually set to an off state when we have it set to off ingame.

[Vizra]

I could frivolously make out with you right now.

I disabled mesh streaming and boom… my game no longer stutters. I have a 7900xtx so the extra VRAM usage isn’t going to be an issue. ALSO MY GAME LOOKS SO CRISP AND SHARP OMG.

I have also disabled direct storage and AMD Smart Access Memory and my game isn’t dropping below 140fps.

Thank you for solving my months of pain and suffering.

[DashHandsome]

Heck yea, love to hear it. Just remember that it’ll be reverted during verification when the game is updated.

If you want to try another touch-up you can also try swapping out the games Fidelityfx file with the latest version from AMD’s GPUOpen Github to maybe make FSR and frame gen look a bit better? Just replace the amd_fidelityfx_dx12.dll in the binaries folder in the main game folder with the one from the FidelityFX-SDK-v1.1.2.zip found in ffx-api\bin. Remember to get the right one for Darktide, not the .lib file or either of the vulkan files.

Darktide still uses the original FSR 3.1 while 3.1.2 released a few weeks ago with “bug fixes” and a memory leak fix but luckily AMD made it so that from this release and onward all it takes to update it for devs (or us) is to swap out the .dll file with the newest one. It should just work but you can save the old file just in case or verify through steam to change it back (and yes, game updates also revert this).

I know people recommend to use XeSS instead but Intel changed the resolution scaling of XeSS 1.3 to upscale from an even lower resolution. Yes there’s less shimmering but the picture is blurrier and doesn’t perform quite as well as it does on other GPUs as it does on Intel’s due to how Intel coded it.

You might think that’s not relevant here but the XeSS 1.3 .dll snuck in to the game files with the Unlocked and Loaded update (extra presets not included).

Hope this helps it look even better!

[Vizra]

I used to use XeSS 1.3 back when FSR was worse. But honestly the implementation of FSR2 was better than what we have now for FSR3. Deffinelt going to update that dll and give it a go