The Fast Texture Loading Rewrite, attempt 2 - #1160
Conversation
| // Try to lazily create the task eagerly, if there's no preload EnsurePublicFields will load it anyway | ||
| // Don't call the event when there's an override | ||
| bool doLazyLoad = lazyOverride ?? | ||
| Everest.Events.VirtualTexture.OnShouldForceLazyLoad((VirtualTexture) (object) this) | ||
| || CoreModule.Settings.LazyLoading; |
There was a problem hiding this comment.
I decided to only call the event when necessary, but I'm not sure this is the right thing to do. I'd appreciate feedback on this.
| Logger.Error("vtex", $"Failed preloading PNG: Expected IHDR marker 0x52444849, got 0x{chunk.ToString("X8")} - {path}"); | ||
| return false; | ||
| // Waits for the texture to finish loading while ensuring no deadlocks occur due to stalling the main thread | ||
| private Texture2D SafeWaitForTextureUnlocked() { |
There was a problem hiding this comment.
Since nothing should ever call _textureTask.Value.Result but this we could make a wrapper class/struct that holds this logic hidden from the VirtualTexture land.
I decided not to do this to reduce boilerplate, but I'd like feedback on whether it was the right call or not.
|
|
||
| if (limit <= 0) { | ||
| // Everest.SystemMemoryMB reports the total memory in the system, so assume a tenth will be available | ||
| limit = (long) (Everest.SystemMemoryMB * 0.1f * 1024f * 1024f); |
There was a problem hiding this comment.
The diffs make it really hard to notice, but i changed a 0.2 to a 0.1f because the memory renting locking was also simplified and made less aggressive. Using 0.1f has no rationale behind it other than my specific machine throttling less. Feedback is not strictly needed but appreciated here. (And in how memory is guesstimated inside TextureLoader.ctor.
|
|
||
| namespace Monocle { | ||
| // We may have concurrent usage of this class due to FTL | ||
| [MakeAllMethodsSynchronized] |
There was a problem hiding this comment.
This was very frowned upon on the last review, but I decided to keep it since I still think it wont be a huge performance hit.
This just exists to ensure the vanilla list assets is used in a synchronous manner, since we now allow multiple threads to create textures. This means we could instead make il patches to just hold a lock when assets is used.
As the title suggests, this PR attempts to rewrite the entirety of the good old FTL, into a more maintainable and future proof codebase, splitting code into a new class TextureContentHelper to keep file sizes small.
The main intention of this PR is to make VirtualTexture fully thread safe, and re-enginneer an implementation of FTL.
For context FTL has simply the following goal: decode the texture data from disk to a CPU buffer asynchronously, then upload to GPU memory on the main thread. This speeds up loading times and fixes some crashes on Nvidia gpus. There's a writeup about most of the significant FTL details at the top of the VirtualTexture file diving deep into this and other topics.
Performance should be on par, the new implementation contains more overhead due to a different and much less aggressive scheduling policy, but at the same time it contains some improvements such as buffering outside of
SynchronizedZipEntryStreaminstead of inside, and copying the zip streams into aMemoryStreambefore doing the heavy loading in order to reduce lock contention across threads.This PR also introduces a bit of new api surface with two events, and some extra utility functions in
MainThreadHelper.Finally I apologize for the huge commit "FTL v2.1", this is the result of an iterative process of trying several designs and optimizations that eventually led to the current state. Because of this I failed to maintain a sane commit history that would be useful in any way for reviewing, and thus I ended up squashing it.
Finally-finally I'm debating on whether making a design document covering the details of why I did the things I did, and also move the text on top of
patch_VirtualTexturein there. Adding the questions that may come up during review to that document could be very fruitful for it too. Let me know if you'd like to see such a thing be made.Edit: I figured I'd sketch out a bit in here how the loading process works now.
Loading starts at
VirtualContent.CreateTexturewhich calls theVirtualTexturector. In there two things happen: aIPreLoaderis created (object in charge of loading the texture dimensions ahead of time, and creating anTextureLoaderlater on), and we initialize the_textureTaskwith aLazy<Task<Texture2D>>. The lazy part exists to model lazy loading and unloading in an easy and safe manner.During the initialization of the value in the lazy
_textureTaskwe jump toTextureContentHelperto queue a job in itsPipeline.This new implementation has two loading pipelines: one for synchronous loading and one for FTL. Both pipelines are identical, they only differ in the fact that the
FTLone has multiple asynchronous workers.The pipelines are modeled using channels and a few tasks that run once data is available, those consist of a head channel that buffers the work, a possibly asynchronous worker that does the cpu bound work, followed by another channel which lands us on a single worker that batches work for the
MainThreadScheduler, details are in theChannelJobManagerclass inTextureContentHelper.cs.Once a job is done its associated task will hold the result, which will be observed in the
VirtualTexturecodebase.Why should we use a pipeline and not simply queue all work as tasks? Because that causes all the tasks to become a work item in the
ThreadPool, so for SJ we end up with 25k queued work items in theThreadPool, this causes any further TPL work that is queued during FTL to get delayed arbitrarily. The new system only containsEnvironment.ProcessorCountworkers, so the total amount of work items thrown at theThreadPoolis really low, causing the TPL to continue being smooth.