Early June Devlog

tldr: Been working hard on the engine development. A small playable part of the Artoria standalone chapter will be coming this month if not more.

Just realised I didn’t share anything in May. I’ve been very deep into the engine & movement system improvements I discussed in the last update in April. I’ve been spending nearly every chance I get after work on this project. But I’ve been feeling like I’ve been deep in a limbo/stuck in development hell so this update is as much for me to keep track of everything I’ve done this past month or so, as it is for anyone interested in following the engine dev progress. I’d be surprised if anybody reads all of this! Lots of technical details below.

Firstly, I spent the final part of April to just slow down & focus on improving my math knowledge to be able to implement the movement system. AI has been a huge help too. It is turning out super good. The IK system is perfect now in both VR & non-VR which is a huge relief. In the current movement mode where you control the head that you’ve all played, now, the body stretches and collapses much better with much smoother movement too. Should be less frustrating for players. Not just that, the avatar can be rotated and positioned every which way - upside down, side ways. It opens up possibilities for many fun scenarios. It was big challenge, took me over a week just for this and this being perfect makes the rest of the movement system work much easier. This is more than half of the work for the locomotion movement mode too. I just need to finish up on the animation/locked movement mode which I would’ve done by now but…

I got side tracked by a lot of other engine improvements for the rest of May. They’re all worth it and serve the future of this project. But all the engine improvements I was planning to make far in the future, I’ve already done most of them. It all started with the Artoria model being too heavy and constantly causing crashes on iOS. And not having a smooth frame rate even on my powerful PC. We’re severely limited due to WebGL being a very antiquated graphics API.

Due to this I ended up working on a lot of different things to reduce both RAM & CPU usage. Mainly:

• Overhauling Dynamic Reflections. I’ve implemented a Dual Paraboloid Cameras based relfection system instead of a Cube Camera system. This results in huge performance gains, especially for VR, while also improving visual quality in a lot of situations.

  • Cube Cameras although the modern option with most graphics library including ours are not properly optimized for WebGL. The 6x cube faces even when rendering an empty scene cost 4ms. If I want to support VR, I cannot yet use WebGPU which optimizes this. Our whole frame budget if we want to run at 120fps is 8.33ms, that’s half gone for nothing. This was kinda fine when our models were light but that’s not the case with this upcoming chapter.

  • DP cameras only use 2x cameras. They have a different set of trade offs such as needing a larger amount of triangles and technically they’re not perfect like Cube Cameras. But I’ve actually gotten them to level where for our particular use case, the benefits even the fidelity is higher than Cube Cameras.

  • I’ll also use them for shadows moving forward for things like chandeliers which will be very nice as that was something I couldn’t do.

• Way better 3d model optimization for memory. Our models were already optimized but only the textures were VRAM optimized and CPU rendering was . The geometry optimization as its tricky to get right, I’d skipped over. Now, the geometry & UV mappings for our domme’s models is quantized without sacrificing visible quality as much as possible which saves 30mb+ RAM & VRAM each.

  • And stages/worlds are quantized even more. Good RAM saving.

• Learnt deeply about performance measurements in browsers especially for memory.

• Learnt how to use the WebAudio API to do reverb optimally. My original implementation was horrible. The cost of per vocal audio in terms of memory has now gone down from 10-20mb per file to 1-2mb per file with a fixed 20mb overhead for the reverb effect.

• I’ve perfected the vocal loading system. Previously, the preloading of upcoming vocal files was hacky, it wasn’t aware of what direction the story will proceed in so just preloaded 5 upcoming files and still failed sometimes during choices (so would have to wait a few seconds before playing audio). Now, so the audio is always ready just before its needed. And we network preload/disk cache all the audio files sequentially to have the game keep on working even if your internet dies. But only decode in RAM the ones that’ll be needed in the next 1-2 lines only.

• I’m in the process of doing the same for animations.

  • Currently, all animations are parsed and decoded into memory/RAM on page load. This is one of the reasons for iOS crashes and also causes the initial 2-3 seconds of stutter when the game initially loads. But it ensures that there are no CPU spikes/main thread freezes later in the game. But this approach cannot scale with the bigger stories I wan to tell.

  • So similar to vocal/speech audio, I’m working on network preloading/disk caching all the animations. And in another thread/worker, decode the animations that are about to be needed by the story to avoid freezing the main thread. And then seamlessly transfer the decoded animation to the main thread and parse them there. Running into some challenges here, nothing we can’t solve though.

I’ve also improved many other things:

• Multiple character support to the story system.

  • Previously, it was only suitable for 1 character/domme per chapter as I was rushing to just release something. Now, its supports many - each with their own animations, vocal lines. And the ability to look at each other as they talk, etc, etc.

  • This was a huge piece of work. Strictly speaking, I could’ve avoided but it needed to be done at some point. And its done now.

  • A chapter with multiple ladies will be coming very soon! May be even in the standalone chapter.

• TTS overhaul. Dialogue generation is much more emotionally consistent now and gives me way more control over expressions and animation timing.

  • Previously, each line was generated as a separate “input group” in a single request. Now for all lines that are together, I make a single request and then break the lines down locally.

  • This ensures emotional consistency for longer sections. And also allows me to granularly customize facial/eye expressions & body movements per word if I choose to.

• Much better lip sync. Using an AI model in tandem with the speech generation to accurate pre-generate realistic mouth movements. 2 AI models in fact and I apply whichever one looks better.

  • This was needed due to Artoria’s model having ARKit expressions which are meant for realistic human facial expression and my old approach looked really bad.

  • There “lipsync” files are also loaded/decoded in an optimal manner to keep memory usage low.

• Improved developer experience for myself.

  • All of the above speech and lipsync stuff is automated. I just need to save my story file and the speech audio and lipsync is automatically generated. Makes it much easier to iterate and experiment. My story system also automatically goes to the latest generated or edited line and replays that. Just much nicer way for me to write & iterate over the story without waiting on things.

  • Overhauled a lot of the code so everything hot reloads as much as possible to avoid full page refreshes. Pretty much nothing fully refreshes the app anymore.

• A better player to domme/NPC look at implementation for non-VR for the view mode during story choices. Now, no matter how the player is rotated, standing, upside down, crawling or whatever, it always perfectly tries to keep domme framed to the screen perfectly.

  • Previously, this was mostly a hacky approach and didn’t work perfectly in many cases. But I got sick of playing around with random values to get this working nicely and this was needed for how the animations I’m planning with Artoria.

  • Camera shake was updated to work properly with this too

• Improving the sky/star system. The stars look much better in VR. This was mainly needed for the DP reflection as the previous implementation was flawed for those. But I noticed the stars were changing size based on device resolution. Now, they’re way more consistent. And flicker less in VR.

• Using a more optimized build for the physics engine, should help lower CPU usage.

• Ability to lower render scale/resolution and actually using a lower starting render scale for retina devices. This should help perf on very low end devices.

• Disabled the fancy UI effects by default on slower devices like mobile phones. Not sure if this helps, but by default on iPhones and Android, the fancy UI effects and animations will be disabled to keep memory usage low.

• Post Processing for non-VR: Anti-aliasing, Depth of field, Bloom & more.

  • This does not and can not work in VR until WebGPU+WebXR binding comes out. And then this API becoming stable is probably another 1-2 years away…

  • But I was sick of waiting for implement this. So, now we have beautiful post processing effect for non-VR players! The bloom and depth of field is really nice. When any domme is in view, anything that’s not near her is blurred out.

  • Quite expensive, off by default for now. But option will be available in settings.

There’s probably a bunch more stuff I’ve forgotten too. Looking back it is kinda ridiculous how much engine work got done over the last month and a week or so. I’m honestly running out of tasks on my backlog. But its time to come back to actually making the next chapter.

Just 2-3 final things left before I can jump back into the art, modeling and story writing:

• Finishing up on the movement system. My main focus for now. The player animation mode needs a lot of testing in VR, and there are many variations of this that are needed. And then a locomotion mode.

• Finishing up that animation optimisation.

And after that?

Using all this new tech, I’m planning to release an early playable build of some part of the Artoria standalone chapter this month. I have a longer story in mind so it will likely some time before the full chapter is ready but I think I can deliver it in small playable bites that can later be enjoyed as a single experience.

I’d like to finally say that despite all these efforts and learning, sadly, I’ve come to realize iOS Safari really caps the amount of RAM a page has access to randomly. Sometimes crazy low amounts 200mb JS memory usage can cause the page to crash. This makes it unlikely that we’ll be able to keep the future biggers chapters stable on it that feature multiple dommes or locations. Not saying we’re dropping support, they’ll work but can crash. I’ll still try to release smaller chapters from time to time that should work fine.

Thank you for reading if you made it this far!