Can't help but make things complicated XD

So, while I've been doing some testing and experimentation with new and better(?) art workflows I've been slowly putting some of the results together in a little mini-game.

It's simple, it's nothing complicated, this is mostly an exercise for myself but if I can make something shareable out of my experiments so much the better :) It's basically going to be a hidden object game in which you have a kind of magical magnifying glass which can see through some materials.

And that's simple: a world-space SDF drives a mask on each sprite to hide or reveal areas. Done. No problem. No need for anything complicated.

Buuuut wouldn't it look nicer and make sense to have a bit of distortion through the lens?

So, we use the same SDF to determine where distortion should occur, offset & scale the UVs of the sprite in that area, it's working immediately on the background but the clothing is acting super weird.

Oooh, right, the background image uses the full 0-1 UV range of its source texture because it's just one single image, but sprites in Unity use smaller sub-ranges and this data isn't automatically passed to shaders. So, new C# script just to extract the sprite UVs on startup and pass them to the material. Now the distortion is working correctly, but hilariously only within the bounds of the original sprite, so the silhouette remains unchanged but the fabric is zooming and distorting within the silhouette!

In hindsight this obvious: by default Unity builds a mesh to match the sprite silhouette and the shader doesn't run on any pixels falling outside of that mesh. Normally, this is exactly what you want, it's a waste to run your shader on pixels which you expect to just be invisible anyway. You can disable this behavior and tell Unity to use the full rect covering the sprite, but this just causes the distorted sprite to get very obviously cut off at the rect bounds which is also awfully ugly.

You could extend the sprite rect to include some padding around the actual image data so the distortion can't push any pixels beyond the edge, but now you're wasting a ton a memory on literally empty pixels for every object in the scene.

So this is getting more complicated and expensive than just doing the distortion as a post-process effect, let's just do that. We have to do a little bit of work to ensure that the post-process shader gets the correct parameters to match the size and location of the distortion with the lens as we move it around, but that's not much.

What is particularly annoying, though, is how much of a pain it is to use an HLSL shader as a post-process effect in URP. It can be done, but rewriting the distortion effect in a shader graph is less work:

Ok! So now we have a lens you can move around, its position gets sent to the materials used on clothing and other objects to determine what's visible as well as to the full-screen distortion material to determine where to apply distortion. It's a little inconvenient to have things split like this, but it's not bad. It looks pretty good in the end.

Oops, it's distorting the magnifying glass, itself. That's kind of awkward. So, we add a second camera just for rendering the magnifying glass sprite after the first camera has applied the distortion effect. Ok! Ok, nice and clean now.

But this is still supposed to be a game, right? It's not just about looking around, you need to be able to interact with things. And what happens if you try to click on something revealed by the magnifying glass which has been distorted? It's not gonna work... the visuals don't match the collision geometry.

So, let's offset the collider to match the visual location. Basically, we just need calculate the distortion for the collider's location and move it there. There's a little bit of juggling coordinate systems to ensure that when we do this we are actually distorting the collider's position in the same coordinate space as the pixels in the full-screen shader, the collider needs to be on a separate GameObject from the sprite itself (or the visuals will get double-distorted), and of course it needs to be written in C#. Once that's all taken care of, though, we have colliders which actually match the on-screen location of a distorted sprite:

One small decision (the magnifying glass should apply some distortion/magnification to objects behind it) led to a cascade of small problems. I've now written the distortion code 3 times in 3 different languages, and while none of the individual problems were a big deal it's fair to say that this feature as a whole was a bit of a headache. This is not unusual at all, which is why developers often recite mantras like KISS -- Keep It Simple, Stupid! And indeed, it's really easy to argue against going down this road at all.

Does the game require this (purely visual) feature? Absolutely not, no real gameplay feature will actually depend on it, and maybe it wouldn't even be something people complained was missing. But I do think in this instance it's worthwhile despite the headache, and for a feature like this it's also worth doing this work early on. It's often tempting (and even logical) to put off features like this until the end--get the stuff the game actually needs done first and then if there's time work on some of the nice visual features once everything else is in place.

This little distortion feature has introduced new constraints and requirements which affect the whole project, though. We now have multiple cameras rendering different parts of the scene, interactable objects must follow a specific hierarchy with collisions and visuals on separate objects, we have shared state multiple shaders need access to, and a number of small little scripts which must be attached to certain objects in the scene or else something will break.

Discovering these requirements at the end of the project after everything was nearly finished and suddenly needing to rebuild a bunch of stuff to work with it is a lot more work than the slightly-larger-than-expected headache to get this up and running now, and with the constraints in mind going forward it's not much extra overhead to just ensure everything built from this point on will work.

So, maybe this has delayed the project a bit but I think the end result will be better for it, and I won't have to wrestle with the dilemma of whether or not to release an early version without this feature while I rebuild all the necessary assets to publish an update with it working.

And on that note: I think I should have a playable prototype of this ready fur subscribers soon! The first build will probably just contain one level, with a later release being more complete.

'til then,

<3

-M