Polygons from paragraphs: turning a 3D model into HTML and CSS

Introduction

Here's a post about something I did over a few days in November 2022. Please familiarise yourself with your browser's “Inspect Element” or “View Source” features if you want to see how the examples work!

Sometime circa 2009, CSS gained the ability to apply arbitrary transformation matrices to HTML elements. This made it possible to animate elements with translation, scaling, rotation and so on, independently of the normal CSS layout system, and with hardware (GPU) acceleration. Interestingly, they didn't limit this to simple 2D affine transformations, and so from this point onwards, the web gained the potential to enter the third dimension.

I'm pretty sure its intended use was for simple 3D transitions like those found in older iOS apps, where pressing a button would sometimes flip the entire screen around, as if it were a card. Here's Steve Jobs showing off such an effect in 2010:

That was probably accomplished with iOS's Core Animation framework, but evidently Apple wanted the web to have the same capabilities. Here, click on this playing card:

Anyway, while that must be how you're intended to use it, I've long wanted to use it to construct a full 3D game scene. In computer graphics, we build 3D scenes out of meshes, and a mesh is built out of polygons, usually triangles. Considering that we can create polygons (any HTML element, e.g. <div>, can work as a quad) and we can position them arbitrarily in 3D space (with CSS transforms), it should be possible to construct an arbitrarily complex 3D scene if we use enough HTML elements. As a simple example, we can construct the front side of a cube by using a rotated <div> for each face:

By adding a copy rotated 180° you can get a full cube, and then with the <img> tag or CSS background images you could texture the faces. If you can make enough cubes, you could make Minecraft! (A younger version of me attempted something like that.)

Most games aren't built with cubes, however. What if we could take arbitrary 3D models (meshes) from existing games and turn them into HTML and CSS? Well, it's surely possible, but nobody would have any good reason to do that. WebGL has existed for almost as long, and is a vastly more efficient, easy and featureful way to display 3D models in a web browser. You'd only want to do this if you have too much free time, enjoy whimsical projects, and were aware of some context where, for some reason, JavaScript (and hence WebGL) isn't available, but full HTML and CSS is…

…well, late last year, I'd recently left my job, and I was itching to write some open-source software again. Also, a new website called cohost, which is a bit like tumblr or Twitter, was becoming popular. Unusually for a modern site, it allows almost unrestricted use of inline CSS within posts, and has a very generous post size limit. Naturally, the site immediately became a hotspot for elaborate CSS shitposting. I wanted to join in, and thought back to my history of playing around with 3D CSS transforms…

Doing the thing

So, I started writing an application that would convert 3D models into HTML and CSS.

I decided early on that I wanted to consume models that use the .obj file format. It's a very old (circa 1992!) format, but it's very popular and easy to support. Here's an example of an .obj file for an untextured equilateral triangle:

v 0.0 0.0 0.0
v 0.5 0.8660254
v 1.0 0.0 0.0
f 3 2 1

As you can see, it's a simple text-based format, so I don't need to pull in a complex dependency to be able to parse it. There are many features of modern 3D model formats (skeletal animation, fancy materials…) that it doesn't support, but we have no hope of supporting those with CSS's limited capabilities, so if anything this is a feature. It keeps expectations in check.

Because this format is so old and simple, it's a popular target format for ripping models from old 3D games. For example, the website Models Resource has a lot of models in this format. For my shitposting goal, I wanted a model that had a very low polygon count (to fit within the cohost post size limit), but still complex enough to be visually interesting, and ideally it would be something iconic, so it would be recognisable and perhaps tap into nostalgia. I ended up choosing Lara Croft from Tomb Raider. Thank you to Cavan Ashton for ripping it!

So, to get things started, I wrote some simple Rust code that parsed the vertices of the model and turned them into a simple point cloud by creating many small <div>'s translated to the appropriate positions. It looked like this:

So far, so good. But this is very much the easy part. How do we turn these bare vertices into faces?

It's (not) hip to be square

As mentioned earlier, meshes in computer graphics consist of polygons, and usually triangles are the primitive of choice. Actually, if we're talking about the 21st century, and specifically meshes in the form used by game engines and submitted to the GPU for rasterisation, triangles are always the primitive of choice. Modern graphics APIs have taken the liberty of choosing for us. There's a lot of good reasons for this that you'll eventually discover if you play around in a 3D modelling application enough. In any case, if we want to consume arbitrary meshes, we'll need to have triangles.

Unfortunately, the web does not have triangles! Undoubtedly the HTML elements get broken down into triangles at some point on their journey to the GPU, but alas CSS does not let us skip to that point. So far as we care, every HTML element is a rectangle, or at least some kind of quad (quadrilateral). How do we solve this?

The first approach I thought of was to try to reconstruct quads from triangles. It sounds so nice on paper! While meshes are always in triangle form when sent to the GPU, they're often authored with quads because they make it easier to create clean topologies. There's also plenty of boxy objects out there that lend themselves to being made out of quads. And of course, the SEGA Saturn used quads rather than triangles, and it could run Tomb Raider, so surely this would work?

Alas, this approach doesn't really work. First of all, I didn't really want to write an algorithm for optimally combining triangles into quads. The killing blow however was that quads aren't precisely what we're dealing with. HTML elements start off as rectangles, and by applying appropriate transformation matrices, we can deform them into other kinds of quadrilateral… but not any quadrilateral. If we forget perspective projection for a moment, we only have affine transformations, and no matter how many affine transformations you apply to a rectangle, it won't stop being a parallelogram:

The problem here is that parallelograms are a small and very annoying subset of quadrilaterals. We can't do much useful with them. 3D models of non-boxy things (like Lara Croft) contain very few parallelograms, if any, and you can't construct other, more useful kinds of quadrilaterals out of parallelograms, at least not without making them overlap eachother. So it seemed like a non-starter. (As for why I ignored perspective projection: if we were only trying to manipulate a shape in 2D and did not care about the resulting z and w co-ordinates, that would provide a solution, but I assume it would blow up spectacularly once we try to apply further 3D transformations to the “2D” shape.)

Constructing triangles from rectangles

Okay, so can we make triangles from rectangles instead? Well, in pure geometric terms, probably not, but the mathematicians never considered alpha transparency. There's an old trick in CSS where you can get a triangle by creative use of the border properties:

Now, don't be deceived. This may look like a triangle, but it is secretly still a square, it's just half-invisible. No, really, the Emperor does have clothes; I swear that this will be important later.

Now that we have a square that looks like a right-angle triangle, all our problems should be gone. Whereas the affine transformations weren't enough for us to get any quad, they are more than enough to get us any triangle. It's pretty easy to do, too, thanks to how transformation matrices work. Say that our goal is the triangle with the points ABC, and we have a unit right-angle triangle with the points D = (0,0,0), E = (1,0,0), and F = (0,1,0). Notice that the point D is the origin, the vector DE is the x basis vector, and the vector DF is the y basis vector. So, if we perform a translation to A and change the x and y basis vectors to AB and AC, we're done. Conveniently, the columns in a transformation matrix are… the basis vectors and the origin! So:

let translation = a;
let x_basis_vector = b - a;
let y_basis_vector = c - a;

let matrix: [f32; 16] = [
    x_basis_vector[0], x_basis_vector[1], x_basis_vector[2], 0f32,
    y_basis_vector[0], y_basis_vector[1], y_basis_vector[2], 0f32,
    0f32, 0f32, 1f32, 0f32,
    translation[0], translation[1], translation[2], 1f32,
];

Combine that with making sure the original triangle is a unit right-angle triangle, by making the square be 1×1 pixels before transformation, and just like that, we can finally display Lara's silhouette:

Hmm, that doesn't look quite right! The broad strokes are definitely correct, but why are there semi-transparent triangles everywhere? I thought I used solid black… and why are there all these jagged edges sticking out everywhere, did I mess up the transform somehow?

Well, remember how I said that it is really still a square, and I made it be 1×1 pixels before transformation? I was assuming that the browser would extract the scale from the CSS transform and draw the triangle at whatever size the square was scaled to. But it seems that some browsers, at the very least Firefox and Safari, don't do what I'd hoped they would. They actually render a 1×1 pixel square to some kind of composition layer (i.e. a texture) and then scale it up afterwards. The 50% alpha transparency is not a mistake: I asked for a square that's half-opaque, half-transparent, and when you only have a single pixel to work with, a semi-transparent pixel is the best available approximation.

This is of course easy to work around: we change the initial size of the square (or rather rectangle, now) and normalise the basis vectors instead. With that fixed, we do get what we were expecting:

Great! But if you look very closely, you can see some blurry edges and spots of grey where there should be just black. That's really the same problem as before, just in a more subtle form. All our triangles are going to be slightly imperfect, because two edges are normal polygon edges, and one edge is an anti-aliased border from a texture. This is what you get for trying to draw triangles using rectangles. That and enough overdraw to make a GPU engineer cry.

One more problem. Once I'd fixed the single-pixel problem, I tried my code at that point in Firefox again, after having been testing in Safari for a while. To my horror, the result was incredibly glitchy! The same was true in Google Chrome, too. Were these browsers' 3D transform implementations far more broken than I thought?

Here was another lesson in how browsers implement these 3D transforms. The problem was in the transformation matrix. If you looked carefully, you may have noticed I didn't touch the z basis vector earlier. I didn't think I needed to: the untransformed rectangle would have no depth whatsoever on the z axis, so it shouldn't matter. And indeed that's true, it's a perfectly fine matrix. But Chrome and Firefox aren't expecting just any transformation matrix: they expect to be able to decompose it into a translation, scale, rotation, etc, and are getting bizarre results. Anyway, setting the z basis vector appropriately fixed the issue:

let z_basis_vector = x_basis_vector.cross_product(y_basis_vector);

With that fixed, harmony was restored, and the browser engines were once again in balance.

The finishing touches

All that remained was to add texture mapping and do some sizecoding (…am I allowed to use that word?) to fit Lara into the 200KB post size limit on cohost. There's not a lot to talk about there, but you can look at the commit history if you're curious. The texture mapping uses tiny PNGs embedded with data: URIs, though as an optimisation, I just use a solid background colour if all the texels covering a polygon have the same colour. Here's the final result:

And here's the chost (cohost post)! I also made a post with an item from Quake and the N64 logo.

Conclusion

Please do not seriously use this. It was great fun, though, and I hope you found it interesting! ^^

You can get the code here, if you want it. Oh, and check out my friend cassie's website, I talked to her a lot when I was working on the original project!