How I Built the First 3D Racing Game for ESP32-S3 (Because Someone Said AI Could Do It Better)

6 min read Original article ↗

David Montero Crespo

Or: What happens when you challenge AI to understand 3D perspective, physics, and good taste

It Started With a Reddit Argument

Someone on Reddit claimed that AI now codes better than any human developer ,citing the latest models like Opus 4.6 and GPT-5.3 Codex. I disagreed.

Don’t get me wrong: AI is great at churning out social media demos, landing pages, and CRUD apps. But when you venture into complex calculations, low-level languages like C++, and spatial reasoning? It still falls flat.

To prove my point, I needed a challenge that barely exists on the internet: a pseudo-3D/3D racing game running on an ESP32-S3 with just 8MB of PSRAM.

Spoiler: I succeeded. And the AI models? They all failed.

“That’s Impossible”

The first thing ChatGPT and Claude told me was that this project was impossible due to hardware limitations and the lack of a traditional Z-buffer (the classic 3D rendering technique for depth sorting).

But here’s where human ingenuity comes in: you don’t need a Z-buffer if you combine real 3D objects with pseudo-3D techniques inspired by classic games like OutRun.

The result? A game that looks and feels fully 3D , complete with a textured 3D car, buildings, tunnels, day/night cycles, drift physics, and collisions ,all running at 240 MHz on a tiny microcontroller.

The Foundation: Pseudo-3D Magic

I built the core engine based on this excellent article about pseudo-3D racing games:

👉 https://codeincomplete.com/articles/javascript-racer

The technique is simple but powerful:

1. Divide the track into segments (like slices of a loaf of bread)

2. Project each segment onto the screen using perspective math

3. Render back-to-front (painter’s algorithm) so distant objects appear behind closer ones

4. Add curves, hills, and tunnels by tweaking segment positions and Y-offsets

Here’s the core projection code (`render_road.cpp`):

void projectSegment(int i, RenderPt &rp) {
 Segment &s = segments[i];
 float camDist = (i * SEG_LEN) - playerZ;
 float scale = camDist / (CAM_H - s.y);
  rp.scale = scale;
 rp.x = SCREEN_W2 + (scale * (-playerX) * SCREEN_W);
 rp.y = SCREEN_H2 - (scale * CAM_H * SCREEN_H);
 rp.w = scale * ROAD_W * SCREEN_W;
}

This transforms 3D world coordinates into 2D screen space , the same math used in games like Out Run, Pole Position, and Lotus Turbo Challenge.

Adding Real 3D: The Player Car

Here’s where things get spicy. I didn’t want a flat sprite ,I wanted a real 3D car model.

So I loaded a .obj mesh with 428 vertices and 312 triangles, applied a 128×128 RGB565 texture, and rendered it using scanline affine texture mapping + painter’s algorithm (sorting triangles by Z-depth).

Here’s the rasterizer loop (`render_player.cpp`):

for (int y = yMin; y <= yMax; y++) {
   if (y < 0 || y >= SCREEN_H) continue;
      float t = (y - v0.sy) / dy;
   int xLeft = v0.sx + t * (v1.sx - v0.sx);
   int xRight = v0.sx + t * (v2.sx - v0.sx);
      for (int x = xLeft; x <= xRight; x++) {
   float u = interpolateU(x, xLeft, xRight);
   float v = interpolateV(y, yMin, yMax);
   uint16_t color = sampleTexture(u, v);
   spr.drawPixel(x, y, color);
   }
}

This draws the car pixel-by-pixel with perspective-correct texture mapping all in software on a microcontroller.

Physics: Drift, Friction, and Gravity

A racing game isn’t just graphics ,it’s feel. I hand-tuned every constant in config.h:

#define SPEED_MULTIPLIER 65.0f // Max speed ≈ 246 km/h
#define FRICTION 0.996f // Coast-down in ~3.3s
#define CENTRIFUGAL 0.25f // Curve drift force
#define GRAVITY_FACTOR 0.15f // Hill acceleration effectrSpeed *= pow(FRICTION, dt * 60.0f);

The physics loop (physics.cpp) handles:

- Acceleration & braking with smooth easing

- Centrifugal drift when cornering

- Gravity assist/resistance on hills

- Collision detection with walls, traffic, and trackside objects

Here’s the drift calculation:

void updatePhysics(float dt) {
 Segment &seg = segments[segIdx];
  // Centrifugal force pushes car outward in curves
 playerX += (seg.curve * playerSpeed * dt * CENTRIFUGAL);
  // Gravity accelerates downhill, decelerates uphill
 float slope = seg.y - segments[(segIdx - 1 + TOTAL_SEGS) % TOTAL_SEGS].y;
 playerSpeed += slope * GRAVITY_FACTOR * dt;
  // Friction slows you down over time
 playerSpeed *= pow(FRICTION, dt * 60.0f);
}

Tuning these values required hours of Excel spreadsheets, Paint sketches, and trial-and-error ,something AI simply cannot do with taste or intuition.

The Development Trick: Build for PC, Deploy to Hardware

Compiling to ESP32 every time would’ve been a nightmare. So I created a Windows emulator using Raylib that lets me run the exact same C++ code on PC.

The magic? A wrapper (`emulator/TFT_eSPI.cpp`) that translates Arduino graphics calls into Raylib:

void TFT_eSprite::fillTriangle(int x0, int y0, int x1, int y1, int x2, int y2, uint16_t color) {
  Color c = rgb565ToRaylib(color);
  DrawTriangle((Vector2){x0, y0}, (Vector2){x1, y1}, (Vector2){x2, y2}, c);
}

Where AI Failed (Spectacularly)

I tested Gemini, ChatGPT, and Claude by asking them to build this game. Here’s what happened:

❌ ChatGPT

- Generated code that didn’t compile

- Perspective calculations were wildly off

- Suggested using a Z-buffer (impossible on ESP32 with this resolution)

❌ Claude

- Better at structure, but broke the rendering pipeline

- Couldn’t grasp the back-to-front drawing order

- Introduced subtle bugs in floating-point math

❌ Gemini 3 Pro

- Actually got closest to correct perspective math

- Still failed at artistic tuning (colors looked awful, physics felt floaty)

- Couldn’t debug its own rendering artifacts

The problem? AI doesn’t understand:

- Spatial reasoning( what’s “behind” vs. “in front”)

- Numerical precision (floating-point drift, rounding errors)

- Artistic taste (does this *feel* good to play?)

The Results

Here’s what I achieved:

Real-time 3D rendering at ~30 FPS on ESP32-S3

Textured 3D car model with 428 vertices

Procedural track generation with curves, hills, tunnels, and buildings

Day/sunset/night cycle with dynamic fog and color palettes

Physics simulation with drift, friction, and gravity

Collision detection and lap timing

Windows emulator for fast iteration

The Code:

GitHub https://github.com/yourusername/esp32-racing-game

Did I Use AI?

Yes , but not how you think.

I used ”vibe coding”tools to scaffold boilerplate, generate headers from OBJ files, and refactor repetitive code.

But every critical calculation was done by hand:

- Drawing perspective grids in Paint

- Building physics models in Excel

- Tuning constants through trial-and-error

- Debugging rendering artifacts pixel-by-pixel

AI was a tool, not the architect.

The Takeaway

AI is incredible at generating code that looks right. But when you need code that works right , especially in domains requiring spatial reasoning, numerical precision, or artistic judgment ,human expertise is still irreplaceable.

Could AI eventually do this? Maybe. But today, if you ask an LLM to build this blindly, you’ll end up with floating rectangles and terrible C++.

The challenge remains open: Can your AI calculate perspective, physics, and good taste?

I doubt it. But I’d love to be proven wrong.

Technical Details

- Hardware:ESP32-S3 @ 240 MHz, 8MB PSRAM, ILI9341 320×240 SPI display

- Language: C++ (Arduino framework)

- Emulator:Windows (Raylib + custom TFT_eSPI wrapper)

- Rendering: Painter’s algorithm + scanline texture mapping

- Assets: OBJ mesh (428 verts), RGB565 texture (128×128)

Build It Yourself

Want to try it? Here’s how:

Emulator (Windows):

cd emulator/

make

./car_game_emu.exe

Hardware (ESP32-S3):

1. Open `car_game.ino` in Arduino IDE

2. Select ESP32-S3 board with 8MB PSRAM

3. Upload to your device

4. Connect ILI9341 display via SPI

What do you think? Could AI build this today? Let me know in the comments.