Home / Forums / Clean WorldToScreen calculation in modern C++ (Row vs Column Major)

UnreliableCode Community

Developer Research, Reverse Engineering & Coding Community

Tutorial

Clean WorldToScreen calculation in modern C++ (Row vs Column Major)

MatrixRecon
3D Math & Vectors
MEMBER
Rep: 225
Join Date: Jan 2025
Posts: 33
Thanks: 55
2y ago · Mar 5, 2024 11:20 AM
#1
A common stumbling block for beginners when writing their first ESP overlay is getting the View Matrix calculation wrong due to Row-Major vs Column-Major matrix layout differences.

The Mathematical Formula:
Given a 4x4 View-Projection Matrix
CPP
M
and a 3D World position
CPP
v
:

CPP
struct Vector3 { float x, y, z; };
struct Vector2 { float x, y; };

bool WorldToScreen(const Vector3& worldPos, Vector2& screenPos, float viewMatrix[16], int screenWidth, int screenHeight) {
    // Matrix transformation (Row-major indexing)
    float clipW = worldPos.x * viewMatrix[3]  + worldPos.y * viewMatrix[7]  + worldPos.z * viewMatrix[11] + viewMatrix[15];
    
    // Behind camera check
    if (clipW < 0.001f) return false;
    
    float clipX = worldPos.x * viewMatrix[0]  + worldPos.y * viewMatrix[4]  + worldPos.z * viewMatrix[8]  + viewMatrix[12];
    float clipY = worldPos.x * viewMatrix[1]  + worldPos.y * viewMatrix[5]  + worldPos.z * viewMatrix[9]  + viewMatrix[13];
    
    // Normalized Device Coordinates (NDC) [-1, 1]
    float ndcX = clipX / clipW;
    float ndcY = clipY / clipW;
    
    // Map NDC to screen pixel coordinates
    screenPos.x = (screenWidth / 2.0f) * (1.0f + ndcX);
    screenPos.y = (screenHeight / 2.0f) * (1.0f - ndcY); // Invert Y for screen space
    
    return true;
}


Pro-tip: If your ESP is rendering inverted (left is right, or top is bottom), simply check whether the game's matrix is column-major
CPP
viewMatrix[0][i]
vs row-major
CPP
viewMatrix[i][0]
.
MatrixRecon · 3D Mathematics & View Matrix Calculations
The following users thanked MatrixRecon for this post:
VectorByte
Graphics & DirectX Dev
VIP
Rep: 420
Join Date: Aug 2022
Posts: 39
Thanks: 115
2y ago · Mar 5, 2024 2:15 PM
#2
Pinning this in my notes! In DirectX 11/12, Y goes down in screen coordinates (0,0 is top-left), which is why
CPP
(1.0f - ndcY)
is crucial. In OpenGL screen space, (0,0) is bottom-left so you wouldn't invert Y.
VectorByte | DirectX 11/12 Hooking & ImGui Overlays
Quote
Math is the language of game engines.
KronoDev
C++ / Game Modder
MEMBER
Rep: 195
Join Date: Jan 2023
Posts: 32
Thanks: 48
2y ago · Mar 6, 2024 8:50 AM
#3
Thanks for the writeup @MatrixRecon! The explanation of
CPP
clipW < 0.001f
preventing mirror artifacts behind the player camera is super clear.
KronoDev - Keep coding, keep learning