Hacking Simple DX9 Meshes with WinDBG

I am now able to set the color of an entire mesh to solid red thanks to windbg and knowledge of some dx9 structures. I am however starting to think there's better ways of doing this. First off, the target example only loads a single mesh, if it were to load multiple meshes, I would end up setting them all to red by using this technique. But for the sake of completeness this post will show how I figured out at least one way of doing it. I really think I'm going to have to start writing at least some basic injection code so I can be more flexible in how I am modifying code or data at runtime. I'm just not sure I want to start down that slippery slope just yet.

My target mesh/texture

My target this time was the DX9 SDK example code 'Meshes' which can be found in the following directory <sdk install dir>\Samples\C++\Direct3D\Tutorials\Tut06_Meshes. This example loads the mesh information from an '.x' file via the D3DXLoadMeshFromX function. These 'X'-files (teehee!) contain all the necessary geometry and texture information required to properly render it to the screen. For more information on materials check out this msdn link. Oh and here is the 'X' file, if you're curious:



After it loads the above file, the function creates a pointer to a D3DXMATERIAL structure by calling pD3DXMtrlBuffer's GetBufferPointer method. It uses this variable to extract the material and texture information. This is of obvious interest to us as it may contain default RGB values. Since a mesh can contain a variable number of materials, it loops over the number of materials and assigns the global material variable the extracted information. Here's the code for reference:

The variables that contain our mesh material information

At first I had no idea what was going on in the above code until I noticed the 'Diffuse' and 'Ambient' members of the g_pMeshMaterials structure. Since I learned all that stuff regarding texture maps, I know now that Diffuse basically means 'color stuff I can muck with'.  So as a test I added a number of lines (passed null to the SetTexture call) and re-ran the application:

Added lines (in red box) with the result.

So that's what I want, except I'm not directly injecting code yet, so I can't just stuff assembly into that part of the code. I need to learn what the structures are and do so I can modify the values prior to assignment. Here are the two structures that I'm concerned with; D3DMATERIAL9 and D3DCOLORVALUE:

typedef struct _D3DMATERIAL9 {
    D3DCOLORVALUE   Diffuse;        /* Diffuse color RGBA */
    D3DCOLORVALUE   Ambient;        /* Ambient color RGB */
    D3DCOLORVALUE   Specular;       /* Specular 'shininess' */
    D3DCOLORVALUE   Emissive;       /* Emissive color RGB */
    float           Power;          /* Sharpness if specular highlight */
} D3DMATERIAL9;

typedef struct _D3DCOLORVALUE {
    float r;
    float g;
    float b;
    float a;
} D3DCOLORVALUE;

On my machine a float is 4 bytes. So to get the red color I should only need to modify the first D3DCOLORVALUE struct (Diffuse) to 1.0f for R, and 0.0f for all other colors. I deleted my modifications to the example and recompiled it in Release mode and took a quick peak with IDA Pro.

Setting material struct in IDA.

I also found the SetTexture call in IDA using previously described methods so I can set a break point on that as well. Finally, I loaded up windbg and set breakpoints where necessary. To get access to the structures I'll need to look at the return value of GetBufferPointer(), this is stored in the eax register. That value is actually the d3dxMaterials struct which members I'll need to modify. Here's the contents of the address of eax, which you can see is the D3DMATERIAL9 struct. It had some default values already set which I needed to change.

D3DMATERIAL9's Diffuse struct

I should only have to change these four floats to 1.0f, 0.0f, 0.0f, 0.0f and my texture should be red after I remove it from the SetTexture call. You may recall from my last post that 1.0f in memory is represented as 0x3f800000, so here's my red RGB values in windbg (also note I set green, blue and alpha to 0x00000000).

The new Diffuse struct values.

Next up came null'ing out the material variable in the call to SetTexture. I actually screwed up here and set the wrong address to null, which caused the application to crash. I set what was pointed to edx to null, but didn't notice the instruction below it showing that edx would be used as a look up in conjunction with esi (mov edx,dword ptr [edx+esi*4]). Ah well, no problem, I restarted and tried again this time setting the value at [edx+esi*4] to null.

[edx+esi*4] set to null

After that, I hit continue, which happened three times before I realized I was in a loop, I cleared my breakpoints (which you can see in the command window in the below screen shot) and it rendered my red tiger to the screen.

Mesh set to red via modifying values with windbg.

It always feels good when you accomplish something, however I'm starting to feel really held back by doing everything in a debugger. I've known for a while that I will need to seriously look at code injection and hooking techniques, I just didn't think it would be so soon. It's not that I don't want to do it now, it's that I'm going to really dive into various levels of hooking and injection (think ring0) and I'd like to learn more graphics stuff before I really dive into that. Because my knowledge of the windows kernel is about 0 (and not ring0, literally 0) it's going to take a while. 

Anyways, I think I'm ready to play around with Havok's SDK and see if they have any examples of loading meshes or something. I think that'll bring me real close to being able to do it to the real game I'm looking at. Until then! /me out.