Fast3D Display List Commands

[Trenavix]

Here, I will explain as many display list commands as I can that are used in Fast3D, aka the graphics engine that is used by Super Mario 64. Geometry layouts point to these scripts. Your imported levels run on this code, your imported models run on this code, all graphics in general almost always run on this. Every command is 8 bytes long.

​0x03 F3D_MOVEMEM
Used in Super Mario 64 to load vector normals (shading) RGB values.

Syntax:
03 aa 0000 bbcccccc
-aa: If 86, loads light values. If 88, loads dark values.
-bb: RAM bank value
-cccccc: RAM bank address

Example:
03 86 0000 0E 00 00 00
03 88 0000 0E 00 00 08
Loads light RGB from 0x0 in RAM bank 0x0E; loads dark RGB from 0x08 in RAM Bank 0x0E



​0x04 F3D_VTX
Fills the vertex buffer with vertex information (ex. coordinates, color values)

Syntax:
04aabbbb ccdddddd
- aa: (bbbb -) Where to start writing vertices inside the vertex buffer
- bbbb: Amount of vertices to write (* 0x10)
- cc: RAM bank in which vertices are stored
- dddddd: Offset at which vertices are stored in RAM bank

Example:
04E000F0 0E000780: Load 15 (0xF0 / 0x10) vertices from 0x0E000780 - offset 0x780 inside bank 0x0E (current map data) - and put them into the vertex buffer, starting at position 0x10 (15 vertices - 14).

Max amount of bytes to load in F3D is 1 00 (16 vertices)



​0x06 F3D_DL
Signifies the start of a Display List. May be used to link data and branch the current DL.

Syntax:
06000000 aabbbbbb
-aa: RAM Bank value
-bbbbbb: RAM Bank adress

Example:
06 00 00 00 07 00 0A 50 - Loads a display list from 0xA50 in bank 0x07.



​0xB5 F3D_QUAD
Renders one quad according to the vertices inside the vertex buffer

Syntax:
B5aabbcc XXddeeff
- aa: First vertex to use for the quad (* 0x0A)
- bb: Second vertex to use for the quad (* 0x0A)
- cc: Third vertex to use for the quad (* 0x0A)
- XX: (unused)
- aa: Fourth vertex to use for the quad (* 0x0A)
- bb: Fifth vertex to use for the quad (* 0x0A)
- cc: Sixth vertex to use for the quad (* 0x0A)

Example:
B5000A14 0000141E: Render a quad using the vertex data specified at the vertex buffer positions 0, 1 (0x0A / 0x0A), 2 (0x14 / 0x0A), 0, 2 (0x14 / 0x0A) and 3 (0x1E / 0x0A).



​0xB6 F3D_CLEARGEOMETRYMODE
Enables or disables certain geometry parameters (ex. lighting, front-/backface culling, Z-buffer). Used  at start of Display List.

Syntax: B6000000 aaaaaaaa
-aaaaaaaa: Various parameters

Examples:
B6 00 00 00 00 02 20 00 - Vertex RGB, no culling
B6 00 00 00 00 02 00 00 - Vertex RGB, culling
B6 00 00 00 00 00 00 00 - No vertex RGB, culling
B6 00 00 00 00 00 22 00 - No vertex RGB, no culling
​



​0xB7 F3D_SETGEOMETRYMODE
Enables or disables certain geometry parameters (ex. lighting, front-/backface culling, Z-buffer). Used at end of Display List.

Syntax: B7000000 aaaaaaaa
-aaaaaaaa: Various parameters

Examples:
B7 00 00 00 00 02 20 00 - Vertex RGB, no culling
B7 00 00 00 00 02 00 00 - Vertex RGB, culling
B7 00 00 00 00 00 00 00 - No vertex RGB, culling
B7 00 00 00 00 00 22 00 - No vertex RGB, no culling



​0xB8 F3D_ENDDL
Terminates the current Display List

Usage:
B800000000000000


​0xBB F3D_TEXTURE
Sets the texture scaling factor

Syntax: Unknown

Examples:
Standard: Set to BB 00 00 01 FF FF FF FF at start for 1 scaling.
Special case: Set to BB 00 00 01 0F 80 07 C0 at start for environment mapping.
Always reset to BB 00 00 00 FF FF FF FF at end of the DL to reset scale to 0.
​



​0xBF F3D_TRI1
Renders one triangle according to the vertices inside the vertex buffer

Syntax:
BF XXXXXX aabbcc
- XXXXXXXX: (unused)
- aa: First vertex to use for the triangle (* 0x0A)
- bb: Second vertex to use for the triangle (* 0x0A)
- cc: Third vertex to use for the triangle (* 0x0A)

Example:
BF 000000 00 0A 14: Render a triangle using the vertex data specified at the vertex buffer positions 0, 1 (0x0A / 0x0A) and 2 (0x14 / 0x0A).



​0xF2 G_SETTILESIZE
Sets the texture coordinates and size

Syntax: F2 00 00 00 00 WW WH HH
Where WWW = (width - 1) << 2 and HHH = (height - 1) << 2
The lower two bits of each WWW and HHH are always zero and setting them to non-zero doesn't appear to have any impact.

Examples:
F2 00 00 00 00 07 C0 7C for  32x32 textures
F2 00 00 00 00 0F C0 7C for  64x32
F2 00 00 00 00 07 C0 FC for  32x64



​0xF3 G_LOADBLOCK
Determines how much data to load after SETTIMG

Syntax: Unknown

Examples:
F3 00 00 00 07 7F F1 00 for 32x64 or 64x32 RGBA Textures
F3 00 00 00 07 3F F1 00 for 32x32 RGBA Textures



​0xF5 G_SETTILE
Sets the texture properties (ex. mirroring)

Syntax: Unknown

Examples:
F5 10 00 00 07 00 00 00 - Always loaded first for normal RGBA, followed by another F5 command
F5 70 00 00 07 00 00 00 - Always loaded first for Grayscale, followed by another F5 command
F5 10 10 00 07 01 40 50 - Normal for 32x32 textures after G_SETTILESIZE
F5 10 20 00 07 01 40 60 - Normal for 64x32 textures after G_SETTILESIZE
F5 70 10 00 07 01 40 50 - Grayscale for 32x32 textures after G_SETTILESIZE



​0xF8 G_SETFOGCOLOR
Sets the fog color

Syntax:
F8 000000 RRGGBBAA? (AA is typically left FF)

Example:
F8 00 00 00 00 FF 00 FF - Will set the fog for the following polygons to pure green.



​0xFB G_SETENVCOLOR
Sets the environment color for combiner

Syntax:
FB 000000 RRGGBBAA

Example:
FB 00 00 00 FF 00 00 8C - Will make everything loaded underneath the command red, and semi-transparent.



​0xFC G_SETCOMBINE
Performs combining operations (ex. multi-texturing)

Syntax: Unknown

Examples:
FC 12 7F FF FF FF F8 38 - Standard usage for solid RGBA textures
FC 12 18 24 FF 33 FF FF - Standard usage for alpha RGBA textures



​0xFD G_SETTIMG
Sets the texture image offset

Syntax:
FD100000 aabbbbbb
-aa: RAM bank value
-bbbbbb: RAM bank address

Example:
FD 10 00 00 04 00 00 90
Loads texture position from 0x90 in RAM bank 0x04
​

Vertex Structure

xxxxyyyyzzzz0000uuuuvvvvrrggbbaa
-xxxx: X Position of vertex
-yyyy: Y Position of vertex
-zzzz: Z Position of vertex
-uuuu: X texture stretch along vertex
-vvvv: Y texture stretch along vertex
-rr: Red channel of vertex, used for vertex colors and/or normals
-gg: Green channel of vertex, used for vertex colors and/or normals
-bb: Blue channel of vertex, used for vertex colors and/or normals
-aa: Alpha channel of vertex, used in vertex colors, ignored in vertex normals

Thanks to Spinout for several notes on F3DEX2, which I just translated to F3D.
Let me know if there are any errors, or commands that you want added.

[queueRAM]

(11-04-2015, 05:20 PM)cpuHacka101 Wrote: ​0xF2 G_SETTILESIZE
Sets the texture coordinates and size

Syntax: Unknown

Examples:
F2 00 00 00 00 07 C0 7C for  32x32 textures
F2 00 00 00 00 0F C0 7C for  64x32
F2 00 00 00 00 07 C0 FC for  32x64 textures

​

With some help from your docs and from Kaze, I was able to decode more of the G_SETTILESIZE encoding.
F2 00 00 00 00 WW WH HH
Where WWW = (width - 1) << 2 and HHH = (height - 1) << 2
The lower two bits of each WWW and HHH are always zero and setting them to non-zero doesn't appear to have any impact.

For example, to decode the command in C:

Code:

​width  = (((cmd[5] << 8) | (cmd[6] & 0xF0)) >> 6) + 1;
height = (((cmd[6] & 0x0F) << 8 | cmd[7]) >> 2) + 1;

For reference, here is a complete list of texture sizes used in SM64:
F2 00 00 00 00 01 C0 1C: 8x8
F2 00 00 00 00 01 C0 7C: 8x32
F2 00 00 00 00 03 C0 0C: 16x4
F2 00 00 00 00 03 C0 1C: 16x8
F2 00 00 00 00 03 C0 3C: 16x16
F2 00 00 00 00 03 C0 7C: 16x32
F2 00 00 00 00 03 C0 FC: 16x64
F2 00 00 00 00 07 C0 0C: 32x4
F2 00 00 00 00 07 C0 1C: 32x8
F2 00 00 00 00 07 C0 3C: 32x16
F2 00 00 00 00 07 C0 7C: 32x32
F2 00 00 00 00 07 C0 FC: 32x64
F2 00 00 00 00 0F C0 1C: 64x8
F2 00 00 00 00 0F C0 3C: 64x16
F2 00 00 00 00 0F C0 7C: 64x32
F2 00 00 00 00 13 C0 4C: 80x20
F2 00 00 00 00 1F C0 3C: 128x16

[Trenavix]

(11-04-2015, 06:38 PM)queueRAM Wrote: With some help from your docs and from Kaze, I was able to decode more of the G_SETTILESIZE encoding.

​
Ah, thanks very much for that. Glad we actually get good information in these forums. Less of a pre-configuration and more of a true understanding now. Copy pasta'd that into the syntax, thanks again

[Tarek701]

(11-04-2015, 05:20 PM)cpuHacka101 Wrote: ​0xFC G_SETCOMBINE
Performs combining operations (ex. multi-texturing)

Syntax: Unknown

Examples:
FC 12 7F FF FF FF F8 38 - Standard usage for solid RGBA textures
FC 12 18 24 FF 33 FF FF - Standard usage for alpha RGBA textures

​

I have some information to post on this. I posted it over skype to you earlier, trent. But now I post it here too. Me and jeremy were looking what this was doing, and discovered that 0xFC is basically for N64's CC (Color Combiner). The manual says following: "The CC is the ultimate paint mixer. It can take two color values from many sources and linearly interpolate between them." The equation:

newcolor = (A-B)*C+D

Here, A, B, C, and D can come from many different sources. Notice that if D=B, then this is a simple linear interpolator.

So, here the notes:
FC ee ee ee ee ee ee ee

e - Equation for blending, color = ( a - b) x c + d and alpha = ( e - f ) x g + h

Bitwise:
Mask for 'e':
[aaaacccc ceeegggi iiikkkkk bbbbjjjj mmmooodd dfffhhhl llnnnppp]​
a  - variable 'a' for mode 1  (color)
b  - variable 'b' for mode 1  (color)
c  - variable 'c' for mode 1  (color)
d  - variable 'd' for mode 1  (color)
e  - variable 'e' for mode 1  (alpha)
f  - variable 'f' for mode 1  (alpha)
g  - variable 'g' for mode 1  (alpha)
h  - variable 'h' for mode 1  (alpha)
i  - variable 'a' for mode 2  (color)
j  - variable 'b' for mode 2  (color)
k  - variable 'c' for mode 2  (color)
l  - variable 'd' for mode 2  (color)
m  - variable 'e' for mode 2  (alpha)
n  - variable 'f' for mode 2  (alpha)
o  - variable 'g' for mode 2  (alpha)
p  - variable 'h' for mode 2  (alpha)

Basically, n64 has a color combiner, basically combiner values, color + alpha values.

Decoding the standard usage for alpha RGBA textures:
FC 12 18 24 FF 33 FF FF - Standard usage for alpha RGBA textures

We decoded it as (the names were taken from N64devkit manual):
TEXEL0 (mode 1)
SHADE  (mode 1)
TEXEL0_ALPHA (mode 1)
SHADE_ALPHA (mode 1)
TEXEL0 (mode 2)
SHADE (mode 2)
K5 (mode 1)
K5 (mode 2)
TEXEL0_ALPHA (mode 2)
SHADE_ALPHA  (mode 2)
NOISE/K4     (mode 1)
0      (mode 1)
0      (mode 1)
NOISE/K4     (mode 2)
0      (mode 2)
0      (mode 2)

TEXEL0 = 1
SHADE = 4
TEXEL0_ALPHA = 1
SHADE_ALPHA = 4
K5 = 15
NOISE/K4 = 7
0 = 31

TEXEL0 is texture map output.
K5 is color convert constant
SHADE is shade color
NOISE is noise.

So, for calculating alpha and color:
color = (a - b)  c + d and alpha = (e - f) * g + h (according to N64DevKit manual)

More information about N64's CC is here:
​N64's Color Combiner

Also, this one here is also helpful if you want to understand a bit of the complexity of N64's color combiner and some examples.
​GLideN64 - Emulating CC

And last, there's the architecture of the CC.
​http://n64devkit.square7.ch/tutorial/graphics/4/4_1.htm

If something is wrong there, feel free to correct it.

[queueRAM]

(11-04-2015, 05:20 PM)cpuHacka101 Wrote: ​0x04 F3D_VTX
Fills the vertex buffer with vertex information (ex. coordinates, color values)

Syntax:
04aabbbb ccdddddd
- aa: (bbbb -) Where to start writing vertices inside the vertex buffer
- bbbb: Amount of vertices to write (* 0x10)
- cc: RAM bank in which vertices are stored
- dddddd: Offset at which vertices are stored in RAM bank

Example:
04E000F0 0E000780: Load 15 (0xF0 / 0x10) vertices from 0x0E000780 - offset 0x780 inside bank 0x0E (current map data) - and put them into the vertex buffer, starting at position 0x10 (15 vertices - 14).

Max amount of bytes to load in F3D is 1 00 (16 vertices)

​

I am a little confused about the first two parameters of the F3D_VTX command. I think the notes here indicate ('bbbb'-'aa') is the offset in the vertex buffer where to start writing ('bbbb'>>4) number of vertices. For all F3D_VTX i've seen in the SM64 ROM, it always loads an entire block at a time, starting at 0, 'aa' = 'bbbb' - 0x10. e.g.:
​04 80 0090 07 006430
04 F0 0100 07 0064C0
04 E0 00F0 07 006880

However, I've been looking in the Blast Corps ROM recently, which uses the same F3D microcode, but it has F3D_VTX values that don't follow this pattern. It sometimes uses multiple VTX commands, each only a couple verts at a time, e.g.:
​04 70 0080 08 000000
04 10 0020 08 000080
04 12 0020 08 0000A0
04 14 0020 08 0000C0
04 16 0020 08 0000E0

I looked at how F3D was interpretted by some of the emulators video plugins, and the all use the following formula (where w0 is the first 32-bit word of the command):

Code:

​
number = ((w0 >> 20) & 0xF) + 1; // Number of vertices to copy
offset =  (w0 >> 16) & 0xF;      // Current vertex

​

Based on these observations, I think F3D_VTX might actually be decoded as:
04 AB CCCC DD EEEEEE
​A = number of vertices - 1
​B = vertex index in buffer to start copying into
​C = length (in bytes = (A+1)*0x10)
​D = segment to copy from
E = offset in segment to copy from

[RobiNERD]

is the sphere enviroment map a 0xF5 G_SETTILE F3D effect?
Y'know, the one used by metal mario.

[Nutta]

no, it's 0xb6 and 0xb5

[RobiNERD]

cool thanks, i will look for it

[David]

I've created 2 simple online tools in javascript to decode and encode 0xFC commands. The decoder will break down the 0xFC into the separate tables to view the different a, b, c, d values. The encoder allows you to create custom 0xFC commands by selecting the a, b, c, d values in each of the tables. These two tools should help figure out what we need to use for custom levels.

You can view Tarek's post above if you want to learn how the color combiner works.

(Decoder) http://bin.smwcentral.net/u/26355/N64_cc_decoder.html
(Encoder) http://bin.smwcentral.net/u/26355/N64_cc_encoder.html

Spoiler: Combiner constants from gbi.h

/* Color combiner constants: */

#define G_CCMUX_COMBINED 0

#define G_CCMUX_TEXEL0 1

#define G_CCMUX_TEXEL1 2

#define G_CCMUX_PRIMITIVE 3

#define G_CCMUX_SHADE 4

#define G_CCMUX_ENVIRONMENT 5

#define G_CCMUX_CENTER 6

#define G_CCMUX_SCALE 6

#define G_CCMUX_COMBINED_ALPHA 7

#define G_CCMUX_TEXEL0_ALPHA 8

#define G_CCMUX_TEXEL1_ALPHA 9

#define G_CCMUX_PRIMITIVE_ALPHA 10

#define G_CCMUX_SHADE_ALPHA 11

#define G_CCMUX_ENV_ALPHA 12

#define G_CCMUX_LOD_FRACTION 13

#define G_CCMUX_PRIM_LOD_FRAC 14

#define G_CCMUX_NOISE 7

#define G_CCMUX_K4 7

#define G_CCMUX_K5 15

#define G_CCMUX_1 6

#define G_CCMUX_0 31



​/* Alpha combiner constants: */

#define G_ACMUX_COMBINED 0

#define G_ACMUX_TEXEL0 1

#define G_ACMUX_TEXEL1 2

#define G_ACMUX_PRIMITIVE 3

#define G_ACMUX_SHADE 4

#define G_ACMUX_ENVIRONMENT 5

#define G_ACMUX_LOD_FRACTION 0

#define G_ACMUX_PRIM_LOD_FRAC 6

#define G_ACMUX_1 6

#define G_ACMUX_0 7

Some of the constants use overlapping id numbers, so I had to put in "or" between the names with matching id numbers. I'm guessing the actual source is based off of context.

Here are four 0xFC commands that are commonly used with custom levels. Below this is an image with the results of all 4 of these commands together for comparison.

Code:

​FC 12 7E 24 FF FF F9 FC = Solid RGBA texture (No fog)
FC 12 18 24 FF 33 FF FF = Alpha RGBA texture (No fog)
FC 12 7F FF FF FF F8 38 = Solid RGBA texture (With fog)
FC FF FF FF FF FC F2 38 = Alpha RGBA texture (With fog)

Spoiler: Decode Results

It appears that the color combiner is in Two-Cycle mode when fog is turned on, and in One-Cycle mode when it's turned off.