[F3D] Material Overflow Preview Fix

[jesusyoshi54]

Fixed the way overflow works on the color combiner.

• Previous behavior was:
  • overflow [-0.5, 1.5) per cycle combined
• New behavior is:
  • c overflows [-1.0, 1.0) on input
  • overflow [-0.5, 1.5) per cycle combined

On 2 cycle, the combiner doesn't actually do the special sign extension [-0.5, 1.5) on the first cycle's output unless it is for alpha compare, but since this will feed into inputs which are getting special sign extension on hardware it replicates the behavior properly.
This will come after #312

[Dragorn421]

to be clear,

overflow [-0.5, 1.5)

means "clamping into the [0, 1] range happens for values in range [-0.5, 1.5), values outside are not clamped properly" right? (not sure on the specifics of "not clamped properly")

---

On 2 cycle, the combiner doesn't actually do the special sign extension [-0.5, 1.5) on the first cycle's output unless it is for alpha compare, but since this will feed into inputs which are getting special sign extension on hardware it replicates the behavior properly.

can you clarify what is fast64 nodes behavior and what is hardware behavior here

[Dragorn421]

I am trying to review this, currently trying to make a test blend, but I stumbled upon

• first Initial preset on material creation is "Custom" #315
• then color interpolation trolling

If you have a test blend already it'd be appreciated

[jesusyoshi54]

to be clear,

overflow [-0.5, 1.5)

means "clamping into the [0, 1] range happens for values in range [-0.5, 1.5), values outside are not clamped properly" right? (not sure on the specifics of "not clamped properly")

Values are coming out of the combiner have a special sign extension and clamp happening at those values, you can see that from AL source here with the definition of the clamp here.

So this means the output has values clamped to [0-1), with the overflow happening on [-0.5, 1.5). Anything [1-1.5) = 1, anything [-0.5-0] = 0.

On 2 cycle, the combiner doesn't actually do the special sign extension [-0.5, 1.5) on the first cycle's output unless it is for alpha compare, but since this will feed into inputs which are getting special sign extension on hardware it replicates the behavior properly.

can you clarify what is fast64 nodes behavior and what is hardware behavior here

I'll just go over this in steps. First the actual N64:

• Inputs receive the sign extensions they're assigned. e.g. A, B, & D get special sign extension, C gets normal one
• First cycle goes through the normal equation, gets right shifted by 8 (so that it is a s1.15 number).
• If there is no second cycle, this gets the special clamping described above, otherwise the number goes straight to cycle 2.
• Second cycle's inputs get the sign extensions assigned.
• Second cycle's output gets the special clamping.

For Fast64 both cycles use the same nodes, so the formula is as follows:

• C input receives normal sign extension
• Combiner equation is calculated
• Output gets special sign extension
• Value is clamped [0-1) heading into blender (which is just fog/no fog for fast64).

[jesusyoshi54]

If you have a test blend already it'd be appreciated

You get the rendering bug when you feed a combined input of 256 into C input. It then undergoes normal sign extension and becomes negative. You can get a value of 256 by using 1.0 on the 1st cycle combiner.

So a 1st cycle value of 1, or (1-prim)*tex+prim, tex+prim etc. will result in a value of 256 output. Just anything where a value between 1-1.5 can be input into C.

[Dragorn421]

has conflicts

[jesusyoshi54]

I messed up with my git history woops. Remade the branch