Tuesday, October 1, 2019

One Steve Shield Limit

Reader Emi asked:
There's something that always bothered me in sonic 3 with super sonic
Why SS can't use the insta shield? Or why you can't use the insta shield with normal invicibility.
I'm not sure if that's a bug but it feels odd since the rest still can use their special moves as super but sonic can't
The reason for this is twofold.

First off, the insta-shield is implemented as regular power-up -- it is actually a separate object which spawns into the player 1 shield shot when playing as Sonic, and Sonic is also awarded one whenever he loses any other shield. This is because unlike other characters' special moves, the insta-shield must animate independently from Sonic, and it loads its art to the same VRAM slot reserved for shields and invincibility stars.

This is probably why Sonic can't use any shield moves while invincible. Since shields and invincibility share a single VRAM slot, they can't be displayed at the same time (thank goodness), and it would be pretty bad design to let players use shield moves without knowing ahead of time which move Sonic would actually perform. As such, all shield moves are disabled while Sonic's shield is hidden, and this includes Super Sonic.

Of course, you could reasonably argue that since Sonic's shield is hidden, the insta-shield should just override all other shield moves for the duration of the Super form. After all, it's what Hyper Sonic's jump dash does. That would probably work, if not for another VRAM conflict: similar to the insta-shield, the large sparks that trail behind Super characters at high speeds load their art to the same VRAM slot as shields and invincibility stars.

Actually, this is probably why they fixed that Sonic 2 bug where invincibility and Super would stack! It wasn't a problem in Sonic 2 because shields and invincibility used separate VRAM slots, since players could potentially have both at the same time in a 2P game. Anyway, it was probably too much trouble to mediate VRAM usage between the insta-shield and the sparks, so Super Sonic specifically has no jump moves. That much is intentional:
    bclr    #Status_RollJump,status(a0)             bclr    #Status_RollJump,status(a0)
    tst.b   (Super_Sonic_Knux_flag).w               tst.b   (Super_Sonic_Knux_flag).w
    beq.s   Sonic_FireShield                        beq.s   Sonic_FireShield
                                                    bmi.w   Sonic_HyperDash
    move.b  #1,double_jump_flag(a0)                 move.b  #1,double_jump_flag(a0)
    rts                                             rts
; --------------------------------------------- ; ---------------------------------------------

Sonic_FireShield:                               Sonic_FireShield:
    btst    #Status_Invincible,$2B(a0)              btst    #Status_Invincible,$2B(a0)
    bne.w   locret_12A20                            bne.w   locret_11A14
    btst    #Status_FireShield,$2B(a0)              btst    #Status_FireShield,$2B(a0)
    beq.s   Sonic_LightningShield                   beq.s   Sonic_LightningShield
Now, what probably wasn't intentional is how Super Sonic can trigger the bubble shield's bounce effect upon touching the ground in S3A!

Note how in the above bit of code, Sonic's double_jump_flag is still set upon a successful double jump, even when the Super_Sonic_Knux_flag is on. If Sonic happens to have a bubble shield, that will cause the Player_TouchFloor routine to trigger a bubble bounce, despite the initial downward plunge bit never actually taking place.

S&K adds a check for Super_Sonic_Knux_flag to this code, as well as a clumsy character ID check since characters other than Sonic sometimes run through this code too I guess:
    tst.b   double_jump_flag(a0)                    tst.b   double_jump_flag(a0)
    beq.s   locret_130BC                            beq.s   locret_12230
                                                    tst.b   character_id(a0)
                                                    bne.s   loc_1222A
                                                    tst.b   (Super_Sonic_Knux_flag).w
                                                    bne.s   loc_1222A
    btst    #Status_BublShield,$2B(a0)              btst    #Status_BublShield,$2B(a0)
    beq.s   loc_130B6                               beq.s   loc_1222A
    bsr.s   BubbleShield_Bounce                     bsr.s   BubbleShield_Bounce

loc_130B6:                                      loc_1222A:
    move.b  #0,double_jump_flag(a0)                 move.b  #0,double_jump_flag(a0)
What baffles me is, why include that code path at all? If you just remove the Super_Sonic_Knux_flag check, then the invincibility check in Sonic_FireShield will also catch Super Sonic and bail without setting the double_jump_flag. Yes, the game will continue checking the A, B and C buttons once every frame until Sonic lands back on the floor, but guess what? The game already does that for invincibility!

Okay, so the insta-shield is just a regular old object in the shield slot. Does that mean that if we award it to characters other than Sonic, they too can benefit from its unique properties? Not really.

You see, all the interesting stuff is done by the Sonic object itself. So long as his double_jump_flag is set to 1, Sonic's attack radius is increased beyond its normal range. The insta-shield simply plays its animation whenever the flag jumps from 0 to 1, and then at the end of the animation, it increases the flag to 2, returning Sonic's attack radius to normal.

Incidentally, this is the source of another S3A bug: the double_jump_flag is only cleared when Sonic touches the floor, so if the insta-shield's animation ends after Sonic has already landed, the flag will remain stuck at 2 until Sonic touches the floor again, and any double jumps prior to that will be considered to have been "already spent", as it were.

S&K fixes this by checking whether the double_jump_flag has already been cleared before attempting to set it to 2.

In my upcoming ROM hack, which I *swear* isn't cancelled, you can have characters other than Tails as your support character. Shields can still be awarded to player 1, though, so what do we do about the insta-shield art conflict?

Well, the spindash dust object is already working overtime as skid dust, water splashes and the drowning timer, so now it also serves as an insta-shield for player 2. Yeah, it disappears when player 2 jumps in/out of water and fails to show up at all when they're drowning, but as reader Emi notes, blocking the insta-shield altogether would just feel... odd.

Sunday, February 24, 2019

Took you long enough

25 years ago to this day, the original Sonic 3 was released for the Sega Mega Drive across European territories. Being from Portugal myself, I still have a full boxed copy!

In celebration of this date, I am proud to announce that a disassembly of Sonic 3 is now available, more than ten years after the original Sonic & Knuckles disassembly was released.

...Wow, I don't even know where to begin. Just five or six years ago I was still exclusively messing around with graphics stuff, afraid to touch any of the scary assembly code. Before I knew it, I was programming entire objects from scratch, writing a blog which is at least 50% assembly code, massively improving the level editor experience, and now this.

None of these things would have happened if not for the people at my side each step of the way. My undying gratitude to Tiddles for always supporting me and getting me on this wild ride. Thanks to MainMemory for always being there to talk and share war stories. Thanks to the readers of my blog for the motivation and for showing interest in a Sonic 3 disassembly. And ultimately, thanks to flamewing for graciously approving my pull requests to the skdisasm repo.

I hope you will continue to support me in my future endeavors. The ride's not over by any stretch of the word.

Monday, January 28, 2019

Green spheres in special stages, part 4

Alright, let's finally start writing our green sphere logic. Because we're implementing them over yellow spheres, there's actually something extra we need to take into account: yellow spheres are the only objects Tails will interact with in a Sonic and Tails game, so we have to skip over that code by adding a check for our green spheres flag at loc_92C4:
    cmpi.b  #5,$44(a0)
    bne.s   loc_9304
    tst.b   (Special_stage_green_spheres).w
    bne.s   loc_9304
    tst.b   (Special_stage_clear_routine).w
    bne.s   loc_9304
Next, we'll go to sub_972E and look for the yellow sphere handling code. It can be found at loc_97EE, so that's where we need to add the object to the collision response list, which will look very similar to the blue sphere handling code:
    cmpi.b  #5,d2
    bne.s   loc_9822
    tst.b   (Special_stage_green_spheres).w
    beq.s   loc_97F4
    bsr.w   Find_SStageCollisionResponseSlot
    bne.s   loc_97BE
    move.b  #3,(a2)
    move.l  a1,4(a2)
    bra.s   loc_97BE

    tst.b   (Special_stage_jump_lock).w
We write the routine and layout pointer to the collision response slot as usual, then jump to loc_97BE to play the blue sphere sound. This time however, we set the routine to 3, which we'll then define in the off_9DFC array as follows:
off_9DFC:   dc.l Touch_SSSprites_Ring
            dc.l Touch_SSSprites_BlueSphere
            dc.l Touch_SSSprites_GreenSphere
We also need to add a dummy green sphere definition to the MapPtr_A10A array, which will serve as our halfway state between touching the green sphere and placing a blue sphere in its stead. Turns out, there's another blue sphere clone in slot $C which is completely unused, so we can just tweak its VDP pattern to point at palette line 4 and use that:
    dc.l Map_SStageChaosEmerald ; $B
    dc.l $E5A70000              ;
    dc.l Map_SStageSphere       ; $C
    dc.l $E6800000              ;
    dc.l Map_SStageSuperEmerald ; $D
    dc.l $E5A70000              ;
Okay, we're now ready to write our implementation of Touch_SSSprites_GreenSphere, which is basically going to be a lighter version of Touch_SSSprites_BlueSphere. Here's what it needs to do:
  • The first time through, we change the object at the layout position to $C, and set up a timer of nine frames.
  • On subsequent loops, we run out the timer, and then wait until any of the top three bits in the fractional part of the player's position are set, signaling that we are no longer right on top of the sphere.
  • After we're done waiting, we write 2 (a blue sphere) to the layout and clear out the collision response slot.

So let's go ahead and turn that into code:
    movea.l 4(a0),a1
    cmpi.b  #$C,(a1)
    beq.s   .checkChangeColor
    move.b  #$C,(a1)
    move.b  #9,1(a0)

    subq.b  #1,1(a0)
    bpl.s   .return
    move.w  (Special_stage_X_pos).w,d0
    or.w    (Special_stage_Y_pos).w,d0
    andi.w  #$E0,d0
    beq.s   .return
    move.b  #2,(a1)
    clr.l   (a0)
    clr.l   4(a0)

That should about do it. If we build and test the ROM right now though, we'll soon realize there is a problem. Although our green spheres correctly only turn blue once we've walked past them, we end up collecting those immediately after, leaving behind a trail of red spheres as we go.

So what is the reason for the discrepancy between red and blue spheres? Why do we collect the blue sphere as we're walking away, but not activate the red sphere in the same circumstances? The answer can be found at sub_972E:
    lea     (Plane_buffer).w,a1
    move.w  (Special_stage_X_pos).w,d0
    addi.w  #$80,d0
    lsr.w   #8,d0
    andi.w  #$1F,d0
    move.w  (Special_stage_Y_pos).w,d1
    addi.w  #$80,d1
    lsr.w   #8,d1
    andi.w  #$1F,d1
    lsl.w   #5,d1
    or.b    d0,d1
    lea     (a1,d1.w),a1
    move.b  (a1),d2
    beq.w   locret_98AE
Note how before we truncate the player's X and Y coordinates, we add onto them a fractional value of $80, or 0.5. This effectively rounds the player's position towards the closest integer, which means that if we're walking from position A to position B, then we will touch the object at position B as soon as we're closer to B then we are to A.

That's not really the relevant part, though. If we continue reading, we'll soon run into some very familiar code:
    cmpi.b  #1,d2
    bne.s   loc_97AA
    move.w  (Special_stage_X_pos).w,d0
    or.w    (Special_stage_Y_pos).w,d0
    andi.w  #$E0,d0
    bne.s   locret_97A8
When the object at the player's current position is 1, a red sphere, we are once again we are checking the top three bits in the fractional part of the player's position. However, this time we're not waiting until any of them are set. We're waiting until all of them are cleared! Red spheres only activate if we're right on top of them!

This explains the mismatched behavior between blue spheres and red spheres. For red spheres, it is sufficient to wait until we are no longer at the center of the intersection before placing them in the layout. Before placing a blue sphere however, we must ensure that the player has since moved to another layout cell.

To do this, we will first write the player's position to the collision response slot at loc_97EE:
    move.b  #3,(a2)
    move.w  d1,2(a2)
    move.l  a1,4(a2)
Now we can repeat the same calculation in Touch_SSSprites_GreenSphere, and then compare it against the stored value. As soon as these are different, it is safe to place the blue sphere in the layout. We preserve the original check so that the sphere still changes from green to blue with the same timing that a blue sphere does to red, except all we write to the layout at that point is $A, a dummy blue sphere:
    movea.l 4(a0),a1
    cmpi.b  #$C,(a1)
    beq.s   .checkChangeColor
    cmpi.b  #$A,(a1)
    beq.s   .checkPlayerMoved
    move.b  #$C,(a1)
    move.b  #9,1(a0)

    subq.b  #1,1(a0)
    bpl.s   .checkPlayerMoved
    move.w  (Special_stage_X_pos).w,d0
    or.w    (Special_stage_Y_pos).w,d0
    andi.w  #$E0,d0
    beq.s   .return
    move.b  #$A,(a1)

    move.w  (Special_stage_X_pos).w,d0
    addi.w  #$80,d0
    lsr.w   #8,d0
    andi.w  #$1F,d0
    move.w  (Special_stage_Y_pos).w,d1
    addi.w  #$80,d1
    lsr.w   #8,d1
    andi.w  #$1F,d1
    lsl.w   #5,d1
    or.b    d0,d1
    cmp.w   2(a0),d1
    beq.s   .return
    move.b  #2,(a1)
    clr.l   (a0)
    clr.l   4(a0)

Note how we change the first return branch to point at the new check, which forces the sphere to become collectible as soon as we move to another layout cell, even if the sphere was still running out its timer. This avoids a bug at very high speeds, where if we collect a green sphere and then immediately run into a bumper, we fail to pick up the resulting blue sphere as we backpedal through it.

And that's it. We've successfully added green spheres to the special stage, while yellow spheres continue to work in all the layouts we didn't touch. Many thanks to flamewing for approving my pull requests to the skdisasm repo over these past few weeks; this series would've been unpalatable without properly labeled variables and subroutines.

Saturday, January 26, 2019

Green spheres in special stages, part 3

We have a lot to cover this time as well, so let's get to it. sub_972E is the function responsible for handling collisions between the player and all other special stage objects, and particularly by the time we get to loc_97AA, the d2 register will hold the contents of the layout cell closest to the player's position:
    cmpi.b  #2,d2
    bne.s   loc_97C8
    bsr.w   Find_SStageCollisionResponseSlot
    bne.s   loc_97BE
    move.b  #2,(a2)
    move.l  a1,4(a2)

    moveq   #$65,d0
    jsr     (Play_Sound_2).l
The check at the top ensures that the rest of this code only executes when the cell the player is on contains a 2, which as we saw before, corresponds to a blue sphere. This is the blue sphere collision code! So what does it do?

Not much, really. Besides playing the blue sphere sound, all this does is call the Find_SStageCollisionResponseSlot function, which if you happen to have read my previous series on responsive object collision, works quite a bit like the Add_SpriteToCollisionResponseList function, in that it allows the object to store information about the collision that just took place into a list to be processed in the future, this time by the Touch_SSSprites function.

In this case, since there are no SSTs associated with special stage objects, all information must be stored directly into the collision response list. Each entry is eight bytes long, and the code at sub_972E consumes five of them right off the bat: byte 0 is the object's routine, while bytes 4-7 store a RAM pointer to the object's location in the layout.

Touch_SSSprites uses the routine byte to index into the off_9DFC array. Note that since zero denotes an empty slot in the collision response list, this array is actually one-based, so the routine value set by loc_97AA above refers to the second function in the array, not the third.

Which is a good thing, because there are only two functions in the array!
off_9DFC:   dc.l Touch_SSSprites_Ring
            dc.l Touch_SSSprites_BlueSphere
Touch_SSSprites_BlueSphere is a bit more involved than the preceding code, so let's break it up into small parts:
    subq.b  #1,2(a0)
    bpl.s   locret_9E86
    move.b  #9,2(a0)
Right at the start, we run a timer in byte 2 of the collision response slot. This timer is decremented every frame, and it prevents the function from doing anything until the result of the decrement is negative. Since the timer is always zero the first time through this code, we skip the branch and reset the timer to 9 in the process.
    movea.l 4(a0),a1
    cmpi.b  #2,(a1)
    bne.s   loc_9E62
Next, we load the object's location into register a1, and check the contents of that layout cell. If the object at that cell is somehow not a blue sphere, then we jump to loc_9E62. This might not make much sense right now, but for the time being, we skip the branch and continue to the code below.
    bsr.w   sub_9E88
    move.b  #$A,(a1)
    bsr.s   sub_9EBC
    beq.s   locret_9E60
    move.b  #4,(a1)
    clr.l   (a0)
    clr.l   4(a0)

When called, sub_9E88 decrements the remaining sphere count once, and if the count has reached zero, disables the player's ability to jump. Meanwhile, sub_9EBC is responsible for clearing out large groups of blue spheres once they are enclosed by red spheres, and returns a non-zero value if such a closed pattern is found.

Let's go over that within the context of our code. When a closed pattern of red spheres is found, we skip the branch to locret_9E60 and set the object at the current layout cell to 4, which corresponds to a ring. The collision response slot is then cleared out, signaling that we are done processing this sphere.

So what happens when a closed pattern isn't found? Well, right before sub_9EBC is called, the contents of the current layout cell are set to $A, and when we take the branch to locret_9E60, that change sticks -- for the nine frames that we told the timer at the start of the function to wait around for.

But what does a value of $A represent? It's not a red sphere, since as we previously saw, those correspond to a value of 1. The answer lies in the MapPtr_A10A array, which defines the mappings pointer and base VDP pattern for all the objects that can be placed in the special stage layout:
    dc.l Map_SStageSphere       ; 0
    dc.l $86800000              ;
    dc.l Map_SStageSphere       ; 1
    dc.l $86800000              ;
    dc.l Map_SStageSphere       ; 2
    dc.l $C6800000              ;
    dc.l Map_SStageSphere       ; $A
    dc.l $C6800000              ;
As it turns out, object $A is identical to object 2, blue sphere, except for the fact that it is not object 2. That means it is not caught by the collision code at loc_97AA, and therefore not re-added to the collision response list while we're still waiting out the timer on the first one. It also means that once the timer expires, we will take that branch to loc_9E62.

How come we didn't change the blue sphere to a red sphere right away, though? Let's keep reading.
    move.b  #0,2(a0)
    move.w  (Special_stage_X_pos).w,d0
    or.w    (Special_stage_Y_pos).w,d0
    andi.w  #$E0,d0
    beq.s   locret_9E86
Here, we take the player's current X and Y positions, and check the three highest bits of their fractional part. If none of those are set, then the function does nothing except clear the timer to ensure it doesn't roll over. This essentially means we're waiting until we're no longer at one of the crossroads in the layout.

You may have guessed it by now, but the reason we're doing this is so we don't run into the red sphere we're trying to place. Once we're in the clear, we can finally write a 1 to the layout and clear out the collision response slot.
    cmpi.b  #$A,(a1)
    bne.s   loc_9E80
    move.b  #1,(a1)

    clr.l   (a0)
    clr.l   4(a0)

Note the sanity check at the top: it ensures we don't accidentally place a red sphere over a layout cell which has since been converted to a ring by the closed pattern algorithm in sub_9EBC.

Okay, that was a lot of words, but not much to show for it. In the next and final part, we'll use everything we've learned to write a custom implementation of green spheres using the existing framework.

Tuesday, January 22, 2019

Green spheres in special stages, part 2

Okay, so the first thing we should do is set up a test stage. For the purpose of this series, I'll just be doing small edits to the first Sonic 3 stage. We could edit one of the Sonic & Knuckles stages, but since those are Kosinski-compressed, it would add the extra inconvenience of having to decompress the file before making our changes, and then recompress the file before building and testing the ROM.

The layout for the first Sonic 3 stage can be found in the General/Special Stage/Layout/S3 1.bin file. We can use a tool such as S3SSEdit to add in a suitable formation of yellow spheres, which later will become our green spheres.

However, as soon will be apparent, it is more than pedagogic to open up the layout file in a hex editor and take a look at what's under the covers. See, if you set the row length to exactly 32 bytes...

Holy smokes! That's the first special stage, all right!

So as it happens, special stage layouts work much in the same as regular level layouts do, with each byte in the layout file describing the contents of each space in the level grid. From the screenshot above, we can see that red spheres correspond to a value of 01, blue spheres correspond to a value of 02, and the yellow spheres we added correspond to a value of 05. This will be important later, so just keep it at the back of your mind for now.

Looking closely though, at the very end of the file there are four extra words which do not correspond to any part of the stage's layout. Each of these words is actually copied into its own RAM variable right before the special stage starts, as part of the layout load code at sub_85B0:
    move.w  (a2)+,(Special_stage_angle).w
    move.w  (a2)+,(Special_stage_X_pos).w
    move.w  (a2)+,(Special_stage_Y_pos).w
    move.w  (a2)+,(Special_stage_rings_left).w
The fourth word is the number of rings necessary to obtain a perfect bonus. Four groups of blue spheres × 16 spheres each = 64 = $40, so that looks about right. Note that the number of blue spheres necessary to clear the stage is not stored; we'll get back to this soon.

The second and third words set the player's starting position on the level grid. These are fixed-point numbers, with the high byte representing the integer part and the low byte representing the fractional part, so the values $200, $200 start us off at point (2,2) on the grid, with (0,0) being the northwest (top-left) corner.

The first word is interesting. It defines the player's starting direction along the level grid, and a key of valid values can be found in sonic3k.constants.asm:
Special_stage_angle =           ramaddr( $FFFFE426 ) ; byte ; $00 = north, $40 = west, $80 = south, $C0 = east

A value of $80 represents a southward-facing direction (down), which lines up with what we see both in S3SSEdit and in-game... but wait a second, it says right there that Special_stage_angle is a byte!

This is not an oversight: if you go through the entire code, you'll find that every reference to the Special_stage_angle variable is a byte access, except for the initial write from the layout seen above. We have an entire byte of RAM that is loaded from the layout file and then left completely unused. Let's go ahead and label that byte:
Special_stage_angle =           ramaddr( $FFFFE426 ) ; byte ; $00 = north, $40 = west, $80 = south, $C0 = east
Special_stage_green_spheres =   ramaddr( $FFFFE427 ) ; byte

We can now flag individual special stages as green sphere stages by setting byte $401 of their layout file to a non-zero value, and then check the Special_stage_green_spheres flag anywhere in the code and branch to custom logic when relevant. Let's set that byte in our layout, build the ROM and confirm that everything still works properly.

Alright, let's finally start coding stuff. Two things stick out to me: the first is that our green spheres aren't green (laughs). We can fix this by adding the following code at loc_82A6, directly after the call to sub_85B0:
    bsr.w   sub_85B0
    tst.b   (Special_stage_green_spheres).w
    beq.s   loc_82BE
    lea     (Target_palette_line_4+4).w,a1
    move.l  #$0C600A0,d0
    move.l  d0,(a1)+
    move.l  #$0600020,(a1)+
    move.l  d0,(a1)+

    lea     ($FFFF5500).l,a1
Let's make sure we understand what we're doing here. First, we load the address of the first yellow sphere color into register a1. The yellow colors start two entries into the fourth palette line, and each Mega Drive color is two bytes long, so that comes to four bytes after Target_palette_line_4.

Next, we load the colors $0C6, $0A0 into the d0 register and write them over the first two yellow colors, incrementing register a1 in the process. We then write the colors $060, $020 over the two middle yellows, increment a1 again, and close it out by writing the colors $0C6, $0A0 from d0 over the last two yellows as well.

The second thing that stuck out to me, which admittedly isn't obvious without a comparison screenshot, is that the total blue sphere count is now incorrect. Well I mean, it's correct right now since green spheres are still springs, but that's going to change once we start turning them into blue spheres on the fly.

So how is the total sphere count determined? The code responsible for this is at sub_9EA0:
    lea     (Plane_buffer).w,a3
    moveq   #0,d1
    move.w  #$3FF,d0

    cmpi.b  #2,(a3)+
    bne.s   loc_9EB2
    addq.w  #1,d1

    dbf     d0,loc_9EAA
    move.w  d1,(Special_stage_spheres_left).w
Quite plainly, this function loops through each byte in the special stage layout, and increments register d1 every time it finds a 2, which as we saw before, corresponds to a blue sphere. We'll extend this by also incrementing d1 whenever we find a 5 (yellow sphere), but only if the stage has been flagged as a green sphere stage:
    move.b  (a3)+,d2
    cmpi.b  #2,d2
    beq.s   loc_9EB0
    tst.b   (Special_stage_green_spheres).w
    beq.s   loc_9EB2
    cmpi.b  #5,d2
    bne.s   loc_9EB2

    addq.w  #1,d1
Build the ROM, run it and verify that the total sphere count is back to 102:

Alright, we've got the look down, so next time we'll jump right into what happens when a blue sphere is touched, and see how we can adapt that behavior for green spheres. See you there!

Monday, January 21, 2019

Green spheres in special stages, part 1

Man, I am just crazy good at updating this blog, aren't I?

First, some housekeeping. Back in October, I finally finished the SonLVL object definitions; you can now find them both in the S&K disasm repo and alongside the main SonLVL download. I then used them to produce a set of full-sized level maps for both Sonic 3 and Sonic & Knuckles, which you can find at the Sonic Retro wiki.

I soon got back to working on my hack as had I longed to, and have been semi-regularly posting a dev log to YouTube. It's still not at a point where I'm ready to share all my plans for it, but you'll get the basic idea from the videos alone.

Anyway, a while ago, Sonic Retro user Travelsonic posted a thread asking for documentation on special stage object behavior. I suggested I could blog a bit about the process of adding green spheres to the special stage, and here we are. It only took me two months to get around to it!

So, what are green spheres, exactly? Green spheres are one of the two new sphere types Stealth implemented in his "Blue Spheres 2" concept demo, which eventually made its way into Sonic Mania as a bonus game:

Quite simply, green spheres function as an additional layer on top of blue spheres: just like a blue sphere becomes red when touched, a green sphere first becomes blue, and must then be touched again before it becomes red.

Pink spheres, on the other hand, are two-way teleporters that would be far more difficult to implement, not only due to the programming overhead involved, but also a much more pressing factor: the palette.

As I alluded to in an early post, all four sphere types use the same set of sprites with different color palettes, in order to cut down on VRAM usage. Specifically, the first half of each palette line corresponds to the red, white, blue and yellow spheres respectively, in which the white palette replaces two otherwise duplicate colors with shades of red, revealing a star design that is invisible in every other sphere type.

And therein lies the rub: this design implies that there can only be four different sphere types at play at any given time, which means that if we want to add a new type, it will have to replace one of the existing colors.

Famously, the goal of the special stage is to Get Blue Spheres, so obviously those cannot be replaced. And since blue spheres become red spheres when touched, red spheres cannot be replaced either. The conclusion is that in order to implement green spheres, we must sacrifice either white spheres or yellow spheres.

Over the course of this series, I will show you how to implement green spheres over yellow spheres, though you could easily go the other way with presumably little complications. I will also implement a method of marking individual stages as green sphere stages, which if present enables all the additional logic surrounding green spheres, and if omitted preserves their behavior as ordinary yellow spheres.

Continued in part 2!

Thursday, September 20, 2018

On the subject of bitwise operators in C#

This subject is a bit off-band for the blog, but I figured it could also double as a status update. The first draft of the object definitions is almost complete; only Death Egg Zone remains at the time of writing. After that, I'll probably go over the entire set and make everything a little bit more consistent, add a few more overlays here and there, etc. I'm currently aiming to get everything done early next month, so we'll see how that goes.

I've also made up my mind about what the focus of my hack will be, so I can't wait to jump on that as well. It's going to be a lot of work up front, but I'm hoping the payoff is worth it. Anyway, time for a rant.

As you may or may not be aware, SonLVL is programmed in C#. Due to this, the most powerful way of writing SonLVL object definitions is to just roll your own C# code against SonLVL's public API, which SonLVL then compiles on the fly by calling up the C# compiler at runtime.

This is good! C# is a great programming language, and one which I regularly work with in my day job, so being able to transfer my existing skill set certainly makes it easier on both sides.

Now, the greatest complexity in writing object definitions comes from wrangling subtypes. Apart from the X/Y flip flags, the subtype is the only way of instructing objects to serve up a different appearance or behavior. As such, more often than not, several different properties are packed into the individual bits of the subtype byte. And therein lies the rub: performing bitwise operations in C# is just sad.

Let's take, for example, the Automatic Tunnel object. These are the high speed chutes found in Launch Base Zone and Lava Reef Zone. They have three properties, which are encoded into the subtype as follows:

  • Bits 0-4 are the Path ID, which defines the set of waypoints the player will be sent through.
  • Bit 6 is the Launch flag; if set, the player will keep their momentum at the end of the tunnel.
  • Bit 7 is the Reverse flag; if set, the player will go through the waypoints in reverse order.
  • Bit 5 is unused.

Here's the above information in graphical form, because humans love graphics:
     0  0  0  0  0  0  0  0 

   Reverse     Launch     Path ID
Alright, so now let's say I want to have a property box where the user can change the path ID, without affecting the other flags. Sounds easy enough. Just blank out the path ID bits already in the subtype, truncate the user value to five bits, and join the two together. So let's write that.
    subtype = (subtype & 0xE0) | (value & 0x1F);
Hit compile and... compilation error. An expression of type int cannot be assigned to the variable subtype, which is of type byte. Oh right, the literals 0xE0 and 0x1F are of type int, so the AND operations are lifted to int: both subtype and value get promoted from byte to int and operator &(int a, int b) is called, which itself returns int. The two resulting ints are then ORed together, so the entire expression is of type int, which cannot be assigned to a variable of type byte.

There's actually no way to write a byte literal in C#; you are expected to cast the int literal to byte. The compiler will do the right thing and not insert a conversion operation, but work with byte from the start. So let's write that.
    subtype = (subtype & (byte)0xE0) | (value & (byte)0x1F);
Hit compile, same error. As it turns out...

Pain point #1: There are no bitwise operators defined on byte

It's not the literals, it's the operators! There actually isn't such a thing as byte operator &(byte a, byte b) in C#; they go down to int and that's it. So when we write subtype & (byte)0xE0, the compiler promotes both bytes to int and then calls int operator &(int a, int b), once again resulting in a subexpression of type int.

The same thing goes for the OR operator, so no matter how we slice it, the whole expression will always evaluate to int. So the correct solution is to cast that instead:
    subtype = (byte)((subtype & 0xE0) | (value & 0x1F));
It's already getting hard to read through all the parentheses, but it's only going to get worse.

Pain point #2: Bitwise operations do not return bool

Let's turn our attention to the flags. In the case of the Reverse flag, I want the user value to be a yes/no toggle, so value is a bool. Then, depending on whether the bool is true or not, we set the relevant bit to 1 or 0. Let's write that.
    subtype = (byte)((subtype & 0x7F) | (value ? 0x80 : 0x00));
Alright, relatively painless. But what about the reverse operation, where we look up the current subtype and figure out the current state of the Launch flag? This time we're assigning to value, which is of type bool. So we write
    value = subtype & 0x80;
which again results in a compilation error, this time stating that an expression of type int cannot be assigned to a variable of type bool.

This is because in C#, unlike C and C++ before it, bools are strongly typed. They can only hold the values true and false, which alleviates the situation where 1 and 2 both mean true, but compare differently to one another. But that means there's no quick way to write a bit test in C#; one must append either != 0 or == 0x80, the former a tautology, the latter a repetition.

Now, since the Reverse flag happens to be the most significant bit, we can sidestep the issue by instead writing:
    value = subtype >= 0x80;
But in the case of the Launch flag, imagine my surprise when I write
    value = subtype & 0x40 != 0;
and I get yet another compilation error: operator & cannot be applied to operands of type byte and bool.

Pain point #3: Bitwise operators are also logical operators

If the previous point was to get rid of legacy C bullcrap, then this one enshrines it. Early versions of C did not have the logical operators && and ||, so to combine two or more equality comparisons into a single conditional expression, you would use the bitwise operators & and |, like so:
    if (day == 25 & month == 12) printf("It's Christmas!\n");
In order for this kind of expression to evaluate correctly, bitwise operations were given lower precedence than equality comparisons, so that the program would first check that the day is 25, then that the month is December, before it combines the results and decides whether it's Christmas or not. When bitwise operators were added to the C# specification, their precedence was kept the same, presumably in order to avoid "gotcha" scenarios when porting over legacy C and C++ code.

So above, when we wrote
    value = subtype & 0x40 != 0;
what the compiler actually does is check 0x40 and 0 for equality, and then attempt to combine the result with the value of subtype, which is the complete opposite of what we were trying to accomplish!

The solution is, again, to add more parentheses to the expression:
    value = (subtype & 0x40) != 0;
But here's the kicker: since in C#, equality comparisons result in bool, not int, they had to introduce separate, eager logical operators &(bool a, bool b) and |(bool a, bool b) to go along with the to the existing short-circuiting logical operators &&(bool a, bool b) and ||(bool a, bool b). So they could have avoided this whole disaster by simply giving the eager logical operators a different notation from the bitwise operators! Grrr.

With all that parenthesizing, it's no surprise that we end up with code that looks a little something like this:
properties[2] = new PropertySpec("Launch", typeof(bool), "Extended",
    "If set, the player will launch off at the end of the path.", null,
    (obj) => (obj.SubType & 0x40) != 0,
    (obj, value) => obj.SubType = (byte)((obj.SubType & 0xBF) | ((bool)value ? 0x40 : 0)));

And that's just a little bit sad.

Update 27/02/2020: Eric Lippert expands on the last point over at his own blog. This post was mostly inspired by Eric's writings there and elsewhere on the the Internet, so being able to finally link back is incredibly delightful to me.

Monday, July 9, 2018

An overview of the new API features in SonLVL

Okay, here we go. As promised in my previous post, here's a rundown of all the features MainMemory has graciously added to SonLVL's API in order to support my ongoing object definition adventure.

Debug overlays: Like I hinted at in last week's post, objects now have the option of drawing a secondary sprite, which is rendered with high priority above all regular sprites and level blocks. The most basic use of this feature is to plot out the movement patterns of continuously moving objects, such as floating platforms.

Beyond that, I also made it so that objects which are configured to move to a predetermined position will plot out the location of their collision box at the end of their movement pattern.

Streamlined sprites: Under the hood, the sprite rendering code has been greatly improved, resulting in faster load times and smoother scrolling when compared to previous versions of SonLVL. Sprites are also refreshed more often now, allowing me to do crazy stuff such as have crushing objects automatically detect the floors and ceilings as you drag them around the level.

Depth and priority: SonLVL now considers VDP priority information when rendering the main level view: high priority level blocks will hide any overlapping sprites, unless they are also set to high priority. Objects can now also optionally report the sprite's SST priority value, known here as depth, for proper sorting between sprites.

Extra colors: Four additional color palettes are now available in each level. These are hidden in the editor, but can be used by sprites in order to render objects which normally overwrite one of the palette lines, such as bosses, as well as grant the Knuckles player start its proper coloring.

XML player starts: Player start markers can now be defined using a limited subset of the XML syntax for regular object defintions, allowing for custom poses in each level, as well as composite sprites where necessary.

Layout swap option: This is a big one. It allows level configurations to define a set of layout copy areas in its level layout, and then swap them into the main layout through a menu option. This is absolutely vital while editing LBZ1, but also quite useful in FBZ, and to a lesser extent AIZ1, LBZ2 and SOZ2.

Animated PLC support: Levels can now optionally load blocks of uncompressed art, which are then rendered in place of the placeholder blocks in the main level view, allowing for a level's animated tiles to be rendered as they appear in-game. This is a really big one because it's not useful for just Sonic 3; MainMemory has already gone ahead and added animated tiles to the Sonic 1 and 2 level configurations.

So that's where we stand. The whole thing is still a work in progress -- only the S3 levels are done at the moment. If you're feeling intrepid enough, you can always grab the current stuff from my personal Git repo; constructive feedback is highly appreciated if you have any.

Monday, July 2, 2018

What I've been up to this whole time

It's been a while, huh?

You may be wondering where the hell I've been. Unfortunately, progress on the hack is pretty much where I left it four months ago -- I've produced a couple of assets and had a couple new ideas, no doubt in part due to all the crazy Mania Plus stuff just around the corner -- but I seem to have a knack for getting into digressions which soon become much more work than anyone could have reasonably anticipated.

Case in point: right now, here's what your average Sonic 3 stage looks like when opened up in the SonLVL level editor:

Urgh. A bunch of floating question marks, the actual level layout covered by a brick wall, and to round it out, a couple of placeholder numerals hanging around near the corner.

Now, let's check out what the same section looks like when using my work-in-progress SonLVL configuration files:

OMG there's so much stuff to talk about in this screenshot.

What you're seeing here is the result of a semi-joint venture between the author of SonLVL, MainMemory and myself, with the goal of providing a complete set of SonLVL object definitions for Sonic 3 & Knuckles. Essentially, what that entails is reverse-engineering the original 68k code, and producing accurate representations of all the objects within the level editor.

This process can be further boiled down into three core points:
  1. Identify every object ID used in a given stage, and give each of them them human-readable names;
  2. Document the effects of the subtype byte and the X/Y flip flags on the object's behavior and appearance, and provide a reasonable way for the user to view and modify these properties;
  3. Render a visual representation of the object, which should first and foremost be accurate to the object's in-game appearance, and if possible, illustrate the object's movement pattern as defined by its properties.

Let's focus on that last point. SonLVL already has a definition for Sonic 2's invisible block object, which highlights the object's actual size by drawing a yellow box around the rather unhelpful Tails block that appears in-game.

Upon porting this object to the Sonic 3 side of things, I quickly realized that a similar concept could be used to illustrate the alternating movement pattern of objects such as retracting spikes, thus making them stand out from their stationary brethren. Just draw the spikes at their "on" position, and the box at the "off" position!

There were two problems with this idea. Since the box was logically part of the object's sprite, you couldn't draw the spikes without the box, which ruined the map export feature. This wasn't an issue with the invisible block object, since its status as a "debug object" meant it didn't appear in the exported map anyway.

The other problem was that because the sprite was now twice as tall, so were the object's selection bounds!


No, if I was doing this, I was doing it right, which meant that somehow, I had to make SonLVL draw an optional "debug overlay" on top of an object's regular sprite. Luckily, I was already in talks with MainMemory at this point, and little did he know, I was about to make him work on SonLVL more than he had during the entire preceding year.

Next time, I'll go over all of the new features, and how I'm currently using them to make the Sonic 3 object definitions vastly superior to the ones available for previous titles.

Sunday, April 1, 2018

The many tendrils of a Sonic 3 level, part 3: hiding in plain sight

Have you ever been on an egg hunt, and ended up walking past the damn things multiple times without actually seeing them?

Shortly after the code for the animal object, at $2CA7C we run across the code for the title card object. The first thing it does in its init routine is check if the current zone is one of the Competition levels, and if so set byte $44 of its own SST.
        cmpi.b  #$E,(Current_zone).w
        bcs.s   loc_2CA96
        cmpi.b  #$12,(Current_zone).w
        bhi.s   loc_2CA96
        st      $44(a0)
        jmp     (Delete_Current_Sprite).l
When this flag is set, various aspects of the object's behavior are changed in order to display a unique set of title cards in Competition mode. However, these are ultimately never shown, because the object calls the Delete_Current_Sprite function immediately after setting the flag.
        lea     TitleCard_LevelGfx,a1
        moveq   #0,d0
        move.b  (Apparent_zone).w,d0
        lsl.w   #2,d0
        movea.l (a1,d0.w),a1
        move.w  #$A9A0,d2
        jsr     (Queue_Kos_Module).l
Shortly after, we find the code which loads the title card graphics. It uses the value of the apparent zone as an index to the TitleCard_LevelGfx array, which is a list of pointers to KosM-compressed archives containing the letters that spell out each zone's name:
TitleCard_LevelGfx:     dc.l ArtKosM_AIZTitleCard
                        dc.l ArtKosM_HCZTitleCard
                        dc.l ArtKosM_MGZTitleCard
                        dc.l ArtKosM_CNZTitleCard
                        dc.l ArtKosM_FBZTitleCard
                        dc.l ArtKosM_ICZTitleCard
                        dc.l ArtKosM_LBZTitleCard
                        dc.l ArtKosM_AIZTitleCard       ; MHZ
                        dc.l ArtKosM_AIZTitleCard       ; SOZ
                        dc.l ArtKosM_AIZTitleCard       ; LRZ
                        dc.l ArtKosM_AIZTitleCard       ; SSZ
                        dc.l ArtKosM_AIZTitleCard       ; DEZ
                        dc.l ArtKosM_AIZTitleCard       ; DDZ
                        dc.l ArtKosM_AIZTitleCard       ; HPZ
                        dc.l ArtKosM_ALZTitleCard
                        dc.l ArtKosM_BPZTitleCard
                        dc.l ArtKosM_DPZTitleCard
                        dc.l ArtKosM_CGZTitleCard
                        dc.l ArtKosM_EMZTitleCard
                        dc.l ArtKosM_BonusTitleCard
                        dc.l ArtKosM_BonusTitleCard
                        dc.l ArtKosM_BonusTitleCard
Then, at $2CC62, the apparent zone is used again, this time to inform the mapping frame used by the "name" portion of the title card:
        move.b  (Apparent_zone).w,d0
        add.b   d0,$22(a0)
The sprite mappings used by the title card object can be found at $2D90C. These mappings have the same layout as their Sonic & Knuckles counterparts, except the S&K stages all point at a blank mapping frame, and there's a mapping frame for the Competition mode title cards which was removed in Sonic & Knuckles.
Map_TitleCard:  dc.w Map_TitleCard_Blank-Map_TitleCard
                dc.w Map_TitleCard_Banner-Map_TitleCard
                dc.w Map_TitleCard_Act-Map_TitleCard
                dc.w Map_TitleCard_Zone-Map_TitleCard
                dc.w Map_TitleCard_AIZ-Map_TitleCard
                dc.w Map_TitleCard_HCZ-Map_TitleCard
                dc.w Map_TitleCard_MGZ-Map_TitleCard
                dc.w Map_TitleCard_CNZ-Map_TitleCard
                dc.w Map_TitleCard_FBZ-Map_TitleCard
                dc.w Map_TitleCard_ICZ-Map_TitleCard
                dc.w Map_TitleCard_LBZ-Map_TitleCard
                dc.w Map_TitleCard_Blank-Map_TitleCard  ; MHZ
                dc.w Map_TitleCard_Blank-Map_TitleCard  ; SOZ
                dc.w Map_TitleCard_Blank-Map_TitleCard  ; LRZ
                dc.w Map_TitleCard_Blank-Map_TitleCard  ; SSZ
                dc.w Map_TitleCard_Blank-Map_TitleCard  ; DEZ
                dc.w Map_TitleCard_Blank-Map_TitleCard  ; DDZ
                dc.w Map_TitleCard_Blank-Map_TitleCard  ; HPZ
                dc.w Map_TitleCard_2PMode-Map_TitleCard
                dc.w Map_TitleCard_Bonus-Map_TitleCard
                dc.w Map_TitleCard_Stage-Map_TitleCard
Note that both TitleCard_LevelGfx and Map_TitleCard contain valid entries for Flying Battery Zone, which is why we can get as far as displaying its title card in Sonic 3.

That's not why we're here, though. At $2CC08, right before being dismissed, the title card is responsible for loading the current level's KosM PLCs. It does this by calling the rather appropriately named LoadEnemyArt function:
        lea     off_2DF60,a6
        move.w  (Apparent_zone_and_act).w,d0
        ror.b   #1,d0
        lsr.w   #6,d0
        adda.w  (a6,d0.w),a6
        move.w  (a6)+,d6
        bmi.s   locret_2DF5E

        movea.l (a6)+,a1
        move.w  (a6)+,d2
        jsr     (Queue_Kos_Module).l
        dbf     d6,loc_2DF50

Once again, it uses the value of the apparent zone (and act) as an index to a pointer array, this time containing pointers to the KosM PLCs for each act in the game.

The thing is, much like TitleCard_LevelGfx and Map_TitleCard before it, this pointer array also contains valid entries for Flying Battery Zone:
off_2DF60:      dc.w PLCKosM_AIZ-off_2DF60
                dc.w PLCKosM_AIZ-off_2DF60
                dc.w PLCKosM_HCZ1-off_2DF60
                dc.w PLCKosM_HCZ2-off_2DF60
                dc.w PLCKosM_MGZ1-off_2DF60
                dc.w PLCKosM_MGZ2-off_2DF60
                dc.w PLCKosM_CNZ-off_2DF60
                dc.w PLCKosM_CNZ-off_2DF60
                dc.w PLCKosM_FBZ-off_2DF60
                dc.w PLCKosM_FBZ-off_2DF60
                dc.w PLCKosM_ICZ-off_2DF60
                dc.w PLCKosM_ICZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
                dc.w PLCKosM_LBZ-off_2DF60
Remember when I said the graphics for Flying Battery's enemies went out the door with the rest of the level load block?
PLCKosM_FBZ:    dc.w 1
                dc.l ArtKosM_Blaster
                dc.w $A000
                dc.l ArtKosM_Technosqueek
                dc.w $A500
Uh... April Fools!

The PAR code 02DF46:7010 will force every level to load Flying Battery Zone's KosM PLC. You can use this alongside the codes 05B58C:7010 and 05B5C2:7010 from my previous post in order to place the FBZ enemies in any level using debug mode. FBZ's palette is long gone (I checked), but luckily Carnival Night Zone's is a suitable replacement.