Sounds

After a year-and-a-half of taking part in Metaplace during alpha and beta, I've finally added sounds to one of my worlds (I don't count the crwth_effects world, as the sounds are only played when the "player" tries them out, and not as part of the world itself.) I'd say the biggest reason is that I rarely finish a world, and something like sound is a finishing touch.

I'm not really sure what made me do it, since the world I've added them to, my Ultima Online workspace world, is nowhere near done (being a lab world, it'll never be done as such). For some reason this past week, I got to thinking about the music of UO, or the sounds of UO... I'm not sure, but I decided to figure out how to pull all of the UO sounds from the datafiles. Once I had those, I browsed through them, playing them to bring back memories (which is silly, since my UO accounts only just expired) and to see the large portion of them that never appeared in the game. Some of the sounds include the player's footsteps on a variety of terrain, and I was thus compelled to add them into the UO world.

The funny thing is that right now, my UO world doesn't have the UO avatars for the players, but the Metaplace ones instead -- the only non-UO thing in the world gets the only UO sounds! For each terrain type (grass, pavement, sand, etc.) there are two sound files, allowing you to alternate the foot falls per-foot; they could have taken the approach of a sound file that played a two-step sound, but it wouldn't match up if you took a single step with your avatar.

This poses a challenge right off the bat, since there's no way in Metaplace to say "play these two sounds in a loop"; all sounds, whether to an individual or the whole Place, or whether always-on or based on distance, are on their own. Because we want to play a different sound right after one finishes -- to give the clip-clop walking effect, we have one of two tools available to us: we can either ask to be informed when a sound ends (and then act to start up the next one), or we can try to time playing the next sound based on the length of the current one.

Trigger sound_done

As part of the PlaySound*() series of API functions (but not exhaustively documented), you may provide an object pointer as a callback object, telling the system that once this sound finishes, this callback object will have a "sound_done" Trigger fire at it. This lets you play the next sound at the right time -- in theory. Of course, being an online world, you have the latency of the internet to contend with, which means that once the sound finishes, your client tells the server (after X milliseconds), the server acts upon this information by firing the sound_done trigger at the specified object, the object decides to play a new sound, and the client is told, again after X milliseconds. All told, these milliseconds can add up, between the latency and the script logic to determine that another sound should be played. This delay is probably fine for playing background music (which I also added to the UO world), because a half-second delay between soundtracks is fine. A half-second between footsteps, when you're stepping every quarter-second, is not.

Here are the basics of this method:

Trigger sound_done()
    local movesounds={
        {"feetpvmta","0:19"},
        {"feetpvmtb","0:18"}
    }

    self.move_number=self.move_number+1
    if self.move_number>#movesounds then
        self.move_number=1
    end
    local moverec=movesounds[self.move_number]
    self.move_sound=PlaySoundTo(self,moverec[2],255,0,self)
end

SendTo()

While we can't completely eliminate the latency problem, we can at least reduce it by not waiting to react to a message from the client, stating that the current sound has finished playing. Instead, after starting the current sound, we'll fire a delayed Trigger to our sound-playing code to just start playing when it makes sense to do so. This means that we need to know how long our sounds are to know how long to wait (for playing a series of background tracks), or we have to know that a given delay will be plenty of time for each of the sounds to play and finish (such as short footstep sounds).

Trigger movesound()
    local movesounds={
        {"feetpvmta","0:19"},
        {"feetpvmtb","0:18"}
    }

    self.move_number=self.move_number+1
    if self.move_number>#movesounds then
        self.move_number=1
    end
    local moverec=movesounds[self.move_number]
    self.move_sound=PlaySoundTo(self,moverec[2],255,0)
    self.soundtrigger=SendTo(self,"movesound",250)
end

I have to admit, I was prepared to trial-and-error the delay value to see what looked good -- in theory, it should be based off of the framerate of your animation, and the number of frames between each foot lands on the ground in the animation), but I really lucked out with my first attempt at 250ms -- go take a look at the world and see if you agree.

That's right, I've gone with this second approach, for the reasons specified -- I've reduced the effect of latency on my sounds loop by not relying on the client to report when the sounds finish, and I'm able to do this because the clip-clop of walking is very regular. And what about my background music? Well, in the end I used to delayed-Trigger method, there, too:

Trigger playmusic()
    local music={
        {"britain1","http://pages.cpsc.ucalgary.ca/~crwth/metaplace/uo/music/Britain1.mp3",39},
        {"Britainpos","http://pages.cpsc.ucalgary.ca/~crwth/metaplace/uo/music/Britainpos.mp3",53},
        {"Stones1","http://pages.cpsc.ucalgary.ca/~crwth/metaplace/uo/music/Stones1.mp3",135},
        {"Walking","http://pages.cpsc.ucalgary.ca/~crwth/metaplace/uo/music/Walking.mp3",343},
        {"Medieval","http://pages.cpsc.ucalgary.ca/~crwth/metaplace/uo/music/Medieval.mp3",150},
        {"Festival","http://pages.cpsc.ucalgary.ca/~crwth/metaplace/uo/music/Festival.mp3",128},
    }
    self.music_number=self.music_number+1
    if self.music_number>#music then
        self.music_number=1
    end
    local musicrec=music[self.music_number]
    self.background_music=PlaySoundRefTo(self,musicrec[2],100,0,self)
    SendTo(self,"playmusic",(musicrec[3]+1)*1000)
end

Note the list of tracks also contains the length, needed because they're of variable length, unlike the footsteps. Why use this method instead of the sound_done() Trigger? I think the main reason is because, when trying to handle both the music and the sound effects, I had code that looked like this:

Trigger sound_done(userid,handleid)
    if handleid==self.background_music then
        SendTo(self,"playmusic",1000)
    elseif handleid==self.move_sound then
        SendTo(self,"movesound",0)
    end
end

I can't imagine all the conditionals when I've got dozens of sound effects, such as spell effects and combat sounds. Sure, I could separate them into different scripts each with their own sound_done() Trigger, but the conditional code would still be the same. One way I see this being a bit more useful is being able to supply not only the callback object, but the callback Trigger as well.

Dynamic footsteps

The other thing I decided to do, because of the variety of footstep sounds available, was to change the sound based on the terrain the player walked upon.

In general, having different events based on the tiletype that a player stands on isn't "easy", because tiles themselves don't support Triggers or events, so you can't just attach a script to a grass tile that sets the player's footstep sound to "feetgrssa" and "feetgrssb". For many cases of tile-based effects, you'd have to have some sort of tick-based Trigger, checking if the tile has changed and then changing the effect in question.

The handy thing about the footsteps firing every 250ms, however, is that this provides its own "tick", and thus every time we're to start a new sound, we can decide if the sound should change.

Trigger movesound()
    if self.moving==0 then return end

    local tile=GetTileAt(self.x,self.y)
    local tilename=stylesheet.places["0:"..GetPlace().placeId].tiles[tile].name
    local movesounds={
        stone={
        {"feetpvmta","0:19"},
        {"feetpvmtb","0:18"}
        },
        grass={
        {"feetgrssa","0:21"},
        {"feetgrssb","0:20"}
        },
        dirt={
        {"feetgrvla","0:25"},
        {"feetgrvlb","0:22"}
        },
        sand={
        {"feetsanda","0:23"},
        {"feetsandb","0:24"}
        },
        water={
        {"feet15a","0:27"},
        {"feet15b","0:28"},
        {"feet15c","0:29"},
        {"feet15d","0:26"},
        }
    }
    local tilesounds=movesounds[tilename]
    if tilesounds then
        self.move_number=self.move_number+1
        if self.move_number>#tilesounds then
            self.move_number=1
        end
        local moverec=tilesounds[self.move_number]
        self.move_sound=PlaySoundTo(self,moverec[2],255,0,self)
    end
    self.soundtrigger=SendTo(self,"movesound",250)
end

This assumes that your tiles are nicely named, as mine are (they're computationally-generated from the Ultima Online datafiles, which conveniently provide names), though this is all in the hands of the world builder anyway. The above is the exact function I use for my footsteps.

Of course, this solution wouldn't work for sounds that have a long "tick" -- Ultima Online played different background music depending on your region, so you'd get some spookier music in the jungle or a dungeon, compared to a forest or city. You wouldn't want to wait three minutes for your current cycle of "city" themed music to end before realizing that you're in a dungeon and should be a little more on edge.

When to walk

The only thing missing is knowing when to start and stop the walking sound loop at all. The last snippet gave a hint on how the stopping is done -- using a property called self.walking. It's used to stop the sound-playing loop, above, but how is it set? By patching into the path_begin()/path_not_found()/path_end() series of Triggers that are part of the pathfinding system.

Trigger path_begin()
    StopSound(self,self.move_sound)
    if self.soundtrigger~=0 then CancelSend(self,self.soundtrigger) end
    self.moving=1
    SendTo(self,"movesound",0)
end

Trigger path_end()
    self.moving=0
    StopSound(self,self.move_sound)
end

Trigger path_not_found()
    self.moving=0
end

The extra code in the path_begin() Trigger is necessary because of the delayed-Trigger method being used; if someone was walking and then clicked elsewhere, they'd start a new pathfinding session but wouldn't have stopped the last delayed-Trigger, causing two (or three, or four) sound loops to play until the player finally stopped moving. This was an interesting effect, but definitely not desired!

Some readers might be thinking that such a problem could be avoided if I had just used SendToRepeat() instead of a recursive delayed Trigger. Then I could just let the SendToRepeat() continue as it would, but I could really do the same thing with the recursive Trigger. The reason I didn't use the SendToRepeat() is because of the possibility of a variable delay for the sounds; while the footsteps are a regular 250ms apart, I might get sounds that aren't so regular, and would thus want to be able to vary the delay, much as done in the music loop. For this use, I could go either way, but I like to code generally, even if Dorian and Sean would rather I didn't.


I'm happy with how the sound turned out, considering it was really a whim. It makes me want to get more sounds into the world, even without the mechanics that really require them (the sounds of combat, the sounds of crafting, monsters roaring...) I also realized, as writing this post, that the footstep sounds are solely for the ears of the maker -- this is a bit silly, since we have PlaySoundRefRadius() to emanate sound from an object and to get the feature of distance affecting the volume built-in. Perhaps I'll go take a look at that now.