Client writing

One of the earliest projects I started when I got into the Metaplace alpha, apart from a handful of demo worlds to try out the API, was a client. Why write a client when the Flash one was available? To see if I could, of course. For me, it's not so much the "doing" as the "can I do" when it comes to programming.

If you're a programmer, you should definitely give the Metaplace wiki a peek; it has a lot of information about how a client connects and all of the Metatags that come down the pipe whenever something happens in a Metaplace world. Even if you have no interest in writing your own Metaplace client, seeing how the communication between the client and server happens can lend some insight to what kind of information is available to the client, and when, and thus can help you design scripts more efficiently for yourself or for modules you're looking to publish.

Lua

My first attempt at a client was in Lua. Because I was learning (and loving) Lua at the time, mainly to learn how to script in Metaplace, I was attempting to apply it to all programming problems that arose -- I find this a very good way to learn a language, even if the language is not necessarily suited to the problem domain. This is a perfect example of that, because while some of the data structures of Lua lend themselves well to writing a Metaplace client, the basics such as encryption and secure HTTP are not supported without adding external libraries, and Lua comes with no standard graphics library. This was where the project stalled -- there were a handful of choices out there, C libraries that could be linked into Lua to provide the visual part of the client (which, admittedly, is really the point). But these were large libraries that I had no interest in learning, especially in a manner that requires calling them awkwardly from within Lua.

In the end, my Lua client was just a text-based client, showing me the tags as they came in (and eventually you develop the ability to "read" them), with a simple command-line interface for querying the state of a world, such as the objects within, the users within, and, perhaps most usefully, for calling some built-in scripts that I had written; the rudimentary beginnings of a Metaplace bot. Those who were around in the alpha days might remember the statue in the earliest Central getting dropped on people's heads, or being tomatoed mercilessly by my auTOMATOn script. The only other use for the Lua client -- one that I've actually used quite a bit -- is as a way of introducing many commands to a world at once. This allowed me to generate the instructions for placing tiles and objects offline, into a file, and then upload the whole series of commands in one fell swoop, mimicking the effect that would otherwise require me to painstakingly do so through the editor tools.

Java/Android

My next attempt was to make a client for Google's Android platform. Not that I had (or have) an Android device, but it sure sounded like an interesting way to learn how to develop for that platform AND still do Metaplace stuff. Because Android is essentially a Java platform, I would be in familiar territory, having coded lots of Java over the years, and my initial trial with the networking in Android (using the emulator) showed that the most worrisome part of writing a Metaplace client on Android -- the actual connectivity -- wasn't going to be a problem.

The approach I took, then, was to write a Java client, and then worry about the Android-specifics after the fact. This may not be the most efficient way to write a Android client, but remember I really just wanted to write "a client", and if I ended up with a Java one first, and later an Android one, that was fine with me. Shortly after I started the project, Akidan, who had been fiddling with Tachevert's short foray into client-writing in .NET, joined the Java project, and he ended up getting it further along than the Lua client got. Unfortunately, Akidan left the Metaplace scene, and while I'm not usually a group-programming kind of guy, it kind of took the steam out of the effort, not having him around as a motivator to continue work made the project stall. This client got so far as to display some rudimentary UI, and the world tiles, and in the end could function as a very simple chat client, something that was only possible in the Lua client if you hand-crafted your conversation into Metaplace commands.

It's too bad that the project stopped -- there are a few Metaplace developers with Android devices that I feel I let down by not coming up with something useable for them. Though, unlike the Lua project, this one isn't "dead" so much as "abandoned", and could still be resurrected, if time permits. However...

Javascript

HTML5, and CSS3, and all of these new Web technologies that are coming out have finally given me reason to give this "web programming" a try. I've never had a need to do it before, and although I've read the occasional Javascript book in the past, I had never had a need to use it, so never really "knew" Javascript. As HTML5 and CSS3 are being developed and standardized, however, I'm trying to stay on top of it, figuring that now is as good a time as any to learn them. And what better way to learn than to write something big -- like a Metaplace client!

I have to say that, by far, this was the easiest client to get going, at least to the state that the other two did. Because (at the time of me writing it) the WebSocket part of HTML5 wasn't supported in any browser, it relies on a Java Applet to do the networking, but otherwise, in no time at all, it caught up to, and surpassed, the functionality of the Lua client. Perhaps two or three days? It's a good testament to either Javascript or the Metaplace client model. Or perhaps to the fact that it was my third client attempt.

The best part about this client is that you can actually try it! I will point out that the username/password become part of the URL, so do get stored in my server logs, but I assure you, I have no interest in them. I don't blame you if that prevents you from giving it a try, but you could also just make another Metaplace account if you really want to see. Now, it's not like it's finished or anything, so you might want to stick to simple worlds, such as the ClientTestWorld, and I think doesn't work on Safari or Opera (and definitely not IE!) so Firefox and Chrome are your best bet. It draws the background, tiles, and objects too, even animating them! And simple keyboard support also works, so you can walk around in the world a little.

This project only paused because this semester has become way too busy (thus my lack of anything Metaplace at all, and lack of posts here). Family, work, teaching and classes doesn't leave much time for anything, even fun Javascript-based Metaplace clients. This project is certainly not dead, just on hold, because it's still a great way for me to learn Javascript and some other HTML5 tags. The best part of this one, though, is that there's nothing stopping you from using View Source and just continuing on with it, if you so choose. Hey, if you do, let me know how far you get!

Go

Of course, no matter how busy I am, I find it hard to ignore when Google comes out with something new and interesting, such as their new Go language. Being an old C hacker and a compiler writer in another life, something like this really got my attention, and after a few little projects to warm up with the language (mainly ports of Lua projects that I had done to learn THAT language), I started thinking about what I could do for a large project in Go, to really test it out. You can see where this is going...

And that's where I am today; my current Metaplace time, which is still very slight for another week or so this semester, is taken up with starting my fourth Metaplace client. Go is very young, still under development, which makes things a little awkward (the first day I used the Vector library, they changed it on me), so things such as encryption and secure HTTP are missing from it as they were in Lua, as are anything but the most basic graphic primitives, but I feel have a better chance of appearing, and soon. Even so, connecting to external libraries is a little cleaner with Go than it was with Lua, and in just a few days, I've put together the start of client #4. Not as fast as the Javascript one, mind you -- I did have years of reading of Javascript behind me -- but it's getting there, slowly but surely.

Until the next language comes along, I guess.

Metascript

The scripting language for Metaplace, Metascript, is strongly based off of Lua, so much so that one might think that it WAS Lua. If you don't know Lua before diving into Metaplace, then that's fine, but if you've got some Lua under your belt, you might hit some hurdles.

This post isn't about learning Metascript, though, or about the differences between it and other programming languages, or how Lua/Metascript sucks compared to your favorite language. Just because you come from a background where arrays are indexed from zero doesn't mean that indexing from one is wrong. And if you want to argue the point, then I'll point out that Lua doesn't have arrays anyway.

Syntactic sugar

One of the things that will strike Lua programmers is the extra set of definitions available in Metascript:

• Define Properties()


• Define Commands()


• Trigger foo()


• Command bar()


• WebTrigger baz()

These all start "special" functions in Metascript, and aren't standard Lua. What post-alpha users might not know, though, is that these are just colourful candy coatings for some mundane-looking counterparts:

• function def_properties()


• function def_commands()


• function trg_foo()


• function cmd_bar()


• function trg_http_baz()

In fact, way back when we also included the "trg_" prefix in the SendTo() calls. As far as I know, these old methods still work; I assume this because

1. I still have old worlds that use it (thought I haven't visited them in a while)
   
2. We were told that they wouldn't stop working
   

Kinda takes some of the mystery away, doesn't it? I can see why the change was made: it helps to emphasize the special nature of these functions from others defined with "function XXX()"; and it hides the unfortunate fact that variable prefixes are used to denote semantic meaning. Don't consider them variables? In Lua,

function foo_bar()
...
end

is equivalent to

foo_bar=function()
...
end

Hrm. I'm now curious whether I could write

def_properties=function() foo_bar=0 end

in Metascript and have it work. I'm going to guess not, which I'll talk about shortly.

So, should you use function trg_foo() instead of Trigger foo() ? One reason you might is to do some offline programming, so you have code that compiles under a standard Lua interpreter, but can still be tested (by implementing your own backend library that knows how to handle Triggers and Commands). I've used this in the past to do some rapid development and to avoid some editor idiosyncrasies.

Define Properties() and Define Commands() (or function def_properties()...)

I've made no secret my dislike for these "functions". Define Properties() is where you define your script properties (where they might collide with others on the object), include scripts (similar to Lua's dofile()), expose properties and export functions.

Define Commands() is where you define your MakeInput()s and your MakeCommand()s. (There used to be a separate function def_inputs() -- I wonder if that still works.)


These functions are handled "specially"; they're actually executed at compile time, which can reveal bugs earlier than runtime, such as attempting to access functions that aren't available (which is almost all of them). That means no Debug(), no AlertToPlace() - no pairs() or ipairs().

They're actually code, though; you can have for loops in here, if there's anything worth looping over, and conditionals, if there's anything to compare. But generally, these functions act as definition blocks, and might better be implemented as such, extending the Metascript further from standard Lua. Then we could have some alternate notation for defining properties, such as

float value;

instead of relying on

float=0.0

to work. Which it won't. This is because behind-the-scenes, any local variables defined in this block are being specially processed into member variables on a C++ object on the server (all supposition -- I'm not actually privy to the source code). Being C++, types need to be known and once defined, permanent. This means that, unlike Lua, Metascript won't let you redefine the type of a property once it has been set (and this is a reason why I advocate table properties). Not only that, but it means that certain Lua types - ones that aren't available in C++ - aren't allowed as properties, such as booleans and functions. I would expect that supporting table properties had to require a bit of work, having the Lua form of the table being converted into something that C++ understands, and can hold, so why not a Lua function? Why not the lowly boolean? Also, this desire to define property type using Lua notation, instead of just creating a custom definition block, leads to ugliness such as

child="_object_"

to define a property as an object. Ouch.

As for Define Commands(), I'll just re-iterate from my previous post that I don't understand why MakeInput() and MakeCommand() cannot be used outside of Define Commands(). Sure, some compile-time checking is nice, but you're not saving me from all sorts of other scripting bugs, so why this? Also, these definitions check whether a Command specified in MakeCommand() is actually defined. Why? What's so bad about a Command being sent that doesn't have a handler? Triggers allow this (and allow multiple handlers), yet Commands must have specifically 1.

And because I just can't let it go: why are these two scripts run in such a tight sandbox? Fine, it might make no sense to call SendTo() while defining properties, or perhaps not even Debug() while defining an input, but no pairs()/ipairs()? Am I really the first person to write "code" in these blocks, instead of just a handful of definitions and pre-structured function calls?

Scope

In Lua, everything is a global variable unless you define it as "local". In Metascript, too, your definitions have a global scope within a script -- this probably bites me in the ass once a week, since I tend to write a lot of recursive functions. The "within a script" is important, though. When first looking at Metascript, and Metaplace, and the idea that multiple scripts are attached to an object, you might want to think that these are all loaded into a shared environment as far as the object is concerned. However, this isn't true, and rightfully so.

Remember how

Trigger foo()

is the same as

function trg_foo()

is the same as

trg_foo=function() ...

? Well this would be problematic if all scripts shared the same scope, because it would mean that an object couldn't have definitions for a Trigger function more than once, as the latter ones would overwrite the former, and having multiple definitions for the same Trigger is a key, powerful part of Metaplace. This scoping is unfortunate, however, because this means if we use IncludeScript() to import a set of functions, we have to do it for every script that needs them, instead of just having it imported once for the object. This makes it awkward to have a library that's used throughout a set of scripts.

Also, the scope affects the idea of "self". Commands and Triggers, usually the largest portion of a script, have an inherent sense of self. But functions do not. Why? Well, they actually used to, I believe before we had IncludeScript(), so there wasn't a question of the context in which a function was being run. Not being privy to the way the Lua sandbox is being run, I'm not exactly sure why "self" can't still be defined in the environment of a function call, but it is no longer supported - you must now pass it in explicitly.

Userdata

Stock Lua also has a userdata type -- it's basically a C++ object -- so this doesn't make Metascript different in that regard. However, because every object in Metaplace is represented by a userdata object, how they interact in script is important.

Knowing the structure of the userdata is usually required, because there's no reflection or introspection by default; you can access self.foo if you know it's there, but if you don't, you have no way to ask (not exactly true, see below). Why is this important? If I don't know that self.foo is there, should I really be using it?

Well, yes, sometimes there are cases where iterating over everything is a good idea. One case is the set of stylesheet API functions that Metaplace provides. These let us look at the stylesheet (basically, the static portion of a world), to peruse the templates, places, sprites, scripts and modules. To do this without any prior knowledge, we need to be able to iterate over them all, much the same way that pairs() and ipairs() allow us to iterate through a table. In some cases, we have a special ._all_ property on the userdata, which returns a table which can indeed be iterated over with ipairs(). But why not all the time?

For instance, if I want to browse the Places in my world, I can access a userdata object with stylesheet.places, and if I know the name of one, I can index into this userdata, such as stylesheet.places["0:1"]. Alternately, I can use stylesheet.places._all_ and loop through them all, finding the one I want. And once I have a specific Place, I can get another userdata from it with someplace.tiles. Not knowing anything about how many tiles there might be, I don't know what to ask for specifically, so I try someplace.tiles._all_, and get back ... nothing. It turns out that instead of a nice table to iterate through, I have to basically guess the indexes of the tiles, with something like

t=0
while someplace.tiles[t] do
...
t=t+1
end

And worse, there are other objects, such as the Place itself, that neither have a way to iterate (with ._all_) nor to enumerate (with a 0->n loop), but that you just have to guess/know the properties of, hoping that the wiki documentation is up-to-date. Again, this isn't something specific to Metascript -- Lua userdata can have this problem too -- but it would be nice if we had the ._all_ table available on all accessible userdata objects. Alternately, all userdata objects could act as tables for purposes of using pairs() or ipairs(), if they really wanted to.

Metatables

Metatables (the "meta" being unrelated to Metaplace) of Lua are a very powerful feature, one that, in my opinion, turns Lua from a simple toy language into a powerful full-featured one. Metatables allow us to redefine our environment, creating a sandbox where a limited set of functions might be available. Metatables allow tables to take on new functionality, such as addition of two tables, or different handling of accessing values that don't exist. And metatables ... aren't available in Metascript.

My only guess is that the current sandboxing provided to the Metaplace scripts makes further access to metatables ... hard? Unwise? Dangerous? Confusing? I'm not sure, but it's a real shame that we don't have them. Granted, they're an advanced feature, and it's quite likely that very few Metaplace users will notice their absence, but heavier coders like myself certainly do, and working around them can be difficult, awkward, or near impossible.

Case in point: I'm writing a vector/matrix library for Metaplace, to help implement a new physics engine that I'm writing. Such a library must exist for Lua somewhere, right? Sure, there are some out there, but most (all?) wisely take advantage of metatables to overwrite the built-in operators, since it makes for much nicer and cleaner usage of such a library if I can say

newvector=v1+v2

instead of

newvector=v1.sum(v2)

which I have to do now. Even something as simple as making the vectors printable:

Debug(newvector)

is a lot nicer than

Debug(newvector.tostring())

Small things? Yes, but these only touch on what metatables can allow. Frankly, I'd like to stop there because I don't want to realize what other powerful things I cannot do in Metascript because of their absence.

Verdict?

All that being said, Metascript isn't a bad language. It's Lua at its heart, with a few surgeries to make it tick a little differently, some of which might have been required, or some only elective. This customized version of Lua hasn't itself prevented me from doing anything in Metaplace, apart from quickly porting in pure-Lua code from the internet; any walls I've hit have been with the Metaplace platform itself, and not the scripting language.