We've Moved


The blog has been retired - it's up for legacy reasons, but these days I'm blogging at blog.theodox.com. All of the content from this site has been replicated there, and that's where all of the new content will be posted. The new feed is here . I'm experimenting with crossposting from the live site, but if you want to keep up to date use blog.theodox.com or just theodox.com

Sunday, May 31, 2015

Boo Who?

Boo Who?

Boo!

Did I scare you?

Evidently somebody’s scared: the Boo language, which has been a part of Unity for several years, is going to be removed from the Unity documentation in favor of C#.

The reason is pretty simple, as this graph explains:

For a lot of Unity developers (99.56% of them, apparently) this is non-news; Boo never really garnered much of a following in the Unity community. For new developers and recent grads, C# is an easy and very well documented option; for former web debs moving to mobile apps, UnityScript feels JavaScript-y enough to ease into a new environment. Boo, on the other hand, never got much traction: it’s got a small but passionate community but it never garnered enough momentum to break out of its niche.

Boo Hoo

Now, I’m kind of a sucker for hopeless causes, so almost inevitably this news inclined me to revisit Boo, which I’ve toyed with a few times but never really tried to learn. I had to write a lot of C# for Moonrise and it made me long for the clarity and concision of Python. Even though C# is a perfectly capable language with lots of modern features (closures, firest class functions, etc) it’s still very chatty. The tantalizing promise of Boo – not completely fulfilled, but pretty close, is that it combines both: the performance, runtime type safety, and intimate access to Unity that C# offers in a language not deformed by punctuation and rendered ridiculous by overly wordy syntax.

Here’s the aesthetic differences in a nutshell:

Boo

import UnityEngine

class JumpingMovementController(MonoBehaviour): 

 _HORIZ = 'Horizontal'
 _VERT = 'Vertical'
 _JUMP = 'Jump'
 _Momentum = 0.0
 _Gravity = 2.0
 public _Speed = 1.0
 public _JumpSpeed = 1.5

 def Update(): 
  frame_speed = _Speed * Time.deltaTime 

  if transform.position.y == 0 :
   _Momentum += Input.GetAxis(_JUMP) * _JumpSpeed

  up =  _Momentum * Time.deltaTime
  left_right = Input.GetAxis(_HORIZ) * frame_speed
  forward_back = Input.GetAxis(_VERT) * frame_speed
  transform.Translate(Vector3(left_right, up, forward_back), Space.Self)

 def LateUpdate():
  if transform.position.y > 0:
   _Momentum -= _Gravity * Time.deltaTime;
  else:
   _Momentum = 0;
   vp = Vector3(transform.position.x, 0, transform.position.z)
   transform.position = vp

C

using UnityEngine;
using System;

public class JumpingMovementController(MonoBehaviour)
{ 

    const static string _HORIZ = "Horizontal";
    const static string _VERT = "Vertical";
    const static string _JUMP = "Jump";
    var _Momentum = 0.0f;
    var _Gravity = 2.0f;
    public var _Speed = 1.0f;
    public var _JumpSpeed = 1.5f;


    void Update()
    {
        var frame_speed = _Speed * Time.deltaTime 

        if (transform.position.y == 0) 
        {
            _Momentum += Input.GetAxis(_JUMP) * _JumpSpeed;
        }

        var up =  _Momentum * Time.deltaTime;
        var left_right = Input.GetAxis(_HORIZ) * frame_speed;
        var forward_back = Input.GetAxis(_VERT) * frame_speed;
        transform.Translate(new (Vector3(left_right, up, forward_back)), Space.Self);
    }

    void LateUpdate()
    {
        if (transform.position.y > 0) 
        {
            _Momentum -= _Gravity * Time.deltaTime;
        }
        else 
        {
            _Momentum = 0;
            vp = new Vector3(transform.position.x, 0, transform.position.z);
            transform.position = vp;
        }
    }
}

I just can’t shake the feeling that the first code is something I don’t mind reading and writing while the latter is a chore. It’s also a whopping 45% more typing for the same result. And that delta only gets bigger if you want to try something a little more complicated: Boo supports offers the same list comprehension syntax as Python, so you can write:

    addresses = [(x,y) for x in range(3) for y in range(3)]

where in C# you’d either get 6 lines of for-loops and nested brackets, or you’d have to use Linq. Even in the most compact form I can manage it’s still much wordier:

        var xs = Enumerable.Range(0, 3).ToList();
        var ys = Enumerable.Range(0, 3).ToList();
        var addresses = (from x in xs
                         from y in ys
                         select new Tuple<int,int>(x, y)).ToList();      

to get to the same place.

Why Boother?

A hardcore programmer might object that this is “all just syntax”. That’s true – but since my everyday experience of working with a language is about 90% syntax I don’t feel like it’s fair to dismiss that concern as if it were irrelevant. That said, it can’t be denied that modern C# 4 includes many language constructs that earlier versions of the language lacked: things like var inferred variables, lambdas, closures, and named default arguments. These things all help make the code less chatty and more legilble: If you’re a good C# programmer you can write very terse, expressive code that’s not absurdly wordy.

Apart from those stupid curly brackets.

On the other hand, the “culture” of the language was set in place before those kinds of features were added. The C# ethos definitely prefers the verbose to the understated, the extremely explicit to the implied.This isn’t a terrible decision – like Java, it’s a language designed to be used by huge teams of enterprise programmers working on titanic projects where only a few people see the whole project scope and most coders are locked away in cubicles on the 18th floor working on isolated modules to be used by other coders they will never meet.That’s why C#’s obssession with visibility management and highly-specified behavior makes sense.C# is a language that’s all about apportioning blame: it forces everything to be very explicit so you know which of the 6000 drones working on your enterprise app to bleame when things go wrong.

In the context of a small game or a solo project, though, the Pythonic ethic of “we’re all adults here” seems more natural and productive.Besides, for hobby projects and one offs fun is a totally legitimate concern: making minigames is something that gets crammed into nooks and crannies left over by work, kids and keeping the house from falling down around my ears.So fun is a totally legit criterion for selecting a language.

And Boo is definitely more fun than C#.

Boo Who?

Like many Pythoneers, I’ve always nursed a secret hunger for the magical button that would take all my tight, laconic Python code and magically make it perform like a “real” language. Boo is not the magic button, but it’s a pretty good preview of what that might look like. As you can see from the code samples above, it looks and feels a lot like Python but under the hood is has similar performance and compile-time constraints to C#: in other words, Boo code can run as much as 20X faster than Python.

That’s what makes Boo so tantalizing. It is almost Python, but you can write Unity games, Winforms apps, or even cross-platform DLLS with it. Plus, since Boo runs on the same dotnet CLR as C#, it runs on any platform with the DotNet framework or Mono installed, so a compiled Boo program can run on Windows, Macs, or Linux boxes. There’s even an interactice shell so you can do one-offs and experiment, just like Python. But – unlike Python – you get the performance gains that come from a compiled, statically typed language.

Typing and the compiler are the key difference between Boo and Python. The a compiler makes sure that all of your variables, return values and method signatures line up and uses that knowledge to optimize the final runtime code for you. You can do this in Python:

fred = 1 
fred = fred +  1
print fred
# 2
fred = "fred"
fred = fred + " flintstone"
print fred
# fred flintstone

In Boo, however, you’ll get an error when you try to change fred from an integer value to a string:

fred = 1
fred = fred +1 
fred = "fred"
#------^
#ERROR: Cannot convert `string` to `int`

In old-school C#, this was made obvious that all variables and to declare a type:

int fred = 1;

In more modern C# you can use the var keyword to make the compiler guess what type you want based on the initial input: when you give it

var fred = 1;

it sees that fred has an integer in it, and treats fred as an integer from then on. If you assign the variable with the result of a method call or another variable, C# uses the expected type of that return value to set the variable type. Boo does more or less the same thing: it uses the assignment value to guess the type of a variable. You can specify it explicitly if you prefer by using the as keyword:

barney as string
barney = "barney"   #OK

The same syntax is used to specify inputs in methods and returns:

def bedrock (name as string) as string:
    return name + "rock"

def inferred_return_type(name as string):
    return name + "inferred"
    # if the compiler can guess the output type
    # you don't need to 'as' it

Once you get out of the habit of re-using variables with different types, this is usually not too bad: 95% of the time the inference “just works” and you can write code that focuses on logic and good design instead of worrying about the types. The other 5% of the time, however, is often very frustrating. It’s particularly tough when Boo’s Python-like, but not exactly Python behavior trips you up. Consider this little puzzle:

def sum (values as (int)) # expect an integer tuple
    result = 0
    for v in values:
        result += v
    return v

# works as expected for this case:
example = (1,2,3)
sum(example)
# 6

However it can be broken if your input list isn’t all of the same type:

example2 = (1,2,3,"X") 
sum(example2)
# ERROR: the best overload to the method sum((int)) is not compatible with the argument list '((object))'

That’s not entirely shocking: the compiler saw a mixed list in example2 and return and array of type object (as in C#, object is the base class of all types). So it is right to reject that as an argument for an int-specific function. Here’s where it gets odd:

#reassign
example2 = (1,2,3,4)
sum (example2)
# 10

This time the compiler “helpfully” casts that array to an array of ints because it can. This is not a crazy behavior, but it’s definitely going to raise some issues where test code works fine but production code contans bad values. The only way to be sure that things are what they seem is to force them into the right types at declaration time:

example3 as (int) == (1,2,3,4,5)
sum(example3)
# 15

example3 = (1,2,3,"one hundred")
#----------^
# ERROR: Cannot convert `(object)` to `(int)`

This example highlights both the usefulness and the limitations of type inference: If you want a statically typed language (and all the compiler tricks that make it speedier than Python) you do have to think about how to manage your types. There’s no way around it. If you’ve got enough C# experience, you can look at Boo as a neat way of writing speedy, statically typed code with less typing and more syntactic freedom - but if you’re looking at it from the standpoint of loosey-goosey Pythonista it can seem like a lot of hurdles to jump.

My (unscientific) impression is that a lot of people from the Python world come over to Boo and the familiar look of the code gives them a false sense of security. It’s easy to write simple bits of code until the subtleties of type management bite you in the behind, and then to give up in frustration when things seem to get cluttered and uptight.

It is, however, part of the territory: lots of other tools for speeding up Python such as Cython expect the same kind of attention to variable types: here’s a sample from Cython

def f(double x):
    return x**2-x

def integrate_f(double a, double b, int N):
    cdef int i
    cdef double s, dx
    s = 0
    dx = (b-a)/N
    for i in range(N):
        s += f(a+i*dx)
    return s * dx

which is just as finicky as C# or Boo.

For me, at any rate, spending more than a year doing C# as a regular part of work made fiddling around with Boo much easier and more productive. The type management hassles strike me as inevitable, or even natural, after a year of typing verbose C# variable types. On the other hand the cleanliness of the layout, the lack of extraneous punctuation, and the clealiness of list comprehensions and Python style loops never gets old.

While there are plenty of minor gotchas, and a few important high-level rules that can’t be forgotten, Boo development flows in with an almost Pythonic fluency. If you put in the time to figure out the type inference behavior and add the annotations, you can get code thats almost as flexible as Python and almost as performant as C# – which, for my kind of pet projects is a great compromise.

Boo-ty is in the eye of the beholder

TL;DR: I’ve gotten pretty fond of Boo. Above all, it serves me well for noodling around in Unity where the API is mostly identical but the logic is cleaner, shorter and easier to read than the same code in C#. Translating the docs is rarely more than trivial, and the very narrow scope of a typical Unity code chunk keeps me from any of Boo’s rough edges.

Another hurdle for many Pythonistas, though one which does not matter in the the context of Unity games, is the lack of the Python standard library. About 70% of what you can do with the ‘batteries included’ in Python can, however, be replicated using the dotnet Base Class Library if you’re running Boo on a Windows box (on Linux or OSX the percentage is lower: Mono has its own base class library but it’s not a complete replica of the one from Microsoft). For many tools tasks and projects, this is more than enough: you’ll be able to read and write XML, to decrypt JSON, to talk to an http server and so on although the function names and APIs will vary. I have to admit I prefer the Python toolkit to the dotnet one, which reflects the same bureaucratic mindset that I dislike in C#’s design, but it’s still a big, robust set of tools. You can also use anything that’s available as a dotnet DLL. Almost anything advertised as a usable with C# will work with Boo.

All That said, I’d definitely think twice before basing a commercial Unity project or a critical pipeline component on Boo. There does seem to be a small but measurable perfromance penalty compared to C# (the performance is, however, pretty much on par with that of UnityScript). More importantly, the Boo’s biggest weakness is documentation: with a small community and (from now on) no docs on the Unity site, finding your way around in the language at first is pretty awkward. The documentation is a sporadic, volunteer effort with some glaring holes – it doesn’t help that Google still sends you to the moribund Boo site on codehaus instead of the current docs, which are in a Github Wiki. The language is officially at version 0.9.4.9 and hasn’t incremented in a long time: it’s still getting commits from the original author and few other devs but it’s a much smaller project than, say, IronPython. In short, it’s a cool language that has not found it’s audience yet, and unless it does it will remain a niche player.

Still, it’s pretty cool too. If, after those caveats, it still sounds interesting, you’ll be relieved to know that Boo is not really ‘going away’: For the forseeable future, the language will still work in Unity, Boo, like C# and UnityScript, runs on Mono, much as Java runs on the JVM. Unity doesn’t distinguish between the source of Mono assemblies: you can still use Boo, and even more exotic dotnet languages such as F# (though not, alas, IronPython!) in Unity today. The only practical result of Unity’s decision to sunset Boo support is the disappearance of the Boo documentation from the Unity website – which , to be honest was rarely adequate – and the lack of a ‘create Boo script’ menu item. Dropping a boo script into your assets folder, however still creates runnable code, and it should continue to do so for the forseeable future.

There’s some question about how Unity’s new cross-platform compiler technology, IL2CPP will handle Boo. In principle, since it compiles the byte code produced by Mono rather than the original source, it too should work with any CLR language, be it Boo or F# or what have you. I’ve been able to compile Boo code to Unity WebGL games, which use IL2CPP, without obvious problems although I haven’t done anything like a scientific test. It’s not beyond belief that bugs which occur only in non-C#, non-UnityScript IL code may go unfixed. And, of course, it’s impossible to say what will happen after Unity 5 – technology, particularly in games, moves too fast for confident future predictions. However, It seems pretty clear Boo will be working in Unity for a while to come even though it is being demoted from “officially supported” status to the same kind of l33t hacker underworld as functional languages.

Boo-Curious?

If you’ve got Unity installed already, you’ve already got everything you need to play with Boo. Just create a text file with a “.boo” extension inside a Unity project and you can write Unity components in Boo. If you don’t have Unity, You can also download Mono directly, which installs MonoDevelop and Boo automatically.

If you’re not fond of MonoDevelop – an editor for which I have… mixed… feelings – You can write Boo using Sublime Text, which has a Boo syntax higlighting package and can run Boo compiles for you.

If you’re curious but don’t want to take the plunge, you can see the language for yourself and play with it online, using this interactive repl

The documentation – which (be warned!) is incomplete and not always up to date – is in the Boo Project GitHub wiki. There’s an older site at boo.codehaus.org which is tends to show up on the Google results but has mostly been ported to the github. In cases of conflicting information, the GitGub wiki is likelier to be right. There’s also a Google Group and a small pool of questions on StackOverflow

If you’re a hardcore type, you can also download and rebuild the source for the entire Boo language yourself from GitHub. Lastly, you might want to check out BooJS, a project which aims to compile Boo into JavaScript.

Wednesday, May 27, 2015

Moonrise Early Access is Live!

Moonrise is up on Steam now! It's a blast to see people logging in from all over and playing.



Now that the cat is out of the bag I'll try to post some tech postmortems on the pipeline and some of the interesting shader work we did.

PS If you get into the game, I share the username 'Alpha' with my kids.  So try sending friend requests, but if you get turned down it might be the kids instead of me! Don't take it personally.

Thursday, May 21, 2015

Saturday, May 16, 2015

spelchek

Spelchek

I’m planning one of the worst things that can happen to a TA: a big massive file move-and-rename operation. Much as I love my team, we have a poor record as a company when it comes to spelling, and it occurred to me that I’d like to at least have some degree of automatic spell checking on the names of the new files, folders and assets.
It turns out that there’s no good spell checker for Python that doesn’t come with some kind of extension module (BTW, I’d love to be wrong about that - if you know one definitely post it in the comments). PyEnchant for example is great, but it’s got 32-bit only Windows extensions that I can’t distribute without a hassle.
I did, however, find a very neat example of Python Power in a little post by Peter Norvig, who put together a simple spellchecker in a few dozen lines of plain, readable Python code and great explanations here.
I shamelessly borrowed his structure, with a couple of minor and not very creative tweaks. Peter’s original is built around Bayesian analysis: it guesses the correct word by looking at the relative frequencies with which variants show up – if ‘meet’ shows up 1000 times in your database but ‘mete’ shows up 5 times, that’s a good indication that ‘meet’ is the correct first guess.
Since I’m in a rush, I didn’t use that functionality very much. I scrounged around for as many sources correctly scored words. Unfortunately the only free source I could rely on turned out to be the venerable ‘GSL’ or ‘General Service List’, which has great data but only for about 2000 words (I used the version found here, by John Bauman as a the core of the list, and then scrounged the internet for other free sources. Since all of these were less common words than the ones in the GSL I gave them pretty arbitrary Bayes scores (4’s and 5’s for common words, 3’s for variants, plurals and participles). This is not sophisticated linguistics, but it’s close enough for horseshoes.
The result is up on github as spelchek, which I affectionately refer to as the cheap-ass spell checker.
It is hardly rocket science, but it does work. You can do something like:
   import spelchek
   spelchek.correct('vhicle')
    # 'vehicle'
or
    spelchek.guesses('flied')
    # ['filed', 'flied', 'flies', 'lied']
I would caution against using this for hard-core text work where perfect accuracy matters -- like database stuff, a customer-facing website, or a word processor -- since I did not go with high quality commercially or academically vetted word lists. I’m reasonably certain that there are some mis-spellings or oddballs in the 75,000 or so words I ended up with from various sources. Still, the module useful for my intended use, which is making sure that we don’t get things like ‘floder’, ‘frunishings’ and ‘vetegation’ (all of which shipped with in State of Decay, I’m sorry to admit).
As always, MIT licensed so go to town.

Saturday, May 9, 2015

Tech art for art directors

I see that the slides from my 2015 GDC talk, Technical Art For Art Directors is now up on the GDC Vault. paywall warning Enjoy – and feedback in the comments is very welcome. I can post the slides if there is interest, but if you watch the video you’ll see that they don’t really work without the verbal context. I defy anybody who wasn't there to make sense of this image, for example....
I’ll say in passing that – despite the lack of text in the finished presentation – I had great luck putting this talk together in Markdown using DeckSet, which let me do the first 90% of the presentation without ever opening PowerPoint or Keynote. The slides were just one big plain text file, which let me put them up on github so I could version them and track changes without getting bogged down in the finicky formatting nonsense that ultimately consumes all slide decks. I particularly liked this workflow because I could work as if it was text but still run it as a slide show very early on, which gave me a chance to get my patter down and listen to the words instead of getting hung up on formatting text. Plain text FTW!

Wednesday, May 6, 2015

Book Review

Computer books are a very hit-or-miss affair. I go through them like comic books but – like comic books – they rarely have much of an impact on me: I read along, nod, and ten minutes after I close the covers I’ve either compressed the whole thing into a crunched-down, dried-up mental summary or filed it away as something to look at for a very specific kind of reference.
It’s hard for a technical book to really engage a reader. Clarity, clean exposition and simply not peddling misinformation by accident or oversight are hard enough and many tech books fail to even hit those mares. Engaging style and memorable insights are even tougher to conjure up.
My personal favorite in this regard is The Art Of Readable Code by Dustin Boswell and Trevor Foucher. Unlike most tech books it’s not about technology, it’s about you. By which I mean it’s about how to approach programming problems with a pragmatic, long term view that’s about making your life more productive and less stressful in the long run by adopting smart practices. It’s not about whiz-bang tech - in fact, the book hops around between languages and problem sets and doesn’t really teach anything about programming per se; rather, it’s about writing: about making your code clear, expressive and humane. Programmer machismo often encourages us to write super-efficient alien artifacts which can be extremely impressive but which nobody wants to touch for fear of getting the Space Odyssey treatment. I’ve started forcing this book on all my new people: it’s better than a style guide because it teaches you things to think about, rather than giving you a list of things to memorize and follow.
(hat tip to @Pat Corwin, by the way, who turned me on to this one)
What got me thinking about tech books just now is the fact that I’ve recently found one that’s almost but not quite in that elite class of game changers. Effective Python, by Brett Slatkin, is more technical and a bit less fluent than The Art Of Readable Code. Still, it caught my eye because it seems like a great mile-marker on the road from “pretty good python user” to “real Pythonista.”
Effective Python does a great job of explaining and illustrating several features of the language which can be overlooked by folks coming from a scripting background rather than computer science. Many of the habits which separate the Python zen masters from the mere acolytes are explained neatly and clearly: the mysteries of the yield keyword, the evils of the mutable default argument, and the unfathomable allure of metaclasses are all dealt with clearly and in ways that don’t make you feel like a chump. Even better, the information is presented with a usable, practical slant: it’s neither an abstract set of rules to memorize nor a comp-sci nerdout about language features. Instead the book presents enough information for you to make good strategic choices: Perennial python puyzzlers like “list comprehension or loop?”, “Is it okay to use Try…except…else?” and “When to use propertites vs descriptors vs plain old fields” are all addressed in a reasonable, non-dogmatic way.
So, I’ll definitely be adding that one to the new-hire reading list in the future. Recommended!
PS if you haven’t checked out the Tech Art Bookstore in a while, I’ve added a bunch of new items over the last few months