Still on holiday blogging wise, as we ponder the year gone by, give a thought to these ancient nuggets from Robert Abel, back from when I was still in high school :)  Back when being a 'technical artist' probably meant 'being the guy who can hand adjust the laser in the slide-rendering machine that produces our graphics on film'

Who else remembers when this was cutting edge graphics? Happy Holidays!

The book I worked on last year is up for pre-order on Amazon! It was a long haul - congrats to Renee and all the other authors who helped get this giant zeppelin filled with hot air!

Techartists doin' it for themselves: A Python REPL in Unity

Last time, I sketched out the basics of embedding a Python intepreter into Unity.   That's cool emough -- but unless you're so desperate for Python that you're willing to script your whole application inside of triple quotes it doesn't have a ton of immediate applications.

So, this time out I'll sketch out how to build a simple script editor inside of Unity (at the risk of repeating myself, I'll just say again that the extensibility of the Unity editor is an incredible aid to game developers of all stripes, and to tech artists in particular -- it's pretty amazing you can hack in something this complex without source code access or esoteric C++ chops.

Prologomena

The basic strategy for this excersize is simply to create a Unity window with two panes - a 'history' pane  and a  'script' pane -- and before you ask, yes, it's just a ripoff of the Maya listener.

Before setting up the GUI, we need to cover the framework - the code that will keep the GUI stat and also set up the Python intepreter. In this example, you'll see a bunch of properties declared for the use of the GUI - notably _historyText and _scriptText, which hold the actual contents of the listener and the history pane.  The other notable feature is the same duo of _ScriptEngine, and _ScriptScope which we went over in the last post. If those terms don't mean anything to you you might want to follow that link before proceeding).

 using UnityEngine;  
 using UnityEditor;  
 using IronPython;  
 using IronPython.Modules;  
 using System.Text;  
 using System.Collections.Generic;  
 using Microsoft.Scripting.Hosting;  
 // derive from EditorWindow for convenience, but this is just a fire-n-forget script  
 public class ScriptExample : EditorWindow  
 {  
     // class member properties
     Vector2 _historyScroll;  
     Vector2 _scriptScroll;  
     bool _showHistory = true;  
     int _historyPaneHeight = 192;  
     string _historyText = "history";  
     string _scriptText = "script";  
     string _lastResult = "";  
     TextEditor _TEditor;  
     GUIStyle consoleStyle = new GUIStyle ();  
     GUIStyle historyStyle = new GUIStyle ();  
     Microsoft.Scripting.Hosting.ScriptEngine _ScriptEngine;  
     Microsoft.Scripting.Hosting.ScriptScope _ScriptScope;  

     // initialization logic (it's Unity, so we don't do this in the constructor!
     public void OnEnable ()  
     {     
         // pure gui stuff
         consoleStyle.normal.textColor = Color.yellow;  
         consoleStyle.margin = new RectOffset (20, 10, 10, 10);  
         historyStyle.normal.textColor = Color.white;  
         historyStyle.margin = new RectOffset (20, 10, 10, 10);  

         // load up the hosting environment  
         _ScriptEngine = IronPython.Hosting.Python.CreateEngine ();  
         _ScriptScope = _ScriptEngine.CreateScope ();  

         // load the assemblies for unity, using types  
         // to resolve assemblies so we don't need to hard code paths  
         _ScriptEngine.Runtime.LoadAssembly (typeof(PythonFileIOModule).Assembly);  
         _ScriptEngine.Runtime.LoadAssembly (typeof(GameObject).Assembly);  
         _ScriptEngine.Runtime.LoadAssembly (typeof(Editor).Assembly);  
         string dllpath = System.IO.Path.GetDirectoryName (  
             (typeof(ScriptEngine)).Assembly.Location).Replace (  
             "\\", "/");  
         // load needed modules and paths  
         StringBuilder init = new StringBuilder ();  
         init.AppendLine ("import sys");  
         init.AppendFormat ("sys.path.append(\"{0}\")\n", dllpath + "/Lib");  
         init.AppendFormat ("sys.path.append(\"{0}\")\n", dllpath + "/DLLs");  
         init.AppendLine ("import UnityEngine as unity");  
         init.AppendLine ("import UnityEditor as editor");  
         init.AppendLine ("import StringIO");  
         init.AppendLine ("unity.Debug.Log(\"Python console initialized\")");  
         init.AppendLine ("__print_buffer = sys.stdout = StringIO.StringIO()");  
         var ScriptSource = _ScriptEngine.CreateScriptSourceFromString (init.ToString ());  
         ScriptSource.Execute (_ScriptScope);  
     }   
 
    public  void OnGUI (){} // see next code snippet
 }  

As in the last example you'll also not that we're manually setting up sys.path to point at the directory where IronPython is installed, with a little extra code to make it portable  (dotNet assemblies can tell you where they live on disk, so it's a cheap shortcut to find your install directory).

The only thing in here that is really 'architecturally' important its this line:

         init.AppendLine ("__print_buffer = sys.stdout = StringIO.StringIO()");  

What's going on there is that we're replacing sys.stdout - which in ordinary Python points at the user's console - with a StringIO object.  StringIO mimicks a file -- and so does sys.stdout. By stuffing __print_buffer in there we are hijacking any calls to print that you might make in a script so we can print them out in our UI.  This is trick should be familiar to tech artists who need to grab the Maya console for nefarious purposes.

Unity GUI - the good, the bad, and the ugly

Unity's GUI toolkit is notoriously wonky, and you'll see as we go along that much of the energy here is devoted to working around it's limitations.  While we can go pretty far just using the basics, the is a certain Rube Goldberg quality to what follows. You've been warned.

First let's just layout out the actual drawing call - the OnGUI method of our window:

 using UnityEngine;  
 using UnityEditor;  
 using IronPython;  
 using IronPython.Modules;  
 using System.Text;  
 using System.Collections.Generic;  
 using Microsoft.Scripting.Hosting;  
 // derive from EditorWindow for convenience, but this is just a fire-n-forget script  
 public class ScriptExample : EditorWindow  
 {  
    /* snip... see previous example for the setup code... */

     public void OnGUI ()  
     {  
         HackyTabSubstitute ();  // this is explained below...

         // top pane with history  
         _showHistory = EditorGUILayout.Foldout (_showHistory, "History");  
         if (_showHistory) {  
             EditorGUILayout.BeginVertical (GUILayout.ExpandWidth (true),   
             GUILayout.Height (_historyPaneHeight));  
             if (GUILayout.Button ("Clear history")) {  
                 _historyText = "";  
             }  
             _historyScroll = EditorGUILayout.BeginScrollView (_historyScroll);  
             EditorGUILayout.TextArea (_historyText,   
                 historyStyle,   
                 GUILayout.ExpandWidth (true),   
                 GUILayout.ExpandHeight (true));          
             EditorGUILayout.EndScrollView ();  
             EditorGUILayout.EndVertical ();  
         }  
         // draggable splitter  
         GUILayout.Box ("", GUILayout.Height (8), GUILayout.ExpandWidth (true));  
         //Lower pane for script editing  
         EditorGUILayout.BeginVertical (GUILayout.ExpandWidth (true),   
             GUILayout.ExpandHeight (true));  
         _scriptScroll = EditorGUILayout.BeginScrollView (_scriptScroll);  
         GUI.SetNextControlName ("script_pane");  
         // note use of GUILayout NOT EditorGUILayout.  
         // TextEditor is not accessible for EditorGUILayout!  
         _scriptText = GUILayout.TextArea (_scriptText,   
             consoleStyle,  
             GUILayout.ExpandWidth (true),   
             GUILayout.ExpandHeight (true));          
         _TEditor = (TextEditor)GUIUtility.GetStateObject (typeof(TextEditor), GUIUtility.keyboardControl);  
         EditorGUILayout.EndScrollView ();  
         EditorGUILayout.BeginHorizontal ();  
         if (GUILayout.Button("Clear", GUILayout.ExpandWidth(true)))  
         {  
             _scriptText = "";  
             GUI.FocusControl("script_pane");  
         }  
         if (GUILayout.Button ("Execute and clear", GUILayout.ExpandWidth (true))) {  
             Intepret (_scriptText);  
             _scriptText = "";  
             GUI.FocusControl("script_pane");  
         }  
         if (GUILayout.Button ("Execute", GUILayout.ExpandWidth (true))) {  
             Intepret (_scriptText);  
         }  
         EditorGUILayout.EndHorizontal ();  
         EditorGUILayout.EndVertical ();      
         // mimic maya Ctrl+enter = execute  
         if (Event.current.isKey &&  
             Event.current.keyCode == KeyCode.Return &&  
             Event.current.type == EventType.KeyUp &&  
             Event.current.control) {  
             Intepret (_scriptText);  
         }  
         // drag the splitter  
         if (Event.current.isMouse & Event.current.type == EventType.mouseDrag)  
         {_historyPaneHeight = (int) Event.current.mousePosition.y - 28;  
             Repaint();  
         }  
     }  
   
 }  

If you're familiar with the dark arts of Unity GUI programming this should be pretty straight forward. If you're not, the key to understanding it is to remember that Unity uses what old-schooler's call Immediate mode GUI , in which each control gets evaluated as it is declared .  There's a case to be made that immediate mode is better for performance sensitive applications, but if you're used to the more typical (aka 'retained') mode GUIs in, for example, QT it's kind of an oddball way to write.

As each GUI element is drawn it reflects and then possibly updates the data that it relies on -- so, for example, we pass the string _scriptText  to the GUI.TextArea that draws the script listener pane - and the results of any changes are immediately passed back into _scriptText without the courtesy of a callback. This makes it tricky to manage complex state - as you run down the GUI draw, it's possible to hit a condition which changes a state and sends you back to the start! This makes it important to keep your state management code very clean and simple.

The one bit that may surprise people who do have some Unity experience its the line

_TEditor = (TextEditor)GUIUtility.GetStateObject (typeof(TextEditor), 
                                                  GUIUtility.keyboardControl);  

The TextEditor class is an undocumented bit of Unity arcana - it is a wrapper on the code that actually handles things like typing, selecting or cutting and pasting into a Unity text field.  It has methods for things like setting the cursor location and executing copy-paste operations. Unfortunately, being undocumented, it's tricky to figure out what to do with it -- in this example I'm only using it to preserve the selection position when I do something crazy - as you'll see in a moment.

Hacktastic

You probably noticed the enigmatic line

HackyTabSubstitute()  

which leads up to the tricky bit of this example  -- and the reason for my earlier hack disclaimer.

Tabs of course are the sine qua non for Pythonistas.  Unfortunately Unity catches the tab key before you can grab it, so it's impossible to 'type' a tab into a Unity text field.  After banging my head against this for a while, I settled on a pathetic workaround: just cheat and use the tilde key, which is above the tab key on most keyboards and doesn't have semantic importance in Python. Our new friend HackyTabSubsitute() makes sure that each time the GUI is drawn we replace and  backtick characters with indents and any tildes (shift-backtick) with dedents.  You can see how we also preserve the cursor position  by use of the _TextEditor.

   
         / // use ` and ~ as substitutes for tab and un-tab  
private void HackyTabSubstitute ()  
     {  
         string _t = _scriptText;  
         string[] lines = _scriptText.Split ('\n');  
         for (int i = 0; i< lines.Length; i++) {  
             if (lines [i].IndexOf ('`') >= 0) {  
                 lines [i] = "  " + lines [i];  
                 _TEditor.selectPos = _TEditor.pos = _TEditor.pos + 3;  
             }  
             if (lines [i].IndexOf ("  ") >= 0 && lines [i].IndexOf ("~") >= 0) {  
                 if (lines [i].StartsWith ("  "))  
                     lines [i] = lines [i].Substring (4);  
                 _TEditor.selectPos = _TEditor.pos = _TEditor.pos - 4;  
             }  
             lines [i] = lines [i].Replace ("~", "");  
             lines [i] = lines [i].Replace ("`", "");  
         }  
         _scriptText = string.Join ("\n", lines);  
         if (_scriptText != _t)  
             Repaint ();  
     }  
    




Assuming you can discipline yourself to use tilde instead of tab, this works like you'd expect, and it supports indents and dedents in any part of the line, which is handy for python edits.

Running the script

As so often happens, it's the damn GUI which takes all the work. The actual point of this whole excersize is to let you type in some python and execute it. If you trigger an evaluation - with the buttons or with command + enter, you'll fire the Interpret function:

 
     // Pass the script text to the interpreter and display results  
     private void Intepret (string text_to_interpret)  
     {  
         object result = null;  
         try {  
             Undo.RegisterSceneUndo ("script");  
             var scriptSrc = _ScriptEngine.CreateScriptSourceFromString (text_to_interpret);  
             _historyText += "\n";  
             _historyText += text_to_interpret;  
             _historyText += "\n";  
             result = scriptSrc.Execute (_ScriptScope);  
         }   
         // Log exceptions to the console too  
         catch (System.Exception e) {  
             Debug.LogException (e);  
             _historyText += "\n";  
             _historyText += "#  " + e.Message + "\n";  
         }   
         finally {  
             // grab the __print_buffer stringIO and get its contents  
             var print_buffer = _ScriptScope.GetVariable ("__print_buffer");  
             var gv = _ScriptEngine.Operations.GetMember (print_buffer, "getvalue");  
             var st = _ScriptEngine.Operations.Invoke (gv);  
             var src = _ScriptEngine.CreateScriptSourceFromString ("__print_buffer = sys.stdout = StringIO.StringIO()");  
             src.Execute (_ScriptScope);  
             if (st.ToString ().Length > 0) {  
                 _historyText += "";  
                 foreach (string l in st.ToString().Split('\n'))  
                 {  
                     _historyText += "  " + l + "\n";  
                 }  
                 _historyText += "\n";  
             }  
             // and print the last value for single-statement evals  
             if (result != null) {  
                 _historyText += "#  " + result.ToString () + "\n";  
             }  
             int lines = _historyText.Split ('\n').Length;  
             _historyScroll.y += (lines * 19);                  
             Repaint ();  
         }  
     }  
 }  

The heart of the whole business is just

result = scriptSrc.Execute (_ScriptScope);

which actually executes the contents of your script window.  As in Maya, we'll copy the evaluated text up to the history pane  (_historyText += ,etc).  If the event of an exception, we print out the exception into the history window as well, and also push a Unity debug message in case you aren't looking at your console window when the problem arises.  Finally, we check to see if the __print_buffer StringIO object has been written to duing the script execution  and copy it's contents to the history window too.

v. 0.1

Before starting the first of this pair of posts I was mostly just musing on how TA-friendly Unity is.   Building out a complete script editor is a perfect example of TA feature creep in action.

If you implement a script editor using the hints here you'll quickly see what's not there things like cut and paste, syntax highlighting, execution of selected text only and support for external files, just to name a few things that would be worth having.  And I should mention that this is demo code, it's not the sort of thing I'd want to turn into a critical path tool without further work.

Even so, it's been a useful little project.  In this holiday season it's taught me to appreciate my blessings - like how many nice little touches you get with a modern text editor. I'm even feeling more charitable towards the Max and Maya script listeners, since I've walked a mile in their sad patheric old worn out shoes.

All that said though, it really is pretty fricking neat that you can add a completely new scripting language to the Unity editor in a couple of hours -- and save your self tons of future time by adding cheapo scripts to automate tedious tasks that aren't worth 200 lines of C# curly brackets.

At some point I'll address the most obvious failings - lack of cut-n-paste is the clear winner! - but first I want to see about implementing the console in a more flexible GUI - for example, I could pop up a WPF window, or maybe even something truly funky like an in-browser python console..  In the mean time, if anybody takes this further I'd love to hear about it.

Python in Unity - minor correction

Going over the last post about Python + Unity, I did a clean install to make sure the steps I was describing were working correctly and it reminded me about an inportant bit I've left out: how to get the Python stdlib into your Unity IronPython

Because Microsoft was the sponsor of the original IronPython project, versions that Microsoft released (including the 2.6.2 that i linked to in the last post) don't include the stdlib, which comes with it's own license that clashes in some mysterious way with MS's licensing terms (even though both MS and the Python Foundation are giving the stuff away for free... *sigh*). So to dance around that, they did not include the stdlib  -- the 'batteries included' -- with base install.

The remedy is simple - grab a copy of the regular python stdlib from a python 2.6 series install and copy it into the /Lib folder next to the location of your IronPython DLL.  I found it simplest to grab the Python26.zip folder from my Maya install and to expand that into the folder.  I did leave the 3 or 4 files that IPy had installed there on its own intact, I believe -- on pure intuition -- that they are different from the corresponding files in the standard python lib.

Caveat emptor

FWIW, this is a good place to point out that some small percentage of stdlib modules don't work under IronPython (an unfortunate example being the handy csv module for reading comma-delimited data files).  AFAIK there is no authoritative list of which modules do and don't work under Ipy. The good news is that, for this application , there is almost always a dotnet native solution to use as an alternative without having to install anything else.

Embedding IronPython in Unity = Tech-art hog heaven

Update 6/2/2015 If you are relatively new to Unity, and you're here because you're looking for ways to work in Python rather than C#, you may also want to check out this 2015 post about Boo - the very obscure, but very Python like language in Mono that lets you write Unity games with fast, compiled code by almost-Python syntax. You don't work long in games without griping about your engine, and I've got my share of complaints about Unity. But I have to admit that it's the best toy in the world for a classic tech-art geek personality. Unlike bigger, more powerful AAA engines, Unity lets you get in under the hood really quickly. The editor environment is extremely extensible - you can add not just dialogs and buttons but 3d widgets and diegetic UI .

When I first got my hands on Unity I was a bit disappointed to note that, unlike Maya, it doesn't include a built in interactive console environment. The console is a wonderful thing for a TA - it's great for printing out data, lightweight automation of the "find everything in the scene named 'foo' and rename it to 'bar'" variety. So, I thought, is there some way to get this into Unity?  The fact that one could even ask it is a tribute to how flexible Unity is - and as it turned out it was not only possible, it wasn't too hard.

You got Python in my Unity!

To start with a console needs some kind of scripting language. Not surprisingly, I wanted to see I could do it in python. Fortunately, this is ridiculously easy thanks to IronPython, the dotnet flavor of Python.  IronPython  runs on dotnet, and so does Unity - so it's not tough to plug IronPython into Unity directly.  Here's how:

1. You need a verstion of IronPython that will run on Unity's version of Mono , which as of this writing (Unity 4.22 , late 2013) is version 2.6.  By a happy coincidence of naming, that points you at IronPython 2.6.2.  (I've tried later versions but without much luck).
2. Locate the IronPython dlls and the IronPython stdlib in the zip file. You will need
   
   • IronPython.dll
   • IronPython.Modules.dll
   • Microsoft.Scripting.Core.dll
   • Microsoft.Scripting.dll
   • Microsoft.Scripting.Debugging.dll
   • Microsoft.Scripting.ExtensionAttribute.dll
   • Microsoft.Dynamic.dll
3. CORRECTION : 12-22-2013 If you want access to the Python stdlib, you'll also need to grab a copy of the python 2.6 /Lib folder -- this is not distributed with IronPython 2.6.  I unzipped the Python26.zip file from my Maya bin directory into the /Lib folder, taking care to leave the handful of IronPython files already there
4. Copy all of the above into an Editor/Plugins/Resources folder in Unity. If you're not sure what that means:
5. • Naming a folder Editor tells Unity it only runs in the editor, not at runtime (IronPython won't run inside Unity on IOS devices, since those never run editor code)
   • Naming it a folder Plugins tells Unity to load dlls from it
   • Naming a folder Resources makes sure it loads before scripts are compiled
   For our application we need all three, hence "Editor/Plugins/Resources/..."  You can stick that whole thing into a top level folder for cleanliness if you want. Note the path names in this example:
   
   
6. Restart Unity and open up MonoDevelop. If you check the Assembly-CSharp Editor info in the Solution panel you should see all of your IronPython DLL's are referenced therein:
   
   
   

Once you've verified that the DLL's are in place, its time to test them.  Hosting an IronPython session in another app is much simpler than it sounds. The best resource for how it works is Michael Foord's Voidspace site (his book on IronPython is a great resource if you plan on going far with this , btw) . However in overview the process is pretty simple (

1. Create a ScriptEngine. This is the actual Python interpreter.
2. Create a ScriptScope ( Microsoft.Hosting.ScriptScope) This corresponds to the global namespace of your interpreter - much like the interpeter namespace in Maya
3. Create  a ScriptSource ( Microsoft.Hosting.ScriptSource) using some text you've entered or composed.
4. Execute the script
5. Rinse & Repeat. You are in business.

Script hosting in practice

Here's an example .cs script that demostrates the process. Put it in an editor folder so that it can access the Unity Editor assembly (it's probably a good idea to keep it in the editor folder where you have your plugins/resources folder for cleanliness).

 using UnityEngine;  
 using UnityEditor;  
 using IronPython;  
 using IronPython.Modules;  
 using System.Text;  
 // derive from EditorWindow for convenience, but this is just a fire-n-forget script  
 public class ScriptExample : EditorWindow {  
     [MenuItem("Python/HelloWorld")]  
     public static void ScriptTest()  
     {  
         // create the engine  
         var ScriptEngine = IronPython.Hosting.Python.CreateEngine();  
         // and the scope (ie, the python namespace)  
         var ScriptScope = ScriptEngine.CreateScope();  
         // execute a string in the interpreter and grab the variable  
         string example = "output = 'hello world'";  
         var ScriptSource = ScriptEngine.CreateScriptSourceFromString(example);  
         ScriptSource.Execute(ScriptScope);  
         string came_from_script = ScriptScope.GetVariable<string>("output");  
         // Should be what we put into 'output' in the script.  
         Debug.Log(came_from_script);              
     }  
 }  

When it compiles you'll get a menu items that activates the script.  When you hit it you should get a debug printout in your console like so.

Like the Maya python interpreter, you need to import the appropriate names so that you can get to them in scripts (it's always rather boggled my mind that Maya's own interpreter requires you to import cmds or pymel every single freakin' time.  IronPython lets you import dotnet assemblies as if they were python modules, and since Unity and the Unity Editor are dotnet assemblies you can get access to the entire Unity environment just by importing them into your interpreter.

First, we need to load the assemblies to make them available to the intpereter itself.  In dotnet land that's done by loading an assembly.  Once an assembly is loaded, it can be imported using typical Python syntax

 [MenuItem("Python/HelloWorldRuntime")]  
 public static void UnityScriptTest()  
 {  
     // create the engine like last time  
     var ScriptEngine = IronPython.Hosting.Python.CreateEngine();  
     var ScriptScope = ScriptEngine.CreateScope();  
     // load the assemblies for unity, using the types of GameObject  
     // and Editor so we don't have to hardcoded paths  
     ScriptEngine.Runtime.LoadAssembly(typeof(GameObject).Assembly);  
     ScriptEngine.Runtime.LoadAssembly(typeof(Editor).Assembly);  
     StringBuilder example = new StringBuilder();  
     example.AppendLine("import UnityEngine as unity");  
     example.AppendLine("import UnityEditor as editor");  
     example.AppendLine("unity.Debug.Log(\"hello from inside the editor\")");  
     var ScriptSource = ScriptEngine.CreateScriptSourceFromString(example.ToString());  
     ScriptSource.Execute(ScriptScope);  
 }  

Running that one generates another console message - but this time from inside the script!

Next time: building a console

That may not seem like much, but in reality it's a big deal. You've got a working script interpreter running in Unity now, with all the power of Python and access to the innards of the Unity environment.  What remains is to build a decent interactive environment. If you're content with something barebones, you can whip up a simple UI to allow you to type code into a text field and see the results in another text block.  That may be enough for some purposes (hell, half the 3d packages on the market do little more than that).  However a more featured editor is a little tougher and involves a  little hacking to work around the limitations of Unity's GUI text handling, which makes it a bit involved for a single post. I'll save that one for next time

Enter, the Chimaera

In addition to grinding my way through the innards of Maya, I spent a large part of my (sadly wasted) youth in graduate school studying ancient history with a sideline in the Italian Renaissance.   Even though  I don't get to use much of this information on a regular basis, I do occasionally get nostalgic for the random bits of weird and wonderful information you can only find in the dim recesses of a big research university library.

One of the greatest treasures in my personal museum of useless information is the genre of Wunderzeichenbücher. The German means "wonder sign books," and it refers to 16th and 17th century genre of illustrated guide to fantastic creatures and strange occurrences. The Wonder Books are a fascinating mix of early scientific investigations, second hand traveller's tales, and just plain wacky stuff that is hard to explain as anything other than as the Renaissance equivalent of a SomethingAwful Photoshop Friday.

My particular favorite is the Monstrorum Historia  (1642) of Aldovandrus, which is chock full of amazing woodcut drawings of every kind of crazy hybrid creature the Renaissance could dream up, from chimaeras to hippogriffs to monopods and blemmyes.

Besides the awesome graphics (how's that for retro, pixel kiddies!)  I've always found these images compelling because Tech Art is the domain of crazy hybrids.  We're artist/programmers, animator/logicians,  scultpor/scripters and graphics-nerd/visionaries. So adding dogs heads or chicken feet to our other attributes doesn't seem like too big a stretch.

So, with that an excuse I give you the Chimaera, my personal totem animal

As an aside I will point out that this one is not exactly pulled out of some Renaissance dude's posteriore.   The artist must have seen this amazing Etruscan bronze:

You can even see how the woodcut artist included the stump of the tail. When the 2000 year original was dug up in 1553 the tail was missing.  It was restored in the 1800s.  Bug fix! 

Python in browsers

In case folks are wondering about the tantalizing prospect of running python in browser (as an alternative to or escape from JavaScript) this is a great roundup of where things sit as of now (late 2013)

...And I thought we had it bad...

Bitching about all the random stuff you have to wire together is the key rituals of the tech-art faith.  You inherit all sorts of crazy decisions from Max, Maya, plugin authors, game engine teams, and random tools you find lying around and then some how have to lash it all together into a Rube Goldberg contraption that (hopefully) hides the wackiness from your users

Of course, this is a huge pain in the butt

But I never realized that we have it easy. Until I sat down to write a web app.

After lots of fruitless searching for commercial asset management system that would help us manage versions, review status and so on for our assets I finally broke down and decided to write my own. While Shotgun has gotten much slicker over the last year and Tactic is both open-source and Python-based, neither works well with Perforce (Tactic has been promising P4 integration for over a year, with no public release that I've been able to find) and both are very heavy on the kind of features you need to handle a huge Hollywood-style team with hundreds of Anonymous Drones.  Our tiny team doesn't need pretty gantt charts and time-stamped hourly activity reports.

Web Dev 0.1B

The basic plan of attack is quite simple. Luckily the problem that usually damns these kinds of setups -- making sure that the database and the game are actually in sync on things like, say, texture size or polycount -- isn't a big problem since all of that is, blessedly, available in Unity with minimal work. The real goal of this system is to make sure people know what's placeholder art, what's work in progress, what has bugs, and so on..  To that, I've got a MySQL database sitting in our data center and some tools in the Unity editor that can collect and forward info to the database.  In Unity I've also hacked the editor so that the current status of the asset is displayed over the asset thumbnail in the project view so the users can see where they stand without going out to another tool.

I love Unity to death, but even I can't convince myself to like Unity's built in procedural GUI language; it's clunky and formulaic -- and because it's orientation is so procedural it is extremely slow for anything with lots of controls.  Big Data -- spreadsheets, long lists, or fancy MVVM views are just not happening in the Unity Editor UI layer.  So -- to return at long last to the original seed of this post - I decided to write a web app to provide the producers and artists with the kind of overview data they'd need to see how things were progressing all across the project, rather than just the status of individual assets.

The Devil You Know

I did a bunch of research trying to figure out how to do this without having to start a whole new career as a web developer. In all honesty I was mostly hoping to avoid having to escape from the comfy confines of Python and C# (I can't really call SQL 'comfy' but at least it's familiar).  In particular my limited experiments with JavaScript (the real kind, not the beefed up Unity version) have been so uniformly unpleasant that I was desperate to avoid tangling with it at all costs.  I played with browser Pythons such as Brython , Skulpt, and the IronPython DLR host.  There's also projects like PyJS and RapydScript, which compile Python (or in the case of RapydScript a 'pythonalike') to Javascript.

All of which are really cool - but none of which are tightly integrated into the bewildering complexity of modern web development. If all I wanted to do was programin the browser, I could stick with Python, take the speed hit and be done.  After more obsessive reading, however, I sadly concluded that the HTML-verse too crazy a place for any tacked-on, ex-post-facto solution -- even one as fricking cool as a complete Python interpreter written in Javascript.  The real hard part is not the programming, which is mostly UI level stuff -- hide this! highlight that! -- it's controlling the infinite number of stylistic choices that are involved in designing a functional and attractive layout in HTML.  The more I looked at it the more I felt like I needed to follow the herd and do what the 'real' web developers do -- which means not only learning HTML itself, but also CSS -- the language that defines style sheets for different graphic elements -- a language which is not the same as HTML - or the same as Javascript. ( Remember I said tech artists have it pretty good compared to web programmers? Imagine having to learn Mel, Python and Maxscript all at the same time just to get started as a TA.  There but for the grace...)

This could be worse. Thank heavens a friend at work pointed me at Bootstrap, which is Twitter's clean and relatively simple to learn web gui framework. It lets you create nicely formatted modern looking pages without knowing too much about what really goes on in the tangled jungle of curly braces that define your CSS.  It still takes some doing to handle the GUI glue "do this when I push the button" stuff but its less annoying than, say, QT or WPF.

 It's still incredibly annoying to write scads of parentheses and curlies for even the most trivial task

numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
evens = [x for x in numbers if x % 2 == 0]

Is aesthetically and morally superior to

numbers=[1,2,3,4,5,6,7,8,9,10];
evens=new Array;
var _$tmp1_data=_$pyva_iter(numbers);
var _$tmp2_len=_$tmp1_data.length;
for(var _$tmp3_index=0;_$tmp3_index<_$tmp2_len;_$tmp3_index++)
    {
    x=_$tmp1_data[_$tmp3_index];
    if((_.isEqual((x % 2),0)))
        {
        evens.append(x);
        }
     }

But on the other hand you get lots of nice graphical goodies to soften the pain. I'm experimenting with CoffeeScript, which has Pythonic brevity and clarity -- but it's not Python and if you go in thinking it is (as I have on a few occasions) you'll get your knickers in a twist.

The backside

Fortunately, the server side code is actually much less of a hassle than I had feared. Given the nature of the problem -- selecting and filtering data from a database and then spitting it out to users -- the natural choice for me was Django, which has several advantages:

1. A great object-relational mapper (ie, you get to program with nice objects instead of gnarly SQL queries, cursors, and rows
2. A decent templating language. When I wrote the first Python build server for the lab I did this all by myself with string.Template, and it was a pain in the pants.
3. Hardly a curly bracket in sight
4. Lots of documentation and tutorials on the web, which is comforting for the terrified novice (= me).

After all of the consternation I went to researching web client stuff, this was a snap choice. And then the fun began.

The server is going to be running on a Mac (the tech art mac that we use for builds) so it will be ready to migrate to a Linux host later.  No problem, I use a mac laptop and do a fair amount of python on the mac already.,  So we want to install Django on the mac.

We need to install easy-setup (not sure why that doesn't come with the default mac python install)
....So we can install pip
.......So we can install Django
...... which needs MySQLdb
.......... which needs MySQL
..............which doesn;t install correctly on OSX the way it is supposed to
.................so we need to install a Ruby package manager that can install MySql and MysqlDb
....................so we install Homebrew
.........................which needs Xcode
............................which requires you to accept the Xcode license after updates
..............................and then to install the Xcode commmand line tools
...........................so we can properly rebuild and install MySQL
........................so we can install MySQLdb
.....................so we can point Django at MySQL

Eh voila! We're done!  It's a good thing Mac's are the elegant operating system for people who don't go in for all that techie stuff.

Evidently. though,  this sort of thing is just business as usual for the real web developers. I guess I'll stop bitching about Autodesk's lousy license manager from now on.  The theme of this project is learning to appreciate just how good we've got it :)

So, with all that done, I grabbed a trial of PyCharm . As an IDE I find it a bit clunky, but it's got good Django integration. Off to the races

SQLitis

In this next bit I'm going to touch lightly on how Django does and doesn't make it easy to work with a database using familiar python techniques. The TL;DR is that it works pretty well, with gotchas. If you're unfamiliar with SQL terminology, databases, etc, you may want skim (and if you're an expert, you'll probably roll your eyes).  This isn't a how-to or tutorial - there are great ones out there.  It's just a quick glance at something that many TA's may find useful in coming years.

The nice part of the Django workflow is that you can get Django to generate your data models for you by analysing your existing database.  Running

python django-admin.py inspectdb

On your database will spit out the Python class definitions for your data models to stdout, where you can cut and paste them into your code.  I already had a decent database schema (I took most of my n00b lumps with that sort of thing building a bug tracking database for SOD) and getting the object models built took about half an hour from 'starting to look at the docs' to 'done'. The little GUI that PyCharm gives you for django-admin is particularly helpful here.

There are a couple of hitches.

Django could be smarter about tables which don't use numerical ID columns as their primary keys -- which bugs my inner Joe Celko -- and I ended up having to add numeric ID columns to some of my tables in order to placate Django. Evidently there are ways to get in under the hood an tweak the actual sql that is emitted from your models to get around this but I've got other things to worry about.

It's also a bit tricky to get exactly the info you want.  For simple queries of the get everything in this table named 'foo'  variety Django works fine.  You can even do SQL joins, where you create a new pseudo table by picking entries from a set of tables with common keys - in Django you can trace back through a chain or table to table relationships just by following a  set of properties on your data objects.  In my case I have a table of assets (which describes the assets' names, location, and so on) and another table of asset types which is basically an Enum describing all the asset types - models, textures, animations and whatnot.  In SQL you'd connect them up using a foreign key   to make sure that all the links were valid, and then use a JOIN statement to produce a combined result.  For example I have two tables somewhat like this:

Assets table Asset String Thumbnail Image Path String Type Int

Asset type table ID Int Description String Name String

And if I want to grab a combined view that shows the assets along with their types I might do something like

SELECT assets.asset, assets.path, asset_types.name
FROM assets
INNER JOIN asset_types
ON asset_types.id = assets.type

Which seems kind of funky and 1970's to read, but it's a very powerful technique - this would give back all of the assets and their types without any tedious loop writing on the receiving end.  Django handles this for you nicely in its object model: as long as I made sure to let Django know that the 'type' I want comes from the asset types table and the 'asset' comes from the assets table, I can just grab it with

asset_list = Assets.objects.all()  # collect all the assets
for each_asset in asset_list:
    print asset.type.name


This is fine and dandy for many cases, but it does have some pitfalls. One of the great things about joins is that you can use them to substitute for conventional if-then logic. For example, in my case the I have a table of 'history' entries, which record changes in status (say from 'ready for review' to 'approved') along with times, dates and comments so we can see when things were OK'd for use in the game.  To get the current status of the object  I join the history table to itself:

SELECT
h1.idhistory, h1.asset, h1.changed
FROM (history h1 left join history h2
        ON
       (h1.asset = h2.asset and  h2.changed > h1.changed)
       )
        WHERE isnull(h2.asset)
ORDER BY h1.asset
The LEFT JOIN tries to run all combinations of dates in h2 against matching assets/date combinations in h1.  In all but one cases these will succeed (since the last date will be larger than all but the latest date). By looking for the failed join (with "isnull (h2.assets)") we can grab the latest entry.  This seems like a lot of work but it's way faster than a conventional loop-through-them-all-and-pick-the-latest-one approach; plus it is done in the server, in highly optimized C code, instead of on the client in (slow) Python.

Unfortunately this is a tricky one to get right with Django - at least, in my current (4 days and counting) aquaintance with Django.  I ended up having to work around it by creating a SQL view - basically a stored query - and grabbing just the id's of the history entries I wanted from their, and then doing a separate query to get the 'real' assets using the object technique I outlined earlier. Works fine but it is two server hits where one would do. C'est la vie.

Last words

So, it's been a mixed bag getting this thing off the ground.  Django is pretty cool - one of the nice things about working with web tech is that itt has several orders of magnitude more users than typical TA tech - the  speed with which things evolve in this space is pretty dizzying if you're used to the typical lackadaisical pace of tech advances from ADSK.  Web tools are clearly going to be a Big Thing in the coming years - the ease of distribution and graphical panache you can get makes the old school batch-file and script world seem pretty pokey.  On the other hand, web tech  really is a Rube Goldberg machine - it's made up of technologies that have vastly outgrown their original purposes, and is plagued by competing vendors and developer faddism.  Javascript is icky, and the idea of having to learn at least 3 different languages ( JS, CSS, and HTML) just to get anything done is pretty irritating.

Fortunately the core of the TA personality is pure stubbornness.  We have our jobs because we don't like to rest until we figure out what the hell is going on. It's a very, very valuable trait to have if you're getting into web tools :)

Late for the revolution

Since the demise of the late, lamented Game Developer Magazine, I've been wondering if it's time to find some other writing outlet. Most of my spare writing energy for last year was consumed by shipping State Of Decay and working on a book project for Focal Press.

Now that both of those are safely in the past I'm going to spend a few weeks noodling on this to see if it's congenial.  I'm working on another book outline at the moment, and as topics of interest come up - or as I need to think out loud about them - i'll try floating them here to see how they look in print.  I also plan on ruminating on knotty tech-art questions, programming problems, and maybe throwing in the occasional classical allusion to make myself look smart.

No cat pictures though.  Not my bag.