We've Moved


The blog has been retired - it's up for legacy reasons, but these days I'm blogging at blog.theodox.com. All of the content from this site has been replicated there, and that's where all of the new content will be posted. The new feed is here . I'm experimenting with crossposting from the live site, but if you want to keep up to date use blog.theodox.com or just theodox.com

Friday, November 16, 2018

Pythonception!


Pythonception!

Posted on Thu 15 November 2018 in blog


Even though serious Maya tools development has been done in Python for almost a decade now, Python still doesn’t get a lot of love from Maya when the application starts up.

You can run a userSetup.py, which isn’t bad — but it isn’t very flexible: you’re going to get one, and only one chance with userSetup, which makes things a lot more complicated if you have support a bunch of possible configurations, like different projects or different tool setups for different specialists; you don’t want to either physically swap userSetup files all the time, or to try to make one that’s complex and flexible enough to handle all your needs. And, of course, a lot of users have their own userSetups and they get mad when you want to take them away.

Unsurprisingly. many teams end up with dedicated launchers — either dedicated applications or just .BAT files — whose job it is to configure maya on startup. For example you might keep parallel userSetup/code combinations in different locations and then launch Maya with an .BAT that manipulates sys.path or PYTHONPATH to get the right version to the top of the list. These kinds of systems work reasonably well, although they also complicate your support structure — when you’re trying to add a new artists to the team you need to train them out of just launching Maya from the applications menu, and if you’re supporting outsources or partner studios you need to distribute the launcher infrastructure as well as the code that does the actual work.

If you stick with the simple stuff, on the other hand… well, BAT files are just nasty — they’re the computing equivalent of truckstop corn dogs that have been rolling under the heatlamp since 1984.
In the words of the great Raymond Hettingerthere’s got to be a better way. A way you can distribute a toolkit that’s:

Easy to activate — It won’t require me to fire up a different program just to use the one I want to use
A good citizen — It doesn’t force me to reorganize my Maya install (particularly if I’m an outsource who might only be using this toolkit for a few weeks!)
Install Free — It doesn’t demand a separate installer or a plugin or anything else I don’t already have.

… but at the same time it’s

Fully functional — you can’t compromise the depth of what you’re delivering just in order to make it easy to deliver.

The first three are actually surprisingly easy. After all, Maya has always has it’s own dedicated script file type — you can always double-click or drag-and-drop a mel file and it will just work. No installation, not configuration, not even any special rules about where the file has to be. You can mail somebody a MEL file and they can just run it from their downloads folder.
Alas, it works in MEL. Which is to say, it’s going to be an uphill battle to make it hit that last bullet point and be fully functional. And how much power can you really pack into a single MEL file?

But, as Ray would say, there is a better way.

If you’ve ever had to write one of those BAT file launchers, you may have seen something like this:
maya.exe -c "python(\"import sys; sys.path.insert(0, 'c:\\path\\to\\mayatools'); import mayatools; maytools.run()\")"
That’s telling Maya to run a MEL command that in turn tells Python to add a folder to the path and then run. Conceptually simple, if ugly as sin; and fine enough for a one liner. Not, however, the stuff of which great programs are made — those string escapes alone are enough to make one want to take a hot shower.

You can make things a bit better if you delegate the work to an actual Python file; that gets you down to something like:
maya.exe -c "python(\"execfile 'c:\\path\\to\\mayatools\\mayatools.py')\")"

One easy thing to do from here is simply to convert these BATs to MEL files — after all, they’re really just calling MEL anyway. That gets you doen to one less layer of string escapes (you’re still stuck quoting all the python, alas) and it will let you run Python from anywhere. In many scenarios this is a good, low-maintenance way forward. You can create dedicated MELlaunchers that can fire up Python as needed without actually mucking around in the depths of your Maya install directory, without dedicated launchers, and without even writing more than the teensiest smidgen of MEL. Heck, this even works with drag-and-drop!

However it’s not perfect. The main weakness of something like this — and with BAT files, for that matter — comes when you start to wonder where, exactly, the relevant python files are. As soon as the logic gets more complex than a hard-coded path name you’re suddenly back in the business of trying to write a complex algorithm in MEL without all of the nice tools you get in Python. Sometimes you need the power of Python to unleash the power of Python… it’s a sort of snake-eating-its-own-tail situation.

To escape from that conundrum you really just need to figure out a way to squeeze the Python logic into thae executable shell of that MEL file. Unfortunately, with all those stupid string escape codes, that’s not a trivial task. Unless — let’s get really meta here for a moment — you use the power of Python to get MEL to help you unleash the power of Python.

Waaaaat?

Using Python to make MEL to make Python…?” Pythonception!
The problem with jamming complex code into your MEL is all the string escaping. But Python comes with built-in tools for dealing with string-safe encoding of arbitrary data — including strings. The base64 moduie is used to encode aribtrary data for transmission over the internet as blocks of characters. For example you can take a string like

example = r'This is a long string, including punctuation and "difficult" characters like quotes\slashes'
and turn it into something that will not require special escapes:

import base64
print base64.b64_urlsafeencode(example)
#  VGhpcyBpcyBhIGxvbmcgc3RyaW5nLCBpbmNsdWRpbmcgcHVuY3R1YXRpb24gYW5kICJkaWZmaWN1bHQiIGNoYXJhY3RlcnMgbGlrZSBxdW90ZXNcc2xhc2hlcw==

Base64encoding doesn’t use the full ascii character set, and the b64_urlsafeencode in particular avoids slashes — so you can be sure that your big blob or random characters will not cause MEL to fritz out.
This kind of packing gives you the means to take a whole Python file — as much code as you need to find your install directory, or download updates from the network, or sync up to perforce — and jam it right into your MEL file. Here’s about all you need to compile a Python file to MEL:

def python_to_mel(pythonfile, melfile):
    with open(pythonfile, 'rt') as reader:
        py_code = reader.read()
    encoded = base64.b64_urlsafeencode(py_code)
    py_to_mel = "import base64; _py_code = b64_urlsafedecode('%s'); exec py_code"  % encoded 
    with open(melfile, 'wt') as writer:
        writer.write('python("%s");' % py_to_mel
Now, when you double-click that Mel file the original code — safely stashed away inside that block of gibberish — will be turned back into code and run as if you’d typed it into the script listener.

Incidentally if something goes wrong you can see the actual text fo the Python code in Maya — in the example above it’s stashed in variable called _py_code, which will be in the global namespace (again exactly as if you had typed it in the listener).

While you can certainly achieve the same end in different ways, this method as a lot of appeal, particularly because it gives you a maximal amount of power without any infrastructure on the receiving end. You could email a MEL file like that to anybody and they could run it as is. Anything in the standard library — including important things like the ability to download an update from an http server or to call out to a source control program — can now be done from a vanilla MEL file. It’s almost enough to make you like MEL again… it’s like the good old days of 2005.

Of course you’ll quickly run into the other big hassle of Maya startup, which is setting up the rest of your Python environment: the endless old struggle of PATH and PYTHONPATH and the rest. But don’t worry! MEL has an answer for that as well… but it’s an answer that will wait until next time.

Sunday, May 6, 2018

State of Decay 2

SOD 2 box
It’s no accident that there have not been a lot of blog entries in the last several months, but hopefully life will be returning to normal soon. I took the time to clean up my site generator and simplify the site theme, though I expect I’ll probably waste a bunch of time fiddling around over the summer.
In any event, the game is looking good — although I’ve got one HUGE bug I must squish before we go out the door! — and it’s been a real blast to bask in some of the reactions from people who’ve seen the game.
The last couple of years have been kind of a wild ride, professionally — I certainly never thought I’d spend more time staring at C++ than anything else ! — but at least, now that the dust is settling, there should be time for blogging again. With all the crazy stuff I’ve had to learn I’m sure there will be lots to chew over, too.
In any event, the press embargo will lift on May 1st, and opening day will be May 22d. In the meantime, here’s a little video teaser from the company stream with a special shout out to my colleague Jeff Sult, who helped me ferret out a truly insidious bug in the Xbox shader compiler.

PS If you’re wondering… sometimes the Xbox decides to ‘optimize’ exponent operators to use a logarithm instead of a multiply… Which is very nice unless your exponent somehow goes negative, in which case the result is a NAN which can really mess up your PBR G-buffers. Sigh…

Thursday, August 24, 2017

mGui available on CreativeCrash

If you’ve been considering taking mGui for a spin, but don’t want to take on the work of cloning the whole project from GitHub, we’ve uploaded a copy of mGui 2.2 to CreativeCrash as a (free) single file download. All you need to do is download the zip file from here, drop it into a folder where you Maya is looking for python scripts, and you can start writing gui’s with the neat mGui syntax, multicast delegates, and all sorts of other features. You can also check out the tutorials in the wiki for some additional help getting started.
If you’re interested in learning how things work under the hood — or if you’re interested in contributing to the project — you’ll probably still want to grab the GitHub version first. However if all you want to do is write some tools quickly, the zipped up version is an easy way to get started.
PS: We’re always looking for new contributors to the project!

Sunday, July 30, 2017

MGui 2.2 release is live

The latest next point release of mGui — the python module to streamline maya gui creation — is up on on Github, thanks in large part to +Bob White, who put in a lot of the work on this release.

Although this is a point release, it has some exciting improvements:

Improved Menu Support

We've added a more refined an elegant method for handling sub-menus with the SubMenu class. This works like a context manager, so the structure of the menu is obvious when reading the code.

    with Menu(label='TestMenu') as food_menu:            # conventional menu items          hotdog = MenuItem(label = 'Hot Dog')          burger = MenuItem(label = 'Burger')          taco = MenuItem(label = 'Taco')            # a submenu          with SubMenu(label='Pizza') as sm:              pepperoni = CheckBoxMenuItem(label='Pepperoni')              sausage = CheckBoxMenuItem(label='Sausage')              pineapples = CheckBoxMenuItem(label='Pineapples')              for each in (pepperoni, sausage, pineapples):                  each.command += pizza_selected              MenuDivider()            with SubMenu(label='Delivery') as sm:              with RadioMenuItemCollection() as radio:                  eatin = RadioMenuItem('Eat In')                  takeout = RadioMenuItem('Take Out')                  delivery = RadioMenuItem('Delivery')  

We've also enhance menus so they support the same kind of dot notation as other mGui controls. In this example that means you could get to the 'sausage' item as

    food_menu.sm.sausage  

which is going to be both more readable and more consistent with other mGui code. If you use the YAML menu loader, that also supports submenus:

        - !MMenuItem              key:  check              label:  checkboxes              annotation: Toggle me!              options:                checkBox: True              command: mGui.examples.menu_loader.checkbox              - !MSubMenu              key: submenu              label: submenus                items:                  - !MMenuItem                    key: sub1                    label: Item 1                    - !MMenuItem                    key: sub2                    label: Item 2  

Working with existing items

At various points we supported three different idioms (Menu.from_existing()gui.derive() and Control.wrap(). for putting an mGui wrapper around existng controls. In this release these have been collapsed into a single function, gui.wrap() which has some cool new features:

  • It's automatically recursive. So if you wrap an existing menu, the returned object will have nested properties for all of the menuItems in the menu. If you wrap an existing layout, you'll have nested properties for all the layout's children.
  • It does a better job matching the underlying maya gui widgets to mGui wrapper classes than any of the previous methods. Because of some irregularities in the way Maya relates commands to objects there are still some edge cases the new wrap()should get most cases.
  • For maya objects with python callbacks, you can ask wrap() to convert them to [multicast delegates for you. This does not support mel callbacks yet, but we may be able to expand to supporting them in the future.

The new, improved gui.wrap() should make it much easier to add mGui controls or layouts to menus and windows that come from regular Maya.

For the time being, we've re-pointed gui.derive() at gui.wrap() under the hood so it won't break existing code — it will however issue a deprecation warning so that when we remove derive() in 2.3 people will have had plenty of warning.

Find() method for child controls

We've added a new method to all layouts that will collect all children of a layout by mGui type. This is handy for big complex layouts where you aren't quite sure what something should be called — or for layouts that you've gotten from gui.wrap().

&c.

We've also cleaned up and expanded the examples, added more test coverage, and fixed a few minor bugs (the most obvious one will be that the StretchForm() classes properly respect margins, at long last!).

We're starting to put more time into the project wiki, including some new tutorials. Feedback and suggestions are much appreciated! And of course if you run into any problems be sure to log them on the project issues page. We've been debated whether or not we should package the project for PyPi and/or the Maya app store — if you have an opinion you should chime in in the comments or on the issues page.

tags: mayamgui

Sunday, June 12, 2016

The New Hotness


I've finally completed rolling over to to a new, self-hosted blog platform!

The process took a bit longer than I wanted, mostly because it web development remains a messy, iterative process - at least for me.  Since I ended up unifying both the blog and the old Character Rigger's Cookbook as well as on old markdown wiki, I had to do a lot of little scripts to groom the old content into a consistent format and linking strategy.  Add in more than a modicum of CSS noodling and whatnot and my 'couple of weekends' project turned into a couple of months.

However all that is behind me now, and all my future updates are going to be coming up at theodox.github.io  (you can also use www.theodox.com).  If you're subscribed to the current feed, you should switch over to either http://theodox.github.io/feeds/atom.xml or http://theodox.github.io/feeds/rss.xml, depending on your reader; one of the nice side effects of the switch is that the new feeds are much cleaner than Blogger's -- no more CSS gibberish instead of article summaries, thank you very much!)

I'm going to leave this site intact, and I'll try to keep monitoring comments and so on here, but the new site is going to be where all the news stuff comes out.  I'm also going to change the blog.theodox.com redirect so it goes to the new site, so if you're using that in a blogroll you won't have to update it.

PS. I had to touch a lot of content during the migration: there are about 150 blog posts, several dozen wiki pages, and a bunch of articles that all had to be repointed and I'd be pretty surprised if nothing odd slipped through.  Please let me know using the comments at the new site so I can fix up anything that's confusing or misleading.

So, here one more link to the new site, just in case.  Hope to see you there!

Thursday, May 12, 2016

Shamless Job Plug

Just wanted to re-flag the two job openings I'm hiring for at Undead Labs right now.

We're looking for one TA with good python chops to ride herd on the python/maya toolkit, and another TA with good shading and GPU skills to help us wring the most out of the Unreal Engine.  Details of both positions are in the link above.

Contact me directly or apply via the Undead Labs website!


Thursday, April 28, 2016

minor minq manifestations

Blogging has been super light, thanks to the triple whammy of a milestone, a cold, and my grand plan to migrate this blog over to a static site generator.  However I did want to mention that I've been trying to update the minq wiki page so it is more accessible. I'd appreciate any feedback, comments or suggestions to make it clearer.  When you write something yourself you just accept as natural the quirks and habits of thought that go with it, so it's handy to have outside eyes when you're writing documentation.

I've added a couple of minor features as well.  There's a  bunch of operators for counting things -- for example you can get the vertex count of meshes with something like Meshes().get(VertCounts) . On a somewhat related note all streams have a count() function which will return the length of the stream, and a first() method which will pull the head item from a stream -- which is handy if you expect to narrow down to a single item and don't want a single-item iterable.

Last but not least I'd love to hear from the community about good minq hacks -- I'll be happy to add cool ones to the examples file. I'd also appreciate any bugs you find going into the issues page!