Vector Matrix multiplication in Maya

Multiply PyMel vectors by matrices will not give you the result you might expect. Try a simple operation like this:

import pymel.core as pm
pm.polySphere() # Make a sphere.
pm.move((0,1,0)) # Move 1 unit up.
wm = pm.SCENE.pSphere1.getMatrix(worldSpace=True) # Get the local->world space matrix.
v = pm.dt.Vector((0,0,0)) # Create a zero vector.
print v*wm # Multiply the zero vector by the matrix and print the resulting vector.

At this point you'd expect to get (0, 1, 0) - the result of multiplying a (0,0,0) vector by a matrix that translates points by 1 in the Y axis. Instead you'll get (0,0,0). The reason is PyMel's vectors are only 3 components long, and the matrix is 4x4. Mathematically that's a no-no, but PyMel is happy to do it, but what you're getting is only the rotation and scale contained in the matrix affecting the vector. These are contained in the 3x3 part of the matrix while translation lives in the fourth row.
To get the translation you need to add the w component. However trying to initialize a PyMel Vector object with 4 components is not possible. Instead you need to use the VectorN class. If we do that in the code above, it will work as expected:

v = pm.dt.VectorN((0,0,0,1)) # Create a zero vector.

Alternatively you can use the Point class, which describes a 4-component vector:

v = pm.dt.Point((0,0,0))

MQuaternion inversion functions

The wording in the documentation for Maya's quaternion inversion functions can lead you to use them erroneously. MQuaternion::invertIt() (and probably inverse() as well) will make the quaternion represent the inverse rotation and not do an inversion of the quaternion's components. To the latter you will need someQuaternion = -someQuaternion. This will correctly invert the XYZ and W components of the quaternion, and not just the XYZ. Because of double cover, inverted quaternions represent an equivalent orientation in 3D space. You might want to invert all the components when interpolating between two orientations.

Performance comparison: MEL vs PyMel vs OpenMaya vs maya.cmds vs C++

Over the years Maya has grown to have quite the myriad of programming APIs. MEL and the C++ API have been joined by Python maya.cmds, the OpenMaya Python API wrapper and PyMel. Now I read Maya 2013 is getting Python API 2.0. I've read a lot of discussion on what's quickest but most of them are based on generic assumptions (simpler is quicker so PyMel must be slowest) and don't quote any specific timings. So I've done some myself, while profiling tools and trying to get the most performance out of them. (All timings are in seconds)
Example: having selected 10000 edges, convert the selection to faces.

PyMel:	1.7
maya.cmds:	7.5
MEL:	7.4
OpenMaya:	0.2

OpenMaya is quickest by far, but PyMel is still 4.5x faster then MEL or maya.cmds. What will slow it down however, is if we add a small change - flattening the selection list (i.e.: "ls -sl -fl"):
Timings with selection list flattening:

PyMel:	7.9
maya.cmds:	8.0
MEL:	7.7
OpenMaya:	0.2 (there is no distinction in the API, same code)

Here it's roughly the same speed as MEL and maya.cmds. What seems to slow it down is creating instances of PyMel classes as we will see in the next example.
Example: custom edge ring select:

PyMel:	2.4
maya.cmds:	1.2
OpenMaya:	0.03

PyMel stumbles here, maya.cmds is quicker but OpenMaya turns out nearly 40x faster still. However it's interesting to note that a lot of the PyMel code actually runs 2 or 3 times faster then the maya.cmds code. What slows it down is one part of it, which creates PyMel objects. To be clear, by creating PyMel objects, I mean something like: edge = pymel.core.general.MeshEdge( someMesh, someEdgeIndex ).

The overall conclusion I can draw is that PyMel is quick. A lot of it builds on top of OpenMaya, which is the quickest of all the scripting APIs. Maya.cmds is really just a thin wrapper on top of MEL, and there is no significant performance difference between the two. As long as you can avoid creating a lot of PyMel objects in your code, you should find it outperforming both maya.cmds and MEL.

We know OpenMaya is very quick, but lets compare it to it's non interpreted, compiled flavour: the C++ API. I have a simple plugin deformer implemented in both C++ and Python. It basically takes a point on a curve, calculates a normal to the curve at that point and then bends geometry along that curve. The timings are:

Python

Finding the point along the curve:	0.027
Calculating a normal:	0.15
Deforming passed points:	0.060

C++

Finding the point along the curve:	0.017
Calculating a normal:	0.0023
Deforming passed points:	0.000062

"Finding the point along the curve" is a single function call and most of the work gets done by Maya. There is only a small difference between Python and C++. "Deforming passed points" however, is a bunch of adding and multiplying. No API function gets called. Here C++ outperforms Python by nearly a thousand-fold. The more I work on this plugin the more dramatic the overall difference becomes. For deformers at least, C++ is massively quicker then Python.

These tests are only very limited, I admit, but the medal standings are: gold for C++, silver for OpenMaya, and bronze for PyMel. MEL and maya.cmds are tied for a close fourth.

Maya normals locking and what it does to tangent space

Apparently locking normals in Maya is no simple matter. There are several problems with using MEL, PyMel or methods accessed through the UI:

• Locking normals might modify them. If you locked a normal before, locking it again might set it to something you didn't want.
• Edge smoothing is re-calculated. Any vertex that has at least one hard edge, will make all the edges that are connected to it, hard. So for example you have a vertex with two soft edges and two hard edges connected to it. Lock that vertex's normal and suddenly all the edges are now hard.

Edge softness is still important even when you lock normals. With locked normals, changing edge softness will make no apparent difference in the viewport. However Maya will still look at edge smoothing to determine if vertices should share tangent and bi-normal vectors. This has  impact on normal maps and also on tri-stripping on graphics cards.
The only reliable way I found of locking normals is:

• Store edge softness in an array
• Save all the normals into an array
• Set all the normals to what they are using this array
• Lock all the normals per vertex, per poly
• Restore edge softness

Nested function variable scope in Javascript

The way scope works for nested functions in Javascript can trip you up:

#target photoshop
function test()
{
this.a10 = function()
{
a = 10;
this.a20();
return a;
}
this.a20 = function()
{
a = 20;
}
}
t = new test();
alert( t.a10() ); // Returns 20!

And now with the added "var" keyword:

#target photoshop
function test()
{
this.a10 = function()
{
var a = 10;
this.a20();
return a;
}
this.a20 = function()
{
var a = 20;
}
}
t = new test();
alert( t.a10() ); // Returns 10

Locking vertex normals using Maya API

OpenMaya.MFnMesh.lockFaceVertexNormals is the function to use but it's got not very clear from the documentation what you need to pass to it. It needs an array of faces and of vertices formed like this:
faces = ( 0, 0, 0, 0, 1, 1, 1, 1 )
verts = ( 1, 2, 3, 4, 1, 2, 5, 6 )
In this example the face with index 0 will have vertices 1, 2, 3 and 4 locked. Face 1 will have vertices 1, 2, 5, 6 locked. This code will lock all vertex face normals:

verts = OpenMaya.MIntArray()
faces = OpenMaya.MIntArray()
vertsForFace = OpenMaya.MIntArray()
for faceID in range( meshFn.numPolygons() ) :
meshFn.getPolygonVertices( faceID, vertsForFace )
for i in range( vertsForFace.length() ) :
faces.append( faceID )
verts.append( vertsForFace[i] )
meshFn.lockFaceVertexNormals( faces, verts )

Context sensitive hotkeys

Want the same hotkey to do different things dependent on which Maya window you're in?

string $panelWithFocus = `getPanel -withFocus`;
if( $panelWithFocus == "polyTexturePlacementPanel1" ){
//we're in the UV Editor!
}
else{
//We're somewhere else
}

Great for having different marking menus under one hotkey.