PyQt Agnostic Tool Launcher

After some weeks of resting and a new challenge at Drakhar Studio where i am developing the nuts & bolts of a pipeline software that communicates with TACTIC, i have now some spare time to talk about one of my recent discoveries concerning Python programming.

I'm always looking on how to improve my code (in general, no matter what the programming language is), although it's certainly true Python is one in a million because of many reasons, one of them being that a bunch of design patterns are an intrinsic part of the language, such as decorators and context managers.

More specifically, i was looking for a way to run my tools regardless or whether it was standalone (this is, running its own QtApplication, or embedded into Maya's). In the past, i didnt have much time to dig into this and consequently, i used two separate launchers.

Last week this was solved by the use of a context manager. I realised i could make good use of it, since in both cases (running standalone or in a host app) i had to make the same two calls:

 tool_window = Gui_class(parent)  
 tool_window.show()  

Where Gui_class() is the main GUI PyQt Class. The difference is the parent argument where in one case it must be None and in the other it must be a pointer to the Maya GUI Main Window.

This difference in the parent argument could be handled just the same way as for example the open() context manager works:

 with open(filepath, 'wb') as f:  
    src_lines = f.readlines()  

In this case, the advantage is the user doesnt have to remember to close the file stream and the open() method yields the stream as 'f'.


LAUNCHER IMPLEMENTATION

 @contextlib.contextmanager  
 def application():    
   if not QtGui.qApp:  
     app = QtGui.QApplication(sys.argv)  
     parent = None  
     yield parent  
     app.exec_()  
   else:  
     parent = get_maya_main_window()  
     yield parent  

This is my implementation of my custom context manager, it is basically an if statement that checks whether the tool is being lauched from a host application or in his own QApplication. This function could be already the main launcher, the only need is to put the previous two lines showed where the yield statement goes. But this goes against one of the OOP principles, Dont Repeat Yourself (DRY).

So if we push a little bit further we end up with:

 def launcher(cls):      
   with application() as e:  
     window = cls(parent=e)  
     window.show()  

Where cls is the name of the main GUI class. This way, we have an agnostic launcher prepared to work as standalone and within a host app like maya.

Needless to say that some pieces of the tool only will work inside Maya but this way at least we can launch the tool for GUI refinement and development.

No more separate launchers with duplicate parts of the code!!

Simple Procedural Texture Generator and Visualizer

INTRODUCTION

I've been thinking about coding something related with perlin noise, just something that could be used as a justification. Normally i would have coded it in C++ with Qt but since ive been digging into the guts of python and PySide/PyQt for the last year, together with the fact that python GUI with PyQt is not that hard like in C++ (something it really does not have much interest once you get how the layouts, widgets, etc, work).

My only concern was performance because i wanted to do all the calculation and send the vertices data to the gpu each time you changed any of the parameter values governing the shape of the noise, the size, the visualization,..etc. I was willing to accept a little lag.

I wont explain deeply how Perlin noise works. For this you can have a look at the wikipedia or in a book i consider very useful: Texturing & Modeling: A Procedural Approach 

My approach basically consists of a function that generates values for a given octave. Then the final result will be a superposition of those octaves depending on the number specified.

INTERPOLATION

One of the options i wanted to explore was to obtain a more organic feel to the noise. With linear interpolation you can get some artifacts horizontally and vertically which really doesnt look well.

Here are the three interpolation methods:

1. linear

2. cosine

3. cubic 

All of the form "interpolate(x0,x1,t)"

 def Linear(a,b,t):  
   return a * (1 - t) + b * t  
 def Cosine(a,b,t):  
   t2 = (1 - math.cos(t * math.pi)) / 2.0  
   return (a * (1 - t2) + b * t2);  
 def Spline(x0,x1,t):  
   a = x0 - x1  
   b = -1.5 * x0 + 1.5 * x1  
   c = -0.5 * x0 + 0.5 * x1  
   d = x0  
   t2= t * t  
   return a * t2 * t + b * t2 + c * t + d  

The spline or cubic interpolation was used in a simplified manner. Normally the cubic interpolation formula uses information of 4 points: the two in the middle plus the rightmost and leftmost of them. For coding purposes, just to simplify, we assumed p0=p1 and p2=p3, hence the above code.

I will quote this page for the cubic interpolation just in case it disappears.

If the values of a function f(x) and its derivative are known at x=0 and x=1, then the function can be interpolated on the interval [0,1] using a third degree polynomial. This is called cubic interpolation. The formula of this polynomial can be easily derived.

A third degree polynomial and its derivative:

For the green curve:

The values of the polynomial and its derivative at x=0 and x=1:

The four equations above can be rewritten to this:

And there we have our cubic interpolation formula.

Interpolation is often used to interpolate between a list of values. In that case we don't know the derivative of the function. We could simply use derivative 0 at every point, but we obtain smoother curves when we use the slope of a line between the previous and the next point as the derivative at a point. In that case the resulting polynomial is called a Catmull-Rom spline. Suppose you have the values p₀, p₁, p₂ and p₃ at respectively x=-1, x=0, x=1, and x=2. Then we can assign the values of f(0), f(1), f'(0) and f'(1) using the formulas below to interpolate between p₁ and p₂.

Combining the last four formulas and the preceding four, we get:

OPENGL and PYTHON

One of the most time consuming aspects of dealing with PyOpenGL is that OpenGL is a C library and hence, if you code in C++ you share the same basic data types specially things like (void *) pointers, C arrays and the casting operation between types... But Python has its own data types!

1) I'll give you an example: Vertex Buffer Objects need to be passed a C array of GL_FLOAT values in order to specify vertex data. I was managing vertex data but in python lists. I discovered i had two options here: whether i used another dependency library such as Numpy with their immediate conversion between lists and arrays...or i could just use the "array" type. I finally chose this last option.

 from array import array  
 vertex_array = array('f', vertex_list)  
 index_array = array('i', index_list)  

where 'f' stands for float and 'i' for integer.

2) Another big problem i faced is how on earth i could update the vertex data sent to the buffer instead of deleting/creating/sending everything again as if i restarted the app.

 glBindBuffer(GL_ARRAY_BUFFER, self.vboId)  
 c_void_ptr = glMapBuffer(GL_ARRAY_BUFFER, GL_READ_WRITE)  
 c_float_array_ptr = cast(c_void_ptr, POINTER(c_float))  
 # change vertex data    
 for i in range(len(vertex_list)):  
    c_float_array_ptr[i] = vertex_list[i]  
 glUnmapBuffer(GL_ARRAY_BUFFER)  

I discovered the buffer in video memory could be mapped to a chunk in RAM so that when changing one, it immediately applies to GPU. This is using "glMapBuffer/glUnmapBuffer".

But this function returns a "C void pointer" which in python terms is just an integer refering to some memory address.

We need a way to cast this void pointer to a float pointer (float array). That is the raison d'être of the next line. Needless to say i needed to import the ctypes module.

Then we can access finally the c_float_array_ptr as an iterable assigning float values from the python vertex_list!

Here is a video snippet of how the app works.

How To Get Rid of PyQt Widgets Correctly

Introduction. The Tool.

In recent weeks i was told it would be nice to have some kind of reference editor outside Maya. Something simple that allowed animators to chose which references they wanted to load in the scene and which ones they didn't want.

What are the advantages for this requirement? The main reason is although we have at the studio powerful workstations in terms of Ram, CPU and Graphics processor some assets like set, props etc are really big, one single prop can take 3 GB!! and depending on the scene you can have almost 400 references. If each prop took that much space.. you can do the math.. it's simply unmanageable. It's not that huge in reality but it remains a big problem also if you take into account the amount of time it takes to load them all and finally open the scene. An animator would normally only want to load the character he/she is about to work with leaving aside all the props and set elements that don't interact with the character. This enables everyone to work faster.

Obviously the external reference editor must be "non-destructive". What i mean for this is it should not delete the reference node in Maya. Why? Obviously this external reference editor is useful for opening a scene file for the first time. Once the scene is loaded in Maya the animator must use the Maya reference editor to load/unload assets. In this case, to load all the necessary assets once the animation is finished, so that everything is in place when the playblast is published. So we need to let the animator the chance to load in Maya the rest of the assets and for this, he needs the reference node of the asset to be present in the scene.

After analyzing the Maya ASCII scene file it was clear what changes to do to the file to unload a specific asset.

Design. The problem.

Here is what i thought it would be a good design: i would use a dynamic list of widgets where each line would be composed of a QCheckBox showing the current state of the reference and the reference node of the asset.

I used the same approach as other times when i needed to code a dynamic list of widgets which consisted mainly in two steps:

A) we have a widget that triggers the fullfillment of the dynamic list. It can be something like a QComboBox to select the file's work area.

B) each time the dynamic list is filled we need to create a "line widget" with its proper layout which contains the QCheckbox and the QLineEdit. Those widgets are created each time which also means they need to be properly deleted, otherwise we will run into memory problems. And that was the origin of the bug i had.

When i first coded a dynamic list like this and wasn't that much versed into python i googled to look for the proper way to delete a widget, and i found this site in stackoverflow to be very useful although somewhat confusing. So many ways to apparently delete QWidgets!!

Digging into the proper solution.

There were three methods that apparently reached the same result:

1) the close() method in the QWidget class

2) the setParent() to None method also in the QWidget class also

3) the deleteLater() also in the QWidget class

I always thought the setParent() to None in each parent widget worked well. So in the method before filling the list i called a cleanup_scrollArea() method which was coded like this:

for i in reversed(range(layout.count())): 
        layout.itemAt(i).widget().setParent(None)

Relying on the fact that in the documentation they say: "the new widget is deleted when its parent is deleted".

I wont explain much. Only tell that this apparently works. Setting the the parent of a widget to None breaks the connection of the PyQt tree and causes all the children to not show anymore.

But there was a big bug. Whenever i tried repeatidly to test the tool with different files the tool crashed within the third or fourth iteration. The dynamic list's behaviour was apparently correct and working well, everything looked alright and i had no error message to give a hint of the problem.

I had the suspicion it had to do with a problem in the deletion of the widgets because the memory increased in each iteration even if the file had less refereneces to show than the previous one!. And obviously it was crashing when you tried to repeatidly use it. It must be a problem in the dynamic list!!

The Solution.

It  took me a short but intense moment  to figure out what was happening. And here my experience with a language such as C/C++ that deals with memory management helped me a lot since PyQt is a bind for Nokia's Qt written in C++.

What was happening?

Setting the  widget's parent to None only breaks the connection in the Qt widgets tree and causes the python reference to be deleted by the garbage collector. But what about the C++ QWidget Object that python was referencing? C++ does not have a garbage collector, so the C++ object's memory has to be deleted manually.

Here is why we have to use deleteLater() 's QWidget method. That's what it does, it frees the memory the C++ object is using..That's what we were missing! Furthermore deleting the C++ object makes the python references invalid therefore we don't need to set anymore to None the parent's widget,

The cleanup_scrollArea() method became:

while aLayout.count() > 0:

    item = aLayout.takeAt(0)

    widget = item.widget()

    if not widget:

        continue

    widget.deleteLater()

Design Improvement Quick Note

Creating and deleting widgets is expensive. It's a very stressful task even for a language like C++ with it's new and delete methods. So it may look like this behaviour for a dynamic list is not the best fit.

In the PyQt documentation they say that for this it may be better to use a QStackedWidget and play with the show() / hide() methods of the widgets which i believe reserves memory for a set of widgets and in the next iteration it reuses the same widgets changing their properties, hiding and adding new widgets on demand as needed. Might want to try this sometime!

Maya Python/PyQt Alignment Plugin Rewritten and Extended

Unfortunately, a few days ago my computer decided to crash forcing me to reinstall the OS. I thought i had all my code files and stuff safe in the Data drive... well I proved to trust wrong because guess where the system got installed with the recovery partition? yes !! in my data drive!!. What pisses me off mostly is that i ve lost all the five programming assignments from the Coursera MOOC "Algorithms, part I", which i ve previously talked about in this blog. Other code I had was some python/pygame computational geometry stuff i did in my spare time.

And finally, my first Maya plugin also was lost. So far from being discouraged, i decided to recode again the plugin adding the feature someone suggested me in the spanish TD facebook group and also i decided to extend the plugin in order to be able to do 3 types of operations with the object's pivot.

Before i post the youtube video showing how it works i will talk a bit about some tips i didn't post previously that i had forgot and that i had to face again.

Qt Designer

I used Qt Designer to visually establish the layout of my window. Two things about this:

- first, i wanted to use two sets of exclusive radioButtons which led me to set two separate QButtonGroups. With the left button on each QButtonGroup i selected the radiobuttons to be members of it.

- Secondly. I wanted to put QGroupBoxes because it is a feature that enhances the visual look of the window as well as helping figure out visually which parameters are part of the same option. For this you have to firstly select the groupbox and drop it in the window layout, and after and only after, put inside the necessary widgets. You will notice if you have done it correctly because the widgets appear to be "parented" to the groupbox in the "Object Inspector" 

This is how it looks like:

UI xml file conversion to python file

The .ui file saved by Qt Designer is no more than an xml file that has to be converted to .py file with all the window layout so that we can instance it in our window class from our Python/PyQt main file.

For this we open the windows command line, we set the path to our .ui file and type:

pyuic4 align.ui > align.py

Our Maya Plugin

We declare our class that this time inherits from QWidget and add the WindowStayOnTopHint.

class AlignWindow(QtGui.QWidget):
    def __init__(self, parent=None):

        QtGui.QWidget.__init__(self, parent, QtCore.Qt.WindowStaysOnTopHint)
        self.ui = Ui_Alignment()
        self.ui.setupUi(self)

Since Maya is running its own QApplication thread this code causes Maya to stall:

if __name__ == "__main__":
    app = QtGui.QApplication(sys.argv)
    mw = AlignWindow()
    mw.show()
    sys.exit(app.exec_())

But instead can make the window visible when running from Visual Studio. To make it work in Maya it suffices to write:

myWindow = AlignWindow()
myWindow.show()

Three different Pivot Alignment Techniques

- pivot in object(s). Aligns the pivot(s) within the bounding box(es) of each of the object(s). That was what my first plugin, yes the one i lost did.

- pivot(s) to object's pivot. Aligns a set of object(s) pivot(s) to the last selected object's pivot.

- object(s) to object. Aligns a set of object(s)'s pivot(s) to the last object selected but this time so that we translate the objects maintaining their relative position regarding their pivots.

My first Maya Python/PyQt plugin

THE PLUGIN

What i'm gonna show in the next video is the first plugin for Maya i've coded. I wanted to do something with GUI so i forced myself to use PyQt and the QTDesigner to get familiar with it. The idea of this plugin came up during one of the sessions of the Máster in audiovisual production with Autodesk Maya that i'm following at CICE here in Madrid.

I recalled that 3ds max has a very complete tool for pivot alignment and orientation. One can do the practically the same with Maya but the local axis alignment. This comes handy when we cannot do a snap to the grid or geometric vertex to position the object's pivot.

Modifying the object's pivot position is a very usual operation when we find ourselves modeling stuff. It allows for example to rotate an object's duplicate around a geometric center that can be found through the bounding box of the proper object itself.

ABOUT MY INTEREST IN MAYA TOOLS DEVELOPMENT

The course is mainly focused on 3d concepts such as NURBS & polygon modeling, lighting, texturing, rendering, dynamics etc, all from a Maya artist point of view so no coding at all taught. I felt very disappointed when the other Master i was about to do was cancelled, this one oriented exclusively to Maya tools development and API understanding. The reason was that there wasnt people enough interested in the course. Truth is we were at first 10 people (enough for it) but 5 of them were being sent by the same company as an education at work thing and this company backed off at the last moment. So there we were only 5 people, which the school though was not a proper amount to dedicate a room during five and a half months, in other words, it wasn't profitable.

'Okay, then i'll try in April 2015 while im working' (the schedule allowed me to balance it with work) but then new surprise "the teacher is leaving to work for MPC in Canada, the course will not be released anymore".

Well then find another teacher!!! Apparently this is such a "high profile" that there is little demand for this kind of education (usually 3d artist dont wanna know anything about coding) and at the same time it's difficult to find a teacher. Is this caused by the fact that there are few Technical Directors or Tech Artists out there? is it a "niche" job?

All this said, that's why im trying to make it up and start learning tools development on my own since there is no place else where to study this here in Spain. It will require more effort and the steps will be smaller but with help from other technical directors in a facebook group and mainly my will power  (yes! i can !) i hope to get enough understanding and experience to land a job as CG Generalist/TD.

Ohhhh excuse me, this post is about the plugin. I 'll let the video do the talking!!

Visual Studio Setup for Maya Python and Qt4 programming

I'm following a course on 3d using Maya and having worked as a programmer for a couple of years i want to learn to code some python tools for Maya as well as some C++ plug-ins. So the first thing to do is to choose the development environment.

I ve heard in some places that people tend to use SublimeText and that it's really easy to configure for Maya python programming. Well the truth is i'm rather used to VS since i ve programmed in C++ and C# doing some things on my own. Plus, lately i ve been programming some stuff in python and pygame and i used VS with the python tools so this is my selected IDE.

Now i will explain how to setup Visual Studio 2013 for Maya Python programming.

VISUAL STUDIO 2013 SETUP FOR MAYA TOOLS PROGRAMMING IN PYTHON

1. Download and install Python Tools for Visual Studio (PTVS).You can get them here.

2. Download and install PyQt4 for python 2.7 (since Maya 2015 is using python version 2.7.3). You can get it here. You need to install PyQt4 in the following path:

C:\Program Files\Autodesk\Maya2015\Python\Lib\site-packages

3. Open VS 2013 and create a Python Application project.
Under project properties->debug you need to add the following folders to the search path:

C:\Program Files\Autodesk\Maya2015\devkit\other\pymel\extras\completion\py
C:\Program Files\Autodesk\Maya2015\Python\Lib\site-packages\maya
C:\Program Files\Autodesk\Maya2015\Python\Lib\site-packages\pymel
C:\Program Files\Autodesk\Maya2015\Python\Lib\site-packages\PySide
C:\Program Files\Autodesk\Maya2015\Python\Lib\site-packages\PyQt4

Now we have autocompletion for maya.cmds, pyqt4 etc!!

Under "interpreter path" add the line:

C:\Program Files\Autodesk\Maya2015\bin\mayapy.exe

You can have the mayapy.exe interpreter if you want or the one in python27 folder if you prefer. We ve made sure both python version are the same 2.7

4. Now we will add the possibility to execute code from VS directly into Maya.
Download the fast script execution (toMaya) by Josbalcaen here.

- once installed, restart VS 2013
- go to Tools->Options->Keyboard
- search for "ToMaya" and create a new shortcut for example ALT+SPACE

5. write the following code in the Maya script editor

1 2 3 4

import maya.cmds as cmds try: cmds.commandPort(name="127.0.0.1:6000", close=True, echoOutput=True) except:    pass cmds.commandPort(name="127.0.0.1:6000", echoOutput=True)

and put it in a custom shelf or configure your userSetup.mel file to execute the script on Maya startup.

Click on the script, now Maya is listening to this port which is the one ToMaya uses for sending the scripts via VS.

That's all, we should have maya.cmds, PyQt and Python auto completion along with direct execution in Maya!!.

If you prefer, i ve gathered all the necessary stuff needed and put it here. (except for the PTVS, sorry!!)