PyQt Agnostic Launcher II

As part of the improvements i have been carrying on to the VFX pipeline we are developing i wanted to dig deeper into the problem of executing a PySide Maya tool outside the DCC, this is, as standalone, as well as being able to execute it inside Maya without doing any changes to the code. I already came out with a first version of the launcher which you can see in http://jiceq.blogspot.com.es/2016/08/pyqt-agnostic-tool-launcher.html  . This basically detects whether there isn´t a Qt host application running and if not, we assume it is Maya running.

 @contextlib.contextmanager  
 def application():    
   if not QtGui.qApp:  
     app = QtGui.QApplication(sys.argv)  
     parent = None  
     yield parent  
     app.exec_()  
   else:  
     parent = get_maya_main_window()  
     yield parent  

This works fine for the beginnings of a VFX pipeline, mostly based in Maya. But as soon as you face the need to integrate other heterogeneous packages (that ship with any version of Python and PyQt, which is becoming a standard in the industry. see: http://www.vfxplatform.com/  ) you will probably want to be able to, at least, run the same GUI embedded in different packages as well as standalone. So the need to distinguish between host apps arises and this first solution falls short.

One poor solution is to query the Operating System whether the maya.exe/maya.bin or nuke.exe/nuke.bin processes were running. In the following fashion, for example:

  
def tdfx_is_maya_process_running():    
   return tdfx_is_process_running('maya')
def tdfx_is_nuke_process_running():    
   return tdfx_is_process_running('nuke')
def tdfx_is_process_running(process_name):
   if os.platform() == 'windows':
      ''' specific os code here '''
      return is_running    
   elif os.platform() == 'linux':
      ''' specific os code here '''
      return is_running
   return False

And use this instead in the previous if-statement to retrieve the corresponding QApplication main window instance.

This is a very poor solution, if we can call it a solution. It doesnt work well: you may have an instance of Maya or Nuke running, but you may want to run in standalone mode your custom script from your preferred IDE. The above functions will both return True, first problem. Second, it will depend on order of evaluation, so if you are testing first "tdfx_is_maya_process_running()" then your launcher will attempt to get the Maya main window instance. And third and most important, your launcher wont work because internally it is detecting Maya, so it is reporting the presence of a QApplication.qApp pointer, when you are in standalone mode and there is no qApp pointer actually!

So basically, this approach is not valid. What we really want to query is not the processes running, but more specifically if my current script is running embedded in a qt host application or not, and if so, i want to be able to know which one is.

I googled a little bit and was surprised that some people had faced this problem and meanly resolved it their own -not so great and elegant- way. I just thought there must be some way in Qt to query the host application. I just cant acknowledge something so basic wasnt taken into account in the framework. After some looking into the documentation..eureka, i found this line:

QtWidgets.QApplication.applicationName()

which returns the name of the host application. In standalone Qt apps, it is a parameter that must be set by the programmer.

def tdfx_qthostapp_is_maya():
    return tdfx_qthostapp_is('Maya-2017')

def tdfx_qthostapp_is_nuke():
    return tdfx_qthostapp_is('Nuke')

def tdfx_qthostapp_is(dcc_name):
    from PySide2 import QtWidgets
    hostappname = QtWidgets.QApplication.applicationName()
    if hostappname == dcc_name:
        return True
    return False

Consequently my new contextmanager version takes the following form:

 @contextlib.contextmanager  
 def application():    
   if tdfx_qthostapp_is_none():  
     app = QtGui.QApplication(sys.argv)  
     parent = None  
     yield parent  
     app.exec_()  
   elif tdfx_qthostapp_is_maya():  
     parent = get_maya_main_window()  
     yield parent
   elif tdfx_qthostapp_is_nuke():
     parent = get_nuke_main_window()
     yield parent  

This is a step improvement towards easing the integration of other PyQt-API-based DCCs in a VFX pipeline and easing the task of the programmer, thus avoiding to produce GUI application-specific code. Nonetheless, there is still some work to do that i will deal with when i have more time. This is, making the GUI code fully portable between PySide2 and PySide (or Qt4 and Qt5). There are already some solutions out there like the "Qt.py module" that intends to abstract the GUI from the Qt4 to Qt5 big jump in recent Maya 2017 Python API.

Python Multithreaded Asset Downloader

We are getting towards the end of the production of El Viaje Imposible 's teaser. A mixed 3d/real image project i ve been working on for the last few months along with other fantastic and experienced co-workers.

For the time we ve been developing the pipeline and artists using it, everyday there were some kind of issue and lately the problem was due to the http protocol we used to do the transfers. This was set from the very beginning hoping to review the different uploading methods our pipe allows in the future where the need to send massive amounts of files aroused.

Well, this time has come, Cloth and Hair Artists are already working and in order to pass on their work to the lighters they need to export caches files. Taking into account that our Hair plugin generates  a cache file per frame (even though one can choose to do inter frame caching also, i.e. to avoid flickering), that there may be a couple of plugin nodes that read/export cache multiplied by the number of characters in a shot this makes hundreds of files if not thousands of files to be sent to the server. Hence the need of a good bulletproof protocol.

Forgot to say, a lot of artists are working remotely! With all the inconvenientes this implies, you see.

This week we have improved a lot our checkin/checkout pipeline. We dont use anymore HTTP but have relied now on Samba as our audiovisual project management system allows this.

From my part, one of the improvements i ve done this week is to "parallelize" the assets downloader tool. The first release was running a unique thread in the background and downloaded each pipeline task assets sequentially. 

This was unbearable when we got deeper in the production as more advanced tasks depended upon all the previous  tasks. This means in order to perform a task, an artist should wait until near more than a hundred tasks were checked taking as long as 10 min sitting just with crossed arms.

IMPLEMENTATION

The goal was to substitute the sequential background thread with a configurable number of independent threads each in charge of checking the assets of a unique task. For this, we identify a class Job that is responsible for holding its own connection through Tactic API and all the metadata needed to tell to Tactic what are the assets it is looking for.

Then we define our Worker Class that will be sharing a thread-safe Queue. This Worker class will ask for the current job indefinitely while there are still jobs in the queue. Actually this is a variant of the Producer/Consumer problem where we fulfill the queue with jobs from the beginning, so there is no need for a producer thread.

class Worker(QtCore.QThread):
        
        '''
        define different signals to emit
        '''
        def __init__(self, queue, report):
            QtCore.QThread.__init__(self)
            self.queue = queue
            self.abort = False
            '''
            rest of variables
            '''        
        def run(self):

            while not self.abort and not self.queue.empty():
                job = self.queue.get()
                
                try:
                                
                    response = job.execute()
                    
                except Exception,e:
                    
                    process(e)

                self.queue.task_done()          

One of the problems left then is how to shutdown all the threads when closing the QDialog. I had quite a hard time figuring out the best way of doing it. 

Googleing a bit, people asked the same questions when your thread is running a while True sort of loop. Most people tend to confirm that the most elegant way is to put a "semaphor" also called "sentinel" which no any other thing that a boolean that is checked within every iteration. This allows to set this boolean from outside the thread, so next time it iterates it will jump out of the loop.

Another possibility is to put a Job None Object in the queue, so that immediately after retrieving it from the queue the thread checks its value and exits accordingly. This would work for a single thread, if we spawn 10 threads we should put 10 None Job Objects in the queue.

This leaves the question..¿how to terminate a specific thread? It's not needed here but rather something to think of later...

I resorted to the first elegant solution, that's the reason of the self.abort. So here is the code that overrides the closeEvent()

    def closeEvent(self, *args, **kwargs):
        
        for t in self.threads:
            if t.isRunning():
                t.abort = True
        import time
        time.sleep(2)
        for t in self.td.threads:
            t.terminate()
       
        return QtGui.QDialog.closeEvent(self, *args, **kwargs)

As you can see, before closing, we set the semaphore of each thread to True. The interesting thing about this code is that if you inmediately after try to terminate the thread (im not gonna discuss here the correctness of terminating/killing a thread) the window gets hung. Not sure why this is happening. All we need to do is sleep() a sufficient amount of time to give all the threads the chance to get out of the job.execute() and check for the semaphore.

My only concern with this solution is: what happens if one of the threads is downloading say 1 GB of data? would 2 seconds like in the example be enough time for it to get to the semaphore checking and then exit ? 

That's why i would really want to tweak the Tactic API and get low level for each downloaded chunk of data for example 10 MB. In 10MB slices this problem would disappear... but i'm stuck with the API for now and its interface.

IS IT REALLY PARALLEL?

Well not really. This same code in C or C++ would work totally parallelized but we are bumping into the GIL here, the Global Interpreter Lock of the MayaPy and CPython interpreters. You can have a look at all the posts regarding this in google. Basically, the GIL is a mechanism that forbids the python code to run more than one thread at the same time. This is to prevent the Interpreter's memory gets corrupted.

if we want full parallelization, we should go into multiprocessing which differs from multithreading in that each spawned process has its own memory space. Ideal when you dont need to share objects between processes for example or the need is little. Apart from the fact that, a lot of benchmarks that some people have done, come to the conclusion that in Python, multithreading tends to take more time in CPU-bound tasks that the same code running in single thread.

So if your task is CPU expensive, then try to go Multiprocessing rather than multithreading. But, if your tasks are I/O, networking ,etc (like it is the case here) i find multithreading more suitable. 

Nevertheless, i would like to give it another spin to the code a run benchmarks this time using the multiprocessing module.

How To Get Rid of PyQt Widgets Correctly

Introduction. The Tool.

In recent weeks i was told it would be nice to have some kind of reference editor outside Maya. Something simple that allowed animators to chose which references they wanted to load in the scene and which ones they didn't want.

What are the advantages for this requirement? The main reason is although we have at the studio powerful workstations in terms of Ram, CPU and Graphics processor some assets like set, props etc are really big, one single prop can take 3 GB!! and depending on the scene you can have almost 400 references. If each prop took that much space.. you can do the math.. it's simply unmanageable. It's not that huge in reality but it remains a big problem also if you take into account the amount of time it takes to load them all and finally open the scene. An animator would normally only want to load the character he/she is about to work with leaving aside all the props and set elements that don't interact with the character. This enables everyone to work faster.

Obviously the external reference editor must be "non-destructive". What i mean for this is it should not delete the reference node in Maya. Why? Obviously this external reference editor is useful for opening a scene file for the first time. Once the scene is loaded in Maya the animator must use the Maya reference editor to load/unload assets. In this case, to load all the necessary assets once the animation is finished, so that everything is in place when the playblast is published. So we need to let the animator the chance to load in Maya the rest of the assets and for this, he needs the reference node of the asset to be present in the scene.

After analyzing the Maya ASCII scene file it was clear what changes to do to the file to unload a specific asset.

Design. The problem.

Here is what i thought it would be a good design: i would use a dynamic list of widgets where each line would be composed of a QCheckBox showing the current state of the reference and the reference node of the asset.

I used the same approach as other times when i needed to code a dynamic list of widgets which consisted mainly in two steps:

A) we have a widget that triggers the fullfillment of the dynamic list. It can be something like a QComboBox to select the file's work area.

B) each time the dynamic list is filled we need to create a "line widget" with its proper layout which contains the QCheckbox and the QLineEdit. Those widgets are created each time which also means they need to be properly deleted, otherwise we will run into memory problems. And that was the origin of the bug i had.

When i first coded a dynamic list like this and wasn't that much versed into python i googled to look for the proper way to delete a widget, and i found this site in stackoverflow to be very useful although somewhat confusing. So many ways to apparently delete QWidgets!!

Digging into the proper solution.

There were three methods that apparently reached the same result:

1) the close() method in the QWidget class

2) the setParent() to None method also in the QWidget class also

3) the deleteLater() also in the QWidget class

I always thought the setParent() to None in each parent widget worked well. So in the method before filling the list i called a cleanup_scrollArea() method which was coded like this:

for i in reversed(range(layout.count())): 
        layout.itemAt(i).widget().setParent(None)

Relying on the fact that in the documentation they say: "the new widget is deleted when its parent is deleted".

I wont explain much. Only tell that this apparently works. Setting the the parent of a widget to None breaks the connection of the PyQt tree and causes all the children to not show anymore.

But there was a big bug. Whenever i tried repeatidly to test the tool with different files the tool crashed within the third or fourth iteration. The dynamic list's behaviour was apparently correct and working well, everything looked alright and i had no error message to give a hint of the problem.

I had the suspicion it had to do with a problem in the deletion of the widgets because the memory increased in each iteration even if the file had less refereneces to show than the previous one!. And obviously it was crashing when you tried to repeatidly use it. It must be a problem in the dynamic list!!

The Solution.

It  took me a short but intense moment  to figure out what was happening. And here my experience with a language such as C/C++ that deals with memory management helped me a lot since PyQt is a bind for Nokia's Qt written in C++.

What was happening?

Setting the  widget's parent to None only breaks the connection in the Qt widgets tree and causes the python reference to be deleted by the garbage collector. But what about the C++ QWidget Object that python was referencing? C++ does not have a garbage collector, so the C++ object's memory has to be deleted manually.

Here is why we have to use deleteLater() 's QWidget method. That's what it does, it frees the memory the C++ object is using..That's what we were missing! Furthermore deleting the C++ object makes the python references invalid therefore we don't need to set anymore to None the parent's widget,

The cleanup_scrollArea() method became:

while aLayout.count() > 0:

    item = aLayout.takeAt(0)

    widget = item.widget()

    if not widget:

        continue

    widget.deleteLater()

Design Improvement Quick Note

Creating and deleting widgets is expensive. It's a very stressful task even for a language like C++ with it's new and delete methods. So it may look like this behaviour for a dynamic list is not the best fit.

In the PyQt documentation they say that for this it may be better to use a QStackedWidget and play with the show() / hide() methods of the widgets which i believe reserves memory for a set of widgets and in the next iteration it reuses the same widgets changing their properties, hiding and adding new widgets on demand as needed. Might want to try this sometime!

Advanced Multithreading GUI Programming with PySide

One of the tasks i was in charge of was to look for a video player in linux that met several requirements from the art director. Some of them were:

- available for linux
- free
- frame by frame backwards and forward playing
- hotkeys for the above, not only buttons
- able to play several clips in a row
- no blank frame between clips
- able to play the audio of the video as well
- autoloop feature
- plays .mov, .mp4, .avi, compatible with the most possible codecs out there.

The autodesk RV Player is the best one according to some experienced animators but we couldn't afford licenceses for each individual.

After testing the obvious ones such as VLC (with jump in time extension), Openshot, MPlayer (with SMPlayer front-end)... All of them were lacking any of the listed features. Concretely i couldnt restrain my disappointment when i found that VLC's extension wasnt working as claimed on the web.

Several days passed by, and in the meantime i was busy doing other stuff and the animators had to deal with openshot which is not properly a video player but rather a video editor.

Anyways i found out that openshot was built upon the media lovin't toolkit video engine (MELT from now on)

So i decided to install it and give it a go.

Couldn't compile the last version 0.9.6 so i tried the 0.9.2 and yeah it worked!!

I was testing it and realized it met all the primary requisites only drawback: it had to be launched from the terminal and this was not as much user-friendly as it should for an animator.

So i was talking to my boss and he suggested coding a gui for maya that listed all the videos in the work directory and launch the player through Maya. That's when i got "hands on".

CODING THE TOOL

I was getting pretty advanced with the gui programming in PySide/Python time to do a test and launch the player came.

And here i discovered a big problem for me that took me some time to resolve. The problem was that the video player was correctly launched if done manually from an opened terminal but when done from the python script with:

import subprocess

command = ["gnome-terminal", "-x", "bash", "-c", "/usr/bin/melt <videofilefortest>"]
subprocess.call(command, shell = False, env = os.environ.copy())

it was giving an error regarding one of the dynamic link libraries (the famous "DLL" files in windows).

TWO WAYS TO SOLVE IT

Further investigation revealed that doing "ldconfig" in that terminal and launching again the player was a solution. "ldconfig" apparently rebuilds a cache file that specifies the name and path of each dynamic library so the system is able to find them. That opened a way to go. I just had to be able to login as root in the script, do a "ldconfig" and then launch MELT.

Another logical way was to mimic the same environment configuration between the two terminals setting and unsetting the ENV variables until both matched. It was clear that both terminals had different configuration. A "printenv > env_good.txt" and "printenv > env_bad.txt" helped to list the ENV variables of both terminals.

SOLUTION

I was jumping from one strategy to the other always testing. Had some difficulties trying to login as root in a script and also i wasn't confortable hardcoding the root password for obvious security reasons.

Finally i noticed that some important ENV variables werent set in the "good terminal" such as PYTHONHOME PYTHONPATH etc. So I decided to UNSET them prior to launching the player. With some help from a linux forum a user suggested also to UNSET not only those variables but also PATH, LD_LIBRARY_PATH. I agreed and finally it worked!!

EXPLANATION

Apparently, the way subprocess.call works is it executes the commands in a child process. Child processes inherit by default the environment from the parent process which in this case is Maya. But Maya itself is launched with specific configuration that doesnt necessarily match the default one.

It was the LD_LIBRARY_PATH what made it work. Makes sense since the concrete problem was the system wasnt finding where the libraries were.


ADVANCED GUI PROGRAMMING

That problem solved the rest was straightforward.

I noticed that the "search-the-filesystem-for-videos" was taking a bit too much time causing the tool to be apparently "frozen". So I decided to add something showing the tool was busy while searching. Something like a "living" waiting pattern.

This obviously led to the use of multithreading: one showing the waiting state while the other performs the search.

First i was wrong because i tried to run in a background process the GUI waiting icon. And i was wrong because a simple search in google showed that all the GUI processing has to be done in the main thread, and not in secondary ones. That is a good thing to know. I just had to switch the tasks and.. voilá.

I was really proud of it, not that it mattered too much since it was more like an aesthetic thing but i have to admit i haven't done much multithreading programming in all my years of programming.