How To Get Rid of PyQt Widgets Correctly

Introduction. The Tool.

In recent weeks i was told it would be nice to have some kind of reference editor outside Maya. Something simple that allowed animators to chose which references they wanted to load in the scene and which ones they didn't want.

What are the advantages for this requirement? The main reason is although we have at the studio powerful workstations in terms of Ram, CPU and Graphics processor some assets like set, props etc are really big, one single prop can take 3 GB!! and depending on the scene you can have almost 400 references. If each prop took that much space.. you can do the math.. it's simply unmanageable. It's not that huge in reality but it remains a big problem also if you take into account the amount of time it takes to load them all and finally open the scene. An animator would normally only want to load the character he/she is about to work with leaving aside all the props and set elements that don't interact with the character. This enables everyone to work faster.

Obviously the external reference editor must be "non-destructive". What i mean for this is it should not delete the reference node in Maya. Why? Obviously this external reference editor is useful for opening a scene file for the first time. Once the scene is loaded in Maya the animator must use the Maya reference editor to load/unload assets. In this case, to load all the necessary assets once the animation is finished, so that everything is in place when the playblast is published. So we need to let the animator the chance to load in Maya the rest of the assets and for this, he needs the reference node of the asset to be present in the scene.

After analyzing the Maya ASCII scene file it was clear what changes to do to the file to unload a specific asset.

Design. The problem.

Here is what i thought it would be a good design: i would use a dynamic list of widgets where each line would be composed of a QCheckBox showing the current state of the reference and the reference node of the asset.

I used the same approach as other times when i needed to code a dynamic list of widgets which consisted mainly in two steps:

A) we have a widget that triggers the fullfillment of the dynamic list. It can be something like a QComboBox to select the file's work area.

B) each time the dynamic list is filled we need to create a "line widget" with its proper layout which contains the QCheckbox and the QLineEdit. Those widgets are created each time which also means they need to be properly deleted, otherwise we will run into memory problems. And that was the origin of the bug i had.

When i first coded a dynamic list like this and wasn't that much versed into python i googled to look for the proper way to delete a widget, and i found this site in stackoverflow to be very useful although somewhat confusing. So many ways to apparently delete QWidgets!!

Digging into the proper solution.

There were three methods that apparently reached the same result:

1) the close() method in the QWidget class

2) the setParent() to None method also in the QWidget class also

3) the deleteLater() also in the QWidget class

I always thought the setParent() to None in each parent widget worked well. So in the method before filling the list i called a cleanup_scrollArea() method which was coded like this:

for i in reversed(range(layout.count())): 
        layout.itemAt(i).widget().setParent(None)

Relying on the fact that in the documentation they say: "the new widget is deleted when its parent is deleted".

I wont explain much. Only tell that this apparently works. Setting the the parent of a widget to None breaks the connection of the PyQt tree and causes all the children to not show anymore.

But there was a big bug. Whenever i tried repeatidly to test the tool with different files the tool crashed within the third or fourth iteration. The dynamic list's behaviour was apparently correct and working well, everything looked alright and i had no error message to give a hint of the problem.

I had the suspicion it had to do with a problem in the deletion of the widgets because the memory increased in each iteration even if the file had less refereneces to show than the previous one!. And obviously it was crashing when you tried to repeatidly use it. It must be a problem in the dynamic list!!

The Solution.

It  took me a short but intense moment  to figure out what was happening. And here my experience with a language such as C/C++ that deals with memory management helped me a lot since PyQt is a bind for Nokia's Qt written in C++.

What was happening?

Setting the  widget's parent to None only breaks the connection in the Qt widgets tree and causes the python reference to be deleted by the garbage collector. But what about the C++ QWidget Object that python was referencing? C++ does not have a garbage collector, so the C++ object's memory has to be deleted manually.

Here is why we have to use deleteLater() 's QWidget method. That's what it does, it frees the memory the C++ object is using..That's what we were missing! Furthermore deleting the C++ object makes the python references invalid therefore we don't need to set anymore to None the parent's widget,

The cleanup_scrollArea() method became:

while aLayout.count() > 0:

    item = aLayout.takeAt(0)

    widget = item.widget()

    if not widget:

        continue

    widget.deleteLater()

Design Improvement Quick Note

Creating and deleting widgets is expensive. It's a very stressful task even for a language like C++ with it's new and delete methods. So it may look like this behaviour for a dynamic list is not the best fit.

In the PyQt documentation they say that for this it may be better to use a QStackedWidget and play with the show() / hide() methods of the widgets which i believe reserves memory for a set of widgets and in the next iteration it reuses the same widgets changing their properties, hiding and adding new widgets on demand as needed. Might want to try this sometime!

Nested References in Maya

I've found myself in the need at work to be able to get all the references a shot had. What you can get with file(q = True, r = True) are only the top level references leaving behind the nested ones.

I've read somewhere that is a good practice trying to avoid such levels of depth, but in this case we were in a closet.

So after a while searching the web and the official Maya python documentation i hit with referenceQuery and the "children" flag.

Finally i developed this simple method that fills a python list with reference filenames:


    def add_nested_references(self,parent_ref_list):
       
        for parent_ref in parent_ref_list:
            child_list = mc.referenceQuery(parent_ref, f = True, ch = True)
            if child_list != None:
                   self.add_nested_references(child_list)
                   parent_ref_list.extend(child_list)

How to Recover Linux Grub Boot Loader

Normally, the order of installation should be first windows then linux so that the grub boot loader is installed and recognizes the windows partition too.

At my job i had to do it the other way around: we had a system with Centos 7 linux and we needed to install Windows to make this machine we useful either to rigging either to production staff.

I inserted the windows dvd and made a new partition in the unallocated space then install.

When i restarted the machine the linux boot loader had disappeared so i was googling how to recover it. Truth is it's pretty simple and after a really few minor mistakes i managed to make it work:

We need first to enter in RESCUE mode with a linux installation dvd/cd.

The system will be most probably mounted in /mnt/sysimage so we make the

chroot /mnt/sysimage

after that we do:

/sbin/grub2-install /dev/sda

to make grub recognize our windows entry just type:

sudo grub2-mkconfig > /dev/null

If everything went well the windows loader entry has been detected, all we need now is to save it to the grub.cfg:

grub2-mkconfig -o /boot/grub2/grub.cfg

Restart and voilà, we have our boot loader back with our windows O.S. entry!!!

Advanced Multithreading GUI Programming with PySide

One of the tasks i was in charge of was to look for a video player in linux that met several requirements from the art director. Some of them were:

- available for linux
- free
- frame by frame backwards and forward playing
- hotkeys for the above, not only buttons
- able to play several clips in a row
- no blank frame between clips
- able to play the audio of the video as well
- autoloop feature
- plays .mov, .mp4, .avi, compatible with the most possible codecs out there.

The autodesk RV Player is the best one according to some experienced animators but we couldn't afford licenceses for each individual.

After testing the obvious ones such as VLC (with jump in time extension), Openshot, MPlayer (with SMPlayer front-end)... All of them were lacking any of the listed features. Concretely i couldnt restrain my disappointment when i found that VLC's extension wasnt working as claimed on the web.

Several days passed by, and in the meantime i was busy doing other stuff and the animators had to deal with openshot which is not properly a video player but rather a video editor.

Anyways i found out that openshot was built upon the media lovin't toolkit video engine (MELT from now on)

So i decided to install it and give it a go.

Couldn't compile the last version 0.9.6 so i tried the 0.9.2 and yeah it worked!!

I was testing it and realized it met all the primary requisites only drawback: it had to be launched from the terminal and this was not as much user-friendly as it should for an animator.

So i was talking to my boss and he suggested coding a gui for maya that listed all the videos in the work directory and launch the player through Maya. That's when i got "hands on".

CODING THE TOOL

I was getting pretty advanced with the gui programming in PySide/Python time to do a test and launch the player came.

And here i discovered a big problem for me that took me some time to resolve. The problem was that the video player was correctly launched if done manually from an opened terminal but when done from the python script with:

import subprocess

command = ["gnome-terminal", "-x", "bash", "-c", "/usr/bin/melt <videofilefortest>"]
subprocess.call(command, shell = False, env = os.environ.copy())

it was giving an error regarding one of the dynamic link libraries (the famous "DLL" files in windows).

TWO WAYS TO SOLVE IT

Further investigation revealed that doing "ldconfig" in that terminal and launching again the player was a solution. "ldconfig" apparently rebuilds a cache file that specifies the name and path of each dynamic library so the system is able to find them. That opened a way to go. I just had to be able to login as root in the script, do a "ldconfig" and then launch MELT.

Another logical way was to mimic the same environment configuration between the two terminals setting and unsetting the ENV variables until both matched. It was clear that both terminals had different configuration. A "printenv > env_good.txt" and "printenv > env_bad.txt" helped to list the ENV variables of both terminals.

SOLUTION

I was jumping from one strategy to the other always testing. Had some difficulties trying to login as root in a script and also i wasn't confortable hardcoding the root password for obvious security reasons.

Finally i noticed that some important ENV variables werent set in the "good terminal" such as PYTHONHOME PYTHONPATH etc. So I decided to UNSET them prior to launching the player. With some help from a linux forum a user suggested also to UNSET not only those variables but also PATH, LD_LIBRARY_PATH. I agreed and finally it worked!!

EXPLANATION

Apparently, the way subprocess.call works is it executes the commands in a child process. Child processes inherit by default the environment from the parent process which in this case is Maya. But Maya itself is launched with specific configuration that doesnt necessarily match the default one.

It was the LD_LIBRARY_PATH what made it work. Makes sense since the concrete problem was the system wasnt finding where the libraries were.


ADVANCED GUI PROGRAMMING

That problem solved the rest was straightforward.

I noticed that the "search-the-filesystem-for-videos" was taking a bit too much time causing the tool to be apparently "frozen". So I decided to add something showing the tool was busy while searching. Something like a "living" waiting pattern.

This obviously led to the use of multithreading: one showing the waiting state while the other performs the search.

First i was wrong because i tried to run in a background process the GUI waiting icon. And i was wrong because a simple search in google showed that all the GUI processing has to be done in the main thread, and not in secondary ones. That is a good thing to know. I just had to switch the tasks and.. voilá.

I was really proud of it, not that it mattered too much since it was more like an aesthetic thing but i have to admit i haven't done much multithreading programming in all my years of programming.

Selection Button In Maya MEL

It's been a while since my last post. The reason is that work's keeping me really busy and i'm learning a lot about Python and MEL. 

The last couple of weeks i 've left temporarily rigging the props and i 've got back to scripting.

It happens that some of the main characters' rigs cannot use FK because the dynamics don't work, so i was talking to one of the lead animators and he came up with an idea, and i was the one to code it and make it possible. I did several scripts that accomplish all the tasks the lead animator asked for the main and secondary characters. Still it has to be tested by the art director and the other lead animators but it feels like it's really useful since it mimics an FK chain with IK controls.

One of the scripts is a panel with selection buttons. Those buttons select each locator so they can animate the locators and then bake the animation.

In MEL we have the "select -r <whatever>" which selects the new by replacing the old. Also we have "select -tgl <whatever>" which adds the selected to the current selection.

The MEL code for a button is:

button -label $label -command $command

Where $command stands for some sort of  "select" as described before. The problem is none of them 

works as it should. I wanted to "select -r" replace but with the ability to Shift Select other locators, this is to add them to the selection if i use Shift just like Maya works.

The solution is to add the following line of code to the $command:

So that finally the button code results like this:

string $command = "string $object=\"object_name_to_select\"; int $mods = `getModifiers`; if (($mods)%2 == 1) select -add $object; else select $object;";

button -label $label -command $command

I'd like to post some rigs i've done but i'm afraid i cannot show the models due to confidentiality terms. 

Shortcut in Linux

How to create a shortcut in linux that executes a shell script

1. We create the shortcut.

Typically it's a ".desktop" ascii file with these entries although it can have more:

[Desktop Entry]
Type=Application
Encoding=UTF-8
Name=Sample Application Name
Comment=A sample application
Exec=application
Icon=application.png
Terminal=false

- The Name specifies the name of the launcher.
- Exec is the most important line as it specifies which script to launch and how it would be launched. We will leave this field blank til we have the shell script written
- Icon we set the path of the app's icon if available.
- Terminal specifies whether the app needs a terminal to run.

2. We create the shell script.

#!/bin/bash

. ~/.bashrc
PATH=$PATH:/srv/XXXX/XXXX/final_directory;
export PATH
cd /srv/projects/myproject

[some commands executing applications]

The first line is a comment indicating which shell will execute the script, in this case BASH.

The second line is a line that loads all the basrc file extensions

Then we perform some tasks such as adding another value to the system environment variable path and export that new entry.

We change our current directory and finally we call some commands that will launch our application.

3. We link the Exec command of the desktop file to the script

Note: if we have successfully coded our script it should work from the command line writing:

sh <ourscriptname>.sh

In our case an example of launching the shell script from our desktop file by double-clicking is

Exec=gnome-terminal -x bash -c "sh /path/to/thescript/scriptname.sh;bash"

Another linux tip: when we try to open a file sometimes we dont get to choose the right program even selecting "open with other programs". The trick to making appear the application in that list is editing as root its corresponding ".desktop" file normally placed in /usr/shared/applications/<name>.desktop and in the "command" or "Exec" field write at the end of the command the following: "<previous_commands><blank space>%U"

We save... and that should do the trick!!!

Job as Rigger/Pipeline TD

After a few frenetic weeks rushing to meet the deadline imposed by the CG Student Awards 2015 (13 May) finally i had something to show. And I say finally because for the last months i've applied for jobs where i was aware i needed at least a demo reel, so instead i used to tell my life in neverending emails hoping to get caught by some samaritan producer.

Well this last week was in the end very productive. I had no more than 3 job opportunities:

- The first one was an internship of a month and a half in Lightbox Entertainment thanks to a master colleague that already works there. You never know how important networking is.

- The second one was an internship at a company in Barcelona but the work could presumably be done from here from Madrid. They told me they really liked my work and that they would put me as rigger in spite of the fact that they were looking for a 3d modeler. They revealed to be very kind: i sure would have loved working for them. The company name is PaperFrog and they are currently making a TV3 show regarding animals and dinosaurs.

- The third one, the one i chose, is at a company in the center of Madrid. They are starting the production of a new feature film but they don't have made many claims regarding this yet. The film is intended to deliver next year and i have a contract for almost the rest of the year. I cannot talk really much about it because of the confidential terms but in the press note they released in April they say Spa Studios has done the character development and that the script has been written by the guy that worked in Madagascar 2 and others. It looks really promising.

I leave you here with the show reel i presented to the CG Student Awards. I very doubt that i win anything but the purpose for me was to do some work and have something to show when applying for a job.

CG STUDENT AWARDS 2015 - Showreel

Juan Cristóbal Quesada Showreel 2015