jen Project Pipeline - Lets talk about project folders and code

So, one thing that you might be wondering is how do i handle the working files remotely without any lost paths or anything and make it seamless. Well thats what i want to talk about today!

Basically, the way im handling the organization of the project is that i made an organized directory in the FTP server, where all the files and information are neatly stored and organized. The way that the tool does not get lost in knowing where the files are regardless of the computer its being ran on, (remembering the previous blog post that mentioned the use of local files pulled from the FTP) is that i am basically reading that directory from the FTP and im pulling it to be saved locally with the same organization. So when the files are pulled, lets say you pull the model from a character named “John”, it will pull from the server to local like this:

Server: //Project/2 Production/Characters/John/Model/publish

Local: C:/Users/user/Documents/Project/2 Production/Characters/John/Model/publish

So when opening referenced files or just handling textures, in reality nothing really changes because its organized the same way in the FTP and in everyone’s personal computers.

The way im handling the pull of those directories is with some cool python coding and using the FTP module for python, which lets me query folder names and files just like the OS module lets me do the same in windows.

import os
import ftplib
import sys

#Ftp log in
    
session = ftplib.FTP('FTPAddress','Username','Password')

#project path
path = 'Project'

#empty path list to store the full directory
pathList = []

#list directory of the main path
pathList.append(path + '/')

Starting simple im just importing the os, ftp and sys modules for use. Im initiating a new log in session through the FTP, so that i can pull the info i want, im declaring my project name. Im doing it this way, because i could easily change the project name and the code still works if i wanna switch to an new project. Then i have an empty list called pathList and then im appending the main path to that list, which in this case is ‘Project/’, since FTP doesnt have any drive string preceding it, its just written as it is.

#function to cycle all directories
def cycle(directory):
    mainD = session.nlst(directory)
    for a in range(0, len(mainD)):
        if mainD[a] in pathList:
            pass
        else:
            pathList.append(mainD[a])

Then i continued by creating this function, which i can just add the directory, hence the pathList variable, and i can cycle through each folder and list its contents down the line. Thanks to the mainD variable, which session.nlist() its like an equivalent of OS’s os.listdir(). VERY CONVENIENT

def folderCreate(folderPath):
        doc = os.path.expanduser('~/')
        
        fullpath = doc + folderPath
        if os.path.exists(fullpath):
            print '---' + folderPath + '---Folder Already Exists'
        else:
            if folderPath.endswith('.txt', '.ma', '.mb', 
                                    '.png', '.jpg', '.mov', 
                                    '.mp4', '.jpeg', '.py', 
                                    '.pyc', '.tiff', '.iff', 
                                    '.json'):
                pass
            else:
                os.mkdir(fullpath)
                print '---'+ folderPath +'--- Folder Has Been Pulled from Server'
                

Now this is like, the heart of it all, here i basically i am taking advantage of the os.path.expanduser so that i can query the documents folder regardless of username and pc owner, so i can create my local directory. Then im just running a check that detects if the folder im trying to create exists, and if the file that im doing a query to, is a working file, like .ma or .mb which im going to use a lot in this project. Then, based on what i pulled from the FTP im creating that same directory with the os.mkdir command.

#run function with main path

for i in range(0, 500):
    try:
        cycle(pathList[i])
        print ('Directory Level ' + str(i + 1) + ' Done')
    except IndexError:
        break

for i in range(0, len(pathList)):
    folderCreate(pathList[i])

#log off from ftp
session.quit()

And finally, this is where it all comes together, Im basically doing a 500 for loop that is just using try and except so it doesnt do the 500 check everytime. That way i dont have to specify how many folders i have in the directory, and i just have a big number so i can keep adding folders to the FTP directory and it will pull it successfully. So im just cycling the cycle function and then based on the pathList variable im looping the creation of the folders. And finishing off with of course, logging off from the FTP itself.

I just saved this file separately so i can call it on command via the Maya script editor or, from the UI itself. For example, in the UI, i added a menu button to update directory, which it calls this code and checks the folders.

And this is how im handling the organization of the whole project itself, so that everyone is on the same page, and we dont have to transfer stuff via wetransfer or dropbox or anything, its all inside Maya. I’ll go more in detail on other aspects later on!

Cheers!

jen Project Pipeline - A journey within a journey

Hello! Sorry i havent posted in a while, things have been pretty busy as of my last blog post. But! Updates! Between then and now, i started a new job in Vancouver as a Senior Rigger and so far it has been a blast!

Anyways to start, with the content of this blog post… a little backstory. I wrote a story for an animated short film im working on in my free time. It has been a long process and theres still lots to do, our team is very small so i have to do a lot of work and develop a lot of stuff to get this short to completion. Hopefully when this project is done, it can begin my journey as a director. (One can hope) Ive had experience running projects in the past, mainly in the areas of organization, pipeline stuff, task management and overall running the a project from start to finish, so im bringing that knowledge for this personal project.

While we started working on this project and the pre production began, i started thinking ahead, since im mostly used to that thinking, i was trying to anticipate any problems we could face during production, one of those problems is the management of assets across the artists. There are many tools out there that can make this easier, like Artella, or Shotgun or any other project management software/asset management. Because one option was, to transfer the info between us via Dropbox or g drive or things like that, but its too much hassle. So i began to think, what if i make like, something like Artella but, by myself? Since Python is pretty much compatible with a LOT of stuff. I started doing my research.

I stumbled upon FTP servers, and what do you know, theres a whole python module for FTP servers. So I mixed that with some python and some Maya commands and i started to develop what i named… jenProjectPipeline. A tool that, in the course of this project, will facilitate the use of assets, props, characters, rigs, textures and animations of the whole short film. This tool is still in development of course, i believe i have done a lot of work on it in the past few months, so i think ive made good progress and ive made some successful test runs.

Here im gonna start breaking down my logistics with the UI, maybe in future blog posts i can go a bit in detail on parts of the code.

Here, we can see what i have so far in the UI, we can see the different sections in tabs for modeling, rigging and animation. So now ill explain how it works.

Here in the section labeled as number 1 in red, is a text list of all the files/items in the server that are under assets. As you can imagine, each type of model, rig an animation tab has the same server list layout in the UI. The purpose of this section is so that the user can see the list of all the available assets(in this case) that are currently in the FTP server. It also tells them what is the latest file currently in the FTP as well, for example in this image. We can see that the Tree asset is still in its first version, versus the Cliff asset is already in its third version. But with that list alone, we cannot handle those files to be worked on. Therefore, below the list, theres a progress bar with a download button. What this does is, when you select the asset you want to work on, you can hit the download to pull that file from the FTP server and store it locally on your computer. Once its fully downloaded, it will appear in the UI as a usable/editable asset.

That takes us to the area labeled number 2 in blue. That area is an icon text button layout that will display all the files you currently have locally with an icon representing them visually. It will also tell you what version you currently have locally. That way, if you are a version behind, you can download from the FTP and get the latest, but still storing the previous versions if you want to roll back. Having those assets locally in the right side opens us up to other options in the UI, marked below as number 3 in green.

These buttons below will allow us to manage what we want to do with the asset we selected in the number 2 area where the local files are stored. Starting with the green buttons, we can do a normal save, its just like a CTRL + S. Then we have Save Version, which is just that, version saving for your working backups. And with the publish button, what this does is that, it saves the official file, as a published file or published asset, and stores it locally, making the UI update on the right, but also, it triggers a python FTP code to upload that published asset back to the FTP server so that other users may see it and download it as well. This is going to be huge and very handy when handling the model deliveries, like, they can just tell me, i finished this character and i published it, and now i know i can just open this tool, download said character and start working on it. Since all of my small team is working remotely, this is going to be crucial to maintain a good synchronization during production.

The orange buttons, Open is just opening the selected file, Open Version is if you want to open any backup versions you saved with the Save Version button and the Open publish version is the same thing, but with the published files.

And finally in blue, we got the referencing, pretty straightforward, Reference its just that, it takes the latest asset selected and does a reference. Update Reference checks your scene for the asset selected in area number 2, and checks if its already referenced and it prompts you if you want to update that selected asset to the latest, or if you want to choose to what remap that asset. And finally, Reference publish version is if you want to bring a new reference that you can look for from the publish files.

I feel it pretty straight forward, im really excited how its working and how it has turned out, in the tests ive made, one of the team members actually uploaded that Cliff asset and i successfully downloaded it on my pc via the tool.

This is a pretty cool start for this tool and i bet its going to be very helpful in this project. Im going to continue developing it, and i will do my best to keep this blog update with the development of the tool and i will talk more in depth about certain areas of the tool as well. Maybe i will also do tech blogs talking about the rigging progress for the main character as well, well see!

Thanks for reading!