jen Project Pipeline - Lets talk about project folders and code

So, one thing that you might be wondering is how do i handle the working files remotely without any lost paths or anything and make it seamless. Well thats what i want to talk about today!

Basically, the way im handling the organization of the project is that i made an organized directory in the FTP server, where all the files and information are neatly stored and organized. The way that the tool does not get lost in knowing where the files are regardless of the computer its being ran on, (remembering the previous blog post that mentioned the use of local files pulled from the FTP) is that i am basically reading that directory from the FTP and im pulling it to be saved locally with the same organization. So when the files are pulled, lets say you pull the model from a character named “John”, it will pull from the server to local like this:

Server: //Project/2 Production/Characters/John/Model/publish

Local: C:/Users/user/Documents/Project/2 Production/Characters/John/Model/publish

So when opening referenced files or just handling textures, in reality nothing really changes because its organized the same way in the FTP and in everyone’s personal computers.

The way im handling the pull of those directories is with some cool python coding and using the FTP module for python, which lets me query folder names and files just like the OS module lets me do the same in windows.

import os
import ftplib
import sys

#Ftp log in
    
session = ftplib.FTP('FTPAddress','Username','Password')

#project path
path = 'Project'

#empty path list to store the full directory
pathList = []

#list directory of the main path
pathList.append(path + '/')

Starting simple im just importing the os, ftp and sys modules for use. Im initiating a new log in session through the FTP, so that i can pull the info i want, im declaring my project name. Im doing it this way, because i could easily change the project name and the code still works if i wanna switch to an new project. Then i have an empty list called pathList and then im appending the main path to that list, which in this case is ‘Project/’, since FTP doesnt have any drive string preceding it, its just written as it is.

#function to cycle all directories
def cycle(directory):
    mainD = session.nlst(directory)
    for a in range(0, len(mainD)):
        if mainD[a] in pathList:
            pass
        else:
            pathList.append(mainD[a])

Then i continued by creating this function, which i can just add the directory, hence the pathList variable, and i can cycle through each folder and list its contents down the line. Thanks to the mainD variable, which session.nlist() its like an equivalent of OS’s os.listdir(). VERY CONVENIENT

def folderCreate(folderPath):
        doc = os.path.expanduser('~/')
        
        fullpath = doc + folderPath
        if os.path.exists(fullpath):
            print '---' + folderPath + '---Folder Already Exists'
        else:
            if folderPath.endswith('.txt', '.ma', '.mb', 
                                    '.png', '.jpg', '.mov', 
                                    '.mp4', '.jpeg', '.py', 
                                    '.pyc', '.tiff', '.iff', 
                                    '.json'):
                pass
            else:
                os.mkdir(fullpath)
                print '---'+ folderPath +'--- Folder Has Been Pulled from Server'
                

Now this is like, the heart of it all, here i basically i am taking advantage of the os.path.expanduser so that i can query the documents folder regardless of username and pc owner, so i can create my local directory. Then im just running a check that detects if the folder im trying to create exists, and if the file that im doing a query to, is a working file, like .ma or .mb which im going to use a lot in this project. Then, based on what i pulled from the FTP im creating that same directory with the os.mkdir command.

#run function with main path

for i in range(0, 500):
    try:
        cycle(pathList[i])
        print ('Directory Level ' + str(i + 1) + ' Done')
    except IndexError:
        break

for i in range(0, len(pathList)):
    folderCreate(pathList[i])

#log off from ftp
session.quit()

And finally, this is where it all comes together, Im basically doing a 500 for loop that is just using try and except so it doesnt do the 500 check everytime. That way i dont have to specify how many folders i have in the directory, and i just have a big number so i can keep adding folders to the FTP directory and it will pull it successfully. So im just cycling the cycle function and then based on the pathList variable im looping the creation of the folders. And finishing off with of course, logging off from the FTP itself.

I just saved this file separately so i can call it on command via the Maya script editor or, from the UI itself. For example, in the UI, i added a menu button to update directory, which it calls this code and checks the folders.

And this is how im handling the organization of the whole project itself, so that everyone is on the same page, and we dont have to transfer stuff via wetransfer or dropbox or anything, its all inside Maya. I’ll go more in detail on other aspects later on!

Cheers!