Template engine (Python)

Muster 8 implements a brand new template engine. The template engine takes care of instruct Muster how to interact with external render engines or batch jobs. The new engine is entirely based on Python 3.3. Through the template engine you can:

  • Configure external applications
  • Build command lines
  • Create submission dialogs
  • Declare special actions
  • Create complex presets dialogs
  • Define the behaviours of clients
  • Create custom execution environments

The template engine expect a .py file for each template that should create a basic class derived from the standard MTemplateAPI.MTemplate class configured with custom capabilities.

A minimal template requires at least the following elements:

  • A class that inherit the base class MTemplateAPI.MTemplate
  • Configuration of the class default values like the template ID, the template name, the template logic (multiframe, image slicing, broadcast or target host)
  • Configuration of the template logics default values
  • Configuration of the template fields that will be mapped directly to a job attributes and will appear in the submission dialog
  • Configuration of the supported platforms (Win / Mac / Linux) specific values
  • Redefinition of the requires virtual functions that spawn the processes, build the command lines and check for the results

After creating the template class, it must be installed into the global template manager singleton using :

MTemplateAPI.MManager.getUniqueClass().installPyTemplate(myTemplateClass)

Each section may contain special values, stepping inside Muster default templates could be a good starting point in learning the template Python classes.

We’ll step across building a template for Maya rendering to learn more: The first check you’ve to do when building a new template, is taking a look at the command line required by the engine, it our case, the basic syntax maya expect is:

Render –r render_type –proj project_directory –sf start_frame –ef end_frame –bf by_frame path_to_scene

Keeping that in mind, let’s build the template using the internal software rendering:

''' Import Muster base modules '''
import MClientAPI
import MTemplateAPI
 
''' Get the template manager singleton '''
manager = MTemplateAPI.MManager.getUniqueClass()
 
''' Define a new class derived from MTemplateAPI.MTemplate base class , override required methods '''
class MayaTemplate(MTemplateAPI.MTemplate):
    def __init__(self):
        MTemplateAPI.MTemplate.__init__(self)
	self.setID(1)
	self.setName("Maya Sw")
	self.setDescription("Maya software render")
	self.setTemplateLogic(MTemplateAPI.kTemplateMultiframe)
	self.setDefaultPriority(1)
	''' Multiframe template specific settings '''
	self.multiframe.setEnableFrameCheck(1)

As you can see, the first section declares the basic template parameters, giving a template name, description and unique identifier. The logic is set to Multiframe that requires a

self.multiframe.setEnableFrameCheck(1)

if you want to enable the frame check feature. This syntax shows how to access specific logics features.

The next section configures the submission dialog either for Console and the web server:

mayaFile = MTemplateAPI.MTemplateItemFile("MAYAFILENAME","Maya scene file name","Specifies the maya scene file to render","",0,1,1,"*.ma;*.mb")
mayaProject = MTemplateAPI.MTemplateItemFolder("MAYAPROJECTPATH","Project directory","Specifies the maya project path","",0,1,1)
mayaRenderPath = MTemplateAPI.MTemplateItemFolder("MAYADESTINATION","Frames destination","Specifies the frames destination","",1,1,1)
mayaStartFrame = MTemplateAPI.MTemplateItemDouble("MAYASTARTFRAME","Start frame","Specifies the starting frame for the rendering job",1.0,0,0,1)
mayaEndFrame = MTemplateAPI.MTemplateItemDouble("MAYAENDFRAME","End frame","Specifies the ending frame for the rendering job",1.0,0,0,1)
mayaByFrame = MTemplateAPI.MTemplateItemDouble("MAYABYFRAME","By frame","Specifies the frame step for the rendering job",1.0,0,0,1)
mayaNumberBy = MTemplateAPI.MTemplateItemInteger("MAYANUMBERBY","Number by","Specifies the start number for frame numbering",1,0,0,1)
mayaStepBy = MTemplateAPI.MTemplateItemInteger("MAYASTEPBY","Step by","Specifies the stepping for the frame numbering",1,0,0,1)
mayaDigits = MTemplateAPI.MTemplateItemInteger("MAYADIGITS","Digits","Specifies the number of digits for the frame numbering",1,0,0,1)
mayaAbortRender = MTemplateAPI.MTemplateItemYesNo("ABORTRENDER","Abort on log","Immediately abort the rendering if an error string is found during log output",0,0,0,1)
self.addSubmissionItem(mayaFile)
self.addSubmissionItem(mayaProject)
self.addSubmissionItem(mayaRenderPath)
self.addSubmissionItem(mayaStartFrame)
self.addSubmissionItem(mayaEndFrame)
self.addSubmissionItem(mayaByFrame)
self.addSubmissionItem(mayaNumberBy)
self.addSubmissionItem(mayaStepBy)
self.addSubmissionItem(mayaDigits)
self.addSubmissionItem(mayaAbortRender)
''' items mapping to Muster default tags '''
self.addMapping("MAYASTARTFRAME","start_frame")
self.addMapping("MAYAENDFRAME","end_frame")
self.addMapping("MAYABYFRAME","by_frame")
self.addMapping("MAYANUMBERBY","number_by")
self.addMapping("MAYASTEPBY","step_by")
self.addMapping("MAYAFILENAME","job_file")
self.addMapping("MAYAPROJECTPATH","job_project")
self.addMapping("MAYARENDERPATH","output_folder")

The previous code builds a set of fields object that are installed into the template using the addSubmissionItem() function call. Each object is a subclass of the MTemplateItem base class.

The addMapping() function let you specify which field maps to the required Muster internal fields like the start frame, end frame and so on..

''' Windows support '''
self.platformWindows.setPlatformEnabled(1)
self.platformWindows.setEnableErrorCheck(1)
self.platformWindows.setEnabledByDefault(1)
self.platformWindows.setFramesFloating(3)
self.platformWindows.setDetectionLogic(MTemplateAPI.kProcessChild)
self.platformWindows.setDetectionLogicProcessName("Mayabatch.exe")
applicationPath = MTemplateAPI.MTemplateItemFile("MAYAEXECUTABLE","Maya batch render executable","Path to Maya render.exe application batch render","c:\\program files\\Autodesk\\Maya 2010\\bin\\render.exe",0,0,1,"*.exe")
self.platformWindows.addClientConfigurationItem(applicationPath)
startingFolderPath = MTemplateAPI.MTemplateItemFolder("MAYASTARTINGFOLDER","Starting folder","Specify the starting folder for the render process","c:\\program files\\Autodesk\\Maya 2010\\bin",0,0,1)
self.platformWindows.addClientConfigurationItem(startingFolderPath)
processorUsage = MTemplateAPI.MTemplateItemCombo("PROCUSAGE","Processors usage","Specify the number of processors to use for each render instance","0",0,0,1)
processorUsage.addItem("All","0")
processorUsage.addItem("1","1")
processorUsage.addItem("2","2")
processorUsage.addItem("3","3")
processorUsage.addItem("4","4")
self.platformWindows.addClientConfigurationItem(processorUsage)

The previous code sample shows the engine declaration section for the Windows platform. It not all sets platform specific attributes, but it also install a set of fields inside the platform itself. Those fields will be available into the client configuration dialog and can be accessed later to construct the command line as well as to provide additional behaviours.

def onBuildCommandLine(self, platform, job, chunk, clientTemplatePreferences,instanceNum):
	renderCmd = "-r sw"
	renderCmd += " -n " + clientTemplatePreferences['PROCUSAGE']
	renderCmd += " -proj \"" + job.attributeGetString(manager.resolveMappingToJob(self.getID(),"MAYAPROJECTPATH")) + "\""
	if job.attributeIsEnabled(manager.resolveMappingToJob(self.getID(),"MAYADESTINATION")):
		renderCmd += " -rd \"" + job.attributeGetString(manager.resolveMappingToJob(self.getID(),"MAYADESTINATION")) + "\""
	renderCmd += " -s " + '%.3f' % chunk.getStartFrame()
	renderCmd += " -e " + '%.3f' % chunk.getEndFrame()
	renderCmd += " -b " + '%.3f' % chunk.getByFrame()
	renderCmd += " -rfs " + str(chunk.getFrameStart())
	renderCmd += " -rfb " + str(chunk.getFrameStep())
	renderCmd += " -pad " + job.attributeGetString("MAYADIGITS")
	renderCmd += " " + job.attributeGetString("ADD_FLAGS")
	renderCmd += " \"" + job.attributeGetString(manager.resolveMappingToJob(self.getID(),"MAYAFILENAME")) + "\""
	return renderCmd

The previous code sample shows how to override a class virtual method to provide a specific function. In this case, we need to tell Muster how to build the final command line.

def onGetApplicationPath(self,job,clientTemplatePreferences,pathOut ):
	pathOut.setString(clientTemplatePreferences['MAYAEXECUTABLE'])
	return MTemplateAPI.MTemplateError()
 
def onGetApplicationStartingFolder(self,job,clientTemplatePreferences,pathOut):
	pathOut.setString(clientTemplatePreferences['MAYASTARTINGFOLDER'])
	return MTemplateAPI.MTemplateError()

In the previous code, we tell Muster how to find the command line and the starting folder for the batch job. In that case, we grab the values from the client configuration values using the clientTemplatePreferences supplied dictionary.

def onCheckLog(self,job,chunk,clientTemplatePreferences,log):
	if log.find("Warning") != -1:
		return MTemplateAPI.MTemplateError(2,"Warning keyword found",MTemplateAPI.MTemplateError.kTemplateErrorTypeWarning)
	if log.find("Error") != -1:
		return MTemplateAPI.MTemplateError(1,"Error keyword found",MTemplateAPI.MTemplateError.kTemplateErrorTypeError)
	if log.find("Total Time For Render") == -1:
		return MTemplateAPI.MTemplateError(3,"Missing \"Total Time For Render\" keyword. Batch render may have crashed",MTemplateAPI.MTemplateError.kTemplateErrorTypeWarning)
	if log.find("Total Elapsed Time For Maya") == -1:
		return MTemplateAPI.MTemplateError(3,"Missing \"Total Elapsed Time For Maya\" keyword. Batch render may have crashed",MTemplateAPI.MTemplateError.kTemplateErrorTypeWarning)
	if log.find("completed.") == -1:
		return MTemplateAPI.MTemplateError(3,"Missing \"completed.\" keyword. Batch render may have crashed",MTemplateAPI.MTemplateError.kTemplateErrorTypeWarning)
	return MTemplateAPI.MTemplateError()
 
def onCheckExitCode(self,job,chunk,clientTemplatePreferences,exitCode):
	if exitCode == 211:
		return MTemplateAPI.MTemplateError(exitCode,"Abnormal render termination",MTemplateAPI.MTemplateError.kTemplateErrorTypeError)
	if exitCode != 0:
		return MTemplateAPI.MTemplateError(exitCode, "%d Exit code different from expected 0 value" % exitCode,MTemplateAPI.MTemplateError.kTemplateErrorTypeWarning)
 
	return MTemplateAPI.MTemplateError()

The previous virtual functions tell Muster how to process the batch job output log as well as the process exit code.

''' Create an instance of the template class and install it into the template manager context '''
template = MayaTemplate()
manager.installPyTemplate(template)

The last code creates an instance of the class and installs it into the template manager.

Further information on the template syntax can be found looking into the API classes reference as well as into the existing templates.

Any template is able to override Muster default behaviours by redefining one or more virtual functions of the base class. This is a list of the available functions and their meaning:

Function name and parametersExpected return variableDescription
onFieldChanged(string fieldName,string fieldValue, ref MTemplateItemsFields fieldsToChange)voidInvoked by Console when a template field value changes, useful to automate fills of different fields and/or invoke actions like scene parsing after the file selection
example
onBuildCommandLine(int platform, MJob job, MChunk chunk,ref stringMap clientTemplatePreferences, int instanceNum)stringInvoked by the client to build the process command line
example
onBuildAdditionalParameters(ref MPropertiesMap attributes)stringInvoked by Console to build additional parameters for the command line
example
onGetApplicationPath(MJob job,ref stringMap clientTemplatePreferences, ref MStringRef pathOut )MTemplateError Invoked by the client to get the process path
example
onGetApplicationStartingFolder(MJob job,ref stringMap clientTemplatePreferences, ref MStringRef pathOut) MTemplateErrorInvoked by the client to get the process execution folder
example
onBuildEnvironment(MJob job, MChunk chunk,ref stringMap clientTemplatePreferences, ref MEnvironment existingEnvironment)MEnvironmentInvoked by the client to build an environment block for the process
example
onCheckForSubframeAdvancingString(string line)boolInvoked by the client on each log line parsing to check for a frame completion string
example
onDetectRunningProcess(MPid mainProcess, vector of MProcessSnapshot hostProcesses, ref stringMap clientTemplatePreferences,ref MPid runningProcessPid)kTemplateFunctionBehaviour (int constant)Invoked by the client to detect the process effective running PID
onCheckForFramesPrefixString(string line, ref MStringRef prefixOut)boolInvoked by the client on each log line parsing to guess the frames path
example
onCheckForSubframeProgress(string line, ref MStringRef progressOut)boolInvoked by the client on each log line parsing to guess the percentage of the current frames
example
onCheckLogLine(MMJob job,MChunk chunk,ref stringMap clientTemplatePreferences, string line)MTemplateErrorInvoked by the client for each log line, this is useful to abort the current render on certain log errors
example
onCheckLog(MJob job, MChunk chunk,ref stringMap clientTemplatePreferences, string log)MTemplateErrorInvoked by the client to check the full log at the end of the render
example
onCheckExitCode(MJob job,MChunk chunk,ref stringMap clientTemplatePreferences, int exitCode)MTemplateErrorInvoked by the client to check the process exit code
example
onApplicationFinder(ref MStringRef moduleRegExp, ref MStringRef& moduleTag)intInvoked by the Services control panel applet to configure automatic application finding
example
onFindApplication(ref MStringRef basePath,ref stringMap clientTemplatePreferences)boolInvoked by the Services control panel applet to find a specific application directly from the template
example
onModuleFound(ref MStringRef moduleExec,ref MStringRef modulePath,ref stringMap clientTemplatePreferences); boolInvoked by the Services control panel applet when an application executable is found
example
onGetSlicesInputFilename(MJob job,MChunk chunk)stringInvoked by image slicing jobs to know each chunk slice file path
example
onGetSlicesOutputFilename(MJob job)stringInvoked by image slicing jobs to know the final assembled image name
example
onGetSliceBoundaries(MJob job,MChunk chunk,MTemplateSliceBoundaries boundaries)voidInvoked by image slicing jobs to know how to expect slices and how to assemble them
example
onStartProcess(MJob job, int instanceNum, MPid& runningProcessPid))voidInvoked by the client when a new process should be started
example
onTerminateProcess(MJob job, MPid process, int instanceNum))voidInvoked by the client when a running process should be terminated
example
onChangeProcessPriority(MJob job, MPid process, int priority, int instanceNum))voidInvoked by the client when the system priority of a process should be changed
example
onCheckProcessTermination(MJob job, MPid process, int instanceNum))voidInvoked by the client to check if the process has been completed
example

The following functions are also called from the global template (ID 0):

Function name and parametersExpected return variableDescription
onValidateJobSubmission(MJob job, ref MStringRef err)boolUsed to validate a job submission
example
onDeleteJob(MJob job)voidInvoked when a job is deleted
example
onSubmitJob(MJob job)voidInvoked when a job is submitted
example
onReinitJob(MJob job)voidInvoked when a job is reinit
example
onEditJob(MJob job)voidInvoked when a job is edited
example
onJobPropertiesChanged(MJob job)voidInvoked when a property of a job is changed
example
onJobStart(MJob job)MTemplateErrorInvoked when the job is started
example
onJobEnd(MJob job)MTemplateErrorInvoked when a job is completed
example
onChunkStart(MJob job,MChunk chunk, ref stringMap clientTemplatePreferences,int instanceId)MTemplateErrorInvoked when a chunk is started
example
onChunkEnd(MJob job,MChunk chunk,ref stringMap clientTemplatePreferences, int instanceId)MTemplateErrorInvoked when a chunk is completed
example