Template engine (Python)
Introduction
Muster 8 implements a brand new template engine. The template engine takes care of instruct Muster how to interact with external render engines or batch jobs. The new engine is entirely based on Python 3.3. Through the template engine you can:
- Configure external applications
- Build command lines
- Create submission dialogs
- Declare special actions
- Create complex presets dialogs
- Define the behaviours of clients
- Create custom execution environments
The template engine expect a .py file for each template that should create a basic class derived from the standard MTemplateAPI.MTemplate class configured with custom capabilities.
A minimal template requires at least the following elements:
- A class that inherit the base class MTemplateAPI.MTemplate
- Configuration of the class default values like the template ID, the template name, the template logic (multiframe, image slicing, broadcast or target host)
- Configuration of the template logics default values
- Configuration of the template fields that will be mapped directly to a job attributes and will appear in the submission dialog
- Configuration of the supported platforms (Win / Mac / Linux) specific values
- Redefinition of the requires virtual functions that spawn the processes, build the command lines and check for the results
After creating the template class, it must be installed into the global template manager singleton using :
MTemplateAPI.MManager.getUniqueClass().installPyTemplate(myTemplateClass)
Each section may contain special values, stepping inside Muster default templates could be a good starting point in learning the template Python classes.
Building a template
We’ll step across building a template for Maya rendering to learn more: The first check you’ve to do when building a new template, is taking a look at the command line required by the engine, it our case, the basic syntax maya expect is:
Render –r render_type –proj project_directory –sf start_frame –ef end_frame –bf by_frame path_to_scene
Keeping that in mind, let’s build the template using the internal software rendering:
''' Import Muster base modules ''' import MClientAPI import MTemplateAPI ''' Get the template manager singleton ''' manager = MTemplateAPI.MManager.getUniqueClass() ''' Define a new class derived from MTemplateAPI.MTemplate base class , override required methods ''' class MayaTemplate(MTemplateAPI.MTemplate): def __init__(self): MTemplateAPI.MTemplate.__init__(self) self.setID(1) self.setName("Maya Sw") self.setDescription("Maya software render") self.setTemplateLogic(MTemplateAPI.kTemplateMultiframe) self.setDefaultPriority(1) ''' Multiframe template specific settings ''' self.multiframe.setEnableFrameCheck(1)
As you can see, the first section declares the basic template parameters, giving a template name, description and unique identifier. The logic is set to Multiframe that requires a
self.multiframe.setEnableFrameCheck(1)
if you want to enable the frame check feature. This syntax shows how to access specific logics features.
The next section configures the submission dialog either for Console and the web server:
mayaFile = MTemplateAPI.MTemplateItemFile("MAYAFILENAME","Maya scene file name","Specifies the maya scene file to render","",0,1,1,"*.ma;*.mb") mayaProject = MTemplateAPI.MTemplateItemFolder("MAYAPROJECTPATH","Project directory","Specifies the maya project path","",0,1,1) mayaRenderPath = MTemplateAPI.MTemplateItemFolder("MAYADESTINATION","Frames destination","Specifies the frames destination","",1,1,1) mayaStartFrame = MTemplateAPI.MTemplateItemDouble("MAYASTARTFRAME","Start frame","Specifies the starting frame for the rendering job",1.0,0,0,1) mayaEndFrame = MTemplateAPI.MTemplateItemDouble("MAYAENDFRAME","End frame","Specifies the ending frame for the rendering job",1.0,0,0,1) mayaByFrame = MTemplateAPI.MTemplateItemDouble("MAYABYFRAME","By frame","Specifies the frame step for the rendering job",1.0,0,0,1) mayaNumberBy = MTemplateAPI.MTemplateItemInteger("MAYANUMBERBY","Number by","Specifies the start number for frame numbering",1,0,0,1) mayaStepBy = MTemplateAPI.MTemplateItemInteger("MAYASTEPBY","Step by","Specifies the stepping for the frame numbering",1,0,0,1) mayaDigits = MTemplateAPI.MTemplateItemInteger("MAYADIGITS","Digits","Specifies the number of digits for the frame numbering",1,0,0,1) mayaAbortRender = MTemplateAPI.MTemplateItemYesNo("ABORTRENDER","Abort on log","Immediately abort the rendering if an error string is found during log output",0,0,0,1) self.addSubmissionItem(mayaFile) self.addSubmissionItem(mayaProject) self.addSubmissionItem(mayaRenderPath) self.addSubmissionItem(mayaStartFrame) self.addSubmissionItem(mayaEndFrame) self.addSubmissionItem(mayaByFrame) self.addSubmissionItem(mayaNumberBy) self.addSubmissionItem(mayaStepBy) self.addSubmissionItem(mayaDigits) self.addSubmissionItem(mayaAbortRender) ''' items mapping to Muster default tags ''' self.addMapping("MAYASTARTFRAME","start_frame") self.addMapping("MAYAENDFRAME","end_frame") self.addMapping("MAYABYFRAME","by_frame") self.addMapping("MAYANUMBERBY","number_by") self.addMapping("MAYASTEPBY","step_by") self.addMapping("MAYAFILENAME","job_file") self.addMapping("MAYAPROJECTPATH","job_project") self.addMapping("MAYARENDERPATH","output_folder")
The previous code builds a set of fields object that are installed into the template using the addSubmissionItem() function call. Each object is a subclass of the MTemplateItem base class.
The addMapping() function let you specify which field maps to the required Muster internal fields like the start frame, end frame and so on..
''' Windows support ''' self.platformWindows.setPlatformEnabled(1) self.platformWindows.setEnableErrorCheck(1) self.platformWindows.setEnabledByDefault(1) self.platformWindows.setFramesFloating(3) self.platformWindows.setDetectionLogic(MTemplateAPI.kProcessChild) self.platformWindows.setDetectionLogicProcessName("Mayabatch.exe") applicationPath = MTemplateAPI.MTemplateItemFile("MAYAEXECUTABLE","Maya batch render executable","Path to Maya render.exe application batch render","c:\\program files\\Autodesk\\Maya 2010\\bin\\render.exe",0,0,1,"*.exe") self.platformWindows.addClientConfigurationItem(applicationPath) startingFolderPath = MTemplateAPI.MTemplateItemFolder("MAYASTARTINGFOLDER","Starting folder","Specify the starting folder for the render process","c:\\program files\\Autodesk\\Maya 2010\\bin",0,0,1) self.platformWindows.addClientConfigurationItem(startingFolderPath) processorUsage = MTemplateAPI.MTemplateItemCombo("PROCUSAGE","Processors usage","Specify the number of processors to use for each render instance","0",0,0,1) processorUsage.addItem("All","0") processorUsage.addItem("1","1") processorUsage.addItem("2","2") processorUsage.addItem("3","3") processorUsage.addItem("4","4") self.platformWindows.addClientConfigurationItem(processorUsage)
The previous code sample shows the engine declaration section for the Windows platform. It not all sets platform specific attributes, but it also install a set of fields inside the platform itself. Those fields will be available into the client configuration dialog and can be accessed later to construct the command line as well as to provide additional behaviours.
def onBuildCommandLine(self, platform, job, chunk, clientTemplatePreferences,instanceNum): renderCmd = "-r sw" renderCmd += " -n " + clientTemplatePreferences['PROCUSAGE'] renderCmd += " -proj \"" + job.attributeGetString(manager.resolveMappingToJob(self.getID(),"MAYAPROJECTPATH")) + "\"" if job.attributeIsEnabled(manager.resolveMappingToJob(self.getID(),"MAYADESTINATION")): renderCmd += " -rd \"" + job.attributeGetString(manager.resolveMappingToJob(self.getID(),"MAYADESTINATION")) + "\"" renderCmd += " -s " + '%.3f' % chunk.getStartFrame() renderCmd += " -e " + '%.3f' % chunk.getEndFrame() renderCmd += " -b " + '%.3f' % chunk.getByFrame() renderCmd += " -rfs " + str(chunk.getFrameStart()) renderCmd += " -rfb " + str(chunk.getFrameStep()) renderCmd += " -pad " + job.attributeGetString("MAYADIGITS") renderCmd += " " + job.attributeGetString("ADD_FLAGS") renderCmd += " \"" + job.attributeGetString(manager.resolveMappingToJob(self.getID(),"MAYAFILENAME")) + "\"" return renderCmd
The previous code sample shows how to override a class virtual method to provide a specific function. In this case, we need to tell Muster how to build the final command line.
def onGetApplicationPath(self,job,clientTemplatePreferences,pathOut ): pathOut.setString(clientTemplatePreferences['MAYAEXECUTABLE']) return MTemplateAPI.MTemplateError() def onGetApplicationStartingFolder(self,job,clientTemplatePreferences,pathOut): pathOut.setString(clientTemplatePreferences['MAYASTARTINGFOLDER']) return MTemplateAPI.MTemplateError()
In the previous code, we tell Muster how to find the command line and the starting folder for the batch job. In that case, we grab the values from the client configuration values using the clientTemplatePreferences supplied dictionary.
def onCheckLog(self,job,chunk,clientTemplatePreferences,log): if log.find("Warning") != -1: return MTemplateAPI.MTemplateError(2,"Warning keyword found",MTemplateAPI.MTemplateError.kTemplateErrorTypeWarning) if log.find("Error") != -1: return MTemplateAPI.MTemplateError(1,"Error keyword found",MTemplateAPI.MTemplateError.kTemplateErrorTypeError) if log.find("Total Time For Render") == -1: return MTemplateAPI.MTemplateError(3,"Missing \"Total Time For Render\" keyword. Batch render may have crashed",MTemplateAPI.MTemplateError.kTemplateErrorTypeWarning) if log.find("Total Elapsed Time For Maya") == -1: return MTemplateAPI.MTemplateError(3,"Missing \"Total Elapsed Time For Maya\" keyword. Batch render may have crashed",MTemplateAPI.MTemplateError.kTemplateErrorTypeWarning) if log.find("completed.") == -1: return MTemplateAPI.MTemplateError(3,"Missing \"completed.\" keyword. Batch render may have crashed",MTemplateAPI.MTemplateError.kTemplateErrorTypeWarning) return MTemplateAPI.MTemplateError() def onCheckExitCode(self,job,chunk,clientTemplatePreferences,exitCode): if exitCode == 211: return MTemplateAPI.MTemplateError(exitCode,"Abnormal render termination",MTemplateAPI.MTemplateError.kTemplateErrorTypeError) if exitCode != 0: return MTemplateAPI.MTemplateError(exitCode, "%d Exit code different from expected 0 value" % exitCode,MTemplateAPI.MTemplateError.kTemplateErrorTypeWarning) return MTemplateAPI.MTemplateError()
The previous virtual functions tell Muster how to process the batch job output log as well as the process exit code.
''' Create an instance of the template class and install it into the template manager context ''' template = MayaTemplate() manager.installPyTemplate(template)
The last code creates an instance of the class and installs it into the template manager.
Further information on the template syntax can be found looking into the API classes reference as well as into the existing templates.
Class overrides descriptions
Any template is able to override Muster default behaviours by redefining one or more virtual functions of the base class. This is a list of the available functions and their meaning:
Function name and parameters | Expected return variable | Description |
---|---|---|
onFieldChanged(string fieldName,string fieldValue, ref MTemplateItemsFields fieldsToChange) | void | Invoked by Console when a template field value changes, useful to automate fills of different fields and/or invoke actions like scene parsing after the file selection |
example | ||
onBuildCommandLine(int platform, MJob job, MChunk chunk,ref stringMap clientTemplatePreferences, int instanceNum) | string | Invoked by the client to build the process command line |
example | ||
onBuildAdditionalParameters(ref MPropertiesMap attributes) | string | Invoked by Console to build additional parameters for the command line |
example | ||
onGetApplicationPath(MJob job,ref stringMap clientTemplatePreferences, ref MStringRef pathOut ) | MTemplateError | Invoked by the client to get the process path |
example | ||
onGetApplicationStartingFolder(MJob job,ref stringMap clientTemplatePreferences, ref MStringRef pathOut) | MTemplateError | Invoked by the client to get the process execution folder |
example | ||
onBuildEnvironment(MJob job, MChunk chunk,ref stringMap clientTemplatePreferences, ref MEnvironment existingEnvironment) | MEnvironment | Invoked by the client to build an environment block for the process |
example | ||
onCheckForSubframeAdvancingString(string line) | bool | Invoked by the client on each log line parsing to check for a frame completion string |
example | ||
onDetectRunningProcess(MPid mainProcess, vector of MProcessSnapshot hostProcesses, ref stringMap clientTemplatePreferences,ref MPid runningProcessPid) | kTemplateFunctionBehaviour (int constant) | Invoked by the client to detect the process effective running PID |
onCheckForFramesPrefixString(string line, ref MStringRef prefixOut) | bool | Invoked by the client on each log line parsing to guess the frames path |
example | ||
onCheckForSubframeProgress(string line, ref MStringRef progressOut) | bool | Invoked by the client on each log line parsing to guess the percentage of the current frames |
example | ||
onCheckLogLine(MMJob job,MChunk chunk,ref stringMap clientTemplatePreferences, string line) | MTemplateError | Invoked by the client for each log line, this is useful to abort the current render on certain log errors |
example | ||
onCheckLog(MJob job, MChunk chunk,ref stringMap clientTemplatePreferences, string log) | MTemplateError | Invoked by the client to check the full log at the end of the render |
example | ||
onCheckExitCode(MJob job,MChunk chunk,ref stringMap clientTemplatePreferences, int exitCode) | MTemplateError | Invoked by the client to check the process exit code |
example | ||
onApplicationFinder(ref MStringRef moduleRegExp, ref MStringRef& moduleTag) | int | Invoked by the Services control panel applet to configure automatic application finding |
example | ||
onFindApplication(ref MStringRef basePath,ref stringMap clientTemplatePreferences) | bool | Invoked by the Services control panel applet to find a specific application directly from the template |
example | ||
onModuleFound(ref MStringRef moduleExec,ref MStringRef modulePath,ref stringMap clientTemplatePreferences); | bool | Invoked by the Services control panel applet when an application executable is found |
example | ||
onGetSlicesInputFilename(MJob job,MChunk chunk) | string | Invoked by image slicing jobs to know each chunk slice file path |
example | ||
onGetSlicesOutputFilename(MJob job) | string | Invoked by image slicing jobs to know the final assembled image name |
example | ||
onGetSliceBoundaries(MJob job,MChunk chunk,MTemplateSliceBoundaries boundaries) | void | Invoked by image slicing jobs to know how to expect slices and how to assemble them |
example | ||
onStartProcess(MJob job, int instanceNum, MPid& runningProcessPid)) | void | Invoked by the client when a new process should be started |
example | ||
onTerminateProcess(MJob job, MPid process, int instanceNum)) | void | Invoked by the client when a running process should be terminated |
example | ||
onChangeProcessPriority(MJob job, MPid process, int priority, int instanceNum)) | void | Invoked by the client when the system priority of a process should be changed |
example | ||
onCheckProcessTermination(MJob job, MPid process, int instanceNum)) | void | Invoked by the client to check if the process has been completed |
example |
The following functions are also called from the global template (ID 0):
Function name and parameters | Expected return variable | Description |
---|---|---|
onValidateJobSubmission(MJob job, ref MStringRef err) | bool | Used to validate a job submission |
example | ||
onDeleteJob(MJob job) | void | Invoked when a job is deleted |
example | ||
onSubmitJob(MJob job) | void | Invoked when a job is submitted |
example | ||
onReinitJob(MJob job) | void | Invoked when a job is reinit |
example | ||
onEditJob(MJob job) | void | Invoked when a job is edited |
example | ||
onJobPropertiesChanged(MJob job) | void | Invoked when a property of a job is changed |
example | ||
onJobStart(MJob job) | MTemplateError | Invoked when the job is started |
example | ||
onJobEnd(MJob job) | MTemplateError | Invoked when a job is completed |
example | ||
onChunkStart(MJob job,MChunk chunk, ref stringMap clientTemplatePreferences,int instanceId) | MTemplateError | Invoked when a chunk is started |
example | ||
onChunkEnd(MJob job,MChunk chunk,ref stringMap clientTemplatePreferences, int instanceId) | MTemplateError | Invoked when a chunk is completed |
example |