Muster Console walk trough
Muster Console is the graphical interface to the Dispatcher Service. Each operation on the render farm and on the queue is performed through it.
Launch the Console. This can be done from the Start menu in Windows, from the Application folder in OS X and from a command line in Linux. You'll get a welcome screen:
The first thing you need to do is to log in on the dispatcher service, so click on the first icon on the left side of the toolbar. A connection window will appear:
Insert your dispatcher service host name or IP address, put admin as username and leave the password field blank, then click on Connect.
If the connection is successfully, you’ll get the Console interface:
The Muster Console interface is made by four different view panels:
The Instances/hosts view: This is the upper view in the screenshot above. This is basically a list of your available and unavailable rendering instances and can be configured in Instance or Host view mode. Consider an host like a physical machine, while an instance is an entity inside Muster. One host is able to spawn multiple instances. In this way, you can have simultaneous batch renders running on the same host and use multi core and multi processors machines even the batch render doesn’t support it. In Instance view mode, each instance has its own entry in the list, while in host view mode, instances are grouped under their host node. Operations that are pertinent to a single instance can be done in Instance view mode, while operations that are pertinent to the host , like starting and stopping services, are permitted in host view mode only.
This is an image of a typical view in host mode:
While this is an image of a typical view in instance mode:
The colored dot at the left of each instance shows the actual status of the machine. When the machine is idle and ready to render, it is green. When it’s idle but paused, it’s yellow. When it’s rendering, it’s red. After your first installation, you should have at least one instance with a green dot. If it’s yellow, you may have configured the client to start paused. You’ll change its status later.
The instances/hosts views can be filtered using the combo controls at the top of the view itself. You can also customize the columns by right-clicking on the view headers. If you’re working on a custom workspace (see the workspaces section) , each change you make in the view, will be stored persistently and left untouched at the next Console startup! This allows you to build your very customized workspace with multiple views filtered in different ways!
Before going further, we’ll check that the hosts we are going to use have the correct paths configured. Just switch to host view mode, right click on a node and select Configuration→Configure. Once you get the configuration window, move to the Templates section, locate the Maya sw template we are going to use for this test, and verify that it’s correctly pointing to the Render.exe file required to launch Maya batch renders, double check that the process starting folder is inside the bin directory of your Maya installation. Also check that it points to the correct Maya version, the one that you used to produce the scene file and that’s enabled with a checkmark near the engine name. Click on Ok and your configuration will be stored persistently on the host. Later, you'll learn to configure multiple versions of the same software if you need it.
As said before, this configuration will apply to any instance belonging to the host and does not require a restart of the service. Your changes will be available immediately.
The next view, the one shown in the central pane contains the dispatcher job queue and it’s called Queue view. Jobs can be organized inside folders so it’s possible to have collapsed or expanded nodes. The icon on the left of the node shows the status of the node itself and in case of a folder, the overall status of the childs.
This is a typical queue view:
After your first installation, this view will be empty. You will need to submit some jobs to populate it.
The bottom pane contains the dispatcher internal log. It’s a resume of the most important events happened on the Dispatcher service. This is a typical log view:
You should always take a look at the log during the rendering process. Every error reported from render clients or directly from the dispatcher will be logged here and is your primary error detection tool.
The last pane, the one on the right, is the submission view. From this window you can submit new jobs, inspect parameters related to jobs inside the queue, load and save presets. The Submission view is a dynamic property sheet. It will change contents depending on the currently selected render engine:
You can configure the submission panel to automatically load the jobs selected in a particular queue view. Just enable the Load selection from view option at the top of the submission dialog.
To explain its functionalities, we will submit our first job!
Before doing that, we have to check that the internal Dispatcher engine is running.
When we talk about the Dispatcher engine, we talk about its internal logic that does all the heavy stuff, select jobs, check nodes and control the entire Muster features. By stopping the Dispatcher engine, you stop any kind of future activity on the Dispatcher. It won’t stop any render in progress but will prevent the submission of new ones. Stopping the Dispatcher may be useful in those situations where you need to configure something on the Dispatcher itself and you don’t want any activity until you complete the configuration.
You can immediately check the activity of the Dispatcher by looking at the lower right corner of the Console. If you see a green moving bar, the Dispatcher is active. If not, just start it by pressing the F10 key, or selecting the Change Engine Status menu entry in the Management menu.
Now check that our instances are available for rendering. If some of them have a yellow icon, it means they are paused. Just select them, right click and select Resume. They should change to the idle status and show a green ball.
As a last step, we want to exactly check what’s generated from the instances during their render, so right-click on the instance you’re going to use, and select Realtime Log Streaming → Enable and Open view. By selecting this menu item, Muster Console will open an output window that will dump the output from the processes spawned on the remote host:
We are ready to select our rendering engine from the submission view. In the general section, locate the Engine combo box, and select “Maya sw” as the render engine. This will tell Muster to render using the Maya software rendering.
Next, select the job file by clicking the button on the right of the Maya scene filename field, in the Maya Sw section of the submission panel (notice that this section has been dynamically built if you had a different render engine selected).
Pick up the file from the network, you should have something like \\MASTER\RENDERFARM\MUSTERTEST\SCENES\TEST.MB inside the field. Muster will detect automatically the project path and will set it to \\MASTER\RENDERFARM\MUSTERTEST
Set the starting frame to 1 and the ending frame to 10. Because we want to send the job to an high number of instances, set the packet size to 1.
The packet size value controls how many frames are assigned to a single instance for each render session (a chunk). Under production, a value of 4 is often a good compromise.
Click on Submit and you should immediately see the job appearing in the queue. If it has a yellow icon, just right click on it and select Resume.
Congratulations, you’ve sent your first render job. In a few minutes, you should get all your instances back to idle and the frames successfully rendered inside the images folder of your project.
If your render does not start and you get a warning icon under the J or T column of your node, that means your job has been blocked by Muster and has been inserted in the "exclusion" lists. The templates exclusion list prevent any jobs belonging to a particular engine to be submitted to a certain host, while the jobs exclusion list prevent single jobs to be submitted to a certain host. 99% of the times a job or a template are blocked because you have a bad configuration of the render engine paths. If this happens, please refer to the Dispatcher botton log, and check for an error line. The reason of the failure is there!
In case you got the J and T column of a node populated, after you find the reason of the problem, you need to “purge” those lists otherwise further jobs won't be processed by that specific instance. Just right click on the instance and select Purge templates or Purge jobs exclusion lists. The renders will resume and if your configuration is fixed, this time, will go on successfully.
During this example, you watched the output from the batch renders using the real time log feature. But what about checking the logs of completed packets ? No problem, just right click on the job you want to inspect and select Chunks view, you'll get a similar window:
Now move to the bottom “Logs inspector pane” and double check on the chunk you're interested in. You'll get a detailed resume of the errors and warnings generated by the render engine. If you want to highlight them inside the entire render log, right click on the error, and select Highlight in log. The packet log is streamed from the instance and you'll get the workstations logs view window with the error highlighted in the log itself.
There are several ways to access the logs. You can use the chunks view as we just explained, or you can manually open the Workstation logs view by right clicking on an instance, and select Workstation Logs.
You can also right click on a Dispatcher bottom log entry that reports the success or the failure of the chunk, right click on the entry, and open the log from there.
The logs have the solution to your problem. Muster is not able to always understand what’s going on during a batch rendering process. The output from the batch render always contains enough hints to understand why a particular job is failing!
Okay, what about your rendered frames ? If your render has been successfully or at least partial frames have been rendered, you can simply right click on your job, and select Open frames in viewer. If Muster has been able to detect your render destination from your logs, the viewer will open automatically, otherwise you may find this menu entry disabled. In that case, you may need to select manually where the frames have been rendered (this also applies to certain render engine that does not output the render path from the log). Just right click on the job, and select Jobs settings → Set frames path mask.
After selecting a frame from your renders, the mask is calculated automatically and you can open the image viewer:
From the image viewer, you can scrub and preview in realtime your renderer frames. You also have access to several image tuning parameters to inspect your HDR renders like .EXR files. If you have launched the image viewer from within a Job, connection between the viewer window and the job is retained. That means you can inspect frames, and re queue chunks and/or singolar frames directly from within the image viewer window. Please refer to the Console reference to look at the available parameters and functions of the Image Viewer.
You can start playing with the various Muster parameters or move to the reference section of each module to gain a deeper knowledge of the software.
If something went wrong, we suggest you to check carefully each section of this chapter and verify your steps. If you still can’t get a valid result, check the Troubleshooting section at the end of this manual.