Alex Schack

I ran some tests using different combinations of the borrow instances feature.

I did run into one main issue. If I set the borrow instances to 3, it only renders on the machine with 4GPUs/4 instances. It doesn’t render on any machines with less than 4 instances, which makes sense. But I don’t think that will work with our configuration.

We might need to consider going back to the first option. Splitting each GPU into a separate instance with the GPU mask. I’m going to run a couple of hardware tests by adding more RAM to the crashing machines to see if that is causing the issue.

Oh wait, I think I see.

Is it the minimum physical GPUs setting?

Thank you Leonardo! This is very helpful info!

Could you explain more about point number 2?

the first one is to not set the GPU affinity mask and set the number of GPUs per instances to 1

Is this basically what my current setup is?

I’d be interested to hear more about the borrow instance option. It think that might be a good option. How would this work with the combination of other machines with only two cards?

Could you walk me through the setup for something like this?

you can send light jobs to 4 instances and heavy jobs to 2 instances with 2GPUs each one.

This seems pretty straight-forward and, in theory, it could also be set to 1 instance with 4GPUs in case we want to use the “old” method correct?

Thanks again!

[MUSTER]Spawning process C:\Program Files\Autodesk\Maya2019\bin\Render.exe inside C:\Program Files\Autodesk\Maya2019\bin using the following command line flags:
[MUSTER]-r redshift -logLevel 1 -gpu {2} -proj “\\192.168.1.6\Xsan\_PROJECTS\[PROJECT_PATH_REMOVED]\3D\Maya” -s 41.000 -e 44.000 -b 1.000 -rfs 41 -rfb 1 -pad 1 -rl particles “\\192.168.1.6\Xsan\_PROJECTS\[PROJECT_PATH_REMOVED]\3D\Maya\scenes\sn05_Fingerprint\sn05_sh01_printHallway\sn05_sh02_PrintHallway_MusterTest_v02.mb”

Starting “C:\Program Files\Autodesk\Maya2019\bin\mayabatch.exe”
VP2 Error : Failed to initialize graphics device.

Cannot open file ‘:/expression.svg’, because: No such file or directory

Cannot open file ‘:/expression.svg’, because: No such file or directory

Cannot open file ‘:/expression.svg’, because: No such file or directory

Cannot open file ‘:/expression.svg’, because: No such file or directory

[Redshift] Redshift for Maya 2019

[Redshift] Version 3.0.13, Dec 20 2019

Installing Redshift MEL overrides…

Sourcing //192.168.1.6/Xsan/_INTERNAL/Utility/Redshift/Plugins/Maya/Common/scripts/override/2019/connectNodeToAttrOverride.mel

Sourcing //192.168.1.6/Xsan/_INTERNAL/Utility/Redshift/Plugins/Maya/Common/scripts/override/2019/createMayaSoftwareCommonGlobalsTab (2).mel

Sourcing //192.168.1.6/Xsan/_INTERNAL/Utility/Redshift/Plugins/Maya/Common/scripts/override/2019/createMayaSoftwareCommonGlobalsTab.mel

Sourcing //192.168.1.6/Xsan/_INTERNAL/Utility/Redshift/Plugins/Maya/Common/scripts/override/2019/createRenderNode.mel

Sourcing //192.168.1.6/Xsan/_INTERNAL/Utility/Redshift/Plugins/Maya/Common/scripts/override/2019/mayaBatchRender.mel

Sourcing //192.168.1.6/Xsan/_INTERNAL/Utility/Redshift/Plugins/Maya/Common/scripts/override/2019/MLdeleteUnused.mel

Sourcing //192.168.1.6/Xsan/_INTERNAL/Utility/Redshift/Plugins/Maya/Common/scripts/override/2019/relationshipEditor.mel

Sourcing //192.168.1.6/Xsan/_INTERNAL/Utility/Redshift/Plugins/Maya/Common/scripts/override/2019/renderWindowPanel.mel

Sourcing //192.168.1.6/Xsan/_INTERNAL/Utility/Redshift/Plugins/Maya/Common/scripts/override/2019/renderWithCurrentRenderer.mel

File read in 4.7 seconds.

Result: //192.168.1.6/Xsan/_PROJECTS/[PROJECT_PATH_REMOVED]/3D/Maya/scenes/sn05_Fingerprint/sn05_sh01_printHallway/sn05_sh02_PrintHallway_MusterTest_v02.mb

[Redshift] Cache path: C:\Users\pixel\AppData\Local\Redshift\Cache

[Redshift]

[Redshift] Redshift Initialized

[Redshift] Version: 3.0.13, Dec 20 2019

[Redshift] Windows Platform (Windows 10 Pro)

[Redshift] Release Build

[Redshift] Number of CPU HW threads: 16

[Redshift] CPU speed: 3.79 GHz

[Redshift] Total system memory: 63.88 GB

[Redshift] TDR delay: 60s

[Redshift] Current working dir: C:\Program Files\Autodesk\Maya2019\bin

[Redshift] redshift_LICENSE=5055@192.168.3.25

[Redshift] Creating CUDA contexts

[Redshift] CUDA init ok

[Redshift] Ordinals: { 2 }

[Redshift] Initializing GPUComputing module (CUDA). Ordinal 2

[Redshift] CUDA Ver: 10020

[Redshift] Device 3/4 : GeForce RTX 2080 Ti

[Redshift] Compute capability: 7.5

[Redshift] Num multiprocessors: 68

[Redshift] PCI busID: 66, deviceID: 0, domainID: 0

[Redshift] Theoretical memory bandwidth: 616.000000 GB/Sec

[Redshift] Measured PCIe bandwidth (pinned CPU->GPU): 6.110065 GB/s

[Redshift] Measured PCIe bandwidth (pinned GPU->CPU): 2.536454 GB/s

[Redshift] Measured PCIe bandwidth (paged CPU->GPU): 3.767165 GB/s

[Redshift] Measured PCIe bandwidth (paged GPU->CPU): 2.396808 GB/s

[Redshift] Estimated GPU->CPU latency (0): 0.049223 ms

[Redshift] Estimated GPU->CPU latency (1): 0.049464 ms

[Redshift] Estimated GPU->CPU latency (2): 0.049904 ms

[Redshift] Estimated GPU->CPU latency (3): 0.050430 ms

[Redshift] New CUDA context created

[Redshift] Available memory: 9248.9625 MB out of 11264.0000 MB

[Redshift]

[Redshift] Loading Redshift procedural extensions…

[Redshift] Done!

[Redshift] Redshift for Maya 2019

[Redshift] Version 3.0.13, Dec 20 2019

[Redshift] Rendering layer ‘rs_particles’, frame 41 (1/4)

[Redshift] Scene translation time: 4.52s

[Redshift] =================================================================================================

[Redshift] Rendering frame 41…

[Redshift] AMM enabled

[Redshift] =================================================================================================

[Redshift] License acquired

[Redshift] License for redshift-core 2020.09 (permanent)

[Redshift]

[Redshift] Rendering time: 17.5s (1 GPU(s) used)

[Redshift] Saved file ‘\\192.168.1.6\Xsan\_PROJECTS\[PROJECT_PATH_REMOVED]\3D\Maya\images\sn05_Fingerprint\sn05_sh02\sn05_sh02_PrintHallway_MusterTest_v02\particles.41.exr’ in 1.34s

[Redshift] Frame done – total time for layer ‘rs_particles’, frame 41 (1/4): 25.11s

[Redshift] License returned

[Redshift] Rendering layer ‘rs_particles’, frame 42 (2/4)

[Redshift] Scene translation time: 2.88s

[Redshift] =================================================================================================

[Redshift] Rendering frame 42…

[Redshift] AMM enabled

[Redshift] =================================================================================================

[Redshift] License acquired

[Redshift] License for redshift-core 2020.09 (permanent)

[Redshift]

[Redshift] Rendering time: 17.1s (1 GPU(s) used)

[Redshift] Saved file ‘\\192.168.1.6\Xsan\_PROJECTS\[PROJECT_PATH_REMOVED]\3D\Maya\images\sn05_Fingerprint\sn05_sh02\sn05_sh02_PrintHallway_MusterTest_v02\particles.42.exr’ in 1.34s

[Redshift] Frame done – total time for layer ‘rs_particles’, frame 42 (2/4): 26.02s

Stack trace:

DependEngine.dll!TplugChildren::find

DependEngine.dll!Tplug::findInBranchingNet

OpenMaya.dll!Autodesk::Maya::OpenMaya20190000::MPlug::isConnected

OpenMaya.dll!MPyUtil::valueToObject<Autodesk::Maya::OpenMaya20190000::MPlug>

python27.dll!PyComplex_AsCComplex

python27.dll!_PyObject_GenericGetAttrWithDict

python27.dll!PyEval_EvalFrameEx

python27.dll!PyEval_EvalCodeEx

python27.dll!PyFunction_SetClosure

python27.dll!PyObject_Call

python27.dll!PyMethod_New

python27.dll!PyObject_Call

python27.dll!PyEval_CallObjectWithKeywords

CommandEngine.dll!TpythonInterpreter::callPythonFunction

OpenMaya.dll!THclient::operator=

OpenMaya.dll!THclient::callbackInScriptNewAPI

OpenMaya.dll!THclient::userAttributeChangedCallback

OpenMaya.dll!THclient::userAttributeChangedCallbackCB

Foundation.dll!TclientServer::notifyClients

DependEngine.dll!TdependNode::sendAttributeChangedMsgInternal

DependEngine.dll!TdrawDbChangeTracker::isListeningToSource

DependEngine.dll!TevaluationManager::executeAsWorker
DependEngine.dll!TparallelExecution::sAddNodesWithinDepthToSet

tbb.dll!tbb::interface7::internal::task_arena_base::internal_execute

DependEngine.dll!TparallelExecution::evaluateImplementation

DependEngine.dll!TevaluationGraph::evaluate

DependEngine.dll!TevaluationManager::evaluate

DependEngine.dll!TcipEvaluationManager::operator=

DependEngine.dll!TdependGraph::setTime

OpenMayaAnim.dll!Autodesk::Maya::OpenMaya20190000::MAnimControl::setCurrentTime

redshift4maya.mll!uninitializePlugin

redshift4maya.mll!uninitializePlugin

redshift4maya.mll!uninitializePlugin

redshift4maya.mll!uninitializePlugin

redshift4maya.mll!uninitializePlugin

redshift4maya.mll!uninitializePlugin

redshift4maya.mll!uninitializePlugin

OpenMaya.dll!THcommandObject::doIt

CommandEngine.dll!TmetaCommand::doCommand

CommandEngine.dll!TmetaCommandPtrArray::catenate

Result: //192.168.1.6/Xsan/_PROJECTS/[PROJECT_PATH_REMOVED]/3D/Maya/scenes/sn05_Fingerprint/sn05_sh01_printHallway/sn05_sh02_PrintHallway_MusterTest_v02.ma

Fatal Error. Attempting to save in C:/Users/pixel/AppData/Local/Temp/pixel.20200210.1521.ma

// Maya exited with status -1073741819

[MUSTER]Process terminated with exit code: -1073741819