API call to create job

Tagged: ,

This topic contains 6 replies, has 2 voices, and was last updated by  Leonardo Bernardini 3 months, 1 week ago.

Viewing 7 posts - 1 through 7 (of 7 total)
  • 2nd May 2019 at 5:33 pm #17479

    Hello,

    I am trying to integrate Muster into pipeline. I need to submit job to farm via API call and add some post-render action (basically python script) when job is done. Is there some documentation for required job data I need to collect and send via HTTP API?

    I came over: https://www.vvertex.com/wiki900/doku.php?id=muster:9.0:restful_http_api – there is example with submit. I tried to pick it up from maya submit plugin (MayaMuster9.py) but that is quite a beast with over 5000 lines. Some hint where to find those data and add custom post render action like python publish_muster.py would be appreciated.

    Thanks,

    Ondrej

    2nd May 2019 at 6:39 pm #17480

    Hi Ondrej,

    If you want to submit a job using the REST APIs do not look at the Maya connector because it uses the Muster command line tool (mrtool) to submit the job so it is of no use for you. Consider that you can use it also to submit a job, but let’s assume you want to use the REST APIs that will be also the default method in the future..

    Basically you need two steps:

    1) Perform a login using a login/password and get back an access token. This requires you to call /api/login and parse the JSON response
    2) Submit a job using the queue method and passing also the access token.
    3) Optional, deallocate your session by performing a logout

    The login URL will be something like this:

    http://musterserver:9890/api/login?username=admin&password=mypassword

    and you’ll get back a JSON like this:

    {
        "ResponseData":
        {
            "authToken": "426200942D0501285315C82F"
        },
        "ResponseStatus":
        {
            "Code": 200,
            "Time": 1339840278,
            "Description": "OK"
        }
    }

    Always check that the response status is 200 as an additional error check , or the description if it different from 200.

    After you get an access token, you can submit the job using a POST REQUEST, the job data must be into the POST data:

    http://musterserver:9890/api/queue/actions?authToken=426200942D0501285315C82F&name=submit
    {
    	"RequestData":
    	{
    		"job" : 
    		{
    			"jobName": "my brand new job!",
                    	"templateId": 2,
                 		"parentId": -1,
                    	"dependIds": [],
                    	"dependMode": 0,
    			"packetType" : 1,
                   		"priority": 1,
                    	"group": "",
                    	"department": "",
                    	"packetSize": 5,
    			"maximumInstances": 0,
                   		"assignedInstances": 0,
                    	"attributes":
                    	{
                        		
    			}
    		}
    	}
    }

    Those are the required steps, also you need to pass specific attributes depending on the render engine you’re targeting and you need also to pass information about the post job action you want to run.

    Now, the easiest way to construct your basic job template is looking at our integrated web server. If you open it , and submit a job from within the interface keeping the developer tools open in Chrome, you can check how the web application sends the job to muster, and collect the entire POST DATA structure, based on your settings. You can them replicate it from your script.

    If you tell me the Maya render engine you’re targeting as well as listing all your job submission parameters, I’ll run it into the web browser for you and give you back the exact POST data to use.

    Hope this helps you !

    2nd May 2019 at 11:01 pm #17483

    Thank you for all those useful informations, they certainly shed some light into it. I’ll try to look into data sent by submission to integrated web server. One more question (for now at least :)) – I’ve got from the submission json:

    ...
    "post_job_action": {
              "value": "python /opt/pipeline/publish_filesequence.py",
              "state": true,
              "subst": true
            },
    ...

    (https://www.vvertex.com/wiki900/doku.php?id=muster:9.0:custom_actions)
    Just for clarification – as this is job action, it will be run by Dispatcher, so dispatcher machine will start that python script, right? This command will be issued in shell of dispatcher machine under user running dispatcher service …

    Quick side note – my very brief dive into Muster left me somehow impressed, good job there!

    3rd May 2019 at 10:48 am #17484

    Hi Ondrej,

    We are happy you’re having good impressions from Muster!

    Yes, a post or pre job action is performed on the Dispatcher, it is launched from a separate thread and won’t stall any dispatcher activity, the job will enter into a post-job action status and the executable is launched.
    The permission are inherited from the user that’s actually running the Dispatcher service, but it won’t be launched from within a shell but as a direct child process. If you need shell environmental variables , you need to launch a shell using /bin/sh -c python…. , this will parent the python process to an effective shell, that depends on what your script needs but in most situation, it’s better to avoid the shell and set all your variables inside your script.

    3rd May 2019 at 10:52 am #17485

    Additional note, in the command line you launch, you can use Python inline substitution as explained in the manual, if you need to pass it any job parameter. For a real life example, the must useful thing it to launch the Muster python interpreter (instead of the system one) that’s the mpython executable and pass the job ID to a script. I.e /usr/local/muster9/mpython myscript.py < %=job.getJobId()%> . In that way, you can then import the muster API inside myscript.py, reconnect to the Dispatcher and get the full MJob object back , and perform all your stuff.
    Off course this is for a situation where you need to extract all the jobs parameters, if you need just a few parameters, you can just pass the parameters you need directly to the command line and use your python interpreter.

    5th May 2019 at 11:53 pm #17488

    Thanks again. I am going through it right now. About the attributes – can you please explain what exactly means that state and subst flag? State is I suppose enabled/disabled, but subst? Also, does it mean for example that:

    "job_file": {
         "value": "/foo/bar.mb",
         "state": True,
         "subst": True
    }

    that if I set state to False, it will ignore the value?

    Thanks,

    O.

    7th May 2019 at 9:11 am #17490

    Sure, state if it is enabled or disabled, like you can have an attribute filled in console but actually disabled.
    The subs flags tells Muster if the attribute should pass the substitution paths engine (i.e. you’re submitting from a different platform than the Dispatcher or in general you’re submitting a path/file path)

Viewing 7 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic.