Databases structure

Muster stores the queue and internal configurations in a database that may be on a local file (using sqlite) or external (Mysql or SQL Server).

An internal abstraction layer is used for certain data types, the field types you’ll find in the descriptions may appear differently depending on the destination database.

The following section lists the internal tables and their purpose. You are just allowed to read data from the Muster database. You are not allowed to write data neither change the data structure of the tables: Muster will check the structure on any service start-up and rebuild them according.

Queue database tables:

  • db_jobs: Table holding the jobs general data either for folders and real jobs
  • db_jobs_data: Table holding attributes for real jobs as well as custom attributes
  • db_chunks: Table holding the chunks data
  • db_logs: Table holding the internal Muster log
  • db_mails: Table holding distribution lists
  • db_pools: Table holding pools configuration
  • db_users: Table holding users configuration
  • db_users_groups: Table holding groups configuration
  • db_paths: Table holding global substitution paths

History database tables:

  • db_history_chunks: Table holding the Muster processing history
  • db_history_stats: Table holding the Muster loads statistics

Backup database tables:

  • backup_db_jobs: Table holding the jobs general data either for folders and real jobs
  • backup_db_jobs_data: Table holding attributes for real jobs as well as custom attributes
  • backup_db_chunks: Table holding the chunks data

db_jobs

Field nameField typeDescription
idintPrimary key
parent_idintID of the parent folder
indentintLevel of indentation
statusintJob status
tasks_maskstringJob tasks mask
chunks_priorityintThe chunks priority type
chunks_interleaveintThe chunks priority interleave value
packet_typeintType of job packets
packet_sizeintSize of each chunk
requeuedintNumber of times job has been requeued
engineintTemplate uid
engine_versionstringThe template version
max_cpuintMaximum number of concurrent instances
max_cpu_workingintNumber of current working instances
depend_idstringDepend from ids
depend_modeintDepend mode used
depend_link_modeintDepend link mode
job_typeintJob type, Folder or real job
job_namestringName of the job
priorityintPriority for the job
is_pausedboolPause status
is_lockedboolLocked status
job_poolstringTarget pools comma separated
Job_excluded_poolstringExcluded pools comma separated
submission_timeint time_tTime of submission
start_timeint time_tTime of job execution
progressintJob progress
end_timeint time_tTime of job termination
start_onint time_tSpecify the scheduled starting time for the job
resume_onint time_tSpecify the scheduled resume time for the job
pause_onint time_tSpecify the scheduled paused time for the job
submitterstringUsername of the job owner
exitcodes_err_check_typeintSpecify the type of exit codes check
logs_err_check_typeintSpecify the type of logs texts error check
emergency_queueboolSpecifies if the job is into the emergency queue
logs_parsing_rulesstringEncoded overriden logs parsing rules
override_notificator_notifications_at_chunk_levelintOverrides of notifications
override_notificator_notifications_at_job_levelintOverrides of notifications
notificator_notifications_at_chunk_level_typeintOverrides of notifications
notificator_notifications_at_job_level_typeintOverrides of notifications
override_mobile_notifications_at_chunk_levelintOverrides of notifications
override_mobile_notifications_at_job_levelintOverrides of notifications
mobile_notifications_at_chunk_level_typeintOverrides of notifications
mobile_notifications_at_job_level_typeintOverrides of notifications
override_mail_notifications_at_chunk_levelintOverrides of notifications
override_mail_notifications_at_job_levelintOverrides of notifications
mail_notifications_at_chunk_level_typeintOverrides of notifications
mail_notifications_at_job_level_typeintOverrides of notifications
override_start_notificator_notifications_at_chunk_levelintOverrides of notifications
override_start_notificator_notifications_at_job_levelintOverrides of notifications
start_notificator_notifications_at_chunk_level_typeintOverrides of notifications
start_notificator_notifications_at_job_level_typeintOverrides of notifications
override_start_mobile_notifications_at_chunk_levelintOverrides of notifications
override_start_mobile_notifications_at_job_levelintOverrides of notifications
start_mobile_notifications_at_chunk_level_typeintOverrides of notifications
start_mobile_notifications_at_job_level_typeintOverrides of notifications
override_start_mail_notifications_at_chunk_levelintOverrides of notifications
override_start_mail_notifications_at_job_levelintOverrides of notifications
start_mail_notifications_at_chunk_level_typeintOverrides of notifications
start_mail_notifications_at_job_level_typeintOverrides of notifications
override_chunks_timeoutbooleanOverrides chunks timeout state
override_chunks_timeout_valueintOverrides chunks timeout value
is_archivedbooleanJob is archived
job_projectstringJob's project
job_departmentstringJob's department
job_camerastringJob's camera
job_shotstringJob's shot
job_sequencestringJob's sequence
override_valid_exit_codesstringOverrides of exit codes
override_warning_exit_codesstringOverrides of exit codes
override_error_exit_codesstringOverrides of exit codes
override_minimum_threadsboolOverrides minimum threads
override_minimum_coresboolOverrides minium cores
override_minimum_physicalboolOverrides physical cores
override_minimum_speedboolOverrides minimum speed
override_minimum_ramboolOverrides minimum RAM
override_minimum_disk_spaceboolOverrides minimum disk space
override_maximum_chunks_requeueboolOverrides maximum chunks requeue
override_maximum_chunks_requeue_valueintOverrides maximums chunks requeue value
override_valid_exit_codes_valuestringOverrides of exit codes value
override_valid_warning_codes_valuestringOverrides of exit codes value
override_error_exit_codes_valuestringOverrides of exit codes value
override_minimum_threads_valueintOverrides minimum threads value
override_minimum_cores_valueintOverrides minimum cores value
override_minimum_physical_valueintOverrides minimum physical value
override_minimum_speed_value floating Overrides mimumum speed value
override_minimum_ram_value floating Overrides minimum RAM value
override_minimum_disk_space_valuefloatingOverrides minimum disk space value

db_jobs_data

Field nameField typeDescription
idintPrimary key
job_idintJob ID reference
key_namestringAttribute name
key_valuestringAttribute value
key_stateboolAttribute activation status
key_substboolAttribute substitution status

db_chunks

Field nameField typeDescription
idintPrimary key
chunk_idintSequential chunk id
job_idintJob ID reference
priorityintThe chunk priority
statusintChunk status
packet_typeintThe chunk packet type
requeuedintNumber of times chunk has been requeued
start_framefloatingStarting frame for the chunk
end_framefloatingEnding frame for the chunk
by_framefloatingBy framing for the chunk
starting_timeint time_cChunk starting time
ending timeint time_cChunk ending time
rendering_hoststringInstance name that processed the chunk
resultstringError string returned by the process
starting_numberintStarting numbering for the chunk
is_assemblerboolFlagged for assembler chunks
taskstringThe chunk's job task
depend_from_chunksstringDepends from chunks IDs
depend_from_chunks_error_levelsintDepends error level
warning_markersstringWarnings markers
error_markersstringError markers
silenced_warning_markersstringWarning silenced markers
silenced_error_markersstringError silenced markers

db_logs

Field nameField typeDescription
idintPrimary key
host_ipstringIP of the host that generated the log
log_timestringFormatted time of the log
host_namestringName of the instance that generated the log
event_userstringUsername of the user
log_entrystringText of the log entry
log_typeintLog type
job_idintThe job ID the log entry reefer too
chunk_idintThe chunk ID the log entry reefer too
instance_idintThe instance ID the log entry reefer too
history_chunk_idintThe history chunk ID the log entry reefer too

db_paths

Field nameField typeDescription
idintPrimary key
namestringRepository name
local_pathstringDispatcher side path
winstringWindows side path
linuxstringLinux side path
macstringMac side path
scopeintPath scope

db_pools

Field nameField typeDescription
idintPrimary key
namestringThe name of the pool or the node entry
nodes_priorityintThe nodes priority overrides
jobs_priorityintThe jobs priority overrides
enable_nodes_priorityboolEnables nodes priority override
enable_jobs_priorityboolEnables jobs priority override
autoPool_ip_filterstringFilter to create an autopool
autoPool_hostname_filterstringHostname to create an autopool
time_rulesstringTime rules for the pool
pool_idintParent pool id for a node entry
host_namestringHost name for a node entry
host_ipstringHost ip for a node entry

db_users

Field nameField typeDescription
idintPrimary key ID
account_namestringAccount name
groupsstringAccount list of groups
encoded_passwordstringEncoded password
email_addressstringEmail address
allowed_rightsstringAllowed rights bit mask
forbidden_rightsstringForbidden rights bit mask
allowed_mail_notificationsstringNotifications bit mask
blocked_mail_notificationsstringNotifications bit mask
allowed_notificator_notificationsstringNotifications bit mask
blocked_notificator_notificationsstringNotifications bit mask
allowed_mobile_notificationsstringNotifications bit mask
blocked_mobile_notificationsstringNotifications bit mask
allowed_templates_stateboolEnable state
forbidden_templates_stateboolEnable state
allowed_pools_stateboolEnable state
forbidden_pools_stateboolEnable state
maximum_instances_stateboolEnable state
maximum_priority_stateboolEnable state
enable_home_folder_stateboolEnable state
constrain_to_home_folder_stateboolEnable state
repositories_stateboolEnable state
limited_pools_skip_on_idle_stateboolEnable state
limited_instances_skip_on_idle_stateboolEnable state
allowed_poolsstringAllowed pools
forbidden_poolsstringForbidden pools
allowed_templatesstringAllowed templates
forbidden_templatesstringForbidden templates
maximum_instancesintMaximum instances
maximum_priorityintMaximum priority
repositoriesstringRepositories IDs list
home_folder_idintHome folder ID
is_ldapboolldap imported object

db_users_groups

Field nameField typeDescription
idintPrimary key ID
group_namestringGroup name
allowed_rightsstringAllowed rights bit mask
allowed_mail_notificationsstringNotifications bit mask
allowed_notificator_notificationsstringNotifications bit mask
allowed_mobile_notificationsstringNotifications bit mask
allowed_templates_stateboolEnable state
forbidden_templates_stateboolEnable state
allowed_pools_stateboolEnable state
forbidden_pools_stateboolEnable state
maximum_instances_stateboolEnable state
maximum_priority_stateboolEnable state
enable_home_folder_stateboolEnable state
constrain_to_home_folder_stateboolEnable state
repositories_stateboolEnable state
limited_pools_skip_on_idle_stateboolEnable state
limited_instances_skip_on_idle_stateboolEnable state
allowed_poolsstringAllowed pools
forbidden_poolsstringForbidden pools
allowed_templatesstringAllowed templates
forbidden_templatesstringForbidden templates
maximum_instancesintMaximum instances
maximum_priorityintMaximum priority
repositoriesstringRepositories IDs list
is_ldapboolldap imported object

db_sequences

Field nameField typeDescription
table_namestringName of the table
next_idintNext valid table ID

db_nodes

Field nameField typeDescription
idintPrimary key
instance_namestringName of the instance
instance_numintInstance number
host_ipstringIp address of the instance
host_namestringHost name of the instance
instance_notesstringNotes of the instance
platformintPlatform of the instance
mac_addressesstringList of instance NICs Mac addresses
wakeup_fullloadboolWakeup when at full load
wakeup_bypoolboolWakeup when required by pool
wakeup_policiesintWakeup policies
wakeup_match_availabilityboolMatch availability status
availability_policiesstringAvailability policies (encoded)
availableByDefaultboolAvailability default flag
statusintInstance status

db_history_chunks

Field nameField typeDescription
idintPrimary key
chunk_idintChunk reference id
job_idintJob reference id
job_namestringJob name
job_projectstringThe job project
job_groupstringThe job group
job_departmentstringThe job department
job_filestringThe job main file
taskstringThe chunk task
statusintChunk final status
start_framefloatingChunk starting frame
end_framefloatingChunk ending frame
by_framefloatingChunk ending frame
starting_timeint time_tChunk starting time
ending_timeint time_tChunk ending time
rendering_hoststringChunk assigned instance name
resultstringChunk result text
starting_numberintChunk starting numbering
step_byintChunk numbering step by
packet_sizeintChunk packet size
packet_typeintChunk packet type
is_assemblerboolChunk assembler flag
warning_markersstringChunk warnings markers
error_markersstringChunk errors markers
silenced_warning_markersstringChunk silenced warning markers
silenced_error_markersstringChunk silenced error markers

db_history_stats

Field nameField typeDescription
idintPrimary key
sample_timeint time_t
connected_rcintNumber of connected clients
connected_exintNumber of connected consoles/mrtool
connected_notintNumber of connected notificators
connected_web_sessionsintNumber of active web sessions
connected_webintNumber of active HTTP connections
connected_httpsintNumber of active HTTPS connections
rc_usedintNumber of working nodes
rc_pausedintNumber of paused nodes
rc_idleintNumber of idle nodes
rc_unavailableintNumber of disconnected nodes
total_in_kbsfloatingTotal amount of incoming KBytes
total_out_kbsfloatingTotal amount of outcoming KBytes
control_out_kbsfloatingConsole/mrtool outcoming KBytes
control_in_kbsfloatingConsole/mrtool incoming KBytes
rc_in_kbsfloatingNodes incoming KBytes
rc_out_kbsfloatingNodes outcoming KBytes
not_in_kbsfloatingNotificators incoming KBytes
not_out_kbsfloatingNotificators outcoming KBytes
web_in_kbsfloatingWeb incoming KBytes
web_out_kbsfloatingWeb outcoming KBytes
connected_web_sessionsintNumber of concurrent HTTP(S) sessions

backup_db_jobs

Field nameField typeDescription
idintPrimary key
job_idintThe ID of the original job this backup refer to
backup_timeint time_tThe time when the job has been backed up
parent_idintID of the parent folder
indentintLevel of indentation
statusintJob status
tasks_maskstringJob tasks mask
chunks_priorityintThe chunks priority type
chunks_interleaveintThe chunks priority interleave value
packet_typeintType of job packets
packet_sizeintSize of each chunk
requeuedintNumber of times job has been requeued
engineintTemplate uid
engine_versionstringThe template version
max_cpuintMaximum number of concurrent instances
max_cpu_workingintNumber of current working instances
depend_idstringDepend from ids
depend_modeintDepend mode used
depend_link_modeintDepend link mode
job_typeintJob type, Folder or real job
job_namestringName of the job
priorityintPriority for the job
is_pausedboolPause status
is_lockedboolLocked status
job_poolstringTarget pools comma separated
Job_excluded_poolstringExcluded pools comma separated
submission_timeint time_tTime of submission
start_timeint time_tTime of job execution
progressintJob progress
end_timeint time_tTime of job termination
start_onint time_tSpecify the scheduled starting time for the job
resume_onint time_tSpecify the scheduled resume time for the job
pause_onint time_tSpecify the scheduled paused time for the job
submitterstringUsername of the job owner
exitcodes_err_check_typeintSpecify the type of exit codes check
logs_err_check_typeintSpecify the type of logs texts error check
emergency_queueboolSpecifies if the job is into the emergency queue
logs_parsing_rulesstringEncoded overriden logs parsing rules
override_notificator_notifications_at_chunk_levelintOverrides of notifications
override_notificator_notifications_at_job_levelintOverrides of notifications
notificator_notifications_at_chunk_level_typeintOverrides of notifications
notificator_notifications_at_job_level_typeintOverrides of notifications
override_mobile_notifications_at_chunk_levelintOverrides of notifications
override_mobile_notifications_at_job_levelintOverrides of notifications
mobile_notifications_at_chunk_level_typeintOverrides of notifications
mobile_notifications_at_job_level_typeintOverrides of notifications
override_mail_notifications_at_chunk_levelintOverrides of notifications
override_mail_notifications_at_job_levelintOverrides of notifications
mail_notifications_at_chunk_level_typeintOverrides of notifications
mail_notifications_at_job_level_typeintOverrides of notifications
override_start_notificator_notifications_at_chunk_levelintOverrides of notifications
override_start_notificator_notifications_at_job_levelintOverrides of notifications
start_notificator_notifications_at_chunk_level_typeintOverrides of notifications
start_notificator_notifications_at_job_level_typeintOverrides of notifications
override_start_mobile_notifications_at_chunk_levelintOverrides of notifications
override_start_mobile_notifications_at_job_levelintOverrides of notifications
start_mobile_notifications_at_chunk_level_typeintOverrides of notifications
start_mobile_notifications_at_job_level_typeintOverrides of notifications
override_start_mail_notifications_at_chunk_levelintOverrides of notifications
override_start_mail_notifications_at_job_levelintOverrides of notifications
start_mail_notifications_at_chunk_level_typeintOverrides of notifications
start_mail_notifications_at_job_level_typeintOverrides of notifications
override_chunks_timeoutbooleanOverrides chunks timeout state
override_chunks_timeout_valueintOverrides chunks timeout value
is_archivedbooleanJob is archived
job_projectstringJob's project
job_departmentstringJob's department
job_camerastringJob's camera
job_shotstringJob's shot
job_sequencestringJob's sequence
override_valid_exit_codesstringOverrides of exit codes
override_warning_exit_codesstringOverrides of exit codes
override_error_exit_codesstringOverrides of exit codes
override_minimum_threadsboolOverrides minimum threads
override_minimum_coresboolOverrides minium cores
override_minimum_physicalboolOverrides physical cores
override_minimum_speedboolOverrides minimum speed
override_minimum_ramboolOverrides minimum RAM
override_minimum_disk_spaceboolOverrides minimum disk space
override_maximum_chunks_requeueboolOverrides maximum chunks requeue
override_maximum_chunks_requeue_valueintOverrides maximums chunks requeue value
override_valid_exit_codes_valuestringOverrides of exit codes value
override_valid_warning_codes_valuestringOverrides of exit codes value
override_error_exit_codes_valuestringOverrides of exit codes value
override_minimum_threads_valueintOverrides minimum threads value
override_minimum_cores_valueintOverrides minimum cores value
override_minimum_physical_valueintOverrides minimum physical value
override_minimum_speed_value floating Overrides mimumum speed value
override_minimum_ram_value floating Overrides minimum RAM value
override_minimum_disk_space_valuefloatingOverrides minimum disk space value

backup_db_jobs_data

Field nameField typeDescription
idintPrimary key
job_idintJob ID reference
key_namestringAttribute name
key_valuestringAttribute value
key_stateboolAttribute activation status
key_substboolAttribute substitution status

backup_db_chunks

Field nameField typeDescription
idintPrimary key
chunk_idintSequential chunk id
job_idintJob ID reference
priorityintThe chunk priority
statusintChunk status
packet_typeintThe chunk packet type
requeuedintNumber of times chunk has been requeued
start_framefloatingStarting frame for the chunk
end_framefloatingEnding frame for the chunk
by_framefloatingBy framing for the chunk
starting_timeint time_cChunk starting time
ending timeint time_cChunk ending time
rendering_hoststringInstance name that processed the chunk
resultstringError string returned by the process
starting_numberintStarting numbering for the chunk
is_assemblerboolFlagged for assembler chunks
taskstringThe chunk's job task
depend_from_chunksstringDepends from chunks IDs
depend_from_chunks_error_levelsintDepends error level
warning_markersstringWarnings markers
error_markersstringError markers
silenced_warning_markersstringWarning silenced markers
silenced_error_markersstringError silenced markers