Databases structure

Muster stores the queue and internal configurations in a database that may be on a local file (using sqlite) or external (Mysql or SQL Server).

An internal abstraction layer is used for certain data types, the field types you’ll find in the descriptions may appear differently depending on the destination database.

The following section lists the internal tables and their purpose. You are just allowed to read data from the Muster database. You are not allowed to write data neither change the data structure of the tables: Muster will check the structure on any service start-up and rebuild them according.

  • db_jobs: Table holding the jobs general data either for folders and real jobs
  • db_jobs_data: Table holding attributes for real jobs as well as custom attributes
  • db_chunks: Table holding the chunks data
  • db_logs: Table holding the internal Muster log
  • db_mails: Table holding distribution lists
  • db_pools: Table holding pools configuration
  • db_users: Table holding users configuration
  • db_users_groups: Table holding groups configuration
  • db_paths: Table holding global substitution paths
  • db_history_chunks: Table holding the Muster processing history (Stored in a separate file/db)

db_jobs

Field nameField typeDescription
id32 bit integerPrimary key
parent_id32 bit integerID of the parent folder
indent32 bit integerLevel of indentation
status32 bit integerJob status
packet_type32 bit integerType of job packets
packet_size32 bit integerSize of each chunk
requeued32 bit integerNumber of times job has been requeued
engine32 bit integerTemplate uid
max_cpu32 bit integerMaximum number of concurrent instances
max_cpu_working32 bit integerNumber of current working instances
depend_id32 bit integerDepend from id
depend_mode32 bit integerDepend mode used
job_type32 bit integerJob type, Folder or real job
job_nameStringName of the job
priority32 bit integerPriority for the job
is_pausedBooleanPause status
is_lockedBooleanLocked status
job_poolStringTarget pools comma separated
Job_excluded_poolStringExcluded pools comma separated
submission_time64 bit integer, c time_t formatTime of submission
start_time64 bit integer, c time_t formatTime of job execution
progress32 bit integerJob progress
end_time64 bit integer, c time_t formatTime of job termination
start_on64 bit integer, c time_t formatSpecify the scheduled starting time for the job
resume_on64 bit integer, c time_t formatSpecify the scheduled resume time for the job
pause_on64 bit integer, c time_t formatSpecify the scheduled paused time for the job
submitterStringUsername of the job owner
exitcodes_err_check_type32 bit integerSpecify the type of exit codes check
logs_err_check_type32 bit integerSpecify the type of logs texts error check
override_mail_addressBooleanEnable mail address override
mail_addressStringMail / distribution for notification (if overridden)
override_job_checkBooleanEnable job notifications override
job_checkBooleanJob check status
job_check_typeStringJob check type
override_chunk_checkBooleanEnable chunk notifications override
chunk_checkBooleanChunk check status
chunk_check_typeStringChunk check type
override_chunks_timeoutBooleanOverride the default value for chunks timeout
override_chunks_timeout_value32 bit integerTimeout value
is_archiviedBooleanSpecify if the job is in archived status
job_groupStringSpecify the job group
job_departmentStringSpecify the job department
override_valid_exit_codesBooleanEnables overrides
override_warning_exit_codes
override_error_exit_codes
override_valid_log_texts
override_warning_log_texts
override_error_log_texts
override_minimum_threads
override_minimum_cores
override_minimum_physical
override_minimum_speed
override_minimum_ram
override_minimum_disk_space
override_maximum_chunks_requeue
override_valid_exit_codes_valueStringOverrides specific values
override_warning_exit_codes_value
override_error_exit_codes_value
override_valid_log_texts_value
override_warning_log_texts_value
override_error_log_texts_value
override_minimum_threads_value
override_minimum_cores_value
override_minimum_physical_value
override_minimum_speed_value
override_minimum_ram_value
override_minimum_disk_space_value
override_maximum_chunks_requeue_value

db_jobs_data

Field nameField typeDescription
id32 bit integerPrimary key
job_id32 bit integerJob ID reference
key_nameStringAttribute name
key_valueStringAttribute value
key_stateBooleanAttribute activation status
key_substBooleanAttribute substitution status

db_chunks

Field nameField typeDescription
id32 bit integerPrimary key
chunk_id32 bit integerSequential chunk id
job_id32 bit integerJob ID reference
status32 bit integerChunk status
requeued32 bit integerNumber of times chunk has been requeued
start_frameDouble floatStarting frame for the chunk
end_frameDouble floatEnding frame for the chunk
by_frameDouble floatBy framing for the chunk
starting_time64 bit integer, c time_t formatChunk starting time
ending time64 bit integer, c time_t formatChunk ending time
rendering_hostStringInstance name that processed the chunk
resultStringError string returned by the process
starting_number32 bit integerStarting numbering for the chunk
is_assemblerBooleanFlagged for assembler chunks

db_logs

Field nameField typeDescription
id32 bit integerPrimary key
host_ipStringIP of the host that generated the log
log_timeStringFormatted time of the log
host_nameStringName of the instance that generated the log
event_userStringUsername of the user
log_entryStringText of the log entry
log_type32 bit integerLog type
job_id32 bit integerThe job ID the log entry reefer too
chunk_id32 bit integerThe chunk ID the log entry reefer too
instance_id32 bit integerThe instance ID the log entry reefer too

db_mails

Field nameField typeDescription
id32 bit integerPrimary key
mail_addressStringList of e-mails associated with the distribution list separated by ;
mail_nameStringName of the distribution list

db_mails_templates

Field nameField typeDescription
template_nameStringThe name of the template
template_contentStringThe content of the template

db_paths

Field nameField typeDescription
id32 bit integerPrimary key
local_pathStringDispatcher side path
win_pathStringWindows side path
linux_pathStringLinux side path
mac_pathStringMac side path
scope32 bit integerPath scope

db_pools

Field nameField typeDescription
id32 bit integerPrimary key
pool_ipStringIP address of the pool entry
pool_nameStringInstance name of the entry
pool_parentStringName of the parent pool
host_nameStringThe host name of the pool entry
featuresStringAdditional features of the pool (encoded)

db_users

Field nameField typeDescription
id32 bit integerPrimary key
usernameStringName of the user
encoded_passwordStringBase64 encoded password
allowed_rightsStringBits mask of the supported user rights as a 64 bit integer
forbidden_rightsStringBits mask of the forbidden user rights as a 64 bit integer
groupsStringList of user's groups separated by ,
paths32 bit integerUser specific substitution paths
max_instances32 bit integerMaximum number of concurrent instances allocable by user
template_limitComma separated list of template IDs integersAllowed templates
pool_limitComma separated list of pool limits, stringsAllowed pools
home_folder32 bit integerID of user home folder, -1 for none
lock_homeBooleanLock user actions and views on the home folder
email_addressStringUser email address

db_users_groups

Field nameField typeDescription
id32 bit integerPrimary key
groupnameStringName of the group
allowed_rightsStringBits mask of the supported user rights as a 64 bit integer
max_instances32 bit integerMaximum number of concurrent instances allocable by user
template_limitComma separated list of template IDs integersAllowed templates
pool_limitComma separated list of pool limits, stringsAllowed pools
paths32 bit integerUser specific substitution paths
enable_home_folder32 bit integerEnables home folders for the group's users
lock_homeBooleanLock user actions and views on the home folder
email_addressStringUser email address

db_sequences

Field nameField typeDescription
table_nameStringName of the table
next_id32 bit integerNext valid table ID

db_nodes

Field nameField typeDescription
id32 bit integerPrimary key
instance_nameStringName of the instance
instance_num32 bit integerInstance number
host_ipStringIp address of the instance
host_nameStringHost name of the instance
instance_notesStringNotes of the instance
platform32 bit integerPlatform of the instance
mac_addressesStringList of instance NICs Mac addresses
wakeup_fullloadBoolean
wakeup_bypoolBooleanWakeup when required by pool
wakeup_policies32 bit integerWakeup policies
wakeup_match_availability32 bit integerMatch availability status
availability_policiesStringAvailability policies (encoded)
availableByDefaultBooleanAvailability default flag

db_history_chunks

Field nameField typeDescription
id32 bit integerPrimary key
chunk_id32 bit integerChunk reference id
job_id32 bit integerJob reference id
job_nameStringJob name
job_projectStringThe job project
job_groupStringThe job group
job_departmentStringThe job department
job_fileStringThe job main file
status32 bit integerChunk final status
start_frameDouble floatChunk starting frame
end_frameDouble floatChunk ending frame
by_frameDouble floatChunk ending frame
starting_time64 bit integer, c time_t formatChunk starting time
ending_time64 bit integer, c time_t formatChunk ending time
rendering_hostStringChunk assigned instance name
resultStringChunk result text
starting_number32 bit integerChunk starting numbering
step_by32 bit integerChunk numbering step by
packet_size32 bit integerChunk packet size
is_assemblerBooleanChunk assembler flag

db_history_stats

Field nameField typeDescription
id32 bit integerPrimary key
sample_time64 bit integer, c time_t format
connected_rc32 bit integerNumber of connected clients
connected_ex32 bit integerNumber of connected consoles/mrtool
connected_not32 bit integerNumber of connected notificators
connected_web_sessions32 bit integerNumber of active web sessions
connected_web 32 bit integerNumber of active HTTP connections
connected_https 32 bit integerNumber of active HTTPS connections
rc_used 32 bit integerNumber of working nodes
rc_paused 32 bit integerNumber of paused nodes
rc_idle 32 bit integerNumber of idle nodes
rc_unavailable 32 bit integerNumber of disconnected nodes
total_in_kbsDouble floatTotal amount of incoming KBytes
total_out_kbsDouble floatTotal amount of outcoming KBytes
control_out_kbsDouble floatConsole/mrtool outcoming KBytes
control_in_kbsDouble floatConsole/mrtool incoming KBytes
rc_in_kbsDouble floatNodes incoming KBytes
rc_out_kbsDouble floatNodes outcoming KBytes
not_in_kbsDouble floatNotificators incoming KBytes
not_out_kbsDouble floatNotificators outcoming KBytes
web_in_kbsDouble floatWeb incoming KBytes
web_out_kbsDouble floatWeb outcoming KBytes