Databases structure
Muster stores the queue and internal configurations in a database that may be on a local file (using sqlite) or external (Mysql or SQL Server).
An internal abstraction layer is used for certain data types, the field types you’ll find in the descriptions may appear differently depending on the destination database.
The following section lists the internal tables and their purpose. You are just allowed to read data from the Muster database. You are not allowed to write data neither change the data structure of the tables: Muster will check the structure on any service start-up and rebuild them according.
Queue database tables:
- db_jobs: Table holding the jobs general data either for folders and real jobs
- db_jobs_data: Table holding attributes for real jobs as well as custom attributes
- db_chunks: Table holding the chunks data
- db_logs: Table holding the internal Muster log
- db_mails: Table holding distribution lists
- db_pools: Table holding pools configuration
- db_users: Table holding users configuration
- db_users_groups: Table holding groups configuration
- db_paths: Table holding global substitution paths
History database tables:
- db_history_chunks: Table holding the Muster processing history
- db_history_stats: Table holding the Muster loads statistics
Backup database tables:
- backup_db_jobs: Table holding the jobs general data either for folders and real jobs
- backup_db_jobs_data: Table holding attributes for real jobs as well as custom attributes
- backup_db_chunks: Table holding the chunks data
Queue database
db_jobs
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key |
| parent_id | int | ID of the parent folder |
| indent | int | Level of indentation |
| status | int | Job status |
| tasks_mask | string | Job tasks mask |
| chunks_priority | int | The chunks priority type |
| chunks_interleave | int | The chunks priority interleave value |
| packet_type | int | Type of job packets |
| packet_size | int | Size of each chunk |
| requeued | int | Number of times job has been requeued |
| engine | int | Template uid |
| engine_version | string | The template version |
| max_cpu | int | Maximum number of concurrent instances |
| max_cpu_working | int | Number of current working instances |
| depend_id | string | Depend from ids |
| depend_mode | int | Depend mode used |
| depend_link_mode | int | Depend link mode |
| job_type | int | Job type, Folder or real job |
| job_name | string | Name of the job |
| priority | int | Priority for the job |
| is_paused | bool | Pause status |
| is_locked | bool | Locked status |
| job_pool | string | Target pools comma separated |
| Job_excluded_pool | string | Excluded pools comma separated |
| submission_time | int time_t | Time of submission |
| start_time | int time_t | Time of job execution |
| progress | int | Job progress |
| end_time | int time_t | Time of job termination |
| start_on | int time_t | Specify the scheduled starting time for the job |
| resume_on | int time_t | Specify the scheduled resume time for the job |
| pause_on | int time_t | Specify the scheduled paused time for the job |
| submitter | string | Username of the job owner |
| exitcodes_err_check_type | int | Specify the type of exit codes check |
| logs_err_check_type | int | Specify the type of logs texts error check |
| emergency_queue | bool | Specifies if the job is into the emergency queue |
| logs_parsing_rules | string | Encoded overriden logs parsing rules |
| override_notificator_notifications_at_chunk_level | int | Overrides of notifications |
| override_notificator_notifications_at_job_level | int | Overrides of notifications |
| notificator_notifications_at_chunk_level_type | int | Overrides of notifications |
| notificator_notifications_at_job_level_type | int | Overrides of notifications |
| override_mobile_notifications_at_chunk_level | int | Overrides of notifications |
| override_mobile_notifications_at_job_level | int | Overrides of notifications |
| mobile_notifications_at_chunk_level_type | int | Overrides of notifications |
| mobile_notifications_at_job_level_type | int | Overrides of notifications |
| override_mail_notifications_at_chunk_level | int | Overrides of notifications |
| override_mail_notifications_at_job_level | int | Overrides of notifications |
| mail_notifications_at_chunk_level_type | int | Overrides of notifications |
| mail_notifications_at_job_level_type | int | Overrides of notifications |
| override_start_notificator_notifications_at_chunk_level | int | Overrides of notifications |
| override_start_notificator_notifications_at_job_level | int | Overrides of notifications |
| start_notificator_notifications_at_chunk_level_type | int | Overrides of notifications |
| start_notificator_notifications_at_job_level_type | int | Overrides of notifications |
| override_start_mobile_notifications_at_chunk_level | int | Overrides of notifications |
| override_start_mobile_notifications_at_job_level | int | Overrides of notifications |
| start_mobile_notifications_at_chunk_level_type | int | Overrides of notifications |
| start_mobile_notifications_at_job_level_type | int | Overrides of notifications |
| override_start_mail_notifications_at_chunk_level | int | Overrides of notifications |
| override_start_mail_notifications_at_job_level | int | Overrides of notifications |
| start_mail_notifications_at_chunk_level_type | int | Overrides of notifications |
| start_mail_notifications_at_job_level_type | int | Overrides of notifications |
| override_chunks_timeout | boolean | Overrides chunks timeout state |
| override_chunks_timeout_value | int | Overrides chunks timeout value |
| is_archived | boolean | Job is archived |
| job_project | string | Job's project |
| job_department | string | Job's department |
| job_camera | string | Job's camera |
| job_shot | string | Job's shot |
| job_sequence | string | Job's sequence |
| override_valid_exit_codes | string | Overrides of exit codes |
| override_warning_exit_codes | string | Overrides of exit codes |
| override_error_exit_codes | string | Overrides of exit codes |
| override_minimum_threads | bool | Overrides minimum threads |
| override_minimum_cores | bool | Overrides minium cores |
| override_minimum_physical | bool | Overrides physical cores |
| override_minimum_speed | bool | Overrides minimum speed |
| override_minimum_ram | bool | Overrides minimum RAM |
| override_minimum_disk_space | bool | Overrides minimum disk space |
| override_maximum_chunks_requeue | bool | Overrides maximum chunks requeue |
| override_maximum_chunks_requeue_value | int | Overrides maximums chunks requeue value |
| override_valid_exit_codes_value | string | Overrides of exit codes value |
| override_valid_warning_codes_value | string | Overrides of exit codes value |
| override_error_exit_codes_value | string | Overrides of exit codes value |
| override_minimum_threads_value | int | Overrides minimum threads value |
| override_minimum_cores_value | int | Overrides minimum cores value |
| override_minimum_physical_value | int | Overrides minimum physical value |
| override_minimum_speed_value | floating | Overrides mimumum speed value |
| override_minimum_ram_value | floating | Overrides minimum RAM value |
| override_minimum_disk_space_value | floating | Overrides minimum disk space value |
db_jobs_data
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key |
| job_id | int | Job ID reference |
| key_name | string | Attribute name |
| key_value | string | Attribute value |
| key_state | bool | Attribute activation status |
| key_subst | bool | Attribute substitution status |
db_chunks
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key |
| chunk_id | int | Sequential chunk id |
| job_id | int | Job ID reference |
| priority | int | The chunk priority |
| status | int | Chunk status |
| packet_type | int | The chunk packet type |
| requeued | int | Number of times chunk has been requeued |
| start_frame | floating | Starting frame for the chunk |
| end_frame | floating | Ending frame for the chunk |
| by_frame | floating | By framing for the chunk |
| starting_time | int time_c | Chunk starting time |
| ending time | int time_c | Chunk ending time |
| rendering_host | string | Instance name that processed the chunk |
| result | string | Error string returned by the process |
| starting_number | int | Starting numbering for the chunk |
| is_assembler | bool | Flagged for assembler chunks |
| task | string | The chunk's job task |
| depend_from_chunks | string | Depends from chunks IDs |
| depend_from_chunks_error_levels | int | Depends error level |
| warning_markers | string | Warnings markers |
| error_markers | string | Error markers |
| silenced_warning_markers | string | Warning silenced markers |
| silenced_error_markers | string | Error silenced markers |
db_logs
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key |
| host_ip | string | IP of the host that generated the log |
| log_time | string | Formatted time of the log |
| host_name | string | Name of the instance that generated the log |
| event_user | string | Username of the user |
| log_entry | string | Text of the log entry |
| log_type | int | Log type |
| job_id | int | The job ID the log entry reefer too |
| chunk_id | int | The chunk ID the log entry reefer too |
| instance_id | int | The instance ID the log entry reefer too |
| history_chunk_id | int | The history chunk ID the log entry reefer too |
db_paths
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key |
| name | string | Repository name |
| local_path | string | Dispatcher side path |
| win | string | Windows side path |
| linux | string | Linux side path |
| mac | string | Mac side path |
| scope | int | Path scope |
db_pools
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key |
| name | string | The name of the pool or the node entry |
| nodes_priority | int | The nodes priority overrides |
| jobs_priority | int | The jobs priority overrides |
| enable_nodes_priority | bool | Enables nodes priority override |
| enable_jobs_priority | bool | Enables jobs priority override |
| autoPool_ip_filter | string | Filter to create an autopool |
| autoPool_hostname_filter | string | Hostname to create an autopool |
| time_rules | string | Time rules for the pool |
| pool_id | int | Parent pool id for a node entry |
| host_name | string | Host name for a node entry |
| host_ip | string | Host ip for a node entry |
db_users
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key ID |
| account_name | string | Account name |
| groups | string | Account list of groups |
| encoded_password | string | Encoded password |
| email_address | string | Email address |
| allowed_rights | string | Allowed rights bit mask |
| forbidden_rights | string | Forbidden rights bit mask |
| allowed_mail_notifications | string | Notifications bit mask |
| blocked_mail_notifications | string | Notifications bit mask |
| allowed_notificator_notifications | string | Notifications bit mask |
| blocked_notificator_notifications | string | Notifications bit mask |
| allowed_mobile_notifications | string | Notifications bit mask |
| blocked_mobile_notifications | string | Notifications bit mask |
| allowed_templates_state | bool | Enable state |
| forbidden_templates_state | bool | Enable state |
| allowed_pools_state | bool | Enable state |
| forbidden_pools_state | bool | Enable state |
| maximum_instances_state | bool | Enable state |
| maximum_priority_state | bool | Enable state |
| enable_home_folder_state | bool | Enable state |
| constrain_to_home_folder_state | bool | Enable state |
| repositories_state | bool | Enable state |
| limited_pools_skip_on_idle_state | bool | Enable state |
| limited_instances_skip_on_idle_state | bool | Enable state |
| allowed_pools | string | Allowed pools |
| forbidden_pools | string | Forbidden pools |
| allowed_templates | string | Allowed templates |
| forbidden_templates | string | Forbidden templates |
| maximum_instances | int | Maximum instances |
| maximum_priority | int | Maximum priority |
| repositories | string | Repositories IDs list |
| home_folder_id | int | Home folder ID |
| is_ldap | bool | ldap imported object |
db_users_groups
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key ID |
| group_name | string | Group name |
| allowed_rights | string | Allowed rights bit mask |
| allowed_mail_notifications | string | Notifications bit mask |
| allowed_notificator_notifications | string | Notifications bit mask |
| allowed_mobile_notifications | string | Notifications bit mask |
| allowed_templates_state | bool | Enable state |
| forbidden_templates_state | bool | Enable state |
| allowed_pools_state | bool | Enable state |
| forbidden_pools_state | bool | Enable state |
| maximum_instances_state | bool | Enable state |
| maximum_priority_state | bool | Enable state |
| enable_home_folder_state | bool | Enable state |
| constrain_to_home_folder_state | bool | Enable state |
| repositories_state | bool | Enable state |
| limited_pools_skip_on_idle_state | bool | Enable state |
| limited_instances_skip_on_idle_state | bool | Enable state |
| allowed_pools | string | Allowed pools |
| forbidden_pools | string | Forbidden pools |
| allowed_templates | string | Allowed templates |
| forbidden_templates | string | Forbidden templates |
| maximum_instances | int | Maximum instances |
| maximum_priority | int | Maximum priority |
| repositories | string | Repositories IDs list |
| is_ldap | bool | ldap imported object |
db_sequences
| Field name | Field type | Description |
|---|---|---|
| table_name | string | Name of the table |
| next_id | int | Next valid table ID |
db_nodes
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key |
| instance_name | string | Name of the instance |
| instance_num | int | Instance number |
| host_ip | string | Ip address of the instance |
| host_name | string | Host name of the instance |
| instance_notes | string | Notes of the instance |
| platform | int | Platform of the instance |
| mac_addresses | string | List of instance NICs Mac addresses |
| wakeup_fullload | bool | Wakeup when at full load |
| wakeup_bypool | bool | Wakeup when required by pool |
| wakeup_policies | int | Wakeup policies |
| wakeup_match_availability | bool | Match availability status |
| availability_policies | string | Availability policies (encoded) |
| availableByDefault | bool | Availability default flag |
| status | int | Instance status |
History database
db_history_chunks
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key |
| chunk_id | int | Chunk reference id |
| job_id | int | Job reference id |
| job_name | string | Job name |
| job_project | string | The job project |
| job_group | string | The job group |
| job_department | string | The job department |
| job_file | string | The job main file |
| task | string | The chunk task |
| status | int | Chunk final status |
| start_frame | floating | Chunk starting frame |
| end_frame | floating | Chunk ending frame |
| by_frame | floating | Chunk ending frame |
| starting_time | int time_t | Chunk starting time |
| ending_time | int time_t | Chunk ending time |
| rendering_host | string | Chunk assigned instance name |
| result | string | Chunk result text |
| starting_number | int | Chunk starting numbering |
| step_by | int | Chunk numbering step by |
| packet_size | int | Chunk packet size |
| packet_type | int | Chunk packet type |
| is_assembler | bool | Chunk assembler flag |
| warning_markers | string | Chunk warnings markers |
| error_markers | string | Chunk errors markers |
| silenced_warning_markers | string | Chunk silenced warning markers |
| silenced_error_markers | string | Chunk silenced error markers |
db_history_stats
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key |
| sample_time | int time_t | |
| connected_rc | int | Number of connected clients |
| connected_ex | int | Number of connected consoles/mrtool |
| connected_not | int | Number of connected notificators |
| connected_web_sessions | int | Number of active web sessions |
| connected_web | int | Number of active HTTP connections |
| connected_https | int | Number of active HTTPS connections |
| rc_used | int | Number of working nodes |
| rc_paused | int | Number of paused nodes |
| rc_idle | int | Number of idle nodes |
| rc_unavailable | int | Number of disconnected nodes |
| total_in_kbs | floating | Total amount of incoming KBytes |
| total_out_kbs | floating | Total amount of outcoming KBytes |
| control_out_kbs | floating | Console/mrtool outcoming KBytes |
| control_in_kbs | floating | Console/mrtool incoming KBytes |
| rc_in_kbs | floating | Nodes incoming KBytes |
| rc_out_kbs | floating | Nodes outcoming KBytes |
| not_in_kbs | floating | Notificators incoming KBytes |
| not_out_kbs | floating | Notificators outcoming KBytes |
| web_in_kbs | floating | Web incoming KBytes |
| web_out_kbs | floating | Web outcoming KBytes |
| connected_web_sessions | int | Number of concurrent HTTP(S) sessions |
backup_db_jobs
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key |
| job_id | int | The ID of the original job this backup refer to |
| backup_time | int time_t | The time when the job has been backed up |
| parent_id | int | ID of the parent folder |
| indent | int | Level of indentation |
| status | int | Job status |
| tasks_mask | string | Job tasks mask |
| chunks_priority | int | The chunks priority type |
| chunks_interleave | int | The chunks priority interleave value |
| packet_type | int | Type of job packets |
| packet_size | int | Size of each chunk |
| requeued | int | Number of times job has been requeued |
| engine | int | Template uid |
| engine_version | string | The template version |
| max_cpu | int | Maximum number of concurrent instances |
| max_cpu_working | int | Number of current working instances |
| depend_id | string | Depend from ids |
| depend_mode | int | Depend mode used |
| depend_link_mode | int | Depend link mode |
| job_type | int | Job type, Folder or real job |
| job_name | string | Name of the job |
| priority | int | Priority for the job |
| is_paused | bool | Pause status |
| is_locked | bool | Locked status |
| job_pool | string | Target pools comma separated |
| Job_excluded_pool | string | Excluded pools comma separated |
| submission_time | int time_t | Time of submission |
| start_time | int time_t | Time of job execution |
| progress | int | Job progress |
| end_time | int time_t | Time of job termination |
| start_on | int time_t | Specify the scheduled starting time for the job |
| resume_on | int time_t | Specify the scheduled resume time for the job |
| pause_on | int time_t | Specify the scheduled paused time for the job |
| submitter | string | Username of the job owner |
| exitcodes_err_check_type | int | Specify the type of exit codes check |
| logs_err_check_type | int | Specify the type of logs texts error check |
| emergency_queue | bool | Specifies if the job is into the emergency queue |
| logs_parsing_rules | string | Encoded overriden logs parsing rules |
| override_notificator_notifications_at_chunk_level | int | Overrides of notifications |
| override_notificator_notifications_at_job_level | int | Overrides of notifications |
| notificator_notifications_at_chunk_level_type | int | Overrides of notifications |
| notificator_notifications_at_job_level_type | int | Overrides of notifications |
| override_mobile_notifications_at_chunk_level | int | Overrides of notifications |
| override_mobile_notifications_at_job_level | int | Overrides of notifications |
| mobile_notifications_at_chunk_level_type | int | Overrides of notifications |
| mobile_notifications_at_job_level_type | int | Overrides of notifications |
| override_mail_notifications_at_chunk_level | int | Overrides of notifications |
| override_mail_notifications_at_job_level | int | Overrides of notifications |
| mail_notifications_at_chunk_level_type | int | Overrides of notifications |
| mail_notifications_at_job_level_type | int | Overrides of notifications |
| override_start_notificator_notifications_at_chunk_level | int | Overrides of notifications |
| override_start_notificator_notifications_at_job_level | int | Overrides of notifications |
| start_notificator_notifications_at_chunk_level_type | int | Overrides of notifications |
| start_notificator_notifications_at_job_level_type | int | Overrides of notifications |
| override_start_mobile_notifications_at_chunk_level | int | Overrides of notifications |
| override_start_mobile_notifications_at_job_level | int | Overrides of notifications |
| start_mobile_notifications_at_chunk_level_type | int | Overrides of notifications |
| start_mobile_notifications_at_job_level_type | int | Overrides of notifications |
| override_start_mail_notifications_at_chunk_level | int | Overrides of notifications |
| override_start_mail_notifications_at_job_level | int | Overrides of notifications |
| start_mail_notifications_at_chunk_level_type | int | Overrides of notifications |
| start_mail_notifications_at_job_level_type | int | Overrides of notifications |
| override_chunks_timeout | boolean | Overrides chunks timeout state |
| override_chunks_timeout_value | int | Overrides chunks timeout value |
| is_archived | boolean | Job is archived |
| job_project | string | Job's project |
| job_department | string | Job's department |
| job_camera | string | Job's camera |
| job_shot | string | Job's shot |
| job_sequence | string | Job's sequence |
| override_valid_exit_codes | string | Overrides of exit codes |
| override_warning_exit_codes | string | Overrides of exit codes |
| override_error_exit_codes | string | Overrides of exit codes |
| override_minimum_threads | bool | Overrides minimum threads |
| override_minimum_cores | bool | Overrides minium cores |
| override_minimum_physical | bool | Overrides physical cores |
| override_minimum_speed | bool | Overrides minimum speed |
| override_minimum_ram | bool | Overrides minimum RAM |
| override_minimum_disk_space | bool | Overrides minimum disk space |
| override_maximum_chunks_requeue | bool | Overrides maximum chunks requeue |
| override_maximum_chunks_requeue_value | int | Overrides maximums chunks requeue value |
| override_valid_exit_codes_value | string | Overrides of exit codes value |
| override_valid_warning_codes_value | string | Overrides of exit codes value |
| override_error_exit_codes_value | string | Overrides of exit codes value |
| override_minimum_threads_value | int | Overrides minimum threads value |
| override_minimum_cores_value | int | Overrides minimum cores value |
| override_minimum_physical_value | int | Overrides minimum physical value |
| override_minimum_speed_value | floating | Overrides mimumum speed value |
| override_minimum_ram_value | floating | Overrides minimum RAM value |
| override_minimum_disk_space_value | floating | Overrides minimum disk space value |
backup_db_jobs_data
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key |
| job_id | int | Job ID reference |
| key_name | string | Attribute name |
| key_value | string | Attribute value |
| key_state | bool | Attribute activation status |
| key_subst | bool | Attribute substitution status |
backup_db_chunks
| Field name | Field type | Description |
|---|---|---|
| id | int | Primary key |
| chunk_id | int | Sequential chunk id |
| job_id | int | Job ID reference |
| priority | int | The chunk priority |
| status | int | Chunk status |
| packet_type | int | The chunk packet type |
| requeued | int | Number of times chunk has been requeued |
| start_frame | floating | Starting frame for the chunk |
| end_frame | floating | Ending frame for the chunk |
| by_frame | floating | By framing for the chunk |
| starting_time | int time_c | Chunk starting time |
| ending time | int time_c | Chunk ending time |
| rendering_host | string | Instance name that processed the chunk |
| result | string | Error string returned by the process |
| starting_number | int | Starting numbering for the chunk |
| is_assembler | bool | Flagged for assembler chunks |
| task | string | The chunk's job task |
| depend_from_chunks | string | Depends from chunks IDs |
| depend_from_chunks_error_levels | int | Depends error level |
| warning_markers | string | Warnings markers |
| error_markers | string | Error markers |
| silenced_warning_markers | string | Warning silenced markers |
| silenced_error_markers | string | Error silenced markers |