Databases structure
Muster stores the queue and internal configurations in a database that may be on a local file (using sqlite) or external (Mysql or SQL Server).
An internal abstraction layer is used for certain data types, the field types you’ll find in the descriptions may appear differently depending on the destination database.
The following section lists the internal tables and their purpose. You are just allowed to read data from the Muster database. You are not allowed to write data neither change the data structure of the tables: Muster will check the structure on any service start-up and rebuild them according.
- db_jobs: Table holding the jobs general data either for folders and real jobs
- db_jobs_data: Table holding attributes for real jobs as well as custom attributes
- db_chunks: Table holding the chunks data
- db_logs: Table holding the internal Muster log
- db_mails: Table holding distribution lists
- db_pools: Table holding pools configuration
- db_users: Table holding users configuration
- db_users_groups: Table holding groups configuration
- db_paths: Table holding global substitution paths
- db_history_chunks: Table holding the Muster processing history (Stored in a separate file/db)
Primary database
db_jobs
| Field name | Field type | Description |
|---|---|---|
| id | 32 bit integer | Primary key |
| parent_id | 32 bit integer | ID of the parent folder |
| indent | 32 bit integer | Level of indentation |
| status | 32 bit integer | Job status |
| packet_type | 32 bit integer | Type of job packets |
| packet_size | 32 bit integer | Size of each chunk |
| requeued | 32 bit integer | Number of times job has been requeued |
| engine | 32 bit integer | Template uid |
| max_cpu | 32 bit integer | Maximum number of concurrent instances |
| max_cpu_working | 32 bit integer | Number of current working instances |
| depend_id | 32 bit integer | Depend from id |
| depend_mode | 32 bit integer | Depend mode used |
| job_type | 32 bit integer | Job type, Folder or real job |
| job_name | String | Name of the job |
| priority | 32 bit integer | Priority for the job |
| is_paused | Boolean | Pause status |
| is_locked | Boolean | Locked status |
| job_pool | String | Target pools comma separated |
| Job_excluded_pool | String | Excluded pools comma separated |
| submission_time | 64 bit integer, c time_t format | Time of submission |
| start_time | 64 bit integer, c time_t format | Time of job execution |
| progress | 32 bit integer | Job progress |
| end_time | 64 bit integer, c time_t format | Time of job termination |
| start_on | 64 bit integer, c time_t format | Specify the scheduled starting time for the job |
| resume_on | 64 bit integer, c time_t format | Specify the scheduled resume time for the job |
| pause_on | 64 bit integer, c time_t format | Specify the scheduled paused time for the job |
| submitter | String | Username of the job owner |
| exitcodes_err_check_type | 32 bit integer | Specify the type of exit codes check |
| logs_err_check_type | 32 bit integer | Specify the type of logs texts error check |
| override_mail_address | Boolean | Enable mail address override |
| mail_address | String | Mail / distribution for notification (if overridden) |
| override_job_check | Boolean | Enable job notifications override |
| job_check | Boolean | Job check status |
| job_check_type | String | Job check type |
| override_chunk_check | Boolean | Enable chunk notifications override |
| chunk_check | Boolean | Chunk check status |
| chunk_check_type | String | Chunk check type |
| override_chunks_timeout | Boolean | Override the default value for chunks timeout |
| override_chunks_timeout_value | 32 bit integer | Timeout value |
| is_archivied | Boolean | Specify if the job is in archived status |
| job_group | String | Specify the job group |
| job_department | String | Specify the job department |
| override_valid_exit_codes | Boolean | Enables overrides |
| override_warning_exit_codes | ||
| override_error_exit_codes | ||
| override_valid_log_texts | ||
| override_warning_log_texts | ||
| override_error_log_texts | ||
| override_minimum_threads | ||
| override_minimum_cores | ||
| override_minimum_physical | ||
| override_minimum_speed | ||
| override_minimum_ram | ||
| override_minimum_disk_space | ||
| override_maximum_chunks_requeue | ||
| override_valid_exit_codes_value | String | Overrides specific values |
| override_warning_exit_codes_value | ||
| override_error_exit_codes_value | ||
| override_valid_log_texts_value | ||
| override_warning_log_texts_value | ||
| override_error_log_texts_value | ||
| override_minimum_threads_value | ||
| override_minimum_cores_value | ||
| override_minimum_physical_value | ||
| override_minimum_speed_value | ||
| override_minimum_ram_value | ||
| override_minimum_disk_space_value | ||
| override_maximum_chunks_requeue_value |
db_jobs_data
| Field name | Field type | Description |
|---|---|---|
| id | 32 bit integer | Primary key |
| job_id | 32 bit integer | Job ID reference |
| key_name | String | Attribute name |
| key_value | String | Attribute value |
| key_state | Boolean | Attribute activation status |
| key_subst | Boolean | Attribute substitution status |
db_chunks
| Field name | Field type | Description |
|---|---|---|
| id | 32 bit integer | Primary key |
| chunk_id | 32 bit integer | Sequential chunk id |
| job_id | 32 bit integer | Job ID reference |
| status | 32 bit integer | Chunk status |
| requeued | 32 bit integer | Number of times chunk has been requeued |
| start_frame | Double float | Starting frame for the chunk |
| end_frame | Double float | Ending frame for the chunk |
| by_frame | Double float | By framing for the chunk |
| starting_time | 64 bit integer, c time_t format | Chunk starting time |
| ending time | 64 bit integer, c time_t format | Chunk ending time |
| rendering_host | String | Instance name that processed the chunk |
| result | String | Error string returned by the process |
| starting_number | 32 bit integer | Starting numbering for the chunk |
| is_assembler | Boolean | Flagged for assembler chunks |
db_logs
| Field name | Field type | Description |
|---|---|---|
| id | 32 bit integer | Primary key |
| host_ip | String | IP of the host that generated the log |
| log_time | String | Formatted time of the log |
| host_name | String | Name of the instance that generated the log |
| event_user | String | Username of the user |
| log_entry | String | Text of the log entry |
| log_type | 32 bit integer | Log type |
| job_id | 32 bit integer | The job ID the log entry reefer too |
| chunk_id | 32 bit integer | The chunk ID the log entry reefer too |
| instance_id | 32 bit integer | The instance ID the log entry reefer too |
db_mails
| Field name | Field type | Description |
|---|---|---|
| id | 32 bit integer | Primary key |
| mail_address | String | List of e-mails associated with the distribution list separated by ; |
| mail_name | String | Name of the distribution list |
db_mails_templates
| Field name | Field type | Description |
|---|---|---|
| template_name | String | The name of the template |
| template_content | String | The content of the template |
db_paths
| Field name | Field type | Description |
|---|---|---|
| id | 32 bit integer | Primary key |
| local_path | String | Dispatcher side path |
| win_path | String | Windows side path |
| linux_path | String | Linux side path |
| mac_path | String | Mac side path |
| scope | 32 bit integer | Path scope |
db_pools
| Field name | Field type | Description |
|---|---|---|
| id | 32 bit integer | Primary key |
| pool_ip | String | IP address of the pool entry |
| pool_name | String | Instance name of the entry |
| pool_parent | String | Name of the parent pool |
| host_name | String | The host name of the pool entry |
| features | String | Additional features of the pool (encoded) |
db_users
| Field name | Field type | Description |
|---|---|---|
| id | 32 bit integer | Primary key |
| username | String | Name of the user |
| encoded_password | String | Base64 encoded password |
| allowed_rights | String | Bits mask of the supported user rights as a 64 bit integer |
| forbidden_rights | String | Bits mask of the forbidden user rights as a 64 bit integer |
| groups | String | List of user's groups separated by , |
| paths | 32 bit integer | User specific substitution paths |
| max_instances | 32 bit integer | Maximum number of concurrent instances allocable by user |
| template_limit | Comma separated list of template IDs integers | Allowed templates |
| pool_limit | Comma separated list of pool limits, strings | Allowed pools |
| home_folder | 32 bit integer | ID of user home folder, -1 for none |
| lock_home | Boolean | Lock user actions and views on the home folder |
| email_address | String | User email address |
db_users_groups
| Field name | Field type | Description |
|---|---|---|
| id | 32 bit integer | Primary key |
| groupname | String | Name of the group |
| allowed_rights | String | Bits mask of the supported user rights as a 64 bit integer |
| max_instances | 32 bit integer | Maximum number of concurrent instances allocable by user |
| template_limit | Comma separated list of template IDs integers | Allowed templates |
| pool_limit | Comma separated list of pool limits, strings | Allowed pools |
| paths | 32 bit integer | User specific substitution paths |
| enable_home_folder | 32 bit integer | Enables home folders for the group's users |
| lock_home | Boolean | Lock user actions and views on the home folder |
| email_address | String | User email address |
db_sequences
| Field name | Field type | Description |
|---|---|---|
| table_name | String | Name of the table |
| next_id | 32 bit integer | Next valid table ID |
db_nodes
| Field name | Field type | Description |
|---|---|---|
| id | 32 bit integer | Primary key |
| instance_name | String | Name of the instance |
| instance_num | 32 bit integer | Instance number |
| host_ip | String | Ip address of the instance |
| host_name | String | Host name of the instance |
| instance_notes | String | Notes of the instance |
| platform | 32 bit integer | Platform of the instance |
| mac_addresses | String | List of instance NICs Mac addresses |
| wakeup_fullload | Boolean | |
| wakeup_bypool | Boolean | Wakeup when required by pool |
| wakeup_policies | 32 bit integer | Wakeup policies |
| wakeup_match_availability | 32 bit integer | Match availability status |
| availability_policies | String | Availability policies (encoded) |
| availableByDefault | Boolean | Availability default flag |
History database
db_history_chunks
| Field name | Field type | Description |
|---|---|---|
| id | 32 bit integer | Primary key |
| chunk_id | 32 bit integer | Chunk reference id |
| job_id | 32 bit integer | Job reference id |
| job_name | String | Job name |
| job_project | String | The job project |
| job_group | String | The job group |
| job_department | String | The job department |
| job_file | String | The job main file |
| status | 32 bit integer | Chunk final status |
| start_frame | Double float | Chunk starting frame |
| end_frame | Double float | Chunk ending frame |
| by_frame | Double float | Chunk ending frame |
| starting_time | 64 bit integer, c time_t format | Chunk starting time |
| ending_time | 64 bit integer, c time_t format | Chunk ending time |
| rendering_host | String | Chunk assigned instance name |
| result | String | Chunk result text |
| starting_number | 32 bit integer | Chunk starting numbering |
| step_by | 32 bit integer | Chunk numbering step by |
| packet_size | 32 bit integer | Chunk packet size |
| is_assembler | Boolean | Chunk assembler flag |
db_history_stats
| Field name | Field type | Description |
|---|---|---|
| id | 32 bit integer | Primary key |
| sample_time | 64 bit integer, c time_t format | |
| connected_rc | 32 bit integer | Number of connected clients |
| connected_ex | 32 bit integer | Number of connected consoles/mrtool |
| connected_not | 32 bit integer | Number of connected notificators |
| connected_web_sessions | 32 bit integer | Number of active web sessions |
| connected_web | 32 bit integer | Number of active HTTP connections |
| connected_https | 32 bit integer | Number of active HTTPS connections |
| rc_used | 32 bit integer | Number of working nodes |
| rc_paused | 32 bit integer | Number of paused nodes |
| rc_idle | 32 bit integer | Number of idle nodes |
| rc_unavailable | 32 bit integer | Number of disconnected nodes |
| total_in_kbs | Double float | Total amount of incoming KBytes |
| total_out_kbs | Double float | Total amount of outcoming KBytes |
| control_out_kbs | Double float | Console/mrtool outcoming KBytes |
| control_in_kbs | Double float | Console/mrtool incoming KBytes |
| rc_in_kbs | Double float | Nodes incoming KBytes |
| rc_out_kbs | Double float | Nodes outcoming KBytes |
| not_in_kbs | Double float | Notificators incoming KBytes |
| not_out_kbs | Double float | Notificators outcoming KBytes |
| web_in_kbs | Double float | Web incoming KBytes |
| web_out_kbs | Double float | Web outcoming KBytes |