Differences

This shows you the differences between two versions of the page.

Link to this comparison view

muster:8.0:databases_structure [2018/01/11 08:21] (current)
Line 1: Line 1:
 +====== Databases structure ======
 +
 +Muster stores the queue and internal configurations in a database that may be on a local file (using sqlite) or external (Mysql or SQL Server).
 +
 +An internal abstraction layer is used for certain data types, the field types you’ll find in the descriptions may appear differently depending on the destination database.
 +
 +The following section lists the internal tables and their purpose. **You are just allowed to read data from the Muster database. You are not allowed to write data neither change the data structure of the tables: Muster will check the structure on any service start-up and rebuild them according.**
 +
 +  * db_jobs: Table holding the jobs general data either for folders and real jobs
 +  * db_jobs_data:​ Table holding attributes for real jobs as well as custom attributes
 +  * db_chunks: Table holding the chunks data
 +  * db_logs: Table holding the internal Muster log
 +  * db_mails: Table holding distribution lists
 +  * db_pools: Table holding pools configuration
 +  * db_users: Table holding users configuration
 +  * db_users_groups:​ Table holding groups configuration
 +  * db_paths: Table holding global substitution paths
 +
 +  * db_history_chunks:​ Table holding the Muster processing history (Stored in a separate file/db)
 +
 +===== Primary database =====
 +
 +==== db_jobs ====
 +^Field name^Field type^Description^
 +|id|32 bit integer|Primary key|
 +|parent_id|32 bit integer|ID of the parent folder|
 +|indent|32 bit integer|Level of indentation|
 +|status|32 bit integer|Job status|
 +|packet_type|32 bit integer|Type of job packets|
 +|packet_size|32 bit integer|Size of each chunk|
 +|requeued|32 bit integer|Number of times job has been requeued|
 +|engine|32 bit integer|Template uid|
 +|max_cpu|32 bit integer|Maximum number of concurrent instances|
 +|max_cpu_working|32 bit integer|Number of current working instances|
 +|depend_id|32 bit integer|Depend from id|
 +|depend_mode|32 bit integer|Depend mode used|
 +|job_type|32 bit integer|Job type, Folder or real job|
 +|job_name|String|Name of the job|
 +|priority|32 bit integer|Priority for the job|
 +|is_paused|Boolean|Pause status|
 +|is_locked|Boolean|Locked status|
 +|job_pool|String|Target pools comma separated|
 +|Job_excluded_pool|String|Excluded pools comma separated|
 +|submission_time|64 bit integer, c time_t format|Time of submission|
 +|start_time|64 bit integer, c time_t format|Time of job execution|
 +|progress|32 bit integer|Job progress|
 +|end_time|64 bit integer, c time_t format|Time of job termination|
 +|start_on|64 bit integer, c time_t format|Specify the scheduled starting time for the job|
 +|resume_on|64 bit integer, c time_t format|Specify the scheduled resume time for the job|
 +|pause_on|64 bit integer, c time_t format|Specify the scheduled paused time for the job|
 +|submitter|String|Username of the job owner|
 +|exitcodes_err_check_type|32 bit integer|Specify the type of exit codes check|
 +|logs_err_check_type|32 bit integer|Specify the type of logs texts error check|
 +|override_mail_address|Boolean|Enable mail address override|
 +|mail_address|String|Mail / distribution for notification (if overridden)|
 +|override_job_check|Boolean|Enable job notifications override|
 +|job_check|Boolean|Job check status|
 +|job_check_type|String|Job check type|
 +|override_chunk_check|Boolean|Enable chunk notifications override|
 +|chunk_check|Boolean|Chunk check status|
 +|chunk_check_type|String|Chunk check type|
 +|override_chunks_timeout|Boolean|Override the default value for chunks timeout|
 +|override_chunks_timeout_value|32 bit integer|Timeout value|
 +|is_archivied|Boolean|Specify if the job is in archived status|
 +|job_group|String|Specify the job group|
 +|job_department|String|Specify the job department|
 +|override_valid_exit_codes|Boolean|Enables overrides|
 +|override_warning_exit_codes|:::​|:::​|
 +|override_error_exit_codes|:::​|:::​|
 +|override_valid_log_texts|:::​|:::​|
 +|override_warning_log_texts|:::​|:::​|
 +|override_error_log_texts|:::​|:::​|
 +|override_minimum_threads|:::​|:::​|
 +|override_minimum_cores|:::​|:::​|
 +|override_minimum_physical|:::​|:::​|
 +|override_minimum_speed|:::​|:::​|
 +|override_minimum_ram|:::​|:::​|
 +|override_minimum_disk_space|:::​|:::​|
 +|override_maximum_chunks_requeue|:::​|:::​|
 +|override_valid_exit_codes_value|String|Overrides specific values|
 +|override_warning_exit_codes_value|:::​|:::​|
 +|override_error_exit_codes_value|:::​|:::​|
 +|override_valid_log_texts_value|:::​|:::​|
 +|override_warning_log_texts_value|:::​|:::​|
 +|override_error_log_texts_value|:::​|:::​|
 +|override_minimum_threads_value|:::​|:::​|
 +|override_minimum_cores_value|:::​|:::​|
 +|override_minimum_physical_value|:::​|:::​|
 +|override_minimum_speed_value|:::​|:::​|
 +|override_minimum_ram_value|:::​|:::​|
 +|override_minimum_disk_space_value|:::​|:::​|
 +|override_maximum_chunks_requeue_value|:::​|:::​|
 +
 +==== db_jobs_data ====
 +^Field name^Field type^Description^
 +|id|32 bit integer|Primary key|
 +|job_id|32 bit integer|Job ID reference|
 +|key_name|String|Attribute name|
 +|key_value|String|Attribute value|
 +|key_state|Boolean|Attribute activation status|
 +|key_subst|Boolean|Attribute substitution status|
 +
 +==== db_chunks ====
 +^Field name^Field type^Description^
 +|id|32 bit integer|Primary key|
 +|chunk_id|32 bit integer|Sequential chunk id|
 +|job_id|32 bit integer|Job ID reference|
 +|status|32 bit integer|Chunk status|
 +|requeued|32 bit integer|Number of times chunk has been requeued|
 +|start_frame|Double float|Starting frame for the chunk|
 +|end_frame|Double float|Ending frame for the chunk|
 +|by_frame|Double float|By framing for the chunk|
 +|starting_time|64 bit integer, c time_t format|Chunk starting time|
 +|ending time|64 bit integer, c time_t format|Chunk ending time|
 +|rendering_host|String|Instance name that processed the chunk|
 +|result|String|Error string returned by the process|
 +|starting_number|32 bit integer|Starting numbering for the chunk|
 +|is_assembler|Boolean|Flagged for assembler chunks|
 +
 +==== db_logs ====
 +^Field name^Field type^Description^
 +|id|32 bit integer|Primary key|
 +|host_ip|String|IP of the host that generated the log|
 +|log_time|String|Formatted time of the log|
 +|host_name|String|Name of the instance that generated the log|
 +|event_user|String|Username of the user|
 +|log_entry|String|Text of the log entry|
 +|log_type|32 bit integer|Log type|
 +|job_id|32 bit integer|The job ID the log entry reefer too|
 +|chunk_id|32 bit integer|The chunk ID the log entry reefer too|
 +|instance_id|32 bit integer|The instance ID the log entry reefer too|
 +
 +==== db_mails ====
 +^Field name^Field type^Description^
 +|id|32 bit integer|Primary key|
 +|mail_address|String|List of e-mails associated with the distribution list separated by ;|
 +|mail_name|String|Name of the distribution list|
 +
 +==== db_mails_templates ====
 +^Field name^Field type^Description^
 +|template_name|String|The name of the template|
 +|template_content|String|The content of the template|
 +
 +==== db_paths ====
 +^Field name^Field type^Description^
 +|id|32 bit integer|Primary key|
 +|local_path|String|Dispatcher side path|
 +|win_path|String|Windows side path|
 +|linux_path|String|Linux side path|
 +|mac_path|String|Mac side path|
 +|scope|32 bit integer|Path scope|
 +
 +==== db_pools ====
 +^Field name^Field type^Description^
 +|id|32 bit integer|Primary key|
 +|pool_ip|String|IP address of the pool entry|
 +|pool_name|String|Instance name of the entry|
 +|pool_parent|String|Name of the parent pool|
 +|host_name|String|The host name of the pool entry|
 +|features|String|Additional features of the pool (encoded)|
 +
 +==== db_users ====
 +^Field name^Field type^Description^
 +|id|32 bit integer|Primary key|
 +|username|String|Name of the user|
 +|encoded_password|String|Base64 encoded password|
 +|allowed_rights|String|Bits mask of the supported user rights as a 64 bit integer|
 +|forbidden_rights|String|Bits mask of the forbidden user rights as a 64 bit integer|
 +|groups|String|List of user's groups separated by ,|
 +|paths|32 bit integer|User specific substitution paths|
 +|max_instances|32 bit integer|Maximum number of concurrent instances allocable by user|
 +|template_limit|Comma separated list of template IDs integers|Allowed templates|
 +|pool_limit|Comma separated list of pool limits, strings|Allowed pools|
 +|home_folder|32 bit integer|ID of user home folder, -1 for none|
 +|lock_home|Boolean|Lock user actions and views on the home folder|
 +|email_address|String|User email address|
 +
 +==== db_users_groups ====
 +^Field name^Field type^Description^
 +|id|32 bit integer|Primary key|
 +|groupname|String|Name of the group|
 +|allowed_rights|String|Bits mask of the supported user rights as a 64 bit integer|
 +|max_instances|32 bit integer|Maximum number of concurrent instances allocable by user|
 +|template_limit|Comma separated list of template IDs integers|Allowed templates|
 +|pool_limit|Comma separated list of pool limits, strings|Allowed pools|
 +|paths|32 bit integer|User specific substitution paths|
 +|enable_home_folder|32 bit integer|Enables home folders for the group'​s users|
 +|lock_home|Boolean|Lock user actions and views on the home folder|
 +|email_address|String|User email address|
 +
 +==== db_sequences ====
 +^Field name^Field type^Description^
 +|table_name|String|Name of the table|
 +|next_id|32 bit integer|Next valid table ID|
 +
 +==== db_nodes ====
 +^Field name^Field type^Description^
 +|id|32 bit integer|Primary key|
 +|instance_name|String|Name of the instance|
 +|instance_num|32 bit integer|Instance number|
 +|host_ip|String|Ip address of the instance|
 +|host_name|String|Host name of the instance|
 +|instance_notes|String|Notes of the instance|
 +|platform|32 bit integer|Platform of the instance|
 +|mac_addresses|String|List of instance NICs Mac addresses|
 +|wakeup_fullload|Boolean|Wakeup when at full load
 +|wakeup_bypool|Boolean|Wakeup when required by pool|
 +|wakeup_policies|32 bit integer|Wakeup policies|
 +|wakeup_match_availability|32 bit integer|Match availability status|
 +|availability_policies|String|Availability policies (encoded)|
 +|availableByDefault|Boolean|Availability default flag|
 +
 +===== History database =====
 +
 +==== db_history_chunks ====
 +^Field name^Field type^Description^
 +|id|32 bit integer|Primary key|
 +|chunk_id|32 bit integer|Chunk reference id|
 +|job_id|32 bit integer|Job reference id|
 +|job_name|String|Job name|
 +|job_project|String|The job project|
 +|job_group|String|The job group|
 +|job_department|String|The job department|
 +|job_file|String|The job main file|
 +|status|32 bit integer|Chunk final status|
 +|start_frame|Double float|Chunk starting frame|
 +|end_frame|Double float|Chunk ending frame|
 +|by_frame|Double float|Chunk ending frame|
 +|starting_time|64 bit integer, c time_t format|Chunk starting time|
 +|ending_time|64 bit integer, c time_t format|Chunk ending time|
 +|rendering_host|String|Chunk assigned instance name|
 +|result|String|Chunk result text|
 +|starting_number|32 bit integer|Chunk starting numbering|
 +|step_by|32 bit integer|Chunk numbering step by|
 +|packet_size|32 bit integer|Chunk packet size|
 +|is_assembler|Boolean|Chunk assembler flag|
 +
 +==== db_history_stats ====
 +^Field name^Field type^Description^
 +|id|32 bit integer|Primary key|
 +|sample_time|64 bit integer, c time_t format|Time of the samples snapshot
 +|connected_rc|32 bit integer|Number of connected clients|
 +|connected_ex|32 bit integer|Number of connected consoles/​mrtool|
 +|connected_not|32 bit integer|Number of connected notificators|
 +|connected_web_sessions|32 bit integer|Number of active web sessions|
 +|connected_web |32 bit integer|Number of active HTTP connections|
 +|connected_https |32 bit integer|Number of active HTTPS connections|
 +|rc_used |32 bit integer|Number of working nodes|
 +|rc_paused |32 bit integer|Number of paused nodes|
 +|rc_idle |32 bit integer|Number of idle nodes|
 +|rc_unavailable |32 bit integer|Number of disconnected nodes|
 +|total_in_kbs|Double float|Total amount of incoming KBytes|
 +|total_out_kbs|Double float|Total amount of outcoming KBytes|
 +|control_out_kbs|Double float|Console/​mrtool outcoming KBytes|
 +|control_in_kbs|Double float|Console/​mrtool incoming KBytes|
 +|rc_in_kbs|Double float|Nodes incoming KBytes|
 +|rc_out_kbs|Double float|Nodes outcoming KBytes|
 +|not_in_kbs|Double float|Notificators incoming KBytes|
 +|not_out_kbs|Double float|Notificators outcoming KBytes|
 +|web_in_kbs|Double float|Web incoming KBytes|
 +|web_out_kbs|Double float|Web outcoming KBytes|
 +