Differences
This shows you the differences between two versions of the page.
muster:8.0:databases_structure [2018/01/11 08:21] |
muster:8.0:databases_structure [2018/01/11 08:21] (current) |
||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Databases structure ====== | ||
+ | |||
+ | Muster stores the queue and internal configurations in a database that may be on a local file (using sqlite) or external (Mysql or SQL Server). | ||
+ | |||
+ | An internal abstraction layer is used for certain data types, the field types you’ll find in the descriptions may appear differently depending on the destination database. | ||
+ | |||
+ | The following section lists the internal tables and their purpose. **You are just allowed to read data from the Muster database. You are not allowed to write data neither change the data structure of the tables: Muster will check the structure on any service start-up and rebuild them according.** | ||
+ | |||
+ | * db_jobs: Table holding the jobs general data either for folders and real jobs | ||
+ | * db_jobs_data: Table holding attributes for real jobs as well as custom attributes | ||
+ | * db_chunks: Table holding the chunks data | ||
+ | * db_logs: Table holding the internal Muster log | ||
+ | * db_mails: Table holding distribution lists | ||
+ | * db_pools: Table holding pools configuration | ||
+ | * db_users: Table holding users configuration | ||
+ | * db_users_groups: Table holding groups configuration | ||
+ | * db_paths: Table holding global substitution paths | ||
+ | |||
+ | * db_history_chunks: Table holding the Muster processing history (Stored in a separate file/db) | ||
+ | |||
+ | ===== Primary database ===== | ||
+ | |||
+ | ==== db_jobs ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |id|32 bit integer|Primary key| | ||
+ | |parent_id|32 bit integer|ID of the parent folder| | ||
+ | |indent|32 bit integer|Level of indentation| | ||
+ | |status|32 bit integer|Job status| | ||
+ | |packet_type|32 bit integer|Type of job packets| | ||
+ | |packet_size|32 bit integer|Size of each chunk| | ||
+ | |requeued|32 bit integer|Number of times job has been requeued| | ||
+ | |engine|32 bit integer|Template uid| | ||
+ | |max_cpu|32 bit integer|Maximum number of concurrent instances| | ||
+ | |max_cpu_working|32 bit integer|Number of current working instances| | ||
+ | |depend_id|32 bit integer|Depend from id| | ||
+ | |depend_mode|32 bit integer|Depend mode used| | ||
+ | |job_type|32 bit integer|Job type, Folder or real job| | ||
+ | |job_name|String|Name of the job| | ||
+ | |priority|32 bit integer|Priority for the job| | ||
+ | |is_paused|Boolean|Pause status| | ||
+ | |is_locked|Boolean|Locked status| | ||
+ | |job_pool|String|Target pools comma separated| | ||
+ | |Job_excluded_pool|String|Excluded pools comma separated| | ||
+ | |submission_time|64 bit integer, c time_t format|Time of submission| | ||
+ | |start_time|64 bit integer, c time_t format|Time of job execution| | ||
+ | |progress|32 bit integer|Job progress| | ||
+ | |end_time|64 bit integer, c time_t format|Time of job termination| | ||
+ | |start_on|64 bit integer, c time_t format|Specify the scheduled starting time for the job| | ||
+ | |resume_on|64 bit integer, c time_t format|Specify the scheduled resume time for the job| | ||
+ | |pause_on|64 bit integer, c time_t format|Specify the scheduled paused time for the job| | ||
+ | |submitter|String|Username of the job owner| | ||
+ | |exitcodes_err_check_type|32 bit integer|Specify the type of exit codes check| | ||
+ | |logs_err_check_type|32 bit integer|Specify the type of logs texts error check| | ||
+ | |override_mail_address|Boolean|Enable mail address override| | ||
+ | |mail_address|String|Mail / distribution for notification (if overridden)| | ||
+ | |override_job_check|Boolean|Enable job notifications override| | ||
+ | |job_check|Boolean|Job check status| | ||
+ | |job_check_type|String|Job check type| | ||
+ | |override_chunk_check|Boolean|Enable chunk notifications override| | ||
+ | |chunk_check|Boolean|Chunk check status| | ||
+ | |chunk_check_type|String|Chunk check type| | ||
+ | |override_chunks_timeout|Boolean|Override the default value for chunks timeout| | ||
+ | |override_chunks_timeout_value|32 bit integer|Timeout value| | ||
+ | |is_archivied|Boolean|Specify if the job is in archived status| | ||
+ | |job_group|String|Specify the job group| | ||
+ | |job_department|String|Specify the job department| | ||
+ | |override_valid_exit_codes|Boolean|Enables overrides| | ||
+ | |override_warning_exit_codes|:::|:::| | ||
+ | |override_error_exit_codes|:::|:::| | ||
+ | |override_valid_log_texts|:::|:::| | ||
+ | |override_warning_log_texts|:::|:::| | ||
+ | |override_error_log_texts|:::|:::| | ||
+ | |override_minimum_threads|:::|:::| | ||
+ | |override_minimum_cores|:::|:::| | ||
+ | |override_minimum_physical|:::|:::| | ||
+ | |override_minimum_speed|:::|:::| | ||
+ | |override_minimum_ram|:::|:::| | ||
+ | |override_minimum_disk_space|:::|:::| | ||
+ | |override_maximum_chunks_requeue|:::|:::| | ||
+ | |override_valid_exit_codes_value|String|Overrides specific values| | ||
+ | |override_warning_exit_codes_value|:::|:::| | ||
+ | |override_error_exit_codes_value|:::|:::| | ||
+ | |override_valid_log_texts_value|:::|:::| | ||
+ | |override_warning_log_texts_value|:::|:::| | ||
+ | |override_error_log_texts_value|:::|:::| | ||
+ | |override_minimum_threads_value|:::|:::| | ||
+ | |override_minimum_cores_value|:::|:::| | ||
+ | |override_minimum_physical_value|:::|:::| | ||
+ | |override_minimum_speed_value|:::|:::| | ||
+ | |override_minimum_ram_value|:::|:::| | ||
+ | |override_minimum_disk_space_value|:::|:::| | ||
+ | |override_maximum_chunks_requeue_value|:::|:::| | ||
+ | |||
+ | ==== db_jobs_data ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |id|32 bit integer|Primary key| | ||
+ | |job_id|32 bit integer|Job ID reference| | ||
+ | |key_name|String|Attribute name| | ||
+ | |key_value|String|Attribute value| | ||
+ | |key_state|Boolean|Attribute activation status| | ||
+ | |key_subst|Boolean|Attribute substitution status| | ||
+ | |||
+ | ==== db_chunks ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |id|32 bit integer|Primary key| | ||
+ | |chunk_id|32 bit integer|Sequential chunk id| | ||
+ | |job_id|32 bit integer|Job ID reference| | ||
+ | |status|32 bit integer|Chunk status| | ||
+ | |requeued|32 bit integer|Number of times chunk has been requeued| | ||
+ | |start_frame|Double float|Starting frame for the chunk| | ||
+ | |end_frame|Double float|Ending frame for the chunk| | ||
+ | |by_frame|Double float|By framing for the chunk| | ||
+ | |starting_time|64 bit integer, c time_t format|Chunk starting time| | ||
+ | |ending time|64 bit integer, c time_t format|Chunk ending time| | ||
+ | |rendering_host|String|Instance name that processed the chunk| | ||
+ | |result|String|Error string returned by the process| | ||
+ | |starting_number|32 bit integer|Starting numbering for the chunk| | ||
+ | |is_assembler|Boolean|Flagged for assembler chunks| | ||
+ | |||
+ | ==== db_logs ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |id|32 bit integer|Primary key| | ||
+ | |host_ip|String|IP of the host that generated the log| | ||
+ | |log_time|String|Formatted time of the log| | ||
+ | |host_name|String|Name of the instance that generated the log| | ||
+ | |event_user|String|Username of the user| | ||
+ | |log_entry|String|Text of the log entry| | ||
+ | |log_type|32 bit integer|Log type| | ||
+ | |job_id|32 bit integer|The job ID the log entry reefer too| | ||
+ | |chunk_id|32 bit integer|The chunk ID the log entry reefer too| | ||
+ | |instance_id|32 bit integer|The instance ID the log entry reefer too| | ||
+ | |||
+ | ==== db_mails ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |id|32 bit integer|Primary key| | ||
+ | |mail_address|String|List of e-mails associated with the distribution list separated by ;| | ||
+ | |mail_name|String|Name of the distribution list| | ||
+ | |||
+ | ==== db_mails_templates ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |template_name|String|The name of the template| | ||
+ | |template_content|String|The content of the template| | ||
+ | |||
+ | ==== db_paths ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |id|32 bit integer|Primary key| | ||
+ | |local_path|String|Dispatcher side path| | ||
+ | |win_path|String|Windows side path| | ||
+ | |linux_path|String|Linux side path| | ||
+ | |mac_path|String|Mac side path| | ||
+ | |scope|32 bit integer|Path scope| | ||
+ | |||
+ | ==== db_pools ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |id|32 bit integer|Primary key| | ||
+ | |pool_ip|String|IP address of the pool entry| | ||
+ | |pool_name|String|Instance name of the entry| | ||
+ | |pool_parent|String|Name of the parent pool| | ||
+ | |host_name|String|The host name of the pool entry| | ||
+ | |features|String|Additional features of the pool (encoded)| | ||
+ | |||
+ | ==== db_users ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |id|32 bit integer|Primary key| | ||
+ | |username|String|Name of the user| | ||
+ | |encoded_password|String|Base64 encoded password| | ||
+ | |allowed_rights|String|Bits mask of the supported user rights as a 64 bit integer| | ||
+ | |forbidden_rights|String|Bits mask of the forbidden user rights as a 64 bit integer| | ||
+ | |groups|String|List of user's groups separated by ,| | ||
+ | |paths|32 bit integer|User specific substitution paths| | ||
+ | |max_instances|32 bit integer|Maximum number of concurrent instances allocable by user| | ||
+ | |template_limit|Comma separated list of template IDs integers|Allowed templates| | ||
+ | |pool_limit|Comma separated list of pool limits, strings|Allowed pools| | ||
+ | |home_folder|32 bit integer|ID of user home folder, -1 for none| | ||
+ | |lock_home|Boolean|Lock user actions and views on the home folder| | ||
+ | |email_address|String|User email address| | ||
+ | |||
+ | ==== db_users_groups ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |id|32 bit integer|Primary key| | ||
+ | |groupname|String|Name of the group| | ||
+ | |allowed_rights|String|Bits mask of the supported user rights as a 64 bit integer| | ||
+ | |max_instances|32 bit integer|Maximum number of concurrent instances allocable by user| | ||
+ | |template_limit|Comma separated list of template IDs integers|Allowed templates| | ||
+ | |pool_limit|Comma separated list of pool limits, strings|Allowed pools| | ||
+ | |paths|32 bit integer|User specific substitution paths| | ||
+ | |enable_home_folder|32 bit integer|Enables home folders for the group's users| | ||
+ | |lock_home|Boolean|Lock user actions and views on the home folder| | ||
+ | |email_address|String|User email address| | ||
+ | |||
+ | ==== db_sequences ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |table_name|String|Name of the table| | ||
+ | |next_id|32 bit integer|Next valid table ID| | ||
+ | |||
+ | ==== db_nodes ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |id|32 bit integer|Primary key| | ||
+ | |instance_name|String|Name of the instance| | ||
+ | |instance_num|32 bit integer|Instance number| | ||
+ | |host_ip|String|Ip address of the instance| | ||
+ | |host_name|String|Host name of the instance| | ||
+ | |instance_notes|String|Notes of the instance| | ||
+ | |platform|32 bit integer|Platform of the instance| | ||
+ | |mac_addresses|String|List of instance NICs Mac addresses| | ||
+ | |wakeup_fullload|Boolean|Wakeup when at full load | ||
+ | |wakeup_bypool|Boolean|Wakeup when required by pool| | ||
+ | |wakeup_policies|32 bit integer|Wakeup policies| | ||
+ | |wakeup_match_availability|32 bit integer|Match availability status| | ||
+ | |availability_policies|String|Availability policies (encoded)| | ||
+ | |availableByDefault|Boolean|Availability default flag| | ||
+ | |||
+ | ===== History database ===== | ||
+ | |||
+ | ==== db_history_chunks ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |id|32 bit integer|Primary key| | ||
+ | |chunk_id|32 bit integer|Chunk reference id| | ||
+ | |job_id|32 bit integer|Job reference id| | ||
+ | |job_name|String|Job name| | ||
+ | |job_project|String|The job project| | ||
+ | |job_group|String|The job group| | ||
+ | |job_department|String|The job department| | ||
+ | |job_file|String|The job main file| | ||
+ | |status|32 bit integer|Chunk final status| | ||
+ | |start_frame|Double float|Chunk starting frame| | ||
+ | |end_frame|Double float|Chunk ending frame| | ||
+ | |by_frame|Double float|Chunk ending frame| | ||
+ | |starting_time|64 bit integer, c time_t format|Chunk starting time| | ||
+ | |ending_time|64 bit integer, c time_t format|Chunk ending time| | ||
+ | |rendering_host|String|Chunk assigned instance name| | ||
+ | |result|String|Chunk result text| | ||
+ | |starting_number|32 bit integer|Chunk starting numbering| | ||
+ | |step_by|32 bit integer|Chunk numbering step by| | ||
+ | |packet_size|32 bit integer|Chunk packet size| | ||
+ | |is_assembler|Boolean|Chunk assembler flag| | ||
+ | |||
+ | ==== db_history_stats ==== | ||
+ | ^Field name^Field type^Description^ | ||
+ | |id|32 bit integer|Primary key| | ||
+ | |sample_time|64 bit integer, c time_t format|Time of the samples snapshot | ||
+ | |connected_rc|32 bit integer|Number of connected clients| | ||
+ | |connected_ex|32 bit integer|Number of connected consoles/mrtool| | ||
+ | |connected_not|32 bit integer|Number of connected notificators| | ||
+ | |connected_web_sessions|32 bit integer|Number of active web sessions| | ||
+ | |connected_web |32 bit integer|Number of active HTTP connections| | ||
+ | |connected_https |32 bit integer|Number of active HTTPS connections| | ||
+ | |rc_used |32 bit integer|Number of working nodes| | ||
+ | |rc_paused |32 bit integer|Number of paused nodes| | ||
+ | |rc_idle |32 bit integer|Number of idle nodes| | ||
+ | |rc_unavailable |32 bit integer|Number of disconnected nodes| | ||
+ | |total_in_kbs|Double float|Total amount of incoming KBytes| | ||
+ | |total_out_kbs|Double float|Total amount of outcoming KBytes| | ||
+ | |control_out_kbs|Double float|Console/mrtool outcoming KBytes| | ||
+ | |control_in_kbs|Double float|Console/mrtool incoming KBytes| | ||
+ | |rc_in_kbs|Double float|Nodes incoming KBytes| | ||
+ | |rc_out_kbs|Double float|Nodes outcoming KBytes| | ||
+ | |not_in_kbs|Double float|Notificators incoming KBytes| | ||
+ | |not_out_kbs|Double float|Notificators outcoming KBytes| | ||
+ | |web_in_kbs|Double float|Web incoming KBytes| | ||
+ | |web_out_kbs|Double float|Web outcoming KBytes| | ||
+ | |||