Page Comparison

Follow the PBS Pro Design Document Guidelines.

Link to discussion on Developer Forum: <http://community.pbspro.org/t/design-for-refactoring-pbs-database-code/2009>

Link to issue: <issue link if available>

Link to pull request: <PR link if available>

For easy maintainability, current database-related code within the PBS server needs to be refactored into a separate pluggable dynamic library. And this design page discusses the APIs using which PBS server can talk to Libdb, the database dynamic library using which PBS can talk to the database. Another motivation behind this refactoring is the added advantage of moving to any database without having to do major code changes within PBS itself.

1.1. Files: List of files PBS will need to have database support.

PBS_EXEC/lib/libdb.so

Description: This dynamic library will have the functionality for the PBS server to access the database.

...

Usage: PBS_EXEC/libexec/pbs_db_utility <install_db | upgrade_db | init_db_env>

Parameters:

install_db: Create and initialize the dataservice instance.

upgrade_db: Upgrade the database.init

PBS_EXEC/libexec/pbs_db_env

...

Description: This script will initialize the environment with information related to the database installation path.

PBS_EXEC/include/pbs_db.h

Description: This header file will provide declarations for APIs and argument structures listed below.

pbsconn_db_conn_thandle: Structure Pointer used to maintain the database connection information. All elements of this structure are generic and are not bound to any particular database.

struct pbs_db_connection {
      void    *conn_db_handle;      /* opaque database handle  */
      int     conn_state;           /* Database connection status */
      int     conn_db_state;        /* Databese states, db up? down? starting? */
}
typedef struct pbs_db_connection pbs_db_conn_t;

/* Values for conn_state */
#define PBS_DB_CONNECT_STATE_NOT_CONNECTED  1 /* Not connected to the db */
#define PBS_DB_CONNECT_STATE_CONNECTING	    2 /* Connecting to the db */
#define PBS_DB_CONNECT_STATE_CONNECTED      3 /* Connected to the db */
#define PBS_DB_CONNECT_STATE_FAILED         4 /* Coonection to db failed */

/* Values for conn_db_state */
#define PBS_DB_DOWN		1  /* Database is down */
#define PBS_DB_STARTING		2  /* Database is starting */
#define PBS_DB_STARTED		3  /* Database is up */

pbs_db_obj_info_t: Wrapper object structure. It contains a pointer to one of the several database structures. Most of the database manipulation/query functions take this structure as a parameter. Depending on the contained structure type, an appropriate internal database manipulation/query function is eventually called. This allows keeping the interface simpler and generic.

struct pbs_db_obj_info {
  int 	pbs_db_obj_type; /* identifies the contained object type */
  union {
	pbs_db_job_info_t	*pbs_db_job; 	/* map database job structure to C */
	pbs_db_jobscr_info_t	*pbs_db_jobscr;	/* map database job script to C */
	pbs_db_resv_info_t	*pbs_db_resv;	/* map database resv structure to C */
	pbs_db_svr_info_t	*pbs_db_svr;	/* map database server structure to C */
	pbs_db_que_info_t	*pbs_db_que;	/* map database queue structure to C */
	pbs_db_node_info_t	*pbs_db_node;	/* map database node structure to C */
	pbs_db_sched_info_t	*pbs_db_sched;	/* map database scheduler structure to C */	
	pbs_db_mominfo_time_t	*pbs_db_mominfo_tm; /* map database mominfo_time structure to C */
  } pbs_db_un;
};
typedef struct pbs_db_obj_info pbs_db_obj_info_t;

/* Structure used to map database job structure to C */
struct pbs_db_job_info {

pbs_db_obj_info_t: Wrapper object structure. It contains a pointer to one of the several database structures. Most of the database manipulation/query functions take this structure as a parameter. Depending on the contained structure type, an appropriate internal database manipulation/query function is eventually called. This allows keeping the interface simpler and generic.

struct pbs_db_obj_info {
  int 	pbs_db_obj_type; /* identifies the contained object type */
  union {
	pbs_db_job_info_t	*pbs_db_job; 	/* map database job structure to C */
	pbs_db_jobscr_info_t	*pbs_db_jobscr;	/* map database job script to C */
	pbs_db_resv_info_t	*pbs_db_resv;	/* map database resv structure to C */
	pbs_db_svr_info_t	*pbs_db_svr;	/* map database server structure to C */
	pbs_db_que_info_t	*pbs_db_que;	/* map database queue structure to C */
	pbs_db_node_info_t	*pbs_db_node;	/* map database node structure to C */
	pbs_db_sched_info_t	*pbs_db_sched;	/* map database scheduler structure to C */	
	pbs_db_mominfo_time_t	*pbs_db_mominfo_tm; /* map database mominfo_time structure to C */
  } pbs_db_un;
};
typedef struct pbs_db_obj_info pbs_db_obj_info_t;

/* Structure used to map database job structure to C */
struct pbs_db_job_info {
	char     ji_jobid[PBS_MAXSVRJOBID + 1]; /* job identifier */
	INTEGER  ji_state; 	/* Internal copy of state */
	INTEGER  ji_substate; 	/* job sub-state */
	INTEGER  ji_svrflags; 	/* server flags */
	INTEGER  ji_numattr; 	/* not used */
	INTEGER  ji_ordering; 	/* special scheduling ordering */
	INTEGER  ji_priority; 	/* priority */
	BIGINT   ji_stime; 	/* time job started execution */
	BIGINT   ji_endtBdry; 	/* estimate upper bound on end time */
	char     ji_queue[PBS_MAXQUEUENAME + 1]; /* name of current queue */
	char     ji_jobiddestin[PBS_MAXSVRJOBIDMAXROUTEDEST + 1]; /* jobdest identifierfrom qmove/route */
	INTEGER  ji_un_statetype;    	/* INTEGERernaljob's copyqueue oftype state */
	INTEGER  ji_substatemomaddr; 	/* host addr jobof sub-stateServer */
	INTEGER  ji_svrflagsmomport; 	/* serverport flags# */
	INTEGER  ji_orderingexitstat; 	/* special scheduling orderingjob exit status from MOM */
	INTEGERBIGINT   ji_priorityquetime; 	/* INTEGERernaltime entered priorityqueue */
	BIGINT   ji_stimerteretry;    	/* route retry time */
	INTEGER  ji_fromsock; 	/* socket job startedcoming executionover */
	BIGINT   ji_endtBdryfromaddr; 	/* estimatehost upperjob boundcoming onfrom end time */
	char     ji_queue[PBS_MAXQUEUENAME + 14jid[8];  	/* nameextended ofjob currentsave queuedata */
	char     ji_destin[PBS_MAXROUTEDEST + 14ash[8];  	/* destextended from qmove/routejob save data */
	INTEGER  ji_un_typecredtype;  	/* job's queuecredential type */
	INTEGERBIGINT   ji_momaddrqrank;    	/* sort hostkey addrfor ofdb Serverquery */
	INTEGER  ji_momport; pbs_db_attr_list_t db_attr_list; /* port # list of attributes for database */
};
	INTEGERtypedef struct ji_exitstatpbs_db_job_info pbs_db_job_info_t;

/* job exit status from MOM */
	BIGINT   ji_quetime;  Structure used to map database resv structure to C */
struct pbs_db_resv_info {
	char    ri_resvid[PBS_MAXSVRJOBID + 1]; /* timereservation enteredidentifier queue */
	BIGINTchar    ji_rteretryri_queue[PBS_MAXQUEUENAME + 1]; /* routequeue used retryby timereservation */
	INTEGER  jiri_fromsockstate; 	/* socketinternal jobcopy comingof overstate */
	BIGINTINTEGER   jiri_fromaddrsubstate; 	/* hostsubstate jobof comingresv fromstate   */
	charBIGINT     ji_4jid[8];  ri_stime;	/* extendedleft jobwindow saveboundry data */
	charBIGINT     ji_4ash[8];  ri_etime;	/* extendedright jobwindow saveboundry data */
	INTEGERBIGINT  jiri_credtypeduration; 	/* credentialreservation typeduration */
	INTEGER  jiri_qrank;    tactive;	/* sorttime keyreservation forbecame dbactive query */
	BIGINTINTEGER   jiri_savetmsvrflags;	/* server flags */* job save time
	INTEGER ri_numattr;	/* not used */
	BIGINTINTEGER   jiri_creattmresvTag;  	/* jobnot createused time */
	pbsINTEGER ri_db_attr_list_t attr_list; un_type;	/* listnot of attributesused */
};
typedef struct pbs_db_job	INTEGER ri_fromsock;	/* resv from sock */
	BIGINT  ri_fromaddr;	/* resv from sock addr */
	pbs_db_attr_list_t db_attr_list; /* list of attributes */
};
typedef struct pbs_db_resv_info pbs_db_jobresv_info_t;

/* Structure used to map database resvserver structure to C */
struct pbs_db_resvsvr_info {
	charBIGINT  sv_jobidnumber;
 ri_resvid[PBS_MAXSVRJOBID + 1]	pbs_db_attr_list_t db_attr_list; /* list reservationof identifierattributes */
	char};
typedef struct  ri_queue[PBS_MAXQUEUENAME + 1]; /* queue used by reservation */
	INTEGER ri_state;	/* internal copy of state */
	INTEGER ri_substate;	/* substate of resv state */
	INTEGER ri_type;	/* "reservation" or "reservation job"*/
	BIGINT  ri_stime;	/* left window boundry  */
	BIGINT  ri_etime;	/* right window boundry */
	BIGINT  ri_duration;	/* reservation durationpbs_db_svr_info pbs_db_svr_info_t;

/* Structure used to map database scheduler structure to C */
struct pbs_db_sched_info {
	char    sched_name[PBS_MAXSCHEDNAME+1]; /* sched name */
	pbs_db_attr_list_t db_attr_list; 	/* list of attributes */
};
typedef struct pbs_db_sched_info pbs_db_sched_info_t;

/* Structure used to map database queue structure to C */
struct pbs_db_que_info {
	char    qu_name[PBS_MAXQUEUENAME +1]; 	/* queue name */
	INTEGER riqu_tactivetype;		 	/* timequeue reservationtype: becameexec, activeroute */
	INTEGER ri_svrflags;pbs_db_attr_list_t db_attr_list; 	/* list serverof flagsattributes */
	INTEGER ri_fromsock;	};
typedef struct pbs_db_que_info pbs_db_que_info_t;

/* resvStructure fromused sockto */map 	BIGINTdatabase  ri_fromaddr;	/* resv from sock addr node structure to C */
	BIGINT  ri_creattm;	/* resv create time on dbstruct pbs_db_node_info {
	char	nd_name[PBS_MAXSERVERNAME+1]; /* vnode's name */
	BIGINTINTEGER  rind_savetmindex;	/* resvglobal save time on dbnode index */
	pbs_db_attr_list_t attr_listBIGINT	mom_modtime; 	/* listnode config ofupdate attributestime */
};
typedef struct pbs_db_resv_info pbs_db_resv_info_t;
	char	nd_hostname[PBS_MAXSERVERNAME+1]; /* Structure used to map database server structure to Cnode hostname */
struct pbs_db_svr_info {
	INTEGER svnd_numjobsstate; 	/* numberstate of job owned by server node */
	INTEGER svnd_numquentype;	/* nubernode of queuestype managed */
	BIGINT  sv_jobidnumberchar	nd_pque[PBS_MAXSERVERNAME+1]; /* next numberqueue to usewhich in new jobid it belongs */
	BIGINT  sv_creattm; 	pbs_db_attr_list_t db_attr_list; /* timelist of serverattributes db create */
};
	BIGINTtypedef struct sv_savetm;	/* time of server db update */
	pbs_db_attr_list_t attr_list; /* list of attributes */
};
typedef struct pbs_db_svr_pbs_db_node_info pbs_db_svrnode_info_t;

/* Structure used to map database schedulermominfo_time structure to C */
struct pbs_db_schedmominfo_infotime {
	char    sched_name[PBS_MAXSCHEDNAME+1]BIGINT	mit_time; /* schedtime nameof */the 	BIGINThost  sched_creattm;	/* sched create time on db to vnode map */
	BIGINTINTEGER  schedmit_savetmgen;	  /* sched save time on dbgeneration of the host to vnode map */
	};
typedef struct pbs_db_attrmominfo_list_t attr_list; /* list of attributes */
};
typedef struct time pbs_db_sched_info pbs_db_sched_info_mominfo_time_t;

/* Structure used to map database queuejob structurescript to C */
struct pbs_db_quejobscr_info {
	char    qu ji_namejobid[PBS_MAXQUEUENAMEMAXSVRJOBID + 1]; /* queuejob nameidentifier */
	INTEGER qu_typeTEXT     script;	/* queue type: exec, route */
	BIGINT  qu_ctime;	/* time queue created */
	BIGINT  qu_mtime;	/* time queue last modified */
	pbs_db_attr_list_t attr_list; /* list of attributes */
};
typedef struct pbs_db_que_info pbs_db_que_info_t;

/* Structure used to map database node structure to C */
struct pbs_db_node_info {
	char	nd_name[PBS_MAXSERVERNAME+1]; /* vnode's name */
	INTEGER nd_index;	/* global node index */
	BIGINT	mom_modtime; 	/* node config update time */
	char	nd_hostname[PBS_MAXSERVERNAME+1]; /* node hostname */
	INTEGER nd_state;	/* state of node */
	INTEGER nd_ntype;	/* node type */
	char	nd_pque[PBS_MAXSERVERNAME+1]; /* queue to which it belongs */
	BIGINT  nd_creattm; 	/* node create time */
	BIGINT  nd_svtime;	/* node save time */
	pbs_db_attr_list_t attr_list; /* list of attributes */
};
typedef struct pbs_db_node_info pbs_db_node_info_t;

/* Structure used to map database mominfo_time structure to C */
struct pbs_db_mominfo_time {
	BIGINT	mit_time; /* time of the host to vnode map */
	INTEGER mit_gen;  /* generation of the host to vnode map */
};
typedef struct pbs_db_mominfo_time pbs_db_mominfo_time_t;

/* Structure used to map database job script to C */
struct pbs_db_jobscr_info {
	char     ji_jobid[PBS_MAXSVRJOBID + 1]; /* job identifier */
	TEXT     script;	/* job script */
};
typedef struct pbs_db_jobscr_info pbs_db_jobscr_info_t;

pbs_db_query_options_t: Structure used to pass database query options to database functions. Flags field can be used to pass any flags to a query function. Timestamp field can be used to pass a timestamp, to return rows that have a modification timestamp newer (more recent) than the timestamp passed. (Basically to return rows that have been modified since a point of time)

Structure used to pass database query options to database functions

struct pbs_db_query_options {
  int	flags; /* Flags field can be used to pass any flags to a query function. */
  time_t timestamp; /* Timestamp field can be used to pass a timestamp, to return rows that have a modification timestamp newer (more recent) than the timestamp passed. (Basically to return rows that have been modified since a point of time) */
};
typedef struct pbs_db_query_options pbs_db_query_options_t;

query_cb_t: Function pointer for call back function to process the data returned by the database.

typedef int (*query_cb_t) (pbs_db_obj_info_t *, int *);

...

Structure used to map database attr structure to C

struct pbs_db_attr_info {
	char	attr_name[PBS_MAXATTRNAME+1]; /* attribute name */
	char	attr_resc[PBS_MAXATTRRESC+1]; /* resource name */
	TEXT    attr_value;		      /* attribute value */
	INTEGER attr_flags; 		      /* attribute flags */
};
typedef struct pbs_db_attr_info pbs_db_attr_info_t;

job script */
};
typedef struct pbs_db_jobscr_info pbs_db_jobscr_info_t;

pbs_db_query_options_t: Structure used to pass database query options to database functions. Flags field can be used to pass any flags to a query function. Timestamp field can be used to pass a timestamp, to return rows that have a modification timestamp newer (more recent) than the timestamp passed. (Basically to return rows that have been modified since a point of time)

The structure used to pass database query options to database functions

struct pbs_db_query_options {
  int	flags; /* Flags field can be used to pass any flags to a query function. */
  time_t timestamp; /* Timestamp field can be used to pass a timestamp, to return rows that have a modification timestamp newer (more recent) than the timestamp passed. (Basically to return rows that have been modified since a point of time) */
};
typedef struct pbs_db_query_options pbs_db_query_options_t;

query_cb_t: Function pointer for call back function to process the data returned by the database. For each row/record returned by the search result, the function pointed by this pointer is called and the result is processed so that PBS can use it.

typedef int (*query_cb_t) (pbs_db_obj_info_t *, int *);

pbs_db_attr_list_t: Structure used to map database attr structure to C.

The structure used to map database attr structure to C

struct pbs_db_attr_list {
	int attr_count;			     /* attribute count */
	pbs_db_attr_info_t *attributeslist_head attrs; /* ptr to attr_info structure */
};
typedef struct pbs_db_attr_list pbs_db_attr_list_t;

1.2. APIs from libdb.so

1.2.1. PBS Objects: Libdb APIs can work with below mentioned PBS objects.

PBS_DB_JOB: To save and update PBS job objects

...

PBS_DB_MOMINFO_TIME: To save/update PBS mominfo_time

1.2.2. pbs_db_connect

Description: Setup a persistent database connection for further use by APIs which will work on PBS objects. On success return the connection information.

Signature:

(int pbs_db_connect(void *conn_t *) pbs_db_connect(handle, char *pbs_ds_host, int pbs_ds_port, int timeout, char **err_msg)

...

)

Parameters:

conn_db_handle[out]: This pointer will have connection information for the database on successfull connection.

pbs_data_service_host[in]: Hostname information where the database is running.

...

timeout[in]: Timeout in seconds before the API will return if it is taking too long to connect.err_msg[out]: Connection/system messages generated by libdb. On connection failure out error messages.

Returns:  Pointer to the initialized connection structure of type (pbs_db_conn_t *)

...

Error code

0: On successful connection setup to the database.

NULL!0: On connection failure.

1.2.3. pbs_db_disconnect

Description: Disconnect the PBS server from a database connection.

Signature:

int pbs_db_disconnect(pbsvoid *conn_db_conn_t *conn, char **err_msghandle)

Parameters:

conn_db_handle[in]: Connection structure with connection handler to the database which was created by pbs_db_connection.err_msg[out] : Connection logs/messages generated by libdb. On connecttion failure out error messages.connect.

Returns: Returns status of the disconnection.

0 - for successful database disconnect.

-1 - for failure

1.2.4. pbs_db_save_obj

Description: PBS can use this API to save any of the PBS Objects listed above to the database. Save operation can be an insert or update.

Signature:

int pbs_db_save_obj(pbsvoid *conn_db_conn_t *connhandle, pbs_db_obj_info_t *obj, int savetype)

Parameters:

conn_db_handle[in]: The database connnection handle : Connection handler to the database which was created by pbs_db_connectionconnect.

obj[in]: The PBS object to save. Based on the type defined within obj structure, right PBS object will be picked and dealt with.

savetype[in]: OBJ_SAVE_QS for quick save and OBJ_SAVE_NEW for insert.

Returns: Error code.

-1 - Execution of prepared statement failed.

...

1 - Execution succeeded but the statement did not affect any rows.

1.2.5. pbs_db_delete_obj

Description: This API lets PBS delete PBS objects PBS_DB_JOB, PBS_DB_RESV, PBS_DB_NODE, PBS_DB_QUE and PBS_DB_SCHED from the database.

Signature:

int pbs_db_delete_obj(pbsvoid *conn_db_conn_t *connhandle, pbs_db_obj_info_t *obj)

Parameters:

conn_db_handle[in]:The database connection handle Connection handler to the database which was created by pbs_db_connectionconnect.

obj[in]: PBS object information to be deleted. Based on the type defined within the obj structure, the right PBS object will be picked and dealt with.

Returns: Error code.

-1 - Failure

0 - Success

1 - Success but no rows deleted

1.2.6. pbs_db_load_obj

Description: This API lets PBS load objects data from the database. This API can work with any of the PBS objects.
Signature:

int pbs_db_load_obj(pbsvoid *conn_db_conn_t *connhandle, pbs_db_obj_info_t *obj)

Parameters:

conn_db_handle[in] - The database connection handle : Connection handler to the database which was created by pbs_db_connectionconnect..
obj[in/out] - PBS object information to be loaded from the database. Based on the type defined within obj structure, right PBS object will be picked and dealt with.

Returns: Error code

-1 - Failure

0 - Success
1 - Success but no rows loaded

1.2.7. pbs_db_

...

search

Description: This API can be used to find/search PBS objects PBS_DB_JOB, PBS_DB_RESV, PBS_DB_NODE, PBS_DB_QUE and PBS_DB_SCHED in the database. This API takes a pointer to the callback function as an argument that will work on the records returned from the database based on the query specified by the PBS.

Signature:

int pbs_db_find_obj(pbs_db_conn_t *connsearch(void *conn_db_handle, pbs_db_obj_info_t *obj, pbs_db_query_options_t *opts, query_cb_t query_cb)

Parameters:

conn_db_handle[in]:The database connection handle Connection handler to the database which was created by pbs_db_connectionconnect..

obj[in/out]: Information of the object to be found.

...

query_cb[in]: Function pointer to a callback function that will process the each record returned by the database for the find query made.

Returns: Error code

-1 - Failure

0 0> - Success
1 0 - Success but no rows found

1.2.8. pbs_db_del_attr_obj

Description: This API can be used to delete attributes of a PBS object.

Signature:

int pbs_db_deldelete_attr_obj(void *conn, pbs_db_connobj_info_t *connobj, void *obj_id, char *sv_time, pbs_db_attr_list_t *attr_list)

Parameters:

conn[in]: The database connection handle which was created by pbs_db_connection..

obj_id[in]: Object id for the delete attributes action.

sv_time[in]: Save time stamp for the delete operation.

attr_list[in]: List of attributes to be deleted.

...

pbs_db_attr_list_t *db_attr_list)

Parameters:

conn_db_handle[in]: Connection handler to the database which was created by pbs_db_connect.

obj[in]: Information of the object to be found.

obj_id[in]: Object id for the delete attributes action.

attr_list[in]: List of attributes to be deleted from DB.

Returns: Error code

0 - Success

Non zero - On Failure

1.2.9. pbs_start_db

Description: This API can be used to start the database instance.

Signature:

int pbs_start_db(char *pbs_ds_host, int pbs_ds_port)

Returns: Status code

0 - Success

1 Non zero - On Failure

1.2.

...

10. pbs_

...

stop_db

Description: This API can be used to start stop the database instance.

Signature:

int pbs_startstop_db(char **errmsg)

Parameters:

errmsg[out]: returns the startup error message if any

...

pbs_ds_host, int pbs_ds_port)

Returns: Status code

0 - Success

Non zero - On Failure

1.2.

...

11. pbs_

...

status_db

Description: This API can be used to shut down check if the database instance is running.

Signature:

int pbs_shutdownstatus_db(char **errmsgpbs_ds_host, int pbs_ds_port)

ParametersReturns:

errmsg[out]: returns the shutdown error message if any

Returns: Error code

0 - Success

...

Status code

1 - Database is not running.

2 - Database is running on a different host.

0 - Database is running on the localhost.

-1 - On Failure

1.2.

...

12. pbs_

...

db_

...

password

Description: This API can be used to check if change the database user and password for the database instance is running.

Signature:

int pbs_status_db(char **errmsg)

Parameters:

errmsg[out]: returns the db error message if anydb_password(void *conn_db_handle, char *user_name, char *password, char *old_user)

Parameters:

conn_db_handle[in]: Connection handler to the database which was created by pbs_db_connect.

user_name[in]: Database user name.

password[in]: New password for the database.

old_user[in]: old database user name used in case of -C option to change database user.

Returns: Error code

0 - Success

Non zero - On Failure

1.2.12. pbs_db_

...

get_errmsg

Description: This API can be used to get the error messages from the database library. When any of the above-listed APIs fail, this API can be used to change the database user and password for the database instanceget the error message from libdb.

Signature:

...

void pbs_db_

...

get_errmsg(int failcode, char *

...

Parameters:

conn[in]: The database connection handle which was created by pbs_db_connection.

user_name[in]: Databse user name.

password[in]: New password for the database.

change_user[in]: Takes values 0 or 1. When set to 1, user_name will be set as new database user.

Returns: Error code

0 - Success

...

*errmsg)

Parameters:

failcode[in]: Error code returned by one of the above APIs.

errmsg[out]: returns the DB error message if any.