This directory contains the MSDSD incremental application. Download and unpack the compressed tar archive, unpack it and read the README.txt file for more information. MSDSD Periodic incremental mechanism version beta01: This is a mechanism for the periodic incremental update of the MSD search database for more details look at http://www.ebi.ac.uk/msd. Requirements: Before running the mechanism check that your host fullfills the following requirements - Standard Java 1.3 runtime (there may be problems with FAST VM's or newer version of Java). - Access to an initial replica database of the MSDSD on Oracle (for more information look at http://www.ebi.ac.uk/msd) - Access to a seperate temporary incremental schema, on the same database with the main schema of MSDSD (for more information look at http://www.ebi.ac.uk/msd) - An Oracle client installation - The oracle environmental variables (like ORACLE_HOME,NLS_LANG) setup properly and the $ORACLE_HOME/bin in the PATH of the user that will run the script. The Oracle client commands imp and sqlplus should be in the PATH and work properly. - A working JDBC driver for your version of ORACLE that works OK with the OCI API. The driver zip file (i.e. $ORACLE_HOME/jdbc/lib/classes12.zip) should be included in the CLASSPATH enviromental variable - Access to the internet and the ability to download files through ftp. The unix GNU Wget (1.6) command in the PATH and working. The "wget" command should be able to download remote files in passive mode. Test wget --passive-ftp - The standard unix "mail" command should be in the path and working properly to forward warning and error mails. - The standard unix "gzip" command should be in the path and working properly to uncompress .gz files - A directory for temporary storage of the downloaded incremental files Setup: ----- In order to properly setup the mechanism you need to download and unzip the appropriate version of the incremental update mechanism that is compatible with your database version (you will get an error message mentioning the appropriate version if you attempt to run one that is incompatible with your replicate database), set up the enviroment as mentioned above (check ORACLE_HOME,NLS_LANG,PATH and CLASSPATH) and then edit and fix the initialisation file ImportPeriodic.ini. There you will have to set the variables "ConnectionString" and "IConnectionString" for connection to your main and incremental users and the variables "User" and "IUser" for the name of the 2 schemas. You will also have to set the "OutputDirectory" variable with the location of the temporary area where the downloaded incremental files will be stored as well as the "ErrorMailAddresses" where messages and warnings will be sent by the incremental mechanism. Next you may wish to specify which "marts" and tables you want to replicate. For that reason you may update the "INSTALLED" column of the corresponding ADMIN_GROUP and ADMIN_TABLE tables of your main schema on the replicated database to the appropriates Y / N values. Tests: ------ After you have downloaded and installed this set of scripts you may perform the following tests to make sure that the setup is correct. - Test java and rhino: Try to run the command "js" included in the downloaded incremental update directory. If you get a correct prompt "js>" give "print('hello');" - Test your connection to the main and incremental user: try "sqlplus " and give SELECT VALUE FROM ADMIN_MSD WHERE PARAMETER_NAME='database_version'; SELECT * from user_indexes t where t.status = 'UNUSABLE' ; if there are indexes unusable rebuild them. SELECT GROUP_NAME,INSTALLED FROM ADMIN_GROUP; SELECT MESSAGE FROM ADMIN_UPDATE WHERE STATUS!='Y' (if this returns rows then there are unresolved errors from older attempts) also try "sqlplus " - Test your JDBC and rhino enviroment. Try to run "js" and issue the following commands: js>load("common/dbUtils.js"); js>var conn=dbConnect("","",""); // Get the values from the ini file js>var admin=getQueryResults(conn,"SELECT * FROM ADMIN_MSD"); js>dumpResults(admin); - Check that the imp, wget and mail commands are in your $PATH by trying the following commands: imp HELP=Y wget --passive-ftp /admin_export.dmp echo test|mail How to run: From a terminal window in the downloaded incremeental directory (after the environmental variables have been setup) give js ImportPeriodic.js |tee ImportPeriodic.log You may check the ouput of the command or your mailbox for any message or warnings In the long run you should: Prepare a cron script that should be run by your cron tab that: - sets up all the enviromental variables - changes directory (cd) to the incremental directory - runs js ImportPeriodic.js >> ImportPeriodic.log You should leave that running periodically and in case of no warnings or errors you may just do some cleanup in the incremental directory to truncate log files etc. and the usual oracle DBA work (i.e. check that tablespaces are not getting full) There are though several cases that you may have to deviate from this routine: a) There is a database patch that has to be applied manually. You will receive a warning e-mail from the inremental mechanism that there is a patch that needs for some reason to be applied manually. The incremental mechanism will then stop working until you have applied this patch by: Getting in contact with the MSD group and/or following the instructions on the MSD web site. After the appropriate application of the manual patch the incremental mechanism will resume working as normal. b) You need to upgrade the incremental mechanism. You will receive a warning e-mail from the incremental update mechanism that the current version is not any more capable of performing the incremental updates on your version of database and you should download a new version. Download the new version and follow the instructions in its README file c) There was an error during the incremental process. You will receive an error e-mail from the incremental update mechanism that will give details of the context of its run and the error message. You will need to take action to resolve the problem of the error (i.e. in case of lack of disk space: add more disk space to your oracle tablespaces) and then check the ADMIN_TEMP table. If there are entries in the ADMIN_TEMP table then this means that the incremental mechanism has stop halfway and that some of the tables have been incremented while others not. You may want to get in contact with the MSD group about how to proceed. Otherwise it may be the case that the incremental mechanism stopped before doing any modifications in the database (ie. there was not enough space in the temporary ftp area, or there was an error in the enviromental variables) in that case remove the error entry from the ADMIN_UPDATE table and try to run the incremental mechanism once more. In any case you may want to get in contact with the MSD group to make sure that you are dealing correctly with the error. Have in mind that if an error has happened once, the incremental mechanism will stop working until you delete manually the error record from the ADMIN_UPDATE table.