Migrate OmicSoftServer v11 to v12
From Array Suite Wiki
OmicSoft recommends migrating all installations of OmicSoft Server on version 11.7 or earlier to version 12.1, to new server instance(s), running on Ubuntu 20, RHEL 8, CentOS 8 or Amazon Linux 2.
1. Make an inventory/list with all places where data is stored by OmicSoft Server
- In the ArrayServerLinuxBeta.exe installation directory, check ArrayServer.cfg, make a copy
- Folders.cfg: Double-check the configuration file that defines where other custom relative paths can be added
- Copy default.template and sample.template: Default.template; Sample.template
- Main directories can be found in the ArrayServer.cfg > Directory Options:
- Other folder mappings:
- Other configurations in ArrayServer.cfg to be aware of: Database backup, Analytic Servers, LDAP section, SearchProject section, Filter section, CloudFolder section, CloudInstanceTag, CloudVolumeTag, CFRPart11 section.
2. Inform the users
Announce all users well in advance for a planned downtime, and ask them to complete any pending jobs before this.
Setup OmicSoft Server on a new server instance
If you are installing on AWS, we recommend using one of our pre-built AMIs
- Ubuntu 20.04 using pre-built AMIs (recommended)
- Red Hat Enterprise Linux 8.5 using pre-built AMIs
- CentOS Stream 8 using pre-built AMIs
- Amazon Linux 2 using pre-built AMIs
If required by your IT policy, you can Install OmicSoft Server on your own, using any of the following operating systems
Make sure the hardware characteristics (processor, storage, memory) are at least equal or better than the old machine. Please consult Hardware/Software Requirements for OmicSoft Server.
Perform data and configuration migration
- Make sure there are no running jobs or connected users (In Studio > Server tab > Manage > Manage sessions: Sessions, Users, Log)
- Stop the existing OmicSoft Server program and all remaining ojob processes and cloud instances connected to it (see Manage Cloud Instances)
- Stop the OmicSoft Server program on the new server instance, if it is running.
- Copy all data identified at step (1) (the server's and users' data) from the old server on the new one, in folders having the same path/name.
- If possible, make sure the old data is backed-up
- do NOT copy the mono or sqlite folders
- Copy the old ArrayServer.cfg and other config files (from step 1.) from the old OmicSoft Server installation folder: default.template, sample.template, Folders.cfg (if it exists) to the new server (backup the existing files first).
- Modify ArrayServer.cfg in order to conform with the requirements of OmicSoft Server v12. Please see section "Update ArrayServer.cfg".
- Attach and mount all existing volumes/devices, that contain data and were not copied, with the same names, on the new server(e.g., for Amazon EBS: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-attaching-volume.html;
- If needed, re-create the local users, ex. - the user used to run Array Server service
- If running on Amazon AWS, associate the same key pair, VPC, IAM users, policy, roles, security group, with the instance (see Steps_to_build_AWS_ArrayServer_with_cloud_plugin )
- Configure the backup policy, if any
Run Omicsoft Service Updater
If the mono process instance running the Omicsoft Server is started, make sure it is stopped first.
$ cd /opt/arrayserver # If not downloaded recently, get the latest version of OmicSoftServiceUpdater: wget -c https://resources.omicsoft.com/software_update/OmicSoftServiceUpdater.exe $ mono ./OmicsoftServiceUpdater.exe
The downloaded template contains the ArrayServer.cfg, which is the key configuration file. The following sections and fields are mandatory:
- License section - server license code for your company
- Port2 and Port3 - define the port number for data communication (gRPC and SFTP data transfer) between OmicSoft Studio client and OmicSoft Server. Make sure to open these ports on firewall.
- CertificateThumbprint - specifies the server certificate thumbprint
- MasterServerUrl - identify the DNS name and the gRPC port on which the server can be accessed
- OmicSoft directories
- BaseDirectory - this will be the working directory of the array server, storing all the raw and possessed data. Depending on the projects, it can take huge amount of disk space.
- OmicsoftDirectory - This directory can be defined locally or on a network drive. All the reference genomes, gene models, Affymetrix CDF files, log files, etc. are stored in this folder.
- TempDirectory - This should be a local directory (i.e. NOT a network drive) for fast read and write access. It can take twice the size of an unzipped fastq file in some NGS tasks (we would suggest use a drive with at least 100GB storage).
- LandDirectory - directory for Land data storage. Depending on the number of Lands installed, this can take several hundred GB.
- Locations of Mono 6 binaries installed on your Oshell instance(s), which are required by jobs started through the server:
- The [Folder] section defines additional local or network folders monitored and available for access by array server users.
To enable cloud-based NGS analysis with the Array Server Cloud add-on, please follow the instructions from ArrayServer_Configuration_with_Cloud. The following properties must be configured within ArrayServer.cfg.
- General Cloud Preferences
Update metadata template files
- default.template: it defines project level meta data, such as requiring project title, and list organism. Admin can customize the project meta data based on this template file and even enforce controlled vocabulary. For first time setup, the default values we prepared should be good to go. Admin can further customized based on the understanding of default.template wiki article.
- sample.template: it defines sample level meta data, such as requiring users to fill organism and tissue for each samples in sample registration. Admin can customize the sample meta based on this template file and even enforce controlled vocabulary. For first time setup, the default values we prepared should be good to go. Admin can further customize based on the understanding of sample.template wiki article.
- Make the necessary DNS changes to point to the new server's IP, if necessary.
- Announce the users about the change, including the fact that they must update OmicSoft Studio to the latest version, before using the software.