Use Case

Over time, we have found some users have needed to migrate OmicSoft Server infrastructure to different physical servers or to cloud based servers (i.e. within AWS). This is also true for storage solutions (i.e. Isilon/NFS to S3). In these cases, data objects generated in OmicSoft may need to be remapped to restore full functionality within projects or within the genome browser. The procedures below provide scripting framework to update the binary files within the OmicSoft projects and genome browswer files.

OmicSoft Project Environment

Within OmicSoft projects, users create NgsData objects that provide metadata and links to the processed data (e.g. bam files) that were generated within the project. These NgsData objects are used to feed into downstream analytic tools, such as quantification of gene expression (ReportGeneTranscriptCounts). When the underlying bam files have been moved due to server migration or hardware failure, users can update the project files to point to the proper files.

Generate Mapping Files

To correct the file locations in OmicSoft projects, users can create mapping files that provide the original and destination file/folder paths. The mapping file is a two column, tab-delimited file without headers:

Col 1 = old file/folder location

Col 2 = new file/folder location

For example:

Update File Paths individually

Recursively Update Folder Paths

Templates:

File:TemplateFile.txt

File:TemplateFolder.txt

Note these can be local locations, cloud locations or server locations, or a combination of these, as required.

Oscript to update Osobj file

Osobj: (Omicsoft object file) these files are streamed from user's local cache to view in the solution explorer in OmicSoft Projects. To update the object, run the following:

Begin UpdateOjobFile /Namespace=NgsLib;
Files "\path\to\old\file\\or\folder\sample.osobj";
Options 
/OutputFile="\path\to\new\file\sample.osobj"
/FileMappings="\file\mapping\file\file_mapping_osobj.txt"
/FolderMappings="\folder\mapping\file\folder_mapping_osobj.txt";
End;

OmicSoft Genome Browser

OmicSoft's Genome Browser provides an integrative viewer of NGS data files, such as BAM files that can stream from server or cloud based locations. Similar to links in OmicSoft projects, when files are moved, the data location link in the genome browser will be broken.

For example, from the genome browser:

]

And you add Bam files from a specific location to this browser:

Now at a later date, if there is a server migration or the files in a remote location are moved around, the server will lose the link to these files loaded onto the browser

Using the oscripts described below along with a mapping file, you can relink these files without having to reimport them as before.

Osgbf mapping file

As opposed to the solution above for osobj, for genome browser files, users can use a mixed mapping file, as genome browser files contain server information in addition to the file paths. The headers for this file are:

Depending on the use case, include all the headers but fill in only the pertinent information

scenario 1) mapping a local file: just specify Path and New_Path

scenario 2) mapping a local folder: just specify path and new path. In this case, the path MUST end in a forward/back slash character

scenario 3) mapping a remote file: in this case, if a server has moved, you need to fill in all 6 columns, with information about the new and old server details.

scenario 4) mapping a remote folder: same as above. do not forget the trailing slash.

For Example:

Template:

File:TemplateMixedMappingFile.txt

2) to update osgbf files

Begin UpdateGBProjectFile /Namespace=NgsLib;
Files "\path\to\old\files\or\folder\sample.osgbf";
Options 
/OutputFile="\path\to\output\sample.osgbf"
/MappingFile="path\to\mapping\file\mapping_files_mixed_osgbf.txt";
End;