Update Browser and Data object file locations
From Array Suite Wiki
Over time, we have found some users have needed to migrate OmicSoft Server infrastructure to different physical servers or to cloud based servers (i.e. within AWS). This is also true for storage solutions (i.e. Isilon/NFS to S3). In these cases, data objects generated in OmicSoft may need to be remapped to restore full functionality within projects or within the genome browser. The procedures below provide scripting framework to update the binary files within the OmicSoft projects and genome browswer files.
OmicSoft Project Environment
Within OmicSoft projects, users create NgsData objects that provide metadata and links to the processed data (e.g. bam files) that were generated within the project. These NgsData objects are used to feed into downstream analytic tools, such as quantification of gene expression (ReportGeneTranscriptCounts). When the underlying bam files have been moved due to server migration or hardware failure, users can update the project files to point to the proper files.
Generate Mapping Files
To correct the file locations in OmicSoft projects, users can create mapping files that provide the original and destination file/folder paths. The mapping file is a two column, tab-delimited file without headers:
Col 1 = old file/folder location
Col 2 = new file/folder location
Note these can be local locations, cloud locations or server locations, or a combination of these, as required.
Oscript to update Osobj file
Osobj: (Omicsoft object file) these files are streamed from user's local cache to view in the solution explorer in OmicSoft Projects. To update the object, run the following:
Begin UpdateOjobFile /Namespace=NgsLib; Files "\path\to\old\file\\or\folder\sample.osobj"; Options /OutputFile="\path\to\new\file\sample.osobj" /FileMappings="\file\mapping\file\file_mapping_osobj.txt" /FolderMappings="\folder\mapping\file\folder_mapping_osobj.txt"; End;
OmicSoft Genome Browser
OmicSoft's Genome Browser provides an integrative viewer of NGS data files, such as BAM files that can stream from server or cloud based locations. Similar to links in OmicSoft projects, when files are moved, the data location link in the genome browser will be broken.
For example, from the genome browser:
And you add Bam files from a specific location to this browser:
Now at a later date, if there is a server migration or the files in a remote location are moved around, the server will lose the link to these files loaded onto the browser
Using the oscripts described below along with a mapping file, you can relink these files without having to reimport them as before.
Osgbf mapping file
As opposed to the solution above for osobj, for genome browser files, users can use a mixed mapping file, as genome browser files contain server information in addition to the file paths. The headers for this file are:
Depending on the use case, include all the headers but fill in only the pertinent information
scenario 1) mapping a local file: just specify Path and New_Path
scenario 2) mapping a local folder: just specify path and new path. In this case, the path MUST end in a forward/back slash character
scenario 3) mapping a remote file: in this case, if a server has moved, you need to fill in all 6 columns, with information about the new and old server details.
scenario 4) mapping a remote folder: same as above. do not forget the trailing slash.
2) to update osgbf files
Begin UpdateGBProjectFile /Namespace=NgsLib; Files "\path\to\old\files\or\folder\sample.osgbf"; Options /OutputFile="\path\to\output\sample.osgbf" /MappingFile="path\to\mapping\file\mapping_files_mixed_osgbf.txt"; End;