ArrayServer folder mapping and management
From Array Suite Wiki
Genomics data can be very large, and users want to avoid moving/copying/duplicating large data files unless necessary. ArrayServer provides folder mapping to map mounted folders into the ArrayServer file system. This allows folders/files outside of the ArrayServer BaseDirectory hierarchy to be accessible within ArrayServer.
ArrayServer admins can add folder mapping lines in the ArrayServer.cfg [Folder] section like the one below:
Linux-based Server folder
[Folder] IData_TestDataSets=/mnt/IData/TestDataSets/ IData_Users=/mnt/IData/Users IData_OmicTest=/mnt/IData/OmicTest IData_ArrayLand=/mnt/IData/ArrayLand InternalData=/InternalData
Windows-based Server folder
[Folder] IData_TestDataSets=F:\IData\TestDataSets IData_Users=F:\IData\Users IData_OmicTest=F:\IData\OmicTest IData_ArrayLand=F:\IData\ArrayLand InternalData=G:\InternalData
cloud S3 folder (more details, including how to map S3 buckets from multiple AWS accounts, can be found here.
[CloudFolder] GaryCloudFolder=/east.test.omicsoft/gary GaryCloudFolder2=/omicsofttest1
After restarting the server, files and subfolders inside these mapped folder will show up in the ArrayServer root directory:
It is better to map the highest level of directory to avoid too many folder mappings. Taking the example above, it is better to map as below:
[Folder] IData=/mnt/IData/ InternalData=/InternalData
All ArrayServer users are accessing these folders through the user account running ArrayServer, such as "sys_omicsoft". The account "sys_omicsoft" should have read and write access to these folders in the server machine. Folder owners or admin can assign read/write access in Server Tab | Manage | Manage users:
Each mapped folder will be a user, such as "_idata_omictest_" user for folder mapping "IData_OmicTest=/IData/OmicTest". Admin can assign a group of users or a single user to its folder reader and editor user groups. By default, all standard (or above) users can read files and all editor (or above) users can write files to these mapped folders.
Manage folder mapping
Since ArrayServer 9.0, admin can manage/add folder mapping on the fly without server restart.
Control User Access to Mapped Subfolders
If a mapped folder contains subfolders that have restricted access, ArrayServer admins can control which users or groups can access these data.
- Create a user group to define those who should have access to restricted subfolders.
- Add Array Studio users to this user group.
- Create a pseudo-user with naming pattern _mappedFolder/path/to/RestrictedFolder_.
- Add the appropriate user group as a Folder Reader group for the pseudo-user's "folder".
For example, consider a mapped folder "workspace", with three subfolders:
/workspace/ ├── Clinicaldata ├── CNSdata └── Commondata
Only researchers in the CNS group should access CNSdata, while only researchers in the Clinical group should access Clinicaldata; both groups should have access to Commondata.
Step 1: Create user groups to define users with access to folders
Under Manage | Manage User Groups, define groups for users that can have access to CNSdata or Clinicaldata:
There is no specific naming scheme required, but consider making the purpose of the user group clear. The new user groups should be listed in Manage User Groups:
Step 2: Add users to the newly-created user groups
In the Manage User Groups window, select one of the new user groups (e.g. workspace-clinicaldata_access), then click Set Members. Choose users that should belong to this group (e.g. user name clinicalresearcher). Repeat for members of workspace-CNSdata_access.
Step 3: Create a "pseudo-user" that specifies the subfolders with restricted access
In Manage | Manage Users, click Add User, and create a user in the pattern _mappedFolder/path/to/RestrictedFolder_. This pseudo-user will represent the path to the subfolder that should be restricted.
In this example, two new users are created: _workspace/clinicaldata_ and _workspace/cnsdata_.
Step 4: Add user groups to the "pseudo-user's" permitted folder readers
In Manage | Manage Users, select one of the "pseudo-users" that represents a restricted folder (e.g. _workspace/clinicaldata_.
In the right window, click the Privilege | [Folder Reader] User groups tab.
Select the User groups that should be able to see this folder (e.g. workspace-clinicaldata_access). Do the same for _workspace/cnsdata_.
Be sure to click Update User to save your changes!
The user "clinicalresearcher" will see the following folders:
while the user "cnsresearcher" will see the following folders:
How to hide a folder from everybody
A folder "pseudo-user" requires at least one assigned "folder reader group"; i.e, if a subfolder with a "pseudo-user" name does not have any "folder reader groups" assigned, all users will be able to see the subfolder.
To make it invisible to all users, create a new User Group (like step 1, above), but do not add any users.
Create the "pseudo-user" representing the path to the folder that should be hidden.
Then assign the empty User Group to the "[FolderReaders] User Group" for the sub-folder.
Array Server administrators will still see the folder,
but no other users.
ArrayServer file paths use a virtual folder directory system, similar to FTP servers. If you map a folder, such as (ArrayServer folder IData=> physical folder /mnt/IData), it will create an empty folder, IData, in the physical ArrayServerBaseDirectory/FtpRoot folder. When a user browses files in ArrayStudio/ArrayServer, it will display the FtpRoot folder as the root folder, and translate the link IData to the actual folder /mnt/IData.
However, if you map a folder such that your ArrayServer BaseDirectory itself is a subfolder of your mapped folder, it will cause a loop in the FTP.
For example, if the BaseDirectory is
/mnt/IData/omicsoft/ArrayServerHome, and /mnt/IData is mapped as a virtual subdirectory, an improper FTP loop will be setup leading to various problems. One tell-tale sign is that clicking the FtpRoot button or navigating to the Root folder displays the full Server path (as opposed to '/'):
If other folders in IData need to be accessed in ArrayServer, the admin should find a better way to add these folders in /mnt/IData, such as
IData_data1=/mnt/IData/data1 IData_data2=/mnt/IData/data2 IData_data3=/mnt/IData/data3