Standard Setup

A standard installation of the CloudDat for Wasabi uses the following ExpeDat configuration options.

• The following line in /etc/servedat.cf ties the “S3” action code supplied by clients to the s3handler.sh shell script on the CloudDat VM instance.  See the Object Handlers documentation.

  ObjectHandler S3,/usr/local/expedat/s3handler.sh

• AuthFile /etc/svpasswd
  ExpeDat users may be defined in /etc/svpasswd, which resembles a unix password file.  The s3user created during initial setup is given the ObjectOnly restriction, which means it cannot access the instance filesystem.  See the ExpeDat AuthFile documentation.  If you customize this file, be sure to answer ‘n’ if you re-run an installer and it asks about replacing existing credentials.
• SysAuth 1
  This option in /etc/servedat.cf allows system accounts on the CloudDat instance, such as admin, to access both Wasabi storage and the instance filesystem.  You must assign a password to a system account for it to be useable with ExpeDat.  For example, “sudo passwd admin".  See the ExpeDat SysAuth documentation.

Following are some of the ways you can modify these settings to customize CloudDat.

Updating to a New Version

New versions of CloudDat software are downloaded directly from your DEI Customer Site.  ExpeDat components may be updated by simply running ExpeDat/Server\ Files/install-servedat.sh script.  If you have not made custom modifications to files in /usr/local/expedat, you can update CloudDat components by running clouddat-others/install.sh.  If you have modified the CloudDat components, particularly /usr/local/expedat/s3handler.sh, you may wish to install the new s3handler.sh manually.

General advice for updating DEI software can be found in Tech Note 0012.

Multiple Users

The initial installation creates a single S3 user.  To create additional credentials, edit /etc/svpasswd, using that first entry as a template.  For example, after adding a second user, svpasswd might look like:

s3user:frYypg…0HCxHA:99:99::ObjectOnly usertwo:SddKMR…jSHh7g:99:99::ObjectOnly

The second field of svpasswd can contain a plain text password, but it is safer to generate an SHA hash using the /usr/local/expedat/mkpasswd utility.  After modifying svpasswd, the server will automatically load the new credentials in about 2 minutes.

With multiple user credentials, you can apply different restrictions and configurations to different accounts, as discussed below.

Access Restrictions

The final field of an AuthFile (svpasswd) record contains a list of access restrictions.  Commonly used restrictions are:

• ObjectOnly — prevents access to the filesystem.
• ReadOnly — no uploads or changes allowed.
• WriteListOnly — downloading prohibited.
• WriteOnly — downloading and listing prohibited.
• GetOnly — download only, listing other actions prohibited.
• NoOverwrite — this option is not supported.
• RestrictHome — the user home directory is prepended to all paths.  See Segmented Buckets below.  The user home directory is otherwise ignored.

See the ExpeDat AuthFile documentation for more options.

Segmented Buckets

The default CloudDat user account, shown above, has access to the entire Wasabi bucket.  You can restrict gateway users to just a portion of the bucket by assigning a RestrictHome prefix.  For example,

s3user:frYypg…0HCxHA:99:99:webfolder:ObjectOnly,RestrictHome

This will insert the prefix “webfolder/” in front of all paths accessed by ExpeDat clients using the “s3user” name and will not allow them to access any other objects.  It is like restricting the user to a single folder.  You can create multiple users, each restricted to a different prefix.  Changes to svpasswd will be automatically loaded about two minutes after your last modification.

Multiple Buckets

Step 1 of the Install process asks you to choose a Wasabi bucket to access.  Setting a fixed name allows clients to access Wasabi storage without knowing the bucket name.  If you leave this blank, clients must specify a bucket name as the first part of the remote path.  This allows you to access multiple buckets through a single CloudDat instance.

If you’ve already setup the gateway, the easiest way to modify the bucket name is by editing /usr/local/expedat/s3handler.sh to change the “S3BUCKET=” setting at the top.

When S3BUCKET is blank, the bucket must be specified either by the client or as part of the svpasswd home prefix.  If you follow the Multiple Users and Segmented Buckets instructions above, you can have different users accessing different buckets all from the same CloudDat instance.

joanne:frYypg…0HCxHA:99:99:bucketone:ObjectOnly,RestrictHome robert:SddKMR…jSHh7g:99:99:buckettwo:ObjectOnly,RestrictHome samuel:Yd3751…PWFgIQ:99:99:bucketthree/sam:ObjectOnly,RestrictHome nancy:nCcR61…TXNybU:99:99:bucketthree/nancy:ObjectOnly,RestrictHome

In the example above, joanne and robert each have access to an entire bucket.  Users samuel and nancy each have access to a psuedo-folder within a third bucket.

To have clients choose the bucket, leave off the RestrictHome option.  When RestrictHome is not set, the home directory field is ignored and clients must specify the bucket name as part of their remote path.  With movedat, this just means typing the bucket name in front of the object name.

movedat local.dat s3user@wasabi.example.com=S3:bucketone/remote.dat

In ExpeDat Desktop, the bucketname will need to be entered into the Remote Prefix field.  Users can type this manually, or you can make it part of an expedat:// URL.

Changes to svpasswd will be automatically loaded about two minutes after your last modification.

Sending Links

You can direct users to a specific location in your Wasabi bucket by creating a gateway link.  The easiest way to do this is to navigate to the location using the ExpeDat Desktop client.  If you want to make a link for downloading a specific file, select that file, then click the Bookmark button.

You will now have an expedat:// link copied into your desktop clipboard.  To create a gateway link, just paste the link onto the end of your gateway URL with a question mark in between.  For example:

http://wasabi.example.com/?expedat://wasabi.example.com:8080/Downloadme.dat?u=&h=S3

When a user clicks that link, they will be taken to your gateway page where they receive instructions, an opportunity to download the client software, and a final link to direct the client to download the targeted object.

You can modify the link by pre-filling the username ( u=s3user ) or by prompting the user to upload a file instead of downloading ( &a=s ).  For example:

http://wasabi.example.com/?expedat://wasabi.example.com:8080/folder/?u=s3user&h=S3&a=s

See the expedat:// URLs chapter of the ExpeDat Documentation for more details about creating links.

Custom Clients

The clients provided to you with your CloudDat purchase are pre-configured to show a “S3” object handler as an option in the Handler pop-up menu.  Standard ExpeDat clients, including free trials, are not pre-configured for S3 and must be setup either by the end-user or by embedding a list of handlers.

If you have a standard ExpeDat package, you can embed custom settings into ExpeDat Desktop executables, including a list of Object Handlers.  This list might include the standard “S3” handler as well as any custom handlers you create.  Once you have embedded this list, the Handler pop-up menu will automatically appear when that ExpeDat Desktop executable is run, without the end-user needing to perform any setup.

Filesystem Access

Instance system users, such as admin can access the full filesystems of the CloudDat instance.  In ExpeDat Desktop, select “None” from the Handler pop-up menu.  In movedat, remove the “=S3” or similar suffix.  System user accounts must have a password assigned to be used with ExpeDat.  For example, “sudo passwd admin".  See the ExpeDat SysAuth documentation for additional information about how system accounts interact with ExpeDat.

Users declared in the AuthFile (/etc/svpasswd) are blocked from accessing the instance filesystem by the ObjectOnly option.  If you remove that option from a user's record, then that user will also be able to access the full filesystem with the privileges of the UID and GID specified.  This is not recommended without additional restrictions such as RestrictHome and AllowPath to control which paths are visible to the user.

If you wish to prevent any users, including system users, from accessing the instance filesystem, add "ObjectOnly" to /etc/servedat.cf.  After modifying servedat.cf, you must signal the server to restart with "sudo killall -HUP servedat".

Custom Handling

This topic requires proficiency with Bash shell scripts.

You can create multiple custom versions of s3handler.sh and access them using different handler names.  Simply make a copy of the /usr/local/expedat/s3handler.sh shell script and modify it as needed.  For example, you might have different S3BUCKET targets or target different SSE options.

After creating a new handler, you must declare it in /etc/servedat.cf.  The following example shows two handlers:

ObjectHandler S3,/usr/local/expedat/s3handler.sh ObjectHandler S3custom,/usr/local/expedat/s3custom.sh

To access the custom handler with movedat, specify its action code on the command line:

movedat local.dat s3user@wasabi.example.com=S3custom:bucketone/remote.dat

To access the custom handler with ExpeDat Desktop, add its action code in the Opt Setup dialog, then select it in the Handler pop-up menu.  Alternatively, you can follow the instructions above to preconfigure ExpeDat Desktop clients with your custom handlers.

After modifying servedat.cf, you must signal the server to restart with “sudo killall -HUP servedat".

More

Each CloudDat VM instance includes a fully functional ExpeDat server.  See the ExpeDat Documentation for more information about customizing it.