AWS S3 Distribution Configuration

This page documents the configuration for Opencast module distribution-service-aws-s3. This configuration is only required on the presentation node, and only if you are using Amazon S3 and/or Cloudfront for distributing media to end users.

Amazon User Configuration

Configuration of Amazon users is beyond the scope of this documentation, instead we suggest referring to Amazon's documentation. You will, however, require to set up proper credentials by either:

AmazonS3FullAccess permission is required, which can be granted using these instructions.

A free Amazon account will work for small scale testing, but be aware that S3 distribution can cost you a lot of money very quickly. Be aware of how much data and how many requests you are making, and be sure to set alarms to notify you of cost overruns.

Amazon Service Configuration

The development and testing it is generally safe to allow the Opencast AWS S3 Distribution service to create the S3 bucket for you. It will create the bucket per its configuration, with public read-only access to the files, and no versioning. For production use we suggest using Amazon CloudFront, which requires additional configuration.

Amazon CloudFront

Amazon CloudFront provides an optional way to better handle distributing your media to end users. While fully configuring CloudFront is outside the scope of this documentation, we wish to note that this does affect one of the keys described below. Please ensure you use the correct distribution base format depending on which service you are using!

Opencast Service Configuration

The Opencast AWS S3 Distribution service has five configuration keys, which can be found in the org.opencastproject.distribution.aws.s3.AwsS3DistributionServiceImpl.cfg configuration file.

Key name Value Example
org.opencastproject.distribution.aws.s3.distribution.enable True to enable S3 distribution, false otherwise true
org.opencastproject.distribution.aws.s3.region The AWS region to set us-west-2
org.opencastproject.distribution.aws.s3.bucket The S3 bucket name example-org-dist
org.opencastproject.distribution.aws.s3.distribution.base Where the S3 files are available from. This value can be derived from the bucket and region values, or is set by CloudFront. http://s3-us-west-2.amazonaws.com/example-org-dist, or DOMAIN_NAME.cloudfront.net
org.opencastproject.distribution.aws.s3.access.id Your access ID 20 alphanumeric characters
org.opencastproject.distribution.aws.s3.secret.key Your secret key 40 characters

If org.opencastproject.distribution.aws.s3.access.id and org.opencastproject.distribution.aws.s3.secret.key are not explicitly provided, search for credentials will be performed in the order specified by the Default Credentials Provider Chain.

Using S3 Distribution

Amazon S3 distribution is already included in the default Opencast workflows, however it must first be enabled. The schedule-and-upload.xml and publish.xml workflow configuration files both contain lines containing the string "Remove this line if you wish to publish to AWS S3". Both of these lines must be removed before publishing to AWS S3 will function correctly.

If you wish to use AWS S3 publishing with your own custom workflow, you must add the publish-aws workflow operation to your workflow. The operation documentation can be found here.

Publishing to multiple distribution services

Currently we do not support publication to multiple distribution services simultaneously. This means that whichever workflow operation is last in the workflow will be the final publication.

Using this handler in custom workflows

If your workflow contains both publish-engage and publish-aws, in that order, and without a conditional you would have publication files stored both locally and in AWS. This is likely not what you want, so protect your workflow operations appropriately. If you really do need these files stored in both places (for example, in cases where you need to make the files available immediately, and only push to AWS in some cases) then remember to add a retract-engage in between the publication operations. Note that if this step is omitted the files will remain available locally, but will not be used. Of further note, if you retract after publication to AWS then your workflow will not be available to users. To summarize, this table presents a subset of the various situations that are possible

Workflow Operations Files present in the Media Module Files present in AWS Files served from
publish-engage Yes No Opencast Media Module
publish-aws No Yes AWS
publish-engage, publish-aws Yes Yes AWS
publish-aws, publish-engage Yes Yes Opencast Media Module
publish-engage, retract-engage, publish-aws Temporary Yes AWS
publish-engage, publish-aws, retract-engage No Yes Not available

Migrating to S3 Distribution with Pre-Existing Data

If you already have data published to your local Opencast install, you should be able to republish the media selecting AWS S3 as the distribution service to use.