ODL Video Service

This is a video hosting platform, designed for MIT's Office of Digital Learning (ODL). Authentication is handled through a Keycloak realm that federates to MIT's Touchstone identity provider.

Installation

You will need to obtain several different pieces of information in order to get this project up and running. Secret and non-secret settings will be stored in environment variables. In order to make it easier to get started, you can copy .env.example to .env.

Django

Create a secret key for Django, and store it in the .env file as SECRET_KEY. You can run this code to create a key:

head -c 50 /dev/urandom | base64

AWS

You'll need an AWS access key ID and secret access key. Store them in the file .env, like this:

AWS_ACCESS_KEY_ID=foo
AWS_SECRET_ACCESS_KEY=bar

You'll also need a CloudFront private key for generating signed URLs for CloudFront. Store the private key file in .env in one single string (careful with the newlines), like this:

CLOUDFRONT_PRIVATE_KEY==----BEGIN RSA PRIVATE KEY-----\nMIICXAIBAAKBgQCQMjkVo9gogtb8DI2bZyFGvnnN81Q4d0crS4S9UDrxHJU/yrKg\n...

Set the key ID as the CLOUDFRONT_KEY_ID environment variable, using the .env file.

You'll also need to set three S3 bucket for storing video files, and a CloudFront distribution that is hooked up to that S3 bucket. The files in the S3 bucket should not be publicly accessible, and the CloudFront distribution should be set up to serve private content. (See the CloudFront documentation for more information.) Set the S3 upload bucket name as the VIDEO_S3_BUCKET environment variable, the transcode bucket name as the VIDEO_S3_TRANSCODE_BUCKET environment variable, the thumbnail bucket name as the VIDEO_S3_THUMBNAIL_BUCKET environment variable, the subtitle bucket name as the VIDEO_S3_SUBTITLE_BUCKET environment variable, and set the CloudFront distribution ID as the VIDEO_CLOUDFRONT_DIST environment variable, using the .env file.

The Buckets should each have a CORS configuration that will allow for cross-origin requests, for example:

You also must have a proper Elastic Transcoder pipeline configured to use the specified 3 bucket names.

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>video.odl.mit.edu</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedMethod>DELETE</AllowedMethod>
    <AllowedMethod>HEAD</AllowedMethod>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

Each of the Cloudfront origins should be configured as follows:

• Restrict Bucket Access
• Origin Access Identity: Use an Existing Identity
• Your Identities: select an existing CloudFront user (create if necessary)

You also need to create cloudfront behaviors for each bucket:

• Allowed HTTP methods: GET, HEAD, OPTIONS
• Whitelist Headers: Access-Control-Request-Headers,`Access-Control-Request-Method`, Origin
• Restrict Viewer Access: No

• VIDEO_S3_TRANSCODE_BUCKET bucket:

  • Precedence: 0
  • Path pattern: transcoded/*

• VIDEO_S3_SUBTITLE_BUCKET bucket:

  • Precedence: 1
  • Path pattern: subtitles/*

• VIDEO_S3_THUMBNAIL_BUCKET

  • Precedence 2:
  • Path pattern: thumbnails/*

• VIDEO_S3_BUCKET

  • Precedence 3:
  • Path pattern: Default(*)

You can also optionally create a public CloudFront distribution for serving static files for the web application. If you want to do this, set the CloudFront distribution ID as the STATIC_CLOUDFRONT_DIST environment variable, using the .env file.

This app expects the transcoding to use HLS or MP4, and the ET_HLS_PRESET_IDS and ET_MP4_PRESET_ID``environment variables, respectively. ``ET_HLS_PRESET_IDS should be a comma-delimited list of Video HLS presets for AWS ElasticTranscode. The defaults are standard presets (2M, 1M, 600K).

Dropbox

Create an app on Dropbox, and store the app key in the file .env, like this:

DROPBOX_KEY=foo

Keycloak (local authentication)

Local development uses Keycloak as the OIDC provider instead of Touchstone. A keycloak service runs in docker-compose with a pre-configured ovs-local realm, seeded users, and an odl-video-app client. See README-keycloak.md for the full setup walkthrough — adding the kc.odl.local entry to /etc/hosts, pulling the realm public key, and populating the SOCIAL_AUTH_KEYCLOAK_* and KEYCLOAK_* values in your .env.

YouTube Integration

• Create a new project at https://console.cloud.google.com/apis/dashboard - Save the project ID in your .env file as YT_PROJECT_ID
• Create an OAuth client ID for the project (type: Other) - Save your client ID and client secret in your .env file (as YT_CLIENT_ID and YT_CLIENT_SECRET)
• Enable the YouTube Data API v3 for your project
• Run the following Django command to generate values for YT_ACCESS_TOKEN and YT_REFRESH_TOKEN:

docker-compose run web python manage.py oauthtokens

• Click on the provided link, follow the prompts, and enter the verification code back in the shell.
• Save the YT_ACCESS_TOKEN and YT_REFRESH_TOKEN values to your .env file

Thumbnail Upload Limits

Two environment variables control the maximum size of uploaded thumbnail images. Both must be kept in sync so the limits are consistent across all layers:

THUMBNAIL_UPLOAD_MAX_SIZE (default: 1048576 — 1 MB)

Maximum allowed thumbnail file size in bytes. Enforced by Django before the image is processed. The value is also passed to the frontend as SETTINGS.thumbnail_upload_max_size for client-side validation.

NGINX_CLIENT_MAX_BODY_SIZE (default: 1m)

nginx client_max_body_size directive. Requests exceeding this limit are rejected by nginx with a 413 before reaching Django. Accepts standard nginx size strings (e.g. 1m, 5m, 500k).

Both variables default to 1 MB. To raise the limit, set both together in your .env file (or k8s/Heroku config vars):

THUMBNAIL_UPLOAD_MAX_SIZE=5242880   # 5 MB in bytes
NGINX_CLIENT_MAX_BODY_SIZE=5m       # 5 MB nginx string

Running

To run the application, install Docker and Docker Compose, then run:

docker-compose up

Tests

To run the tests, install the development dependencies and then run the test suite, like this:

./scripts/test/test_suite.sh

Commits

To ensure commits to github are safe, you should install the following first: .. code-block:: bash

pip install pre_commit detect-secrets pre-commit install

To automatically install precommit hooks when cloning a repo, you can run this: .. code-block:: bash

git config --global init.templateDir ~/.git-template pre-commit init-templatedir ~/.git-template