Calculation Engine

The Calculation Engine (CE) is the application that integrates the MUSES modules by providing an API for running scientific calculation workflows involving one or more modules. This document explains how to run the CE locally in Docker Compose and develop its code.

To run the CE in Kubernetes, take a look at the Helm chart driving our production deployment.

Installation

Follow carefully the Docker Compose installation instructions for your operating system.

Clone the git repository from the desired release (replace main with a numbered release tag like 1.2.3 to checkout a specific version):

# Specify the target released version
RELEASE_TAG=1.2.3
# Clone the git repo
git clone -b ${RELEASE_TAG} https://gitlab.com/nsf-muses/calculation-engine.git
# Enter the cloned repo directory for subsequent commands
cd calculation-engine

Configuration

Environment

Customize the configuration as desired by setting environment variable values in a file called .env.dev, following the example in the default values .env.default.

Module registration and manifests

The CE configuration is specified by config/config.yaml. Modules are “registered” for integration into the CE by adding an entry in the config/config.yaml#/modules section of the YAML document.

Build the CE image

$ docker build . -f docker/Dockerfile -t registry.gitlab.com/nsf-muses/calculation-engine:dev

Start the application

The first time you run the CE, you will need to initialize a few items:

# DEV_MODE will configure the api-server and celery-worker replicas to restart
# automatically upon changes to Python source files for rapid code iteration.
echo "DEV_MODE = true" > docker/.env.dev
# When running multiple replicas of the api-server, there is a semaphore file
# ".initializing_db" that is created to prevent a race condition during database
# provisioning. In development, you should only run one replica. The 
# FORCE_INITIALIZATION env var prevents an errant initialization from leaving a 
# stale semaphore file that blocks subsequent initializations.
echo "FORCE_INITIALIZATION = true" >> docker/.env.dev

Customize other parameter values from docker/.env.default as desired by redefining them in docker/.env.dev.

Launch the application with

bash docker/launch.sh dev up

By default you should be able to access the service by opening http://127.0.0.1:4000/ in your browser. Watch logs for the api-server with

bash docker/launch.sh dev logs

You will need to manually run the docker compose logs command if you want to watch other services, such as the Celery workers. Use the command echoed by the launch.sh dev logs script as a template, appending celery-worker for example.

To stop and destroy all containers, use

bash docker/launch.sh dev down

If you need to start with a clean slate (e.g. an empty database), delete the persistent Docker volumes using:

bash docker/launch.sh dev destroy-volumes

You may need to delete the app/calculation_engine/migrations folder when starting fresh since no database migrations are necessary in that case.

Develop CE code

When running locally, you can use the default admin user for testing since the OIDC authentication will not work out-of-the-box. The Python example below shows how to obtain an object for interacting with the CE API as well as an example showing how to list the jobs owned by that user:

from calculation_engine_api import CalculationEngineApi
if __name__ == "__main__":
    api = CalculationEngineApi()
    api.job_list()

Celery tasks can be monitored at http://localhost:4000/flower/tasks. The object storage bucket with files generated by the job can be viewed at http://localhost:9001/browser/calculation-engine.

Users that authenticate via OIDC will not have passwords associated with their Django user accounts. In that case, the user must obtain an API token by visiting http://localhost:4000/, logging in, and then visiting http://localhost:4000/api/v0/token.

Unit tests

The suite of tests can be executing using the command below. Optionally, use the destroy-volumes option to ensure a consistent, empty initial state.

bash docker/launch.sh test destroy-volumes
bash docker/launch.sh test up

This runs the bare minimum services (api-server and database) to execute the run/run_tests.sh script via the run/run_api_server.sh entrypoint script, where the dedicated environment file docker/.env.test sets the TEST_MODE env var.

Staging deployment via Docker Compose

We have a dev/staging deployment running in Docker Compose on a virtual machine hosted on Jetstream2 for rapid collaborative code iteration and testing. It also acts as a staging area for proposed updates before applying the the production instance running in Kubernetes.

The Compose command is different for this staging instance deployment:

bash docker/launch.sh prod up