# Ingress and TLS Certificates

Connecting external HTTP requests to the services running in Kubernetes in a high-availability (HA) manner requires several components, including a load balancer and an ingress controller.

## [Traefik](https://doc.traefik.io/traefik) ingress controller

The ingress controller is Traefik v2. It is configured as a LoadBalancer service, bound to the static internal IP address controlled by MetalLB that is associated with the external floating IP address provisioned by Radiant (i.e. OpenStack) and assigned to our DNS records.

## [MetalLB](https://metallb.universe.tf/) load balancer

The top half of the schematic below illustrates how to think about it from the other direction. Inbound requests to `example.com` resolve to the external IP address `141.142.x.y` due to the DNS records we request of the NCSA networking staff. The external IP address is associated in OpenStack with the MetalLB-controlled internal network IP address `192.168.a.b`.

In the event the worker node running Traefik goes down, Kubernetes will respawn Traefik on another worker node. MetalLB, which itself runs as a DaemonSet on all worker nodes, will notice this event and will reassign the internal IP address `192.168.a.b` to this new worker node running Traefik. This is illustrated by the bottom half of the figure.

![](k8s_ingress.png)

## SSL Certificate Management

Currently we configure Traefik to automatically obtain TLS certificates via LetsEncrypt.

**Note**: Although Traefik provides automatic TLS certificate services, it cannot do this in HA mode (meaning no more than a single replica of Traefik can be online). To support HA, we *should* use [Certificate Manager](https://cert-manager.io/docs/) to obtain and automatically renew TLS certificates from LetsEncrypt for our HTTPS web service ingresses.

The certificates themselves are stored in `/data/acme.json` accessible from within the Traefik container like so:

```bash
$ kubectl exec -it -n traefik traefik-8476fcb9db-t42gk -- cat /data/acme.json
{
  "le": {
    "Account": {
      "Email": "devnull@ncsa.illinois.edu",
      "Registration": {
        "body": {
          "status": "valid",
          "contact": [
            "mailto:devnull@ncsa.illinois.edu"
          ]
        },
        "uri": "https://acme-v02.api.letsencrypt.org/acme/acct/1...60"
      },
      "PrivateKey": "MI...9R",
      "KeyType": "4096"
    },
    "Certificates": [
      {
        "domain": {
          "main": "musesframework.io"
        },
        "certificate": "LS0...Cg==",
        ...
      },
      ...
    ]
  }
}
```