Ingress and TLS Certificates

Connecting external HTTP requests to the services running in Kubernetes in a high-availability (HA) manner requires several components, including a load balancer and an ingress controller.

Traefik ingress controller

The ingress controller is Traefik v2. It is configured as a LoadBalancer service, bound to the static internal IP address controlled by MetalLB that is associated with the external floating IP address provisioned by Radiant (i.e. OpenStack) and assigned to our DNS records.

MetalLB load balancer

The top half of the schematic below illustrates how to think about it from the other direction. Inbound requests to example.com resolve to the external IP address 141.142.x.y due to the DNS records we request of the NCSA networking staff. The external IP address is associated in OpenStack with the MetalLB-controlled internal network IP address 192.168.a.b.

In the event the worker node running Traefik goes down, Kubernetes will respawn Traefik on another worker node. MetalLB, which itself runs as a DaemonSet on all worker nodes, will notice this event and will reassign the internal IP address 192.168.a.b to this new worker node running Traefik. This is illustrated by the bottom half of the figure.

../_images/k8s_ingress.png

SSL Certificate Management

Currently we configure Traefik to automatically obtain TLS certificates via LetsEncrypt.

Note: Although Traefik provides automatic TLS certificate services, it cannot do this in HA mode (meaning no more than a single replica of Traefik can be online). To support HA, we should use Certificate Manager to obtain and automatically renew TLS certificates from LetsEncrypt for our HTTPS web service ingresses.

The certificates themselves are stored in /data/acme.json accessible from within the Traefik container like so:

$ kubectl exec -it -n traefik traefik-8476fcb9db-t42gk -- cat /data/acme.json
{
  "le": {
    "Account": {
      "Email": "devnull@ncsa.illinois.edu",
      "Registration": {
        "body": {
          "status": "valid",
          "contact": [
            "mailto:devnull@ncsa.illinois.edu"
          ]
        },
        "uri": "https://acme-v02.api.letsencrypt.org/acme/acct/1...60"
      },
      "PrivateKey": "MI...9R",
      "KeyType": "4096"
    },
    "Certificates": [
      {
        "domain": {
          "main": "musesframework.io"
        },
        "certificate": "LS0...Cg==",
        ...
      },
      ...
    ]
  }
}