[elasticsearch] Set secureSettings only for a specific nodeSet

[ebuildy]

Proposal

Use case. Why is this important?

Architecture is a 3 node-sets clusters: master / data / client nodes. I want enable azureAD auth only on client nodes (because only client nodes are accessible). Auth OIDC requires HTTPS and specific secureSettings.

ECK operator take Elasticsearch CRD config to deploy a StatefulSet by nodeSet, who share secureSettings.

Feature

We must be able to add secureSettings by nodeSet.

Workaround

We can add an init container to add some secrets (inside Elasticsearch node spec):

initContainers:
   - name: post-elastic-internal-init-keystore
      image: docker.elastic.co/elasticsearch/elasticsearch:8.8.1
      command:
        - /usr/bin/env
        - bash
        - '-c'
      args:
      - |
          # add all existing secret entries into it
          for filename in /mnt/elastic-internal/secure-settings/*; do
            [[ -e "$filename" ]] || continue # glob does not match
            key=$(basename "$filename")
            echo "Adding "$key" to the keystore."
            /usr/share/elasticsearch/bin/elasticsearch-keystore add-file "$key" "$filename"
          done
      resources:
        limits:
          cpu: 500m
          memory: 196Mi
        requests:
          cpu: 500m
          memory: 196Mi
      volumeMounts:
      - name: auth-azuread-secure-settings
        mountPath: /mnt/elastic-internal/secure-settings
        readOnly: true
      - name: elastic-internal-elasticsearch-config-local
        mountPath: /usr/share/elasticsearch/config
      - name: elastic-internal-elasticsearch-config
        readOnly: true
        mountPath: /mnt/elastic-internal/elasticsearch-config
volumes:
   - name: auth-azuread-secure-settings
      secret:
        secretName: auth-azuread-secure-settings
        defaultMode: 420
        optional: false

(btw: I am not comfortable with ECK operator reading my secrets ...)

[pebrc]

Any security related configuration should alwasy be applied to all nodes in the cluster. If you want to restric external access to the coordinating-only nodes you should do this via other means (e.g. ingress)

Please use our discuss forum at https://discuss.elastic.co/c/eck for any further questions around this.