Service exposition with listener classes

Spark History services

The operator deploys a Listener for each Spark History Server pod. The default is to only being accessible from within the Kubernetes cluster, but this can be changed by setting .spec.nodes.roleConfig.listenerClass:

apiVersion: spark.stackable.tech/v1alpha1
kind: SparkHistoryServer
metadata:
  name: spark-history
spec:
  nodes:
    roleConfig:
      listenerClass: external-unstable  (1)
1 Specify one of external-stable, external-unstable, cluster-internal (the default setting is cluster-internal).

For the example above, the listener operator creates a service named spark-history-node where spark-history is the name of the SparkHistoryServer and node is the service role (the only service role available for history servers).

Spark Connect services

Connect pods can be exposed using listener classes in exactly tha same fashion as History Servers (with the exception for the role):

apiVersion: spark.stackable.tech/v1alpha1
kind: SparkConnectServer
metadata:
  name: spark-connect
spec:
  servers:
    roleConfig:
      listenerClass: external-unstable  (1)
1 Specify one of external-stable, external-unstable, cluster-internal (the default setting is cluster-internal).

For the example above, the listener operator creates a service named spark-connect-server where spark-connect is the name of the SparkConnectServer and server is the service role.