Update KafkaRestProxies crd with option to specify external and internal listeneres, as Kafka, KsqlDb and Schemaregistry already has

[oscarbm1998]

Description:

We are currently running CFK (Confluent for kubernetes) on our kubernetes cluster and deploying all components such as Kafka, SchemaRegistry, KsqlDb, KafkaRestProxy, Connect and Control-center. Our cluster currently has two sets of certificate issuers, one internal and one external. The internal one is used for when the components send information between them selves and is how we have enabled tls on our system. The external one is used for when the components talk directly with something outside the Kubernetes cluster and uses a verified CA (Certificate Authority) to generate certificates. This is important because trying to use self-signed certificates outside the Kubernetes cluster, requires extra configuration for all users and is therefore something we do not wish to do.

Current solution for some components (Kafka, Schema Registry and KsqlDb):

For components such as Kafka, Schema Registry and KsqlDb we are able to set a configuration such as this:

  listeners:
    internal:
      tls:
        enabled: true
    external:
      tls:
        enabled: true
        secretRef: schemaregistry-external-tls
      externalAccess:
        type: loadBalancer
        loadBalancer:
          domain: test.example.com

This allows for example the schema registry as shown in the example to return different certificates for tls handshakes based on if the client it interacts with is internal to the kubernetes cluster or external.

Requested solution for KafkaRestProxies, Control-center and Connect:

The requested solution is the same as shown in the schema registry example above. However for these 3 components, their CRDS do not have the option of configuring listeners, they only allow for configuring externalAccess load balancer which seems to not work for our solution.

To implement the solution, changes are required to these specified files:

platform.confluent.io_kafkarestproxies.yaml
platform.confluent.io_controlcenters.yaml
platform.confluent.io_connects.yaml

There may also be additional code implementation required for the components if they do not already support the internal/external listeners method.

Environment information:

Kubernetes version: 1.27.9
Confluent operator version: 0.824.33
cp-kafka-rest-version: 7.4.3
cp-enterprise-control-center version: 7.4.3
cp-server-connect version: 7.4.3