GEP-1713: ListenerSets - Standard Mechanism to Merge Gateway Listeners (rev 2) #3213

dprotaso · 2024-07-23T23:33:04Z

This replaces #1863
What type of PR is this?

/kind gep

What this PR does / why we need it:

Outlines a mechanism to merge Gateway Listeners

Which issue(s) this PR fixes:

Does this PR introduce a user-facing change?:

NONE

youngnick

Quite a few comments here, thanks for keeping working on pushing this forward.

geps/gep-1713/index.md

robscott

Thanks for all the work on this @dprotaso!

geps/gep-1713/index.md

…nd is not allowed

geps/gep-1713/index.md

…status

youngnick

This LGTM for Provisional (or maybe even Implementable, since we'll need to implement it and try it out) now.

robscott · 2024-08-02T01:01:46Z

Status update: We're at the end of the 2-day extension for this GEP to make it into this release, so unfortunately I think this is going to have to switch to targeting v1.3. Fortunately, this new shorter release cycle means that's actually not that far away.

Ultimately, I think you're right. For this to make sense, we'd need some structural changes to Gateway, particularly making Listeners optional. That's a huge change to a GA API (even if it would initially be limited to experimental channel, breaking changes to something labeled as "v1"can be confusing). Combined with the huge scope of the PR, and most of the feedback just coming from Nick and I, I think this particular GEP would benefit from some more soak time for additional reviewers + to smooth out any areas we're uncertain. Unfortunately, I think it's just too big to try to squeeze in today.

With that said, I would like to see this GEP make it in early in the next release cycle (similar to how named Route rules just barely missed the previous cycle but was merged at the beginning of this one).

I think it would be useful to do the following between now and then:

Continue to iterate on this GEP, discussing in meetings as necessary (this can include merging this as provisional with some unresolved sections and iterating in smaller PRs)
Build out a clearer vision for how this can be expanded to cover more use cases in the future (particularly the ones we discussed in the hierarchy doc) - that will help make a stronger case for why this is important
Present these ideas at the SIG-Network review session for v1.2 so we can get early feedback on this overall idea and adjust the proposal accordingly
Warn all controller authors that it's possible that Listeners may become optional in a future release of Gateway API and they should make sure they can handle that case

Thanks again for all the work you've been putting in to push this forward @dprotaso! I really do appreciate it.

kate-osborn · 2024-09-03T23:03:30Z

geps/gep-1713/index.md

+type ListenerSetSpec struct {
+	// ParentRefs references the Gateway that the listeners are attached to.
+	//
+	// +kubebuilder:validation:MaxItems=2


Just curious, why limit ParentRefs to 2?

My guess is to allow "moving" a set of listeners from one Gateway to another without downtime, but limiting the ability to attach the same set of listeners to many different Gateways - if we find sufficient demand for that pattern loosening this constraint in the future could be possible, but it feels like a pattern that likely should be avoided.

+1 to Mike's answer. @dprotaso do you mind capturing this rationale somewhere in the GEP?

Circling back on this I want to check the assumption - we sorta added multiple parents for 'future' proofing without a use case. See: #3213 (comment)

@mikemorris mentioned the migration as one use case but I don't particularly see that being any simpler than creating a copy of the ListenerSet.

I'd be willing to hear other use cases but so far I'm not convinced a list is the right thing here - especially if we're limiting it to 2 parents. It makes the status more complicated when we know the 80% of uses will be a single parent Gateway.

Again if we didn't have multiple parents a user could simply create two ListenerSets.

Again if we didn't have multiple parents a user could simply create two ListenerSets.

We could use the same argument for routes, but routes support multiple ParentRefs. Was that a mistake?

If we're coming from the perspective of ListenerSets being generated by a controller, it does seem unlikely that multiple parent refs would be useful. If on the other hand, we're thinking about users manually configuring ListenerSets, multiple ParentRefs seem like they'd be just as useful here as they are in Routes.

dprotaso · 2024-12-09T16:18:31Z

/retest

robscott

Thanks for all the work on this @dprotaso! Took a fresh pass and I think this largely makes sense.

geps/gep-1713/index.md

robscott · 2024-12-12T01:33:35Z

geps/gep-1713/index.md

+type ListenerSetSpec struct {
+	// ParentRefs references the Gateway that the listeners are attached to.
+	//
+	// +kubebuilder:validation:MaxItems=2


+1 to Mike's answer. @dprotaso do you mind capturing this rationale somewhere in the GEP?

geps/gep-1713/index.md

robscott · 2024-12-12T01:44:54Z

geps/gep-1713/index.md

+> 1. Should we have made it easier to leave port empty or do port ranges?
+> 2. Should we support multiple hostnames?
+> 3. Are there any validations that we wish we'd tightened up?


Thanks for capturing this!

+1 to past me on all points 🙃. I'd argue that we should probably make attempts at resolving both 1 and 2 here.

For multiple hostnames, I can see the utility, but it risks making the rules about Listener -> Route hostname matching even more complicated.

Optional port would be nice - then let the implementation choose it. Though it's a rare use case for me.

My votes on the above:

Yes

No

Not that I can think of

geps/gep-1713/index.md

robscott · 2024-12-12T02:14:15Z

geps/gep-1713/index.md

+    protocol: HTTP
+    port: 80
+---
+apiVersion: gateway.networking.k8s.io/v1alpha1


Would like to get some resolution on #3497 before we release this API, as I'd prefer to switch this group to gateway.networking.x-k8s.io

robscott · 2024-12-12T02:17:16Z

geps/gep-1713/index.md

+}
+// ListenerEntry embodies the concept of a logical endpoint where a Gateway accepts
+// network connections.
+type ListenerEntry struct {


There's been some renewed interest in pushing L4 + ClusterIP Gateways forward, and one thing that could help with that is if Listeners could point directly to backends. I'd personally like to explore that in this GEP as a follow up to the similar idea in the doc. (No need to take action here until we get some more comments/feedback on this idea).

I think that including L4 considerations at the very first stage of the design, before having the actual API in place is too much of a stretch. Getting rid of L4 routes in favor of including L4 routing information (i.e., backends) is a controversial one (for which I am on the fence honestly) and I don't think that should block this GEP which has been in the work for a long time and needs to be moved forward.

Putting here L4 will likely require a lot of discussion, partially off-topic with the broader goal of this GEP.

geps/gep-1713/index.md

youngnick · 2025-01-14T04:15:13Z

Reading back over this, I think the main things outstanding are:

Will we start this off with the separate experimental API group (as per New Experimental APIs should start in x-k8s.io API Group #3497)?
Will we make port optional in the ListenerSet version of Listener?
Will we keep the parentRef as a list or move it back to a singular one?

robscott · 2025-01-14T07:09:56Z

Will we start this off with the separate experimental API group (as per New Experimental APIs should start in x-k8s.io API Group #3497)?

I'm strongly in favor of this, and the idea seems to be gaining some traction on the corresponding issue.

Will we make port optional in the ListenerSet version of Listener?

I would like to ensure that this has a well documented path forward to supporting all ports on a single Listener. That could either be a port range or an empty port. We don't have to start with either of those, I'd just like to have a documented plan for how we could get there.

Will we keep the parentRef as a list or move it back to a singular one?

I'm on the fence on this one, left a comment above.

mikemorris · 2025-01-14T14:57:03Z

I would like to ensure that this has a well documented path forward to supporting all ports on a single Listener.

Could make sense to align with NetPol conventions on this (omitting an optional port field implies all), refs kubernetes-sigs/network-policy-api#247 (comment)

dprotaso · 2025-01-14T15:08:47Z

I was thinking an empty port could be allow the implementation to pick a random port. Though I realize now we could also accomplish that by relaxing our constraints on PortNumber and allowing 0 (unix semantics). If we do that then I don't particularly need an optional port - so I'm curious about other people's use cases.

I think use of port in netpol is fundamentally different than that of a listener - eg. firewall rules semantics (no port == all) vs listener semantics (0 port = wildcard)

mlavacca · 2025-01-23T11:24:08Z

geps/gep-1713/index.md

+
+## API
+
+This proposal introduces a new `ListenerSet` resource that has the ability to attach to a set of listeners to multiple parent `Gateways`.


Suggested change

This proposal introduces a new `ListenerSet` resource that has the ability to attach to a set of listeners to multiple parent `Gateways`.

This proposal introduces a new `ListenerSet` resource that has the ability to attach a set of listeners to multiple parent `Gateways`.

mlavacca · 2025-01-23T11:28:27Z

geps/gep-1713/index.md

+	// +kubebuilder:default={from: None}
+	Namespaces *ListenerNamespaces `json:"namespaces,omitempty"`


Why defaulting to None? This way we'd need users to explicitly opt-in for the feature even for the same namespace as the Gateway. Wouldn't it make sense to default to Same?

mlavacca · 2025-01-23T11:43:52Z

geps/gep-1713/index.md

+}
+// ListenerEntry embodies the concept of a logical endpoint where a Gateway accepts
+// network connections.
+type ListenerEntry struct {


I think that including L4 considerations at the very first stage of the design, before having the actual API in place is too much of a stretch. Getting rid of L4 routes in favor of including L4 routing information (i.e., backends) is a controversial one (for which I am on the fence honestly) and I don't think that should block this GEP which has been in the work for a long time and needs to be moved forward.

Putting here L4 will likely require a lot of discussion, partially off-topic with the broader goal of this GEP.

mlavacca · 2025-01-23T11:58:33Z

geps/gep-1713/index.md

+Valid reasons for `Accepted` being `False` are:
+
+- `NotAllowed` - the `parentRef` doesn't allow attachment
+- `ParentNotAccepted` - the `parentRef` isn't accepted (eg. invalid address)


I'm struggling to understand what this reason means: is it about the Accepted condition of the referenced parent? If that's the case, we are setting here a cross-object dependency. What should we use here if the parent doesn't exist at all (problem raised by the ResolvedRefs condition)?

Second Revision of GEP-1713

71a9bab

k8s-ci-robot requested review from LiorLieberman and youngnick July 23, 2024 23:33

dprotaso mentioned this pull request Jul 23, 2024

GEP-1713: Standard Mechanism to Merge Multiple Gateways #1863

Closed

youngnick reviewed Jul 24, 2024

View reviewed changes

robscott reviewed Jul 24, 2024

View reviewed changes

dprotaso added 5 commits July 29, 2024 22:09

fix minor nits

02c6a05

Drop section highlighting failure cases we can't handle

72b6b1e

add enum validation

bb40ff6

Describe the 'meta' details about this feature

9baad11

attaching to the Gateway's sectionName but through the ListenerSet ki…

41b1b21

…nd is not allowed

guicassolato reviewed Jul 30, 2024

View reviewed changes

geps/gep-1713/index.md Outdated Show resolved Hide resolved

dprotaso added 12 commits July 30, 2024 14:00

ListenerSet can have multiple parents

e221a16

Allow multiple parentRefs and adjust language around PolicyAttachment

67ec44f

clarify which section has listener ordering

228a4f8

fix heading levels

4ff2eb0

allow no listeners on the Gateway spec

879c011

add back istio use case

dd175c6

update gateway conditions paragraph

6890bc3

don't say concatenate

460576f

drop clause saying invalid listeners affects other listeners

a46a001

clarify route attachment

70261a1

split listenersetconditions into two sections: top-level and listener…

aa382cc

…status

clarify where the conflicted condition is set

32fc59f

youngnick reviewed Jul 31, 2024

View reviewed changes

dprotaso added 2 commits August 1, 2024 16:54

Gateways that fail should also fail their attached ListenerSets

54de622

don't leak secrets

8b338bc

robscott mentioned this pull request Aug 10, 2024

Use unique Listener hostnames in modify listeners tests #3220

Closed

gcs278 mentioned this pull request Aug 22, 2024

Support hostnames in K8S Gateway kubernetes-sigs/external-dns#4402

Open

kate-osborn reviewed Sep 3, 2024

View reviewed changes

youngnick mentioned this pull request Sep 4, 2024

Make the maximum number of listeners in Gateway CRD configurable #2869

Closed

danielloader mentioned this pull request Sep 5, 2024

Detecting Gateway hostnames based on attached HTTPRoutes cert-manager/cert-manager#6051

Open

shaneutt added the priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. label Sep 18, 2024

shaneutt added this to the v1.3.0 milestone Sep 18, 2024

dprotaso changed the title ~~GEP-1713: Standard Mechanism to Gateway Merge Listeners (rev 2)~~ GEP-1713: Standard Mechanism to Merge Gateway Listeners (rev 2) Nov 12, 2024

dprotaso changed the title ~~GEP-1713: Standard Mechanism to Merge Gateway Listeners (rev 2)~~ GEP-1713: ListenerSets - Standard Mechanism to Merge Gateway Listeners (rev 2) Nov 25, 2024

robscott reviewed Dec 12, 2024

View reviewed changes

youngnick reviewed Dec 12, 2024

View reviewed changes

geps/gep-1713/index.md Show resolved Hide resolved

dprotaso added 9 commits December 19, 2024 22:21

update title

c656a65

mention dummy listener for the first release

67fbed4

tweak the name of the AttachedListenerSets condition

02a50a2

rob's suggestions

4ca38a6

bump listener count

3fdd031

just re-use GatewayTLSConfig

1c33b84

use v1 api version for httproute

59cc00b

replace route with ListenerSet in godoc

cb906b9

include listener precedence in godoc

c015961

dprotaso mentioned this pull request Jan 16, 2025

Traefik as Knative Ingress? traefik/traefik#5081

Open

mlavacca reviewed Jan 23, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GEP-1713: ListenerSets - Standard Mechanism to Merge Gateway Listeners (rev 2) #3213

GEP-1713: ListenerSets - Standard Mechanism to Merge Gateway Listeners (rev 2) #3213

dprotaso commented Jul 23, 2024 •

edited

Loading

youngnick left a comment

robscott left a comment

youngnick left a comment

robscott commented Aug 2, 2024

kate-osborn Sep 3, 2024

mikemorris Sep 4, 2024 •

edited

Loading

robscott Dec 12, 2024

dprotaso Dec 20, 2024

robscott Jan 14, 2025

dprotaso commented Dec 9, 2024

robscott left a comment

robscott Dec 12, 2024

robscott Dec 12, 2024

youngnick Dec 12, 2024

dprotaso Dec 20, 2024

robscott Jan 14, 2025

robscott Dec 12, 2024

robscott Dec 12, 2024

mlavacca Jan 23, 2025

youngnick commented Jan 14, 2025

robscott commented Jan 14, 2025

mikemorris commented Jan 14, 2025

dprotaso commented Jan 14, 2025

mlavacca Jan 23, 2025

mlavacca Jan 23, 2025

mlavacca Jan 23, 2025

mlavacca Jan 23, 2025


		## API

		This proposal introduces a new `ListenerSet` resource that has the ability to attach to a set of listeners to multiple parent `Gateways`.

		// +kubebuilder:default={from: None}
		Namespaces *ListenerNamespaces `json:"namespaces,omitempty"`

GEP-1713: ListenerSets - Standard Mechanism to Merge Gateway Listeners (rev 2) #3213

Are you sure you want to change the base?

GEP-1713: ListenerSets - Standard Mechanism to Merge Gateway Listeners (rev 2) #3213

Conversation

dprotaso commented Jul 23, 2024 • edited Loading

youngnick left a comment

Choose a reason for hiding this comment

robscott left a comment

Choose a reason for hiding this comment

youngnick left a comment

Choose a reason for hiding this comment

robscott commented Aug 2, 2024

Choose a reason for hiding this comment

mikemorris Sep 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dprotaso commented Dec 9, 2024

robscott left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

youngnick commented Jan 14, 2025

robscott commented Jan 14, 2025

mikemorris commented Jan 14, 2025

dprotaso commented Jan 14, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dprotaso commented Jul 23, 2024 •

edited

Loading

mikemorris Sep 4, 2024 •

edited

Loading