-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for Traefik 2's IngressRoute, IngressRouteTCP and IngressRouteUDP #3055
Conversation
Welcome @ThomasK33! |
@seanmalloy @stevehipwell @szuecs @Raffo, could someone of you please review this? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ThomasK33 the Helm chart changes look good to me, but I think you probably want to separate them into a different PR as the release cycle of the Helm chart differs from the binary.
Sorry but I'm not qualified to review the rest of the code.
@ThomasK33 @stevehipwell any updates here? Would love this feature. |
/assign @szuecs |
/assign njuettner |
@njuettner can you click the "run tests" button please? @ThomasK33 as all other new providers get the same answer: |
Hey @szuecs, As this is not a new provider but a new source, the aforementioned #3063 PR is not applicable here. Could you, @njuettner, @Raffo, or @seanmalloy, please review this PR? |
/assign @szuecs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a bunch of comments. I mostly skimmed the code and it looks legit.
docs/tutorials/traefik-proxy.md
Outdated
|
||
## Manifest (for clusters with RBAC enabled) | ||
|
||
Could be change if you have mulitple sources |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure what this sentence mean and it contains a typo. Can you please correct it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is a fair point. I referenced the gloo proxy tutorial and carried it over.
Would you like me to fix/remove it from them as well?
Bumping for visibility. Should we expect a merge of this PR in the near future? Asking since our current implementation is using |
@heliochronix I made it work by adding these annotation to my ingressroute
so it looks like this
I also had to build from master to be able to make it work |
Fantastic, thank you for the clarification! I see you updated the documentation as well with the example. I think the key thing I was missing was the |
@heliochronix That would be a very nice improvement indeed, i spent quite some time around this. |
@ThomasK33 the So while this works if you have a record already setup for traefik.example.com it doesn't help create the traefik.example.com domain to begin with.
Currently my traefik is deployed creating an AWS NLB which can be used with Ingress and Traefik using the following helm values:
technically the
|
Edit: Opps, i overlooked the fact i didnt request Once i did all worked as expected |
y do we need the target annotation at all? where does ingress get the external lb hostname from? and y do we create cnames instead of a records? |
Ingress gets the external lb hostname from the Ingress Controller Loadbalancer status, so you shouldn't be required to hardcode a target like this. |
i dont see a status added to |
Yeah, that's what I thought too - where does the status for ingress resources come from? |
It’s a flag that needs to be enabled.
…On Wed, Feb 7, 2024 at 3:16 AM Timo Behrmann ***@***.***> wrote:
i dont see a status added to IngressRoute kind's which is why target is
required
Yeah, that's what I thought too - where does the status for ingress
resources come from?
—
Reply to this email directly, view it on GitHub
<#3055 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFPZYKPUMMUGUQ4F3QNH4DYSMZ67AVCNFSM6AAAAAAQYD6WSSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZRGUYDAMRQHE>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
are you talking about this? if so i do have it enabled if not what is this flag?
|
That is the flag for Ingress to see the status. Ingress Route however
doesn’t see this which is why I had to create Ingress for even the Traefik
dashboard. I rely on that field for external-dns to update.
…On Wed, Feb 7, 2024 at 9:01 PM Jack ***@***.***> wrote:
@cdenneen <https://github.com/cdenneen>
are you talking about this? if so i do have it enabled if not what is this
flag?
providers:
kubernetesIngress:
publishedService:
enabled: true
—
Reply to this email directly, view it on GitHub
<#3055 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFPZYMK5UKTPPQC5OTC6ZDYSQWV5AVCNFSM6AAAAAAQYD6WSSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZTGI2DQMBZGI>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
@z0mt3c what we do is we have 1 ingress that creates the alb, external dns sets domain i.e lb.MYDOMAN then each app's ingressroute use the target: lb.MYDOMAIN so then external dns just goes okay lets copy what lb.MYDOMAIN points at. |
@phyzical yeah had the same setup... but it sets cname- instead of a-records, doesn't it? |
Yes it's cnames |
Is giving the kubernetes.io/ingress.class annotation needed? First of all I would expect IngressRoute to use Traefik as default anyway since it is part of Traefik's CRDs. Furthermore, the default ingress class in my cluster is in fact |
Can someone explain what actually got merged here and how to use it? I'm trying to get a record into route53 with my grpc endpoint but clearly I'm not understanding something. apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
annotations:
# This one has no effect
# external-dns.alpha.kubernetes.io/hostname: thanos-remote-grpc.example.com
# This one crashes external dns pod * cname loop error
# external-dns.alpha.kubernetes.io/target: thanos-remote-grpc.example.com
kubernetes.io/ingress.class: traefik
name: thanos-receive-remote-grpc
spec:
entryPoints:
- websecure
routes:
- kind: Rule
match: Host(`thanos-remote-grpc.example.com`)
services:
- name: monitoring-thanos-receive
namespace: monitoring
passHostHeader: true
port: 19291
scheme: h2c
tls:
secretName: letsencrypt
|
from what i understood ( i made it work like this ) thinnk it's a limitation atm, ingressRoute make only CNAME made by A records from the IP traefik svc |
@DrummyFloyd Can you explain the two domains being used? I do not understand why all these examples are using two different domains completely unrelated to one another. I am trying to expose my service at |
The /target is the destination for the Host match thanos-remote-grpc.example.com your setting up in the IngressRoute which is why the CNAME loop(you can't have it CNAME to itself). If you know the endpoint /target works. |
I found a workaround which works alright. Create an ingress along side the IngressRoute but make sure the grpc service has higher priority than the http ingress. I think it can also be used in case there are multiple endpoints on the same service where some are http/https and others grpc. # Using this to get external dns to create a dns record (ingressroute not supported)
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: websecure
traefik.ingress.kubernetes.io/router.priority: "10"
traefik.ingress.kubernetes.io/router.tls: "true"
cert-manager.io/cluster-issuer: letsencrypt
name: thanos-query-grpc-dummy
spec:
rules:
- host: thanos-query-grpc.example.com
http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service:
name: monitoring-thanos-query-grpc
port:
# could probably specify different port here as well?
number: 10901
tls:
- secretName: letsencrypt
hosts:
- example.com
- "*.example.com"
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: thanos-query-grpc
spec:
entryPoints:
- websecure
routes:
- match: Host(`thanos-query-grpc.example.com`) && Headers(`Content-Type`, `application/grpc`)
kind: Rule
priority: 11
services:
- name: monitoring-thanos-query-grpc
namespace: monitoring
port: 10901
scheme: h2c
passHostHeader: true
tls:
secretName: letsencrypt |
You're using the Ingress to create the DNS entry and the IngressRoute is just jumping on board. The underlying issue is that Ingress resources will get the controller SVC LB hostname from its status and External-DNS uses that to create the target for the dns entry. IngressRoute however doesn't seem to get the LB hostname from the controller SVC and that's why you have to use a /target. Others have done this by creating the Ingress with a DNS set to lb.domain.com and then set /target on IngressRoute to point Host match of app.domain.com to lb.domain.com. Either way you're using utilizing the fact that Ingress actually gets the hostname back from controller svc. Why can't IngressRoute do the same and avoid all this? |
I opted for this approach because having to know the loadbalancer name by manually picking it out of AWS defeats the point of using external dns for automation. |
Hello is this officialy supported? It works correctly with traditional Ingress Resources |
It works, but it's a bit finnicky. The important steps are:
apiVersion: v1
kind: Service
metadata:
annotations:
external-dns.alpha.kubernetes.io/hostname: traefik.example.com
labels:
app.kubernetes.io/name: traefik
name: traefik
spec:
type: LoadBalancer
ports:
- name: web
port: 80
protocol: TCP
targetPort: web
- name: websecure
port: 443
protocol: TCP
targetPort: websecure
selector:
app.kubernetes.io/name: traefik
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: test
annotations:
external-dns.alpha.kubernetes.io/hostname: "test.example.com"
external-dns.alpha.kubernetes.io/target: "traefik.example.com"
spec:
entryPoints:
- web
- websecure
routes:
- match: Host(`test.example.com`)
kind: Rule
services:
- name: my-service
port: http Tested and working on k8s v1.28.6, Traefik v3.0.2 (Helm chart v28.3.0), external-dns v0.14.2.
|
Thanks @itspngu , that worked. In my case though, I had to also add |
@itspngu You're a hero, and this process needs to be documented. I've been bashing my head against this issue before I found these steps. The external-dns.alpha.kubernetes.io/target: annotation that makes this work is barely documented, and doesn't explain what it does and why it's needed. |
@nralbers This project is a community one, with very few people on their spare time. Feel free to open a PR on this subject if you have a good idea on how to improve it. |
Description
Added support for Traefik 2's custom resource definition.
The traefik-proxy source extracts hostnames from annotations and Traefik's host rule syntax.
Fixes #2286
Fixes traefik/traefik#4655
Checklist