You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
My Setup:
1 Node k3s deployment on IPv4 ONLY lab network
Other workloads up and working as expected
KubeMQ Community latest release (as of 2/25/22) deployed using default config
What happens:
When the community edition is deployed it starts, however all listeners only bind to the containers IPv6 address. This makes it impossible to connect to the container with the CLI from an IPv4 network.
What's expected:
Listeners should bind to both IPv4 and IPv6 interfaces.
Digging in the code, the raw TCP socket listeners appear to be correctly setting the bind port according to the net.Listen documentation. As well the echo servers appear to be setup correctly as well.
Ref: grpc/server.go:111 lis, err := net.Listen("tcp", fmt.Sprintf(":%d", port))
Ref: rest/server.go:172 _ = s.echoWebServer.Start(fmt.Sprintf(":%d", conf.Port))
However as seen from the startup log output, it appears to ignore this.
2022-02-26T02:07:21.322Z INFO starting kubemq community edition version: , cores: 16 {"host": "f1a8683371dc", "module": "server"}
2022-02-26T02:07:21.323Z INFO ⇨ http server started on [::]:8080
{"host": "f1a8683371dc", "module": "server-api"}
2022-02-26T02:07:22.324Z INFO Starting broker-server version 2.1.9 {"host": "f1a8683371dc", "module": "broker"}
2022-02-26T02:07:22.324Z INFO Git commit [not set] {"host": "f1a8683371dc", "module": "broker"}
2022-02-26T02:07:22.324Z INFO Listening for client connections on 0.0.0.0:39983 {"host": "f1a8683371dc", "module": "broker"}
2022-02-26T02:07:22.324Z INFO Server id is NAT3D4HT53N6MRBB4T35ZVRXEPEC3CSRAWKTVZ5JXGCI3TWT2772BX2E {"host": "f1a8683371dc", "module": "broker"}
2022-02-26T02:07:22.324Z INFO Server is ready {"host": "f1a8683371dc", "module": "broker"}
2022-02-26T02:07:22.325Z INFO In-Memory client server connection pipe: ya3PsP8PSdGJarJJX9lIIC started {"host": "f1a8683371dc", "module": "broker"}
2022-02-26T02:07:22.350Z INFO STREAM: Starting broker-streaming-server[kubemq] version 0.19.0 {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.350Z INFO STREAM: ServerID: 08rN2ijHYGp37AYGf7AYqI {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.350Z INFO STREAM: Go version: go1.17.7 {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.350Z INFO STREAM: Git commit: [not set] {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.352Z INFO STREAM: Recovering the state... {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.352Z INFO STREAM: No recovered state {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.603Z INFO STREAM: Message store is FILE {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.603Z INFO STREAM: Store location: store/f1a8683371dc {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.603Z INFO STREAM: ---------- Store Limits ---------- {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.603Z INFO STREAM: Channels: unlimited {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.603Z INFO STREAM: --------- Channels Limits -------- {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.603Z INFO STREAM: Subscriptions: unlimited {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.603Z INFO STREAM: Messages : unlimited {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.603Z INFO STREAM: Bytes : unlimited {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.603Z INFO STREAM: Age : 24h0m0s {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.603Z INFO STREAM: Inactivity : 24h0m0s {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.603Z INFO STREAM: ---------------------------------- {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:22.603Z INFO STREAM: Streaming Server is ready {"host": "f1a8683371dc", "module": "broker-store-server"}
2022-02-26T02:07:23.603Z INFO routing service loaded {"host": "f1a8683371dc", "module": "services"}
2022-02-26T02:07:23.606Z WARN started insecure grpc server at port 50000 {"host": "f1a8683371dc", "module": "grpc"}
2022-02-26T02:07:23.606Z WARN grpc interface is accepting traffic {"host": "f1a8683371dc", "module": "grpc"}
2022-02-26T02:07:23.606Z WARN rest interface is accepting traffic {"host": "f1a8683371dc", "module": "rest-interface"}
2022-02-26T02:07:23.606Z INFO started insecure rest server at port 9090 {"host": "f1a8683371dc", "module": "rest-interface"}
2022-02-26T02:07:23.606Z INFO ⇨ http server started on [::]:9090
{"host": "f1a8683371dc", "module": "rest-api"}
2022-02-26T02:07:23.606Z INFO STREAM: Channel "_QUEUE_DELAY_" has been created {"host": "f1a8683371dc", "module": "broker-store-server"}
This is confirmed by remoting into the container and listing the bound ipv4 and ipv6 ports
Open IPv4 ports
I have no name!@f1a8683371dc:/kubemq$ declare -a array=($(tail /proc/net/tcp | cut -d":" -f"3"|cut -d" " -f"1")) && for port in ${array[@]}; do echo $((0x$port)); done | sort | uniq
57560
Open IPv6 ports
I have no name!@f1a8683371dc:/kubemq$ declare -a array=($(tail /proc/net/tcp6 | cut -d":" -f"3"|cut -d" " -f"1")) && for port in ${array[@]}; do echo $((0x$port)); done | sort | uniq
36742
39983
42349
50000
8080
9090
Things I've tried:
Rebuild container from source
Rebuild with different base image (bullseye-slim)
Hardcoded address/ports (ex 0.0.0.0:50000 for grpc)
Hardcoded ListenerNetwork for echo servers to tcp
The only thing that has successfully allowed the project to bind on ipv4 was to set the first param of Listen and the ListenerNetwork property of the echo servers to tcp4, which of course forced IPv4 only.
2022-02-26T03:00:41.577Z INFO ⇨ http server started on 0.0.0.0:9090
I've run out of things to try to allow both ipv4 and ipv6, any ideas?
The text was updated successfully, but these errors were encountered:
I found this post in the golang issues golang/go#9334 which seems to be a long standing issue with the net package of golang. The issue doesn't appear to be going anywhere at the moment.
I found a similar issue in echo servers history labstack/echo#1658. Their solution was to add the aforementioned configuration setting I referenced in the issue. I think that may be the solution to go with.
I have put in a PR #21 to add transport as a config setting for api, grpc, and rest if that is the direction we want to take.
My Setup:
1 Node k3s deployment on IPv4 ONLY lab network
Other workloads up and working as expected
KubeMQ Community latest release (as of 2/25/22) deployed using default config
What happens:
When the community edition is deployed it starts, however all listeners only bind to the containers IPv6 address. This makes it impossible to connect to the container with the CLI from an IPv4 network.
What's expected:
Listeners should bind to both IPv4 and IPv6 interfaces.
Digging in the code, the raw TCP socket listeners appear to be correctly setting the bind port according to the
net.Listen
documentation. As well the echo servers appear to be setup correctly as well.Ref: grpc/server.go:111
lis, err := net.Listen("tcp", fmt.Sprintf(":%d", port))
Ref: rest/server.go:172
_ = s.echoWebServer.Start(fmt.Sprintf(":%d", conf.Port))
However as seen from the startup log output, it appears to ignore this.
This is confirmed by remoting into the container and listing the bound ipv4 and ipv6 ports
Open IPv4 ports
Open IPv6 ports
Things I've tried:
ListenerNetwork
for echo servers totcp
The only thing that has successfully allowed the project to bind on ipv4 was to set the first param of Listen and the
ListenerNetwork
property of the echo servers totcp4
, which of course forced IPv4 only.2022-02-26T03:00:41.577Z INFO ⇨ http server started on 0.0.0.0:9090
I've run out of things to try to allow both ipv4 and ipv6, any ideas?
The text was updated successfully, but these errors were encountered: