Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sporaidc csharp/Ice/adapterDeactivation test failure #2061

Closed
externl opened this issue Apr 19, 2024 · 2 comments
Closed

Sporaidc csharp/Ice/adapterDeactivation test failure #2061

externl opened this issue Apr 19, 2024 · 2 comments

Comments

@externl
Copy link
Member

externl commented Apr 19, 2024

This test started failing after I updated the ci workflow to build and run all of the languages and tests in one step.
The test fails sporadically but always at the same place and with the same configuration

ws,compress,ipv6,serialize,mx
testing object adapter with router...

The exception causing the error seems to be:

Caused by: System.Net.Sockets.SocketException: Connection refused
Unhandled exception. System.ArgumentException: The supplied System.Net.SocketAddress is an invalid size for the System.Net.IPEndPoint end point. (Parameter 'socketAddress')
*** [292/358] Running csharp/Ice/adapterDeactivation tests ***
[ running client/server test - 04/19/24 14:49:53 ]
- Config: ssl
(dotnet /Users/runner/work/ice/ice/csharp/test/Ice/adapterDeactivation/msbuild/server/net8.0/server.dll --Ice.Default.Host=127.0.0.1 --Test.BasePort=14000 --Ice.Warn.Connections=1 --Ice.Default.Protocol=ssl --Ice.IPv6=0 --Ice.Plugin.IceSSL= --IceSSL.*** --IceSSL.DefaultDir=/Users/runner/work/ice/ice/certs --IceSSL.CAs=cacert.pem --IceSSL.VerifyPeer=2 --IceSSL.CertFile=server.p12 --Ice.ThreadPool.Server.Size=1 --Ice.ThreadPool.Server.SizeMax=3 --Ice.ThreadPool.Server.SizeWarn=0 --Ice.PrintAdapterReady=1)
(dotnet /Users/runner/work/ice/ice/csharp/test/Ice/adapterDeactivation/msbuild/client/net8.0/client.dll --Ice.Default.Host=127.0.0.1 --Test.BasePort=14000 --Ice.Warn.Connections=1 --Ice.Default.Protocol=ssl --Ice.IPv6=0 --Ice.Plugin.IceSSL= --IceSSL.*** --IceSSL.DefaultDir=/Users/runner/work/ice/ice/certs --IceSSL.CAs=cacert.pem --IceSSL.VerifyPeer=2 --IceSSL.CertFile=client.p12)
testing stringToProxy... ok
testing checked cast... ok
creating/destroying/recreating object adapter... ok
creating/activating/deactivating object adapter in one operation... ok
testing connection closure... ok
testing object adapter published endpoints... ok
testing object adapter with bi-dir connection... ok
testing object adapter with router... ok
testing object adapter creation with port in use... ok
deactivating object adapter in the server... ok
testing whether server is gone... ok
testing server idle time...ok
[ running client/server test - 04/19/24 14:49:57 ]
- Config: ws,compress,ipv6,serialize,mx
(dotnet /Users/runner/work/ice/ice/csharp/test/Ice/adapterDeactivation/msbuild/server/net8.0/server.dll --Ice.Default.Host=::1 --Test.BasePort=14000 --Ice.Warn.Connections=1 --Ice.Default.Protocol=ws --Ice.Override.Compress=1 --Ice.ThreadPool.Server.Serialize=1 --Ice.IPv6=1 --Ice.PreferIPv6Address=1 --Ice.Admin.Endpoints="tcp -h \"::1\"" --Ice.Admin.InstanceName=Server --IceMX.Metrics.Debug.GroupBy=id --IceMX.Metrics.Parent.GroupBy=parent --IceMX.Metrics.All.GroupBy=none --Ice.ThreadPool.Server.Size=1 --Ice.ThreadPool.Server.SizeMax=3 --Ice.ThreadPool.Server.SizeWarn=0 --Ice.PrintAdapterReady=1)
(dotnet /Users/runner/work/ice/ice/csharp/test/Ice/adapterDeactivation/msbuild/client/net8.0/client.dll --Ice.Default.Host=::1 --Test.BasePort=14000 --Ice.Warn.Connections=1 --Ice.Default.Protocol=ws --Ice.Override.Compress=1 --Ice.ThreadPool.Server.Serialize=1 --Ice.IPv6=1 --Ice.PreferIPv6Address=1 --Ice.Admin.Endpoints="tcp -h \"::1\"" --Ice.Admin.InstanceName=Client --IceMX.Metrics.Debug.GroupBy=id --IceMX.Metrics.Parent.GroupBy=parent --IceMX.Metrics.All.GroupBy=none)
testing stringToProxy... ok
testing checked cast... ok
creating/destroying/recreating object adapter... ok
creating/activating/deactivating object adapter in one operation... ok
testing connection closure... ok
testing object adapter published endpoints... ok
testing object adapter with bi-dir connection... ok
testing object adapter with router... -! 4/19/2024 14:49:58:171 client: warning: connection exception:
   Ice.ConnectionLostException
       error = 0
      at IceInternal.StreamSocket.finishRead(Buffer buf) in /Users/runner/work/ice/ice/csharp/src/Ice/StreamSocket.cs:line 205
      at IceInternal.TcpTransceiver.finishRead(Buffer buf) in /Users/runner/work/ice/ice/csharp/src/Ice/TcpTransceiver.cs:line 61
      at IceInternal.WSTransceiver.finishRead(Buffer buf) in /Users/runner/work/ice/ice/csharp/src/Ice/WSTransceiver.cs:line 558
      at Ice.ConnectionI.finishAsync(Int32 operation) in /Users/runner/work/ice/ice/csharp/src/Ice/ConnectionI.cs:line 987
   local address = ::1:52005
   remote address = ::1:14000
Ice.ConnectionRefusedException
    error = 0
   at Ice.RouterPrxHelper.getClientProxy(Optional`1& hasRoutingTable, OptionalContext context) in /Users/runner/work/ice/ice/csharp/src/Ice/generated/Router.cs:line 282
   at IceInternal.RouterInfo.getClientEndpoints() in /Users/runner/work/ice/ice/csharp/src/Ice/RouterInfo.cs:line 73
   at IceInternal.OutgoingConnectionFactory.setRouterInfo(RouterInfo routerInfo) in /Users/runner/work/ice/ice/csharp/src/Ice/ConnectionFactory.cs:line 178
   at Ice.ObjectAdapterI..ctor(Instance instance, Communicator communicator, ObjectAdapterFactory objectAdapterFactory, String name, RouterPrx router, Boolean noConfig, SslServerAuthenticationOptions serverAuthenticationOptions) in /Users/runner/work/ice/ice/csharp/src/Ice/ObjectAdapterI.cs:line 955
   at IceInternal.ObjectAdapterFactory.createObjectAdapter(String name, RouterPrx router, SslServerAuthenticationOptions serverAuthenticationOptions) in /Users/runner/work/ice/ice/csharp/src/Ice/ObjectAdapterFactory.cs:line 172
   at Ice.CommunicatorI.createObjectAdapterWithRouter(String name, RouterPrx router) in /Users/runner/work/ice/ice/csharp/src/Ice/CommunicatorI.cs:line 108
   at Ice.adapterDeactivation.AllTests.allTests(TestHelper helper) in /Users/runner/work/ice/ice/csharp/test/Ice/adapterDeactivation/AllTests.cs:line 124
   at Ice.adapterDeactivation.Client.runAsync(String[] args) in /Users/runner/work/ice/ice/csharp/test/Ice/adapterDeactivation/Client.cs:line 14
   at Test.TestDriver.runTestAsync[T](String[] args) in /Users/runner/work/ice/ice/csharp/test/TestCommon/TestHelper.cs:line 232
Caused by: System.Net.Sockets.SocketException: Connection refused
Unhandled exception. System.ArgumentException: The supplied System.Net.SocketAddress is an invalid size for the System.Net.IPEndPoint end point. (Parameter 'socketAddress')
   at System.Net.IPEndPoint.Create(SocketAddress socketAddress)
   at System.Net.Sockets.SocketAsyncEventArgs.FinishOperationAccept(SocketAddress remoteSocketAddress)
   at System.Net.Sockets.SocketAsyncEventArgs.FinishOperationSyncSuccess(Int32 bytesTransferred, SocketFlags flags)
   at System.Net.Sockets.SocketAsyncEventArgs.FinishOperationAsyncSuccess(Int32 bytesTransferred, SocketFlags flags)
   at System.Net.Sockets.SocketAsyncEventArgs.AcceptCompletionCallback(IntPtr acceptedFileDescriptor, Memory`1 socketAddress, SocketError socketError)
   at System.Net.Sockets.SocketAsyncEngine.System.Threading.IThreadPoolWorkItem.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()
@externl
Copy link
Member Author

externl commented Apr 24, 2024

I can reproduce this locally, but it seems to only happen when my system is under load.

To encourage the crash, I've been using stress:

brew install stress
stress --cpu 10

Also maybe relevant: dotnet/runtime#40913

@pepone
Copy link
Member

pepone commented Sep 5, 2024

The underlying cause is likely this .NET bug dotnet/runtime#102663, see also #2677

The unhandled exception issue was fixed in #2704

@pepone pepone closed this as completed Sep 5, 2024
@pepone pepone added the bug label Sep 5, 2024
externl added a commit to externl/ice that referenced this issue Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants