We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Received a test flake in CI: https://github.com/grafana/mimir/actions/runs/10856690387/job/30131835759#step:8:136
--- FAIL: TestDistributor_Push_ShouldSupportWriteBothToIngestersAndPartitions (0.00s) --- FAIL: TestDistributor_Push_ShouldSupportWriteBothToIngestersAndPartitions/should_shard_series_across_all_partitions_when_shuffle_sharding_is_disabled (3.00s) distributor_ingest_storage_test.go:455: Error Trace: /__w/mimir/mimir/pkg/distributor/distributor_ingest_storage_test.go:455 Error: Not equal: expected: map[int32][]string{0:[]string{"series_four", "series_one", "series_three"}, 1:[]string{"series_two"}, 2:[]string{"series_five"}} actual : map[int32][]string{2:[]string{"series_five"}} Diff: --- Expected +++ Actual @@ -1,10 +1,2 @@ -(map[int32][]string) (len=3) { - (int32) 0: ([]string) (len=3) { - (string) (len=11) "series_four", - (string) (len=10) "series_one", - (string) (len=12) "series_three" - }, - (int32) 1: ([]string) (len=1) { - (string) (len=10) "series_two" - }, +(map[int32][]string) (len=1) { (int32) 2: ([]string) (len=1) { Test: TestDistributor_Push_ShouldSupportWriteBothToIngestersAndPartitions/should_shard_series_across_all_partitions_when_shuffle_sharding_is_disabled
Github actions CI.
The text was updated successfully, but these errors were encountered:
i couldn't reproduce after about 600 runs.
After reducing the time to poll from kafka from 1s to 10ms I could get a similar failure: although there it's missing all 3 partitions, not just 2/3.
diff --git a/pkg/distributor/distributor_ingest_storage_test.go b/pkg/distributor/distributor_ingest_storage_test.go index b598ac6a8f..0ac85ac77a 100644 --- a/pkg/distributor/distributor_ingest_storage_test.go +++ b/pkg/distributor/distributor_ingest_storage_test.go @@ -451,7 +451,7 @@ func TestDistributor_Push_ShouldSupportWriteBothToIngestersAndPartitions(t *test require.NoError(t, err) // Ensure series has been correctly sharded to partitions. - actualSeriesByPartition := readAllMetricNamesByPartitionFromKafka(t, kafkaCluster.ListenAddrs(), testConfig.ingestStoragePartitions, time.Second) + actualSeriesByPartition := readAllMetricNamesByPartitionFromKafka(t, kafkaCluster.ListenAddrs(), testConfig.ingestStoragePartitions, time.Second/100) assert.Equal(t, testData.expectedMetricsByPartition, actualSeriesByPartition) // Ensure series have been correctly sharded to ingesters.
distributor_ingest_storage_test.go:455: Error Trace: /Users/dimitar/grafana/mimir/pkg/distributor/distributor_ingest_storage_test.go:455 Error: Not equal: expected: map[int32][]string{0:[]string{"series_four", "series_one", "series_three"}, 1:[]string{"series_two"}, 2:[]string{"series_five"}} actual : map[int32][]string{} Diff: --- Expected +++ Actual @@ -1,13 +1,2 @@ -(map[int32][]string) (len=3) { - (int32) 0: ([]string) (len=3) { - (string) (len=11) "series_four", - (string) (len=10) "series_one", - (string) (len=12) "series_three" - }, - (int32) 1: ([]string) (len=1) { - (string) (len=10) "series_two" - }, - (int32) 2: ([]string) (len=1) { - (string) (len=11) "series_five" - } +(map[int32][]string) { }
I opened a PR to log the end offsets of the topic on a failure: #9323 so we can verify if the timeout is just too tight
Sorry, something went wrong.
No branches or pull requests
Describe the bug
Received a test flake in CI: https://github.com/grafana/mimir/actions/runs/10856690387/job/30131835759#step:8:136
Environment
Github actions CI.
The text was updated successfully, but these errors were encountered: