Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run one build per platform in recipes cq. #5226

Closed
wants to merge 1 commit into from

Conversation

godofredoc
Copy link
Contributor

Flakiness in packages build block recipe changes very frequently and slows them down. Given that the same recipe is used in all the builds we only need to run one build per platform in the cq.

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the relevant style guides and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/packages repo does use dart format.)
  • I signed the CLA.
  • The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
  • I listed at least one issue that this PR fixes in the description above.
  • I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.
  • I updated CHANGELOG.md to add a description of the change, following repository CHANGELOG style.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is test-exempt.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Flakiness in packages build block recipe changes very frequently and
slows them down. Given that the same recipe is used in all the builds we
only need to run one build per platform in the cq.
@stuartmorgan
Copy link
Contributor

Given that the same recipe is used in all the builds we only need to run one build per platform in the cq.

How does that follow? Not every build uses every feature of the recipes. For instance, we need to add secrets support for the maps API keys, and none of the builds chosen here would make use of them. None of these use emulator support either, which IIRC has a recipe component.

Shouldn't we have one of each kind of build, and just remove shards beyond shard 1?

@stuartmorgan
Copy link
Contributor

@godofredoc Ping on the question above.

@stuartmorgan
Copy link
Contributor

@godofredoc Should this still be open?

@godofredoc
Copy link
Contributor Author

Given that the same recipe is used in all the builds we only need to run one build per platform in the cq.

How does that follow? Not every build uses every feature of the recipes. For instance, we need to add secrets support for the maps API keys, and none of the builds chosen here would make use of them. None of these use emulator support either, which IIRC has a recipe component.

Shouldn't we have one of each kind of build, and just remove shards beyond shard 1?

Recipes CQ is intended to validate changes on recipes and/or recipe modules. Just enough builds to test the packages recipe paths is desirable. Also inspecting the packages recipe seems really flat not sure why we need tens of builds to validate it.

The main reason I created this PR is because some of these builds are very slow and fail frequently blocking recipes CLs for several hours. If we would like to continue running all these builds can we at least remove the slow and flaky ones?

@stuartmorgan
Copy link
Contributor

Recipes CQ is intended to validate changes on recipes and/or recipe modules. Just enough builds to test the packages recipe paths is desirable.

Then we need at least one build using everything the recipe does. E.g., emulators, as I mentioned above. Just one per platform isn't sufficient to test all the recipe functionality.

The main reason I created this PR is because some of these builds are very slow and fail frequently blocking recipes CLs for several hours.

I'm fine with trimming the list down, but it needs to do so in a way that doesn't remove coverage of critical functionality.

If we would like to continue running all these builds can we at least remove the slow and flaky ones?

Emulators are only used for integration tests, so there is no way to ensure that emulator support isn't broken without one such test being included.

@stuartmorgan
Copy link
Contributor

@godofredoc How do you want to proceed with this? Do you need me to identify a more minimal set without losing key coverage?

@stuartmorgan
Copy link
Contributor

From triage: I'll update this to a smaller (but less minimal) set.

@stuartmorgan
Copy link
Contributor

Closing in favor of #6673

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants