Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RunAllTests] Fix #3752: Introduce Bazel test batching in CI #3757

Merged
merged 13 commits into from
Sep 8, 2021

Commits on Sep 1, 2021

  1. Initial commit.

    This is needed to open a PR on GitHub. This commit is being made so that
    the PR can start off in a broken Actions state.
    
    This also initially disables most non-Bazel workflows to make workflow
    iteration faster and less impacting on other team members.
    BenHenning committed Sep 1, 2021
    Configuration menu
    Copy the full SHA
    366ab4d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c38b95c View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2021

  1. Introduce infrastructure for batching.

    This introduces a new mechanism for passing lists of tests to sharded
    test targets in CI, and hooks it up. No actual sharding is occurring
    yet. This led to some simplifications in the CI workflow since the
    script can be more dynamic in computing the full list of targets (which
    also works around a previous bug with instrumentation tests being run).
    Java proto lite also needed to be upgraded for the scripts to be able to
    use it.
    
    More testing/documentation needed as this functionality continues to
    expand.
    BenHenning committed Sep 2, 2021
    Configuration menu
    Copy the full SHA
    ac73dd6 View commit details
    Browse the repository at this point in the history
  2. Add bucketing strategy.

    This simply partitions bucketed groups of targets into chunks of 10 for
    each run. Only 3 buckets are currently retained to test sharding in CI
    before introducing full support.
    BenHenning committed Sep 2, 2021
    Configuration menu
    Copy the full SHA
    3aec233 View commit details
    Browse the repository at this point in the history
  3. Fix caching & stabilize builds.

    Fixes some caching bucket and output bugs. Also, introduces while loop &
    keep_going to introduce resilience against app test build failures (or
    just test failures in general).
    BenHenning committed Sep 2, 2021
    Configuration menu
    Copy the full SHA
    650570b View commit details
    Browse the repository at this point in the history
  4. Increase sharding & add randomization.

    Also, enable other workflows.
    
    Note that CI shouldn't fully pass yet since some documentation and
    testing needs to be added yet, but this is meant to be a more realistic
    test of the CI environment before the PR is finished.
    BenHenning committed Sep 2, 2021
    Configuration menu
    Copy the full SHA
    d66bb2c View commit details
    Browse the repository at this point in the history
  5. Improving partitionin & readability.

    Adds a human-readable prefix to make the shards look a bit nicer.
    
    Also, adds more fine-tuned partitioning to bucket & reduce shard counts
    to improve overall timing. Will need to be tested in CI.
    BenHenning committed Sep 2, 2021
    Configuration menu
    Copy the full SHA
    3969a6d View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    e7b2a0d View commit details
    Browse the repository at this point in the history
  7. Fix script.

    A newly computed variable wasn't updated to be used in an earlier
    change.
    BenHenning committed Sep 2, 2021
    Configuration menu
    Copy the full SHA
    d1da067 View commit details
    Browse the repository at this point in the history
  8. Fix broken tests & test configuration.

    Add docstrings for proto.
    BenHenning committed Sep 2, 2021
    Configuration menu
    Copy the full SHA
    fbb3838 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    6c91da9 View commit details
    Browse the repository at this point in the history
  10. Try 10 max parallel actions instead.

    See
    #3757 (comment)
    for context.
    BenHenning committed Sep 2, 2021
    Configuration menu
    Copy the full SHA
    e3eb6f2 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    62576a1 View commit details
    Browse the repository at this point in the history