Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI convergence (flaky tests support, add internet tests) #2424

Closed
wants to merge 6 commits into from
Closed

CI convergence (flaky tests support, add internet tests) #2424

wants to merge 6 commits into from

Commits on Aug 25, 2015

  1. test: runner support for flaky tests 

    Adding --flaky-tests option, to allow regarding flaky tests failures
as non-fatal.

Currently only observed by the TapProgressIndicator, which will
add a # TODO directive to tests classified as flaky. According to the
TAP specification, the test harness is supposed to treat failures
that have a # TODO directive as non-fatal.

Ported from nodejs/node-v0.x-archive@df3a2b2
    @orangemocha @joaocgreis
    orangemocha authored and joaocgreis committed Aug 25, 2015
    Configuration menu
    Copy the full SHA
    bc4a5b9 View commit details
    Browse the repository at this point in the history

  2. test: runner should return 0 on flaky tests 

    Make the test runner return a 0 exit code when only
flaky tests fail and --flaky-tests=dontcare is specified.

Ported from nodejs/node-v0.x-archive@a9b642c
    @orangemocha @joaocgreis
    orangemocha authored and joaocgreis committed Aug 25, 2015
    Configuration menu
    Copy the full SHA
    50319ed View commit details
    Browse the repository at this point in the history

  3. test: add test configuration templates 

    Template configuration files for marking tests as flaky.
    @orangemocha @joaocgreis
    orangemocha authored and joaocgreis committed Aug 25, 2015
    Configuration menu
    Copy the full SHA
    b448b83 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2015

  1. test: support flaky tests in test-ci 

    Adding support for specifying flaky test mode to
the test runner:
- via an environment variable FLAKY_TESTS for Makefile
- via an argument ignore-flaky for vcbuild.bat

Ported from nodejs/node-v0.x-archive@2d2494c
    @orangemocha
    orangemocha committed Aug 26, 2015
    Configuration menu
    Copy the full SHA
    fd05d46 View commit details
    Browse the repository at this point in the history

  2. test: pass args to test-ci via env variable

    @orangemocha
    orangemocha committed Aug 26, 2015
    Configuration menu
    Copy the full SHA
    8567b2a View commit details
    Browse the repository at this point in the history

  3. test: initial list of flaky tests 

    This poplulates the lists of flaky tests with tests that failed recently
in Jenkins.
    @orangemocha
    orangemocha committed Aug 26, 2015
    Configuration menu
    Copy the full SHA
    8e60b28 View commit details
    Browse the repository at this point in the history