Skip to content

Commit

Permalink
Copy most of tests/run-coverage into tests/coverage-map/status-quo
Browse files Browse the repository at this point in the history
The output of these tests is too complicated to comfortably verify by hand, but
we can still use them to observe changes to the underlying mappings produced by
codegen/LLVM.

If these tests fail due to non-coverage changes (e.g. in HIR-to-MIR lowering or
MIR optimizations), it should usually be OK to just `--bless` them, as long as
the `run-coverage` test suite still works.
  • Loading branch information
Zalathar committed Sep 5, 2023
1 parent 004db47 commit 3141177
Show file tree
Hide file tree
Showing 81 changed files with 5,527 additions and 0 deletions.
13 changes: 13 additions & 0 deletions tests/coverage-map/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
The tests in `./status-quo` were copied from `tests/run-coverage` in order to
capture the current behavior of the instrumentor on non-trivial programs.
The actual mappings have not been closely inspected.

## Maintenance note

These tests can be sensitive to small changes in MIR spans or MIR control flow,
especially in HIR-to-MIR lowering or MIR optimizations.

If you haven't touched the coverage code directly, and the `run-coverage` test
suite still works, then it should usually be OK to just `--bless` these
coverage mapping tests as necessary, without worrying too much about the exact
changes.
57 changes: 57 additions & 0 deletions tests/coverage-map/status-quo/abort.cov-map
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
Function name: abort::main
Raw bytes (105): 0x[01, 01, 12, 01, 47, 05, 09, 03, 0d, 42, 11, 03, 0d, 11, 3e, 42, 11, 03, 0d, 3b, 15, 11, 3e, 42, 11, 03, 0d, 15, 36, 3b, 15, 11, 3e, 42, 11, 03, 0d, 05, 09, 0d, 01, 0d, 01, 01, 1b, 03, 02, 0b, 00, 18, 42, 01, 0c, 00, 19, 11, 00, 1a, 02, 0a, 3e, 02, 0a, 00, 0b, 3b, 02, 0c, 00, 19, 15, 00, 1a, 00, 31, 36, 00, 31, 00, 32, 33, 04, 0c, 00, 19, 05, 00, 1a, 00, 31, 09, 00, 31, 00, 32, 47, 01, 09, 00, 17, 0d, 02, 05, 01, 02]
Number of files: 1
- file 0 => global file 1
Number of expressions: 18
- expression 0 operands: lhs = Counter(0), rhs = Expression(17, Add)
- expression 1 operands: lhs = Counter(1), rhs = Counter(2)
- expression 2 operands: lhs = Expression(0, Add), rhs = Counter(3)
- expression 3 operands: lhs = Expression(16, Sub), rhs = Counter(4)
- expression 4 operands: lhs = Expression(0, Add), rhs = Counter(3)
- expression 5 operands: lhs = Counter(4), rhs = Expression(15, Sub)
- expression 6 operands: lhs = Expression(16, Sub), rhs = Counter(4)
- expression 7 operands: lhs = Expression(0, Add), rhs = Counter(3)
- expression 8 operands: lhs = Expression(14, Add), rhs = Counter(5)
- expression 9 operands: lhs = Counter(4), rhs = Expression(15, Sub)
- expression 10 operands: lhs = Expression(16, Sub), rhs = Counter(4)
- expression 11 operands: lhs = Expression(0, Add), rhs = Counter(3)
- expression 12 operands: lhs = Counter(5), rhs = Expression(13, Sub)
- expression 13 operands: lhs = Expression(14, Add), rhs = Counter(5)
- expression 14 operands: lhs = Counter(4), rhs = Expression(15, Sub)
- expression 15 operands: lhs = Expression(16, Sub), rhs = Counter(4)
- expression 16 operands: lhs = Expression(0, Add), rhs = Counter(3)
- expression 17 operands: lhs = Counter(1), rhs = Counter(2)
Number of file 0 mappings: 13
- Code(Counter(0)) at (prev + 13, 1) to (start + 1, 27)
- Code(Expression(0, Add)) at (prev + 2, 11) to (start + 0, 24)
= (c0 + (c1 + c2))
- Code(Expression(16, Sub)) at (prev + 1, 12) to (start + 0, 25)
= ((c0 + (c1 + c2)) - c3)
- Code(Counter(4)) at (prev + 0, 26) to (start + 2, 10)
- Code(Expression(15, Sub)) at (prev + 2, 10) to (start + 0, 11)
= (((c0 + (c1 + c2)) - c3) - c4)
- Code(Expression(14, Add)) at (prev + 2, 12) to (start + 0, 25)
= (c4 + (((c0 + (c1 + c2)) - c3) - c4))
- Code(Counter(5)) at (prev + 0, 26) to (start + 0, 49)
- Code(Expression(13, Sub)) at (prev + 0, 49) to (start + 0, 50)
= ((c4 + (((c0 + (c1 + c2)) - c3) - c4)) - c5)
- Code(Expression(12, Add)) at (prev + 4, 12) to (start + 0, 25)
= (c5 + ((c4 + (((c0 + (c1 + c2)) - c3) - c4)) - c5))
- Code(Counter(1)) at (prev + 0, 26) to (start + 0, 49)
- Code(Counter(2)) at (prev + 0, 49) to (start + 0, 50)
- Code(Expression(17, Add)) at (prev + 1, 9) to (start + 0, 23)
= (c1 + c2)
- Code(Counter(3)) at (prev + 2, 5) to (start + 1, 2)

Function name: abort::might_abort
Raw bytes (21): 0x[01, 01, 01, 01, 05, 03, 01, 04, 01, 01, 14, 05, 02, 09, 01, 24, 02, 02, 0c, 03, 02]
Number of files: 1
- file 0 => global file 1
Number of expressions: 1
- expression 0 operands: lhs = Counter(0), rhs = Counter(1)
Number of file 0 mappings: 3
- Code(Counter(0)) at (prev + 4, 1) to (start + 1, 20)
- Code(Counter(1)) at (prev + 2, 9) to (start + 1, 36)
- Code(Expression(0, Sub)) at (prev + 2, 12) to (start + 3, 2)
= (c0 - c1)

66 changes: 66 additions & 0 deletions tests/coverage-map/status-quo/abort.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
#![feature(c_unwind)]
#![allow(unused_assignments)]

extern "C" fn might_abort(should_abort: bool) {
if should_abort {
println!("aborting...");
panic!("panics and aborts");
} else {
println!("Don't Panic");
}
}

fn main() -> Result<(), u8> {
let mut countdown = 10;
while countdown > 0 {
if countdown < 5 {
might_abort(false);
}
// See discussion (below the `Notes` section) on coverage results for the closing brace.
if countdown < 5 { might_abort(false); } // Counts for different regions on one line.
// For the following example, the closing brace is the last character on the line.
// This shows the character after the closing brace is highlighted, even if that next
// character is a newline.
if countdown < 5 { might_abort(false); }
countdown -= 1;
}
Ok(())
}

// Notes:
// 1. Compare this program and its coverage results to those of the similar tests
// `panic_unwind.rs` and `try_error_result.rs`.
// 2. This test confirms the coverage generated when a program includes `UnwindAction::Terminate`.
// 3. The test does not invoke the abort. By executing to a successful completion, the coverage
// results show where the program did and did not execute.
// 4. If the program actually aborted, the coverage counters would not be saved (which "works as
// intended"). Coverage results would show no executed coverage regions.
// 6. If `should_abort` is `true` and the program aborts, the program exits with a `132` status
// (on Linux at least).

/*
Expect the following coverage results:
```text
16| 11| while countdown > 0 {
17| 10| if countdown < 5 {
18| 4| might_abort(false);
19| 6| }
```
This is actually correct.
The condition `countdown < 5` executed 10 times (10 loop iterations).
It evaluated to `true` 4 times, and executed the `might_abort()` call.
It skipped the body of the `might_abort()` call 6 times. If an `if` does not include an explicit
`else`, the coverage implementation injects a counter, at the character immediately after the `if`s
closing brace, to count the "implicit" `else`. This is the only way to capture the coverage of the
non-true condition.
As another example of why this is important, say the condition was `countdown < 50`, which is always
`true`. In that case, we wouldn't have a test for what happens if `might_abort()` is not called.
The closing brace would have a count of `0`, highlighting the missed coverage.
*/
40 changes: 40 additions & 0 deletions tests/coverage-map/status-quo/assert.cov-map
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
Function name: assert::main
Raw bytes (65): 0x[01, 01, 08, 01, 1b, 05, 1f, 09, 0d, 03, 11, 16, 05, 03, 11, 05, 1f, 09, 0d, 09, 01, 09, 01, 01, 1b, 03, 02, 0b, 00, 18, 16, 01, 0c, 00, 1a, 05, 00, 1b, 02, 0a, 12, 02, 13, 00, 20, 09, 00, 21, 02, 0a, 0d, 02, 0a, 00, 0b, 1b, 01, 09, 00, 17, 11, 02, 05, 01, 02]
Number of files: 1
- file 0 => global file 1
Number of expressions: 8
- expression 0 operands: lhs = Counter(0), rhs = Expression(6, Add)
- expression 1 operands: lhs = Counter(1), rhs = Expression(7, Add)
- expression 2 operands: lhs = Counter(2), rhs = Counter(3)
- expression 3 operands: lhs = Expression(0, Add), rhs = Counter(4)
- expression 4 operands: lhs = Expression(5, Sub), rhs = Counter(1)
- expression 5 operands: lhs = Expression(0, Add), rhs = Counter(4)
- expression 6 operands: lhs = Counter(1), rhs = Expression(7, Add)
- expression 7 operands: lhs = Counter(2), rhs = Counter(3)
Number of file 0 mappings: 9
- Code(Counter(0)) at (prev + 9, 1) to (start + 1, 27)
- Code(Expression(0, Add)) at (prev + 2, 11) to (start + 0, 24)
= (c0 + (c1 + (c2 + c3)))
- Code(Expression(5, Sub)) at (prev + 1, 12) to (start + 0, 26)
= ((c0 + (c1 + (c2 + c3))) - c4)
- Code(Counter(1)) at (prev + 0, 27) to (start + 2, 10)
- Code(Expression(4, Sub)) at (prev + 2, 19) to (start + 0, 32)
= (((c0 + (c1 + (c2 + c3))) - c4) - c1)
- Code(Counter(2)) at (prev + 0, 33) to (start + 2, 10)
- Code(Counter(3)) at (prev + 2, 10) to (start + 0, 11)
- Code(Expression(6, Add)) at (prev + 1, 9) to (start + 0, 23)
= (c1 + (c2 + c3))
- Code(Counter(4)) at (prev + 2, 5) to (start + 1, 2)

Function name: assert::might_fail_assert
Raw bytes (21): 0x[01, 01, 01, 01, 05, 03, 01, 04, 01, 02, 0f, 02, 02, 25, 00, 3d, 05, 01, 01, 00, 02]
Number of files: 1
- file 0 => global file 1
Number of expressions: 1
- expression 0 operands: lhs = Counter(0), rhs = Counter(1)
Number of file 0 mappings: 3
- Code(Counter(0)) at (prev + 4, 1) to (start + 2, 15)
- Code(Expression(0, Sub)) at (prev + 2, 37) to (start + 0, 61)
= (c0 - c1)
- Code(Counter(1)) at (prev + 1, 1) to (start + 0, 2)

32 changes: 32 additions & 0 deletions tests/coverage-map/status-quo/assert.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#![allow(unused_assignments)]
// failure-status: 101

fn might_fail_assert(one_plus_one: u32) {
println!("does 1 + 1 = {}?", one_plus_one);
assert_eq!(1 + 1, one_plus_one, "the argument was wrong");
}

fn main() -> Result<(), u8> {
let mut countdown = 10;
while countdown > 0 {
if countdown == 1 {
might_fail_assert(3);
} else if countdown < 5 {
might_fail_assert(2);
}
countdown -= 1;
}
Ok(())
}

// Notes:
// 1. Compare this program and its coverage results to those of the very similar test
// `panic_unwind.rs`, and similar tests `abort.rs` and `try_error_result.rs`.
// 2. This test confirms the coverage generated when a program passes or fails an `assert!()` or
// related `assert_*!()` macro.
// 3. Notably, the `assert` macros *do not* generate `TerminatorKind::Assert`. The macros produce
// conditional expressions, `TerminatorKind::SwitchInt` branches, and a possible call to
// `begin_panic_fmt()` (that begins a panic unwind, if the assertion test fails).
// 4. `TerminatoKind::Assert` is, however, also present in the MIR generated for this test
// (and in many other coverage tests). The `Assert` terminator is typically generated by the
// Rust compiler to check for runtime failures, such as numeric overflows.
Loading

0 comments on commit 3141177

Please sign in to comment.