Line(s) omitted from coverage, may be related to unfolding

[jaltmayerpizzorno]

Describe the bug
I noticed that some lines were missing from the coverage report, possibly as a result of line unfolding by coveragepy.

To Reproduce
How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

1. What version of Python are you using?
   Python 3.9.10

2. What version of coverage.py shows the problem? The output of coverage debug sys is helpful.

$ python3 -m coverage debug sys
-- sys -------------------------------------------------------
               coverage_version: 6.3.1
                coverage_module: /usr/local/lib/python3.9/site-packages/coverage/__init__.py
                         tracer: -none-
                        CTracer: available
           plugins.file_tracers: -none-
            plugins.configurers: -none-
      plugins.context_switchers: -none-
              configs_attempted: .coveragerc
                   configs_read: /Users/juan/tmp/scikit-learn/.coveragerc
                    config_file: /Users/juan/tmp/scikit-learn/.coveragerc
                config_contents: b'[run]\nbranch = False\nsource = sklearn\nomit =\n    */sklearn/externals/*\n    */sklearn/_build_utils/*\n    */benchmarks/*\n    **/setup.py\n'
                      data_file: -none-
                         python: 3.9.10 (main, Jan 15 2022, 11:48:04) [Clang 13.0.0 (clang-1300.0.29.3)]
                       platform: macOS-12.3-x86_64-i386-64bit
                 implementation: CPython
                     executable: /usr/local/opt/python@3.9/bin/python3.9
                   def_encoding: utf-8
                    fs_encoding: utf-8
                            pid: 16078
                            cwd: /Users/juan/tmp/scikit-learn
                           path: /Users/juan/tmp/scikit-learn
                                 /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python39.zip
                                 /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9
                                 /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload
                                 /usr/local/lib/python3.9/site-packages
                                 /usr/local/lib/python3.9/site-packages/GPUtil-1.4.0-py3.9.egg
                                 /Users/juan/tmp/flask/src
                                 /Users/juan/tmp/scikit-learn
                                 /Users/juan/tmp/sqlalchemy/lib
                    environment: HOME = /Users/juan
                   command_line: /usr/local/lib/python3.9/site-packages/coverage/__main__.py debug sys
                sqlite3_version: 2.6.0
         sqlite3_sqlite_version: 3.38.2
             sqlite3_temp_store: 0
        sqlite3_compile_options: ATOMIC_INTRINSICS=1; COMPILER=clang-13.1.6; DEFAULT_AUTOVACUUM
                                 DEFAULT_CACHE_SIZE=-2000; DEFAULT_FILE_FORMAT=4; DEFAULT_JOURNAL_SIZE_LIMIT=-1
                                 DEFAULT_MMAP_SIZE=0; DEFAULT_PAGE_SIZE=4096; DEFAULT_PCACHE_INITSZ=20
                                 DEFAULT_RECURSIVE_TRIGGERS; DEFAULT_SECTOR_SIZE=4096; DEFAULT_SYNCHRONOUS=2
                                 DEFAULT_WAL_AUTOCHECKPOINT=1000; DEFAULT_WAL_SYNCHRONOUS=2; DEFAULT_WORKER_THREADS=0
                                 ENABLE_COLUMN_METADATA; ENABLE_FTS3; ENABLE_FTS3_PARENTHESIS
                                 ENABLE_FTS4; ENABLE_FTS5; ENABLE_GEOPOLY
                                 ENABLE_MATH_FUNCTIONS; ENABLE_PREUPDATE_HOOK; ENABLE_RTREE
                                 ENABLE_SESSION; MALLOC_SOFT_LIMIT=1024; MAX_ATTACHED=10
                                 MAX_COLUMN=2000; MAX_COMPOUND_SELECT=500; MAX_DEFAULT_PAGE_SIZE=8192
                                 MAX_EXPR_DEPTH=1000; MAX_FUNCTION_ARG=127; MAX_LENGTH=1000000000
                                 MAX_LIKE_PATTERN_LENGTH=50000; MAX_MMAP_SIZE=0x7fff0000; MAX_PAGE_COUNT=1073741823
                                 MAX_PAGE_SIZE=65536; MAX_SQL_LENGTH=1000000000; MAX_TRIGGER_DEPTH=1000
                                 MAX_VARIABLE_NUMBER=250000; MAX_VDBE_OP=250000000; MAX_WORKER_THREADS=8
                                 MUTEX_PTHREADS; SYSTEM_MALLOC; TEMP_STORE=1
                                 THREADSAFE=1

3. What versions of what packages do you have installed? The output of pip freeze is helpful.

absl-py==0.14.0
astunparse==1.6.3
attrs==21.4.0
bytecode==0.13.0
cachetools==4.2.4
certifi==2021.10.8
charset-normalizer==2.0.9
click==8.1.2
cloudpickle==1.6.0
colorama==0.4.4
commonmark==0.9.1
coverage==6.3.1
cycler==0.10.0
DateTime==4.4
distlib==0.3.4
execnet==1.9.0
filelock==3.4.2
find-libpython==0.2.0
-e git+ssh://git@github.com/pallets/flask.git@a03719b01076a5bfdc2c8f4024eda7b874614bc1#egg=Flask
flatbuffers==2.0
fonttools==4.29.0
gast==0.4.0
google-auth==2.3.3
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
GPUtil==1.4.0
greenlet==1.1.2
grpcio==1.42.0
h5py==3.6.0
hypothesis==6.39.3
idna==3.3
imageio==2.14.1
importlib-metadata==4.8.3
iniconfig==1.1.1
itsdangerous==2.1.2
jax==0.2.20
Jinja2==3.1.1
joblib==1.0.1
keras==2.7.0
Keras-Preprocessing==1.1.2
kiwisolver==1.3.2
libclang==12.0.0
Markdown==3.3.6
MarkupSafe==2.1.1
matplotlib==3.5.1
networkx==2.6.3
numpy==1.21.2
nvidia-ml-py==11.450.51
oauthlib==3.1.1
opencv-python==4.5.4.58
opt-einsum==3.3.0
packaging==21.3
pandas==1.3.2
# Editable install with no version control (Pillow==9.1.0)
-e /usr/local/lib/python3.9/site-packages
platformdirs==2.4.1
plotly==5.5.0
pluggy==1.0.0
protobuf==3.19.1
psutil==5.9.0
py==1.11.0
py-cpuinfo==8.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycairo==1.20.1
pycoral==2.0.0
pyerfa==2.0.0.1
Pygments==2.10.0
PyGObject==3.42.0
pyparsing==2.4.7
pyperf==2.3.0
pyperformance==1.0.4
pytest==7.0.1
pytest-arraydiff==0.5.0
pytest-astropy==0.9.0
pytest-astropy-header==0.1.2
pytest-cov==3.0.0
pytest-doctestplus==0.12.0
pytest-filter-subpackage==0.1.1
pytest-forked==1.4.0
pytest-mock==3.7.0
pytest-openfiles==0.5.0
pytest-remotedata==0.3.3
pytest-xdist==2.5.0
python-dateutil==2.8.2
pytz==2021.1
PyWavelets==1.2.0
PyYAML==6.0
requests-oauthlib==1.3.0
rich==10.9.0
rsa==4.8
scalene==1.5.2
scikit-image==0.19.1
-e git+https://github.com/scikit-learn/scikit-learn.git@4cf932d98e529dc0fd234342f168d9536ffac51a#egg=scikit_learn
scipy==1.7.1
six==1.16.0
sklearn==0.0
slipcover @ file:///Users/juan/project/slipcover
sortedcontainers==2.4.0
-e git+ssh://git@github.com/sqlalchemy/sqlalchemy.git@42c20b015d01ced441c42a8a6c5e9ed823316682#egg=SQLAlchemy
svgwrite==1.4.1
tabulate==0.8.9
tenacity==8.0.1
tensorboard==2.7.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.0
tensorflow==2.7.0
tensorflow-estimator==2.7.0
tensorflow-io-gcs-filesystem==0.23.1
termcolor==1.1.0
tflite-runtime==2.5.0.post1
threadpoolctl==2.2.0
tifffile==2021.11.2
toml==0.10.2
tomli==2.0.1
torch==1.10.0
typing_extensions==4.1.1
urllib3==1.26.7
virtualenv==20.13.0
Werkzeug==2.0.2
wrapt==1.13.3
zipp==3.6.0
zope.interface==5.4.0

4. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.

$ cat >test.py <<EOF
cond = True
foo = {
    "const": 42,
    "computed": 666 if cond \
                else 999,
    "other": 0
}
EOF
$ python3 -m coverage run test.py 
$ python3 -m coverage report -m
Name      Stmts   Miss  Cover   Missing
---------------------------------------
test.py       2      0   100%
---------------------------------------
TOTAL         2      0   100%

5. What commands did you run?
   Please see above.

Expected behavior
I'd expect line 5 (with the "else 999") to show up as missing.

Additional context

Hi Ned, we haven't met but I'm on the Boston Python slack.
I have been working with Emery Berger (@emeryberger) at UMass Amherst, noticed this issue and thought we'd let you know.
Given the cases I've seen this happen for before, I think it's related to unfolding certain lines.
Line 5 doesn't show as executed, missing or excluded.

I think the issue lies in how coveragepy detects executable lines.
I didn't dig super deep into this, but I think it may have to do with reading them from the AST
attributes.

[nedbat]

Hi fellow New Englander! :)

The missing execution you're showing is in an expression, not a statement. Coverage.py uses the word "line" a lot, but it's not measuring lines, it's measuring statements. In the future, we could try to measure coverage within expressions, but it's not something that coverage.py can do now. It could be that coverage.py's idea of statements is cruder than Python's also, so maybe we could look into that.

What trace event are you seeing for line 5? I see that it generates an event for line 4, but not for 5.

[jaltmayerpizzorno]

Hi fellow New Englander! :)

Hi!!

What trace event are you seeing for line 5? I see that it generates an event for line 4, but not for 5.

Argh, none at all, sorry about that! I fixed the issue description now.