Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

To work around an issue in pytest, run tests from empty temp dir #1566

Merged

Conversation

oleksandr-pavlyk
Copy link
Collaborator

Create empty temporary directory to run "pytest" command from to work-around an issue in pytest accessing files/directories it may not have access to during test collection.

pytest-dev/pytest#11904

This should fixed test_windows step failures.

  • Have you provided a meaningful PR description?
  • Have you added a test, reproducer or referred to an issue with a reproducer?
  • Have you tested your changes locally for CPU and GPU devices?
  • Have you made sure that new changes do not introduce compiler warnings?
  • Have you checked performance impact of proposed changes?
  • If this PR is a work in progress, are you opening the PR as a draft?

Copy link

github-actions bot commented Feb 27, 2024

Deleted rendered PR docs from intelpython.github.com/dpctl, latest should be updated shortly. 🤞

@coveralls
Copy link
Collaborator

coveralls commented Feb 27, 2024

Coverage Status

coverage: 91.099%. remained the same
when pulling b961b1a on deal-with-pytest-permission-failure-on-window
into be6bced on master.

Copy link

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_30 ran successfully.
Passed: 904
Failed: 2
Skipped: 94

@oleksandr-pavlyk oleksandr-pavlyk force-pushed the deal-with-pytest-permission-failure-on-window branch 2 times, most recently from ebff1bd to b37b0da Compare February 27, 2024 17:01
Copy link

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_30 ran successfully.
Passed: 904
Failed: 2
Skipped: 94

Create empty temporary directory to run "pytest" command from to
work-around an issue in pytest accessing files/directories it may
not have access to during test collection.

pytest-dev/pytest#11904
@oleksandr-pavlyk oleksandr-pavlyk force-pushed the deal-with-pytest-permission-failure-on-window branch from b37b0da to 4441b9f Compare February 27, 2024 19:20
Copy link

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_30 ran successfully.
Passed: 904
Failed: 2
Skipped: 94

@oleksandr-pavlyk
Copy link
Collaborator Author

Creating an empty sub-directory within working directory and running pytest from there as suggested in pytest-dev/pytest#11904 did not work on Windows.

conda activate test_dpctl && python -m pytest -v -s --pyargs dpctl
  shell: C:\Windows\system[3](https://github.com/IntelPython/dpctl/actions/runs/8070208629/job/22049391309#step:19:3)2\cmd.EXE /C CALL {0}
  env:
    PACKAGE_NAME: dpctl
    MODULE_NAME: dpctl
    TEST_ENV_NAME: test_dpctl
    VER_SCRIPT1: import json; f = open('ver.json', 'r'); j = json.load(f); f.close(); 
    VER_SCRIPT2: d = j['dpctl'][0]; print('='.join((d[s] for s in ('version', 'build'))))
    workdir: D:\a\dpctl\dpctl
    CHANNELS: -c intel -c conda-forge --override-channels
    INPUT_RUN_POST: true
    CONDA: C:\Users\runneradmin\miniconda3
    CONDA_PKGS_DIR: C:\Users\runneradmin\conda_pkgs_dir
    SYCL_CACHE_PERSISTENT: 1
============================= test session starts =============================
platform win32 -- Python 3.11.8, pytest-8.0.2, pluggy-1.3.0 -- C:\Users\runneradmin\miniconda3\envs\test_dpctl\python.exe
cachedir: .pytest_cache
rootdir: D:\a\dpctl\dpctl\test_tmp
plugins: cov-[4](https://github.com/IntelPython/dpctl/actions/runs/8070208629/job/22049391309#step:19:4).1.0
collecting ... collected 170[5](https://github.com/IntelPython/dpctl/actions/runs/8070208629/job/22049391309#step:19:5)1 items / 1 error

=================================== ERRORS ====================================
________________________ ERROR collecting test session ________________________
C:\Users\runneradmin\miniconda3\envs\test_dpctl\Lib\site-packages\_pytest\runner.py:342: in from_call
    result: Optional[TResult] = func()
C:\Users\runneradmin\miniconda3\envs\test_dpctl\Lib\site-packages\_pytest\runner.py:38[7](https://github.com/IntelPython/dpctl/actions/runs/8070208629/job/22049391309#step:19:7): in collect
    return list(collector.collect())
C:\Users\runneradmin\miniconda3\envs\test_dpctl\Lib\site-packages\_pytest\main.py:50[8](https://github.com/IntelPython/dpctl/actions/runs/8070208629/job/22049391309#step:19:8): in collect
    for direntry in scandir(self.path):
C:\Users\runneradmin\miniconda3\envs\test_dpctl\Lib\site-packages\_pytest\pathlib.py:701: in scandir
    with os.scandir(path) as s:
E   PermissionError: [WinError 5] Access is denied: 'C:\\Documents and Settings'
=========================== short test summary info ===========================
ERROR ::::Documents and Settings - PermissionError: [WinError 5] Access is denied: 'C:\\Documents and Settings'
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
============================== 1 error in 5.41s ===============================
Error: Process completed with exit code 2.

Notice that pytest reports "rootdir" as that empty temporary directory, yet it still attempts to traverse "C:\Documents and Settings" and got PermissionError.

So, giving up, I have pushed a change to use "pytest'<8'" when installing pytest in conda environment.

Copy link

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_31 ran successfully.
Passed: 904
Failed: 2
Skipped: 94

@oleksandr-pavlyk oleksandr-pavlyk force-pushed the deal-with-pytest-permission-failure-on-window branch from a253a0a to b7a07fa Compare February 27, 2024 22:09
Copy link

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_31 ran successfully.
Passed: 904
Failed: 2
Skipped: 94

@oleksandr-pavlyk oleksandr-pavlyk force-pushed the deal-with-pytest-permission-failure-on-window branch from b7a07fa to b961b1a Compare February 27, 2024 23:50
@oleksandr-pavlyk oleksandr-pavlyk marked this pull request as ready for review February 28, 2024 01:13
Copy link

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_31 ran successfully.
Passed: 904
Failed: 2
Skipped: 94

- name: Run tests
shell: cmd /C CALL {0}
env:
SYCL_CACHE_PERSISTENT: 1
working-directory: ${{ env.workdir }}\test_tmp
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would allow default path, since it should also in writable location (how would conda install else work)? So conda environment location is located in writable path. Putting conda env in working folder could result in accidental removal and unexpected CI behavior. Same for other job.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, but this newly created folder is writable too. It is created by the workflow at the prior step as a sub-folder of the default path.

Copy link
Collaborator

@ZzEeKkAa ZzEeKkAa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGMT! Just small nit.

@oleksandr-pavlyk oleksandr-pavlyk merged commit be4a01c into master Feb 28, 2024
49 checks passed
@oleksandr-pavlyk oleksandr-pavlyk deleted the deal-with-pytest-permission-failure-on-window branch February 28, 2024 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants