tmpdir name clashes with parametrization

[DariusL]

It seems that tmpdir numbering fails when fixtures have similar start names.
Reproduction:

import pytest
CLASSES = ['JasminTest', 'JAssert', 'JAssert2', 'STest5', 'StrictClass', 'Strictfp']
PARAMS = [("emulator-5554", p) for p in CLASSES]

@pytest.mark.parametrize("one, two", PARAMS)
def test_asdasdsa(one, two, tmpdir):
    pass

Result:

test_repro.py::test_asdasdsa[emulator-5554-JasminTest] PASSED
test_repro.py::test_asdasdsa[emulator-5554-JAssert] ERROR
test_repro.py::test_asdasdsa[emulator-5554-JAssert2] ERROR
test_repro.py::test_asdasdsa[emulator-5554-STest5] PASSED
test_repro.py::test_asdasdsa[emulator-5554-StrictClass] ERROR
test_repro.py::test_asdasdsa[emulator-5554-Strictfp] ERROR

The error is the similar for all:

____________ ERROR at setup of test_asdasdsa[emulator-5554-JAssert] ____________

self = <CallInfo when='setup' exception: [File exists]: mkdir('/private/var/folders/bz/sr5j7mgn5919c2k1n03m5x2w0000gn/T/pytest-of-dlapunas/pytest-337/test_asdasdsa_emulator_5554_JA0',)>
func = <function <lambda> at 0x109c10cf8>, when = 'setup'

    def __init__(self, func, when):
        #: context of invocation: one of "setup", "call",
        #: "teardown", "memocollect"
        self.when = when
        self.start = time()
        try:
>           self.result = func()

../venv/lib/python2.7/site-packages/_pytest/runner.py:157: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../venv/lib/python2.7/site-packages/_pytest/runner.py:145: in <lambda>
    return CallInfo(lambda: ihook(item=item, **kwds), when=when)
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:745: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:339: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:334: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:613: in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:254: in _wrapped_call
    return call_outcome.get_result()
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:280: in get_result
    _reraise(*ex)  # noqa
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:265: in __init__
    self.result = func()
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:614: in execute
    res = hook_impl.function(*args)
../venv/lib/python2.7/site-packages/_pytest/runner.py:94: in pytest_runtest_setup
    item.session._setupstate.prepare(item)
../venv/lib/python2.7/site-packages/_pytest/runner.py:449: in prepare
    col.setup()
../venv/lib/python2.7/site-packages/_pytest/python.py:1597: in setup
    fixtures.fillfixtures(self)
../venv/lib/python2.7/site-packages/_pytest/fixtures.py:237: in fillfixtures
    request._fillfixtures()
../venv/lib/python2.7/site-packages/_pytest/fixtures.py:382: in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
../venv/lib/python2.7/site-packages/_pytest/fixtures.py:424: in getfixturevalue
    return self._get_active_fixturedef(argname).cached_result[0]
../venv/lib/python2.7/site-packages/_pytest/fixtures.py:449: in _get_active_fixturedef
    result = self._getfixturevalue(fixturedef)
../venv/lib/python2.7/site-packages/_pytest/fixtures.py:514: in _getfixturevalue
    val = fixturedef.execute(request=subrequest)
../venv/lib/python2.7/site-packages/_pytest/fixtures.py:771: in execute
    return ihook.pytest_fixture_setup(fixturedef=self, request=request)
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:745: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:339: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:334: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:613: in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:254: in _wrapped_call
    return call_outcome.get_result()
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:280: in get_result
    _reraise(*ex)  # noqa
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:265: in __init__
    self.result = func()
../venv/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:614: in execute
    res = hook_impl.function(*args)
../venv/lib/python2.7/site-packages/_pytest/fixtures.py:801: in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
../venv/lib/python2.7/site-packages/_pytest/fixtures.py:706: in call_fixture_func
    res = fixturefunc(**kwargs)
../venv/lib/python2.7/site-packages/_pytest/tmpdir.py:125: in tmpdir
    x = tmpdir_factory.mktemp(name, numbered=True)
../venv/lib/python2.7/site-packages/_pytest/tmpdir.py:41: in mktemp
    keep=0, rootdir=basetemp, lock_timeout=None)
../venv/lib/python2.7/site-packages/py/_path/local.py:825: in make_numbered_dir
    udir = rootdir.mkdir(prefix + str(maxnum+1))
../venv/lib/python2.7/site-packages/py/_path/local.py:459: in mkdir
    py.error.checked_call(os.mkdir, fspath(p))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <py._error.ErrorMaker object at 0x1095654d0>
func = <built-in function mkdir>
args = ('/private/var/folders/bz/sr5j7mgn5919c2k1n03m5x2w0000gn/T/pytest-of-dlapunas/pytest-337/test_asdasdsa_emulator_5554_JA0',)
kwargs = {}, __tracebackhide__ = False, cls = <class 'py.error.EEXIST'>
value = OSError(17, 'File exists'), tb = <traceback object at 0x109d2c560>
errno = 17

    def checked_call(self, func, *args, **kwargs):
        """ call a function and raise an errno-exception if applicable. """
        __tracebackhide__ = True
        try:
            return func(*args, **kwargs)
        except self.Error:
            raise
        except (OSError, EnvironmentError):
            cls, value, tb = sys.exc_info()
            if not hasattr(value, 'errno'):
                raise
            __tracebackhide__ = False
            errno = value.errno
            try:
                if not isinstance(value, WindowsError):
                    raise NameError
            except NameError:
                # we are not on Windows, or we got a proper OSError
                cls = self._geterrnoclass(errno)
            else:
                try:
                    cls = self._geterrnoclass(_winerrnomap[errno])
                except KeyError:
                    raise value
>           raise cls("%s%r" % (func.__name__, args))
E           EEXIST: [File exists]: mkdir('/private/var/folders/bz/sr5j7mgn5919c2k1n03m5x2w0000gn/T/pytest-of-dlapunas/pytest-337/test_asdasdsa_emulator_5554_JA0',)

../venv/lib/python2.7/site-packages/py/_error.py:85: EEXIST

Basically, test_asdasds[emulator-5554-JasminTest], test_asdasds[emulator-5554-JAssert] and test_asdasds[emulator-5554-JAssert2] tried to use the same temporary directory test_asdasdsa_emulator_5554_JA0, the last two failing.

Seems to be directly related to full test name length - if I shorten the test name, I only get two errors, where the names have a longer match:

test_repro.py::test_asdasds[emulator-5554-JasminTest] PASSED
test_repro.py::test_asdasds[emulator-5554-JAssert] ERROR
test_repro.py::test_asdasds[emulator-5554-JAssert2] ERROR
test_repro.py::test_asdasds[emulator-5554-STest5] PASSED
test_repro.py::test_asdasds[emulator-5554-StrictClass] PASSED
test_repro.py::test_asdasds[emulator-5554-Strictfp] PASSED

Running pytest 3.1.0, MacOS 10.12.6 beta.
Fresh virtualenv,

appdirs (1.4.3)
packaging (16.8)
pip (9.0.1)
pyparsing (2.2.0)
setuptools (35.0.2)
six (1.10.0)
wheel (0.29.0)

[nicoddemus]

This should be fixed with the release of py-1.4.34, could you please try that?

[DariusL]

Sorry for the delay, but this is still happening with

$ pip freeze
py==1.4.34
pytest==3.1.1

I used the reproduction I posted earlier as $ pytest repro.py -v

[nicoddemus]

That's strange, I can reproduce it with py-1.4.33 but not with py-1.4.34 on Windows (which also has case-insensitive path names). Can somebody with OSX give it a try to see if they can reproduce it?

[isaulv]

I get a huge stack trace with pytest 3.1.2, py 1.4.34 using MacOS 10.12.5 using Darius's code.

[nicoddemus]

Thanks @Dude-x.

That's unfortunate, I can't reproduce it on Windows using py 1.4.34 and don't have a Mac around to try on it. If any Mac users want to tackle this, the code which supposedly should prevent names clashing because of case differences in the folder name is here, from pytest-dev/py#121.

[RonnyPfannschmidt]

@nicoddemus afair osx does unicode normalization for path names - which is even worse than just case normalization and could be the source of the issue

[nicoddemus]

does unicode normalization for path names

Sorry, what do you mean by that?

[RonnyPfannschmidt]

HFS+ stores filenames in decomposed form, so an umlaut for example will be turned from a single codepoint to 2

[nicoddemus]

I see, thanks.

I wonder if that's the problem though, the example posted by @DariusL only uses ASCII. Also doesn't the form how the filenames are stored abstract away when obtaining the file names (os.listdir()) and comparing it with the prefix (already unicode)?

[RonnyPfannschmidt]

true, as of now i can only guess as well

[asottile]

This should be fixed after py==1.5.4: pytest-dev/py#186 #3451 -- try again with that?

[DariusL]

Seems to pass correctly with

$ pip freeze
atomicwrites==1.1.5
attrs==18.1.0
funcsigs==1.0.2
more-itertools==4.2.0
pluggy==0.6.0
py==1.5.4
pytest==3.6.3
six==1.11.0

which I get with pip install pytest. Running on macosx 10.13.6.

$ pytest repro.py -v
=================================== test session starts ====================================
platform darwin -- Python 2.7.10, pytest-3.6.3, py-1.5.4, pluggy-0.6.0 -- /Users/dlapunas/dev/python/repro/venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/dlapunas/dev/python/repro, inifile:
collected 6 items                                                                          

repro.py::test_asdasdsa[emulator-5554-JasminTest] PASSED                             [ 16%]
repro.py::test_asdasdsa[emulator-5554-JAssert] PASSED                                [ 33%]
repro.py::test_asdasdsa[emulator-5554-JAssert2] PASSED                               [ 50%]
repro.py::test_asdasdsa[emulator-5554-STest5] PASSED                                 [ 66%]
repro.py::test_asdasdsa[emulator-5554-StrictClass] PASSED                            [ 83%]
repro.py::test_asdasdsa[emulator-5554-Strictfp] PASSED                               [100%]

================================= 6 passed in 0.05 seconds =================================

Looks like a fix to me.

[asottile]

Sweeeeeeeeet