bug: pynvml.NVMLError_DriverNotLoaded: Driver Not Loaded

[rlleshi]

Describe the bug

When starting the server the following cuda-related error is thrown:

❯ bentoml serve . --verbose --debug
Traceback (most recent call last):
  File "/home/user/.pyenv/versions/neoface-swapper/bin/bentoml", line 8, in <module>
    sys.exit(cli())
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/bentoml_cli/utils.py", line 372, in wrapper
    return func(*args, **kwargs)
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/bentoml_cli/utils.py", line 343, in wrapper
    return_value = func(*args, **kwargs)
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/bentoml_cli/env_manager.py", line 123, in wrapper
    return func(*args, **kwargs)
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/bentoml_cli/serve.py", line 313, in serve
    serve_http(
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/_bentoml_impl/server/serving.py", line 204, in serve_http
    allocator = ResourceAllocator()
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/_bentoml_impl/server/allocator.py", line 21, in __init__
    self.system_resources = system_resources()
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/bentoml/_internal/resource.py", line 46, in system_resources
    res[resource_kind] = resource.from_system()
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/bentoml/_internal/resource.py", line 248, in from_system
    pynvml.nvmlInit()
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/pynvml.py", line 1793, in nvmlInit
    nvmlInitWithFlags(0)
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/pynvml.py", line 1783, in nvmlInitWithFlags
    _nvmlCheckReturn(ret)
  File "/home/user/.pyenv/versions/3.9.0/envs/neoface-swapper/lib/python3.9/site-packages/pynvml.py", line 855, in _nvmlCheckReturn
    raise NVMLError(ret)
pynvml.NVMLError_DriverNotLoaded: Driver Not Loaded

To reproduce

No response

Expected behavior

I have nvidia drivers and nvidia toolkit installed because I sometimes use an eGPU. But bento only runs when the eGPU is attached. The current application code is supposed to work both with CPU & GPU. Actually it was working a couple of weeks ago when I first developed the solution in another machine .

Environment

bentoml: 1.3.0
python: 3.9.0
ubuntu 22.04