Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker build failure - Add Tini and set version range for numpy #1514

Merged
merged 2 commits into from
Sep 27, 2016
Merged

Docker build failure - Add Tini and set version range for numpy #1514

merged 2 commits into from
Sep 27, 2016

Conversation

fzagarzazu
Copy link
Contributor

First time user, I spent some time setting it up and I ran into a few issues, the purpose of this pull request is to bring it to your attention, maybe it will help somebody else. Please close it if it doesn't apply.

Background:

  • First time user.
  • OS X El Capitan + VirtualBox
  • Docker version 1.9.1,

Docker build failure:

Obtaining file:///zipline
    Complete output from command python setup.py egg_info:
    warning: no files found matching '*.pyx' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.pxd' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.h' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.pxd' under directory 'Cython/Utility'
    Unable to find pgen, not compiling formal grammar.

    Installed /zipline/.eggs/Cython-0.24.1-py2.7-linux-x86_64.egg
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/zipline/setup.py", line 300, in <module>
        **conditional_arguments
      File "/usr/local/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 315, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
        replace_conflicting=True,
      File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 851, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1116, in best_match
        dist = working_set.find(req)
      File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 720, in find
        raise VersionConflict(dist, req)
    pkg_resources.VersionConflict: (numpy 1.9.2 (/usr/local/lib/python2.7/site-packages), Requirement.parse('numpy>=1.11.1'))

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /zipline/

Possible solution / workaround:

  • Set numpy version range. (included in commit).

Jupyter notebooks: Dead kernels / automatic restart failures.

Thank you for your work.

@coveralls
Copy link

coveralls commented Sep 25, 2016

Coverage Status

Coverage remained the same at 86.55% when pulling 68f6e6e on fzagarzazu:fix-dockerfile into be0ecb6 on quantopian:master.

@lssilva
Copy link

lssilva commented Sep 26, 2016

Thank you I had the same issue, I hope this is integrated soon

Copy link
Contributor

@ssanderson ssanderson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had one request and one question, otherwise looks reasonable to me.

@@ -49,13 +54,14 @@ RUN mkdir ${PROJECT_DIR} \

WORKDIR /ta-lib

RUN pip install numpy==1.9.2 \
RUN pip install 'numpy>=1.11.1,<2.0.0' \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Latest zipline is also on pandas 0.18.1 and scipy 0.17.1 if you want to bump those as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, latest commit sets the version range for pandas and scipy.

Copy link
Contributor Author

@fzagarzazu fzagarzazu Sep 27, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note I added && pip install matplotlib \ to the Dockerfile in order to fix an error when executing zipline/examples/buyapple.ipynb .

ENV TINI_VERSION v0.10.0
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /tini
RUN chmod +x /tini
ENTRYPOINT ["/tini", "--"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a good executive summary somewhere of the benefits of using tini as the container pid 1? I'm aware vaguely that it solves issues with zombie processes not being correctly reaped, but I don't have a clear sense of concrete cases where it fixes issues.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't dig much into it, but this is the best explanation I found krallin/tini#8 .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For what it's worth I am running it with docker-compose .

@coveralls
Copy link

coveralls commented Sep 27, 2016

Coverage Status

Coverage decreased (-0.006%) to 86.544% when pulling 4ae4934 on fzagarzazu:fix-dockerfile into be0ecb6 on quantopian:master.

@ssanderson
Copy link
Contributor

ssanderson commented Sep 27, 2016

👍, thanks @fzagarzazu!

@ssanderson ssanderson merged commit 4a22afd into quantopian:master Sep 27, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants