[chore] Fix lint errors that broke master #348

anj-s · 2021-02-01T18:22:13Z

Before submitting

Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
[X ] Did you read the contributor guideline?
Did you make sure to update the docs?
Did you write any new necessary tests?

What does this PR do?

Fixes broken master: https://app.circleci.com/pipelines/github/facebookresearch/fairscale?branch=master

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

blefaudeux

LGTM, thanks Anjali !

msbaines

I think the correct solution is to not include dataclasses in know_third_party.

blefaudeux · 2021-02-01T18:48:39Z

I think the correct solution is to not include dataclasses in know_third_party.

linting used to be done with python3.6, which does not have dataclasses, which means that just following our contributing guide (like, setup pre-commit and such) would auto-add dataclasses to the dependencies. Now if the reference python version changes that's ok, but it would be nice to publicize this a little, this is the real change behind the breakage

blefaudeux · 2021-02-01T18:55:57Z

I think the correct solution is to not include dataclasses in know_third_party.

linting used to be done with python3.6, which does not have dataclasses, which means that just following our contributing guide (like, setup pre-commit and such) would auto-add dataclasses to the dependencies. Now if the reference python version changes that's ok, but it would be nice to publicize this a little, this is the real change behind the breakage

looks like we changed the default to 3.7 sometime in the last month, updating my environments

anj-s · 2021-02-01T19:13:27Z

I think the correct solution is to not include dataclasses in know_third_party.

Done.

blefaudeux

Thanks @anj-s !

authored-by: Anjali Sridhar <anj@devfair0443.h2.fair>

* [mypy]: fixed all the mypy errors * make older version python happy with typing hints * [chore] Fix lint errors that broke master (#348) authored-by: Anjali Sridhar <anj@devfair0443.h2.fair> Co-authored-by: anj-s <32556631+anj-s@users.noreply.github.com>

* [chore] Fix lint errors that broke master (#348) authored-by: Anjali Sridhar <anj@devfair0443.h2.fair> * [fix] ShardedDDP - cpu testfix - remove Gloo/CPU (#350) * no idea about the root issue, but it proved to be fairly narrowed (gloo+cpu+python3.8+no cuda installed) so I guess that's out of scope for fairscale * [feat][OSS] elastic and pytorch compatible checkpoints (#310) * adding a test to prove the inter operability with upstream pytorch * updating the changelog * eager state pruning * pytorch 1.5 compat * [fix] ShardedDDP - properly handle post device change (#353) * adding the .to(device) support + unit testing * doc update * [feat] Add AdaScaleWrapper (#347) * [feat] Add AdaScaleWrapper - This enables a different API for wrapping an optimizer with AdaScale. - This also enables AdaScale to be wrapped by OSS. - However, OSS wrapping AdaScale results in different optimization, which future research will be needed to study its effects. testing: add unit tests. * addressed comment: typo * [refactor] Refactor and enable multiprocess nn.Pipe benchmarks. (#319) * mp cleanup * round of multiprocess refactoring * test golden run * print cuda stats * fix lint errors * enable multiprocess pipe benchmarks * set world size to be available gpus * more changes * use synthetic loaders for intermediate pipeline stages * merged master * fix for the devices property * dataloader fix * modify rank check * print wps stats * enable verification * fix logging * fix flag name * fix flag name * check for rank * fix indent * pass args * pass args * modify golden data * remove unused print messsage * fix lint errors * add comments * fix benchmarks Co-authored-by: Anjali Sridhar <anj@devfair0443.h2.fair> * [refactor] pipe: simplify balance and module checks (#346) * [chore] v0.1.5 (#355) * [chore] disheartening switch off of a OSS cpu test (#356) * precise skip, only if agent has only cpu * [feat][minor] OSS Benchmark - regression test + background testing new optims (#352) * restoring the regression test, adding a test of the for_each optims * fix the regression test on circleci * removing unused flags * [refactor] multiprocess_pipe: cleanup __init__ (#357) * [refactor] multiprocess_pipe: remove retain_graph __init__ param (#358) It is not currently being used so we can simplify the interface by removing it. * [refactor] multiprocess_pipe: focus on LazyModule usage (#360) * [feat] ShardedDDP : Adding a proper DDP parity / AMP unit test, overdue (#361) * Adding a proper ddp parity / AMP unit test, overdue * catch non-AMP pytorch * [perf][OSS] Clip grad norm : minor obvious speedup (#363) cache this iterator, easy speed up * [refactor] multiprocess_pipe: remove pipelined_backward (#362) * [perf] ShardedDDP - small memory use reduction - minor speedup (#366) * minor * minor * [fix] repro+fix (#365) fix a broken earlier commit, only worked for the first step * [refactor] OSS only use flat buffers (#371) * flat params all along, way simpler * updating the docstring * [refactor] AsyncPipe: do not sub-class MultiProcessPipe (#370) * [refactor] remove multiprocess dependency on async (#373) * [fix] Workaround need for pip --no-build-isolation (#375) * Add fairscale.nn.misc.checkpoint_activations (#376) * Add fairscale.utils.containers Co-authored-by: Min Xu <24926999+min-xu-ai@users.noreply.github.com> * Add fairscale.nn.misc.checkpoint_activations Co-authored-by: Sam Shleifer <sshleifer@gmail.com> Co-authored-by: Min Xu <24926999+min-xu-ai@users.noreply.github.com> Co-authored-by: Sam Shleifer <sshleifer@gmail.com> * [chore] v0.1.6 (#377) * v0.1.6 Co-authored-by: anj-s <32556631+anj-s@users.noreply.github.com> Co-authored-by: Benjamin Lefaudeux <blefaudeux@users.noreply.github.com> Co-authored-by: Anjali Sridhar <anj@devfair0443.h2.fair> Co-authored-by: msbaines <35972327+msbaines@users.noreply.github.com> Co-authored-by: Leonard Lausen <leonard@lausen.nl> Co-authored-by: Myle Ott <myleott@fb.com> Co-authored-by: Sam Shleifer <sshleifer@gmail.com>

fix-lint errors for pipe

bcea8ad

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 1, 2021

anj-s requested review from blefaudeux and msbaines February 1, 2021 18:22

blefaudeux approved these changes Feb 1, 2021

View reviewed changes

min-xu-ai approved these changes Feb 1, 2021

View reviewed changes

msbaines suggested changes Feb 1, 2021

View reviewed changes

Anjali Sridhar added 2 commits February 1, 2021 11:12

remove dataclasses

09e4993

fix-lint errors

160f3cf

anj-s requested a review from msbaines February 1, 2021 19:13

anj-s mentioned this pull request Feb 1, 2021

[chore] Pre commit to python 3.7 #349

Closed

4 tasks

fix python version

2f703b6

blefaudeux approved these changes Feb 1, 2021

View reviewed changes

blefaudeux merged commit dc05dd8 into master Feb 1, 2021

blefaudeux pushed a commit that referenced this pull request Feb 1, 2021

[chore] Fix lint errors that broke master (#348)

bf2a352

authored-by: Anjali Sridhar <anj@devfair0443.h2.fair>

blefaudeux mentioned this pull request Feb 1, 2021

[feat][OSS] elastic and pytorch compatible checkpoints #310

Merged

4 tasks

blefaudeux deleted the fix-pipe-lint branch February 16, 2021 23:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[chore] Fix lint errors that broke master #348

[chore] Fix lint errors that broke master #348

anj-s commented Feb 1, 2021

blefaudeux left a comment

msbaines left a comment

blefaudeux commented Feb 1, 2021 •

edited

Loading

blefaudeux commented Feb 1, 2021

anj-s commented Feb 1, 2021

blefaudeux left a comment

[chore] Fix lint errors that broke master #348

[chore] Fix lint errors that broke master #348

Conversation

anj-s commented Feb 1, 2021

Before submitting

What does this PR do?

PR review

Did you have fun?

blefaudeux left a comment

Choose a reason for hiding this comment

msbaines left a comment

Choose a reason for hiding this comment

blefaudeux commented Feb 1, 2021 • edited Loading

blefaudeux commented Feb 1, 2021

anj-s commented Feb 1, 2021

blefaudeux left a comment

Choose a reason for hiding this comment

blefaudeux commented Feb 1, 2021 •

edited

Loading