Notebook failure status

[lamberta]

The tensorflow.org docs are continuously updated for clarity and to make sure they're compatible with the latest TensorFlow release (this makes for a challenging translation project). And because they're Jupyter notebooks, we have some automation to run and test these docs to ensure everything on the site works.

Because translated notebooks are downstream of the source documents, there's some inevitable lag for updates to the translation. Nonetheless, we also run and test the translated notebooks because it's important for all users to have working docs.

To help the community and vendors find (and fix) broken translations, we're now publishing a list of notebooks that did not run to completion. This usually indicates that a translated file is out-of-date and not synced to its source counterpart. Notebooks can be run in Colab and synced using the GitLocalize project. These notebook failures are listed in the /site/<lang>/FAILURES.md file. Notebooks are tested on a periodic basis (usually weekly or bi-weekly) so the failure list may not reflect the most recent updates.

Still figuring out how to best broadcast this information and the removal rules for broken notebooks (grace period, etc.), but the first step is to make this available to the community. These status lists can be found here:

• site/ar/FAILURES.md
• site/es-419/FAILURES.md
• site/fr/FAILURES.md [Currently unavailable]
• site/id/FAILURES.md
• site/it/FAILURES.md
• site/ja/FAILURES.md
• site/ko/FAILURES.md
• site/pt-br/FAILURES.md
• site/ru/FAILURES.md
• site/zh-cn/FAILURES.md

[ilyaspiridonov]

@joefernandez (writing here as issues seem to be disabled in the repo).
The script that updates notebook statuses (tracks failed notebooks) doesn't seem to run on a regular basis.

• The weekly GithubAction that I added earlier seems to run fine.
• That GitHub action triggers the script Billy wrote earlier that downloads the list (and some headers like 'Last modified') from this endpoint: https://storage.googleapis.com/tensorflow_docs/failed_notebooks/${lang}/failed_notebooks.txt
• My understanding is that there is some script internally that runs the actual tests
• It appears for some languages like Korean the last time that script ran was a year ago and the results are now out of date: (site/ko/FAILURES.md)
  Billy @lamberta, I know this is no longer your responsibility, but perhaps you could point me and @joefernandez in the right direction to resolve this? :)
  Thank you!

[lamberta]

@ilyaspiridonov @joefernandez Correct, the tools/notebook-status.sh updates the various <lang>/FAILURES.md files by parsing a generated error log. The error log is created when the internal mechanisms import and execute the notebooks. It is not run automatically ... you may be able to run it externally (don't know, haven't tried). cc: @MarkDaoust