v8.0.2: New map_list layer, bug fixes for saving to Pathy paths and more

✨ New features and improvements

• Add map_list layer (#472)

🔴 Bug fixes

• Fix issue #465: Fix saving models to Pathy paths
• Fix issue #466: Avoid initializing with Y if X is set
• Fix issue #470: Reset torch tensor type in require_cpu
• Fix issue #484: Ensure consistency of nO dim for BiLSTM

v8.0.1: Bug fixes for list2padded and LayerNorm

🔴 Bug fixes

• Fix issue #464: Fix list2padded op
• Add nO to LayerNorm

v8.0.0: Full rewrite, compose models using any framework such as PyTorch or TensorFlow, built-in type checking, config system and more

🔮 This version of Thinc has been rewritten from the ground up and will be used to power the upcoming spaCy v3.0. The new Thinc v8.0 is a lightweight deep learning library that offers an elegant, type-checked, functional-programming API for composing models, with support for layers defined in other frameworks such as PyTorch, TensorFlow or MXNet. You can use Thinc as an interface layer, a standalone toolkit or a flexible way to develop new models. For more details, see the documentation.

✨ New features and improvements

• Use any framework: Switch between PyTorch, TensorFlow and MXNet models without changing your application, or even create mutant hybrids using zero-copy array interchange.
• Type checking: Develop faster and catch bugs sooner with sophisticated type checking. Trying to pass a 1-dimensional array into a model that expects two dimensions? That’s a type error. Your editor can pick it up as the code leaves your fingers.
• Config system: Configuration is a major pain for ML. Thinc lets you describe trees of objects with references to your own functions, so you can stop passing around blobs of settings. It's simple, clean, and it works for both research and production.
• Super lightweight: Small and easy to install with very few required dependencies, available on pip and conda for Linux, macOS and Windows. Simple source with a consistent API.
• Concise functional-programming approach to model definition using composition rather than inheritance.
• First-class support for variable-length sequences: multiple built-in sequence representations and your layers can use any object.

v7.4.5: Fix numpy compatibility in binary wheels

🔴 Bug fixes

• Fix numpy compatibility in binary wheel releases.
• Fix cupy-cuda111 extra requirement.

v7.4.4: Update for cupy v8 and update package setup

🔴 Bug fixes

• Update for compatibility with cupy v8.
• Remove f-strings from PyTorchWrapper.
• Remove detailed numpy build constraints from pyproject.toml.
• Update Cython extension setup.

v7.4.3: Fix memory leak in Beam and random seed in ParametricAttention

✨ New features and improvements

• Add seed argument to ParametricAttention.
• Dynamically include numpy headers and add numpy build constraints.
• Update tests to support hypothesis v5.

🔴 Bug fixes

• Fix memory leak in Beam.

v7.4.2: Update compatible cupy versions and for python 3.9

🔴 Bug fixes

• Restrict compatible cupy versions to <8.0.0.
• Update setup for python 3.9.

v7.4.1: Fix OOV vectors bug

🔴 Bug fixes

• Use 0-vector for OOV in StaticVectors to fix similarity bug in spaCy
• Fix murmurhash on platforms where long type was not 64 bit

v7.3.1: Relax dependecy requirements

🔴 Bug fixes

• Relax version range of plac to match spaCy.

v7.3.0: Mish activation and experimental optimizers

✨ New features and improvements

• Add Mish activation. Use via the thinc.v2v.Mish layer, which computes f(X) = mish(W @ X + b). CUDA and Cython kernels are included to make the activation efficient.
• Add experimental support for RAdam to the optimizer. Enable it with the keyword argument use_radam to True. In preliminary testing, it's a small change that's worth enabling.
• Add experimental support for Lookahead to the optimizer. Enable it by setting the keyword argument lookahead_k to a positive integer. In preliminary testing, it helps if you're not using parameter averaging, but with averaging it's a bit worse.
• Add experimental support for LARS to the optimizer. Enable it by setting use_lars to True. In preliminary testing, this hasn't worked well at all – possibly our implementation is broken.

🙏 Acknowledgements

Big thanks to @digantamisra98 for the Mish activation, especially the extensive experiments and simple gradient calculation. We expect to be using the activation in the next round of spaCy models.

Gratitude to the fast.ai community for their crowd-sourced experiments, and especially to users @lessw2020, @mgrankin and others for their optimizer implementations, which we referenced heavily when implementing the optimizers for Thinc. More importantly, it's super helpful to have a community filtering the deluge of papers for techniques that work on a few different datasets. This thread on optimization research was particularly helpful.