v6.9.0: Reorganize layers, bug fix to Layer Normalization

✨ Major features and improvements

• Add new namespace modules thinc.v2v, thinc.i2v, thinc.t2t, thinc.t2v that group layer implementations by input and output type v indicates vector, i indicates integer ID, t indicates tensor. The input type refers to the logical unit, i.e. what constitutes a sample.

🔴 Bug fixes

• Fix bug in layer normalization. The bug fix means that models trained with Thinc 6.8 are incompatible with Thinc 6.9. For convenience, a backwards compatibility flag has been added, which can be set with thinc.neural._classes.layernorm.set_compat_six_eight. This flag is off by default.

v6.8.2: Fix packaging of gpu_ops

🔴 Bug fixes

• Fix incorrect packaging of thinc.neural.gpu_ops, introduced in v6.8.1.
• Fix bad data type in thinc.extra.search.MaxViolation, which caused segfaults on some platforms.

v6.8.1: Fix Windows support

✨ Major features and improvements

• Add new foreach layer combinator, which maps a layer across elements of a sequence.
• Add support for predict methods to more layers, for use during decoding.
• Improve correctness of batch normalization. Previously, some layers would force batch normalization to run in training mode, even during prediction. This led to decreased accuracy in some situations.
• Improved efficiency of Maxout layer.

🔴 Bug fixes

• Fix compiler flags for MSVC
• Remove unnecessary Chainer dependency. Now depends on Chainer's cupy package.
• Fix LSTM layer.
• Small bug fixes to beam search

v6.8.0: SELU layer, attention, improved GPU/CPU compatibility

✨ Major features and improvements

• Add SELU layer, from Klambauer et al. (2017).
• Add parametric soft attention layer, as in Yang et al. (2016).
• New higher-order function uniqued, which wraps layers giving them a per-batch cache.
• Improve batch normalization, by tracking activation moving averages.

🔴 Bug fixes

• Fix GPU usage in pooling operations.
• Add optimized code for extracting ngram features.
• Improve CPU/GPU compatibility.
• Improve compatibility of LinearModel class.

👥 Contributors

Thanks to @tammoippen for the pull request!

v6.7.3: Fix convolution on GPU

🔴 Bug fixes

• Convolution is now computed the same on CPU and GPU.

v6.7.2: Bug fixes to serialization

🔴 Bug fixes

• Make order of dicts stable when serializing model.

v6.7.1: Improve serialization

✨ Major features and improvements

• Temporarily revert change to CuPy.
• Improve efficiency of Adam optimizer.

v6.7.0: Fixes to serialization, hash embeddings and flatten ops

✨ Major features and improvements

• Add Model.to_bytes() and Model.from_bytes() methods, to support serialization that's compatible between Python versions.
• Remove code depending on Chainer, and instead depend explicitly on the new cupy subpackage, for simpler GPU installation.
• Improve accuracy for HashEmbed table, by using 4 conditionally independent keys.
• Support padding in flatten and with_flatten ops.
• Use the same hash function on both CPU and GPU, for model compatibility.

🔴 Bug fixes

• HashEmbed now returns correct results for arrays of length not divisible by 16.
• Provide .cu source files in the source distribution.
• Remove unnecessary allocations from the CPU maxout op.
• Fix issue #27: Remove Python2-specific code from setup.py.

v6.6.0: Improved GPU usage and examples

✨ Major features and improvements

• Add GPU kernels for max and mean pool using variable-length sequences.
• thinc.api.FeatureExtractor, for getting features from spaCy Doc objects.

🔴 Bug fixes

• Improve multi-device handling
• thinc.api.add now accepts a variable number of layers.
• Improve Residual class.

⚠️ Backwards incompatibilities

• Some of the example code may be out of date.

📖 Documentation and examples

• Add reader for WikiNER corpora.
• Add example for Twitter NER.
• Add Siamese network example.

v6.5.1: Improved linear class and Windows fix

✨ Major features and improvements

• Add hash kernel linear class.

🔴 Bug fixes

• Fix issue #22: Remove random_bytes method from Ops.
• Fix termcolor dependency.

📖 Documentation and examples

• Add IMDB to datasets.
• Add linear BOW example, using hash kernel.

👥 Contributors

Thanks to @rolando and @ogrisel for the pull requests!