Quantization of LSTM layers - a Technical White Paper

As announced before Christmas Imagimob AI supports quantization of LSTM layers. What does this mean and why is it important?

By Johan Malm, Imagimob, Ph.D and Product Owner

The Need for Quantization
LSTM stands for Long Short-Term Memory and is a vital part in Machine Learning when it comes to time-series data. This type of neural network building block has been used successfully in numerous state-of-the-art models that treat time-dependencies and can be credited with the success for solving the difficult problems in language modelling, translation, robotics, and gaming.

Quantization has to do with the actual implementation of the neural network on a digital computer and is important for the tiniest MCUs on the market that don’t have a floating-point unit (e.g., ARM’s M0-M3 cores).

Heuristic Approach to the Problem
LSTMs in particular are known to be hard to quantize due to their capability to remember features over long sequences of inputs, and this capability can amplify small errors. In this article we look at a way to attack this tricky problem. We show how full integer post-training quantization works and how one can go about to solve dependencies between parameters using a heuristic approach.

Imagimob’s Solution
In Imagimob AI, quantization of a pretrained model is done with the click of a button in a user-friendly user interface, which generates C code without any floating-point operations, ready for deployment.

Click here to download the Technical White Paper about Imagimob's approach to quantization.