Imperial College London > Talks@ee.imperial > CAS Talks > Multi-precision training for CNNs

Multi-precision training for CNNs

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact George A Constantinides.

Large-scale Convolutional Neural Netowrks (CNNs) suffer from very long training times that can range from a hours to a couple days. One way to reduce the training time is to perform training at lower precisions as this improves both memory and computational bandwidth. However, training at low precisions suffers from losses in accuracy due to the problem of vanishing gradients. This work introduces the concept multi-precision training as a tool to combat this loss in accuracy. Multi-precision training starts the training process at a 2 to 4-bit dynamic fixed point representation, and over the training epochs increases precision up until 32-bit floating point (FP32). This work also proposes a strategy based on the gradients of the network that can be used to decide during run-time the best times to perform this change in precision.

This talk is part of the CAS Talks series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

 

Changes to Talks@imperial | Privacy and Publicity