Log inImperial users Other users No account?Information onFinding a talk Adding a talk Syndicating talks Who we are Everything else |
Multi-precision training for CNNsAdd to your list(s) Download to your calendar using vCal
If you have a question about this talk, please contact George A Constantinides. Large-scale Convolutional Neural Netowrks (CNNs) suffer from very long training times that can range from a hours to a couple days. One way to reduce the training time is to perform training at lower precisions as this improves both memory and computational bandwidth. However, training at low precisions suffers from losses in accuracy due to the problem of vanishing gradients. This work introduces the concept multi-precision training as a tool to combat this loss in accuracy. Multi-precision training starts the training process at a 2 to 4-bit dynamic fixed point representation, and over the training epochs increases precision up until 32-bit floating point (FP32). This work also proposes a strategy based on the gradients of the network that can be used to decide during run-time the best times to perform this change in precision. This talk is part of the CAS Talks series. This talk is included in these lists:
Note that ex-directory lists are not shown. |
Other listsAndrea Picciau's list TalksOther talksAnalysis and Synthesis of Floating-Point Routines An IP provider’s perspective on functional safety The Challenge of Economic Wave Energy - A Control Perspective What's inside Xilinx's 7nm Versal Adaptive Compute Acceleration Platforms |