Best-Paper Award at the 27th International Conference on Artificial Neural Networks2018 November 20
Dr Joeran Beel and Mark Collier
ICANN 2018 Best Paper Award – Implementing Neural Turing Machines
Above: Mark Collier, who presented the paper Implementing Neural Turing at the 27th International Conference on Artificial Neural Networks (ICANN 2018)
Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. Neural Turing Machines have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A number of open source implementations of Neural Turing Machines exist but are unstable during training and/or fail to replicate the reported performance of NTMs. This paper presents the details of our successful implementation of a Neural Turing Machine. Our implementation learns to solve three sequential learning tasks from the original NTM paper. We find that the choice of memory contents initialization scheme is crucial in successfully implementing a Neural Turing Machine. Networks with memory contents initialized to small constant values converge on average 2 times faster than the next best memory contents initialization scheme.
Posted by: Catherine O'Connor, Head of External Relations, School of Computer Science and Statistics, Trinity College Dublin.
catherine.oconnor at tcd.ie