Memory Networks 2015

Back to Research-Papers

datascience #machinelearning

Facebook AI research. Many machine learning models lack read/write capability to long term memory

Memory Networks

Memory Networks have m, an array of objects, and four learned components:

A memory network takes some input x, x is a word, sentence, audio, image

  1. convert x to internal representation: I(x)
  2. update memory with generalization: m'i = G(mi, I(x), m), forall i
  3. compute output features o given the new input and memory: o = O(I(x), m')
  4. decode the output features o to give the response: r=R(o)

Components of the memory network can be composed of other learning algorithms.

Memory network applied to a set of problems

LSTM looks at a sequence, are questions a sequence?

A MemNN Implementation for Text

One implementation of a memory network where the components are neural networks

LSTM

Specific RNN architecture, designed to learn from experience to classify and predict time series

Pointer Network

Neural Turing Machine

Coupling neural networks with external memory resources