Attention is not quite all you need

Attention is not quite all you need

While working on MacGraph, a neural network that answers questions using knowledge graphs, we came across a problem: How to tell if something’s not present in a list.

We (Octavian.AI) have developed a novel and robust neural network solution to detecting if an item is present in a list. We've created this as an extension of the popular attention mechanism.

F9706b4491576b6d61563b3256d31a79?s=128

Ashwath Salimath

January 05, 2019
Tweet