Matrix Recurrent Units updated as attention alternative

View on Reddit

The MRU is a linear-time sequence architecture that transforms embeddings into a matrix representation, offering an alternative to attention. The update includes new implementations and experiments.