98.7% Accuracy on Length Generalization: What RewriteNets Reveals About Transformers

According to the paper "RewriteNets: End-to-End Trainable String-Rewriting for Generative Sequence Modeling" by Harshil Vejendla, explicit string rewriting rules achieve 98.7% accuracy on SCAN length generalization while maintaining linear computational complexity. This means our assumptions about needing quadratic attention for systematic reasoning may need rethinking.