Adapting Linear-Time Architectures for Tabular In-Context Learning

Tabular foundation models achieve strong performance by conditioning on labelled examples in context, but softmax attention limits their use on large datasets.