IceBERT2 min read
Beyond IceBERT: Open-Data ModernBERT Encoders for Icelandic from 12M to 1B Parameters
IceBERT, the reference encoder for Icelandic, found that a cleaned web crawl is enough to train a strong model and that starting from a multilingual model can help on some tasks. We revisit both findings with the ModernBERT architecture.
By Hafsteinn EinarssonarXiv - cs.CL (NLP)
Read the full story
Originally published by arXiv - cs.CL (NLP) on Oct 28, 2026.
Comments
Sign in to join the discussion.