Know a great AI tool or AI news source? Submit it to AiByts
IceBERT2 min read

Beyond IceBERT: Open-Data ModernBERT Encoders for Icelandic from 12M to 1B Parameters

IceBERT, the reference encoder for Icelandic, found that a cleaned web crawl is enough to train a strong model and that starting from a multilingual model can help on some tasks. We revisit both findings with the ModernBERT architecture.

By Hafsteinn EinarssonarXiv - cs.CL (NLP)

Read the full story

Originally published by arXiv - cs.CL (NLP) on Oct 28, 2026.

Read on arXiv - cs.CL (NLP)(opens in a new tab)

Comments

Sign in to join the discussion.

No comments yet — be the first to share your thoughts.