Running a 4-Language Next.js 16 Site for Humans and AI Crawlers: hreflang, llms.txt, JSON-LD and IndexNow
I run ModernWebSEO, a small web design and SEO studio in Istanbul. Our own site is in four languages: Turkish (the main market, at the root), English (/en), Spanish (/es) and Arabic (/ar, right to left). The sitemap holds 700 URLs.
This article details technical SEO strategies for a multilingual Next.js 16 website, focusing on handling AI crawlers, hreflang implementation, and structured data for global content delivery.
What happened
The author describes their implementation of multilingual SEO techniques for a Next.js 16 site, including folder-based routing, custom hreflang mappings, AI-optimized robots.txt, llms.txt for AI agents, JSON-LD structured data, and automated IndexNow submissions.
Why it matters
For developers and SEO professionals managing multilingual sites, this article provides actionable insights into optimizing content delivery for both human users and AI crawlers, ensuring proper indexing and language-specific content visibility.
- Use explicit hreflang maps for translated slugs instead of URL prefix strategies
- Implement self-referencing canonical tags per locale to avoid content duplication issues
- Configure robots.txt with AI crawler identifiers and content usage signals
- Maintain honest lastmod dates in sitemaps for effective IndexNow updates
- Use JSON-LD @id references for unified structured data across multiple languages
The bigger picture
The described techniques improve search engine visibility for multilingual content while accommodating AI crawlers' requirements, providing a framework for maintaining consistent SEO practices across multiple languages.
The article provides a comprehensive technical guide for managing multilingual SEO in Next.js applications, emphasizing strategies that accommodate both human users and AI crawlers through structured data, proper routing, and search engine optimization practices.
Technical details
- Folder-per-locale structure with manual route tree management
- Hreflang maps using slug-based translation dictionaries
- Content-Signal headers for AI crawler permissions
- Markdown negotiation via middleware for llms.txt
- JSON-LD @graph with @id-based entity referencing
- IndexNow API integration with 24-hour lastmod filtering
FAQ
How does the site handle hreflang for translated slugs?
The site uses explicit slug mapping dictionaries to create reciprocal hreflang links, ensuring each translated page correctly references its counterparts in other languages.
What's the purpose of llms.txt in this implementation?
llms.txt provides context about the site's content and services to AI agents, including language-specific information to prevent misleading users about available content.
How are AI crawlers accounted for in the robots.txt configuration?
The robots.txt includes specific user-agent rules for AI crawlers and adds Content-Signal headers to specify permitted content usage for search and AI training.
What's the benefit of using JSON-LD with @id references?
This approach creates a unified structured data graph where entities are referenced by ID rather than duplicated, improving consistency and entity recognition by search engines.
How does the IndexNow implementation ensure accurate search engine updates?
The site uses honest lastmod dates in sitemaps and submits only recently modified URLs via IndexNow, maintaining trust with search engines while automating updates.
This summary and analysis were generated with AI from the original article and may contain errors. Read the original for the full story.
Read the full story
Originally published by Dev.to AI on Oct 10, 2026.
Comments
Sign in to join the discussion.