IdeaShift-X

IdeaShift-X tests idea-provenance detection outside English. It holds about 10,000 human-written web pages in 24 languages from FineWeb2. Each page has two AI versions in the same language, written by Gemini 3.7 Flash:

• Level 0: written from a brief naming only the document type, so the ideas are the AI's.
• Level 5: written from the page's full role-labelled outline with an instruction to add nothing, so the ideas are the page's and the prose is the AI's.

A detector should not flag the human pages. It should flag level 0 for AI ideas. An idea detector should not flag level 5; a prose detector should.

Construction

The dataset merges two builds.

• 9-language build: Arabic, Chinese, German, Indonesian, Japanese, Russian, Spanish, Tamil, Urdu. About 610-740 pages per language.
• 24-language build: adds 15 languages at about 250 pages each.

Pages were format-classified with WebOrganizer's annotation prompt on an LLM. Only pages in the eight long-form formats IdeaLens was trained on were kept, some of them force-fitted to the closest format. Outlines are always extracted in English, whatever the page's language.

Splits

Split	Rows	What it holds
source	10,160	the human-written FineWeb2 pages
level_0	10,152	the AI version written from a brief naming only the document type (AI ideas)
level_5	10,116	the AI version written from the page's full outline (human ideas, AI prose)

Rows in different splits share seed_id; a few pages lack one of their AI versions. language selects a language, e.g. load_dataset("rishanthrajendhran/IdeaShift-X", split="level_5").filter(lambda r: r["language"] == "fra_Latn").

Fields

Field	Description
id	<language>#<FineWeb2 id> for human pages; …|level0 / …|level5 for AI versions
seed_id	the human page the row belongs to
language, language_name	FineWeb2 language code and name
build	9-language or 24-language
level	null for the human page; 0 or 5 for AI versions
kind, prose_author, idea_author	what the row is and who wrote its prose and ideas
generator	gemini-3.7-flash for AI versions
format	the page's format (WebOrganizer taxonomy, one of the eight long-form formats)
words, text	the document
seed_crawl_dump, seed_crawl_date	Common Crawl snapshot and crawl date of the human page
system_prompt, prompt	the exact generation request of a level-0 document (system_prompt is always "You are a skilled writer."); empty for the human pages and for level 5
outline	the English, role-labelled outline of this document as IdeaLens scored it (JSON string: document_description, global_themes, items of role and content)
idealens_p_ai, proselens_p_ai, pangram4_p_ai	detector P(AI) as reported in the paper (IdeaLens on the outline, ProseLens and Pangram 4 on the document)

Caveats

• URLs are not included. The FineWeb2 id identifies each page in the source dataset.
• Outlines are the untreated outlines each document was scored on, not the paraphrased outlines used for training. The level-5 prompt is assembled from a template for each document and was not stored, so system_prompt and prompt are empty there.
• Scores are stored at full precision. Rounding them ties the very small P(AI) values and moves the AUC. pangram4_p_ai is empty for a few documents Pangram returned nothing for.
• Flag a document at the 1% global cut, as in the paper: IdeaLens when idealens_p_ai is above 0.86288, Pangram 4 when pangram4_p_ai is at least 0.5, ProseLens when proselens_p_ai is above 0.39461.
• Outlines and page text can contain web addresses or e-mail addresses that were on the original page.

Scoring

The idealens package (PyPI) runs IdeaLens, ProseLens and the other released detectors on these documents: pip install idealens, then idealens run on a JSONL file with a text field.

All IdeaLens models and datasets are in the IdeaLens collection.

License

The dataset is released under CC BY-NC-SA 4.0: free to share and adapt for non-commercial purposes, with attribution and under the same license. The human pages come from FineWeb2 (ODC-By 1.0) and remain subject to the terms of their original sources and of Common Crawl. The AI versions were produced with Google Gemini models.

Citation

@article{idealens2026,
  title         = {IdeaLens: Detecting AI Ideas in Long-form Writing},
  author        = {Rajendhran, Rishanth and Choi, Minjoon and Russell, Jenna and Namuduri, Ramya and B{\"o}l{\"o}ni-Turgut, Deniz and Karpinska, Marzena and Wieting, John and Iyyer, Mohit},
  journal       = {arXiv preprint arXiv:2610.06778},
  year          = {2026},
  eprint        = {2610.06778},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2610.06778}
}