Less unnecessary information
Normal pages include menus, styling, and visual elements. A shorter text response could reduce material an AI tool needs to process, although fewer bytes do not automatically mean fewer model tokens.
The idea, without the jargon
Why might this work?
Think of a webpage like a brochure: people need the design and navigation, while software answering a question may only need the relevant facts and where they came from. RankGen tests whether a simpler, organized version helps.
Answers organized for understanding
WARP connects direct answers to related questions and supporting sources. The idea is to make each answer and its context easier to locate.
Evidence that can be checked
Evidence links let readers inspect supporting sources and give AI tools references they could check. Whether AI systems use them more effectively remains an open research question.
What this experiment can show today: The server can deliver two formats and compare their response sizes, timing, and matching answer/evidence IDs. It does not yet prove token savings, answer accuracy, or higher AI citation rates.
How the website works
One source. Two ways to read it.
You are viewing the regular website now. Software that specifically requests Markdown can receive a simpler text version. Both versions reuse the same important answers and supporting evidence.
Ask for a format.
A normal browser receives HTML. Software can explicitly ask for Markdown. The server checks the requested format rather than guessing whether a visitor is human or a bot.
Keep the facts together.
Direct answers, related questions, and evidence references come from one publisher-controlled source. Important claims stay consistent across the two formats.
Give each a useful view.
People keep the full visual experience. Markdown clients get an answer-focused text version. The live experiment compares what each request receives, not whether rankings improve.
A working delivery demonstration
Same ideas.
Different surfaces.
Switch between a familiar webpage and a simpler text format. This small preview illustrates the difference. The live Markdown link shows what the server actually sends.
Try from a terminal: curl -H "Accept: text/markdown" https://rankgen.ai/
Proof, not a promise
REAL HTTP RESPONSES / USER-INITIATED
The live protocol lab
Compare the formats.
See the results.
Click once to request this homepage in two formats: the regular webpage and a simpler text version. Compare actual response sizes, timing, and evidence IDs. This is a delivery test, not an AI citation test.
Regular webpage
GET / Accept: text/htmlAnswer-focused text
GET / Accept: text/markdownThe test requests this site's public homepage twice. It compares formats and WARP IDs, not model token counts, answer quality, or AI citations.
Delivery profiler / RankGen.ai
REQUEST-TIME FORMAT
Website / HTML
Not tested
Response pending
Browser view · — ms
Text / Markdown
Not tested
Response pending
Agent format · — ms
Body size comparison
Awaiting test
UTF-8 bytes before compression. Not LLM tokens.
WARP identifier consistency
Awaiting test
Compares answer and evidence IDs, not semantic equivalence.
Ready. The next numbers shown here will come from your browser's actual responses, not from a fabricated demonstration.
EXPERIMENTAL · Request timing varies by client and CDN · Not a crawler-citation benchmark.
WARP: Answer → Fan-out → Evidence
Answers are only useful when you can check them.
This page contains the same stable answer IDs and source references as the Markdown version. Expand each follow-up to inspect the evidence. These publisher-authored relationships are experimental WARP markup, not a claimed search-engine ranking factor.
What is adaptive content delivery for AI crawlers?
Adaptive content delivery is a server-side technique that offers different representations of the same substantive information according to an HTTP request. On this site, normal visitors receive an accessible HTML page; clients explicitly requesting text/markdown can receive a compact text representation.
[a1-f1] Are there two independent versions of the facts?
No. RankGen keeps one publisher-authored answer and evidence source. Its HTML and Markdown views are generated from the same content rather than maintained as competing pages.
[a1-f1-e1] Evidence: HTTP Semantics RFC 9110: content negotiation ↗How does the server decide which representation to return?
The request-time router checks the explicit HTTP Accept preference. An accepted text/markdown request receives Markdown; ordinary browser requests receive HTML. Vary: Accept signals that representations differ. The experiment disables page caching at the WordPress layer until end-to-end cache behavior is verified.
[a2-f1] Does naming Googlebot or another crawler automatically trigger a different message?
No. User-agent names are easily spoofed, and this demonstration does not send different claims to search crawlers. The meaningful format switch is initiated by explicit content negotiation, not by hiding information from humans.
[a2-f1-e1] Evidence: RFC 9110: Accept and Vary headers ↗What does WARP add beyond automatic HTML-to-Markdown conversion?
WARP, the experimental Web Answer Retrieval Protocol, organizes publisher answers into stable A nodes, related fan-out F nodes and linked evidence E nodes. The proposed advantage is selective retrieval of the smallest useful answer and supporting evidence rather than a wholesale conversion of every visible page element.
[a3-f1] Do AI search platforms currently understand WARP as a standard?
There is no demonstrated platform-level adoption. WARP is an experiment, not an industry standard or a ranking signal. RankGen compares it against normal HTML and generic Markdown and publishes only measured outcomes.
[a3-f1-e1] Evidence: WARP specification and limitations ↗Is serving HTML to people and Markdown to an agent considered cloaking?
Different formats are not inherently deceptive. The critical safeguard is keeping material facts, qualifications and supporting evidence equivalent and accessible. This experiment does not promise crawler-only benefits or withhold material information from a normal browser.
[a4-f1] Why keep the Google-search experience crawlable as HTML?
Google recommends server-side or static rendering over bot-specific dynamic rendering workarounds. A well-structured HTML page is a safe baseline for search discovery while explicit machine-readable alternatives remain optional.
[a4-f1-e1] Evidence: Google Search Central: dynamic rendering guidance ↗Does the Markdown version save tokens or increase AI citations?
A shorter retrieval payload can reduce transmitted bytes, but actual language-model token savings depend on what a retriever consumes. Higher citation rates are unproven. RankGen treats transfer size, tokenizer-specific input length, latency, answer fidelity and citation outcomes as separate measurements.
[a5-f1] What makes this different from Markdown for Agents?
Cloudflare offers HTML-to-Markdown conversion on request. RankGen is testing publisher-authored answer-and-evidence packaging and selective retrieval as a separate, measurable improvement hypothesis.
[a5-f1-e1] Evidence: Cloudflare: Markdown for Agents ↗The evidence standard
No cloaking.
No magic rankings.
Search engines and human readers should have access to equivalent material claims. RankGen explores different representations, not conflicting messages.
- Content parity. Pricing, qualifications, limitations and evidence cannot disappear from a special bot version.
- Search baseline. Googlebot continues to receive crawlable HTML; Markdown is an explicit client preference.
- Testable hypothesis. Token counts, bytes, retrieval accuracy and citations must be measured separately.
- Honest status. WARP is experimental; no AI platform adoption or citation advantage is asserted.
Inspect the real response
The best way to judge the idea is to request it.
Open the live Markdown representation, compare its answers with this HTML page, and inspect the cited evidence. This is a transparent engineering experiment, not a finished benchmark.