AI Access Policy

Purpose

Date Time Encyclopedia is a technical reference built around structured, modular, and machine-readable date, calendar, time zone, and timestamp content.

Because the site receives automated traffic from AI systems and search engines, this policy defines the conditions under which crawlers may access Date Time Encyclopedia.

Support For Pay Per Crawl

Date Time Encyclopedia supports Cloudflare's Pay Per Crawl model and is preparing its access controls for paid AI crawling when the protocol is available for this zone.

After Pay Per Crawl is active, AI crawler access to site content must occur through authorized Cloudflare channels and may require compensation for successful crawl access.

Unpaid AI Crawling

Unpaid AI crawling is not authorized as a permanent access model for training, enrichment, retrieval, or commercial AI use.

Until Pay Per Crawl enforcement is active, recognized official AI crawlers may temporarily access public content so semantic indexing, citations, and dataset refresh workflows can continue without interruption.

Recognized AI Crawlers

The currently recognized AI crawler user agents include GPTBot, ClaudeBot, PerplexityBot, Bytespider, Amazonbot, Applebot, Meta-ExternalAgent, and Google-CloudVertexBot.

These crawlers must identify themselves with a transparent user agent, respect reasonable request rates, avoid aggressive scraping, and must not impersonate another crawler or browser.

Recognized Search Crawlers

Search indexing crawlers such as Googlebot, Bingbot, Applebot, and DuckDuckBot are treated separately from AI crawler access because their primary role is discovery and search indexing.

Search crawlers must continue to identify themselves accurately, respect technical controls, and comply with published crawler guidance.

Unknown Or Non-Compliant Crawlers

Unknown AI crawlers, incomplete user agents, empty user agents, generic scraping clients, and crawlers that imitate official services may be challenged, rate limited, blocked, or reviewed for further action.

Cloudflare logs, AI Crawl Control, bot verification signals, and traffic analytics may be used to distinguish official crawlers from spoofed or non-compliant traffic.

Technical Requirements

Crawler operators must use a stable and transparent User-Agent header, provide a meaningful product identity, respect robots.txt, and avoid bypassing rate limits or security controls.

Automated clients must not use deceptive identities, rotate user agents to evade policy, overload origin resources, scrape private or restricted paths, or ignore Cloudflare access decisions.

Future Enforcement

When Pay Per Crawl is active, unauthorized AI crawling may receive a payment-required response, a block response, a challenge, or another Cloudflare-managed enforcement action.

Date Time Encyclopedia may update this policy, Cloudflare rules, robots.txt references, crawler lists, rate limits, and access requirements as crawler behavior and Cloudflare capabilities evolve.

Contact

Crawler operators, AI companies, and search providers can request access clarification, licensing discussions, or technical review by emailing ai-access@datetimeencyclopedia.com.

Email Directory

AI access and crawler policy: ai-access@datetimeencyclopedia.com