API Documentation — llmstxts.com

Overview

The llmstxts.com API provides programmatic access to our domain scanning and AI access intelligence platform. Use our API to integrate domain audits, llms.txt analysis, and adoption statistics into your applications.

Base URL:

https://llmstxts.com/api/v1

Authentication

API requests require authentication using an API key. Include your API key in the request header:

Authorization: Bearer YOUR_API_KEY

Note: API keys are currently available by request. Contact us to obtain an API key.

Rate Limits

API requests are rate-limited to ensure fair usage:

Free tier: 100 requests per hour
Pro tier: 1,000 requests per hour
Enterprise: Custom limits available

Rate limit information is included in response headers:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 95
X-RateLimit-Reset: 1640995200

Endpoints

GET /audit/{domain}

Audit a domain for llms.txt presence and robots.txt AI bot directives.

Parameters

domain (required) - The domain to audit (e.g., "example.com")

Example Request

curl -H "Authorization: Bearer YOUR_API_KEY" \
  https://llmstxts.com/api/v1/audit/openai.com

Example Response

{
  "domain": "openai.com",
  "grade": "A+",
  "summary": "Excellent AI access transparency",
  "llms_txt": {
    "found": true,
    "size": "2.4 KB",
    "content": "# OpenAI\n> ..."
  },
  "robots_txt": {
    "blocked_bots": [],
    "allowing": true
  },
  "checks": [
    {
      "label": "llms.txt file present",
      "detail": "File found at /llms.txt",
      "pass": true
    }
  ],
  "scanned_at": "2026-01-15T10:30:00Z"
}

GET /statistics

Get aggregated adoption statistics and trends.

Example Request

curl -H "Authorization: Bearer YOUR_API_KEY" \
  https://llmstxts.com/api/v1/statistics

Example Response

{
  "total_domains": 2417892,
  "llms_txt_adoption": 18.3,
  "ai_blocking_rate": 34.7,
  "avg_score": "B+",
  "updated_at": "2026-01-15T10:00:00Z"
}

GET /statistics/tlds

Get adoption statistics grouped by top-level domain (TLD).

Example Response

{
  "tlds": [
    {
      "tld": ".com",
      "adoption": 22,
      "blocking": 35,
      "total_domains": 892000
    },
    {
      "tld": ".org",
      "adoption": 18,
      "blocking": 28,
      "total_domains": 645000
    }
  ]
}

GET /statistics/bots

Get blocking rates for different AI bots.

Example Response

{
  "bots": [
    {
      "name": "GPTBot",
      "blocking_rate": 38
    },
    {
      "name": "Google-Extended",
      "blocking_rate": 31
    }
  ]
}

Error Handling

The API uses standard HTTP status codes to indicate success or failure:

200 OK - Request successful
400 Bad Request - Invalid request parameters
401 Unauthorized - Missing or invalid API key
404 Not Found - Resource not found
429 Too Many Requests - Rate limit exceeded
500 Internal Server Error - Server error

Error Response Format

{
  "error": {
    "code": "RATE_LIMIT_EXCEEDED",
    "message": "Rate limit exceeded. Please try again later.",
    "retry_after": 3600
  }
}

SDKs and Libraries

Official SDKs and community libraries are coming soon. For now, you can use any HTTP client to interact with our REST API.

Coming soon: Python, JavaScript, PHP, and Go SDKs

Get Started

Ready to integrate our API? Contact us to request an API key and get started.

Request API Key Try Web Interface