Content API Rate Limit Increases
The Content API enforces rate limits to safeguard the underlying Arc XP infrastructure from potential failure caused by unforeseen spikes in customer website traffic, including instances of bot crawling or incorrect implementations affecting content retrieval.
Content API rate limits start at a default level of 30 requests per minute. This default limit is set by Arc XP when your organization is provisioned on the Arc XP platform. It applies to all Content API endpoints, including fetching individual documents or collections, and searching or scrolling across several documents using the /search or /scan endpoints.
Once your website is live on Arc XP, these default limits are adjusted after a period of analyzing your website's traffic. If higher rate limits are necessary to handle the load without causing HTTP 429 errors, they are applied accordingly.
Note
This review and adjustment process is standard practice for all Arc XP customers, and you do not need to request an adjustment.
Important information about rate limit increases:
Rate limit increases granted will vary depending on your website's traffic patterns to ensure optimal performance.
If unexpected or unidentifiable traffic is detected by Arc XP, we may refrain from increasing the rate limit. In such cases, clients will be promptly notified. As part of our shared responsibility model, clients are expected to address any issues stemming from changes on their end.
However, there may still be cases where the adjusted limits don't match your website's request patterns. If this happens, we encourage you to notify Arc XP about the discrepancy. We're happy to accommodate and grant the necessary adjustment to the Content API rate limit.