How To Write Content API Queries: Example Queries
Content API is a powerful tool for controlling and selecting the content that gets surfaced to readers. All Content API requests are performed in the context of a specific website. If your organization has multiple websites that it manages, your requests to Content API must specify which site will be rendering the content. By mandating a website on each request, Content API is able to gracefully handle content restrictions and circulations on your content.
Content API provides two types of endpoints: direct fetches and searching.
Direct Fetch
Direct fetches should be used when you know exactly which piece of content you want to surface, based on its Arc ID or Canonical URL. These endpoints can only return one piece of content.
GET https://api.{org}.arcpublishing.com/content/v4/_id=ABC123&website=the-herald
GET https://api.{org}.arcpublishing.com/content/v4/website_url=/sports/2021/03/15/team-wins-game&website=the-herald
Content API also provides endpoints that are specific to different content types, /content/v4/stories, /content/v4/galleries, /content/v4/videos. These are also direct fetch endpoints but narrow the retrievable content to just the specified type.
Searching Examples
The /content/v4/search endpoints of Content API allow for more complicated searches that return feeds of multiple content items rather than just one specific item. A few common use cases are listed in this document. For more detailed documentation, see Query Reference.
When searching for content, ensure that the query (q=) values are wrapped in quotes (do not wrap query keys). Some key values often contain certain special characters, which can cause query analysis issues resulting in inconsistent results. For example, a query for q=taxonomy.tags.slug:national-news will likely return different results compared to q=taxonomy.tags.slug:“national-news”.
Your use case for building a feed or searching for content may require accessing pages and pages of results. While Content API can support this, as of ElasticSearch 7, there is a hard limit of 10,000 documents in a single result set. We recommend adding a filter for a particular date range that will keep your result set under 10,000. Depending on your organization, the number of days that make sense may vary. By making a series of smaller queries, you can access all of the results and perform any required aggregation in the downstream system.
Note
All of the examples use the content/v4/search/published endpoint, which limits results to only published documents. If you want to retrieve unpublished documents, use content/v4/search?published=false. This retrieves the unpublished (draft) revision of the documents. Keep in mind that all published documents have an unpublished revision, so if you do not include the &published= parameter, you receive two copies of each story. Also remember using &published=false also returns the unpublished revisions of published documents.
A count of stories:
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=type:story&size=1&_sourceInclude=_id&track_total_hits=true
The track_total_hits parameter tracks the true total hit count, even if total count is above 10,000
A feed of stories:
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=type:story
A feed of stories, sorted by most recent and limited to 10:
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=type:story&sort=display_date:desc&size=10
A feed of stories in the Politics section:
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=type:story+AND+taxonomy.primary_section._id:"/politics"&sort=display_date:desc&size=10
A feed of stories in the Politics section that are tagged with “obama:”
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=type:story+AND+taxonomy.primary_section._id:"/politics"+AND+taxonomy.tags.slug:obama&sort=display_date:desc&size=10
A feed of stories in the Politics section limited to those set to display on a specific date range:
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=type:story+AND+taxonomy.primary_section._id:"/politics"+AND+display_date:[2021-01-01+TO+2021-01-05]&sort=display_date:desc&size=10
A feed of stories by a certain author:
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=type:story+AND+credits.by._id:john-smith&sort=display_date:desc&size=10
A feed of stories that have a video in the body:
In this example, you can sub video for any object that can appear in content_elements: image, gallery, reference, quote, etc.
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=type:story+AND+content_elements.type:video&sort=display_date:desc&size=10
A feed of stories, but I don’t actually need anything in content_elements so let’s make that response smaller:
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=type:story+AND+taxonomy.primary_section._id:"/politics"+AND+content_elements.type:reference&sort=display_date:desc&size=10&_sourceExclude=content_elements,related_content,additional_properties
A feed of stories filtered to a particular year of content:
This query uses DSL query language in the body= query param; the q= param is not used.
https://api.{org}.arcpublishing.com/content/v4/search?website={website_id}&body={"query":{"filtered":{"query":{"term":{"type":"story"}},"filter":{"bool":{"must":[{"range":{"created_date":{"gte":"2017-01-01","lte":"2017-12-31"}}}]}}}}}&size=100
A single story based on a search that may return more than one story:
In this example, even though the results of the search are greater than 1, we coerce the response to match that of the direct fetch endpoints listed above.
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=type:story+AND+taxonomy.primary_section._id:"/politics"&sort=display_date:desc&size=10&single=true
A single story, looked up by its source_id:
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=source.source_id:123456789&single=true
A feed of videos:
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=type:video&sort=display_date:desc&size=10
A feed of galleries:
https://api.{org}.arcpublishing.com/content/v4/search/published?website=the-herald&q=type:gallery&sort=display_date:desc&size=10