Skip to main content

AWS Kinesis Streams

Arc XP supports customers who need to receive event streams from our content platform in a variety of methods. One of the most advanced methods is integrating with Amazon Kinesis Data Streams.

Event streams provide a stream of events instead of individual requests and responses common in API integrations, or even webhooks. Event streams are better when you are dealing with a high volume of real-time messages, such as every update to every story in the CMS.

Note

Arc XP recommends the Integration Framework (IFX) as the preferred method of connecting to other systems. While it doesn’t support streams, the events are much cleaner, and the development framework is much more powerful in allowing you to choose the events you need on an application-by-application basis.

If you want to use Kinesis Streams, you must host a Kinesis “client” that reads the stream and sends the data to your application. AWS has example code and documentation for this.

platform_kinesis.png

The Event Stream product allows you to create up to four clients for your stream. Each client runs in parallel and receives all messages. Some reasons for using different client connections include:

  • You have multiple applications (analytics, archiving, alerting).

  • You have a production/live connection but want to try new logic using a sandbox/test connection.

  • You want to do parallel reads/processing. Note that there is a special feature in Amazon Kinesis for “fan-out” connections. We don’t support this in our standard configuration.

Note

Some partners of Arc XP have integrations that are based on Event Streams. When you work with these partners, they may request that you enable the feature in order to allow them to access your information. We encourage you to set up individual streams per partner as a way to ensure security, because each stream has its own security configuration in the Arc XP platform. Using a connection to the same stream isn’t optimal.

Also note the following items:

  • Each stream sends events from a specific "topic". These include:

    • Content API - normalized story, video, and collection events

    • Stories - normalized story events only

    • Authors - author events only

    • Collections - collection events only

    • Images - image events only

    • URLs - story url events only

    • Video - video events only

  • Starting in June 2025, Event Streams become a paid, optional service for new Arc XP customers. The cost is based on the number of streams and if special customer hosting is needed.

Learn more:

In this section: