What's New in Migration Center

Release dates

Release summary

This release contains small patches to dependencies and adds a new API to allow you to query the completion estimate and amount of time remaining for a content stream. These estimations are available only for existing supported ANS types and is not available to source content uploads at this time. For more information, see Migration Center API.

You can now programmatically send smaller batches of content query for a completion date and use that date to programmatically send more content. These estimations can help with projecting dates for planning and scheduling workloads and resources for migrations.

Users affected

Developers

Action required

None

Release dates

Production - Monday, July 15, 2024

Release summary

Arc XP is excited to announce that Migration Center is now offering an extended set of tools to help customer developers who want a more streamlined process for getting content into Arc XP for both large one-time data migrations and for ongoing inbound data feeds.

Some key benefits include: 

Reduced Developer Time

Lower Risk of Data Errors

Value Adds

Users affected

Developers, Migration Center users

Action required

To learn more about how Migration Center Inbound Content Pipelines can help you, contact Arc XP Customer Support.

Release dates

Release summary

This release improves stability around retry scenarios for story content. 

Users affected

Developers

Action required

None

Release dates

Release summary

The /migrations/v3/report/summary API now supports the query parameter limit to allow callers to return a smaller result set for their records. 

Users affected 

Developers

Action required

None 

Release dates

Release summary

Users affected

Developers

Action required

None

Release dates

Release summary

This release introduces inbound content pipelines, which are a set of capabilities that let you self-configure inbound content feeds from external content sources, such as legacy CMS systems or third-party content providers (like The Associated Press and Getty Images). This new feature set extends Migration Center’s existing loading capabilities to help you more quickly map external content formats from external sources to Arc XP’s internal format by generating content maps and then using our extended Migration Center APIs to transform and load content at scale.

Whether you are a customer developer, an Arc XP development partner tasked with migrating large volumes of content to Arc XP from a legacy CMS, or a client who needs to create ‘live ingest’ pipelines of content from third-party providers, Arc XP’s inbound content pipelines provide a toolkit to help you create those integrations with more speed and reliability than writing your integrations directly to Draft API. Additionally, the toolkit is designed to be completely self-service, meaning you have control over your integration without needing input or assistance from Arc XP development staff.

Users affected

Developer beta participants

Action required

None

Launch dates

Release summary

In this release, we upgraded components within Migration Center to meet the demands of our growing customer base. Changes include replacing default messaging components with auto-scaling elements, so messages generated by ingested documents are always delivered to the content queues without failure. Additionally, we updated retry mechanisms and the application configuration to help improve the stability of the migration pipeline.

Also in this release, we deployed a new batch processing framework to service long-running processes separately and asynchronously. At this time, we are not shipping any customer-facing services that leverage the new batch framework, but we will be releasing new APIs, including a redesigned Reset and Counts API for public use, after an internal assessment of the framework. More details will be available at the time of the release.

Users affected

All changes in this release affect all Migration Center customer environments.

Action required

None

Hotfixes

3.4.1

Release dates

Bug fix summary

We consolidated the number of internal system calls required to migrate stories with many circulations. This fix has the dual benefit of optimizing the performance of the story migration queue as well as further mitigating the frequency of 429 errors that the Arc XP platform throws.

Users affected

All the bug fixes in this patch release are applied to all Migration Center customer environments.

Action required

None

3.4.2

Release dates

Bug fix summary

Users affected

All the bug fixes in this patch release are applied to all Migration Center customer environments.

Action required

None

3.4.3

Release dates

Bug fix summary

Users affected

All the bug fixes in this hotfix release are applied to all Migration Center customer environments.

Action required

None

3.4.4

Release dates

Bug fix summary

Migration Center now correctly logs errors caused during circulations. Previously, the application ignored such errors, leading to them failing silently without communicating to the end user. With this update, as with any error status, the /report/detail API response body returns circulation errors within the errorMessage property, along with the status set to FailStory.

For more information, see Using Migration Center to circulate documents during a migration.

Users affected

All the bug fixes in this hotfix release are applied to all Migration Center customer environments.

Action required

None

Launch dates

Bug fix summary

Users affected

All the bug fixes in this patch release will be applied to all Migration Center customer environments.

Action required

None

Release dates

Sandbox: Monday, December 12, 2022

Production: Friday, December 16, 2022

Improvements

Webhook integration for auto-notification of migration status changes

Migration Center is now able to send notifications to customer applications when a document has been fully migrated to Arc. This occurs when you register a webhook URL with Migration Center's Webhook registration service. If the webhook setting is enabled for a tenant, Migration Center sends a message to the URL whenever an ingested document of any content type reaches one of the following statuses:

The webhook registration service lets customers create and update a webhook with Migration Center. Only one webhook is permitted in a customer's Arc XP environment. You can create and update a webhook through HTTP calls as the following code samples illustrate.

Create a webhook in Migration Center

curl --location --request POST 'https://api.{org}.arcpublishing.com/migrations/v3/webhook' \
--header 'Authorization: Bearer <INSERT YOUR ARC BEARER TOKEN HERE' \
--header 'Content-Type: application/json' \
--data-raw '{
    "endpoint":"https://api.{org}.arcpublishing.com/notify/stage/bug_bush_3_3",
    "authToken":" <INSERT YOUR WEBHOOK BEARER TOKEN HEREl",
    "isEnabled":true,
    "headers":{
        "Accept-Language":"en-US"
    },
    "subscriber": "sandbox_webhook_1"
}'

The POST request sent to the /migrations/v3/webhook endpoint requires an authorization token and a Content-type header as is the case with any other API endpoints within Migration Center. The request payload must contain the following properties:

Update a webhook in Migration Center

curl --location --request PATCH 'https://api.{org}.arcpublishing.com/migrations/v3/webhook/1' \
--header 'Authorization: Bearer INSERT YOUR ARC BEARER TOKEN HERE' \
--header 'Content-Type: application/json' \
--data-raw '{
    "isEnabled":false
}'

You can update the webhook through a PATCH call to the webhook registration service as shown in the previous sample and passing in the ID returned in the API response to the POST /migrations/v3/webhook call. This process lets you update your authorization tokens when they are recycled or disable notifications from Migration Center.

Get a list of all webhooks in Migration Center

curl --location --request GET 'https://api.{org}.arcpublishing.com/migrations/v3/webhook' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer INSERT YOUR ARC BEARER TOKEN HERE' \
--data-raw ''

And the response looks something like this:

{
    "hasMore": false,
    "webhooks": [
        {
            "id": 1,
            "endpoint": "https://api.{org}.arcpublishing.com/migrations/v3/mywebhookapi",
            "headers": {
                "Content-Type": "application/json"
            },
            "isEnabled": true,
            "createDate": "2022-12-08T17:33:56.990Z",
            "updateDate": "2022-12-08T17:44:20.804Z",
            "subscriber": "test"
        }
    ]
}

For security reasons, the response does not include the authorization token.

Using the subscriber to link a POST /ans request to a specific webhook

In the case where the client application maintains multiple webhooks with Migration Center, the subscriber name can be utilized to match specific migration records to webhooks. Passing the subscriber name in the POST /ans or POST /ans/bulk API call will link the ANS document(s) in the POST call to a particular webhook integration on the client side. Migration Center will call the webhook that matches the subscriber name.

{
    "tags": {
        "webhookSubscriber": "sandbox_webhook_1"
    },
    "ANS": {...}
}

Notification message by Migration Center to the webhook

If a webhook has been registered and its isEnabled setting is set to true, Migration Center will POST a message to the webhook URL using the authorization token and HTTP headers provided during registration, whenever an ingested document reaches a terminal status. The list of statuses can be found at the beginning of this section. The POST message payload will look something like this and will include the ansId, ansType and status of the migrated document, along with the document's metadata.

"record": {
            "id": 229338,
            "sourceLocation": null,
            "sourceAdditionalProperties": null,
            "ansId": "T3GBOT63EBA4JGFU4RPTCQPK4Q",
            "ansType": "author",
            "ansLocation": "{org}/stage/author/7fc2b6cb-badb-438a-8d13-8ec8d9fde628/ans.json",
            "status": "Circulated",
            "website": null,
            "operations": null,
            "circulationLocation": "{org}/stage/author/7fc2b6cb-badb-438a-8d13-8ec8d9fde628/circulations.json",
            "createDate": "2022-12-07T19:49:00.157Z",
            "updateDate": "2022-12-07T19:49:00.157Z",
            "errorMessage": null,
            "priority": "historical",
            "arcAdditionalProperties": null,
            "groupId": null
        }

PATCH inventory-data API endpoint supports ansId and ansType as request parameters

The PATCH /inventory-data API now accepts ansId and ansType as request parameters. The following values can be updated when ansId and ansType are used. The ansId field cannot be updated through the request body when the request parameter also contains ansId.

{
  "sourceAdditionalProperties": {},
  "groupId": "string",
  "website": "string",
  "arcAdditionalProperties": {}
}

The request should contain either sourceId and sourceType OR ansId and ansType to be passed but not all parameters in the same request. If a request contains all of the parameters, only ansId and ansType will be used to look up the record in Migration Center.

Bug Fixes

This release fixes an issue to retry previously failed galleries due to missing images. This improvement also ensures that records are always updated to the most current status and reduces the incidence of silent failures. For example, a silent failure can occur when a record fails to migrate to Arc XP, but its status remains Queued due to an unhandled exception thrown during migration.

Roll-out dates

Sandbox: Wednesday, November 16, 2022

Production: Wednesday, November 30, 2022

Improvements

Update arcAdditionalProperties using the PATCH API endpoint

arcAdditionalProperties can be now updated using the PATCH /inventory-data API. This eliminates the need to pass in the entire ANS object a second time to make changes to the ANS metadata. For example, to publish a story that was previously ingested, the client can now submit a request to the PATCH /inventory-data API as shown below:

curl --location --request PATCH 'https://api.{{org}}.arcpublishing.com/migrations/v3/inventory-data?sourceId=123_abc&sourceType=test_record' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer' \
--data-raw '{
    "arcAdditionalProperties": {
        "story": {
            "publish": true
        }
    }
}'

A successful response would look like this:

{
    "success": true,
    "message": "Inventory data has been accepted."
}

Ingestion API response returns status information of ingested record

The POST /ans API endpoint has been updated to return information about the state of the resource (ANS record) through hypermedia in the response body. The updated response body format conforms to the HATEOS principle of the REST application architecture. A sample response is shown below:

{
    "message": "ANS content has been accepted and added to the queue for migration.",
    "success": true,
    "_links": {
        "self": "https://api.{{org}}.arcpublishing.com/migrations/v3/report/detail?ansId=&ansType="
    }
}

In the above example, the HATEOS hypermedia “_links.self” informs clients of the URL to fetch the status of the ANS record in Migration Center.

Pass URL collision behavior when migrating stories

Users can pass options in the API request to control how the Arc XP Publishing Platform handles requests to register or generate URLs that already exist. The option is defined through a new property in arcAdditionalProperties.

{
    ANS:...,
    "arcAdditionalProperties": {
        "story": {
            "circulation": {
                "collision_behavior": "increment|fail (default)"
            }
        }
    }
}

Additional filters in Report Summary API

The GET /report/summary API endpoint now accepts two optional query parameters to sort items inside the source and/or ANS record collection returned in the response. The parameters are shown below.

Below is an example CURL request with the new parameters. While the sort parameter values are case-sensitive, the sortOrder parameter values are not.

curl --location --request GET 'https://api.{{org}}.arcpublishing.com/migrations/v3/report/summary?sort=createDate&sortOrder=DESC'

Bug Fixes

Roll-out dates

Sandbox: Friday September 16, 2022

Production: Monday September 19, 2022

Bug Fixes

Roll-out dates

Sandbox: Tuesday, July 26, 2022

Production: Wednesday July 27. 2022

Bug Fixes

Roll-out dates

Sandbox: Thursday, June 30, 2022

Production: Tuesday, July 12, 2022

Improvements

New request parameters with the Count API

priority is now an optional query parameter that can be passed in the /report/status/count request to filter the count by document priority. Accepted values are live or historical (default). documentType is now an optional query parameter that can be passed in the /report/status/count request to filter the count by document type. Accepted values are source or ans (default). An example is shown below:

https://api.sandbox.{org}.arcpublishing.com/migrations/v3/report/status/count?documentType=ans&priority=live

Content size limit increased in the Bulk Ingestion API

The maximum allowed items that can be passed within a single request to the Bulk Ingestion API has been increased to 50 which enables users to migrate up to 50 ANS documents using a single API request. The content property can now accept up to 50 items in the array. If you want to optimize bulk ingestion, we recommend continuing to limit requests to 25 items. 

{
    "defaultPriority": "live",
    "defaultWebsite": "defaultWebsite",
    "defaultCorrelationId": "test_batch",
    "content": [
        {
            "ANS": {
            },
            "ANS": {
            },
            .
            .
            .
            <!-- upto 50 ANS documents allowed -->
        }
    ]
}

New validation for circulation and content operations

The ingestion API now validates the circulation and content operations objects submitted in the request. If any of the required properties are missing or if the format violates the expected JSON schema, the API will return a user friendly error message. An example error message is shown below.

 {
    "success": false,
    "message": "[{\"instancePath\":\"/circulations/0\",\"schemaPath\":\"#/properties/circulations/items/oneOf/0/required\",\"keyword\":\"required\",\"params\":{\"missingProperty\":\"website_id\"},\"message\":\"must have required property 'website_id'\"},{\"instancePath\":\"/circulations/0\",\"schemaPath\":\"#/properties/circulations/items/oneOf\",\"keyword\":\"oneOf\",\"params\":{\"passingSchemas\":null},\"message\":\"must match exactly one schema in oneOf\"}]"
}

Upon inspecting the message closely, it shows that the circulations object is missing a required property called website_id.

\“keyword\“:\“required\“,\“params\“:{\“missingProperty\“:\“website_id\“}

Additional properties for ingesting videos

The ingestion API now accepts useLastUpdated for videos which is passed to Video Center. The expected format for this request is shown below.

{
    "ANS": {
        ...
    },
    "arcAdditionalProperties": {
        "video": {
            "useLastUpdated": true
        }
    }
}

Bug Fixes

Roll-out dates

Sandbox: Friday, April 1, 2022

Production: Monday, April 25, 2022

Migration Center v3.0 is now live. In addition, we are excited to reintroduce our UAT process flow. With UAT, you will have access to the latest release as soon as it is coded, tested, and deployed in the sandbox environment. During UAT, we welcome all feedback in order to incorporate any additional changes to the platform.

Enhancements

Bug Fixes

Planning an Upgrade

Coming Soon

Migration Center 3.1 Release (ETA May 2022)

Our next scheduled release will be backwards compatible with 3.0 and customers who have integrated with Migration Center 3.0 do not have to update their API clients to take advantage of the following platform improvements coming in 3.1.