Sitecore CDP data lake export service
Are you looking to transfer your data from Sitecore CDP with Azure Synapse? This blog post will provide step-by-step instructions on copying your Sitecore CDP data from an AWS S3 bucket to Azure Data Lake using Azure Synapse and Sitecore data lake export service.
This is a fully no-code implementation that can be completed in just under 30 minutes, with the main focus on unlocking Sitecore CDP capability around AI, Custom dashboards, and implementation of Data Mesh.
Configure permissions in AWS:
- Using IAM, Create new policy, follow recommendations listed here to set optimal level of access:
- Create new user, apply new policy
- Create Access Key and save it (download file)
Configure Azure Synapse
- Create Linked Service, using Access key form prevous Step
- Create AWS and Azure Data Lake datasets
- Create Copy Pipeline
- Configure Seververless Pool (Spark or Dedicated SQL pool should be used to build Data product, Serverless should be used for smaller datasets or to analyze data structures)
- Create External Table
- Query Data using SQL notations
Sitecore CDP export service schema: https://doc.sitecore.com/cdp/en/developers/api/index-en.html#UUID-d7b5b76b-bde8-c468-1b74-d8c15734e040
Getting started with Azure Synapse: https://learn.microsoft.com/en-us/azure/synapse-analytics/get-started