[ad_1]
Enterprise intelligence (BI) and IT operations (BIOps) groups usually must automate and speed up the deployment of BI property to make sure enterprise continuity. We heard that you simply wished an automatic and scalable solution to deploy, again up, and replicate Amazon QuickSight property at scale in order that BIOps groups inside your group can work in an agile method.
At the moment, we’re releasing six new QuickSight APIs to permit programmatic entry to export and import QuickSight property—dashboards, analyses, datasets together with ingestion schedules, knowledge sources, themes, and VPC configurations—throughout accounts and environments. These new APIs will let you work together with a group of property in a lift-and-shift method for deployment throughout QuickSight accounts, allow backup and restore, and help replication so you’ll be able to automate workflows and obtain your required infrastructure setup. These new capabilities carry larger agility to your BIOps groups, permitting you to automate and seamlessly combine QuickSight property into present infrastructure.
Previous to this launch, you wanted to have an in-depth understanding of QuickSight asset relationships and couldn’t deploy, again up, or replicate at scale in an automatic method. On this put up, we cowl the capabilities of the brand new APIs intimately and go over frequent use instances.
Export APIs
You should utilize the next APIs to provoke, observe, and describe the export jobs that produce the bundle information from the supply account. A bundle file is a zipper file (with the .qs extension) that incorporates property specified by the caller, and optionally all dependencies of the property. The APIs are as follows:
- StartAssetBundleExportJob – Use this asynchronous API to export an asset bundle file.
- DescribeAssetBundleExportJob – Use this synchronous API to get the standing of your export job. When profitable, this API name response can have a presigned URL to fetch the asset bundle.
- ListAssetBundleExportJobs – Use this synchronous API to record previous export jobs. The record will comprise each completed and operating jobs from the previous 15 days.
Import APIs
These APIs provoke, observe, and describe the import jobs that take the bundle file as enter and create or replace property within the vacation spot account:
Supported property
You possibly can export and import the next property with these APIs:
- Analyses
- Dashboards
- Knowledge sources
- Datasets together with scheduled and incremental refreshes
- Themes
- VPC connections
Asset bundle output format
The output of the export job is a single zip file with the .qs extension. This zip file incorporates a separate folder for every asset sort. Every folder incorporates a single JSON file for every asset with the resourceId because the file title. This folder construction makes it simple to commit the contents right into a model management system like Git, so you will get the advantages of an entire model historical past.
The Asset-bundle API may also export QuickSight property as AWS CloudFormation templates, one of the in style infrastructure as code (IaC) frameworks. It makes it simple to handle your QuickSight property at scale and automate your deployments. AWS CloudFormation additionally has built-in transaction and rollback capabilities, making certain that each one your environments are constant and your property are deployed accurately each time. Lastly, you should utilize the CloudFormation templates to recreate your QuickSight assets in case of a catastrophe.
Permissions required
These APIs can be found to customers with AWS Identification and Entry Administration (IAM) permissions to run these APIs. The next IAM coverage permits an IAM person to get entry to those APIs:
Use case overview
Let’s take into account a fictional firm, AnyCompany, which owns healthcare services throughout the globe. They’ve arrange a improvement QuickSight account for authors to create and replace QuickSight property and a separate manufacturing account. In some instances due, to knowledge residency regulation, they’ve to keep up the identical property throughout a number of Areas. AnyCompany is scaling their enterprise and so they wish to automate deployment inside and throughout a number of QuickSight accounts and again up QuickSight property on a schedule.
AnyCompany has the next key deployment and backup necessities:
- Deployment – Deploy QuickSight property throughout Areas and a number of accounts:
- Deployment to the manufacturing account – AnyCompany needs to automate the deployment of QuickSight property from their improvement to their manufacturing account.
- Deployment to totally different Areas in the identical account – AnyCompany’s central IT crew must deploy dashboards and datasets throughout varied Areas to fulfill knowledge residency necessities.
- Deployment to a number of accounts in numerous Areas – To fulfill their finish buyer necessities of separate QuickSight accounts, AnyCompany must deploy the dashboards and datasets throughout a number of accounts.
- Deployment in the identical account and similar Area – AnyCompany consolidates all non-production surroundings into one QuickSight account. Nevertheless, there must be totally different dashboards and datasets for every non-production surroundings, resembling improvement and testing.
- Backup and restore – As AnyCompany rolls out essential dashboards for enterprise, it wants to make sure excessive availability of the dashboards. As a part of their technique, AnyCompany needs to keep up a backup of property to revive in case of disasters.
- Deployment historical past – As a part of the ruled course of, AnyCompany’s central IT crew must have a historical past of deployments in every surroundings.
Within the following sections, we focus on learn how to meet these necessities.
Deploy to a manufacturing account
The next determine reveals the sequence of steps within the deployment course of by means of the brand new asset deployment APIs.
For deployments, the import job API gives the potential to go knowledge supply configurations to level to the respective take a look at or manufacturing situations of knowledge sources. Within the previous sequence stream, we use the AWS Command Line Interface (AWS CLI) to showcase the potential, however you’ll be able to invoke the APIs by means of your automation pipeline utilizing AWS SDKs.
For this use case, AnyCompany used Amazon Easy Storage Service (Amazon S3) to retailer their asset bundles..
On the event QuickSight account, full the next steps:
- Use the
StartAssetBundleExportJobAPI to export the dashboard and its dependencies. - Place the output asset bundle in an S3 bucket within the manufacturing account.
On the manufacturing QuickSight account, full the next steps:
- Use the
StartAssetBundleImportJobAPI with the asset bundle in Amazon S3 because the supply, overriding the info supply particulars. - Run the import job.
The next code reveals their StartAssetBundleExportJob API name to export the dashboard and its dependencies:
The next code is for the DescribeAssetBundleExportJob API:
The output of the DescribeAssetBundleExportJob API name incorporates the presigned URL, which you employ to obtain your respective property and subsequently add them to a devoted S3 bucket within the goal account.
The import job (StartAssetBundleImportJob) is initiated within the goal account utilizing S3Uri as one of many enter parameters. You may as well change the info supply configuration whereas initiating the job. Within the following instance, the S3 manifest file location for the S3 knowledge supply is overridden:
DescribeAssetBundleImportJob permits you to to watch the standing of the import job:
The next screenshot reveals the response.
Deploy to totally different Areas within the totally different accounts
To adjust to knowledge residency rules, knowledge can’t be moved outdoors a Area in sure international locations. Due to this fact, the dashboards should be deployed in every of those Areas. QuickSight gives the choice to go an asset bundle extracted from the supply surroundings as a base64 encoded string for the import job (StartAssetBundleImportJob):
Deploy inside a single account in the identical Area
When utilizing a centralized account method for all of the decrease environments, AnyCompany wished to have the identical dashboards inside a single Area to have the ability to join with totally different knowledge sources. To realize this, they used the non-obligatory parameter resource-id-prefix within the import job (StartAssetBundleImportJob) to create a novel ID for every surroundings:
Backup and model management
AnyCompany deploys business-critical dashboards, and it’s vital for them to have correct backup and model management processes. They run scheduled export jobs at common intervals together with asset deployments. Moreover, they use asset bundle APIs to fulfill their model management necessities.
The next screenshot reveals the content material of a pattern bundle.
Deployment historical past
AnyCompany wants to trace the deployment historical past of all of the property in all environments. They achieved this objective by utilizing the ListAssetBundleExportJobs and ListAssetBundleImportJobs APIs to fetch the deployment historical past in a given account.
The next code is for ListAssetBundleExportJobs:
The next code is for ListAssetBundleImportJobs:
Conclusion
Asset bundle APIs present strategies for automation and acceleration within the deployment course of throughout a number of environments. This put up illustrated varied use instances the place you’ll be able to apply these APIs for automation and scale. For extra info, discuss with Amazon QuickSight and What’s New within the Amazon QuickSight Person Information.
In case you have any questions or suggestions, please go away a remark. For added discussions and assist getting solutions to your questions, try the QuickSight Neighborhood.
In regards to the authors
Vetri Natarajan is a Specialist Options Architect for Amazon QuickSight. Vetri has 15 years of expertise implementing enterprise enterprise intelligence (BI) options and greenfield knowledge merchandise. Vetri makes a speciality of integration of BI options with enterprise functions and allow data-driven choices.
Zhao Pan is a software program improvement supervisor for Amazon QuickSight. He’s working to supply a pleasant developer expertise to our clients to automate and streamline their BI operations. He has 20 years of software program improvement expertise in varied tech stacks. Previous to QuickSight he was a individuals and technical chief at ADP constructing a next-gen platform for human capital administration. When he’s not at his desk, he can often be present in his storage constructing one contraption or one other.
Mayank Agarwal is a product supervisor for Amazon QuickSight, AWS’ cloud-native, totally managed BI service. He focuses on embedded analytics and developer expertise. He began his profession as an embedded software program engineer creating handheld gadgets. Previous to QuickSight he was main engineering groups at Credence ID, creating customized cell embedded machine and net options utilizing AWS companies that make biometric enrollment and identification quick, intuitive, and cost-effective for Authorities sector, healthcare and transaction safety functions.
[ad_2]