CloudQuery Product Updates #8
September 28, 2023
- Name
- Michal Brutvan
- pilvikala
Hey everyone! Our monthly round-up of CloudQuery product and roadmap updates is here! Here are the main highlights:
- Open source: A new SDK, new plugins, performance improvements
- Premium (closed source): New policies
SDKs!
In case you missed it, we published Python (opens in a new tab) and JavaScript (opens in a new tab) SDKs last month and this month, we followed with Java!
- Check out our Java announcement blog (opens in a new tab)!
- Creating a Java source (opens in a new tab)
Sources
New Source Plugins
- Bitbucket (opens in a new tab) - First plugin utilizing our Java SDK!
- Notion (opens in a new tab) - Loads databases, pages, and users from Notion to any database, data warehouse or data lake supported by CloudQuery, such as PostgreSQL, BigQuery, Athena, and many more. Shout-out to Gagan Gaurav (opens in a new tab) for this community contribution!
AWS, GCP, Azure Highlights
-
CloudQuery is now an official AWS Partner! Read more about this in our announcement (opens in a new tab).
-
We launched a closed beta for event-based sync with the AWS plugin. This helps with keeping your data fresh without running full syncs. Sign up for beta (opens in a new tab) or read the full announcement (opens in a new tab) first.
-
The AWS plugin has a new scheduler strategy that reduces the number of concurrent calls made to AWS APIs in syncs with multiple tables. This means the probability of hitting rate limits is lower. Read more about this in our documentation on performance tuning (opens in a new tab).
Check out the full changelog for AWS (opens in a new tab), GCP (opens in a new tab) and Azure (opens in a new tab) for full updates.
Destinations
- We have made performance fixes to our PostgreSQL destination plugin which resulted in a significant improvement in data throughput and resource utilization.
Policies
- We released AWS Foundational Security Best Practices for Snowflake (opens in a new tab)! With this release, there are now over 200 controls with coverage for more than 40 AWS services.
- Bug fixes on AWS policies
