CloudQuery Product Updates #6
June 10, 2023
Hey everyone! Our monthly round-up of CloudQuery product and roadmap updates is here!
Apache Arrow
We were heads down adopting Apache Arrow as our in-memory data format!
TLDR:
- Support for more than 30 types (opens in a new tab) for any source plugin including repeated and nested types like lists, structs, maps.
- Database source plugins (with CDC) will be able to sync their schema accurately using Apache Arrow type system.
- Any database that already supports Arrow format natively like ClickHouse, DuckDB will be able to enjoy enhanced performance.
Sources
New Sources
Welcoming 4 new partner and community plugins!
- Coinpaprika (opens in a new tab)
- Onfleet (opens in a new tab)
- CrowdStrike (opens in a new tab)
- SharePoint (opens in a new tab)
New Official plugins:
- OracleDB
- Firestore Thanks to @BubbaJoe (opens in a new tab) for initial contribution!
Community Contributions to Security Policies:
- Azure CIS (opens in a new tab) Thanks to
aruneko
(opens in a new tab) for community contribution!
New Resources
-
AWS: 18 New tables:
aws_config_config_rule_compliance_details
aws_config_remediation_configurations
aws_acmpca_certificate_authorities
aws_cloudformation_stack_templates
aws_cloudformation_template_summaries
aws_cloudwatchlogs_log_group_data_protection_policies
aws_config_config_rule_compliance_details
aws_config_remediation_configurations
aws_config_configuration_aggregators
aws_config_delivery_channels
aws_config_retention_configurations
aws_costexplorer_cost_30d
aws_costexplorer_cost_forecast_30d
aws_efs_access_points
aws_networkfirewall_firewall_policies
aws_networkfirewall_rule_groups
aws_securityhub_enabled_standards
aws_securityhub_hubs
-
Azure: 39 New tables:
azure_consumption_billing_*
azure_consumption_subscription_*
azure_dns_record_sets
azure_network_private_link_services
azure_network_watcher_flow_logs
azure_security_sub_assessments
azure_sql_server_advanced_threat_protection_settings
azure_sql_server_firewall_rules
azure_storage_queue_acl
-
GCP: New Table:
gcp_dns_resource_record_sets
-
Datadog: 3 Tables:
datadog_rum_events
datadog_slo_corrections
datadog_slos
-
Digital Ocean: 2 Tables:
digitalocean_load_balancers
digitalocean_regions
-
GitLab: 2 Tables:
gitlab_group_billable_members
gitlab_project_members
-
Oracle: 5 Tables:
oracle_compute_volume_attachments
oracle_virtualnetwork_assigned_public_ips
oracle_virtualnetwork_ephemeral_public_ips
oracle_virtualnetwork_private_ips
oracle_virtualnetwork_reserved_public_ips
-
Snyk: 2 Tables:
snyk_groups
snyk_group_members
-
Stripe: 2 Tables:
stripe_checkout_session_line_items
stripe_payment_link_line_items
Policies
- Thanks to @aruneko (opens in a new tab) for contribution to our Azure CIS Policies (opens in a new tab)!
Other notable blogs and use-cases
Engineering
- How to Manage AWS Resource and Usage Limits with CloudQuery and Service Quotas
- Video: Deploying CloudQuery to a Virtual Machine on Google Cloud (opens in a new tab)
- Adopting Apache Arrow at CloudQuery (opens in a new tab)
- Video: Neo4j and CloudQuery (opens in a new tab)
- Powering Community Driven Cloud Security CSPM with CloudQuery and SumoLogic (opens in a new tab)