bg-leftbg-right

Store and Retrieve Objects with Amazon S3 in a Secure, Scalable, and Reliable Way

Schedule a Demo
quarter-light
image-right

Astro + Amazon S3

Orchestrate object storage in Amazon S3 buckets with Astro, the modern data orchestration platform powered by Apache Airflow. Astro offers easy Amazon S3 integration using the AWS provider package. Leverage a full set of core Airflow tools that not only make data storage and retrieval from AWS S3 effortless, but also allow you to dynamically restructure your S3 buckets directly within Astro.


img-left
Amazon S3

About Amazon S3

Amazon S3 object storage is one of the most commonly used solutions to store large amounts of data. AWS S3 buckets provide a scalable and secure option to store the data created through your Astro pipeline, no matter the format.

data-tools

Use Case

Storing large amounts of data objects, such as raw files ingested from various sources or logs, is one of the most common reasons to combine Astro with Amazon S3. With Astro, you can easily orchestrate pipelines to start once a file lands in S3 in an asynchronous resource-friendly way by using specialized deferrable operators to watch for changes. Thanks to Astro’s support of Airflow 2.4 it’s possible to use dataset-driven scheduling with S3 objects to further optimize your data pipeline.


See Astro in Action: Get a demo that’s customized around your unique data orchestration workflows and pain points.

Schedule a Demo