Cloud Storage and Data Acquisition
Cloud Storage and Data Acquisition on the APIs.io network is a curated area collecting 6 resources — specifications, tools, and documentation — for this subject.
24 providers on the network work in this area, including Estuary Flow, Backblaze, Amazon S3, Amazon Redshift, Amazon S3 API, Azure Blob Storage, and 18 more — each links out to that provider’s APIs, schemas, and governance artifacts.
Related areas: Cloud Storage. Browse every area at areas.apis.io.
About this area
Cloud Storage and Data Acquisition is a topic profile in the API Evangelist Network covering APIs and tooling for ingesting, moving, and persisting bulk and streaming data into cloud-resident storage. It groups object storage services, data-lake foundations, managed ingestion pipelines, change-data-capture connectors, transfer appliances, and data-broker APIs that provide source material for cloud-storage workloads. The topic is intended as an entry point for developers and architects evaluating how data lands in cloud storage from on-premises systems, SaaS APIs, IoT devices, public data sources, and partner exchanges.
Curated resources (6)
Object Storage Surface
The object-storage surface includes Amazon S3, Google Cloud Storage, and Azure Blob Storage REST APIs. These APIs provide the canonical landing zone for cloud data acquisition and are the most common targets for inges...
Streaming Ingest Surface
The streaming-ingest surface covers Amazon Kinesis Data Streams, Google Cloud Pub/Sub, and Azure Event Hubs. These services ingest high-volume event streams and make them durable for downstream landing into object sto...
Managed Ingestion Pipelines
Managed ingestion pipelines such as AWS Glue, Google Cloud Dataflow, and Azure Data Factory expose REST APIs for orchestrating extract-transform-load and extract-load-transform jobs that land source data in cloud stor...
Change Data Capture Connectors
Change Data Capture connectors (Debezium, AWS DMS, Fivetran, Striim) replicate row-level changes from operational databases to cloud storage and warehouses, providing a low-latency feed for analytics and data-lake hyd...
Bulk Transfer Services
Bulk transfer services (AWS DataSync and Snow family, Google Storage Transfer Service, Azure Data Box) move large datasets from on-premises and edge locations into cloud storage over network or via offline appliances.
Data Marketplaces
Data marketplaces and open-data registries expose third-party datasets through REST APIs, subscription delivery, and shared buckets. They are an increasingly common acquisition channel for cloud-storage data lakes.
Related providers (231)
Top 24 of 231 network providers tagged for this area — search the full set on apis.io.
| Provider | Description | APIs | Rating |
|---|---|---|---|
| Estuary Flow | Estuary Flow is a real-time data movement and transformation platform combining streaming infrastructure, a runtime, and an open-source ecosystem of connectors. It supports chan... | 4 | thin |
| Backblaze | Backblaze is a cloud storage and data backup provider offering B2 Cloud Storage - a low-cost, S3-compatible object storage service. Backblaze provides both a native B2 API and a... | 2 | strong |
| Amazon S3 | Amazon Simple Storage Service (S3) is an object storage service offering industry-leading scalability, data availability, security, and performance. | 3 | strong |
| Amazon Redshift | Amazon Redshift is a fast, fully managed cloud data warehouse that makes it simple and cost-effective to analyze all your data using standard SQL and your existing Business Inte... | 3 | developing |
| Amazon S3 API | Amazon Simple Storage Service (S3) is an object storage service offering industry-leading scalability, data availability, security, and performance for storing and retrieving an... | 1 | developing |
| Azure Blob Storage | Microsoft Azure Blob Storage is a service for storing large amounts of unstructured object data, such as text or binary data, that can be accessed from anywhere in the world via... | 3 | developing |
| Azure Blob Storage | Microsoft Azure Blob Storage is a service for storing large amounts of unstructured object data, such as text or binary data, that can be accessed from anywhere in the world via... | 3 | developing |
| Google Cloud Datastream | Google Cloud Datastream is a serverless change data capture (CDC) and replication service that allows you to synchronize data across heterogeneous databases, storage systems, an... | 1 | developing |
| Google Cloud Storage | Object storage service offering high durability, availability, and scalability for storing and accessing data on Google Cloud Platform. | 2 | developing |
| Pure Storage | Pure Storage is an American publicly traded technology company specializing in all-flash data storage hardware and software products. The company provides enterprise data storag... | 4 | developing |
| WarpStream | WarpStream is a diskless, Apache Kafka-compatible data streaming platform built directly on top of cloud object storage such as S3, GCP, and Azure. It eliminates the need for lo... | 1 | developing |
| Filebase | Filebase is an S3-compatible object storage and IPFS pinning platform that combines familiar cloud storage APIs with decentralized, blockchain-backed infrastructure. Developers ... | 4 | thin |
| Talend | Talend (now part of Qlik) provides data integration, quality, and API management capabilities through cloud-native APIs for ETL, data pipelines, and application integration. The... | 6 | thin |
| Cloudflare R2 | Cloudflare R2 is S3-compatible object storage with zero egress fees. It provides both an S3-compatible REST API and a Cloudflare API for managing buckets, objects, and storage p... | 2 | thin |
| Apache Flume | Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log and event data. It provides a simple and ... | 2 | thin |
| Apache SeaTunnel | Apache SeaTunnel is a high-performance, distributed data integration platform that supports real-time and batch data synchronization. It provides a connector API with support fo... | 1 | thin |
| Flatfile | Flatfile is a data exchange platform that helps teams import, transform, validate, and collaborate on file-based data. The Flatfile API provides programmatic access to spaces, w... | 1 | thin |
| Sequin | Sequin is an open-source Postgres change data capture (CDC) engine that streams Postgres rows and changes to streams, queues, and search indexes - Kafka, SQS, SNS, Kinesis, Redi... | 5 | thin |
| Streamkap | Streamkap is a real-time streaming ETL and change data capture (CDC) platform built on Apache Kafka and Apache Flink. It streams data from operational databases (PostgreSQL, MyS... | 9 | thin |
| ByteArk | ByteArk is a Thailand-based video streaming and content delivery platform founded in 2012 and headquartered in Bangkok. It provides video-on-demand (ByteArk Stream), live stream... | 5 | thin |
| Apache NiFi | Apache NiFi is a dataflow management system designed to automate the flow of data between systems. It provides a web-based user interface for designing, controlling, and monitor... | 3 | thin |
| Apache Beam | Apache Beam is a unified, open-source programming model developed by the Apache Software Foundation for defining both batch and streaming data processing pipelines. It provides ... | 2 | thin |
| Kafka Connect | Kafka Connect is a tool for scalably and reliably streaming data between Apache Kafka and other systems. It makes it simple to quickly define connectors that move large collecti... | 1 | thin |
| Wasabi | Wasabi Hot Cloud Storage is an S3-compatible object storage service offering a REST API that mirrors the Amazon S3 API for storing, retrieving, and managing objects and buckets.... | 4 | thin |
Related areas
Tags: Bulk Transfer, Change Data Capture, Cloud Storage, Data Acquisition, Data Ingestion, Data Lake, ETL, Object Storage, Pipelines, Streaming.
Sourced from the api-evangelist/cloud-storage-and-data-acquisition topic repo.