Topic · concept
MapReduce
MapReduce is a programming model and processing technique for computing large data sets in parallel across a distributed cluster. Originally developed by Google and popularized by Apache Hadoop, it splits data processing into map and reduce phases to enable scalable big data analytics.
24 related providers on APIs.io
Source repository →
Providers working in MapReduce
Providers whose own tags share at least two of this topic's tags, most shared first.
| Provider | About | Rating | APIs |
|---|---|---|---|
| Apache Hadoop | Apache Hadoop is an open-source framework for distributed storage and processing of large datasets across clusters of computers using simple programming models. It includes HDFS for distributed storage, YARN for resource management, and Ma… | thin | 1 |
| Google Cloud Dataproc | Google Cloud Dataproc is a fully managed and highly scalable service for running Apache Spark, Apache Hadoop, Apache Flink, Presto, and other open-source data processing frameworks. It enables batch processing, querying, streaming, and mac… | strong | 1 |
| Apache PySpark | Python API for Apache Spark - A unified analytics engine for large-scale data processing supporting batch processing, streaming, machine learning, and graph computing. | emerging | 5 |
| Apache Spark | Open-source unified analytics engine for large-scale data processing with built-in modules for streaming, SQL, machine learning and graph processing. | minimal | 0 |
| Amazon EMR | Amazon EMR is a cloud big data platform for running large-scale distributed data processing jobs, interactive SQL queries, and machine learning applications using open-source analytics frameworks such as Apache Spark, Apache Hive, Apache H… | 1 | |
| AWS Kinesis | Amazon Kinesis is a family of fully managed AWS services for collecting, processing, and analyzing real-time streaming data. The family includes Kinesis Data Streams for scalable record ingestion, Amazon Data Firehose (formerly Kinesis Dat… | developing | 4 |
| Apache HBase | Apache HBase is an open-source, distributed, versioned, non-relational database modeled after Google's Bigtable. It provides random, real-time read/write access to big data and runs on top of Apache Hadoop HDFS, offering a REST API (Starga… | thin | 1 |
| Apache Spark | Apache Spark is a unified analytics engine for large-scale data processing. It provides high-level APIs in Java, Scala, Python, and R, and an optimized engine that supports general execution graphs. Spark offers a comprehensive suite of AP… | thin | 6 |
| Apache Oozie | Apache Oozie is a workflow scheduler system for managing Apache Hadoop jobs. It enables users to define workflows as directed acyclic graphs (DAGs) of actions including MapReduce, Pig, Hive, Sqoop, and custom Java/shell steps. Coordinator… | thin | 1 |
| Apache Hive | Apache Hive is a data warehouse software that facilitates reading, writing, and managing large datasets residing in distributed storage using SQL. It provides a SQL-like interface called HiveQL for querying data stored in Hadoop, along wit… | thin | 1 |
| Apache Giraph | Apache Giraph is an iterative graph processing system built for high scalability on Apache Hadoop. It is modeled after Google's Pregel and provides a simple yet flexible Java API for graph algorithms at massive scale using the Bulk Synchro… | thin | 1 |
| Cask | Cask Data, Inc. was the enterprise big-data startup, backed by Andreessen Horowitz, that built and open-sourced CDAP — the Cask Data Application Platform — a framework for building and running data analytic applications and data pipelines… | thin | 12 |
| Apache Atlas | Apache Atlas is a scalable and extensible set of core foundational data governance services developed by the Apache Software Foundation. It enables enterprises to effectively meet their compliance requirements within Hadoop and allows inte… | thin | 6 |
| Apache Samza | Apache Samza is a distributed stream processing framework that provides a simple API for building stateful stream processing applications. It integrates with Apache Kafka for messaging and supports both stream and batch processing. | thin | 1 |
| Apache Flink | Apache Flink is an open-source framework and distributed processing engine for stateful computations over unbounded and bounded data streams. It is designed to run in all common cluster environments and to perform computations at in-memory… | emerging | 1 |
| Cask Data | Cask Data was the developer-tools company (originally Continuuity) behind CDAP, the Cask Data Application Platform — a 100% open source, integrated framework for building and running batch and real-time data-analytics applications and self… | emerging | 0 |
| Apache ORC | Apache ORC is a self-describing, type-aware columnar file format designed for Hadoop workloads. It provides high compression ratios and fast read performance for large-scale data processing with support for complex data types. | emerging | 3 |
| Apache Tez | Apache Tez is an application framework that allows for complex directed-acyclic-graph (DAG) based processing of data built on Apache Hadoop YARN. It is designed as a successor to MapReduce for executing Hive and Pig queries, providing a fl… | emerging | 2 |
| Apache Pig | Apache Pig is a platform for analyzing large data sets that provides a high-level language (Pig Latin) for expressing data analysis programs. It compiles Pig Latin programs into MapReduce/Tez jobs and runs them on Hadoop clusters. | emerging | 2 |
| Apache Sqoop | Apache Sqoop is a command-line tool designed for efficiently transferring bulk data between Apache Hadoop and structured data stores such as relational databases. It supports parallel import and export of data, incremental loads, and direc… | emerging | 1 |
| Flarion | Flarion is a data-processing acceleration company that speeds up distributed compute engines — Apache Spark, Hadoop, and Ray — without code or infrastructure changes. It ships an Arrow-native, DataFusion/Polars-based execution engine writt… | emerging | 0 |
| Dualbird | Dualbird is a data and AI infrastructure company building accelerated data processing systems that speed up Apache Spark and Amazon EMR workloads by 10x-30x at 50%-90% lower cost, without code changes or migration. Founded by Gilad Tal (CE… | minimal | 0 |
| Concord Systems | Concord Systems was a Brooklyn, New York stream-processing company founded in December 2014 by Alexander Gallego and Emilio Del Tesoro, and backed by Bloomberg Beta. It built a high-performance distributed stream processing framework writt… | 0 | |
| Platfora | Platfora was a big-data analytics company founded in 2011 by Ben Werther that provided in-memory business intelligence and data visualization directly on top of Apache Hadoop, letting analysts run interactive queries, build "lenses," and s… | minimal | 0 |
A term in the Database area.
Tags
Big DataData ProcessingDistributed ComputingHadoopMapReduce