ADAM analyzes genomics data with specialized file formats built on Apache Spark, Avro, and Parquet.
Open source
24 alternativesSpark
Apache Spark is a unified engine for large-scale data processing and analytics.
About Spark
Spark is an open-source engine for large-scale data processing and analytics. It provides high-level APIs in Scala, Java, Python, and R, noted as deprecated, with an engine that supports general computation graphs for data analysis.
It includes Spark SQL for SQL and DataFrames, pandas API on Spark for pandas workloads, MLlib for machine learning, GraphX for graph processing, and Structured Streaming for stream processing. Source builds use Apache Maven, and use with HDFS or other Hadoop-supported storage requires building against the same Hadoop version as the cluster.
Key features
- High-level APIs in Scala, Java, Python, and deprecated R
- Spark SQL for SQL queries and DataFrames
- pandas API on Spark for pandas workloads
- MLlib for machine learning
- GraphX for graph processing
- Structured Streaming for stream processing
Best for: Developers and data teams building large-scale batch and streaming analytics with SQL, Python, Scala, or Java.
Good for
- Supports Scala, Java, Python, SQL, DataFrame, and pandas-style interfaces
- Includes modules for SQL, machine learning, graph, and streaming work
- Can run locally or submit jobs to a cluster via spark:// or YARN
Watch out for
- R API is marked as deprecated
- Must be built against the same Hadoop version as the cluster
- Testing requires building Spark first
Spark at a glance
- Website
- github.com/apache/spark
- Category
- AI & Machine Learning › AI Data & Analytics
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
Spark alternatives
Tools and sites you could use instead, ranked by how closely they match.
Enables data-driven, interactive data analytics and collaborative documents.
Hex is an AI analytics platform for exploring and analyzing data through conversation grounded in governed context.
Hex combines data notebooks, applications, and AI tools so teams can ask questions and build with organizational data.
pytorch tabular provides a unified framework for deep learning models on tabular data.
Open source
24 alternatives
NALP covers Natural Adversarial Language Processing through a dedicated library.
Open source
24 alternativesPopular in AI Data & Analytics
Well-known listings from the same topic.
Dotgetdot.aiForvioforvio.comFluidAI Medicalfluidai.mdSola Securitysola.securityQuadraticquadratichq.comIntellectia AIintellectia.ai
Spark FAQ
How do I build Spark from source?
Run ./build/mvn -DskipTests clean package. This is not needed if you downloaded a pre-built package.
How do I start Spark interactively?
Use ./bin/spark-shell for Scala or ./bin/pyspark for Python.
How do I run the included example programs?
Run ./bin/run-example [params], for example ./bin/run-example SparkPi, and set the MASTER variable to submit to a cluster.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.