Apache Arrow DataFusion

Star Fork

DataFusion is a very fast, extensible query engine for building high-quality data-centric systems in Rust, using the Apache Arrow in-memory format.

DataFusion offers SQL and Dataframe APIs, excellent performance, built-in support for CSV, Parquet, JSON, and Avro, extensive customization, and a great community.

The example usage section in the user guide and the datafusion-examples code in the crate contain information on using DataFusion.

Please see the developer’s guide for contributing and communication for getting in touch with us.

Links

• Github and Issue Tracker
• crates.io
• API Docs
• Code of conduct

User Guide

• Introduction
• Example Usage
• DataFusion CLI
• DataFrame API
• Expression API
• SQL Reference
• Configuration Settings
• Frequently Asked Questions
• How does DataFusion Compare with XYZ?

Library User Guide

• Introduction
• Using the SQL API
• Working with Exprs
• Using the DataFrame API
• Write DataFrame to Files
• Building Logical Plans
• Catalogs, Schemas, and Tables
• Adding User Defined Functions: Scalar/Window/Aggregate/Table Functions
• Custom Table Provider
• Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
• Profiling Cookbook

Contributor Guide

• Introduction
• Developer’s guide
• Reviewing Pull Requests
• Communication
• Architecture
• Roadmap
• Quarterly Roadmap
• Specifications

next

Introduction