BrickFlow¶

BrickFlow is a CLI tool for development and deployment of Python based Databricks Workflows in a declarative way.

Concept¶

brickflow aims to improve development experience for building any pipelines on databricks via:

Providing a declarative way to describe workflows via decorators
Provide intelligent defaults to compute targets
Provide a code and git first approach to managing and deploying workflows
Use databricks asset bundles to deploy workflows seamlessly. It is powered using terraform which helps manage state across deployments.
CLI tool helps facilitate setting up a projects
Provides additional functionality through the context library to be able to do additional things for workflows.

Issues with brickflow? Found a bug? Have a great idea for an addition? Want to improve the documentation? Please feel free to file an issue.

To contribute please fork and create a pull request. Here is a guide to help you through this process.