Deprecated Airflow-based Operators¶
As of the Airflow-free release, brickflow no longer depends on
apache-airflow at runtime. The following operators used to subclass
Airflow's BaseOperator and required apache-airflow installed on the
Databricks cluster:
BashOperatorBranchPythonOperatorShortCircuitOperator
They are still importable so existing user code fails loudly at usage
rather than silently at import, but instantiating any of them now raises
RuntimeError with a pointer to the native replacement.
Migration¶
| Removed | Replacement |
|---|---|
BashOperator |
Run your shell command from a Databricks notebook cell (%sh) invoked via RunJobTask, or dbutils.notebook.run a helper notebook. |
BranchPythonOperator |
Use IfElseConditionTask on your workflow. |
ShortCircuitOperator |
Use IfElseConditionTask to skip downstream tasks. |
TaskDependencySensor |
Use AirflowTaskDependencySensor from brickflow_plugins. |
AirflowProxyOktaClusterAuth |
Compute the bearer token yourself and pass it into the plain AirflowCluster dataclass. |
BrickflowSecretsBackend |
Use brickflow_plugins.secrets.resolve_secret(url) directly or call the Cerberus/Base64 helpers. |
API Reference¶
Deprecation stubs for airflow-based operators that were part of brickflow
prior to the removal of the apache-airflow dependency.
These stubs exist purely to give a clear migration message; instantiating
them raises :class:RuntimeError with a pointer to the native replacement.
Classes¶
brickflow_plugins.operators.deprecated_airflow_operators.AirflowProxyOktaClusterAuth(*args, **kwargs)
¶
Bases: _RemovedAirflowOperator
Deprecated: replaced by the plain AirflowCluster dataclass in
brickflow_plugins.sensors.airflow_task_dependency_sensor. Compute
the token yourself (from Okta or another IdP) and pass it in directly.
Source code in brickflow_plugins/operators/deprecated_airflow_operators.py
brickflow_plugins.operators.deprecated_airflow_operators.BashOperator(*args, **kwargs)
¶
Bases: _RemovedAirflowOperator
Deprecated: use dbutils.notebook.run for shell commands, or run a
native Databricks task with brickflow.RunJobTask and invoke your
shell script from within a notebook cell.
Source code in brickflow_plugins/operators/deprecated_airflow_operators.py
brickflow_plugins.operators.deprecated_airflow_operators.BranchPythonOperator(*args, **kwargs)
¶
Bases: _RemovedAirflowOperator
Deprecated: use IfElseConditionTask for conditional task execution.
Source code in brickflow_plugins/operators/deprecated_airflow_operators.py
brickflow_plugins.operators.deprecated_airflow_operators.BrickflowSecretsBackend(*args, **kwargs)
¶
Bases: _RemovedAirflowOperator
Deprecated: replaced by brickflow_plugins.secrets.resolve_secret(url) or
the Cerberus / Base64 helper classes.
Source code in brickflow_plugins/operators/deprecated_airflow_operators.py
brickflow_plugins.operators.deprecated_airflow_operators.ShortCircuitOperator(*args, **kwargs)
¶
Bases: _RemovedAirflowOperator
Deprecated: use IfElseConditionTask to short-circuit downstream tasks.
Source code in brickflow_plugins/operators/deprecated_airflow_operators.py
brickflow_plugins.operators.deprecated_airflow_operators.TaskDependencySensor(*args, **kwargs)
¶
Bases: _RemovedAirflowOperator
Deprecated: replaced by
brickflow_plugins.sensors.airflow_task_dependency_sensor.AirflowTaskDependencySensor.