Skip to content

Utils

koheesio.integrations.spark.databricks.utils.get_dbutils #

get_dbutils(spark_session: SparkSession) -> DBUtils
Source code in src/koheesio/integrations/spark/databricks/utils.py 
 8
 9
10
11
12
13
14
15
16
def get_dbutils(spark_session: SparkSession) -> DBUtils:  # type: ignore  # noqa: F821
    if not on_databricks():
        raise RuntimeError("dbutils not available")

    from pyspark.dbutils import DBUtils  # pylint: disable=E0611,E0401 # type: ignore

    dbutils = DBUtils(spark_session)

    return dbutils