Snowflake

This module contains the SnowflakeWriter class, which is used to write data to Snowflake.

koheesio.spark.writers.snowflake.SnowflakeWriter

Class for writing to Snowflake

See Also

• koheesio.spark.writers.Writer
• koheesio.spark.writers.BatchOutputMode
• koheesio.spark.writers.StreamingOutputMode

format class-attribute instance-attribute

format: str = Field(
    "snowflake",
    description="The format to use when writing to Snowflake",
)

insert_type class-attribute instance-attribute

insert_type: Optional[BatchOutputMode] = Field(
    APPEND,
    alias="mode",
    description="The insertion type, append or overwrite",
)

table class-attribute instance-attribute

table: str = Field(
    default=..., description="Target table name"
)

execute

execute() -> Output

Write to Snowflake

Source code in src/koheesio/integrations/spark/snowflake.py

def execute(self) -> SnowflakeWriter.Output:
    """Write to Snowflake"""
    self.log.debug(f"writing to {self.table} with mode {self.insert_type}")
    self.df.write.format(self.format).options(**self.get_options()).option("dbtable", self.table).mode(
        self.insert_type
    ).save()