Skip to main content

Row Distributor

Create multiple dataframes based on provided filter conditions from an input dataframe.

This is useful for cases where rows from the input dataframe needs to be distributed into multiple dataframes in different ways for the downstream application/pipeline.

Parameters

ParameterDescriptionRequired
DataframeInput dataframe for which rows needs to be distributed into multiple dataframesTrue
Filter ConditionsBooleanType column or boolean expression for each output tab. Supports sql, python and scala expressionsTrue

Example

Row distributor 1

info

Number of outputs can be changed as per need by clicking the + button.

Generated Code

def RowDistributor(spark: SparkSession, in0: DataFrame) -> (DataFrame, DataFrame, DataFrame):
df1 = in0.filter((col("order_status") == lit("Started")))
df2 = in0.filter((col("order_status") == lit("Approved")))
df3 = in0.filter((col("order_status") == lit("Finished")))

return df1, df2, df3