Script
Provides a SparkSession and allows you to run custom code
Parameters
| Parameter | Meaning | Required |
|---|---|---|
| Input Dataframe(s) | Input dataframe(s) | False |
| Output Dataframe(s) | Output dataframe(s) | False |
| Code | Custom code to be executed | True |
info
To remove input/output dataframe(s), simply click icon on the left sidebar
Examples
Script Gem with Input and Output: Un-pivoting a Dataframe
We'll perform the unpivot operation using our custom code
Script Gem with only Output: Generating a Dataframe
We'll use the provided SparkSession to create and return a dataframe
note
Since we removed the input port, we don't see input dataframe in the method signature
