State Reader API for Spark Structured Streaming on Databricks
Databricks Runtime 14.3 includes a new capability that allows users to access and analyze Structured Streaming‘s internal state data: the State Reader API. The State Reader API sets itself apart from well-known Spark data formats such as JSON, CSV, Avro, and Protobuf. Its primary purpose is facilitating the development, debugging, and troubleshooting of stateful Structured […]
Continue Reading