Allison Wang
Allison Wang is a Software Engineer at Databricks and an Apache Spark Committer, specializing in Spark SQL and PySpark. She’s passionate about bridging Python with the big data ecosystem. Allison holds a bachelor’s degree in Computer Science from Carnegie Mellon University.
Sessions
11-07
15:20
40min
Faster PySpark with Apache Arrow
Allison Wang
PySpark is the Python API for Apache Spark, an open-source distributed computing framework that enables large-scale, real-time data processing. In this talk, we will show how integrating Apache Arrow—a high-performance in-memory format—makes PySpark faster and more efficient.
Music Box