Session Description

In recent years, several table formats for large datasets have emerged to help data engineers deal with complexity of handling substantial amounts of data in a flexible, performant and safe way. One of the most popular among those formats is Apache Iceberg.

In this workshop, you will gain up-to-date, hands-on experience on how to work with Iceberg. Using a local coding environment based on Jupyter Notebooks and a Docker Compose stack, you are going to:

1. Learn about required components of a data processing system that uses Iceberg.
2. Practice examples of how to update and query Iceberg using several query engines and libraries.
3. Use advanced features of Iceberg, like flexible partitioning scheme, time travel or dataset branching.
4. Learn about optimisation techniques and configuration “levers” you can pull to improve the overall performance and query speed of workloads using Iceberg.
5. Peek under the hood of an Iceberg dataset, to understand its metadata and ways it improves query speed and supports data audits and lineage.

This workshop is recommended for Data Engineers, Analytics Engineers and Machine Learning Engineers wanting to improve their data pipelines and data processing workflows.