Module 6 of 8

Distributed Systems: Ray, Dask, Celery & Kafka

Scale Python workloads with Ray and Dask, build resilient task queues with Celery, and process event streams with Kafka.

4 lessons

~32 hours

195 XP

Scale functions and classes with @ray.remote, use actors for stateful services, and process data with Ray Data.

Scale NumPy/pandas with dask.array/dask.dataframe, build custom task graphs, and deploy on clusters with dask-gateway.

Design idempotent tasks, compose workflows with canvas (chain, group, chord), and operate with Flower, retries, and dead letters.

Build exactly-once producers, consumer groups with rebalance listeners, and stream processing with Faust or ksqlDB.