Course Overview
This course serves as an appropriate entry point to learn Apache Spark Programming with Databricks.
Below, we describe each of the four, four-hour modules included in this course.
Introduction to Apache Spark
This course offers essential knowledge of Apache Spark, with a focus on its distributed architecture and practical applications for large-scale data processing. Participants will explore programming frameworks, learn the Spark DataFrame API, and develop skills for reading, writing, and transforming data using Python-based Spark workflows.
Developing Applications with Apache Spark
Master scalable data processing with Apache Spark in this hands-on course. Learn to build efficient ETL pipelines, perform advanced analytics, and optimize distributed data transformations using Spark’s DataFrame API. Explore grouping, aggregation, joins, set operations, and window functions. Work with complex data types like arrays, maps, and structs while applying best practices for performance optimization.
Stream Processing and Analysis with Apache Spark
Learn the essentials of stream processing and analysis with Apache Spark in this course. Gain a solid understanding of stream processing fundamentals and develop applications using the Spark Structured Streaming API. Explore advanced techniques such as stream aggregation and window analysis to process real-time data efficiently. This course equips you with the skills to create scalable and fault-tolerant streaming applications for dynamic data environments.
Monitoring and Optimizing Apache Spark Workloads on Databricks
This course explores the Lakehouse architecture and Medallion design for scalable data workflows, focusing on Unity Catalog for secure data governance, access control, and lineage tracking. The curriculum includes building reliable, ACID-compliant pipelines with Delta Lake. You’ll examine Spark optimization techniques, such as partitioning, caching, and query tuning, and learn performance monitoring, troubleshooting, and best practices for efficient data engineering and analytics to address real-world challenges.
What are the skills covered
- Introduction to Apache Spark
- Developing Applications with Apache Spark
- Stream Processing and Analysis with Apache Spark
- Monitoring and Optimizing Apache Spark Workloads on Databricks
Who should attend this course
- Everyone who is interested
Course Curriculum
Course Modules
Exam & Certification
Databricks Certified Associate Developer for Apache Spark exam.
The Databricks Certified Associate Developer for Apache Spark certification exam assesses the understanding of the Apache Spark Architecture and Components and the ability to apply the Spark DataFrame API to complete basic data manipulation tasks within a Spark session. These tasks include selecting, renaming and manipulating columns; filtering, dropping, sorting, and aggregating rows; handling missing data; combining, reading, writing and partitioning DataFrames with schemas; and working with UDFs and Spark SQL functions.
In addition, the exam will assess the basics of the Spark architecture like execution/deployment modes, the execution hierarchy, fault tolerance, garbage collection, lazy evaluation, Shuffling and usage of Actions and broadcasting, Structured Streaming, Spark Connect, and common troubleshooting and tuning techniques. Individuals who pass this certification exam can be expected to complete basic Spark DataFrame tasks using Python.
This exam covers:
-
Apache Spark Architecture and Components – 20%
-
Using Spark SQL – 20%
-
Developing Apache Spark™ DataFrame/DataSet API Applications – 30%
-
Troubleshooting and Tuning Apache Spark DataFrame API Applications – 10%
-
Structured Streaming – 10%
-
Using Spark Connect to deploy applications – 5%
-
Using Pandas API on Apache Spark – 5%
