# Free DASD practice questions

> 10 free DASD practice questions with the correct answer and an explanation.

Page: https://devopscodelab.com/exams/databricks-apache-spark-developer/free-practice-questions/
Practice exams: https://devopscodelab.com/exams/databricks-apache-spark-developer/

### Question 1 of 10

In a Spark cluster, what is an executor?

- A. A physical machine in the cluster that the cluster manager reserves for a single Spark job
- B. A thread inside the driver that converts each stage into bytecode before it is shipped
- C. A daemon shared by all applications on a node that keeps shuffle files after jobs finish
- D. A per-application JVM process on a worker node that runs tasks and caches data

**Answer:** D

**Explanation:** Executors are processes launched for a specific application on worker nodes; they run tasks in their cores (slots) and hold cached data and shuffle output. Code generation happens in the driver's planning phase, not in a dedicated executor thread. A shared node daemon that serves shuffle files is the external shuffle service, not an executor, and a worker node can host executors of several applications, so an executor is not a reserved machine.

### Question 2 of 10

Which statement correctly describes lazy evaluation in Apache Spark?

- A. Lazy evaluation only applies to RDDs, because DataFrame transformations are executed eagerly by the Catalyst optimizer
- B. Spark delays reading the source files until the first transformation, then materializes each intermediate DataFrame
- C. Actions are queued and executed in the background, while transformations run immediately to validate the data
- D. Transformations only build up a logical plan; nothing executes until an action such as count() or write is called

**Answer:** D

**Explanation:** Transformations such as select, filter and join return a new DataFrame describing the computation; Spark executes the plan only when an action requests a result. Actions run synchronously and trigger execution rather than being queued. Spark does not materialize intermediate DataFrames after each transformation, and DataFrames are lazy too, which is exactly what allows Catalyst to optimize the whole plan before running it.

### Question 3 of 10

A developer needs a persistent table where data is hashed into 16 buckets by user_id and sorted within each bucket by event_ts, to speed up later joins on user_id. Which code is valid?

- A. df.write.bucketBy(16, "user_id").sortBy("event_ts").saveAsTable("events_b")
- B. df.write.bucketBy(16, "user_id").sortBy("event_ts").parquet("/data/events_b")
- C. df.write.bucketBy("user_id", 16).orderBy("event_ts").saveAsTable("events_b")
- D. df.write.sortBy("event_ts").partitionBy("user_id").saveAsTable("events_b")

**Answer:** A

**Explanation:** Bucketing requires the number of buckets first, then the columns, and it only works with saveAsTable because bucket metadata is stored in the metastore; sortBy is valid only together with bucketBy. Writing bucketed output with parquet(path) raises an error. sortBy without bucketBy is not allowed, and partitioning by a high-cardinality user_id would create far too many directories. The writer has no orderBy method and the arguments of bucketBy are reversed.

### Question 4 of 10

A developer reads a PostgreSQL table with the JDBC source. The read is slow because only one task pulls all 80 million rows. Which option set lets Spark read the table in parallel?

- A. partitionColumn, lowerBound, upperBound and numPartitions on a numeric or date column
- B. numPartitions on its own, since Spark then splits the table automatically by the primary key
- C. pushDownPredicate and queryTimeout so the database splits the query across several cursors
- D. fetchsize and batchsize set high enough so that each executor opens its own connection

**Answer:** A

**Explanation:** Parallel JDBC reads require partitionColumn together with lowerBound, upperBound and numPartitions; Spark generates one range query per partition. fetchsize controls rows per round trip and batchsize affects writes, neither adds parallelism. numPartitions alone does not split a read without a partition column, and pushDownPredicate/queryTimeout control filter pushdown and timeouts, not parallelism.

### Question 5 of 10

Two DataFrames, orders (500 million rows) and countries (250 rows), are joined on country_code. The developer wants to make sure the large table is not shuffled. Which code achieves this?

- A. broadcast(orders).join(countries, "country_code")
- B. orders.join(countries.repartition(1), "country_code")
- C. orders.join(broadcast(countries), "country_code")
- D. orders.join(countries, "country_code").coalesce(1)

**Answer:** C

**Explanation:** Wrapping the small DataFrame in broadcast() ships a copy of it to every executor, so each partition of orders is joined locally with no shuffle of the large side. Broadcasting the large table would send 500 million rows to every executor. Repartitioning the small side to one partition does not stop a sort-merge join from shuffling orders, and coalescing after the join changes only the output partitioning.

### Question 6 of 10

A pandas UDF computes a value from two numeric columns. Which definition is correct for Spark 3.x/4.x?

- A. @pandas_udf("double", PandasUDFType.SCALAR_ITER)
def ratio(a, b): return a.values / b.values
- B. @pandas_udf("double")
def ratio(a: pd.Series, b: pd.Series) -> pd.Series: return a / b
- C. @udf("double")
def ratio(a: pd.Series, b: pd.Series) -> pd.Series: return a / b
- D. @pandas_udf
def ratio(a: float, b: float) -> float: return a / b

**Answer:** B

**Explanation:** Modern pandas UDFs are declared with @pandas_udf(returnType) and Python type hints; pd.Series inputs and output define a Series-to-Series UDF. PandasUDFType is the deprecated pre-3.0 style, and SCALAR_ITER would expect iterator input. @udf creates a row-at-a-time Python UDF that receives scalars, not Series. Scalar float hints are not a valid pandas UDF signature and no return type is given.

### Question 7 of 10

A UDF that normalizes phone numbers must be used in DataFrame code and in SQL queries. Which code registers it for both uses?

- A. norm = udf(normalize, StringType()).alias("norm_phone")
- B. spark.catalog.createFunction("norm_phone", normalize)
- C. norm = udf(normalize, StringType()); spark.sql("CREATE FUNCTION norm_phone AS norm")
- D. norm = spark.udf.register("norm_phone", normalize, StringType())

**Answer:** D

**Explanation:** spark.udf.register registers the function for SQL under the given name and returns a UDF object that can also be used in DataFrame expressions. A udf() object is not visible to SQL, and CREATE FUNCTION expects a class or a SQL body rather than a Python variable. Aliasing a UDF names an output column, and the catalog API has no createFunction method for registering Python functions.

### Question 8 of 10 (choose 2)

Which TWO optimizations does Adaptive Query Execution (AQE) perform at runtime using shuffle statistics? (Choose two.)

- A. Choosing the number of executors for the next application
- B. Rewriting Python UDFs into native JVM expressions
- C. Splitting skewed partitions in a sort-merge join into smaller tasks
- D. Collecting table statistics before the query starts running
- E. Coalescing many small shuffle partitions into fewer, larger ones

**Answer:** C, E

**Explanation:** AQE re-optimizes the plan at stage boundaries using real shuffle statistics: it coalesces small shuffle partitions, splits skewed join partitions, and can switch a sort-merge join to a broadcast join. It never rewrites Python UDFs, it does not size executors (that is dynamic allocation or the cluster configuration), and it relies on runtime shuffle statistics rather than collecting table statistics in advance like ANALYZE TABLE.

### Question 9 of 10

A stream may deliver the same order event several times within a few minutes. The query must drop duplicates by order_id while keeping state bounded. Which code is the best fit in Spark 3.5 or later?

- A. df.distinct() applied after the sink so the files written are de-duplicated afterwards
- B. df.dropDuplicates(["order_id"]) without any watermark, so every order_id is kept in state
- C. df.dropDuplicates(["order_id", "event_time"]) without a watermark on event_time
- D. df.withWatermark("event_time", "10 minutes").dropDuplicatesWithinWatermark(["order_id"])

**Answer:** D

**Explanation:** dropDuplicatesWithinWatermark (Spark 3.5+) removes duplicates by key that arrive within the watermark delay, and state older than the watermark is purged, so state stays bounded even when duplicates have different timestamps. dropDuplicates without a watermark keeps every key forever. distinct cannot be applied to data after it is written by a sink, and including event_time without a watermark both misses duplicates with different timestamps and grows state without limit.

### Question 10 of 10

Which deployment mode is MOST suitable for unit testing PySpark code on a CI runner without a cluster?

- A. Local mode, for example master local[2]
- B. Client mode against a YARN cluster
- C. Cluster mode on a Kubernetes cluster
- D. Standalone cluster with high availability

**Answer:** A

**Explanation:** Local mode runs the driver and executor in one process on the runner, which is ideal for tests with small data and no external dependencies. Cluster and client modes require a running cluster manager and cluster resources, and a highly available standalone cluster adds infrastructure that a unit test does not need.

Ready for a timed run? Take the free 15-question DASD mini exam: https://devopscodelab.com/app/exams/databricks-apache-spark-developer/mini/

DevOpsCode Lab is not affiliated with Databricks. DASD and related names are trademarks of their owners. We write every question ourselves; these are not real exam questions.
