DCDEP

Free DCDEP practice questions

10 questions from our DCDEP practice tests, each with the correct answer and a written explanation. Read them with the answers shown, or switch to quiz mode and answer first.

Back to the DCDEP practice exams

Question 1 of 10, Developing Code for Data Processing using Python and SQL

In a Lakeflow Pipeline, a gold dataset joins a slowly changing product dimension to sales facts and computes revenue per category. Product attributes are corrected retroactively several times a week, and the result must always reflect those corrections. Dashboards read the dataset every hour. Which dataset type should the engineer use?

  1. AA materialized view, because its refreshed result always matches the query, including corrected dimension rows Correct answer
  2. BA streaming table, because it reads each new sales row once and therefore keeps the per-category revenue current at the lowest compute cost
  3. CA streaming table with a stream-static join, because the static product side is re-read on every micro-batch and updates past aggregates
  4. DA temporary view, because it is evaluated at query time by every dashboard and therefore never needs a pipeline refresh to stay accurate

Answer and explanation: A materialized view's result always matches its defining query, including changes to rows that were already processed, which is what retroactive dimension corrections require. A streaming table processes appended input once and does not revisit earlier output when dimension rows change. A temporary view exists only within the pipeline and is not published for dashboards. In a stream-static join the latest static data is used only for new stream rows; previously emitted aggregates are not recalculated.

Question 2 of 10, Developing Code for Data Processing using Python and SQL

A team compares hand-written Spark Structured Streaming with Lakeflow Pipelines for a new bronze-silver-gold ETL. Requirements are automatic dependency ordering between tables, built-in data quality rules with metrics, and managed retries and autoscaling, with as little orchestration code as possible. Which option fits best?

  1. ALakeflow Pipelines in development mode only, because production mode disables expectations and requires manual table ordering
  2. BLakeflow Pipelines, because dependencies, expectations, retries and compute are managed declaratively from the dataset definitions Correct answer
  3. CStructured Streaming in notebooks chained with Lakeflow Jobs tasks, because job retries and task dependencies cover the same requirements
  4. DStructured Streaming with foreachBatch per table, because each batch function can log quality metrics and manage its own retries

Answer and explanation: Lakeflow Pipelines infer the dependency graph from dataset definitions, record expectation metrics in the event log, and manage retries and autoscaling of compute, minimising orchestration code. Chaining notebooks with job tasks gives retries but requires you to code and order each stream and build quality checks yourself. foreachBatch per table puts quality logging and retry logic in custom code. Production mode does not disable expectations; it differs in retry behaviour and cluster reuse.

Question 3 of 10, Data Ingestion & Acquisition

A data provider on Databricks wants to share a curated orders table and its ongoing changes with a partner who also uses Databricks in a different account and cloud region. The partner must be able to read the table incrementally as a stream. Which setup should the provider use?

  1. ALakehouse Federation: create a connection in the partner's workspace to the provider's SQL warehouse and expose the table as a foreign catalog
  2. BClean Rooms: add the orders table to a clean room and let the partner run approved notebooks against it to produce output tables
  3. CDatabricks-to-Databricks sharing: add the table to a share WITH HISTORY for a recipient keyed on the partner's sharing identifier Correct answer
  4. DOpen sharing: add the table to a share without history and send the partner a bearer-token credential file to download for their client

Answer and explanation: Databricks-to-Databricks sharing uses the recipient's metastore sharing identifier, and sharing a table WITH HISTORY enables the recipient to read it as a stream and use time travel. A share without history cannot be streamed, and token-based open sharing is meant for non-Databricks clients. Lakehouse Federation gives query-time access through a connection but does not deliver the table's change history for streaming reads. Clean Rooms is for joint analysis without exposing raw data, not for giving a partner a readable replica.

Question 4 of 10, Data Manipulation

In a Lakeflow Pipeline, an engineer wants invalid rows written to a separate quarantine table for review while valid rows continue to the silver table, using a single set of rules. Which approach implements this?

  1. AApply expect_all on silver, which writes failed rows to a generated _quarantine table
  2. BDefine the rules once, use them with expect_all_or_drop on silver, and filter a quarantine table on the negated rules Correct answer
  3. CApply expect_all_or_fail on the silver table so invalid rows are moved to the event log, then read them from there for review
  4. DApply the rules as CHECK constraints on the silver table, which divert violating rows to a rejected-records path automatically

Answer and explanation: Expectations do not route rows themselves, so the quarantine pattern reuses the same rule definitions: drop invalid rows from the valid table and create a second table selecting rows where the rules are not met. expect_all_or_fail stops the update; the event log stores metrics, not rows. expect_all only records metrics and keeps the rows; there is no automatic quarantine table. CHECK constraints reject the whole write rather than diverting rows.

Question 5 of 10, Monitoring and Alerting

A FinOps lead asks for last month's DBU consumption per Lakeflow Job, converted to list-price dollars, using governed data in Unity Catalog. Which query design is correct?

  1. AAggregate system.query.history by statement_id and sum total_duration_ms for statements executed by job service principals last month
  2. BAggregate system.compute.clusters by cluster_id and join to system.access.audit to find which user started each job cluster during the month
  3. CSum usage_quantity in system.billing.usage per usage_metadata.job_id, priced by joining system.billing.list_prices on SKU and time Correct answer
  4. DAggregate system.lakeflow.job_run_timeline by job_id and multiply run duration by the cluster's node count and the hourly VM price from the cloud console

Answer and explanation: system.billing.usage records DBU usage with usage_metadata such as job_id, and joining system.billing.list_prices on SKU and effective time gives list-price cost. Job run timelines give durations, not DBUs, and VM prices are not DBU list prices. Cluster and audit tables describe configuration and actions, not consumption. Query history covers SQL statements and durations, not job DBU spend.

Question 6 of 10, Cost & Performance Optimization

A team migrates several hundred Delta tables to Unity Catalog managed tables. They want routine file compaction, VACUUM and statistics collection to happen without scheduling maintenance jobs. Which feature delivers this?

  1. APredictive optimization, which runs OPTIMIZE, VACUUM and ANALYZE on managed tables as needed Correct answer
  2. BAuto compaction, which runs OPTIMIZE, VACUUM and ANALYZE after each write on both managed and external tables
  3. COptimized writes, which bin-pack data during each write and remove files no longer referenced by the table
  4. DA Lakeflow Job with a For each task that runs OPTIMIZE and VACUUM on every table in the catalog nightly

Answer and explanation: Predictive optimization automatically identifies Unity Catalog managed tables that benefit from maintenance and runs OPTIMIZE, VACUUM and ANALYZE, removing the need for scheduled jobs. Auto compaction only compacts small files after writes; it does not vacuum or collect statistics. Optimized writes improve file sizes at write time but do not delete stale files. A nightly job works but is exactly the maintenance scheduling the team wants to avoid.

Question 7 of 10, Cost & Performance Optimization

A Delta table with 3 billion rows receives MERGE statements every five minutes, each updating about 2,000 scattered rows. Write latency is dominated by rewriting large Parquet files that contain only a few changed rows. Which change most directly reduces this write amplification?

  1. AEnable deletion vectors so changed rows are marked as removed and only the new row versions are written Correct answer
  2. BRun OPTIMIZE after each MERGE so the files that MERGE has to rewrite are compacted into larger files
  3. CSet delta.targetFileSize to 1 GB so that each MERGE touches fewer but larger files when it rewrites data
  4. DEnable the Change Data Feed so MERGE writes only the changed rows to the _change_data folder of the table

Answer and explanation: With deletion vectors, MERGE, UPDATE and DELETE mark affected rows in a small vector file and write new versions, avoiding a full rewrite of each touched data file. Change Data Feed records changes for consumers but does not stop the table files from being rewritten. Compacting after every MERGE adds work and larger files make rewrites more expensive. Larger target files increase the cost of rewriting a file that contains one changed row.

Question 8 of 10, Ensuring Data Security and Compliance. Choose 2.

A streaming pipeline writes customer events containing email addresses to a bronze table. Compliance requires that analysts never see raw emails, while analysts must still be able to count distinct customers and join events across tables by customer. Which TWO techniques meet these requirements? (Choose two.)

  1. AApply a column mask on the bronze email column so that only the compliance group sees the raw value Correct answer
  2. BSuppress the email column entirely in all layers, including bronze, so that no one including compliance holds it
  3. CGeneralize email in silver to only its domain part so that analysts can still join events to individual customers
  4. DReplace email in silver with a keyed hash such as sha2 over the email concatenated with a secret salt from a secret scope Correct answer
  5. EReplace email in silver with a random UUID generated per event so that no value can be linked to a person

Answer and explanation: A salted hash produces a stable pseudonym that supports joins and distinct counts without exposing the email, and a mask on bronze restricts raw values to the authorised group. A random UUID per event breaks linkage across events. Generalizing to the domain loses the per-customer key. Removing the value everywhere prevents legitimate compliance use and erasure processing.

Question 9 of 10, Data Governance

A company wants tag values for data sensitivity to be restricted to an approved list (public, internal, confidential) and applied consistently across all workspaces. Which Unity Catalog capability supports this?

  1. AWorkspace-local tags created per workspace, with the same names typed manually by each steward
  2. BGoverned tags with a tag policy that defines the allowed values and who may assign them Correct answer
  3. CTable comments starting with a sensitivity keyword, enforced by a CHECK constraint on each table
  4. DFree-form tags on each table, with a monthly audit query that reports invalid tag values

Answer and explanation: Governed tags are defined at the account level with policies restricting allowed values and assignment permissions, ensuring consistency. Free-form tags allow any value and only detect problems after the fact. Comments are free text and CHECK constraints validate data, not metadata. Typing tags manually per workspace is inconsistent.

Question 10 of 10, Debugging and Deploying

A Spark job's stage shows 200 tasks, and the Spark UI Executors tab reports high GC time and several executors lost with "ExecutorLostFailure ... exceeding memory limits". Shuffle read per task is several GB. What is the most effective remediation?

  1. ARaise the shuffle partition count so each task reads less data, letting AQE coalesce small ones Correct answer
  2. BDisable AQE so Spark stops changing partition counts mid-stage
  3. CIncrease spark.driver.memory because executor loss in a stage is caused by the driver running out of memory
  4. DDecrease the number of shuffle partitions to 50 so each executor runs fewer tasks and has more memory per task

Answer and explanation: Tasks reading several gigabytes of shuffle data exhaust executor memory; more partitions reduce data per task, and AQE can still coalesce small ones. Fewer partitions make each task larger and worsen the problem. Executor loss here is an executor memory issue, not driver memory. Disabling AQE removes optimisations without addressing the oversized partitions.