Application Development: Databricks Generative AI Engineer Associate Practice Questions

Correct answer: B

WHY B is correct: In LCEL, the pipe operator (|) composes Runnable objects into a pipeline. The chain is defined as retriever | prompt | llm | StrOutputParser() and streaming is activated by calling .stream(input) on the composed chain object. The LLM node automatically yields tokens when .stream() is called at the chain level. WHY NOT A: .stream() is an invocation method, not a pipe-operator argument. Writing llm.stream() inline with | produces a syntax error — the pipe operator expects Runnable objects, not coroutine call results. WHY NOT C: LLMChain is the legacy pre-LCEL class. It does not accept retriever= as a constructor parameter and does not support stream=True as a run() keyword argument in the standard API. WHY NOT D: Manually chaining .invoke(), .format(), and .predict() produces sequential blocking synchronous calls. It does not compose a streaming pipeline and provides no real-time token delivery.

Correct answer: B

WHY B is correct: A classifier-based intent guardrail accurately distinguishes the user's semantic intent — 'I have chest pain, what's wrong with me?' (seeking diagnosis) versus 'How does the heart pump blood?' (seeking general education). This allows the assistant to refuse and redirect diagnosis requests while remaining useful for educational queries. WHY NOT A: A keyword blocklist is overly broad and brittle. Terms like 'symptoms' or 'treatment' appear in legitimate general health education questions. Blocking on keywords would incorrectly refuse a large proportion of benign queries. WHY NOT C: A short max_tokens limit prevents detailed responses but does not prevent the model from partially providing a diagnosis within 50 tokens. It also truncates legitimate general health explanations. WHY NOT D: Removing medical training data degrades the model's ability to answer general health education questions — which is a core use case. This approach is also impractical for foundation models whose pretraining is fixed.

Correct answer: C

WHY C is correct: Applying both constraints: Execution Accuracy > 0.87 — Model X (0.88 ✓), Model Y (0.91 ✓), Model Z (0.86 ✗). Cost < $0.015 — Model X ($0.012 ✓), Model Y ($0.047 ✗), Model Z ($0.003 ✓). Only Model X satisfies both constraints simultaneously. WHY NOT A: Model Y achieves the highest accuracy but its $0.047 cost is more than 3× the $0.015 budget ceiling. It fails the cost constraint and is ineligible. WHY NOT B: Model Z costs only $0.003 per query but its Execution Accuracy of 0.86 falls below the required 0.87 threshold. It fails the accuracy constraint. WHY NOT D: Model Y fails the cost constraint ($0.047 > $0.015), so it does not qualify. There is only one model that meets both criteria, making team familiarity irrelevant as a selection factor.

Correct answer: B

WHY B is correct: This template correctly separates the retrieved, targeted context (earnings data fetched using the extracted tickers NVDA, MSFT and period Q3) from the user question, and explicitly grounds the model with 'Answer using only the data provided above.' This is the canonical retrieval-augmented prompting structure. WHY NOT A: Directing the model to answer from training data is the opposite of augmentation. Training data may contain stale or incorrect quarterly figures. WHY NOT C: Injecting only the ticker symbols without the actual retrieved earnings data leaves the model without factual context. It may still hallucinate specific revenue figures. WHY NOT D: Injecting all company data indiscriminately bloats the context window with irrelevant records and introduces noise that degrades the quality of the targeted comparison.

Correct answer: C

WHY C is correct: Per the Databricks Agent Framework documentation, after logging the agent, it must be registered to Unity Catalog using mlflow.set_registry_uri('databricks-uc') followed by mlflow.register_model(model_uri=logged_agent_info.model_uri, name='catalog.schema.model_name'). Registration packages the agent as a Unity Catalog model, which is required before deployment to a Model Serving endpoint. WHY NOT A: mlflow.end_run() closes the active MLflow experiment run but does not register or deploy the model. A logged model that is not registered cannot be deployed to Model Serving. WHY NOT B: MLflow handles artifact storage automatically in its configured artifact store. Manual cloud storage upload is neither required nor part of the Agent Framework deployment workflow. WHY NOT D: Code-based logging captures the code file path to execute at serving time. The artifact is the code path reference, not a time-limited generated artifact. Re-running the notebook is not part of the deployment process.

Correct answer: C

WHY C is correct: Faithfulness hallucination occurs when a model generates content inconsistent with or contradictory to the provided retrieval context. The model produced '60 days' despite the retrieved document stating '30 days' — a direct faithfulness violation. WHY NOT A: Prompt injection involves a malicious or cleverly crafted user input that causes the model to bypass its instructions (e.g., 'ignore the above'). The user's question here is direct and benign. WHY NOT B: Context stuffing refers to retrieving too many document chunks that may confuse the model. A single document being contradicted by the model's output is a faithfulness issue, not a retrieval volume issue. WHY NOT D: Token limit truncation results in missing or incomplete information, typically visible as abrupt cut-off generation. Here the model produced a confident, complete — but factually incorrect — answer, which is characteristic of hallucination, not truncation.

Correct answer: C

WHY C is correct: Offline evaluation is a static, pre-deployment quality gate run on a fixed benchmark dataset to decide whether to release. Online monitoring is continuous and post-deployment, applying evaluation logic (including LLM judges) to live production samples to detect quality regression, data drift, and safety violations over time. The key differences are: data source (static benchmark vs. live traffic), frequency (point-in-time vs. continuous), and purpose (deployment gate vs. production health tracking). WHY NOT A: Databricks explicitly supports using the same LLM judge configuration — including faithfulness — across both offline evaluation and online monitoring. Faithfulness measurement is not restricted to offline use. WHY NOT B: Modern Gen AI monitoring systems (including Databricks Agent Evaluation) use LLM judges to assess quality dimensions well beyond system-level latency and error rates. WHY NOT D: While the same LLM judge configuration is commonly reused across both phases, the processes differ fundamentally in dataset type, cadence, and purpose. Stating they are identical would eliminate the distinction that the Gen AI application lifecycle explicitly makes between them.

Correct answer: B

WHY B is correct: For a coding assistant, performance on code benchmarks (HumanEval measures pass@k for synthesized Python functions; MBPP evaluates diverse programming problems) directly predicts the ability to generate syntactically correct, idiomatic code. Code-completion fine-tuning further specializes the model for the exact task. WHY NOT A: Knowledge cutoff primarily matters for factual or current-events tasks. Python and SQL syntax, standard libraries, and best practices evolve slowly; a recent cutoff is far less predictive of code quality than code-specific training. WHY NOT C: Conversational empathy and sentiment analysis are needed for customer-facing emotional support chatbots — completely unrelated to technical code generation. WHY NOT D: Creative narrative generation capabilities do not transfer to code synthesis correctness. These are orthogonal model attributes.