Add 2026-06-llmops-quickstart blog code

2026-06-llmops-quickstart/LICENSE.md

@@ -0,0 +1,21 @@
+Copyright (2022) Databricks, Inc.
+
+This library (the "Software") may not be used except in connection with the Licensee's use of the Databricks Platform Services pursuant to an Agreement (defined below) between Licensee (defined below) and Databricks, Inc. ("Databricks"). The Object Code version of the Software shall be deemed part of the Downloadable Services under the Agreement, or if the Agreement does not define Downloadable Services, Subscription Services, or if neither are defined then the term in such Agreement that refers to the applicable Databricks Platform Services (as defined below) shall be substituted herein for “Downloadable Services.” Licensee's use of the Software must comply at all times with any restrictions applicable to the Downlodable Services and Subscription Services, generally, and must be used in accordance with any applicable documentation. For the avoidance of doubt, the Software constitutes Databricks Confidential Information under the Agreement.
+
+Additionally, and notwithstanding anything in the Agreement to the contrary:
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+you may view, make limited copies of, and may compile the Source Code version of the Software into an Object Code version of the Software. For the avoidance of doubt, you may not make derivative works of Software (or make any any changes to the Source Code version of the unless you have agreed to separate terms with Databricks permitting such modifications (e.g., a contribution license agreement)).
+If you have not agreed to an Agreement or otherwise do not agree to these terms, you may not use the Software or view, copy or compile the Source Code of the Software.
+
+This license terminates automatically upon the termination of the Agreement or Licensee's breach of these terms. Additionally, Databricks may terminate this license at any time on notice. Upon termination, you must permanently delete the Software and all copies thereof (including the Source Code).
+
+Agreement: the agreement between Databricks and Licensee governing the use of the Databricks Platform Services, which shall be, with respect to Databricks, the Databricks Terms of Service located at www.databricks.com/termsofservice, and with respect to Databricks Community Edition, the Community Edition Terms of Service located at www.databricks.com/ce-termsofuse, in each case unless Licensee has entered into a separate written agreement with Databricks governing the use of the applicable Databricks Platform Services.
+
+Databricks Platform Services: the Databricks services or the Databricks Community Edition services, according to where the Software is used.
+
+Licensee: the user of the Software, or, if the Software is being used on behalf of a company, the company.
+
+Object Code: is version of the Software produced when an interpreter or a compiler translates the Source Code into recognizable and executable machine code.
+
+Source Code: the human readable portion of the Software.

2026-06-llmops-quickstart/README.md

@@ -0,0 +1,152 @@
+# LLMOps Quickstart for Databricks
+
+A minimal but complete end-to-end **LLMOps** example on Databricks, demonstrating the full lifecycle of an LLM-powered application:
+
+**Data Ingestion → Agent Build → Evaluation → Deployment → Inference**
+
+Use case: a **customer support ticket classifier** that uses a Databricks Foundation Model to categorize free-text tickets into `billing`, `technical_issue`, `feature_request`, `account_management`, or `other`.
+
+> 📝 **Blog post:** _link to be added once published on the [Databricks Community](https://community.databricks.com/) platform._
+
+> ⚠️ **Disclaimer:** This is **not production-ready code**. It is provided **as-is**, for educational purposes, and support is available on a **best-effort basis**. If you run into problems, please [open an issue](https://github.com/databricks-solutions/databricks-blogposts/issues).
+
+This example accompanies the blog post and is intended for **educational purposes**. It is unofficial and unsupported (see [Licensing](#licensing)).
+
+---
+
+## What it demonstrates
+
+The repository shows how to wrap a hosted LLM in the standard Databricks MLOps building blocks so that an LLM application becomes reproducible, governed, and repeatably deployable:
+
+- An agent logged to **MLflow** as a `ChatAgent`, calling a **Foundation Model API** endpoint (default `databricks-claude-sonnet-4-6`) through the OpenAI-compatible client.
+- An **evaluation gate** that promotes a model version to the **Champion** alias in **Unity Catalog** only when accuracy clears a threshold (default 80%).
+- A **Databricks Asset Bundle** that deploys the Unity Catalog schema, the MLflow experiment, and every job with one command, with separate `dev` and `prod` targets.
+- Both **batch** and **real-time** inference against the deployed **Mosaic AI Model Serving** endpoint.
+
+---
+
+## Prerequisites
+
+- [Databricks CLI](https://docs.databricks.com/dev-tools/cli/install.html) v0.200+
+- A Databricks workspace with:
+  - Unity Catalog enabled
+  - Foundation Model APIs enabled (for the default `databricks-claude-sonnet-4-6` endpoint)
+  - Permissions to create schemas, registered models, jobs, and Model Serving endpoints
+
+---
+
+## Quickstart
+
+### 1. Authenticate
+
+```bash
+databricks auth login --host https://<your-workspace>.cloud.databricks.com
+```
+
+### 2. Deploy the bundle
+
+From this folder:
+
+```bash
+databricks bundle deploy
+```
+
+This creates the Unity Catalog schema, MLflow experiment, and all jobs in your workspace under your user directory.
+
+### 3. Run the pipeline
+
+```bash
+# Step 1 — ingest sample support tickets into a Delta table
+databricks bundle run data_preprocessing_job
+
+# Step 2 — build and evaluate the classifier; promote to Champion if accuracy >= 80%
+databricks bundle run model_build_evaluation_job
+
+# Step 3 — deploy the Champion model to a Model Serving endpoint
+databricks bundle run model_deployment_job
+
+# Step 4 — run batch inference over all tickets
+databricks bundle run batch_inference_job
+```
+
+---
+
+## Configuration
+
+All configuration is exposed as bundle variables with sensible defaults — no edits to source files are needed for most workspaces.
+
+| Variable | Default | Description |
+|---|---|---|
+| `catalog_name` | `main` | Unity Catalog catalog (must already exist) |
+| `schema_name` | `llmops_quickstart` | UC schema (created by the bundle) |
+| `model_name` | `support_ticket_classifier` | Registered model name |
+| `llm_endpoint` | `databricks-claude-sonnet-4-6` | Foundation Model API endpoint used by the agent |
+
+Override at deploy time, e.g.:
+
+```bash
+databricks bundle deploy \
+  -v catalog_name=my_catalog \
+  -v llm_endpoint=databricks-meta-llama-3-3-70b-instruct
+```
+
+The `prod` target uses `llmops_quickstart_prod` as the schema name:
+
+```bash
+databricks bundle deploy --target prod
+```
+
+---
+
+## Project structure
+
+```
+notebooks/
+  1_data_preprocessing/
+    data_ingestion.py         # Creates support_tickets Delta table (30 labelled rows)
+  2_model_build_and_deploy/
+    quickstart_agent.py       # MLflow ChatAgent definition
+    model_config.yml          # Default agent config (llm_endpoint)
+    model_build.py            # Logs agent to MLflow
+    model_evaluation.py       # Evaluates agent; promotes to Champion if accuracy >= threshold
+    model_deployment.py       # Deploys Champion to Mosaic AI Model Serving
+  3_inference/
+    batch_inference.py        # Batch predictions written to inference_results table
+    realtime_inference.py     # Live queries via the OpenAI-compatible API
+resources/                    # Bundle job + schema/experiment definitions
+databricks.yml                # Bundle entry point — targets, variables
+docs/img/                     # Architecture diagrams used in the blog post
+```
+
+---
+
+## How it works
+
+1. **Data Ingestion** — 30 hand-written, synthetic support tickets (6 per category) are written to a Delta table in Unity Catalog. No real or PII data is used.
+2. **Model Build** — `quickstart_agent.py` is logged as an MLflow `ChatAgent`. The configured LLM endpoint is baked into the model artifact via `mlflow.models.ModelConfig`.
+3. **Evaluation** — the logged agent runs predictions on all tickets. If accuracy meets the threshold (default 80%), the model is registered in Unity Catalog and aliased as **Champion**.
+4. **Deployment** — the Champion version is deployed to a Mosaic AI Model Serving endpoint via `databricks.agents.deploy()`.
+5. **Inference** — batch inference loads the Champion model directly; real-time inference queries the serving endpoint via the OpenAI-compatible API.
+
+---
+
+## Data
+
+The only dataset is **30 small, synthetic support tickets** generated by hand in `notebooks/1_data_preprocessing/data_ingestion.py`. There is no external dataset, no customer data, and no PII.
+
+---
+
+## Licensing
+
+- This folder is provided under the repository's Databricks license — see [`LICENSE.md`](./LICENSE.md). The Databricks license is **not modified**.
+- Dependencies used by this example and their licenses:
+  - [MLflow](https://github.com/mlflow/mlflow) — Apache License 2.0
+  - [Databricks SDK for Python](https://github.com/databricks/databricks-sdk-py) — Apache License 2.0
+  - [`databricks-agents`](https://docs.databricks.com/en/generative-ai/agent-framework/build-genai-apps.html) — Databricks
+  - [OpenAI Python client](https://github.com/openai/openai-python) — Apache License 2.0
+
+---
+
+## Acknowledgements
+
+The structure of this example follows the established [databricks-solutions/mlops-quickstart](https://github.com/databricks-solutions/mlops-quickstart) repository, adapted for an LLM/agent workload. Portions of the code and the accompanying blog draft were developed with the assistance of AI tooling and reviewed by the author.

2026-06-llmops-quickstart/databricks.yml

@@ -0,0 +1,38 @@
+bundle:
+  name: llmops-quickstart
+
+include:
+  - resources/*.yml
+
+workspace:
+  root_path: /Workspace/Users/${workspace.current_user.userName}/.bundle/${bundle.name}/${bundle.target}
+
+variables:
+  catalog_name:
+    description: "Unity Catalog catalog name (must already exist)"
+    default: main
+  schema_name:
+    description: "Unity Catalog schema name (created by the bundle if absent)"
+    default: llmops_quickstart
+  model_name:
+    description: "Registered model name in Unity Catalog"
+    default: support_ticket_classifier
+  llm_endpoint:
+    description: "Databricks Foundation Model API endpoint used by the agent"
+    default: databricks-claude-sonnet-4-6
+  environment:
+    description: "Deployment environment label"
+    default: dev
+
+targets:
+  dev:
+    mode: development
+    default: true
+    variables:
+      environment: dev
+
+  prod:
+    mode: production
+    variables:
+      schema_name: llmops_quickstart_prod
+      environment: prod

2026-06-llmops-quickstart/notebooks/1_data_preprocessing/data_ingestion.py

@@ -0,0 +1,88 @@
+# Databricks notebook source
+# Uses Databricks Serverless Environment v5 (configured in job resource YAML).
+# No %pip install needed — all required packages are pre-installed.
+
+# COMMAND ----------
+# MAGIC %md
+# MAGIC # Data Ingestion
+# MAGIC
+# MAGIC Creates sample customer support tickets in a Unity Catalog Delta table.
+# MAGIC These records serve as both the evaluation dataset and batch inference input.
+
+# COMMAND ----------
+
+dbutils.widgets.text("catalog_name", "main")
+dbutils.widgets.text("schema_name", "llmops_quickstart")
+catalog_name = dbutils.widgets.get("catalog_name")
+schema_name = dbutils.widgets.get("schema_name")
+
+# COMMAND ----------
+# MAGIC %md
+# MAGIC ## Create schema
+
+# COMMAND ----------
+
+spark.sql(f"CREATE SCHEMA IF NOT EXISTS {catalog_name}.{schema_name}")
+
+# COMMAND ----------
+# MAGIC %md
+# MAGIC ## Create support tickets table
+
+# COMMAND ----------
+
+tickets = [
+    # billing
+    (1,  "I was charged twice for my subscription last month. Please refund the extra charge.", "billing"),
+    (2,  "My invoice shows a different amount than what I was quoted. Can you explain the discrepancy?", "billing"),
+    (3,  "I cancelled my plan three weeks ago but still got charged. I need an immediate refund.", "billing"),
+    (4,  "How do I update my payment method? My credit card expired.", "billing"),
+    (5,  "I'd like to downgrade my plan to save money. What are the pricing options?", "billing"),
+    (6,  "Can I get an itemized invoice for my last three months of service?", "billing"),
+    # technical_issue
+    (7,  "The mobile app crashes every time I try to upload a photo. Running iOS 17.", "technical_issue"),
+    (8,  "I keep getting a 500 error when trying to export my reports to PDF.", "technical_issue"),
+    (9,  "The dashboard stopped loading after your last update. I just see a blank screen.", "technical_issue"),
+    (10, "Two-factor authentication is not sending me the SMS code.", "technical_issue"),
+    (11, "My data sync between devices stopped working two days ago.", "technical_issue"),
+    (12, "Search results are returning empty even though I know the data exists.", "technical_issue"),
+    # feature_request
+    (13, "It would be great if you could add dark mode to the dashboard.", "feature_request"),
+    (14, "Can you add bulk export functionality to the reporting module?", "feature_request"),
+    (15, "Please add keyboard shortcuts to the editor — it would speed up my workflow a lot.", "feature_request"),
+    (16, "I'd love to see a Slack integration so I get notifications in my team channel.", "feature_request"),
+    (17, "Would it be possible to add an undo button to the data editor?", "feature_request"),
+    (18, "Please add a public API so we can integrate with our internal tools.", "feature_request"),
+    # account_management
+    (19, "I need to transfer ownership of my account to a colleague.", "account_management"),
+    (20, "How do I add team members to my organization account?", "account_management"),
+    (21, "My password reset email never arrived. I've been locked out for 2 days.", "account_management"),
+    (22, "I want to delete my account and all associated data.", "account_management"),
+    (23, "Can I merge two accounts under the same email address?", "account_management"),
+    (24, "How do I enable SSO login for my company?", "account_management"),
+    # other
+    (25, "What are your business hours for live chat support?", "other"),
+    (26, "Do you offer any discounts for non-profit organizations?", "other"),
+    (27, "I'd like to leave a positive review — your support team was amazing.", "other"),
+    (28, "Is your platform SOC 2 Type II certified?", "other"),
+    (29, "What data centers do you use and where are they located?", "other"),
+    (30, "Do you have a referral program? I'd like to recommend you to clients.", "other"),
+]
+
+from pyspark.sql.types import StructType, StructField, IntegerType, StringType
+
+schema = StructType([
+    StructField("id", IntegerType(), False),
+    StructField("ticket", StringType(), False),
+    StructField("category", StringType(), False),
+])
+
+df = spark.createDataFrame(tickets, schema=schema)
+
+df.write.mode("overwrite").saveAsTable(f"{catalog_name}.{schema_name}.support_tickets")
+
+display(spark.read.table(f"{catalog_name}.{schema_name}.support_tickets"))
+
+# COMMAND ----------
+
+print(f"Created table: {catalog_name}.{schema_name}.support_tickets")
+print(f"Row count: {spark.read.table(f'{catalog_name}.{schema_name}.support_tickets').count()}")

2026-06-llmops-quickstart/notebooks/2_model_build_and_deploy/model_build.py

@@ -0,0 +1,70 @@
+# Databricks notebook source
+# Uses Databricks Serverless Environment v5 (configured in job resource YAML).
+# databricks-openai is added via the job's environment spec; no %pip install needed here.
+
+# COMMAND ----------
+# MAGIC %md
+# MAGIC # Model Build
+# MAGIC
+# MAGIC Logs the `TicketClassifierAgent` to an MLflow experiment run and stores the
+# MAGIC run ID for use by the evaluation task.
+
+# COMMAND ----------
+
+dbutils.widgets.text("catalog_name", "main")
+dbutils.widgets.text("schema_name", "llmops_quickstart")
+dbutils.widgets.text("model_name", "support_ticket_classifier")
+dbutils.widgets.text("experiment_name", f"/Users/{dbutils.notebook.entry_point.getDbutils().notebook().getContext().userName().get()}/llmops_quickstart")
+dbutils.widgets.text("llm_endpoint", "databricks-claude-sonnet-4-6")
+
+catalog_name = dbutils.widgets.get("catalog_name")
+schema_name = dbutils.widgets.get("schema_name")
+model_name = dbutils.widgets.get("model_name")
+experiment_name = dbutils.widgets.get("experiment_name")
+llm_endpoint = dbutils.widgets.get("llm_endpoint")
+
+registered_model_name = f"{catalog_name}.{schema_name}.{model_name}"
+
+# COMMAND ----------
+# MAGIC %md
+# MAGIC ## Log agent to MLflow
+
+# COMMAND ----------
+
+import mlflow
+import datetime
+from mlflow.models.resources import DatabricksServingEndpoint
+
+mlflow.set_registry_uri("databricks-uc")
+mlflow.set_experiment(experiment_name)
+
+resources = [DatabricksServingEndpoint(endpoint_name=llm_endpoint)]
+
+timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
+
+# quickstart_agent.py lives in the same directory as this notebook.
+# model_config is written into the artifact so the agent reads llm_endpoint at serving time.
+with mlflow.start_run(run_name=f"build_{timestamp}") as run:
+    logged_model_info = mlflow.pyfunc.log_model(
+        artifact_path="agent",
+        python_model="quickstart_agent.py",
+        model_config={"llm_endpoint": llm_endpoint},
+        resources=resources,
+        pip_requirements=[
+            "mlflow",
+            "databricks-openai",
+            "databricks-agents",
+            "databricks-sdk",
+            "typing_extensions",
+        ],
+    )
+    print(f"Logged model: {logged_model_info.model_uri}")
+
+# COMMAND ----------
+# MAGIC %md
+# MAGIC ## Pass run ID to downstream evaluation task
+
+# COMMAND ----------
+
+dbutils.jobs.taskValues.set(key="logged_run_id", value=logged_model_info.run_id)
+print(f"run_id: {logged_model_info.run_id}")

2026-06-llmops-quickstart/notebooks/2_model_build_and_deploy/model_config.yml

@@ -0,0 +1 @@
+llm_endpoint: "databricks-claude-sonnet-4-6"