feat(dq): add create_dq_job + prepare_create_dq_job MCP tools by vvaks0 · Pull Request #88 · collibra/chip

vvaks0 · 2026-07-01T20:27:17Z

🎯 What does this PR do?

Adds a guided, multi-turn data-quality job creation flow for DEV-197672.

prepare_create_dq_job (read-only) walks the discovery chain — resolves the edge connection, detects the job type (PUSHDOWN/PULLUP) from its capabilities, and enumerates data sources, schemas, tables and columns — returning a ready-to-use plan. It also resolves the location from a catalog Table asset (id / URL / name) for the table-asset-page entry point.
create_dq_job builds and submits the job via the PUBLIC DQ API (POST /rest/dq/1.0/jobs) behind a confirm-checkpoint preview. Supports column selection, a single-column row filter, row sampling, time-slice scheduling (${rd}/${rdEnd} composed into a dialect-aware source query), recurring schedules, monitors + adaptive settings, notifications, Pullup sizing / Parallel JDBC, and Pushdown compute.
Discovery uses internal BFF endpoints, since the public DQ API exposes no connection/edge metadata browse.

Validated end-to-end on a live stack: create -> persist -> run FINISHED.

Refs: DEV-197672

✅ Checklist

My code follows the style guidelines of this project.
I have performed a self-review of my own code.
I have commented my code, particularly in hard-to-understand areas.
I have made corresponding changes to the documentation (if needed).
My commit messages follow the Conventional Commits standard.

EricWarnerCollibra · 2026-07-02T10:28:47Z

+	return &chip.Tool[Input, Output]{
+		Name:  "prepare_create_dq_job",
+		Title: "Prepare to Create Data Quality Job",
+		Description: "Read-only companion to create_dq_job. Walks the data-quality wizard's discovery chain — resolve the edge " +


@vvaks0 I would be careful with too many Collibra-coded words within this which LLMs might have a challenge understanding. I know this is a companion tool, but it might make sense to explain a bit more what create_dq_job is, as well as be more explicit with words like 'job', 'edge', etc

@EricWarnerCollibra My one concern is that job might be too generic and could collide with other tools. Would you prefer we unpack dq to data_quality?

@vvaks0 yes, for sure on the tool name. I also might suggest adding more detail in the description, as an LLM may be viewing this tool independently. Overall, a few examples of user questions per tool, especially ones that are vague, are really helpful for us to understand behavior via evaluations

@EricWarnerCollibra addressed

aberkowCollibra · 2026-07-14T18:05:15Z

@vvaks0 do we have tests on the DQ side to ensure these API contracts will not change and will fail if someone modifies?

aberkowCollibra · 2026-07-14T19:16:36Z

@vvaks0

We'd like any DQ admin tools (create/edit/delete) behind a flag, similar to --experimental we have for the Data Product skills
We need more in depth descriptions
We need to map to personas. I'm told existing examples in repo. https://engineering-collibra.atlassian.net/wiki/spaces/AIENG/pages/19152601130/Chip+Tool+Personas+and+Permission+Mapping?atlOrigin=eyJpIjoiNjY2YzdkOGZhNDA1NDUwN2JmMTI0MzM2MTAzYzYxYjgiLCJwIjoiY29uZmx1ZW5jZS1jaGF0cy1pbnQifQ

…imental, clarify descriptions Addresses review feedback on #88. - Drop the internal /jobs/name, /jobs/{name}/exists, and /jobs/{name}/validJobName calls. Naming now relies on the PUBLIC create API: omitting jobName lets the server auto-assign a collision-free name (returned in the response); a user-supplied name is validated client-side against the server's rules (charset + no leading hyphen). - Rename the tools to create_data_quality_job / prepare_create_data_quality_job and expand their descriptions with plain-language framing (job/edge/connection) plus example user requests, for better LLM comprehension and evaluations. - Gate both tools behind a new "data-quality" experimental feature (off by default), like the Data Product skills, since they write to Collibra. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

svc-snyk-github-jira · 2026-07-15T23:34:14Z

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues
✅	Code Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Adds a guided, multi-turn data-quality job creation flow for DEV-197672. prepare_create_dq_job (read-only) walks the discovery chain — resolve the edge connection, detect the job type (PUSHDOWN/PULLUP) from its capabilities, and enumerate data sources, schemas, tables and columns — returning a ready-to-use plan. It also resolves the location from a catalog Table asset (id / URL / name) for the table-asset-page entry point. create_dq_job builds and submits the job via the PUBLIC DQ API (POST /rest/dq/1.0/jobs) behind a confirm-checkpoint preview. It supports column selection, a single-column row filter, row sampling, time-slice scheduling (${rd}/${rdEnd} composed into a dialect-aware source query), recurring schedules, monitors + adaptive settings, notifications, Pullup sizing / Parallel JDBC, and Pushdown compute. Discovery uses internal BFF endpoints (the public DQ API exposes no connection/edge metadata browse). Validated end-to-end on a live stack: create -> persist -> run FINISHED. Refs: DEV-197672 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…imental, clarify descriptions Addresses review feedback on #88. - Drop the internal /jobs/name, /jobs/{name}/exists, and /jobs/{name}/validJobName calls. Naming now relies on the PUBLIC create API: omitting jobName lets the server auto-assign a collision-free name (returned in the response); a user-supplied name is validated client-side against the server's rules (charset + no leading hyphen). - Rename the tools to create_data_quality_job / prepare_create_data_quality_job and expand their descriptions with plain-language framing (job/edge/connection) plus example user requests, for better LLM comprehension and evaluations. - Gate both tools behind a new "data-quality" experimental feature (off by default), like the Data Product skills, since they write to Collibra. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

vvaks0 · 2026-07-15T23:56:08Z

@vvaks0 do we have tests on the DQ side to ensure these API contracts will not change and will fail if someone modifies?

@aberkow Collibra Here is what I am going to propose internally but it will take a bit of time to work through it with the team as there is some team level messaging required here.

https://github.com/collibra/dq/pull/7591

vvaks0 · 2026-07-15T23:58:17Z

@vvaks0

We'd like any DQ admin tools (create/edit/delete) behind a flag, similar to --experimental we have for the Data Product skills

We need more in depth descriptions

We need to map to personas. I'm told existing examples in repo. https://engineering-collibra.atlassian.net/wiki/spaces/AIENG/pages/19152601130/Chip+Tool+Personas+and+Permission+Mapping?atlOrigin=eyJpIjoiNjY2YzdkOGZhNDA1NDUwN2JmMTI0MzM2MTAzYzYxYjgiLCJwIjoiY29uZmx1ZW5jZS1jaGF0cy1pbnQifQ

@aberkowCollibra Done

aberkowCollibra · 2026-07-16T17:37:51Z

@EricWarnerCollibra @vvaks0 all changes requested have been made and verified.

selectedColumns projects the chosen columns into the source query's SELECT list (falling back to SELECT * only when omitted), per dqColumnsPart in pkg/clients/dq_source_query.go. The schema description wrongly claimed it left the query as SELECT *. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

vvaks0 requested a review from a team as a code owner July 1, 2026 20:27

EricWarnerCollibra reviewed Jul 2, 2026

View reviewed changes

EricWarnerCollibra mentioned this pull request Jul 14, 2026

feat(dq): add DQ rule lifecycle + job observability MCP tools #91

Draft

5 tasks

vvaks0 and others added 2 commits July 15, 2026 19:43

vvaks0 force-pushed the feat/dev-197672-create-dq-job branch from 749732a to deb3c62 Compare July 15, 2026 23:44

aberkowCollibra requested a review from EricWarnerCollibra July 16, 2026 17:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(dq): add create_dq_job + prepare_create_dq_job MCP tools#88

feat(dq): add create_dq_job + prepare_create_dq_job MCP tools#88
vvaks0 wants to merge 3 commits into
mainfrom
feat/dev-197672-create-dq-job

vvaks0 commented Jul 1, 2026 •

edited

Loading

Uh oh!

EricWarnerCollibra Jul 2, 2026

Uh oh!

vvaks0 Jul 14, 2026

Uh oh!

EricWarnerCollibra Jul 14, 2026

Uh oh!

vvaks0 Jul 15, 2026

Uh oh!

aberkowCollibra commented Jul 14, 2026

Uh oh!

aberkowCollibra commented Jul 14, 2026

Uh oh!

svc-snyk-github-jira commented Jul 15, 2026 •

edited

Loading

Uh oh!

vvaks0 commented Jul 15, 2026 •

edited

Loading

Uh oh!

vvaks0 commented Jul 15, 2026

Uh oh!

aberkowCollibra commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

vvaks0 commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🎯 What does this PR do?

✅ Checklist

Uh oh!

EricWarnerCollibra Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

vvaks0 Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

EricWarnerCollibra Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

vvaks0 Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

aberkowCollibra commented Jul 14, 2026

Uh oh!

aberkowCollibra commented Jul 14, 2026

Uh oh!

svc-snyk-github-jira commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Snyk checks have passed. No issues have been found so far.

Uh oh!

vvaks0 commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vvaks0 commented Jul 15, 2026

Uh oh!

aberkowCollibra commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vvaks0 commented Jul 1, 2026 •

edited

Loading

svc-snyk-github-jira commented Jul 15, 2026 •

edited

Loading

vvaks0 commented Jul 15, 2026 •

edited

Loading