Improve sandbox application guidance to reduce auto-close rate

[angellk]

Summary

Add inline validation guidance to the sandbox application template to prevent common blockers that cause applications to be auto-closed without TOC review.

Based on Q2 2026 analysis: 78% of applications (7 of 9) were auto-closed due to preventable issues.

---

Changes

1. MAINTAINERS File Guidance Enhancement

Problem: 33% of applications (3/9) provided invalid MAINTAINERS file links:

• Contributors graph instead of actual file
• "N/A" or "will be added after acceptance"
• External website links

Solution:

• Require direct GitHub /blob/ link to file
• Mandate Company/Organization column in table format
• Provide clear valid/invalid examples
• Show example table structure

2. Organization Diversity Clarification

Problem: 78% of applications (7/9) failed org diversity requirement due to confusion between:

• Employer organizations (what counts)
• GitHub organization memberships (what doesn't count)

Solution:

• Add prominent notice after MAINTAINERS field explaining:
  • ✅ Different employers count (Acme Corp, Example Inc)
  • ❌ Different GitHub orgs don't count (kubernetes, prometheus, envoy)
• Emphasize that all maintainers from same company = single org
• Require Company/Organization column to demonstrate diversity

3. License Compliance Timing

Problem: Applications with BSL (Business Source License) promising to "convert to Apache 2.0 upon acceptance"

Solution:

• Emphasize license must be compliant BEFORE acceptance
• Explicitly list prohibited licenses: GPL, LGPL, AGPL, BSL
• No "will convert later" promises accepted
• Clarify allowlist vs preferred licenses

4. Reference Architecture/Implementation Check

Problem: Projects that are reference architectures/implementations submitted as reusable projects

Solution:

• New required checkbox distinguishing:
  • Reusable projects (builds NEW tool)
  • Reference architectures/implementations (shows HOW to use existing tools)
• Routes reference architectures to architecture.cncf.io
• Uses key terms: "reference architecture", "reference implementation", "implementation of"

5. Subproject Separation Requirement

Problem: Subprojects under existing project organizations without parent maintainer approval

Solution:

• Require parent project maintainer vote for ALL subprojects (not just CNCF parents)
• Must link to public GitHub/GitLab issue documenting vote
• Clarifies this applies to ANY parent project, not just CNCF projects

6. Pre-Submission Checklist

Solution:

• Self-validation checklist before submit
• Lists all critical auto-close criteria
• Common mistakes to avoid
• Expected to reduce wasted applications significantly

---

Expected Impact

Current (Q2 2026):

• 78% auto-close rate (7 of 9 applications)
• Top blockers: Org diversity (78%), Invalid MAINTAINERS (33%)

Target after changes:

• <20% auto-close rate
• 60% reduction in org diversity failures
• 80% reduction in invalid MAINTAINERS links
• Zero reference architecture mis-submissions

---

Examples of Improved Guidance

Before:

label: Maintainers file
description: Provide the URL of the project's maintainers file

After:

label: Maintainers file
description: |
  Requirements:
  - Must be direct GitHub /blob/ link
  - Table with Name, GitHub ID, Company/Organization columns
  - Minimum 3 maintainers from 2+ organizations
  
  ❌ Invalid: https://github.com/org/repo/graphs/contributors
  ✅ Valid: https://github.com/org/repo/blob/main/MAINTAINERS.md

---

Testing

• Template syntax validates in GitHub
• All markdown renders correctly
• Checkboxes work as expected
• No specific project names used as examples
• Language is clear and actionable

---

Related

• Based on findings from TOC sandbox evaluation agent improvements
• Addresses root causes identified in Q2 2026 application review
• Complements automation work in toc-sandbox-evaluate tool

🤖 Generated with Claude Code

[joshgav]

These are useful additions! Frankly, I didn't know that number and diversity of maintainers are requirements for sandbox.

Does this proposal view .github/ISSUE_TEMPLATE/application.yaml as the canonical place for project guidance? Is there another place where these requirements are listed? Should there be a guidance document instead of including it in the issue form? On the other hand, including details in the form makes it more likely people will read it, so LGTM!

[joshgav]

Wanted to bring up - is it a requirement for sandbox projects to have maintainers from multiple orgs? I looked but haven't yet found it documented.