Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/adr/0001-learning-extension-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ Each import records whether every list page and reply page was present. The impo

Hub hosts the loop and runs the `replica-learning` command for every step, so the store's rules stay the only rules. Hub keeps one store per repository in its own storage, outside any checkout. A background job imports the repository's review comments from Hub's database as one complete collection, removes the text of comments deleted on Hub, asks Copilot to label each new or edited comment, and consolidates the labels into proposals. When someone deletes a comment, another job removes its text by its Hub comment ID, which imports nothing, so it works without an active grant and adds no other comments. Repository administrators accept, edit, or reject proposals on a Learnings page. The Replica prototype plugin adds accepted learnings to the prompts a person submits in that repository. [Learnings](../../hub/docs/learnings.md) describes the product.

Hub asks Copilot through the [Copilot SDK](https://github.com/github/copilot-sdk), with a token that the person who runs Hub provides and whose only permission is Copilot requests, as [Copilot](../../hub/docs/copilot.md) describes. The session has no tools, refuses every permission request, and never sees who wrote a comment. Copilot chooses the model unless the store's configuration names one, and each run records the usage Copilot reports.
Hub asks Copilot through the [Copilot SDK](https://github.com/github/copilot-sdk), with a token whose only permission is Copilot requests: one that the person who runs Hub provides, or, with organization billing, a one-hour token that Hub's GitHub App creates for each request, as [Copilot](../../hub/docs/copilot.md) describes. The session has no tools, refuses every permission request, and never sees who wrote a comment. Copilot chooses the model unless the store's configuration names one, and each run records the usage Copilot reports.

The command still runs in a checkout, with data under `.replica/learning/`, for contributors and for other hosts. This complements #230: its discovery output can become labels, and its judge can become a consolidator.

Expand All @@ -114,7 +114,7 @@ Learning happens in explicit, inspectable memory. A training-purpose export is t
- The generated validator file is large, as in `packages/controls`.
- Hub keeps one SQLite file per repository. A team that needs a shared database server brings its own provider and runs the conformance suite against it.
- Hub's production image includes Node.js and the command, which adds about 260 MB.
- Copilot usage counts toward the token owner's plan, not necessarily the administrator who enabled the connection. The operator supplies a token intended to have only the Copilot requests permission; Hub keeps it in process configuration, not its database. The operator must verify its actual scope and manage expiry, rotation and revocation. Token replacement requires a full stop/start with no old/new process overlap, including learning and SDK children; rolling replacement is unsupported. Credential custody, subprocess isolation and deployed authentication still need security review.
- By default, Copilot usage counts toward the token owner's plan, not necessarily the administrator who enabled the connection. With organization billing, the organization that installed Hub's GitHub App pays, and anyone who controls the App's private key, which Hub already stores, can spend that organization's Copilot budget. The operator supplies a token intended to have only the Copilot requests permission; Hub keeps it in process configuration, not its database. The operator must verify its actual scope and manage expiry, rotation and revocation. Token replacement requires a full stop/start with no old/new process overlap, including learning and SDK children; rolling replacement is unsupported. Credential custody, subprocess isolation and deployed authentication still need security review.

## Alternatives considered

Expand All @@ -124,7 +124,7 @@ Learning happens in explicit, inspectable memory. A training-purpose export is t
- Dropping records that fail their checks and continuing. Rejected because dropping a damaged withdrawal or retraction silently restores a permission.
- A knowledge graph or vector store as the default. Rejected; the default should be simple and local, and graph or semantic search can arrive as provider capabilities.
- An agent collects comments through the canvas, and a starter skill runs each step. Replaced: a person had to start every step, and every comment passed through an agent session, which cost tokens and could introduce copying errors.
- Billing Copilot to the organization through the App's installation token. Deferred: it needs the App's Copilot requests permission and an installation on every repository in the organization.
- Organization billing through the App's installation tokens as the only option. Rejected: it needs Copilot Business or Copilot Enterprise and an App installed on an organization, and GitHub offers it as a preview. Hub offers it alongside the operator's token.
- PostgreSQL or Neo4j providers in this version. Deferred: SQLite meets one Hub's needs, and the provider interface and conformance suite let a team add one later.
- Model-only consolidation. Kept as a plug-in option; the reference stays deterministic so tests and reviews are reproducible.

Expand Down
3 changes: 3 additions & 0 deletions hub/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ REPLICA_ARTIFACT_ROOT=storage/artifacts
# Learnings asks Copilot with this token: a fine-grained personal access token whose only
# permission is Copilot requests. Hub never stores it. See docs/copilot.md.
# REPLICA_COPILOT_TOKEN=
# To bill the organization that installed Hub's GitHub App instead of the token's owner, give the
# App the Copilot requests organization permission and set this. Hub then ignores the token.
# REPLICA_COPILOT_BILLING=organization
# Use a non-empty secret. A long random value is recommended for public deployments.
REPLICA_BOOTSTRAP_SECRET=
# Production deployments should set independent random values for these keys.
Expand Down
29 changes: 20 additions & 9 deletions hub/app/controllers/copilot_connections_controller.rb
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# Lets Hub administrators connect Copilot with the operator's token, see who connected it, and
# disconnect it. Both changes need fresh GitHub proof, like Hub's other administrator settings.
# Hub keeps no Copilot credential; see CopilotConnection.
# Lets Hub administrators connect Copilot with the credential the operator configured, see who
# connected it, and disconnect it. Both changes need fresh GitHub proof, like Hub's other
# administrator settings. Hub keeps no Copilot credential; see CopilotConnection.
class CopilotConnectionsController < SuperglueController
include FreshAuthorizationRequired

before_action :require_github_hub
before_action :authorize_admin
before_action :confirm_change, only: %i[create destroy]
before_action :confirm_connect, only: :create
before_action :confirm_disconnect, only: :destroy
before_action :authorize_connect, only: :create
before_action :authorize_disconnect, only: :destroy

Expand All @@ -21,16 +22,20 @@ def show

# @route POST /copilot (copilot_connection)
def create
CopilotConnection.connect!(actor: Current.user, browser_session: Current.session)
CopilotConnection.connect! \
actor: Current.user,
browser_session: Current.session,
credential: @confirmed_credential
@security_audit_recorded = true
redirect_to copilot_connection_path, notice: t("copilot_connection.connected"), status: :see_other
rescue CopilotConnection::Refused, Learning::Error => error
reason, outcome = case error
when CopilotConnection::PermissionMissing then [ :permission_missing, "denied" ]
when CopilotConnection::Refused then [ :refused, "denied" ]
else [ error.code == "not_installed" ? :not_installed : :unavailable, "failed" ]
end
record_security_audit(outcome:)
redirect_to copilot_connection_path, alert: t("copilot_connection.errors.#{reason}"), status: :see_other
redirect_to copilot_connection_path, alert: helpers.copilot_billing_t("errors.#{reason}"), status: :see_other
end

# @route DELETE /copilot (copilot_connection)
Expand All @@ -42,7 +47,7 @@ def destroy

private

# Local development has no Copilot token, so it uses the developer's own Copilot sign-in instead.
# Local development has no Copilot credential, so it uses the developer's own Copilot sign-in instead.
def require_github_hub
head :not_found if LocalDevelopment.enabled?
end
Expand All @@ -57,7 +62,13 @@ def authorize_admin
render :show, status: :forbidden
end

def confirm_change
def confirm_connect
return change_denied(CopilotConnection::ChangeDenied.new(:confirmation_required)) if params[:confirmed] != "1"

@confirmed_credential = CopilotConnection.confirmed_credential!(params[:billing_confirmation])
end

def confirm_disconnect
return if params[:confirmed] == "1"

change_denied(CopilotConnection::ChangeDenied.new(:confirmation_required))
Expand Down Expand Up @@ -85,7 +96,7 @@ def change_denied(error)
else
redirect_to \
copilot_connection_path,
alert: t("copilot_connection.errors.#{error.reason}"),
alert: helpers.copilot_billing_t("errors.#{error.reason}"),
status: :see_other
end
end
Expand Down
10 changes: 10 additions & 0 deletions hub/app/helpers/copilot_connections_helper.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Describes the Copilot credential the operator configured, in the words that fit who Copilot bills.
module CopilotConnectionsHelper
# Translates key under scope. When Copilot bills the organization that installed Hub's GitHub App,
# the organization's wording under scope.organization replaces the token owner's where it differs.
def copilot_billing_t(key, scope: "copilot_connection", **options)
return t("#{scope}.#{key}", **options) if !CopilotConnection.organization_billing?

t("#{scope}.organization.#{key}", default: :"#{scope}.#{key}", **options)
end
end
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@ describe("CopilotConnectionPage()", () => {
it("asks for confirmation before connecting and says who pays for the usage", () => {
renderPage(story.args.data)
expect(screen.getByRole("heading", { name: "Copilot is not connected" })).toBeInTheDocument()
expect(
screen.getByText(
"Copilot counts the usage toward the Copilot plan of the person who owns the token in REPLICA_COPILOT_TOKEN.",
),
).toBeInTheDocument()
expect(
screen.getByText(/The usage counts toward the Copilot plan of the token's owner/),
).toBeInTheDocument()
Expand All @@ -31,6 +36,9 @@ describe("CopilotConnectionPage()", () => {
expect(form).toHaveAttribute("method", "post")
expect(form?.querySelector('[name="_method"]')).toBeNull()
expect(form?.querySelector('[name="authenticity_token"]')).toHaveValue("storybook-token")
expect(form?.querySelector('[name="billing_confirmation"]')).toHaveValue(
"signed-billing-confirmation",
)
expect(screen.queryByRole("button", { name: "Disconnect Copilot" })).not.toBeInTheDocument()
})

Expand Down
10 changes: 8 additions & 2 deletions hub/app/javascript/pages/copilot-connection-page/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { HubAdministrationPage } from "@features/hub-administration-page"
import type { CopilotConnectionPageData } from "./types"
import "./styles.css"

/** Shows who connected Copilot, whether Hub has a token, and each change's consequence beside its confirmation. */
/** Shows who connected Copilot, who Copilot bills, and each change's consequence beside its confirmation. */
export function CopilotConnectionPage({ data }: { data: CopilotConnectionPageData }) {
return (
<HubAdministrationPage data={data}>
Expand All @@ -27,7 +27,8 @@ export function CopilotConnectionPage({ data }: { data: CopilotConnectionPageDat
) : (
<Text as="p">{data.emptyLabel}</Text>
)}
{data.tokenLabel ? <HubAlert title={data.tokenLabel} variant="warning" /> : null}
{data.billingLabel ? <Text as="p">{data.billingLabel}</Text> : null}
{data.setupLabel ? <HubAlert title={data.setupLabel} variant="warning" /> : null}
{data.refusedLabel ? <HubAlert title={data.refusedLabel} variant="critical" /> : null}
</section>

Expand All @@ -41,6 +42,11 @@ export function CopilotConnectionPage({ data }: { data: CopilotConnectionPageDat
className="copilot-connection-page__form"
>
<input type="hidden" name="authenticity_token" value={data.authenticityToken} />
<input
type="hidden"
name="billing_confirmation"
value={data.connect.billingConfirmation}
/>
<label className="copilot-connection-page__confirmation">
<input type="checkbox" name="confirmed" value="1" required />
<span>{data.connect.confirmLabel}</span>
Expand Down
75 changes: 73 additions & 2 deletions hub/app/javascript/pages/copilot-connection-page/stories.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,17 @@ const shared = {
"A Hub administrator connects Copilot once for the whole Hub. Hub then asks Copilot to suggest learnings from prototype review comments, with a token that the person who runs Hub provides, so it works while everyone is signed out.",
emptyLabel: "Learnings stays off until a Hub administrator connects Copilot.",
submitPath: "/copilot",
tokenLabel: null,
billingLabel:
"Copilot counts the usage toward the Copilot plan of the person who owns the token in REPLICA_COPILOT_TOKEN.",
setupLabel: null,
}

const verification =
"This change requires GitHub verification within the last five minutes on this browser. If verification is required, Hub continues your confirmed change after it succeeds."

const connect = {
label: "Connect Copilot",
billingConfirmation: "signed-billing-confirmation",
consequence:
"Hub will ask Copilot, with the configured token, to read new review comments from every repository on this Hub and suggest learnings, including while you are signed out. Copilot never sees who wrote a comment. The usage counts toward the Copilot plan of the token's owner. Hub checks the token with Copilot first, and never stores or shows it.",
confirmLabel: "I understand that Hub will use this Copilot token for this Hub.",
Expand Down Expand Up @@ -79,7 +82,8 @@ export const NoToken: Story = {
args: {
data: {
...data,
tokenLabel:
billingLabel: null,
setupLabel:
"Hub has no Copilot token. The person who runs Hub sets REPLICA_COPILOT_TOKEN to a fine-grained personal access token whose only permission is Copilot requests, then restarts Hub.",
connect: null,
},
Expand All @@ -98,6 +102,73 @@ export const Refused: Story = {
},
},
}
const organizationDescription =
"A Hub administrator connects Copilot once for the whole Hub. Hub then asks Copilot to suggest learnings from prototype review comments, with tokens from Hub's GitHub App, so it works while everyone is signed out."

const organizationBilling =
"Copilot bills the usage to the github organization, which installed Hub's GitHub App, and applies its Copilot policies and budgets."

const organizationConnect = {
...connect,
consequence:
"Hub will ask Copilot to read new review comments from every repository on this Hub and suggest learnings, including while you are signed out. Copilot never sees who wrote a comment. Copilot bills the usage to the github organization. For each request, Hub's GitHub App creates a new one-hour token that can only make Copilot requests. Hub checks with Copilot first, and never stores or shows a token.",
confirmLabel: "I understand that Hub will bill this Hub's Copilot usage to github.",
}

const organizationDisconnect = {
...disconnect,
consequence:
"Hub stops looking for learnings and turns off Learnings for every repository. Hub keeps every learning and decision, so connecting again restores them. Hub's GitHub App keeps its Copilot Requests permission until someone removes it in the App's settings.",
}

export const OrganizationBilling: Story = {
args: {
data: {
...data,
description: organizationDescription,
billingLabel: organizationBilling,
connect: organizationConnect,
},
},
}
export const OrganizationRefused: Story = {
args: {
data: {
...connected,
description: organizationDescription,
billingLabel: organizationBilling,
refusedLabel:
"Copilot refused Hub's GitHub App on October 22, 2026, so Hub stopped looking for learnings. Check that the App has the Copilot Requests organization permission and that github allows Copilot requests from GitHub Apps, then check Copilot again.",
connect: { ...organizationConnect, label: "Check Copilot again" },
disconnect: organizationDisconnect,
},
},
}
export const OrganizationUnconfirmed: Story = {
args: {
data: {
...connected,
description: organizationDescription,
billingLabel: organizationBilling,
setupLabel:
"Hub bills Copilot to an organization only after a Hub administrator confirms it, so Hub stopped looking for learnings. Connect Copilot to let Copilot bill github.",
connect: organizationConnect,
disconnect: organizationDisconnect,
},
},
}
export const NoOrganizationInstallation: Story = {
args: {
data: {
...data,
description: organizationDescription,
billingLabel: null,
setupLabel:
"Hub's GitHub App has no active installation on an organization, so Hub cannot bill Copilot to one. Install the App on an organization, or the person who runs Hub can remove REPLICA_COPILOT_BILLING and give Hub a token instead.",
connect: null,
},
},
}
export const AccessRemoved: Story = {
args: {
data: {
Expand Down
4 changes: 3 additions & 1 deletion hub/app/javascript/pages/copilot-connection-page/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ export interface CopilotConnectionChange {
label: string
consequence: string
confirmLabel: string
billingConfirmation?: string
verification: string
}

Expand All @@ -19,7 +20,8 @@ export type CopilotConnectionPageData =
connected: boolean
statusLabel: string
submitPath: string
tokenLabel: string | null
billingLabel: string | null
setupLabel: string | null
connectedBy?: string
verifiedLabel?: string
usageLabel?: string
Expand Down
Loading
Loading