Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ Examples of unacceptable behavior include:
* Other conduct which could reasonably be considered inappropriate in a
professional setting

## Large Language Model usage policy

* [LLM Usage Policy](docs/policies/llm-usage-policy.md)

## Enforcement Responsibilities

Community leaders are responsible for clarifying and enforcing our standards of
Expand Down
5 changes: 5 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ free to propose changes to this document in a pull request.
[Style Guides](#style-guides)
* [Principles of Keyman Code Changes](#principles-of-keyman-code-changes)
* [Pull Request and Commit Workflow](#pull-request-and-commit-workflow)
* [Large Language Model usage policy](#large-language-model-usage-policy)
* [Code Style Guide](#code-style-guide)
* [User Testing](#user-testing)
* [Documentation Style Guide](#documentation-style-guide)
Expand Down Expand Up @@ -311,6 +312,10 @@ We can review your pull request faster if you follow these guidelines:
* [Pull Request and Commit workflow
wiki](https://github.com/keymanapp/keyman/wiki/Pull-Request-and-Commit-workflow-notes)

### Large Language Model usage policy

* [LLM Usage Policy](docs/policies/llm-usage-policy.md)

### Code Style Guide

The Keyman style guide will help you to prepare code in a way that
Expand Down
181 changes: 181 additions & 0 deletions docs/policies/llm-usage-policy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
# LLM Usage Policy
This is a moderation policy for how LLMs are used in `keymanapp` repositories.

For additional information about the policy itself, see [the appendix](#appendix).

## Overview
Using LLMs while working on `keymanapp` repositories is conditionally allowed, when done with care.
LLMs are not a substitute for thought,
and we do not allow them to be used in ways that risk losing our shared social and technical understanding of the project,
nor in ways that hurt our goals of creating a strong community.

We are aware that many clauses in this policy are unenforceable.
Our goal is *not* to catch every violation (see "It's not your job to play detective", below).
Instead, our goal is to remove plausible deniability: to force a choice between following the policy and intentionally violating it.

The policy's guidelines are roughly as follows:

> It's fine to use LLMs to answer questions, analyze, distill, refine, check, suggest, review. But not to **create**.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
> It's fine to use LLMs to answer questions, analyze, distill, refine, check, suggest, review. But not to **create**.
> It's fine to use LLMs to answer your own questions, analyze, distill, refine, check, suggest, review. But not to **create**.


> LLMs work best when used as a tool to write *better*, not *faster*.

## Rules
### Legend
- ✅ Allowed.
- ❌ Banned.
- ⚠️ Allowed with caveats. Must disclose that an LLM was used.
- ℹ️ Adds additional detail to the policy. These bullets are normative.
- 🔨 Violating this clause counts as a violation of the Code of Conduct.
Comment on lines +23 to +28

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because Legend has the same heading level as the following and comes before Summary I find that a little bit hard to understand.

Wondering if something like this would be better:

Suggested change
### Legend
- ✅ Allowed.
- ❌ Banned.
- ⚠️ Allowed with caveats. Must disclose that an LLM was used.
- ℹ️ Adds additional detail to the policy. These bullets are normative.
- 🔨 Violating this clause counts as a violation of the Code of Conduct.
> [!NOTE]
> Legend:
> - ✅ Allowed.
> - ❌ Banned.
> - ⚠️ Allowed with caveats. Must disclose that an LLM was used.
> - ℹ️ Adds additional detail to the policy. These bullets are normative.
> - 🔨 Violating this clause counts as a violation of the Code of Conduct.

or

Suggested change
### Legend
- ✅ Allowed.
- ❌ Banned.
- ⚠️ Allowed with caveats. Must disclose that an LLM was used.
- ℹ️ Adds additional detail to the policy. These bullets are normative.
- 🔨 Violating this clause counts as a violation of the Code of Conduct.
Legend:
- ✅ Allowed.
- ❌ Banned.
- ⚠️ Allowed with caveats. Must disclose that an LLM was used.
- ℹ️ Adds additional detail to the policy. These bullets are normative.
- 🔨 Violating this clause counts as a violation of the Code of Conduct.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we cross-link to CODE_OF_CONDUCT.md somewhere?


### Summary
- ✅ Allowed: Private use.
- ❌ Banned: LLM-created comments, docs, or diagnostics. Replacing human judgement with LLM judgement. Requiring people to use an LLM to contribute.
- ⚠️ Conditionally allowed: Trivial changes, machine translation, LLM reviews and review bots, LLM-created code *under the experiment rules*.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"under the experiment rules" - can't find those

Also, to be consistent with the rest of the doc:

Suggested change
- ⚠️ Conditionally allowed: Trivial changes, machine translation, LLM reviews and review bots, LLM-created code *under the experiment rules*.
- ⚠️ Conditionally allowed: Trivial changes, machine-translation, LLM reviews and review bots, LLM-created code *under the experiment rules*.

- 🔨 Carries a moderation penalty: Lying.

### Non-exhaustive policy

This policy does not aim to be exhaustive.
If you have a use of LLMs in mind that isn't on this list, talk to people about it, and judge it in the spirit of this overview:
- Using an LLM for your own personal use is likely allowed ✅
- Showing LLM output to another human without solicitation is likely banned ❌
- Making a decision that affects others based on LLM output requires disclosure ⚠️

### ✅ Allowed
The following are allowed.
- Any use of an LLM where you are the only one who sees the output. For example:
- Asking an LLM questions about an existing codebase.
- Asking an LLM to summarize comments on an issue or PR.
- ℹ️ This does not allow reposting the summary publicly. This only includes your own personal use.
- Asking an LLM to privately review your code or prose.
- ℹ️ This does not apply to public comments by the LLM. See "review bots" under ⚠️ below.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"review bots" - we don't have a section that explicitly talks about that.

- Writing dev-tools for your own personal use using an LLM.
- Using an LLM to generate possible solutions to an issue, learning from them, and then writing something from scratch in your own style.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just re-read the policy. In prose, when you want to repeat something notewrothy without plagiarism, you cite and then quote or paraphrase. But if an LLM produces the exact right contextual code fix to a problem and you can defend it, you should be able to disclose it and there's no need (and sometimes no possibility) to rewrite it in your own style. The risk isn't an AI-styled correct solution. The actual risks (as I see them) are:

  1. With the international exposure, you end up with something clearly or subtly nefarious slipping in without review by the submitter and overworked reviewer.
  2. The second worst risk is wasting the reviewer's time (especially for Keyman since there are too few) with cleaning something that is unreviewed by the submitter and not disclosed as unreviewed.
  3. Code that doesn't solve the problem or makes things worse.
  4. Code that works in isolation but doesn't account for decisions elsewhere in the codebase.
  5. Contributor time is wasted restyling something that was already clearly good.

The crazy thing that is that those risks are exactly the same when an LLM isn't involved at all. The standard for merging AI-assisted code should be identical to human-written code: the author human must understand it, and reviewers must be satisfied with it. The rewrite process doesn't address risk one; review does. Disclosure makes that review more informed. Rewriting from scratch just adds time without adding safety.

I'd like to propose replacing this line with:

Directing an LLM to generate code, which you then review, understand, and take full responsibility for, including being able to explain any part of it in review, is allowed with disclosure.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I deleted the duplicate comment)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is another.
6. Code that solves the problem but makes unnecessary changes that still require the reviewer to spend precious time to verify they are not needed as it is not obvious at first.
This is not common with code not written by a LLM. That is because the LLM code seems correct at first specially with the verbose explanations around the change so the person submitting the PR believes they understand it but really they don't. You don't see this in human generated PRs(well not often).
I agree that sometimes the code is very close to as you the contributor would write it (see my comment below about creater). Therfore re-writing from scratch is over the top in some cases, but it puts the burden on the submitter to really make sure the PR is valid.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ermshiperete I wish you hadn't deleted your comment, as you expressed some things in ways that I hadn't. I thought they were complimentary, especially since I hadn't read yours before I wrote mine.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

He didn't delete his comment he deleted your verbatim duplicate comment.

@ermshiperete ermshiperete Jul 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, the exact same comment from you appeared twice, so I deleted one so that others wouldn't have to figure out that it's a duplicate... "Hide" might have been the better option than delete, sorry, didn't think of that.


### ❌ Banned
The following are banned.
- Comments from a personal user account that are originally created by an LLM.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Comments from a personal user account that are originally created by an LLM.
- <a name="comments-from-personal-account"></a>Comments from a personal user account that are originally created by an LLM.

- ℹ️ This also applies to issue bodies and PR descriptions.
- ℹ️ This also applies to voice/video content originally created or scripted by an LLM.
- ℹ️ This does not apply if the LLM content is clearly quoted and marked; you can post that.
However, the content of the comment must stand on its own even without the LLM content; it's not a substitute for your own words.
- ℹ️ See also "machine-translation" in ⚠️ below.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- ℹ️ See also "machine-translation" in ⚠️ below.
- ℹ️ See also ["machine-translation"](#machine-translation) in ⚠️ below.

- ℹ️ See also "Scope" in the appendix below.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- ℹ️ See also "Scope" in the appendix below.
- ℹ️ See also ["Scope"](#scope) in the appendix below.

- Documentation that is originally created by an LLM.
- ℹ️ This includes non-trivial source comments, such as doc-comments, safety comments, or multiple paragraphs of non-doc-comments.
- ℹ️ This includes compiler diagnostics.
LLMs are conditionally allowed to assist with the *logic* surrounding a diagnostic (see "Experiment: LLM-created code changes" below),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is "Experiment: LLM-created code changes" missing in our doc?

but they must not be used to create the message itself.
- ℹ️ This does not include "trivial" changes (see ⚠️ below).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- ℹ️ This does not include "trivial" changes (see ⚠️ below).
- ℹ️ This does not include "trivial" changes (see [⚠️](#%EF%B8%8F-allowed-with-caveats) below).

Comment on lines +57 to +69

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We list comments and documentation as banned if created by an LLM, but not code. I think we should add code changes as well.

- Policies or processes that are written such that an LLM is required to execute them.
- Documentation must be authored for humans primarily, and LLM documentation may only summarize it, not add new detail.
- Treating an LLM review as a sufficient condition to merge or reject a change. (see ⚠️ below).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Treating an LLM review as a sufficient condition to merge or reject a change. (see ⚠️ below).
- Treating an LLM review as a sufficient condition to merge or reject a change. (see [⚠️](#%EF%B8%8F-allowed-with-caveats) below).


### ⚠️ Allowed with caveats
These uses are allowed on a case-by-case basis, under the rules below.
If you are a new contributor, you should expect to be scrutinized more heavily than existing contributors,
since you haven't yet established trust with your reviewers.

All uses under "⚠️ Allowed with caveats" **must** disclose that an LLM was used.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where should this be disclosed for code changes? In the PR description, in the commit message, or in a code comment?

Should it just mention that it's generated by a LLM, or should it include more detail, e.g. the model or service used?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
My vote is commit message and PR description.


- Using machine-translation (e.g. Google Translate) from your native language without posting your original message is allowed but discouraged.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Using machine-translation (e.g. Google Translate) from your native language without posting your original message is allowed but discouraged.
- <a name="machine-translation></a>Using machine-translation (e.g. Google Translate) from your native language without posting your original message is allowed but discouraged.

Doing so can introduce new miscommunications that weren't there originally, and prevents someone who speaks the language from providing a better translation.
- ℹ️ Posting both your original message and the translated version is always ok, but you must still disclose that machine-translation was used.
- "Trivial" code or prose changes.
- ℹ️ Changes are trivial if there is no other way to write them, or the other ways to write them are nearly identical. For example, the following are all trivial:
- Typo fixes
- Markdown links
- Changing a word to a synonym
Comment on lines +85 to +88

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we gain by knowing that such a trivial change was generated by a LLM if a reasonable human would do the same thing?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Human or a numerous other editing tools that are not LLM. I feel this is not really the intent of the llm policy.

- ℹ️ Be cautious about PRs that consist solely of trivial changes.
- Using an LLM to discover bugs, as long as you personally verify the bug.
- ℹ️ This also includes reviewers who use LLMs to discover flaws in unmerged code.
- ℹ️ See also "Comments from a personal user account" under ❌ above.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- ℹ️ See also "Comments from a personal user account" under ❌ above.
- ℹ️ See also ["Comments from a personal user account"](#comments-from-personal-account) under ❌ above.

- LLM reviews, if enabled, **must** be advisory-only, and should be performed _after_ a human review.
- ℹ️ An LLM review does not substitute for self-review. Authors are expected to review their own code before posting and after each change.

## Appendix

### Source

This policy has been adapted from a [draft policy on rust-lang/rust](https://github.com/rust-lang/rust-forge/pull/1040).
It was copied and adapted on 14 July 2026.

### Scope

This policy applies to repositories in the `keymanapp` org.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This policy applies to repositories in the `keymanapp` org.
This policy applies to repositories in the [`keymanapp` org](https://github.com/keymanapp).


### Motivation and guiding principles

There is not a consensus within the Keyman project—and likely never will be—about when/how/where it is acceptable to use AI-based tools.
Many members of the Keyman project and community find value in AI;
many others feel that its negative impact on society and the climate are severe enough that no use is acceptable.
Still others are working out their opinion.

Despite these differences, there are many values we all share:

- Building a community of deep experts in our collective projects.
- Building an inclusive community where all feel welcome and respected.

And many facts we agree on:

- Many people find LLM-generated code and writing deeply unpleasant to read or review.
- Many people find LLMs to be a significant aid to learning and discovery.
- LLMs are a new technology, and we are still learning how to use, moderate, and improve them.

With those facts and values in mind, the policy is designed with the following goals:

- Build an intentionally conservative policy that lets us maintain the standard of quality that Keyman is known for.
- Limit LLM contributions to the very highest standard of quality, to show that our guideline of "better, not faster" isn't just words.
- Make the policy enforceable and easy to moderate.
- Make the policy consistent and easy to understand and summarize, even for people who haven't read it in detail.
- Avoid making LLMs a requirement to contribute, to be inclusive of contributors who choose not to use them.

### Moderation policy
#### It's not your job to play detective
["The optimal amount of fraud is not zero"](https://www.bitsaboutmoney.com/archive/optimal-amount-of-fraud/).
Don't try to be the police for whether someone has used an LLM.
You are not required to "actively look" for whether an LLM was involved.

Style is not evidence, and English-as-a-second-language speakers, neurodivergent people, and over-explainers are the most likely to be accused of writing like an LLM.

If it's clear someone's broken the rules, point them to this policy;
otherwise, discuss with team leads on #internal on Slack.

#### Be honest
Conversely, you're expected to be honest about your use of LLMs and the extent of that use.
If you are not sure where something you would like to do falls in this policy, please ask on the [Community Site](https://community.software.sil.org/c/keyman).
Don't try to hide it.

Deliberately misrepresenting your use of LLMs is not welcome and may result in moderation action.

#### Penalties
The policies marked with a 🔨 follow the same guidelines as the code of conduct:
Violations will first result in a warning, and repeated violations may result in a ban.
- 🔨 Violations of the "Be honest" section

Other violations are left up to the discretion of reviewers and moderators.
For minor violations we recommend telling the author that we can't review the PR until it complies with the policy, with pointers to exactly what they need to do.
For major violations or extractive PRs, we recommend closing the PR or issue.

It is **not** ok to harass a contributor for using an LLM.
All contributors must be treated with respect.
The Code of Conduct applies to *all* conversations in the Keyman project.

### Responsibility

Your contributions are your responsibility; you cannot place any blame on an LLM.
- ℹ️ This includes when asking people to address review comments originally created by an LLM. See "review bots" under ⚠️ above.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"review bots" - we don't have a section that explicitly talks about that.


### The meaning of "originally created by an LLM"

This document uses the phrase "originally created by an LLM" to mean "text that was generated by an LLM (and then possibly edited by a human)".
No amount of editing can change how it was originally created;
the origin sets the initial style, and that style is very hard to change once it's set.
Comment on lines +172 to +173

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I agree where"text that was generated..." if the text refers to documentation, code comments I agree.
For code it is not clear cut and the style is not different since the human set it.
Where was it created though?
For example the other day, I wrote a prompt that described the class I had designed with the member types and the pubic member functions. I also told it the files/classes where it would be called from. The agent basically did my biding and typed the code. I created it the LLM typed it. When I re-write it it is pretty much the same as it is my design (the code with in the functions is deterministic and usually only one way of doing it. I'm ok with the re-write. I just disagre with assumtion being made in these statements about the "original created" when it comes to code.


For more background about analogous reasoning, see ["What Colour are your bits?"](https://ansuz.sooke.bc.ca/entry/23).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The top of this link the authour says
"As of Summer 2024, my article "What Colour are your bits?" has been online 20 years, and people are still linking to it as a benchmark. It's clear that people still care about intellectual property in general and copyright in particular, and the difference, if any, between identical copies of things is still important; but the most salient issues today are not directly related to the verbatim copying that was a big deal in 2004 and was the main topic of "What Colour." I've written other articles about today's issues, and I wish my more recent articles would get the attention that the 20-year-old one still commands; they are more relevant now.

In particular, people in 2024 care a lot about how intellectual property issues and "creator's rights" relate to material that is not actually created by humans - like the output of so-called "generative AI....."


This policy makes no distinction between LLM output that comes from a chat interface and output that comes from editor auto-completion.
In most cases the output is "trivial" (see above under ⚠️), but regardless, it is not treated specially by this policy.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In most cases the output is "trivial" (see above under ⚠️), but regardless, it is not treated specially by this policy.
In most cases the output is "trivial" (see above under [⚠️](#%EF%B8%8F-allowed-with-caveats)), but regardless, it is not treated specially by this policy.


### Conditions for modification or dissolution
This policy is not set in stone, and we can evolve it as we gain more experience working with LLMs.