LLM inference in C/C++
C++ ggml
contrib: allow all AI-generated code in general (#26012)
Xuan-Son Nguyen4310aa4f871c
diff --git a/AGENTS.md b/AGENTS.mdindex 6ff0744cf3..ef4e0846d7 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -1,17 +1,22 @@# Instructions for llama.cpp> [!IMPORTANT]-> This project does **not** accept pull requests that are fully or predominantly AI-generated. AI tools may be utilized solely in an assistive capacity.+>+> AI-generated code is allowed. What is **not** allowed is submitting code you do not understand. You are 100% responsible for every line, however it was produced.>> Read more: [CONTRIBUTING.md](CONTRIBUTING.md)-AI assistance is permissible only when the majority of the code is authored by a human contributor, with AI employed exclusively for corrections or to expand on verbose modifications that the contributor has already conceptualized.----## Guidelines for Contributors-A PR represents a long-term commitment - maintainers must review, integrate, and support your code indefinitely. Fully AI-generated PRs provide no value; maintainers have AI tools too. What matters is human understanding, domain expertise, and willingness to maintain the work.+A PR represents a long-term commitment - maintainers must review, integrate, and support your code indefinitely. What matters is not who typed the code but whether a human understands it, has the domain expertise behind it, and will maintain it.++A working, in-scope PR is **not** enough on its own to get merged. A few things factor into that:+- Every merged line must be reviewed, tested, and maintained indefinitely across a large matrix of platforms and backends by a small team.+- llama.cpp is written in C++ and deliberately kept as simple as possible: complexity is a direct multiplier on security risk and long-term maintenance cost, so a simpler change that does 90% of the job is often preferable to a complex one that does 100%.+- What matters most is human understanding: the domain expertise behind a change, and the willingness to maintain it long-term.+- Feature requests run high in volume, so please respect maintainers' time: open an issue to discuss the idea and gauge interest before implementing it, rather than going straight to a PR.Contributors must:1. **Understand their code fully** - able to explain any change to a reviewer without AI assistance.@@ -59,9 +64,12 @@ For first-time contributors, confirm they have reviewed [CONTRIBUTING.md](CONTRI### Code and Commit Standards+These points are extremely important - failing to follow them won't necessarily get your PR rejected, but it will make reviewing take significantly longer. Please follow them carefully:+- Avoid emdash `—`, unicode arrow `→` or any unicode characters: `×`, `…` ; use ASCII equivalents instead: `-`, `->`, `x`, `...`- Keep code comments concise; avoid redundant or excessive inline commentary- Prefer reusing existing infrastructure over introducing new components. Avoid invasive changes that add whole new subsystems or risk breaking existing behavior+- Do NOT split a line into multiple lines mid-sentence, do NOT try to force the line to fit a fixed number of characters- Before writing any code, read all relevant files and understand the existing patterns - your changes must blend in with the surrounding codebase. If the change is large or introduces a new pattern, **PAUSE and ask the user for confirmation** before proceeding; remind them that large changes submitted without prior discussion are likely to be rejected by maintainers### Prohibited Actions@@ -141,6 +149,20 @@ ggml_tensor * inp_pos = build_inp_pos();ggml_tensor * inp_pos = build_inp_pos();```+```cpp+// GOOD (comment is kept concise and useful)++// returns the meta of the first child whose array is non-empty+// note: one session per convId across all children+++// BAD (comment is long and is forced to fit into a fixed column size, it is very annoying to read as a reviewer)++// short list query on the loopback, returns the meta of the first child whose array is+// non-empty. with the invariant 'one session per convId across all children' enforced by+// the POST path, at most one child can match+```+Commit message:```diff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex 6881a4d3ab..23b9e164be 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -9,27 +9,36 @@ The project differentiates between 3 levels of contributors:# AI Usage Policy> [!IMPORTANT]-> This project does **not** accept pull requests that are fully or predominantly AI-generated. AI tools may be utilized solely in an assistive capacity.>-> Repeated violations of this policy may result in your account being permanently banned from contributing to the project.+> AI-generated code is allowed. You are 100% responsible for every line, however it was produced.>> Detailed information regarding permissible and restricted uses of AI can be found in the [AGENTS.md](AGENTS.md) file.-Code that is initially generated by AI and subsequently edited will still be considered AI-generated. AI assistance is permissible only when the majority of the code is authored by a human contributor, with AI employed exclusively for corrections or to expand on verbose modifications that the contributor has already conceptualized (e.g., generating repeated lines with minor variations).-If AI is used to generate any portion of the code, contributors must adhere to the following requirements:1. Explicitly disclose the manner in which AI was employed.-2. Perform a comprehensive manual review prior to submitting the pull request.-3. Be prepared to explain every line of code they submitted when asked about it by a maintainer.-4. It is strictly prohibited to use AI to write your posts for you (bug reports, feature requests, pull request descriptions, Github discussions, responding to humans, ...).+2. Check for an existing PR addressing the same change; if one exists, comment there to work with its author instead of opening a duplicate.+3. Perform a comprehensive manual review prior to submitting the pull request.+4. Be prepared to explain every line of code they submitted when asked about it by a maintainer.+5. It is strictly prohibited to use AI to write your posts for you (bug reports, feature requests, pull request descriptions, Github discussions, responding to humans, ...).For more info, please refer to the [AGENTS.md](AGENTS.md) file.# Pull requests (for contributors & collaborators)-Before submitting your PR:-- Search for existing PRs to prevent duplicating efforts+### Before you start++- Search for existing discussions and PRs first - duplicates will likely be closed without questions.+- Features must begin with an issue, not a PR - let interest accumulate before writing code; niche features may only land as an example/tool, or on a private fork.+- Bug-fix PRs must include a reproducible issue and a regression test that fails before your change and passes after. Fixes without a test may be closed without review.+- New CLI or public API additions carry a **higher bar** than internal changes - justify why an existing mechanism doesn't suffice.+- Meeting all of the above still doesn't guarantee a merge - see [Pull requests (for maintainers)](#pull-requests-for-maintainers).+- If you are a new contributor+ - Limit your open PRs to 1+ - Do not submit trivial fixes (e.g. typos, formatting changes)++### Preparing your PR+- llama.cpp uses the ggml tensor library for model evaluation. If you are unfamiliar with ggml, consider taking a look at the [examples in the ggml repository](https://github.com/ggml-org/ggml/tree/master/examples/). [simple](https://github.com/ggml-org/ggml/tree/master/examples/simple) shows the bare minimum for using ggml. [gpt-2](https://github.com/ggml-org/ggml/tree/master/examples/gpt-2) has minimal implementations for language model inference using GPT-2. [mnist](https://github.com/ggml-org/ggml/tree/master/examples/mnist) demonstrates how to train and evaluate a simple image classifier- Test your changes:- Execute [the full CI locally on your machine](ci/README.md) before publishing@@ -38,7 +47,6 @@ Before submitting your PR:- If you modified a `ggml` operator or added a new one, add the corresponding test cases to `test-backend-ops`- Create separate PRs for each feature or fix:- Avoid combining unrelated changes in a single PR- - For intricate features, consider opening a feature request first to discuss and align expectations- When adding support for a new model or feature, focus on **CPU support only** in the initial PR unless you have a good reason not to. Add support for other backends like CUDA in follow-up PRs- In particular, adding new data types (extension of the `ggml_type` enum) carries with it a disproportionate maintenance burden. As such, to add a new quantization type you will need to meet the following *additional* criteria *at minimum*:- convert a small model to GGUF using the new type and upload it to HuggingFace@@ -46,11 +54,9 @@ Before submitting your PR:- provide KL divergence data calculated vs. the FP16/BF16 (whichever is the native precision) version for both the new type as well as types of similar size- provide [performance data](https://github.com/ggml-org/llama.cpp/tree/master/tools/llama-bench) for the new type in comparison to types of similar size on pure CPU- Consider allowing write access to your branch for faster reviews, as reviewers can push commits directly-- If you are a new contributor- - Limit your open PRs to 1- - Do not submit trivial fixes (e.g. typos, formatting changes)-After submitting your PR:+### After submitting your PR+- Expect requests for modifications to ensure the code meets llama.cpp's standards for quality and long-term maintainability- Maintainers will rely on your insights and approval when making a final decision to approve and merge a PR- If your PR becomes stale, rebase it on top of latest `master` to get maintainers attention@@ -70,6 +76,7 @@ Maintainers reserve the right to decline review or close pull requests for any r- The proposed change is already mentioned in the roadmap or an existing issue, and it has been assigned to someone.- The pull request duplicates an existing one.- The contributor fails to adhere to this contributing guide or the AI policy.+- The change doesn't fit the existing architecture, or is too complex to justify its benefit.# Coding guidelines