Apache Solr open-source search software
= Adding a Changelog Entry to Apache Solr
:toc:
:toclevels: 2
:icons: font
The project no longer maintains a monolithic *CHANGES.txt* file directly.
Instead, we create small YAML fragments **per bug-fix or feature**, checked into
the `solr/changelog/unreleased/ folder`. At release time these fragments are
organized in versioned folders and collated into a human-readable *CHANGELOG.md* file.
This document walks you through the workflow, explains the Gradle helpers
available, and finishes with a concrete example fragment. We use a tool called
https://logchange.dev/tools/logchange/[logchange] as a gradle plugin to manage
the changelog.
== 1. Workflow Overview
. Make a feature branch with a descriptive name like 'SOLR-12345-fix-memory-leak'.
. Implement your change
. Run `./gradlew writeChangelog`
. Edit the generated YAML file in `changelog/unreleased/` to add a clear, user-focused description.
. Commit the YAML fragment along with the code change.
. Open your PR as usual.
== 2. Directory Layout
[source]
----
solr/
└── changelog/
├── unreleased/ ← new fragments live here
| └── SOLR-12345-fix-memory-leak.yml
├── v9.8.0/ ← changes in already released versions
├── v9.8.0/
└── v9.9.0/
----
== 3. The YAML format
Below is an example of a changelog yaml fragment. The full yaml format is https://logchange.dev/tools/logchange/reference/#yaml-entry-format[documented here], but we normally only need `title`, `type`, `authors` and `links`. For a change without a JIRA, you can add the PR number in `issues`:
For the author section, the `name` key is the only one required. You may optionally want to link to either your github user (`nick`) or e.g. your ASF user (`url`) as shown below.
[source, yaml]
----
title: Fix nasty bug in CoreContainer shutdown
# added, changed, fixed, deprecated, removed, dependency_update, security, other
type: fixed
authors:
- name: Alex Doe
nick: alex_githubid
url: https://home.apache.org/phonebook.html?uid=alex_asfid
links:
- name: SOLR-12345
url: https://issues.apache.org/jira/browse/SOLR-12345
----
=== Changelog Types Reference
[cols="1,2,3",options="header"]
|===
| Type | Description | When to Use (Examples)
| `added`
| For changes requiring a user to take action to use (opt-in).
| Could be completely new features or simply new configuration values on existing features. Typically documented in the Ref Guide.
| `changed`
| For improvements; not opt-in.
| Modifying behavior or performance of existing requests/configuration.
| `fixed`
| For improvements that are deemed to have fixed buggy behavior.
| Fixing a `NullPointerException`, correcting data corruption issues, or resolving UI glitches.
| `deprecated`
| For marking things deprecated.
| Declaring a configuration option, class, or API endpoint as deprecated ahead of a future removal.
| `removed`
| For code removed.
| Removing a previously deprecated parameter, class, or legacy module.
| `dependency_update`
| For updates to dependencies.
| Bumping third-party library versions (e.g., Lucene, Jetty, Jackson).
| `other`
| For anything else. Most such changes are too small/minor to bother with a changelog entry.
| Large/significant refactorings, build changes, test infrastructure, or documentation.
|===
=== 3.1 Tool to draft a YAML for your change
We have two gradle tasks that bootstraps a YAML file in the `changelog/unreleased/` directory.
`writeChangeLog` will infer the JIRA id from the current branch if possible, otherwise you'll need to add the JIRA link manually.
If the change isn't worth a JIRA yet a changelog would still be nice, use `writeChangelogPr` after you've created the PR. It will incorporate a link to the PR in the entry.
If `writeChangelogPr` can't auto-detect the PR for your current branch (e.g. `gh` can't correlate a differently-named local branch to it), pass the PR explicitly: `./gradlew writeChangelogPr -Ppr=4677` or `-Ppr=https://github.com/apache/solr/pull/4677`.
Invoke the task with:
[source, bash]
----
./gradlew writeChangelog
----
The task will use your git `user.name` as display name by default, but if you
add your name, GitHub username, and optionally Apache ID to the
`gradle.properties` file at the root of the Solr git checkout, this task will
use that information. Example:
[source, properties]
----
# Changelog generator settings (./gradlew writeChangelog)
user.name=Alex Doe
user.githubid=alex_githubid
user.asfid=alex_asfid
----
The changelog has comments to help you fill it out. Remove them after filling out the form.
TIP: Aliases for the `writeChangelog` task are `changelog` and `newChangelog`.
== 4. Changelog Validation in Pull Requests
The `validate-changelog` GitHub workflow checks that code changes come with a
changelog entry, and that any entry added under `changelog/unreleased/` is
structurally valid.
Pull requests that only touch documentation, tests, build files or CI configuration
are exempt and pass without an entry. Subjectively, some minor changes don't need one either.
=== 4.1 Opting out of the changelog requirement
If your change does not require a changelog entry (refactoring, internal cleanup,
and so on), it is still possible to merge the PR. Mark it by adding the
`no-changelog` label, which you can set in either of two ways:
* Comment `/no-changelog` on the pull request, alone on its own line. This works
for the PR author as well as for committers, so a contributor without write
access does not need to ask anyone to set the label. Comment `/needs-changelog`
to undo it. Use a regular comment in the "Conversation" tab; commands inside
code review comments are not picked up.
* Add or remove the `no-changelog` label directly (committers only).
Either way the check re-runs automatically on the label change, so an already
failed "Check changelog entry" run turns green without pushing a new commit.
== 5. For release managers
=== 5.1 Gradle tasks for logchange
The logchange gradle plugin offers some tasks, here are the two most important:
[cols="1,2", options="header"]
|===
| Task | Purpose
| `logchangeGenerate`
| Generates changelog file `/CHANGELOG.md` based on `.yml` entries and archives (does not moves any files)
| `logchangeRelease`
| Creates a new changelog release by moving files from `changelog/unreleased/` directory to `changelog/vX.Y.Z` directory
| `logchangeArchive`
| Archives the list of released versions up to (and including) the specified version by transferring their summaries to the `archive.md` file, merging all existing archives, and deleting the corresponding version directories.
|===
The `logchangeRelease` and `logchangeGenerate` tasks are used by ReleaseWizard. The `logchangeArchive` task can be ran once for every major release or when the number of versioned changelog folders grow too large.
These are integrated in the Release Wizard via `dev-tools/scripts/logchange.py`, which also handles RC2+ scenarios — see section 5.4.
=== 5.2 Migration tool
There is a migration tool in `dev-tools/scripts/changes2logchange.py` for one-time use during the transition.
It will bulk convert the entire `solr/CHANGES.txt` file to files in the `changelog/` folder and versioned sub folders.
The tool can also be used by a developer who needs to convert many entries from work-in-progress after the migration
cutover. You can paste one or more entries and get them converted in the Terminal as follows:
[source,bash]
----
cat << EOF | python3 dev-tools/scripts/changes2logchange.py -
* SOLR-17960: Removed TikaLanguageIdentifierUpdateProcessor.
Use LangDetectLanguageIdentifierUpdateProcessor or
OpenNLPLangDetectUpdateProcessor instead. (janhoy)
EOF
----
which will output to `stdout`:
[source,yaml]
----
title: Removed TikaLanguageIdentifierUpdateProcessor. Use LangDetectLanguageIdentifierUpdateProcessor
or OpenNLPLangDetectUpdateProcessor instead.
type: other # added, changed, fixed, deprecated, removed, dependency_update, security, other
authors:
- name: janhoy
links:
- name: SOLR-17960
url: https://issues.apache.org/jira/browse/SOLR-17960
----
=== 5.3 Changelog validation tool
There is a tool `dev-tools/scripts/validateChangelogs.py` that will do a
cross-branch validation of changelog folders. It takes no arguments and
discovers the branch structure automatically, and checks that each branch
(unstable, stable, release, previous-stable, previous-bugfix) are in sync
with remote and "clean". Then it checks that the changelog folders are the
same across branches and computes some statistics on how many unreleased
features are from each branch.
The tool can also output a consolidated `CHANGELOG.md` file which calculates
what (unreleased) features are likely to be released in what version. This
resembles what we used to have in our `CHANGES.txt` on main branch before.
The tool is integrated in the releaseWizard.
usage: validateChangelogs.py -h
Example report output (Json or Markdown):
[source,json,title=Example json report]
----
{
"success": false,
"errors": [
"Folder v9.8.1 not found on branches: {'branch_9_9'}",
"Folder v9.9.0 not found on branches: {'branch_9_9'}"
],
"warnings": [],
"branch_report": {
"branch_9_9": {
"version": "9.9.1",
"has_changelog_folder": false,
"id": "previous_major_bugfix"
},
"branch_9x": {
"version": "9.10.0",
"unreleased_count": 31,
"id": "previous_major_stable",
"new_count": 31,
"new": [
"SOLR-17541-deprecate -cloudhttp2solrclient-builder.yml",
"SOLR-17619 Use logchange for changelog management.yml"
],
"not_in_newer_count": 1,
"not_in_newer": [
"SOLR-17541-deprecate -cloudhttp2solrclient-builder.yml"
]
},
"branch_10_0": {
"version": "10.0.0",
"unreleased_count": 146,
"id": "release",
"new_count": 5,
"new": [
"GITHUB#3666-removing redundant check if field exists in.yml",
"SOLR-12089-remove deprecated -breaksugestiontiebreaker-.yml",
"SOLR-14070-deprecate cloudsolrclient-s zookeeper hosts.yml"
],
"not_in_newer_count": 2,
"not_in_newer": [
"SOLR-16562-solr-16578- upgrade caffeine to 3-1-4 and.yml",
"SOLR-17012-update apache hadoop to 3-3-6 and apache curator.yml"
]
},
"branch_10x": {
"version": "10.1.0",
"unreleased_count": 147,
"id": "stable",
"new_count": 2,
"new": [
"PR#3758-logs- removed webapp-solr and also removed from.yml",
"SOLR-17963-these solrcloud commands no longer aquire locks.yml"
]
},
"main": {
"version": "11.0.0",
"unreleased_count": 147,
"id": "main",
"new_count": 0,
"new": []
}
}
}
----
=== 5.4 Release candidate changelog script
`dev-tools/scripts/logchange.py` handles changelog git operations for each RC
and the final post-vote forward-porting. It is integrated into the Release
Wizard but can also be run standalone. Run with `--help` for usage.
=== 5.5 Dependency changes summary
`dev-tools/scripts/dependencyChanges.py` summarizes the third-party dependency changes
since the previous release by diffing `solr/licenses/`. By default it prints to stdout;
with `--write`, the Release Wizard uses it to write `dependency_update` changelog entries
(one each for added, upgraded, and removed).
== 6. Further Reading
* https://github.com/logchange/logchange[Logchange web page]
* https://keepachangelog.com/en/1.1.0/[keepachangelog.com website]