Self-hosted project management for devs and their clients: tickets, time tracking, docs and meetings, with an AI assistant that cites its sources. AGPL-3.0.
Ruby agplai-assistantdockerdsgvogdprhotwireissue-trackerjira-alternativeknowledge-basellmmcp-servermeeting-notesopen-sourceproject-managementredmine-alternativeruby-on-railsself-hostedticketingtime-tracking
Refuse unstorable Idempotency-Keys and widen the error-body guard
Guido Boyke5efde7ed1c73
diff --git a/CLAUDE.md b/CLAUDE.mdindex df725ed..39f3085 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -793,14 +793,15 @@ feature by feature):`unauthorized`); an unmapped actor code is `refused` with the old value as `reason`. Everybody comes from `Api::V1::ErrorBody` through `BaseController#render_error`; guard`spec/architecture/api_v1_error_body_guard_spec.rb`. Answers outside the controllers carry- no `code`: the Rack::Attack 429 (plain text) and a 500 from the exceptions app. The 401 of `/api/mcp` (same concern,+ no `code`: the Rack::Attack 429 (plain text), a 500 from the exceptions app, and a 400 for a+ query string Rack cannot decode (`?x=%`; raised in middleware before any controller). The 401 of `/api/mcp` (same concern,`render_unauthorized`) keeps its old body.* **`Idempotency-Key`** on every write (`idempotent :create` in the controller,`Api::V1::Idempotent`): the same key and request replays the first 2xx answer (status andbody), another request answers 422 `idempotency_key_reused`. Same table and store as MCP(`Mcp::IdempotencyStore`); the REST fingerprint includes method and path, so a key nevercrosses surfaces. Runs inside authentication and the read-only gate. A key is at most 255- bytes (longer: 422 `invalid_idempotency_key`, nothing written). A replay checks no rights+ bytes of valid UTF-8 without NUL (else: 422 `invalid_idempotency_key`, nothing written). A replay checks no rights(it repeats what the same token already got; keys do not expire), and a retry must sendthe same bytes. Every write route must be `idempotent`(`spec/architecture/api_v1_writes_idempotent_guard_spec.rb`).diff --git a/app/controllers/concerns/api/v1/idempotent.rb b/app/controllers/concerns/api/v1/idempotent.rbindex 637357e..6accc2d 100644--- a/app/controllers/concerns/api/v1/idempotent.rb+++ b/app/controllers/concerns/api/v1/idempotent.rb@@ -22,7 +22,11 @@ module Api# same token already received, nothing it has not seen; a new key goes# through every check. Keys do not expire. A retry must send the same bytes:# the fingerprint sorts only the top-level keys, so another encoding of the- # same request is a mismatch (safe: it writes nothing).+ # same request is a mismatch (safe: it writes nothing). With a key the body is+ # parsed even where the action ignores it, so malformed JSON answers 400 there.+ # A multipart body is never repeatable (the upload's temp path is in the+ # fingerprint); reduce files to name, size and digest when an upload endpoint+ # arrives.module Idempotentextend ActiveSupport::Concern@@ -53,7 +57,7 @@ module Apidef with_idempotency_keykey = request.headers[HEADER].to_s.strip.presencereturn yield unless key- return render_invalid_idempotency_key if key.bytesize > MAX_KEY_LENGTH+ return render_invalid_idempotency_key unless storable_key?(key)store = Mcp::IdempotencyStore.new(token: current_token, tool_name: "rest:#{controller_path}##{action_name}",key: key, args: idempotency_fingerprint_args)@@ -65,6 +69,13 @@ module Apistore.remember({ status: response.status, body: response.body }.to_json) if response.successful?end+ # Within the index (bytes) and storable in a PostgreSQL text column: valid+ # UTF-8 without NUL, which the database refuses with an error otherwise.+ def storable_key?(key)+ utf8 = key.dup.force_encoding(Encoding::UTF_8)+ key.bytesize <= MAX_KEY_LENGTH && utf8.valid_encoding? && !utf8.include?("\0")+ end+def idempotency_fingerprint_args{ 'method' => request.request_method, 'path' => request.path,'params' => request.query_parameters.merge(request.request_parameters) }@@ -82,7 +93,8 @@ module Apidef render_invalid_idempotency_keyskip_authorizationrender_error(error: INVALID_KEY, code: 'invalid_input',- message: "#{HEADER} is longer than #{MAX_KEY_LENGTH} bytes", status: :unprocessable_entity)+ message: "#{HEADER} must be valid UTF-8 without NUL, at most #{MAX_KEY_LENGTH} bytes",+ status: :unprocessable_entity)enddef render_idempotency_mismatchdiff --git a/spec/architecture/api_v1_error_body_guard_spec.rb b/spec/architecture/api_v1_error_body_guard_spec.rbindex e546fe4..e95723f 100644--- a/spec/architecture/api_v1_error_body_guard_spec.rb+++ b/spec/architecture/api_v1_error_body_guard_spec.rb@@ -13,17 +13,33 @@ module ApiV1ErrorBodyGuardGLOBS = %w[app/controllers/api/v1/**/*.rb app/controllers/concerns/api/v1/**/*.rb].freezeOWNER = 'app/controllers/concerns/api/v1/error_body.rb'- ERROR_STATUS = 'bad_request|unauthorized|forbidden|not_found|unprocessable_entity|unprocessable_content|' \- 'conflict|internal_server_error|[45]\\d\\d'+ # The two renders whose status is a variable by design, each named by the+ # line it may contain. Anything else in these files is still checked.+ ALLOWED_LINES = {+ 'app/controllers/api/v1/base_controller.rb' =>+ 'render json: Api::V1::ErrorBody.build(error: error, code: code, **extra), status: status',+ 'app/controllers/concerns/api/v1/idempotent.rb' =>+ "render json: answer.fetch('body'), status: answer.fetch('status')"+ }.freeze- # Any of: a render with an `error`/`errors` key (one line or wrapped), a- # string `"error" =>` key, a bodyless `head` with an error status, or a plain- # `render` (not `render_error`) that sets an error status on its own line.+ # Every error status Rack knows by name, and any 4xx/5xx number.+ ERROR_STATUS = (Rack::Utils::SYMBOL_TO_STATUS_CODE.select { |_name, code| code >= 400 }.keys.map(&:to_s) ++ ['[45]\\d\\d']).join('|')+ SUCCESS_STATUS = 'ok|created|accepted|no_content|2\\d\\d'++ # Any of:+ # * a render with an `error`/`errors` key, one line or wrapped;+ # * a string or symbol key: `"error" =>`, `"error":`, `:error =>`;+ # * a bodyless `head` with an error status;+ # * a plain `render` (not `render_error`) whose `status:`, on its line or a+ # continuation line after a comma, is anything but a success status: an+ # error symbol, a number, or a variable such as a status lookup.HAND_BUILT = Regexp.union(/render\s*\(?\s*json:\s*\{[^}]*\berrors?:/m,- /["']errors?["']\s*=>/,+ /["']errors?["']\s*(?:=>|:)/,+ /:errors?\s*=>/,/\bhead\s*\(?\s*:?(?:#{ERROR_STATUS})\b/,- /\brender(?!_)\b[^\n]*\bstatus:\s*:?(?:#{ERROR_STATUS})\b/+ /\brender(?!_)\b(?:[^\n]*,[ \t]*\n)*[^\n]*\bstatus:[ \t]*(?![ \t])(?!:?(?:#{SUCCESS_STATUS})\b)/)def self.offences@@ -31,7 +47,9 @@ module ApiV1ErrorBodyGuardrelative = path.relative_path_from(Rails.root).to_snext if relative == OWNER- relative if path.read.match?(HAND_BUILT)+ source = path.read+ source = source.sub(ALLOWED_LINES[relative], '') if ALLOWED_LINES.key?(relative)+ relative if source.match?(HAND_BUILT)endendend@@ -52,10 +70,22 @@ RSpec.describe 'API V1 error body guard', :ci_guard doexpect('head :forbidden').to match(ApiV1ErrorBodyGuard::HAND_BUILT)expect('render json: { data: rows }').not_to match(ApiV1ErrorBodyGuard::HAND_BUILT)expect('render json: { data: row }, status: :created').not_to match(ApiV1ErrorBodyGuard::HAND_BUILT)+ expect("render json: { message: x },\n status: :forbidden").to match(ApiV1ErrorBodyGuard::HAND_BUILT)+ expect('render json: { "error": "x" }, status: http_status').to match(ApiV1ErrorBodyGuard::HAND_BUILT)+ expect("render json: { :error => 'x' }").to match(ApiV1ErrorBodyGuard::HAND_BUILT)+ expect('render json: x, status: ACTOR_ERROR_STATUS.fetch(c)').to match(ApiV1ErrorBodyGuard::HAND_BUILT)+ expect('head :too_many_requests').to match(ApiV1ErrorBodyGuard::HAND_BUILT)+ expect("render json: { data: row },\n status: :created").not_to match(ApiV1ErrorBodyGuard::HAND_BUILT)expect('head :ok').not_to match(ApiV1ErrorBodyGuard::HAND_BUILT)expect("render_error(error: 'forbidden', status: :forbidden)").not_to match(ApiV1ErrorBodyGuard::HAND_BUILT)end+ it 'still finds each allowed line, so an allowance cannot outlive its code' do+ ApiV1ErrorBodyGuard::ALLOWED_LINES.each do |file, line|+ expect(Rails.root.join(file).read).to include(line), "#{file} no longer contains the allowed render"+ end+ end+it 'leaves every error body to render_error' doexpect(ApiV1ErrorBodyGuard.offences).to be_emptyenddiff --git a/spec/requests/api/v1/idempotency_spec.rb b/spec/requests/api/v1/idempotency_spec.rbindex 3b1e9d4..fd50352 100644--- a/spec/requests/api/v1/idempotency_spec.rb+++ b/spec/requests/api/v1/idempotency_spec.rb@@ -167,6 +167,29 @@ RSpec.describe 'API V1 idempotency', :ci_guard, type: :request doexpect(response.parsed_body).to include('error' => 'invalid_idempotency_key', 'code' => 'invalid_input')end+ # Bytes, not characters: 128 umlauts are 256 bytes.+ it 'counts the limit in bytes' do+ expect { comment('Hallo', key: 'ä' * 128) }.not_to(change { posts_count })+ expect(response).to have_http_status(:unprocessable_content)+ expect { comment('Hallo', key: "#{'ä' * 127}k") }.to change { posts_count }.by(1)+ end++ # PostgreSQL refuses both in a text column; they used to answer a 500.+ def expect_invalid_key+ expect(response).to have_http_status(:unprocessable_content)+ expect(response.parsed_body).to include('error' => 'invalid_idempotency_key', 'code' => 'invalid_input')+ end++ it 'refuses a key that is not valid UTF-8, writing nothing' do+ expect { comment('Hallo', key: "k\xFFx".b) }.not_to(change { posts_count })+ expect_invalid_key+ end++ it 'refuses a key with a NUL byte, writing nothing' do+ expect { comment('Hallo', key: "k\u0000x") }.not_to(change { posts_count })+ expect_invalid_key+ end+it 'accepts a key of exactly the limit' dokey = 'k' * Api::V1::Idempotent::MAX_KEY_LENGTHexpect { 2.times { comment('Hallo', key: key) } }.to change { posts_count }.by(1)