Self-hosted project management for devs and their clients: tickets, time tracking, docs and meetings, with an AI assistant that cites its sources. AGPL-3.0.
Ruby agplai-assistantdockerdsgvogdprhotwireissue-trackerjira-alternativeknowledge-basellmmcp-servermeeting-notesopen-sourceproject-managementredmine-alternativeruby-on-railsself-hostedticketingtime-tracking
Fail over across the checked addresses of a customer endpoint
Guido Boykebe52d319a1a8
diff --git a/app/services/lalabase/llm/public_endpoint_adapter.rb b/app/services/lalabase/llm/public_endpoint_adapter.rbindex bbe90f5..48de633 100644--- a/app/services/lalabase/llm/public_endpoint_adapter.rb+++ b/app/services/lalabase/llm/public_endpoint_adapter.rb@@ -8,9 +8,9 @@ require 'faraday/net_http'# can point at the app server when the decrypted key is sent (DNS rebinding).## So for every connection it resolves the host through Lalabase::PublicAddress, refuses-# unless every address is public, and pins the socket to the first of them-# (Net::HTTP#ipaddr=). Net::HTTP keeps the NAME for SNI and the certificate check, so-# TLS still verifies the host the customer named.+# unless every address is public, and connects only to those addresses, in the order+# the resolver gave them (PinnedAddresses). Net::HTTP keeps the NAME for SNI and the+# certificate check, so TLS still verifies the host the customer named.## Through a proxy the proxy resolves the name and the pinned address means nothing, so# a proxied request is refused. An operator who needs one configures the endpoint on a@@ -24,11 +24,49 @@ module Lalabase# retry_on in the jobs, ConnectionCheck), with no port or reason in its text.class EndpointNotPublicError < Faraday::ConnectionFailed; end+ # Mixed into the one Net::HTTP object of a connection. Given the name, Net::HTTP+ # let TCPSocket try every address of it; given one pinned address, a dual-stack+ # endpoint with broken IPv6 would fail although its IPv4 address answers. So the+ # TCP connect walks the checked addresses in order and gives up with the last+ # error.+ #+ # Only around Net::HTTP#connect, which opens the socket and does the TLS+ # handshake and nothing else: the request is written after it returns, so+ # nothing has been sent when the next address is tried, and a request is never+ # sent twice. On failure #connect closes the socket it opened. #connect is+ # private, but it is the seam both the net-http gem in the bundle (0.6.0) and the+ # one in Ruby 4.0 share; the helper below it differs between the two.+ module PinnedAddresses+ CONNECT_ERRORS = [SystemCallError, IO::TimeoutError, Net::OpenTimeout].freeze++ attr_accessor :pinned_addresses++ private++ def connect+ *earlier, last = pinned_addresses+ earlier.each do |address|+ self.ipaddr = address+ return super+ rescue *CONNECT_ERRORS+ next+ end+ self.ipaddr = last+ super+ end+ end+def net_http_connection(env)raise EndpointNotPublicError, 'the endpoint is not a public address' if env[:request][:proxy]- address = PublicAddress.resolve!(env[:url].hostname).first- super.tap { |http| http.ipaddr = address }+ addresses = PublicAddress.resolve!(env[:url].hostname)+ super.tap do |http|+ http.extend(PinnedAddresses)+ http.pinned_addresses = addresses+ # What Net::HTTP reports as the address it connects to (#ipaddr, and the+ # "Failed to open TCP connection to" message) is the first checked one.+ http.ipaddr = addresses.first+ endrescue PublicAddress::NotPublicErrorraise EndpointNotPublicError, 'the endpoint is not a public address'enddiff --git a/spec/services/lalabase/llm/public_endpoint_adapter_spec.rb b/spec/services/lalabase/llm/public_endpoint_adapter_spec.rbindex c6cfecf..2d10454 100644--- a/spec/services/lalabase/llm/public_endpoint_adapter_spec.rb+++ b/spec/services/lalabase/llm/public_endpoint_adapter_spec.rb@@ -65,6 +65,62 @@ RSpec.describe Lalabase::Llm::PublicEndpointAdapter, :ci_guard doendend+ # With the name, Net::HTTP let TCPSocket try every address of it. Pinned to one+ # address, a dual-stack endpoint with broken IPv6 would fail every request although+ # its IPv4 address answers. So the TCP connect walks the checked addresses in order,+ # and only the connect: nothing has been sent at that point, so trying the next+ # address cannot send a request twice.+ describe 'failover across the checked addresses' do+ let(:server) { TCPServer.new('127.0.0.1', 0) }+ let(:attempts) { [] }++ after { server.close }++ def connect_through(unreachable:)+ allow(TCPSocket).to receive(:open).and_wrap_original do |original, address, _port, *rest, **options|+ attempts << address+ raise Errno::ENETUNREACH, address if unreachable.include?(address)++ original.call('127.0.0.1', server.addr[1], *rest, **options)+ end++ http = connection_for("http://dual.provider.example:#{server.addr[1]}/v1")+ http.start(&:started?)+ end++ it 'connects to the next checked address when the first is unreachable' do+ resolve_names('dual.provider.example' => ['2001:db8::1', '203.0.113.20'])++ expect(connect_through(unreachable: ['2001:db8::1'])).to be(true)+ expect(attempts).to eq(['2001:db8::1', '203.0.113.20'])+ end++ it 'tries only the checked addresses and fails with the last error when none answers' do+ resolve_names('dual.provider.example' => ['2001:db8::1', '203.0.113.20'])++ expect { connect_through(unreachable: ['2001:db8::1', '203.0.113.20']) }+ .to raise_error(Errno::ENETUNREACH)+ expect(attempts).to eq(['2001:db8::1', '203.0.113.20'])+ end++ # Once connected, the request may have been written. A failure from there on must+ # not go to the next address: a chat request would run, and be billed, twice.+ it 'does not try the next address when the request fails after the connect' do+ resolve_names('dual.provider.example' => ['203.0.113.20', '203.0.113.21'])+ listener = Thread.new { loop { server.accept.close } }+ allow(TCPSocket).to receive(:open).and_wrap_original do |original, address, _port, *rest, **options|+ attempts << address+ original.call('127.0.0.1', server.addr[1], *rest, **options)+ end++ http = connection_for("http://dual.provider.example:#{server.addr[1]}/v1")+ expect { http.request(Net::HTTP::Post.new('/v1/chat/completions')) }.to raise_error(StandardError)+ expect(attempts).to eq(['203.0.113.20'])+ ensure+ listener&.kill+ end+ end+# End to end, with the real OpenAI client: a listener on loopback stands for a# service on the app server, and a public-looking name resolves to it.describe 'Lalabase::Llm::OpenAiCompatibleProvider at request time' do