Send and receive SMS messages using your Android phone programmatically via a simple HTTP API
Remove docs for superpowers
Acho Arnold5ffdcb0b7ce6
diff --git a/docs/superpowers/plans/2026-04-11-mms-attachments.md b/docs/superpowers/plans/2026-04-11-mms-attachments.mddeleted file mode 100644index a759e1c4..00000000--- a/docs/superpowers/plans/2026-04-11-mms-attachments.md+++ /dev/null@@ -1,1139 +0,0 @@-# MMS Attachment Support Implementation Plan--> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.--**Goal:** Add MMS attachment upload/download support to the httpSMS API so received MMS attachments are stored in cloud storage and downloadable via a public URL.--**Architecture:** Android sends base64-encoded attachments in the receive request. The API decodes and uploads them to GCS (or in-memory storage) via a storage interface, stores download URLs in the existing `Message.Attachments` field, and exposes an unauthenticated download endpoint. The webhook event payload includes attachment URLs.--**Tech Stack:** Go, Fiber v2, GORM, `cloud.google.com/go/storage`, `errgroup`, `stacktrace`-------## File Structure--**New files:**--| File | Responsibility |-| --------------------------------------------------- | -------------------------------------------------------------------------------- |-| `api/pkg/repositories/attachment_storage.go` | `AttachmentStorage` interface + content-type-to-extension mapping + sanitization |-| `api/pkg/repositories/gcs_attachment_storage.go` | GCS implementation of `AttachmentStorage` |-| `api/pkg/repositories/memory_attachment_storage.go` | In-memory implementation of `AttachmentStorage` |-| `api/pkg/repositories/attachment_storage_test.go` | Unit tests for content-type mapping and filename sanitization |-| `api/pkg/handlers/attachment_handler.go` | Download endpoint handler (`GET /v1/attachments/...`) |--**Modified files:**--| File | Change |-| ------------------------------------------------- | ------------------------------------------------------------------------------- |-| `api/pkg/requests/message_receive_request.go` | Add `Attachments` field + `MessageAttachment` struct |-| `api/pkg/services/message_service.go` | Add `Attachments` to params, upload logic in `ReceiveMessage()`, set on message |-| `api/pkg/validators/message_handler_validator.go` | Add attachment count/size/content-type validation |-| `api/pkg/events/message_phone_received_event.go` | Add `Attachments []string` to payload |-| `api/pkg/di/container.go` | Wire `AttachmentStorage`, `AttachmentHandler`, `RegisterAttachmentRoutes()` |-| `api/.env.docker` | Add `GCS_BUCKET_NAME` |-| `api/go.mod` / `api/go.sum` | Add `cloud.google.com/go/storage` and `golang.org/x/sync` (errgroup) |-------### Task 1: Add GCS SDK and errgroup dependencies--**Files:**--- Modify: `api/go.mod`--- [ ] **Step 1: Add dependencies**--```bash-cd api && go get cloud.google.com/go/storage && go get golang.org/x/sync-```--- [ ] **Step 2: Verify build still works**--Run: `cd api && go build ./...`-Expected: Build succeeds--- [ ] **Step 3: Commit**--```bash-cd api && git add go.mod go.sum && git commit -m "chore: add cloud.google.com/go/storage and golang.org/x/sync deps"-```-------### Task 2: Storage interface, content-type mapping, and filename sanitization--**Files:**--- Create: `api/pkg/repositories/attachment_storage.go`-- Create: `api/pkg/repositories/attachment_storage_test.go`--- [ ] **Step 1: Write the test file**--Create `api/pkg/repositories/attachment_storage_test.go`:--```go-package repositories--import "testing"--func TestExtensionFromContentType(t *testing.T) {- tests := []struct {- contentType string- expected string- }{- {"image/jpeg", ".jpg"},- {"image/png", ".png"},- {"image/gif", ".gif"},- {"image/webp", ".webp"},- {"image/bmp", ".bmp"},- {"video/mp4", ".mp4"},- {"video/3gpp", ".3gp"},- {"audio/mpeg", ".mp3"},- {"audio/ogg", ".ogg"},- {"audio/amr", ".amr"},- {"application/pdf", ".pdf"},- {"text/vcard", ".vcf"},- {"text/x-vcard", ".vcf"},- {"application/octet-stream", ".bin"},- {"unknown/type", ".bin"},- {"", ".bin"},- }- for _, tt := range tests {- t.Run(tt.contentType, func(t *testing.T) {- got := ExtensionFromContentType(tt.contentType)- if got != tt.expected {- t.Errorf("ExtensionFromContentType(%q) = %q, want %q", tt.contentType, got, tt.expected)- }- })- }-}--func TestSanitizeFilename(t *testing.T) {- tests := []struct {- name string- index int- expected string- }{- {"photo.jpg", 0, "photo"},- {"../../etc/passwd", 0, "etcpasswd"},- {"hello/world\\test", 0, "helloworldtest"},- {"normal_file", 0, "normal_file"},- {"", 0, "attachment-0"},- {" ", 0, "attachment-0"},- {"...", 1, "attachment-1"},- }- for _, tt := range tests {- t.Run(tt.name, func(t *testing.T) {- got := SanitizeFilename(tt.name, tt.index)- if got != tt.expected {- t.Errorf("SanitizeFilename(%q, %d) = %q, want %q", tt.name, tt.index, got, tt.expected)- }- })- }-}-```--- [ ] **Step 2: Run tests to verify they fail**--Run: `cd api && go test ./pkg/repositories/ -run "TestExtensionFromContentType|TestSanitizeFilename" -v`-Expected: FAIL — functions not defined--- [ ] **Step 3: Write the storage interface and utility functions**--Create `api/pkg/repositories/attachment_storage.go`:--```go-package repositories--import (- "context"- "fmt"- "path/filepath"- "strings"-)--// AttachmentStorage is the interface for storing and retrieving message attachments-type AttachmentStorage interface {- // Upload stores attachment data at the given path- Upload(ctx context.Context, path string, data []byte) error- // Download retrieves attachment data from the given path- Download(ctx context.Context, path string) ([]byte, error)- // Delete removes an attachment at the given path- Delete(ctx context.Context, path string) error-}--// contentTypeExtensions maps MIME types to file extensions-var contentTypeExtensions = map[string]string{- "image/jpeg": ".jpg",- "image/png": ".png",- "image/gif": ".gif",- "image/webp": ".webp",- "image/bmp": ".bmp",- "video/mp4": ".mp4",- "video/3gpp": ".3gp",- "audio/mpeg": ".mp3",- "audio/ogg": ".ogg",- "audio/amr": ".amr",- "application/pdf": ".pdf",- "text/vcard": ".vcf",- "text/x-vcard": ".vcf",-}--// AllowedContentTypes returns the set of allowed MIME types for attachments-func AllowedContentTypes() map[string]bool {- allowed := make(map[string]bool, len(contentTypeExtensions))- for ct := range contentTypeExtensions {- allowed[ct] = true- }- return allowed-}--// ExtensionFromContentType returns the file extension for a MIME content type.-// Returns ".bin" if the content type is not recognized.-func ExtensionFromContentType(contentType string) string {- if ext, ok := contentTypeExtensions[contentType]; ok {- return ext- }- return ".bin"-}--// ContentTypeFromExtension returns the MIME content type for a file extension.-// Returns "application/octet-stream" if the extension is not recognized.-func ContentTypeFromExtension(ext string) string {- for ct, e := range contentTypeExtensions {- if e == ext {- return ct- }- }- return "application/octet-stream"-}--// SanitizeFilename removes path separators and traversal sequences from a filename.-// Returns "attachment-{index}" if the sanitized name is empty.-func SanitizeFilename(name string, index int) string {- name = strings.TrimSuffix(name, filepath.Ext(name))- name = strings.ReplaceAll(name, "/", "")- name = strings.ReplaceAll(name, "\\", "")- name = strings.ReplaceAll(name, "..", "")- name = strings.TrimSpace(name)-- if name == "" {- return fmt.Sprintf("attachment-%d", index)- }- return name-}-```--- [ ] **Step 4: Run tests to verify they pass**--Run: `cd api && go test ./pkg/repositories/ -run "TestExtensionFromContentType|TestSanitizeFilename" -v`-Expected: PASS--- [ ] **Step 5: Commit**--```bash-cd api && git add -A && git commit -m "feat: add AttachmentStorage interface and content-type utilities"-```-------### Task 3: Memory storage implementation--**Files:**--- Create: `api/pkg/repositories/memory_attachment_storage.go`--- [ ] **Step 1: Write the implementation**--Create `api/pkg/repositories/memory_attachment_storage.go`:--```go-package repositories--import (- "context"- "fmt"- "sync"-- "github.com/NdoleStudio/httpsms/pkg/telemetry"- "github.com/palantir/stacktrace"-)--// MemoryAttachmentStorage stores attachments in memory-type MemoryAttachmentStorage struct {- logger telemetry.Logger- tracer telemetry.Tracer- data sync.Map-}--// NewMemoryAttachmentStorage creates a new MemoryAttachmentStorage-func NewMemoryAttachmentStorage(- logger telemetry.Logger,- tracer telemetry.Tracer,-) *MemoryAttachmentStorage {- return &MemoryAttachmentStorage{- logger: logger.WithService(fmt.Sprintf("%T", &MemoryAttachmentStorage{})),- tracer: tracer,- }-}--// Upload stores attachment data at the given path-func (s *MemoryAttachmentStorage) Upload(ctx context.Context, path string, data []byte) error {- _, span := s.tracer.Start(ctx)- defer span.End()-- s.data.Store(path, data)- s.logger.Info(fmt.Sprintf("stored attachment at path [%s] with size [%d]", path, len(data)))- return nil-}--// Download retrieves attachment data from the given path-func (s *MemoryAttachmentStorage) Download(ctx context.Context, path string) ([]byte, error) {- _, span := s.tracer.Start(ctx)- defer span.End()-- value, ok := s.data.Load(path)- if !ok {- return nil, stacktrace.NewError(fmt.Sprintf("attachment not found at path [%s]", path))- }- return value.([]byte), nil-}--// Delete removes an attachment at the given path-func (s *MemoryAttachmentStorage) Delete(ctx context.Context, path string) error {- _, span := s.tracer.Start(ctx)- defer span.End()-- s.data.Delete(path)- s.logger.Info(fmt.Sprintf("deleted attachment at path [%s]", path))- return nil-}-```--- [ ] **Step 2: Verify build**--Run: `cd api && go build ./...`-Expected: Build succeeds--- [ ] **Step 3: Commit**--```bash-cd api && git add -A && git commit -m "feat: add MemoryAttachmentStorage implementation"-```-------### Task 4: GCS storage implementation--**Files:**--- Create: `api/pkg/repositories/gcs_attachment_storage.go`--- [ ] **Step 1: Write the implementation**--Create `api/pkg/repositories/gcs_attachment_storage.go`:--```go-package repositories--import (- "context"- "fmt"- "io"-- "cloud.google.com/go/storage"- "github.com/NdoleStudio/httpsms/pkg/telemetry"- "github.com/palantir/stacktrace"-)--// GCSAttachmentStorage stores attachments in Google Cloud Storage-type GCSAttachmentStorage struct {- logger telemetry.Logger- tracer telemetry.Tracer- client *storage.Client- bucket string-}--// NewGCSAttachmentStorage creates a new GCSAttachmentStorage-func NewGCSAttachmentStorage(- logger telemetry.Logger,- tracer telemetry.Tracer,- client *storage.Client,- bucket string,-) *GCSAttachmentStorage {- return &GCSAttachmentStorage{- logger: logger.WithService(fmt.Sprintf("%T", &GCSAttachmentStorage{})),- tracer: tracer,- client: client,- bucket: bucket,- }-}--// Upload stores attachment data at the given path in GCS-func (s *GCSAttachmentStorage) Upload(ctx context.Context, path string, data []byte) error {- ctx, span := s.tracer.Start(ctx)- defer span.End()-- writer := s.client.Bucket(s.bucket).Object(path).NewWriter(ctx)- if _, err := writer.Write(data); err != nil {- return s.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, fmt.Sprintf("cannot write attachment to GCS path [%s]", path)))- }-- if err := writer.Close(); err != nil {- return s.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, fmt.Sprintf("cannot close GCS writer for path [%s]", path)))- }-- s.logger.Info(fmt.Sprintf("uploaded attachment to GCS path [%s/%s] with size [%d]", s.bucket, path, len(data)))- return nil-}--// Download retrieves attachment data from the given path in GCS-func (s *GCSAttachmentStorage) Download(ctx context.Context, path string) ([]byte, error) {- ctx, span := s.tracer.Start(ctx)- defer span.End()-- reader, err := s.client.Bucket(s.bucket).Object(path).NewReader(ctx)- if err != nil {- return nil, s.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, fmt.Sprintf("cannot open GCS reader for path [%s]", path)))- }- defer reader.Close()-- data, err := io.ReadAll(reader)- if err != nil {- return nil, s.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, fmt.Sprintf("cannot read attachment from GCS path [%s]", path)))- }-- return data, nil-}--// Delete removes an attachment at the given path in GCS-func (s *GCSAttachmentStorage) Delete(ctx context.Context, path string) error {- ctx, span := s.tracer.Start(ctx)- defer span.End()-- if err := s.client.Bucket(s.bucket).Object(path).Delete(ctx); err != nil {- return s.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, fmt.Sprintf("cannot delete GCS object at path [%s]", path)))- }-- s.logger.Info(fmt.Sprintf("deleted attachment from GCS path [%s/%s]", s.bucket, path))- return nil-}-```--- [ ] **Step 2: Verify build**--Run: `cd api && go build ./...`-Expected: Build succeeds--- [ ] **Step 3: Commit**--```bash-cd api && git add -A && git commit -m "feat: add GCSAttachmentStorage implementation"-```-------### Task 5: Update request and event structs--**Files:**--- Modify: `api/pkg/requests/message_receive_request.go` (full file)-- Modify: `api/pkg/services/message_service.go:290-300` (MessageReceiveParams)-- Modify: `api/pkg/events/message_phone_received_event.go:14-23` (payload struct)--**Important:** The `requests` package already imports `services` (for `ToMessageReceiveParams`), so we **cannot** import `requests` from `services`. Define a `ServiceAttachment` struct in the services package to avoid a circular import.--- [ ] **Step 1: Add ServiceAttachment to services package**--In `api/pkg/services/message_service.go`, add after the imports (before `MessageService` struct at line 22):--```go-// ServiceAttachment represents attachment data passed to the service layer-type ServiceAttachment struct {- Name string- ContentType string- Content string // base64-encoded-}-```--Update `MessageReceiveParams` (lines 290-300) to add `Attachments`:--```go-type MessageReceiveParams struct {- Contact string- UserID entities.UserID- Owner phonenumbers.PhoneNumber- Content string- SIM entities.SIM- Timestamp time.Time- Encrypted bool- Source string- Attachments []ServiceAttachment-}-```--- [ ] **Step 2: Add MessageAttachment struct and update MessageReceive request**--In `api/pkg/requests/message_receive_request.go`, add the `MessageAttachment` struct before the `MessageReceive` struct, and add the `Attachments` field:--```go-// MessageAttachment represents a single MMS attachment in a receive request-type MessageAttachment struct {- // Name is the original filename of the attachment- Name string `json:"name" example:"photo.jpg"`- // ContentType is the MIME type of the attachment- ContentType string `json:"content_type" example:"image/jpeg"`- // Content is the base64-encoded attachment data- Content string `json:"content" example:"base64data..."`-}--// MessageReceive is the payload for receiving an SMS/MMS message-type MessageReceive struct {- request- From string `json:"from" example:"+18005550199"`- To string `json:"to" example:"+18005550100"`- Content string `json:"content" example:"This is a sample text message received on a phone"`- // Encrypted is used to determine if the content is end-to-end encrypted- Encrypted bool `json:"encrypted" example:"false"`- // SIM card that received the message- SIM entities.SIM `json:"sim" example:"SIM1"`- // Timestamp is the time when the event was emitted- Timestamp time.Time `json:"timestamp" example:"2022-06-05T14:26:09.527976+03:00"`- // Attachments is the list of MMS attachments received with the message- Attachments []MessageAttachment `json:"attachments"`-}-```--Update `ToMessageReceiveParams` to convert attachments:--```go-func (input *MessageReceive) ToMessageReceiveParams(userID entities.UserID, source string) *services.MessageReceiveParams {- phone, _ := phonenumbers.Parse(input.To, phonenumbers.UNKNOWN_REGION)-- attachments := make([]services.ServiceAttachment, len(input.Attachments))- for i, a := range input.Attachments {- attachments[i] = services.ServiceAttachment{- Name: a.Name,- ContentType: a.ContentType,- Content: a.Content,- }- }-- return &services.MessageReceiveParams{- Source: source,- Contact: input.From,- UserID: userID,- Timestamp: input.Timestamp,- Encrypted: input.Encrypted,- Owner: *phone,- Content: input.Content,- SIM: input.SIM,- Attachments: attachments,- }-}-```--- [ ] **Step 3: Update MessagePhoneReceivedPayload**--In `api/pkg/events/message_phone_received_event.go`, add `Attachments` field to the payload struct (after the `SIM` field):--```go-type MessagePhoneReceivedPayload struct {- MessageID uuid.UUID `json:"message_id"`- UserID entities.UserID `json:"user_id"`- Owner string `json:"owner"`- Encrypted bool `json:"encrypted"`- Contact string `json:"contact"`- Timestamp time.Time `json:"timestamp"`- Content string `json:"content"`- SIM entities.SIM `json:"sim"`- Attachments []string `json:"attachments"`-}-```--- [ ] **Step 4: Verify build compiles (will fail until service constructor is updated)**--Run: `cd api && go vet ./pkg/requests/... ./pkg/events/...`-Expected: No errors in these packages--- [ ] **Step 5: Commit**--```bash-cd api && git add -A && git commit -m "feat: add attachment fields to request, params, and event structs"-```-------### Task 6: Add attachment validation--**Files:**--- Modify: `api/pkg/validators/message_handler_validator.go:49-77`--- [ ] **Step 1: Update ValidateMessageReceive to validate attachments**--In `api/pkg/validators/message_handler_validator.go`, replace the `ValidateMessageReceive` method (lines 49-77) with:--```go-const (- maxAttachmentCount = 10- maxAttachmentSize = (3 * 1024 * 1024) / 2 // 1.5 MB-)--// ValidateMessageReceive validates the requests.MessageReceive request-func (validator MessageHandlerValidator) ValidateMessageReceive(_ context.Context, request requests.MessageReceive) url.Values {- v := govalidator.New(govalidator.Options{- Data: &request,- Rules: govalidator.MapData{- "to": []string{- "required",- phoneNumberRule,- },- "from": []string{- "required",- },- "content": []string{- "required",- "min:1",- "max:2048",- },- "sim": []string{- "required",- "in:" + strings.Join([]string{- string(entities.SIM1),- string(entities.SIM2),- }, ","),- },- },- })-- errors := v.ValidateStruct()-- if len(request.Attachments) > 0 {- attachmentErrors := validator.validateAttachments(request.Attachments)- for key, values := range attachmentErrors {- for _, value := range values {- errors.Add(key, value)- }- }- }-- return errors-}--func (validator MessageHandlerValidator) validateAttachments(attachments []requests.MessageAttachment) url.Values {- errors := url.Values{}- allowedTypes := repositories.AllowedContentTypes()-- if len(attachments) > maxAttachmentCount {- errors.Add("attachments", fmt.Sprintf("attachment count [%d] exceeds maximum of [%d]", len(attachments), maxAttachmentCount))- return errors- }-- for i, attachment := range attachments {- if !allowedTypes[attachment.ContentType] {- errors.Add("attachments", fmt.Sprintf("attachment [%d] has unsupported content type [%s]", i, attachment.ContentType))- continue- }-- decoded, err := base64.StdEncoding.DecodeString(attachment.Content)- if err != nil {- errors.Add("attachments", fmt.Sprintf("attachment [%d] has invalid base64 content", i))- continue- }-- if len(decoded) > maxAttachmentSize {- errors.Add("attachments", fmt.Sprintf("attachment [%d] size [%d] exceeds maximum of [%d] bytes", i, len(decoded), maxAttachmentSize))- }- }-- return errors-}-```--Add these imports to the file: `"encoding/base64"`, `"github.com/NdoleStudio/httpsms/pkg/repositories"`.--- [ ] **Step 2: Verify build**--Run: `cd api && go vet ./pkg/validators/...`-Expected: No errors--- [ ] **Step 3: Commit**--```bash-cd api && git add -A && git commit -m "feat: add attachment count, size, and content-type validation"-```-------### Task 7: Upload logic in MessageService.ReceiveMessage()--**Files:**--- Modify: `api/pkg/services/message_service.go:22-47` (struct + constructor)-- Modify: `api/pkg/services/message_service.go:302-337` (ReceiveMessage)-- Modify: `api/pkg/services/message_service.go:550-581` (storeReceivedMessage)--- [ ] **Step 1: Add AttachmentStorage and apiBaseURL to MessageService**--Update the `MessageService` struct (lines 22-30):--```go-type MessageService struct {- service- logger telemetry.Logger- tracer telemetry.Tracer- eventDispatcher *EventDispatcher- phoneService *PhoneService- repository repositories.MessageRepository- attachmentStorage repositories.AttachmentStorage- apiBaseURL string-}-```--Update `NewMessageService` (lines 33-47) to accept the new parameters:--```go-func NewMessageService(- logger telemetry.Logger,- tracer telemetry.Tracer,- repository repositories.MessageRepository,- eventDispatcher *EventDispatcher,- phoneService *PhoneService,- attachmentStorage repositories.AttachmentStorage,- apiBaseURL string,-) (s *MessageService) {- return &MessageService{- logger: logger.WithService(fmt.Sprintf("%T", s)),- tracer: tracer,- repository: repository,- phoneService: phoneService,- eventDispatcher: eventDispatcher,- attachmentStorage: attachmentStorage,- apiBaseURL: apiBaseURL,- }-}-```--- [ ] **Step 2: Add the uploadAttachments helper method**--Add this after `storeReceivedMessage`. Add imports: `"encoding/base64"`, `"golang.org/x/sync/errgroup"`:--```go-func (service *MessageService) uploadAttachments(ctx context.Context, userID entities.UserID, messageID uuid.UUID, attachments []ServiceAttachment) ([]string, error) {- ctx, span := service.tracer.Start(ctx)- defer span.End()-- ctxLogger := service.tracer.CtxLogger(service.logger, span)-- g, gCtx := errgroup.WithContext(ctx)- urls := make([]string, len(attachments))- paths := make([]string, len(attachments))-- for i, attachment := range attachments {- i, attachment := i, attachment- g.Go(func() error {- decoded, err := base64.StdEncoding.DecodeString(attachment.Content)- if err != nil {- return stacktrace.Propagate(err, fmt.Sprintf("cannot decode base64 content for attachment [%d]", i))- }-- sanitizedName := repositories.SanitizeFilename(attachment.Name, i)- ext := repositories.ExtensionFromContentType(attachment.ContentType)- filename := sanitizedName + ext-- path := fmt.Sprintf("attachments/%s/%s/%d/%s", userID, messageID, i, filename)- paths[i] = path-- if err = service.attachmentStorage.Upload(gCtx, path, decoded); err != nil {- return stacktrace.Propagate(err, fmt.Sprintf("cannot upload attachment [%d] to path [%s]", i, path))- }-- urls[i] = fmt.Sprintf("%s/v1/attachments/%s/%s/%d/%s", service.apiBaseURL, userID, messageID, i, filename)- ctxLogger.Info(fmt.Sprintf("uploaded attachment [%d] to [%s]", i, path))- return nil- })- }-- if err := g.Wait(); err != nil {- for _, path := range paths {- if path != "" {- _ = service.attachmentStorage.Delete(ctx, path)- }- }- return nil, stacktrace.Propagate(err, "cannot upload attachments")- }-- return urls, nil-}-```--- [ ] **Step 3: Update ReceiveMessage to upload attachments before event dispatch**--Replace the `ReceiveMessage` method (lines 302-337):--```go-func (service *MessageService) ReceiveMessage(ctx context.Context, params *MessageReceiveParams) (*entities.Message, error) {- ctx, span := service.tracer.Start(ctx)- defer span.End()-- ctxLogger := service.tracer.CtxLogger(service.logger, span)-- messageID := uuid.New()- var attachmentURLs []string-- if len(params.Attachments) > 0 {- ctxLogger.Info(fmt.Sprintf("uploading [%d] attachments for message [%s]", len(params.Attachments), messageID))- var err error- attachmentURLs, err = service.uploadAttachments(ctx, params.UserID, messageID, params.Attachments)- if err != nil {- msg := fmt.Sprintf("cannot upload attachments for message [%s]", messageID)- return nil, service.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, msg))- }- }-- eventPayload := events.MessagePhoneReceivedPayload{- MessageID: messageID,- UserID: params.UserID,- Encrypted: params.Encrypted,- Owner: phonenumbers.Format(¶ms.Owner, phonenumbers.E164),- Contact: params.Contact,- Timestamp: params.Timestamp,- Content: params.Content,- SIM: params.SIM,- Attachments: attachmentURLs,- }-- ctxLogger.Info(fmt.Sprintf("creating cloud event for received with ID [%s]", eventPayload.MessageID))-- event, err := service.createMessagePhoneReceivedEvent(params.Source, eventPayload)- if err != nil {- msg := fmt.Sprintf("cannot create %T from payload with message id [%s]", event, eventPayload.MessageID)- return nil, service.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, msg))- }-- ctxLogger.Info(fmt.Sprintf("created event [%s] with id [%s] and message id [%s]", event.Type(), event.ID(), eventPayload.MessageID))-- if err = service.eventDispatcher.Dispatch(ctx, event); err != nil {- msg := fmt.Sprintf("cannot dispatch event type [%s] and id [%s]", event.Type(), event.ID())- return nil, service.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, msg))- }- ctxLogger.Info(fmt.Sprintf("event [%s] dispatched successfully", event.ID()))-- return service.storeReceivedMessage(ctx, eventPayload)-}-```--- [ ] **Step 4: Update storeReceivedMessage to set Attachments on message**--In the `storeReceivedMessage` method (lines 550-581), add `Attachments` to the message construction:--```go- message := &entities.Message{- ID: params.MessageID,- Owner: params.Owner,- UserID: params.UserID,- Contact: params.Contact,- Content: params.Content,- Attachments: params.Attachments,- SIM: params.SIM,- Encrypted: params.Encrypted,- Type: entities.MessageTypeMobileOriginated,- Status: entities.MessageStatusReceived,- RequestReceivedAt: params.Timestamp,- CreatedAt: time.Now().UTC(),- UpdatedAt: time.Now().UTC(),- OrderTimestamp: params.Timestamp,- ReceivedAt: ¶ms.Timestamp,- }-```--- [ ] **Step 5: Verify the services package compiles**--Run: `cd api && go vet ./pkg/services/...`-Expected: No errors (the full build may still fail until DI container is updated)--- [ ] **Step 6: Commit**--```bash-cd api && git add -A && git commit -m "feat: add attachment upload logic to MessageService.ReceiveMessage()"-```-------### Task 8: Attachment download handler--**Files:**--- Create: `api/pkg/handlers/attachment_handler.go`--- [ ] **Step 1: Write the handler**--Create `api/pkg/handlers/attachment_handler.go`:--```go-package handlers--import (- "fmt"- "path/filepath"-- "github.com/NdoleStudio/httpsms/pkg/repositories"- "github.com/NdoleStudio/httpsms/pkg/telemetry"- "github.com/gofiber/fiber/v2"- "github.com/palantir/stacktrace"-)--// AttachmentHandler handles attachment download requests-type AttachmentHandler struct {- handler- logger telemetry.Logger- tracer telemetry.Tracer- storage repositories.AttachmentStorage-}--// NewAttachmentHandler creates a new AttachmentHandler-func NewAttachmentHandler(- logger telemetry.Logger,- tracer telemetry.Tracer,- storage repositories.AttachmentStorage,-) (h *AttachmentHandler) {- return &AttachmentHandler{- logger: logger.WithService(fmt.Sprintf("%T", h)),- tracer: tracer,- storage: storage,- }-}--// RegisterRoutes registers the routes for the AttachmentHandler (no auth middleware — public endpoint)-func (h *AttachmentHandler) RegisterRoutes(router fiber.Router) {- router.Get("/v1/attachments/:userID/:messageID/:attachmentIndex/:filename", h.GetAttachment)-}--// GetAttachment downloads an attachment-// @Summary Download a message attachment-// @Description Download an MMS attachment by its path components-// @Tags Attachments-// @Produce octet-stream-// @Param userID path string true "User ID"-// @Param messageID path string true "Message ID"-// @Param attachmentIndex path string true "Attachment index"-// @Param filename path string true "Filename with extension"-// @Success 200 {file} binary-// @Failure 404 {object} responses.NotFoundResponse-// @Failure 500 {object} responses.InternalServerError-// @Router /attachments/{userID}/{messageID}/{attachmentIndex}/{filename} [get]-func (h *AttachmentHandler) GetAttachment(c *fiber.Ctx) error {- ctx, span := h.tracer.StartFromFiberCtx(c)- defer span.End()-- ctxLogger := h.tracer.CtxLogger(h.logger, span)-- userID := c.Params("userID")- messageID := c.Params("messageID")- attachmentIndex := c.Params("attachmentIndex")- filename := c.Params("filename")-- path := fmt.Sprintf("attachments/%s/%s/%s/%s", userID, messageID, attachmentIndex, filename)-- ctxLogger.Info(fmt.Sprintf("downloading attachment from path [%s]", path))-- data, err := h.storage.Download(ctx, path)- if err != nil {- msg := fmt.Sprintf("cannot download attachment from path [%s]", path)- ctxLogger.Warn(stacktrace.Propagate(err, msg))- return h.responseNotFound(c, "attachment not found")- }-- ext := filepath.Ext(filename)- contentType := repositories.ContentTypeFromExtension(ext)-- c.Set("Content-Type", contentType)- c.Set("Content-Disposition", "attachment")- c.Set("X-Content-Type-Options", "nosniff")-- return c.Send(data)-}-```--- [ ] **Step 2: Verify build**--Run: `cd api && go vet ./pkg/handlers/...`-Expected: No errors--- [ ] **Step 3: Commit**--```bash-cd api && git add -A && git commit -m "feat: add AttachmentHandler for downloading attachments"-```-------### Task 9: Wire everything in the DI container and env config--**Files:**--- Modify: `api/pkg/di/container.go:104-163` (NewContainer)-- Modify: `api/pkg/di/container.go:1424-1434` (MessageService creation)-- Modify: `api/.env.docker`--- [ ] **Step 1: Add GCS_BUCKET_NAME to .env.docker**--In `api/.env.docker`, add after the `REDIS_URL=redis://@redis:6379` line (line 49):--```env--# Google Cloud Storage bucket for MMS attachments. Leave empty to use in-memory storage.-GCS_BUCKET_NAME=-```--- [ ] **Step 2: Add `attachmentStorage` field to Container struct**--In `api/pkg/di/container.go`, add `attachmentStorage` to the `Container` struct (around line 82-90):--```go-type Container struct {- projectID string- db *gorm.DB- dedicatedDB *gorm.DB- version string- app *fiber.App- eventDispatcher *services.EventDispatcher- logger telemetry.Logger- attachmentStorage repositories.AttachmentStorage-}-```--- [ ] **Step 3: Add AttachmentStorage, APIBaseURL, and AttachmentHandler getters to container.go**--Add these methods to `api/pkg/di/container.go`. Also add required imports: `"cloud.google.com/go/storage"` and `"context"`:--```go-// AttachmentStorage creates a cached AttachmentStorage based on configuration-func (container *Container) AttachmentStorage() repositories.AttachmentStorage {- if container.attachmentStorage != nil {- return container.attachmentStorage- }-- bucket := os.Getenv("GCS_BUCKET_NAME")- if bucket != "" {- container.logger.Debug("creating GCSAttachmentStorage")- client, err := storage.NewClient(context.Background())- if err != nil {- container.logger.Fatal(stacktrace.Propagate(err, "cannot create GCS client"))- }- container.attachmentStorage = repositories.NewGCSAttachmentStorage(- container.Logger(),- container.Tracer(),- client,- bucket,- )- } else {- container.logger.Debug("creating MemoryAttachmentStorage (GCS_BUCKET_NAME not set)")- container.attachmentStorage = repositories.NewMemoryAttachmentStorage(- container.Logger(),- container.Tracer(),- )- }-- return container.attachmentStorage-}--// APIBaseURL returns the API base URL derived from EVENTS_QUEUE_ENDPOINT-func (container *Container) APIBaseURL() string {- endpoint := os.Getenv("EVENTS_QUEUE_ENDPOINT")- return strings.TrimSuffix(endpoint, "/v1/events")-}--// AttachmentHandler creates a new AttachmentHandler-func (container *Container) AttachmentHandler() (handler *handlers.AttachmentHandler) {- container.logger.Debug(fmt.Sprintf("creating %T", handler))- return handlers.NewAttachmentHandler(- container.Logger(),- container.Tracer(),- container.AttachmentStorage(),- )-}--// RegisterAttachmentRoutes registers routes for the /attachments prefix-func (container *Container) RegisterAttachmentRoutes() {- container.logger.Debug(fmt.Sprintf("registering %T routes", &handlers.AttachmentHandler{}))- container.AttachmentHandler().RegisterRoutes(container.App())-}-```--- [ ] **Step 3: Update MessageService creation to pass new parameters**--Update the `MessageService()` getter (around line 1424-1434):--```go-func (container *Container) MessageService() (service *services.MessageService) {- container.logger.Debug(fmt.Sprintf("creating %T", service))- return services.NewMessageService(- container.Logger(),- container.Tracer(),- container.MessageRepository(),- container.EventDispatcher(),- container.PhoneService(),- container.AttachmentStorage(),- container.APIBaseURL(),- )-}-```--- [ ] **Step 4: Register attachment routes in NewContainer**--In the `NewContainer` function (lines 104-163), add `container.RegisterAttachmentRoutes()` after `container.RegisterMessageRoutes()` (after line 120):--```go- container.RegisterMessageRoutes()- container.RegisterAttachmentRoutes()- container.RegisterBulkMessageRoutes()-```--- [ ] **Step 5: Verify full build**--Run: `cd api && go build ./...`-Expected: Build succeeds — all components are now wired--- [ ] **Step 6: Run all tests**--Run: `cd api && go test ./...`-Expected: All tests pass--- [ ] **Step 7: Commit**--```bash-cd api && git add -A && git commit -m "feat: wire attachment storage and handler in DI container--- Add AttachmentStorage selection (GCS vs memory) based on GCS_BUCKET_NAME env var-- Wire AttachmentHandler for public download endpoint-- Pass storage and API base URL to MessageService-- Add GCS_BUCKET_NAME to .env.docker"-```-------### Task 10: Final verification--- [ ] **Step 1: Run full build**--Run: `cd api && go build -o ./tmp/main.exe .`-Expected: Build succeeds--- [ ] **Step 2: Run all tests**--Run: `cd api && go test ./... -v`-Expected: All tests pass including `TestExtensionFromContentType` and `TestSanitizeFilename`--- [ ] **Step 3: Verify go vet**--Run: `cd api && go vet ./...`-Expected: No issues--- [ ] **Step 4: Final commit if any remaining changes**--```bash-cd api && git add -A && git diff --cached --stat-```diff --git a/docs/superpowers/plans/2026-05-03-integration-test-setup.md b/docs/superpowers/plans/2026-05-03-integration-test-setup.mddeleted file mode 100644index 9a8f8cb6..00000000--- a/docs/superpowers/plans/2026-05-03-integration-test-setup.md+++ /dev/null@@ -1,1107 +0,0 @@-# Integration Test Setup Implementation Plan--> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.--**Goal:** Create a CI-gated integration test that validates the full SMS send/receive flow using Docker, a phone emulator, and real FCM code paths redirected to the emulator.--**Architecture:** Docker Compose brings up PostgreSQL + Redis + API + Emulator. The API's Firebase SDK is configured to route FCM traffic to the emulator via a custom HTTP transport. A Go test runner on the host exercises the API and asserts on message state.--**Tech Stack:** Go, Docker Compose, PostgreSQL, Redis, Firebase Admin Go SDK, GitHub Actions-------## File Structure--```-tests/-├── docker-compose.yml # orchestrates all services-├── seed.sql # seeds test user, phone, API keys-├── .env.test # API environment config for tests-├── firebase-credentials.json # fake service account JSON-├── go.mod # test runner Go module-├── go.sum-├── integration_test.go # test cases (send SMS, receive SMS)-├── helpers_test.go # HTTP client, polling, constants-└── emulator/- ├── Dockerfile # builds emulator binary- ├── go.mod # emulator Go module- ├── go.sum- ├── main.go # entry point, HTTP server setup- ├── fcm_handler.go # fake FCM endpoint handler- ├── token_handler.go # fake OAuth2 token endpoint- └── events.go # fires SENT/DELIVERED events to API--api/pkg/di/container.go # modified: FCM transport redirect-.github/workflows/integration-test.yml # new CI workflow-```-------### Task 1: Create Feature Branch--**Files:**--- None (git operations only)--- [ ] **Step 1: Create and switch to feature branch from main**--```bash-cd C:\Users\Arnold\Work\NdoleStudio\httpsms.com-git checkout main-git pull origin main-git checkout -b feature/integration-tests-```--- [ ] **Step 2: Verify branch**--Run: `git branch --show-current`-Expected: `feature/integration-tests`-------### Task 2: API Modification — FCM Transport Override--**Files:**--- Modify: `api/pkg/di/container.go:396-405` (FirebaseApp method)--- [ ] **Step 1: Add the FCM redirect transport and modify FirebaseApp**--In `api/pkg/di/container.go`, modify the `FirebaseApp()` method to check for `FCM_ENDPOINT` env var. When set, use ONLY a custom HTTP client (no credentials). When not set, use credentials as before.--**Important:** `option.WithHTTPClient()` takes precedence over all other options in the Firebase SDK. Do NOT combine it with `option.WithAuthCredentialsJSON()`. Use one or the other.--Create a new file `api/pkg/di/fcm_transport.go`:--```go-package di--import (- "net/http"- "net/url"-)--// fcmRedirectTransport rewrites Firebase SDK HTTP requests to a custom endpoint.-// Used in integration tests to redirect FCM traffic to the emulator.-type fcmRedirectTransport struct {- target *url.URL- base http.RoundTripper-}--func (t *fcmRedirectTransport) RoundTrip(req *http.Request) (*http.Response, error) {- req.URL.Scheme = t.target.Scheme- req.URL.Host = t.target.Host- return t.base.RoundTrip(req)-}-```--Then modify `FirebaseApp()` in `container.go`:--```go-// FirebaseApp creates a new instance of firebase.App-func (container *Container) FirebaseApp() (app *firebase.App) {- container.logger.Debug(fmt.Sprintf("creating %T", app))-- var opts []option.ClientOption-- if fcmEndpoint := os.Getenv("FCM_ENDPOINT"); fcmEndpoint != "" {- container.logger.Info(fmt.Sprintf("using FCM endpoint override: %s", fcmEndpoint))- targetURL, err := url.Parse(fcmEndpoint)- if err != nil {- container.logger.Fatal(stacktrace.Propagate(err, "cannot parse FCM_ENDPOINT"))- }- opts = append(opts, option.WithHTTPClient(&http.Client{- Transport: &fcmRedirectTransport{- target: targetURL,- base: http.DefaultTransport,- },- }))- } else {- opts = append(opts, option.WithAuthCredentialsJSON(option.ServiceAccount, container.FirebaseCredentials()))- }-- app, err := firebase.NewApp(context.Background(), nil, opts...)- if err != nil {- msg := "cannot initialize firebase application"- container.logger.Fatal(stacktrace.Propagate(err, msg))- }- return app-}-```--- [ ] **Step 2: Add `net/url` import if not already present**--Ensure the `net/url` package is imported in `container.go` (or the new file).--- [ ] **Step 3: Verify API still builds**--Run: `cd api && go build ./...`-Expected: Build succeeds with no errors.--- [ ] **Step 4: Commit**--```bash-git add api/pkg/di/-git commit -m "feat(api): add FCM_ENDPOINT transport override for integration tests"-```-------### Task 3: Emulator — Project Scaffolding--**Files:**--- Create: `tests/emulator/go.mod`-- Create: `tests/emulator/emulator.go`-- Create: `tests/emulator/Dockerfile`--Note: `main.go` references `NewEmulator()` and handlers, so we create the struct first. `main.go` is created AFTER all handlers exist (Task 6b).--- [ ] **Step 1: Initialize emulator Go module**--```bash-mkdir -p tests/emulator-cd tests/emulator-go mod init github.com/NdoleStudio/httpsms/tests/emulator-```--- [ ] **Step 2: Create `tests/emulator/emulator.go`**--```go-package main--import "net/http"--// Emulator acts as a fake Android phone that receives FCM pushes-// and responds with message events.-type Emulator struct {- apiBaseURL string- phoneAPIKey string- httpClient *http.Client-}--// NewEmulator creates a new Emulator instance.-func NewEmulator(apiBaseURL, phoneAPIKey string) *Emulator {- return &Emulator{- apiBaseURL: apiBaseURL,- phoneAPIKey: phoneAPIKey,- httpClient: &http.Client{},- }-}--// HealthHandler returns 200 OK for health checks.-func (e *Emulator) HealthHandler(w http.ResponseWriter, r *http.Request) {- w.WriteHeader(http.StatusOK)- w.Write([]byte("ok"))-}-```--- [ ] **Step 3: Create `tests/emulator/Dockerfile`**--```dockerfile-FROM golang:1.22 AS builder--WORKDIR /app-COPY go.mod go.sum ./-RUN go mod download-COPY . .-RUN CGO_ENABLED=0 GOOS=linux go build -o /bin/emulator .--FROM alpine:latest-RUN apk add --no-cache ca-certificates-COPY --from=builder /bin/emulator /bin/emulator-EXPOSE 9090-ENTRYPOINT ["/bin/emulator"]-```--- [ ] **Step 4: Commit**--```bash-git add tests/emulator/-git commit -m "feat(tests): scaffold emulator Go project"-```-------### Task 4: Emulator — Token Handler--**Files:**--- Create: `tests/emulator/token_handler.go`--- [ ] **Step 1: Create `tests/emulator/token_handler.go`**--```go-package main--import (- "encoding/json"- "net/http"-)--// TokenHandler returns a fake OAuth2 access token.-// The Firebase Admin SDK calls this endpoint to get an access token-// before making FCM API calls.-func (e *Emulator) TokenHandler(w http.ResponseWriter, r *http.Request) {- response := map[string]interface{}{- "access_token": "fake-access-token",- "token_type": "Bearer",- "expires_in": 3600,- }-- w.Header().Set("Content-Type", "application/json")- json.NewEncoder(w).Encode(response)-}-```--- [ ] **Step 2: Commit**--```bash-git add tests/emulator/token_handler.go-git commit -m "feat(tests): add fake OAuth2 token handler to emulator"-```-------### Task 5: Emulator — FCM Handler--**Files:**--- Create: `tests/emulator/fcm_handler.go`--- [ ] **Step 1: Create `tests/emulator/fcm_handler.go`**--```go-package main--import (- "encoding/json"- "fmt"- "log"- "net/http"-)--// fcmRequest represents the FCM v1 API request body-type fcmRequest struct {- Message struct {- Data map[string]string `json:"data"`- Token string `json:"token"`- Android struct {- Priority string `json:"priority"`- } `json:"android"`- } `json:"message"`-}--// fcmResponse represents the FCM v1 API response-type fcmResponse struct {- Name string `json:"name"`-}--// FCMHandler handles fake FCM send requests from the Firebase Admin SDK.-func (e *Emulator) FCMHandler(w http.ResponseWriter, r *http.Request) {- var req fcmRequest- if err := json.NewDecoder(r.Body).Decode(&req); err != nil {- http.Error(w, "invalid request body", http.StatusBadRequest)- return- }-- messageID := req.Message.Data["KEY_MESSAGE_ID"]- if messageID == "" {- http.Error(w, "missing KEY_MESSAGE_ID in data", http.StatusBadRequest)- return- }-- log.Printf("received FCM push for message: %s", messageID)-- // Respond with success immediately (like real FCM would)- resp := fcmResponse{- Name: fmt.Sprintf("projects/httpsms-test/messages/fake-%s", messageID),- }- w.Header().Set("Content-Type", "application/json")- json.NewEncoder(w).Encode(resp)-- // Process the message asynchronously (like a real phone would)- go e.processMessage(messageID)-}-```--- [ ] **Step 3: Commit**--```bash-git add tests/emulator/emulator.go tests/emulator/fcm_handler.go-git commit -m "feat(tests): add FCM handler to emulator"-```-------### Task 6: Emulator — Event Firing--**Files:**--- Create: `tests/emulator/events.go`--- [ ] **Step 1: Create `tests/emulator/events.go`**--```go-package main--import (- "bytes"- "encoding/json"- "fmt"- "log"- "net/http"- "time"-)--// messageEvent is the payload for posting a message event to the API-type messageEvent struct {- Timestamp time.Time `json:"timestamp"`- EventName string `json:"event_name"`-}--// processMessage simulates a phone receiving an FCM push and sending the SMS.-// It calls /messages/outstanding, then fires SENT and DELIVERED events.-func (e *Emulator) processMessage(messageID string) {- // Step 1: Fetch outstanding message (like real phone does)- e.fetchOutstanding(messageID)-- // Step 2: Wait briefly then fire SENT- time.Sleep(200 * time.Millisecond)- if err := e.fireEvent(messageID, "SENT"); err != nil {- log.Printf("error firing SENT event for message %s: %v", messageID, err)- return- }-- // Step 3: Wait briefly then fire DELIVERED- time.Sleep(200 * time.Millisecond)- if err := e.fireEvent(messageID, "DELIVERED"); err != nil {- log.Printf("error firing DELIVERED event for message %s: %v", messageID, err)- return- }-- log.Printf("completed processing message: %s", messageID)-}--// fetchOutstanding calls GET /v1/messages/outstanding to mimic the real phone behavior-func (e *Emulator) fetchOutstanding(messageID string) {- url := fmt.Sprintf("%s/v1/messages/outstanding?message_id=%s", e.apiBaseURL, messageID)-- req, _ := http.NewRequest("GET", url, nil)- req.Header.Set("x-api-key", e.phoneAPIKey)-- resp, err := e.httpClient.Do(req)- if err != nil {- log.Printf("error fetching outstanding message %s: %v", messageID, err)- return- }- defer resp.Body.Close()- log.Printf("fetched outstanding message %s: status %d", messageID, resp.StatusCode)-}--// fireEvent posts a message event (SENT or DELIVERED) to the API-func (e *Emulator) fireEvent(messageID, eventName string) error {- url := fmt.Sprintf("%s/v1/messages/%s/events", e.apiBaseURL, messageID)-- event := messageEvent{- Timestamp: time.Now().UTC(),- EventName: eventName,- }-- body, _ := json.Marshal(event)- req, _ := http.NewRequest("POST", url, bytes.NewReader(body))- req.Header.Set("Content-Type", "application/json")- req.Header.Set("x-api-key", e.phoneAPIKey)-- resp, err := e.httpClient.Do(req)- if err != nil {- return fmt.Errorf("HTTP error: %w", err)- }- defer resp.Body.Close()-- if resp.StatusCode >= 400 {- return fmt.Errorf("API returned status %d for %s event", resp.StatusCode, eventName)- }-- log.Printf("fired %s event for message %s: status %d", eventName, messageID, resp.StatusCode)- return nil-}-```--- [ ] **Step 2: Commit**--```bash-git add tests/emulator/events.go-git commit -m "feat(tests): add event firing to emulator"-```-------### Task 6b: Emulator — Main Entry Point--**Files:**--- Create: `tests/emulator/main.go`--- [ ] **Step 1: Create `tests/emulator/main.go`**--Now that all handlers exist (HealthHandler, TokenHandler, FCMHandler), create the entry point:--```go-package main--import (- "log"- "net/http"- "os"-)--func main() {- apiBaseURL := os.Getenv("API_BASE_URL")- if apiBaseURL == "" {- apiBaseURL = "http://api:8000"- }-- phoneAPIKey := os.Getenv("PHONE_API_KEY")- if phoneAPIKey == "" {- phoneAPIKey = "pk_test-phone-api-key"- }-- emulator := NewEmulator(apiBaseURL, phoneAPIKey)-- mux := http.NewServeMux()- mux.HandleFunc("GET /health", emulator.HealthHandler)- mux.HandleFunc("POST /token", emulator.TokenHandler)- mux.HandleFunc("POST /v1/projects/{project}/messages:send", emulator.FCMHandler)-- port := os.Getenv("PORT")- if port == "" {- port = "9090"- }-- log.Printf("emulator listening on :%s", port)- if err := http.ListenAndServe(":"+port, mux); err != nil {- log.Fatalf("server error: %v", err)- }-}-```--- [ ] **Step 2: Verify emulator builds**--```bash-cd tests/emulator-go build ./...-```--Expected: Build succeeds.--- [ ] **Step 3: Commit**--```bash-git add tests/emulator/main.go-git commit -m "feat(tests): add emulator main entry point"-```-------### Task 7: Test Infrastructure — Seed Data & Config--**Files:**--- Create: `tests/seed.sql`-- Create: `tests/.env.test`-- Create: `tests/firebase-credentials.json`--- [ ] **Step 1: Create `tests/seed.sql`**--This script must match the exact table schema from entities. The tables are auto-migrated by GORM, so we insert after API startup. Actually — since we need the user to exist BEFORE the API processes requests, we seed via Docker's postgres init scripts.--Note: GORM auto-migrates tables on API startup. The seed SQL runs AFTER table creation. We use a Docker healthcheck + depends_on to ensure ordering. Alternatively, we can use a startup script that waits for the API to be ready, then seeds. The simplest approach: mount `seed.sql` as a Postgres init script — but that runs before GORM migrates.--**Better approach:** Create a `tests/seed.sh` script that waits for the API to start (which runs GORM migrations), then seeds the database via `psql`.--```sql--- tests/seed.sql--- Seed test data for integration tests--- Run AFTER GORM has migrated the schema (i.e., after API starts)---- Test user-INSERT INTO users (id, email, api_key, timezone, subscription_name, created_at, updated_at)-VALUES (- 'test-user-id',- '[email protected]',- 'test-user-api-key',- 'UTC',- 'pro-monthly',- NOW(),- NOW()-) ON CONFLICT (id) DO NOTHING;---- System user (for event queue auth)-INSERT INTO users (id, email, api_key, timezone, subscription_name, created_at, updated_at)-VALUES (- 'system-user-id',- '[email protected]',- 'system-user-api-key',- 'UTC',- 'pro-monthly',- NOW(),- NOW()-) ON CONFLICT (id) DO NOTHING;---- Test phone-INSERT INTO phones (id, user_id, fcm_token, phone_number, messages_per_minute, sim, max_send_attempts, message_expiration_seconds, created_at, updated_at)-VALUES (- 'a1b2c3d4-e5f6-7890-abcd-ef1234567890',- 'test-user-id',- 'fake-fcm-token',- '+18005550199',- 60,- 'SIM1',- 2,- 600,- NOW(),- NOW()-) ON CONFLICT (id) DO NOTHING;---- Phone API key (for emulator to authenticate as phone)-INSERT INTO phone_api_keys (id, name, user_id, user_email, phone_numbers, phone_ids, api_key, created_at, updated_at)-VALUES (- 'b2c3d4e5-f6a7-8901-bcde-f12345678901',- 'Integration Test Phone Key',- 'test-user-id',- '[email protected]',- '{"+18005550199"}',- '{"a1b2c3d4-e5f6-7890-abcd-ef1234567890"}',- 'pk_test-phone-api-key',- NOW(),- NOW()-) ON CONFLICT (id) DO NOTHING;-```--- [ ] **Step 2: Create `tests/.env.test`**--```env-ENV=production-GCP_PROJECT_ID=httpsms-test-USE_HTTP_LOGGER=true-ENTITLEMENT_ENABLED=false-EVENTS_QUEUE_TYPE=emulator-EVENTS_QUEUE_NAME=events-local-EVENTS_QUEUE_ENDPOINT=http://localhost:8000/v1/events-EVENTS_QUEUE_USER_API_KEY=system-user-api-key-EVENTS_QUEUE_USER_ID=system-user-id-FCM_ENDPOINT=http://emulator:9090-DATABASE_URL=postgresql://dbusername:dbpassword@postgres:5432/httpsms-DATABASE_URL_DEDICATED=postgresql://dbusername:dbpassword@postgres:5432/httpsms-REDIS_URL=redis://@redis:6379-APP_PORT=8000-APP_NAME=httpSMS-APP_URL=http://localhost:8000-SWAGGER_HOST=localhost:8000-SMTP_FROM_NAME=httpSMS-SMTP_USERNAME=-SMTP_PASSWORD=-SMTP_HOST=localhost-SMTP_PORT=2525-PUSHER_APP_ID=-PUSHER_KEY=-PUSHER_SECRET=-PUSHER_CLUSTER=-GCS_BUCKET_NAME=-UPTRACE_DSN=-CLOUDFLARE_TURNSTILE_SECRET_KEY=-```--- [ ] **Step 3: Create `tests/firebase-credentials.json`**--Generate an RSA private key for the fake service account. This must be a valid RSA key so the Firebase SDK can sign JWT tokens (even though the emulator won't validate them).--```bash-cd tests-openssl genrsa -out /tmp/test-key.pem 2048-```--Then create the JSON file with the key embedded:--```json-{- "type": "service_account",- "project_id": "httpsms-test",- "private_key_id": "test-key-id",- "private_key": "<contents of /tmp/test-key.pem with newlines as \\n>",- "client_email": "[email protected]",- "client_id": "123456789",- "auth_uri": "http://emulator:9090/auth",- "token_uri": "http://emulator:9090/token",- "auth_provider_x509_cert_url": "http://emulator:9090/certs",- "client_x509_cert_url": "http://emulator:9090/certs/test"-}-```--Note: The `FIREBASE_CREDENTIALS` env var in `.env.test` should be set to the full contents of this JSON file (single-line). The docker-compose will handle this.--- [ ] **Step 4: Commit**--```bash-git add tests/seed.sql tests/.env.test tests/firebase-credentials.json-git commit -m "feat(tests): add seed data and test environment config"-```-------### Task 8: Docker Compose for Tests--**Files:**--- Create: `tests/docker-compose.yml`--- [ ] **Step 1: Create `tests/docker-compose.yml`**--```yaml-services:- postgres:- image: postgres:alpine- environment:- POSTGRES_DB: httpsms- POSTGRES_PASSWORD: dbpassword- POSTGRES_USER: dbusername- ports:- - "5435:5432"- healthcheck:- test: ["CMD-SHELL", "pg_isready -U dbusername -d httpsms"]- interval: 5s- timeout: 5s- retries: 10- start_period: 5s-- redis:- image: redis:latest- command: redis-server- ports:- - "6379:6379"- healthcheck:- test: ["CMD", "redis-cli", "ping"]- interval: 5s- timeout: 5s- retries: 10-- emulator:- build:- context: ./emulator- ports:- - "9090:9090"- environment:- API_BASE_URL: http://api:8000- PHONE_API_KEY: pk_test-phone-api-key- PORT: "9090"- healthcheck:- test: ["CMD", "wget", "--spider", "-q", "http://localhost:9090/health"]- interval: 5s- timeout: 5s- retries: 10-- api:- build:- context: ../api- ports:- - "8000:8000"- depends_on:- postgres:- condition: service_healthy- redis:- condition: service_healthy- emulator:- condition: service_healthy- env_file:- - .env.test- environment:- FIREBASE_CREDENTIALS: "${FIREBASE_CREDENTIALS}"- healthcheck:- test: ["CMD", "wget", "--spider", "-q", "http://localhost:8000/"]- interval: 5s- timeout: 10s- retries: 20- start_period: 30s-- seed:- image: postgres:alpine- depends_on:- api:- condition: service_healthy- environment:- PGPASSWORD: dbpassword- volumes:- - ./seed.sql:/seed.sql:ro- entrypoint:- [- "psql",- "-h",- "postgres",- "-U",- "dbusername",- "-d",- "httpsms",- "-f",- "/seed.sql",- ]- restart: "no"-```--- [ ] **Step 2: Commit**--```bash-git add tests/docker-compose.yml-git commit -m "feat(tests): add docker-compose for integration test stack"-```-------### Task 9: Test Runner — Go Module & Helpers--**Files:**--- Create: `tests/go.mod`-- Create: `tests/helpers_test.go`--- [ ] **Step 1: Initialize test runner Go module**--```bash-cd tests-go mod init github.com/NdoleStudio/httpsms/tests-go get github.com/stretchr/testify-```--- [ ] **Step 2: Create `tests/helpers_test.go`**--```go-package tests--import (- "encoding/json"- "fmt"- "io"- "net/http"- "testing"- "time"-- "github.com/stretchr/testify/require"-)--const (- apiBaseURL = "http://localhost:8000"- userAPIKey = "test-user-api-key"- phoneAPIKey = "pk_test-phone-api-key"- testPhone = "+18005550199"- testContact = "+18005550100"-)--// apiClient returns an HTTP client configured for API calls-func apiClient() *http.Client {- return &http.Client{Timeout: 10 * time.Second}-}--// doRequest performs an HTTP request with the given API key-func doRequest(t *testing.T, method, url string, body io.Reader, apiKey string) *http.Response {- t.Helper()- req, err := http.NewRequest(method, url, body)- require.NoError(t, err)- req.Header.Set("Content-Type", "application/json")- req.Header.Set("x-api-key", apiKey)-- resp, err := apiClient().Do(req)- require.NoError(t, err)- return resp-}--// pollMessageStatus polls GET /v1/messages/{id} until the message reaches the target status or times out-func pollMessageStatus(t *testing.T, messageID, targetStatus string, timeout time.Duration) map[string]interface{} {- t.Helper()- deadline := time.Now().Add(timeout)-- for time.Now().Before(deadline) {- url := fmt.Sprintf("%s/v1/messages/%s", apiBaseURL, messageID)- resp := doRequest(t, "GET", url, nil, userAPIKey)-- body, err := io.ReadAll(resp.Body)- resp.Body.Close()- require.NoError(t, err)-- if resp.StatusCode == http.StatusOK {- var result map[string]interface{}- require.NoError(t, json.Unmarshal(body, &result))-- data, ok := result["data"].(map[string]interface{})- if ok && data["status"] == targetStatus {- return data- }- }-- time.Sleep(200 * time.Millisecond)- }-- t.Fatalf("message %s did not reach status %q within %v", messageID, targetStatus, timeout)- return nil-}-```--- [ ] **Step 3: Commit**--```bash-git add tests/go.mod tests/go.sum tests/helpers_test.go-git commit -m "feat(tests): add test runner module and helpers"-```-------### Task 10: Test Runner — Integration Tests--**Files:**--- Create: `tests/integration_test.go`--- [ ] **Step 1: Create `tests/integration_test.go`**--```go-package tests--import (- "bytes"- "encoding/json"- "fmt"- "io"- "net/http"- "testing"- "time"-- "github.com/stretchr/testify/assert"- "github.com/stretchr/testify/require"-)--func TestSendSMS_E2E(t *testing.T) {- // Step 1: Send an SMS via the API- sendPayload := map[string]interface{}{- "from": testPhone,- "to": testContact,- "content": "Hello from integration test",- }- body, _ := json.Marshal(sendPayload)-- url := fmt.Sprintf("%s/v1/messages/send", apiBaseURL)- resp := doRequest(t, "POST", url, bytes.NewReader(body), userAPIKey)- defer resp.Body.Close()-- respBody, err := io.ReadAll(resp.Body)- require.NoError(t, err)- require.Equal(t, http.StatusOK, resp.StatusCode, "send response: %s", string(respBody))-- // Step 2: Extract message ID- var sendResult map[string]interface{}- require.NoError(t, json.Unmarshal(respBody, &sendResult))- data := sendResult["data"].(map[string]interface{})- messageID := data["id"].(string)- require.NotEmpty(t, messageID)-- t.Logf("sent message with ID: %s", messageID)-- // Step 3: Poll until message is delivered- message := pollMessageStatus(t, messageID, "delivered", 15*time.Second)-- // Step 4: Assert final state- assert.Equal(t, "delivered", message["status"])- assert.Equal(t, testPhone, message["owner"])- assert.Equal(t, testContact, message["contact"])- assert.Equal(t, "Hello from integration test", message["content"])-}--func TestReceiveSMS_E2E(t *testing.T) {- // Step 1: Simulate receiving an SMS (phone -> API)- receivePayload := map[string]interface{}{- "from": testContact,- "to": testPhone,- "content": "Hi there from integration test",- "encrypted": false,- "sim": "SIM1",- "timestamp": time.Now().UTC().Format(time.RFC3339),- }- body, _ := json.Marshal(receivePayload)-- url := fmt.Sprintf("%s/v1/messages/receive", apiBaseURL)- resp := doRequest(t, "POST", url, bytes.NewReader(body), phoneAPIKey)- defer resp.Body.Close()-- respBody, err := io.ReadAll(resp.Body)- require.NoError(t, err)- require.Equal(t, http.StatusOK, resp.StatusCode, "receive response: %s", string(respBody))-- // Step 2: Extract message ID- var receiveResult map[string]interface{}- require.NoError(t, json.Unmarshal(respBody, &receiveResult))- data := receiveResult["data"].(map[string]interface{})- messageID := data["id"].(string)- require.NotEmpty(t, messageID)-- t.Logf("received message with ID: %s", messageID)-- // Step 3: Verify message exists via GET- getURL := fmt.Sprintf("%s/v1/messages/%s", apiBaseURL, messageID)- getResp := doRequest(t, "GET", getURL, nil, userAPIKey)- defer getResp.Body.Close()-- getBody, err := io.ReadAll(getResp.Body)- require.NoError(t, err)- require.Equal(t, http.StatusOK, getResp.StatusCode)-- var getMessage map[string]interface{}- require.NoError(t, json.Unmarshal(getBody, &getMessage))- messageData := getMessage["data"].(map[string]interface{})-- // Step 4: Assert message fields- assert.Equal(t, "received", messageData["status"])- assert.Equal(t, testPhone, messageData["owner"])- assert.Equal(t, testContact, messageData["contact"])- assert.Equal(t, "Hi there from integration test", messageData["content"])-}-```--- [ ] **Step 2: Verify test file compiles**--```bash-cd tests-go vet ./...-```--Expected: No errors (tests won't pass yet without the stack running).--- [ ] **Step 3: Commit**--```bash-git add tests/integration_test.go-git commit -m "feat(tests): add send and receive SMS integration tests"-```-------### Task 11: GitHub Actions Workflow--**Files:**--- Create: `.github/workflows/integration-test.yml`--- [ ] **Step 1: Create `.github/workflows/integration-test.yml`**--```yaml-name: integration-test--on:- push:- branches:- - main- pull_request:- branches:- - main--jobs:- integration-test:- runs-on: ubuntu-latest- steps:- - name: Checkout 🛎- uses: actions/checkout@v4-- - name: Set up Go- uses: actions/setup-go@v5- with:- go-version: "1.22"-- - name: Load Firebase credentials- run: |- echo "FIREBASE_CREDENTIALS=$(jq -c . tests/firebase-credentials.json)" >> $GITHUB_ENV-- - name: Start services 🐳- working-directory: ./tests- run: docker compose up -d --build --wait-- - name: Wait for seed to complete- working-directory: ./tests- run: |- echo "Waiting for seed container to finish..."- docker compose wait seed || true- sleep 2-- - name: Run integration tests 🧪- working-directory: ./tests- run: go test -v -timeout 120s ./...-- - name: Collect logs on failure 📋- if: failure()- working-directory: ./tests- run: |- docker compose logs api- docker compose logs emulator-- - name: Stop services 🛑- if: always()- working-directory: ./tests- run: docker compose down -v-```--- [ ] **Step 2: Commit**--```bash-git add .github/workflows/integration-test.yml-git commit -m "ci: add integration test workflow"-```-------### Task 12: Local End-to-End Verification--**Files:**--- None (verification only)--- [ ] **Step 1: Generate the fake Firebase credentials file**--```bash-cd tests-openssl genrsa 2048 > /tmp/test-key.pem-# Create firebase-credentials.json with the key (use a script or manually format)-```--- [ ] **Step 2: Build and start the stack**--```bash-cd tests-export FIREBASE_CREDENTIALS=$(jq -c . firebase-credentials.json)-docker compose up -d --build-```--- [ ] **Step 3: Wait for all services to be healthy**--```bash-docker compose ps-# All services should show "healthy" or "exited (0)" for seed-```--- [ ] **Step 4: Run the tests**--```bash-cd tests-go test -v -timeout 120s ./...-```--Expected: Both tests pass.--- [ ] **Step 5: Tear down**--```bash-docker compose down -v-```--- [ ] **Step 6: Push branch and create PR**--```bash-git push -u origin feature/integration-tests-gh pr create --title "feat: add integration test setup for API" --body "Adds E2E integration tests that validate the full SMS send/receive flow using Docker and a phone emulator."-```diff --git a/docs/superpowers/plans/2026-05-03-scheduling-send-refactor.md b/docs/superpowers/plans/2026-05-03-scheduling-send-refactor.mddeleted file mode 100644index f92b60ed..00000000--- a/docs/superpowers/plans/2026-05-03-scheduling-send-refactor.md+++ /dev/null@@ -1,956 +0,0 @@-# Scheduling Send Refactor Implementation Plan--> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.--**Goal:** Allow users to send SMS at an exact time (bypassing scheduling) when `SendAt` is specified, and replace the 1-second bulk hack with rate-based dispatch delays.--**Related docs:**--- [Scheduling SMS Messages](https://docs.httpsms.com/features/scheduling-sms-messages) — the existing `SendAt`/`SendTime` feature-- [Control SMS Send Rate](https://docs.httpsms.com/features/control-sms-send-rate) — the existing `MessagesPerMinute` rate-limiting feature--**Architecture:** Add a transient `ExactSendTime` flag flowing through the event system. When true, bypass [rate-limit](https://docs.httpsms.com/features/control-sms-send-rate) and schedule window logic in notification scheduling. For bulk sends without explicit time, compute dispatch delay from `MessagesPerMinute` per-phone instead of hardcoded 1s.--**Tech Stack:** Go, Fiber, GORM, CockroachDB, Google Cloud Tasks (CloudEvents)--**Spec:** `docs/superpowers/specs/2026-05-03-scheduling-send-refactor-design.md`--**Build/Test commands:**--```bash-cd api && go build ./...-cd api && go test -vet=off ./...-```-------## Task 1: Add ExactSendTime to Event Payload--**Files:**--- Modify: `api/pkg/events/message_api_sent_event.go`--- [ ] **Step 1: Add `ExactSendTime` field to `MessageAPISentPayload`**--In `api/pkg/events/message_api_sent_event.go`, add to the struct:--```go-ExactSendTime bool `json:"exact_send_time"`-```--Add it after line 22 (`ScheduledSendTime *time.Time`).--- [ ] **Step 2: Build to verify no compile errors**--Run: `cd api && go build ./...`-Expected: success--- [ ] **Step 3: Commit**--```bash-cd api && git add -A && git commit -m "feat(events): add ExactSendTime field to MessageAPISentPayload"-```-------## Task 2: Add Index and ExactSendTime to MessageSendParams + Update getSendDelay--**Files:**--- Modify: `api/pkg/services/message_service.go`--- [ ] **Step 1: Add `Index` field to `MessageSendParams`**--In `api/pkg/services/message_service.go` at line ~453, add `Index int` to the struct:--```go-type MessageSendParams struct {- Owner *phonenumbers.PhoneNumber- Contact string- Encrypted bool- Content string- Attachments []string- Source string- SendAt *time.Time- RequestID *string- UserID entities.UserID- RequestReceivedAt time.Time- Index int-}-```--- [ ] **Step 2: Update `phoneSettings` to also return `MessagesPerMinute`**--Change the `phoneSettings` method signature and body at line ~1014:--```go-func (service *MessageService) phoneSettings(ctx context.Context, userID entities.UserID, owner string) (uint, entities.SIM, uint) {- ctx, span := service.tracer.Start(ctx)- defer span.End()-- ctxLogger := service.tracer.CtxLogger(service.logger, span)-- phone, err := service.phoneService.Load(ctx, userID, owner)- if err != nil {- msg := fmt.Sprintf("cannot load phone for userID [%s] and owner [%s]. using default max send attempt of 2", userID, owner)- ctxLogger.Error(stacktrace.Propagate(err, msg))- return 2, entities.SIM1, 0- }-- return phone.MaxSendAttemptsSanitized(), phone.SIM, phone.MessagesPerMinute-}-```--- [ ] **Step 3: Update `SendMessage` to use new `phoneSettings` return value and set `ExactSendTime`**--Update `SendMessage` at line ~467. Key changes: get `messagesPerMinute` from `phoneSettings`, derive `ExactSendTime` from `SendAt != nil`, pass `messagesPerMinute` to `getSendDelay`:--```go-func (service *MessageService) SendMessage(ctx context.Context, params MessageSendParams) (*entities.Message, error) {- ctx, span := service.tracer.Start(ctx)- defer span.End()-- ctxLogger := service.tracer.CtxLogger(service.logger, span)-- sendAttempts, sim, messagesPerMinute := service.phoneSettings(ctx, params.UserID, phonenumbers.Format(params.Owner, phonenumbers.E164))-- eventPayload := events.MessageAPISentPayload{- MessageID: uuid.New(),- UserID: params.UserID,- Encrypted: params.Encrypted,- MaxSendAttempts: sendAttempts,- RequestID: params.RequestID,- Owner: phonenumbers.Format(params.Owner, phonenumbers.E164),- Contact: params.Contact,- RequestReceivedAt: params.RequestReceivedAt,- Content: params.Content,- Attachments: params.Attachments,- ScheduledSendTime: params.SendAt,- ExactSendTime: params.SendAt != nil,- SIM: sim,- }-- event, err := service.createMessageAPISentEvent(params.Source, eventPayload)- if err != nil {- msg := fmt.Sprintf("cannot create %T from payload with message id [%s]", event, eventPayload.MessageID)- return nil, service.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, msg))- }- ctxLogger.Info(fmt.Sprintf("created event [%s] with id [%s] and message id [%s] and user [%s]", event.Type(), event.ID(), eventPayload.MessageID, eventPayload.UserID))-- message, err := service.storeSentMessage(ctx, eventPayload)- if err != nil {- msg := fmt.Sprintf("cannot store message with id [%s]", eventPayload.MessageID)- return nil, service.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, msg))- }-- timeout := service.getSendDelay(ctxLogger, eventPayload, params, messagesPerMinute)- if _, err = service.eventDispatcher.DispatchWithTimeout(ctx, event, timeout); err != nil {- msg := fmt.Sprintf("cannot dispatch event type [%s] and id [%s]", event.Type(), event.ID())- return nil, service.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, msg))- }-- ctxLogger.Info(fmt.Sprintf("[%s] event with ID [%s] dispatched succesfully for message [%s] with user [%s] and delay [%s]", event.Type(), event.ID(), eventPayload.MessageID, eventPayload.UserID, timeout))- return message, err-}-```--- [ ] **Step 4: Rewrite `getSendDelay` to handle rate-based delay**--Replace the existing `getSendDelay` method. New signature takes `messagesPerMinute` as a separate arg:--```go-func (service *MessageService) getSendDelay(ctxLogger telemetry.Logger, eventPayload events.MessageAPISentPayload, params MessageSendParams, messagesPerMinute uint) time.Duration {- // Exact send time: delay until that time (clamped to 0 if in the past)- if params.SendAt != nil {- delay := params.SendAt.Sub(time.Now().UTC())- if delay < 0 {- ctxLogger.Info(fmt.Sprintf("message [%s] has send time [%s] in the past. sending immediately", eventPayload.MessageID, params.SendAt.String()))- return time.Duration(0)- }- return delay- }-- // Rate-based delay for bulk messages (Index > 0)- if params.Index > 0 && messagesPerMinute > 0 {- interval := time.Minute / time.Duration(messagesPerMinute)- delay := time.Duration(params.Index) * interval- ctxLogger.Info(fmt.Sprintf("message [%s] bulk index [%d] rate-based delay [%s]", eventPayload.MessageID, params.Index, delay))- return delay- }-- return time.Duration(0)-}-```--- [ ] **Step 5: Build to verify no compile errors**--Run: `cd api && go build ./...`-Expected: success--- [ ] **Step 6: Run tests**--Run: `cd api && go test -vet=off ./...`-Expected: all pass--- [ ] **Step 7: Commit**--```bash-cd api && git add -A && git commit -m "feat(services): add rate-based dispatch delay and ExactSendTime to SendMessage"-```-------## Task 3: Add ScheduleExact to Repository Interface and Implementation--**Files:**--- Modify: `api/pkg/repositories/phone_notification_repository.go`-- Modify: `api/pkg/repositories/gorm_phone_notification_repository.go`--- [ ] **Step 1: Add `ScheduleExact` to the repository interface**--In `api/pkg/repositories/phone_notification_repository.go`:--```go-// PhoneNotificationRepository loads and persists an entities.PhoneNotification-type PhoneNotificationRepository interface {- // Schedule a new entities.PhoneNotification- Schedule(ctx context.Context, messagesPerMinute uint, schedule *entities.MessageSendSchedule, notification *entities.PhoneNotification) error-- // ScheduleExact stores a phone notification with a fixed ScheduledAt time,- // bypassing rate-limit and schedule window logic.- ScheduleExact(ctx context.Context, notification *entities.PhoneNotification) error-- // UpdateStatus of a notification- UpdateStatus(ctx context.Context, notificationID uuid.UUID, status entities.PhoneNotificationStatus) error-- // DeleteAllForUser deletes all entities.PhoneNotification for a user- DeleteAllForUser(ctx context.Context, userID entities.UserID) error-}-```--- [ ] **Step 2: Implement `ScheduleExact` on `gormPhoneNotificationRepository`**--In `api/pkg/repositories/gorm_phone_notification_repository.go`, add after the `Schedule` method:--```go-// ScheduleExact stores a phone notification with an exact ScheduledAt time.-// It performs a dedupe check — if a pending notification for the same message already exists, it's a no-op.-func (repository *gormPhoneNotificationRepository) ScheduleExact(- ctx context.Context,- notification *entities.PhoneNotification,-) error {- ctx, span := repository.tracer.Start(ctx)- defer span.End()-- // Dedupe: check if a pending notification for this message already exists- var count int64- if err := repository.db.WithContext(ctx).- Model(&entities.PhoneNotification{}).- Where("message_id = ? AND status = ?", notification.MessageID, entities.PhoneNotificationStatusPending).- Count(&count).Error; err != nil {- return repository.tracer.WrapErrorSpan(- span,- stacktrace.Propagate(err, "cannot check for existing notification for message [%s]", notification.MessageID),- )- }-- if count > 0 {- return nil- }-- if err := repository.db.WithContext(ctx).Create(notification).Error; err != nil {- return repository.tracer.WrapErrorSpan(- span,- stacktrace.Propagate(err, "cannot create exact-time notification with id [%s]", notification.ID),- )- }-- return nil-}-```--- [ ] **Step 3: Build to verify no compile errors**--Run: `cd api && go build ./...`-Expected: success--- [ ] **Step 4: Commit**--```bash-cd api && git add -A && git commit -m "feat(repositories): add ScheduleExact method for exact-time notifications"-```-------## Task 4: Update PhoneNotificationService to Support ExactSendTime--**Files:**--- Modify: `api/pkg/services/phone_notification_service.go`--- [ ] **Step 1: Add fields to `PhoneNotificationScheduleParams`**--Update the struct at line ~162:--```go-// PhoneNotificationScheduleParams are parameters for sending a notification-type PhoneNotificationScheduleParams struct {- UserID entities.UserID- Owner string- Source string- Encrypted bool- Contact string- Content string- SIM entities.SIM- MessageID uuid.UUID- ExactSendTime bool- ScheduledSendTime *time.Time-}-```--- [ ] **Step 2: Add bypass logic at the start of `Schedule` method**--Update `Schedule` method at line ~175. Add the bypass path after loading the phone:--```go-// Schedule a notification to be sent to a phone-func (service *PhoneNotificationService) Schedule(ctx context.Context, params *PhoneNotificationScheduleParams) error {- ctx, span := service.tracer.Start(ctx)- defer span.End()-- ctxLogger := service.tracer.CtxLogger(service.logger, span)-- phone, err := service.phoneRepository.Load(ctx, params.UserID, params.Owner)- if err != nil {- msg := fmt.Sprintf("cannot load phone with userID [%s] and phone [%s]", params.UserID, params.Owner)- return service.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, msg))- }-- notification := &entities.PhoneNotification{- ID: uuid.New(),- MessageID: params.MessageID,- UserID: params.UserID,- PhoneID: phone.ID,- Status: entities.PhoneNotificationStatusPending,- ScheduledAt: time.Now().UTC(),- CreatedAt: time.Now().UTC(),- UpdatedAt: time.Now().UTC(),- }-- // Bypass rate-limit and schedule window logic for exact send time- if params.ExactSendTime && params.ScheduledSendTime != nil {- scheduledAt := *params.ScheduledSendTime- // Clamp past times to now (send immediately)- if scheduledAt.Before(time.Now().UTC()) {- scheduledAt = time.Now().UTC()- }- notification.ScheduledAt = scheduledAt- if err = service.phoneNotificationRepository.ScheduleExact(ctx, notification); err != nil {- msg := fmt.Sprintf("cannot schedule exact notification for message [%s] to phone [%s]", params.MessageID, phone.ID)- return service.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, msg))- }-- if err = service.dispatchMessageNotificationScheduled(ctx, params, notification); err != nil {- ctxLogger.Error(err)- }-- if err = service.dispatchMessageNotificationSend(ctx, params.Source, notification); err != nil {- return service.tracer.WrapErrorSpan(span, err)- }-- ctxLogger.Info(fmt.Sprintf(- "message with id [%s] exact notification scheduled for [%s] with id [%s]",- params.MessageID,- notification.ScheduledAt,- notification.ID,- ))- return nil- }-- // Standard path: apply rate-limit + schedule window logic- var schedule *entities.MessageSendSchedule- if phone.ScheduleID != nil {- schedule, err = service.sendScheduleRepository.Load(ctx, params.UserID, *phone.ScheduleID)- if stacktrace.GetCode(err) == repositories.ErrCodeNotFound {- schedule = nil- err = nil- }- if err != nil {- msg := fmt.Sprintf("cannot load send schedule [%s] for phone [%s]", *phone.ScheduleID, phone.ID)- return service.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, msg))- }- }-- if err = service.phoneNotificationRepository.Schedule(ctx, phone.MessagesPerMinute, schedule, notification); err != nil {- msg := fmt.Sprintf("cannot schedule notification for message [%s] to phone [%s]", params.MessageID, phone.ID)- return service.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, msg))- }-- if err = service.dispatchMessageNotificationScheduled(ctx, params, notification); err != nil {- ctxLogger.Error(err)- }-- if err = service.dispatchMessageNotificationSend(ctx, params.Source, notification); err != nil {- return service.tracer.WrapErrorSpan(span, err)- }-- ctxLogger.Info(fmt.Sprintf(- "message with id [%s] notification scheduled for [%s] with id [%s]",- params.MessageID,- notification.ScheduledAt,- notification.ID,- ))- return nil-}-```--- [ ] **Step 3: Build to verify no compile errors**--Run: `cd api && go build ./...`-Expected: success--- [ ] **Step 4: Commit**--```bash-cd api && git add -A && git commit -m "feat(services): add ExactSendTime bypass in PhoneNotificationService.Schedule"-```-------## Task 5: Update Phone Notification Listener to Pass ExactSendTime--**Files:**--- Modify: `api/pkg/listeners/phone_notification_listener.go`--- [ ] **Step 1: Pass ExactSendTime and ScheduledSendTime from event payload to service params**--Update the `onMessageAPISent` method at line ~44:--```go-func (listener *PhoneNotificationListener) onMessageAPISent(ctx context.Context, event cloudevents.Event) error {- ctx, span := listener.tracer.Start(ctx)- defer span.End()-- var payload events.MessageAPISentPayload- if err := event.DataAs(&payload); err != nil {- msg := fmt.Sprintf("cannot decode [%s] into [%T]", event.Data(), payload)- return listener.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, msg))- }-- sendParams := &services.PhoneNotificationScheduleParams{- UserID: payload.UserID,- Owner: payload.Owner,- Contact: payload.Contact,- Content: payload.Content,- SIM: payload.SIM,- Encrypted: payload.Encrypted,- Source: event.Source(),- MessageID: payload.MessageID,- ExactSendTime: payload.ExactSendTime,- ScheduledSendTime: payload.ScheduledSendTime,- }-- if err := listener.service.Schedule(ctx, sendParams); err != nil {- msg := fmt.Sprintf("cannot send notification with params [%s] for event with ID [%s]", spew.Sdump(sendParams), event.ID())- return listener.tracer.WrapErrorSpan(span, stacktrace.Propagate(err, msg))- }-- return nil-}-```--- [ ] **Step 2: Build to verify no compile errors**--Run: `cd api && go build ./...`-Expected: success--- [ ] **Step 3: Commit**--```bash-cd api && git add -A && git commit -m "feat(listeners): pass ExactSendTime to PhoneNotificationService from event"-```-------## Task 6: Update Bulk Send Request + Handler--**Files:**--- Modify: `api/pkg/requests/message_bulk_send_request.go`-- Modify: `api/pkg/handlers/message_handler.go`--- [ ] **Step 1: Remove per-index SendAt from `MessageBulkSend.ToMessageSendParams()`**--In `api/pkg/requests/message_bulk_send_request.go`, update `ToMessageSendParams`:--```go-// ToMessageSendParams converts MessageSend to services.MessageSendParams-func (input *MessageBulkSend) ToMessageSendParams(userID entities.UserID, source string) []services.MessageSendParams {- from, _ := phonenumbers.Parse(input.From, phonenumbers.UNKNOWN_REGION)-- var result []services.MessageSendParams- for index, to := range input.To {- result = append(result, services.MessageSendParams{- Source: source,- Owner: from,- Encrypted: input.Encrypted,- RequestID: input.sanitizeStringPointer(input.RequestID),- UserID: userID,- RequestReceivedAt: time.Now().UTC(),- Contact: to,- Content: input.Content,- Attachments: input.Attachments,- Index: index,- })- }-- return result-}-```--Key changes: removed `SendAt` assignment and added `Index: index`.--- [ ] **Step 2: Remove the `index * 1s` hack from `BulkSend` handler**--In `api/pkg/handlers/message_handler.go`, update the `BulkSend` handler goroutine (around line 160-175). Remove the `if message.SendAt == nil` block:--Replace:--```go-for index, message := range params {- wg.Add(1)- go func(message services.MessageSendParams, index int) {- count.Add(1)- if message.SendAt == nil {- sentAt := time.Now().UTC().Add(time.Duration(index) * time.Second)- message.SendAt = &sentAt- }-- response, err := h.service.SendMessage(ctx, message)-```--With:--```go-for index, message := range params {- wg.Add(1)- go func(message services.MessageSendParams, index int) {- count.Add(1)- response, err := h.service.SendMessage(ctx, message)-```--- [ ] **Step 3: Remove unused `time` import if needed**--Check if `time` is still used in `message_handler.go`. It likely is (used elsewhere), so skip this step if so.--- [ ] **Step 4: Build to verify no compile errors**--Run: `cd api && go build ./...`-Expected: success--- [ ] **Step 5: Commit**--```bash-cd api && git add -A && git commit -m "feat(handlers): replace 1s hack with rate-based delay for bulk send"-```-------## Task 7: Update CSV Bulk Message Request + Handler--**Files:**--- Modify: `api/pkg/requests/bulk_message_request.go`-- Modify: `api/pkg/handlers/bulk_message_handler.go`--- [ ] **Step 1: Add `Index` parameter to `BulkMessage.ToMessageSendParams()`**--In `api/pkg/requests/bulk_message_request.go`, change the method signature to accept index:--```go-// ToMessageSendParams converts BulkMessage to services.MessageSendParams-func (input *BulkMessage) ToMessageSendParams(userID entities.UserID, requestID uuid.UUID, source string, index int) services.MessageSendParams {- from, _ := phonenumbers.Parse(input.FromPhoneNumber, phonenumbers.UNKNOWN_REGION)-- return services.MessageSendParams{- Source: source,- Owner: from,- RequestID: input.sanitizeStringPointer(fmt.Sprintf("bulk-%s", requestID.String())),- UserID: userID,- SendAt: input.SendTime,- RequestReceivedAt: time.Now().UTC(),- Contact: input.sanitizeAddress(input.ToPhoneNumber),- Content: input.Content,- Attachments: input.removeEmptyStrings(strings.Split(input.AttachmentURLs, ",")),- Index: index,- }-}-```--- [ ] **Step 2: Update `BulkMessageHandler.Store()` to compute per-phone index**--In `api/pkg/handlers/bulk_message_handler.go`, update the Store method to compute per-phone indices:--```go-func (h *BulkMessageHandler) Store(c *fiber.Ctx) error {- ctx, span, ctxLogger := h.tracer.StartFromFiberCtxWithLogger(c, h.logger)- defer span.End()-- file, err := c.FormFile("document")- if err != nil {- msg := fmt.Sprintf("cannot fetch file with name [%s] from request", "document")- ctxLogger.Warn(stacktrace.Propagate(err, msg))- return h.responseBadRequest(c, err)- }-- messages, validationErrors := h.validator.ValidateStore(ctx, h.userIDFomContext(c), file)- if len(validationErrors) != 0 {- msg := fmt.Sprintf("validation errors [%s], while sending bulk sms from CSV file [%s] for [%s]", spew.Sdump(validationErrors), file.Filename, h.userIDFomContext(c))- ctxLogger.Warn(stacktrace.NewError(msg))- return h.responseUnprocessableEntity(c, validationErrors, "validation errors while sending bulk SMS")- }-- if msg := h.billingService.IsEntitledWithCount(ctx, h.userIDFomContext(c), uint(len(messages))); msg != nil {- ctxLogger.Warn(stacktrace.NewError(fmt.Sprintf("user with ID [%s] is not entitled to send [%d] messages", h.userIDFomContext(c), len(messages))))- return h.responsePaymentRequired(c, *msg)- }-- requestID := uuid.New()- wg := sync.WaitGroup{}- count := atomic.Int64{}-- // Compute per-phone index for rate-based dispatch delay- phoneIndexMap := make(map[string]int)- for _, message := range messages {- if message.SendTime != nil {- continue // Exact-time messages don't need indexing- }- phone := message.FromPhoneNumber- phoneIndexMap[phone]++ // Pre-count not needed, we'll compute inline- }-- // Reset for actual iteration- phoneIndexCounter := make(map[string]int)-- for _, message := range messages {- wg.Add(1)- var perPhoneIndex int- if message.SendTime == nil {- perPhoneIndex = phoneIndexCounter[message.FromPhoneNumber]- phoneIndexCounter[message.FromPhoneNumber]++- }-- go func(message *requests.BulkMessage, index int) {- count.Add(1)- _, err = h.messageService.SendMessage(- ctx,- message.ToMessageSendParams(h.userIDFomContext(c), requestID, c.OriginalURL(), index),- )- if err != nil {- count.Add(-1)- msg := fmt.Sprintf("cannot send message with paylod [%s] at index [%d]", spew.Sdump(message), index)- ctxLogger.Error(stacktrace.Propagate(err, msg))- }- wg.Done()- }(message, perPhoneIndex)- }-- wg.Wait()- return h.responseAccepted(c, fmt.Sprintf("Added %d out of %d messages to the queue", count.Load(), len(messages)))-}-```--- [ ] **Step 3: Clean up unused `phoneIndexMap` variable**--The `phoneIndexMap` is computed but unused. Remove it — we only need `phoneIndexCounter`:--```go-// Compute per-phone index for rate-based dispatch delay-phoneIndexCounter := make(map[string]int)--for _, message := range messages {- wg.Add(1)- var perPhoneIndex int- if message.SendTime == nil {- perPhoneIndex = phoneIndexCounter[message.FromPhoneNumber]- phoneIndexCounter[message.FromPhoneNumber]++- }-- go func(message *requests.BulkMessage, index int) {- // ... same as above- }(message, perPhoneIndex)-}-```--- [ ] **Step 4: Build to verify no compile errors**--Run: `cd api && go build ./...`-Expected: success--- [ ] **Step 5: Run tests**--Run: `cd api && go test -vet=off ./...`-Expected: all pass--- [ ] **Step 6: Commit**--```bash-cd api && git add -A && git commit -m "feat(handlers): add per-phone index for CSV bulk messages"-```-------## Task 8: Add Unit Tests for getSendDelay--**Files:**--- Create: `api/pkg/services/message_service_test.go`--- [ ] **Step 1: Write tests for the new `getSendDelay` logic**--Create `api/pkg/services/message_service_test.go`:--```go-package services--import (- "testing"- "time"-- "github.com/NdoleStudio/httpsms/pkg/events"- "github.com/NdoleStudio/httpsms/pkg/telemetry"- "github.com/google/uuid"- "github.com/stretchr/testify/assert"- "go.opentelemetry.io/otel/trace"-)--func TestGetSendDelay_WithSendAt_ReturnsTimeUntil(t *testing.T) {- service := &MessageService{}- logger := &noopLogger{}-- sendAt := time.Now().UTC().Add(5 * time.Minute)- params := MessageSendParams{SendAt: &sendAt}- payload := events.MessageAPISentPayload{MessageID: uuid.New()}-- delay := service.getSendDelay(logger, payload, params, 10)-- // Should be approximately 5 minutes (within 2 seconds tolerance)- assert.InDelta(t, float64(5*time.Minute), float64(delay), float64(2*time.Second))-}--func TestGetSendDelay_WithSendAtInPast_ReturnsZero(t *testing.T) {- service := &MessageService{}- logger := &noopLogger{}-- sendAt := time.Now().UTC().Add(-5 * time.Minute)- params := MessageSendParams{SendAt: &sendAt}- payload := events.MessageAPISentPayload{MessageID: uuid.New()}-- delay := service.getSendDelay(logger, payload, params, 10)-- assert.Equal(t, time.Duration(0), delay)-}--func TestGetSendDelay_BulkIndex_RateBasedDelay(t *testing.T) {- service := &MessageService{}- logger := &noopLogger{}-- params := MessageSendParams{Index: 3}- payload := events.MessageAPISentPayload{MessageID: uuid.New()}-- // 10 messages per minute = 6 seconds interval- delay := service.getSendDelay(logger, payload, params, 10)-- expected := time.Duration(3) * (time.Minute / time.Duration(10))- assert.Equal(t, expected, delay)-}--func TestGetSendDelay_BulkIndex_ZeroRate_ReturnsZero(t *testing.T) {- service := &MessageService{}- logger := &noopLogger{}-- params := MessageSendParams{Index: 5}- payload := events.MessageAPISentPayload{MessageID: uuid.New()}-- delay := service.getSendDelay(logger, payload, params, 0)-- assert.Equal(t, time.Duration(0), delay)-}--func TestGetSendDelay_IndexZero_ReturnsZero(t *testing.T) {- service := &MessageService{}- logger := &noopLogger{}-- params := MessageSendParams{Index: 0}- payload := events.MessageAPISentPayload{MessageID: uuid.New()}-- delay := service.getSendDelay(logger, payload, params, 10)-- assert.Equal(t, time.Duration(0), delay)-}--func TestGetSendDelay_NoSendAtNoIndex_ReturnsZero(t *testing.T) {- service := &MessageService{}- logger := &noopLogger{}-- params := MessageSendParams{}- payload := events.MessageAPISentPayload{MessageID: uuid.New()}-- delay := service.getSendDelay(logger, payload, params, 10)-- assert.Equal(t, time.Duration(0), delay)-}--// noopLogger implements telemetry.Logger for testing-type noopLogger struct{}--var _ telemetry.Logger = (*noopLogger)(nil)--func (l *noopLogger) Error(_ error) {}-func (l *noopLogger) WithService(_ string) telemetry.Logger { return l }-func (l *noopLogger) WithString(_, _ string) telemetry.Logger { return l }-func (l *noopLogger) WithSpan(_ trace.SpanContext) telemetry.Logger { return l }-func (l *noopLogger) Trace(_ string) {}-func (l *noopLogger) Info(_ string) {}-func (l *noopLogger) Warn(_ error) {}-func (l *noopLogger) Debug(_ string) {}-func (l *noopLogger) Fatal(_ error) {}-func (l *noopLogger) Printf(_ string, _ ...interface{}) {}-```--- [ ] **Step 2: Run the tests**--Run: `cd api && go test -vet=off ./pkg/services/ -run TestGetSendDelay -v`-Expected: all pass--- [ ] **Step 3: Commit**--```bash-cd api && git add -A && git commit -m "test(services): add unit tests for getSendDelay rate-based logic"-```-------## Task 9: Add Unit Test for ResolveScheduledAt (Existing, Verify No Regression)--**Files:**--- Create: `api/pkg/entities/send_schedule_test.go`--- [ ] **Step 1: Write tests to lock existing ResolveScheduledAt behavior**--Create `api/pkg/entities/send_schedule_test.go`:--```go-package entities--import (- "testing"- "time"-- "github.com/stretchr/testify/assert"-)--func TestResolveScheduledAt_NilSchedule_ReturnsCurrentUTC(t *testing.T) {- now := time.Now()- var schedule *MessageSendSchedule- result := schedule.ResolveScheduledAt(now)- assert.Equal(t, now.UTC(), result)-}--func TestResolveScheduledAt_InactiveSchedule_ReturnsCurrentUTC(t *testing.T) {- now := time.Now()- schedule := &MessageSendSchedule{IsActive: false}- result := schedule.ResolveScheduledAt(now)- assert.Equal(t, now.UTC(), result)-}--func TestResolveScheduledAt_NoWindows_ReturnsCurrentUTC(t *testing.T) {- now := time.Now()- schedule := &MessageSendSchedule{- IsActive: true,- Timezone: "UTC",- Windows: []MessageSendScheduleWindow{},- }- result := schedule.ResolveScheduledAt(now)- assert.Equal(t, now.UTC(), result)-}--func TestResolveScheduledAt_WithinWindow_ReturnsCurrentUTC(t *testing.T) {- // Wednesday at 10:00 UTC, window is Wed 9:00-17:00 (540-1020 minutes)- now := time.Date(2025, 1, 1, 10, 0, 0, 0, time.UTC) // Wednesday- schedule := &MessageSendSchedule{- IsActive: true,- Timezone: "UTC",- Windows: []MessageSendScheduleWindow{- {DayOfWeek: int(now.Weekday()), StartMinute: 540, EndMinute: 1020},- },- }- result := schedule.ResolveScheduledAt(now)- assert.Equal(t, now.UTC(), result)-}--func TestResolveScheduledAt_BeforeWindow_ReturnsWindowStart(t *testing.T) {- // Wednesday at 7:00 UTC, window is Wed 9:00-17:00- now := time.Date(2025, 1, 1, 7, 0, 0, 0, time.UTC) // Wednesday- schedule := &MessageSendSchedule{- IsActive: true,- Timezone: "UTC",- Windows: []MessageSendScheduleWindow{- {DayOfWeek: int(now.Weekday()), StartMinute: 540, EndMinute: 1020},- },- }- result := schedule.ResolveScheduledAt(now)- expected := time.Date(2025, 1, 1, 9, 0, 0, 0, time.UTC)- assert.Equal(t, expected, result)-}-```--- [ ] **Step 2: Run the tests**--Run: `cd api && go test -vet=off ./pkg/entities/ -run TestResolveScheduledAt -v`-Expected: all pass--- [ ] **Step 3: Commit**--```bash-cd api && git add -A && git commit -m "test(entities): add regression tests for ResolveScheduledAt"-```-------## Task 10: Final Build + Integration Verification--**Files:** None (verification only)--- [ ] **Step 1: Full build**--Run: `cd api && go build ./...`-Expected: success--- [ ] **Step 2: Full test suite**--Run: `cd api && go test -vet=off ./...`-Expected: all pass--- [ ] **Step 3: Generate Swagger docs (if API annotations changed)**--The API request structs' annotations haven't changed for swagger (no new endpoints, `SendAt` already documented). Skip swagger regen unless compile errors appear.--- [ ] **Step 4: Verify git status is clean**--Run: `cd api && git status`-Expected: clean working tree-------## Notes--- The `noopLogger` in tests implements the full `telemetry.Logger` interface (Error, WithService, WithString, WithSpan, Trace, Info, Warn, Debug, Fatal, Printf).-- The `ExactSendTime` field is transient — no database migrations needed.-- **Dedupe strategy**: `ScheduleExact` uses a `SELECT COUNT` check before insert. This is not fully race-proof but acceptable given: (a) Cloud Tasks at-least-once duplicates are rare, and (b) the existing `Schedule` path also has this same theoretical gap. Adding a DB unique constraint on `(message_id, status='pending')` would require a partial index migration — this is deferred as a future improvement if duplicates become a problem in practice.-- The existing `Schedule` method already handles concurrency via CockroachDB's serializable transactions (`crdbgorm.ExecuteTx`), which retries automatically on conflicts. No additional dedupe is added there.-- All existing behavior for single messages without `SendAt` is preserved (delay = 0, standard scheduling path).-- Past `SendAt` times are handled at both layers: `getSendDelay` returns 0 (immediate dispatch), and `Schedule` clamps `ScheduledAt` to `now` (no past timestamps persisted).diff --git a/docs/superpowers/plans/2026-05-15-hedging-repository.md b/docs/superpowers/plans/2026-05-15-hedging-repository.mddeleted file mode 100644index e3b838fa..00000000--- a/docs/superpowers/plans/2026-05-15-hedging-repository.md+++ /dev/null@@ -1,265 +0,0 @@-# Hedging Repository Implementation Plan--> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.--> **Status:** All tasks implemented. This plan was written retroactively to document and verify the implementation.--**Goal:** Create composite hedging repositories that dual-write to GORM (primary) and Turso (secondary) with fail-open semantics on secondary failures.--**Architecture:** Two new repository files implement the existing `HeartbeatRepository` and `HeartbeatMonitorRepository` interfaces by delegating reads to primary only and writes to both. Secondary write failures are logged and counted via an OTel metric but never propagated. Activated via `HEARTBEAT_DB_BACKEND=hedging`.--**Tech Stack:** Go, OpenTelemetry metrics (`go.opentelemetry.io/otel/metric`), existing repository interfaces--**Spec:** `docs/superpowers/specs/2026-05-15-hedging-repository-design.md`-------### Task 1: Create hedging heartbeat repository ✅--**Files:**--- Created: `api/pkg/repositories/hedging_heartbeat_repository.go`--- [ ] **Step 1: Create the hedging heartbeat repository file**--```go-package repositories--import (- "context"- "fmt"-- otelMetric "go.opentelemetry.io/otel/metric"-- "github.com/NdoleStudio/httpsms/pkg/entities"- "github.com/NdoleStudio/httpsms/pkg/telemetry"- "github.com/palantir/stacktrace"-)--type hedgingHeartbeatRepository struct {- logger telemetry.Logger- tracer telemetry.Tracer- primary HeartbeatRepository- secondary HeartbeatRepository- failureCounter otelMetric.Int64Counter-}--func NewHedgingHeartbeatRepository(- logger telemetry.Logger,- tracer telemetry.Tracer,- primary HeartbeatRepository,- secondary HeartbeatRepository,- failureCounter otelMetric.Int64Counter,-) HeartbeatRepository {- return &hedgingHeartbeatRepository{- logger: logger.WithService(fmt.Sprintf("%T", &hedgingHeartbeatRepository{})),- tracer: tracer,- primary: primary,- secondary: secondary,- failureCounter: failureCounter,- }-}-```--Implement 4 methods:--- `Store` — write to primary, then secondary (fail-open with log + metric)-- `Index` — delegate to primary only-- `Last` — delegate to primary only-- `DeleteAllForUser` — write to primary, then secondary (fail-open with log + metric)--Write methods follow this pattern:--```go-func (r *hedgingHeartbeatRepository) Store(ctx context.Context, heartbeat *entities.Heartbeat) error {- ctx, span := r.tracer.Start(ctx)- defer span.End()-- if err := r.primary.Store(ctx, heartbeat); err != nil {- return err- }-- if err := r.secondary.Store(ctx, heartbeat); err != nil {- r.logger.Error(stacktrace.Propagate(err, fmt.Sprintf("hedging: secondary write failed for heartbeat [%s]", heartbeat.ID)))- r.failureCounter.Add(ctx, 1)- }-- return nil-}-```--Read methods simply delegate:--```go-func (r *hedgingHeartbeatRepository) Index(ctx context.Context, userID entities.UserID, owner string, params IndexParams) (*[]entities.Heartbeat, error) {- return r.primary.Index(ctx, userID, owner, params)-}-```--- [ ] **Step 2: Verify build**--Run: `cd api && go build ./...`-Expected: exit code 0--- [ ] **Step 3: Commit**--```bash-git add api/pkg/repositories/hedging_heartbeat_repository.go-git commit -m "feat(api): add hedging heartbeat repository"-```-------### Task 2: Create hedging heartbeat monitor repository ✅--**Files:**--- Created: `api/pkg/repositories/hedging_heartbeat_monitor_repository.go`--- [ ] **Step 1: Create the hedging heartbeat monitor repository file**--Same struct pattern as Task 1 but wrapping `HeartbeatMonitorRepository` interface.--Implement 7 methods:--| Method | Behavior |-| ------------------- | ---------------------- |-| `Store` | Write both (fail-open) |-| `Load` | Primary only |-| `Exists` | Primary only |-| `UpdateQueueID` | Write both (fail-open) |-| `Delete` | Write both (fail-open) |-| `UpdatePhoneOnline` | Write both (fail-open) |-| `DeleteAllForUser` | Write both (fail-open) |--All write methods follow the same fail-open pattern: primary must succeed, secondary logs + increments counter on failure.--- [ ] **Step 2: Verify build**--Run: `cd api && go build ./...`-Expected: exit code 0--- [ ] **Step 3: Commit**--```bash-git add api/pkg/repositories/hedging_heartbeat_monitor_repository.go-git commit -m "feat(api): add hedging heartbeat monitor repository"-```-------### Task 3: Add HedgingFailureCounter to DI container ✅--**Files:**--- Modified: `api/pkg/di/container.go` (added `HedgingFailureCounter()` method after `TursoDB()`, ~line 320)--- [ ] **Step 1: Add the HedgingFailureCounter method**--```go-func (container *Container) HedgingFailureCounter() otelMetric.Int64Counter {- meter := otel.GetMeterProvider().Meter(- container.projectID,- otelMetric.WithInstrumentationVersion(otel.Version()),- )- counter, err := meter.Int64Counter(- "hedging.secondary.write.failures",- otelMetric.WithUnit("1"),- otelMetric.WithDescription("Number of failed secondary writes in hedging repositories"),- )- if err != nil {- container.logger.Fatal(stacktrace.Propagate(err, "cannot create hedging failure counter"))- }- return counter-}-```--- [ ] **Step 2: Verify build**--Run: `cd api && go build ./...`-Expected: exit code 0-------### Task 4: Wire hedging mode in DI container ✅--**Files:**--- Modified: `api/pkg/di/container.go`-- - `HeartbeatRepository()` (~line 1768)- - `HeartbeatMonitorRepository()` (~line 930)--- [ ] **Step 1: Change both methods from if/else to switch**--Replace the existing `if os.Getenv("HEARTBEAT_DB_BACKEND") == "turso"` with a switch:--```go-func (container *Container) HeartbeatRepository() repositories.HeartbeatRepository {- switch os.Getenv("HEARTBEAT_DB_BACKEND") {- case "turso":- // existing libSQL path- case "hedging":- return repositories.NewHedgingHeartbeatRepository(- container.Logger(),- container.Tracer(),- repositories.NewGormHeartbeatRepository(container.Logger(), container.Tracer(), container.DedicatedDB()),- repositories.NewLibsqlHeartbeatRepository(container.Logger(), container.Tracer(), container.TursoDB()),- container.HedgingFailureCounter(),- )- default:- // existing GORM path- }-}-```--Same pattern for `HeartbeatMonitorRepository()`.--- [ ] **Step 2: Verify build**--Run: `cd api && go build ./...`-Expected: exit code 0--- [ ] **Step 3: Run pre-commit hooks**--Run: `cd api && gofumpt -w pkg/repositories/hedging_heartbeat_repository.go pkg/repositories/hedging_heartbeat_monitor_repository.go pkg/di/container.go`-Expected: exit code 0--- [ ] **Step 4: Final commit**--```bash-git add api/pkg/di/container.go-git commit -m "feat(api): wire hedging mode in DI container (HEARTBEAT_DB_BACKEND=hedging)"-```-------### Task 5: Final verification--- [ ] **Step 1: Full build**--Run: `cd api && go build ./...`-Expected: exit code 0--- [ ] **Step 2: Run tests**--Run: `cd api && go test ./...`-Expected: all tests pass--- [ ] **Step 3: go vet (no new warnings)**--Run: `cd api && go vet ./... 2>&1 | Select-String "hedging"`-Expected: only pre-existing `non-constant format string` warnings (same category as all other repos)--- [ ] **Step 4: Pre-commit hooks pass**--Run: `git add -A && git commit --dry-run`-Expected: all hooks pass (go-fumpt, go-lint, go-imports, go-mod-tidy)--- [ ] **Step 5: Verify three modes work (code review)**--Check that the DI container correctly handles all three values:--- Default (unset) → GORM only-- `turso` → libSQL only-- `hedging` → GORM primary + libSQL secondarydiff --git a/docs/superpowers/specs/2026-04-11-mms-attachments-design.md b/docs/superpowers/specs/2026-04-11-mms-attachments-design.mddeleted file mode 100644index 7beea85f..00000000--- a/docs/superpowers/specs/2026-04-11-mms-attachments-design.md+++ /dev/null@@ -1,220 +0,0 @@-# MMS Attachment Support — Design Spec--## Problem--The Android app now forwards MMS attachments (as base64-encoded data) when receiving MMS messages via `HttpSmsApiService.receive()`. The API server needs to:--1. Accept attachment data in the receive endpoint-2. Upload attachments to cloud storage (GCS or in-memory)-3. Store download URLs in the Message entity-4. Serve a download endpoint for retrieving attachments-5. Include attachment URLs in webhook event payloads--## Approach--**Approach A: Storage Interface + Minimal New Code** — Add an `AttachmentStorage` interface with GCS and memory implementations. Upload logic lives in the existing `MessageService.ReceiveMessage()` flow (synchronous). A new `AttachmentHandler` serves downloads. No new database tables — content type is encoded in the URL file extension.--## Design--### 1. Storage Interface--**New file: `pkg/repositories/attachment_storage.go`**--```go-type AttachmentStorage interface {- Upload(ctx context.Context, path string, data []byte) error- Download(ctx context.Context, path string) ([]byte, error)- Delete(ctx context.Context, path string) error-}-```--**GCS Implementation** (`pkg/repositories/gcs_attachment_storage.go`):--- Uses `cloud.google.com/go/storage` SDK-- Configured with bucket name from `GCS_BUCKET_NAME` env var-- Stores objects at: `attachments/{userID}/{messageID}/{index}/{name}.{ext}`-- Extension derived from content type (e.g., `image/jpeg` → `.jpg`); falls back to `.bin` only when no mapping exists--**Memory Implementation** (`pkg/repositories/memory_attachment_storage.go`):--- `sync.Map`-backed in-memory store-- Used when `GCS_BUCKET_NAME` is empty/unset (local dev, testing)--**DI selection** (in `container.go`):--```go-if os.Getenv("GCS_BUCKET_NAME") != "" {- return NewGCSAttachmentStorage(bucket, tracer, logger)-}-return NewMemoryAttachmentStorage(tracer, logger)-```--### 2. Environment Variables--| Variable | Description | Default |-| ----------------- | ------------------------------------------------------- | ---------------------------------- |-| `GCS_BUCKET_NAME` | GCS bucket for attachments. Empty = use memory storage. | `httpsms-86c51.appspot.com` (prod) |--The API base URL for constructing download links is derived from `EVENTS_QUEUE_ENDPOINT` by stripping the `/v1/events` suffix.--### 3. Request & Validation Changes--**Updated `MessageReceive` request** (`pkg/requests/`):--```go-type MessageReceive struct {- From string `json:"from"`- To string `json:"to"`- Content string `json:"content"`- Encrypted bool `json:"encrypted"`- SIM entities.SIM `json:"sim"`- Timestamp time.Time `json:"timestamp"`- Attachments []MessageAttachment `json:"attachments"` // NEW-}--type MessageAttachment struct {- Name string `json:"name"`- ContentType string `json:"content_type"`- Content string `json:"content"` // base64-encoded-}-```--**Updated `MessageReceiveParams`** (`pkg/services/`):-The `ToMessageReceiveParams()` method must propagate attachments to the service layer:--```go-type MessageReceiveParams struct {- // ... existing fields ...- Attachments []requests.MessageAttachment // NEW — raw attachment data for upload-}-```--**Filename sanitization:**-The `Name` field from the Android client must be sanitized to prevent path traversal attacks. Strip all path separators (`/`, `\`), directory traversal sequences (`..`), and non-printable characters. If the sanitized name is empty, use a fallback like `attachment-{index}`.--**Content type allowlist:**-Only allow known-safe MIME types from the extension mapping table (Section 5). Reject attachments with unrecognized content types with a 400 error.--**Validation rules** (in `pkg/validators/`):--- Attachment count must be ≤ 10-- Each decoded attachment must be ≤ 1.5 MB (1,572,864 bytes)-- Content type must be in the allowlist-- If any limit is exceeded → **reject entire request with 400 Bad Request**-- Validation happens before any upload or storage--### 4. Upload Flow (Synchronous in Receive)--In `MessageService.ReceiveMessage()`:--1. Validate attachment count, sizes, and content types-2. Upload attachments **in parallel** using `errgroup`:- a. Decode base64 content- b. Sanitize `name` (strip path separators, `..`, non-printable chars; fallback to `attachment-{index}`)- c. Map `content_type` → file extension (e.g., `image/jpeg` → `.jpg`, unknown → `.bin`)- d. Upload to storage at path: `attachments/{userID}/{messageID}/{index}/{sanitizedName}.{ext}`- e. Build download URL: `{apiBaseURL}/v1/attachments/{userID}/{messageID}/{index}/{sanitizedName}.{ext}`-3. If any upload fails → best-effort delete of already-uploaded files, then return 500-4. Collect download URLs into `message.Attachments` (existing `pq.StringArray` field)-5. Set `Attachments` on `MessagePhoneReceivedPayload` before dispatching event-6. `storeReceivedMessage()` copies `payload.Attachments` → `message.Attachments`-7. Store message in database-8. Fire `message.phone.received` event (includes attachment URLs)--### 5. Content Type → Extension Mapping--A utility function maps MIME types to file extensions:--| Content Type | Extension |-| ----------------- | --------- |-| `image/jpeg` | `.jpg` |-| `image/png` | `.png` |-| `image/gif` | `.gif` |-| `image/webp` | `.webp` |-| `image/bmp` | `.bmp` |-| `video/mp4` | `.mp4` |-| `video/3gpp` | `.3gp` |-| `audio/mpeg` | `.mp3` |-| `audio/ogg` | `.ogg` |-| `audio/amr` | `.amr` |-| `application/pdf` | `.pdf` |-| `text/vcard` | `.vcf` |-| `text/x-vcard` | `.vcf` |-| _(default)_ | `.bin` |--This covers common MMS content types. New mappings can be added as needed.--### 6. Download Handler--**New file: `pkg/handlers/attachment_handler.go`**--**Route:** `GET /v1/attachments/:userID/:messageID/:attachmentIndex/:filename`--- Registered **without authentication middleware** — publicly accessible, consistent with outgoing attachment URLs-- The `{userID}/{messageID}/{attachmentIndex}` path components provide sufficient obscurity (UUIDs are unguessable)--**Download flow:**--1. Parse URL params (userID, messageID, attachmentIndex, filename)-2. Construct storage path: `attachments/{userID}/{messageID}/{attachmentIndex}/{filename}`-3. Fetch bytes from `AttachmentStorage.Download(ctx, path)`-4. Derive `Content-Type` from filename extension-5. Set security headers: `Content-Disposition: attachment`, `X-Content-Type-Options: nosniff`-6. Respond with binary data + correct `Content-Type` header-7. Return 404 if attachment not found in storage--### 7. Webhook Event Changes--**Updated `MessagePhoneReceivedPayload`** (`pkg/events/message_phone_received_event.go`):--```go-type MessagePhoneReceivedPayload struct {- MessageID uuid.UUID `json:"message_id"`- UserID entities.UserID `json:"user_id"`- Owner string `json:"owner"`- Encrypted bool `json:"encrypted"`- Contact string `json:"contact"`- Timestamp time.Time `json:"timestamp"`- Content string `json:"content"`- SIM entities.SIM `json:"sim"`- Attachments []string `json:"attachments"` // NEW — download URLs-}-```--Webhook subscribers will receive the array of download URLs. They can `GET` each URL directly — no authentication required.--### 8. Files Changed / Created--**New files:**--- `pkg/repositories/attachment_storage.go` — Interface definition-- `pkg/repositories/gcs_attachment_storage.go` — GCS implementation-- `pkg/repositories/memory_attachment_storage.go` — Memory implementation-- `pkg/handlers/attachment_handler.go` — Download endpoint handler-- `pkg/validators/attachment_handler_validator.go` — Download param validation--**Modified files:**--- `pkg/requests/message_receive.go` (or wherever `MessageReceive` is defined) — Add `Attachments` field-- `pkg/validators/message_handler_validator.go` — Add attachment count/size validation-- `pkg/services/message_service.go` — Add upload logic to `ReceiveMessage()`-- `pkg/events/message_phone_received_event.go` — Add `Attachments` field to payload-- `pkg/di/container.go` — Wire storage, new handler, pass storage to message service-- `api/.env.docker` — Add `GCS_BUCKET_NAME` variable-- `go.mod` / `go.sum` — Add `cloud.google.com/go/storage` dependency--### 9. Validation Constraints--| Constraint | Value | Behavior |-| ------------------------------- | ------------------------ | ---------------------------------------------------- |-| Max attachment count | 10 | 400 Bad Request |-| Max attachment size (decoded) | 1.5 MB (1,572,864 bytes) | 400 Bad Request |-| Content type not in allowlist | — | 400 Bad Request |-| Missing/empty attachments array | — | Message stored without attachments (normal SMS flow) |--### 10. Error Handling--- Storage upload failure → Best-effort delete of already-uploaded attachments, then return 500; message is NOT stored-- Storage download failure → Return 404 or 500 depending on error type-- Invalid base64 content → Return 400 Bad Request-- All errors wrapped with `stacktrace.Propagate()` per project conventiondiff --git a/docs/superpowers/specs/2026-05-03-entitlement-service-design.md b/docs/superpowers/specs/2026-05-03-entitlement-service-design.mddeleted file mode 100644index 8ec3893c..00000000--- a/docs/superpowers/specs/2026-05-03-entitlement-service-design.md+++ /dev/null@@ -1,188 +0,0 @@-# Entitlement Service Design--## Problem--The [MessageSendSchedule](./2026-05-03-scheduling-send-refactor-design.md#messagesendschedule-send-windows--new-feature) feature (and future features) need usage limits based on the user's subscription plan. Free users should be limited to 1 send schedule; paid users get unlimited. The system must be:--- **Scalable**: Easy to add new entity limits without architectural changes-- **Configurable**: Disabled by default for self-hosted deployments, enabled via env var for cloud-- **Non-invasive**: Enforced at the handler layer, before business logic executes--## Approach--Create a dedicated `EntitlementService` in `pkg/services/` that:--1. Reads `ENTITLEMENT_ENABLED` from environment (defaults to `false`)-2. Defines a code-based map of entity limits per subscription plan-3. Exposes a single `Check()` method that handlers call before creating resources-4. Returns 402 Payment Required when a free user exceeds their limit--## Configuration--### Environment Variable--```env-# Set to "true" on cloud deployment; self-hosted defaults to false (no limits)-ENTITLEMENT_ENABLED=false-```--### Entity Limits (code-based)--```go-// entityLimits maps entity name → subscription plan → max count-// A limit of 0 means unlimited. If a plan is not listed, it defaults to unlimited.-var entityLimits = map[string]map[entities.SubscriptionName]int{- "MessageSendSchedule": {- entities.SubscriptionNameFree: 1,- },- // Future: add more entities here- // "Webhook": {- // entities.SubscriptionNameFree: 3,- // },-}-```--## Service Interface--```go-// EntitlementService checks whether a user can create more of a given entity.-type EntitlementService struct {- logger telemetry.Logger- tracer telemetry.Tracer- enabled bool- userRepository repositories.UserRepository-}--// NewEntitlementService creates the service. `enabled` comes from ENTITLEMENT_ENABLED env var.-func NewEntitlementService(- logger telemetry.Logger,- tracer telemetry.Tracer,- enabled bool,- userRepository repositories.UserRepository,-) *EntitlementService--// CheckResult holds the outcome of an entitlement check.-type CheckResult struct {- Allowed bool- Message string-}--// Check verifies if the user can create another instance of the given entity.-// - If entitlements are disabled (self-hosted), always returns Allowed: true.-// - Loads the user's subscription plan.-// - Looks up the limit for the entity + plan combination.-// - Compares currentCount against the limit.-func (s *EntitlementService) Check(- ctx context.Context,- userID entities.UserID,- entityName string,- currentCount int,-) (*CheckResult, error)-```--## Handler Integration--In `SendScheduleHandler.Store()`:--```go-func (h *SendScheduleHandler) Store(c *fiber.Ctx) error {- // 1. Validate request (existing logic)- // 2. Get current count (efficient COUNT query)- count, err := h.service.CountByUser(ctx, userID)- if err != nil { ... }- // 3. Check entitlement- result, err := h.entitlementService.Check(ctx, userID, "MessageSendSchedule", count)- if err != nil {- return h.responseInternalServerError(c)- }- if !result.Allowed {- return h.responsePaymentRequired(c, result.Message)- }- // 4. Proceed with creating schedule (existing logic)-}-```--## Repository Addition--Add to `SendScheduleRepository` interface and GORM implementation:--```go-// CountByUser returns the number of schedules owned by a user.-CountByUser(ctx context.Context, userID entities.UserID) (int, error)-```--````--## Error Response--HTTP 402 Payment Required:--```json-{- "message": "Upgrade to a paid plan to create more than 1 send schedule. Visit https://httpsms.com/pricing for details.",- "status": "payment_required"-}-````--## Files to Create/Modify--| Action | File | Change |-| ------ | --------------------------------------------------- | ------------------------------------------------------------ |-| Create | `pkg/services/entitlement_service.go` | New service with limits map, `Check()`, `CheckResult` |-| Modify | `pkg/handlers/handler.go` | Add `responsePaymentRequired()` helper method |-| Modify | `pkg/handlers/send_schedule_handler.go` | Inject `EntitlementService`, add check in `Store()` |-| Modify | `pkg/di/container.go` | Wire `EntitlementService`, read env var, inject into handler |-| Modify | `pkg/repositories/send_schedule_repository.go` | Add `CountByUser()` to interface |-| Modify | `pkg/repositories/gorm_send_schedule_repository.go` | Implement `CountByUser()` with SQL COUNT |-| Modify | `pkg/services/send_schedule_service.go` | Add `CountByUser()` pass-through method |-| Modify | `.env.example` or `.env` | Add `ENTITLEMENT_ENABLED=false` |--## Concurrency & Race Conditions--The handler-level check (`count → check → create`) is not atomic. Two concurrent requests could both see `count=0` and both proceed. Mitigations:--1. **Repository count method**: Use `CountByUser(ctx, userID)` instead of loading all records (efficient SQL `SELECT COUNT(*)`).-2. **Acceptable race window**: For a limit of 1, the worst case is 2 schedules created. This is acceptable because:- - The window is extremely small (single user, same millisecond)- - The consequence is minor (user has 2 schedules instead of 1)- - A DB-level unique constraint is impractical here (limit is per-user count, not per-row uniqueness)-3. **Future hardening**: If stricter enforcement is needed, add an advisory lock or transaction-based count+insert.--## Counting Semantics--All schedules owned by the user count toward the limit, regardless of `is_active` status. A user must delete a schedule to free up their quota.--## Error Handling When Enabled--- **Entitlements disabled** (`ENTITLEMENT_ENABLED=false`): Always returns `Allowed: true`, zero DB calls.-- **Entitlements enabled, DB error loading user**: Return error (surfaces as 500). Do NOT fail-open — this is a monetized feature gate.-- **Entitlements enabled, entity not in limits map**: Returns `Allowed: true` (entity has no restrictions).--## Design Decisions--1. **Handler-layer enforcement**: The handler gets the count and calls `Check()`. This keeps the entitlement service free of domain-specific repository dependencies.-2. **Entity name as key**: Using the entity struct name (e.g., `"MessageSendSchedule"`) makes it self-documenting and matches the user's preference for entity-based naming.-3. **Fail-open when disabled**: Self-hosted users never hit limits. The `enabled` flag short-circuits all checks.-4. **Fail-closed on error when enabled**: If the user can't be loaded and entitlements are enabled, the request fails with 500.-5. **Separate from BillingService**: BillingService handles SMS message counting/billing. EntitlementService handles feature-level access gating. Different concerns.-6. **No caching**: User plan data is already fast to load. Caching can be added later if needed.--## Swagger & Handler Updates--- Add `@Failure 402 {object} responses.PaymentRequired` annotation to `Store` route-- Add `responsePaymentRequired` helper to base handler struct-- Update handler constructor to accept `*services.EntitlementService`--## Testing Strategy--- Unit test `EntitlementService.Check()` with:- - Disabled mode → always allowed- - Free user at limit → denied- - Free user under limit → allowed- - Paid user → always allowed- - Unknown entity → allowed (no restrictions defined)- - User load error when enabled → returns error-- Handler test for `Store`:- - Free user with 0 schedules → 201 Created- - Free user with 1 schedule → 402 Payment Required- - Paid user with N schedules → 201 Createddiff --git a/docs/superpowers/specs/2026-05-03-integration-test-setup-design.md b/docs/superpowers/specs/2026-05-03-integration-test-setup-design.mddeleted file mode 100644index 1b4ced8e..00000000--- a/docs/superpowers/specs/2026-05-03-integration-test-setup-design.md+++ /dev/null@@ -1,248 +0,0 @@-# Integration Test Setup for httpSMS API--## Problem--The httpSMS API has no integration tests that verify the full SMS send/receive flow end-to-end. We need a CI-gated integration test that runs the entire stack in Docker and validates the core message lifecycle before deploying the API.--## Approach--Run the full application stack (API + PostgreSQL + Redis) in Docker alongside an **emulator** service that acts as a fake Android phone. The emulator implements a fake FCM server endpoint so the API's Firebase messaging client sends push notifications to it (instead of Google). The emulator then responds with SENT/DELIVERED events, completing the SMS lifecycle. A Go test runner exercises the API externally and asserts on final message state.--## Architecture--```-┌─────────────────────────────────────────────────────────┐-│ Docker Compose (tests/docker-compose.yml) │-│ │-│ ┌──────────┐ ┌───────┐ ┌──────────────────────────┐ │-│ │PostgreSQL│ │ Redis │ │ API (existing Dockerfile)│ │-│ └──────────┘ └───────┘ └────────────┬─────────────┘ │-│ │ FCM push │-│ ▼ │-│ ┌──────────────────────────┐ │-│ │ Emulator (fake phone) │ │-│ │ - Fake FCM server :9090 │ │-│ │ - Fires SENT/DELIVERED │ │-│ │ events back to API │ │-│ └──────────────────────────┘ │-└─────────────────────────────────────────────────────────┘- ▲- │ HTTP calls (send SMS, get message, etc.)- │-┌────────┴──────────┐-│ Test Runner (Go) │ ← runs on host / in CI-│ go test ./... │-└───────────────────┘-```--## Components--### 1. `tests/docker-compose.yml`--Brings up the full stack:--- **postgres** — Same as root `docker-compose.yml`, seeded with `tests/seed.sql`-- **redis** — Standard Redis-- **api** — Built from `api/Dockerfile`, configured with `FCM_ENDPOINT=http://emulator:9090` to redirect Firebase messaging to the emulator-- **emulator** — Built from `tests/emulator/Dockerfile`, receives FCM pushes and fires events back--### 2. `tests/emulator/` (Go project)--A lightweight Go HTTP server that:--- Exposes `POST /v1/projects/{project}/messages:send` — mimics the FCM v1 API. Receives push notification payloads from the API's Firebase messaging client.-- Exposes `POST /token` — returns a fake OAuth2 access token (the Firebase SDK calls this before sending FCM). Response format: `{"access_token": "fake-token", "token_type": "Bearer", "expires_in": 3600}`-- Exposes `GET /health` — health check endpoint-- On receiving a push with `KEY_MESSAGE_ID` in the data payload:- 1. Calls `GET http://api:8000/v1/messages/outstanding?message_id={messageID}` (using phone API key) to fetch the message like a real phone would- 2. Waits a brief delay (e.g., 200ms)- 3. Calls `POST http://api:8000/v1/messages/{messageID}/events` with event `SENT` (using phone API key)- 4. Waits another brief delay (e.g., 200ms)- 5. Calls `POST http://api:8000/v1/messages/{messageID}/events` with event `DELIVERED` (using phone API key)-- All API calls authenticated with the seeded phone API key (`x-api-key` header)-- Asserts it received the correct FCM payload structure (path, data.KEY_MESSAGE_ID present)--### 3. `tests/seed.sql`--SQL script that runs on PostgreSQL startup to create:--- A test user: `id='test-user-id'`, `email='[email protected]'`, `api_key='test-user-api-key'`, `subscription_name='pro'`-- A system user (for event queue): `id='system-user-id'`, `api_key='system-user-api-key'`-- A phone: `id=<uuid>`, `user_id='test-user-id'`, `phone_number='+18005550199'`, `fcm_token='fake-fcm-token'`-- A phone API key: `id=<uuid>`, `user_id='test-user-id'`, `api_key='test-phone-api-key'`, `phone_numbers=['+18005550199']`--### 4. API Modification — FCM Transport Override--In `api/pkg/di/container.go`, modify `FirebaseMessagingClient()`:--- When `FCM_ENDPOINT` env var is set, create the Firebase App with a custom HTTP client whose `Transport` rewrites request URLs from `https://fcm.googleapis.com` to the value of `FCM_ENDPOINT`-- This requires no changes to business logic — the messaging client works normally but routes traffic to the emulator-- The Firebase credentials must be a syntactically valid fake service account JSON with `token_uri` pointing to `http://emulator:9090/token`--### 4b. `tests/.env.test` — API environment for tests--```env-ENV=production-GCP_PROJECT_ID=httpsms-test-EVENTS_QUEUE_TYPE=emulator-EVENTS_QUEUE_NAME=events-local-EVENTS_QUEUE_ENDPOINT=http://localhost:8000/v1/events-EVENTS_QUEUE_USER_API_KEY=system-user-api-key-EVENTS_QUEUE_USER_ID=system-user-id-FCM_ENDPOINT=http://emulator:9090-DATABASE_URL=postgresql://dbusername:dbpassword@postgres:5432/httpsms-DATABASE_URL_DEDICATED=postgresql://dbusername:dbpassword@postgres:5432/httpsms-REDIS_URL=redis://@redis:6379-APP_PORT=8000-ENTITLEMENT_ENABLED=false-USE_HTTP_LOGGER=true-FIREBASE_CREDENTIALS=<fake service account JSON with token_uri=http://emulator:9090/token>-```--### 5. `tests/integration_test.go` (Go test files)--Go tests using the standard `testing` package + `testify` for assertions:--**Test 1: Send SMS E2E**--1. `POST /v1/messages/send` with `from=<test_phone>`, `to=+18005550100`, `content="Hello"` (using user API key `x-api-key` header)-2. Extract message ID from response-3. Poll `GET /v1/messages/{id}` every 200ms with max 15s timeout (using user API key)-4. Assert message status reaches `delivered`-5. Assert message events include both `SENT` and `DELIVERED`--**Test 2: Receive SMS**--1. `POST /v1/messages/receive` (using phone API key auth) with `from=+18005550100`, `to=+18005550199`, `content="Hi there"`, `sim="SIM1"`, `timestamp=<now>`-2. Extract message ID from response-3. `GET /v1/messages/{id}` (using user API key auth)-4. Assert message exists with correct content, from, to fields-5. Assert status is `received`--### 6. `.github/workflows/integration-test.yml`--GitHub Actions workflow:--```yaml-name: integration-test-on:- push:- branches: [main]- pull_request:- branches: [main]--jobs:- integration-test:- runs-on: ubuntu-latest- steps:- - Checkout- - Docker Compose up (tests/docker-compose.yml)- - Wait for health checks (API + emulator)- - Run: cd tests && go test -v -timeout 120s ./...- - Docker Compose down-- deploy-api:- needs: integration-test- # existing deploy logic-```--The `deploy-api` job depends on `integration-test` passing.--## FCM Redirect Implementation Detail--The Firebase Admin Go SDK's messaging client sends HTTP POST requests to:--```-https://fcm.googleapis.com/v1/projects/{project_id}/messages:send-```--We intercept this by providing a custom `http.RoundTripper`:--```go-type fcmRedirectTransport struct {- target string // e.g., "http://emulator:9090"- base http.RoundTripper-}--func (t *fcmRedirectTransport) RoundTrip(req *http.Request) (*http.Response, error) {- // Rewrite: https://fcm.googleapis.com/... → http://emulator:9090/...- req.URL.Scheme = "http"- req.URL.Host = strings.TrimPrefix(t.target, "http://")- return t.base.RoundTrip(req)-}-```--This is injected via `option.WithHTTPClient()` when creating the Firebase App in the DI container.--## Fake Firebase Credentials--For the integration test environment, we provide a minimal fake service account JSON:--```json-{- "type": "service_account",- "project_id": "httpsms-test",- "private_key_id": "test",- "private_key": "-----BEGIN RSA PRIVATE KEY-----\n<test key>\n-----END RSA PRIVATE KEY-----\n",- "client_email": "[email protected]",- "client_id": "123456789",- "auth_uri": "https://accounts.google.com/o/oauth2/auth",- "token_uri": "http://emulator:9090/token",- "auth_provider_x509_cert_url": "http://emulator:9090/certs",- "client_x509_cert_url": "http://emulator:9090/certs/test"-}-```--The emulator implements:--- `POST /token` — Accepts JWT assertion grant, returns `{"access_token": "fake-token", "token_type": "Bearer", "expires_in": 3600}`-- Does NOT validate the JWT signature — just returns a valid token response--## Docker Health Checks & Orchestration--Services start in order with health dependencies:--1. **postgres** — healthy when `pg_isready` passes-2. **redis** — healthy when accepting connections-3. **emulator** — healthy when `GET /health` returns 200-4. **api** — starts after postgres+redis+emulator healthy, healthy when `GET /v1/` returns (or a dedicated health endpoint)--Test runner waits for all services healthy before executing `go test`.--## File Structure--```-tests/-├── docker-compose.yml-├── seed.sql-├── go.mod-├── go.sum-├── integration_test.go-├── helpers_test.go # shared HTTP client, polling helpers-├── .env.test # env vars for the API in test mode-└── emulator/- ├── Dockerfile- ├── go.mod- ├── go.sum- ├── main.go # entry point, starts HTTP server- ├── fcm_handler.go # fake FCM endpoint- ├── token_handler.go # fake OAuth2 token endpoint- └── events.go # fires SENT/DELIVERED events to API-```--## Key Design Decisions--1. **DB seeding over Firebase Auth emulator** — Simpler, keeps focus on SMS flow testing. Auth is not what we're validating.-2. **Real FCM code path with redirected transport** — Tests the actual Firebase SDK integration, payload construction, and error handling. More confidence than a noop mock.-3. **Emulator as separate Go project** — Clean separation, own Dockerfile, own module. Doesn't pollute the API codebase.-4. **Test runner runs on host (not in Docker)** — Simpler debugging, standard `go test` output, easier CI integration.-5. **Polling with timeout for async assertions** — The send flow is async (event-driven). Polling with backoff is the pragmatic approach.--## Out of Scope--- Testing the web frontend-- Testing the Android app-- Load/performance testing-- Testing auth flows (login, registration)-- Testing billing/entitlements-- MMS/attachment testing (can be added later)diff --git a/docs/superpowers/specs/2026-05-03-scheduling-send-refactor-design.md b/docs/superpowers/specs/2026-05-03-scheduling-send-refactor-design.mddeleted file mode 100644index 1a02bc08..00000000--- a/docs/superpowers/specs/2026-05-03-scheduling-send-refactor-design.md+++ /dev/null@@ -1,188 +0,0 @@-# Scheduling Send Refactor Design--## Related Documentation--- [Scheduling SMS Messages](https://docs.httpsms.com/features/scheduling-sms-messages) — existing `SendAt`/`SendTime` scheduling feature-- [Control SMS Send Rate](https://docs.httpsms.com/features/control-sms-send-rate) — existing `MessagesPerMinute` rate-limiting feature--## Problem Statement--The current SMS scheduling logic has two issues:--1. **No way to send at an exact time without scheduling interference.** When a user specifies a `SendTime`/`SendAt` (see [Scheduling SMS Messages](https://docs.httpsms.com/features/scheduling-sms-messages)), the system still applies rate-limiting and schedule window logic, which may shift the actual send time.--2. **Bulk message contention.** When bulk messages (API or CSV) are sent, all events arrive at the Cloud Tasks queue near-simultaneously, causing DB serialization conflicts in `PhoneNotificationRepository.Schedule()` (which uses `SELECT ... ORDER BY scheduled_at DESC` in a transaction). The current workaround is a hardcoded 1-second spacing hack.--## Proposed Solution--### Core Principle--- **Explicit `SendTime`** = send at exactly that time, bypass all scheduling logic. See [Scheduling SMS Messages](https://docs.httpsms.com/features/scheduling-sms-messages) for how `SendAt` works.-- **No `SendTime`** = apply full scheduling logic ([rate-limit](https://docs.httpsms.com/features/control-sms-send-rate) + schedule windows), with rate-based Cloud Task dispatch delay to prevent DB contention.--### Design--#### 1. ExactSendTime Flag (Transient — not persisted)--A boolean `ExactSendTime` flows through the event system:--```-Request → MessageSendParams → MessageAPISentPayload → PhoneNotificationScheduleParams-```--When `true`, the notification scheduling layer sets `ScheduledAt` to the exact time and skips rate-limit + window logic.--#### 2. Rate-Based Dispatch Delay--For bulk messages without an explicit `SendTime`, instead of the `index * 1s` hack, the service computes:--```go-interval := time.Minute / time.Duration(messagesPerMinute)-delay := time.Duration(index) * interval-```--Where `index` is **per-phone** (not global across the batch). This spreads Cloud Task deliveries at the phone's actual send rate, eliminating DB contention naturally. Duration math avoids integer truncation issues for rates > 60/min or non-divisors of 60.--#### 3. Per-Endpoint Behavior--| Endpoint | `SendAt` provided | `SendAt` absent |-| --------------------------------------- | ---------------------------------------------------- | --------------------------------------------------------- |-| Single SMS API (`/v1/messages/send`) | `ExactSendTime=true`, delay = `time.Until(SendAt)` | `ExactSendTime=false`, delay = 0 |-| Bulk SMS API (`/v1/messages/bulk-send`) | N/A (no SendAt field) | `ExactSendTime=false`, delay = `perPhoneIndex * interval` |-| CSV Upload | `ExactSendTime=true`, delay = `time.Until(SendTime)` | `ExactSendTime=false`, delay = `perPhoneIndex * interval` |--**Index is per-phone**: In a CSV with messages to multiple phones, each phone maintains its own index counter. Messages to Phone A get indices 0, 1, 2... and messages to Phone B get separate indices 0, 1, 2... This ensures correct rate-limiting per phone without over-throttling unrelated phones.--#### 4. Notification Scheduling Bypass--In `PhoneNotificationService.Schedule()`:--```go-if params.ExactSendTime && params.ScheduledSendTime != nil {- notification.ScheduledAt = *params.ScheduledSendTime- // Skip rate-limit and schedule window logic- // Insert directly-} else {- // Existing logic: rate-limit + schedule window-}-```--### Changes by File--| File | Change |-| -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |-| `pkg/events/message_api_sent_event.go` | Add `ExactSendTime bool` field to `MessageAPISentPayload` |-| `pkg/services/message_service.go` | Add `Index int` to `MessageSendParams`; update `getSendDelay()` to compute rate-based delay when `Index > 0` and `SendAt == nil`; set `ExactSendTime` on event payload when `SendAt != nil` |-| `pkg/services/phone_notification_service.go` | Add `ExactSendTime bool` + `ScheduledSendTime *time.Time` to `PhoneNotificationScheduleParams`; add bypass path in `Schedule()` when `ExactSendTime && ScheduledSendTime != nil` — insert notification directly without transaction/rate logic |-| `pkg/repositories/gorm_phone_notification_repository.go` | Add `ScheduleExact(ctx, notification)` method that inserts with a fixed `ScheduledAt` (no transaction, no rate query). Add unique constraint or dedupe check on `(message_id)` for pending notifications to ensure idempotency. |-| `pkg/repositories/phone_notification_repository.go` | Add `ScheduleExact` to the repository interface |-| `pkg/listeners/phone_notification_listener.go` | Pass `ExactSendTime` + `ScheduledSendTime` from event payload to service params |-| `pkg/requests/message_bulk_send_request.go` | Remove per-index `SendAt` computation; add `Index` to each `MessageSendParams` |-| `pkg/requests/bulk_message_request.go` | Propagate `Index` into params for CSV rows |-| `pkg/handlers/message_handler.go` | Remove `index * 1s` hack in `BulkSend` handler |-| `pkg/handlers/bulk_message_handler.go` | Compute per-phone index for CSV rows; remove any concurrent scheduling; ensure `Index` is passed to `MessageSendParams` |--### Data Flow--```-User sends request- → Handler creates MessageSendParams (with Index for bulk, ExactSendTime derived from SendAt presence)- → MessageService.SendMessage()- → Computes dispatch delay:- - ExactSendTime: time.Until(SendAt)- - Bulk without SendAt: Index * (60/MessagesPerMinute)s- - Single without SendAt: 0- → Sets ExactSendTime on MessageAPISentPayload- → DispatchWithTimeout(event, delay) → Cloud Tasks- → [delay elapses] → PhoneNotificationListener.onMessageAPISent()- → PhoneNotificationService.Schedule(params with ExactSendTime)- → If ExactSendTime: insert with exact ScheduledAt- → Else: apply rate-limit + schedule window logic-```--### Edge Cases--- **SendAt in the past**: Send immediately (existing behavior preserved).-- **MessagesPerMinute = 0**: No rate limiting; bulk messages dispatch immediately (existing behavior — `Schedule()` already handles this). Rate-based delay uses 0 when rate is 0.-- **No schedule attached to phone**: Window logic returns current time unchanged (existing behavior).-- **CSV with mixed rows**: Some rows have `SendTime`, others don't. Each row is processed independently — those with `SendTime` get exact dispatch, those without get rate-based delay.-- **Cloud Task duplicate delivery**: `ScheduleExact` and `Schedule` use a dedupe check (unique active notification per `message_id`) to prevent duplicate notification creation on at-least-once delivery.-- **Retries for exact-send messages**: When an exact-send message expires and triggers a retry, the retry does NOT preserve exact-send semantics — it falls through to standard scheduling. The explicit time was a one-shot intent.--### Terminology Note--"Send at exactly that time" means the system will not apply additional rate-limit or schedule-window adjustments. It does NOT guarantee precise handset delivery timing (which depends on Cloud Tasks delivery, FCM push, and device state).--### What Does NOT Change--- The `MessageSendSchedule` entity and its `ResolveScheduledAt()` logic-- The `MessageSendScheduleService` CRUD operations-- The phone notification entity schema (no new DB columns)-- The Android app behavior-- The web frontend (models auto-generated from Swagger)-------## MessageSendSchedule (Send Windows) — New Feature--This is the only scheduling mechanism that does **not** have a dedicated documentation page yet. Unlike [Scheduling SMS Messages](https://docs.httpsms.com/features/scheduling-sms-messages) (one-time `SendAt`) and [Control SMS Send Rate](https://docs.httpsms.com/features/control-sms-send-rate) (`MessagesPerMinute` throttling), MessageSendSchedule defines **recurring availability windows** that control when a phone is allowed to send outgoing SMS messages.--### Concept--A `MessageSendSchedule` is a named set of time windows (per day of week) that define when the phone can send. Messages arriving outside those windows are delayed until the next available window opens.--### Entity--```go-type MessageSendSchedule struct {- ID uuid.UUID- UserID UserID- Name string // e.g. "Business Hours"- Timezone string // IANA timezone e.g. "Europe/Tallinn"- IsActive bool- Windows []MessageSendScheduleWindow // per-day availability slots- CreatedAt time.Time- UpdatedAt time.Time-}--type MessageSendScheduleWindow struct {- DayOfWeek int // 0=Sunday, 6=Saturday- StartMinute int // minutes from midnight (e.g. 540 = 9:00)- EndMinute int // minutes from midnight (e.g. 1020 = 17:00)-}-```--### How It Works--1. A user creates a schedule via `POST /v1/send-schedules` with a name, timezone, and one or more windows.-2. The schedule is linked to a phone via a `ScheduleID` field on the phone entity.-3. When a message is queued (without an explicit `SendAt`), the `PhoneNotificationRepository.Schedule()` method calls `MessageSendSchedule.ResolveScheduledAt(now)` to find the next allowed send time.-4. If the current time falls within a window, the message sends immediately. If not, it's delayed to the start of the next available window.--### API Endpoints--| Method | Endpoint | Description |-| ------ | --------------------------------- | --------------------------- |-| GET | `/v1/send-schedules` | List all user schedules |-| POST | `/v1/send-schedules` | Create a new schedule |-| PUT | `/v1/send-schedules/{scheduleID}` | Update an existing schedule |-| DELETE | `/v1/send-schedules/{scheduleID}` | Delete a schedule |--### Validation Rules--- `name`: required, 2–100 characters-- `timezone`: required, valid IANA timezone-- `windows[].day_of_week`: 0–6-- `windows[].start_minute`: 0–1439-- `windows[].end_minute`: 1–1440, must be greater than `start_minute`-- Max 6 windows per day-- No overlapping windows on the same day--### Entitlement--Free users are limited to 1 schedule. Paid users get unlimited schedules. Enforced via `EntitlementService.Check()` in the handler before creation.--### Interaction with Other Scheduling Features--- **[Scheduling SMS Messages](https://docs.httpsms.com/features/scheduling-sms-messages)** (`SendAt`): When provided, bypasses send windows entirely (exact send time).-- **[Control SMS Send Rate](https://docs.httpsms.com/features/control-sms-send-rate)** (`MessagesPerMinute`): Applied independently — rate-limiting still applies within allowed windows. Both constraints compose: the message must be within a window AND respect the rate limit.diff --git a/docs/superpowers/specs/2026-05-05-integration-tests-wiremock-design.md b/docs/superpowers/specs/2026-05-05-integration-tests-wiremock-design.mddeleted file mode 100644index f2383bff..00000000--- a/docs/superpowers/specs/2026-05-05-integration-tests-wiremock-design.md+++ /dev/null@@ -1,304 +0,0 @@-# Integration Tests: WireMock + httpsms-go Client Refactor--## Problem--The current integration tests use raw `net/http` calls and a custom emulator (120+ lines of Go) to simulate phone behavior. This makes tests harder to maintain and doesn't cover encryption, rate limiting, or webhook verification. We need to:--1. Refactor tests to use the official `httpsms-go` client SDK-2. Replace the custom emulator with WireMock (stub server + request journal)-3. Add E2E encryption tests (outgoing + incoming)-4. Add rate-limit verification test-5. Assert webhook delivery with JWT authentication in all tests--## Architecture--```-┌────────────────────────────────────────────────────────┐-│ Docker Compose (tests/docker-compose.yml) │-│ │-│ ┌──────────┐ ┌───────┐ ┌─────────────────────────┐ │-│ │PostgreSQL│ │ Redis │ │ API │ │-│ └──────────┘ └───────┘ └────────────┬────────────┘ │-│ │FCM push │-│ │Webhook calls │-│ ▼ │-│ ┌─────────────────────────┐ │-│ │ WireMock 3.x (:8080) │ │-│ │ - Fake FCM endpoint │ │-│ │ - Fake OAuth token │ │-│ │ - Webhook receiver │ │-│ │ - Request journal │ │-│ └─────────────────────────┘ │-└────────────────────────────────────────────────────────┘- ▲- │ httpsms-go client + go-wiremock client-┌────────┴──────────┐-│ Test Runner (Go) │-│ go test ./... │-└───────────────────┘-```--### Key Design Decisions--- **WireMock replaces the custom emulator entirely**. It serves as both the fake FCM endpoint (receives push notifications from the API) and the webhook receiver (captures webhook events).-- **Tests fire SENT/DELIVERED events directly** to the API via HTTP. No WireMock callbacks needed — the test controls the flow deterministically.-- **Each test creates its own phone** with a random phone number for parallel test isolation.-- **go-wiremock** (`github.com/wiremock/go-wiremock`) is used to configure stubs and query the request journal from test code.--## Test Flow (per test)--```-1. SETUP- ├─ Create phone (random number, test-specific messages_per_minute)- ├─ Create phone API key for that phone- ├─ Create webhook pointing to WireMock with a signing key- └─ Configure WireMock stubs (if not pre-loaded)--2. ACT- ├─ Send/receive message via httpsms-go client- └─ (For send tests) Query WireMock journal → extract KEY_MESSAGE_ID from FCM push--3. SIMULATE PHONE- ├─ Fire SENT event to API (POST /v1/messages/{id}/events)- └─ Fire DELIVERED event to API--4. ASSERT- ├─ Verify message reached expected status via httpsms-go client- ├─ Query WireMock journal for webhook events- ├─ Validate JWT token: signature (HMAC-SHA256), issuer, subject, audience, expiry- └─ Validate webhook payload contains correct event type and message data-```--## Components--### 1. Docker Compose Changes--**Remove:**--- `tests/emulator/` directory entirely (Dockerfile, Go source, go.mod)--**Replace with WireMock:**--```yaml-wiremock:- image: wiremock/wiremock:3x- ports:- - "8080:8080"- volumes:- - ./wiremock/mappings:/home/wiremock/mappings:ro- healthcheck:- test: ["CMD", "curl", "-f", "http://localhost:8080/__admin/health"]- interval: 5s- timeout: 5s- retries: 10-```--**Pre-loaded WireMock mappings** (`tests/wiremock/mappings/`):--- `fcm-send.json` — Stub for `POST /v1/projects/*/messages:send` → returns `{"name": "projects/httpsms-test/messages/fake-id"}`-- `oauth-token.json` — Stub for `POST /token` → returns `{"access_token": "fake-access-token", "token_type": "Bearer", "expires_in": 3600}`-- `webhook-receiver.json` — Stub for `POST /webhooks/test` → returns 200 (catches all webhook calls)--### 2. API Configuration Updates--**`.env.test` changes:**--- `FCM_ENDPOINT=http://wiremock:8080` (was `http://emulator:9090`)--**Firebase credentials** `token_uri` points to `http://wiremock:8080/token`--### 3. Seed SQL (simplified)--Only seeds:--- Test user (`test-user-id`, `api_key='test-user-api-key'`)-- System user (`system-user-id`, for event queue auth)--Phones, phone API keys, and webhooks are created per-test via the API.--### 4. httpsms-go Client Additions--New services to add to `github.com/NdoleStudio/httpsms-go`:--#### `PhoneService`--```go-type PhoneUpsertParams struct {- PhoneNumber string `json:"phone_number"`- FcmToken string `json:"fcm_token"`- MessagesPerMinute uint `json:"messages_per_minute"`- MaxSendAttempts uint `json:"max_send_attempts"`- MessageExpirationSeconds uint `json:"message_expiration_seconds"`- SIM string `json:"sim"`-}--func (service *PhoneService) Upsert(ctx, params) → (*PhoneResponse, *Response, error)-// PUT /v1/phones — authenticated with user API key-```--#### `PhoneService` (FCM Token binding)--```go-type PhoneFCMTokenParams struct {- PhoneNumber string `json:"phone_number"`- FcmToken string `json:"fcm_token"`- SIM string `json:"sim"`-}--func (service *PhoneService) UpsertFCMToken(ctx, params) → (*PhoneResponse, *Response, error)-// PUT /v1/phones/fcm-token — authenticated with phone API key-// This binds the phone to the phone API key via the auth context-```--#### `PhoneAPIKeyService`--```go-type PhoneAPIKeyStoreParams struct {- Name string `json:"name"`-}--func (service *PhoneAPIKeyService) Store(ctx, params) → (*PhoneAPIKeyResponse, *Response, error)-// POST /v1/phone-api-keys/ — authenticated with user API key-// Returns the created phone API key including its api_key value-```--#### `WebhookService`--```go-type WebhookStoreParams struct {- SigningKey string `json:"signing_key"`- URL string `json:"url"`- PhoneNumbers []string `json:"phone_numbers"`- Events []string `json:"events"`-}--func (service *WebhookService) Store(ctx, params) → (*WebhookResponse, *Response, error)-// POST /v1/webhooks — authenticated with user API key-```--#### Phone Setup Flow (per test)--The real Android phone registers via this flow, and tests must replicate it:--1. `PUT /v1/phones` (user API key) — creates phone with phone_number + fcm_token + messages_per_minute-2. `POST /v1/phone-api-keys/` (user API key) — creates a phone API key, returns the `api_key` value-3. `PUT /v1/phones/fcm-token` (phone API key) — re-registers FCM token, which binds the phone to the API key via `PhoneAPIKeyListener.onPhoneUpdated`--After step 3, the phone API key is authorized to act on behalf of that phone (fire events, receive messages, etc.).--### 5. Test Cases--#### `TestSendSMS_Encrypted`--1. Generate random encryption key-2. Create phone + phone API key + webhook-3. Encrypt plaintext using `client.Cipher.Encrypt(key, "secret message")`-4. Send message with `Encrypted: true` and encrypted content-5. Query WireMock journal → verify FCM push arrived with `KEY_MESSAGE_ID` (FCM only carries the message ID, not content)-6. Call `GET /v1/messages/outstanding?message_id={id}` (phone API key) — verify response has `encrypted: true` and content is ciphertext (not plaintext)-7. Fire SENT + DELIVERED events-8. Fetch message via user API key → verify `encrypted: true`, content is ciphertext-9. Decrypt with `client.Cipher.Decrypt(key, content)` → assert equals original plaintext-10. Verify webhook event in WireMock with valid JWT--#### `TestReceiveSMS_Encrypted`--1. Generate random encryption key-2. Create phone + phone API key + webhook-3. Encrypt plaintext using `client.Cipher.Encrypt(key, "incoming secret")`-4. Simulate receiving an encrypted SMS (POST /v1/messages/receive with phone API key)-5. Fetch message via user API key → verify `encrypted: true`-6. Decrypt content → assert equals original plaintext-7. Verify webhook event (`message.phone.received`) in WireMock with valid JWT--#### `TestSendSMS_RateLimit`--1. Create phone with `messages_per_minute: 10` (= 6s gap)-2. Create phone API key + webhook-3. Send 2 messages simultaneously-4. Query WireMock journal for FCM pushes (correlate by message IDs from send responses)-5. Assert the timestamps of the two FCM pushes have ≥6 second gap-6. Fire SENT + DELIVERED for both messages-7. Verify both messages reach `delivered` status-8. Verify webhook events for both messages--#### `TestSendSMS_OutstandingFlow`--Validates the real phone flow (`/v1/messages/outstanding`):--1. Create phone + phone API key + webhook-2. Send message via httpsms-go client-3. Query WireMock journal → extract `KEY_MESSAGE_ID` from FCM push-4. Call `GET /v1/messages/outstanding?message_id={id}` (phone API key) — assert returns the message with correct content, owner, contact-5. Fire SENT + DELIVERED events-6. Verify message reaches `delivered` status-7. Verify webhook events--#### Webhook Verification (shared helper)--For all tests, a helper function:--```go-func assertWebhookEvent(t *testing.T, wiremockClient *wiremock.Client, signingKey string, expectedEventType string) {- // 1. Query WireMock journal for POST /webhooks/test requests- // 2. Find request with X-Event-Type header matching expectedEventType- // 3. Extract Authorization header → parse JWT- // 4. Validate signature with signingKey (HMAC-SHA256)- // 5. Assert claims:- // - Issuer == "api.httpsms.com"- // - Subject == "test-user-id"- // - Audience contains webhook URL- // - ExpiresAt is in the future- // - NotBefore is in the past-}-```--### 6. Test Helper Structure--```-tests/-├── docker-compose.yml (updated: wiremock replaces emulator)-├── wiremock/-│ └── mappings/-│ ├── fcm-send.json-│ ├── oauth-token.json-│ └── webhook-receiver.json-├── seed.sql (simplified: user + system user only)-├── .env.test (updated: FCM_ENDPOINT → wiremock)-├── go.mod (add httpsms-go, go-wiremock, golang-jwt)-├── helpers_test.go (shared constants, setup helpers)-├── webhook_helpers_test.go (JWT verification helpers)-├── integration_test.go (all test cases)-└── README.md-```--### 7. Dependencies--**Test module (`tests/go.mod`):**--- `github.com/NdoleStudio/httpsms-go` — API client-- `github.com/wiremock/go-wiremock` — WireMock stub configuration + journal queries-- `github.com/golang-jwt/jwt/v5` — JWT parsing and validation-- `github.com/stretchr/testify` — assertions (already present)--### 8. Parallel Test Execution & Request Correlation--Each test creates its own phone with a unique random number (e.g. `+1800555XXXX` where XXXX is random). This ensures:--- No message cross-contamination between tests-- Webhooks scoped to specific phone numbers don't fire for other tests--**Correlation strategy for WireMock journal queries:**--- **FCM pushes**: Correlate by message ID. The test gets the message ID from the send response, then searches WireMock journal for FCM push requests containing that `KEY_MESSAGE_ID` in the JSON body.-- **Webhook events**: Each test uses a **unique webhook URL path** (e.g. `/webhooks/{testUUID}`). This ensures journal queries for webhook assertions only match events for that specific test. Additionally, match on `X-Event-Type` header and message ID in payload body.-- **Unique FCM token per phone**: Each test generates a unique `fcm_token` string. Since WireMock captures the FCM push including the `token` field, this can be used as a secondary correlation key if needed.--Tests use `t.Parallel()` where safe (encryption tests can run in parallel; rate-limit test may need serial execution due to timing assertions).--## Migration Notes--- The `tests/emulator/` directory is deleted entirely-- The CI workflow (`.github/workflows/integration-test.yml`) needs updating to remove emulator references-- Firebase credentials `token_uri` must point to `http://wiremock:8080/token`-- WireMock image is Java-based (~300MB) vs the old Alpine emulator (~15MB), but eliminates maintenance of custom codediff --git a/docs/superpowers/specs/2026-05-15-hedging-repository-design.md b/docs/superpowers/specs/2026-05-15-hedging-repository-design.mddeleted file mode 100644index 2afc6ace..00000000--- a/docs/superpowers/specs/2026-05-15-hedging-repository-design.md+++ /dev/null@@ -1,164 +0,0 @@-# Hedging Repository for Heartbeat & Monitor--**Date:** 2026-05-15-**Status:** Approved--## Overview--Create composite "hedging" repositories for `HeartbeatRepository` and `HeartbeatMonitorRepository` that write to both GORM (primary) and Turso (secondary). Reads only hit the primary. Secondary writes are fail-open — errors are logged and a metric is emitted, but the operation succeeds from the caller's perspective.--## Motivation--Gradually migrate heartbeat data to Turso by dual-writing. The GORM/PostgreSQL backend remains the source of truth while Turso builds up a complete dataset. If Turso has issues, the system is unaffected.--## Configuration--Activated via `HEARTBEAT_DB_BACKEND=hedging`. The three modes are now:--| Value | Behavior |-| ----------------- | ---------------------------------------------------------------------- |-| _(unset/default)_ | GORM/PostgreSQL only |-| `turso` | Turso/libSQL only |-| `hedging` | GORM primary (reads+writes) + Turso secondary (writes only, fail-open) |--## Architecture--### New Files--| File | Purpose |-| -------------------------------------------------------------- | -------------------------------------- |-| `api/pkg/repositories/hedging_heartbeat_repository.go` | Composite `HeartbeatRepository` |-| `api/pkg/repositories/hedging_heartbeat_monitor_repository.go` | Composite `HeartbeatMonitorRepository` |--### Modified Files--| File | Change |-| ------------------------- | ------------------------------------------------------------------ |-| `api/pkg/di/container.go` | Add `hedging` case to switch, add `HedgingFailureCounter()` method |--## Method Delegation--### HeartbeatRepository--| Method | Primary (GORM) | Secondary (Turso) |-| ------------------ | -------------- | -------------------- |-| `Store` | ✅ write | ✅ write (fail-open) |-| `Index` | ✅ read | ❌ skip |-| `Last` | ✅ read | ❌ skip |-| `DeleteAllForUser` | ✅ write | ✅ write (fail-open) |--### HeartbeatMonitorRepository--| Method | Primary (GORM) | Secondary (Turso) |-| ------------------- | -------------- | -------------------- |-| `Store` | ✅ write | ✅ write (fail-open) |-| `Load` | ✅ read | ❌ skip |-| `Exists` | ✅ read | ❌ skip |-| `UpdateQueueID` | ✅ write | ✅ write (fail-open) |-| `Delete` | ✅ write | ✅ write (fail-open) |-| `UpdatePhoneOnline` | ✅ write | ✅ write (fail-open) |-| `DeleteAllForUser` | ✅ write | ✅ write (fail-open) |--## Struct Design--```go-type hedgingHeartbeatRepository struct {- logger telemetry.Logger- tracer telemetry.Tracer- primary HeartbeatRepository- secondary HeartbeatRepository- failureCounter otelMetric.Int64Counter-}--type hedgingHeartbeatMonitorRepository struct {- logger telemetry.Logger- tracer telemetry.Tracer- primary HeartbeatMonitorRepository- secondary HeartbeatMonitorRepository- failureCounter otelMetric.Int64Counter-}-```--## Error Handling (Fail-Open Pattern)--```go-func (r *hedgingHeartbeatRepository) Store(ctx context.Context, heartbeat *entities.Heartbeat) error {- ctx, span := r.tracer.Start(ctx)- defer span.End()-- // Primary: must succeed- if err := r.primary.Store(ctx, heartbeat); err != nil {- return err- }-- // Secondary: fail-open (log + metric)- if err := r.secondary.Store(ctx, heartbeat); err != nil {- r.logger.Error(stacktrace.Propagate(err, fmt.Sprintf(- "hedging: secondary write failed for heartbeat [%s]", heartbeat.ID,- )))- r.failureCounter.Add(ctx, 1)- }-- return nil-}-```--**Rules:**--- Primary error → return immediately, no secondary attempt-- Secondary error → log at ERROR level, increment `failureCounter`, return nil-- Read methods → delegate directly to primary, no secondary involvement-- Each method has its own tracing span via `tracer.Start(ctx)`--## Observability--**Metric:**--- Name: `hedging.secondary.write.failures`-- Unit: `1` (count)-- Description: `Number of failed secondary writes in hedging repositories`-- Created once in DI container, shared by both hedging repos--**Logging:** Each secondary failure logs at ERROR with the method context (entity ID, user ID, etc.)--## DI Container Changes--```go-func (container *Container) HeartbeatRepository() repositories.HeartbeatRepository {- switch os.Getenv("HEARTBEAT_DB_BACKEND") {- case "turso":- return repositories.NewLibsqlHeartbeatRepository(...)- case "hedging":- return repositories.NewHedgingHeartbeatRepository(- container.Logger(),- container.Tracer(),- repositories.NewGormHeartbeatRepository(container.Logger(), container.Tracer(), container.DedicatedDB()),- repositories.NewLibsqlHeartbeatRepository(container.Logger(), container.Tracer(), container.TursoDB()),- container.HedgingFailureCounter(),- )- default:- return repositories.NewGormHeartbeatRepository(...)- }-}--func (container *Container) HedgingFailureCounter() otelMetric.Int64Counter {- meter := otel.GetMeterProvider().Meter(container.projectID)- counter, err := meter.Int64Counter("hedging.secondary.write.failures",- otelMetric.WithUnit("1"),- otelMetric.WithDescription("Number of failed secondary writes in hedging repositories"),- )- if err != nil {- container.logger.Fatal(...)- }- return counter-}-```--Same switch pattern for `HeartbeatMonitorRepository()`.--## Safety--- Default behavior (no env var) is unchanged — pure GORM/PostgreSQL-- `turso` mode remains available for pure Turso usage-- `hedging` mode never fails due to Turso issues — secondary is fully fail-open-- Service layer requires no changes — same interfacesdiff --git a/docs/superpowers/specs/2026-05-15-turso-heartbeat-backend-design.md b/docs/superpowers/specs/2026-05-15-turso-heartbeat-backend-design.mddeleted file mode 100644index b501650f..00000000--- a/docs/superpowers/specs/2026-05-15-turso-heartbeat-backend-design.md+++ /dev/null@@ -1,162 +0,0 @@-# Turso/libSQL Backend for Heartbeat & Monitor Repositories--**Date:** 2026-05-15-**Status:** Approved--## Overview--Add a libSQL/Turso alternative implementation for `HeartbeatRepository` and `HeartbeatMonitorRepository`, switchable via the `HEARTBEAT_DB_BACKEND` environment variable. When set to `turso`, the API connects to a cloud-hosted Turso database instead of the dedicated PostgreSQL instance.--## Motivation--Move heartbeat storage to a dedicated Turso database for cost efficiency and edge performance, while keeping the existing PostgreSQL path as the default fallback.--## Configuration--| Env Var | Purpose | Example |-| ---------------------- | -------------------------------------------------------------- | ---------------------------------------------------------------------- |-| `HEARTBEAT_DB_BACKEND` | Selects backend (`turso` = libSQL, anything else = PostgreSQL) | `turso` |-| `TURSO_DATABASE_DSN` | Turso database DSN (URL with authToken query param) | `libsql://httpsms-ndolestudio.aws-us-east-1.turso.io?authToken=eyJ...` |--When `HEARTBEAT_DB_BACKEND` is not set or is any value other than `turso`, the existing GORM/PostgreSQL path is used unchanged.--## Architecture--### Approach--Direct `database/sql` with the Turso Go remote driver (`github.com/tursodatabase/libsql-client-go`). No ORM — raw SQL queries for a simple 2-table schema. This is the pure-Go HTTP driver for remote Turso Cloud access (no CGo required).--### New Files--| File | Purpose |-| ------------------------------------------------------------- | -------------------------------------------------------- |-| `api/pkg/repositories/libsql.go` | Connection factory, table auto-creation, shared helpers |-| `api/pkg/repositories/libsql_heartbeat_repository.go` | `HeartbeatRepository` implementation using libSQL |-| `api/pkg/repositories/libsql_heartbeat_monitor_repository.go` | `HeartbeatMonitorRepository` implementation using libSQL |--### Modified Files--| File | Change |-| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |-| `api/pkg/di/container.go` | Add `tursoDB *sql.DB` field, `TursoDB()` method, conditional wiring in `HeartbeatRepository()` and `HeartbeatMonitorRepository()` |-| `api/go.mod` | Add `github.com/tursodatabase/libsql-client-go` dependency |--## Database Schema--```sql-CREATE TABLE IF NOT EXISTS heartbeats (- id TEXT PRIMARY KEY,- owner TEXT NOT NULL,- version TEXT NOT NULL,- charging INTEGER NOT NULL DEFAULT 0,- user_id TEXT NOT NULL,- timestamp DATETIME NOT NULL-);-CREATE INDEX IF NOT EXISTS idx_heartbeats_owner_timestamp ON heartbeats(owner, timestamp);-CREATE INDEX IF NOT EXISTS idx_heartbeats_user_id ON heartbeats(user_id);--CREATE TABLE IF NOT EXISTS heartbeat_monitors (- id TEXT PRIMARY KEY,- phone_id TEXT NOT NULL,- user_id TEXT NOT NULL,- queue_id TEXT NOT NULL DEFAULT '',- owner TEXT NOT NULL,- phone_online INTEGER NOT NULL DEFAULT 1,- created_at DATETIME NOT NULL,- updated_at DATETIME NOT NULL-);-CREATE INDEX IF NOT EXISTS idx_heartbeat_monitors_user_owner ON heartbeat_monitors(user_id, owner);-```--**Type mappings from PostgreSQL:**--- UUID → TEXT-- BOOLEAN → INTEGER (0/1)-- TIMESTAMP → DATETIME (ISO 8601 text)--## Repository Implementations--### `libsql.go` (shared)--- `NewTursoDB(url, authToken string) (*sql.DB, error)` — opens connection with libSQL driver, executes CREATE TABLE/INDEX statements-- Returns `*sql.DB` for use by both repository implementations--### `libsql_heartbeat_repository.go`--Implements `HeartbeatRepository`:--| Method | SQL |-| ------------------ | ------------------------------------------------------------------------------------------------------------------------ |-| `Store` | `INSERT INTO heartbeats (id, owner, version, charging, user_id, timestamp) VALUES (?, ?, ?, ?, ?, ?)` |-| `Index` | `SELECT ... WHERE user_id = ? AND owner = ? ORDER BY timestamp DESC LIMIT ? OFFSET ?` (optional `version LIKE ?` filter) |-| `Last` | `SELECT ... WHERE user_id = ? AND owner = ? ORDER BY timestamp DESC LIMIT 1` |-| `DeleteAllForUser` | `DELETE FROM heartbeats WHERE user_id = ?` |--### `libsql_heartbeat_monitor_repository.go`--Implements `HeartbeatMonitorRepository`:--| Method | SQL |-| ------------------- | --------------------------------------------------------------------------- |-| `Store` | INSERT all fields |-| `Load` | SELECT by user_id + owner |-| `Exists` | `SELECT COUNT(*) FROM ... WHERE user_id = ? AND id = ?` (returns count > 0) |-| `UpdateQueueID` | UPDATE queue_id + updated_at WHERE id = ? |-| `Delete` | DELETE WHERE user_id = ? AND owner = ? |-| `UpdatePhoneOnline` | UPDATE phone_online + updated_at WHERE id = ? AND user_id = ? |-| `DeleteAllForUser` | DELETE WHERE user_id = ? |--### Error Handling--- `sql.ErrNoRows` → wrap with `stacktrace.PropagateWithCode(err, ErrCodeNotFound, msg)` to match GORM behavior expected by the service layer-- All other errors → wrap with `stacktrace.Propagate(err, msg)`--### Observability--Every method follows the existing tracing pattern:--```go-ctx, span := repository.tracer.Start(ctx)-defer span.End()-```--## DI Container Changes--```go-// New field on Container struct-tursoDB *sql.DB--// New method-func (container *Container) TursoDB() *sql.DB {- if container.tursoDB != nil {- return container.tursoDB- }- db, err := repositories.NewTursoDB(os.Getenv("TURSO_DATABASE_DSN"))- if err != nil {- container.logger.Fatal(err)- }- container.tursoDB = db- return container.tursoDB-}--// Modified methods-func (container *Container) HeartbeatRepository() repositories.HeartbeatRepository {- if os.Getenv("HEARTBEAT_DB_BACKEND") == "turso" {- return repositories.NewLibsqlHeartbeatRepository(container.Logger(), container.Tracer(), container.TursoDB())- }- return repositories.NewGormHeartbeatRepository(container.Logger(), container.Tracer(), container.DedicatedDB())-}--func (container *Container) HeartbeatMonitorRepository() repositories.HeartbeatMonitorRepository {- if os.Getenv("HEARTBEAT_DB_BACKEND") == "turso" {- return repositories.NewLibsqlHeartbeatMonitorRepository(container.Logger(), container.Tracer(), container.TursoDB())- }- return repositories.NewGormHeartbeatMonitorRepository(container.Logger(), container.Tracer(), container.DedicatedDB())-}-```--## Safety--- When `HEARTBEAT_DB_BACKEND != "turso"`, `TursoDB()` is never called — no Turso connection is opened-- The existing PostgreSQL path remains the default and is completely unaffected-- Both implementations satisfy the same interface — the service layer requires no changes