Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

16.9 KB
Raw Blame
package ai

import (
	"context"
	"encoding/json"
	"errors"
	"fmt"
	"strings"
	"time"

	"github.com/google/uuid"
	"github.com/rcourtman/pulse-go-rewrite/internal/ai/cost"
	"github.com/rcourtman/pulse-go-rewrite/internal/ai/providers"
	"github.com/rcourtman/pulse-go-rewrite/internal/config"
	"github.com/rcourtman/pulse-go-rewrite/pkg/reporting"
)

// reportNarratorUseCase is the cost-ledger label for AI-narrated report
// generations. Operators see this value in the AI usage dashboard so
// report-narrative spend is distinguishable from chat or patrol.
const reportNarratorUseCase = "report_narrative"

// reportNarratorMaxTokens caps the response budget for narrative generation.
// The output is short prose plus a small JSON envelope, so a tight ceiling
// keeps cost predictable and discourages padding.
const reportNarratorMaxTokens = 1500

// reportNarratorSystemPrompt instructs the model to ground every claim in
// the provided structured data and to refuse interpretation it cannot
// support from the input. Severity is constrained to the set the renderer
// understands so unknown values do not silently render as muted.
const reportNarratorSystemPrompt = `You are Pulse Assistant generating the executive summary section of a sysadmin performance report.

You MUST:
- Ground every observation and recommendation in the structured data provided in the user message. If the data does not support a claim, do not make it.
- Reference the deterministic data tables that accompany this narrative (Performance Summary, Active Alerts, Storage, Disks). Do NOT fabricate metric values, alert messages, disk identifiers, or finding titles.
- Use observation severity strictly from this set: "ok", "info", "warning", "critical". Map clean state to "ok", informational facts to "info", concerning trends to "warning", and immediate-action items to "critical".
- Keep observations to short, concrete sentences. Avoid hedging adverbs ("perhaps", "seems"). State the fact and its implication.
- When prior-period data is supplied, write a period_comparison paragraph describing the most material deltas (resource trends, new or resolved alerts, new findings). When no prior data is supplied, leave period_comparison empty.
- Do NOT invent recommendations for problems that aren't in the data. If everything is healthy, say so plainly.
- An alert with a "resolution" closed without recovering, for example because the Pulse agent now monitoring the machine took its metric over. It is not a recovery: never count it as resolved or call the resource healthy because of it, and say where it went using the resolution text. When it also has "successor_alert_listed": true, the alerts list holds the alert the resource it moved to raised for that metric; describe the condition once, from that alert.

DETECTION BOUNDARY — this is critical:
Pulse Patrol is the canonical detection layer for this product. The patrol_findings array, when populated, contains issues Patrol has already classified with its own severity. Your job is to SUMMARIZE those findings, not to function as a parallel detector competing with them.

- When a patrol_finding exists for an issue, reference it by its title in observations and recommendations. Do not paraphrase it as if you discovered it independently.
- You may flag a "warning" or "critical" observation only when it is grounded in:
    - a patrol_finding in the input, OR
    - an alert in the input, OR
    - a hard-threshold breach you can point to in metric_stats (e.g. cpu max > 90, memory avg > 85, disk avg > 85), OR
    - a disk with health = "FAILED" or wear_level <= 30, OR
    - a storage pool with usage_perc >= 90.
- If you notice a pattern in the metric data that is NOT covered by any of the above, you may mention it as an "info" severity observation describing what you saw — but you MUST NOT promote it to "warning" or "critical". Classification at those levels is Patrol's job; you are reporting on Patrol's work, not replacing it.
- This rule applies to recommendations too: do not recommend remediation for an inferred issue that has no finding, alert, or threshold breach to back it.

Respond ONLY with a single JSON object matching this exact schema (no markdown fences, no commentary outside the JSON):

{
  "health_status": "HEALTHY" | "WARNING" | "CRITICAL",
  "health_message": "<one short sentence>",
  "executive_summary": "<2-4 sentence paragraph>",
  "observations": [
    { "text": "<one sentence>", "severity": "ok" | "info" | "warning" | "critical" }
  ],
  "recommendations": [ "<one sentence imperative>" ],
  "period_comparison": "<optional paragraph; empty string if no prior data>"
}`

// reportNarratorPayload is the structured data sent to the model. Keeping
// it explicit (rather than json.Marshal-ing reporting types directly) makes
// the prompt surface stable as internal types evolve.
type reportNarratorPayload struct {
	Title        string                       `json:"title"`
	ResourceType string                       `json:"resource_type"`
	ResourceID   string                       `json:"resource_id"`
	Period       reportNarratorPeriod         `json:"period"`
	Resource     *reportNarratorResource      `json:"resource,omitempty"`
	MetricStats  map[string]reportNarratorMS  `json:"metric_stats"`
	PriorPeriod  *reportNarratorPriorPeriod   `json:"prior_period,omitempty"`
	Alerts       []reportNarratorAlert        `json:"alerts,omitempty"`
	Storage      []reportNarratorStorage      `json:"storage,omitempty"`
	Disks        []reportNarratorDisk         `json:"disks,omitempty"`
	Findings     []reportNarratorFindingEntry `json:"patrol_findings,omitempty"`
}

type reportNarratorPeriod struct {
	Start string `json:"start"`
	End   string `json:"end"`
	Hours int    `json:"hours"`
}

type reportNarratorResource struct {
	Name        string  `json:"name,omitempty"`
	DisplayName string  `json:"display_name,omitempty"`
	Status      string  `json:"status,omitempty"`
	Node        string  `json:"node,omitempty"`
	UptimeDays  int64   `json:"uptime_days,omitempty"`
	CPUCores    int     `json:"cpu_cores,omitempty"`
	MemoryGB    float64 `json:"memory_gb,omitempty"`
}

type reportNarratorMS struct {
	Min     float64 `json:"min"`
	Max     float64 `json:"max"`
	Avg     float64 `json:"avg"`
	Current float64 `json:"current"`
	Count   int     `json:"count"`
}

type reportNarratorPriorPeriod struct {
	Start       string                      `json:"start"`
	End         string                      `json:"end"`
	MetricStats map[string]reportNarratorMS `json:"metric_stats"`
}

type reportNarratorAlert struct {
	Type      string  `json:"type"`
	Level     string  `json:"level"`
	Message   string  `json:"message"`
	Value     float64 `json:"value,omitempty"`
	Threshold float64 `json:"threshold,omitempty"`
	// Resolved is true only for a recovery. An alert that closed without
	// recovering, such as a node alert that moved to its Pulse agent, carries
	// the alert engine's account in Resolution instead, and
	// SuccessorAlertListed when this list also holds the alert the resource
	// it moved to raised for the same metric.
	Resolved             bool   `json:"resolved"`
	Resolution           string `json:"resolution,omitempty"`
	SuccessorAlertListed bool   `json:"successor_alert_listed,omitempty"`
}

type reportNarratorStorage struct {
	Name      string  `json:"name"`
	Type      string  `json:"type"`
	UsagePerc float64 `json:"usage_perc"`
	ZFSHealth string  `json:"zfs_health,omitempty"`
}

type reportNarratorDisk struct {
	Device    string `json:"device"`
	Type      string `json:"type"`
	Health    string `json:"health,omitempty"`
	WearLevel int    `json:"wear_level,omitempty"`
}

type reportNarratorFindingEntry struct {
	Severity       string `json:"severity"`
	Category       string `json:"category"`
	Title          string `json:"title"`
	Description    string `json:"description,omitempty"`
	Recommendation string `json:"recommendation,omitempty"`
	Resolved       bool   `json:"resolved"`
}

// reportNarratorResponse is the JSON envelope the model is asked to return.
type reportNarratorResponse struct {
	HealthStatus     string                         `json:"health_status"`
	HealthMessage    string                         `json:"health_message"`
	ExecutiveSummary string                         `json:"executive_summary"`
	Observations     []reportNarratorResponseBullet `json:"observations"`
	Recommendations  []string                       `json:"recommendations"`
	PeriodComparison string                         `json:"period_comparison"`
}

type reportNarratorResponseBullet struct {
	Text     string `json:"text"`
	Severity string `json:"severity"`
}

// Compile-time assertion the Service satisfies the reporting.Narrator interface.
var _ reporting.Narrator = (*Service)(nil)

// Narrate implements reporting.Narrator. It builds a single-turn prompt
// from the supplied input, asks the configured provider for a JSON
// response, and parses it into a reporting.Narrative. Returning an error
// causes the engine to fall back to the heuristic narrator, so callers do
// not need to distinguish "AI disabled" from "AI failed".
func (s *Service) Narrate(ctx context.Context, in reporting.NarrativeInput) (reporting.Narrative, error) {
	s.mu.RLock()
	provider := s.provider
	cfg := s.cfg
	costStore := s.costStore
	s.mu.RUnlock()

	if provider == nil {
		return reporting.Narrative{}, errors.New("Pulse Assistant is not configured")
	}

	model := ""
	if cfg != nil {
		if cfg.PatrolModel != "" {
			model = cfg.PatrolModel
		} else {
			model = cfg.GetChatModel()
		}
	}

	if err := s.enforceBudget(reportNarratorUseCase); err != nil {
		return reporting.Narrative{}, err
	}

	payload := buildReportNarratorPayload(in)
	body, err := json.Marshal(payload)
	if err != nil {
		return reporting.Narrative{}, fmt.Errorf("encode report payload: %w", err)
	}

	chatReq := providers.ChatRequest{
		Messages: []providers.Message{
			{Role: "user", Content: string(body)},
		},
		Model:       model,
		System:      reportNarratorSystemPrompt,
		MaxTokens:   reportNarratorMaxTokens,
		ExecutionID: uuid.NewString(),
	}
	if sanitizer := s.requestSanitizerForModel(model); sanitizer != nil {
		chatReq = sanitizer(chatReq)
	}

	resp, err := provider.Chat(ctx, chatReq)
	if err != nil {
		return reporting.Narrative{}, fmt.Errorf("provider chat: %w", err)
	}

	// Record token usage in the operator-facing cost ledger so AI-narrated
	// report generation shows up in the AI usage dashboard alongside chat
	// and patrol spend. Recording happens regardless of whether the
	// response parses, so failed-but-billed calls are still visible.
	if costStore != nil {
		providerName, _ := config.ParseModelString(model)
		if providerName == "" {
			providerName = provider.Name()
		}
		costStore.Record(cost.UsageEvent{
			Timestamp:                time.Now(),
			Provider:                 providerName,
			RequestModel:             model,
			ResponseModel:            resp.Model,
			UseCase:                  reportNarratorUseCase,
			InputTokens:              resp.InputTokens,
			OutputTokens:             resp.OutputTokens,
			CacheCreationInputTokens: resp.CacheCreationInputTokens,
			CacheReadInputTokens:     resp.CacheReadInputTokens,
			TargetType:               in.ResourceType,
			TargetID:                 in.ResourceID,
		})
	}

	content := strings.TrimSpace(resp.Content)
	if content == "" {
		return reporting.Narrative{}, errors.New("provider returned empty narrative")
	}

	parsed, err := parseReportNarratorResponse(content)
	if err != nil {
		return reporting.Narrative{}, err
	}

	narrative := reporting.Narrative{
		Source:           reporting.NarrativeSourceAI,
		HealthStatus:     normalizeReportHealthStatus(parsed.HealthStatus),
		HealthMessage:    strings.TrimSpace(parsed.HealthMessage),
		ExecutiveSummary: strings.TrimSpace(parsed.ExecutiveSummary),
		PeriodComparison: strings.TrimSpace(parsed.PeriodComparison),
		Disclaimer:       "Narrative generated by Pulse Assistant. Verify against the data tables in this report.",
	}
	for _, b := range parsed.Observations {
		text := strings.TrimSpace(b.Text)
		if text == "" {
			continue
		}
		narrative.Observations = append(narrative.Observations, reporting.NarrativeBullet{
			Text:     text,
			Severity: normalizeBulletSeverity(b.Severity),
		})
	}
	for _, r := range parsed.Recommendations {
		r = strings.TrimSpace(r)
		if r != "" {
			narrative.Recommendations = append(narrative.Recommendations, r)
		}
	}
	if narrative.HealthStatus == "" || (len(narrative.Observations) == 0 && len(narrative.Recommendations) == 0) {
		return reporting.Narrative{}, errors.New("provider returned empty or invalid narrative")
	}
	return narrative, nil
}

func buildReportNarratorPayload(in reporting.NarrativeInput) reportNarratorPayload {
	payload := reportNarratorPayload{
		Title:        in.Title,
		ResourceType: in.ResourceType,
		ResourceID:   in.ResourceID,
		Period: reportNarratorPeriod{
			Start: in.Period.Start.UTC().Format("2006-01-02T15:04:05Z"),
			End:   in.Period.End.UTC().Format("2006-01-02T15:04:05Z"),
			Hours: int(in.Period.End.Sub(in.Period.Start).Hours()),
		},
		MetricStats: convertMetricStats(in.MetricStats),
	}

	if in.Resource != nil {
		payload.Resource = &reportNarratorResource{
			Name:        in.Resource.Name,
			DisplayName: in.Resource.DisplayName,
			Status:      in.Resource.Status,
			Node:        in.Resource.Node,
			CPUCores:    in.Resource.CPUCores,
		}
		if in.Resource.Uptime > 0 {
			payload.Resource.UptimeDays = in.Resource.Uptime / 86400
		}
		if in.Resource.MemoryTotal > 0 {
			payload.Resource.MemoryGB = float64(in.Resource.MemoryTotal) / (1024 * 1024 * 1024)
		}
	}

	if in.PriorPeriod != nil {
		payload.PriorPeriod = &reportNarratorPriorPeriod{
			Start:       in.PriorPeriod.Period.Start.UTC().Format("2006-01-02T15:04:05Z"),
			End:         in.PriorPeriod.Period.End.UTC().Format("2006-01-02T15:04:05Z"),
			MetricStats: convertMetricStats(in.PriorPeriod.MetricStats),
		}
	}

	for _, alert := range in.Alerts {
		payload.Alerts = append(payload.Alerts, reportNarratorAlert{
			Type:                 alert.Type,
			Level:                alert.Level,
			Message:              alert.Message,
			Value:                alert.Value,
			Threshold:            alert.Threshold,
			Resolved:             alert.Recovered(),
			Resolution:           alert.ResolutionSummary(),
			SuccessorAlertListed: alert.SuccessorAlertListed(in.Alerts),
		})
	}

	for _, st := range in.Storage {
		payload.Storage = append(payload.Storage, reportNarratorStorage{
			Name:      st.Name,
			Type:      st.Type,
			UsagePerc: st.UsagePerc,
			ZFSHealth: st.ZFSHealth,
		})
	}

	for _, disk := range in.Disks {
		payload.Disks = append(payload.Disks, reportNarratorDisk{
			Device:    disk.Device,
			Type:      disk.Type,
			Health:    disk.Health,
			WearLevel: disk.WearLevel,
		})
	}

	for _, f := range in.Findings {
		payload.Findings = append(payload.Findings, reportNarratorFindingEntry{
			Severity:       f.Severity,
			Category:       f.Category,
			Title:          f.Title,
			Description:    f.Description,
			Recommendation: f.Recommendation,
			Resolved:       f.Resolved,
		})
	}

	return payload
}

func convertMetricStats(stats map[string]reporting.MetricStats) map[string]reportNarratorMS {
	if len(stats) == 0 {
		return nil
	}
	out := make(map[string]reportNarratorMS, len(stats))
	for k, v := range stats {
		out[k] = reportNarratorMS{
			Min:     v.Min,
			Max:     v.Max,
			Avg:     v.Avg,
			Current: v.Current,
			Count:   v.Count,
		}
	}
	return out
}

// parseReportNarratorResponse strips an optional code fence and unmarshals
// the JSON envelope. Models occasionally wrap the response despite the
// "no markdown fences" instruction; tolerate it rather than fall through to
// the heuristic for a stylistic miss.
func parseReportNarratorResponse(raw string) (reportNarratorResponse, error) {
	trimmed := strings.TrimSpace(raw)
	if strings.HasPrefix(trimmed, "```") {
		trimmed = strings.TrimPrefix(trimmed, "```")
		// drop optional language tag like "json"
		if newline := strings.IndexByte(trimmed, '\n'); newline >= 0 {
			trimmed = trimmed[newline+1:]
		}
		if idx := strings.LastIndex(trimmed, "```"); idx >= 0 {
			trimmed = trimmed[:idx]
		}
		trimmed = strings.TrimSpace(trimmed)
	}
	var parsed reportNarratorResponse
	if err := json.Unmarshal([]byte(trimmed), &parsed); err != nil {
		return reportNarratorResponse{}, fmt.Errorf("decode narrative JSON: %w", err)
	}
	return parsed, nil
}

func normalizeReportHealthStatus(raw string) string {
	switch strings.ToUpper(strings.TrimSpace(raw)) {
	case "HEALTHY":
		return "HEALTHY"
	case "WARNING":
		return "WARNING"
	case "CRITICAL":
		return "CRITICAL"
	default:
		return ""
	}
}

func normalizeBulletSeverity(raw string) string {
	switch strings.ToLower(strings.TrimSpace(raw)) {
	case reporting.NarrativeSeverityCritical, "high", "danger":
		return reporting.NarrativeSeverityCritical
	case reporting.NarrativeSeverityWarning, "medium":
		return reporting.NarrativeSeverityWarning
	case reporting.NarrativeSeverityInfo:
		return reporting.NarrativeSeverityInfo
	case reporting.NarrativeSeverityOK, "good", "healthy":
		return reporting.NarrativeSeverityOK
	default:
		return reporting.NarrativeSeverityInfo
	}
}