Skip to content

The only tool you need to know what is happening and how to fix it.

fix: emails cleanup

narrow-adventuref69ad0db76e5
diff --git a/CLAUDE.md b/CLAUDE.md
index c4b1e732..80ffb200 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -263,6 +263,9 @@ DUCKDB_CHECKPOINT_THRESHOLD= # e.g. 256MB. Unset = DuckDB default (16MB
INGEST_MAX_CONCURRENT= # max concurrently processed ingest requests. Unset = 2×CPU cores, min 4. Bounds ingest memory so overload sheds load with 503s instead of the process being OOM-killed (on DuckDB an OOM death is followed by a minutes-long WAL-replay stall on restart).
INGEST_ADMISSION_WAIT_SECONDS=5 # how long a request may wait for a slot before the 503 + Retry-After; 0 = reject immediately when saturated
+# Email
+EMAIL_PREVIEW_ENABLED=false # "true" registers GET /api/email-preview[/:template], rendering every email template with sample data for design review. Off by default; never enable in production.
+
# Notifications
NOTIFICATION_POLL_SECONDS=60 # polled rule evaluation interval; minimum 5, invalid values fall back to 60
ONCALL_POLL_SECONDS=30 # on-call escalation worker interval; minimum 5, invalid values fall back to 30. Kept separate from NOTIFICATION_POLL_SECONDS so raising rule-evaluation intervals never delays paging. A buffered Wake() channel makes freshly opened pages notify L1 near-instantly regardless of this interval.
@@ -1212,15 +1215,17 @@ if err != nil {
- **Validation errors** (user-facing): `c.JSON(422, gin.H{"error": "message"})` for form validation
- **Always** wrap errors with `traceway.NewStackTraceErrorf` or `fmt.Errorf` using `%w` — never discard the original error
-### Notification Email Templates (HTML)
+### Emails
+
+Every email Traceway sends is a hardcoded template in `backend/app/services/emailtemplates/` (embedded with `go:embed`, parsed once at init). The copy of an email lives in its own `.gohtml` file; the payload is the only thing it interpolates. There is no shared layout document, so changing one email cannot reshape the others, and every file is a complete HTML document styled like the rest.
-Notification/alert/monitor emails are sent as styled HTML through one shared layout; only the email adapter renders HTML, every other channel stays plaintext.
+Templates: `new_error`, `error_regression`, `check_down`, `check_recovered`, `alert` (the threshold email every rate/latency/apdex/throughput/task/impact/cost rule sends), `ai_flagged`, `page` (on-call escalation), `test` (both test buttons), `invitation`, `password_reset`.
-- **`backend/app/services/emailtemplate/`** — the layout (`base.gohtml`, embedded via `go:embed`, parsed once with `template.Must`) and `Render(Data)`. `Data` fields: `Title`, `Badge`/`BadgeColor` (severity chip; `ColorCritical`/`ColorWarning`/`ColorInfo`), `Paragraphs` (internal `\n` becomes `<br>` via the `nl2br` func), `Details` (label/value rows, monospace values), `CodeBlock` (stack traces, probe errors), `Button` (always the dashboard's outline style — white background, `#d1d9e0` border; severity never colors the button, only the badge), `FooterNote`, `LogoURL` (`LogoURL(baseURL)` = `{APP_BASE_URL}/traceway-mark.png`), `Preheader` (defaults to the first paragraph). Table-based markup with inline styles for Gmail/Outlook; all content is escaped by `html/template`.
-- **`emailtemplate.BuildMIME(from, to, subject, textBody, htmlBody)`** — composes `multipart/alternative` (plaintext part first, then HTML), quoted-printable both parts, RFC 2047 subject. Empty `htmlBody` yields a single-part plaintext message. Used by the notifications email adapter; the invitation/password-reset emails in `services/email.service.go` are deliberately still plaintext.
-- **Structured message fields** (`models.NotificationMessage`): optional `Intro`, `Details []NotificationMessageDetail`, `CodeBlock`, `ActionLabel`, all `json:",omitempty"`. When any of `Intro`/`Details`/`CodeBlock` is set, `renderMessageHTML` in `adapter_email.go` renders them instead of `Body`; otherwise `Body` is split into paragraphs on blank lines. `Body` must therefore always stay complete on its own — Slack/Telegram/SMS/webhook and `fired_notifications` only ever see `Body`. The fields round-trip through the outbox (the whole `Message` is persisted as JSON), so no migration is needed when adding more; rows enqueued before an upgrade fall back to the generic rendering.
-- **Buttons/links**: relative `msg.URL`s are absolutized against `services.EmailService.BaseURL()`; `ActionLabel` overrides the default "View in Traceway" (builders use "View Issue", "View Monitor", "Acknowledge Page"). `msg.HTMLBody`, when set, bypasses the template entirely.
-- **Adding a new rule type's email**: nothing to do for a plain sentence (generic path). For richer emails, set `Intro`/`Details`/`CodeBlock`/`ActionLabel` on the `Message` in `messages.go` while keeping `Body` self-contained (see `buildNewErrorMessage` / `buildCheckDownMessage`).
+- **One send path** — `services.SendEmail(ctx, services.Email{...})` in `backend/app/services/email.service.go` is the only way mail leaves the process: it renders the template, wraps it as `multipart/alternative` (plaintext part first, quoted-printable), and talks SMTP. With `SMTP_ENABLED` off it logs the plaintext instead. `services.RenderEmail` is the render-only half, used by the preview endpoint.
+- **`services.Email`** carries the chrome every template shares (`Title`, `Badge`/`BadgeColor` via `EmailColor*`, `URL`, `Footer`, `LogoURL` defaulting to `{APP_BASE_URL}/traceway-mark.png`) plus `Template` and `Data`, the typed payload reached as `{{.Data.X}}`. `Text` is the plaintext alternative and must stand on its own.
+- **Notification payloads** live on `models.NotificationMessage.Email` (`models.NotificationEmail`: a template name plus one typed struct — `EmailException`, `EmailCheck`, `EmailAlert`, `EmailFlagged`, `EmailPage`, `EmailTest`). The builders in `notifications/messages.go` attach it; `services.NotificationEmail` turns a message into the `Email` at send time (severity chip, absolute link, "why you got this" footer). It rides through the notification outbox as JSON, so the shape is a wire format: add fields, never repurpose them. Only the email adapter reads it — Slack/Telegram/SMS/webhook and `fired_notifications` still see `Body` alone, so `Body` must always stay complete.
+- **Adding an email**: add `emailtemplates/<name>.gohtml`, add its payload struct, and call `SendEmail` with `Template: "<name>"`. For a new rule type that only needs a sentence and a link, reuse `alert` by filling `models.EmailAlert`.
+- **Preview**: `EMAIL_PREVIEW_ENABLED=true` registers `GET /api/email-preview` (index) and `GET /api/email-preview/:template`, which renders any template with sample data (`?format=text` shows the plaintext alternative). Off by default, so it is never a route in production. Sample data lives in `controllers/email_preview.controller.go`.
---
diff --git a/backend/app/config/config.go b/backend/app/config/config.go
index b6389c99..bee7cb9a 100644
--- a/backend/app/config/config.go
+++ b/backend/app/config/config.go
@@ -84,6 +84,7 @@ type Cfg struct {
SMTPFrom string
AppBaseURL string
+ EmailPreviewEnabled string
CloudMode string
MonitoringTracewayURL string
APIOnly string
@@ -221,6 +222,7 @@ func LoadFromEnv() *Cfg {
SMTPFrom: os.Getenv("SMTP_FROM"),
AppBaseURL: os.Getenv("APP_BASE_URL"),
+ EmailPreviewEnabled: os.Getenv("EMAIL_PREVIEW_ENABLED"),
CloudMode: os.Getenv("CLOUD_MODE"),
MonitoringTracewayURL: os.Getenv("MONITORING_TRACEWAY_URL"),
APIOnly: os.Getenv("API_ONLY"),
diff --git a/backend/app/controllers/contact_method.controller.go b/backend/app/controllers/contact_method.controller.go
index a76aefa1..f5351083 100644
--- a/backend/app/controllers/contact_method.controller.go
+++ b/backend/app/controllers/contact_method.controller.go
@@ -405,13 +405,7 @@ func (c *contactMethodController) Test(ctx *gin.Context) {
ctx.JSON(http.StatusUnprocessableEntity, gin.H{"error": err.Error()})
return
}
- testMsg := notifications.Message{
- Subject: "Traceway on-call test",
- Body: "This is a test of your on-call contact method. If you received this, pages will reach you here.",
- Severity: notifications.SeverityInfo,
- RuleType: "test",
- RuleName: "Test",
- }
+ testMsg := notifications.TestContactMethodMessage()
if err := adapter.Send(ctx.Request.Context(), testMsg); err != nil {
ctx.JSON(http.StatusUnprocessableEntity, gin.H{"error": "Test notification failed: " + err.Error()})
return
diff --git a/backend/app/controllers/email_preview.controller.go b/backend/app/controllers/email_preview.controller.go
new file mode 100644
index 00000000..f4a810b7
--- /dev/null
+++ b/backend/app/controllers/email_preview.controller.go
@@ -0,0 +1,177 @@
+package controllers
+
+import (
+ "fmt"
+ "html"
+ "net/http"
+ "sort"
+
+ "github.com/gin-gonic/gin"
+ "github.com/tracewayapp/traceway/backend/app/models"
+ "github.com/tracewayapp/traceway/backend/app/services"
+)
+
+type emailPreviewController struct{}
+
+var EmailPreviewController = &emailPreviewController{}
+
+func emailPreviews() map[string]services.Email {
+ return map[string]services.Email{
+ "invitation": {
+ Subject: "You've been invited to join Acme Inc on Traceway",
+ Title: "Join Acme Inc on Traceway",
+ URL: "https://traceway.example.com/accept-invitation?token=sample",
+ Footer: "This invitation expires in 7 days. If you did not expect it, you can safely ignore this email.",
+ Data: map[string]string{"InviterName": "Dana Whitfield", "OrgName": "Acme Inc"},
+ },
+ "password_reset": {
+ Subject: "Reset your Traceway password",
+ Title: "Reset your password",
+ URL: "https://traceway.example.com/reset-password?token=sample",
+ Footer: "This link expires in 1 hour. If you did not ask for a reset, you can safely ignore this email and your password stays unchanged.",
+ },
+ "new_error": exceptionPreview("[Acme Production] New error: *fmt.wrapError"),
+ "error_regression": exceptionPreview("[Acme Production] Resolved error reappeared: *fmt.wrapError"),
+ "check_down": {
+ Subject: "[CRITICAL] [Acme Production] Check \"Checkout API\" is down",
+ Title: "[Acme Production] Check \"Checkout API\" is down",
+ Badge: "CRITICAL",
+ BadgeColor: services.EmailColorCritical,
+ URL: "https://traceway.example.com/monitors/7",
+ Footer: "You are receiving this because the notification rule \"Monitor alerts\" fired.",
+ Data: &models.NotificationEmail{Check: &models.EmailCheck{
+ ProjectName: "Acme Production",
+ CheckName: "Checkout API",
+ ConsecutiveFailures: 3,
+ LastError: "dial tcp 10.0.0.4:443: i/o timeout",
+ }},
+ },
+ "check_recovered": {
+ Subject: "[INFO] [Acme Production] Check \"Checkout API\" recovered",
+ Title: "[Acme Production] Check \"Checkout API\" recovered",
+ Badge: "INFO",
+ BadgeColor: services.EmailColorInfo,
+ URL: "https://traceway.example.com/monitors/7",
+ Footer: "You are receiving this because the notification rule \"Monitor alerts\" fired.",
+ Data: &models.NotificationEmail{Check: &models.EmailCheck{
+ ProjectName: "Acme Production",
+ CheckName: "Checkout API",
+ }},
+ },
+ "alert": {
+ Subject: "[WARNING] [Acme Production] P95 latency 2480ms on GET /api/users/:id",
+ Title: "[Acme Production] P95 latency 2480ms on GET /api/users/:id",
+ Badge: "WARNING",
+ BadgeColor: services.EmailColorWarning,
+ URL: "https://traceway.example.com/endpoints",
+ Footer: "You are receiving this because the notification rule \"Latency watch\" fired.",
+ Data: &models.NotificationEmail{Alert: &models.EmailAlert{
+ ProjectName: "Acme Production",
+ Headline: "The P95 latency for GET /api/users/:id has reached 2480ms over the last 15 minutes (threshold: 800ms).",
+ ScopeLabel: "Endpoint",
+ Scope: "GET /api/users/:id",
+ Observed: "P95 2480ms",
+ Threshold: "800ms",
+ WindowMins: 15,
+ }},
+ },
+ "ai_flagged": {
+ Subject: "[WARNING] [Acme Production] AI conversation flagged: password, ssn",
+ Title: "[Acme Production] AI conversation flagged: password, ssn",
+ Badge: "WARNING",
+ BadgeColor: services.EmailColorWarning,
+ URL: "https://traceway.example.com/ai-traces/conversations",
+ Footer: "You are receiving this because the notification rule \"AI content review\" fired.",
+ Data: &models.NotificationEmail{Flagged: &models.EmailFlagged{
+ ProjectName: "Acme Production",
+ ConversationId: "conv_7f21ac",
+ UserId: "8412",
+ Terms: []string{"password", "ssn"},
+ }},
+ },
+ "page": {
+ Subject: "[CRITICAL] [Page] [Acme Production] New error: *fmt.wrapError",
+ Title: "[Page] [Acme Production] New error: *fmt.wrapError",
+ Badge: "CRITICAL",
+ BadgeColor: services.EmailColorCritical,
+ URL: "https://traceway.example.com/ack/twk_sample",
+ Data: &models.NotificationEmail{Page: &models.EmailPage{
+ BodyText: "A new error has been detected: *fmt.wrapError\n\nException ID: 01930f8c-2b1a-7f3e-9c4d-5a6b7c8d9e0f\nHash: 9c7bc73da37328f2\nEndpoint: GET /api/users/:id\n\nStack trace:\nruntime error: invalid memory address or nil pointer dereference\n\tgithub.com/acme/api/handlers.(*UserHandler).Get\n\t\t/app/handlers/user.go:142",
+ RuleName: "Critical errors",
+ EventCount: 4,
+ }},
+ },
+ "test": {
+ Subject: "[INFO] Traceway Test Notification",
+ Title: "Traceway Test Notification",
+ Badge: "INFO",
+ BadgeColor: services.EmailColorInfo,
+ Data: &models.NotificationEmail{Test: &models.EmailTest{Target: "notification channel"}},
+ },
+ }
+}
+
+func exceptionPreview(title string) services.Email {
+ return services.Email{
+ Subject: "[CRITICAL] " + title,
+ Title: title,
+ Badge: "CRITICAL",
+ BadgeColor: services.EmailColorCritical,
+ URL: "https://traceway.example.com/issues/9c7bc73da37328f2",
+ Footer: "You are receiving this because the notification rule \"Critical errors\" fired.",
+ Data: &models.NotificationEmail{Exception: &models.EmailException{
+ ProjectName: "Acme Production",
+ ErrorType: "*fmt.wrapError",
+ ExceptionId: "01930f8c-2b1a-7f3e-9c4d-5a6b7c8d9e0f",
+ Hash: "9c7bc73da37328f2",
+ OccurredAt: "2026-08-24 09:41:12 UTC",
+ AppVersion: "1.9.19",
+ ServerName: "api-blue-3",
+ TraceLabel: "Endpoint",
+ TraceName: "GET /api/users/:id",
+ Attributes: []models.EmailField{{Label: "tenant", Value: "acme-prod"}, {Label: "user_id", Value: "8412"}},
+ StackTrace: "runtime error: invalid memory address or nil pointer dereference\n\tgithub.com/acme/api/handlers.(*UserHandler).Get\n\t\t/app/handlers/user.go:142\n\tgithub.com/acme/api/store.(*Users).ByID\n\t\t/app/store/users.go:88",
+ }},
+ }
+}
+
+func (c *emailPreviewController) Index(ctx *gin.Context) {
+ previews := emailPreviews()
+ names := make([]string, 0, len(previews))
+ for name := range previews {
+ names = append(names, name)
+ }
+ sort.Strings(names)
+
+ page := `<!DOCTYPE html><html><head><meta charset="utf-8"><title>Traceway email previews</title>` +
+ `<style>body{font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;background:#f6f8fa;color:#1f2328;padding:32px}` +
+ `a{color:#0969da;text-decoration:none}li{margin:8px 0}code{color:#59636e}</style></head><body><h1>Email previews</h1><ul>`
+ for _, name := range names {
+ page += fmt.Sprintf(`<li><a href="/api/email-preview/%s">%s</a> <code>%s</code></li>`,
+ name, name, html.EscapeString(previews[name].Subject))
+ }
+ page += `</ul></body></html>`
+ ctx.Data(http.StatusOK, "text/html; charset=utf-8", []byte(page))
+}
+
+func (c *emailPreviewController) Show(ctx *gin.Context) {
+ name := ctx.Param("template")
+ preview, ok := emailPreviews()[name]
+ if !ok {
+ ctx.String(http.StatusNotFound, "no email template named %q", name)
+ return
+ }
+ preview.Template = name
+
+ if ctx.Query("format") == "text" {
+ ctx.Data(http.StatusOK, "text/plain; charset=utf-8", []byte("Subject: "+preview.Subject+"\n\n"+preview.Text))
+ return
+ }
+
+ rendered, err := services.RenderEmail(preview)
+ if err != nil {
+ ctx.String(http.StatusInternalServerError, "%v", err)
+ return
+ }
+ ctx.Data(http.StatusOK, "text/html; charset=utf-8", []byte(rendered))
+}
diff --git a/backend/app/controllers/notification_channel.controller.go b/backend/app/controllers/notification_channel.controller.go
index d5650ee8..50669b21 100644
--- a/backend/app/controllers/notification_channel.controller.go
+++ b/backend/app/controllers/notification_channel.controller.go
@@ -303,13 +303,7 @@ func (ctrl *notificationChannelController) Test(ctx *gin.Context) {
return
}
- testMsg := notifications.Message{
- Subject: "Traceway Test Notification",
- Body: "This is a test notification from Traceway. If you received this, your notification channel is configured correctly.",
- Severity: notifications.SeverityInfo,
- RuleType: "test",
- RuleName: "Test",
- }
+ testMsg := notifications.TestChannelMessage()
if err := adapter.Send(ctx.Request.Context(), testMsg); err != nil {
ctx.JSON(http.StatusUnprocessableEntity, gin.H{"error": "Test notification failed: " + err.Error()})
diff --git a/backend/app/controllers/routes.go b/backend/app/controllers/routes.go
index e81d64f4..60dd0229 100644
--- a/backend/app/controllers/routes.go
+++ b/backend/app/controllers/routes.go
@@ -82,6 +82,11 @@ func RegisterControllers(router *gin.RouterGroup) {
router.GET("/health/deep", middleware.UseHealthDeepToken, HealthDeepController.Get)
+ if config.Config.EmailPreviewEnabled == "true" {
+ router.GET("/email-preview", EmailPreviewController.Index)
+ router.GET("/email-preview/:template", EmailPreviewController.Show)
+ }
+
router.POST("/stats", middleware.UseAppAuth, middleware.RequireProjectAccess, MetricRecordController.FindHomepageStats)
router.GET("/dashboard", middleware.UseAppAuth, middleware.RequireProjectAccess, DashboardController.GetDashboard)
router.GET("/dashboard/overview", middleware.UseAppAuth, middleware.RequireProjectAccess, DashboardController.GetDashboardOverview)
diff --git a/backend/app/models/notification_message.model.go b/backend/app/models/notification_message.model.go
index 0e3992f3..77c59c4c 100644
--- a/backend/app/models/notification_message.model.go
+++ b/backend/app/models/notification_message.model.go
@@ -21,14 +21,7 @@ type NotificationMessage struct {
URL string
Endpoint string
- // Optional structured presentation for HTML-capable adapters (email).
- // When any of Intro/Details/CodeBlock is set, the email adapter renders
- // them instead of the plaintext Body; every other adapter keeps using
- // Body, so builders must keep Body complete on its own.
- Intro string `json:",omitempty"`
- Details []NotificationMessageDetail `json:",omitempty"`
- CodeBlock string `json:",omitempty"`
- ActionLabel string `json:",omitempty"`
+ Email *NotificationEmail `json:",omitempty"`
// DedupToken is the stable identity of what fired within the rule
// (exception hash, endpoint, task or metric name; empty for rule-level
@@ -38,9 +31,77 @@ type NotificationMessage struct {
DedupToken string `json:"-"`
}
-// NotificationMessageDetail is one label/value row rendered as a details
-// table in HTML emails (exception id, hash, server, ...).
-type NotificationMessageDetail struct {
+const (
+ EmailTemplateNewError = "new_error"
+ EmailTemplateErrorRegression = "error_regression"
+ EmailTemplateCheckDown = "check_down"
+ EmailTemplateCheckRecovered = "check_recovered"
+ EmailTemplateAlert = "alert"
+ EmailTemplateAiFlagged = "ai_flagged"
+ EmailTemplatePage = "page"
+ EmailTemplateTest = "test"
+)
+
+type NotificationEmail struct {
+ Template string
+ Exception *EmailException `json:",omitempty"`
+ Check *EmailCheck `json:",omitempty"`
+ Alert *EmailAlert `json:",omitempty"`
+ Flagged *EmailFlagged `json:",omitempty"`
+ Page *EmailPage `json:",omitempty"`
+ Test *EmailTest `json:",omitempty"`
+}
+
+type EmailField struct {
Label string
Value string
}
+
+type EmailException struct {
+ ProjectName string
+ ErrorType string
+ ExceptionId string
+ Hash string
+ OccurredAt string
+ AppVersion string
+ ServerName string
+ TraceLabel string
+ TraceName string
+ Attributes []EmailField `json:",omitempty"`
+ StackTrace string
+}
+
+type EmailCheck struct {
+ ProjectName string
+ CheckName string
+ ConsecutiveFailures int
+ LastError string
+}
+
+type EmailAlert struct {
+ ProjectName string
+ Headline string
+ ScopeLabel string
+ Scope string
+ Observed string
+ Threshold string
+ WindowMins int
+}
+
+type EmailFlagged struct {
+ ProjectName string
+ ConversationId string
+ UserId string
+ Terms []string
+}
+
+type EmailPage struct {
+ BodyText string
+ RuleName string
+ EventCount int
+ EscalationLevel int
+}
+
+type EmailTest struct {
+ Target string
+}
diff --git a/backend/app/notifications/adapter_email.go b/backend/app/notifications/adapter_email.go
index cc3d1fa5..3b1b4348 100644
--- a/backend/app/notifications/adapter_email.go
+++ b/backend/app/notifications/adapter_email.go
@@ -2,20 +2,13 @@ package notifications
import (
"context"
- "crypto/tls"
"fmt"
- "net"
- "net/smtp"
"strings"
- "time"
- "github.com/tracewayapp/traceway/backend/app/config"
+ "github.com/tracewayapp/traceway/backend/app/models"
"github.com/tracewayapp/traceway/backend/app/services"
- "github.com/tracewayapp/traceway/backend/app/services/emailtemplate"
)
-const smtpTimeout = 10 * time.Second
-
type EmailAdapter struct {
Recipients []string `json:"recipients"`
}
@@ -38,154 +31,8 @@ func (a *EmailAdapter) Validate() error {
}
func (a *EmailAdapter) Send(ctx context.Context, msg Message) error {
- emailSvc := services.EmailService
- if emailSvc == nil {
- return fmt.Errorf("email service not initialized")
- }
-
- prefix := ""
- switch msg.Severity {
- case SeverityCritical:
- prefix = "[CRITICAL] "
- case SeverityWarning:
- prefix = "[WARNING] "
- case SeverityInfo:
- prefix = "[INFO] "
+ if msg.Email == nil {
+ msg.Email = &models.NotificationEmail{Template: models.EmailTemplateAlert, Alert: &models.EmailAlert{Headline: msg.Body}}
}
-
- subject := prefix + msg.Subject
-
- if !emailSvc.IsEnabled() {
- config.Logf("[EMAIL LOG] To: %s\nSubject: %s\nBody:\n%s", strings.Join(a.Recipients, ", "), subject, msg.Body)
- return nil
- }
-
- cfg := config.Config
- from := cfg.SMTPFrom
- // No username means an auth-less relay; smtp.PlainAuth would refuse to
- // run over an unencrypted connection anyway.
- var auth smtp.Auth
- if cfg.SMTPUsername != "" {
- auth = smtp.PlainAuth("", cfg.SMTPUsername, cfg.SMTPPassword, cfg.SMTPHost)
- }
- addr := fmt.Sprintf("%s:%s", cfg.SMTPHost, cfg.SMTPPort)
-
- htmlBody := msg.HTMLBody
- if htmlBody == "" {
- var err error
- htmlBody, err = renderMessageHTML(msg, emailSvc.BaseURL())
- if err != nil {
- return fmt.Errorf("failed to render email HTML: %w", err)
- }
- }
- emailMsg, err := emailtemplate.BuildMIME(from, a.Recipients, subject, msg.Body, htmlBody)
- if err != nil {
- return fmt.Errorf("failed to build email message: %w", err)
- }
-
- return sendMailWithTimeout(ctx, addr, auth, from, a.Recipients, emailMsg)
-}
-
-// renderMessageHTML renders any notification Message through the shared email
-// layout. Structured fields (Intro/Details/CodeBlock) win over the plaintext
-// Body, which stays the source for every text-only adapter.
-func renderMessageHTML(msg Message, baseURL string) (string, error) {
- d := emailtemplate.Data{
- LogoURL: emailtemplate.LogoURL(baseURL),
- Title: msg.Subject,
- }
-
- switch msg.Severity {
- case SeverityCritical:
- d.Badge, d.BadgeColor = "CRITICAL", emailtemplate.ColorCritical
- case SeverityWarning:
- d.Badge, d.BadgeColor = "WARNING", emailtemplate.ColorWarning
- case SeverityInfo:
- d.Badge, d.BadgeColor = "INFO", emailtemplate.ColorInfo
- }
-
- if msg.Intro != "" || len(msg.Details) > 0 || msg.CodeBlock != "" {
- if msg.Intro != "" {
- d.Paragraphs = []string{msg.Intro}
- }
- for _, det := range msg.Details {
- d.Details = append(d.Details, emailtemplate.Detail{Label: det.Label, Value: det.Value})
- }
- d.CodeBlock = msg.CodeBlock
- } else if body := strings.TrimSpace(msg.Body); body != "" {
- d.Paragraphs = strings.Split(body, "\n\n")
- }
-
- if msg.URL != "" {
- u := msg.URL
- if strings.HasPrefix(u, "/") {
- u = strings.TrimRight(baseURL, "/") + u
- }
- label := msg.ActionLabel
- if label == "" {
- label = "View in Traceway"
- }
- d.Button = &emailtemplate.Button{Label: label, URL: u}
- }
-
- if msg.RuleName != "" {
- d.FooterNote = fmt.Sprintf("You are receiving this because the notification rule %q fired.", msg.RuleName)
- }
-
- return emailtemplate.Render(d)
-}
-
-func sendMailWithTimeout(ctx context.Context, addr string, auth smtp.Auth, from string, to []string, msg []byte) error {
- dialer := net.Dialer{Timeout: smtpTimeout}
- conn, err := dialer.DialContext(ctx, "tcp", addr)
- if err != nil {
- return fmt.Errorf("SMTP dial failed: %w", err)
- }
-
- // Must be set before smtp.NewClient, which blocks reading the server greeting.
- deadline := time.Now().Add(smtpTimeout)
- if ctxDeadline, ok := ctx.Deadline(); ok && ctxDeadline.Before(deadline) {
- deadline = ctxDeadline
- }
- conn.SetDeadline(deadline)
-
- host, _, _ := net.SplitHostPort(addr)
- client, err := smtp.NewClient(conn, host)
- if err != nil {
- conn.Close()
- return fmt.Errorf("SMTP client failed: %w", err)
- }
- defer client.Close()
-
- if ok, _ := client.Extension("STARTTLS"); ok {
- if err := client.StartTLS(&tls.Config{ServerName: host}); err != nil {
- return fmt.Errorf("SMTP STARTTLS failed: %w", err)
- }
- }
-
- if auth != nil {
- if err := client.Auth(auth); err != nil {
- return fmt.Errorf("SMTP auth failed: %w", err)
- }
- }
- if err := client.Mail(from); err != nil {
- return fmt.Errorf("SMTP MAIL failed: %w", err)
- }
- for _, r := range to {
- if err := client.Rcpt(r); err != nil {
- return fmt.Errorf("SMTP RCPT failed: %w", err)
- }
- }
- w, err := client.Data()
- if err != nil {
- return fmt.Errorf("SMTP DATA failed: %w", err)
- }
- if _, err := w.Write(msg); err != nil {
- return fmt.Errorf("SMTP write failed: %w", err)
- }
- if err := w.Close(); err != nil {
- return fmt.Errorf("SMTP close data failed: %w", err)
- }
- client.Quit()
- return nil
+ return services.SendEmail(ctx, services.NotificationEmail(msg, a.Recipients))
}
diff --git a/backend/app/notifications/messages.go b/backend/app/notifications/messages.go
index 8687663e..68b84976 100644
--- a/backend/app/notifications/messages.go
+++ b/backend/app/notifications/messages.go
@@ -9,16 +9,28 @@ import (
"github.com/tracewayapp/traceway/backend/app/models"
)
+func alertEmail(alert models.EmailAlert) *models.NotificationEmail {
+ return &models.NotificationEmail{Template: models.EmailTemplateAlert, Alert: &alert}
+}
+
func buildErrorRateMessage(rate float64, threshold float64, window int, projectName string) Message {
severity := SeverityWarning
if rate >= threshold*2 {
severity = SeverityCritical
}
+ headline := fmt.Sprintf("The error rate has reached %.1f%% over the last %d minutes (threshold: %.1f%%).", rate, window, threshold)
return Message{
Subject: fmt.Sprintf("[%s] Error rate %.1f%% exceeds %.1f%%", projectName, rate, threshold),
- Body: fmt.Sprintf("The error rate has reached %.1f%% over the last %d minutes (threshold: %.1f%%).", rate, window, threshold),
+ Body: headline,
Severity: severity,
URL: "/issues?preset=1h",
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ Observed: fmt.Sprintf("%.1f%% of requests", rate),
+ Threshold: fmt.Sprintf("%.1f%%", threshold),
+ WindowMins: window,
+ }),
}
}
@@ -29,12 +41,22 @@ func endpointTimeRangeURL(now time.Time) string {
}
func buildEndpointLatencyMessage(percentile string, latencyMs float64, thresholdMs float64, endpoint string, window int, projectName string) Message {
+ headline := fmt.Sprintf("The %s latency for %s has reached %.0fms over the last %d minutes (threshold: %.0fms).", percentile, endpoint, latencyMs, window, thresholdMs)
return Message{
Subject: fmt.Sprintf("[%s] %s latency %.0fms on %s", projectName, percentile, latencyMs, endpoint),
- Body: fmt.Sprintf("The %s latency for %s has reached %.0fms over the last %d minutes (threshold: %.0fms).", percentile, endpoint, latencyMs, window, thresholdMs),
+ Body: headline,
Severity: SeverityWarning,
URL: endpointTimeRangeURL(time.Now()),
DedupToken: endpoint,
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ ScopeLabel: "Endpoint",
+ Scope: endpoint,
+ Observed: fmt.Sprintf("%s %.0fms", percentile, latencyMs),
+ Threshold: fmt.Sprintf("%.0fms", thresholdMs),
+ WindowMins: window,
+ }),
}
}
@@ -43,11 +65,19 @@ func buildApdexDropMessage(apdex float64, threshold float64, total int64, window
if apdex < 0.5 {
severity = SeverityCritical
}
+ headline := fmt.Sprintf("The Apdex score has dropped to %.2f across %d requests over the last %d minutes (threshold: %.2f).", apdex, total, window, threshold)
return Message{
Subject: fmt.Sprintf("[%s] Apdex dropped to %.2f (threshold: %.2f)", projectName, apdex, threshold),
- Body: fmt.Sprintf("The Apdex score has dropped to %.2f across %d requests over the last %d minutes (threshold: %.2f).", apdex, total, window, threshold),
+ Body: headline,
Severity: severity,
URL: endpointTimeRangeURL(time.Now()),
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ Observed: fmt.Sprintf("%.2f across %d requests", apdex, total),
+ Threshold: fmt.Sprintf("%.2f", threshold),
+ WindowMins: window,
+ }),
}
}
@@ -63,22 +93,40 @@ func buildMetricThresholdMessage(metricName string, value float64, operator stri
if aggregation == "" {
aggregation = "avg"
}
+ headline := fmt.Sprintf("The metric %s has a %s of %.2f over the last %d minutes which violates the threshold %s %.2f.", metricName, aggregation, value, window, operator, threshold)
return Message{
Subject: fmt.Sprintf("[%s] Metric %s is %.2f (threshold: %s %.2f)", projectName, metricName, value, operator, threshold),
- Body: fmt.Sprintf("The metric %s has a %s of %.2f over the last %d minutes which violates the threshold %s %.2f.", metricName, aggregation, value, window, operator, threshold),
+ Body: headline,
Severity: severity,
URL: "/metrics?preset=1h",
DedupToken: metricName,
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ ScopeLabel: "Metric",
+ Scope: metricName,
+ Observed: fmt.Sprintf("%s %.2f", aggregation, value),
+ Threshold: fmt.Sprintf("%s %.2f", operator, threshold),
+ WindowMins: window,
+ }),
}
}
func buildNoDataMessage(dataType string, silenceMinutes int, projectName string) Message {
+ headline := fmt.Sprintf("No %s data has been received for the last %d minutes.", dataType, silenceMinutes)
return Message{
Subject: fmt.Sprintf("[%s] No %s data for %d minutes", projectName, dataType, silenceMinutes),
- Body: fmt.Sprintf("No %s data has been received for the last %d minutes.", dataType, silenceMinutes),
+ Body: headline,
Severity: SeverityCritical,
URL: "/",
DedupToken: dataType,
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ ScopeLabel: "Signal",
+ Scope: dataType,
+ Observed: fmt.Sprintf("silent for %d minutes", silenceMinutes),
+ }),
}
}
@@ -87,21 +135,39 @@ func buildErrorCountMessage(count int64, threshold int64, window int, projectNam
if count >= threshold*5 {
severity = SeverityCritical
}
+ headline := fmt.Sprintf("There have been %d errors in the last %d minutes (threshold: %d).", count, window, threshold)
return Message{
Subject: fmt.Sprintf("[%s] %d errors in last %d minutes", projectName, count, window),
- Body: fmt.Sprintf("There have been %d errors in the last %d minutes (threshold: %d).", count, window, threshold),
+ Body: headline,
Severity: severity,
URL: "/issues?preset=1h",
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ Observed: fmt.Sprintf("%d errors", count),
+ Threshold: fmt.Sprintf("%d errors", threshold),
+ WindowMins: window,
+ }),
}
}
func buildTaskDurationMessage(taskName string, p95Ms float64, thresholdMs float64, window int, projectName string) Message {
+ headline := fmt.Sprintf("The task %s P95 duration is %.0fms over the last %d minutes (threshold: %.0fms).", taskName, p95Ms, window, thresholdMs)
return Message{
Subject: fmt.Sprintf("[%s] Task %s P95 %.0fms exceeds %.0fms", projectName, taskName, p95Ms, thresholdMs),
- Body: fmt.Sprintf("The task %s P95 duration is %.0fms over the last %d minutes (threshold: %.0fms).", taskName, p95Ms, window, thresholdMs),
+ Body: headline,
Severity: SeverityWarning,
URL: "/tasks?preset=1h",
DedupToken: taskName,
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ ScopeLabel: "Task",
+ Scope: taskName,
+ Observed: fmt.Sprintf("P95 %.0fms", p95Ms),
+ Threshold: fmt.Sprintf("%.0fms", thresholdMs),
+ WindowMins: window,
+ }),
}
}
@@ -110,12 +176,22 @@ func buildTaskFailureRateMessage(taskName string, rate float64, threshold float6
if rate >= threshold*2 {
severity = SeverityCritical
}
+ headline := fmt.Sprintf("The task %s failed %d of %d executions (%.1f%%) over the last %d minutes (threshold: %.1f%%).", taskName, failed, total, rate, window, threshold)
return Message{
Subject: fmt.Sprintf("[%s] Task %s failure rate %.1f%% exceeds %.1f%%", projectName, taskName, rate, threshold),
- Body: fmt.Sprintf("The task %s failed %d of %d executions (%.1f%%) over the last %d minutes (threshold: %.1f%%).", taskName, failed, total, rate, window, threshold),
+ Body: headline,
Severity: severity,
URL: "/tasks?preset=1h",
DedupToken: taskName,
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ ScopeLabel: "Task",
+ Scope: taskName,
+ Observed: fmt.Sprintf("%d of %d failed (%.1f%%)", failed, total, rate),
+ Threshold: fmt.Sprintf("%.1f%%", threshold),
+ WindowMins: window,
+ }),
}
}
@@ -124,11 +200,19 @@ func buildThroughputDropMessage(dropPercent float64, current int64, expected flo
if dropPercent >= 80 {
severity = SeverityCritical
}
+ headline := fmt.Sprintf("Request throughput has dropped by %.0f%%: %d requests in the last %d minutes vs an expected %.0f based on the baseline window.", dropPercent, current, window, expected)
return Message{
Subject: fmt.Sprintf("[%s] Throughput dropped %.0f%% vs baseline", projectName, dropPercent),
- Body: fmt.Sprintf("Request throughput has dropped by %.0f%%: %d requests in the last %d minutes vs an expected %.0f based on the baseline window.", dropPercent, current, window, expected),
+ Body: headline,
Severity: severity,
URL: endpointTimeRangeURL(time.Now()),
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ Observed: fmt.Sprintf("%d requests", current),
+ Threshold: fmt.Sprintf("%.0f expected", expected),
+ WindowMins: window,
+ }),
}
}
@@ -137,46 +221,57 @@ func buildEndpointErrorRateMessage(endpoint string, rate float64, threshold floa
if rate >= threshold*2 {
severity = SeverityCritical
}
+ headline := fmt.Sprintf("The endpoint %s has an error rate of %.1f%% (threshold: %.1f%%).", endpoint, rate, threshold)
return Message{
Subject: fmt.Sprintf("[%s] %s error rate %.1f%%", projectName, endpoint, rate),
- Body: fmt.Sprintf("The endpoint %s has an error rate of %.1f%% (threshold: %.1f%%).", endpoint, rate, threshold),
+ Body: headline,
Severity: severity,
URL: endpointTimeRangeURL(time.Now()),
DedupToken: endpoint,
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ ScopeLabel: "Endpoint",
+ Scope: endpoint,
+ Observed: fmt.Sprintf("%.1f%% of requests", rate),
+ Threshold: fmt.Sprintf("%.1f%%", threshold),
+ }),
}
}
-func buildImpactScoreCriticalMessage(endpoint string, score float64, reason string, projectName string) Message {
+func buildImpactScoreMessage(level string, severity Severity, endpoint string, score float64, reason string, projectName string) Message {
+ headline := fmt.Sprintf("The endpoint %s has become %s impact (impact score: %.2f). Reason: %s", endpoint, level, score, reason)
+ if level == "critical" {
+ headline = fmt.Sprintf("The endpoint %s has become critical (impact score: %.2f). Reason: %s", endpoint, score, reason)
+ }
return Message{
- Subject: fmt.Sprintf("[%s] Endpoint %s impact became critical", projectName, endpoint),
- Body: fmt.Sprintf("The endpoint %s has become critical (impact score: %.2f). Reason: %s", endpoint, score, reason),
- Severity: SeverityCritical,
+ Subject: fmt.Sprintf("[%s] Endpoint %s impact became %s", projectName, endpoint, level),
+ Body: headline,
+ Severity: severity,
URL: endpointTimeRangeURL(time.Now()),
Endpoint: endpoint,
DedupToken: endpoint,
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ ScopeLabel: "Endpoint",
+ Scope: endpoint,
+ Observed: fmt.Sprintf("impact score %.2f", score),
+ Threshold: reason,
+ }),
}
}
+func buildImpactScoreCriticalMessage(endpoint string, score float64, reason string, projectName string) Message {
+ return buildImpactScoreMessage("critical", SeverityCritical, endpoint, score, reason, projectName)
+}
+
func buildImpactScoreHighMessage(endpoint string, score float64, reason string, projectName string) Message {
- return Message{
- Subject: fmt.Sprintf("[%s] Endpoint %s impact became high", projectName, endpoint),
- Body: fmt.Sprintf("The endpoint %s has become high impact (impact score: %.2f). Reason: %s", endpoint, score, reason),
- Severity: SeverityWarning,
- URL: endpointTimeRangeURL(time.Now()),
- Endpoint: endpoint,
- DedupToken: endpoint,
- }
+ return buildImpactScoreMessage("high", SeverityWarning, endpoint, score, reason, projectName)
}
func buildImpactScoreMediumMessage(endpoint string, score float64, reason string, projectName string) Message {
- return Message{
- Subject: fmt.Sprintf("[%s] Endpoint %s impact became medium", projectName, endpoint),
- Body: fmt.Sprintf("The endpoint %s has become medium impact (impact score: %.2f). Reason: %s", endpoint, score, reason),
- Severity: SeverityInfo,
- URL: endpointTimeRangeURL(time.Now()),
- Endpoint: endpoint,
- DedupToken: endpoint,
- }
+ return buildImpactScoreMessage("medium", SeverityInfo, endpoint, score, reason, projectName)
}
func formatCostForMessage(cost float64) string {
@@ -191,12 +286,21 @@ func buildAiTraceCostMessage(traceName string, cost float64, threshold float64,
if cost >= threshold*3 {
severity = SeverityCritical
}
+ headline := fmt.Sprintf("The AI trace %q cost %s, exceeding the threshold of %s.", traceName, formatCostForMessage(cost), formatCostForMessage(threshold))
return Message{
Subject: fmt.Sprintf("[%s] AI trace %s cost %s exceeds %s", projectName, traceName, formatCostForMessage(cost), formatCostForMessage(threshold)),
- Body: fmt.Sprintf("The AI trace \"%s\" cost %s, exceeding the threshold of %s.", traceName, formatCostForMessage(cost), formatCostForMessage(threshold)),
+ Body: headline,
Severity: severity,
URL: "/ai-traces?preset=1h",
DedupToken: traceName,
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ ScopeLabel: "AI trace",
+ Scope: traceName,
+ Observed: formatCostForMessage(cost),
+ Threshold: formatCostForMessage(threshold),
+ }),
}
}
@@ -205,12 +309,21 @@ func buildAiConversationCostMessage(conversationId string, cost, threshold float
if cost >= threshold*3 {
severity = SeverityCritical
}
+ headline := fmt.Sprintf("The AI conversation %q has cost %s over the last 24 hours, exceeding the threshold of %s.", conversationId, formatCostForMessage(cost), formatCostForMessage(threshold))
return Message{
Subject: fmt.Sprintf("[%s] AI conversation cost %s exceeds %s", projectName, formatCostForMessage(cost), formatCostForMessage(threshold)),
- Body: fmt.Sprintf("The AI conversation \"%s\" has cost %s over the last 24 hours, exceeding the threshold of %s.", conversationId, formatCostForMessage(cost), formatCostForMessage(threshold)),
+ Body: headline,
Severity: severity,
URL: "/ai-traces/conversations?preset=24h",
DedupToken: conversationId,
+ Email: alertEmail(models.EmailAlert{
+ ProjectName: projectName,
+ Headline: headline,
+ ScopeLabel: "Conversation",
+ Scope: conversationId,
+ Observed: formatCostForMessage(cost) + " in 24 hours",
+ Threshold: formatCostForMessage(threshold),
+ }),
}
}
@@ -218,7 +331,7 @@ func buildAiFlaggedContentMessage(conversationId, userId string, terms []string,
termList := strings.Join(terms, ", ")
body := fmt.Sprintf("An AI conversation matched flagged terms: %s.", termList)
if conversationId != "" {
- body = fmt.Sprintf("AI conversation \"%s\" matched flagged terms: %s.", conversationId, termList)
+ body = fmt.Sprintf("AI conversation %q matched flagged terms: %s.", conversationId, termList)
}
if userId != "" {
body += fmt.Sprintf(" User: %s.", userId)
@@ -228,6 +341,15 @@ func buildAiFlaggedContentMessage(conversationId, userId string, terms []string,
Body: body,
Severity: SeverityWarning,
URL: "/ai-traces/conversations?preset=24h&flagged=1",
+ Email: &models.NotificationEmail{
+ Template: models.EmailTemplateAiFlagged,
+ Flagged: &models.EmailFlagged{
+ ProjectName: projectName,
+ ConversationId: conversationId,
+ UserId: userId,
+ Terms: terms,
+ },
+ },
}
}
@@ -252,74 +374,63 @@ func (d ExceptionDetails) endpointForMessage() string {
}
func buildNewErrorMessage(details ExceptionDetails, projectName string) Message {
- headline := "A new error has been detected: " + details.ErrorType
return Message{
- Subject: fmt.Sprintf("[%s] New error: %s", projectName, details.ErrorType),
- Body: buildExceptionBody(headline, details),
- Severity: SeverityCritical,
- URL: fmt.Sprintf("/issues/%s", details.Hash),
- Endpoint: details.endpointForMessage(),
- DedupToken: details.Hash,
- Intro: headline,
- Details: exceptionMessageDetails(details),
- CodeBlock: details.StackTrace,
- ActionLabel: "View Issue",
+ Subject: fmt.Sprintf("[%s] New error: %s", projectName, details.ErrorType),
+ Body: buildExceptionBody("A new error has been detected: "+details.ErrorType, details),
+ Severity: SeverityCritical,
+ URL: fmt.Sprintf("/issues/%s", details.Hash),
+ Endpoint: details.endpointForMessage(),
+ DedupToken: details.Hash,
+ Email: exceptionEmail(models.EmailTemplateNewError, details, projectName),
}
}
func buildErrorRegressionMessage(details ExceptionDetails, projectName string) Message {
- headline := "A previously resolved error has reappeared: " + details.ErrorType
return Message{
- Subject: fmt.Sprintf("[%s] Resolved error reappeared: %s", projectName, details.ErrorType),
- Body: buildExceptionBody(headline, details),
- Severity: SeverityCritical,
- URL: fmt.Sprintf("/issues/%s", details.Hash),
- Endpoint: details.endpointForMessage(),
- DedupToken: details.Hash,
- Intro: headline,
- Details: exceptionMessageDetails(details),
- CodeBlock: details.StackTrace,
- ActionLabel: "View Issue",
+ Subject: fmt.Sprintf("[%s] Resolved error reappeared: %s", projectName, details.ErrorType),
+ Body: buildExceptionBody("A previously resolved error has reappeared: "+details.ErrorType, details),
+ Severity: SeverityCritical,
+ URL: fmt.Sprintf("/issues/%s", details.Hash),
+ Endpoint: details.endpointForMessage(),
+ DedupToken: details.Hash,
+ Email: exceptionEmail(models.EmailTemplateErrorRegression, details, projectName),
}
}
-// exceptionMessageDetails mirrors the detail lines of buildExceptionBody as
-// structured rows for the HTML email layout.
-func exceptionMessageDetails(d ExceptionDetails) []models.NotificationMessageDetail {
- var rows []models.NotificationMessageDetail
- add := func(label, value string) {
- if value != "" {
- rows = append(rows, models.NotificationMessageDetail{Label: label, Value: value})
- }
+func exceptionEmail(template string, d ExceptionDetails, projectName string) *models.NotificationEmail {
+ payload := &models.EmailException{
+ ProjectName: projectName,
+ ErrorType: d.ErrorType,
+ ExceptionId: d.Id,
+ Hash: d.Hash,
+ AppVersion: d.AppVersion,
+ ServerName: d.ServerName,
+ TraceLabel: "Endpoint",
+ TraceName: d.TraceName,
+ StackTrace: d.StackTrace,
}
- add("Exception ID", d.Id)
- add("Hash", d.Hash)
if !d.RecordedAt.IsZero() {
- add("Occurred at", d.RecordedAt.UTC().Format("2006-01-02 15:04:05 UTC"))
+ payload.OccurredAt = d.RecordedAt.UTC().Format("2006-01-02 15:04:05 UTC")
}
- add("App version", d.AppVersion)
- add("Server", d.ServerName)
- if d.TraceName != "" {
- switch d.TraceType {
- case "task":
- add("Task", d.TraceName)
- case "ai_trace":
- add("AI trace", d.TraceName)
- default:
- add("Endpoint", d.TraceName)
- }
+ switch d.TraceType {
+ case "task":
+ payload.TraceLabel = "Task"
+ case "ai_trace":
+ payload.TraceLabel = "AI trace"
}
- if len(d.Attributes) > 0 {
- keys := make([]string, 0, len(d.Attributes))
- for k := range d.Attributes {
- keys = append(keys, k)
- }
- sort.Strings(keys)
- for _, k := range keys {
- add(k, d.Attributes[k])
- }
+ for _, key := range sortedKeys(d.Attributes) {
+ payload.Attributes = append(payload.Attributes, models.EmailField{Label: key, Value: d.Attributes[key]})
+ }
+ return &models.NotificationEmail{Template: template, Exception: payload}
+}
+
+func sortedKeys(values map[string]string) []string {
+ keys := make([]string, 0, len(values))
+ for key := range values {
+ keys = append(keys, key)
}
- return rows
+ sort.Strings(keys)
+ return keys
}
func buildExceptionBody(headline string, d ExceptionDetails) string {
@@ -359,12 +470,7 @@ func buildExceptionBody(headline string, d ExceptionDetails) string {
if len(d.Attributes) > 0 {
b.WriteString("\n\nAttributes:")
- keys := make([]string, 0, len(d.Attributes))
- for k := range d.Attributes {
- keys = append(keys, k)
- }
- sort.Strings(keys)
- for _, k := range keys {
+ for _, k := range sortedKeys(d.Attributes) {
fmt.Fprintf(&b, "\n %s: %s", k, d.Attributes[k])
}
}
@@ -377,30 +483,69 @@ func buildExceptionBody(headline string, d ExceptionDetails) string {
}
func buildCheckDownMessage(check *models.SyntheticCheck, errorMsg string, projectName string) Message {
- intro := fmt.Sprintf("The synthetic check %q is failing (%d consecutive failures).", check.Name, check.ConsecutiveFailures)
- body := intro
+ body := fmt.Sprintf("The synthetic check %q is failing (%d consecutive failures).", check.Name, check.ConsecutiveFailures)
if errorMsg != "" {
body += "\n\nLast error: " + errorMsg
}
return Message{
- Subject: fmt.Sprintf("[%s] Check %q is down", projectName, check.Name),
- Body: body,
- Severity: SeverityCritical,
- URL: fmt.Sprintf("/monitors/%d", check.Id),
- DedupToken: fmt.Sprintf("check:%d", check.Id),
- Intro: intro,
- CodeBlock: errorMsg,
- ActionLabel: "View Monitor",
+ Subject: fmt.Sprintf("[%s] Check %q is down", projectName, check.Name),
+ Body: body,
+ Severity: SeverityCritical,
+ URL: fmt.Sprintf("/monitors/%d", check.Id),
+ DedupToken: fmt.Sprintf("check:%d", check.Id),
+ Email: &models.NotificationEmail{
+ Template: models.EmailTemplateCheckDown,
+ Check: &models.EmailCheck{
+ ProjectName: projectName,
+ CheckName: check.Name,
+ ConsecutiveFailures: check.ConsecutiveFailures,
+ LastError: errorMsg,
+ },
+ },
}
}
func buildCheckRecoveredMessage(check *models.SyntheticCheck, projectName string) Message {
return Message{
- Subject: fmt.Sprintf("[%s] Check %q recovered", projectName, check.Name),
- Body: fmt.Sprintf("The synthetic check %q is passing again.", check.Name),
- Severity: SeverityInfo,
- URL: fmt.Sprintf("/monitors/%d", check.Id),
- DedupToken: fmt.Sprintf("check:%d", check.Id),
- ActionLabel: "View Monitor",
+ Subject: fmt.Sprintf("[%s] Check %q recovered", projectName, check.Name),
+ Body: fmt.Sprintf("The synthetic check %q is passing again.", check.Name),
+ Severity: SeverityInfo,
+ URL: fmt.Sprintf("/monitors/%d", check.Id),
+ DedupToken: fmt.Sprintf("check:%d", check.Id),
+ Email: &models.NotificationEmail{
+ Template: models.EmailTemplateCheckRecovered,
+ Check: &models.EmailCheck{
+ ProjectName: projectName,
+ CheckName: check.Name,
+ },
+ },
+ }
+}
+
+func TestChannelMessage() Message {
+ return Message{
+ Subject: "Traceway Test Notification",
+ Body: "This is a test notification from Traceway. If you received this, your notification channel is configured correctly.",
+ Severity: SeverityInfo,
+ RuleType: "test",
+ RuleName: "Test",
+ Email: &models.NotificationEmail{
+ Template: models.EmailTemplateTest,
+ Test: &models.EmailTest{Target: "notification channel"},
+ },
+ }
+}
+
+func TestContactMethodMessage() Message {
+ return Message{
+ Subject: "Traceway on-call test",
+ Body: "This is a test of your on-call contact method. If you received this, pages will reach you here.",
+ Severity: SeverityInfo,
+ RuleType: "test",
+ RuleName: "Test",
+ Email: &models.NotificationEmail{
+ Template: models.EmailTemplateTest,
+ Test: &models.EmailTest{Target: "on-call contact method"},
+ },
}
}
diff --git a/backend/app/oncall/escalator.go b/backend/app/oncall/escalator.go
index ec83c62a..068c9050 100644
--- a/backend/app/oncall/escalator.go
+++ b/backend/app/oncall/escalator.go
@@ -533,8 +533,10 @@ func claimChannelDelivery(tx *sql.Tx, page *models.Page, level int, channelId in
// link.
func buildPageMessage(page *models.Page, level int, ackURL string) notifications.Message {
prefix := "[Page] "
+ escalationLevel := 0
if level > 0 {
- prefix = fmt.Sprintf("[Page - escalation L%d] ", level+1)
+ escalationLevel = level + 1
+ prefix = fmt.Sprintf("[Page - escalation L%d] ", escalationLevel)
}
body := page.Body
if body != "" {
@@ -547,13 +549,21 @@ func buildPageMessage(page *models.Page, level int, ackURL string) notifications
severity = notifications.SeverityCritical
}
return notifications.Message{
- Subject: prefix + page.Subject,
- Body: body,
- Severity: severity,
- RuleType: page.RuleType,
- RuleName: page.RuleName,
- URL: ackURL,
- ActionLabel: "Acknowledge Page",
+ Subject: prefix + page.Subject,
+ Body: body,
+ Severity: severity,
+ RuleType: page.RuleType,
+ RuleName: page.RuleName,
+ URL: ackURL,
+ Email: &models.NotificationEmail{
+ Template: models.EmailTemplatePage,
+ Page: &models.EmailPage{
+ BodyText: page.Body,
+ RuleName: page.RuleName,
+ EventCount: page.EventCount,
+ EscalationLevel: escalationLevel,
+ },
+ },
}
}
diff --git a/backend/app/services/email.service.go b/backend/app/services/email.service.go
index 1ed11ca1..f0532eb8 100644
--- a/backend/app/services/email.service.go
+++ b/backend/app/services/email.service.go
@@ -1,159 +1,160 @@
package services
import (
+ "bytes"
+ "context"
"crypto/tls"
+ "embed"
"fmt"
+ "html/template"
+ "mime"
+ "mime/multipart"
+ "mime/quotedprintable"
"net"
"net/smtp"
- "strconv"
+ "net/textproto"
+ "strings"
"time"
"github.com/tracewayapp/traceway/backend/app/config"
+ "github.com/tracewayapp/traceway/backend/app/models"
)
-type emailService struct {
- enabled bool
- host string
- port int
- username string
- password string
- from string
- baseUrl string
-}
+//go:embed emailtemplates/*.gohtml
+var emailTemplateFS embed.FS
-var EmailService *emailService
+var emailTemplates = template.Must(template.New("emails").ParseFS(emailTemplateFS, "emailtemplates/*.gohtml"))
-func InitEmail() {
- cfg := config.Config
+const (
+ EmailColorCritical = "#cf222e"
+ EmailColorWarning = "#9a6700"
+ EmailColorInfo = "#0969da"
+)
- enabled := cfg.SMTPEnabled == "true"
- port, _ := strconv.Atoi(cfg.SMTPPort)
- if port == 0 {
- port = 587
- }
+const smtpTimeout = 10 * time.Second
+
+type Email struct {
+ To []string
+ Subject string
+ Text string
+ Template string
+ Data any
+
+ Title string
+ Badge string
+ BadgeColor string
+ URL string
+ Footer string
+ LogoURL string
+}
- baseUrl := cfg.AppBaseURL
- if baseUrl == "" {
- baseUrl = "http://localhost:5173"
- }
+type emailService struct{}
- EmailService = &emailService{
- enabled: enabled,
- host: cfg.SMTPHost,
- port: port,
- username: cfg.SMTPUsername,
- password: cfg.SMTPPassword,
- from: cfg.SMTPFrom,
- baseUrl: baseUrl,
- }
+var EmailService = &emailService{}
- if enabled {
+func InitEmail() {
+ if emailEnabled() {
config.Logln("Email service initialized with SMTP")
} else {
config.Logln("Email service initialized in log-only mode (SMTP disabled)")
}
}
-func (e *emailService) SendInvitation(toEmail string, inviterName string, orgName string, token string) error {
- inviteUrl := fmt.Sprintf("%s/accept-invitation?token=%s", e.baseUrl, token)
+func (e *emailService) IsEnabled() bool { return emailEnabled() }
- subject := fmt.Sprintf("You've been invited to join %s on Traceway", orgName)
- body := fmt.Sprintf(`Hello,
+func (e *emailService) BaseURL() string { return emailBaseURL() }
-%s has invited you to join %s on Traceway.
+func emailEnabled() bool { return config.Config != nil && config.Config.SMTPEnabled == "true" }
-Click the link below to accept the invitation:
-%s
-
-This invitation will expire in 7 days.
+func emailBaseURL() string {
+ if config.Config != nil && config.Config.AppBaseURL != "" {
+ return strings.TrimRight(config.Config.AppBaseURL, "/")
+ }
+ return "http://localhost:5173"
+}
-If you did not expect this invitation, you can safely ignore this email.
+func RenderEmail(email Email) (string, error) {
+ if email.LogoURL == "" {
+ email.LogoURL = emailBaseURL() + "/traceway-mark.png"
+ }
+ var out bytes.Buffer
+ if err := emailTemplates.ExecuteTemplate(&out, email.Template+".gohtml", email); err != nil {
+ return "", fmt.Errorf("failed to render email template %q: %w", email.Template, err)
+ }
+ return out.String(), nil
+}
-Best regards,
-The Traceway Team
-`, inviterName, orgName, inviteUrl)
+func SendEmail(ctx context.Context, email Email) error {
+ if len(email.To) == 0 {
+ return fmt.Errorf("no recipients")
+ }
- if !e.enabled {
- config.Logf("[EMAIL LOG] To: %s\nSubject: %s\nBody:\n%s", toEmail, subject, body)
+ if !emailEnabled() {
+ config.Logf("[EMAIL LOG] To: %s\nSubject: %s\nBody:\n%s", strings.Join(email.To, ", "), email.Subject, email.Text)
return nil
}
- msg := fmt.Sprintf("From: %s\r\nTo: %s\r\nSubject: %s\r\n\r\n%s",
- e.from, toEmail, subject, body)
-
- err := e.sendMail([]string{toEmail}, []byte(msg))
+ html, err := RenderEmail(email)
if err != nil {
- config.Logf("Failed to send invitation email to %s: %v", toEmail, err)
return err
}
- config.Logf("Invitation email sent to %s for organization %s", toEmail, orgName)
- return nil
-}
-
-func (e *emailService) IsEnabled() bool {
- return e.enabled
-}
-
-// BaseURL is the dashboard origin used to absolutize links and reference the
-// logo in HTML notification emails.
-func (e *emailService) BaseURL() string {
- return e.baseUrl
-}
-
-func (e *emailService) SendPasswordReset(toEmail string, token string) error {
- resetUrl := fmt.Sprintf("%s/reset-password?token=%s", e.baseUrl, token)
-
- subject := "Reset your Traceway password"
- body := fmt.Sprintf(`Hello,
-
-You requested to reset your password for your Traceway account.
-
-Click the link below to reset your password:
-%s
-
-This link will expire in 1 hour.
-
-If you did not request this password reset, you can safely ignore this email.
-
-Best regards,
-The Traceway Team
-`, resetUrl)
-
- if !e.enabled {
- config.Logf("[EMAIL LOG] To: %s\nSubject: %s\nBody:\n%s", toEmail, subject, body)
- return nil
+ from := config.Config.SMTPFrom
+ var buf bytes.Buffer
+ fmt.Fprintf(&buf, "From: %s\r\n", from)
+ fmt.Fprintf(&buf, "To: %s\r\n", strings.Join(email.To, ", "))
+ fmt.Fprintf(&buf, "Subject: %s\r\n", mime.QEncoding.Encode("utf-8", email.Subject))
+ fmt.Fprintf(&buf, "Date: %s\r\n", time.Now().Format(time.RFC1123Z))
+ buf.WriteString("MIME-Version: 1.0\r\n")
+
+ writer := multipart.NewWriter(&buf)
+ fmt.Fprintf(&buf, "Content-Type: multipart/alternative; boundary=%q\r\n\r\n", writer.Boundary())
+ for _, part := range []struct{ contentType, body string }{
+ {"text/plain; charset=utf-8", email.Text},
+ {"text/html; charset=utf-8", html},
+ } {
+ w, err := writer.CreatePart(textproto.MIMEHeader{
+ "Content-Type": {part.contentType},
+ "Content-Transfer-Encoding": {"quoted-printable"},
+ })
+ if err != nil {
+ return err
+ }
+ qp := quotedprintable.NewWriter(w)
+ if _, err := qp.Write([]byte(part.body)); err != nil {
+ return err
+ }
+ if err := qp.Close(); err != nil {
+ return err
+ }
}
-
- msg := fmt.Sprintf("From: %s\r\nTo: %s\r\nSubject: %s\r\n\r\n%s",
- e.from, toEmail, subject, body)
-
- err := e.sendMail([]string{toEmail}, []byte(msg))
- if err != nil {
- config.Logf("Failed to send password reset email to %s: %v", toEmail, err)
+ if err := writer.Close(); err != nil {
return err
}
- config.Logf("Password reset email sent to %s", toEmail)
- return nil
+ return sendSMTP(ctx, from, email.To, buf.Bytes())
}
-func (e *emailService) sendMail(to []string, msg []byte) error {
- addr := net.JoinHostPort(e.host, strconv.Itoa(e.port))
- var auth smtp.Auth
- if e.username != "" {
- auth = smtp.PlainAuth("", e.username, e.password, e.host)
+func sendSMTP(ctx context.Context, from string, to []string, msg []byte) error {
+ host := config.Config.SMTPHost
+ port := config.Config.SMTPPort
+ if port == "" {
+ port = "587"
}
- conn, err := net.DialTimeout("tcp", addr, 10*time.Second)
+ dialer := net.Dialer{Timeout: smtpTimeout}
+ conn, err := dialer.DialContext(ctx, "tcp", net.JoinHostPort(host, port))
if err != nil {
return fmt.Errorf("SMTP dial failed: %w", err)
}
- // Must be set before smtp.NewClient, which blocks reading the server greeting.
- conn.SetDeadline(time.Now().Add(10 * time.Second))
+ deadline := time.Now().Add(smtpTimeout)
+ if ctxDeadline, ok := ctx.Deadline(); ok && ctxDeadline.Before(deadline) {
+ deadline = ctxDeadline
+ }
+ conn.SetDeadline(deadline)
- client, err := smtp.NewClient(conn, e.host)
+ client, err := smtp.NewClient(conn, host)
if err != nil {
conn.Close()
return fmt.Errorf("SMTP client failed: %w", err)
@@ -161,21 +162,22 @@ func (e *emailService) sendMail(to []string, msg []byte) error {
defer client.Close()
if ok, _ := client.Extension("STARTTLS"); ok {
- if err := client.StartTLS(&tls.Config{ServerName: e.host}); err != nil {
+ if err := client.StartTLS(&tls.Config{ServerName: host}); err != nil {
return fmt.Errorf("SMTP STARTTLS failed: %w", err)
}
}
- if auth != nil {
- if err := client.Auth(auth); err != nil {
+ if username := config.Config.SMTPUsername; username != "" {
+ if err := client.Auth(smtp.PlainAuth("", username, config.Config.SMTPPassword, host)); err != nil {
return fmt.Errorf("SMTP auth failed: %w", err)
}
}
- if err := client.Mail(e.from); err != nil {
+
+ if err := client.Mail(from); err != nil {
return fmt.Errorf("SMTP MAIL failed: %w", err)
}
- for _, r := range to {
- if err := client.Rcpt(r); err != nil {
+ for _, recipient := range to {
+ if err := client.Rcpt(recipient); err != nil {
return fmt.Errorf("SMTP RCPT failed: %w", err)
}
}
@@ -191,3 +193,111 @@ func (e *emailService) sendMail(to []string, msg []byte) error {
}
return client.Quit()
}
+
+type invitationData struct {
+ InviterName string
+ OrgName string
+}
+
+func (e *emailService) SendInvitation(toEmail string, inviterName string, orgName string, token string) error {
+ inviteUrl := fmt.Sprintf("%s/accept-invitation?token=%s", emailBaseURL(), token)
+ subject := fmt.Sprintf("You've been invited to join %s on Traceway", orgName)
+
+ err := SendEmail(context.Background(), Email{
+ To: []string{toEmail},
+ Subject: subject,
+ Template: "invitation",
+ Title: fmt.Sprintf("Join %s on Traceway", orgName),
+ URL: inviteUrl,
+ Footer: "This invitation expires in 7 days. If you did not expect it, you can safely ignore this email.",
+ Data: invitationData{InviterName: inviterName, OrgName: orgName},
+ Text: fmt.Sprintf(`Hello,
+
+%s has invited you to join %s on Traceway.
+
+Click the link below to accept the invitation:
+%s
+
+This invitation will expire in 7 days.
+
+If you did not expect this invitation, you can safely ignore this email.
+
+Best regards,
+The Traceway Team
+`, inviterName, orgName, inviteUrl),
+ })
+ if err != nil {
+ config.Logf("Failed to send invitation email to %s: %v", toEmail, err)
+ return err
+ }
+
+ config.Logf("Invitation email sent to %s for organization %s", toEmail, orgName)
+ return nil
+}
+
+func (e *emailService) SendPasswordReset(toEmail string, token string) error {
+ resetUrl := fmt.Sprintf("%s/reset-password?token=%s", emailBaseURL(), token)
+
+ err := SendEmail(context.Background(), Email{
+ To: []string{toEmail},
+ Subject: "Reset your Traceway password",
+ Template: "password_reset",
+ Title: "Reset your password",
+ URL: resetUrl,
+ Footer: "This link expires in 1 hour. If you did not ask for a reset, you can safely ignore this email and your password stays unchanged.",
+ Text: fmt.Sprintf(`Hello,
+
+You requested to reset your password for your Traceway account.
+
+Click the link below to reset your password:
+%s
+
+This link will expire in 1 hour.
+
+If you did not request this password reset, you can safely ignore this email.
+
+Best regards,
+The Traceway Team
+`, resetUrl),
+ })
+ if err != nil {
+ config.Logf("Failed to send password reset email to %s: %v", toEmail, err)
+ return err
+ }
+
+ config.Logf("Password reset email sent to %s", toEmail)
+ return nil
+}
+
+func NotificationEmail(msg models.NotificationMessage, recipients []string) Email {
+ email := Email{
+ To: recipients,
+ Subject: msg.Subject,
+ Text: msg.Body,
+ Template: msg.Email.Template,
+ Title: msg.Subject,
+ URL: msg.URL,
+ Data: msg.Email,
+ }
+
+ switch msg.Severity {
+ case models.NotificationSeverityCritical:
+ email.Subject = "[CRITICAL] " + msg.Subject
+ email.Badge, email.BadgeColor = "CRITICAL", EmailColorCritical
+ case models.NotificationSeverityWarning:
+ email.Subject = "[WARNING] " + msg.Subject
+ email.Badge, email.BadgeColor = "WARNING", EmailColorWarning
+ case models.NotificationSeverityInfo:
+ email.Subject = "[INFO] " + msg.Subject
+ email.Badge, email.BadgeColor = "INFO", EmailColorInfo
+ }
+
+ if strings.HasPrefix(email.URL, "/") {
+ email.URL = emailBaseURL() + email.URL
+ }
+ if msg.RuleType != "test" && msg.RuleName != "" {
+ email.Footer = fmt.Sprintf("You are receiving this because the notification rule %q fired.", msg.RuleName)
+ }
+
+ return email
+}
diff --git a/backend/app/services/email.service_test.go b/backend/app/services/email.service_test.go
new file mode 100644
index 00000000..9d3a8374
--- /dev/null
+++ b/backend/app/services/email.service_test.go
@@ -0,0 +1,198 @@
+package services
+
+import (
+ "strings"
+ "testing"
+
+ "github.com/tracewayapp/traceway/backend/app/models"
+)
+
+func TestEveryTemplateRenders(t *testing.T) {
+ payload := &models.NotificationEmail{
+ Exception: &models.EmailException{
+ ProjectName: "Acme <Prod>",
+ ErrorType: "*fmt.wrapError",
+ ExceptionId: "01930f8c",
+ Hash: "9c7bc73da37328f2",
+ OccurredAt: "2026-08-24 09:41:12 UTC",
+ AppVersion: "1.9.19",
+ ServerName: "api-blue-3",
+ TraceLabel: "Endpoint",
+ TraceName: "GET /api/users/:id",
+ Attributes: []models.EmailField{{Label: "user_id", Value: "8412"}},
+ StackTrace: "err := <nil>\n\tat main.go:1",
+ },
+ Check: &models.EmailCheck{ProjectName: "Acme", CheckName: "Checkout API", ConsecutiveFailures: 3, LastError: "i/o timeout"},
+ Alert: &models.EmailAlert{ProjectName: "Acme", Headline: "The error rate has reached 12.4%.", Observed: "12.4%", Threshold: "5.0%", WindowMins: 15},
+ Flagged: &models.EmailFlagged{ProjectName: "Acme", ConversationId: "conv_1", UserId: "8412", Terms: []string{"password"}},
+ Page: &models.EmailPage{BodyText: "Something broke", RuleName: "Critical errors", EventCount: 4},
+ Test: &models.EmailTest{Target: "notification channel"},
+ }
+
+ templates := []string{
+ models.EmailTemplateNewError,
+ models.EmailTemplateErrorRegression,
+ models.EmailTemplateCheckDown,
+ models.EmailTemplateCheckRecovered,
+ models.EmailTemplateAlert,
+ models.EmailTemplateAiFlagged,
+ models.EmailTemplatePage,
+ models.EmailTemplateTest,
+ }
+
+ for _, name := range templates {
+ html, err := RenderEmail(Email{
+ Template: name,
+ Title: "Something happened",
+ Badge: "CRITICAL",
+ BadgeColor: EmailColorCritical,
+ URL: "https://traceway.example.com/issues/abc",
+ Footer: "You are receiving this because a rule fired.",
+ Data: payload,
+ })
+ if err != nil {
+ t.Fatalf("%s: %v", name, err)
+ }
+ for _, want := range []string{"<!DOCTYPE html>", "Something happened", EmailColorCritical, "traceway-mark.png", "Traceway &middot; Error tracking"} {
+ if !strings.Contains(html, want) {
+ t.Errorf("%s: rendered HTML missing %q", name, want)
+ }
+ }
+ }
+}
+
+func TestTemplatesEscapeContent(t *testing.T) {
+ html, err := RenderEmail(Email{
+ Template: models.EmailTemplateNewError,
+ Title: "[Project <X>] New error",
+ URL: "https://traceway.example.com/issues/abc",
+ Data: &models.NotificationEmail{Exception: &models.EmailException{
+ ProjectName: "Acme <Prod>",
+ ErrorType: "*fmt.wrapError",
+ StackTrace: "err := <nil>",
+ }},
+ })
+ if err != nil {
+ t.Fatalf("RenderEmail failed: %v", err)
+ }
+ for _, want := range []string{"[Project &lt;X&gt;] New error", "Acme &lt;Prod&gt;", "err := &lt;nil&gt;", "View Issue"} {
+ if !strings.Contains(html, want) {
+ t.Errorf("rendered HTML missing %q", want)
+ }
+ }
+ if strings.Contains(html, "<nil>") {
+ t.Error("stack trace was not escaped")
+ }
+}
+
+func TestRenderEmailRejectsUnknownTemplate(t *testing.T) {
+ if _, err := RenderEmail(Email{Template: "nope"}); err == nil {
+ t.Error("expected an error for an unknown template")
+ }
+}
+
+func TestTransactionalTemplatesRender(t *testing.T) {
+ invitation, err := RenderEmail(Email{
+ Template: "invitation",
+ Title: "Join Acme on Traceway",
+ URL: "https://traceway.example.com/accept-invitation?token=t",
+ Data: invitationData{InviterName: "Dana", OrgName: "Acme <Inc>"},
+ })
+ if err != nil {
+ t.Fatalf("invitation failed: %v", err)
+ }
+ for _, want := range []string{"Dana", "Acme &lt;Inc&gt;", "Accept Invitation", `href="https://traceway.example.com/accept-invitation?token=t"`} {
+ if !strings.Contains(invitation, want) {
+ t.Errorf("invitation missing %q", want)
+ }
+ }
+
+ reset, err := RenderEmail(Email{Template: "password_reset", Title: "Reset your password", URL: "https://traceway.example.com/reset-password?token=t"})
+ if err != nil {
+ t.Fatalf("password reset failed: %v", err)
+ }
+ for _, want := range []string{"Reset Password", `href="https://traceway.example.com/reset-password?token=t"`} {
+ if !strings.Contains(reset, want) {
+ t.Errorf("password reset missing %q", want)
+ }
+ }
+}
+
+func TestNotificationEmailFillsChrome(t *testing.T) {
+ msg := models.NotificationMessage{
+ Subject: "[Acme] New error: *fmt.wrapError",
+ Body: "A new error has been detected.",
+ Severity: models.NotificationSeverityCritical,
+ RuleType: "new_error",
+ RuleName: "Critical errors",
+ URL: "/issues/abc",
+ Email: &models.NotificationEmail{Template: models.EmailTemplateNewError, Exception: &models.EmailException{}},
+ }
+
+ email := NotificationEmail(msg, []string{"[email protected]"})
+
+ if email.Subject != "[CRITICAL] "+msg.Subject {
+ t.Errorf("subject = %q, want the severity prefix", email.Subject)
+ }
+ if email.Title != msg.Subject {
+ t.Errorf("title = %q, want the unprefixed subject", email.Title)
+ }
+ if email.Badge != "CRITICAL" || email.BadgeColor != EmailColorCritical {
+ t.Errorf("badge = %q/%q, want CRITICAL", email.Badge, email.BadgeColor)
+ }
+ if email.URL != emailBaseURL()+"/issues/abc" {
+ t.Errorf("url = %q, want the relative link absolutized", email.URL)
+ }
+ if !strings.Contains(email.Footer, `"Critical errors"`) {
+ t.Errorf("footer = %q, want it to name the rule", email.Footer)
+ }
+ if email.Text != msg.Body {
+ t.Error("plaintext alternative must stay the message body")
+ }
+}
+
+func TestNotificationEmailKeepsAbsoluteURLsAndSkipsTestFooter(t *testing.T) {
+ msg := models.NotificationMessage{
+ Severity: models.NotificationSeverityInfo,
+ RuleType: "test",
+ RuleName: "Test",
+ URL: "https://traceway.example.com/ack/twk_1",
+ Email: &models.NotificationEmail{Template: models.EmailTemplateTest, Test: &models.EmailTest{Target: "notification channel"}},
+ }
+
+ email := NotificationEmail(msg, []string{"[email protected]"})
+
+ if email.URL != msg.URL {
+ t.Errorf("url = %q, want the already absolute link untouched", email.URL)
+ }
+ if email.Footer != "" {
+ t.Errorf("footer = %q, want none: the test template says it is a test", email.Footer)
+ }
+}
+
+func TestPageTemplateSwitchesCopyOnEscalation(t *testing.T) {
+ render := func(level int) string {
+ t.Helper()
+ html, err := RenderEmail(Email{
+ Template: models.EmailTemplatePage,
+ Title: "[Page] Something broke",
+ URL: "https://traceway.example.com/ack/twk_1",
+ Data: &models.NotificationEmail{Page: &models.EmailPage{BodyText: "Something broke", EscalationLevel: level}},
+ })
+ if err != nil {
+ t.Fatalf("level %d: %v", level, err)
+ }
+ return html
+ }
+
+ if first := render(0); !strings.Contains(first, "You are on call and a page needs your attention.") {
+ t.Error("first delivery should read as a fresh page")
+ }
+ escalated := render(2)
+ if !strings.Contains(escalated, "escalated to level 2") {
+ t.Error("escalated delivery should name the level it reached, matching the subject prefix")
+ }
+ if strings.Contains(escalated, "You are on call and a page needs your attention. Acknowledge") {
+ t.Error("escalated delivery should not also show the first-delivery copy")
+ }
+}
diff --git a/backend/app/services/emailtemplate/mime.go b/backend/app/services/emailtemplate/mime.go
deleted file mode 100644
index 5a200a4a..00000000
--- a/backend/app/services/emailtemplate/mime.go
+++ /dev/null
@@ -1,69 +0,0 @@
-package emailtemplate
-
-import (
- "bytes"
- "fmt"
- "io"
- "mime"
- "mime/multipart"
- "mime/quotedprintable"
- "strings"
- "time"
-)
-
-// BuildMIME composes a multipart/alternative message carrying both the
-// plaintext and the rendered HTML body, quoted-printable encoded so long
-// stack-trace and markup lines stay within SMTP line limits. An empty
-// htmlBody yields a plain single-part message.
-func BuildMIME(from string, to []string, subject, textBody, htmlBody string) ([]byte, error) {
- var buf bytes.Buffer
-
- fmt.Fprintf(&buf, "From: %s\r\n", from)
- fmt.Fprintf(&buf, "To: %s\r\n", strings.Join(to, ", "))
- fmt.Fprintf(&buf, "Subject: %s\r\n", mime.QEncoding.Encode("utf-8", subject))
- fmt.Fprintf(&buf, "Date: %s\r\n", time.Now().Format(time.RFC1123Z))
- buf.WriteString("MIME-Version: 1.0\r\n")
-
- if htmlBody == "" {
- buf.WriteString("Content-Type: text/plain; charset=utf-8\r\n")
- buf.WriteString("Content-Transfer-Encoding: quoted-printable\r\n\r\n")
- if err := writeQuotedPrintable(&buf, textBody); err != nil {
- return nil, err
- }
- return buf.Bytes(), nil
- }
-
- mw := multipart.NewWriter(&buf)
- fmt.Fprintf(&buf, "Content-Type: multipart/alternative; boundary=%q\r\n\r\n", mw.Boundary())
-
- for _, part := range []struct {
- contentType string
- body string
- }{
- {"text/plain; charset=utf-8", textBody},
- {"text/html; charset=utf-8", htmlBody},
- } {
- pw, err := mw.CreatePart(map[string][]string{
- "Content-Type": {part.contentType},
- "Content-Transfer-Encoding": {"quoted-printable"},
- })
- if err != nil {
- return nil, err
- }
- if err := writeQuotedPrintable(pw, part.body); err != nil {
- return nil, err
- }
- }
- if err := mw.Close(); err != nil {
- return nil, err
- }
- return buf.Bytes(), nil
-}
-
-func writeQuotedPrintable(w io.Writer, body string) error {
- qp := quotedprintable.NewWriter(w)
- if _, err := qp.Write([]byte(body)); err != nil {
- return err
- }
- return qp.Close()
-}
diff --git a/backend/app/services/emailtemplate/template.go b/backend/app/services/emailtemplate/template.go
deleted file mode 100644
index f34be6a8..00000000
--- a/backend/app/services/emailtemplate/template.go
+++ /dev/null
@@ -1,74 +0,0 @@
-// Package emailtemplate renders every outgoing Traceway email through one
-// embedded HTML layout so they all share the same look: centered logo,
-// headline, white content card with optional badge/details/code block, a
-// single action button, a plain-link fallback and a footer.
-package emailtemplate
-
-import (
- "bytes"
- _ "embed"
- "html/template"
- "strings"
-)
-
-//go:embed base.gohtml
-var baseTemplateSource string
-
-var baseTemplate = template.Must(template.New("base").Funcs(template.FuncMap{
- "nl2br": func(s string) template.HTML {
- return template.HTML(strings.ReplaceAll(template.HTMLEscapeString(s), "\n", "<br>"))
- },
-}).Parse(baseTemplateSource))
-
-const (
- ColorCritical = "#cf222e"
- ColorWarning = "#9a6700"
- ColorInfo = "#0969da"
-)
-
-type Detail struct {
- Label string
- Value string
-}
-
-type Button struct {
- Label string
- URL string
-}
-
-type Data struct {
- // Preheader is the hidden inbox preview text.
- Preheader string
- LogoURL string
- Title string
- // Badge is an optional severity chip above the content (e.g. "CRITICAL").
- Badge string
- BadgeColor string
- Paragraphs []string
- Details []Detail
- CodeBlock string
- // Button renders in the dashboard's outline style; severity only ever
- // colors the badge.
- Button *Button
- FooterNote string
-}
-
-func Render(d Data) (string, error) {
- if d.BadgeColor == "" {
- d.BadgeColor = ColorInfo
- }
- if d.Preheader == "" && len(d.Paragraphs) > 0 {
- d.Preheader = d.Paragraphs[0]
- }
-
- var buf bytes.Buffer
- if err := baseTemplate.Execute(&buf, d); err != nil {
- return "", err
- }
- return buf.String(), nil
-}
-
-// LogoURL builds the logo image reference served by the dashboard.
-func LogoURL(baseURL string) string {
- return strings.TrimRight(baseURL, "/") + "/traceway-mark.png"
-}
diff --git a/backend/app/services/emailtemplate/template_test.go b/backend/app/services/emailtemplate/template_test.go
deleted file mode 100644
index 9cfc2497..00000000
--- a/backend/app/services/emailtemplate/template_test.go
+++ /dev/null
@@ -1,119 +0,0 @@
-package emailtemplate
-
-import (
- "io"
- "mime"
- "mime/multipart"
- "mime/quotedprintable"
- "net/mail"
- "strings"
- "testing"
-)
-
-func TestRenderEscapesAndIncludesContent(t *testing.T) {
- html, err := Render(Data{
- LogoURL: "https://traceway.example.com/traceway-mark.png",
- Title: "[Project <X>] New error: *traceway.StackTraceError",
- Badge: "CRITICAL",
- BadgeColor: ColorCritical,
- Paragraphs: []string{"A new error has been detected.\nSecond line."},
- Details: []Detail{{Label: "Hash", Value: "9c7bc73da37328f2"}},
- CodeBlock: "err := <nil>\n\tat main.go:1",
- Button: &Button{Label: "View Issue", URL: "https://traceway.example.com/issues/abc"},
- FooterNote: "You are receiving this because a rule fired.",
- })
- if err != nil {
- t.Fatalf("Render failed: %v", err)
- }
-
- for _, want := range []string{
- "[Project &lt;X&gt;] New error: *traceway.StackTraceError",
- "CRITICAL",
- "A new error has been detected.<br>Second line.",
- "9c7bc73da37328f2",
- "err := &lt;nil&gt;",
- `href="https://traceway.example.com/issues/abc"`,
- "View Issue",
- "Button not working?",
- "You are receiving this because a rule fired.",
- } {
- if !strings.Contains(html, want) {
- t.Errorf("rendered HTML missing %q", want)
- }
- }
-}
-
-func TestRenderButtonIsOutlineStyle(t *testing.T) {
- html, err := Render(Data{Title: "t", Button: &Button{Label: "Go", URL: "https://x"}})
- if err != nil {
- t.Fatalf("Render failed: %v", err)
- }
- if !strings.Contains(html, "border:1px solid #d1d9e0;border-radius:6px;background-color:#ffffff;") {
- t.Error("expected the outline button style")
- }
-}
-
-func TestBuildMIMEMultipart(t *testing.T) {
- raw, err := BuildMIME("[email protected]", []string{"[email protected]"},
- "[CRITICAL] New error: über long subject", "plain text body", "<html><body>hi</body></html>")
- if err != nil {
- t.Fatalf("BuildMIME failed: %v", err)
- }
-
- msg, err := mail.ReadMessage(strings.NewReader(string(raw)))
- if err != nil {
- t.Fatalf("message does not parse: %v", err)
- }
-
- dec := new(mime.WordDecoder)
- subject, err := dec.DecodeHeader(msg.Header.Get("Subject"))
- if err != nil {
- t.Fatalf("subject does not decode: %v", err)
- }
- if subject != "[CRITICAL] New error: über long subject" {
- t.Errorf("unexpected subject: %q", subject)
- }
-
- mediaType, params, err := mime.ParseMediaType(msg.Header.Get("Content-Type"))
- if err != nil || mediaType != "multipart/alternative" {
- t.Fatalf("unexpected content type %q (err %v)", mediaType, err)
- }
-
- mr := multipart.NewReader(msg.Body, params["boundary"])
- var bodies []string
- for {
- part, err := mr.NextPart()
- if err == io.EOF {
- break
- }
- if err != nil {
- t.Fatalf("reading part: %v", err)
- }
- body, err := io.ReadAll(quotedprintable.NewReader(part))
- if err != nil {
- t.Fatalf("decoding part: %v", err)
- }
- bodies = append(bodies, string(body))
- }
- if len(bodies) != 2 || bodies[0] != "plain text body" || bodies[1] != "<html><body>hi</body></html>" {
- t.Errorf("unexpected part bodies: %#v", bodies)
- }
-}
-
-func TestBuildMIMEPlainOnly(t *testing.T) {
- raw, err := BuildMIME("[email protected]", []string{"[email protected]"}, "subject", "just text", "")
- if err != nil {
- t.Fatalf("BuildMIME failed: %v", err)
- }
- msg, err := mail.ReadMessage(strings.NewReader(string(raw)))
- if err != nil {
- t.Fatalf("message does not parse: %v", err)
- }
- body, err := io.ReadAll(quotedprintable.NewReader(msg.Body))
- if err != nil {
- t.Fatalf("decoding body: %v", err)
- }
- if string(body) != "just text" {
- t.Errorf("unexpected body: %q", body)
- }
-}
diff --git a/backend/app/services/emailtemplates/ai_flagged.gohtml b/backend/app/services/emailtemplates/ai_flagged.gohtml
new file mode 100644
index 00000000..18b0f4b5
--- /dev/null
+++ b/backend/app/services/emailtemplates/ai_flagged.gohtml
@@ -0,0 +1,51 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>{{.Title}}</title>
+</head>
+<body style="margin:0;padding:0;background-color:#f6f8fa;">
+<div style="display:none;max-height:0;overflow:hidden;mso-hide:all;">An AI conversation matched flagged terms in {{.Data.Flagged.ProjectName}}.</div>
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background-color:#f6f8fa;">
+<tr><td align="center" style="padding:24px 16px;">
+<table role="presentation" width="600" cellpadding="0" cellspacing="0" style="max-width:600px;width:100%;">
+<tr><td align="center" style="padding:16px 0;">
+<img src="{{.LogoURL}}" alt="Traceway" width="36" height="36" style="display:block;border:0;">
+</td></tr>
+<tr><td align="center" style="padding:0 8px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:22px;line-height:30px;font-weight:600;color:#1f2328;">{{.Title}}</td></tr>
+<tr><td style="background-color:#ffffff;border:1px solid #d1d9e0;border-radius:8px;padding:24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;line-height:21px;color:#1f2328;">
+{{if .Badge}}<div style="margin:0 0 16px;"><span style="display:inline-block;padding:2px 10px;border-radius:12px;font-size:11px;font-weight:600;letter-spacing:0.5px;color:#ffffff;background-color:{{.BadgeColor}};">{{.Badge}}</span></div>{{end}}
+<p style="margin:0 0 16px;">An AI conversation in <strong>{{.Data.Flagged.ProjectName}}</strong> matched terms you asked Traceway to flag.</p>
+<div style="margin:0 0 16px;">{{range .Data.Flagged.Terms}}<span style="display:inline-block;margin:0 6px 6px 0;padding:2px 10px;border:1px solid #d1d9e0;border-radius:12px;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:12px;color:#1f2328;background-color:#f6f8fa;">{{.}}</span>{{end}}</div>
+<table role="presentation" cellpadding="0" cellspacing="0" style="margin:0 0 16px;">
+{{if .Data.Flagged.ConversationId}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Conversation</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Flagged.ConversationId}}</td>
+</tr>
+{{end}}{{if .Data.Flagged.UserId}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">User</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Flagged.UserId}}</td>
+</tr>
+{{end}}</table>
+<table role="presentation" cellpadding="0" cellspacing="0" align="center" style="margin:20px auto 4px;">
+<tr><td align="center" style="border:1px solid #d1d9e0;border-radius:6px;background-color:#ffffff;">
+<a href="{{.URL}}" style="display:inline-block;padding:10px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;font-weight:500;color:#1f2328;text-decoration:none;border-radius:6px;">View Conversation</a>
+</td></tr>
+</table>
+</td></tr>
+<tr><td align="center" style="padding:24px 8px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;">
+Button not working? Paste the following link into your browser:<br>
+<a href="{{.URL}}" style="color:#0969da;text-decoration:none;word-break:break-all;">{{.URL}}</a>
+</td></tr>
+<tr><td style="padding:24px 0 0;">
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0"><tr><td style="border-top:1px solid #d1d9e0;font-size:0;line-height:0;">&nbsp;</td></tr></table>
+</td></tr>
+<tr><td align="center" style="padding:8px 8px 32px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:12px;line-height:18px;color:#59636e;">
+{{if .Footer}}{{.Footer}}<br>{{end}}Traceway &middot; Error tracking &amp; monitoring
+</td></tr>
+</table>
+</td></tr>
+</table>
+</body>
+</html>
diff --git a/backend/app/services/emailtemplates/alert.gohtml b/backend/app/services/emailtemplates/alert.gohtml
new file mode 100644
index 00000000..c4a9f4d1
--- /dev/null
+++ b/backend/app/services/emailtemplates/alert.gohtml
@@ -0,0 +1,62 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>{{.Title}}</title>
+</head>
+<body style="margin:0;padding:0;background-color:#f6f8fa;">
+<div style="display:none;max-height:0;overflow:hidden;mso-hide:all;">{{.Data.Alert.Headline}}</div>
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background-color:#f6f8fa;">
+<tr><td align="center" style="padding:24px 16px;">
+<table role="presentation" width="600" cellpadding="0" cellspacing="0" style="max-width:600px;width:100%;">
+<tr><td align="center" style="padding:16px 0;">
+<img src="{{.LogoURL}}" alt="Traceway" width="36" height="36" style="display:block;border:0;">
+</td></tr>
+<tr><td align="center" style="padding:0 8px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:22px;line-height:30px;font-weight:600;color:#1f2328;">{{.Title}}</td></tr>
+<tr><td style="background-color:#ffffff;border:1px solid #d1d9e0;border-radius:8px;padding:24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;line-height:21px;color:#1f2328;">
+{{if .Badge}}<div style="margin:0 0 16px;"><span style="display:inline-block;padding:2px 10px;border-radius:12px;font-size:11px;font-weight:600;letter-spacing:0.5px;color:#ffffff;background-color:{{.BadgeColor}};">{{.Badge}}</span></div>{{end}}
+<p style="margin:0 0 16px;">{{.Data.Alert.Headline}}</p>
+<table role="presentation" cellpadding="0" cellspacing="0" style="margin:0 0 16px;">
+{{if .Data.Alert.ProjectName}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Project</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Alert.ProjectName}}</td>
+</tr>
+{{end}}{{if .Data.Alert.Scope}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">{{.Data.Alert.ScopeLabel}}</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Alert.Scope}}</td>
+</tr>
+{{end}}{{if .Data.Alert.Observed}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Observed</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Alert.Observed}}</td>
+</tr>
+{{end}}{{if .Data.Alert.Threshold}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Threshold</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Alert.Threshold}}</td>
+</tr>
+{{end}}{{if .Data.Alert.WindowMins}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Window</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">last {{.Data.Alert.WindowMins}} minutes</td>
+</tr>
+{{end}}</table>
+<table role="presentation" cellpadding="0" cellspacing="0" align="center" style="margin:20px auto 4px;">
+<tr><td align="center" style="border:1px solid #d1d9e0;border-radius:6px;background-color:#ffffff;">
+<a href="{{.URL}}" style="display:inline-block;padding:10px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;font-weight:500;color:#1f2328;text-decoration:none;border-radius:6px;">View in Traceway</a>
+</td></tr>
+</table>
+</td></tr>
+<tr><td align="center" style="padding:24px 8px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;">
+Button not working? Paste the following link into your browser:<br>
+<a href="{{.URL}}" style="color:#0969da;text-decoration:none;word-break:break-all;">{{.URL}}</a>
+</td></tr>
+<tr><td style="padding:24px 0 0;">
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0"><tr><td style="border-top:1px solid #d1d9e0;font-size:0;line-height:0;">&nbsp;</td></tr></table>
+</td></tr>
+<tr><td align="center" style="padding:8px 8px 32px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:12px;line-height:18px;color:#59636e;">
+{{if .Footer}}{{.Footer}}<br>{{end}}Traceway &middot; Error tracking &amp; monitoring
+</td></tr>
+</table>
+</td></tr>
+</table>
+</body>
+</html>
diff --git a/backend/app/services/emailtemplates/check_down.gohtml b/backend/app/services/emailtemplates/check_down.gohtml
new file mode 100644
index 00000000..d27d4a84
--- /dev/null
+++ b/backend/app/services/emailtemplates/check_down.gohtml
@@ -0,0 +1,52 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>{{.Title}}</title>
+</head>
+<body style="margin:0;padding:0;background-color:#f6f8fa;">
+<div style="display:none;max-height:0;overflow:hidden;mso-hide:all;">The monitor {{.Data.Check.CheckName}} is down.</div>
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background-color:#f6f8fa;">
+<tr><td align="center" style="padding:24px 16px;">
+<table role="presentation" width="600" cellpadding="0" cellspacing="0" style="max-width:600px;width:100%;">
+<tr><td align="center" style="padding:16px 0;">
+<img src="{{.LogoURL}}" alt="Traceway" width="36" height="36" style="display:block;border:0;">
+</td></tr>
+<tr><td align="center" style="padding:0 8px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:22px;line-height:30px;font-weight:600;color:#1f2328;">{{.Title}}</td></tr>
+<tr><td style="background-color:#ffffff;border:1px solid #d1d9e0;border-radius:8px;padding:24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;line-height:21px;color:#1f2328;">
+{{if .Badge}}<div style="margin:0 0 16px;"><span style="display:inline-block;padding:2px 10px;border-radius:12px;font-size:11px;font-weight:600;letter-spacing:0.5px;color:#ffffff;background-color:{{.BadgeColor}};">{{.Badge}}</span></div>{{end}}
+<p style="margin:0 0 16px;">The monitor <strong>{{.Data.Check.CheckName}}</strong> is failing and Traceway has opened an incident for it.</p>
+<table role="presentation" cellpadding="0" cellspacing="0" style="margin:0 0 16px;">
+<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Project</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Check.ProjectName}}</td>
+</tr>
+<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Consecutive failures</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Check.ConsecutiveFailures}}</td>
+</tr>
+</table>
+{{if .Data.Check.LastError}}<div style="margin:0 0 6px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:11px;font-weight:600;letter-spacing:0.5px;text-transform:uppercase;color:#59636e;">Last error</div>
+<pre style="margin:0 0 16px;padding:12px;background-color:#f6f8fa;border:1px solid #d1d9e0;border-radius:6px;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:12px;line-height:18px;color:#1f2328;white-space:pre-wrap;word-break:break-word;">{{.Data.Check.LastError}}</pre>
+{{end}}<table role="presentation" cellpadding="0" cellspacing="0" align="center" style="margin:20px auto 4px;">
+<tr><td align="center" style="border:1px solid #d1d9e0;border-radius:6px;background-color:#ffffff;">
+<a href="{{.URL}}" style="display:inline-block;padding:10px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;font-weight:500;color:#1f2328;text-decoration:none;border-radius:6px;">View Monitor</a>
+</td></tr>
+</table>
+</td></tr>
+<tr><td align="center" style="padding:24px 8px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;">
+Button not working? Paste the following link into your browser:<br>
+<a href="{{.URL}}" style="color:#0969da;text-decoration:none;word-break:break-all;">{{.URL}}</a>
+</td></tr>
+<tr><td style="padding:24px 0 0;">
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0"><tr><td style="border-top:1px solid #d1d9e0;font-size:0;line-height:0;">&nbsp;</td></tr></table>
+</td></tr>
+<tr><td align="center" style="padding:8px 8px 32px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:12px;line-height:18px;color:#59636e;">
+{{if .Footer}}{{.Footer}}<br>{{end}}Traceway &middot; Error tracking &amp; monitoring
+</td></tr>
+</table>
+</td></tr>
+</table>
+</body>
+</html>
diff --git a/backend/app/services/emailtemplate/base.gohtml b/backend/app/services/emailtemplates/check_recovered.gohtml
similarity index 63%
rename from backend/app/services/emailtemplate/base.gohtml
rename to backend/app/services/emailtemplates/check_recovered.gohtml
index add71bca..a2a273ef 100644
--- a/backend/app/services/emailtemplate/base.gohtml
+++ b/backend/app/services/emailtemplates/check_recovered.gohtml
@@ -6,56 +6,38 @@
<title>{{.Title}}</title>
</head>
<body style="margin:0;padding:0;background-color:#f6f8fa;">
-{{if .Preheader}}<div style="display:none;max-height:0;overflow:hidden;mso-hide:all;">{{.Preheader}}</div>{{end}}
+<div style="display:none;max-height:0;overflow:hidden;mso-hide:all;">The monitor {{.Data.Check.CheckName}} is back up.</div>
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background-color:#f6f8fa;">
<tr><td align="center" style="padding:24px 16px;">
<table role="presentation" width="600" cellpadding="0" cellspacing="0" style="max-width:600px;width:100%;">
-{{if .LogoURL}}
<tr><td align="center" style="padding:16px 0;">
<img src="{{.LogoURL}}" alt="Traceway" width="36" height="36" style="display:block;border:0;">
</td></tr>
-{{end}}
<tr><td align="center" style="padding:0 8px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:22px;line-height:30px;font-weight:600;color:#1f2328;">{{.Title}}</td></tr>
<tr><td style="background-color:#ffffff;border:1px solid #d1d9e0;border-radius:8px;padding:24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;line-height:21px;color:#1f2328;">
-{{if .Badge}}
-<div style="margin:0 0 16px;"><span style="display:inline-block;padding:2px 10px;border-radius:12px;font-size:11px;font-weight:600;letter-spacing:0.5px;color:#ffffff;background-color:{{.BadgeColor}};">{{.Badge}}</span></div>
-{{end}}
-{{range .Paragraphs}}
-<p style="margin:0 0 16px;">{{. | nl2br}}</p>
-{{end}}
-{{if .Details}}
+{{if .Badge}}<div style="margin:0 0 16px;"><span style="display:inline-block;padding:2px 10px;border-radius:12px;font-size:11px;font-weight:600;letter-spacing:0.5px;color:#ffffff;background-color:{{.BadgeColor}};">{{.Badge}}</span></div>{{end}}
+<p style="margin:0 0 16px;">The monitor <strong>{{.Data.Check.CheckName}}</strong> is passing again. Traceway has resolved the incident it opened.</p>
<table role="presentation" cellpadding="0" cellspacing="0" style="margin:0 0 16px;">
-{{range .Details}}
<tr>
-<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">{{.Label}}</td>
-<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Value}}</td>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Project</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Check.ProjectName}}</td>
</tr>
-{{end}}
</table>
-{{end}}
-{{if .CodeBlock}}
-<pre style="margin:0 0 16px;padding:12px;background-color:#f6f8fa;border:1px solid #d1d9e0;border-radius:6px;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:12px;line-height:18px;color:#1f2328;white-space:pre-wrap;word-break:break-word;">{{.CodeBlock}}</pre>
-{{end}}
-{{if .Button}}
<table role="presentation" cellpadding="0" cellspacing="0" align="center" style="margin:20px auto 4px;">
<tr><td align="center" style="border:1px solid #d1d9e0;border-radius:6px;background-color:#ffffff;">
-<a href="{{.Button.URL}}" style="display:inline-block;padding:10px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;font-weight:500;color:#1f2328;text-decoration:none;border-radius:6px;">{{.Button.Label}}</a>
+<a href="{{.URL}}" style="display:inline-block;padding:10px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;font-weight:500;color:#1f2328;text-decoration:none;border-radius:6px;">View Monitor</a>
</td></tr>
</table>
-{{end}}
</td></tr>
-{{if .Button}}
<tr><td align="center" style="padding:24px 8px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;">
Button not working? Paste the following link into your browser:<br>
-<a href="{{.Button.URL}}" style="color:#0969da;text-decoration:none;word-break:break-all;">{{.Button.URL}}</a>
+<a href="{{.URL}}" style="color:#0969da;text-decoration:none;word-break:break-all;">{{.URL}}</a>
</td></tr>
-{{end}}
<tr><td style="padding:24px 0 0;">
<table role="presentation" width="100%" cellpadding="0" cellspacing="0"><tr><td style="border-top:1px solid #d1d9e0;font-size:0;line-height:0;">&nbsp;</td></tr></table>
</td></tr>
<tr><td align="center" style="padding:8px 8px 32px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:12px;line-height:18px;color:#59636e;">
-{{if .FooterNote}}{{.FooterNote}}<br>{{end}}
-Traceway &middot; Error tracking &amp; monitoring
+{{if .Footer}}{{.Footer}}<br>{{end}}Traceway &middot; Error tracking &amp; monitoring
</td></tr>
</table>
</td></tr>
diff --git a/backend/app/services/emailtemplates/error_regression.gohtml b/backend/app/services/emailtemplates/error_regression.gohtml
new file mode 100644
index 00000000..d300f11c
--- /dev/null
+++ b/backend/app/services/emailtemplates/error_regression.gohtml
@@ -0,0 +1,75 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>{{.Title}}</title>
+</head>
+<body style="margin:0;padding:0;background-color:#f6f8fa;">
+<div style="display:none;max-height:0;overflow:hidden;mso-hide:all;">An error that was resolved has come back in {{.Data.Exception.ProjectName}}.</div>
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background-color:#f6f8fa;">
+<tr><td align="center" style="padding:24px 16px;">
+<table role="presentation" width="600" cellpadding="0" cellspacing="0" style="max-width:600px;width:100%;">
+<tr><td align="center" style="padding:16px 0;">
+<img src="{{.LogoURL}}" alt="Traceway" width="36" height="36" style="display:block;border:0;">
+</td></tr>
+<tr><td align="center" style="padding:0 8px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:22px;line-height:30px;font-weight:600;color:#1f2328;">{{.Title}}</td></tr>
+<tr><td style="background-color:#ffffff;border:1px solid #d1d9e0;border-radius:8px;padding:24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;line-height:21px;color:#1f2328;">
+{{if .Badge}}<div style="margin:0 0 16px;"><span style="display:inline-block;padding:2px 10px;border-radius:12px;font-size:11px;font-weight:600;letter-spacing:0.5px;color:#ffffff;background-color:{{.BadgeColor}};">{{.Badge}}</span></div>{{end}}
+<div style="margin:0 0 16px;padding:12px;background-color:#f6f8fa;border-left:3px solid {{.BadgeColor}};border-radius:4px;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:14px;line-height:20px;color:#1f2328;word-break:break-word;">{{.Data.Exception.ErrorType}}</div>
+<p style="margin:0 0 16px;">An error that had been resolved is happening again in <strong>{{.Data.Exception.ProjectName}}</strong>.</p>
+<table role="presentation" cellpadding="0" cellspacing="0" style="margin:0 0 16px;">
+{{if .Data.Exception.ExceptionId}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Exception ID</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Exception.ExceptionId}}</td>
+</tr>
+{{end}}{{if .Data.Exception.Hash}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Hash</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Exception.Hash}}</td>
+</tr>
+{{end}}{{if .Data.Exception.OccurredAt}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Occurred at</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Exception.OccurredAt}}</td>
+</tr>
+{{end}}{{if .Data.Exception.TraceName}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">{{.Data.Exception.TraceLabel}}</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Exception.TraceName}}</td>
+</tr>
+{{end}}{{if .Data.Exception.AppVersion}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">App version</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Exception.AppVersion}}</td>
+</tr>
+{{end}}{{if .Data.Exception.ServerName}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Server</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Exception.ServerName}}</td>
+</tr>
+{{end}}</table>
+{{if .Data.Exception.Attributes}}<div style="margin:0 0 6px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:11px;font-weight:600;letter-spacing:0.5px;text-transform:uppercase;color:#59636e;">Attributes</div>
+<table role="presentation" cellpadding="0" cellspacing="0" style="margin:0 0 16px;">
+{{range .Data.Exception.Attributes}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">{{.Label}}</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Value}}</td>
+</tr>
+{{end}}</table>
+{{end}}{{if .Data.Exception.StackTrace}}<pre style="margin:0 0 16px;padding:12px;background-color:#f6f8fa;border:1px solid #d1d9e0;border-radius:6px;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:12px;line-height:18px;color:#1f2328;white-space:pre-wrap;word-break:break-word;">{{.Data.Exception.StackTrace}}</pre>
+{{end}}<table role="presentation" cellpadding="0" cellspacing="0" align="center" style="margin:20px auto 4px;">
+<tr><td align="center" style="border:1px solid #d1d9e0;border-radius:6px;background-color:#ffffff;">
+<a href="{{.URL}}" style="display:inline-block;padding:10px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;font-weight:500;color:#1f2328;text-decoration:none;border-radius:6px;">View Issue</a>
+</td></tr>
+</table>
+</td></tr>
+<tr><td align="center" style="padding:24px 8px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;">
+Button not working? Paste the following link into your browser:<br>
+<a href="{{.URL}}" style="color:#0969da;text-decoration:none;word-break:break-all;">{{.URL}}</a>
+</td></tr>
+<tr><td style="padding:24px 0 0;">
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0"><tr><td style="border-top:1px solid #d1d9e0;font-size:0;line-height:0;">&nbsp;</td></tr></table>
+</td></tr>
+<tr><td align="center" style="padding:8px 8px 32px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:12px;line-height:18px;color:#59636e;">
+{{if .Footer}}{{.Footer}}<br>{{end}}Traceway &middot; Error tracking &amp; monitoring
+</td></tr>
+</table>
+</td></tr>
+</table>
+</body>
+</html>
diff --git a/backend/app/services/emailtemplates/invitation.gohtml b/backend/app/services/emailtemplates/invitation.gohtml
new file mode 100644
index 00000000..a06af86a
--- /dev/null
+++ b/backend/app/services/emailtemplates/invitation.gohtml
@@ -0,0 +1,40 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>{{.Title}}</title>
+</head>
+<body style="margin:0;padding:0;background-color:#f6f8fa;">
+<div style="display:none;max-height:0;overflow:hidden;mso-hide:all;">{{.Data.InviterName}} invited you to join {{.Data.OrgName}} on Traceway.</div>
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background-color:#f6f8fa;">
+<tr><td align="center" style="padding:24px 16px;">
+<table role="presentation" width="600" cellpadding="0" cellspacing="0" style="max-width:600px;width:100%;">
+<tr><td align="center" style="padding:16px 0;">
+<img src="{{.LogoURL}}" alt="Traceway" width="36" height="36" style="display:block;border:0;">
+</td></tr>
+<tr><td align="center" style="padding:0 8px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:22px;line-height:30px;font-weight:600;color:#1f2328;">Join {{.Data.OrgName}} on Traceway</td></tr>
+<tr><td style="background-color:#ffffff;border:1px solid #d1d9e0;border-radius:8px;padding:24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;line-height:21px;color:#1f2328;">
+<p style="margin:0 0 16px;"><strong>{{.Data.InviterName}}</strong> has invited you to join <strong>{{.Data.OrgName}}</strong> on Traceway.</p>
+<p style="margin:0 0 16px;">Traceway tracks errors, performance and uptime for your team's projects. Accept the invitation to get access.</p>
+<table role="presentation" cellpadding="0" cellspacing="0" align="center" style="margin:20px auto 4px;">
+<tr><td align="center" style="border:1px solid #d1d9e0;border-radius:6px;background-color:#ffffff;">
+<a href="{{.URL}}" style="display:inline-block;padding:10px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;font-weight:500;color:#1f2328;text-decoration:none;border-radius:6px;">Accept Invitation</a>
+</td></tr>
+</table>
+</td></tr>
+<tr><td align="center" style="padding:24px 8px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;">
+Button not working? Paste the following link into your browser:<br>
+<a href="{{.URL}}" style="color:#0969da;text-decoration:none;word-break:break-all;">{{.URL}}</a>
+</td></tr>
+<tr><td style="padding:24px 0 0;">
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0"><tr><td style="border-top:1px solid #d1d9e0;font-size:0;line-height:0;">&nbsp;</td></tr></table>
+</td></tr>
+<tr><td align="center" style="padding:8px 8px 32px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:12px;line-height:18px;color:#59636e;">
+This invitation expires in 7 days. If you did not expect it, you can safely ignore this email.<br>Traceway &middot; Error tracking &amp; monitoring
+</td></tr>
+</table>
+</td></tr>
+</table>
+</body>
+</html>
diff --git a/backend/app/services/emailtemplates/new_error.gohtml b/backend/app/services/emailtemplates/new_error.gohtml
new file mode 100644
index 00000000..13d54beb
--- /dev/null
+++ b/backend/app/services/emailtemplates/new_error.gohtml
@@ -0,0 +1,75 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>{{.Title}}</title>
+</head>
+<body style="margin:0;padding:0;background-color:#f6f8fa;">
+<div style="display:none;max-height:0;overflow:hidden;mso-hide:all;">A new error was reported in {{.Data.Exception.ProjectName}}.</div>
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background-color:#f6f8fa;">
+<tr><td align="center" style="padding:24px 16px;">
+<table role="presentation" width="600" cellpadding="0" cellspacing="0" style="max-width:600px;width:100%;">
+<tr><td align="center" style="padding:16px 0;">
+<img src="{{.LogoURL}}" alt="Traceway" width="36" height="36" style="display:block;border:0;">
+</td></tr>
+<tr><td align="center" style="padding:0 8px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:22px;line-height:30px;font-weight:600;color:#1f2328;">{{.Title}}</td></tr>
+<tr><td style="background-color:#ffffff;border:1px solid #d1d9e0;border-radius:8px;padding:24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;line-height:21px;color:#1f2328;">
+{{if .Badge}}<div style="margin:0 0 16px;"><span style="display:inline-block;padding:2px 10px;border-radius:12px;font-size:11px;font-weight:600;letter-spacing:0.5px;color:#ffffff;background-color:{{.BadgeColor}};">{{.Badge}}</span></div>{{end}}
+<div style="margin:0 0 16px;padding:12px;background-color:#f6f8fa;border-left:3px solid {{.BadgeColor}};border-radius:4px;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:14px;line-height:20px;color:#1f2328;word-break:break-word;">{{.Data.Exception.ErrorType}}</div>
+<p style="margin:0 0 16px;">A new error was reported in <strong>{{.Data.Exception.ProjectName}}</strong>. This is the first time Traceway has seen it.</p>
+<table role="presentation" cellpadding="0" cellspacing="0" style="margin:0 0 16px;">
+{{if .Data.Exception.ExceptionId}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Exception ID</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Exception.ExceptionId}}</td>
+</tr>
+{{end}}{{if .Data.Exception.Hash}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Hash</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Exception.Hash}}</td>
+</tr>
+{{end}}{{if .Data.Exception.OccurredAt}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Occurred at</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Exception.OccurredAt}}</td>
+</tr>
+{{end}}{{if .Data.Exception.TraceName}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">{{.Data.Exception.TraceLabel}}</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Exception.TraceName}}</td>
+</tr>
+{{end}}{{if .Data.Exception.AppVersion}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">App version</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Exception.AppVersion}}</td>
+</tr>
+{{end}}{{if .Data.Exception.ServerName}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Server</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Exception.ServerName}}</td>
+</tr>
+{{end}}</table>
+{{if .Data.Exception.Attributes}}<div style="margin:0 0 6px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:11px;font-weight:600;letter-spacing:0.5px;text-transform:uppercase;color:#59636e;">Attributes</div>
+<table role="presentation" cellpadding="0" cellspacing="0" style="margin:0 0 16px;">
+{{range .Data.Exception.Attributes}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">{{.Label}}</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Value}}</td>
+</tr>
+{{end}}</table>
+{{end}}{{if .Data.Exception.StackTrace}}<pre style="margin:0 0 16px;padding:12px;background-color:#f6f8fa;border:1px solid #d1d9e0;border-radius:6px;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:12px;line-height:18px;color:#1f2328;white-space:pre-wrap;word-break:break-word;">{{.Data.Exception.StackTrace}}</pre>
+{{end}}<table role="presentation" cellpadding="0" cellspacing="0" align="center" style="margin:20px auto 4px;">
+<tr><td align="center" style="border:1px solid #d1d9e0;border-radius:6px;background-color:#ffffff;">
+<a href="{{.URL}}" style="display:inline-block;padding:10px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;font-weight:500;color:#1f2328;text-decoration:none;border-radius:6px;">View Issue</a>
+</td></tr>
+</table>
+</td></tr>
+<tr><td align="center" style="padding:24px 8px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;">
+Button not working? Paste the following link into your browser:<br>
+<a href="{{.URL}}" style="color:#0969da;text-decoration:none;word-break:break-all;">{{.URL}}</a>
+</td></tr>
+<tr><td style="padding:24px 0 0;">
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0"><tr><td style="border-top:1px solid #d1d9e0;font-size:0;line-height:0;">&nbsp;</td></tr></table>
+</td></tr>
+<tr><td align="center" style="padding:8px 8px 32px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:12px;line-height:18px;color:#59636e;">
+{{if .Footer}}{{.Footer}}<br>{{end}}Traceway &middot; Error tracking &amp; monitoring
+</td></tr>
+</table>
+</td></tr>
+</table>
+</body>
+</html>
diff --git a/backend/app/services/emailtemplates/page.gohtml b/backend/app/services/emailtemplates/page.gohtml
new file mode 100644
index 00000000..e56762c1
--- /dev/null
+++ b/backend/app/services/emailtemplates/page.gohtml
@@ -0,0 +1,52 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>{{.Title}}</title>
+</head>
+<body style="margin:0;padding:0;background-color:#f6f8fa;">
+<div style="display:none;max-height:0;overflow:hidden;mso-hide:all;">You are on call and a page needs your attention.</div>
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background-color:#f6f8fa;">
+<tr><td align="center" style="padding:24px 16px;">
+<table role="presentation" width="600" cellpadding="0" cellspacing="0" style="max-width:600px;width:100%;">
+<tr><td align="center" style="padding:16px 0;">
+<img src="{{.LogoURL}}" alt="Traceway" width="36" height="36" style="display:block;border:0;">
+</td></tr>
+<tr><td align="center" style="padding:0 8px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:22px;line-height:30px;font-weight:600;color:#1f2328;">{{.Title}}</td></tr>
+<tr><td style="background-color:#ffffff;border:1px solid #d1d9e0;border-radius:8px;padding:24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;line-height:21px;color:#1f2328;">
+{{if .Badge}}<div style="margin:0 0 16px;"><span style="display:inline-block;padding:2px 10px;border-radius:12px;font-size:11px;font-weight:600;letter-spacing:0.5px;color:#ffffff;background-color:{{.BadgeColor}};">{{.Badge}}</span></div>{{end}}
+{{if .Data.Page.EscalationLevel}}<p style="margin:0 0 16px;">This page was <strong>escalated to level {{.Data.Page.EscalationLevel}}</strong> because nobody acknowledged it yet. You are on call and it needs your attention.</p>
+{{end}}{{if not .Data.Page.EscalationLevel}}<p style="margin:0 0 16px;">You are on call and a page needs your attention. Acknowledge it to stop the escalation.</p>
+{{end}}<pre style="margin:0 0 16px;padding:12px;background-color:#f6f8fa;border:1px solid #d1d9e0;border-radius:6px;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:12px;line-height:18px;color:#1f2328;white-space:pre-wrap;word-break:break-word;">{{.Data.Page.BodyText}}</pre>
+<table role="presentation" cellpadding="0" cellspacing="0" style="margin:0 0 16px;">
+{{if .Data.Page.RuleName}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Rule</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Page.RuleName}}</td>
+</tr>
+{{end}}{{if .Data.Page.EventCount}}<tr>
+<td style="padding:3px 16px 3px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;white-space:nowrap;vertical-align:top;">Events</td>
+<td style="padding:3px 0;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;line-height:20px;color:#1f2328;word-break:break-all;">{{.Data.Page.EventCount}}</td>
+</tr>
+{{end}}</table>
+<table role="presentation" cellpadding="0" cellspacing="0" align="center" style="margin:20px auto 4px;">
+<tr><td align="center" style="border:1px solid #d1d9e0;border-radius:6px;background-color:#ffffff;">
+<a href="{{.URL}}" style="display:inline-block;padding:10px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;font-weight:500;color:#1f2328;text-decoration:none;border-radius:6px;">Acknowledge Page</a>
+</td></tr>
+</table>
+</td></tr>
+<tr><td align="center" style="padding:24px 8px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;">
+Button not working? Paste the following link into your browser:<br>
+<a href="{{.URL}}" style="color:#0969da;text-decoration:none;word-break:break-all;">{{.URL}}</a>
+</td></tr>
+<tr><td style="padding:24px 0 0;">
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0"><tr><td style="border-top:1px solid #d1d9e0;font-size:0;line-height:0;">&nbsp;</td></tr></table>
+</td></tr>
+<tr><td align="center" style="padding:8px 8px 32px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:12px;line-height:18px;color:#59636e;">
+{{if .Footer}}{{.Footer}}<br>{{end}}Traceway &middot; Error tracking &amp; monitoring
+</td></tr>
+</table>
+</td></tr>
+</table>
+</body>
+</html>
diff --git a/backend/app/services/emailtemplates/password_reset.gohtml b/backend/app/services/emailtemplates/password_reset.gohtml
new file mode 100644
index 00000000..e9916b21
--- /dev/null
+++ b/backend/app/services/emailtemplates/password_reset.gohtml
@@ -0,0 +1,40 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>{{.Title}}</title>
+</head>
+<body style="margin:0;padding:0;background-color:#f6f8fa;">
+<div style="display:none;max-height:0;overflow:hidden;mso-hide:all;">Reset the password for your Traceway account.</div>
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background-color:#f6f8fa;">
+<tr><td align="center" style="padding:24px 16px;">
+<table role="presentation" width="600" cellpadding="0" cellspacing="0" style="max-width:600px;width:100%;">
+<tr><td align="center" style="padding:16px 0;">
+<img src="{{.LogoURL}}" alt="Traceway" width="36" height="36" style="display:block;border:0;">
+</td></tr>
+<tr><td align="center" style="padding:0 8px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:22px;line-height:30px;font-weight:600;color:#1f2328;">Reset your password</td></tr>
+<tr><td style="background-color:#ffffff;border:1px solid #d1d9e0;border-radius:8px;padding:24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;line-height:21px;color:#1f2328;">
+<p style="margin:0 0 16px;">Someone asked to reset the password for your Traceway account.</p>
+<p style="margin:0 0 16px;">Choose a new one with the button below.</p>
+<table role="presentation" cellpadding="0" cellspacing="0" align="center" style="margin:20px auto 4px;">
+<tr><td align="center" style="border:1px solid #d1d9e0;border-radius:6px;background-color:#ffffff;">
+<a href="{{.URL}}" style="display:inline-block;padding:10px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;font-weight:500;color:#1f2328;text-decoration:none;border-radius:6px;">Reset Password</a>
+</td></tr>
+</table>
+</td></tr>
+<tr><td align="center" style="padding:24px 8px 0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:13px;line-height:20px;color:#59636e;">
+Button not working? Paste the following link into your browser:<br>
+<a href="{{.URL}}" style="color:#0969da;text-decoration:none;word-break:break-all;">{{.URL}}</a>
+</td></tr>
+<tr><td style="padding:24px 0 0;">
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0"><tr><td style="border-top:1px solid #d1d9e0;font-size:0;line-height:0;">&nbsp;</td></tr></table>
+</td></tr>
+<tr><td align="center" style="padding:8px 8px 32px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:12px;line-height:18px;color:#59636e;">
+This link expires in 1 hour. If you did not ask for a reset, you can safely ignore this email and your password stays unchanged.<br>Traceway &middot; Error tracking &amp; monitoring
+</td></tr>
+</table>
+</td></tr>
+</table>
+</body>
+</html>
diff --git a/backend/app/services/emailtemplates/test.gohtml b/backend/app/services/emailtemplates/test.gohtml
new file mode 100644
index 00000000..c2a2e2e1
--- /dev/null
+++ b/backend/app/services/emailtemplates/test.gohtml
@@ -0,0 +1,32 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>{{.Title}}</title>
+</head>
+<body style="margin:0;padding:0;background-color:#f6f8fa;">
+<div style="display:none;max-height:0;overflow:hidden;mso-hide:all;">Your {{.Data.Test.Target}} is configured correctly.</div>
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background-color:#f6f8fa;">
+<tr><td align="center" style="padding:24px 16px;">
+<table role="presentation" width="600" cellpadding="0" cellspacing="0" style="max-width:600px;width:100%;">
+<tr><td align="center" style="padding:16px 0;">
+<img src="{{.LogoURL}}" alt="Traceway" width="36" height="36" style="display:block;border:0;">
+</td></tr>
+<tr><td align="center" style="padding:0 8px 24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:22px;line-height:30px;font-weight:600;color:#1f2328;">{{.Title}}</td></tr>
+<tr><td style="background-color:#ffffff;border:1px solid #d1d9e0;border-radius:8px;padding:24px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:14px;line-height:21px;color:#1f2328;">
+{{if .Badge}}<div style="margin:0 0 16px;"><span style="display:inline-block;padding:2px 10px;border-radius:12px;font-size:11px;font-weight:600;letter-spacing:0.5px;color:#ffffff;background-color:{{.BadgeColor}};">{{.Badge}}</span></div>{{end}}
+<p style="margin:0 0 16px;">This is a test email from Traceway.</p>
+<p style="margin:0 0 16px;">Your <strong>{{.Data.Test.Target}}</strong> is configured correctly. Real alerts will arrive here, styled just like this one.</p>
+</td></tr>
+<tr><td style="padding:24px 0 0;">
+<table role="presentation" width="100%" cellpadding="0" cellspacing="0"><tr><td style="border-top:1px solid #d1d9e0;font-size:0;line-height:0;">&nbsp;</td></tr></table>
+</td></tr>
+<tr><td align="center" style="padding:8px 8px 32px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;font-size:12px;line-height:18px;color:#59636e;">
+You are receiving this because someone used a test button in Traceway.<br>Traceway &middot; Error tracking &amp; monitoring
+</td></tr>
+</table>
+</td></tr>
+</table>
+</body>
+</html>