Notification channels

Beta

Being rolled out. The interface described here is the one of the next release: it may still change, and may not be enabled on your tenant yet.

Monitoring is being rolled out; email channels are not available yet.

A notification channel is a destination for alerts: a webhook, Slack, Discord, Microsoft Teams or an email address. You declare it once, then each monitor says which channels to notify when it goes down and when it comes back.

Prerequisites

A tenant session (sng tenant use acme) and the deploy:write right.
A monitor, or the intention to create one.
The incoming webhook URL provided by your tool (Slack, Discord, Teams) or by your service.

1. Declare a webhook channel

infra/NotificationChannel/ops.yml
apiVersion: monitoring.stackngo.io/v1alpha1
kind: NotificationChannel
metadata:
  name: ops
spec:
  type: Webhook
  webhook:
    url: https://hooks.example.com/uptime    # https required
  sendResolved: true                         # also notifies of the return to normal (default)
  repeatInterval: 4h                         # reminder as long as the incident lasts (default)

sng apply infra/NotificationChannel/ops.yml

There is no dedicated sng command for channels: start from the template sng template gen NotificationChannel ops > ops.yml, then sng apply.

2. Connect a monitor to the channel

sng monitor create api --http https://api.example.com/health --interval 5m --channel ops

or, in the monitor's manifest:

spec:
  notificationChannelRefs:
    - name: ops          # 10 channels at most per monitor

A monitor that references a nonexistent or refused channel is itself refused (ChannelNotFound): declare the channel first, or in the same sng apply.

3. Channel types

`type`	Block	Destination
`Webhook`	`webhook`	`url: https://…` or `urlSecretRef`
`Discord`	`discord`	`url` or `urlSecretRef` (Discord webhook URL)
`MSTeams`	`msteams`	`url` or `urlSecretRef` (Teams incoming webhook URL)
`Slack`	`slack`	`webhookSecretRef` required (the Slack URL is a secret), `channel` optional (`#ops`)
`Email`	`email`	`to: [ops@example.com]` (10 addresses at most) — not available yet

The block must match the type: type: Slack requires slack and forbids the others.

infra/NotificationChannel/discord.yml
apiVersion: monitoring.stackngo.io/v1alpha1
kind: NotificationChannel
metadata:
  name: discord
spec:
  type: Discord
  discord:
    url: https://discord.com/api/webhooks/123/abc

4. Keep the URL out of the manifest

A webhook URL grants the right to post in your channel: treat it as a secret. urlSecretRef (and webhookSecretRef for Slack) designates a tenant secret and the key that contains the URL. This secret must be declared with materialize: true, which makes it usable by your notification channels (see Declare a secret by manifest):

infra/SSMSecret/slack-webhook.yml
apiVersion: ssm.stackngo.io/v1alpha1
kind: SSMSecret
metadata:
  name: slack-webhook
spec:
  length: 0          # you set the value yourself
  materialize: true  # usable by the notification channel

sng apply infra/SSMSecret/slack-webhook.yml
printf '%s' 'https://hooks.slack.com/services/T000/B000/XXXX' > /tmp/slack-url
sng secret set slack-webhook --from-file /tmp/slack-url
rm /tmp/slack-url

infra/NotificationChannel/slack.yml
apiVersion: monitoring.stackngo.io/v1alpha1
kind: NotificationChannel
metadata:
  name: slack
spec:
  type: Slack
  slack:
    webhookSecretRef: { name: slack-webhook, key: value }
    channel: "#ops"

If the secret is missing, does not have the given key, or does not contain an https:// URL, only that channel is set aside: it becomes Accepted=False with the reason SecretNotFound (or Invalid for a URL that is not https://), and your other channels keep sending their alerts. The channel resumes on its own as soon as the secret is fixed.

Allow up to 15 minutes between the sng secret set and the channel taking the URL into account; a later change of the value, rotation included, is taken into account within the same delay.

5. What you receive

Alert	When	Severity
`MonitorDown` — "`<moniteur>` est indisponible"	the monitor switches to `Down`	`critical`
`TLSCertificateExpiring`	the certificate expires in less than `tls.expiryWarningDays` days (14 by default), for 10 min	`warning`
`TLSCertificateExpiryCritical`	less than `tls.expiryCriticalDays` days (3 by default)	`critical`
Return to normal	the alert stops, if `sendResolved: true`	—

Alerts of the same monitor are grouped: the first one goes out after 30 seconds, and a reminder is sent every repeatInterval (4 h by default) as long as the incident lasts. TLS alerts only concern monitors with tls (HTTPS, or TCP with tcp.tls: true).

The message follows the format of each tool (Slack, Discord, Teams). For a Webhook, the body is JSON ("version": "4", list alerts, each alert with status, labels and annotations); the monitor name is in the sng_monitor label.

Verification

Create a test monitor on an address that does not respond, connected to the channel, with --threshold 1 and the minimum interval of your tier:

sng monitor create test-alerte --http https://does-not-exist.example.com/ --interval 5m --threshold 1 --channel ops

You should receive MonitorDown within the following minutes. To also check the return notification, rerun the command with a URL that responds (--http https://example.com/) and wait for the resolution message. Then delete the monitor: sng monitor delete test-alerte --yes. Deleting or suspending a monitor that is down does not necessarily send a return notification: it is no longer attached to the channel.

Going further

Number of channels: bounded by the tier (2 in 3XS, 5 in 2XS, 10 in XS…). Beyond that, the most recent channels are refused (QuotaExceeded), as are the monitors that reference them.
A channel without a monitor sends nothing.
repeatInterval format: an integer followed by m or h (30m, 4h, 24h).
Maintenance windows: not active yet; see Status page.

Common errors

Symptom	Cause	Fix
`sng apply` refuses: `slack : requis pour le type Slack, interdit sinon`	Block missing or block of another type	A single block, the one of the `type`
`sng apply` refuses: `url ou urlSecretRef : exactement un des deux`	Both given, or neither	Keep one
`sng apply` refuses: `url : https:// requis`	`http://` URL	`https://` URL
Email channel without effect, monitors that reference it refused	Email channels are not available yet (`NotSupported`)	Use a webhook in the meantime
Channel `Accepted=False`, `SecretNotFound`; the other channels work	`urlSecretRef` / `webhookSecretRef` secret not found, key missing, or secret declared without `materialize: true`	Declare the secret with `materialize: true` (section 4); check the key with `sng secret describe`
Channel `Accepted=False`, `Invalid`	The secret's value does not start with `https://`	Fix the value with `sng secret set`
Monitor refused `ChannelNotFound`	Misspelled channel name, or channel refused (quota, email)	Fix the name; check the channel

Prerequisites​

1. Declare a webhook channel​

2. Connect a monitor to the channel​

3. Channel types​

4. Keep the URL out of the manifest​

5. What you receive​

Verification​

Going further​

Common errors​

See also​