Consumer-facing summary of wiki changes that matter to a reader (new pages, removed features, behavior corrections). Format: Keep a Changelog. The full forensic history is in log.md.
- Notifications Engine — persistent, RBAC-scoped notifications (severity
info/warn/error/critical; monitor/system/admin sources) delivered as OS toasts, in-app banners, and a browser SSE stream (GET /exactapi/notifications/stream); optional email per notification. New reference page: Notifications Engine; Python + TypeScript client SDKs (React/Angular bindings) documented at Notification SDKs.
- Mail / SMTP service — outbound email via an SMTP client (lettre) with a SQLite store-and-forward queue (exponential-backoff retry), a JSONL audit log, and an admin config UI at
/admin/smtp_ui. New page: Mail / SMTP.
- CloudSync — a background task that syncs/migrates recent time-series + collection metadata from the local instance to a remote Clarity instance, controlled via
/exactapi/cloudsync/*. New page: CloudSync. (Not HA-leader-gated — noted as a limitation.)
- Ingest v2 fast path —
POST /ingest/v2/{client}/{config} with the same wire format as v1 but a streaming borrowed-key parse + cached ring bindings. See Processing API.
- Two-tier staging — a sequential
staging.clog chunk log between the ring buffer and day files that removes the write-amplification wall on second-level ingest (shipped enabled). See Storage Engine.
- LoopBack-compatible
/exactapi/collections CRUD (list/get/create/replace/patch/delete + bulk update). See SQLite API.
- HA VIP guard — an out-of-process Windows SYSTEM sidecar that strips the floating IP if the app dies uncleanly, preventing split-brain after a hard kill. See HA.
- Binary query values widened
f32 → f64 (exact 0.001 readback at any magnitude). The wire format_version bumped: basic /exactapi/fast_query_binary v1→v2, optimized /exactapi/fast_query_optimised v2→v3. On-disk storage is unchanged (scaled i32); the SDK decoders accept both old and new. See Queries API.
- JavaScript SDK is now read/query-only — the
write/writeFast/writeBuffered methods and HA shadow dual-write helpers were removed; use the Python SDK or REST for writes. See JavaScript SDK.
- Storage engine
storage.rs split into a storage/ module and the monitor's data reader now delegates to the shared storage read path (staging + ring aware). Behavior-preserving.
- New config sections: notifications retention, SMTP/mail queue, cloudsync, staging, and hardened auth (per-IP login limiter + progressive delay). ⚠️ Note: the shipped
clarity.properties still pins the login limit to 30 and omits the new per-IP keys. See API Server § Config.
- Runtime Authenticode self-verification replaces kernel
/INTEGRITYCHECK; an unsigned/tampered Windows binary refuses to start unless opted out.
- PI connector setup is now a UI-first walkthrough — Set up the PI System connector is rewritten around the in-app flow (add connection → browse AF tree → onboard → map tags → backfill) with screenshot placeholders, instead of raw REST calls. API-based onboarding moved to the PI Connectors API.
- Get-started tutorial — Your first 15 minutes with Pulse Historian (write → read → set a limit → get alerted, one happy path) plus a Get started landing. The
start/ tier is now populated.
- Two new concept explainers — How you get notified (how an alarm reaches a person: desktop toast, bell badge, browser SSE feed, email; RBAC scoping) and Time & timezones (epoch-ms everywhere, the sampling grid, UTC day files, timezone = display only), the latter with a hand-drawn grid/slot diagram.
- Five how-to guides — Create an alarm rule, Back up & restore, Subscribe to live data over MQTT, Enable High Availability, and an SDK quickstart.
- Notifications REST reference — Notifications API group page (send, list, mark-read, unread-count, SSE stream), adopted from the backend developer docs and hardened against the mounted routes.
- How monitoring & alarms work now explains where alarms go — the notification fan-out — closing the earlier "where the results go" gap.
- SQLite API body reconciled — removed the stale narration of the removed typed-macro route layer and the
DYNAMIC_TABLES per-path gate (both gone); documented what actually survives (the CrudModel model layer + its few remaining custom-handler callers) and how dispatch precedence works now.
- REST API reference now in the wiki — a new API reference section (conventions + 10 endpoint-group pages: auth & users, collections & writes, queries/blobs/sensordata, ingest, aggregation, entity CRUD, special resources, monitoring, PI connectors, HA/backup/platform), adopted from the clarity backend developer docs and spot-verified against the routes. Fixes previously-dead API links in the reading guide and indexes. The running backend also serves live Swagger UI at
/docs.
- The landing page is now the Reading Guide (previously the platform Overview). The reader-routing front door is served at the wiki home; the end-to-end platform Overview is now a regular page at overview.md.
- Benchmarks + performance now sourced from the Pulse Historian Benchmark Report (a two-machine field run — an 8-core workstation PC and a 4-core cloud VM). Benchmarks and the Performance brief now report: 50,000 tags @ 1 Hz sustained with 0 dropped points on both machines, 1.0–1.66 M points/sec bulk backfill, a full read-latency grid, and concurrency to ~587 req/s with 0 errors. The earlier single-laptop micro-benchmark figures (and the JSON-vs-binary wire-format-efficiency tables) have been removed.
- HA zero-loss failover (secondary shadow cache) — clients or the server can mirror each write to a Secondary via
POST /exactapi/shadow_write; on promotion the new Leader drains the buffer and gap-fills writes lost during the failover window. Disabled by default (clarity.ha.shadow_cache.* / clarity.ha.shadow_targets); the JS SDK gains shadowTargets/shadowTimeoutMs. See HA § Secondary Shadow Cache.
GET /exactapi/tag_mappings is now reachable — the custom tag-mappings routes (previously 405 on every verb) are mounted, and a filtered GET ?filter={"where":…} returns all matching rows. See SQLite API.CLARITY_BU env var — the default business-unit list stamped into auto-created org/site/unit metadata is now configurable (default ["ems"]).
- Python SDK
mapped_tags on read now resolves via a filtered GET /exactapi/tag_mappings, so one spec can resolve to many tags (any omitted field is unconstrained) and reads never create mapping rows. The JS SDK read path is unchanged (still one-row find-or-create) — the two SDKs now differ here. See Python SDK § mapped_tags.
- Shipped
clarity.properties default flips: clarity.ingest.write.method → fast (code default is buffered), and the hot-ring drain interval 30 s → 60 s.
- Aggregation op parameter values are now lowercase-canonical (
trim/filter/score order & boundary — UPPERCASE still accepted).
- Windows build ships a
requireAdministrator manifest — the app requests UAC elevation once at launch (required for HA VIP/firewall management).
- HA cold-tier reconciliation — sealed
.bin.czst / blob cold-tier days are now digested (over decompressed logical bytes) and repaired between HA peers; previously historic sealed days silently no-op'd, so two nodes could diverge undetected. See HA § Sealed File Reconciliation.
- Gap-fill aggregation timestamps — the
gaps op now anchors to the query start like the rest of the pipeline instead of snapping null points to UTC/epoch boundaries (fixes a mismatch between gap-filled nulls and real aggregated values).
- Windows disk-space guard — the 507 backpressure guard now resolves the data volume on Windows (it previously never blocked there, due to the
\\?\ verbatim path prefix).
- HA admin API — mutating endpoints (
promote/demote/fence/unfence/ops/*) are now bearer-token gated.
- Bulk
POST /{entity}/update now works for TEXT primary keys (e.g. configurations) as well as integer PKs.
- String/array tag values (blob storage) — new
/exactapi/blob_write|blob_query|blob_lastlist|blob_search|blob_state_durations endpoints backed by .blob/.bref sidecar files; see Storage Engine and API Server. (clarity@c41a03b)
POST /exactapi/seal_now — force cold-tier sealing on demand.- Disk-space guard — ingest returns HTTP 507 when free disk drops below
clarity.storage.min_free_disk_mb (default 2 GB).
- Hot ring buffer for ≤1 s-interval collections (RAM row-buffer, background drain).
report-scheduler Python service (port 18000) driving scheduled PDF reports.- SQLite layer:
PATCH /{entity}/:id (LoopBack updateAttributes) and introspection endpoints GET /_tables / GET /_schema.
- Reading Guide (home.md) — the Wiki.js landing page; a front door that routes readers by reader type and by product/deployment.
- Concepts section (explanation layer): Data model & scope, plus The historian, end to end, The agentic AI layer, and High availability, explained.
- How-to Guides — PI connector setup, plus end-user guides Query your data from the terminal and Using the Pulse app (UI scaffold pending screenshots).
- Product introduction (Introducing Pulse) — the two-product platform (Pulse Historian + Pulse Copilot), a legacy-historian comparison (OSI PI, Honeywell, IP.21, Canary), the migrate-vs-keep adoption fork, and deployment.
- Copilot use cases (copilot-use-cases.md) — the plain-English questions Copilot answers, by role, each mapped to a shipping agent capability, with honest boundaries.
- A canonical running example (
acme-power / plant-1 / boiler-2) now used in examples.
- The wiki is being re-oriented from code-structured to reader-need-first (a human "reader layer" now sits on top of the code-cited reference layer). Existing
dev/ and hld/ pages are unchanged.
- Wiki structure: provenance stamps (
> Last updated: … @sha) moved from page tops to a --- footer on all reference pages (tops keep only orientation notes); five thin sub-index.md files (concepts, guides, hld, dev/adr, dev/integrations) folded into the master index and dev index; a styled SVG stack diagram now heads System Context.
- Product taxonomy adopted (matches the website): Pulse (platform) with Pulse Historian (storage engine Pulse Chronos) and Pulse Copilot (Clarity AI engine), deployed edge/on-prem or managed cloud — replacing the old
historian / standalone / cloud-copilot modes.
- Naming reconciled in the reader/product layers to the Pulse names;
dev/ prose stays neutral and code citations stay clarity: (repo name). "Clarity" now denotes only the AI engine, not the platform.
- Overview rewritten to the Pulse taxonomy — products × deployment axes plus a technical-topology table (Historian-only / AI bundled (edge) / AI separate (cloud)) that maps the retired mode names.
[mode:] tags retired wiki-wide. dev/ and hld/ pages now carry a one-line Deployment note or Platform note only where behavior actually differs; deployment pages and hld/deployment-views.md sections are titled by topology; reader/product pages use Applies to: <product> · <deployment>.- OPC-UA reframed — from a flat "not implemented" to: no native connector, with the external OPCConnect gateway (Node.js OPC-UA → MQTT-over-WebSocket) documented from
Docs/OPConnect.pdf, including how its northbound endpoints line up with the historian's /mqtt and /exactapi surfaces.
- New collections use an on-disk V2 XOR codec (zero missing-sentinel → day files born sparse, no fill burst at day roll); existing collections stay V1 and remain readable.
- Cold-tier sealing default: everything before today is compressed (
hot_days=0, hourly sealer — was 7 days, nightly); new seals use a delta+varint codec.
- Ingest restructured: historian write happens first (in-process,
buffered|fast|http selectable) and returns real backpressure (503) / size errors (413); MQTT publishing moved to a decoupled lossy freshest-wins fan-out.
- Buffered-write durability default relaxed: WAL is a flusher-owned journal with
wal_fsync off by default; flush 50 ms / capacity 200k (config-driven).
- SQLite update semantics:
json_patch deep-merge → LoopBack shallow merge on all update paths; neq/nin SQL now matches NULLs (LoopBack semantics).
- Windows installer is now per-machine, code-signed, and preserves
clarity.properties across upgrades; monitor agent polls every 1 s (properties).
- Python SDK
query_optimised(pipeline=…) reroutes to the binary endpoint (same output shape, much faster); both SDK decoders rewritten for bulk decoding.
python_runner.rs / the run_python_binary Tauri command (agent binary is ProcessManager-supervised).- The SQLite query-result cache (
query_cache.rs) — its config knobs are dead.
api/start_adk.rs (dead code) and the HeatRate model.
- 50k-tag query stack overflow (rayon 8 MB stacks + flattened read fan-out) and random Python-service failures after reboot (restore no longer double-owns Python services). (clarity@c41a03b)
- Slow (>90 s) tagmeta nested routes via JSON functional indexes.
- Stale data-plane endpoint paths in the overview:
/api/write, /api/fast_query, … corrected to /exactapi/… (verified against source).
- OPC-UA page citation rot: the
/opc-network stub moved (now main.rs:4066-4077), and opcuser.json was deleted upstream — the opcuser account is now seeded from users.json with its password from the secret pack.
- Windows binaries + bundled assets are Authenticode-signed; the elevated Mosquitto installer refuses to run on manifest mismatch (SHA-256 integrity gate).
- License hardening: fingerprint anchors reject OEM-junk values, activation refuses on insufficient entropy, anti-rollback is a mirrored monotonic high-water mark, and the build manifest is bound to the embedded signing key.
- XSS fix in the process-logs UI; console-window and DLL-search hardening across all child-process spawns.
- HA sealed-file reconciliation does not cover the compressed cold tier — with default sealing, historic days are never checksummed between HA peers; divergence goes undetected. See HA.
- The JS SDK still targets
/api/… paths that the server no longer serves (Python SDK uses /exactapi/…) — see JS SDK.