There are two completely distinct implementations named tag_resolver in the codebase. They serve different purposes and are not related.
tag_resolver.rsSource: clarity:backend/src-tauri/src/api/tag_resolver.rs
Maps tag names to their containing scope (org/site/unit/grid) by scanning the on-disk data/ directory structure. Enables tagname-only writes and queries without requiring callers to supply the full 4-tuple.
pub struct Scope {
pub organization: String,
pub site: String,
pub unit: String,
pub grid: String,
}
// Global in-memory map: tag_name → Arc<Scope>
// Pre-sized to 2,000,000 tags (raised from 200,000) to avoid rehashing on large deployments.
// As of c41a03b the map type is FastMap (std HashMap over ahash::RandomState — see fast_hash.rs).
pub static TAG_SCOPE_MAP: Lazy<RwLock<FastMap<String, Arc<Scope>>>>;
// Build/rebuild the map by scanning data/ directory
pub async fn build_tag_map(storage: Storage);
// Look up scopes for a list of tag names
// Returns HashMap<Arc<Scope>, Vec<String>> — groups tags by scope
pub fn resolve_from_tags(tags: &[String]) -> HashMap<Arc<Scope>, Vec<String>>;
// Incrementally register/remove tags without a full filesystem rebuild
pub fn upsert_tags(scope: Scope, tags: &[String]);
pub fn remove_tags(tags: &[String]);
// Readiness gate (watch channel) — lets build_tag_map run off the boot critical path
pub fn mark_tag_map_ready(); // idempotent; flips ready → true after first population
pub async fn wait_tag_map_ready(); // resolves immediately if ready, else awaits first flip
clarity:backend/src-tauri/src/api/tag_resolver.rs:86-87 (declaration + preallocated capacity), clarity:backend/src-tauri/src/api/tag_resolver.rs:135-196 (resolve_from_tags, upsert_tags, remove_tags)
clarity:backend/src-tauri/src/main.rs:78-81
build_tag_map now runs off the boot critical path — it is spawned as a background task at startup (clarity:backend/src-tauri/src/main.rs:3330-3340) concurrently with HA-agent/Warp init, gated by a tokio::sync::watch readiness channel: build_tag_map calls mark_tag_map_ready() after its first population, and each warp::serve awaits wait_tag_map_ready() before serving, so no handler ever reads a half-built map. clarity:backend/src-tauri/src/api/tag_resolver.rs:102-123; gates at clarity:backend/src-tauri/src/main.rs:4829, 5102create_collection (clarity:backend/src-tauri/src/main.rs:378) and write_tag_mapping (clarity:backend/src-tauri/src/main.rs:1309) call via tokio::spawn57c553a). Rather than waiting for the next full build_tag_map scan, the SQLite tagmeta sync registers newly-added tags into the map immediately via upsert_tags: sync_tagmeta_created upserts each Added tag, and sync_tagmeta_bulk upserts the Added tags per collection. Without this, a write_fast call arriving between a tagmeta upload and the next scan would report the tag as unresolved and drop its data. clarity:backend/src-tauri/src/sqlite_api/sync.rs:584-597, 1064-1078 — see SQLite API § tagmeta sync warm-registration.upsert_tags / remove_tags / snapshot_scope_map are also the primitives the Leader uses to broadcast incremental TagScopeUpsert events so a Secondary's map never goes cold. clarity:backend/src-tauri/src/api/tag_resolver.rs:160-209c41a03b). handle_list_collections is now pub(crate) so the ingest subsystem refreshes TAG_SCOPE_MAP by walking local Storage directly (refresh_tag_scope_map) instead of the old HTTPS GET /collections round-trip. clarity:backend/src-tauri/src/api/tag_resolver.rs:20-23 — see Processing API § Ingest.When a write or query request omits org/site/unit/grid, the handler calls resolve_from_tags to group the requested tags by scope. Tags belonging to different scopes are split into separate write/query operations.
As of
c41a03bthe oldparse_write_requestinmain.rsis removed — the write path is restructured intoresolve_write_scopes(clarity:backend/src-tauri/src/main.rs:177),build_scoped_writes(:203), andrun_write(:400), with the same resolve-and-group semantics.
Refresh-and-retry on stale map (new in 57c553a). The /write, /write_fast, and /write_buffered handlers no longer silently drop tags missing from the map. If resolve_from_tags leaves any request tag unresolved, the handler calls build_tag_map once and retries before reporting the tag as skipped (/write, /write_fast) or erroring (/write_buffered single). See API Server § Scope resolution, refresh-and-retry, and skipped tags.
tag_resolver.pySource: pulse_multi_agents:pulse_manager/sub_agents/meta_data_agent/tag_resolver.py
Resolves natural language tag descriptions (e.g., "outlet temperature of heat exchanger") to exact tag names using semantic similarity matching. Operates within the meta_data_agent to handle user queries that describe sensors in natural language rather than by exact tag ID.
sentence-transformers (all-MiniLM-L6-v2 or equivalent)| Aspect | clarity tag_resolver.rs | pulse tag_resolver.py |
|---|---|---|
| Language | Rust | Python |
| Mechanism | Filesystem directory scan | NLP embedding similarity |
| Purpose | Map tag name → storage scope | Map natural language → tag name |
| Cache | RwLock<HashMap> (in-process, permanent; pre-sized to 2,000,000 tags) |
In-memory, 24h TTL |
| Location | clarity backend | pulse_multi_agents meta_data_agent |
Confirmed from pulse_multi_agents:pulse_manager/sub_agents/meta_data_agent/tag_resolver.py:
"all-mpnet-base-v2" (HuggingFace sentence-transformers; ~80 MB; 768-dimension embeddings, pre-normalized for cosine similarity). Comment notes this is "Upgraded from all-MiniLM-L6-v2 for better accuracy".search_tags(unit_id, query, top_k=10, filters=None, similarity_threshold=0.0):
0.0 (all results pass; caller can raise it)[] (empty list) — no fallback to keyword searchtop_k default is 10; returns results sorted by similarity score descendingLast updated: 2026-07-12 from clarity@45a686e + pulse_multi_agents@e278054