Hands, eyes, voice
and ears.

Dom Sentinel · eyes

A browser-side engine that reads the live web and heals itself when it changes. You describe the elements you care about by intent, each with an ordered chain of fallback selectors. Sentinel watches the page; when a site ships new markup and your selectors break, it detects that, proposes replacements, and you swap them in — without shipping a new build.

Getting started

import { DOMSentinel } from '@sekondbrainailabs/kora-sentinel';
import { GroqProvider } from '@sekondbrainailabs/kora-sentinel/providers/groq';

const sentinel = new DOMSentinel({
  appId: 'my-app',
  elements: {
    postContainer: {
      selectors: ['[role="listitem"]', 'article', '.feed-item'],  // ordered fallback chain
      description: 'A feed post container with title and author',
      required: true,
      expectedCount: { min: 1, max: 50 },
    },
  },
  visionAdapter: new GroqProvider({ apiKey: '<YOUR_PROVIDER_KEY>' }),
  onResetDetected: async (result) => {
    if (result.reason !== 'dom_reset') return;
    const { proposedSelectors } = await sentinel.analyze();
    if (proposedSelectors.postContainer) {
      sentinel.updateSelectors('postContainer', proposedSelectors.postContainer);
    }
  },
});

sentinel.start();
sentinel.watch('postContainer', (els) => console.log(`${els.length} posts`));

Constructor options

Methods

Attach and detach the observer. Attributes are deliberately not observed — modern frameworks rewrite data- and aria- on every render, and watching them is pure noise.

Walks the selector chain and returns the first non-empty result. Invalid selectors are skipped rather than thrown.

Pulls named fields out of a matched element using its extract specs.

Per-element found / count / matched-selector / in-expected-range, and a document-level healthy flag that is false when any required element matched nothing.

Subscribe to a debounced batch. Callback errors are caught, so one bad handler can't stop the observer.

Health check, then a capture, then ask the vision provider for replacement selectors. Returns healthBefore, proposedSelectors and visionAvailable. Provider failures are caught, not thrown.

Swap a chain at runtime. Both fire onSelectorsUpdated so you can persist the repair.

Add elements after construction, or reach the registry directly.

Element definitions

An element is described by what it is, not only by how to find it — the description is what a model reads when proposing a repair.

{
  selectors: ['[data-testid="msg"]', 'article', '.message'],  // required, ordered
  description: 'A chat message bubble',                        // required — read by the healer
  required: true,                    // zero matches ⇒ structural break
  expectedCount: { min: 1, max: 200 },
  scope: 'document',                 // or an Element
  fields: {
    author: { selectors: ['.author'], extract: 'text', description: 'Sender name' },
    href:   { selectors: ['a'], extract: 'attr:href | text', description: 'Permalink' },
    posted: { selectors: ['time'], extract: 'attr:datetime',
              validator: '^\\d{4}-\\d{2}-\\d{2}', description: 'ISO timestamp' }
  }
}

A validator regex discards values that don't match, so a mis-healed selector yields nothing rather than garbage. Definitions are validated on registration with precise errors — a missing description or an expectedCount whose min exceeds its max throws immediately rather than failing mysteriously later.

Self-healing

Two independent mechanisms — use either, or both.

Detection

Every debounced mutation batch runs a check. Results carry a reason:

Detections are rate-limited by cooldown — a six-hour window for structural breaks, an hour for quality drops — so a thrashing page can't trigger a repair storm. State persists in extension storage where available, memory otherwise.

Repair, without a model

The interesting path needs no LLM at all. Fingerprint an element while it works, and later find it again by similarity:

A fingerprint captures role, ARIA, heading presence, depth, child count, text shape and neighbouring text. Candidates are scored against it, a stable selector is synthesised for the winner, and an oracle decides whether to accept — rejecting on zero-match, count-explosion, includes-other-role or low-similarity.

Selector synthesis prefers what survives a redesign: roles and ARIA first, then stable data attributes, then ids and classes — explicitly skipping framework-generated hashes and build-time class noise. A proposal is only accepted if it matches tightly, and healElement returns a proposal without mutating your registry, so you decide whether to trust it.

Vision providers

Bring your own. A provider implements three calls — propose selectors from a DOM chunk, count elements visually, and verify an injection. Groq and OpenAI providers ship; the interface is small enough to implement against any model.

Media capture

Materialises images before their object URLs expire, scaling the longest edge down and encoding to base64. Cross-origin images are skipped rather than silently producing a tainted canvas, and avatars, logos and tracking pixels are filtered out by default.

Intercepts a user's file selection before the page uploads it, returning name, MIME type, size and contents. It attaches both directly and via capture-phase delegation, so inputs a framework creates later are still caught. Oversized files are reported with metadata and no payload.

Browser Control · hands

Reading a page is half of it. Browser Control drives one — opening and navigating conversations, scrolling long or virtualised feeds until they settle, and clicking through multi-step task interfaces — so an agent can operate a web app rather than only observe it. It runs on the same self-healing selector engine, so actions survive a redesign.

Scroll until settled, not scroll N times

Infinite feeds don't finish — they converge. Control scrolls in bounded passes and stops when the content stops growing across several consecutive passes, rather than guessing a fixed count. It finds the real scroll container instead of scrolling the window, waits for the message count to hold steady before extracting, and recognises "load more" affordances. Every loop carries a hard pass and time ceiling, so a pathological page ends the crawl instead of hanging it.

Supported surfaces

Each surface has an adapter supplying its navigation, conversation list and message extraction, behind one interface. Where a platform offers no navigable links, Control enumerates by clicking through the list instead — with stall detection so it stops rather than spinning.

Capture is resilient by design: consecutive extraction failures trip a threshold that marks the session degraded and triggers a heal attempt, rather than quietly returning empty results.

Kora Voice · voice

A complete spoken-conversation layer for React: speech-to-text, text-to-speech, voice-activity detection, break detection, a conversation state machine and a waveform UI. You supply one function — how to answer a message — and it handles the rest.

import { VoiceConversationProvider, VoiceConversationUI } from '@kora/voice-conversation';

<VoiceConversationProvider
  config={{
    breakDetection: { enabled: true, silenceThreshold: 1500,
                      volumeThreshold: 20, minSpeechDuration: 500,
                      continuationWindow: 2000 },
    autoSend: { enabled: true, confirmationRequired: false, showTranscript: true },
    providers: {
      stt: { provider: 'openai', apiKey: '<YOUR_KEY>', model: 'whisper-1' },
      tts: { provider: 'openai', apiKey: '<YOUR_KEY>', voice: 'alloy' },
    },
  }}
  onSendMessage={async (text, history) => (await myBackend(text, history)).reply}
>
  <VoiceConversationUI />
</VoiceConversationProvider>

Providers are pluggable: browser-native speech (no key), OpenAI, or Groq for transcription; browser or OpenAI for speech. A custom provider is two methods.

Voice configuration

Break detection — knowing when you've finished

The hard part of voice isn't transcription, it's deciding when a person has actually stopped talking rather than drawn breath. Break detection is a pure, testable function over volume and elapsed time:

That continuation window is why a pause mid-sentence doesn't fire the message off early — the outcomes are explicit: speaking, silence, continuation, new_utterance or break.

Other settings

Pass providers and ui complete — they replace defaults rather than merging.

Hooks & state

The provider component is a convenience. Every layer is available on its own:

The state machine is explicit

idle → listening → processing → sending → thinking → speaking → idle, with waiting_send when you require confirmation and error as a dead end that only reset leaves. A continuation returns listening to itself; an interrupt during thinking or speaking goes straight back to listening. Transitions are exported and validatable, so illegal states are unreachable rather than merely unlikely.

Transcription results carry more than text where the provider supports it — timed segments, per-word timings and speaker labels — which is what makes the diarisation below possible.

Diarisation · ears

Voice tells you what was said. Diarisation tells you who said it. Enrol a speaker once, then identify them in real time in a room of several people — with per-person context and permissions.

Client

A useSpeakerIdentification hook wraps this with attempt limits and caching — once identified it stops asking — and a drop-in VoiceEnrollment component handles the capture flow, requesting a 16 kHz mono stream with echo cancellation and noise suppression.

Identification is two-stage, and says when it isn't sure

A single similarity score is not enough to act on. The engine returns a decision built from several signals:

Enrolment is collision-checked: a new voice too close to a different enrolled user is rejected rather than quietly creating a second identity for the same person.

Beyond identification

Voice signatures are compact embeddings — two engines are selectable, and audio is conditioned first: noise reduction, a speech-band filter, pre-emphasis and level normalisation. Multiple enrolments average into one profile per speaker.

Your data

On the web. Sentinel and Control read and act only on the pages you enable, and only to do the job you asked. During a self-heal, page structure is sent to the vision provider you configure, with conversational content redacted before it leaves. You hold the provider key. The fingerprint-based repair path sends nothing anywhere.

Availability & access

HEVE is in preview. All four capabilities are built and run inside SeKondBrain today. They ship as part of the platform — licensed, not open source — and the packages are distributed to integration partners rather than published openly, so the APIs above describe what you get on access rather than something you can install unaided.

Where it runs

Sentinel, Browser Control and the Voice library run in your own application — they are client-side, and the pages and audio they touch never pass through us. Only diarisation needs a service behind it, and that follows the platform model:

What you get when you're set up