TUI integration for extensions and custom tools

The current TUI contract used by packages/coding-agent and hosts/terminal/engine for extension UI, custom tool UI, and custom renderers.

What this subsystem is

The runtime has two layers:

• Rendering engine (hosts/terminal/engine): differential terminal renderer, input dispatch, focus, overlays, cursor placement.
• Integration layer (packages/coding-agent): mounts extension/custom-tool components, wires keybindings/theme, and restores editor state.

Runtime behavior by mode

Screen takeover is a capability the host reports, not a method every host declares. ctx.ui.terminal is undefined wherever there is no terminal, so an extension that needs it checks for it and states what it cannot do:

const terminal = ctx.ui.terminal;
if (!terminal) {
	ctx.ui.notify("This picker needs an interactive terminal.", "warning");
	return;
}
const picked = await terminal.custom<string | undefined>((tui, theme, keybindings, done) => {
	// ...
});

Core component contract (@veyyon/tui)

hosts/terminal/engine/src/core/component-types.ts defines:

export interface Component {
  render(width: number): readonly string[];
  measureHeight?(width: number): number;
  handleInput?(data: string): void;
  wantsKeyRelease?: boolean;
  invalidate?(): void;
  dispose?(): void;
}

Render results are component-owned and immutable to callers; a component that did not change should return the same array reference it returned last time (reference equality is what enables the renderer’s memoization and row virtualization), and must return a new array whenever its content changed.

measureHeight(width) returns the same nonnegative integer as render(width).length without constructing output or advancing render state. Home-screen layout uses this measurement before bounded-tail or full rendering. Components without the method retain their existing measurement path.

Focusable is separate:

export interface Focusable {
  focused: boolean;
  setUseTerminalCursor?(useTerminalCursor: boolean): void;
}

Cursor behavior uses CURSOR_MARKER (not getCursorPosition). Focused components emit the marker in rendered text; TUI extracts it and positions the hardware cursor.

Rendering constraints (terminal safety)

Your render(width) output must be terminal-safe:

1. Do not intentionally exceed width on any line. The renderer truncates overwide non-image lines as a last-resort guard, but components should still return width-safe output.
2. Measure visual width, not string length: use visibleWidth().
3. Truncate/wrap ANSI-aware text with truncateToWidth() / wrapTextWithAnsi().
4. Sanitize tabs/content from external sources using replaceTabs() (and higher-level sanitizers in coding-agent render paths).

The terminal ToolView renderer replaces tabs and shortens embedded home-directory paths in text spans, metadata, notices, and generic argument previews. Path shortening precedes syntax highlighting, diff rendering, and argument-preview truncation. Captured terminal rows use styleTerminalRow().

Minimal pattern:

import { truncateToWidth } from "@veyyon/utils/width";
import { replaceTabs } from "@veyyon/utils/tab-width";

render(width: number): readonly string[] {
  return this.lines.map(line => truncateToWidth(replaceTabs(line), width));
}

Input handling and keybindings

Raw key matching

Use matchesKey(data, "...") for navigation keys and combos.

Match app keybinding actions

Extension UI factories receive a KeybindingsManager (interactive mode; an in-memory instance containing the default bindings, not the user’s keybindings.yml) so you can match action ids instead of hardcoding keys:

if (keybindings.matches(data, "app.interrupt")) {
  done(undefined);
  return;
}

Key release/repeat events

Key release events are filtered unless your component sets:

wantsKeyRelease = true;

Then use isKeyRelease() / isKeyRepeat() if needed.

Focus, overlays, and cursor

• TUI.setFocus(component) routes input to that component.
• Overlay APIs exist in TUI (showOverlay, OverlayHandle). In interactive extension/custom UI, custom(..., { overlay: true }) mounts your component through TUI.showOverlay(...); without overlay, it replaces the editor component area directly.
• Overlay custom UI is anchored at bottom-center with full terminal width and is removed through the returned overlay handle when done(...) closes the flow. It is placed with aboveFooter: true: the overlay covers the transcript region only, and the composer zone (prompt, status line, footline) stays painted under it. Size the component from tui.terminal.rows - tui.pinnedFooterRows; a taller render is clipped to the rows above the footer.
• OverlayOptions.aboveFooter is the general form of that placement. With a pinned footer (TUI.setPinnedFooterChildCount), the screen rows from the footer’s top down are added to the overlay’s bottom margin every frame, so the reserve follows a footer that grows or a frame shorter than the viewport. Without a pinned footer the option does nothing.

Mount points and return contracts

1) Extension UI (ExtensionUIContext)

Current signature (extensibility/extensions/types.ts):

custom<T>(
  factory: (
    tui: TUI,
    theme: Theme,
    keybindings: KeybindingsManager,
    done: (result: T) => void,
  ) => (Component & { dispose?(): void }) | Promise<Component & { dispose?(): void }>,
  options?: { overlay?: boolean | OverlayOptions },
): Promise<T>

Behavior in interactive mode (extension-ui-controller.ts):

• Saves editor text.
• Without options.overlay, replaces the editor component with your component.
• With options.overlay: true, mounts your component as a bottom-centered overlay above the composer zone instead of replacing the editor; the prompt and status line remain visible. An OverlayOptions object mounts it with that geometry instead (a centered card, a fixed width). Either way a component that implements MouseRoutable receives the wheel and click reports inside its bounds.
• Focuses your component.
• On done(result): calls component.dispose?.(), hides the overlay if present, restores editor + text for non-overlay flows, focuses editor, resolves promise. So done(...) is mandatory for completion.

2) Hook/custom-tool UI context (legacy typing)

HookUIContext.terminal.custom is typed as (tui, theme, done) in extensibility/terminal-capability.ts. Underlying interactive implementation calls factories with (tui, theme, keybindings, done). JS consumers can use the extra arg; type-level compatibility still reflects the 3-arg legacy signature.

Custom tools typically use the same UI entrypoint via the factory-scoped pi.ui object, then return the selected value in normal tool content:

async execute(toolCallId, params, onUpdate, ctx, signal) {
  const terminal = pi.ui.terminal;
  if (!terminal) {
    return { content: [{ type: "text", text: "UI unavailable" }] };
  }

  const picked = await terminal.custom<string | undefined>((tui, theme, keybindings, done) => {
    const component = new MyPickerComponent(done, signal);
    return component;
  });

  return { content: [{ type: "text", text: picked ? `Picked: ${picked}` : "Cancelled" }] };
}

3) Custom tool call/result renderers

Custom tools and extension tools define two optional renderers:

• renderCall(args, options, theme)
• renderResult(result, options, theme, args?)

options currently includes:

• expanded: boolean
• isPartial: boolean
• spinnerFrame?: number

Both return HostView, which is whatever the active host draws. In the terminal that is a @veyyon/tui Component, and ToolExecutionComponent mounts it.

The view alternative returns host-independent ToolView values; see custom tool rendering hooks.

Lifecycle and cancellation

• dispose() is optional at type level but should be implemented when you own timers, subprocesses, watchers, sockets, or overlays.
• Container.dispose() and Box.dispose() dispose their children; clear() and removeChild() only detach them.
• Tool cards dispose replaced renderer components and retain reused component instances. Disposing a card also stops its animation clocks and detaches its presentation subscription.
• done(...) should be called exactly once from your component flow.
• For cancellable long-running UI, pair CancellableLoader with AbortSignal and call done(...) from onAbort.

Example cancellation pattern:

const loader = new CancellableLoader(
  tui,
  theme.fg("accent"),
  theme.fg("muted"),
  "Working...",
);
loader.onAbort = () => done(undefined);
void doWork(loader.signal).then((result) => done(result));
return loader;

Realistic custom component example (extension command)

import type { Component } from "@veyyon/tui";
import { SelectList } from "@veyyon/tui";
import { matchesKey } from "@veyyon/utils/keys";
import { truncateToWidth } from "@veyyon/utils/width";
import { replaceTabs } from "@veyyon/utils/tab-width";
import {
  getSelectListTheme,
  type ExtensionAPI,
} from "@veyyon/coding-agent";

class Picker implements Component {
  list: SelectList;
  keybindings: any;
  done: (value: string | undefined) => void;

  constructor(
    items: Array<{ value: string; label: string }>,
    keybindings: any,
    done: (value: string | undefined) => void,
  ) {
    this.list = new SelectList(items, 8, getSelectListTheme());
    this.keybindings = keybindings;
    this.done = done;
    this.list.onSelect = (item) => this.done(item.value);
    this.list.onCancel = () => this.done(undefined);
  }

  handleInput(data: string): void {
    if (this.keybindings.matches(data, "app.interrupt")) {
      this.done(undefined);
      return;
    }
    this.list.handleInput(data);
  }

  render(width: number): readonly string[] {
    return this.list
      .render(width)
      .map((line) => truncateToWidth(replaceTabs(line), width));
  }

  invalidate(): void {
    this.list.invalidate();
  }
}

export default function extension(pi: ExtensionAPI): void {
  pi.registerCommand("pick-model", {
    description: "Pick a model profile",
    handler: async (_args, ctx) => {
      const terminal = ctx.ui.terminal;
      if (!terminal) return;

      const selected = await terminal.custom<string | undefined>(
        (tui, theme, keybindings, done) => {
          const items = [
            { value: "fast", label: theme.fg("accent", "Fast") },
            { value: "balanced", label: "Balanced" },
            { value: "quality", label: "Quality" },
          ];
          return new Picker(items, keybindings, done);
        },
      );

      if (selected) ctx.ui.notify(`Selected profile: ${selected}`, "info");
    },
  });
}

Key implementation files

• hosts/terminal/engine/src/core/tui.ts: terminal rendering, focus, overlays, and input dispatch.
• packages/utils/src/width.ts: width/truncation/sanitization primitives.
• packages/utils/src/keys.ts / keybindings.ts: key parsing and configurable action mapping.
• packages/coding-agent/src/modes/terminal/controllers/extension-ui-controller.ts: interactive mounting/unmounting for extension/hook/custom-tool UI.
• packages/coding-agent/src/extensibility/extensions/types.ts: extension UI and renderer contracts.
• packages/coding-agent/src/extensibility/hooks/types.ts: hook UI contract (legacy custom signature).
• packages/coding-agent/src/extensibility/custom-tools/types.ts: custom tool execute/render contracts.
• packages/coding-agent/src/modes/terminal/components/transcript/tool-execution.ts: mounting renderCall/renderResult components and partial-state options.
• packages/coding-agent/src/modes/terminal/components/transcript/chat-transcript-builder.ts: shared persisted-message replay and live-message dispatch for interactive chat and transcript viewers.
• packages/coding-agent/src/tools/core/context.ts: tool UI context propagation (hasUI, ui).