AI Agent & Script Plugins: Deck's Self-Evolving System

A Real Example

You copy a raw JSON response from a terminal and want it pretty-printed. Open Deck and type:

"Format the JSON I just copied."

Deck's AI Agent then does the following:

1. Searched clipboard history, located the JSON copied 30 seconds ago
2. No existing formatter plugin found, so it wrote one in JavaScript
3. Saved it to ~/.deck/scripts/json-formatter/
4. Ran the plugin, formatted the JSON with proper indentation
5. Result is ready in your clipboard history

This isn't a chat UI bolted onto an LLM. Deck's AI has its own tool set, permission boundaries, execution loop, and the ability to extend itself by creating plugins.

Multi-Provider Abstraction: Connecting Any LLM

Most AI apps lock you into a single vendor. Deck adopted a multi-provider architecture from day one. You can freely switch between OpenAI, Anthropic, Ollama local models, or even use the ChatGPT macOS client for free.

Provider Enumeration

enum AIProviderType {
    case chatgpt     // ChatGPT (Web/App)
    case openaiAPI   // OpenAI API (direct)
    case anthropic   // Anthropic Claude
    case ollama      // Ollama local models
}

Unified Protocol

All providers conform to a single AIProviderProtocol with a unified interface:

Send messages with streaming and tool definitions Whether the provider is configured Current model identifier Cancel the in-flight request

AIService acts as the orchestrator, maintaining a providerCache to avoid redundant initialization, and routes to the correct provider with a simple switch:

func configuredProvider() -> AIProviderProtocol {
    switch config.providerType {
    case .chatgpt:   return ChatGPTProvider.shared
    case .openaiAPI: return OpenAIAPIProvider.shared
    case .anthropic: return AnthropicProvider.shared
    case .ollama:    return OllamaProvider.shared
    }
}

The entire Agent system, including tool calls, plugin execution, and Smart Rules, is completely agnostic to the underlying model. Swap a provider, and everything keeps working.

12 Tools: The AI's Capability Boundary

An agent's capability is defined by its available tools. Deck defines 12 tools for its AI, covering the full clipboard management lifecycle:

Authorization Mechanism

Note the [Y] marks: all write, delete, and create operations require explicit user authorization. The auth logic is injected by the UI layer and defaults to rejection when absent. The AI can do a lot, but critical operations always need your approval.

Another detail: tool results are truncated at maxToolSnippetCharacters = 2500 to prevent oversized outputs from exhausting the context window.

Agent Loop: Think, Act, Observe

Deck's AI is not a simple Q&A chatbot. It is a loop-driven Agent with an explicitly defined behavior pattern in the System Prompt:

<agent_loop>
You are a capable agent. Act, don't just describe.

For each request, follow this cycle:
1. Understand the user's intent.
2. Plan internally...
3. Execute: call the necessary tool(s)...
4. Observe and verify each tool result before proceeding.
5. Respond to the user with the final result.

When issuing a tool call, output ONLY the tool call — no surrounding narration.
</agent_loop>

Act, don't just describe sums up the whole approach. When you say "find the links I recently copied," the AI won't respond with "you can use the search feature to look for links." It calls search_clipboard directly, gets results, and tells you what it found.

Streaming Execution

The entire process is streamed via AsyncThrowingStream<AIStreamEvent, Error> with these event types:

Text fragment Tool call request Completion Error

The full tool call flow:

sequenceDiagram
    participant U as 用户 / User
    participant AI as AIService
    participant P as Provider
    participant T as AIToolService

    U->>AI: "格式化最近的 JSON"
    AI->>P: sendMessage(messages, tools)
    P-->>AI: toolCall(search_clipboard)
    AI->>T: execute(search_clipboard)
    T-->>AI: [找到 JSON 记录]
    AI->>P: 继续生成（含工具结果）
    P-->>AI: toolCall(run_script_transform)
    AI->>T: execute(run_script_transform)
    T-->>AI: [格式化结果]
    AI->>P: 继续生成
    P-->>AI: textDelta("已格式化完成...")
    AI-->>U: 最终响应 + 格式化结果

Each tool result is appended to message history as AIMessage.tool(toolCallId:text:). The provider generates the next response based on these results. The Agent loop can chain multiple tool calls until the task is complete.

JavaScriptCore Sandbox: A Secure Plugin Runtime

The plugin system is one of Deck's core features. Each plugin is essentially a JavaScript file running inside a JavaScriptCore sandbox.

Plugin Structure

~/.deck/scripts/
└── my-plugin/
    ├── manifest.json    # 元数据 / Metadata
    └── main.js          # 必须导出 transform(input) / Must export transform(input)

Plugin IDs follow ^[a-z0-9._-]{1,64}$, manifests are capped at 64KB, and scripts at 1MB.

Here's what an AI-generated JSON formatter plugin looks like:

// main.js
function transform(input) {
    try {
        var obj = JSON.parse(input);
        return JSON.stringify(obj, null, 2);
    } catch (e) {
        return "Invalid JSON: " + e.message;
    }
}

Sandbox Security Constraints

Deck imposes strict constraints on the plugin runtime:

How the 5-Second Timeout Works

JavaScriptCore doesn't expose a public execution timeout API. Deck achieves this through private APIs:

static let scriptExecutionTimeout: TimeInterval = 5.0

The approach uses dlopen to load the JavaScriptCore framework, dlsym to resolve JSGlobalContextGetGroup and JSContextGroupSetExecutionTimeLimit, wrapped in a JSExecutionTimeLimiter class. When creating a JS context, it calls setLimit(group, timeLimit, callback, nil), ensuring no script can run longer than 5 seconds.

Plugins requiring network access must declare requiresNetwork: true in the manifest. Runtime uses an isolated ephemeral URLSession, so no cookies or caches from the main app are leaked.

AI-Generated Plugins: A Self-Extending System

This is the most interesting part of Deck's Agent system. The AI can not only use plugins, it can create and modify them.

Generating a New Plugin

generate_script_plugin (Tool #10) lets the AI create a plugin from scratch:

Validate manifest and source code Security validation: Must contain a transform(input) function

• 禁止危险 API：fs、child_process、eval、Function / Dangerous APIs blocked: fs, child_process, eval, Function plugin_id must match naming rules Write to disk at ~/.deck/scripts/{plugin-id}/

Incremental Patching

modify_script_plugin (Tool #11) uses apply_patch-style incremental patching. The AI doesn't need to rewrite the entire file, just describe the changes:

• preparePatchDraft → applyPatch Supports multiple edits in one operation

Cumulative Plugin Effect

Back to the opening example:

User says: "Format my JSON" AI searches history, finds the JSON AI checks existing plugins, no JSON formatter AI writes one, passes security checks, installs it AI immediately uses the new plugin to format the JSON Next time you need JSON formatted? The plugin is already there.

Every time the AI solves a new problem, it leaves behind a reusable plugin. As you use Deck over time, more plugins accumulate, and the system becomes increasingly capable.

Smart Rule Engine: When Conditions Meet AI

Everything so far has been user-initiated. The Smart Rule engine turns the AI into a background automation system. The moment you copy something, rules start firing.

Condition Types

enum RuleCondition {
    case contentContains(String)           // 内容包含 / Content contains
    case contentMatches(String)            // 正则匹配 / Regex match
    case contentType(String)              // 类型匹配 / Type match (text/image/file/url/code/color)
    case sourceApp(String)                // 来源应用 / Source application
    case contentLength(CompareOp, Int)    // 长度比较 / Length comparison
    case hasCustomTitle                   // 有自定义标题 / Has custom title
}

Conditions can be combined with .all (AND) or .any (OR).

Action Types

enum RuleAction {
    case addTag(String)          // 添加标签 / Add tag
    case markSensitive           // 标记敏感 / Mark as sensitive
    case autoDelete(Int)         // N 分钟后自动删除 / Auto-delete after N minutes
    case transform(String)       // 文本转换 / Text transform (built-in or plugin)
    case aiPrompt(String)        // 触发 AI 处理 / Trigger AI processing
    case ignore                  // 不保存 / Don't save to history
}

Execution Flow

flowchart TD
    A[ClipboardService 检测到复制<br/>Copy detected] --> B[processParsedItemWithRules]
    B --> C[SmartRuleService.processItem]
    C --> D{条件求值<br/>Evaluate conditions}
    D -->|命中 ignore| E[丢弃，不入库<br/>Discard]
    D -->|命中 transform| F[先转换再入库<br/>Transform then save]
    D -->|其他动作| G[入库<br/>Save to store]
    F --> G
    G --> H[applyActions]
    H --> I[addTag / markSensitive / autoDelete]
    H --> J[aiPrompt → AI 自动化处理<br/>AI automation]

AI Action Chain

When a rule triggers aiPrompt(prompt), the AI takes over automatically:

Construct a SmartRuleSourceSnapshot with full context of the triggering item (ID, type, source app, content, timestamp)

Call AIService.shared.runSmartRuleAutomation(prompt:source:)

Execute in smartRule(source:) context with auto-approval (autoApprove: true)

Tool scope is restricted to the triggering item only

The AI uses a restricted tool set (Tools 1–7), with no plugin creation or modification allowed

Real-World Scenarios

System Architecture Overview

graph TB
    subgraph Orchestration["编排层 / Orchestration"]
        AIS[AIService<br/>编排中心 / Orchestrator]
    end

    subgraph Providers["模型层 / Model Layer"]
        PP[AIProviderProtocol]
        PP --> CGP[ChatGPTProvider]
        PP --> OAP[OpenAIAPIProvider]
        PP --> ANP[AnthropicProvider]
        PP --> OLP[OllamaProvider]
    end

    subgraph Tools["工具层 / Tool Layer"]
        ATS[AIToolService<br/>12 个工具 / 12 Tools]
    end

    subgraph Plugins["插件层 / Plugin Layer"]
        SPS[ScriptPluginService<br/>JS 沙箱 / JS Sandbox]
        APGS[AIPluginGeneratorService<br/>生成·修改·删除 / Generate·Patch·Delete]
    end

    subgraph Automation["自动化层 / Automation"]
        SRS[SmartRuleService<br/>条件匹配 + 动作 / Conditions + Actions]
    end

    subgraph Storage["数据层 / Data Layer"]
        DDS[DeckDataStore<br/>存储与规则入口 / Storage & Rule Entry]
    end

    AIS <--> PP
    AIS <--> ATS
    ATS --> SPS
    ATS --> APGS
    APGS --> SPS
    SRS --> AIS
    DDS --> SRS
    ATS --> DDS

Power and Constraints: Where to Draw the Line

Deck's Agent design constantly balances one thing: giving the AI enough power to solve problems while ensuring it doesn't do anything the user didn't ask for.

What the AI can do:

12 tools covering the full clipboard lifecycle Write and install plugins to extend its own capabilities Streaming Agent loop for multi-step reasoning and chained tool calls Run autonomously in the background via Smart Rules

What's constrained:

Write and delete operations require user authorization Plugins run in a strict sandbox (no filesystem, no shell, 5s timeout) In Smart Rule context, the AI cannot create new plugins, only use existing tools Tool result truncation prevents context window exhaustion Network permissions require explicit declaration and user confirmation

These constraints aren't about distrusting the AI. In practice, the AI can freely create plugins, but you have to confirm. It can run autonomously in the background, but only with read and search permissions; it can't install new plugins on its own. Power where it matters, limits where they're needed.

That's Deck's AI Agent system: an architecture where AI genuinely participates in the product workflow, rather than just chatting on the side.