diff --git a/.agents/skills/add-block/SKILL.md b/.agents/skills/add-block/SKILL.md new file mode 100644 index 00000000000..0133b14e14f --- /dev/null +++ b/.agents/skills/add-block/SKILL.md @@ -0,0 +1,825 @@ +--- +name: add-block +description: Create or update a Sim integration block with correct subBlocks, conditions, dependsOn, modes, canonicalParamId usage, outputs, and tool wiring. Use when working on `apps/sim/blocks/blocks/{service}.ts` or aligning a block with its tools. +--- + +# Add Block Skill + +You are an expert at creating block configurations for Sim. You understand the serializer, subBlock types, conditions, dependsOn, modes, and all UI patterns. + +## Your Task + +When the user asks you to create a block: +1. Create the block file in `apps/sim/blocks/blocks/{service}.ts` +2. Configure all subBlocks with proper types, conditions, and dependencies +3. Wire up tools correctly + +## Block Configuration Structure + +```typescript +import { {ServiceName}Icon } from '@/components/icons' +import type { BlockConfig } from '@/blocks/types' +import { AuthMode } from '@/blocks/types' +import { getScopesForService } from '@/lib/oauth/utils' + +export const {ServiceName}Block: BlockConfig = { + type: '{service}', // snake_case identifier + name: '{Service Name}', // Human readable + description: 'Brief description', // One sentence + longDescription: 'Detailed description for docs', + docsLink: 'https://docs.sim.ai/tools/{service}', + category: 'tools', // 'tools' | 'blocks' | 'triggers' + bgColor: '#HEXCOLOR', // Brand color + icon: {ServiceName}Icon, + + // Auth mode + authMode: AuthMode.OAuth, // or AuthMode.ApiKey + + subBlocks: [ + // Define all UI fields here + ], + + tools: { + access: ['tool_id_1', 'tool_id_2'], // Array of tool IDs this block can use + config: { + tool: (params) => `{service}_${params.operation}`, // Tool selector function + params: (params) => ({ + // Transform subBlock values to tool params + }), + }, + }, + + inputs: { + // Optional: define expected inputs from other blocks + }, + + outputs: { + // Define outputs available to downstream blocks + }, +} +``` + +## SubBlock Types Reference + +**Critical:** Every subblock `id` must be unique within the block. Duplicate IDs cause conflicts even with different conditions. + +### Text Inputs +```typescript +// Single-line input +{ id: 'field', title: 'Label', type: 'short-input', placeholder: '...' } + +// Multi-line input +{ id: 'field', title: 'Label', type: 'long-input', placeholder: '...', rows: 6 } + +// Password input +{ id: 'apiKey', title: 'API Key', type: 'short-input', password: true } +``` + +### Selection Inputs +```typescript +// Dropdown (static options) +{ + id: 'operation', + title: 'Operation', + type: 'dropdown', + options: [ + { label: 'Create', id: 'create' }, + { label: 'Update', id: 'update' }, + ], + value: () => 'create', // Default value function +} + +// Combobox (searchable dropdown) +{ + id: 'field', + title: 'Label', + type: 'combobox', + options: [...], + searchable: true, +} +``` + +### Code/JSON Inputs +```typescript +{ + id: 'code', + title: 'Code', + type: 'code', + language: 'javascript', // 'javascript' | 'json' | 'python' + placeholder: '// Enter code...', +} +``` + +### OAuth/Credentials +```typescript +{ + id: 'credential', + title: 'Account', + type: 'oauth-input', + serviceId: '{service}', // Must match OAuth provider service key + requiredScopes: getScopesForService('{service}'), // Import from @/lib/oauth/utils + placeholder: 'Select account', + required: true, +} +``` + +**Scopes:** Always use `getScopesForService(serviceId)` from `@/lib/oauth/utils` for `requiredScopes`. Never hardcode scope arrays — the single source of truth is `OAUTH_PROVIDERS` in `lib/oauth/oauth.ts`. + +**Scope descriptions:** When adding a new OAuth provider, also add human-readable descriptions for all scopes in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`. + +### Selectors (with dynamic options) +```typescript +// Channel selector (Slack, Discord, etc.) +{ + id: 'channel', + title: 'Channel', + type: 'channel-selector', + serviceId: '{service}', + placeholder: 'Select channel', + dependsOn: ['credential'], +} + +// Project selector (Jira, etc.) +{ + id: 'project', + title: 'Project', + type: 'project-selector', + serviceId: '{service}', + dependsOn: ['credential'], +} + +// File selector (Google Drive, etc.) +{ + id: 'file', + title: 'File', + type: 'file-selector', + serviceId: '{service}', + mimeType: 'application/pdf', + dependsOn: ['credential'], +} + +// User selector +{ + id: 'user', + title: 'User', + type: 'user-selector', + serviceId: '{service}', + dependsOn: ['credential'], +} +``` + +### Other Types +```typescript +// Switch/toggle +{ id: 'enabled', type: 'switch' } + +// Slider +{ id: 'temperature', title: 'Temperature', type: 'slider', min: 0, max: 2, step: 0.1 } + +// Table (key-value pairs) +{ id: 'headers', title: 'Headers', type: 'table', columns: ['Key', 'Value'] } + +// File upload +{ + id: 'files', + title: 'Attachments', + type: 'file-upload', + multiple: true, + acceptedTypes: 'image/*,application/pdf', +} +``` + +## File Input Handling + +When your block accepts file uploads, use the basic/advanced mode pattern with `normalizeFileInput`. + +### Basic/Advanced File Pattern + +```typescript +// Basic mode: Visual file upload +{ + id: 'uploadFile', + title: 'File', + type: 'file-upload', + canonicalParamId: 'file', // Both map to 'file' param + placeholder: 'Upload file', + mode: 'basic', + multiple: false, + required: true, + condition: { field: 'operation', value: 'upload' }, +}, +// Advanced mode: Reference from other blocks +{ + id: 'fileRef', + title: 'File', + type: 'short-input', + canonicalParamId: 'file', // Both map to 'file' param + placeholder: 'Reference file (e.g., {{file_block.output}})', + mode: 'advanced', + required: true, + condition: { field: 'operation', value: 'upload' }, +}, +``` + +**Critical constraints:** +- `canonicalParamId` must NOT match any subblock's `id` in the same block +- Values are stored under subblock `id`, not `canonicalParamId` + +### Normalizing File Input in tools.config + +Use `normalizeFileInput` to handle all input variants: + +```typescript +import { normalizeFileInput } from '@/blocks/utils' + +tools: { + access: ['service_upload'], + config: { + tool: (params) => { + // Check all field IDs: uploadFile (basic), fileRef (advanced), fileContent (legacy) + const normalizedFile = normalizeFileInput( + params.uploadFile || params.fileRef || params.fileContent, + { single: true } + ) + if (normalizedFile) { + params.file = normalizedFile + } + return `service_${params.operation}` + }, + }, +} +``` + +**Why this pattern?** +- Values come through as `params.uploadFile` or `params.fileRef` (the subblock IDs) +- `canonicalParamId` only controls UI/schema mapping, not runtime values +- `normalizeFileInput` handles JSON strings from advanced mode template resolution + +### File Input Types in `inputs` + +Use `type: 'json'` for file inputs: + +```typescript +inputs: { + uploadFile: { type: 'json', description: 'Uploaded file (UserFile)' }, + fileRef: { type: 'json', description: 'File reference from previous block' }, + // Legacy field for backwards compatibility + fileContent: { type: 'string', description: 'Legacy: base64 encoded content' }, +} +``` + +### Multiple Files + +For multiple file uploads: + +```typescript +{ + id: 'attachments', + title: 'Attachments', + type: 'file-upload', + multiple: true, // Allow multiple files + maxSize: 25, // Max size in MB per file + acceptedTypes: 'image/*,application/pdf,.doc,.docx', +} + +// In tools.config: +const normalizedFiles = normalizeFileInput( + params.attachments || params.attachmentRefs, + // No { single: true } - returns array +) +if (normalizedFiles) { + params.files = normalizedFiles +} +``` + +## Condition Syntax + +Controls when a field is shown based on other field values. + +### Simple Condition +```typescript +condition: { field: 'operation', value: 'create' } +// Shows when operation === 'create' +``` + +### Multiple Values (OR) +```typescript +condition: { field: 'operation', value: ['create', 'update'] } +// Shows when operation is 'create' OR 'update' +``` + +### Negation +```typescript +condition: { field: 'operation', value: 'delete', not: true } +// Shows when operation !== 'delete' +``` + +### Compound (AND) +```typescript +condition: { + field: 'operation', + value: 'send', + and: { + field: 'type', + value: 'dm', + not: true, + } +} +// Shows when operation === 'send' AND type !== 'dm' +``` + +### Complex Example +```typescript +condition: { + field: 'operation', + value: ['list', 'search'], + not: true, + and: { + field: 'authMethod', + value: 'oauth', + } +} +// Shows when operation NOT in ['list', 'search'] AND authMethod === 'oauth' +``` + +## DependsOn Pattern + +Controls when a field is enabled and when its options are refetched. + +### Simple Array (all must be set) +```typescript +dependsOn: ['credential'] +// Enabled only when credential has a value +// Options refetch when credential changes + +dependsOn: ['credential', 'projectId'] +// Enabled only when BOTH have values +``` + +### Complex (all + any) +```typescript +dependsOn: { + all: ['authMethod'], // All must be set + any: ['credential', 'apiKey'] // At least one must be set +} +// Enabled when authMethod is set AND (credential OR apiKey is set) +``` + +## Required Pattern + +Can be boolean or condition-based. + +### Simple Boolean +```typescript +required: true +required: false +``` + +### Conditional Required +```typescript +required: { field: 'operation', value: 'create' } +// Required only when operation === 'create' + +required: { field: 'operation', value: ['create', 'update'] } +// Required when operation is 'create' OR 'update' +``` + +## Mode Pattern (Basic vs Advanced) + +Controls which UI view shows the field. + +### Mode Options +- `'basic'` - Only in basic view (default UI) +- `'advanced'` - Only in advanced view +- `'both'` - Both views (default if not specified) +- `'trigger'` - Only in trigger configuration + +### canonicalParamId Pattern + +Maps multiple UI fields to a single serialized parameter: + +```typescript +// Basic mode: Visual selector +{ + id: 'channel', + title: 'Channel', + type: 'channel-selector', + mode: 'basic', + canonicalParamId: 'channel', // Both map to 'channel' param + dependsOn: ['credential'], +} + +// Advanced mode: Manual input +{ + id: 'channelId', + title: 'Channel ID', + type: 'short-input', + mode: 'advanced', + canonicalParamId: 'channel', // Both map to 'channel' param + placeholder: 'Enter channel ID manually', +} +``` + +**How it works:** +- In basic mode: `channel` selector value → `params.channel` +- In advanced mode: `channelId` input value → `params.channel` +- The serializer consolidates based on current mode + +**Critical constraints:** +- `canonicalParamId` must NOT match any other subblock's `id` in the same block (causes conflicts) +- `canonicalParamId` must be unique per block (only one basic/advanced pair per canonicalParamId) +- ONLY use `canonicalParamId` to link basic/advanced mode alternatives for the same logical parameter +- Do NOT use it for any other purpose + +## WandConfig Pattern + +Enables AI-assisted field generation. + +```typescript +{ + id: 'query', + title: 'Query', + type: 'code', + language: 'json', + wandConfig: { + enabled: true, + prompt: 'Generate a query based on the user request. Return ONLY the JSON.', + placeholder: 'Describe what you want to query...', + generationType: 'json-object', // Optional: affects AI behavior + maintainHistory: true, // Optional: keeps conversation context + }, +} +``` + +### Generation Types +- `'javascript-function-body'` - JS code generation +- `'json-object'` - Raw JSON (adds "no markdown" instruction) +- `'json-schema'` - JSON Schema definitions +- `'sql-query'` - SQL statements +- `'timestamp'` - Adds current date/time context + +## Tools Configuration + +**Important:** `tools.config.tool` runs during serialization before variable resolution. Put `Number()` and other type coercions in `tools.config.params` instead, which runs at execution time after variables are resolved. + +**Preferred:** Use tool names directly as dropdown option IDs to avoid switch cases: +```typescript +// Dropdown options use tool IDs directly +options: [ + { label: 'Create', id: 'service_create' }, + { label: 'Read', id: 'service_read' }, +] + +// Tool selector just returns the operation value +tool: (params) => params.operation, +``` + +### With Parameter Transformation +```typescript +tools: { + access: ['service_action'], + config: { + tool: (params) => 'service_action', + params: (params) => ({ + id: params.resourceId, + data: typeof params.data === 'string' ? JSON.parse(params.data) : params.data, + }), + }, +} +``` + +### V2 Versioned Tool Selector +```typescript +import { createVersionedToolSelector } from '@/blocks/utils' + +tools: { + access: [ + 'service_create_v2', + 'service_read_v2', + 'service_update_v2', + ], + config: { + tool: createVersionedToolSelector({ + baseToolSelector: (params) => `service_${params.operation}`, + suffix: '_v2', + fallbackToolId: 'service_create_v2', + }), + }, +} +``` + +## Outputs Definition + +**IMPORTANT:** Block outputs have a simpler schema than tool outputs. Block outputs do NOT support: +- `optional: true` - This is only for tool outputs +- `items` property - This is only for tool outputs with array types + +Block outputs only support: +- `type` - The data type ('string', 'number', 'boolean', 'json', 'array') +- `description` - Human readable description +- Nested object structure (for complex types) + +```typescript +outputs: { + // Simple outputs + id: { type: 'string', description: 'Resource ID' }, + success: { type: 'boolean', description: 'Whether operation succeeded' }, + + // Use type: 'json' for complex objects or arrays (NOT type: 'array' with items) + items: { type: 'json', description: 'List of items' }, + metadata: { type: 'json', description: 'Response metadata' }, + + // Nested outputs (for structured data) + user: { + id: { type: 'string', description: 'User ID' }, + name: { type: 'string', description: 'User name' }, + email: { type: 'string', description: 'User email' }, + }, +} +``` + +### Typed JSON Outputs + +When using `type: 'json'` and you know the object shape in advance, **describe the inner fields in the description** so downstream blocks know what properties are available. For well-known, stable objects, use nested output definitions instead: + +```typescript +outputs: { + // BAD: Opaque json with no info about what's inside + plan: { type: 'json', description: 'Zone plan information' }, + + // GOOD: Describe the known fields in the description + plan: { + type: 'json', + description: 'Zone plan information (id, name, price, currency, frequency, is_subscribed)', + }, + + // BEST: Use nested output definition when the shape is stable and well-known + plan: { + id: { type: 'string', description: 'Plan identifier' }, + name: { type: 'string', description: 'Plan name' }, + price: { type: 'number', description: 'Plan price' }, + currency: { type: 'string', description: 'Price currency' }, + }, +} +``` + +Use the nested pattern when: +- The object has a small, stable set of fields (< 10) +- Downstream blocks will commonly access specific properties +- The API response shape is well-documented and unlikely to change + +Use `type: 'json'` with a descriptive string when: +- The object has many fields or a dynamic shape +- It represents a list/array of items +- The shape varies by operation + +## V2 Block Pattern + +When creating V2 blocks (alongside legacy V1): + +```typescript +// V1 Block - mark as legacy +export const ServiceBlock: BlockConfig = { + type: 'service', + name: 'Service (Legacy)', + hideFromToolbar: true, // Hide from toolbar + // ... rest of config +} + +// V2 Block - visible, uses V2 tools +export const ServiceV2Block: BlockConfig = { + type: 'service_v2', + name: 'Service', // Clean name + hideFromToolbar: false, // Visible + subBlocks: ServiceBlock.subBlocks, // Reuse UI + tools: { + access: ServiceBlock.tools?.access?.map(id => `${id}_v2`) || [], + config: { + tool: createVersionedToolSelector({ + baseToolSelector: (params) => (ServiceBlock.tools?.config as any)?.tool(params), + suffix: '_v2', + fallbackToolId: 'service_default_v2', + }), + params: ServiceBlock.tools?.config?.params, + }, + }, + outputs: { + // Flat, API-aligned outputs (not wrapped in content/metadata) + }, +} +``` + +## Registering Blocks + +After creating the block, remind the user to: +1. Import in `apps/sim/blocks/registry.ts` +2. Add to the `registry` object (alphabetically): + +```typescript +import { ServiceBlock } from '@/blocks/blocks/service' + +export const registry: Record = { + // ... existing blocks ... + service: ServiceBlock, +} +``` + +## Complete Example + +```typescript +import { ServiceIcon } from '@/components/icons' +import type { BlockConfig } from '@/blocks/types' +import { AuthMode } from '@/blocks/types' +import { getScopesForService } from '@/lib/oauth/utils' + +export const ServiceBlock: BlockConfig = { + type: 'service', + name: 'Service', + description: 'Integrate with Service API', + longDescription: 'Full description for documentation...', + docsLink: 'https://docs.sim.ai/tools/service', + category: 'tools', + bgColor: '#FF6B6B', + icon: ServiceIcon, + authMode: AuthMode.OAuth, + + subBlocks: [ + { + id: 'operation', + title: 'Operation', + type: 'dropdown', + options: [ + { label: 'Create', id: 'create' }, + { label: 'Read', id: 'read' }, + { label: 'Update', id: 'update' }, + { label: 'Delete', id: 'delete' }, + ], + value: () => 'create', + }, + { + id: 'credential', + title: 'Service Account', + type: 'oauth-input', + serviceId: 'service', + requiredScopes: getScopesForService('service'), + placeholder: 'Select account', + required: true, + }, + { + id: 'resourceId', + title: 'Resource ID', + type: 'short-input', + placeholder: 'Enter resource ID', + condition: { field: 'operation', value: ['read', 'update', 'delete'] }, + required: { field: 'operation', value: ['read', 'update', 'delete'] }, + }, + { + id: 'name', + title: 'Name', + type: 'short-input', + placeholder: 'Resource name', + condition: { field: 'operation', value: ['create', 'update'] }, + required: { field: 'operation', value: 'create' }, + }, + ], + + tools: { + access: ['service_create', 'service_read', 'service_update', 'service_delete'], + config: { + tool: (params) => `service_${params.operation}`, + }, + }, + + outputs: { + id: { type: 'string', description: 'Resource ID' }, + name: { type: 'string', description: 'Resource name' }, + createdAt: { type: 'string', description: 'Creation timestamp' }, + }, +} +``` + +## Connecting Blocks with Triggers + +If the service supports webhooks, connect the block to its triggers. + +```typescript +import { getTrigger } from '@/triggers' + +export const ServiceBlock: BlockConfig = { + // ... basic config ... + + triggers: { + enabled: true, + available: ['service_event_a', 'service_event_b', 'service_webhook'], + }, + + subBlocks: [ + // Tool subBlocks first... + { id: 'operation', /* ... */ }, + + // Then spread trigger subBlocks + ...getTrigger('service_event_a').subBlocks, + ...getTrigger('service_event_b').subBlocks, + ...getTrigger('service_webhook').subBlocks, + ], +} +``` + +See the `/add-trigger` skill for creating triggers. + +## Icon Requirement + +If the icon doesn't already exist in `@/components/icons.tsx`, **do NOT search for it yourself**. After completing the block, ask the user to provide the SVG: + +``` +The block is complete, but I need an icon for {Service}. +Please provide the SVG and I'll convert it to a React component. + +You can usually find this in the service's brand/press kit page, or copy it from their website. +``` + +## Advanced Mode for Optional Fields + +Optional fields that are rarely used should be set to `mode: 'advanced'` so they don't clutter the basic UI. This includes: +- Pagination tokens +- Time range filters (start/end time) +- Sort order options +- Reply settings +- Rarely used IDs (e.g., reply-to tweet ID, quote tweet ID) +- Max results / limits + +```typescript +{ + id: 'startTime', + title: 'Start Time', + type: 'short-input', + placeholder: 'ISO 8601 timestamp', + condition: { field: 'operation', value: ['search', 'list'] }, + mode: 'advanced', // Rarely used, hide from basic view +} +``` + +## WandConfig for Complex Inputs + +Use `wandConfig` for fields that are hard to fill out manually, such as timestamps, comma-separated lists, and complex query strings. This gives users an AI-assisted input experience. + +```typescript +// Timestamps - use generationType: 'timestamp' to inject current date context +{ + id: 'startTime', + title: 'Start Time', + type: 'short-input', + mode: 'advanced', + wandConfig: { + enabled: true, + prompt: 'Generate an ISO 8601 timestamp based on the user description. Return ONLY the timestamp string.', + generationType: 'timestamp', + }, +} + +// Comma-separated lists - simple prompt without generationType +{ + id: 'mediaIds', + title: 'Media IDs', + type: 'short-input', + mode: 'advanced', + wandConfig: { + enabled: true, + prompt: 'Generate a comma-separated list of media IDs. Return ONLY the comma-separated values.', + }, +} +``` + +## Naming Convention + +All tool IDs referenced in `tools.access` and returned by `tools.config.tool` MUST use `snake_case` (e.g., `x_create_tweet`, `slack_send_message`). Never use camelCase or PascalCase. + +## Checklist Before Finishing + +- [ ] All subBlocks have `id`, `title` (except switch), and `type` +- [ ] Conditions use correct syntax (field, value, not, and) +- [ ] DependsOn set for fields that need other values +- [ ] Required fields marked correctly (boolean or condition) +- [ ] OAuth inputs have correct `serviceId` and `requiredScopes: getScopesForService(serviceId)` +- [ ] Scope descriptions added to `SCOPE_DESCRIPTIONS` in `lib/oauth/utils.ts` for any new scopes +- [ ] Tools.access lists all tool IDs (snake_case) +- [ ] Tools.config.tool returns correct tool ID (snake_case) +- [ ] Outputs match tool outputs +- [ ] Block registered in registry.ts +- [ ] If icon missing: asked user to provide SVG +- [ ] If triggers exist: `triggers` config set, trigger subBlocks spread +- [ ] Optional/rarely-used fields set to `mode: 'advanced'` +- [ ] Timestamps and complex inputs have `wandConfig` enabled + +## Final Validation (Required) + +After creating the block, you MUST validate it against every tool it references: + +1. **Read every tool definition** that appears in `tools.access` — do not skip any +2. **For each tool, verify the block has correct:** + - SubBlock inputs that cover all required tool params (with correct `condition` to show for that operation) + - SubBlock input types that match the tool param types (e.g., dropdown for enums, short-input for strings) + - `tools.config.params` correctly maps subBlock IDs to tool param names (if they differ) + - Type coercions in `tools.config.params` for any params that need conversion (Number(), Boolean(), JSON.parse()) +3. **Verify block outputs** cover the key fields returned by all tools +4. **Verify conditions** — each subBlock should only show for the operations that actually use it diff --git a/.agents/skills/add-block/agents/openai.yaml b/.agents/skills/add-block/agents/openai.yaml new file mode 100644 index 00000000000..2817f341be4 --- /dev/null +++ b/.agents/skills/add-block/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Add Block" + short_description: "Build a Sim block definition" + brand_color: "#2563EB" + default_prompt: "Use $add-block to create or update the block for a Sim integration." diff --git a/.agents/skills/add-connector/SKILL.md b/.agents/skills/add-connector/SKILL.md new file mode 100644 index 00000000000..b26718f92f8 --- /dev/null +++ b/.agents/skills/add-connector/SKILL.md @@ -0,0 +1,437 @@ +--- +name: add-connector +description: Add or update a Sim knowledge base connector for syncing documents from an external source, including auth mode, config fields, pagination, document mapping, tags, and registry wiring. Use when working in `apps/sim/connectors/{service}/` or adding a new external document source. +--- + +# Add Connector Skill + +You are an expert at adding knowledge base connectors to Sim. A connector syncs documents from an external source (Confluence, Google Drive, Notion, etc.) into a knowledge base. + +## Your Task + +When the user asks you to create a connector: +1. Use Context7 or WebFetch to read the service's API documentation +2. Determine the auth mode: **OAuth** (if Sim already has an OAuth provider for the service) or **API key** (if the service uses API key / Bearer token auth) +3. Create the connector directory and config +4. Register it in the connector registry + +## Directory Structure + +Create files in `apps/sim/connectors/{service}/`: +``` +connectors/{service}/ +├── index.ts # Barrel export +└── {service}.ts # ConnectorConfig definition +``` + +## Authentication + +Connectors use a discriminated union for auth config (`ConnectorAuthConfig` in `connectors/types.ts`): + +```typescript +type ConnectorAuthConfig = + | { mode: 'oauth'; provider: OAuthService; requiredScopes?: string[] } + | { mode: 'apiKey'; label?: string; placeholder?: string } +``` + +### OAuth mode +For services with existing OAuth providers in `apps/sim/lib/oauth/types.ts`. The `provider` must match an `OAuthService`. The modal shows a credential picker and handles token refresh automatically. + +### API key mode +For services that use API key / Bearer token auth. The modal shows a password input with the configured `label` and `placeholder`. The API key is encrypted at rest using AES-256-GCM and stored in a dedicated `encryptedApiKey` column on the connector record. The sync engine decrypts it automatically — connectors receive the raw access token in `listDocuments`, `getDocument`, and `validateConfig`. + +## ConnectorConfig Structure + +### OAuth connector example + +```typescript +import { createLogger } from '@sim/logger' +import { {Service}Icon } from '@/components/icons' +import { fetchWithRetry } from '@/lib/knowledge/documents/utils' +import type { ConnectorConfig, ExternalDocument, ExternalDocumentList } from '@/connectors/types' + +const logger = createLogger('{Service}Connector') + +export const {service}Connector: ConnectorConfig = { + id: '{service}', + name: '{Service}', + description: 'Sync documents from {Service} into your knowledge base', + version: '1.0.0', + icon: {Service}Icon, + + auth: { + mode: 'oauth', + provider: '{service}', // Must match OAuthService in lib/oauth/types.ts + requiredScopes: ['read:...'], + }, + + configFields: [ + // Rendered dynamically by the add-connector modal UI + // Supports 'short-input' and 'dropdown' types + ], + + listDocuments: async (accessToken, sourceConfig, cursor) => { + // Paginate via cursor, extract text, compute SHA-256 hash + // Return { documents: ExternalDocument[], nextCursor?, hasMore } + }, + + getDocument: async (accessToken, sourceConfig, externalId) => { + // Return ExternalDocument or null + }, + + validateConfig: async (accessToken, sourceConfig) => { + // Return { valid: true } or { valid: false, error: 'message' } + }, + + // Optional: map source metadata to semantic tag keys (translated to slots by sync engine) + mapTags: (metadata) => { + // Return Record with keys matching tagDefinitions[].id + }, +} +``` + +### API key connector example + +```typescript +export const {service}Connector: ConnectorConfig = { + id: '{service}', + name: '{Service}', + description: 'Sync documents from {Service} into your knowledge base', + version: '1.0.0', + icon: {Service}Icon, + + auth: { + mode: 'apiKey', + label: 'API Key', // Shown above the input field + placeholder: 'Enter your {Service} API key', // Input placeholder + }, + + configFields: [ /* ... */ ], + listDocuments: async (accessToken, sourceConfig, cursor) => { /* ... */ }, + getDocument: async (accessToken, sourceConfig, externalId) => { /* ... */ }, + validateConfig: async (accessToken, sourceConfig) => { /* ... */ }, +} +``` + +## ConfigField Types + +The add-connector modal renders these automatically — no custom UI needed. + +Three field types are supported: `short-input`, `dropdown`, and `selector`. + +```typescript +// Text input +{ + id: 'domain', + title: 'Domain', + type: 'short-input', + placeholder: 'yoursite.example.com', + required: true, +} + +// Dropdown (static options) +{ + id: 'contentType', + title: 'Content Type', + type: 'dropdown', + required: false, + options: [ + { label: 'Pages only', id: 'page' }, + { label: 'Blog posts only', id: 'blogpost' }, + { label: 'All content', id: 'all' }, + ], +} +``` + +## Dynamic Selectors (Canonical Pairs) + +Use `type: 'selector'` to fetch options dynamically from the existing selector registry (`hooks/selectors/registry.ts`). Selectors are always paired with a manual fallback input using the **canonical pair** pattern — a `selector` field (basic mode) and a `short-input` field (advanced mode) linked by `canonicalParamId`. + +The user sees a toggle button (ArrowLeftRight) to switch between the selector dropdown and manual text input. On submit, the modal resolves each canonical pair to the active mode's value, keyed by `canonicalParamId`. + +### Rules + +1. **Every selector field MUST have a canonical pair** — a corresponding `short-input` (or `dropdown`) field with the same `canonicalParamId` and `mode: 'advanced'`. +2. **`required` must be set identically on both fields** in a pair. If the selector is required, the manual input must also be required. +3. **`canonicalParamId` must match the key the connector expects in `sourceConfig`** (e.g. `baseId`, `channel`, `teamId`). The advanced field's `id` should typically match `canonicalParamId`. +4. **`dependsOn` references the selector field's `id`**, not the `canonicalParamId`. The modal propagates dependency clearing across canonical siblings automatically — changing either field in a parent pair clears dependent children. + +### Selector canonical pair example (Airtable base → table cascade) + +```typescript +configFields: [ + // Base: selector (basic) + manual (advanced) + { + id: 'baseSelector', + title: 'Base', + type: 'selector', + selectorKey: 'airtable.bases', // Must exist in hooks/selectors/registry.ts + canonicalParamId: 'baseId', + mode: 'basic', + placeholder: 'Select a base', + required: true, + }, + { + id: 'baseId', + title: 'Base ID', + type: 'short-input', + canonicalParamId: 'baseId', + mode: 'advanced', + placeholder: 'e.g. appXXXXXXXXXXXXXX', + required: true, + }, + // Table: selector depends on base (basic) + manual (advanced) + { + id: 'tableSelector', + title: 'Table', + type: 'selector', + selectorKey: 'airtable.tables', + canonicalParamId: 'tableIdOrName', + mode: 'basic', + dependsOn: ['baseSelector'], // References the selector field ID + placeholder: 'Select a table', + required: true, + }, + { + id: 'tableIdOrName', + title: 'Table Name or ID', + type: 'short-input', + canonicalParamId: 'tableIdOrName', + mode: 'advanced', + placeholder: 'e.g. Tasks', + required: true, + }, + // Non-selector fields stay as-is + { id: 'maxRecords', title: 'Max Records', type: 'short-input', ... }, +] +``` + +### Selector with domain dependency (Jira/Confluence pattern) + +When a selector depends on a plain `short-input` field (no canonical pair), `dependsOn` references that field's `id` directly. The `domain` field's value maps to `SelectorContext.domain` automatically via `SELECTOR_CONTEXT_FIELDS`. + +```typescript +configFields: [ + { + id: 'domain', + title: 'Jira Domain', + type: 'short-input', + placeholder: 'yoursite.atlassian.net', + required: true, + }, + { + id: 'projectSelector', + title: 'Project', + type: 'selector', + selectorKey: 'jira.projects', + canonicalParamId: 'projectKey', + mode: 'basic', + dependsOn: ['domain'], + placeholder: 'Select a project', + required: true, + }, + { + id: 'projectKey', + title: 'Project Key', + type: 'short-input', + canonicalParamId: 'projectKey', + mode: 'advanced', + placeholder: 'e.g. ENG, PROJ', + required: true, + }, +] +``` + +### How `dependsOn` maps to `SelectorContext` + +The connector selector field builds a `SelectorContext` from dependency values. For the mapping to work, each dependency's `canonicalParamId` (or field `id` for non-canonical fields) must exist in `SELECTOR_CONTEXT_FIELDS` (`lib/workflows/subblocks/context.ts`): + +``` +oauthCredential, domain, teamId, projectId, knowledgeBaseId, planId, +siteId, collectionId, spreadsheetId, fileId, baseId, datasetId, serviceDeskId +``` + +### Available selector keys + +Check `hooks/selectors/types.ts` for the full `SelectorKey` union. Common ones for connectors: + +| SelectorKey | Context Deps | Returns | +|-------------|-------------|---------| +| `airtable.bases` | credential | Base ID + name | +| `airtable.tables` | credential, `baseId` | Table ID + name | +| `slack.channels` | credential | Channel ID + name | +| `gmail.labels` | credential | Label ID + name | +| `google.calendar` | credential | Calendar ID + name | +| `linear.teams` | credential | Team ID + name | +| `linear.projects` | credential, `teamId` | Project ID + name | +| `jira.projects` | credential, `domain` | Project key + name | +| `confluence.spaces` | credential, `domain` | Space key + name | +| `notion.databases` | credential | Database ID + name | +| `asana.workspaces` | credential | Workspace GID + name | +| `microsoft.teams` | credential | Team ID + name | +| `microsoft.channels` | credential, `teamId` | Channel ID + name | +| `webflow.sites` | credential | Site ID + name | +| `outlook.folders` | credential | Folder ID + name | + +## ExternalDocument Shape + +Every document returned from `listDocuments`/`getDocument` must include: + +```typescript +{ + externalId: string // Source-specific unique ID + title: string // Document title + content: string // Extracted plain text + mimeType: 'text/plain' // Always text/plain (content is extracted) + contentHash: string // SHA-256 of content (change detection) + sourceUrl?: string // Link back to original (stored on document record) + metadata?: Record // Source-specific data (fed to mapTags) +} +``` + +## Content Hashing (Required) + +The sync engine uses content hashes for change detection: + +```typescript +async function computeContentHash(content: string): Promise { + const data = new TextEncoder().encode(content) + const hashBuffer = await crypto.subtle.digest('SHA-256', data) + return Array.from(new Uint8Array(hashBuffer)).map(b => b.toString(16).padStart(2, '0')).join('') +} +``` + +## tagDefinitions — Declared Tag Definitions + +Declare which tags the connector populates using semantic IDs. Shown in the add-connector modal as opt-out checkboxes. +On connector creation, slots are **dynamically assigned** via `getNextAvailableSlot` — connectors never hardcode slot names. + +```typescript +tagDefinitions: [ + { id: 'labels', displayName: 'Labels', fieldType: 'text' }, + { id: 'version', displayName: 'Version', fieldType: 'number' }, + { id: 'lastModified', displayName: 'Last Modified', fieldType: 'date' }, +], +``` + +Each entry has: +- `id`: Semantic key matching a key returned by `mapTags` (e.g. `'labels'`, `'version'`) +- `displayName`: Human-readable name shown in the UI (e.g. "Labels", "Last Modified") +- `fieldType`: `'text'` | `'number'` | `'date'` | `'boolean'` — determines which slot pool to draw from + +Users can opt out of specific tags in the modal. Disabled IDs are stored in `sourceConfig.disabledTagIds`. +The assigned mapping (`semantic id → slot`) is stored in `sourceConfig.tagSlotMapping`. + +## mapTags — Metadata to Semantic Keys + +Maps source metadata to semantic tag keys. Required if `tagDefinitions` is set. +The sync engine calls this automatically and translates semantic keys to actual DB slots +using the `tagSlotMapping` stored on the connector. + +Return keys must match the `id` values declared in `tagDefinitions`. + +```typescript +mapTags: (metadata: Record): Record => { + const result: Record = {} + + // Validate arrays before casting — metadata may be malformed + const labels = Array.isArray(metadata.labels) ? (metadata.labels as string[]) : [] + if (labels.length > 0) result.labels = labels.join(', ') + + // Validate numbers — guard against NaN + if (metadata.version != null) { + const num = Number(metadata.version) + if (!Number.isNaN(num)) result.version = num + } + + // Validate dates — guard against Invalid Date + if (typeof metadata.lastModified === 'string') { + const date = new Date(metadata.lastModified) + if (!Number.isNaN(date.getTime())) result.lastModified = date + } + + return result +} +``` + +## External API Calls — Use `fetchWithRetry` + +All external API calls must use `fetchWithRetry` from `@/lib/knowledge/documents/utils` instead of raw `fetch()`. This provides exponential backoff with retries on 429/502/503/504 errors. It returns a standard `Response` — all `.ok`, `.json()`, `.text()` checks work unchanged. + +For `validateConfig` (user-facing, called on save), pass `VALIDATE_RETRY_OPTIONS` to cap wait time at ~7s. Background operations (`listDocuments`, `getDocument`) use the built-in defaults (5 retries, ~31s max). + +```typescript +import { VALIDATE_RETRY_OPTIONS, fetchWithRetry } from '@/lib/knowledge/documents/utils' + +// Background sync — use defaults +const response = await fetchWithRetry(url, { + method: 'GET', + headers: { Authorization: `Bearer ${accessToken}` }, +}) + +// validateConfig — tighter retry budget +const response = await fetchWithRetry(url, { ... }, VALIDATE_RETRY_OPTIONS) +``` + +## sourceUrl + +If `ExternalDocument.sourceUrl` is set, the sync engine stores it on the document record. Always construct the full URL (not a relative path). + +## Sync Engine Behavior (Do Not Modify) + +The sync engine (`lib/knowledge/connectors/sync-engine.ts`) is connector-agnostic. It: +1. Calls `listDocuments` with pagination until `hasMore` is false +2. Compares `contentHash` to detect new/changed/unchanged documents +3. Stores `sourceUrl` and calls `mapTags` on insert/update automatically +4. Handles soft-delete of removed documents +5. Resolves access tokens automatically — OAuth tokens are refreshed, API keys are decrypted from the `encryptedApiKey` column + +You never need to modify the sync engine when adding a connector. + +## Icon + +The `icon` field on `ConnectorConfig` is used throughout the UI — in the connector list, the add-connector modal, and as the document icon in the knowledge base table (replacing the generic file type icon for connector-sourced documents). The icon is read from `CONNECTOR_REGISTRY[connectorType].icon` at runtime — no separate icon map to maintain. + +If the service already has an icon in `apps/sim/components/icons.tsx` (from a tool integration), reuse it. Otherwise, ask the user to provide the SVG. + +## Registering + +Add one line to `apps/sim/connectors/registry.ts`: + +```typescript +import { {service}Connector } from '@/connectors/{service}' + +export const CONNECTOR_REGISTRY: ConnectorRegistry = { + // ... existing connectors ... + {service}: {service}Connector, +} +``` + +## Reference Implementations + +- **OAuth**: `apps/sim/connectors/confluence/confluence.ts` — multiple config field types, `mapTags`, label fetching +- **API key**: `apps/sim/connectors/fireflies/fireflies.ts` — GraphQL API with Bearer token auth + +## Checklist + +- [ ] Created `connectors/{service}/{service}.ts` with full ConnectorConfig +- [ ] Created `connectors/{service}/index.ts` barrel export +- [ ] **Auth configured correctly:** + - OAuth: `auth.provider` matches an existing `OAuthService` in `lib/oauth/types.ts` + - API key: `auth.label` and `auth.placeholder` set appropriately +- [ ] **Selector fields configured correctly (if applicable):** + - Every `type: 'selector'` field has a canonical pair (`short-input` or `dropdown` with same `canonicalParamId` and `mode: 'advanced'`) + - `required` is identical on both fields in each canonical pair + - `selectorKey` exists in `hooks/selectors/registry.ts` + - `dependsOn` references selector field IDs (not `canonicalParamId`) + - Dependency `canonicalParamId` values exist in `SELECTOR_CONTEXT_FIELDS` +- [ ] `listDocuments` handles pagination and computes content hashes +- [ ] `sourceUrl` set on each ExternalDocument (full URL, not relative) +- [ ] `metadata` includes source-specific data for tag mapping +- [ ] `tagDefinitions` declared for each semantic key returned by `mapTags` +- [ ] `mapTags` implemented if source has useful metadata (labels, dates, versions) +- [ ] `validateConfig` verifies the source is accessible +- [ ] All external API calls use `fetchWithRetry` (not raw `fetch`) +- [ ] All optional config fields validated in `validateConfig` +- [ ] Icon exists in `components/icons.tsx` (or asked user to provide SVG) +- [ ] Registered in `connectors/registry.ts` diff --git a/.agents/skills/add-connector/agents/openai.yaml b/.agents/skills/add-connector/agents/openai.yaml new file mode 100644 index 00000000000..0ea2afd147e --- /dev/null +++ b/.agents/skills/add-connector/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Add Connector" + short_description: "Build a Sim knowledge connector" + brand_color: "#0F766E" + default_prompt: "Use $add-connector to add or update a Sim knowledge connector for a service." diff --git a/.agents/skills/add-integration/SKILL.md b/.agents/skills/add-integration/SKILL.md new file mode 100644 index 00000000000..ecfda6b0fac --- /dev/null +++ b/.agents/skills/add-integration/SKILL.md @@ -0,0 +1,760 @@ +--- +name: add-integration +description: Add a complete Sim integration from API docs, covering tools, block, icon, optional triggers, registrations, and integration conventions. Use when introducing a new service under `apps/sim/tools`, `apps/sim/blocks`, and `apps/sim/triggers`. +--- + +# Add Integration Skill + +You are an expert at adding complete integrations to Sim. This skill orchestrates the full process of adding a new service integration. + +## Overview + +Adding an integration involves these steps in order: +1. **Research** - Read the service's API documentation +2. **Create Tools** - Build tool configurations for each API operation +3. **Create Block** - Build the block UI configuration +4. **Add Icon** - Add the service's brand icon +5. **Create Triggers** (optional) - If the service supports webhooks +6. **Register** - Register tools, block, and triggers in their registries +7. **Generate Docs** - Run the docs generation script + +## Step 1: Research the API + +Before writing any code: +1. Use Context7 to find official documentation: `mcp__plugin_context7_context7__resolve-library-id` +2. Or use WebFetch to read API docs directly +3. Identify: + - Authentication method (OAuth, API Key, both) + - Available operations (CRUD, search, etc.) + - Required vs optional parameters + - Response structures + +## Step 2: Create Tools + +### Directory Structure +``` +apps/sim/tools/{service}/ +├── index.ts # Barrel exports +├── types.ts # TypeScript interfaces +├── {action1}.ts # Tool for action 1 +├── {action2}.ts # Tool for action 2 +└── ... +``` + +### Key Patterns + +**types.ts:** +```typescript +import type { ToolResponse } from '@/tools/types' + +export interface {Service}{Action}Params { + accessToken: string // For OAuth services + // OR + apiKey: string // For API key services + + requiredParam: string + optionalParam?: string +} + +export interface {Service}Response extends ToolResponse { + output: { + // Define output structure + } +} +``` + +**Tool file pattern:** +```typescript +export const {service}{Action}Tool: ToolConfig = { + id: '{service}_{action}', + name: '{Service} {Action}', + description: '...', + version: '1.0.0', + + oauth: { required: true, provider: '{service}' }, // If OAuth + + params: { + accessToken: { type: 'string', required: true, visibility: 'hidden', description: '...' }, + // ... other params + }, + + request: { url, method, headers, body }, + + transformResponse: async (response) => { + const data = await response.json() + return { + success: true, + output: { + field: data.field ?? null, // Always handle nullables + }, + } + }, + + outputs: { /* ... */ }, +} +``` + +### Critical Rules +- `visibility: 'hidden'` for OAuth tokens +- `visibility: 'user-only'` for API keys and user credentials +- `visibility: 'user-or-llm'` for operation parameters +- Always use `?? null` for nullable API response fields +- Always use `?? []` for optional array fields +- Set `optional: true` for outputs that may not exist +- Never output raw JSON dumps - extract meaningful fields +- When using `type: 'json'` and you know the object shape, define `properties` with the inner fields so downstream consumers know the structure. Only use bare `type: 'json'` when the shape is truly dynamic + +## Step 3: Create Block + +### File Location +`apps/sim/blocks/blocks/{service}.ts` + +### Block Structure +```typescript +import { {Service}Icon } from '@/components/icons' +import type { BlockConfig } from '@/blocks/types' +import { AuthMode } from '@/blocks/types' +import { getScopesForService } from '@/lib/oauth/utils' + +export const {Service}Block: BlockConfig = { + type: '{service}', + name: '{Service}', + description: '...', + longDescription: '...', + docsLink: 'https://docs.sim.ai/tools/{service}', + category: 'tools', + bgColor: '#HEXCOLOR', + icon: {Service}Icon, + authMode: AuthMode.OAuth, // or AuthMode.ApiKey + + subBlocks: [ + // Operation dropdown + { + id: 'operation', + title: 'Operation', + type: 'dropdown', + options: [ + { label: 'Operation 1', id: 'action1' }, + { label: 'Operation 2', id: 'action2' }, + ], + value: () => 'action1', + }, + // Credential field + { + id: 'credential', + title: '{Service} Account', + type: 'oauth-input', + serviceId: '{service}', + requiredScopes: getScopesForService('{service}'), + required: true, + }, + // Conditional fields per operation + // ... + ], + + tools: { + access: ['{service}_action1', '{service}_action2'], + config: { + tool: (params) => `{service}_${params.operation}`, + }, + }, + + outputs: { /* ... */ }, +} +``` + +### Key SubBlock Patterns + +**Condition-based visibility:** +```typescript +{ + id: 'resourceId', + title: 'Resource ID', + type: 'short-input', + condition: { field: 'operation', value: ['read', 'update', 'delete'] }, + required: { field: 'operation', value: ['read', 'update', 'delete'] }, +} +``` + +**DependsOn for cascading selectors:** +```typescript +{ + id: 'project', + type: 'project-selector', + dependsOn: ['credential'], +}, +{ + id: 'issue', + type: 'file-selector', + dependsOn: ['credential', 'project'], +} +``` + +**Basic/Advanced mode for dual UX:** +```typescript +// Basic: Visual selector +{ + id: 'channel', + type: 'channel-selector', + mode: 'basic', + canonicalParamId: 'channel', + dependsOn: ['credential'], +}, +// Advanced: Manual input +{ + id: 'channelId', + type: 'short-input', + mode: 'advanced', + canonicalParamId: 'channel', +} +``` + +**Critical Canonical Param Rules:** +- `canonicalParamId` must NOT match any subblock's `id` in the block +- `canonicalParamId` must be unique per operation/condition context +- Only use `canonicalParamId` to link basic/advanced alternatives for the same logical parameter +- `mode` only controls UI visibility, NOT serialization. Without `canonicalParamId`, both basic and advanced field values would be sent +- Every subblock `id` must be unique within the block. Duplicate IDs cause conflicts even with different conditions +- **Required consistency:** If one subblock in a canonical group has `required: true`, ALL subblocks in that group must have `required: true` (prevents bypassing validation by switching modes) +- **Inputs section:** Must list canonical param IDs (e.g., `fileId`), NOT raw subblock IDs (e.g., `fileSelector`, `manualFileId`) +- **Params function:** Must use canonical param IDs, NOT raw subblock IDs (raw IDs are deleted after canonical transformation) + +## Step 4: Add Icon + +### File Location +`apps/sim/components/icons.tsx` + +### Pattern +```typescript +export function {Service}Icon(props: SVGProps) { + return ( + + {/* SVG paths from user-provided SVG */} + + ) +} +``` + +### Getting Icons +**Do NOT search for icons yourself.** At the end of implementation, ask the user to provide the SVG: + +``` +I've completed the integration. Before I can add the icon, please provide the SVG for {Service}. +You can usually find this in the service's brand/press kit page, or copy it from their website. + +Paste the SVG code here and I'll convert it to a React component. +``` + +Once the user provides the SVG: +1. Extract the SVG paths/content +2. Create a React component that spreads props +3. Ensure viewBox is preserved from the original SVG + +## Step 5: Create Triggers (Optional) + +If the service supports webhooks, create triggers using the generic `buildTriggerSubBlocks` helper. + +### Directory Structure +``` +apps/sim/triggers/{service}/ +├── index.ts # Barrel exports +├── utils.ts # Trigger options, setup instructions, extra fields +├── {event_a}.ts # Primary trigger (includes dropdown) +├── {event_b}.ts # Secondary triggers (no dropdown) +└── webhook.ts # Generic webhook (optional) +``` + +### Key Pattern + +```typescript +import { buildTriggerSubBlocks } from '@/triggers' +import { {service}TriggerOptions, {service}SetupInstructions, build{Service}ExtraFields } from './utils' + +// Primary trigger - includeDropdown: true +export const {service}EventATrigger: TriggerConfig = { + id: '{service}_event_a', + subBlocks: buildTriggerSubBlocks({ + triggerId: '{service}_event_a', + triggerOptions: {service}TriggerOptions, + includeDropdown: true, // Only for primary trigger! + setupInstructions: {service}SetupInstructions('Event A'), + extraFields: build{Service}ExtraFields('{service}_event_a'), + }), + // ... +} + +// Secondary triggers - no dropdown +export const {service}EventBTrigger: TriggerConfig = { + id: '{service}_event_b', + subBlocks: buildTriggerSubBlocks({ + triggerId: '{service}_event_b', + triggerOptions: {service}TriggerOptions, + // No includeDropdown! + setupInstructions: {service}SetupInstructions('Event B'), + extraFields: build{Service}ExtraFields('{service}_event_b'), + }), + // ... +} +``` + +### Connect to Block +```typescript +import { getTrigger } from '@/triggers' + +export const {Service}Block: BlockConfig = { + triggers: { + enabled: true, + available: ['{service}_event_a', '{service}_event_b'], + }, + subBlocks: [ + // Tool fields... + ...getTrigger('{service}_event_a').subBlocks, + ...getTrigger('{service}_event_b').subBlocks, + ], +} +``` + +See `/add-trigger` skill for complete documentation. + +## Step 6: Register Everything + +### Tools Registry (`apps/sim/tools/registry.ts`) + +```typescript +// Add import (alphabetically) +import { + {service}Action1Tool, + {service}Action2Tool, +} from '@/tools/{service}' + +// Add to tools object (alphabetically) +export const tools: Record = { + // ... existing tools ... + {service}_action1: {service}Action1Tool, + {service}_action2: {service}Action2Tool, +} +``` + +### Block Registry (`apps/sim/blocks/registry.ts`) + +```typescript +// Add import (alphabetically) +import { {Service}Block } from '@/blocks/blocks/{service}' + +// Add to registry (alphabetically) +export const registry: Record = { + // ... existing blocks ... + {service}: {Service}Block, +} +``` + +### Trigger Registry (`apps/sim/triggers/registry.ts`) - If triggers exist + +```typescript +// Add import (alphabetically) +import { + {service}EventATrigger, + {service}EventBTrigger, + {service}WebhookTrigger, +} from '@/triggers/{service}' + +// Add to TRIGGER_REGISTRY (alphabetically) +export const TRIGGER_REGISTRY: TriggerRegistry = { + // ... existing triggers ... + {service}_event_a: {service}EventATrigger, + {service}_event_b: {service}EventBTrigger, + {service}_webhook: {service}WebhookTrigger, +} +``` + +## Step 7: Generate Docs + +Run the documentation generator: +```bash +bun run scripts/generate-docs.ts +``` + +This creates `apps/docs/content/docs/en/tools/{service}.mdx` + +## V2 Integration Pattern + +If creating V2 versions (API-aligned outputs): + +1. **V2 Tools** - Add `_v2` suffix, version `2.0.0`, flat outputs +2. **V2 Block** - Add `_v2` type, use `createVersionedToolSelector` +3. **V1 Block** - Add `(Legacy)` to name, set `hideFromToolbar: true` +4. **Registry** - Register both versions + +```typescript +// In registry +{service}: {Service}Block, // V1 (legacy, hidden) +{service}_v2: {Service}V2Block, // V2 (visible) +``` + +## Complete Checklist + +### Tools +- [ ] Created `tools/{service}/` directory +- [ ] Created `types.ts` with all interfaces +- [ ] Created tool file for each operation +- [ ] All params have correct visibility +- [ ] All nullable fields use `?? null` +- [ ] All optional outputs have `optional: true` +- [ ] Created `index.ts` barrel export +- [ ] Registered all tools in `tools/registry.ts` + +### Block +- [ ] Created `blocks/blocks/{service}.ts` +- [ ] Defined operation dropdown with all operations +- [ ] Added credential field with `requiredScopes: getScopesForService('{service}')` +- [ ] Added conditional fields per operation +- [ ] Set up dependsOn for cascading selectors +- [ ] Configured tools.access with all tool IDs +- [ ] Configured tools.config.tool selector +- [ ] Defined outputs matching tool outputs +- [ ] Registered block in `blocks/registry.ts` +- [ ] If triggers: set `triggers.enabled` and `triggers.available` +- [ ] If triggers: spread trigger subBlocks with `getTrigger()` + +### OAuth Scopes (if OAuth service) +- [ ] Defined scopes in `lib/oauth/oauth.ts` under `OAUTH_PROVIDERS` +- [ ] Added scope descriptions in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` +- [ ] Used `getCanonicalScopesForProvider()` in `auth.ts` (never hardcode) +- [ ] Used `getScopesForService()` in block `requiredScopes` (never hardcode) + +### Icon +- [ ] Asked user to provide SVG +- [ ] Added icon to `components/icons.tsx` +- [ ] Icon spreads props correctly + +### Triggers (if service supports webhooks) +- [ ] Created `triggers/{service}/` directory +- [ ] Created `utils.ts` with options, instructions, and extra fields helpers +- [ ] Primary trigger uses `includeDropdown: true` +- [ ] Secondary triggers do NOT have `includeDropdown` +- [ ] All triggers use `buildTriggerSubBlocks` helper +- [ ] Created `index.ts` barrel export +- [ ] Registered all triggers in `triggers/registry.ts` + +### Docs +- [ ] Ran `bun run scripts/generate-docs.ts` +- [ ] Verified docs file created + +### Final Validation (Required) +- [ ] Read every tool file and cross-referenced inputs/outputs against the API docs +- [ ] Verified block subBlocks cover all required tool params with correct conditions +- [ ] Verified block outputs match what the tools actually return +- [ ] Verified `tools.config.params` correctly maps and coerces all param types + +## Example Command + +When the user asks to add an integration: + +``` +User: Add a Stripe integration + +You: I'll add the Stripe integration. Let me: + +1. First, research the Stripe API using Context7 +2. Create the tools for key operations (payments, subscriptions, etc.) +3. Create the block with operation dropdown +4. Register everything +5. Generate docs +6. Ask you for the Stripe icon SVG + +[Proceed with implementation...] + +[After completing steps 1-5...] + +I've completed the Stripe integration. Before I can add the icon, please provide the SVG for Stripe. +You can usually find this in the service's brand/press kit page, or copy it from their website. + +Paste the SVG code here and I'll convert it to a React component. +``` + +## File Handling + +When your integration handles file uploads or downloads, follow these patterns to work with `UserFile` objects consistently. + +### What is a UserFile? + +A `UserFile` is the standard file representation in Sim: + +```typescript +interface UserFile { + id: string // Unique identifier + name: string // Original filename + url: string // Presigned URL for download + size: number // File size in bytes + type: string // MIME type (e.g., 'application/pdf') + base64?: string // Optional base64 content (if small file) + key?: string // Internal storage key + context?: object // Storage context metadata +} +``` + +### File Input Pattern (Uploads) + +For tools that accept file uploads, **always route through an internal API endpoint** rather than calling external APIs directly. This ensures proper file content retrieval. + +#### 1. Block SubBlocks for File Input + +Use the basic/advanced mode pattern: + +```typescript +// Basic mode: File upload UI +{ + id: 'uploadFile', + title: 'File', + type: 'file-upload', + canonicalParamId: 'file', // Maps to 'file' param + placeholder: 'Upload file', + mode: 'basic', + multiple: false, + required: true, + condition: { field: 'operation', value: 'upload' }, +}, +// Advanced mode: Reference from previous block +{ + id: 'fileRef', + title: 'File', + type: 'short-input', + canonicalParamId: 'file', // Same canonical param + placeholder: 'Reference file (e.g., {{file_block.output}})', + mode: 'advanced', + required: true, + condition: { field: 'operation', value: 'upload' }, +}, +``` + +**Critical:** `canonicalParamId` must NOT match any subblock `id`. + +#### 2. Normalize File Input in Block Config + +In `tools.config.tool`, use `normalizeFileInput` to handle all input variants: + +```typescript +import { normalizeFileInput } from '@/blocks/utils' + +tools: { + config: { + tool: (params) => { + // Normalize file from basic (uploadFile), advanced (fileRef), or legacy (fileContent) + const normalizedFile = normalizeFileInput( + params.uploadFile || params.fileRef || params.fileContent, + { single: true } + ) + if (normalizedFile) { + params.file = normalizedFile + } + return `{service}_${params.operation}` + }, + }, +} +``` + +#### 3. Create Internal API Route + +Create `apps/sim/app/api/tools/{service}/{action}/route.ts`: + +```typescript +import { createLogger } from '@sim/logger' +import { NextResponse, type NextRequest } from 'next/server' +import { z } from 'zod' +import { checkInternalAuth } from '@/lib/auth/hybrid' +import { generateRequestId } from '@/lib/core/utils/request' +import { FileInputSchema, type RawFileInput } from '@/lib/uploads/utils/file-schemas' +import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' +import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server' + +const logger = createLogger('{Service}UploadAPI') + +const RequestSchema = z.object({ + accessToken: z.string(), + file: FileInputSchema.optional().nullable(), + // Legacy field for backwards compatibility + fileContent: z.string().optional().nullable(), + // ... other params +}) + +export async function POST(request: NextRequest) { + const requestId = generateRequestId() + + const authResult = await checkInternalAuth(request, { requireWorkflowId: false }) + if (!authResult.success) { + return NextResponse.json({ success: false, error: 'Unauthorized' }, { status: 401 }) + } + + const body = await request.json() + const data = RequestSchema.parse(body) + + let fileBuffer: Buffer + let fileName: string + + // Prefer UserFile input, fall back to legacy base64 + if (data.file) { + const userFiles = processFilesToUserFiles([data.file as RawFileInput], requestId, logger) + if (userFiles.length === 0) { + return NextResponse.json({ success: false, error: 'Invalid file' }, { status: 400 }) + } + const userFile = userFiles[0] + fileBuffer = await downloadFileFromStorage(userFile, requestId, logger) + fileName = userFile.name + } else if (data.fileContent) { + // Legacy: base64 string (backwards compatibility) + fileBuffer = Buffer.from(data.fileContent, 'base64') + fileName = 'file' + } else { + return NextResponse.json({ success: false, error: 'File required' }, { status: 400 }) + } + + // Now call external API with fileBuffer + const response = await fetch('https://api.{service}.com/upload', { + method: 'POST', + headers: { Authorization: `Bearer ${data.accessToken}` }, + body: new Uint8Array(fileBuffer), // Convert Buffer for fetch + }) + + // ... handle response +} +``` + +#### 4. Update Tool to Use Internal Route + +```typescript +export const {service}UploadTool: ToolConfig = { + id: '{service}_upload', + // ... + params: { + file: { type: 'file', required: false, visibility: 'user-or-llm' }, + fileContent: { type: 'string', required: false, visibility: 'hidden' }, // Legacy + }, + request: { + url: '/api/tools/{service}/upload', // Internal route + method: 'POST', + body: (params) => ({ + accessToken: params.accessToken, + file: params.file, + fileContent: params.fileContent, + }), + }, +} +``` + +### File Output Pattern (Downloads) + +For tools that return files, use `FileToolProcessor` to store files and return `UserFile` objects. + +#### In Tool transformResponse + +```typescript +import { FileToolProcessor } from '@/executor/utils/file-tool-processor' + +transformResponse: async (response, context) => { + const data = await response.json() + + // Process file outputs to UserFile objects + const fileProcessor = new FileToolProcessor(context) + const file = await fileProcessor.processFileData({ + data: data.content, // base64 or buffer + mimeType: data.mimeType, + filename: data.filename, + }) + + return { + success: true, + output: { file }, + } +} +``` + +#### In API Route (for complex file handling) + +```typescript +// Return file data that FileToolProcessor can handle +return NextResponse.json({ + success: true, + output: { + file: { + data: base64Content, + mimeType: 'application/pdf', + filename: 'document.pdf', + }, + }, +}) +``` + +### Key Helpers Reference + +| Helper | Location | Purpose | +|--------|----------|---------| +| `normalizeFileInput` | `@/blocks/utils` | Normalize file params in block config | +| `processFilesToUserFiles` | `@/lib/uploads/utils/file-utils` | Convert raw inputs to UserFile[] | +| `downloadFileFromStorage` | `@/lib/uploads/utils/file-utils.server` | Get file Buffer from UserFile | +| `FileToolProcessor` | `@/executor/utils/file-tool-processor` | Process tool output files | +| `isUserFile` | `@/lib/core/utils/user-file` | Type guard for UserFile objects | +| `FileInputSchema` | `@/lib/uploads/utils/file-schemas` | Zod schema for file validation | + +### Advanced Mode for Optional Fields + +Optional fields that are rarely used should be set to `mode: 'advanced'` so they don't clutter the basic UI. Examples: pagination tokens, time range filters, sort order, max results, reply settings. + +### WandConfig for Complex Inputs + +Use `wandConfig` for fields that are hard to fill out manually: +- **Timestamps**: Use `generationType: 'timestamp'` to inject current date context into the AI prompt +- **JSON arrays**: Use `generationType: 'json-object'` for structured data +- **Complex queries**: Use a descriptive prompt explaining the expected format + +```typescript +{ + id: 'startTime', + title: 'Start Time', + type: 'short-input', + mode: 'advanced', + wandConfig: { + enabled: true, + prompt: 'Generate an ISO 8601 timestamp. Return ONLY the timestamp string.', + generationType: 'timestamp', + }, +} +``` + +### OAuth Scopes (Centralized System) + +Scopes are maintained in a single source of truth and reused everywhere: + +1. **Define scopes** in `lib/oauth/oauth.ts` under `OAUTH_PROVIDERS[provider].services[service].scopes` +2. **Add descriptions** in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` for the OAuth modal UI +3. **Reference in auth.ts** using `getCanonicalScopesForProvider(providerId)` from `@/lib/oauth/utils` +4. **Reference in blocks** using `getScopesForService(serviceId)` from `@/lib/oauth/utils` + +**Never hardcode scope arrays** in `auth.ts` or block `requiredScopes`. Always import from the centralized source. + +```typescript +// In auth.ts (Better Auth config) +scopes: getCanonicalScopesForProvider('{service}'), + +// In block credential sub-block +requiredScopes: getScopesForService('{service}'), +``` + +### Common Gotchas + +1. **OAuth serviceId must match** - The `serviceId` in oauth-input must match the OAuth provider configuration +2. **All tool IDs MUST be snake_case** - `stripe_create_payment`, not `stripeCreatePayment`. This applies to tool `id` fields, registry keys, `tools.access` arrays, and `tools.config.tool` return values +3. **Block type is snake_case** - `type: 'stripe'`, not `type: 'Stripe'` +4. **Alphabetical ordering** - Keep imports and registry entries alphabetically sorted +5. **Required can be conditional** - Use `required: { field: 'op', value: 'create' }` instead of always true +6. **DependsOn clears options** - When a dependency changes, selector options are refetched +7. **Never pass Buffer directly to fetch** - Convert to `new Uint8Array(buffer)` for TypeScript compatibility +8. **Always handle legacy file params** - Keep hidden `fileContent` params for backwards compatibility +9. **Optional fields use advanced mode** - Set `mode: 'advanced'` on rarely-used optional fields +10. **Complex inputs need wandConfig** - Timestamps, JSON arrays, and other hard-to-type values should have `wandConfig` enabled +11. **Never hardcode scopes** - Use `getScopesForService()` in blocks and `getCanonicalScopesForProvider()` in auth.ts +12. **Always add scope descriptions** - New scopes must have entries in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` diff --git a/.agents/skills/add-integration/agents/openai.yaml b/.agents/skills/add-integration/agents/openai.yaml new file mode 100644 index 00000000000..16c7ecc5ed2 --- /dev/null +++ b/.agents/skills/add-integration/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Add Integration" + short_description: "Build a full Sim integration" + brand_color: "#7C3AED" + default_prompt: "Use $add-integration to add a complete Sim integration for a service." diff --git a/.agents/skills/add-tools/SKILL.md b/.agents/skills/add-tools/SKILL.md new file mode 100644 index 00000000000..66f6f88d047 --- /dev/null +++ b/.agents/skills/add-tools/SKILL.md @@ -0,0 +1,321 @@ +--- +name: add-tools +description: Create or update Sim tool configurations from service API docs, including typed params, request mapping, response transforms, outputs, and registry entries. Use when working in `apps/sim/tools/{service}/` or fixing tool definitions for an integration. +--- + +# Add Tools Skill + +You are an expert at creating tool configurations for Sim integrations. Your job is to read API documentation and create properly structured tool files. + +## Your Task + +When the user asks you to create tools for a service: +1. Use Context7 or WebFetch to read the service's API documentation +2. Create the tools directory structure +3. Generate properly typed tool configurations + +## Directory Structure + +Create files in `apps/sim/tools/{service}/`: +``` +tools/{service}/ +├── index.ts # Barrel export +├── types.ts # Parameter & response types +└── {action}.ts # Individual tool files (one per operation) +``` + +## Tool Configuration Structure + +Every tool MUST follow this exact structure: + +```typescript +import type { {ServiceName}{Action}Params } from '@/tools/{service}/types' +import type { ToolConfig } from '@/tools/types' + +interface {ServiceName}{Action}Response { + success: boolean + output: { + // Define output structure here + } +} + +export const {serviceName}{Action}Tool: ToolConfig< + {ServiceName}{Action}Params, + {ServiceName}{Action}Response +> = { + id: '{service}_{action}', // snake_case, matches tool name + name: '{Service} {Action}', // Human readable + description: 'Brief description', // One sentence + version: '1.0.0', + + // OAuth config (if service uses OAuth) + oauth: { + required: true, + provider: '{service}', // Must match OAuth provider ID + }, + + params: { + // Hidden params (system-injected, only use hidden for oauth accessToken) + accessToken: { + type: 'string', + required: true, + visibility: 'hidden', + description: 'OAuth access token', + }, + // User-only params (credentials, api key, IDs user must provide) + someId: { + type: 'string', + required: true, + visibility: 'user-only', + description: 'The ID of the resource', + }, + // User-or-LLM params (everything else, can be provided by user OR computed by LLM) + query: { + type: 'string', + required: false, // Use false for optional + visibility: 'user-or-llm', + description: 'Search query', + }, + }, + + request: { + url: (params) => `https://api.service.com/v1/resource/${params.id}`, + method: 'POST', + headers: (params) => ({ + Authorization: `Bearer ${params.accessToken}`, + 'Content-Type': 'application/json', + }), + body: (params) => ({ + // Request body - only for POST/PUT/PATCH + // Trim ID fields to prevent copy-paste whitespace errors: + // userId: params.userId?.trim(), + }), + }, + + transformResponse: async (response: Response) => { + const data = await response.json() + return { + success: true, + output: { + // Map API response to output + // Use ?? null for nullable fields + // Use ?? [] for optional arrays + }, + } + }, + + outputs: { + // Define each output field + }, +} +``` + +## Critical Rules for Parameters + +### Visibility Options +- `'hidden'` - System-injected (OAuth tokens, internal params). User never sees. +- `'user-only'` - User must provide (credentials, api keys, account-specific IDs) +- `'user-or-llm'` - User provides OR LLM can compute (search queries, content, filters, most fall into this category) + +### Parameter Types +- `'string'` - Text values +- `'number'` - Numeric values +- `'boolean'` - True/false +- `'json'` - Complex objects (NOT 'object', use 'json') +- `'file'` - Single file +- `'file[]'` - Multiple files + +### Required vs Optional +- Always explicitly set `required: true` or `required: false` +- Optional params should have `required: false` + +## Critical Rules for Outputs + +### Output Types +- `'string'`, `'number'`, `'boolean'` - Primitives +- `'json'` - Complex objects (use this, NOT 'object') +- `'array'` - Arrays with `items` property +- `'object'` - Objects with `properties` property + +### Optional Outputs +Add `optional: true` for fields that may not exist in the response: +```typescript +closedAt: { + type: 'string', + description: 'When the issue was closed', + optional: true, +}, +``` + +### Typed JSON Outputs + +When using `type: 'json'` and you know the object shape in advance, **always define the inner structure** using `properties` so downstream consumers know what fields are available: + +```typescript +// BAD: Opaque json with no info about what's inside +metadata: { + type: 'json', + description: 'Response metadata', +}, + +// GOOD: Define the known properties +metadata: { + type: 'json', + description: 'Response metadata', + properties: { + id: { type: 'string', description: 'Unique ID' }, + status: { type: 'string', description: 'Current status' }, + count: { type: 'number', description: 'Total count' }, + }, +}, +``` + +For arrays of objects, define the item structure: +```typescript +items: { + type: 'array', + description: 'List of items', + items: { + type: 'object', + properties: { + id: { type: 'string', description: 'Item ID' }, + name: { type: 'string', description: 'Item name' }, + }, + }, +}, +``` + +Only use bare `type: 'json'` without `properties` when the shape is truly dynamic or unknown. + +## Critical Rules for transformResponse + +### Handle Nullable Fields +ALWAYS use `?? null` for fields that may be undefined: +```typescript +transformResponse: async (response: Response) => { + const data = await response.json() + return { + success: true, + output: { + id: data.id, + title: data.title, + body: data.body ?? null, // May be undefined + assignee: data.assignee ?? null, // May be undefined + labels: data.labels ?? [], // Default to empty array + closedAt: data.closed_at ?? null, // May be undefined + }, + } +} +``` + +### Never Output Raw JSON Dumps +DON'T do this: +```typescript +output: { + data: data, // BAD - raw JSON dump +} +``` + +DO this instead - extract meaningful fields: +```typescript +output: { + id: data.id, + name: data.name, + status: data.status, + metadata: { + createdAt: data.created_at, + updatedAt: data.updated_at, + }, +} +``` + +## Types File Pattern + +Create `types.ts` with interfaces for all params and responses: + +```typescript +import type { ToolResponse } from '@/tools/types' + +// Parameter interfaces +export interface {Service}{Action}Params { + accessToken: string + requiredField: string + optionalField?: string +} + +// Response interfaces (extend ToolResponse) +export interface {Service}{Action}Response extends ToolResponse { + output: { + field1: string + field2: number + optionalField?: string | null + } +} +``` + +## Index.ts Barrel Export Pattern + +```typescript +// Export all tools +export { serviceTool1 } from './{action1}' +export { serviceTool2 } from './{action2}' + +// Export types +export * from './types' +``` + +## Registering Tools + +After creating tools, remind the user to: +1. Import tools in `apps/sim/tools/registry.ts` +2. Add to the `tools` object with snake_case keys: +```typescript +import { serviceActionTool } from '@/tools/{service}' + +export const tools = { + // ... existing tools ... + {service}_{action}: serviceActionTool, +} +``` + +## V2 Tool Pattern + +If creating V2 tools (API-aligned outputs), use `_v2` suffix: +- Tool ID: `{service}_{action}_v2` +- Variable name: `{action}V2Tool` +- Version: `'2.0.0'` +- Outputs: Flat, API-aligned (no content/metadata wrapper) + +## Naming Convention + +All tool IDs MUST use `snake_case`: `{service}_{action}` (e.g., `x_create_tweet`, `slack_send_message`). Never use camelCase or PascalCase for tool IDs. + +## Checklist Before Finishing + +- [ ] All tool IDs use snake_case +- [ ] All params have explicit `required: true` or `required: false` +- [ ] All params have appropriate `visibility` +- [ ] All nullable response fields use `?? null` +- [ ] All optional outputs have `optional: true` +- [ ] No raw JSON dumps in outputs +- [ ] Types file has all interfaces +- [ ] Index.ts exports all tools + +## Final Validation (Required) + +After creating all tools, you MUST validate every tool before finishing: + +1. **Read every tool file** you created — do not skip any +2. **Cross-reference with the API docs** to verify: + - All required params are marked `required: true` + - All optional params are marked `required: false` + - Param types match the API (string, number, boolean, json) + - Request URL, method, headers, and body match the API spec + - `transformResponse` extracts the correct fields from the API response + - All output fields match what the API actually returns + - No fields are missing from outputs that the API provides + - No extra fields are defined in outputs that the API doesn't return +3. **Verify consistency** across tools: + - Shared types in `types.ts` match all tools that use them + - Tool IDs in the barrel export match the tool file definitions + - Error handling is consistent (error checks, meaningful messages) diff --git a/.agents/skills/add-tools/agents/openai.yaml b/.agents/skills/add-tools/agents/openai.yaml new file mode 100644 index 00000000000..1acc4fd7126 --- /dev/null +++ b/.agents/skills/add-tools/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Add Tools" + short_description: "Build Sim tools from API docs" + brand_color: "#EA580C" + default_prompt: "Use $add-tools to create or update Sim tool definitions from service API docs." diff --git a/.agents/skills/add-trigger/SKILL.md b/.agents/skills/add-trigger/SKILL.md new file mode 100644 index 00000000000..fbf27ef625d --- /dev/null +++ b/.agents/skills/add-trigger/SKILL.md @@ -0,0 +1,708 @@ +--- +name: add-trigger +description: Create or update Sim webhook triggers using the generic trigger builder, service-specific setup instructions, outputs, and registry wiring. Use when working in `apps/sim/triggers/{service}/` or adding webhook support to an integration. +--- + +# Add Trigger Skill + +You are an expert at creating webhook triggers for Sim. You understand the trigger system, the generic `buildTriggerSubBlocks` helper, and how triggers connect to blocks. + +## Your Task + +When the user asks you to create triggers for a service: +1. Research what webhook events the service supports +2. Create the trigger files using the generic builder +3. Register triggers and connect them to the block + +## Directory Structure + +``` +apps/sim/triggers/{service}/ +├── index.ts # Barrel exports +├── utils.ts # Service-specific helpers (trigger options, setup instructions, extra fields) +├── {event_a}.ts # Primary trigger (includes dropdown) +├── {event_b}.ts # Secondary trigger (no dropdown) +├── {event_c}.ts # Secondary trigger (no dropdown) +└── webhook.ts # Generic webhook trigger (optional, for "all events") +``` + +## Step 1: Create utils.ts + +This file contains service-specific helpers used by all triggers. + +```typescript +import type { SubBlockConfig } from '@/blocks/types' +import type { TriggerOutput } from '@/triggers/types' + +/** + * Dropdown options for the trigger type selector. + * These appear in the primary trigger's dropdown. + */ +export const {service}TriggerOptions = [ + { label: 'Event A', id: '{service}_event_a' }, + { label: 'Event B', id: '{service}_event_b' }, + { label: 'Event C', id: '{service}_event_c' }, + { label: 'Generic Webhook (All Events)', id: '{service}_webhook' }, +] + +/** + * Generates HTML setup instructions for the trigger. + * Displayed to users to help them configure webhooks in the external service. + */ +export function {service}SetupInstructions(eventType: string): string { + const instructions = [ + 'Copy the Webhook URL above', + 'Go to {Service} Settings > Webhooks', + 'Click Add Webhook', + 'Paste the webhook URL', + `Select the ${eventType} event type`, + 'Save the webhook configuration', + 'Click "Save" above to activate your trigger', + ] + + return instructions + .map((instruction, index) => + `
${index + 1}. ${instruction}
` + ) + .join('') +} + +/** + * Service-specific extra fields to add to triggers. + * These are inserted between webhookUrl and triggerSave. + */ +export function build{Service}ExtraFields(triggerId: string): SubBlockConfig[] { + return [ + { + id: 'projectId', + title: 'Project ID (Optional)', + type: 'short-input', + placeholder: 'Leave empty for all projects', + description: 'Optionally filter to a specific project', + mode: 'trigger', + condition: { field: 'selectedTriggerId', value: triggerId }, + }, + ] +} + +/** + * Build outputs for this trigger type. + * Outputs define what data is available to downstream blocks. + */ +export function build{Service}Outputs(): Record { + return { + eventType: { type: 'string', description: 'The type of event that triggered this workflow' }, + resourceId: { type: 'string', description: 'ID of the affected resource' }, + timestamp: { type: 'string', description: 'When the event occurred (ISO 8601)' }, + // Nested outputs for complex data + resource: { + id: { type: 'string', description: 'Resource ID' }, + name: { type: 'string', description: 'Resource name' }, + status: { type: 'string', description: 'Current status' }, + }, + webhook: { type: 'json', description: 'Full webhook payload' }, + } +} +``` + +## Step 2: Create the Primary Trigger + +The **primary trigger** is the first one listed. It MUST include `includeDropdown: true` so users can switch between trigger types. + +```typescript +import { {Service}Icon } from '@/components/icons' +import { buildTriggerSubBlocks } from '@/triggers' +import { + build{Service}ExtraFields, + build{Service}Outputs, + {service}SetupInstructions, + {service}TriggerOptions, +} from '@/triggers/{service}/utils' +import type { TriggerConfig } from '@/triggers/types' + +/** + * {Service} Event A Trigger + * + * This is the PRIMARY trigger - it includes the dropdown for selecting trigger type. + */ +export const {service}EventATrigger: TriggerConfig = { + id: '{service}_event_a', + name: '{Service} Event A', + provider: '{service}', + description: 'Trigger workflow when Event A occurs', + version: '1.0.0', + icon: {Service}Icon, + + subBlocks: buildTriggerSubBlocks({ + triggerId: '{service}_event_a', + triggerOptions: {service}TriggerOptions, + includeDropdown: true, // PRIMARY TRIGGER - includes dropdown + setupInstructions: {service}SetupInstructions('Event A'), + extraFields: build{Service}ExtraFields('{service}_event_a'), + }), + + outputs: build{Service}Outputs(), + + webhook: { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + }, + }, +} +``` + +## Step 3: Create Secondary Triggers + +Secondary triggers do NOT include the dropdown (it's already in the primary trigger). + +```typescript +import { {Service}Icon } from '@/components/icons' +import { buildTriggerSubBlocks } from '@/triggers' +import { + build{Service}ExtraFields, + build{Service}Outputs, + {service}SetupInstructions, + {service}TriggerOptions, +} from '@/triggers/{service}/utils' +import type { TriggerConfig } from '@/triggers/types' + +/** + * {Service} Event B Trigger + */ +export const {service}EventBTrigger: TriggerConfig = { + id: '{service}_event_b', + name: '{Service} Event B', + provider: '{service}', + description: 'Trigger workflow when Event B occurs', + version: '1.0.0', + icon: {Service}Icon, + + subBlocks: buildTriggerSubBlocks({ + triggerId: '{service}_event_b', + triggerOptions: {service}TriggerOptions, + // NO includeDropdown - secondary trigger + setupInstructions: {service}SetupInstructions('Event B'), + extraFields: build{Service}ExtraFields('{service}_event_b'), + }), + + outputs: build{Service}Outputs(), + + webhook: { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + }, + }, +} +``` + +## Step 4: Create index.ts Barrel Export + +```typescript +export { {service}EventATrigger } from './event_a' +export { {service}EventBTrigger } from './event_b' +export { {service}EventCTrigger } from './event_c' +export { {service}WebhookTrigger } from './webhook' +``` + +## Step 5: Register Triggers + +### Trigger Registry (`apps/sim/triggers/registry.ts`) + +```typescript +// Add import +import { + {service}EventATrigger, + {service}EventBTrigger, + {service}EventCTrigger, + {service}WebhookTrigger, +} from '@/triggers/{service}' + +// Add to TRIGGER_REGISTRY +export const TRIGGER_REGISTRY: TriggerRegistry = { + // ... existing triggers ... + {service}_event_a: {service}EventATrigger, + {service}_event_b: {service}EventBTrigger, + {service}_event_c: {service}EventCTrigger, + {service}_webhook: {service}WebhookTrigger, +} +``` + +## Step 6: Connect Triggers to Block + +In the block file (`apps/sim/blocks/blocks/{service}.ts`): + +```typescript +import { {Service}Icon } from '@/components/icons' +import { getTrigger } from '@/triggers' +import type { BlockConfig } from '@/blocks/types' + +export const {Service}Block: BlockConfig = { + type: '{service}', + name: '{Service}', + // ... other config ... + + // Enable triggers and list available trigger IDs + triggers: { + enabled: true, + available: [ + '{service}_event_a', + '{service}_event_b', + '{service}_event_c', + '{service}_webhook', + ], + }, + + subBlocks: [ + // Regular tool subBlocks first + { id: 'operation', /* ... */ }, + { id: 'credential', /* ... */ }, + // ... other tool fields ... + + // Then spread ALL trigger subBlocks + ...getTrigger('{service}_event_a').subBlocks, + ...getTrigger('{service}_event_b').subBlocks, + ...getTrigger('{service}_event_c').subBlocks, + ...getTrigger('{service}_webhook').subBlocks, + ], + + // ... tools config ... +} +``` + +## Automatic Webhook Registration (Preferred) + +If the service's API supports programmatic webhook creation, implement automatic webhook registration instead of requiring users to manually configure webhooks. This provides a much better user experience. + +### When to Use Automatic Registration + +Check the service's API documentation for endpoints like: +- `POST /webhooks` or `POST /hooks` - Create webhook +- `DELETE /webhooks/{id}` - Delete webhook + +Services that support this pattern include: Grain, Lemlist, Calendly, Airtable, Webflow, Typeform, etc. + +### Implementation Steps + +#### 1. Add API Key to Extra Fields + +Update your `build{Service}ExtraFields` function to include an API key field: + +```typescript +export function build{Service}ExtraFields(triggerId: string): SubBlockConfig[] { + return [ + { + id: 'apiKey', + title: 'API Key', + type: 'short-input', + placeholder: 'Enter your {Service} API key', + description: 'Required to create the webhook in {Service}.', + password: true, + required: true, + mode: 'trigger', + condition: { field: 'selectedTriggerId', value: triggerId }, + }, + // Other optional fields (e.g., campaign filter, project filter) + { + id: 'projectId', + title: 'Project ID (Optional)', + type: 'short-input', + placeholder: 'Leave empty for all projects', + mode: 'trigger', + condition: { field: 'selectedTriggerId', value: triggerId }, + }, + ] +} +``` + +#### 2. Update Setup Instructions for Automatic Creation + +Change instructions to indicate automatic webhook creation: + +```typescript +export function {service}SetupInstructions(eventType: string): string { + const instructions = [ + 'Enter your {Service} API Key above.', + 'You can find your API key in {Service} at Settings > API.', + `Click "Save Configuration" to automatically create the webhook in {Service} for ${eventType} events.`, + 'The webhook will be automatically deleted when you remove this trigger.', + ] + + return instructions + .map((instruction, index) => + `
${index + 1}. ${instruction}
` + ) + .join('') +} +``` + +#### 3. Add Webhook Creation to API Route + +In `apps/sim/app/api/webhooks/route.ts`, add provider-specific logic after the database save: + +```typescript +// --- {Service} specific logic --- +if (savedWebhook && provider === '{service}') { + logger.info(`[${requestId}] {Service} provider detected. Creating webhook subscription.`) + try { + const result = await create{Service}WebhookSubscription( + { + id: savedWebhook.id, + path: savedWebhook.path, + providerConfig: savedWebhook.providerConfig, + }, + requestId + ) + + if (result) { + // Update the webhook record with the external webhook ID + const updatedConfig = { + ...(savedWebhook.providerConfig as Record), + externalId: result.id, + } + await db + .update(webhook) + .set({ + providerConfig: updatedConfig, + updatedAt: new Date(), + }) + .where(eq(webhook.id, savedWebhook.id)) + + savedWebhook.providerConfig = updatedConfig + logger.info(`[${requestId}] Successfully created {Service} webhook`, { + externalHookId: result.id, + webhookId: savedWebhook.id, + }) + } + } catch (err) { + logger.error( + `[${requestId}] Error creating {Service} webhook subscription, rolling back webhook`, + err + ) + await db.delete(webhook).where(eq(webhook.id, savedWebhook.id)) + return NextResponse.json( + { + error: 'Failed to create webhook in {Service}', + details: err instanceof Error ? err.message : 'Unknown error', + }, + { status: 500 } + ) + } +} +// --- End {Service} specific logic --- +``` + +Then add the helper function at the end of the file: + +```typescript +async function create{Service}WebhookSubscription( + webhookData: any, + requestId: string +): Promise<{ id: string } | undefined> { + try { + const { path, providerConfig } = webhookData + const { apiKey, triggerId, projectId } = providerConfig || {} + + if (!apiKey) { + throw new Error('{Service} API Key is required.') + } + + // Map trigger IDs to service event types + const eventTypeMap: Record = { + {service}_event_a: 'eventA', + {service}_event_b: 'eventB', + {service}_webhook: undefined, // Generic - no filter + } + + const eventType = eventTypeMap[triggerId] + const notificationUrl = `${getBaseUrl()}/api/webhooks/trigger/${path}` + + const requestBody: Record = { + url: notificationUrl, + } + + if (eventType) { + requestBody.eventType = eventType + } + + if (projectId) { + requestBody.projectId = projectId + } + + const response = await fetch('https://api.{service}.com/webhooks', { + method: 'POST', + headers: { + Authorization: `Bearer ${apiKey}`, + 'Content-Type': 'application/json', + }, + body: JSON.stringify(requestBody), + }) + + const responseBody = await response.json() + + if (!response.ok) { + const errorMessage = responseBody.message || 'Unknown API error' + let userFriendlyMessage = 'Failed to create webhook in {Service}' + + if (response.status === 401) { + userFriendlyMessage = 'Invalid API Key. Please verify and try again.' + } else if (errorMessage) { + userFriendlyMessage = `{Service} error: ${errorMessage}` + } + + throw new Error(userFriendlyMessage) + } + + return { id: responseBody.id } + } catch (error: any) { + logger.error(`Exception during {Service} webhook creation`, { error: error.message }) + throw error + } +} +``` + +#### 4. Add Webhook Deletion to Provider Subscriptions + +In `apps/sim/lib/webhooks/provider-subscriptions.ts`: + +1. Add a logger: +```typescript +const {service}Logger = createLogger('{Service}Webhook') +``` + +2. Add the delete function: +```typescript +export async function delete{Service}Webhook(webhook: any, requestId: string): Promise { + try { + const config = getProviderConfig(webhook) + const apiKey = config.apiKey as string | undefined + const externalId = config.externalId as string | undefined + + if (!apiKey || !externalId) { + {service}Logger.warn(`[${requestId}] Missing apiKey or externalId, skipping cleanup`) + return + } + + const response = await fetch(`https://api.{service}.com/webhooks/${externalId}`, { + method: 'DELETE', + headers: { + Authorization: `Bearer ${apiKey}`, + }, + }) + + if (!response.ok && response.status !== 404) { + {service}Logger.warn(`[${requestId}] Failed to delete webhook (non-fatal): ${response.status}`) + } else { + {service}Logger.info(`[${requestId}] Successfully deleted webhook ${externalId}`) + } + } catch (error) { + {service}Logger.warn(`[${requestId}] Error deleting webhook (non-fatal)`, error) + } +} +``` + +3. Add to `cleanupExternalWebhook`: +```typescript +export async function cleanupExternalWebhook(...): Promise { + // ... existing providers ... + } else if (webhook.provider === '{service}') { + await delete{Service}Webhook(webhook, requestId) + } +} +``` + +### Key Points for Automatic Registration + +- **API Key visibility**: Always use `password: true` for API key fields +- **Error handling**: Roll back the database webhook if external creation fails +- **External ID storage**: Save the external webhook ID in `providerConfig.externalId` +- **Graceful cleanup**: Don't fail webhook deletion if cleanup fails (use non-fatal logging) +- **User-friendly errors**: Map HTTP status codes to helpful error messages + +## The buildTriggerSubBlocks Helper + +This is the generic helper from `@/triggers` that creates consistent trigger subBlocks. + +### Function Signature + +```typescript +interface BuildTriggerSubBlocksOptions { + triggerId: string // e.g., 'service_event_a' + triggerOptions: Array<{ label: string; id: string }> // Dropdown options + includeDropdown?: boolean // true only for primary trigger + setupInstructions: string // HTML instructions + extraFields?: SubBlockConfig[] // Service-specific fields + webhookPlaceholder?: string // Custom placeholder text +} + +function buildTriggerSubBlocks(options: BuildTriggerSubBlocksOptions): SubBlockConfig[] +``` + +### What It Creates + +The helper creates this structure: +1. **Dropdown** (only if `includeDropdown: true`) - Trigger type selector +2. **Webhook URL** - Read-only field with copy button +3. **Extra Fields** - Your service-specific fields (filters, options, etc.) +4. **Save Button** - Activates the trigger +5. **Instructions** - Setup guide for users + +All fields automatically have: +- `mode: 'trigger'` - Only shown in trigger mode +- `condition: { field: 'selectedTriggerId', value: triggerId }` - Only shown when this trigger is selected + +## Trigger Outputs & Webhook Input Formatting + +### Important: Two Sources of Truth + +There are two related but separate concerns: + +1. **Trigger `outputs`** - Schema/contract defining what fields SHOULD be available. Used by UI for tag dropdown. +2. **`formatWebhookInput`** - Implementation that transforms raw webhook payload into actual data. Located in `apps/sim/lib/webhooks/utils.server.ts`. + +**These MUST be aligned.** The fields returned by `formatWebhookInput` should match what's defined in trigger `outputs`. If they differ: +- Tag dropdown shows fields that don't exist (broken variable resolution) +- Or actual data has fields not shown in dropdown (users can't discover them) + +### When to Add a formatWebhookInput Handler + +- **Simple providers**: If the raw webhook payload structure already matches your outputs, you don't need a handler. The generic fallback returns `body` directly. +- **Complex providers**: If you need to transform, flatten, extract nested data, compute fields, or handle conditional logic, add a handler. + +### Adding a Handler + +In `apps/sim/lib/webhooks/utils.server.ts`, add a handler block: + +```typescript +if (foundWebhook.provider === '{service}') { + // Transform raw webhook body to match trigger outputs + return { + eventType: body.type, + resourceId: body.data?.id || '', + timestamp: body.created_at, + resource: body.data, + } +} +``` + +**Key rules:** +- Return fields that match your trigger `outputs` definition exactly +- No wrapper objects like `webhook: { data: ... }` or `{service}: { ... }` +- No duplication (don't spread body AND add individual fields) +- Use `null` for missing optional data, not empty objects with empty strings + +### Verify Alignment + +Run the alignment checker: +```bash +bunx scripts/check-trigger-alignment.ts {service} +``` + +## Trigger Outputs + +Trigger outputs use the same schema as block outputs (NOT tool outputs). + +**Supported:** +- `type` and `description` for simple fields +- Nested object structure for complex data + +**NOT Supported:** +- `optional: true` (tool outputs only) +- `items` property (tool outputs only) + +```typescript +export function buildOutputs(): Record { + return { + // Simple fields + eventType: { type: 'string', description: 'Event type' }, + timestamp: { type: 'string', description: 'When it occurred' }, + + // Complex data - use type: 'json' + payload: { type: 'json', description: 'Full event payload' }, + + // Nested structure + resource: { + id: { type: 'string', description: 'Resource ID' }, + name: { type: 'string', description: 'Resource name' }, + }, + } +} +``` + +## Generic Webhook Trigger Pattern + +For services with many event types, create a generic webhook that accepts all events: + +```typescript +export const {service}WebhookTrigger: TriggerConfig = { + id: '{service}_webhook', + name: '{Service} Webhook (All Events)', + // ... + + subBlocks: buildTriggerSubBlocks({ + triggerId: '{service}_webhook', + triggerOptions: {service}TriggerOptions, + setupInstructions: {service}SetupInstructions('All Events'), + extraFields: [ + // Event type filter (optional) + { + id: 'eventTypes', + title: 'Event Types', + type: 'dropdown', + multiSelect: true, + options: [ + { label: 'Event A', id: 'event_a' }, + { label: 'Event B', id: 'event_b' }, + ], + placeholder: 'Leave empty for all events', + mode: 'trigger', + condition: { field: 'selectedTriggerId', value: '{service}_webhook' }, + }, + // Plus any other service-specific fields + ...build{Service}ExtraFields('{service}_webhook'), + ], + }), +} +``` + +## Checklist Before Finishing + +### Utils +- [ ] Created `{service}TriggerOptions` array with all trigger IDs +- [ ] Created `{service}SetupInstructions` function with clear steps +- [ ] Created `build{Service}ExtraFields` for service-specific fields +- [ ] Created output builders for each trigger type + +### Triggers +- [ ] Primary trigger has `includeDropdown: true` +- [ ] Secondary triggers do NOT have `includeDropdown` +- [ ] All triggers use `buildTriggerSubBlocks` helper +- [ ] All triggers have proper outputs defined +- [ ] Created `index.ts` barrel export + +### Registration +- [ ] All triggers imported in `triggers/registry.ts` +- [ ] All triggers added to `TRIGGER_REGISTRY` +- [ ] Block has `triggers.enabled: true` +- [ ] Block has all trigger IDs in `triggers.available` +- [ ] Block spreads all trigger subBlocks: `...getTrigger('id').subBlocks` + +### Automatic Webhook Registration (if supported) +- [ ] Added API key field to `build{Service}ExtraFields` with `password: true` +- [ ] Updated setup instructions for automatic webhook creation +- [ ] Added provider-specific logic to `apps/sim/app/api/webhooks/route.ts` +- [ ] Added `create{Service}WebhookSubscription` helper function +- [ ] Added `delete{Service}Webhook` function to `provider-subscriptions.ts` +- [ ] Added provider to `cleanupExternalWebhook` function + +### Webhook Input Formatting +- [ ] Added handler in `apps/sim/lib/webhooks/utils.server.ts` (if custom formatting needed) +- [ ] Handler returns fields matching trigger `outputs` exactly +- [ ] Run `bunx scripts/check-trigger-alignment.ts {service}` to verify alignment + +### Testing +- [ ] Run `bun run type-check` to verify no TypeScript errors +- [ ] Restart dev server to pick up new triggers +- [ ] Test trigger UI shows correctly in the block +- [ ] Test automatic webhook creation works (if applicable) diff --git a/.agents/skills/add-trigger/agents/openai.yaml b/.agents/skills/add-trigger/agents/openai.yaml new file mode 100644 index 00000000000..a22247a70c9 --- /dev/null +++ b/.agents/skills/add-trigger/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Add Trigger" + short_description: "Build Sim webhook triggers" + brand_color: "#DC2626" + default_prompt: "Use $add-trigger to create or update webhook triggers for a Sim integration." diff --git a/.agents/skills/validate-connector/SKILL.md b/.agents/skills/validate-connector/SKILL.md new file mode 100644 index 00000000000..4bafaa07dcb --- /dev/null +++ b/.agents/skills/validate-connector/SKILL.md @@ -0,0 +1,316 @@ +--- +name: validate-connector +description: Audit an existing Sim knowledge base connector against the service API docs and repository conventions, then report and fix issues in auth, config fields, pagination, document mapping, tags, and registry entries. Use when validating or repairing code in `apps/sim/connectors/{service}/`. +--- + +# Validate Connector Skill + +You are an expert auditor for Sim knowledge base connectors. Your job is to thoroughly validate that an existing connector is correct, complete, and follows all conventions. + +## Your Task + +When the user asks you to validate a connector: +1. Read the service's API documentation (via Context7 or WebFetch) +2. Read the connector implementation, OAuth config, and registry entries +3. Cross-reference everything against the API docs and Sim conventions +4. Report all issues found, grouped by severity (critical, warning, suggestion) +5. Fix all issues after reporting them + +## Step 1: Gather All Files + +Read **every** file for the connector — do not skip any: + +``` +apps/sim/connectors/{service}/{service}.ts # Connector implementation +apps/sim/connectors/{service}/index.ts # Barrel export +apps/sim/connectors/registry.ts # Connector registry entry +apps/sim/connectors/types.ts # ConnectorConfig interface, ExternalDocument, etc. +apps/sim/connectors/utils.ts # Shared utilities (computeContentHash, htmlToPlainText, etc.) +apps/sim/lib/oauth/oauth.ts # OAUTH_PROVIDERS — single source of truth for scopes +apps/sim/lib/oauth/utils.ts # getCanonicalScopesForProvider, getScopesForService, SCOPE_DESCRIPTIONS +apps/sim/lib/oauth/types.ts # OAuthService union type +apps/sim/components/icons.tsx # Icon definition for the service +``` + +If the connector uses selectors, also read: +``` +apps/sim/hooks/selectors/registry.ts # Selector key definitions +apps/sim/hooks/selectors/types.ts # SelectorKey union type +apps/sim/lib/workflows/subblocks/context.ts # SELECTOR_CONTEXT_FIELDS +``` + +## Step 2: Pull API Documentation + +Fetch the official API docs for the service. This is the **source of truth** for: +- Endpoint URLs, HTTP methods, and auth headers +- Required vs optional parameters +- Parameter types and allowed values +- Response shapes and field names +- Pagination patterns (cursor, offset, next token) +- Rate limits and error formats +- OAuth scopes and their meanings + +Use Context7 (resolve-library-id → query-docs) or WebFetch to retrieve documentation. If both fail, note which claims are based on training knowledge vs verified docs. + +## Step 3: Validate API Endpoints + +For **every** API call in the connector (`listDocuments`, `getDocument`, `validateConfig`, and any helper functions), verify against the API docs: + +### URLs and Methods +- [ ] Base URL is correct for the service's API version +- [ ] Endpoint paths match the API docs exactly +- [ ] HTTP method is correct (GET, POST, PUT, PATCH, DELETE) +- [ ] Path parameters are correctly interpolated and URI-encoded where needed +- [ ] Query parameters use correct names and formats per the API docs + +### Headers +- [ ] Authorization header uses the correct format: + - OAuth: `Authorization: Bearer ${accessToken}` + - API Key: correct header name per the service's docs +- [ ] `Content-Type` is set for POST/PUT/PATCH requests +- [ ] Any service-specific headers are present (e.g., `Notion-Version`, `Dropbox-API-Arg`) +- [ ] No headers are sent that the API doesn't support or silently ignores + +### Request Bodies +- [ ] POST/PUT body fields match API parameter names exactly +- [ ] Required fields are always sent +- [ ] Optional fields are conditionally included (not sent as `null` or empty unless the API expects that) +- [ ] Field value types match API expectations (string vs number vs boolean) + +### Input Sanitization +- [ ] User-controlled values interpolated into query strings are properly escaped: + - OData `$filter`: single quotes escaped with `''` (e.g., `externalId.replace(/'/g, "''")`) + - SOQL: single quotes escaped with `\'` + - GraphQL variables: passed as variables, not interpolated into query strings + - URL path segments: `encodeURIComponent()` applied +- [ ] URL-type config fields (e.g., `siteUrl`, `instanceUrl`) are normalized: + - Strip `https://` / `http://` prefix if the API expects bare domains + - Strip trailing `/` + - Apply `.trim()` before validation + +### Response Parsing +- [ ] Response structure is correctly traversed (e.g., `data.results` vs `data.items` vs `data`) +- [ ] Field names extracted match what the API actually returns +- [ ] Nullable fields are handled with `?? null` or `|| undefined` +- [ ] Error responses are checked before accessing data fields + +## Step 4: Validate OAuth Scopes (if OAuth connector) + +Scopes must be correctly declared and sufficient for all API calls the connector makes. + +### Connector requiredScopes +- [ ] `requiredScopes` in the connector's `auth` config lists all scopes needed by the connector +- [ ] Each scope in `requiredScopes` is a real, valid scope recognized by the service's API +- [ ] No invalid, deprecated, or made-up scopes are listed +- [ ] No unnecessary excess scopes beyond what the connector actually needs + +### Scope Subset Validation (CRITICAL) +- [ ] Every scope in `requiredScopes` exists in the OAuth provider's `scopes` array in `lib/oauth/oauth.ts` +- [ ] Find the provider in `OAUTH_PROVIDERS[providerGroup].services[serviceId].scopes` +- [ ] Verify: `requiredScopes` ⊆ `OAUTH_PROVIDERS scopes` (every required scope is present in the provider config) +- [ ] If a required scope is NOT in the provider config, flag as **critical** — the connector will fail at runtime + +### Scope Sufficiency +For each API endpoint the connector calls: +- [ ] Identify which scopes are required per the API docs +- [ ] Verify those scopes are included in the connector's `requiredScopes` +- [ ] If the connector calls endpoints requiring scopes not in `requiredScopes`, flag as **warning** + +### Token Refresh Config +- [ ] Check the `getOAuthTokenRefreshConfig` function in `lib/oauth/oauth.ts` for this provider +- [ ] `useBasicAuth` matches the service's token exchange requirements +- [ ] `supportsRefreshTokenRotation` matches whether the service issues rotating refresh tokens +- [ ] Token endpoint URL is correct + +## Step 5: Validate Pagination + +### listDocuments Pagination +- [ ] Cursor/pagination parameter name matches the API docs +- [ ] Response pagination field is correctly extracted (e.g., `next_cursor`, `nextPageToken`, `@odata.nextLink`, `offset`) +- [ ] `hasMore` is correctly determined from the response +- [ ] `nextCursor` is correctly passed back for the next page +- [ ] `maxItems` / `maxRecords` cap is correctly applied across pages using `syncContext.totalDocsFetched` +- [ ] Page size is within the API's allowed range (not exceeding max page size) +- [ ] Last page precision: when a `maxItems` cap exists, the final page request uses `Math.min(PAGE_SIZE, remaining)` to avoid fetching more records than needed +- [ ] No off-by-one errors in pagination tracking +- [ ] The connector does NOT hit known API pagination limits silently (e.g., HubSpot search 10k cap) + +### Pagination State Across Pages +- [ ] `syncContext` is used to cache state across pages (user names, field maps, instance URLs, portal IDs, etc.) +- [ ] Cached state in `syncContext` is correctly initialized on first page and reused on subsequent pages + +## Step 6: Validate Data Transformation + +### ExternalDocument Construction +- [ ] `externalId` is a stable, unique identifier from the source API +- [ ] `title` is extracted from the correct field and has a sensible fallback (e.g., `'Untitled'`) +- [ ] `content` is plain text — HTML content is stripped using `htmlToPlainText` from `@/connectors/utils` +- [ ] `mimeType` is `'text/plain'` +- [ ] `contentHash` is computed using `computeContentHash` from `@/connectors/utils` +- [ ] `sourceUrl` is a valid, complete URL back to the original resource (not relative) +- [ ] `metadata` contains all fields referenced by `mapTags` and `tagDefinitions` + +### Content Extraction +- [ ] Rich text / HTML fields are converted to plain text before indexing +- [ ] Important content is not silently dropped (e.g., nested blocks, table cells, code blocks) +- [ ] Content is not silently truncated without logging a warning +- [ ] Empty/blank documents are properly filtered out +- [ ] Size checks use `Buffer.byteLength(text, 'utf8')` not `text.length` when comparing against byte-based limits (e.g., `MAX_FILE_SIZE` in bytes) + +## Step 7: Validate Tag Definitions and mapTags + +### tagDefinitions +- [ ] Each `tagDefinition` has an `id`, `displayName`, and `fieldType` +- [ ] `fieldType` matches the actual data type: `'text'` for strings, `'number'` for numbers, `'date'` for dates, `'boolean'` for booleans +- [ ] Every `id` in `tagDefinitions` is returned by `mapTags` +- [ ] No `tagDefinition` references a field that `mapTags` never produces + +### mapTags +- [ ] Return keys match `tagDefinition` `id` values exactly +- [ ] Date values are properly parsed using `parseTagDate` from `@/connectors/utils` +- [ ] Array values are properly joined using `joinTagArray` from `@/connectors/utils` +- [ ] Number values are validated (not `NaN`) +- [ ] Metadata field names accessed in `mapTags` match what `listDocuments`/`getDocument` store in `metadata` + +## Step 8: Validate Config Fields and Validation + +### configFields +- [ ] Every field has `id`, `title`, `type` +- [ ] `required` is set explicitly (not omitted) +- [ ] Dropdown fields have `options` with `label` and `id` for each option +- [ ] Selector fields follow the canonical pair pattern: + - A `type: 'selector'` field with `selectorKey`, `canonicalParamId`, `mode: 'basic'` + - A `type: 'short-input'` field with the same `canonicalParamId`, `mode: 'advanced'` + - `required` is identical on both fields in the pair +- [ ] `selectorKey` values exist in the selector registry +- [ ] `dependsOn` references selector field `id` values, not `canonicalParamId` + +### validateConfig +- [ ] Validates all required fields are present before making API calls +- [ ] Validates optional numeric fields (checks `Number.isNaN`, positive values) +- [ ] Makes a lightweight API call to verify access (e.g., fetch 1 record, get profile) +- [ ] Uses `VALIDATE_RETRY_OPTIONS` for retry budget +- [ ] Returns `{ valid: true }` on success +- [ ] Returns `{ valid: false, error: 'descriptive message' }` on failure +- [ ] Catches exceptions and returns user-friendly error messages +- [ ] Does NOT make expensive calls (full data listing, large queries) + +## Step 9: Validate getDocument + +- [ ] Fetches a single document by `externalId` +- [ ] Returns `null` for 404 / not found (does not throw) +- [ ] Returns the same `ExternalDocument` shape as `listDocuments` +- [ ] Handles all content types that `listDocuments` can produce (e.g., if `listDocuments` returns both pages and blogposts, `getDocument` must handle both — not hardcode one endpoint) +- [ ] Forwards `syncContext` if it needs cached state (user names, field maps, etc.) +- [ ] Error handling is graceful (catches, logs, returns null or throws with context) +- [ ] Does not redundantly re-fetch data already included in the initial API response (e.g., if comments come back with the post, don't fetch them again separately) + +## Step 10: Validate General Quality + +### fetchWithRetry Usage +- [ ] All external API calls use `fetchWithRetry` from `@/lib/knowledge/documents/utils` +- [ ] No raw `fetch()` calls to external APIs +- [ ] `VALIDATE_RETRY_OPTIONS` used in `validateConfig` +- [ ] If `validateConfig` calls a shared helper (e.g., `linearGraphQL`, `resolveId`), that helper must accept and forward `retryOptions` to `fetchWithRetry` +- [ ] Default retry options used in `listDocuments`/`getDocument` + +### API Efficiency +- [ ] APIs that support field selection (e.g., `$select`, `sysparm_fields`, `fields`) should request only the fields the connector needs — in both `listDocuments` AND `getDocument` +- [ ] No redundant API calls: if a helper already fetches data (e.g., site metadata), callers should reuse the result instead of making a second call for the same information +- [ ] Sequential per-item API calls (fetching details for each document in a loop) should be batched with `Promise.all` and a concurrency limit of 3-5 + +### Error Handling +- [ ] Individual document failures are caught and logged without aborting the sync +- [ ] API error responses include status codes in error messages +- [ ] No unhandled promise rejections in concurrent operations + +### Concurrency +- [ ] Concurrent API calls use reasonable batch sizes (3-5 is typical) +- [ ] No unbounded `Promise.all` over large arrays + +### Logging +- [ ] Uses `createLogger` from `@sim/logger` (not `console.log`) +- [ ] Logs sync progress at `info` level +- [ ] Logs errors at `warn` or `error` level with context + +### Registry +- [ ] Connector is exported from `connectors/{service}/index.ts` +- [ ] Connector is registered in `connectors/registry.ts` +- [ ] Registry key matches the connector's `id` field + +## Step 11: Report and Fix + +### Report Format + +Group findings by severity: + +**Critical** (will cause runtime errors, data loss, or auth failures): +- Wrong API endpoint URL or HTTP method +- Invalid or missing OAuth scopes (not in provider config) +- Incorrect response field mapping (accessing wrong path) +- SOQL/query fields that don't exist on the target object +- Pagination that silently hits undocumented API limits +- Missing error handling that would crash the sync +- `requiredScopes` not a subset of OAuth provider scopes +- Query/filter injection: user-controlled values interpolated into OData `$filter`, SOQL, or query strings without escaping + +**Warning** (incorrect behavior, data quality issues, or convention violations): +- HTML content not stripped via `htmlToPlainText` +- `getDocument` not forwarding `syncContext` +- `getDocument` hardcoded to one content type when `listDocuments` returns multiple (e.g., only pages but not blogposts) +- Missing `tagDefinition` for metadata fields returned by `mapTags` +- Incorrect `useBasicAuth` or `supportsRefreshTokenRotation` in token refresh config +- Invalid scope names that the API doesn't recognize (even if silently ignored) +- Private resources excluded from name-based lookup despite scopes being available +- Silent data truncation without logging +- Size checks using `text.length` (character count) instead of `Buffer.byteLength` (byte count) for byte-based limits +- URL-type config fields not normalized (protocol prefix, trailing slashes cause API failures) +- `VALIDATE_RETRY_OPTIONS` not threaded through helper functions called by `validateConfig` + +**Suggestion** (minor improvements): +- Missing incremental sync support despite API supporting it +- Overly broad scopes that could be narrowed (not wrong, but could be tighter) +- Source URL format could be more specific +- Missing `orderBy` for deterministic pagination +- Redundant API calls that could be cached in `syncContext` +- Sequential per-item API calls that could be batched with `Promise.all` (concurrency 3-5) +- API supports field selection but connector fetches all fields (e.g., missing `$select`, `sysparm_fields`, `fields`) +- `getDocument` re-fetches data already included in the initial API response (e.g., comments returned with post) +- Last page of pagination requests full `PAGE_SIZE` when fewer records remain (`Math.min(PAGE_SIZE, remaining)`) + +### Fix All Issues + +After reporting, fix every **critical** and **warning** issue. Apply **suggestions** where they don't add unnecessary complexity. + +### Validation Output + +After fixing, confirm: +1. `bun run lint` passes +2. TypeScript compiles clean +3. Re-read all modified files to verify fixes are correct + +## Checklist Summary + +- [ ] Read connector implementation, types, utils, registry, and OAuth config +- [ ] Pulled and read official API documentation for the service +- [ ] Validated every API endpoint URL, method, headers, and body against API docs +- [ ] Validated input sanitization: no query/filter injection, URL fields normalized +- [ ] Validated OAuth scopes: `requiredScopes` ⊆ OAuth provider `scopes` in `oauth.ts` +- [ ] Validated each scope is real and recognized by the service's API +- [ ] Validated scopes are sufficient for all API endpoints the connector calls +- [ ] Validated token refresh config (`useBasicAuth`, `supportsRefreshTokenRotation`) +- [ ] Validated pagination: cursor names, page sizes, hasMore logic, no silent caps +- [ ] Validated data transformation: plain text extraction, HTML stripping, content hashing +- [ ] Validated tag definitions match mapTags output, correct fieldTypes +- [ ] Validated config fields: canonical pairs, selector keys, required flags +- [ ] Validated validateConfig: lightweight check, error messages, retry options +- [ ] Validated getDocument: null on 404, all content types handled, no redundant re-fetches, syncContext forwarding +- [ ] Validated fetchWithRetry used for all external calls (no raw fetch), VALIDATE_RETRY_OPTIONS threaded through helpers +- [ ] Validated API efficiency: field selection used, no redundant calls, sequential fetches batched +- [ ] Validated error handling: graceful failures, no unhandled rejections +- [ ] Validated logging: createLogger, no console.log +- [ ] Validated registry: correct export, correct key +- [ ] Reported all issues grouped by severity +- [ ] Fixed all critical and warning issues +- [ ] Ran `bun run lint` after fixes +- [ ] Verified TypeScript compiles clean diff --git a/.agents/skills/validate-connector/agents/openai.yaml b/.agents/skills/validate-connector/agents/openai.yaml new file mode 100644 index 00000000000..dc4d0366da8 --- /dev/null +++ b/.agents/skills/validate-connector/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Validate Connector" + short_description: "Audit a Sim knowledge connector" + brand_color: "#059669" + default_prompt: "Use $validate-connector to audit and fix a Sim knowledge connector against its API docs." diff --git a/.agents/skills/validate-integration/SKILL.md b/.agents/skills/validate-integration/SKILL.md new file mode 100644 index 00000000000..63a4875798f --- /dev/null +++ b/.agents/skills/validate-integration/SKILL.md @@ -0,0 +1,289 @@ +--- +name: validate-integration +description: Audit an existing Sim integration against the service API docs and repository conventions, then report and fix issues across tools, blocks, outputs, OAuth scopes, triggers, and registry entries. Use when validating or repairing a service integration under `apps/sim/tools`, `apps/sim/blocks`, or `apps/sim/triggers`. +--- + +# Validate Integration Skill + +You are an expert auditor for Sim integrations. Your job is to thoroughly validate that an existing integration is correct, complete, and follows all conventions. + +## Your Task + +When the user asks you to validate an integration: +1. Read the service's API documentation (via WebFetch or Context7) +2. Read every tool, the block, and registry entries +3. Cross-reference everything against the API docs and Sim conventions +4. Report all issues found, grouped by severity (critical, warning, suggestion) +5. Fix all issues after reporting them + +## Step 1: Gather All Files + +Read **every** file for the integration — do not skip any: + +``` +apps/sim/tools/{service}/ # All tool files, types.ts, index.ts +apps/sim/blocks/blocks/{service}.ts # Block definition +apps/sim/tools/registry.ts # Tool registry entries for this service +apps/sim/blocks/registry.ts # Block registry entry for this service +apps/sim/components/icons.tsx # Icon definition +apps/sim/lib/auth/auth.ts # OAuth config — should use getCanonicalScopesForProvider() +apps/sim/lib/oauth/oauth.ts # OAuth provider config — single source of truth for scopes +apps/sim/lib/oauth/utils.ts # Scope utilities, SCOPE_DESCRIPTIONS for modal UI +``` + +## Step 2: Pull API Documentation + +Fetch the official API docs for the service. This is the **source of truth** for: +- Endpoint URLs, HTTP methods, and auth headers +- Required vs optional parameters +- Parameter types and allowed values +- Response shapes and field names +- Pagination patterns (which param name, which response field) +- Rate limits and error formats + +## Step 3: Validate Tools + +For **every** tool file, check: + +### Tool ID and Naming +- [ ] Tool ID uses `snake_case`: `{service}_{action}` (e.g., `x_create_tweet`, `slack_send_message`) +- [ ] Tool `name` is human-readable (e.g., `'X Create Tweet'`) +- [ ] Tool `description` is a concise one-liner describing what it does +- [ ] Tool `version` is set (`'1.0.0'` or `'2.0.0'` for V2) + +### Params +- [ ] All required API params are marked `required: true` +- [ ] All optional API params are marked `required: false` +- [ ] Every param has explicit `required: true` or `required: false` — never omitted +- [ ] Param types match the API (`'string'`, `'number'`, `'boolean'`, `'json'`) +- [ ] Visibility is correct: + - `'hidden'` — ONLY for OAuth access tokens and system-injected params + - `'user-only'` — for API keys, credentials, and account-specific IDs the user must provide + - `'user-or-llm'` — for everything else (search queries, content, filters, IDs that could come from other blocks) +- [ ] Every param has a `description` that explains what it does + +### Request +- [ ] URL matches the API endpoint exactly (correct base URL, path segments, path params) +- [ ] HTTP method matches the API spec (GET, POST, PUT, PATCH, DELETE) +- [ ] Headers include correct auth pattern: + - OAuth: `Authorization: Bearer ${params.accessToken}` + - API Key: correct header name and format per the service's docs +- [ ] `Content-Type` header is set for POST/PUT/PATCH requests +- [ ] Body sends all required fields and only includes optional fields when provided +- [ ] For GET requests with query params: URL is constructed correctly with query string +- [ ] ID fields in URL paths are `.trim()`-ed to prevent copy-paste whitespace errors +- [ ] Path params use template literals correctly: `` `https://api.service.com/v1/${params.id.trim()}` `` + +### Response / transformResponse +- [ ] Correctly parses the API response (`await response.json()`) +- [ ] Extracts the right fields from the response structure (e.g., `data.data` vs `data` vs `data.results`) +- [ ] All nullable fields use `?? null` +- [ ] All optional arrays use `?? []` +- [ ] Error cases are handled: checks for missing/empty data and returns meaningful error +- [ ] Does NOT do raw JSON dumps — extracts meaningful, individual fields + +### Outputs +- [ ] All output fields match what the API actually returns +- [ ] No fields are missing that the API provides and users would commonly need +- [ ] No phantom fields defined that the API doesn't return +- [ ] `optional: true` is set on fields that may not exist in all responses +- [ ] When using `type: 'json'` and the shape is known, `properties` defines the inner fields +- [ ] When using `type: 'array'`, `items` defines the item structure with `properties` +- [ ] Field descriptions are accurate and helpful + +### Types (types.ts) +- [ ] Has param interfaces for every tool (e.g., `XCreateTweetParams`) +- [ ] Has response interfaces for every tool (extending `ToolResponse`) +- [ ] Optional params use `?` in the interface (e.g., `replyTo?: string`) +- [ ] Field names in types match actual API field names +- [ ] Shared response types are properly reused (e.g., `XTweetResponse` shared across tweet tools) + +### Barrel Export (index.ts) +- [ ] Every tool is exported +- [ ] All types are re-exported (`export * from './types'`) +- [ ] No orphaned exports (tools that don't exist) + +### Tool Registry (tools/registry.ts) +- [ ] Every tool is imported and registered +- [ ] Registry keys use snake_case and match tool IDs exactly +- [ ] Entries are in alphabetical order within the file + +## Step 4: Validate Block + +### Block ↔ Tool Alignment (CRITICAL) + +This is the most important validation — the block must be perfectly aligned with every tool it references. + +For **each tool** in `tools.access`: +- [ ] The operation dropdown has an option whose ID matches the tool ID (or the `tools.config.tool` function correctly maps to it) +- [ ] Every **required** tool param (except `accessToken`) has a corresponding subBlock input that is: + - Shown when that operation is selected (correct `condition`) + - Marked as `required: true` (or conditionally required) +- [ ] Every **optional** tool param has a corresponding subBlock input (or is intentionally omitted if truly never needed) +- [ ] SubBlock `id` values are unique across the entire block — no duplicates even across different conditions +- [ ] The `tools.config.tool` function returns the correct tool ID for every possible operation value +- [ ] The `tools.config.params` function correctly maps subBlock IDs to tool param names when they differ + +### SubBlocks +- [ ] Operation dropdown lists ALL tool operations available in `tools.access` +- [ ] Dropdown option labels are human-readable and descriptive +- [ ] Conditions use correct syntax: + - Single value: `{ field: 'operation', value: 'x_create_tweet' }` + - Multiple values (OR): `{ field: 'operation', value: ['x_create_tweet', 'x_delete_tweet'] }` + - Negation: `{ field: 'operation', value: 'delete', not: true }` + - Compound: `{ field: 'op', value: 'send', and: { field: 'type', value: 'dm' } }` +- [ ] Condition arrays include ALL operations that use that field — none missing +- [ ] `dependsOn` is set for fields that need other values (selectors depending on credential, cascading dropdowns) +- [ ] SubBlock types match tool param types: + - Enum/fixed options → `dropdown` + - Free text → `short-input` + - Long text/content → `long-input` + - True/false → `dropdown` with Yes/No options (not `switch` unless purely UI toggle) + - Credentials → `oauth-input` with correct `serviceId` +- [ ] Dropdown `value: () => 'default'` is set for dropdowns with a sensible default + +### Advanced Mode +- [ ] Optional, rarely-used fields are set to `mode: 'advanced'`: + - Pagination tokens / next tokens + - Time range filters (start/end time) + - Sort order / direction options + - Max results / per page limits + - Reply settings / threading options + - Rarely used IDs (reply-to, quote-tweet, etc.) + - Exclude filters +- [ ] **Required** fields are NEVER set to `mode: 'advanced'` +- [ ] Fields that users fill in most of the time are NOT set to `mode: 'advanced'` + +### WandConfig +- [ ] Timestamp fields have `wandConfig` with `generationType: 'timestamp'` +- [ ] Comma-separated list fields have `wandConfig` with a descriptive prompt +- [ ] Complex filter/query fields have `wandConfig` with format examples in the prompt +- [ ] All `wandConfig` prompts end with "Return ONLY the [format] - no explanations, no extra text." +- [ ] `wandConfig.placeholder` describes what to type in natural language + +### Tools Config +- [ ] `tools.access` lists **every** tool ID the block can use — none missing +- [ ] `tools.config.tool` returns the correct tool ID for each operation +- [ ] Type coercions are in `tools.config.params` (runs at execution time), NOT in `tools.config.tool` (runs at serialization time before variable resolution) +- [ ] `tools.config.params` handles: + - `Number()` conversion for numeric params that come as strings from inputs + - `Boolean` / string-to-boolean conversion for toggle params + - Empty string → `undefined` conversion for optional dropdown values + - Any subBlock ID → tool param name remapping +- [ ] No `Number()`, `JSON.parse()`, or other coercions in `tools.config.tool` — these would destroy dynamic references like `` + +### Block Outputs +- [ ] Outputs cover the key fields returned by ALL tools (not just one operation) +- [ ] Output types are correct (`'string'`, `'number'`, `'boolean'`, `'json'`) +- [ ] `type: 'json'` outputs either: + - Describe inner fields in the description string (GOOD): `'User profile (id, name, username, bio)'` + - Use nested output definitions (BEST): `{ id: { type: 'string' }, name: { type: 'string' } }` +- [ ] No opaque `type: 'json'` with vague descriptions like `'Response data'` +- [ ] Outputs that only appear for certain operations use `condition` if supported, or document which operations return them + +### Block Metadata +- [ ] `type` is snake_case (e.g., `'x'`, `'cloudflare'`) +- [ ] `name` is human-readable (e.g., `'X'`, `'Cloudflare'`) +- [ ] `description` is a concise one-liner +- [ ] `longDescription` provides detail for docs +- [ ] `docsLink` points to `'https://docs.sim.ai/tools/{service}'` +- [ ] `category` is `'tools'` +- [ ] `bgColor` uses the service's brand color hex +- [ ] `icon` references the correct icon component from `@/components/icons` +- [ ] `authMode` is set correctly (`AuthMode.OAuth` or `AuthMode.ApiKey`) +- [ ] Block is registered in `blocks/registry.ts` alphabetically + +### Block Inputs +- [ ] `inputs` section lists all subBlock params that the block accepts +- [ ] Input types match the subBlock types +- [ ] When using `canonicalParamId`, inputs list the canonical ID (not the raw subBlock IDs) + +## Step 5: Validate OAuth Scopes (if OAuth service) + +Scopes are centralized — the single source of truth is `OAUTH_PROVIDERS` in `lib/oauth/oauth.ts`. + +- [ ] Scopes defined in `lib/oauth/oauth.ts` under `OAUTH_PROVIDERS[provider].services[service].scopes` +- [ ] `auth.ts` uses `getCanonicalScopesForProvider(providerId)` — NOT a hardcoded array +- [ ] Block `requiredScopes` uses `getScopesForService(serviceId)` — NOT a hardcoded array +- [ ] No hardcoded scope arrays in `auth.ts` or block files (should all use utility functions) +- [ ] Each scope has a human-readable description in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` +- [ ] No excess scopes that aren't needed by any tool + +## Step 6: Validate Pagination Consistency + +If any tools support pagination: +- [ ] Pagination param names match the API docs (e.g., `pagination_token` vs `next_token` vs `cursor`) +- [ ] Different API endpoints that use different pagination param names have separate subBlocks in the block +- [ ] Pagination response fields (`nextToken`, `cursor`, etc.) are included in tool outputs +- [ ] Pagination subBlocks are set to `mode: 'advanced'` + +## Step 7: Validate Error Handling + +- [ ] `transformResponse` checks for error conditions before accessing data +- [ ] Error responses include meaningful messages (not just generic "failed") +- [ ] HTTP error status codes are handled (check `response.ok` or status codes) + +## Step 8: Report and Fix + +### Report Format + +Group findings by severity: + +**Critical** (will cause runtime errors or incorrect behavior): +- Wrong endpoint URL or HTTP method +- Missing required params or wrong `required` flag +- Incorrect response field mapping (accessing wrong path in response) +- Missing error handling that would cause crashes +- Tool ID mismatch between tool file, registry, and block `tools.access` +- OAuth scopes missing in `auth.ts` that tools need +- `tools.config.tool` returning wrong tool ID for an operation +- Type coercions in `tools.config.tool` instead of `tools.config.params` + +**Warning** (follows conventions incorrectly or has usability issues): +- Optional field not set to `mode: 'advanced'` +- Missing `wandConfig` on timestamp/complex fields +- Wrong `visibility` on params (e.g., `'hidden'` instead of `'user-or-llm'`) +- Missing `optional: true` on nullable outputs +- Opaque `type: 'json'` without property descriptions +- Missing `.trim()` on ID fields in request URLs +- Missing `?? null` on nullable response fields +- Block condition array missing an operation that uses that field +- Hardcoded scope arrays instead of using `getScopesForService()` / `getCanonicalScopesForProvider()` +- Missing scope description in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` + +**Suggestion** (minor improvements): +- Better description text +- Inconsistent naming across tools +- Missing `longDescription` or `docsLink` +- Pagination fields that could benefit from `wandConfig` + +### Fix All Issues + +After reporting, fix every **critical** and **warning** issue. Apply **suggestions** where they don't add unnecessary complexity. + +### Validation Output + +After fixing, confirm: +1. `bun run lint` passes with no fixes needed +2. TypeScript compiles clean (no type errors) +3. Re-read all modified files to verify fixes are correct + +## Checklist Summary + +- [ ] Read ALL tool files, block, types, index, and registries +- [ ] Pulled and read official API documentation +- [ ] Validated every tool's ID, params, request, response, outputs, and types against API docs +- [ ] Validated block ↔ tool alignment (every tool param has a subBlock, every condition is correct) +- [ ] Validated advanced mode on optional/rarely-used fields +- [ ] Validated wandConfig on timestamps and complex inputs +- [ ] Validated tools.config mapping, tool selector, and type coercions +- [ ] Validated block outputs match what tools return, with typed JSON where possible +- [ ] Validated OAuth scopes use centralized utilities (getScopesForService, getCanonicalScopesForProvider) — no hardcoded arrays +- [ ] Validated scope descriptions exist in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` for all scopes +- [ ] Validated pagination consistency across tools and block +- [ ] Validated error handling (error checks, meaningful messages) +- [ ] Validated registry entries (tools and block, alphabetical, correct imports) +- [ ] Reported all issues grouped by severity +- [ ] Fixed all critical and warning issues +- [ ] Ran `bun run lint` after fixes +- [ ] Verified TypeScript compiles clean diff --git a/.agents/skills/validate-integration/agents/openai.yaml b/.agents/skills/validate-integration/agents/openai.yaml new file mode 100644 index 00000000000..90b57edf73f --- /dev/null +++ b/.agents/skills/validate-integration/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Validate Integration" + short_description: "Audit a Sim service integration" + brand_color: "#B45309" + default_prompt: "Use $validate-integration to audit and fix a Sim integration against its API docs." diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000000..004252ac6a4 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,383 @@ +# Sim Development Guidelines + +You are a professional software engineer. All code must follow best practices: accurate, readable, clean, and efficient. + +## Global Standards + +- **Logging**: Import `createLogger` from `@sim/logger`. Use `logger.info`, `logger.warn`, `logger.error` instead of `console.log` +- **Comments**: Use TSDoc for documentation. No `====` separators. No non-TSDoc comments +- **Styling**: Never update global styles. Keep all styling local to components +- **Package Manager**: Use `bun` and `bunx`, not `npm` and `npx` + +## Architecture + +### Core Principles +1. Single Responsibility: Each component, hook, store has one clear purpose +2. Composition Over Complexity: Break down complex logic into smaller pieces +3. Type Safety First: TypeScript interfaces for all props, state, return types +4. Predictable State: Zustand for global state, useState for UI-only concerns + +### Root Structure +``` +apps/sim/ +├── app/ # Next.js app router (pages, API routes) +├── blocks/ # Block definitions and registry +├── components/ # Shared UI (emcn/, ui/) +├── executor/ # Workflow execution engine +├── hooks/ # Shared hooks (queries/, selectors/) +├── lib/ # App-wide utilities +├── providers/ # LLM provider integrations +├── stores/ # Zustand stores +├── tools/ # Tool definitions +└── triggers/ # Trigger definitions +``` + +### Naming Conventions +- Components: PascalCase (`WorkflowList`) +- Hooks: `use` prefix (`useWorkflowOperations`) +- Files: kebab-case (`workflow-list.tsx`) +- Stores: `stores/feature/store.ts` +- Constants: SCREAMING_SNAKE_CASE +- Interfaces: PascalCase with suffix (`WorkflowListProps`) + +## Imports + +**Always use absolute imports.** Never use relative imports. + +```typescript +// ✓ Good +import { useWorkflowStore } from '@/stores/workflows/store' + +// ✗ Bad +import { useWorkflowStore } from '../../../stores/workflows/store' +``` + +Use barrel exports (`index.ts`) when a folder has 3+ exports. Do not re-export from non-barrel files; import directly from the source. + +### Import Order +1. React/core libraries +2. External libraries +3. UI components (`@/components/emcn`, `@/components/ui`) +4. Utilities (`@/lib/...`) +5. Stores (`@/stores/...`) +6. Feature imports +7. CSS imports + +Use `import type { X }` for type-only imports. + +## TypeScript + +1. No `any` - Use proper types or `unknown` with type guards +2. Always define props interface for components +3. `as const` for constant objects/arrays +4. Explicit ref types: `useRef(null)` + +## Components + +```typescript +'use client' // Only if using hooks + +const CONFIG = { SPACING: 8 } as const + +interface ComponentProps { + requiredProp: string + optionalProp?: boolean +} + +export function Component({ requiredProp, optionalProp = false }: ComponentProps) { + // Order: refs → external hooks → store hooks → custom hooks → state → useMemo → useCallback → useEffect → return +} +``` + +Extract when: 50+ lines, used in 2+ files, or has own state/logic. Keep inline when: < 10 lines, single use, purely presentational. + +## Hooks + +```typescript +interface UseFeatureProps { id: string } + +export function useFeature({ id }: UseFeatureProps) { + const idRef = useRef(id) + const [data, setData] = useState(null) + + useEffect(() => { idRef.current = id }, [id]) + + const fetchData = useCallback(async () => { ... }, []) // Empty deps when using refs + + return { data, fetchData } +} +``` + +## Zustand Stores + +Stores live in `stores/`. Complex stores split into `store.ts` + `types.ts`. + +```typescript +import { create } from 'zustand' +import { devtools } from 'zustand/middleware' + +const initialState = { items: [] as Item[] } + +export const useFeatureStore = create()( + devtools( + (set, get) => ({ + ...initialState, + setItems: (items) => set({ items }), + reset: () => set(initialState), + }), + { name: 'feature-store' } + ) +) +``` + +Use `devtools` middleware. Use `persist` only when data should survive reload with `partialize` to persist only necessary state. + +## React Query + +All React Query hooks live in `hooks/queries/`. All server state must go through React Query — never use `useState` + `fetch` in components for data fetching or mutations. + +### Query Key Factory + +Every file must have a hierarchical key factory with an `all` root key and intermediate plural keys for prefix invalidation: + +```typescript +export const entityKeys = { + all: ['entity'] as const, + lists: () => [...entityKeys.all, 'list'] as const, + list: (workspaceId?: string) => [...entityKeys.lists(), workspaceId ?? ''] as const, + details: () => [...entityKeys.all, 'detail'] as const, + detail: (id?: string) => [...entityKeys.details(), id ?? ''] as const, +} +``` + +### Query Hooks + +- Every `queryFn` must forward `signal` for request cancellation +- Every query must have an explicit `staleTime` +- Use `keepPreviousData` only on variable-key queries (where params change), never on static keys + +```typescript +export function useEntityList(workspaceId?: string) { + return useQuery({ + queryKey: entityKeys.list(workspaceId), + queryFn: ({ signal }) => fetchEntities(workspaceId as string, signal), + enabled: Boolean(workspaceId), + staleTime: 60 * 1000, + placeholderData: keepPreviousData, // OK: workspaceId varies + }) +} +``` + +### Mutation Hooks + +- Use targeted invalidation (`entityKeys.lists()`) not broad (`entityKeys.all`) when possible +- For optimistic updates: use `onSettled` (not `onSuccess`) for cache reconciliation — `onSettled` fires on both success and error +- Don't include mutation objects in `useCallback` deps — `.mutate()` is stable in TanStack Query v5 + +```typescript +export function useUpdateEntity() { + const queryClient = useQueryClient() + return useMutation({ + mutationFn: async (variables) => { /* ... */ }, + onMutate: async (variables) => { + await queryClient.cancelQueries({ queryKey: entityKeys.detail(variables.id) }) + const previous = queryClient.getQueryData(entityKeys.detail(variables.id)) + queryClient.setQueryData(entityKeys.detail(variables.id), /* optimistic */) + return { previous } + }, + onError: (_err, variables, context) => { + queryClient.setQueryData(entityKeys.detail(variables.id), context?.previous) + }, + onSettled: (_data, _error, variables) => { + queryClient.invalidateQueries({ queryKey: entityKeys.lists() }) + queryClient.invalidateQueries({ queryKey: entityKeys.detail(variables.id) }) + }, + }) +} +``` + +## Styling + +Use Tailwind only, no inline styles. Use `cn()` from `@/lib/utils` for conditional classes. + +```typescript +
+``` + +## EMCN Components + +Import from `@/components/emcn`, never from subpaths (except CSS files). Use CVA when 2+ variants exist. + +## Testing + +Use Vitest. Test files: `feature.ts` → `feature.test.ts`. See `.cursor/rules/sim-testing.mdc` for full details. + +### Global Mocks (vitest.setup.ts) + +`@sim/db`, `drizzle-orm`, `@sim/logger`, `@/blocks/registry`, `@trigger.dev/sdk`, and store mocks are provided globally. Do NOT re-mock them unless overriding behavior. + +### Standard Test Pattern + +```typescript +/** + * @vitest-environment node + */ +import { createMockRequest } from '@sim/testing' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockGetSession } = vi.hoisted(() => ({ + mockGetSession: vi.fn(), +})) + +vi.mock('@/lib/auth', () => ({ + auth: { api: { getSession: vi.fn() } }, + getSession: mockGetSession, +})) + +import { GET } from '@/app/api/my-route/route' + +describe('my route', () => { + beforeEach(() => { + vi.clearAllMocks() + mockGetSession.mockResolvedValue({ user: { id: 'user-1' } }) + }) + it('returns data', async () => { ... }) +}) +``` + +### Performance Rules + +- **NEVER** use `vi.resetModules()` + `vi.doMock()` + `await import()` — use `vi.hoisted()` + `vi.mock()` + static imports +- **NEVER** use `vi.importActual()` — mock everything explicitly +- **NEVER** use `mockAuth()`, `mockConsoleLogger()`, `setupCommonApiMocks()` from `@sim/testing` — they use `vi.doMock()` internally +- **Mock heavy deps** (`@/blocks`, `@/tools/registry`, `@/triggers`) in tests that don't need them +- **Use `@vitest-environment node`** unless DOM APIs are needed (`window`, `document`, `FormData`) +- **Avoid real timers** — use 1ms delays or `vi.useFakeTimers()` + +Use `@sim/testing` mocks/factories over local test data. + +## Utils Rules + +- Never create `utils.ts` for single consumer - inline it +- Create `utils.ts` when 2+ files need the same helper +- Check existing sources in `lib/` before duplicating + +## Adding Integrations + +New integrations require: **Tools** → **Block** → **Icon** → (optional) **Trigger** + +Always look up the service's API docs first. + +### 1. Tools (`tools/{service}/`) + +``` +tools/{service}/ +├── index.ts # Barrel export +├── types.ts # Params/response types +└── {action}.ts # Tool implementation +``` + +**Tool structure:** +```typescript +export const serviceTool: ToolConfig = { + id: 'service_action', + name: 'Service Action', + description: '...', + version: '1.0.0', + oauth: { required: true, provider: 'service' }, + params: { /* ... */ }, + request: { url: '/api/tools/service/action', method: 'POST', ... }, + transformResponse: async (response) => { /* ... */ }, + outputs: { /* ... */ }, +} +``` + +Register in `tools/registry.ts`. + +### 2. Block (`blocks/blocks/{service}.ts`) + +```typescript +export const ServiceBlock: BlockConfig = { + type: 'service', + name: 'Service', + description: '...', + category: 'tools', + bgColor: '#hexcolor', + icon: ServiceIcon, + subBlocks: [ /* see SubBlock Properties */ ], + tools: { access: ['service_action'], config: { tool: (p) => `service_${p.operation}`, params: (p) => ({ /* type coercions here */ }) } }, + inputs: { /* ... */ }, + outputs: { /* ... */ }, +} +``` + +Register in `blocks/registry.ts` (alphabetically). + +**Important:** `tools.config.tool` runs during serialization (before variable resolution). Never do `Number()` or other type coercions there — dynamic references like `` will be destroyed. Use `tools.config.params` for type coercions (it runs during execution, after variables are resolved). + +**SubBlock Properties:** +```typescript +{ + id: 'field', title: 'Label', type: 'short-input', placeholder: '...', + required: true, // or condition object + condition: { field: 'op', value: 'send' }, // show/hide + dependsOn: ['credential'], // clear when dep changes + mode: 'basic', // 'basic' | 'advanced' | 'both' | 'trigger' +} +``` + +**condition examples:** +- `{ field: 'op', value: 'send' }` - show when op === 'send' +- `{ field: 'op', value: ['a','b'] }` - show when op is 'a' OR 'b' +- `{ field: 'op', value: 'x', not: true }` - show when op !== 'x' +- `{ field: 'op', value: 'x', not: true, and: { field: 'type', value: 'dm', not: true } }` - complex + +**dependsOn:** `['field']` or `{ all: ['a'], any: ['b', 'c'] }` + +**File Input Pattern (basic/advanced mode):** +```typescript +// Basic: file-upload UI +{ id: 'uploadFile', type: 'file-upload', canonicalParamId: 'file', mode: 'basic' }, +// Advanced: reference from other blocks +{ id: 'fileRef', type: 'short-input', canonicalParamId: 'file', mode: 'advanced' }, +``` + +In `tools.config.tool`, normalize with: +```typescript +import { normalizeFileInput } from '@/blocks/utils' +const file = normalizeFileInput(params.uploadFile || params.fileRef, { single: true }) +if (file) params.file = file +``` + +For file uploads, create an internal API route (`/api/tools/{service}/upload`) that uses `downloadFileFromStorage` to get file content from `UserFile` objects. + +### 3. Icon (`components/icons.tsx`) + +```typescript +export function ServiceIcon(props: SVGProps) { + return /* SVG from brand assets */ +} +``` + +### 4. Trigger (`triggers/{service}/`) - Optional + +``` +triggers/{service}/ +├── index.ts # Barrel export +├── webhook.ts # Webhook handler +└── {event}.ts # Event-specific handlers +``` + +Register in `triggers/registry.ts`. + +### Integration Checklist + +- [ ] Look up API docs +- [ ] Create `tools/{service}/` with types and tools +- [ ] Register tools in `tools/registry.ts` +- [ ] Add icon to `components/icons.tsx` +- [ ] Create block in `blocks/blocks/{service}.ts` +- [ ] Register block in `blocks/registry.ts` +- [ ] (Optional) Create and register triggers +- [ ] (If file uploads) Create internal API route with `downloadFileFromStorage` +- [ ] (If file uploads) Use `normalizeFileInput` in block config diff --git a/apps/sim/AGENTS.md b/apps/sim/AGENTS.md new file mode 100644 index 00000000000..83a44c7939b --- /dev/null +++ b/apps/sim/AGENTS.md @@ -0,0 +1,33 @@ +# Sim App Scope + +These rules apply to files under `apps/sim/` in addition to the repository root [AGENTS.md](/AGENTS.md). + +## Architecture + +- Follow the app structure already established under `app/`, `blocks/`, `components/`, `executor/`, `hooks/`, `lib/`, `providers/`, `stores/`, `tools/`, and `triggers/`. +- Keep single responsibility for components, hooks, and stores. +- Prefer composition over large mixed-responsibility modules. +- Use `lib/` for app-wide helpers, feature-local `utils/` only when 2+ files share the helper, and inline single-use helpers. + +## Imports And Types + +- Always use absolute imports from `@/...`; do not add relative imports. +- Use barrel exports only when a folder has 3+ exports; do not re-export through non-barrel files. +- Use `import type` for type-only imports. +- Do not use `any`; prefer precise types or `unknown` with guards. + +## Components And Styling + +- Use `'use client'` only when hooks or browser-only APIs are required. +- Define a props interface for every component. +- Extract constants with `as const` where appropriate. +- Use Tailwind classes and `cn()` for conditional classes; avoid inline styles unless CSS variables are the intended mechanism. +- Keep styling local to the component; do not modify global styles for feature work. + +## Testing + +- Use Vitest. +- Prefer `@vitest-environment node` unless DOM APIs are required. +- Use `vi.hoisted()` + `vi.mock()` + static imports; do not use `vi.resetModules()` + `vi.doMock()` + dynamic imports except for true module-scope singletons. +- Do not use `vi.importActual()`. +- Prefer mocks and factories from `@sim/testing`. diff --git a/apps/sim/blocks/AGENTS.md b/apps/sim/blocks/AGENTS.md new file mode 100644 index 00000000000..c82d65915ba --- /dev/null +++ b/apps/sim/blocks/AGENTS.md @@ -0,0 +1,12 @@ +# Blocks Scope + +These rules apply to block definitions under `apps/sim/blocks/**`. + +- Keep block `type` values and tool mappings aligned with the actual integration tool IDs. +- Every subblock `id` must be unique within the block, even across different conditions. +- Use `condition`, `required`, `dependsOn`, and `mode` deliberately to reflect the UX and execution requirements. +- Use `canonicalParamId` only to link alternative inputs for the same logical parameter; do not reuse it as a subblock `id`. +- If one field in a canonical group is required, all alternatives in that group must also be required. +- Put type coercion in `tools.config.params`, never in `tools.config.tool`. +- When supporting file inputs, follow the basic/advanced pattern and normalize with `normalizeFileInput`. +- Keep block outputs aligned with what the referenced tools actually return. diff --git a/apps/sim/components/emcn/AGENTS.md b/apps/sim/components/emcn/AGENTS.md new file mode 100644 index 00000000000..6050383d489 --- /dev/null +++ b/apps/sim/components/emcn/AGENTS.md @@ -0,0 +1,11 @@ +# EMCN Components Scope + +These rules apply to `apps/sim/components/emcn/**`. + +- Import from `@/components/emcn`, never from subpaths except CSS files. +- Use Radix UI primitives for accessibility where applicable. +- Use CVA when a component has 2+ variants; use direct `className` composition for single-style components. +- Export both the component and its variants helper when using CVA. +- Keep tokens consistent with the existing library style such as `font-medium`, `text-[12px]`, and `rounded-[4px]`. +- Prefer `transition-colors` for interactive hover and active states. +- Use TSDoc when documenting public components or APIs. diff --git a/apps/sim/hooks/AGENTS.md b/apps/sim/hooks/AGENTS.md new file mode 100644 index 00000000000..c682642a34f --- /dev/null +++ b/apps/sim/hooks/AGENTS.md @@ -0,0 +1,11 @@ +# Hooks Scope + +These rules apply to custom hooks under `apps/sim/**/hooks/**` and `apps/sim/**/use-*.ts`. + +- Give each hook a single clear responsibility. +- Define a props interface for hook inputs. +- Use refs for stable callback dependencies when avoiding dependency churn. +- Wrap returned operations in `useCallback` when the consumer benefits from stable function identity. +- Track loading and error states for async hooks when relevant. +- Use `try`/`catch` around async operations that can fail. +- Keep hook logic separate from rendering concerns. diff --git a/apps/sim/hooks/queries/AGENTS.md b/apps/sim/hooks/queries/AGENTS.md new file mode 100644 index 00000000000..2a461d88a9c --- /dev/null +++ b/apps/sim/hooks/queries/AGENTS.md @@ -0,0 +1,13 @@ +# React Query Scope + +These rules apply to `apps/sim/hooks/queries/**`. + +- All server state must go through React Query; do not use ad hoc `useState` + `fetch` patterns in components for server data. +- Define a query key factory in each file with an `all` root key and intermediate plural keys such as `lists()` and `details()` for prefix invalidation. +- Do not use inline query keys. +- Every `queryFn` must forward `signal` for cancellation. +- Every query must set an explicit `staleTime`. +- Use `keepPreviousData` only for variable-key queries where params change. +- Prefer targeted invalidation such as `entityKeys.lists()` over broad invalidation such as `entityKeys.all`. +- For optimistic updates, reconcile caches in `onSettled`, not only `onSuccess`. +- Do not include stable TanStack mutation objects in `useCallback` deps just to call `.mutate()`. diff --git a/apps/sim/stores/AGENTS.md b/apps/sim/stores/AGENTS.md new file mode 100644 index 00000000000..2e46265f94c --- /dev/null +++ b/apps/sim/stores/AGENTS.md @@ -0,0 +1,12 @@ +# Stores Scope + +These rules apply to Zustand stores under `apps/sim/**/stores/**` and `apps/sim/**/store.ts`. + +- Use `devtools` middleware for stores unless there is a clear reason not to. +- Use `persist` only when state should survive reloads. +- When persisting, use `partialize` to store only the minimum required fields. +- Use immutable updates only. +- Use `set((state) => ...)` when the next value depends on previous state. +- Expose a `reset()` action for non-trivial stores. +- Split more complex stores into `store.ts` and `types.ts`. +- Use hydration tracking such as `_hasHydrated` when persisted stores need it. diff --git a/apps/sim/tools/AGENTS.md b/apps/sim/tools/AGENTS.md new file mode 100644 index 00000000000..dcdf51aa14e --- /dev/null +++ b/apps/sim/tools/AGENTS.md @@ -0,0 +1,13 @@ +# Tools Scope + +These rules apply to integration tool definitions under `apps/sim/tools/**`. + +- Start from the service API docs before adding or changing a tool. +- Keep each service under `tools/{service}/` with `index.ts`, `types.ts`, and one file per action. +- Tool IDs must use `snake_case` and match registry keys exactly. +- Use `visibility: 'hidden'` only for system-injected params such as OAuth access tokens. +- Use `visibility: 'user-only'` for credentials and account-specific values the user must provide. +- Use `visibility: 'user-or-llm'` for ordinary operation parameters. +- In `transformResponse`, extract meaningful fields instead of dumping raw JSON. +- Use `?? null` for nullable response fields and `?? []` for optional arrays where appropriate. +- Register every tool in `tools/registry.ts` and keep entries aligned with the exported tool IDs. diff --git a/apps/sim/triggers/AGENTS.md b/apps/sim/triggers/AGENTS.md new file mode 100644 index 00000000000..2438ea43458 --- /dev/null +++ b/apps/sim/triggers/AGENTS.md @@ -0,0 +1,10 @@ +# Triggers Scope + +These rules apply to trigger definitions under `apps/sim/triggers/**`. + +- Research the service webhook model before implementing triggers. +- Keep each service under `triggers/{service}/` with barrel exports and event-specific files. +- Use shared helpers for setup instructions, extra fields, and output definitions when multiple triggers in the same service need them. +- Ensure the primary trigger supports switching between trigger types when the integration pattern requires it. +- Keep trigger outputs explicit and useful for downstream blocks. +- Register triggers in `triggers/registry.ts` and keep IDs aligned with the integration naming scheme.