Search docs...
DocsArchitectureSystem Architecture

System Architecture

Process boundaries, monorepo packages, and local-first topologies in LeadForge OS.

6 min read•Edit on GitHub

LeadForge OS is built on a local-first architecture. The client machine's desktop hardware serves as the primary engine for heavy operations—running headless browser scrapes, website crawling, database queries, and AI qualifications—while external networks act as synchronization layers.

Overall Architecture

The diagram below details the boundary limits and communication channels between client and cloud processes.

Rendering Outline

Monorepo Structure & Package Boundaries

The workspace is organized as a monorepo powered by pnpm workspaces and Turborepo.

text
├── apps/
│   ├── api/                   # REST API Server built with Hono and MongoDB
│   └── desktop/               # Desktop application built with Electron, React, and SQLite
│   └── marketing/             # Marketing web app (Vite / Next.js)
└── packages/
    ├── agent-core/            # LLM orchestrator (agents, tools, memory, tracing)
    ├── agent-runtime/         # Dynamic agent session runtime & tool executors
    ├── ai/                    # LLM Provider integrations (OpenRouter & Ollama)
    ├── auth/                  # Hono middleware and better-auth configurations
    ├── core/                  # Core guards, pagination helpers, and schemas
    ├── logger/                # Daily-rotating file logging utilities
    ├── schema/                # TypeScript interfaces and IPC contract definitions
    ├── sdk/                   # HTTP transport wrapper for desktop sync engine
    └── workflow-engine/       # Drip sequence action runner

Dependency Rules

  • schema has zero dependencies and is imported by all packages.
  • core depends only on schema.
  • ai depends on schema.
  • agent-core depends on schema, core, and ai.
  • agent-runtime depends on agent-core and workflow-engine.
  • apps/desktop is the master consumer that imports all packages, but no package may import from apps/desktop or apps/api.

These boundaries are validated automatically in CI using dependency-cruiser (.dependency-cruiser.cjs).

Desktop Process Model

The Electron desktop application is split into three processes to enforce security and maintain UI responsiveness:

  1. Renderer Process (Chromium): Renders the React UI dashboard. It cannot access Node APIs directly and communicates only via safe channel bridges.
  2. Preload Script (contextBridge): Exposes a whitelisted set of IPC invoke methods to the window context, preventing arbitrary shell command executions.
  3. Main Process (Node.js): Runs SRE diagnostics, database migrations, updates settings, pings network sockets, and manages background worker child processes.

Background Job Scheduler

The job scheduler manages long-running collection tasks without exhausting local system resources.

Rendering Outline

Lifecycle Controls

  • Heartbeat Watchdog: Every 10 seconds, the main process pings the worker. The worker must respond with pong. If no reply is received within 30 seconds, the scheduler considers the worker stalled, terminates it with SIGKILL, and marks the job for retry.
  • State Checkpointing: Long-running jobs regularly write progress to the checkpointData column in SQLite. If paused or interrupted, the job starts from the last recorded offset.
  • Graceful Cancellation: Sends a cancel IPC command. The worker is given 15 seconds to gracefully close Playwright browsers and write database logs before being killed with SIGKILL.

AI & Agent Runtime

The AI runtime supports both cloud integrations and offline operations.

Provider Architecture

  • OpenRouter (Cloud): Used if openRouterKey is present in settings (defaults to Gemini Flash and Llama).
  • Ollama (Local): Integrates via local HTTP endpoint http://localhost:11434 for complete offline and privacy-oriented qualification.
  • Mock Fallback: A rule-based template engine that serves as a fallback if keys are missing or API boundaries timeout.

Tool Registry & Catalog

The ToolRegistry defines standard executable wrappers for core scrapers, making them available as LLM tools:

  • search_local_businesses: Playwright Maps Scraper.
  • crawl_company_website: Cheerio crawler.
  • search_linkedin_profiles: Voyager LinkedIn API wrapper.
  • send_outreach_email: Nodemailer SMTP dispatcher.
  • score_lead_opportunity: Opportunity scoring analyzer.

Local-First Data Model & Cloud Sync

LeadForge OS provides multi-workspace isolation. Each workspace represents a separate physical SQLite database file (leadforge_${workspaceId}.db).

The sequence below illustrates how local SQLite updates are reconciled with the MongoDB Atlas cloud cluster.

Rendering Outline
  • Conflict Resolution: The system uses Last-Write-Wins (LWW) resolution based on the updatedAt timestamp.
  • Sync Failure Handling: In case of failures, the engine halts queue processing, records the error in lastError, and retries with backoff to prevent out-of-order execution.

Logging, Telemetry & Diagnostics

Daily-Rotating Logging

  • System logs are written to rotating files using @leadforge/logger.
  • When exporting a Support Bundle, log files and the current configuration are packed into a ZIP file. All sensitive parameters (passwords, SMTP credentials, OpenRouter API keys) are replaced with [MASKED].

Telemetry & Diagnostics

  • No metrics are automatically uploaded to remote servers. All telemetry is stored locally.
  • SRE diagnostics query the local system state, testing:
    • SMTP Nodemailer port connection.
    • IMAP login and reply folder checks.
    • DNS resolution of OpenRouter endpoints.
    • SQLite database integrity check (PRAGMA integrity_check).
    • System memory consumption (RSS threshold: 800MB).
NEXT Setup & Installation