Reference

Architecture Overview

Understand how Swarmient works under the hood. (Don’t worry—you don’t need to know this to use Swarmient, but it’s useful context.)

High-Level Flow

Developer          Control Plane          Miner
    |                    |                   |
    |---[1. Submit Job]->|                   |
    |                    |---[2. Offer Job]->|
    |                    |                   |
    |                    |<-[3. Accept]----- |
    |                    |                   |
    |<----[4. Stream Results]--[5. Execute]-|
    |                    |                   |
    |                    |<-[6. Proof]------|
    |                    |                   |
    |<---[7. Verify]-----|                   |
    |                    |                   |
    |<-[8. Results]------|                   |
    |                    |---[9. Settle]--->|
    |                    |                   |

Step by step:

  1. Developer submits a job via API
  2. Control plane validates and queues the job
  3. Control plane offers job to eligible miners
  4. Miner accepts the job
  5. Miner executes the task in sandbox
  6. Miner generates cryptographic proof and returns result
  7. Control plane verifies the proof
  8. Results streamed to developer
  9. Credits settled (automatic)

Five Core Services

Swarmient consists of five services that work together:

1. Protocol Core

Purpose: Cryptographic foundation for all data formats and transactions.

Responsibilities:

  • Define job and task schemas
  • Implement Ed25519 signing/verification
  • Handle serialization (ensuring deterministic output)
  • Generate proofs of execution

Language: TypeScript/JavaScript

Used by: All other services

Key insight: The protocol is the source of truth. All data flows through it.

2. Control Plane

Purpose: Central coordinator of the network.

Responsibilities:

  • Accept job submissions from developers
  • Validate job structure and developer credits
  • Match jobs to available miners
  • Verify proofs of execution
  • Settle payments (debit developer, credit miner)
  • Track miner reputation
  • Manage the job queue

Runs on: Central servers (Swarmient-hosted)

Key insight: The control plane is the “brain” of the network. It ensures fair matching and honest execution.

3. Gateway Proxy

Purpose: Developer’s local interface to Swarmient.

Responsibilities:

  • Provide API endpoint for developers
  • Manage developer accounts and API keys
  • Stream real-time job updates
  • Cache results locally
  • Provide dashboard UI

Runs on: Swarmient’s servers (developers access via HTTP)

Key insight: Developers never directly connect to miners. All communication goes through the gateway.

4. Compute Daemon

Purpose: Miner’s worker process.

Responsibilities:

  • Poll control plane for available jobs
  • Execute tasks in isolated sandbox
  • Generate cryptographic proofs
  • Submit results back to control plane
  • Track local resources (CPU, memory, disk)
  • Manage job lifecycle (setup, execution, cleanup)

Runs on: Miner’s machine (installed locally)

Key insight: This is what miners run. It’s lightweight and only uses resources when executing jobs.

5. Landing Site

Purpose: Public website and documentation.

Responsibilities:

  • Serve documentation (guides, API reference)
  • Serve homepage (marketing)
  • Serve login/registration pages
  • Serve user dashboards

Runs on: Swarmient’s CDN and servers

Key insight: Everything is open and transparent. Documentation is versioned with code.

Key Design Decisions

Trustless Architecture

Swarmient doesn’t require trusting any single party:

  • Developers don’t trust miners — Cryptographic proofs verify execution
  • Miners don’t trust developers — Control plane verifies payment before jobs execute
  • No one trusts the control plane — Proofs are mathematically verifiable (can be audited independently)

Deterministic Proofs

All proofs are deterministic:

  • Same input → Same output (always)
  • Proofs are cryptographically verifiable
  • Results are reproducible

This matters for: compliance, research, audit trails.

Atomic Payments

Payments settle in a single atomic transaction:

  • Job completes successfully
  • Cryptographic proof is verified
  • Credits transfer instantly (miner credited, developer debited)
  • No disputes, no escrow delays

Peer-to-Peer (Eventually)

Current design: Centralized control plane, peer-to-peer miners.

Future design: Fully peer-to-peer (control plane functions distributed).

Sandboxing

All jobs run in isolated sandboxes:

  • No access to filesystem (except temp directory)
  • No network access
  • No privileged syscalls
  • No inter-process communication

This ensures miners’ systems stay safe.

Data Flow

Submission

Developer submits job via API
           ↓
API validates job structure
           ↓
Check developer has sufficient credits (reserve them)
           ↓
Add job to queue
           ↓
Respond with job_id and estimated_cost

Matching

Job enters queue
           ↓
Control plane looks for eligible miners
  (checking: resource requirements, reputation, uptime)
           ↓
Send job offer to miner
           ↓
Miner accepts or rejects
           ↓
If accepted: send full job details to miner
If rejected: try next miner

Execution

Miner receives job
           ↓
Extract task details (prompt, timeout, etc.)
           ↓
Spin up sandbox (isolated container)
           ↓
Execute task in sandbox (run code, generate output)
           ↓
Sandbox terminates (clean up resources)
           ↓
Generate cryptographic proof (sign result with private key)
           ↓
Send proof + result back to control plane

Verification & Settlement

Control plane receives result + proof
           ↓
Verify proof signature (using miner's public key)
           ↓
Verify proof format is correct
           ↓
Calculate cost (execution_time × hourly_rate)
           ↓
Debit developer's account
           ↓
Credit miner's account
           ↓
Stream result to developer

Concurrency & Parallelism

Developer Side

Developers submit DAGs (directed acyclic graphs) of tasks:

Task A (no dependencies)
    ├─> Task B (depends on A)
    ├─> Task C (depends on A)
    └─> Task D (depends on B and C)

The control plane executes:

  • Task A immediately
  • Tasks B and C in parallel (both depend only on A)
  • Task D after B and C complete

Miner Side

Miners execute multiple tasks concurrently (if they have resources):

  • 8-core miner with 2-core concurrent limit: 2 tasks at a time
  • Tasks run in separate containers for isolation

Network Concurrency

Multiple developers and miners interact simultaneously:

  • 1000s of developers submitting jobs
  • 1000s of miners accepting work
  • Control plane orchestrates all of it

Security Model

Threat Model

Assumptions:

  • Developers might submit malicious code
  • Miners might try to fake results
  • Network might be unreliable

Protections:

Threat Protection
Malicious code Sandboxing (isolated execution)
Fake results Cryptographic proofs + control plane verification
Miner disconnect Automatic retry on different miner
Untrusted network TLS (encrypted transport) + signed proofs
Developer fraud Credit pre-reservation before job execution

Cryptographic Guarantees

Every result has a proof that proves:

  1. A specific miner executed the task
  2. The result hasn’t been tampered with
  3. The execution happened at a specific time

Why cryptography matters:

  • You can verify results without trusting Swarmient
  • Developers can audit miners independently
  • Regulators can verify compliance (reproducible, auditable)

Scalability

Control Plane Bottleneck

The control plane is currently centralized. Scalability limited by:

  • Job matching throughput (how fast can we assign jobs?)
  • Payment settlement throughput (how many txns/sec?)
  • Network bandwidth to miners

Current capacity: 10,000s of jobs/day

Future: Distributed control plane (sharded by region or job type)

Miner Scalability

Individual miners scale based on hardware:

  • 2-core CPU → 1–2 concurrent tasks → ~300 jobs/day
  • 16-core CPU + GPU → 4–8 concurrent tasks → 1000+ jobs/day

Swarmient can support millions of miners (each independent).

Developer Scalability

Developers are limited by API rate limits and credit balance:

  • 100 jobs/minute per API key (can request higher limit)
  • Unlimited total jobs if credit balance allows

Deployment

Current (Centralized)

Developer → [Gateway Proxy (Swarmient-hosted)]
                        ↓
                [Control Plane (Swarmient-hosted)]
                        ↓
Miner 1 ←→ [Compute Daemon (your machine)]
Miner 2 ←→ [Compute Daemon (your machine)]
...

Pros: Simple, reliable, centrally managed Cons: Swarmient is a single point of failure

Future (Distributed)

Developer → [Gateway Proxy (distributed)]
                        ↓
        [Control Plane (distributed/sharded)]
                        ↓
Miner 1 ←→ [Compute Daemon (peer-to-peer)]
Miner 2 ←→ [Compute Daemon (peer-to-peer)]
...

Pros: Decentralized, censorship-resistant Cons: More complex, eventual consistency

Monitoring & Observability

Developer Observability

Developers see:

  • Job status (PENDING → RUNNING → COMPLETED)
  • Real-time progress (via streaming)
  • Results and costs
  • Error messages and stack traces

Miner Observability

Miners see:

  • Current and completed jobs
  • Earnings per task
  • Hardware utilization
  • Network latency

Swarmient Observability

Swarmient monitors:

  • Job completion rates
  • Miner reliability
  • Network health
  • Cost calculations

Performance Characteristics

Operation Latency Notes
Submit job <100ms Depends on API latency
Job pickup 1–5s Depends on miner poll interval
Simple task execution 1–10s Depends on task complexity
Verification <100ms Cryptographic operations
Payment settlement <100ms Atomic database transaction
Result streaming <1s Depends on result size

Total end-to-end latency: 5–30 seconds (from submission to developer seeing results)

Future Roadmap

Planned Improvements

  1. Sharded control plane — Distribute coordination across regions
  2. Peer-to-peer matching — Miners discover jobs without central server
  3. Light client SDK — Developers can run mini control plane locally
  4. L2 scaling — Use rollups or sidechains for payment settlement
  5. Custom runtimes — Support Docker, WASM, or custom environments
  6. Reputation markets — Miners publish ratings publicly (on-chain)

Architecture Resources

For deeper understanding: