Terradev

# Terradev CLI v5.6.5

**An imperative command-line-interface for AI workload orchestration.**

![Terradev Demo](https://raw.githubusercontent.com/theoddden/Terradev/main/demo/terradev-demo.gif)

**License: Apache 2.0** - Free and open source for commercial and personal use.

pypi.org/project/terradev-cli/
<!-- mcp-name: io.github.theoddden/terradev -->

Terradev is a cross-cloud compute control plane for AI workloads, not just a provisioning wrapper. 

Combines quoting, provisioning, topology optimization, training orchestration, inference tuning, and cost analytics in one CLI, with a Rust-accelerated idempotent runtime underneath.

Continued focus on lower cost, faster provisioning, and topology-aware execution with local credential storage.

Model agnostic. Dataset agnostic. GPU agnostic. Provider agnostic. The only thing Terradev is not agnostic about is correctness: it enforces topology, idempotency, and sequencing.

**NOTES ON 5.6.1**

Added **Database Connection System** and **Unified Observability & Scheduling** commands:

- **Database Connection System** (`core/database_connection.py`): SQLite and PostgreSQL support with in-memory connection storage, connection ID system, auto-table creation for 4 standard metadata tables (dataset_versions, workflow_runs, idempotency_keys, node_executions), and database operations (query, upsert, get connection).

- **Observe Command** (`terradev observe`): Unified monitoring pipeline that wires API Gateway traffic into W&B, Phoenix, and Cost Analytics with a shared trace ID across all three destinations. Commands: `terradev observe gateway`, `terradev observe status`.

- **Schedule Command** (`terradev schedule`): Spot-aware scheduling for cost-optimized job execution with pricing window awareness (not just generic cron). Commands: `terradev schedule job`, `terradev schedule list`, `terradev schedule windows`.

- **MCP Tools**: 5 new database tools for connection management and queries (create_sqlite_connection, create_postgresql_connection, query_database, upsert_database, get_database_connection).

**NOTES ON 5.6.0**

Added **API Gateway for inference serving** with OpenAI/Anthropic/custom API entry and exit points:

- **Gateway Service** (`core/gateway_service.py`): FastAPI-based gateway that provides OpenAI-compatible, Anthropic-compatible, and custom workflow API endpoints for inference serving. Integrates with Terradev's inference router and KV cache management for intelligent routing.

- **Gateway CLI command** (`terradev gateway`):
  ```bash
  terradev gateway
  terradev gateway --host 0.0.0.0 --port 8080
  terradev gateway --no-anthropic --max-concurrent 50
  terradev gateway --model meta-llama/Llama-3.1-8B-Instruct
  ```

- **OpenAI-compatible endpoints**:
  - POST /v1/chat/completions
  - POST /v1/completions

- **Anthropic-compatible endpoints**:
  - POST /v1/messages
  - POST /v1/messages/batches

- **Custom workflow endpoints**:
  - POST /v1/custom/entry/{workflow_id}
  - POST /v1/custom/exit/{workflow_id}

- **Management endpoints**:
  - GET /health
  - GET /v1/gateway/status

- **Features**:
  - Streaming response support
  - Configurable CORS, concurrent requests, timeouts
  - Integration with inference router for intelligent routing
  - Request/response transformation and validation

**NOTES ON 5.3.9**

Added **LoRAX (LoRA eXchange) integration** and **HuggingFace PEFT import** for production-grade multi-LoRA inference serving:

- **LoRAX Service** (`ml_services/lorax_service.py`): Async HTTP client for Predibase LoRAX multi-LoRA inference server that serves thousands of fine-tuned models on a single GPU with dynamic adapter loading, heterogeneous continuous batching, and adapter exchange scheduling.

- **LoRAX CLI commands** (`terradev lora lorax`):
  ```bash
  terradev lora lorax deploy -m mistralai/Mistral-7B-Instruct-v0.1 --docker
  terradev lora lorax test --host localhost --port 8080
  terradev lora lorax list-adapters
  terradev lora lorax load-adapter -a vineetsharma/qlora-adapter-Mistral-7B-Instruct-v0.1-gsm8k
  terradev lora lorax unload-adapter -a my-adapter
  terradev lora lorax generate -p "What is 2+2?" -a my-adapter
  terradev lora lorax sync-registry
  ```

- **PEFT Import Service** (`ml_services/peft_import_service.py`): Download, validate, and prepare LoRA adapters from HuggingFace using the PEFT library with auto-detection of rank, alpha, and target modules.

- **PEFT CLI commands** (`terradev lora peft`):
  ```bash
  terradev lora peft import -a vineetsharma/qlora-adapter-Mistral-7B-Instruct-v0.1-gsm8k
  terradev lora peft import -a username/adapter --local-name my-adapter --register --base-model mistralai/Mistral-7B-Instruct-v0.1
  terradev lora peft list
  terradev lora peft validate -p ~/.terradev/peft_adapters/username--adapter
  terradev lora peft delete -a username/adapter
  ```

- **LoRAX Helm Template** (`clusters/lorax-template/helm/`): Production-ready Kubernetes manifests with GPU resource limits, storage configuration, and Prometheus metrics support.

- **Registry Integration**: One-step import from HuggingFace and automatic registration in Terradev LoRA registry with version tracking, cross-replica sync, and cost attribution.

**NOTES ON 5.3.3**

Added **provider registration and profiling system** for intelligent quirk-aware routing across 23 cloud providers, and registration for custom providers from .yaml import:

- **ProviderProfile schema** (`providers/types.py`): Encodes provider-specific behaviors including API style (REST/GraphQL/JSON:API), authentication type (Bearer/Basic/HMAC/X-Api-Key), rate limits, spot instance support, egress costs, fallback routing, capacity checks, container image pinning, and spot interruption handling.

- **Built-in profiles** (`providers/provider_profiles.py`): Pre-configured profiles for all 23 providers (RunPod, Vast.ai, Lambda Labs, AWS, GCP, Azure, Oracle, Crusoe, CoreWeave, DigitalOcean, Yotta Labs, E2E Networks, FluidStack, Alibaba, OVHcloud, Hetzner, SiliconFlow, TensorDock, Baseten, HuggingFace, Hyperstack, InferX, Latitude).

- **Dynamic registration**: Users can register custom provider profiles programmatically or load from YAML/JSON files for internal clusters or proprietary cloud providers.

- **Profile-aware routing** (`providers/registry.py`): `ProviderRegistry.ranked_providers()` now incorporates provider profiles into scoring, using egress costs, fallback routing preferences, and spot preemption rates for intelligent provider selection.

- **CLI commands** (`terradev providers`): New command group for managing custom provider profiles:
  ```bash
  terradev providers load-profiles ~/.terradev/custom_providers.yaml
  terradev providers list-profiles
  terradev providers show-profile runpod
  terradev providers remove-profile my_custom_provider
  terradev providers export-example -o ~/.terradev/custom_providers.yaml
  ```

**NOTES ON 5.2.1**

Added two new BYOAPI providers: **Yotta Labs (Shakti Cloud)** and **E2E Networks** — India's leading GPU clouds. Yotta Labs uses a pod-based compute model (similar to RunPod), and E2E Networks is a traditional VM-style hyperscaler that is NSE-listed and MeitY empanelled. Both are BYOAPI: your key, stored locally, never touches a Terradev server.

```bash
terradev configure --provider yottalabs
terradev configure --provider e2enetworks
```

**NOTES ON 5.0.0**

We removed the paywall, open-sourced Terradev, and added Rust accelerators for safe and snappy delivery...

With the Rust DAG orchestrator, the execution graph enforces correct sequencing and idempotency at the runtime level. You or the agent can issue commands freely... the orchestrator ensures they're safe to execute.

218 tools not including subcommand/flags require heavy context. The Rust MCP orchestrator processes tool calls with minimal overhead: deserializing, routing, executing, and responding faster than pure-Python-based MCP servers by an order of magnitude. For an agent running a complex provisioning workflow across 23 cloud providers, that compounds across every tool call in the chain.

## BYOAPI Configuration

Your API keys are stored locally at ~/.terradev/credentials.json and never sent to Terradev servers.

```bash
# Configure multiple providers
terradev configure --provider runpod
terradev configure --provider vastai
terradev configure --provider aws
terradev configure --provider gcp
```

## Performance

- **2-8x throughput improvements** with vLLM optimization
- **30-50% bandwidth penalty eliminated** with NUMA topology
- **2-5x CUDA Graph speedup** with optimal topology
- **Up to 90% cost savings** with automatic provider switching
- **<2 minute spot recovery** with KV cache checkpointing
- **up to 3.6x faster cold starts** with weight streaming
- **Up to 50% cost savings** with MLA-aware VRAM estimation
  
## Complete Tutorial

### Step 1: Install Terradev
```bash
pip install terradev-cli
```

For all cloud provider SDKs and ML integrations:
```bash
pip install terradev-cli[all]
```

Verify and list commands:
```bash
terradev --help
```

### Step 2: Configure Your First Cloud Provider
Terradev supports 23 GPU cloud providers. Start with one, RunPod is the fastest to set up:

```bash
terradev setup runpod --quick
```

This shows you where to get your API key. Then configure it:

```bash
terradev configure --provider runpod
```

Paste your API key when prompted. It's stored locally at ~/.terradev/credentials.json, never sent to a Terradev server. Add more providers later:

```bash
terradev configure --provider vastai
terradev configure --provider lambda_labs
terradev configure --provider aws
```

The more providers you configure, the better your price coverage.

### Step 3: Get Real-Time GPU Prices
Check pricing across every provider you've configured:

```bash
terradev quote -g A100
```

Output is a table sorted cheapest-first: price/hour, provider, region, spot vs. on-demand. Try different GPUs:

```bash
terradev quote -g H100
terradev quote -g L40S
ter