Integrations & Adapters

InferiaLLM uses a modular adapter architecture that allows you to integrate with various external services. This page documents all currently supported integrations.

Compute Providers

Compute providers supply the GPU resources for running LLM inference. InferiaLLM supports decentralized (DePIN) providers.

Nosana

Type: DePIN (Decentralized Physical Infrastructure Network)

Nosana provides decentralized GPU compute on the Solana blockchain. Ideal for cost-effective inference at scale.

Configuration:

1. Navigate to Dashboard > Settings > Providers > DePIN > Nosana
2. Enter your Solana Wallet Private Key (Base58 encoded)
3. Save the configuration

The DePIN sidecar automatically polls for configuration updates every 10 seconds.

Supported Engines:

How it Works:

1. Configure wallet in Dashboard
2. Create a compute pool with provider type "Nosana"
3. Select GPU market (e.g., nosana-rtx4090)
4. Deploy models - the adapter handles job creation, monitoring, and auto-extension

Akash Network

Type: DePIN (Decentralized Cloud)

Akash is a decentralized cloud marketplace on the Cosmos blockchain.

Configuration:

1. Navigate to Dashboard > Settings > Providers > DePIN > Akash
2. Enter your 24-word BIP39 Mnemonic for your Akash wallet
3. Save the configuration

Supported Workloads:

How it Works:

1. Configure mnemonic in Dashboard
2. Create a compute pool with provider type "Akash"
3. Deploy models - the adapter generates SDL manifests, handles bidding, and creates leases

Vector Databases

Vector databases store embeddings for Retrieval Augmented Generation (RAG) workflows.

ChromaDB

Type: Open-source vector database

ChromaDB is the default vector database for InferiaLLM's knowledge base feature.

Configuration:

Navigate to Dashboard > Settings > Providers > Vector Database > Chroma

Mode A: Local/Self-Hosted

If no URL is provided, InferiaLLM uses an embedded persistent client at ./chroma_db.

Mode B: Chroma Cloud

Organization Isolation:

Collections are automatically scoped by organization using the prefix: org_{org_id}_{collection_name}

Guardrail Providers

Guardrail providers perform safety checks on inputs and outputs to prevent harmful content, prompt injection, and PII leakage.

LLM Guard (Local)

Type: Local/self-hosted guardrails

LLM Guard provides local guardrail capabilities without external API calls. This is the default provider.

Configuration:

Set via environment variables:

Llama Guard (via Groq)

Type: LLM-based content moderation

Llama Guard is Meta's safety-tuned model for content classification. InferiaLLM accesses it via Groq's ultra-fast inference API.

Configuration (Option A - Dashboard):

1. Navigate to Dashboard > Settings > Guardrails
2. Enable "Llama Guard" provider
3. Enter your Groq API key
4. Save configuration

Configuration (Option B - Environment Variable):

GUARDRAIL_GROQ_API_KEY=your-groq-api-key
GUARDRAIL_DEFAULT_GUARDRAIL_ENGINE=llama-guard

Dashboard configuration takes precedence if both are set.

Supported Categories:

• Violence and threats
• Sexual content
• Hate speech
• Self-harm
• Criminal activity
• Dangerous instructions

Lakera Guard

Type: Specialized prompt security

Lakera Guard specializes in detecting prompt injection attacks and jailbreak attempts.

Configuration (Option A - Dashboard):

1. Navigate to Dashboard > Settings > Guardrails
2. Enable "Lakera" provider
3. Enter your Lakera API key
4. Save configuration

Configuration (Option B - Environment Variable):

GUARDRAIL_LAKERA_API_KEY=your-lakera-api-key

Dashboard configuration takes precedence if both are set.

Detection Capabilities:

• Direct prompt injection
• Indirect prompt injection
• Jailbreak attempts
• PII extraction attempts
• System prompt leakage

Adding New Integrations

InferiaLLM's adapter architecture makes it easy to add new providers.

Compute Provider Adapters

See Extending Orchestration for implementing custom compute adapters.

Guardrail Providers

See Guardrails Configuration for implementing custom guardrail providers.

Integration Status Summary