Quickstart

You can run Inferia LLM using our Python package (recommended) or via Docker.

1. Using Python Package

The easiest way to get started is by installing the inferiallm package.

Prerequisites

• Python 3.10+
• PostgreSQL 14+
• Redis 6+

Installation

1. Install the package:

   python3 -m venv .venv
   source .venv/bin/activate
   pip install inferiallm

2. Configure Environment:

   curl -o .env https://raw.githubusercontent.com/InferiaAI/InferiaLLM/main/.env.sample

   Edit the .env file to set your database credentials and secrets.

   Critical Variables:

   Variable	Description
   DATABASE_URL	PostgreSQL connection string (e.g., postgresql://inferia:inferia@localhost:5432/inferia)
   REDIS_HOST	Redis host (default: localhost)
   REDIS_PORT	Redis port (default: 6379)
   JWT_SECRET_KEY	Secret for signing access tokens (generate a long random string)
   INTERNAL_API_KEY	Secret for service-to-service authentication
   SECRET_ENCRYPTION_KEY	32-byte base64 key for encrypting provider credentials
   SUPERADMIN_EMAIL	Email for the initial admin account
   SUPERADMIN_PASSWORD	Password for the initial admin account

3. Initialize the Database: This creates the necessary tables, roles, and initial admin user.

   inferiallm init

4. Start Services: Run all gateways and the dashboard in a single command.

   inferiallm start

   Access Points:

   Service	URL
   Dashboard	http://localhost:3001
   Orchestration API	http://localhost:8080
   Filtration Gateway	http://localhost:8000
   Inference Gateway	http://localhost:8001

2. Using Docker Compose

If you prefer containerization, use the deploy directory.

Prerequisites

• Docker & Docker Compose

Steps

1. Pull Docker Image:

   docker pull inferiaai/inferiallm:latest

2. Configure Environment: Edit .env with your secrets. The Docker Compose file will use these variables.

   curl -L https://raw.githubusercontent.com/InferiaAI/InferiaLLM/main/.env.sample -o .env
   nano .env

   [!IMPORTANT] At minimum, set JWT_SECRET_KEY, INTERNAL_API_KEY, and SECRET_ENCRYPTION_KEY.

3. Start Services:

   docker run -d \
     --name inferia-app \
     --env-file .env \
     -p 8000:8000 -p 8001:8001 -p 8080:8080 -p 3000:3000 -p 3001:3001 \
     inferiaai/inferiallm:latest

4. Access Dashboard: Open http://localhost:3001.

   Default credentials: admin@example.com / admin123

Docker Services

The unified Docker image starts these services: