> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gmicloud.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Inference Overview

> Serverless and Dedicated endpoints for running ML models in production on GMI Cloud.

GMI Inference runs production ML models behind two endpoint types: **Serverless** for instant access to managed models, and **Dedicated** for fully customized, isolated deployments.

<CardGroup cols={2}>
  <Card title="Serverless Endpoints" icon="bolt">
    Pre-configured, OpenAI-compatible APIs. No infrastructure to manage. Pay per token. Best for prototyping and variable workloads.
  </Card>

  <Card title="Dedicated Endpoints" icon="server">
    Your own models on dedicated GPUs. Full control over hardware, scaling, and isolation. No rate limits. Best for steady or sensitive production traffic.
  </Card>
</CardGroup>

## Serverless Endpoints

Instant access to popular models through OpenAI-compatible APIs.

* **Zero setup.** Models are ready behind a single API key.
* **Autoscaling.** Capacity grows and shrinks with demand.
* **Per-token billing.** No idle compute charges.

Good for prototypes, small apps, and any workload where you'd rather not run infrastructure.

## Dedicated Endpoints

Provision your own endpoint on dedicated GPUs.

* **Bring your own model.** Deploy fine-tuned or proprietary weights.
* **Predictable performance.** Reserved GPU resources, consistent latency.
* **Isolated.** Private network, separate from other tenants.
* **No rate limits.** Cap is set by the hardware you provision.

Good for enterprise production, latency-sensitive applications, or large continuous workloads.

## Inference in the console

A tour of the screens you'll use inside the [GMI Cloud Console](https://console.gmicloud.ai).

### Dashboard

Landing screen for the Inference tab. Recent activity, usage trends, and shortcuts to your most-used resources.

<Frame>
  <img src="https://mintcdn.com/gmicloud/wwBqG4D8qH5TFQuz/images/inference-dashboard-.png?fit=max&auto=format&n=wwBqG4D8qH5TFQuz&q=85&s=ce703fd37c316155ac72906219f862e6" alt="Inference dashboard" width="1733" height="1011" data-path="images/inference-dashboard-.png" />
</Frame>

### Model Hub

Browse the full catalog of available models, filter by modality (text, image, video, audio), and open a model card for API examples and parameters.

<Frame>
  <img src="https://mintcdn.com/gmicloud/wwBqG4D8qH5TFQuz/images/inference-model-hub.png?fit=max&auto=format&n=wwBqG4D8qH5TFQuz&q=85&s=a567708796d9eaa4a16f88bf98800131" alt="Model Hub" width="3024" height="1477" data-path="images/inference-model-hub.png" />
</Frame>

### Playground

Try any serverless model in the browser. Useful for prompt testing and parameter exploration before integrating via API.

<Frame>
  <img src="https://mintcdn.com/gmicloud/wwBqG4D8qH5TFQuz/images/inference-playground.png?fit=max&auto=format&n=wwBqG4D8qH5TFQuz&q=85&s=5271aa1b3ff261722868afa33c0f4736" alt="Playground" width="3024" height="1484" data-path="images/inference-playground.png" />
</Frame>

### My Models

Your uploaded or fine-tuned models. Manage versions and visibility from here.

<Frame>
  <img src="https://mintcdn.com/gmicloud/wwBqG4D8qH5TFQuz/images/inference-my-models.png?fit=max&auto=format&n=wwBqG4D8qH5TFQuz&q=85&s=69e3bbac8a6bb468ec559d2deeb8bf12" alt="My Models" width="3026" height="1482" data-path="images/inference-my-models.png" />
</Frame>

### Deployments

Manage Dedicated Endpoints: scale settings, model versions, and traffic routing.

<Frame>
  <img src="https://mintcdn.com/gmicloud/wwBqG4D8qH5TFQuz/images/inference-deployments.png?fit=max&auto=format&n=wwBqG4D8qH5TFQuz&q=85&s=60dcc2c8e98b4fa5444f17d0e530d4cd" alt="Deployments" width="3024" height="1482" data-path="images/inference-deployments.png" />
</Frame>

### Storage

Inference Storage holds inputs, outputs, and other artifacts referenced by your endpoints.

<Frame>
  <img src="https://mintcdn.com/gmicloud/wwBqG4D8qH5TFQuz/images/inference-storage.png?fit=max&auto=format&n=wwBqG4D8qH5TFQuz&q=85&s=ad4efecbcbc119c0e11890e89f9d6ad1" alt="Inference Storage" width="3024" height="1479" data-path="images/inference-storage.png" />
</Frame>

<Note>
  Workflows, Team Space, and generated media are part of GMI Studio. See [Managing Workflows](/gmi-studio/gmi-studio-user-manual/managing-workflows) under the GMI Studio tab.
</Note>