diff --git a/src/content/docs/getting-started/quickstart.mdx b/src/content/docs/getting-started/quickstart.mdx index 3fd3320..2ac21db 100644 --- a/src/content/docs/getting-started/quickstart.mdx +++ b/src/content/docs/getting-started/quickstart.mdx @@ -78,7 +78,35 @@ Run your first evaluation with EvalHub. -3. **Submit an Evaluation** +3. **Connect to EvalHub** + + + + + Point the CLI at your EvalHub route or service URL: + + ```bash + evalhub config set base_url https://evalhub.apps.my-cluster.example.com + ``` + + + + + Start the server with a config file, then point the CLI at it: + + ```bash + evalhub config set server_config_file server-config.yaml + evalhub server start + evalhub config set base_url http://localhost:8080 + ``` + + See [Local Mode](/guides/local-mode/) for full local setup including + configuration, MLflow, and provider registration. + + + + +4. **Submit an Evaluation** @@ -156,7 +184,7 @@ Run your first evaluation with EvalHub. -4. **Wait for Results** +5. **Wait for Results** diff --git a/src/content/docs/home.mdx b/src/content/docs/home.mdx index 2045b5e..13c0911 100644 --- a/src/content/docs/home.mdx +++ b/src/content/docs/home.mdx @@ -2,7 +2,7 @@ title: "EvalHub" --- -import { Card, CardGrid, LinkCard } from '@astrojs/starlight/components'; +import { Card, CardGrid, LinkCard, Steps } from '@astrojs/starlight/components'; Open source evaluation orchestration platform for Large Language Models. @@ -62,17 +62,54 @@ Community providers with a `provider.yaml` are listed in the [Provider Catalog]( ## Quick Taste -```bash -pip install "eval-hub-sdk[cli]" - -evalhub eval run \ - --name my-first-eval \ - --model-url http://localhost:11434/v1 \ - --model-name qwen2.5:1.5b \ - --provider lm_evaluation_harness \ - --benchmark mmlu \ - --wait -``` + + +1. **Install the SDK** (includes the server and CLI): + + ```bash + pip install "eval-hub-sdk[server,cli]" + ``` + +2. **Start a model server** (e.g. [Ollama](https://ollama.com)): + + ```bash + ollama run qwen2.5:1.5b + ``` + + Keep Ollama running in this terminal and execute Steps 3–4 in a separate terminal. + +3. **Start EvalHub locally**: + + ```bash + evalhub server start + ``` + + Point the CLI at the local server: + + ```bash + evalhub config set base_url http://localhost:8080 + ``` + + For provider registration and other advanced options (e.g. `server_config_file`), + see [Local Mode](/guides/local-mode/). + +4. **Run an evaluation**: + + ```bash + evalhub eval run \ + --name my-first-eval \ + --model-url http://localhost:11434/v1 \ + --model-name qwen2.5:1.5b \ + --provider lm_evaluation_harness \ + --benchmark mmlu \ + --wait + ``` + + + +See the [Quick Start](/getting-started/quickstart/) for a full walkthrough or +[Local Mode](/guides/local-mode/) for advanced local configuration (MLflow, +custom server config, provider registration). ## Get Started