feat: ground answers in the docs and let the AI propose console actions

README.md

@@ -64,10 +64,39 @@ The prompt supports:
 - User and AI turns in the transcript.
 - Session history with delete controls.
 - Uploaded file references for future capability-specific parsing.
-- Action preview cards, such as Fleet-Ops create-order previews.
+- Action preview cards, such as Fleet-Ops create-order previews (several per answer).
+- Confirmation cards for console actions the AI offers, such as opening a page or a create dialog.
+- Thumbs up/down feedback on answers.
 
 Closing the prompt only hides it. A chat session continues until the user starts a new chat, ends the current chat, or deletes the session from history.
 
+## How Answers Are Grounded
+
+With OpenAI or Claude enabled, each prompt runs as a tool-calling conversation. The model looks information up before answering instead of relying on memory:
+
+- **Documentation**: `search_docs` and `read_doc` search the official Fleetbase documentation (fleetbase.io/docs). Answers link the pages they used.
+- **Company data**: `count_records`, `group_count`, and `list_records` query the allowlisted resources extensions register, scoped to the organization and the user's permissions. Invalid filters return errors instead of being ignored.
+- **Console actions**: `find_console_commands` and `propose_console_command` offer to open a page or a dialog (for example IAM › Users › Create user). The user sees a confirmation card and nothing runs until they confirm; the server re-checks permissions at that moment.
+- **Extension tools**: modules add their own, such as Fleet-Ops record search and order drafts.
+
+Every tool call is recorded as a task step. Turning off **Look up answers with tools** in the provider settings falls back to the older single-request mode.
+
+### Who sees what
+
+Only users whose `type` is `admin` are system administrators. Everyone else, including organization "Administrator" roles, is answered as an organization user: documentation pages and sections about system setup, service credentials, environment variables, and self-hosting are filtered out on the server, admin console actions are never offered, and the system prompt tells the model to refer those needs to the system administrator.
+
+### Documentation index
+
+Documentation is indexed from the fleetbase.io sitemap into `ai_knowledge_documents` and `ai_knowledge_chunks` (MySQL full-text search), refreshed weekly, and tagged by audience (`server/config/ai.php` → `knowledge.docs`). A snapshot ships in `server/resources/ai-knowledge` and loads automatically when nothing is indexed, so instances without internet access still have documentation.
+
+```bash
+php artisan ai:sync-docs                     # crawl fleetbase.io/docs
+php artisan ai:sync-docs --snapshot          # load the packaged snapshot
+php artisan ai:sync-docs --write-snapshot=server/resources/ai-knowledge/docs-snapshot.json.gz
+```
+
+System administrators can check the index and trigger a sync in **Admin → AI Config → Knowledge Base**.
+
 ## Capability Framework
 
 Fleetbase AI does not give providers arbitrary database access. Modules expose AI functionality explicitly by registering capabilities.
@@ -95,6 +124,40 @@ Create-order actions use a dedicated compact preview component designed for the
 
 Orders are not created until the user confirms the preview.
 
+## Console Commands for Extensions
+
+Extensions register console actions from their service providers. Definitions are plain arrays, so an extension does not need to depend on this package's classes:
+
+```php
+$this->callAfterResolving(\Fleetbase\Ai\Support\Commands\AiCommandRegistry::class, function ($commands) {
+    $commands->registerMany([[
+        'id'          => 'my-extension.widgets.create',
+        'label'       => 'Create widget',
+        'breadcrumb'  => 'My Extension › Widgets',
+        'description' => 'Open the new widget form.',
+        'steps'       => [
+            ['type' => 'navigate', 'route' => 'console.my-extension.widgets.index'],
+            ['type' => 'service', 'engine' => '@vendor/my-extension-engine', 'service' => 'widget-actions', 'method' => 'modal.create'],
+        ],
+        'permissions' => ['my-extension create widget'],
+    ]]);
+});
+```
+
+`service` steps call a method on the engine's resource-action service, so the same dialog opens from the AI prompt as from the extension's own pages. Verify routes and service methods resolve with:
+
+```bash
+node scripts/verify-ai-commands.mjs --console ../../console --packages ..
+```
+
+## Logs, Feedback, and Evaluation
+
+- Users rate answers with thumbs up or down. Ratings are filterable in **Admin → AI Config → Task & Chat Logs**, alongside degraded answers (a capability failed) and cut-off answers.
+- The log view shows each conversation in full: every prompt and answer, and, per answer, every step including each tool call's arguments and results and the exact system prompt and messages sent to the model. It requires the `ai view audit logs` permission.
+- Export logs from the admin view, or with `php artisan ai:export-logs --from=2026-09-01 --format=jsonl`. Exports use the same filters as the log view, contain user content, require `ai view audit logs`, and are recorded in the access log.
+- `php artisan ai:eval` runs the golden cases in `server/resources/ai-eval/cases.json` (drawn from real conversations) against the configured provider and reports a pass rate. It calls the provider, so it asks for confirmation; use `--model` to compare models.
+- `php artisan ai:replay <task-uuid>` re-runs a recorded turn through the current runtime and prints both answers.
+
 ## Development Checks
 
 Frontend checks:
@@ -108,8 +171,8 @@ Frontend checks:
 Backend checks:
 
 ```bash
-php -l server/src/routes.php
-php -l server/src/Http/Controllers/Internal/AiSessionController.php
+composer test:unit
+composer test:lint
 ```
 
 The local template-lint configuration may print `Invalid rule configuration found: no-down-event-binding` while still exiting successfully.

addon/components/admin/ai-admin/filter-bar.hbs

@@ -0,0 +1,46 @@
+<div class="fleetbase-ai-admin-toolbar" ...attributes>
+    <div class="fleetbase-ai-admin-toolbar-row">
+        {{yield to="before"}}
+        {{#each @primary as |field|}}
+            <Admin::AiAdmin::FilterField
+                @field={{field}}
+                @filters={{@filters}}
+                @metadata={{@metadata}}
+                @companySource={{@companySource}}
+                @userSource={{@userSource}}
+                @searchPlaceholder={{@searchPlaceholder}}
+                @onChange={{@onChange}}
+            />
+        {{/each}}
+        {{#if this.secondary.length}}
+            <Button
+                @icon="sliders"
+                @text={{if this.secondaryActiveCount (concat "More filters (" this.secondaryActiveCount ")") "More filters"}}
+                @type={{if this.showMore "primary" "default"}}
+                aria-expanded={{if this.showMore "true" "false"}}
+                data-test-more-filters
+                @onClick={{this.toggleMore}}
+            />
+        {{/if}}
+        {{#if this.canClear}}
+            <Button @type="link" @text="Clear" data-test-clear-filters @onClick={{this.clear}} />
+        {{/if}}
+        <div class="fleetbase-ai-admin-toolbar-actions">
+            {{yield to="actions"}}
+        </div>
+    </div>
+    {{#if this.showMore}}
+        <div class="fleetbase-ai-admin-toolbar-row fleetbase-ai-admin-toolbar-secondary">
+            {{#each this.secondary as |field|}}
+                <Admin::AiAdmin::FilterField
+                    @field={{field}}
+                    @filters={{@filters}}
+                    @metadata={{@metadata}}
+                    @companySource={{@companySource}}
+                    @userSource={{@userSource}}
+                    @onChange={{@onChange}}
+                />
+            {{/each}}
+        </div>
+    {{/if}}
+</div>

addon/components/admin/ai-admin/filter-bar.js

@@ -0,0 +1,33 @@
+import Component from '@glimmer/component';
+import { tracked } from '@glimmer/tracking';
+import { action } from '@ember/object';
+
+/**
+ * The compact toolbar at the top of the AI admin views. `@primary` fields sit in the toolbar itself,
+ * `@secondary` fields in a second row opened with "More filters", whose count shows how many of them
+ * are set. Changes are reported through `@onChange(field)`; the view applies them.
+ */
+export default class AdminAiAdminFilterBarComponent extends Component {
+    @tracked showMore = false;
+
+    get secondary() {
+        return this.args.secondary ?? [];
+    }
+
+    get secondaryActiveCount() {
+        return this.args.filters.activeCount(this.secondary.map((field) => (field === 'review' ? ['degraded', 'truncated'] : field === 'session_status' ? 'status' : field)).flat());
+    }
+
+    get canClear() {
+        return this.args.filters.hasAny;
+    }
+
+    @action toggleMore() {
+        this.showMore = !this.showMore;
+    }
+
+    @action clear() {
+        this.args.filters.clear();
+        this.args.onChange?.('clear');
+    }
+}

addon/components/admin/ai-admin/filter-field.hbs

@@ -0,0 +1,81 @@
+{{#if (eq @field "search")}}
+    <div class="fleetbase-ai-admin-toolbar-search">
+        <FaIcon @icon="magnifying-glass" class="fleetbase-ai-admin-toolbar-search-icon" />
+        <input
+            type="search"
+            class="form-input form-input-sm w-full"
+            value={{@filters.search}}
+            placeholder={{or @searchPlaceholder "Search"}}
+            aria-label={{or @searchPlaceholder "Search"}}
+            {{on "input" (fn this.setFromInput "search")}}
+        />
+    </div>
+{{else if (eq @field "session_status")}}
+    <Select class="fleetbase-ai-admin-toolbar-select form-input-sm" aria-label="Session status" @value={{@filters.status}} @options={{this.sessionStatusOptions}} @optionLabel="label" @optionValue="value" @onSelect={{fn this.set "status"}} />
+{{else if (eq @field "task_status")}}
+    <Select class="fleetbase-ai-admin-toolbar-select form-input-sm" aria-label="Answer status" @value={{@filters.task_status}} @options={{this.taskStatusOptions}} @optionLabel="label" @optionValue="value" @onSelect={{fn this.set "task_status"}} />
+{{else if (eq @field "feedback")}}
+    <Select class="fleetbase-ai-admin-toolbar-select form-input-sm" aria-label="Feedback" @value={{@filters.feedback}} @options={{this.feedbackOptions}} @optionLabel="label" @optionValue="value" @onSelect={{fn this.set "feedback"}} />
+{{else if (eq @field "review")}}
+    <div class="fleetbase-ai-admin-toolbar-group" role="group" aria-label="Needs review">
+        <Button
+            @type={{if @filters.degraded "primary" "default"}}
+            @icon="triangle-exclamation"
+            @text="Degraded"
+            @helpText="Answers given while a capability had failed"
+            aria-pressed={{if @filters.degraded "true" "false"}}
+            data-test-filter-degraded
+            @onClick={{fn this.toggle "degraded"}}
+        />
+        <Button
+            @type={{if @filters.truncated "primary" "default"}}
+            @icon="scissors"
+            @text="Cut off"
+            @helpText="Answers cut off at the output limit"
+            aria-pressed={{if @filters.truncated "true" "false"}}
+            data-test-filter-truncated
+            @onClick={{fn this.toggle "truncated"}}
+        />
+    </div>
+{{else if (eq @field "provider")}}
+    <Select class="fleetbase-ai-admin-toolbar-select form-input-sm" aria-label="Provider" @value={{@filters.provider}} @options={{this.providerOptions}} @optionLabel="label" @optionValue="value" @onSelect={{this.setProvider}} />
+{{else if (eq @field "model")}}
+    <Select class="fleetbase-ai-admin-toolbar-select form-input-sm" aria-label="Model" @value={{@filters.model}} @options={{this.modelOptions}} @optionLabel="label" @optionValue="value" @onSelect={{fn this.set "model"}} />
+{{else if (eq @field "company")}}
+    <div class="fleetbase-ai-admin-toolbar-model">
+        <ModelSelect
+            @modelName="company"
+            @source={{@companySource}}
+            @selectedModel={{@filters.selectedCompany}}
+            @placeholder="Any organization"
+            @triggerClass="form-select form-input form-input-sm"
+            @allowClear={{true}}
+            @renderInPlace={{true}}
+            @onChange={{this.setCompany}}
+            as |company|
+        >
+            {{or company.name company.public_id company.uuid}}
+        </ModelSelect>
+    </div>
+{{else if (eq @field "user")}}
+    <div class="fleetbase-ai-admin-toolbar-model">
+        <ModelSelect
+            @modelName="user"
+            @source={{@userSource}}
+            @query={{@filters.userQuery}}
+            @selectedModel={{@filters.selectedUser}}
+            @placeholder="Any user"
+            @triggerClass="form-select form-input form-input-sm"
+            @allowClear={{true}}
+            @renderInPlace={{true}}
+            @onChange={{this.setUser}}
+            as |user|
+        >
+            {{or user.name user.email user.public_id user.uuid}}
+        </ModelSelect>
+    </div>
+{{else if (eq @field "date")}}
+    <div class="fleetbase-ai-admin-toolbar-date">
+        <DatePicker @value={{@filters.dateRange}} @range={{true}} @onSelect={{this.setDateRange}} @autoClose={{true}} @buttons={{array "clear"}} @placeholder="Any date" class="form-input form-input-sm w-full" />
+    </div>
+{{/if}}

addon/components/admin/ai-admin/filter-field.js

@@ -0,0 +1,59 @@
+import Component from '@glimmer/component';
+import { action } from '@ember/object';
+import { FEEDBACK_OPTIONS, SESSION_STATUS_OPTIONS, TASK_STATUS_OPTIONS } from '../../../utils/ai-admin-filters';
+
+/**
+ * One filter control in the AI admin toolbar, chosen by `@field`. Every change updates `@filters` and
+ * then calls `@onChange(field)`, so the view decides when to reload.
+ */
+export default class AdminAiAdminFilterFieldComponent extends Component {
+    sessionStatusOptions = SESSION_STATUS_OPTIONS;
+    taskStatusOptions = TASK_STATUS_OPTIONS;
+    feedbackOptions = FEEDBACK_OPTIONS;
+
+    get providerOptions() {
+        return this.args.filters.providerOptions(this.args.metadata);
+    }
+
+    get modelOptions() {
+        return this.args.filters.modelOptions(this.args.metadata);
+    }
+
+    changed(field) {
+        this.args.onChange?.(field);
+    }
+
+    @action set(field, value) {
+        this.args.filters.set(field, value);
+        this.changed(field);
+    }
+
+    @action setFromInput(field, event) {
+        this.set(field, event.target.value);
+    }
+
+    @action toggle(field) {
+        this.args.filters.toggle(field);
+        this.changed(field);
+    }
+
+    @action setProvider(value) {
+        this.args.filters.setProvider(value);
+        this.changed('provider');
+    }
+
+    @action setCompany(company) {
+        this.args.filters.setCompany(company);
+        this.changed('company');
+    }
+
+    @action setUser(user) {
+        this.args.filters.setUser(user);
+        this.changed('user');
+    }
+
+    @action setDateRange(selection) {
+        this.args.filters.setDateRange(selection);
+        this.changed('date');
+    }
+}