Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1334,4 +1334,4 @@ firecrawl alexandria feedback --rating partial \
--rationale "Found summaries but could not retrieve attachments" --json
```

The optional `--objective` is the underlying goal of the session: what you or your user were ultimately trying to accomplish, beyond the single website. No job ID is required. It must be sent within 20 minutes of the team's most recent Alexandria search, discovery, or execution; later submissions are rejected with `FEEDBACK_WINDOW_EXPIRED`. Each submission refunds 1 credit, up to 10 credits per website and 100 credits per team each UTC day; past either cap, submissions are still recorded and return `websiteCapReached` or `dailyCapReached`. Optional `--provider-feedback` and `--capability-feedback` accept JSON arrays; see `firecrawl alexandria feedback --help` for their fields and issue codes. Capability issue codes are `new_capability_request` (requires `requestedFunctionality`), `missing_capability` (the provider exists but lacks this capability), `insufficient_functionality`, `incorrect_result`, `execution_error`, and `other`. Each eligible provider call prints a `Feedback ref` (for example `c3`) so ratings are attributed to the calls they are about. The CLI keeps up to 20 call tokens for 24 hours in its config directory and sends them with the session rating, then clears them after a successful submission. Rate individual calls with a repeatable `--call-rating c3=good`, or pass tokens yourself with a repeatable `--token <token>` (these replace the stored ones). Existing `feedback` and `search-feedback` commands retain their job-specific behavior. Endpoint feedback opt-out environment variables also apply to this command.
The optional `--objective` is the underlying goal of the session: what you or your user were ultimately trying to accomplish, beyond the single website. No job ID is required. It must be sent within 20 minutes of the team's most recent Alexandria search, discovery, or execution; later submissions are rejected with `FEEDBACK_WINDOW_EXPIRED`. A submission refunds 1 credit, once per call and website, up to 10 credits per website and 100 credits per team each UTC day. A repeat about the same calls and website, or a submission past either cap (`websiteCapReached`, `dailyCapReached`), is still recorded and refunds nothing. Optional `--provider-feedback` and `--capability-feedback` accept JSON arrays; see `firecrawl alexandria feedback --help` for their fields and issue codes. Capability issue codes are `new_capability_request` (requires `requestedFunctionality`), `missing_capability` (the provider exists but lacks this capability), `insufficient_functionality`, `incorrect_result`, `execution_error`, and `other`. Each eligible provider call prints a `Feedback ref` (for example `c3`) so ratings are attributed to the calls they are about. The CLI keeps up to 20 call tokens for 24 hours in its config directory and sends them with the session rating, then clears them after a successful submission. Rate individual calls with a repeatable `--call-rating c3=good`, or pass tokens yourself with a repeatable `--token <token>` (these replace the stored ones). Existing `feedback` and `search-feedback` commands retain their job-specific behavior. Endpoint feedback opt-out environment variables also apply to this command.
2 changes: 1 addition & 1 deletion skills/firecrawl-agent/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,4 +70,4 @@ firecrawl agent "<job-id>" --cancel

## Alexandria session feedback

To report an Alexandria session outcome or a provider/capability gap, use `firecrawl alexandria feedback --rating good|partial|bad --url <website> --requested-functionality '<what was needed>' --objective '<the underlying goal of the task>' --rationale '<what happened>' --json`. Use observed results in the rationale. No job ID is needed; send it within 20 minutes of your last Alexandria search, discovery, or execution, or it is rejected. Each submission refunds 1 credit, up to 10 per website and 100 per team each UTC day. Optional `--provider-feedback` and `--capability-feedback` JSON arrays describe specific gaps; inspect `firecrawl alexandria feedback --help` for their fields. Use the capability issue `missing_capability` when a provider exists but lacks the needed capability, and `new_capability_request` (with `requestedFunctionality`) to ask for one.
To report an Alexandria session outcome or a provider/capability gap, use `firecrawl alexandria feedback --rating good|partial|bad --url <website> --requested-functionality '<what was needed>' --objective '<the underlying goal of the task>' --rationale '<what happened>' --json`. Use observed results in the rationale. No job ID is needed; send it within 20 minutes of your last Alexandria search, discovery, or execution, or it is rejected. A submission refunds 1 credit, once per call and website, up to 10 per website and 100 per team each UTC day. Optional `--provider-feedback` and `--capability-feedback` JSON arrays describe specific gaps; inspect `firecrawl alexandria feedback --help` for their fields. Use the capability issue `missing_capability` when a provider exists but lacks the needed capability, and `new_capability_request` (with `requestedFunctionality`) to ask for one.
3 changes: 2 additions & 1 deletion skills/firecrawl-alexandria/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Use ordinary web results when they answer the question; use a provider tool when

## Alexandria feedback (refunds 1 credit)

Alexandria coverage grows from what agents report. Send one `firecrawl alexandria feedback` per website you needed data from, right after your last Alexandria call for it. No job ID is needed. Each feedback refunds 1 credit, up to 10 credits per website and 100 per team each UTC day.
Alexandria coverage grows from what agents report. Send one `firecrawl alexandria feedback` per website you needed data from, right after your last Alexandria call for it. No job ID is needed. Feedback refunds 1 credit, once per call and website, up to 10 credits per website and 100 per team each UTC day.

Feedback can describe any of these outcomes:

Expand All @@ -33,6 +33,7 @@ Feedback can describe any of these outcomes:
- **`--provider-feedback`** is a JSON array of `{name, issue, why}` for providers that were missing, thin, or unavailable. Issues: `missing_provider` (no provider covers the site), `insufficient_coverage` (exists, but data was thin, stale, or partial for this market or segment), `provider_unavailable` (could not be called), `other`.
- **`--capability-feedback`** is a JSON array of `{name, provider, issue, why, requestedFunctionality?}` for capabilities that were missing, wrong, or failed. Issues: `new_capability_request` (ask the provider to add one; `requestedFunctionality` required), `missing_capability` (provider exists but lacks it), `insufficient_functionality` (exists but cannot take the input or filter you needed), `incorrect_result`, `execution_error`, `other`. Use `name` and `provider` exactly as discovery returned them; for a capability that does not exist yet, name what it should be.
- **Rate honestly:** `good` when a tool answered the need, `partial` when it answered some of it or with gaps, `bad` when nothing available answered it or what ran was wrong or failed. Every rating gets the same refund.
- **One refund per call and website.** Feedback about a website refunds once for the Alexandria calls it covers. Sending it again about the same calls and website is still recorded but refunds nothing.
- **Website refund cap (per website, per UTC day, default 10 credits).** Past it, feedback about that website is still recorded but refunds nothing, and the response sets `websiteCapReached: true`. Feedback about other websites still refunds.
- **Daily refund cap (per team, per UTC day, default 100 credits).** Past the cap, feedback is still recorded but refunds nothing. The response includes `creditsRefundedToday`, `dailyRefundCap`, and `dailyCapReached`. **When `dailyCapReached: true`, stop sending Alexandria feedback for the rest of the UTC day.**
- **`--silent &`** is the right pattern: exit code 0 even on failure, so a rejected call never crashes your pipeline.
Expand Down
2 changes: 1 addition & 1 deletion skills/firecrawl-search/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ Use `list` for category/provider browsing and selected contracts. For a known we

If no returned tool covers the country/market/segment or required inputs, continue with ordinary web results. Do not exhaust the catalogue or pay for adjacent tools just to probe coverage. With an API key, `--no-domain-tools` turns off domain matches only and semantic tool matches stay on; keyless searches return web results only.

For Alexandria feedback about a provider result or coverage gap (each refunds 1 credit, up to 10 per website and 100 per team each UTC day), see [firecrawl-alexandria](../firecrawl-alexandria/SKILL.md).
For Alexandria feedback about a provider result or coverage gap (refunds 1 credit, once per call and website, up to 10 per website and 100 per team each UTC day), see [firecrawl-alexandria](../firecrawl-alexandria/SKILL.md).

## Progressive discovery and output handling

Expand Down
2 changes: 1 addition & 1 deletion skills/firecrawl/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ Single format outputs raw content. Multiple formats (e.g., `--format markdown,li

After using search results, send `firecrawl search-feedback` (the first feedback per search refunds 1 credit). The full pattern, guard, and rules live in [firecrawl-search](../firecrawl-search/SKILL.md).

For Alexandria feedback about a provider result or coverage gap (each refunds 1 credit, up to 10 per website and 100 per team each UTC day), see [firecrawl-alexandria](../firecrawl-alexandria/SKILL.md).
For Alexandria feedback about a provider result or coverage gap (refunds 1 credit, once per call and website, up to 10 per website and 100 per team each UTC day), see [firecrawl-alexandria](../firecrawl-alexandria/SKILL.md).

For non-search endpoint jobs, use `firecrawl feedback <endpoint> <jobId>` to send concise job-level feedback through `/v2/feedback`. Supported endpoints are `search`, `scrape`, `parse`, and `map`.

Expand Down
32 changes: 32 additions & 0 deletions src/__tests__/alexandria-beta.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -579,6 +579,38 @@ it('sends provider calls to Scrape with a stable retry ID and preserves the rece
});
});

it('prints the next step the API attaches to a provider answer', async () => {
const nextStep = {
instruction:
'Next: once you have used this result, run this command to rate it. Eligible feedback can refund 1 credit.',
command:
'firecrawl alexandria feedback --rating "<good|partial|bad>" --url "<website the data is about>" --requested-functionality "<what you needed from it>" --objective "<the underlying goal>" --rationale "<what was right, wrong or missing>"',
};
response.data.alexandria = [
{
provider: 'fred',
capability: 'series/observations',
data: { value: 42 },
creditsCost: 1,
nextStep,
},
];
const result = await cli([
'scrape',
'--alexandria',
'fred/series/observations',
'--json',
]);
expect(result.code).toBe(0);
expect(JSON.parse(result.stdout).data.alexandria[0]).toEqual({
provider: 'fred',
capability: 'series/observations',
data: { value: 42 },
nextStep,
});
expect(requests[0].body).toMatchObject({ integration: 'cli' });
});

it('relays terms refusals and keeps the request ID on failure', async () => {
status = 403;
response = {
Expand Down
2 changes: 1 addition & 1 deletion src/commands/alexandria-feedback.ts
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ export function parseAlexandriaFeedbackArray(
export function createAlexandriaFeedbackCommand(): Command {
return new Command('feedback')
.description(
'Report Alexandria session results, provider gaps, or capability issues. No job ID needed; send within 20 minutes of your last Alexandria search, discovery, or execution. Refunds 1 credit per submission, up to 10 per website and 100 per team each UTC day.'
'Report Alexandria session results, provider gaps, or capability issues. No job ID needed; send within 20 minutes of your last Alexandria search, discovery, or execution. Refunds 1 credit, once per call and website, up to 10 per website and 100 per team each UTC day.'
)
.requiredOption(
'--rating <rating>',
Expand Down
Loading