SCR-395: plugin distribution metadata, Cursor/Codex manifests and install-path fixes

.agents/skills/scrapingbee-cli/SKILL.md

@@ -39,7 +39,9 @@ schedule a recurring job, or keep a large page out of your context window.
 
 Reach for the **MCP** in two cases: the host already has it connected *and* the task is a single
 page or search whose result belongs in the conversation anyway; or you cannot run a CLI at all —
-no shell, no filesystem, or installation is blocked.
+no shell, no filesystem, installation is blocked, or the shell cannot reach
+`app.scrapingbee.com` (a "Temporary failure in name resolution" from the CLI means a network
+allowlist is in the way — the MCP connector is not subject to it).
 
 MCP endpoint: `https://mcp.scrapingbee.com/mcp` — remote, Streamable HTTP, authenticated with an
 `Authorization: Bearer <SCRAPINGBEE_API_KEY>` header. Host-specific connection steps belong to the
@@ -81,7 +83,11 @@ for.
 
 1. **Install:** `uv tool install scrapingbee-cli` (recommended) or `pip install scrapingbee-cli`.
    Every command including `crawl` works immediately — no extras.
-2. **Authenticate:** `scrapingbee auth`, or set `SCRAPINGBEE_API_KEY`.
+   If `scrapingbee` is then "command not found", the executable is in `~/.local/bin`, which
+   minimal shells don't search: run `export PATH="$HOME/.local/bin:$PATH"` once, or call
+   `~/.local/bin/scrapingbee` by full path. Do not fall back to another tool because of this.
+2. **Authenticate:** `scrapingbee auth`, or set `SCRAPINGBEE_API_KEY` (a `.env` file in the
+   working directory is read too).
 3. **Verify:** `scrapingbee usage` — confirms the key works and shows remaining credits.
 
 ## Commands

.agents/skills/scrapingbee-cli/rules/install.md

@@ -72,6 +72,16 @@ scrapingbee logout
 If `scrapingbee` is not found after install:
 
 1. Activate the environment where `pip install scrapingbee-cli` was run (e.g. `source .venv/bin/activate`). Pip puts the `scrapingbee` script in that env’s bin (e.g. `.venv/bin`), so it’s on PATH only when that env is active.
-2. Reinstall: `pip install --force-reinstall scrapingbee-cli`.
+2. If pip warned that the script went to `~/.local/bin` (a user install, common in sandboxed shells), that directory is not on PATH in minimal shells: run `export PATH="$HOME/.local/bin:$PATH"`, or call `~/.local/bin/scrapingbee` by full path.
+3. Reinstall: `pip install --force-reinstall scrapingbee-cli`.
+
+## Network allowlist (Claude Cowork and other locked-down sandboxes)
+
+If every command fails with `Cannot connect to host app.scrapingbee.com:443 ... Temporary failure in name resolution`, the shell can reach PyPI but not the ScrapingBee API: the host restricts outbound traffic to an allowlist. Two ways through:
+
+1. An org Owner adds `app.scrapingbee.com` to the allowed hosts (in Claude Cowork: Admin settings → Capabilities). The CLI then works unchanged.
+2. Use the ScrapingBee MCP connector (`https://mcp.scrapingbee.com/mcp`, `Authorization: Bearer <key>`) for single-page and search work — connectors are not subject to the sandbox allowlist. Crawls, batches, file output and scheduling still need the CLI, so prefer option 1 when those matter.
+
+Do not fall back to the host's built-in fetch tool as a substitute: it is subject to the same allowlist and cannot render JavaScript or pass anti-bot protection.
 
 **See also:** [rules/security.md](rules/security.md) (credits, output safety, shell safety).

.github/skills/scrapingbee-cli/SKILL.md

@@ -39,7 +39,9 @@ schedule a recurring job, or keep a large page out of your context window.
 
 Reach for the **MCP** in two cases: the host already has it connected *and* the task is a single
 page or search whose result belongs in the conversation anyway; or you cannot run a CLI at all —
-no shell, no filesystem, or installation is blocked.
+no shell, no filesystem, installation is blocked, or the shell cannot reach
+`app.scrapingbee.com` (a "Temporary failure in name resolution" from the CLI means a network
+allowlist is in the way — the MCP connector is not subject to it).
 
 MCP endpoint: `https://mcp.scrapingbee.com/mcp` — remote, Streamable HTTP, authenticated with an
 `Authorization: Bearer <SCRAPINGBEE_API_KEY>` header. Host-specific connection steps belong to the
@@ -81,7 +83,11 @@ for.
 
 1. **Install:** `uv tool install scrapingbee-cli` (recommended) or `pip install scrapingbee-cli`.
    Every command including `crawl` works immediately — no extras.
-2. **Authenticate:** `scrapingbee auth`, or set `SCRAPINGBEE_API_KEY`.
+   If `scrapingbee` is then "command not found", the executable is in `~/.local/bin`, which
+   minimal shells don't search: run `export PATH="$HOME/.local/bin:$PATH"` once, or call
+   `~/.local/bin/scrapingbee` by full path. Do not fall back to another tool because of this.
+2. **Authenticate:** `scrapingbee auth`, or set `SCRAPINGBEE_API_KEY` (a `.env` file in the
+   working directory is read too).
 3. **Verify:** `scrapingbee usage` — confirms the key works and shows remaining credits.
 
 ## Commands

.github/skills/scrapingbee-cli/rules/install.md

@@ -72,6 +72,16 @@ scrapingbee logout
 If `scrapingbee` is not found after install:
 
 1. Activate the environment where `pip install scrapingbee-cli` was run (e.g. `source .venv/bin/activate`). Pip puts the `scrapingbee` script in that env’s bin (e.g. `.venv/bin`), so it’s on PATH only when that env is active.
-2. Reinstall: `pip install --force-reinstall scrapingbee-cli`.
+2. If pip warned that the script went to `~/.local/bin` (a user install, common in sandboxed shells), that directory is not on PATH in minimal shells: run `export PATH="$HOME/.local/bin:$PATH"`, or call `~/.local/bin/scrapingbee` by full path.
+3. Reinstall: `pip install --force-reinstall scrapingbee-cli`.
+
+## Network allowlist (Claude Cowork and other locked-down sandboxes)
+
+If every command fails with `Cannot connect to host app.scrapingbee.com:443 ... Temporary failure in name resolution`, the shell can reach PyPI but not the ScrapingBee API: the host restricts outbound traffic to an allowlist. Two ways through:
+
+1. An org Owner adds `app.scrapingbee.com` to the allowed hosts (in Claude Cowork: Admin settings → Capabilities). The CLI then works unchanged.
+2. Use the ScrapingBee MCP connector (`https://mcp.scrapingbee.com/mcp`, `Authorization: Bearer <key>`) for single-page and search work — connectors are not subject to the sandbox allowlist. Crawls, batches, file output and scheduling still need the CLI, so prefer option 1 when those matter.
+
+Do not fall back to the host's built-in fetch tool as a substitute: it is subject to the same allowlist and cannot render JavaScript or pass anti-bot protection.
 
 **See also:** [rules/security.md](rules/security.md) (credits, output safety, shell safety).

.kiro/skills/scrapingbee-cli/SKILL.md

@@ -39,7 +39,9 @@ schedule a recurring job, or keep a large page out of your context window.
 
 Reach for the **MCP** in two cases: the host already has it connected *and* the task is a single
 page or search whose result belongs in the conversation anyway; or you cannot run a CLI at all —
-no shell, no filesystem, or installation is blocked.
+no shell, no filesystem, installation is blocked, or the shell cannot reach
+`app.scrapingbee.com` (a "Temporary failure in name resolution" from the CLI means a network
+allowlist is in the way — the MCP connector is not subject to it).
 
 MCP endpoint: `https://mcp.scrapingbee.com/mcp` — remote, Streamable HTTP, authenticated with an
 `Authorization: Bearer <SCRAPINGBEE_API_KEY>` header. Host-specific connection steps belong to the
@@ -81,7 +83,11 @@ for.
 
 1. **Install:** `uv tool install scrapingbee-cli` (recommended) or `pip install scrapingbee-cli`.
    Every command including `crawl` works immediately — no extras.
-2. **Authenticate:** `scrapingbee auth`, or set `SCRAPINGBEE_API_KEY`.
+   If `scrapingbee` is then "command not found", the executable is in `~/.local/bin`, which
+   minimal shells don't search: run `export PATH="$HOME/.local/bin:$PATH"` once, or call
+   `~/.local/bin/scrapingbee` by full path. Do not fall back to another tool because of this.
+2. **Authenticate:** `scrapingbee auth`, or set `SCRAPINGBEE_API_KEY` (a `.env` file in the
+   working directory is read too).
 3. **Verify:** `scrapingbee usage` — confirms the key works and shows remaining credits.
 
 ## Commands

.kiro/skills/scrapingbee-cli/rules/install.md

@@ -72,6 +72,16 @@ scrapingbee logout
 If `scrapingbee` is not found after install:
 
 1. Activate the environment where `pip install scrapingbee-cli` was run (e.g. `source .venv/bin/activate`). Pip puts the `scrapingbee` script in that env’s bin (e.g. `.venv/bin`), so it’s on PATH only when that env is active.
-2. Reinstall: `pip install --force-reinstall scrapingbee-cli`.
+2. If pip warned that the script went to `~/.local/bin` (a user install, common in sandboxed shells), that directory is not on PATH in minimal shells: run `export PATH="$HOME/.local/bin:$PATH"`, or call `~/.local/bin/scrapingbee` by full path.
+3. Reinstall: `pip install --force-reinstall scrapingbee-cli`.
+
+## Network allowlist (Claude Cowork and other locked-down sandboxes)
+
+If every command fails with `Cannot connect to host app.scrapingbee.com:443 ... Temporary failure in name resolution`, the shell can reach PyPI but not the ScrapingBee API: the host restricts outbound traffic to an allowlist. Two ways through:
+
+1. An org Owner adds `app.scrapingbee.com` to the allowed hosts (in Claude Cowork: Admin settings → Capabilities). The CLI then works unchanged.
+2. Use the ScrapingBee MCP connector (`https://mcp.scrapingbee.com/mcp`, `Authorization: Bearer <key>`) for single-page and search work — connectors are not subject to the sandbox allowlist. Crawls, batches, file output and scheduling still need the CLI, so prefer option 1 when those matter.
+
+Do not fall back to the host's built-in fetch tool as a substitute: it is subject to the same allowlist and cannot render JavaScript or pass anti-bot protection.
 
 **See also:** [rules/security.md](rules/security.md) (credits, output safety, shell safety).

.opencode/skills/scrapingbee-cli/SKILL.md

@@ -39,7 +39,9 @@ schedule a recurring job, or keep a large page out of your context window.
 
 Reach for the **MCP** in two cases: the host already has it connected *and* the task is a single
 page or search whose result belongs in the conversation anyway; or you cannot run a CLI at all —
-no shell, no filesystem, or installation is blocked.
+no shell, no filesystem, installation is blocked, or the shell cannot reach
+`app.scrapingbee.com` (a "Temporary failure in name resolution" from the CLI means a network
+allowlist is in the way — the MCP connector is not subject to it).
 
 MCP endpoint: `https://mcp.scrapingbee.com/mcp` — remote, Streamable HTTP, authenticated with an
 `Authorization: Bearer <SCRAPINGBEE_API_KEY>` header. Host-specific connection steps belong to the
@@ -81,7 +83,11 @@ for.
 
 1. **Install:** `uv tool install scrapingbee-cli` (recommended) or `pip install scrapingbee-cli`.
    Every command including `crawl` works immediately — no extras.
-2. **Authenticate:** `scrapingbee auth`, or set `SCRAPINGBEE_API_KEY`.
+   If `scrapingbee` is then "command not found", the executable is in `~/.local/bin`, which
+   minimal shells don't search: run `export PATH="$HOME/.local/bin:$PATH"` once, or call
+   `~/.local/bin/scrapingbee` by full path. Do not fall back to another tool because of this.
+2. **Authenticate:** `scrapingbee auth`, or set `SCRAPINGBEE_API_KEY` (a `.env` file in the
+   working directory is read too).
 3. **Verify:** `scrapingbee usage` — confirms the key works and shows remaining credits.
 
 ## Commands

.opencode/skills/scrapingbee-cli/rules/install.md

@@ -72,6 +72,16 @@ scrapingbee logout
 If `scrapingbee` is not found after install:
 
 1. Activate the environment where `pip install scrapingbee-cli` was run (e.g. `source .venv/bin/activate`). Pip puts the `scrapingbee` script in that env’s bin (e.g. `.venv/bin`), so it’s on PATH only when that env is active.
-2. Reinstall: `pip install --force-reinstall scrapingbee-cli`.
+2. If pip warned that the script went to `~/.local/bin` (a user install, common in sandboxed shells), that directory is not on PATH in minimal shells: run `export PATH="$HOME/.local/bin:$PATH"`, or call `~/.local/bin/scrapingbee` by full path.
+3. Reinstall: `pip install --force-reinstall scrapingbee-cli`.
+
+## Network allowlist (Claude Cowork and other locked-down sandboxes)
+
+If every command fails with `Cannot connect to host app.scrapingbee.com:443 ... Temporary failure in name resolution`, the shell can reach PyPI but not the ScrapingBee API: the host restricts outbound traffic to an allowlist. Two ways through:
+
+1. An org Owner adds `app.scrapingbee.com` to the allowed hosts (in Claude Cowork: Admin settings → Capabilities). The CLI then works unchanged.
+2. Use the ScrapingBee MCP connector (`https://mcp.scrapingbee.com/mcp`, `Authorization: Bearer <key>`) for single-page and search work — connectors are not subject to the sandbox allowlist. Crawls, batches, file output and scheduling still need the CLI, so prefer option 1 when those matter.
+
+Do not fall back to the host's built-in fetch tool as a substitute: it is subject to the same allowlist and cannot render JavaScript or pass anti-bot protection.
 
 **See also:** [rules/security.md](rules/security.md) (credits, output safety, shell safety).

AGENTS.md

@@ -30,7 +30,9 @@ schedule a recurring job, or keep a large page out of your context window.
 
 Reach for the **MCP** in two cases: the host already has it connected *and* the task is a single
 page or search whose result belongs in the conversation anyway; or you cannot run a CLI at all —
-no shell, no filesystem, or installation is blocked.
+no shell, no filesystem, installation is blocked, or the shell cannot reach
+`app.scrapingbee.com` (a "Temporary failure in name resolution" from the CLI means a network
+allowlist is in the way — the MCP connector is not subject to it).
 
 MCP endpoint: `https://mcp.scrapingbee.com/mcp` — remote, Streamable HTTP, authenticated with an
 `Authorization: Bearer <SCRAPINGBEE_API_KEY>` header. Host-specific connection steps belong to the
@@ -72,7 +74,11 @@ for.
 
 1. **Install:** `uv tool install scrapingbee-cli` (recommended) or `pip install scrapingbee-cli`.
    Every command including `crawl` works immediately — no extras.
-2. **Authenticate:** `scrapingbee auth`, or set `SCRAPINGBEE_API_KEY`.
+   If `scrapingbee` is then "command not found", the executable is in `~/.local/bin`, which
+   minimal shells don't search: run `export PATH="$HOME/.local/bin:$PATH"` once, or call
+   `~/.local/bin/scrapingbee` by full path. Do not fall back to another tool because of this.
+2. **Authenticate:** `scrapingbee auth`, or set `SCRAPINGBEE_API_KEY` (a `.env` file in the
+   working directory is read too).
 3. **Verify:** `scrapingbee usage` — confirms the key works and shows remaining credits.
 
 ## Commands

CHANGELOG.md

@@ -5,6 +5,18 @@ All notable changes to this project are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+
+- **Plugin metadata:** `plugins/scrapingbee-cli/` now ships its own `README.md` and `LICENSE`, and the plugin manifest declares `license`, `homepage` and `repository`, as plugin directories require.
+- **Cursor and ChatGPT / Codex manifests:** the same plugin folder now carries `.cursor-plugin/plugin.json` (with a `SCRAPINGBEE_API_KEY` variable and `mcp.json` that connects the ScrapingBee MCP server) and `.codex-plugin/plugin.json`, plus logo assets, so one folder can be submitted to each marketplace.
+
+### Fixed
+
+- **Skill install path:** the skill now tells agents that `uv tool install` places `scrapingbee` in `~/.local/bin` and how to run it when that directory is not on `PATH`, instead of giving up on the CLI.
+- **Sandbox network allowlists:** documented what a "Temporary failure in name resolution" from the CLI means in a locked-down sandbox and that the MCP connector is the way around it.
+
 ## [1.6.0] - 2026-09-11
 
 ### Added

CONTRIBUTING.md

@@ -28,8 +28,12 @@ Keep these in sync whenever you bump the version:
 - `pyproject.toml` → `[project] version`
 - `src/scrapingbee_cli/__init__.py` → `__version__`
 - `.claude-plugin/marketplace.json` → `plugins[0].version`
-- `.claude-plugin/plugin.json` → `version`
-- `skills/scrapingbee-cli/SKILL.md` → frontmatter `version`
+- `plugins/scrapingbee-cli/.claude-plugin/plugin.json` → `version` (Claude Code)
+- `plugins/scrapingbee-cli/.cursor-plugin/plugin.json` → `version` (Cursor)
+- `plugins/scrapingbee-cli/.codex-plugin/plugin.json` → `version` (ChatGPT / Codex)
+- `.agents/skills/scrapingbee-cli/SKILL.md` and `scrapingbee-cli-guard/SKILL.md` → frontmatter `version`, then `scripts/sync-skills.sh`
+
+Plugin directories re-review a plugin when its version changes, so bump all three manifests in the same release PR.
 
 The CI `check-version` job enforces that `pyproject.toml` and `__init__.py` stay in sync.
 

plugins/scrapingbee-cli/.claude-plugin/plugin.json

@@ -2,6 +2,9 @@
   "name": "scrapingbee",
   "description": "Fetch and read any web page, search the web, crawl a site, or pull structured data out of pages \u2014 whenever a task needs content from a website. Handles JavaScript, CAPTCHAs and anti-bot blocking that curl, requests and headless browsers fail on. Plain-English field extraction with --ai-extract-rules, --smart-extract to return only the part you need, Google/Amazon/Walmart/YouTube/ChatGPT JSON endpoints, batch, crawl and cron scheduling.",
   "version": "1.6.0",
+  "license": "MIT",
+  "homepage": "https://www.scrapingbee.com/documentation/cli/",
+  "repository": "https://github.com/ScrapingBee/scrapingbee-cli",
   "author": {
     "name": "ScrapingBee"
   },

plugins/scrapingbee-cli/.codex-plugin/plugin.json

@@ -0,0 +1,31 @@
+{
+  "name": "scrapingbee",
+  "version": "1.6.0",
+  "description": "Fetch and read any web page, search the web, crawl a site, or pull structured data out of pages — whenever a task needs content from a website. Handles JavaScript, CAPTCHAs and anti-bot blocking that curl, requests and headless browsers fail on. Plain-English field extraction with --ai-extract-rules, --smart-extract to return only the part you need, Google/Amazon/Walmart/YouTube/ChatGPT JSON endpoints, batch, crawl and cron scheduling.",
+  "author": {
+    "name": "ScrapingBee"
+  },
+  "homepage": "https://www.scrapingbee.com/documentation/cli/",
+  "repository": "https://github.com/ScrapingBee/scrapingbee-cli",
+  "license": "MIT",
+  "skills": "./skills",
+  "interface": {
+    "displayName": "ScrapingBee",
+    "shortDescription": "Read, search and crawl the web",
+    "longDescription": "ScrapingBee fetches any web page as clean markdown, text or JSON — including JavaScript-rendered pages and sites behind CAPTCHAs or anti-bot blocking that plain HTTP tools fail on. This plugin teaches the agent when and how to use the ScrapingBee CLI: scrape a single page or a CSV of URLs, crawl a whole site into files for a RAG index, search Google, Amazon, Walmart or YouTube and get structured JSON back, extract fields described in plain English, and schedule recurring checks — while keeping credit spend to the minimum that works. Requires a ScrapingBee account and API key.",
+    "category": "Developer Tools",
+    "capabilities": ["Read"],
+    "composerIcon": "./assets/icon.png",
+    "logo": "./assets/icon.png",
+    "developerName": "ScrapingBee",
+    "websiteURL": "https://www.scrapingbee.com/documentation/cli/",
+    "privacyPolicyURL": "https://www.scrapingbee.com/privacy-policy/",
+    "termsOfServiceURL": "https://www.scrapingbee.com/terms-and-conditions/",
+    "defaultPrompt": [
+      "Fetch this pricing page as markdown and save it to a file",
+      "Crawl this docs site and save every guide page for my RAG index",
+      "Get the current price and rating for this Amazon ASIN",
+      "I have a CSV of product URLs — refresh the price column in place"
+    ]
+  }
+}