Skip to content

feat: add URL and link management tools #785

Description

@avoidwork

Summary

Add URL shortening, link preview/metadata extraction, and bookmark management tools.

Motivation

The existing webExtract tool fetches raw page content, but there is no dedicated URL management capability. Office and marketing workflows frequently need to: shorten URLs for sharing, extract link metadata (title, description, image) before posting, manage bookmarks/favorites, and validate link health. Currently the agent must fall back to shell commands (curl, wget) or rely on ad-hoc LLM reasoning, which is inconsistent.

Proposed Solution

Create URL management tools:

  • URL Shortening: Generate short URLs via services like Bitly, Rebrandly, or custom shorteners
  • Link Preview: Extract metadata (title, description, image, Open Graph tags) from a URL without fetching full content
  • Bookmark Management: Create, read, update, delete bookmarks with tags and notes
  • Link Validation: Check if URLs are reachable, return HTTP status codes, detect redirects
  • Link Analysis: Extract all links from a page, detect broken links, analyze link structure

Each tool should follow the existing tool pattern in src/tools/ — a zod schema, an impl function, and registration in index.js with appropriate permissions.

Alternatives Considered

  • Shell-based tools (curl, wget): fragile, no structured output, requires user-installed dependencies.
  • Rely on webExtract: overkill for metadata-only extraction, no shortening or bookmarking capability.

OpenSpec Note

This project uses OpenSpec for feature development. If this request is approved, I will:

  1. Run /opsx:propose to generate a full proposal with specs and tasks
  2. Iterate on the design before any code is written
  3. Follow the task-driven implementation workflow

Additional Context

This should integrate with the text processing gap (#784) — e.g., analyzing link metadata as part of SEO workflows. The existing webExtract tool handles full content extraction; these tools would provide lighter-weight URL operations.

Dependencies

  • URL shortening: bitly npm package (official Bitly SDK) — requires API key via env var BITLY_API_KEY. Alternative: custom shortener via configurable base URL.
  • Link preview: cheerio for HTML parsing of Open Graph/meta tags — lightweight, no browser required.
  • Link validation: node-fetch with timeout — already a project dependency or easily added.
  • Bookmark storage: SQLite via better-sqlite3 (single-file, no server) or JSON file fallback for simple use cases. SQLite preferred for concurrent access and query capability.

Testing Strategy

  • Unit tests: Mock HTTP responses for URL shortening, metadata extraction, and validation. Verify Zod schema validation for all inputs.
  • Integration test: Generate a short URL via mock Bitly response, verify redirect chain, validate metadata extraction against known HTML.
  • Bookmark CRUD: Test create/read/update/delete against in-memory or temp SQLite database.
  • Edge cases: Invalid URLs, DNS failures, HTTP timeouts, redirect loops, non-HTTP schemes (mailto:, tel:).

Security Considerations

  • URL allowlist: All outbound URL operations must validate against an allowlist per AGENTS.md 1.2. Disallow file://, gopher://, dict:// schemes.
  • Redirect handling: Follow redirects up to a configurable max depth (default: 5) to prevent redirect loop attacks. Reject redirects to disallowed schemes.
  • API key storage: Bitly API key stored in process.env.BITLY_API_KEY only — never in config files or logs.
  • Bookmark data: User-created bookmarks stored in SQLite with no execution of stored content (no SQL injection via bookmark titles/URLs — use parameterized queries).
  • Rate limiting: Implement client-side rate limiting (default: 10 requests/second) to avoid triggering provider blocks.

Implementation Notes

  • Consider splitting into two separate tools: url (shortening, preview, validation) and bookmarks (CRUD operations). This keeps each tool focused and makes testing easier.
  • Zod schema for URL operations: { url: z.string().url(), options?: { timeout?: number, followRedirects?: boolean } }.
  • Bookmark schema: { id?: string, url: z.string().url(), title?: string, tags: z.array(z.string()), notes?: string, createdAt?: string }.
  • Link analysis (extract all links from a page) can reuse webExtract for the initial fetch, then parse with cheerio.
  • Consider adding a --cache flag to skip re-fetching metadata for recently seen URLs (TTL-based cache in SQLite).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions