Company Enrichment System (Hackathon Submission)

A robust, lightweight, and deterministic B2B intelligence and scraping pipeline that extracts company profiles directly from target domains and automatically enriches them using Generative AI (Gemini or OpenAI).

This system is structured into a backend FastAPI + SQLite API service, a React + Vite + TypeScript dashboard client, and a self-contained Google Colab bulk processing notebook.

---

🏗️ Architecture Overview

The system follows a modular monorepo structure:

• backend/: FastAPI web service, SQLite storage via SQLAlchemy, Pydantic data schemas, BeautifulSoup prioritized crawler, and Generative AI client wrappers.
• frontend/: Single-Page Application (SPA) dashboard built using React, Vite, TypeScript, TailwindCSS v4, Axios, and React Query.
• notebooks/: Standalone bulk parsing notebooks for Colab environments.

├── backend/                  # FastAPI Web Server
│   ├── app/
│   │   ├── database.py       # SQLAlchemy SQLite connection
│   │   ├── models.py         # DB entity mapping
│   │   ├── schemas.py        # Pydantic JSON contracts
│   │   ├── crud.py           # DB query helpers
│   │   ├── scraper.py        # prioritized crawling pipeline
│   │   ├── ai.py             # Gemini / OpenAI wrappers
│   │   ├── routes.py         # API endpoints
│   │   └── main.py           # app startup & CORS middleware
│   ├── tests/                # Deterministic phone test suite
│   ├── requirements.txt      # Python dependencies
│   └── runtime.txt           # Python engine version
├── frontend/                 # React Single Page App
│   ├── src/
│   │   ├── App.tsx           # Dashboard view, forms, query hooks
│   │   ├── main.tsx          # App entrypoint
│   │   └── index.css         # Tailwind v4 import
│   ├── vite.config.ts        # Vite configuration
│   └── package.json          # Node dependencies
├── notebooks/                # Google Colab Integration
│   └── company_enrichment.ipynb
├── render.yaml               # Render blueprint file
└── .gitignore                # Global git ignores

---

⚡ Scraper & Phone Extraction Strategy

Crawling & HTML Cleaning

The scraper reads the target URL and crawls up to 5 local pages dynamically matching about, contact, services, products, and solutions target keywords using fuzzy string matching via RapidFuzz to locate business details. HTML content is cleaned by stripping script, style, nav, footer, and svg tags to create a clean text corpus, avoiding token bloat.

Phone Number Extraction (Deterministic)

Extraction is strictly deterministic and follows a strict hierarchy of sources to return only the most relevant business contact:

• Priority 1: tel: DOM links.
• Priority 2: Visible text located on pages matched as the Contact page.
• Priority 3: Visible text within <footer> elements.
• Priority 4: Visible page text from other crawled targets.

All matches are normalized to a standard digit string (retaining the optional leading + country prefix) and deduplicated. Only the highest-priority number is stored under mobile_number.

---

🤖 AI Enrichment Strategy

• Context Limiting: Trims the collected cleaned text corpus to 12,000 characters before sending it to the model to optimize token usage.
• Strict Anti-Hallucination Prompting: Explicit system guidelines forbid the model from inventing company names, addresses, emails, or phone numbers. If a field cannot be derived, it returns "".
• JSON Schema Enforcement: Uses Pydantic structured output mapping (via Gemini's response_schema or OpenAI's .beta.chat.completions.parse) to guarantee response JSON matches the required schema exactly.
• Heuristic Backup: Falls back gracefully to rule-based templates if no API keys are present.

---

📂 Required JSON Schema

The API and exports strictly conform to the frozen hackathon output format:

{
  "website_name": "https://example.com",
  "company_name": "Example Corp",
  "address": "123 Main St, City, CA 90210",
  "mobile_number": "+15551234567",
  "mail": ["info@example.com"],
  "core_service": "Enterprise cloud services",
  "target_customer": "Mid-market tech companies",
  "probable_pain_point": "Server overhead costs",
  "outreach_opener": "Hi Team, I noticed your work in..."
}

---

⚙️ Environment Variables

Backend (backend/.env)

• GEMINI_API_KEY: API Key for Google Gemini (takes precedence).
• OPENAI_API_KEY: API Key for OpenAI models.

Frontend (frontend/.env)

• VITE_API_BASE_URL: The URL of your deployed backend service. Default is http://127.0.0.1:8000 for local runs.

---

🛠️ Local Setup

Prerequisite

Ensure Python 3.10+ and Node.js are installed on your machine.

Backend Setup

1. Navigate to the backend folder:

   cd backend

2. Initialize virtual environment and install dependencies:

   python -m venv .venv
   .venv\Scripts\activate      # Windows
   source .venv/bin/activate    # Mac/Linux
   pip install -r requirements.txt

3. Run the uvicorn server:

   uvicorn app.main:app --reload

   Backend will be running at http://127.0.0.1:8000.

Frontend Setup

1. Navigate to the frontend folder:

   cd frontend

2. Install npm packages:

   npm install

3. Run the development client:

   npm run dev

   Open http://localhost:5173/ in your browser.

Run Unit Tests

To verify phone extraction and normalization reliability, run:

python -m pytest tests/test_phone_extractor.py

---

📝 Colab Integration (Task 1)

judges and developers can run bulk queries inside Google Colab:

1. Open Google Colab.
2. Upload the notebook file: company_enrichment.ipynb.
3. Run cells in sequence to install packages, input array parameters, run crawling, print schemas, and export the final results.json output file.

---

🚀 Deployment Instructions

Backend (Render)

1. Commit and push your code to a GitHub repository.
2. Log into Render, choose Blueprints, and connect your repository.
3. Render will parse the render.yaml configuration and launch your FastAPI backend automatically.
4. Set GEMINI_API_KEY or OPENAI_API_KEY in Render environment variables.

Frontend (Vercel)

1. Add a new project in Vercel and connect your GitHub repository.
2. Set the build directory to frontend/.
3. Set the Environment Variable: VITE_API_BASE_URL pointing to your deployed Render URL.
4. Deploy the application. Vercel will bundle the code and host your dashboard on a public domain.