🚀 Google Maps Lead Scraper & Contact Finder

A professional, high-speed Python command-line utility that extracts local business prospects directly from Google Maps without requiring API keys or quota fees.

The scraper targets businesses lacking website links (your ideal candidates for free website builds or modern design outreach), retrieves their public contact information, searches DuckDuckGo for their company contact email addresses, and persists results incrementally with automatic deduplication.

---

📖 Table of Contents

1. System Features
2. Prerequisites & System Requirements
3. Step-by-Step Installation
4. Step-by-Step Configuration & API Keys
5. Comprehensive Usage Examples
6. Understanding the Output Data Schema
7. Troubleshooting & FAQs

---

🔍 System Features

• Zero Maps API Fees: Leverages Playwright to scrape public Google Maps results in headless Chromium.
• Smart Filter: Automatically skips any local listing that already has an established website link.
• DuckDuckGo Email Locator: Searches public indexing dynamically for the company's contact email address.
• Incremental Saving & Deduplication: Saves lists progressively after every query to protect data and guarantees zero duplicates via URL and phone hashing.
• Self-Healing County Database: Automatically downloads a localized list of all 3,143 counties in the United States offline for seamless state-wide target queries.
• Highly Customizable CLI: Run campaigns easily across predefined state arrays, custom county list strings, or custom search queries.

---

💻 Prerequisites & System Requirements

• Operating System: macOS, Linux, or Windows (tested and optimized for macOS & Linux environments).
• Python: Python 3.9, 3.10, or 3.11 recommended (avoid Homebrew Python 3.14 due to dynamic linking library issues with pyexpat).
• Node.js (Optional): Only required if connecting to the downstream SMS outreach client.

---

🛠️ Step-by-Step Installation

Follow these exact steps to set up your local environment and download the headless browser binaries.

Step 1: Clone the Repository

Clone the repository and navigate into the project directory:

git clone https://github.com/Robj1925/google-maps-scraper.git
cd google-maps-scraper

Step 2: Initialize Virtual Environment

Isolate your Python dependencies using a local virtual environment.

# Create a virtual environment named "venv" using the system python3
python3 -m venv venv

# Activate the virtual environment
# On macOS and Linux:
source venv/bin/activate

# On Windows (Command Prompt):
# venv\Scripts\activate.bat

# On Windows (PowerShell):
# venv\Scripts\Activate.ps1

Step 3: Install Required Libraries

Upgrade pip and install the mandatory Python packages (playwright and pandas):

pip install --upgrade pip
pip install -r requirements.txt

Step 4: Install Headless Playwright Browser

Download and configure the headless Chromium binary required for crawling Google Maps:

playwright install chromium

---

🔑 Step-by-Step Configuration & API Keys (OPTIONAL)

Important

Getting and configuring API keys is completely optional. The core Google Maps Lead Scraper works 100% out-of-the-box to extract names, addresses, emails, and phone numbers without these keys.

Why use Phone Validation APIs?

If you are planning to feed the harvested phone numbers into a downstream automated cold-messaging engine (such as the SMS outreach client), phone validation APIs are highly recommended.

• What the APIs do: These lookup services check the carrier record of each phone number to identify if it is a Mobile (cell) number, a Landline number, or a VoIP line.
• If you do NOT provide them: The system will still harvest all leads and numbers. However, landline numbers will be included in your SMS campaign, resulting in failed text message deliveries to those specific landlines. Providing the API keys filters your CSV lists to target only verified cell phones.

---

Step 1: Copy the Environment Template

Create your active .env file from the provided template:

cp .env.template .env

Step 2: (Optional) Obtain free API Keys

If you choose to use validation, open the newly created .env file and populate the following keys:

1. NUMVERIFY_API_KEY (Primary Validation)

• How to get it (Free):
  1. Visit https://numverify.com/.
  2. Click Sign Up and select the Free Plan (includes 100 free requests per month).
  3. Copy your API access key from your dashboard and paste it into .env:

     NUMVERIFY_API_KEY=your_key_here

2. ABSTRACT_API_KEY (Fallback Validation)

• How to get it (Free):
  1. Go to https://www.abstractapi.com/.
  2. Select the Phone Validation API and register for a free account.
  3. Copy your unique API key and paste it into .env:

     ABSTRACT_API_KEY=your_key_here

---

🚀 Comprehensive Usage Examples

Execute campaigns dynamically by running commands inside your active virtual environment:

Example 1: State-Wide County Campaign

Crawl an entire state's counties for a specific sector. The script will automatically resolve state abbreviations or full names and download counties_by_state.json offline:

# Scrape roofing contractors across all 254 counties in Texas
python scraper.py --industry "roofing" --state TX

# Scrape construction companies across all 62 counties in New York
python scraper.py --industry "construction" --state "New York"

# Scrape plumbing services across all 21 counties in New Jersey
python scraper.py --industry "plumbing" --state NJ

Example 2: Target Custom Counties list

To scrape only specific counties or municipal regions, pass a comma-separated list to the --counties flag:

# Scrape landscaping businesses in specific counties of New Jersey and New York
python scraper.py --industry "landscaping" --counties "Bergen County NJ, Orange County NY, Westchester County NY"

Example 3: Direct Maps Query

To run a quick search query directly on Google Maps (bypassing state or county loops), use the --query flag:

# Search for hair salons specifically in Salem, NY
python scraper.py --query "hair salons in Salem, NY" --max-results 5

Example 4: Limit Results per Query

Control your crawl speed and resource consumption by specifying the max results to fetch per location loop using --max-results (default is 10):

# Get at most 3 electrical contractors per county in New Jersey
python scraper.py --industry "electrician" --state NJ --max-results 3

Example 5: Custom Output CSV File

Specify a custom destination CSV file path using --csv (default is leads.csv):

# Save scraped roofing leads to a distinct target sheet
python scraper.py --industry "roofing" --state NJ --csv "new_jersey_roofers.csv"

---

📊 Understanding the Output Data Schema

All scraped leads are appended progressively to your output CSV file. The CSV contains the following columns:

Column Name	Example Value	Description
Name	Navarro Family Roofing	The public business name listed on Google Maps.
Phone	(551) 310-2400	The validated local phone number.
Address	260 Ryders Ln, Milltown, NJ 08850	The public street address of the business.
Email	info@navarroroofing.com	Scraped email address from DuckDuckGo contact index (N/A if not found).
URL	https://www.google.com/maps/place/...	The direct Google Maps page link (used for indexing/deduplication).

---

🛠️ Troubleshooting & FAQs

Q: Why do I see HTTP Error 403: Forbidden during email lookup?

• A: DuckDuckGo has temporarily flagged your IP address for high-volume automated searching. The scraper is built to handle this gracefully: it logs the error, sets the Email column to N/A, and continues crawling phone numbers and addresses without stopping.

Q: Playwright fails with "Executable doesn't exist"?

• A: You missed installing the browser binaries. Run the following command inside your active virtual environment to download Chromium:

  playwright install chromium

Q: Can I run this script on a recurring cron job?

• A: Yes. Ensure your cron script activates the virtual environment first:

  #!/bin/bash
  cd /path/to/google-maps-scraper
  source venv/bin/activate
  python scraper.py --industry "construction" --state NJ --max-results 5