AlexNet Implementation From Scratch

A complete, faithful implementation of the AlexNet convolutional neural network architecture from scratch using PyTorch. This implementation follows the original paper specifications and includes comprehensive training infrastructure.

🎯 About AlexNet

AlexNet was introduced in the groundbreaking paper "ImageNet Classification with Deep Convolutional Neural Networks" by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. It revolutionized computer vision by:

• Winning ImageNet 2012 with a top-5 error rate of 15.3%
• Popularizing deep learning in computer vision
• Introducing key innovations like ReLU activations, dropout, and GPU training

🚀 Key Features

• ✅ Accurate Architecture: Faithful reproduction of the original AlexNet paper
• ✅ Comprehensive Training: Full training pipeline with validation and checkpointing
• ✅ Mixed Precision: Support for faster training with maintained accuracy
• ✅ Exponential Moving Average: Model weight averaging for better generalization
• ✅ Tensorboard Logging: Real-time monitoring of training progress
• ✅ Flexible Configuration: Easy customization via environment variables
• ✅ Resume Training: Robust checkpoint system for interrupted training

🔧 Installation

Prerequisites

• Python 3.7+
• CUDA-compatible GPU (recommended)
• 8GB+ GPU memory for training

Install Dependencies

git clone https://github.com/yourusername/alexnet-pytorch.git
cd alexnet-pytorch
pip install -r requirements.txt

📊 Architecture Details

Model Structure

• Input: 227×227×3 RGB images
• 5 Convolutional Layers with ReLU activation
• 3 Fully Connected Layers with dropout
• Local Response Normalization after Conv1 and Conv2
• Max Pooling after Conv1, Conv2, and Conv5

Layer Specifications

Layer	Filters	Kernel Size	Stride	Padding	Output Size
Conv1	96	11×11	4	0	55×55×96
Conv2	256	5×5	1	2	27×27×256
Conv3	384	3×3	1	1	13×13×384
Conv4	384	3×3	1	1	13×13×384
Conv5	256	3×3	1	1	13×13×256
FC1	-	-	-	-	4096
FC2	-	-	-	-	4096
FC3	-	-	-	-	num_classes

🎮 Usage

Quick Start

# Copy environment template
cp .env.example .env

# Edit configuration (optional)
nano .env

# Start training
python train.py

Configuration Options

Create a .env file with your desired settings:

# Model configuration
model_num_classes=100
exp_name=my_alexnet_experiment

# Training parameters
epochs=100
batch_size=128
model_lr=0.01

# Dataset paths
train_image_dir=./data

Using Jupyter Notebook

jupyter notebook train.ipynb

📈 Training Tips

Recommended Settings

• Batch Size: 128-256 (adjust based on GPU memory)
• Learning Rate: 0.01 with cosine annealing
• Optimizer: SGD with momentum (0.9)
• Weight Decay: 0.0005
• Dropout: 0.5 in fully connected layers

Data Preprocessing

• Resize images to 227×227 pixels
• Use ImageNet normalization: mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]
• Apply data augmentation for better generalization

📁 Project Structure

alexnet-pytorch/
├── model.py                    # AlexNet architecture
├── train.py                    # Training script
├── utils.py                    # Utility functions
├── train.ipynb                 # Jupyter notebook
├── requirements.txt            # Dependencies
├── .env.example               # Configuration template
├── README.md                  # This file
├── PROJECT_DESCRIPTION.md     # Detailed project info
└── ISSUES_FOUND_AND_FIXED.md # Bug fixes documentation

🔍 Monitoring Training

Tensorboard

tensorboard --logdir=results/logs

Training Output

The training script provides:

• Real-time loss and accuracy metrics
• Top-1 and Top-5 accuracy tracking
• Learning rate monitoring
• Automatic checkpointing

🙏 Acknowledgments

• Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton for the original AlexNet paper
• PyTorch team for the excellent deep learning framework
• ImageNet dataset creators for providing the benchmark dataset

📚 References

1. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105).
2. Original AlexNet Paper
3. ImageNet Dataset
4. PyTorch Documentation

---

⭐ Star this repository if you find it helpful! ⭐