Deep learning is easier to understand when the equations, implementation, and engineering decisions are visible together. This hub pairs readable, from-scratch implementations with mathematical explanations, practical guides, and concise reference material.
- Readable implementations of optimizers, hyperparameter-search methods, neural-network building blocks, and training techniques.
- Mathematical explanations that derive the algorithms rather than treating them as black boxes.
- Practical how-to guides for choosing optimizers, checking gradients, tuning learning rates, and running hyperparameter searches.
- Reference pages for equations, defaults, tensor shapes, and the public Python API.
- Tests grounded in the mathematics, including hand-computed values, closed-form results, and finite-difference checks.
The documentation follows Diátaxis, so learning, problem-solving, lookup, and deeper understanding each have a clear home:
| If you want to... | Start here |
|---|---|
| Learn by building something | Tutorials |
| Complete a specific task | How-to guides |
| Look up an equation, default, shape, or API | Reference |
| Understand why a method works | Explanation |
The hub is a reference rather than a fixed course, but these routes provide a useful order through the material that exists today.
- Forward and backward propagation
- Network shapes and dimensions
- Matrix calculus for neural networks
- Run a gradient check
- Parameter initialization
- Regularization
- Optimization algorithms
- Choose an optimizer
- Tune a learning rate
Clone the repository and install the NumPy reference package:
git clone https://github.com/alshawai/Deep-Learning-Reference-Hub.git
cd Deep-Learning-Reference-Hub
python -m pip install -e .The base installation includes NumPy, SciPy, and Matplotlib. NumPy is the reference platform used by the published implementation modules.
Some documentation also demonstrates the same ideas with PyTorch 2.3+ and TensorFlow 2.17+ using its bundled Keras API. The framework badges communicate the intended minimum compatibility for those optional examples. They do not claim that every NumPy implementation already has a framework port; expanding that parity is part of the hub's future growth.
Install the optional frameworks when you need those examples:
python -m pip install -e ".[frameworks]"Install the development and documentation dependencies:
python -m pip install -e ".[dev,docs]"Then run the same checks enforced by CI:
python tools/hubcheck.py all
mkdocs build --strict
ruff format --check .
ruff check .
python -m pytest -qSee CONTRIBUTING.md for the repository conventions and contribution workflow.
- Starting out: forward propagation, activation functions, parameter initialization, and the optimizer-selection guide.
- Building confidence: backpropagation, regularization, gradient checking, learning-rate schedules, and random search.
- Going deeper: matrix calculus, adaptive optimization, Bayesian optimization, multi-fidelity methods, and population-based training.
These labels describe the background a page assumes, not the importance of its subject. The documentation navigation is the authoritative index as the hub grows.
- Neural-network foundations: propagation, activations, tensor shapes, and parameter updates.
- Training techniques: initialization, regularization, gradient checking, early stopping, optimizers, and learning-rate schedules.
- Hyperparameter optimization: random and Bayesian search, ASHA and multi-fidelity methods, population-based training, and learning-rate finding.
Computer vision, natural language processing, and generative modelling are natural future directions, but the README does not list them as current coverage until the repository contains material readers can use.
- NumPy: the current from-scratch reference implementations.
- PyTorch: optional examples targeting PyTorch 2.3 and newer.
- TensorFlow/Keras: optional examples targeting TensorFlow 2.17 and newer, using the Keras API bundled with TensorFlow.
The hub aims to be educational without becoming approximate. Contributions are expected to preserve:
- Mathematical accuracy: equations and numerical behavior agree.
- Readable implementations: teaching code exposes the algorithm's important steps instead of hiding them behind abstractions.
- Behavioral tests: tests assert meaningful values and invariants, not only output shapes.
- Reproducibility: examples state seeds, shapes, dtypes, and tolerances when those details affect the result.
- Documentation integrity: strict site builds reject broken links, missing navigation entries, and invalid cross-references.
- Consistent Python quality: Ruff enforces formatting, imports, and NumPy docstring conventions.
- Total Documents: 21
- Code Examples: 17 implementations
- Frameworks Covered: NumPy
The framework badges above describe optional example compatibility. The checked statistics count published implementation modules, which are currently NumPy based.
Corrections, clearer explanations, stronger tests, new implementations, and carefully chosen references are welcome. Start with CONTRIBUTING.md, and use GitHub Issues to propose or discuss larger changes.
This project builds on the work of researchers, educators, and open-source maintainers who make deep learning knowledge accessible. In particular, it owes much to Andrew Ng and the Deep Learning Specialization, the authors of the papers and books cited throughout the documentation, and the NumPy, SciPy, Matplotlib, PyTorch, TensorFlow, and Keras communities.
This project is available under the MIT License.