EdgeBench: Live Streaming & HDF5 Extensions for AURA

This project extends the original AURA framework with live-style video processing, live camera input, rolling live retraining, direct HDF5 layer streaming, and HDF5-to-MP4 conversion. It keeps the core detector architecture intact while adapting the input side so video, camera, and HDF5 image stacks can flow through the same threshold / ML / hybrid detection pipeline.

The framework supports:

Statistical threshold-based anomaly detection
Machine learning–based anomaly detection
Hybrid threshold + ML detection
Live video and USB camera processing
Rolling live retraining for stream-adaptive ML
Direct streaming of HDF5 image layers
HDF5-to-MP4 conversion for replayable test streams
Cross-modality dataset benchmarking (thermal, optical, event camera, HDF5)

🚀 Features

AES-style timing-compatible payload pipeline preserved for streaming inputs
Frame-to-frame image change converted into a timing-like scalar signal
Live rolling history window with bounded memory growth
Random Forest retraining on recent stream history
Hybrid anomaly decision logic (threshold OR ML)
OpenCV video file and USB camera input support
HDF5 dataset layer streaming for Peregrine-style build imagery
HDF5 conversion to MP4 for repeated playback and faster testing
CSV anomaly logging for live streams
Batch and live execution modes for video workflows

🧠 System Overview

The original AURA detector was designed around timing-like records. Each record includes an index, a time value, an anomaly type, and sometimes an original block. This project preserves that detector interface by converting frame-to-frame image change into a timing-like scalar value:

change_intensity = mean(abs(current_gray_frame - previous_gray_frame))

That value is stored in the existing time field, allowing the same threshold, ML, and hybrid detectors to operate regardless of whether the source is:

AES timing data
a video file
a USB webcam
a camera index
a direct HDF5 image stack

The key design decision is that the adapters are responsible only for producing frames, while the shared live detection path is responsible for turning those frames into payloads and running detection.

The core detector remains unchanged. The adapter converts source-specific inputs into a compatible statistical signal used by the shared detection pipeline.

AURA supports two primary workflows.

AES Timing Anomaly Detection

AES Execution
↓
Timing Feature Extraction
↓
Threshold / ML / Hybrid Detection
↓
Anomaly Report

Video / Camera / HDF5 Live Benchmarking

Video / Camera / HDF5 Input
↓
Frame-to-frame statistical signal extraction
↓
Live rolling history window
↓
Threshold / ML / Hybrid Detection
↓
Anomaly CSV or formatted report

🚀 Setup Instructions

Prerequisites

Python 3.7+
Virtual environment (recommended: venv or conda)
Dependencies: pandas, numpy, scikit-learn, opencv-python, h5py, openpyxl, psutil
- pycryptodome remains optional for legacy AES-only workflows

Installation

Clone the repository:
```
git clone <repository-url>
cd EdgeBench
```

Create and activate a virtual environment:

python3 -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

Install dependencies:
```
pip install -r AURA/Requirements.txt
```

If you are working directly from the root of this repository, the live/HDF5 scripts can also be run without the submodule being installed as a standalone package.

🧪 Usage

AES Detection Experiments

Threshold-Based AES Detection

python3 Threshold_Detect.py

ML-Based AES Detection

python3 ML_Detect.py

Results are saved as:

aes_anomaly_report.xlsx

📹 Video Dataset and Live Stream Benchmarking

Run anomaly detection on a video file:

python3 thermal_adapter.py <path_to_video_file>

Example:

python3 thermal_adapter.py "/path/to/Laser Welds.mp4"

The script prompts for the execution mode:

batch
live
detection mode: threshold, ml, or hybrid

Batch mode

Reads the full video, stores payloads in memory, and produces an Excel report at the end.

<video_name>_thermal_report.xlsx

Live mode

Reads frames one at a time and detects anomalies immediately against a rolling history buffer. It does not wait for the entire file to finish and writes anomaly logs as the stream progresses.

<video_name>_live_anomalies.csv

📷 Live Camera Input

The same live pipeline can read a USB camera source or numeric camera index using OpenCV. Supported aliases include:

camera
cam
usb
usb_camera
webcam

Numeric camera indexes are also supported:

0
1
2

Run the default camera:

python3 thermal_adapter.py camera

Or a specific camera index:

python3 thermal_adapter.py 1

Then choose:

Select pipeline (batch / live): live
Select detection mode (threshold / ml / hybrid): threshold

Camera sources are intentionally rejected in batch mode because a live camera has no natural end point and should not be loaded into a full batch list. When a camera stream is running, OpenCV displays the current frame. Press q to stop. If an anomaly is detected, the frame gets an ANOMALY DETECTED overlay and the event is appended to a CSV file such as:

usb_camera_0_live_anomalies.csv

⚙️ Detection Modes

threshold → statistical detection against the recent history window
ml → Random Forest detection once enough stream history exists for training
hybrid → threshold OR ML, with anomaly if either detector triggers

The script will:

process frames one by one
extract frame-change statistical signals
update the rolling history window
run the selected mode
log anomalies immediately for live workflows

🔁 Shared Live Detection Path

The live detection state is created via:

create_live_detection_state(window_size=300, min_train_samples=60, retrain_every=15)

This returns a dictionary that includes:

history_window
rf_model
rf_feature_columns
frames_since_retrain
min_train_samples
retrain_every

The most important component is:

history_window = deque(maxlen=window_size)

This bounded rolling buffer keeps only the most recent payloads, preventing unbounded memory growth during long camera or HDF5 streams. Each payload is processed by:

detect_live_payload(payload, mode, state)

This shared function handles the threshold, ML, and hybrid paths across regular video, camera, and HDF5 inputs.

🧠 Frame-to-Payload Conversion

All live frame sources use:

process_single_frame(frame, prev_frame, index)

The function converts the current frame to grayscale, compares it with the previous grayscale frame, and returns a payload like:

{
    "index": index,
    "time": change_intensity,
    "anomaly_type": None,
    "original_block": [0] * 16
}

The first frame does not produce a payload because there is no previous frame to compare against. The original_block field is a compatibility placeholder so the ML feature extractor continues to see the same schema used by older AES-style workloads.

🔄 Live Retraining

Live ML mode cannot rely on future frames, so this project trains the Random Forest from the recent rolling history window. The live ML workflow includes logic for:

extract_live_features(results)
make_live_pseudo_labels(df)
train_live_rf(results)
predict_live_rf(model, feature_columns, history_results, payload)

The default live retraining settings are:

window_size = 300
min_train_samples = 60
retrain_every = 15

These values are tuned to keep the model responsive to stream drift without retraining on every frame. The live ML path builds pseudo-labels from the recent time / change-intensity distribution, because livestream inputs do not provide supervised ground-truth anomaly labels in the same way AES timing traces do.

This enables behavior such as:

adapting to lighting changes
handling camera motion or slight geometry drift
keeping the model tied to the most recent window instead of a stale baseline

📦 HDF5 Layer Streaming

This repository includes direct HDF5 streaming for Peregrine-style layer datasets. The HDF5 adapter reads one layer image at a time, converts it to a BGR OpenCV frame, and sends it through the same live detection logic used for videos and cameras.

The default dataset path is:

slices/camera_data/visible/0

It can also use the post-recoat stream:

slices/camera_data/visible/1

Run a short HDF5 threshold test:

python3 hdf5_live_adapter.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --mode threshold \
  --max-layers 50 \
  --progress-every 10

Run HDF5 hybrid mode:

python3 hdf5_live_adapter.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --mode hybrid \
  --max-layers 100 \
  --progress-every 10

Show a display window:

python3 hdf5_live_adapter.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --mode threshold \
  --display

Use post-recoat images instead of post-melt images:

python3 hdf5_live_adapter.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --dataset slices/camera_data/visible/1 \
  --mode threshold

HDF5 Live Flags

hdf5_file
--mode {threshold,ml,hybrid}
--dataset DATASET
--window-size WINDOW_SIZE
--start-layer START_LAYER
--max-layers MAX_LAYERS
--step STEP
--display
--display-width DISPLAY_WIDTH
--delay-ms DELAY_MS
--output-csv OUTPUT_CSV
--progress-every PROGRESS_EVERY

This streaming approach preserves the original HDF5 source data path and allows correctness checks on a small layer range without converting the whole dataset first.

🎬 HDF5-to-MP4 Conversion

The project also includes utilities to convert HDF5 image stacks into MP4 files for repeated testing and smoother OpenCV playback.

Default behavior:

input:  slices/camera_data/visible/0
output: <hdf5_stem>_visible_0.mp4
fps:    30
codec:  mp4v

Convert the default post-melt image stack:

python3 hdf5_to_mp4.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --output tcr_visible0.mp4

Convert both visible streams:

python3 hdf5_to_mp4.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --both-images \
  --output tcr_visible_both.mp4

Convert only the first 100 images:

python3 hdf5_to_mp4.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --max-images 100 \
  --output tcr_visible0_100.mp4

Generate a 10-minute repeated test video:

python3 hdf5_to_mp4.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --output tcr_visible0_18000_30fps.mp4 \
  --fps 30 \
  --max-images 18000 \
  --repeat \
  --progress-every 500

HDF5 Conversion Flags

hdf5_file
--output OUTPUT
--fps FPS
--dataset DATASET
--both-images
--max-images MAX_IMAGES
--start-layer START_LAYER
--step STEP
--repeat
--resize-width RESIZE_WIDTH
--codec CODEC
--progress-every PROGRESS_EVERY

This conversion step is useful when the same image sequence will be tested repeatedly, since it avoids repeatedly paying the heavy cost of HDF5 decompression on every experiment.

🏃 Running AURA On Converted MP4s

Once an MP4 has been created, it can be processed with the normal video workflow:

python3 thermal_adapter.py tcr_visible0.mp4

Then choose:

Select pipeline (batch / live): live
Select detection mode (threshold / ml / hybrid): hybrid

This is often the fastest setup for repeated experiments because OpenCV can read a native video stream smoothly instead of repeatedly decompressing HDF5 chunks.

Quick Command Reference

Live video file

python3 thermal_adapter.py test.mp4

Live camera

python3 thermal_adapter.py camera

HDF5 live, post-melt

python3 hdf5_live_adapter.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --mode threshold \
  --max-layers 50

HDF5 live, post-recoat

python3 hdf5_live_adapter.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --dataset slices/camera_data/visible/1 \
  --mode threshold \
  --max-layers 50

Convert HDF5 post-melt to MP4

python3 hdf5_to_mp4.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --output tcr_visible0.mp4

Convert both HDF5 streams to MP4

python3 hdf5_to_mp4.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --both-images \
  --output tcr_visible_both.mp4

Repeatable 10-minute test stream

python3 hdf5_to_mp4.py \
  '2021-04-16 TCR Phase 1 Build 2.hdf5' \
  --output tcr_visible0_18000_30fps.mp4 \
  --fps 30 \
  --max-images 18000 \
  --repeat \
  --progress-every 500

📂 Repository Structure

EdgeBench/
├── README.md                    # Project documentation
├── LIVE_AND_HDF5_ADDITIONS.md   # Live/HDF5 design and usage notes
├── thermal_adapter.py           # Video, camera, and live detection adapter
├── hdf5_live_adapter.py         # HDF5 layer streaming through the live path
├── hdf5_to_mp4.py              # HDF5 image stack to MP4 conversion utility
├── ML_Detect.py                # ML detection, feature extraction, and retraining logic
├── Threshold_Detect.py         # Threshold detection logic
├── secom_adapter.py            # Additional adapter/workflow support
├── AURA/                       # AURA submodule / baseline implementation
│   ├── README.md
│   ├── Requirements.txt
│   ├── ML_Detect.py
│   ├── Threshold_Detect.py
│   ├── thermal_adapter.py
│   └── Testing/
└── ...

🎓 Credits

This project builds on the original AURA framework developed by Nishant Chinnasami with advisor Dr. Rasha Karakchi at the University of South Carolina.

The live streaming, camera input, HDF5 streaming, and HDF5-to-MP4 extensions were designed by Vito Spatafora with advisor Dr. Rasha Karakchi at the University of South Carolina. This extension was developed to preserve the original AURA detection philosophy while adding streaming behavior and dataset-access workflows needed for modern camera-style and layer-wise experimentation.

🤝 Acknowledgments

This work builds on the foundation of the AURA framework and its supporting research environment.

The original project was supported under the McNair Junior Fellowship and Magellan Scholar Program at the University of South Carolina. Special thanks to Rye Stahle-Smith for hardware testing and experimental support.

Name		Name	Last commit message	Last commit date
AURA @ 1387451		AURA @ 1387451
.gitmodules		.gitmodules
ML_Detect.py		ML_Detect.py
README.md		README.md
Threshold_Detect.py		Threshold_Detect.py
hdf5_live_adapter.py		hdf5_live_adapter.py
hdf5_to_mp4.py		hdf5_to_mp4.py
secom_adapter.py		secom_adapter.py
thermal_adapter.py		thermal_adapter.py

Latest commit

History

Folders and files

Repository files navigation