FoodShutter

Photo-based nutrition tracking, with a native iOS app and a web companion.

MIT License · iOS 26+ · FastAPI · Next.js 16

Demo · Getting started · Architecture · Documentation · 简体中文

FoodShutter turns a meal photograph into detected dishes, ingredient estimates, calories and macronutrients. The iOS app captures meals and presents analysis in stages. A Python service combines vision models with nutrition retrieval, while Supabase stores accounts and meal records for both clients.

The project was developed by a six-person team for a software project management course. This repository contains the working prototype, model experiments and engineering documentation. Running the full application requires configured cloud services; the setup guide and release readiness notes describe the current requirements.

The complete course film includes sign-in, onboarding, capture, analysis, ingredient detail and recommendations. The silent preview loops at two-thirds speed, about 90 seconds per loop, with higher-resolution imagery. Expand the player to watch the original-speed film with audio.

Play the complete film with audio, about 60 seconds

foodshutter-film.mp4

Complete video file

Screens captured from the same recording:

Capture a meal	Review meal nutrition	Inspect ingredients

Full web walkthrough, about 35 seconds · Short iOS excerpt · Demo guide and media sources

Web companion: meal history and nutrition trends

The web client reads shared meal records and provides a dashboard, history, weekly or monthly statistics, and appearance settings.

web-walkthrough.mp4

The recording shows the course demonstration build. Its displayed nutrition values are estimates; the current source and deployment requirements are documented separately.

Capability	Implementation
Meal capture and staged results	Native camera capture, authenticated upload, early dish list, then nutrition details
Multiple dishes in one image	Vision detection, bounding-box filtering, per-dish analysis with full meal context
Ingredient nutrition retrieval	USDA Foundation Foods, BGE embeddings, ChromaDB semantic and lexical search
Traceable estimates	Ingredient source labels, food database IDs and qualitative confidence grades
Personal dietary context	iOS profile, nutrition targets, meal advice and next-meal recommendation
History across clients	Supabase meal records, iOS history, web dashboard and nutrition statistics
Progress and achievements	iOS nutrition rings, target ranges and trophy views

Some course scope items remain incomplete: web suggestions use sample messages, web nutrition search needs a matching backend endpoint, and PDF/Excel export is not implemented in this tree. See the capability status for the distinction between implemented features, external dependencies and planned work.

The mobile client sends a photograph and its Supabase access token to POST /analyze. The API accepts the job with HTTP 202; results are written to Supabase as processing advances. iOS observes meal status through Realtime and concurrent REST polling. The web companion queries the same meal and dish tables through its Supabase client.

The model endpoint is configurable through an OpenAI-compatible API. Qwen3.5-9B fine-tuning is a separate research and deployment path; it is not a required dependency for running the client or the mocked backend tests.

Component responsibilities · Data contracts · API reference

1. Detect dish names and bounding boxes from the complete photograph.
2. Remove overlapping or contained boxes, apply area limits and flag image edges.
3. Check retained dish count and recover detections after extreme filtering.
4. Analyze each dish with an annotated image and the other dishes as context.
5. Retrieve nutrition candidates for each ingredient.
6. Select database entries or label model-generated nutrition estimates.
7. Scale nutrients by estimated weight and calculate dish and meal totals.
8. Generate advice from the current meal, user profile and recent dietary history.

Steps 4–6 use up to four worker threads per stage. Model calls request schema-constrained JSON. Missing nutrition data can continue through an explicitly labeled estimate, and an advice-generation failure does not discard completed nutrition results.

Pipeline details and limits · Fine-tuning and model serving

FoodShutter/
├── iOSApp/
│   ├── FoodShutter/App/           Application state and onboarding
│   ├── FoodShutter/Core/          Models, camera, API and Supabase repositories
│   ├── FoodShutter/Features/      Analysis, history, authentication and profile UI
│   ├── FoodShutter/Shared/        Reusable nutrition and suggestion components
│   ├── FoodShutterTesting/        Seven XCTest source files
│   └── project.yml               XcodeGen project definition
├── Backend/
│   ├── api/                      FastAPI routes, token verification and jobs
│   ├── pipeline/                 Eight analysis steps and model client
│   ├── nutrition_db/             USDA cleaning, ChromaDB build and search
│   ├── Finetune/finetune/         Qwen3.5-9B training and comparison scripts
│   ├── tests/                    Pytest suite
│   └── docs/                     Original backend design and integration notes
├── web/
│   ├── app/                      Next.js App Router pages
│   ├── components/               Layout, motion and theme components
│   ├── lib/                      Supabase adapters and application data access
│   └── supabase/functions/       Edge Function integration scaffolding
└── docs/                         Architecture, setup, evaluation and demo assets

Component	Requirements	Entry point
Backend	Python 3.12+, Supabase project, compatible vision/model endpoint	Backend/api/main.py
Web	Node.js satisfying Next.js 16 requirements, npm, Supabase public configuration	web/app/
iOS	macOS, Xcode 26 with iOS 26 SDK, signing for a physical device	iOSApp/FoodShutter.xcodeproj

Clone the repository:

git clone https://github.com/FoodShutter-Development-Team/FoodShutter.git
cd FoodShutter

Backend:

cd Backend
cp .env.example .env
# Fill in your server-side credentials and model configuration.
uv sync
uv run uvicorn api.main:app --host 0.0.0.0 --port 8000

Web, in another terminal:

cd web
cp .env.example .env.local
# Fill in the URL and public anon key for your Supabase project.
npm ci
npm run dev

On macOS, open iOSApp/FoodShutter.xcodeproj, configure your own Supabase client and backend URL, select the FoodShutter scheme, and run on an iOS 26 device. The camera workflow requires a physical device.

Database migrations, RLS policies and trained weights are not included. Complete service provisioning before expecting authenticated meal analysis to work. The checked-in iOS target is iOS 26; the earlier course proposal's iOS 17 target does not describe the current build.

The final presentation reports a five-angle demonstration of a seven-dish meal. Reported calorie variation was 6.4% for the pipeline and 22.1% for the presentation's baseline. This small comparison measures consistency across photographs, and does not establish nutrition accuracy. Evaluation methods, results and limitations preserve that distinction.

The existing backend test suite was rerun on 14 October 2026 against source baseline 3a7bf20: 237 tests passed. Overall statement coverage across pipeline, api and nutrition_db was 66.52%; the pipeline directory reached 96.92%. External model and cloud interactions are mocked. iOS XCTest sources are included, but were not executed in the Windows documentation environment. Testing guide

The course work also covers requirements, scope, estimation, Scrum planning, risk management and configuration control. Estimates and simulated reliability figures are identified as course exercises in the project context.

Roles reported in the final course presentation:

Contributor	Area
Liu Xiangyu	iOS development, AI pipeline and model fine-tuning
Xie Tingyang	Testing, documentation, presentation and deployment
Liu Simiao	Presentation and illustration design
Zhou Yunhui	Documentation
Li Peichen	Web frontend and data synchronization
Han Chengxu	Backend development and web integration support

These roles credit the course team. Git authorship remains attached to actual repository commits.

Read CONTRIBUTING.md for development and review expectations, and SECURITY.md for credential handling and the current security scope.

FoodShutter's source code is available under the MIT License. Third-party models, training data and nutrition datasets have their own terms. The application provides estimated nutrition information and is outside the course's scope for medical diagnosis or treatment.