Skip to content
BranchesTags

Repository files navigation

AI4IFC / AI4BIM

A small monorepo of AI-assisted BIM demonstrators built during Summer Semester 2026.

It contains:

  • a Flask backend that exposes endpoints for IFC metadata extraction + Q&A, room photo classification, house price prediction, and (optional) Stable Diffusion layout generation
  • a Next.js (App Router) frontend that provides a UI for the demonstrators

What’s included

Demonstrators (Frontend)

  • IFC Explorer: upload an IFC file, extract basic building metadata + rooms, and ask questions about the model
  • House Price Predictor: XGBoost-based price estimate from a small set of house features
  • Layout Generator: text-to-image and sketch-to-image layout generation (Stable Diffusion; optional)
  • Room Classifier: classify a room photo using a Hugging Face image classification model

Backend APIs

The Flask backend provides:

  • an LLM Q&A endpoint using an OpenAI-compatible API (optional; falls back to simple rule-based answers)
  • ML inference endpoints for price prediction and room image classification
  • Stable Diffusion generation endpoint (only enabled when the diffusion pipeline loads successfully)

Repo structure

.
├── backend/                 # Flask API + ML inference code
│   ├── flask_app.py
│   ├── requirements.txt
│   ├── diffusion/           # Custom UNet checkpoints (if available)
│   ├── house_price_predict/ # XGBoost model + predictor
│   └── room_classifier/     # Hugging Face room classifier + training utilities
├── website/                 # Next.js frontend
└── docs/                    # Learning material (house price notebook + data)

Prerequisites

  • Python: 3.10 or 3.11 required (newer versions may fail to install TensorFlow/PyTorch dependencies)
  • Node.js: 18+ recommended (Next.js 16)
  • Optional for faster generation:
    • NVIDIA GPU + CUDA (for Stable Diffusion)

Quickstart (local development)

1) Set up Environment Variables

Optional: Open website/.env.local and add your Supabase project keys (NEXT_PUBLIC_SUPABASE_URL and NEXT_PUBLIC_SUPABASE_ANON_KEY). The Next.js frontend will crash if these are missing.

2) Download Custom ML Models

For the Layout Generator (Stable Diffusion) to work, you need the custom UNet checkpoint:

  1. Download the unet_epoch_19 folder (safetensors and config) from https://drive.google.com/drive/folders/1B5hX9o55sh_kYn8yhfQnvvEF_3hwLSPl?usp=sharing.
  2. Place the folder inside the backend/diffusion/ directory so the path looks like backend/diffusion/unet_epoch_19/.

3) Start the backend (Flask)

Important: run from the backend/ folder so imports resolve correctly.

# Open Terminal 1
cd backend
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python flask_app.py

Note: The first time you run this, it will take several minutes to download gigabytes of base ML models (like Stable Diffusion).

Backend listens on http://127.0.0.1:5001.

Check:

curl http://127.0.0.1:5001/health

4) Start the frontend (Next.js)

# Open a NEW Terminal (Terminal 2)
cd website
npm install
npm run dev

Frontend is available at http://localhost:3000.

Tip: From the repo root you can also run npm run dev, but you still need npm install inside website/ at least once.

Configuration (Backend Environment variables)

The backend reads these variables from the process environment:

The Flask app does NOT auto-load .env.example.

# Optional: enables IFC Q&A with an OpenAI-compatible endpoint
LLM_API_KEY=...
LLM_BASE_URL=...
LLM_MODEL=openai-gpt-oss-120b

If LLM_API_KEY is not set, the backend falls back to a small built-in answer function.

Supabase setup (optional)

The frontend includes login/sign-up pages and expects a profiles table.

  • Create a Supabase project
  • Run the SQL in website/scripts/001_create_profiles.sql in the Supabase SQL editor
  • Set the NEXT_PUBLIC_SUPABASE_URL + NEXT_PUBLIC_SUPABASE_ANON_KEY variables

Flask API reference

Base URL (default): http://127.0.0.1:5001

GET /health

Returns service status and which models loaded.

curl http://127.0.0.1:5001/health

POST /api/upload-ifc

Upload an IFC file and extract basic metadata + room boxes.

  • Content-Type: multipart/form-data
  • Field name: file
curl -F "file=@/path/to/model.ifc" \
  http://127.0.0.1:5001/api/upload-ifc

Response includes a server-side filename token (used for queries) and data (rooms, storeys, summary).

POST /api/query-ifc

Ask a natural language question about a previously uploaded IFC.

curl -X POST http://127.0.0.1:5001/api/query-ifc \
  -H "Content-Type: application/json" \
  -d '{"filename":"<from upload>","question":"How many rooms are there?"}'

POST /predict-price

Predict a house price using a pre-trained XGBoost model.

curl -X POST http://127.0.0.1:5001/predict-price \
  -H "Content-Type: application/json" \
  -d '{
    "GrLivArea": 1500,
    "BedroomAbvGr": 3,
    "FullBath": 2,
    "YearBuilt": 2005,
    "KitchenAbvGr": 1
  }'

Notes:

  • The model expects feature keys matching the training columns stored in backend/house_price_predict/columns_xgb.pkl.
  • To inspect available columns:
python - <<'PY'
import joblib
cols = joblib.load('backend/house_price_predict/columns_xgb.pkl')
print('n_columns =', len(cols))
print('sample =', cols[:25])
PY

POST /api/classify-room

Classify an image of a room.

  • Content-Type: multipart/form-data
  • Field name: image
curl -F "image=@/path/to/room.jpg" \
  http://127.0.0.1:5001/api/classify-room

POST /api/generate-layout

Generate an architectural layout image.

curl -X POST http://127.0.0.1:5001/api/generate-layout \
  -H "Content-Type: application/json" \
  -d '{"prompt":"A modern house layout with 3 bedrooms and an open kitchen"}'

Optional img2img (sketch-to-layout):

  • Provide image as a base64 PNG (data URL is accepted)
  • Provide strength in [0.0, 1.0]

If the diffusion pipeline is not available, this endpoint returns 503.

Notes on the IFC viewer

  • The backend extracts rooms (IfcSpace) and derives approximate 3D boxes (x, z, width, depth, height).
  • The frontend renders these as colored three.js boxes (not full IFC geometry).
  • Extracted IFC data is stored in-memory in the backend; restarting the backend clears uploaded model context.

Stable Diffusion (Layout generator) notes

  • The backend loads diffusion pipelines on startup (this can take time).
  • It is configured to load Stable Diffusion v1.5 (runwayml/stable-diffusion-v1-5) with a custom UNet checkpoint from backend/diffusion/unet_epoch_19.
  • If the diffusion pipeline can’t be initialized (missing checkpoint files, incompatible weights, download/auth issues), /health reports diffusion as not_loaded and POST /api/generate-layout returns 503.
  • Depending on your environment, you may need to authenticate with Hugging Face / accept model terms to download the base model.

Development scripts

Frontend

From website/:

  • npm run dev
  • npm run build
  • npm run start
  • npm run lint

Root convenience scripts

From repo root:

  • npm run dev (delegates to website)
  • npm run build / start / lint

Troubleshooting

  • Backend import errors: run python flask_app.py from inside backend/.
  • ifcopenshell issues: it can be platform-specific; ensure your Python version matches a supported wheel for your OS.
  • Layout generator returns 503: diffusion pipeline didn’t load (missing or incompatible checkpoint / model download issues).
  • Slow first run: Hugging Face / Diffusers may download model weights on first use.
  • Supabase errors in the UI: set NEXT_PUBLIC_SUPABASE_URL and NEXT_PUBLIC_SUPABASE_ANON_KEY.

Acknowledgements

Built with:

  • Flask + flask-cors
  • IfcOpenShell (ifcopenshell)
  • Hugging Face Transformers (pipeline)
  • XGBoost + scikit-learn
  • Diffusers (Stable Diffusion)
  • Next.js + Tailwind CSS + Radix UI
  • Supabase (Auth)

About

No description, website, or topics provided.

Resources

Readme
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages