Skip to content

Update Gemma3 multimodal SFT Jupyter notebook#4154

Open
SurbhiJainUSC wants to merge 1 commit into
mainfrom
multimodal_notebook
Open

Update Gemma3 multimodal SFT Jupyter notebook#4154
SurbhiJainUSC wants to merge 1 commit into
mainfrom
multimodal_notebook

Conversation

@SurbhiJainUSC

@SurbhiJainUSC SurbhiJainUSC commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR updates the Gemma 3 multimodal Supervised Fine-Tuning (SFT) Jupyter notebook to align with the latest MaxText training utilities and the native SFT pipeline. The primary goal is to provide a streamlined, end-to-end example for users to convert checkpoint, and fine-tune Gemma 3 multimodal model.

Key Changes:

  • Notebook Update: The multimodal_gemma3_demo.ipynb has been updated to use maxtext.trainers.post_train.sft.train_sft_native.
  • Workflow Enhancements: Updated the Checkpoint Conversion section to demonstrate converting Orbax checkpoints from Hugging Face for Gemma 3 models.
  • Trainer Alignment: Minor updates to train_sft_native.py to better align with train.py.
  • Dependency Management: Updated instructions for installing post-training dependencies within the notebook environment.

Tests

  • Notebook Execution: Verified the end-to-end flow in VsCode on TPU v5p-8.
  • Checkpoint Conversion: Confirmed successful conversion of Gemma3-4B weights from Hugging Face.
  • SFT Run: Successfully executed a 10-step SFT run on the ChartQA dataset using the native trainer.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@SurbhiJainUSC SurbhiJainUSC force-pushed the multimodal_notebook branch 2 times, most recently from 095e754 to 39df681 Compare June 12, 2026 00:17

@YixuanWang-99 YixuanWang-99 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants