feat: add opt-in TwelveLabs backend (Marengo search + Pegasus answers)#9
Open
mohit-twelvelabs wants to merge 1 commit into
Open
feat: add opt-in TwelveLabs backend (Marengo search + Pegasus answers)#9mohit-twelvelabs wants to merge 1 commit into
mohit-twelvelabs wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi! I'm Mohit, I work at TwelveLabs (@mohit-twelvelabs).
This PR adds an opt-in alternative backend for StreamRAG powered by TwelveLabs, so the same GPT/agent can run on TwelveLabs' video-native models:
/search./video/{id}.Why it helps
StreamRAG is a video search agent; TwelveLabs is purpose-built for video understanding and multimodal retrieval. This gives users a second backend option without changing the GPT prompt or
openapi.yaml— the provider returns the exact same{compilationVideo, chunks}/{video: {...}}shapes the existing routes return.Opt-in / non-breaking
VIDEO_SEARCH_PROVIDER=twelvelabsis set. Unset (the default) keeps the existing VideoDB behavior completely untouched —twelvelabs_providerisn't even imported..env.sample;twelvelabs==1.2.8added torequirements.txt.How it was tested
python -m unittest test_twelvelabs_provider— unit tests (mocked SDK, no network) verify the VideoDB-compatible response shapes and search/analyze wiring.TWELVELABS_API_KEY+TWELVELABS_INDEX_ID, skipped otherwise) were run against the real API: Marengo text embedding (512-dim), Marengo search, and a Pegasus answer all succeed.You can grab a free API key at https://twelvelabs.io — there's a generous free tier.