diff --git a/AGENTS.md b/AGENTS.md index 82f491c7..6d84bf79 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -309,7 +309,9 @@ default glob; the Rakefile's `test` task is what sweeps both. ### RubyLLM - Uses `ruby_llm` gem for unified access to 15+ providers - RubyLLM manages its own API keys via `RubyLLM.configure` -- Model ID determines which provider is used automatically +- Model ID determines which provider is used automatically; `platform:` + (maps to RubyLLM's `provider:`) pins it when a model ID is served by + more than one, e.g. `platform: :vertexai` for Gemini models on Vertex AI - Supports prompts, embeddings, tool calling, and streaming ## The dashboard: a second gem in this repo diff --git a/CHANGELOG.md b/CHANGELOG.md index 9868b6d8..58aa2ebf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,7 +5,30 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [1.3.1] - 2026-08-19 +## [Unreleased] + +### Added + +- **RubyLLM backend pinning via `platform:`.** RubyLLM resolves which of its + providers serves a request from the model ID, and a model served by more + than one — `gemini-2.5-flash` exists on both the Gemini API and Vertex + AI — lands on whichever RubyLLM's registry prefers, with no way to say + otherwise from ActiveAgent. The new `platform:` option + (`generate_with :ruby_llm, model: "gemini-2.5-flash", platform: :vertexai`) + forwards to RubyLLM's `provider:` and pins the backend, for embeddings as + well as prompts. It is not named `provider:` because a provider reference + is already the first argument to `generate_with`. Omitting it keeps + model-based routing unchanged. (#373) + +### Fixed + +- **`service: "RubyLLM"` loads when the ruby_llm railtie has run.** The + ruby_llm gem registers `RubyLLM` as an inflector acronym in Rails apps, + which turns `"RubyLLM".underscore` into `rubyllm` — so provider loading + required a nonexistent `rubyllm_provider.rb` and failed with + `cannot load such file`. An alias file now covers that require path, the + same fix `openai_provider.rb` applies for `OpenAI`. (#371, fixed in #372 + by @aoki-ryusei; regression tests in #374) ### Fixed diff --git a/docs/providers/ruby_llm.md b/docs/providers/ruby_llm.md index a78d4a43..e261f56c 100644 --- a/docs/providers/ruby_llm.md +++ b/docs/providers/ruby_llm.md @@ -75,12 +75,48 @@ class FlexibleAgent < ApplicationAgent end ``` +### Pinning the Platform + +When the same model ID is served by more than one of RubyLLM's providers, RubyLLM picks one by its own registry preference — `gemini-2.5-flash` resolves to the Gemini API even when you have configured Vertex AI credentials. Set `platform:` to pin the request to a specific RubyLLM provider; it maps to RubyLLM's own `provider:` option: + +```ruby +class VertexAgent < ApplicationAgent + generate_with :ruby_llm, model: "gemini-2.5-flash", platform: :vertexai +end +``` + +Or in `config/active_agent.yml`: + +```yaml +production: + ruby_llm: + service: "RubyLLM" + model: "gemini-2.5-flash" + platform: "vertexai" +``` + +Authentication and region stay in RubyLLM's configuration: + +```ruby +# config/initializers/ruby_llm.rb +RubyLLM.configure do |config| + config.vertexai_project_id = "your-project-id" + config.vertexai_location = "us-central1" +end +``` + +Valid values are RubyLLM's provider keys — `:openai`, `:anthropic`, `:gemini`, `:vertexai`, `:bedrock`, `:openrouter`, `:ollama`, and so on. Omitting `platform:` keeps RubyLLM's automatic model-based routing. The option applies to embeddings as well as prompts. + ## Provider-Specific Parameters ### Required Parameters - **`model`** - Model identifier (e.g., "gpt-4o-mini", "claude-sonnet-5") +### Routing Parameters + +- **`platform`** - Pins which RubyLLM provider serves the model (maps to RubyLLM's `provider:`), e.g. `:vertexai` for Gemini models on Vertex AI. See [Pinning the Platform](#pinning-the-platform) + ### Sampling Parameters - **`temperature`** - Controls randomness (0.0 to 1.0) diff --git a/lib/active_agent/providers/ruby_llm/options.rb b/lib/active_agent/providers/ruby_llm/options.rb index 7ca1ca95..0654f0eb 100644 --- a/lib/active_agent/providers/ruby_llm/options.rb +++ b/lib/active_agent/providers/ruby_llm/options.rb @@ -11,6 +11,10 @@ module RubyLLM # provider-specific API key attributes are needed here. class Options < Common::BaseModel attribute :model, :string + # Pins which RubyLLM backend serves the model (RubyLLM's provider:, + # e.g. :vertexai, :gemini, :bedrock). A model ID served by several + # backends otherwise resolves by RubyLLM's registry preference. + attribute :platform, :string attribute :temperature, :float attribute :max_tokens, :integer diff --git a/lib/active_agent/providers/ruby_llm_provider.rb b/lib/active_agent/providers/ruby_llm_provider.rb index 7e647aa2..9bc9ed8b 100644 --- a/lib/active_agent/providers/ruby_llm_provider.rb +++ b/lib/active_agent/providers/ruby_llm_provider.rb @@ -12,6 +12,10 @@ module Providers # Provider for RubyLLM's unified API, supporting 15+ LLM providers # (OpenAI, Anthropic, Gemini, Bedrock, Azure, Ollama, etc.). # + # RubyLLM resolves which backend serves a request from the model ID; the + # platform option pins it when a model ID is served by more than one + # (e.g. Gemini models on the Gemini API vs Vertex AI). + # # Uses RubyLLM's provider-level API (provider.complete()) rather than # the high-level Chat object to avoid conflicts with ActiveAgent's own # conversation management and tool execution loop. @@ -254,13 +258,22 @@ def process_function_calls(tool_calls) # Reuses the cached provider if the model hasn't changed (e.g., during # multi-turn tool calling loops). # + # The platform option is forwarded as RubyLLM's provider: so a model ID + # served by several backends (e.g. gemini-2.5-flash on the Gemini API + # and Vertex AI) can be pinned instead of resolving by RubyLLM's + # registry preference. + # # @param model_id [String] model identifier # @return [void] def resolve_ruby_llm_provider!(model_id) return if @ruby_llm_provider && @cached_model_id == model_id @cached_model_id = model_id - @ruby_llm_model, @ruby_llm_provider = ::RubyLLM::Models.resolve(model_id, config: ::RubyLLM.config) + @ruby_llm_model, @ruby_llm_provider = ::RubyLLM::Models.resolve( + model_id, + provider: options.platform&.to_sym, + config: ::RubyLLM.config + ) end # Converts ActiveAgent messages to RubyLLM message format. diff --git a/test/providers/ruby_llm/provider_loading_test.rb b/test/providers/ruby_llm/provider_loading_test.rb new file mode 100644 index 00000000..06e45def --- /dev/null +++ b/test/providers/ruby_llm/provider_loading_test.rb @@ -0,0 +1,87 @@ +# frozen_string_literal: true + +require "test_helper" + +# The require_gem! guard in ruby_llm_provider.rb only checks that the +# ruby_llm gem's namespace exists, so the loading paths can be exercised +# without the gem installed. +module ::RubyLLM; end unless defined?(::RubyLLM) + +class RubyLLMProviderLoadingTest < ActiveSupport::TestCase + test "loads RubyLLMProvider via ruby_llm_provider path" do + require "active_agent/providers/ruby_llm_provider" + + assert defined?(ActiveAgent::Providers::RubyLLMProvider) + assert defined?(ActiveAgent::Providers::RubyLLM::Options) + end + + test "loads RubyLLMProvider via rubyllm_provider path" do + require "active_agent/providers/rubyllm_provider" + + assert defined?(ActiveAgent::Providers::RubyLLMProvider) + end + + test "provider concern loads the RubyLLM service with the gem's acronym registered" do + already_registered = "RubyLLM".underscore == "rubyllm" + + with_rubyllm_acronym do + assert_equal "rubyllm", "RubyLLM".underscore + + klass = ActiveAgent::Base.provider_load("RubyLLM") + assert_equal ActiveAgent::Providers::RubyLLMProvider, klass + end + + unless already_registered + assert_equal "ruby_llm", "RubyLLM".underscore, "acronym leaked out of with_rubyllm_acronym" + end + end + + test "provider concern loads the RubyLLM service without the acronym" do + skip "the ruby_llm railtie registered its acronym in this process" if "RubyLLM".underscore == "rubyllm" + + assert_equal "ruby_llm", "RubyLLM".underscore + + klass = ActiveAgent::Base.provider_load("RubyLLM") + assert_equal ActiveAgent::Providers::RubyLLMProvider, klass + end + + test "service name remap handles Rubyllm and RubyLlm variations" do + remaps = ActiveAgent::Provider::PROVIDER_SERVICE_NAMES_REMAPS + + assert_equal "RubyLLM", remaps["Rubyllm"] + assert_equal "RubyLLM", remaps["RubyLlm"] + end + + private + + # Registers the RubyLLM acronym the way the ruby_llm gem's railtie does. + # Edge Rails freezes every Inflections instance after boot, so the acronym + # goes on an unfrozen dup swapped in for the duration (dup support is what + # Inflections#initialize_dup exists for), and the original instance -- + # frozen or not -- is restored afterwards. + def with_rubyllm_acronym + original = ActiveSupport::Inflector.inflections(:en) + swap_en_inflections(original.dup) + + ActiveSupport::Inflector.inflections(:en) do |inflect| + inflect.acronym "RubyLLM" + end + + yield + ensure + swap_en_inflections(original) if original + end + + # Installs an :en Inflections instance in the slot this Rails version + # reads from: a dedicated @__en_instance__ where defined (8.1+), the + # @__instance__ map otherwise (7.2). + def swap_en_inflections(instance) + klass = ActiveSupport::Inflector::Inflections + + if klass.instance_variable_defined?(:@__en_instance__) + klass.instance_variable_set(:@__en_instance__, instance) + else + klass.instance_variable_get(:@__instance__)[:en] = instance + end + end +end diff --git a/test/providers/ruby_llm/ruby_llm_provider_test.rb b/test/providers/ruby_llm/ruby_llm_provider_test.rb index f5ad06a2..29ad0795 100644 --- a/test/providers/ruby_llm/ruby_llm_provider_test.rb +++ b/test/providers/ruby_llm/ruby_llm_provider_test.rb @@ -873,6 +873,112 @@ def complete(messages, **kwargs) end end + # --- platform pinning (RubyLLM's provider:) --- + + test "platform option pins the RubyLLM backend when resolving the model" do + resolve_kwargs = nil + capturing_resolve = ->(model_id, **kwargs) { + resolve_kwargs = kwargs + [ stub_model_info(model_id), ::RubyLLM::StubProvider.new ] + } + + ::RubyLLM::Models.stub(:resolve, capturing_resolve) do + provider = ActiveAgent::Providers::RubyLLMProvider.new( + service: "RubyLLM", + model: "gemini-2.5-flash", + platform: :vertexai, + messages: [ { role: "user", content: "hello" } ] + ) + + provider.prompt + end + + assert_equal :vertexai, resolve_kwargs[:provider] + end + + test "platform option accepts a string" do + resolve_kwargs = nil + capturing_resolve = ->(model_id, **kwargs) { + resolve_kwargs = kwargs + [ stub_model_info(model_id), ::RubyLLM::StubProvider.new ] + } + + ::RubyLLM::Models.stub(:resolve, capturing_resolve) do + provider = ActiveAgent::Providers::RubyLLMProvider.new( + service: "RubyLLM", + model: "gemini-2.5-flash", + platform: "vertexai", + messages: [ { role: "user", content: "hello" } ] + ) + + provider.prompt + end + + assert_equal :vertexai, resolve_kwargs[:provider] + end + + test "model routing is unchanged when platform is not set" do + resolve_kwargs = nil + capturing_resolve = ->(model_id, **kwargs) { + resolve_kwargs = kwargs + [ stub_model_info(model_id), ::RubyLLM::StubProvider.new ] + } + + ::RubyLLM::Models.stub(:resolve, capturing_resolve) do + provider = ActiveAgent::Providers::RubyLLMProvider.new( + service: "RubyLLM", + model: "gpt-4o-mini", + messages: [ { role: "user", content: "hello" } ] + ) + + provider.prompt + end + + assert_nil resolve_kwargs[:provider] + end + + test "platform option pins the RubyLLM backend for embeddings" do + resolve_kwargs = nil + capturing_resolve = ->(model_id, **kwargs) { + resolve_kwargs = kwargs + [ stub_model_info(model_id), ::RubyLLM::StubProvider.new ] + } + + ::RubyLLM::Models.stub(:resolve, capturing_resolve) do + provider = ActiveAgent::Providers::RubyLLMProvider.new( + service: "RubyLLM", + input: "test text", + model: "text-embedding-004", + platform: :vertexai + ) + + provider.embed + end + + assert_equal :vertexai, resolve_kwargs[:provider] + end + + test "platform set via generate_with reaches the provider options" do + agent_class = Class.new(ApplicationAgent) do + def self.name = "PlatformProbeAgent" + generate_with :ruby_llm, model: "gemini-2.5-flash", platform: :vertexai + + def ping + prompt(message: "hello") + end + end + + agent = agent_class.new + agent.params = {} + agent.process(:ping) + parameters = agent.send(:prepare_prompt_parameters) + + assert_equal :vertexai, parameters[:platform] + + provider = agent.prompt_provider_klass.new(**parameters) + assert_equal "vertexai", provider.options.platform + end + # --- stop_reason from RubyLLM response --- test "stop_reason from RubyLLM response is preserved" do