Load unsupported model types instead of raising KeyError in AutoLigerKernelForCausalLM

[nileshpatil6]

Summary

AutoLigerKernelForCausalLM raises KeyError for any model type without a Liger patching function, instead of loading the model unpatched.

AutoLigerKernelForCausalLM.from_pretrained("gpt2")   # KeyError: 'gpt2'

Details

_apply_liger_kernel already handles this case deliberately:

if model_type not in MODEL_TYPE_TO_APPLY_LIGER_FN.keys():
    logger.info(f"There are currently no Liger kernels supported for model type: {model_type}.")
    return

But auto_model.py then indexes the same dict unconditionally, to strip the kwargs that function consumed:

_apply_liger_kernel(model_type, **kwargs)

apply_fn = MODEL_TYPE_TO_APPLY_LIGER_FN[model_type]   # KeyError for anything unsupported
apply_fn_signature = inspect.signature(apply_fn)

so the graceful path above is unreachable from the wrapper. Both from_pretrained (line 36) and from_config (line 55) do it.

This contradicts the documented contract. The class docstring says it is "a drop-in replacement for AutoModelForCausalLM that applies the Liger Kernel to the model if applicable", and the README says "If the model type is supported, the modeling code will be automatically patched." Both read as: unsupported models still load.

The intended usage is a one-line swap in a training script:

model = AutoLigerKernelForCausalLM.from_pretrained(cfg.model_path)   # was AutoModelForCausalLM

which now hard-fails the moment that script is pointed at a model Liger does not patch. It also means each new model type in transformers is a crash here until a patch function is added.

Change

An unsupported model type consumed no kwargs, so all of them belong to the underlying AutoModel call. A small helper returns the kwargs unchanged in that case, and both classmethods use it.

It also fixes from_config returning None when model_type is falsy: it logged "No Liger kernels will be applied" and then returned no model at all, so callers got an AttributeError on None downstream. The redundant model_type = config.model_type on the very next line shows the block was meant to fall through rather than bail.

Testing

Three cases added to test/transformers/test_auto_model.py, following the existing mock style. All are CPU-only.

Without the change:

src/liger_kernel/transformers/auto_model.py:36: KeyError: 'unsupported_model_type'
src/liger_kernel/transformers/auto_model.py:55: KeyError: 'unsupported_model_type'
test/transformers/test_auto_model.py:141: AssertionError: Expected 'from_config' to be called once. Called 0 times.
3 failed, 2 passed

With it:

5 passed

Also checked against a real config, no mocks:

AutoLigerKernelForCausalLM.from_config(GPT2Config()) -> GPT2LMHeadModel
same class as AutoModelForCausalLM: True

Regression, comparing a clean checkout with the patched tree over test_auto_model.py, test_monkey_patch.py, test_utils.py and test_trainer_integration.py:

clean:    89 passed, 8 skipped
patched:  92 passed, 8 skipped

The delta is exactly the three new tests; the 8 skips are GPU-gated and identical in both runs. ruff check and ruff format --check both pass.

Hardware Type: CPU only. This change touches no kernel code, so I ran the CPU-runnable suites above rather than the full GPU matrix. Happy to have CI cover the rest.