vLLM model mismatch and tool calling configuration errors in Fast-Agent

Issue

Fast-Agent encountered two critical failures during its upstream migration with vLLM:

1. A 404 Model Not Found error because vLLM served the wrong model (Qwen/Qwen3-4B-Instruct-2507-FP8) instead of the expected Llama 3.1 model.
2. A 400 BadRequestError when using tool_choice: "auto", as the required vLLM configuration flags (--enable-auto-tool-choice and --tool-call-parser) were missing.

Culprit

The issues stemmed from:

• Model Mismatch: Fast-Agent’s configuration referenced a Llama 3.1 model, while vLLM was configured to deploy a Qwen 4B model. This inconsistency was masked by an outdated integration layer that bypassed model validation.
• Missing Tool Calling Flags: vLLM’s default settings did not include the necessary flags for OpenAI-compatible tool calling. Specifically, the hermes parser (required for Llama 3.1’s JSON-based function calls) was disabled by default.

Solution

The following steps resolved the issues:

1. Model Alignment: Updated vLLM’s deployment configuration to serve the correct Llama 3.1 model with proper quantization settings, ensuring alignment with Fast-Agent’s expectations.
2. Tool Calling Enablement: Enabled the required vLLM flags (--enable-auto-tool-choice and --tool-call-parser=hermes) to support automatic tool selection and JSON-based function calling.

These changes allowed Fast-Agent to successfully execute MCP tools and process function calls as intended.