Python servers crash on startup with OpenAI SDK connection errors

Issue

Three critical components of the FastMCP 2.0 system—specifically generator.py, evaluator.py, and orchestrator.py—failed to start due to improper dependency usage and integration issues. Key errors included:

• Runtime errors related to connection failures during initialization.
• Missing module errors for openai in production environments.
• I/O operation failures on closed files during server shutdown.
• Test suite failures due to experimental sampling handler usage.

These issues prevented any language model (LLM)-related functionality from operating correctly, causing the entire system to become unusable.

Culprit

The problems stemmed from violations of FastMCP 2.0 architecture guidelines:

1. Improper dependency usage: Direct imports of the openai SDK (which added unnecessary bloat) were used instead of adhering to FastMCP’s requirement for httpx.AsyncClient.
2. Incorrect API integration: The system used OpenAI() and AsyncOpenAI() clients directly, despite OpenRouter providing an OpenAI-compatible endpoint (/chat/completions) that could be accessed via standard HTTP.
3. Shutdown violations: The system attempted to print to stdout during the asynchronous client closure, causing I/O errors.
4. Experimental code usage: Undocumented and untested code patterns, such as OpenAISamplingHandler, were employed in test suites.

The FastMCP documentation explicitly mandates the use of httpx.AsyncClient for all external HTTP calls, particularly for interacting with OpenRouter.

Solution

To resolve these issues, the system was refactored to comply with FastMCP’s architecture guidelines:

Pattern Migration

All OpenAI SDK usage was replaced with FastMCP-compliant httpx.AsyncClient implementations. This included:

• Setting up the HTTP client with appropriate headers for OpenRouter, such as Authorization, HTTP-Referer, and X-Title.
• Configuring the client with a timeout to prevent indefinite hangs.

Lifespan Management

Proper asynchronous context management was implemented to ensure clean shutdowns and prevent I/O errors:

• Used asynccontextmanager to manage the lifecycle of the HTTP client.
• Ensured silent closure of the client to avoid printing to stdout during shutdown.

File-Specific Changes

1. generator.py: Replaced the OpenAI() client with httpx.AsyncClient in the lifespan function and updated the LLM calling logic to use HTTP requests.
2. evaluator.py: Fixed shutdown errors by removing print statements during client closure and migrated all LLM calls to use the new httpx.AsyncClient pattern.
3. orchestrator.py: Replaced AsyncOpenAI() with httpx.AsyncClient in functions like analyze_content_themes() and added proper async context management.
4. Dependency cleanup: Removed the openai SDK from the dependency list, resulting in a significant reduction in package size and installation overhead.

Verification

The refactored system was thoroughly tested to ensure:

• Compilation: All Python files compile without any references to the OpenAI SDK.
• Startup: All servers initialize successfully within a two-second timeout.
• Dependency checks: Zero OpenAI SDK references remain in the codebase.
• Functionality: The OpenRouter API calls function identically via httpx.AsyncClient, maintaining the same level of performance and reliability.

This solution ensures compliance with FastMCP 2.0 architecture standards, eliminates dependency-related errors, and maintains full compatibility with the OpenRouter API while reducing overhead.