Content
# NewLife.AI
[](https://www.nuget.org/packages/NewLife.AI/)
[](https://www.nuget.org/packages/NewLife.AI/)
[](https://dotnet.microsoft.com)
[](LICENSE)
**NewLife.AI** is a unified AI gateway base library that supports the access and orchestration of 48 mainstream AI service providers.
**NewLife.ChatAI** is a complete AI dialogue application built based on NewLife.AI, providing a conversational web front-end, API gateway routing, usage statistics, and other business capabilities.
## Core Features
- **Support for 48 Service Providers**: OpenAI, Azure OpenAI, AWS Bedrock, Anthropic (Claude), Google Gemini, Alibaba Bailian (Tongyi Qianwen), DeepSeek, Kimi, DouBao, Zhipu, Baidu, Xunfei, Tencent Hunyuan, Ollama (local), etc.
- **Unified Interface `IChatClient`**: Shields protocol differences, stream and non-stream dual modes, benchmarking Microsoft MEAI specifications
- **`AiClientRegistry` Automatic Registration**: Through `[AiClient]` attribute tagging, reflection scanning automatically discovers all service providers
- **`ChatClientBuilder` Middleware Pipeline**: `UseFilters()` + `UseTools()` chain assembly, flexible expansion
- **Native Function Calling**: `[ToolDescription]` attribute + `ToolSchemaBuilder` automatically generates JSON Schema, `ToolChatClient` multi-round call loop
- **MCP Protocol Support**: `HttpMcpServer` tool call, `NewLife.AI.Extensions` quickly extends ASP.NET applications to MCP Server
- **Thinking Mode**: Auto / Think / Fast three gears, supporting staggered thinking (think–tool–think)
- **Multi-modal**: Image understanding (Vision), image generation, image editing (Inpainting)
- **Planner**: `FunctionCallingPlanner` breaks down goals into tool call steps and executes
- **MultiAgent Framework**: `GroupChat` multi-agent polling, `ParallelGroupChat` parallel collaboration, `AgentAsTool` nesting
- **Semantic Memory Abstraction**: `ISemanticMemory` + `IVectorStore` interface, memory version implementation out of the box
## Supported Protocols and Service Providers
### Communication Protocols (6 types)
| Protocol | Client Class | Applicable Service Providers | Authentication Method |
|------|---------|---------|---------|
| **OpenAI** | `OpenAiChatClient` | ~44 OpenAI compatible service providers | `Authorization: Bearer` |
| **Azure OpenAI** | `AzureAIChatClient` | Azure OpenAI (deployment name URL) | `api-key` request header |
| **Anthropic** | `AnthropicChatClient` | Anthropic Claude series | `x-api-key` request header |
| **Google Gemini** | `GeminiChatClient` | Google Gemini series | `?key=` query parameter |
| **AWS Bedrock** | `BedrockChatClient` | AWS Bedrock Converse API | AWS SigV4 signature |
| **DashScope** | `DashScopeChatClient` | Alibaba Bailian (compatible with OpenAI routing) | `Authorization: Bearer` |
| **Ollama** | `OllamaChatClient` | Local Ollama service | No authentication required |
### Domestic Service Providers (18)
| Code | Service Provider | Representative Model | Features |
|------|-------|---------|------|
| `DeepSeek` | DeepSeek | DeepSeek-V3 / R1 | Strong reasoning, open source friendly |
| `DashScope` | Alibaba Bailian / Tongyi Qianwen | Qwen-Max / Qwen-Plus | Alibaba cloud ecosystem, most comprehensive models |
| `VolcEngine` | ByteDance Volcano (enterprise version) | DouBao 1.5 Pro | ByteDance enterprise entrance |
| `Doubao` | Douyin DouBao (consumer version) | — | ByteDance consumer entrance |
| `Zhipu` | Zhipu AI | GLM-4 / CogView-3 | Code + image generation |
| `Moonshot` | Moonshot Kimi | Kimi-K1.5 / 128K | Ultra-long context, web reading |
| `Hunyuan` | Tencent Hunyuan | Hunyuan T1 | Tencent cloud ecosystem |
| `Qianfan` | Baidu Wenxin Qianfan | ERNIE 4.5 Turbo | Baidu search ecosystem |
| `Spark` | Xunfei Spark | Spark 4.0 Ultra | Voice + education scenarios |
| `Stepfun` | Jieyue Xingchen | Step-2 | Strong reasoning, multi-modal |
| `Baichuan` | Baichuan Intelligence | Baichuan4 Turbo | Medical / legal specialized |
| `SenseNova` | Shangtang Rixin | SenseNova series | Shangtang visual AI fusion |
| `MiniMax` | MiniMax | MiniMax-Text-01 | Ultra-long context |
| `SiliconFlow` | Silicon Flow | Qwen / DeepSeek hosting | Domestic open source model reasoning platform |
| `Yi` | Zero One Thousand Things | Yi-Large | Li Kaifu's company |
| `MiMo` | Xiaomi MiMo | MiMo series | Xiaomi reasoning model |
| `Infini` | Wuwu Xinqiong | Open source model hosting | Domestic AI computing power platform |
| `XiaomaPower` | Xiaoma Computing Power | Open source model hosting | GPU computing power platform |
### International Mainstream Service Providers (7)
| Code | Service Provider | Representative Model | Features |
|------|-------|---------|------|
| `OpenAI` | OpenAI | GPT-4o / o3 / o4-mini | Industry standard, most mature function call |
| `AzureAI` | Azure OpenAI | GPT-4o deployment | Enterprise compliance, private deployment |
| `Anthropic` | Anthropic Claude | Claude 3.7 Sonnet | Long text reasoning, security alignment |
| `Gemini` | Google Gemini | Gemini 2.0 Flash | Multi-modal, code execution |
| `Bedrock` | AWS Bedrock | Claude / Llama / Nova | Cloud-native, SigV4 authentication |
| `XAI` | xAI Grok | Grok-3 / Grok-3 Mini | Musk's company, real-time information |
| `Mistral` | Mistral AI | Mistral Large | EU compliance, efficient model |
### International Aggregation / Emerging Platforms (16)
| Code | Service Provider | Features |
|------|-------|------|
| `OpenRouter` | OpenRouter | Unified routing 300+ models, price comparison |
| `GitHubModels` | GitHub Models | GitHub built-in model market |
| `HuggingFace` | Hugging Face | Thousands of open source model reasoning routing |
| `NvidiaNIM` | Nvidia NIM | GPU-specific reasoning microservices, Llama / Nemotron |
| `Groq` | Groq | Extremely fast reasoning (LPU chip), Llama/Gemma |
| `Cerebras` | Cerebras | Wafer-level chip, ultra-high throughput |
| `Perplexity` | Perplexity | Search-enhanced reasoning |
| `Cohere` | Cohere | Enterprise RAG optimization, Command series |
| `TogetherAI` | Together AI | Open source model aggregation platform |
| `Fireworks` | Fireworks AI | Production-level open source model hosting |
| `SambaNova` | SambaNova | RDU architecture, Llama series ultra-fast reasoning |
| `DeepInfra` | DeepInfra | Economical open source model reasoning |
| `Hyperbolic` | Hyperbolic | Decentralized GPU reasoning |
| `NovitaAI` | Novita AI | Image + text multi-model platform |
| `AI21` | AI21 Labs | Jamba (SSM+Transformer hybrid architecture) |
| `CloudflareAI` | Cloudflare AI | Workers AI, edge reasoning |
### Local / Private Deployment (4)
| Code | Tool | Features |
|------|-----|------|
| `Ollama` | Ollama | One-click local running Llama/Qwen/Gemma |
| `LMStudio` | LM Studio | Desktop GUI management local model |
| `vLLM` | vLLM | High-throughput production-level reasoning engine |
| `OneAPI` | OneAPI | Open source LLM API management and distribution |
> ★ represents the most commonly used service provider in each category. For a complete list of service providers, see [BuiltinChatClient.cs](NewLife.AI/Clients/BuiltinChatClient.cs).
## Quick Start
### Install NuGet Package
```bash
# Core base library
dotnet add package NewLife.AI
# ASP.NET Core DI extension (optional)
dotnet add package NewLife.AI.Extensions
```
### IChatClient Four Usage Modes
#### Mode 1: Direct `new` (command line / script / single file, simplest)
```csharp
// All 6 clients provide (apiKey, model?, endpoint?) convenient construction
var client = new DashScopeChatClient("your-api-key", "qwen-plus");
// Optional: replace underlying HttpClient (set proxy, timeout, etc.)
client.HttpClient = new HttpClient { Timeout = TimeSpan.FromSeconds(30) };
// Send a single message
var reply = await client.ChatAsync("Hello, please introduce yourself");
Console.WriteLine(reply);
// Multi-role message (tuple array, no need to construct ChatMessage)
var reply2 = await client.ChatAsync([
("system", "You are a professional C# development assistant"),
("user", "Please explain what dependency injection is"),
]);
```
Available convenient constructions:
| Client | Description |
|--------|------|
| `new OpenAIChatClient(apiKey, model?, endpoint?)` | OpenAI and all compatible service providers (~30) |
| `new DashScopeChatClient(apiKey, model?, endpoint?)` | Alibaba Bailian |
| `new AnthropicChatClient(apiKey, model?, endpoint?)` | Claude |
| `new GeminiChatClient(apiKey, model?, endpoint?)` | Google Gemini |
| `new AzureAIChatClient(apiKey, model?, endpoint?)` | Azure OpenAI (deployment name as model) |
| `new BedrockChatClient(accessKeyId, secretAccessKey, model?, region?)` | AWS Bedrock (SigV4 signature authentication) |
| `new OllamaChatClient(apiKey?, model?, endpoint?)` | Ollama (local deployment apiKey can be null) |
| `new NewLifeAIChatClient(apiKey, model?, endpoint?)` | Cascading NewLife.AI instance |
#### Mode 2: AiClientRegistry (configuration-driven / dynamic switching service provider)
```csharp
// Create through service provider code + options (suitable for reading parameters from database/configuration file)
var client = AiClientRegistry.Default.CreateClient("DashScope", "your-api-key", "qwen-plus");
// Create in batches from AiClientOptions (including Code property)
var opts = new AiClientOptions { Code = "OpenAI", ApiKey = "sk-xxx", Model = "gpt-4o" };
var client2 = AiClientRegistry.Default.CreateClient(opts.Code!, opts);
```
#### Mode 3: ChatClientBuilder (middleware pipeline)
```csharp
// MEAI style: set service provider first, then chain middleware
var client = new ChatClientBuilder()
.UseDashScope("your-api-key", "qwen-plus") // UseOpenAI / UseAnthropic / UseGemini / UseOllama / UseNewLifeAI
.UseFilters(new MyAuditFilter()) // Log / audit / rate limit (IChatFilter implementation)
.UseTools(toolRegistry) // Automatic multi-round Function Calling
.Build();
```
#### Mode 4: DI Injection (ASP.NET Core, requires `NewLife.AI.Extensions`)
```csharp
// Single service provider - dedicated convenient registration (recommended)
services.AddDashScope("your-api-key", "qwen-plus");
// Inject: IChatClient
// Multiple service providers (.NET 8+, Keyed Services)
services.AddKeyedDashScope("fast", "sk-xxx", "qwen3.5-flash");
services.AddKeyedOpenAI ("strong", "sk-xxx", "gpt-4o");
// Inject: [FromKeyedServices("fast")] IChatClient fastClient
```
All dedicated DI methods: `AddOpenAI` / `AddDashScope` / `AddAnthropic` / `AddGemini` / `AddAzureAI` / `AddBedrock` / `AddOllama` / `AddNewLifeAI`, all have corresponding `AddKeyed*` variants (.NET 8+).
#### Multi-modal Streaming Output (Visual Understanding)
```csharp
var message = new ChatMessage
{
Role = "user",
Contents = [
new ImageContent { Uri = "https://example.com/image.jpg" },
new TextContent("Please describe the content of this image"),
]
};
await foreach (var chunk in client.GetStreamingResponseAsync([message]))
{
var text = chunk.Text;
if (!String.IsNullOrEmpty(text))
Console.Write(text);
}
```
#### Custom Tool (native .NET method)
```csharp
// Use [ToolDescription] to mark tool methods, ToolRegistry automatically scans and registers through reflection
public class WeatherService
{
[ToolDescription("get_weather")]
public async Task<String> GetWeatherAsync(
[Description("City name")] String city)
{
return $"{city} is sunny today, 25°C";
}
}
// Non-DI scenario: directly construct
var toolRegistry = new ToolRegistry();
toolRegistry.AddTools(new WeatherService());
// DI scenario: register as IToolProvider
services.AddSingleton<IToolProvider>(_ =>
{
var registry = new ToolRegistry();
registry.AddTools(new WeatherService());
return registry;
});
```
#### MultiAgent Collaboration
```csharp
var toolRegistry = new ToolRegistry();
toolRegistry.AddTools(new WebSearchService());
var researcher = new ConversableAgent("researcher", researchClient)
{
Tools = [..toolRegistry.Tools],
};
var writer = new ConversableAgent("writer", writingClient);
// Default use RoundRobinSelector polling scheduling
var groupChat = new GroupChat([researcher, writer]);
await foreach (var msg in groupChat.RunAsync(new TextMessage { Content = "Analyze the AI industry trend and write a report" }))
{
if (msg is TextMessage text)
Console.WriteLine($"[{msg.Source}] {text.Content}");
}
```
```text
NewLife.AI.sln
├── NewLife.AI/ # Core foundation library (netstandard2.1)
│ ├── Clients/ # IChatClient implementation (OpenAI/Anthropic/Gemini, etc.)
│ ├── Providers/ # AiClientRegistry + ChatClientBuilder
│ ├── Filters/ # IChatFilter filter system
│ ├── Tools/ # Tool registration + built-in tools (search/weather/translation)
│ ├── Agents/ # MultiAgent framework
│ ├── Planner/ # FunctionCallingPlanner
│ ├── Memory/ # ISemanticMemory + IVectorStore
│ └── ModelContextProtocol/ # MCP protocol implementation
│
├── NewLife.AI.Extensions/ # ASP.NET Core extensions (net6/8/10)
│ └── AspNetMcpServer.cs # Extend ASP.NET application to MCP Server
│
├── NewLife.ChatAI/ # Complete web application (net8/10)
│ ├── Controllers/ # 16 API controllers
│ ├── Services/ # Business service layer
│ ├── Entity/ # XCode entity classes (8 tables)
│ └── wwwroot/ # (embedded in DLL)
│
└── Web/ # Frontend source code (React 19 + TypeScript + Vite)
```
---
## NewLife.ChatAI Complete Application
### Main Features
| Feature | Description |
|------|------|
| conversation | SSE streaming output, visualization, three modes: thinking/quick/auto |
| Skill system | Built user-defined skills, @ input completion selection, prompt word automatic injection into context |
| User memory | Automatic extraction of memories from conversations and injection into context, supporting manual management of memory entries |
| Recommended questions | Welcome page displays guiding recommended questions, assisting users in getting |
| Tool calling | Function MCP tools, frontend folding displayBlock |
| Image multimodal | understanding (drag/drop/paste), text-to-image, image editing (Inpainting) |
| API gateway | Compatible with OpenAI/Anthropic/Gemini standard protocols, AppKey authentication |
| Session management | Pinning, grouping, keyword full-text search, sharing links |
| Usage statistics | Token consumption statistics by user/AppKey dimension |
| Management backend | Based on NewLife.Cube, providing model configuration, service provider management, user management |
### Deployment Methods
#### Independent Executable (Recommended)
```bash
# Start from release package
dotnet NewLife.ChatAI.dll
# Or run directly
./NewLife.ChatAI
```
#### Run from Source Code
```bash
git clone https://github.com/NewLifeX/NewLife.AI.git
cd NewLife.AI
cd Web
pnpm install
pnpm build
cd ../NewLife.ChatAI
dotnet build
dotnet run --framework net8.0
```
Access `http://localhost:5000` after application startup. Configure service providers and models through the Cube management backend (`/Admin first time.
### APIpoints
| Endpoint | Protocol | Description |
|------|------|------ `POST /v1/chat/completions` | OpenAI | Chat supporting streaming/function calling/vision |
| `POST /v1/res | OpenAI Responses | Reasoning models (o3/gpt- etc.)|
| `v1/messages` | Anthropic | Claude series |
| `POST/gemini` | | Gemini series |
| /v1/images/generations` | OpenAI | Text-to-image |
| `GET /v1/models` | OpenAI | Available model list |
All gateway endpoints are authenticated with `Authorization: Bearer sk- AppKey.
---
## Custom Extensions
### Add AI Service Providers
```csharpAiClient("", "MyAI", "https.myai.com/v1")]
[AiClientModel("myatest", FunctionCalling = true = true)]
public class MyAiChatClient : DelegatingChatClient
{
// Implement protocol conversion}
// AiClientRegistry automatically discovers and registers at startup, no manual
```
### Add IChatFilter
```csharp
public classFilter : IChatFilter
{
// Onion ring mode: process responses before and after calling public async Task OnChatAsync(ChatFilterContext ctx,
Func<ChatFilterContext, CancellationToken, Task> next,
Cancellation)
{
// before detection, sensitive word filtering (can modify ctx.Request)
await next(ctx, ct);
// after — read ctx.Response, write audit log
}
// Callback after stream completion, can trigger self-learning analysis
public async Task OnStreamCompletedAsync(ChatFilterContext ctx, CancellationToken ct)
{
// Audit log, learning trigger
}
}
```
---
## Documentation
| Document | Description |
|------|------|
| [Requirements Specification](Doc/Requirements-Specification.md) | Complete, including SSE protocol, details |
| [Architecture Design](Doc/Architecture-Design.md System architecture, data model, specification |
| [API](-Gateway- | Detailed API gateway design [AI Orchestration](Doc/AI-Orchestration-Framework-Requirements.md) | Planner/MultiAgent/Memory detailed design |
---
## Dependencies
- **NewLife.Core**: Basic tool library (string extensions, configuration, serialization)
- **NewLife.Cube** (ChatAI application): Web framework, providing permissions, management backend, configuration center
- **XCode** (ChatAI application): ORM database framework, supporting SQLite/MySQL/SQL Server, etc.
---
## License
MIT © NewLife Dev Team
