Support multiple plugins

use the router to dispatch different models: background,longcontext and think
release v1.0.3
2025-06-15 16:58:11 +08:00 · 2025-06-14 19:48:29 +08:00 · 2025-06-13 06:23:45 +08:00 · 2025-06-12 09:58:05 +08:00 · 2025-06-10 21:43:01 +08:00 · 2025-06-10 16:46:14 +08:00
23 changed files with 434 additions and 319 deletions
--- a/.dockerignore
+++ b/.dockerignore
@@ -0,0 +1,2 @@
+node_modules
+npm-debug.log
--- a/21
+++ b/21
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 musistudio
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
--- a/README.md
+++ b/README.md
@@ -1,6 +1,9 @@
 # Claude Code Router

-> This is a repository for testing routing Claude Code requests to different models.
+> This is a tool for routing Claude Code requests to different models, and you can customize any request.
+
+
+![](screenshoots/claude-code.png)

 ## Usage

@@ -23,7 +26,7 @@ ccr code
 ```


-## Plugin
+## Plugin[Beta]

 The plugin allows users to rewrite Claude Code prompt and custom router. The plugin path is in `$HOME/.claude-code-router/plugins`. Currently, there are two demos available: 
 1. [custom router](https://github.com/musistudio/claude-code-router/blob/dev/custom-prompt/plugins/deepseek.js)
@@ -43,5 +46,20 @@ You need to move them to the `$HOME/.claude-code-router/plugins` directory and c

 ## Features
 - [x] Plugins
- [] Support change models
- [] Suport scheduled tasks
+- [ ] Support change models
+- [ ] Support scheduled tasks
+
+
+## Some tips:
+If you’re using the DeepSeek API provided by the official website, you might encounter an “exceeding context” error after several rounds of conversation (since the official API only supports a 64K context window). In this case, you’ll need to discard the previous context and start fresh. Alternatively, you can use ByteDance’s DeepSeek API, which offers a 128K context window and supports KV cache.
+
+![](screenshoots/contexterror.jpg)
+
+Note: claude code consumes a huge amount of tokens, but thanks to DeepSeek’s low cost, you can use claude code at a fraction of Claude’s price, and you don’t need to subscribe to the Claude Max plan.
+
+Some interesting points: Based on my testing, including a lot of context information can help narrow the performance gap between these LLM models. For instance, when I used Claude-4 in VSCode Copilot to handle a Flutter issue, it messed up the files in three rounds of conversation, and I had to roll everything back. However, when I used claude code with DeepSeek, after three or four rounds of conversation, I finally managed to complete my task—and the cost was less than 1 RMB!
+
+
+## Buy me a coffee
+If you find this project helpful, you can choose to sponsor the author with a cup of coffee.
+[Buy me a coffee](http://paypal.me/musistudio1999)
--- a/config.json
+++ b/config.json
@@ -3,5 +3,6 @@
  "LOG": true,
  "OPENAI_API_KEY": "",
  "OPENAI_BASE_URL": "",
-  "OPENAI_MODEL": ""
-}
+  "OPENAI_MODEL": "",
+  "modelProviders": {}
+}
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -0,0 +1,13 @@
+version: "3.8"
+
+services:
+  claude-code-reverse:
+    build: .
+    ports:
+      - "3456:3456"
+    environment:
+      - ENABLE_ROUTER=${ENABLE_ROUTER}
+      - OPENAI_API_KEY=${OPENAI_API_KEY}
+      - OPENAI_BASE_URL=${OPENAI_BASE_URL}
+      - OPENAI_MODEL=${OPENAI_MODEL}
+    restart: unless-stopped
--- a/12
+++ b/12
@@ -0,0 +1,12 @@
+FROM node:20-alpine
+
+WORKDIR /app
+
+COPY package*.json ./
+RUN npm i
+
+COPY . .
+
+EXPOSE 3456
+
+CMD ["node", "index.mjs"]
--- a/package.json
+++ b/package.json
@@ -1,14 +1,21 @@
 {
  "name": "@musistudio/claude-code-router",
-  "version": "1.0.0",
+  "version": "1.0.3",
  "description": "Use Claude Code without an Anthropics account and route it to another LLM provider",
  "bin": {
    "ccr": "./dist/cli.js"
  },
  "scripts": {
-    "build": "esbuild src/cli.ts --bundle --platform=node --outfile=dist/cli.js"
+    "build": "esbuild src/cli.ts --bundle --platform=node --outfile=dist/cli.js",
+    "buildserver": "esbuild src/index.ts --bundle --platform=node --outfile=dist/index.js"
  },
-  "keywords": ["claude", "code", "router", "llm", "anthropic"],
+  "keywords": [
+    "claude",
+    "code",
+    "router",
+    "llm",
+    "anthropic"
+  ],
  "author": "musistudio",
  "license": "MIT",
  "dependencies": {
@@ -16,7 +23,10 @@
    "dotenv": "^16.4.7",
    "express": "^4.21.2",
    "https-proxy-agent": "^7.0.6",
-    "openai": "^4.85.4"
+    "lru-cache": "^11.1.0",
+    "openai": "^4.85.4",
+    "tiktoken": "^1.0.21",
+    "uuid": "^11.1.0"
  },
  "devDependencies": {
    "@types/express": "^5.0.0",
--- a/plugins/deepseek.js
+++ b/plugins/deepseek.js
@@ -1,139 +0,0 @@
-const {
-  log,
-  streamOpenAIResponse,
-  createClient,
-} = require("claude-code-router");
-
-const thinkRouter = {
-  name: "think",
-  description: `This agent is used solely for complex reasoning and thinking tasks. It should not be called for information retrieval or repetitive, frequent requests. Only use this agent for tasks that require deep analysis or problem-solving. If there is an existing result from the Thinker agent, do not call this agent again. You are only responsible for deep thinking to break down tasks, no coding or tool calls are needed. Finally, return the broken-down steps in order, for example:\n1. xxx\n2. xxx\n3. xxx`,
-  run(args) {
-    const client = createClient({
-      apiKey: process.env.THINK_AGENT_API_KEY,
-      baseURL: process.env.THINK_AGENT_BASE_URL,
-    });
-    const messages = JSON.parse(JSON.stringify(args.messages));
-    messages.forEach((msg) => {
-      if (Array.isArray(msg.content)) {
-        msg.content = JSON.stringify(msg.content);
-      }
-    });
-
-    let startIdx = messages.findIndex((msg) => msg.role !== "system");
-    if (startIdx === -1) startIdx = messages.length;
-
-    for (let i = startIdx; i < messages.length; i++) {
-      const expectedRole = (i - startIdx) % 2 === 0 ? "user" : "assistant";
-      messages[i].role = expectedRole;
-    }
-
-    if (
-      messages.length > 0 &&
-      messages[messages.length - 1].role === "assistant"
-    ) {
-      messages.push({
-        role: "user",
-        content:
-          "Please follow the instructions provided above to resolve the issue.",
-      });
-    }
-    delete args.tools;
-    return client.chat.completions.create({
-      ...args,
-      messages,
-      model: process.env.THINK_AGENT_MODEL,
-    });
-  },
-};
-
-class Router {
-  constructor() {
-    this.routers = [thinkRouter];
-    this.client = createClient({
-      apiKey: process.env.ROUTER_AGENT_API_KEY,
-      baseURL: process.env.ROUTER_AGENT_BASE_URL,
-    });
-  }
-  async route(args) {
-    log(`Request Router: ${JSON.stringify(args, null, 2)}`);
-    const res = await this.client.chat.completions.create({
-      ...args,
-      messages: [
-        ...args.messages,
-        {
-          role: "system",
-          content: `## **Guidelines:**  
-  - **Trigger the "think" mode when the user's request involves deep thinking, complex reasoning, or multi-step analysis.**  
-  - **Criteria:**  
-      - Involves multi-layered logical reasoning or causal analysis  
-      - Requires establishing connections or pattern recognition between different pieces of information  
-      - Involves cross-domain knowledge integration or weighing multiple possibilities  
-      - Requires creative thinking or non-direct inference  
-  ### **Special Case:**  
-  - **When the user sends "test", respond with "success" only.**  
-  
-  ### **Format requirements:**  
-  - When you need to trigger the "think" mode, return the following JSON format:  
-  \`\`\`json
-  {
-    "use": "think"
-  }
-  \`\`\`
-  `,
-        },
-      ],
-      model: process.env.ROUTER_AGENT_MODEL,
-      stream: false,
-    });
-    let result;
-    try {
-      const text = res.choices[0].message.content;
-      if (!text) {
-        throw new Error("No text");
-      }
-      result = JSON.parse(
-        text.slice(text.indexOf("{"), text.lastIndexOf("}") + 1)
-      );
-    } catch (e) {
-      res.choices[0].delta = res.choices[0].message;
-      log(`No Router: ${JSON.stringify(res.choices[0].message)}`);
-      return [res];
-    }
-    const router = this.routers.find((item) => item.name === result.use);
-    if (!router) {
-      res.choices[0].delta = res.choices[0].message;
-      log(`No Router: ${JSON.stringify(res.choices[0].message)}`);
-      return [res];
-    }
-    log(`Use Router: ${router.name}`);
-    if (router.name === "think") {
-      const agentResult = await router.run({
-        ...args,
-        stream: false,
-      });
-      try {
-        args.messages.push({
-          role: "user",
-          content:
-            `${router.name} Agent Result: ` +
-            agentResult.choices[0].message.content,
-        });
-        log(
-          `${router.name} Agent Result: ` +
-            agentResult.choices[0].message.content
-        );
-        return await this.route(args);
-      } catch (error) {
-        console.log(agentResult);
-        throw error;
-      }
-    }
-    return router.run(args);
-  }
-}
-
-const router = new Router();
-module.exports = async function handle(req, res, next) {
-  const completions = await router.route(req.body);
-  streamOpenAIResponse(res, completions, req.body.model);
-};
--- a/plugins/gemini.js
+++ b/plugins/gemini.js
@@ -1,23 +0,0 @@
-module.exports = async function handle(req, res, next) {
-  if (Array.isArray(req.body.tools)) {
-    // rewrite tools definition
-    req.body.tools.forEach((tool) => {
-      if (tool.function.name === "BatchTool") {
-        // HACK: Gemini does not support objects with empty properties
-        tool.function.parameters.properties.invocations.items.properties.input.type =
-          "number";
-        return;
-      }
-      Object.keys(tool.function.parameters.properties).forEach((key) => {
-        const prop = tool.function.parameters.properties[key];
-        if (
-          prop.type === "string" &&
-          !["enum", "date-time"].includes(prop.format)
-        ) {
-          delete prop.format;
-        }
-      });
-    });
-  }
-  next();
-};
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -5,9 +5,6 @@ settings:
  excludeLinksFromLockfile: false

 dependencies:
-  '@anthropic-ai/claude-code':
-    specifier: ^0.2.53
-    version: 0.2.53
  '@anthropic-ai/sdk':
    specifier: ^0.39.0
    version: 0.39.0
@@ -20,9 +17,18 @@ dependencies:
  https-proxy-agent:
    specifier: ^7.0.6
    version: 7.0.6
+  lru-cache:
+    specifier: ^11.1.0
+    version: 11.1.0
  openai:
    specifier: ^4.85.4
    version: 4.86.1
+  tiktoken:
+    specifier: ^1.0.21
+    version: 1.0.21
+  uuid:
+    specifier: ^11.1.0
+    version: 11.1.0

 devDependencies:
  '@types/express':
@@ -37,18 +43,6 @@ devDependencies:

 packages:

-  /@anthropic-ai/claude-code@0.2.53:
-    resolution: {integrity: sha512-DKXGjSsu2+rc1GaAdOjRqD7fMLvyQgwi/sqf6lLHWQAarwYxR/ahbSheu7h1Ub0wm0htnuIqgNnmNZUM43w/3Q==}
-    engines: {node: '>=18.0.0'}
-    hasBin: true
-    requiresBuild: true
-    optionalDependencies:
-      '@img/sharp-darwin-arm64': 0.33.5
-      '@img/sharp-linux-arm': 0.33.5
-      '@img/sharp-linux-x64': 0.33.5
-      '@img/sharp-win32-x64': 0.33.5
-    dev: false
-
  /@anthropic-ai/sdk@0.39.0:
    resolution: {integrity: sha512-eMyDIPRZbt1CCLErRCi3exlAvNkBtRe+kW5vvJyef93PmNr/clstYgHhtvmkxN82nlKgzyGPCyGxrm0JQ1ZIdg==}
    dependencies:
@@ -288,72 +282,6 @@ packages:
    dev: true
    optional: true

-  /@img/sharp-darwin-arm64@0.33.5:
-    resolution: {integrity: sha512-UT4p+iz/2H4twwAoLCqfA9UH5pI6DggwKEGuaPy7nCVQ8ZsiY5PIcrRvD1DzuY3qYL07NtIQcWnBSY/heikIFQ==}
-    engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0}
-    cpu: [arm64]
-    os: [darwin]
-    requiresBuild: true
-    optionalDependencies:
-      '@img/sharp-libvips-darwin-arm64': 1.0.4
-    dev: false
-    optional: true
-
-  /@img/sharp-libvips-darwin-arm64@1.0.4:
-    resolution: {integrity: sha512-XblONe153h0O2zuFfTAbQYAX2JhYmDHeWikp1LM9Hul9gVPjFY427k6dFEcOL72O01QxQsWi761svJ/ev9xEDg==}
-    cpu: [arm64]
-    os: [darwin]
-    requiresBuild: true
-    dev: false
-    optional: true
-
-  /@img/sharp-libvips-linux-arm@1.0.5:
-    resolution: {integrity: sha512-gvcC4ACAOPRNATg/ov8/MnbxFDJqf/pDePbBnuBDcjsI8PssmjoKMAz4LtLaVi+OnSb5FK/yIOamqDwGmXW32g==}
-    cpu: [arm]
-    os: [linux]
-    requiresBuild: true
-    dev: false
-    optional: true
-
-  /@img/sharp-libvips-linux-x64@1.0.4:
-    resolution: {integrity: sha512-MmWmQ3iPFZr0Iev+BAgVMb3ZyC4KeFc3jFxnNbEPas60e1cIfevbtuyf9nDGIzOaW9PdnDciJm+wFFaTlj5xYw==}
-    cpu: [x64]
-    os: [linux]
-    requiresBuild: true
-    dev: false
-    optional: true
-
-  /@img/sharp-linux-arm@0.33.5:
-    resolution: {integrity: sha512-JTS1eldqZbJxjvKaAkxhZmBqPRGmxgu+qFKSInv8moZ2AmT5Yib3EQ1c6gp493HvrvV8QgdOXdyaIBrhvFhBMQ==}
-    engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0}
-    cpu: [arm]
-    os: [linux]
-    requiresBuild: true
-    optionalDependencies:
-      '@img/sharp-libvips-linux-arm': 1.0.5
-    dev: false
-    optional: true
-
-  /@img/sharp-linux-x64@0.33.5:
-    resolution: {integrity: sha512-opC+Ok5pRNAzuvq1AG0ar+1owsu842/Ab+4qvU879ippJBHvyY5n2mxF1izXqkPYlGuP/M556uh53jRLJmzTWA==}
-    engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0}
-    cpu: [x64]
-    os: [linux]
-    requiresBuild: true
-    optionalDependencies:
-      '@img/sharp-libvips-linux-x64': 1.0.4
-    dev: false
-    optional: true
-
-  /@img/sharp-win32-x64@0.33.5:
-    resolution: {integrity: sha512-MpY/o8/8kj+EcnxwvrP4aTJSWw/aZ7JIGR4aBeZkZw5B7/Jn+tY9/VNwtcoGmdT7GfggGIU4kygOMSbYnOrAbg==}
-    engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0}
-    cpu: [x64]
-    os: [win32]
-    requiresBuild: true
-    dev: false
-    optional: true
-
  /@types/body-parser@1.19.5:
    resolution: {integrity: sha512-fB3Zu92ucau0iQ0JMCFQE7b/dv8Ot07NI3KaZIkIUNXq82k4eBAqUaneXfleGY9JWskeS9y+u0nXMyspcuQrCg==}
    dependencies:
@@ -853,6 +781,11 @@ packages:
    engines: {node: '>= 0.10'}
    dev: false

+  /lru-cache@11.1.0:
+    resolution: {integrity: sha512-QIXZUBJUx+2zHUdQujWejBkcD9+cs94tLn0+YL8UrCh+D5sCXZ4c7LaEH48pNwRY3MLDgqUFyhlCyjJPf1WP0A==}
+    engines: {node: 20 || >=22}
+    dev: false
+
  /math-intrinsics@1.1.0:
    resolution: {integrity: sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==}
    engines: {node: '>= 0.4'}
@@ -1084,6 +1017,10 @@ packages:
    engines: {node: '>= 0.8'}
    dev: false

+  /tiktoken@1.0.21:
+    resolution: {integrity: sha512-/kqtlepLMptX0OgbYD9aMYbM7EFrMZCL7EoHM8Psmg2FuhXoo/bH64KqOiZGGwa6oS9TPdSEDKBnV2LuB8+5vQ==}
+    dev: false
+
  /toidentifier@1.0.1:
    resolution: {integrity: sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA==}
    engines: {node: '>=0.6'}
@@ -1120,6 +1057,11 @@ packages:
    engines: {node: '>= 0.4.0'}
    dev: false

+  /uuid@11.1.0:
+    resolution: {integrity: sha512-0/A9rDy9P7cJ+8w1c9WD9V//9Wj15Ce2MPz8Ri6032usz+NfePxx5AcN3bN+r6ZL6jEo066/yNYB3tn4pQEx+A==}
+    hasBin: true
+    dev: false
+
  /vary@1.1.2:
    resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==}
    engines: {node: '>= 0.8'}
--- a/screenshoots/claude-code.png
+++ b/screenshoots/claude-code.png
--- a/screenshoots/contexterror.jpg
+++ b/screenshoots/contexterror.jpg
--- a/src/cli.ts
+++ b/src/cli.ts
@@ -3,7 +3,7 @@ import { run } from "./index";
 import { closeService } from "./utils/close";
 import { showStatus } from "./utils/status";
 import { executeCodeCommand } from "./utils/codeCommand";
-import { isServiceRunning } from "./utils/processCheck";
+import { cleanupPidFile, isServiceRunning } from "./utils/processCheck";
 import { version } from "../package.json";

 const command = process.argv[2];
@@ -43,13 +43,36 @@ async function waitForService(
  return false;
 }

+import { spawn } from "child_process";
+import { PID_FILE, REFERENCE_COUNT_FILE } from "./constants";
+import { existsSync, readFileSync } from "fs";
+
 async function main() {
  switch (command) {
    case "start":
-      await run({ daemon: true });
+      run();
      break;
    case "stop":
-      await closeService();
+      try {
+        const pid = parseInt(readFileSync(PID_FILE, "utf-8"));
+        process.kill(pid);
+        cleanupPidFile();
+        if (existsSync(REFERENCE_COUNT_FILE)) {
+          try {
+            require("fs").unlinkSync(REFERENCE_COUNT_FILE);
+          } catch (e) {
+            // Ignore cleanup errors
+          }
+        }
+        console.log(
+          "claude code router service has been successfully stopped."
+        );
+      } catch (e) {
+        console.log(
+          "Failed to stop the service. It may have already been stopped."
+        );
+        cleanupPidFile();
+      }
      break;
    case "status":
      showStatus();
@@ -57,8 +80,10 @@ async function main() {
    case "code":
      if (!isServiceRunning()) {
        console.log("Service not running, starting service...");
-        await run({ daemon: true });
-        // Wait for service to start, exit with error if timeout
+        spawn("ccr", ["start"], {
+          detached: true,
+          stdio: "ignore",
+        }).unref();
        if (await waitForService()) {
          executeCodeCommand(process.argv.slice(3));
        } else {
--- a/src/constants.ts
+++ b/src/constants.ts
@@ -9,6 +9,8 @@ export const PLUGINS_DIR = `${HOME_DIR}/plugins`;

 export const PID_FILE = path.join(HOME_DIR, '.claude-code-router.pid');

+export const REFERENCE_COUNT_FILE = '/tmp/claude-code-reference-count.txt';
+

 export const DEFAULT_CONFIG = {
  log: false,
--- a/src/index.ts
+++ b/src/index.ts
@@ -4,10 +4,16 @@ import { getOpenAICommonOptions, initConfig, initDir } from "./utils";
 import { createServer } from "./server";
 import { formatRequest } from "./middlewares/formatRequest";
 import { rewriteBody } from "./middlewares/rewriteBody";
+import { router } from "./middlewares/router";
 import OpenAI from "openai";
 import { streamOpenAIResponse } from "./utils/stream";
-import { isServiceRunning, savePid } from "./utils/processCheck";
-import { fork } from "child_process";
+import {
+  cleanupPidFile,
+  isServiceRunning,
+  savePid,
+} from "./utils/processCheck";
+import { LRUCache } from "lru-cache";
+import { log } from "./utils/log";

 async function initializeClaudeConfig() {
  const homeDir = process.env.HOME;
@@ -31,12 +37,16 @@ async function initializeClaudeConfig() {

 interface RunOptions {
  port?: number;
-  daemon?: boolean;
+}
+
+interface ModelProvider {
+  name: string;
+  api_base_url: string;
+  api_key: string;
+  models: string[];
 }

 async function run(options: RunOptions = {}) {
-  const port = options.port || 3456;
-
  // Check if service is already running
  if (isServiceRunning()) {
    console.log("✅ Service is already running in the background.");
@@ -45,31 +55,96 @@ async function run(options: RunOptions = {}) {

  await initializeClaudeConfig();
  await initDir();
-  await initConfig();
+  const config = await initConfig();
+
+  const Providers = new Map<string, ModelProvider>();
+  const providerCache = new LRUCache<string, OpenAI>({
+    max: 10,
+    ttl: 2 * 60 * 60 * 1000,
+  });
+
+  function getProviderInstance(providerName: string): OpenAI {
+    const provider: ModelProvider | undefined = Providers.get(providerName);
+    if (provider === undefined) {
+      throw new Error(`Provider ${providerName} not found`);
+    }
+    let openai = providerCache.get(provider.name);
+    if (!openai) {
+      openai = new OpenAI({
+        baseURL: provider.api_base_url,
+        apiKey: provider.api_key,
+        ...getOpenAICommonOptions(),
+      });
+      providerCache.set(provider.name, openai);
+    }
+    return openai;
+  }
+
+  if (Array.isArray(config.Providers)) {
+    config.Providers.forEach((provider) => {
+      try {
+        Providers.set(provider.name, provider);
+      } catch (error) {
+        console.error("Failed to parse model provider:", error);
+      }
+    });
+  }
+
+  if (config.OPENAI_API_KEY && config.OPENAI_BASE_URL && config.OPENAI_MODEL) {
+    const defaultProvider = {
+      name: "default",
+      api_base_url: config.OPENAI_BASE_URL,
+      api_key: config.OPENAI_API_KEY,
+      models: [config.OPENAI_MODEL],
+    };
+    Providers.set("default", defaultProvider);
+  } else if (Providers.size > 0) {
+    const defaultProvider = Providers.values().next().value!;
+    Providers.set("default", defaultProvider);
+  }
+  const port = options.port || 3456;

  // Save the PID of the background process
  savePid(process.pid);

+  // Handle SIGINT (Ctrl+C) to clean up PID file
+  process.on("SIGINT", () => {
+    console.log("Received SIGINT, cleaning up...");
+    cleanupPidFile();
+    process.exit(0);
+  });
+
+  // Handle SIGTERM to clean up PID file
+  process.on("SIGTERM", () => {
+    cleanupPidFile();
+    process.exit(0);
+  });
+
  // Use port from environment variable if set (for background process)
  const servicePort = process.env.SERVICE_PORT
    ? parseInt(process.env.SERVICE_PORT)
    : port;

-  const server = createServer(servicePort);
-  server.useMiddleware(formatRequest);
-  server.useMiddleware(rewriteBody);
-
-  const openai = new OpenAI({
-    apiKey: process.env.OPENAI_API_KEY,
-    baseURL: process.env.OPENAI_BASE_URL,
-    ...getOpenAICommonOptions(),
+  const server = await createServer(servicePort);
+  server.useMiddleware((req, res, next) => {
+    console.log("Middleware triggered for request:", req.body.model);
+    req.config = config;
+    next();
  });
+  server.useMiddleware(rewriteBody);
+  if (
+    config.Router?.background &&
+    config.Router?.think &&
+    config?.Router?.longContext
+  ) {
+    server.useMiddleware(router);
+  }
+  server.useMiddleware(formatRequest);
+
  server.app.post("/v1/messages", async (req, res) => {
    try {
-      if (process.env.OPENAI_MODEL) {
-        req.body.model = process.env.OPENAI_MODEL;
-      }
-      const completion: any = await openai.chat.completions.create(req.body);
+      const provider = getProviderInstance(req.provider || "default");
+      const completion: any = await provider.chat.completions.create(req.body);
      await streamOpenAIResponse(res, completion, req.body.model, req.body);
    } catch (e) {
      console.error("Error in OpenAI API call:", e);
@@ -80,3 +155,4 @@ async function run(options: RunOptions = {}) {
 }

 export { run };
+// run();
--- a/src/middlewares/formatRequest.ts
+++ b/src/middlewares/formatRequest.ts
@@ -1,5 +1,4 @@
 import { Request, Response, NextFunction } from "express";
-import { ContentBlockParam } from "@anthropic-ai/sdk/resources";
 import { MessageCreateParamsBase } from "@anthropic-ai/sdk/resources/messages";
 import OpenAI from "openai";
 import { streamOpenAIResponse } from "../utils/stream";
@@ -181,6 +180,7 @@ export const formatRequest = async (
    res.setHeader("Cache-Control", "no-cache");
    res.setHeader("Connection", "keep-alive");
    req.body = data;
+    console.log(JSON.stringify(data.messages, null, 2));
  } catch (error) {
    console.error("Error in request processing:", error);
    const errorCompletion: AsyncIterable<OpenAI.Chat.Completions.ChatCompletionChunk> =
@@ -189,7 +189,7 @@ export const formatRequest = async (
          yield {
            id: `error_${Date.now()}`,
            created: Math.floor(Date.now() / 1000),
-            model: "gpt-3.5-turbo",
+            model,
            object: "chat.completion.chunk",
            choices: [
              {
--- a/src/middlewares/rewriteBody.ts
+++ b/src/middlewares/rewriteBody.ts
@@ -28,16 +28,18 @@ export const rewriteBody = async (
  res: Response,
  next: NextFunction
 ) => {
-  if (!process.env.usePlugin) {
+  if (!req.config.usePlugins) {
    return next();
  }
-  const pluginPath = path.join(PLUGINS_DIR, `${process.env.usePlugin}.js`);
-  try {
-    await access(pluginPath);
-    const rewritePlugin = require(pluginPath);
-    rewritePlugin(req, res, next);
-  } catch (e) {
-    console.error(e);
-    next();
+  for (const plugin of req.config.usePlugins) {
+    const pluginPath = path.join(PLUGINS_DIR, `${plugin.trim()}.js`);
+    try {
+      await access(pluginPath);
+      const rewritePlugin = require(pluginPath);
+      await rewritePlugin(req, res);
+    } catch (e) {
+      console.error(e);
+    }
  }
+  next();
 };
--- a/src/middlewares/router.ts
+++ b/src/middlewares/router.ts
@@ -0,0 +1,110 @@
+import { MessageCreateParamsBase } from "@anthropic-ai/sdk/resources/messages";
+import { Request, Response, NextFunction } from "express";
+import { get_encoding } from "tiktoken";
+import { log } from "../utils/log";
+
+const enc = get_encoding("cl100k_base");
+
+const getUseModel = (req: Request, tokenCount: number) => {
+  // if tokenCount is greater than 32K, use the long context model
+  if (tokenCount > 1000 * 32) {
+    log("Using long context model due to token count:", tokenCount);
+    const [provider, model] = req.config.Router!.longContext.split(",");
+    return {
+      provider,
+      model,
+    };
+  }
+  // If the model is claude-3-5-haiku, use the background model
+  if (req.body.model?.startsWith("claude-3-5-haiku")) {
+    log("Using background model for ", req.body.model);
+    const [provider, model] = req.config.Router!.background.split(",");
+    return {
+      provider,
+      model,
+    };
+  }
+  // if exits thinking, use the think model
+  if (req.body.thinking) {
+    log("Using think model for ", req.body.thinking);
+    const [provider, model] = req.config.Router!.think.split(",");
+    return {
+      provider,
+      model,
+    };
+  }
+  const [provider, model] = req.body.model.split(",");
+  if (provider && model) {
+    return {
+      provider,
+      model,
+    };
+  }
+  return {
+    provider: "default",
+    model: req.config.OPENAI_MODEL,
+  };
+};
+
+export const router = async (
+  req: Request,
+  res: Response,
+  next: NextFunction
+) => {
+  const { messages, system = [], tools }: MessageCreateParamsBase = req.body;
+  try {
+    let tokenCount = 0;
+    if (Array.isArray(messages)) {
+      messages.forEach((message) => {
+        if (typeof message.content === "string") {
+          tokenCount += enc.encode(message.content).length;
+        } else if (Array.isArray(message.content)) {
+          message.content.forEach((contentPart) => {
+            if (contentPart.type === "text") {
+              tokenCount += enc.encode(contentPart.text).length;
+            } else if (contentPart.type === "tool_use") {
+              tokenCount += enc.encode(
+                JSON.stringify(contentPart.input)
+              ).length;
+            } else if (contentPart.type === "tool_result") {
+              tokenCount += enc.encode(contentPart.content || "").length;
+            }
+          });
+        }
+      });
+    }
+    if (typeof system === "string") {
+      tokenCount += enc.encode(system).length;
+    } else if (Array.isArray(system)) {
+      system.forEach((item) => {
+        if (item.type !== "text") return;
+        if (typeof item.text === "string") {
+          tokenCount += enc.encode(item.text).length;
+        } else if (Array.isArray(item.text)) {
+          item.text.forEach((textPart) => {
+            tokenCount += enc.encode(textPart || "").length;
+          });
+        }
+      });
+    }
+    if (tools) {
+      tools.forEach((tool) => {
+        if (tool.description) {
+          tokenCount += enc.encode(tool.name + tool.description).length;
+        }
+        if (tool.input_schema) {
+          tokenCount += enc.encode(JSON.stringify(tool.input_schema)).length;
+        }
+      });
+    }
+    const { provider, model } = getUseModel(req, tokenCount);
+    req.provider = provider;
+    req.body.model = model;
+  } catch (error) {
+    log("Error in router middleware:", error.message);
+    req.provider = "default";
+    req.body.model = req.config.OPENAI_MODEL;
+  } finally {
+    next();
+  }
+};
--- a/src/server.ts
+++ b/src/server.ts
@@ -6,7 +6,7 @@ interface Server {
  start: () => void;
 }

-export const createServer = (port: number): Server => {
+export const createServer = async (port: number): Promise<Server> => {
  const app = express();
  app.use(express.json({ limit: "500mb" }));
  return {
--- a/src/utils/close.ts
+++ b/src/utils/close.ts
@@ -1,21 +1,25 @@
-import { isServiceRunning, cleanupPidFile } from './processCheck';
-import { existsSync, readFileSync } from 'fs';
-import { homedir } from 'os';
+import { isServiceRunning, cleanupPidFile, getReferenceCount } from './processCheck';
+import { readFileSync } from 'fs';
+import { HOME_DIR } from '../constants';
 import { join } from 'path';

 export async function closeService() {
-    const PID_FILE = join(homedir(), '.claude-code-router.pid');
+    const PID_FILE = join(HOME_DIR, '.claude-code-router.pid');
    
    if (!isServiceRunning()) {
        console.log("No service is currently running.");
        return;
    }

+    if (getReferenceCount() > 0) {
+        return;
+    }
+
    try {
        const pid = parseInt(readFileSync(PID_FILE, 'utf-8'));
        process.kill(pid);
        cleanupPidFile();
-        console.log("Service has been successfully stopped.");
+        console.log("claude code router service has been successfully stopped.");
    } catch (e) {
        console.log("Failed to stop the service. It may have already been stopped.");
        cleanupPidFile();
--- a/src/utils/codeCommand.ts
+++ b/src/utils/codeCommand.ts
@@ -1,31 +1,42 @@
-import { spawn } from 'child_process';
-import { isServiceRunning } from './processCheck';
+import { spawn } from "child_process";
+import {
+  incrementReferenceCount,
+  decrementReferenceCount,
+} from "./processCheck";
+import { closeService } from "./close";

 export async function executeCodeCommand(args: string[] = []) {
-    // Service check is now handled in cli.ts
+  // Set environment variables
+  const env = {
+    ...process.env,
+    DISABLE_PROMPT_CACHING: "1",
+    ANTHROPIC_AUTH_TOKEN: "test",
+    ANTHROPIC_BASE_URL: `http://127.0.0.1:3456`,
+    API_TIMEOUT_MS: "600000",
+  };

-    // Set environment variables
-    const env = {
-        ...process.env,
-        DISABLE_PROMPT_CACHING: '1',
-        ANTHROPIC_BASE_URL: 'http://127.0.0.1:3456',
-        API_TIMEOUT_MS: '600000'
-    };
+  // Increment reference count when command starts
+  incrementReferenceCount();

-    // Execute claude command
-    const claudeProcess = spawn('claude', args, {
-        env,
-        stdio: 'inherit',
-        shell: true
-    });
+  // Execute claude command
+  const claudeProcess = spawn("claude", args, {
+    env,
+    stdio: "inherit",
+    shell: true,
+  });

-    claudeProcess.on('error', (error) => {
-        console.error('Failed to start claude command:', error.message);
-        console.log('Make sure Claude Code is installed: npm install -g @anthropic-ai/claude-code');
-        process.exit(1);
-    });
+  claudeProcess.on("error", (error) => {
+    console.error("Failed to start claude command:", error.message);
+    console.log(
+      "Make sure Claude Code is installed: npm install -g @anthropic-ai/claude-code"
+    );
+    decrementReferenceCount();
+    process.exit(1);
+  });

-    claudeProcess.on('close', (code) => {
-        process.exit(code || 0);
-    });
+  claudeProcess.on("close", (code) => {
+    decrementReferenceCount();
+    closeService();
+    process.exit(code || 0);
+  });
 }
--- a/src/utils/index.ts
+++ b/src/utils/index.ts
@@ -13,6 +13,8 @@ export function getOpenAICommonOptions(): ClientOptions {
  const options: ClientOptions = {};
  if (process.env.PROXY_URL) {
    options.httpAgent = new HttpsProxyAgent(process.env.PROXY_URL);
+  } else if (process.env.HTTPS_PROXY) {
+    options.httpAgent = new HttpsProxyAgent(process.env.HTTPS_PROXY);
  }
  return options;
 }
@@ -78,6 +80,7 @@ export const writeConfigFile = async (config: any) => {
 export const initConfig = async () => {
  const config = await readConfigFile();
  Object.assign(process.env, config);
+  return config;
 };

 export const createClient = (options: ClientOptions) => {
--- a/src/utils/processCheck.ts
+++ b/src/utils/processCheck.ts
@@ -1,6 +1,30 @@
 import { existsSync, readFileSync, writeFileSync } from 'fs';
-import { PID_FILE } from '../constants';
+import { PID_FILE, REFERENCE_COUNT_FILE } from '../constants';

+export function incrementReferenceCount() {
+    let count = 0;
+    if (existsSync(REFERENCE_COUNT_FILE)) {
+        count = parseInt(readFileSync(REFERENCE_COUNT_FILE, 'utf-8')) || 0;
+    }
+    count++;
+    writeFileSync(REFERENCE_COUNT_FILE, count.toString());
+}
+
+export function decrementReferenceCount() {
+    let count = 0;
+    if (existsSync(REFERENCE_COUNT_FILE)) {
+        count = parseInt(readFileSync(REFERENCE_COUNT_FILE, 'utf-8')) || 0;
+    }
+    count = Math.max(0, count - 1);
+    writeFileSync(REFERENCE_COUNT_FILE, count.toString());
+}
+
+export function getReferenceCount(): number {
+    if (!existsSync(REFERENCE_COUNT_FILE)) {
+        return 0;
+    }
+    return parseInt(readFileSync(REFERENCE_COUNT_FILE, 'utf-8')) || 0;
+}

 export function isServiceRunning(): boolean {
    if (!existsSync(PID_FILE)) {
@@ -55,6 +79,7 @@ export function getServiceInfo() {
        pid,
        port: 3456,
        endpoint: 'http://127.0.0.1:3456',
-        pidFile: PID_FILE
+        pidFile: PID_FILE,
+        referenceCount: getReferenceCount()
    };
 }
Author	SHA1	Message	Date
jinhui.li	5e70bc70c0	Support multiple plugins	2025-06-15 16:58:11 +08:00
jinhui.li	9a89250d79	use the router to dispatch different models: background,longcontext and think	2025-06-14 19:48:29 +08:00
musi	7a5d712444	release v1.0.3	2025-06-13 06:23:45 +08:00
jinhui.li	84e76f24b0	fix the issue of multiple calude using one server by claude code	2025-06-12 09:58:05 +08:00
musi	c9059f146d	fix miss api	2025-06-10 21:43:01 +08:00
jinhui.li	9cffebf081	add LICENSE	2025-06-10 16:46:14 +08:00
jinhui.li	111492b908	add screenshot	2025-06-10 13:30:18 +08:00
jinhui.li	edc8ecbcba	add screenshot	2025-06-10 13:28:41 +08:00
jinhui.li	ea68b2ea55	fix typo	2025-06-10 13:19:02 +08:00
jinhui.li	3b0d7bac0c	add doc	2025-06-10 13:15:36 +08:00
jinhui.li	6912572fbb	Merge branch 'feature/cli' # Conflicts: # .gitignore # index.mjs	2025-06-10 12:58:00 +08:00
musi	aa3f72f390	Merge pull request #8 from sbtobb/feature-docker-config Feature add docker config	2025-05-06 08:47:49 +08:00
TOBB	6e4022b6f1	config(gitignore): Update .gitignore to exclude common files	2025-05-05 23:38:05 +08:00
TOBB	30bf711a2a	config(docker): Add Docker configuration files	2025-05-05 23:37:00 +08:00
TOBB	2ade113c2a	refactor(index.mjs): change listen ip to 0/32 Make server listen on all network interfaces Allow external connections by binding to 0.0.0.0 instead of localhost only	2025-05-05 23:36:06 +08:00