diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..671d4eb --- /dev/null +++ b/.env.example @@ -0,0 +1,31 @@ +## If you don't want to use multi-model routing +## set ENABLE_ROUTER to false, and define the following variables +## the model needs to support function calling +ENABLE_ROUTER=false +OPENAI_API_KEY="" +OPENAI_BASE_URL="" +OPENAI_MODEL="" + + +## If you want to use multi-model routing, set ENABLE_ROUTER to true +# ENABLE_ROUTER=true + +## Define the model for the tool agent, the model needs to support function calling +# TOOL_AGENT_API_KEY="" +# TOOL_AGENT_BASE_URL="" +# TOOL_AGENT_MODEL="" + +## Define the model for the coder agent +# CODER_AGENT_API_KEY="" +# CODER_AGENT_BASE_URL="" +# CODER_AGENT_MODEL="" + +## Define the model for the thinker agent, using a model that supports reasoning will yield better results +# THINK_AGENT_API_KEY="" +# THINK_AGENT_BASE_URL="" +# THINK_AGENT_MODEL="" + +## Define the model for the router agent, this model is the entry point for each request, it will consume a lot of tokens, please choose a small model to reduce costs +# ROUTER_AGENT_API_KEY="" +# ROUTER_AGENT_BASE_URL="" +# ROUTER_AGENT_MODEL="" diff --git a/README.md b/README.md index 2f3ef4d..9bd1e90 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,8 @@ ## Implemented +- [x] Mormal Mode and Router Mode + - [x] Using the qwen2.5-coder-3b-instruct model as the routing dispatcher (since it’s currently free on Alibaba Cloud’s official website) - [x] Using the qwen-max-0125 model as the tool invoker @@ -42,6 +44,7 @@ npm i ```shell # Alternatively, you can create an .env file in the repo directory +# You can refer to the .env.example file to create the .env file ## disable router ENABLE_ROUTER=false @@ -79,3 +82,22 @@ export ANTHROPIC_BASE_URL="http://127.0.0.1:3456" export API_TIMEOUT_MS=600000 claude ``` + +## Normal Mode + +The initial version uses a single model to accomplish all tasks. This model needs to support function calling and must allow for a sufficiently large tool description length, ideally greater than 1754. If the model used in this mode does not support KV Cache, it will consume a significant number of tokens. + +![normal mode](https://github.com/musistudio/claude-code-reverse/blob/main/screenshoots/normal.png) + +## Router Mode + +Using multiple models to handle different tasks, this mode requires setting ENABLE_ROUTER to true and configuring four models: ROUTER_AGENT_MODEL, TOOL_AGENT_MODEL, CODER_AGENT_MODEL, and THINK_AGENT_MODEL. + +ROUTER_AGENT_MODEL does not require high intelligence and is only responsible for request routing. A small model is sufficient for this task (testing has shown that the qwen-coder-3b model performs well). +TOOL_AGENT_MODEL must support function calling and allow for a sufficiently large tool description length, ideally greater than 1754. If the model used in this mode does not support KV Cache, it will consume a significant number of tokens. + +CODER_AGENT_MODEL and THINK_AGENT_MODEL can use the DeepSeek series of models. + +The purpose of router mode is to separate tool invocation from coding tasks, enabling the use of inference models like r1, which do not support function calling. + +![router mode](https://github.com/musistudio/claude-code-reverse/blob/main/screenshoots/router.png) diff --git a/index.mjs b/index.mjs index 275b4c0..ebc1282 100644 --- a/index.mjs +++ b/index.mjs @@ -321,7 +321,7 @@ async function initializeClaudeConfig() { async function run() { await initializeClaudeConfig(); - app.listen(port, () => { + app.listen(port, "127.0.0.1", () => { console.log(`Example app listening on port ${port}`); }); } diff --git a/package.json b/package.json index 9b4088f..fe18595 100644 --- a/package.json +++ b/package.json @@ -1,11 +1,11 @@ { - "name": "claude-code-reverse", + "name": "claude-code-router", "version": "1.0.0", "description": "You can switch the API endpoint by modifying the ANTHROPIC_BASE_URL environment variable.", - "main": "server.js", + "main": "index.mjs", "scripts": { "test": "echo \"Error: no test specified\" && exit 1", - "start": "node server.js" + "start": "node index.mjs" }, "keywords": [], "author": "", diff --git a/screenshoots/normal.png b/screenshoots/normal.png new file mode 100644 index 0000000..80796c9 Binary files /dev/null and b/screenshoots/normal.png differ diff --git a/screenshoots/router.png b/screenshoots/router.png new file mode 100644 index 0000000..3ad6cba Binary files /dev/null and b/screenshoots/router.png differ