feat: optimize Docker build with multi-stage deps and npm retry config

This commit is contained in:
czlonkowski
2025-06-13 21:11:43 +02:00
parent b8b5e674dc
commit 8878c99c3b
3 changed files with 77 additions and 21 deletions

View File

@@ -66,6 +66,9 @@ jobs:
labels: ${{ steps.meta.outputs.labels }} labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha cache-from: type=gha
cache-to: type=gha,mode=max cache-to: type=gha,mode=max
build-args: |
BUILDKIT_INLINE_CACHE=1
provenance: false
# Nginx build commented out until Phase 2 # Nginx build commented out until Phase 2
# build-nginx: # build-nginx:

View File

@@ -2,10 +2,24 @@
FROM node:20-alpine AS deps FROM node:20-alpine AS deps
WORKDIR /app WORKDIR /app
COPY package*.json ./ COPY package*.json ./
# Configure npm for better reliability in CI
RUN npm config set fetch-retries 5 && \
npm config set fetch-retry-mintimeout 20000 && \
npm config set fetch-retry-maxtimeout 120000 && \
npm config set fetch-timeout 300000
# Install all dependencies including dev for building # Install all dependencies including dev for building
RUN npm ci RUN npm ci
# Stage 2: Builder # Stage 2: Production Dependencies
FROM node:20-alpine AS prod-deps
WORKDIR /app
COPY package*.json ./
# Copy all dependencies from deps stage
COPY --from=deps /app/node_modules ./node_modules
# Prune to production only (much faster than fresh install)
RUN npm prune --omit=dev
# Stage 3: Builder
FROM node:20-alpine AS builder FROM node:20-alpine AS builder
WORKDIR /app WORKDIR /app
COPY package*.json ./ COPY package*.json ./
@@ -14,7 +28,7 @@ COPY . .
# Build TypeScript # Build TypeScript
RUN npm run build RUN npm run build
# Stage 3: Simple Runtime # Stage 4: Runtime
FROM node:20-alpine AS runtime FROM node:20-alpine AS runtime
WORKDIR /app WORKDIR /app
@@ -22,10 +36,9 @@ WORKDIR /app
RUN apk add --no-cache curl su-exec util-linux && \ RUN apk add --no-cache curl su-exec util-linux && \
rm -rf /var/cache/apk/* rm -rf /var/cache/apk/*
# Install production dependencies only # Copy production dependencies from prod-deps stage
COPY --from=prod-deps /app/node_modules ./node_modules
COPY package*.json ./ COPY package*.json ./
RUN npm ci --omit=dev && \
npm cache clean --force
# Copy built application # Copy built application
COPY --from=builder /app/dist ./dist COPY --from=builder /app/dist ./dist

View File

@@ -18,6 +18,10 @@ ERROR: failed to solve: failed to read dockerfile: open Dockerfile.nginx: no suc
ERROR: failed to solve: process "/bin/sh -c npm ci --only=production && npm cache clean --force" did not complete successfully: exit code: 1 ERROR: failed to solve: process "/bin/sh -c npm ci --only=production && npm cache clean --force" did not complete successfully: exit code: 1
``` ```
### 4. Network Timeout in GitHub Actions
```
npm error network If you are behind a proxy, please make sure that the 'proxy' config is set properly
```
## Root Causes & Solutions ## Root Causes & Solutions
### 1. Invalid COPY Syntax ### 1. Invalid COPY Syntax
@@ -35,25 +39,43 @@ The `--only=production` flag is deprecated in newer npm versions.
**Solution**: Changed to `--omit=dev` which is the current syntax. **Solution**: Changed to `--omit=dev` which is the current syntax.
### 4. Network Timeouts During npm Install
Installing production dependencies fresh was causing network timeouts in GitHub Actions.
**Solution**:
- Added npm retry configuration for reliability
- Created separate stage for production dependencies
- Use `npm prune` instead of fresh install to avoid network issues
- Copy pre-pruned dependencies to runtime stage
## Changes Made ## Changes Made
### Dockerfile Changes ### Complete Dockerfile Optimization
```diff
- # Pre-initialize database during build
- RUN mkdir -p /app/data && npm run rebuild || echo "Database will be initialized at runtime"
+ # Build TypeScript only
+ RUN npm run build
- # Copy pre-built database if it exists 1. **Added npm configuration for reliability**:
- COPY --from=builder /app/data/nodes.db ./data/nodes.db 2>/dev/null || true ```dockerfile
+ # Create data directory RUN npm config set fetch-retries 5 && \
+ RUN mkdir -p /app/data npm config set fetch-retry-mintimeout 20000 && \
npm config set fetch-retry-maxtimeout 120000 && \
- RUN npm ci --only=production && \ npm config set fetch-timeout 300000
+ RUN npm ci --omit=dev && \
npm cache clean --force
``` ```
2. **Created separate production dependencies stage**:
```dockerfile
# Stage 2: Production Dependencies
FROM node:20-alpine AS prod-deps
WORKDIR /app
COPY package*.json ./
COPY --from=deps /app/node_modules ./node_modules
RUN npm prune --omit=dev
```
3. **Optimized runtime stage**:
```dockerfile
# Copy pre-pruned production dependencies
COPY --from=prod-deps /app/node_modules ./node_modules
# No need for npm install in runtime!
```
### GitHub Actions Workflow Changes ### GitHub Actions Workflow Changes
```diff ```diff
- name: Log in to GitHub Container Registry - name: Log in to GitHub Container Registry
@@ -68,11 +90,20 @@ The `--only=production` flag is deprecated in newer npm versions.
``` ```
## Result ## Result
✅ Docker build now succeeds ✅ Docker build now succeeds without network timeouts
✅ Optimized build process with 4 stages
✅ Production dependencies are pruned efficiently
✅ Database initialization happens at container startup ✅ Database initialization happens at container startup
✅ GitHub Actions workflow will work properly ✅ GitHub Actions workflow will work properly
✅ No manual intervention required ✅ No manual intervention required
## Key Improvements
1. **Network Reliability**: Added npm retry configuration
2. **Build Efficiency**: Only hit npm registry once, then prune
3. **Stage Optimization**: 4 stages for clear separation of concerns
4. **No Redundant Installs**: Eliminated duplicate npm operations
5. **GitHub Actions Ready**: Added build optimizations
## Testing ## Testing
```bash ```bash
# Build locally # Build locally
@@ -82,4 +113,13 @@ docker build -t n8n-mcp:test .
docker run -d --name test -e MCP_MODE=http -e AUTH_TOKEN=test -p 3000:3000 n8n-mcp:test docker run -d --name test -e MCP_MODE=http -e AUTH_TOKEN=test -p 3000:3000 n8n-mcp:test
docker logs test docker logs test
curl http://localhost:3000/health curl http://localhost:3000/health
```
# Check image size
docker images n8n-mcp:test
```
## Performance Impact
- Build time: Reduced by avoiding duplicate npm installs
- Network usage: Single npm ci instead of two
- Reliability: 5 retries with exponential backoff
- Cache efficiency: Better layer caching with separate stages