How to Deploy Python ML Models Alongside a Node.js API on Railway

Running Python machine learning code alongside a Node.js API on a PaaS platform involves a set of decisions that aren't well-documented anywhere. Do you run them as separate services? As a monorepo with a single Dockerfile? As subprocess calls? After working through all three approaches, this post covers what actually works in production — specifically for the kind of ML inference workloads that come up in video processing pipelines.

This is the deployment architecture for the Python/Node.js hybrid running ClipSpeedAI on Railway.

The Three Approaches

Approach 1: Separate services (Python API + Node.js API)

• Pros: Independent scaling, language isolation, clean HTTP interface
• Cons: Network latency between services, two services to manage, auth between them

Approach 2: Single container, Python as subprocess

• Pros: Simple deployment, no network hop, shared filesystem
• Cons: One container for everything, can't scale independently

Approach 3: Embedded Python via node-gyp bindings

• Cons: Extremely complex, fragile, don't do this

For video processing specifically, Approach 2 wins. The reason: video files are large. Passing them between services means either duplicating them to shared storage or making HTTP requests with multi-hundred-MB bodies. Using subprocess calls with a shared /tmp filesystem is dramatically simpler.

The Dockerfile

FROM node:20-slim

# Install Python and system dependencies
RUN apt-get update && apt-get install -y \
    python3 \
    python3-pip \
    python3-venv \
    ffmpeg \
    libgl1-mesa-glx \
    libglib2.0-0 \
    libgomp1 \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

# Install Python deps first (better layer caching)
COPY python/requirements.txt ./python/requirements.txt
RUN pip3 install --no-cache-dir -r python/requirements.txt

# Install Node deps
COPY package*.json ./
RUN npm ci --only=production

# Copy app code
COPY . .

EXPOSE 3000
CMD ["node", "src/index.js"]

Python requirements.txt

mediapipe==0.10.9
opencv-python-headless==4.9.0.80
numpy==1.26.4

Note opencv-python-headless — not opencv-python. The headless version omits Qt display dependencies that cause import failures in headless server environments.

railway.toml Configuration

[build]
builder = "DOCKERFILE"
dockerfilePath = "./Dockerfile"

[deploy]
startCommand = "node src/index.js"
healthcheckPath = "/health"
healthcheckTimeout = 30
restartPolicyType = "ON_FAILURE"
restartPolicyMaxRetries = 3

[[services]]
name = "api"
source = "."

The Subprocess Bridge Pattern

// lib/python_bridge.js
import { execa } from 'execa';
import { randomUUID } from 'crypto';
import fs from 'fs/promises';
import path from 'path';

const PYTHON_SCRIPTS = {
  faceDetect: path.resolve('./python/face_detector.py'),
  tracker: path.resolve('./python/tracker.py'),
};

const PYTHON_ENV = {
  ...process.env,
  MEDIAPIPE_DISABLE_GPU: '1',
  OMP_NUM_THREADS: '2',
  OPENBLAS_NUM_THREADS: '2',
  MKL_NUM_THREADS: '2',
  PYTHONUNBUFFERED: '1'
};

export async function runPythonScript(scriptName, args, timeoutMs = 120_000) {
  const scriptPath = PYTHON_SCRIPTS[scriptName];
  if (!scriptPath) throw new Error(`Unknown Python script: ${scriptName}`);

  const outputPath = path.join('/tmp', `py_out_${randomUUID()}.json`);

  try {
    const { exitCode, stderr } = await execa('python3', [
      scriptPath,
      ...args,
      '--output', outputPath
    ], {
      timeout: timeoutMs,
      env: PYTHON_ENV,
      reject: false  // don't throw on non-zero exit — handle manually
    });

    if (exitCode !== 0) {
      throw new Error(`Python script failed (exit ${exitCode}): ${stderr}`);
    }

    const result = JSON.parse(await fs.readFile(outputPath, 'utf8'));
    return result;

  } finally {
    try { await fs.unlink(outputPath); } catch {}
  }
}

The reject: false option in execa means you get the exit code back instead of an automatic throw. This lets you log the stderr (which contains useful Python tracebacks) before raising the error.

Startup Health Check

Railway's health check hits your /health endpoint. Use it to verify the Python environment is intact:

// routes/health.js
import { execa } from 'execa';

export async function healthCheck(req, res) {
  const checks = {
    node: true,
    python: false,
    ffmpeg: false
  };

  try {
    await execa('python3', ['-c', 'import mediapipe; import cv2']);
    checks.python = true;
  } catch {}

  try {
    await execa('ffmpeg', ['-version']);
    checks.ffmpeg = true;
  } catch {}

  const healthy = Object.values(checks).every(Boolean);
  res.status(healthy ? 200 : 503).json({ status: healthy ? 'ok' : 'degraded', checks });
}

If python is false on startup, Railway will restart the container — which surfaces environment issues immediately rather than at job runtime.

Managing Container Memory

MediaPipe models load into memory when first used. On a Railway Starter plan (512MB RAM), that can be tight. Strategies:

# In the Python script, unload model immediately after use
with mp_face.FaceDetection(model_selection=1) as detector:
    # ... process frames ...
    pass  # model freed here when context exits

# Force garbage collection
import gc
gc.collect()

Each Python subprocess is a fresh process with no shared memory. MediaPipe loads cold on every invocation. This means no memory accumulation between jobs — but ~2-3 seconds of cold-start overhead per job. Acceptable for async processing queues, not for real-time APIs.

Railway-Specific Pitfalls

Thread limits: Railway containers have a thread limit (varies by plan). MediaPipe's default thread pool = number of CPU cores. On a 2-core Railway instance, cap to 2 threads with OMP_NUM_THREADS=2 to stay under the limit.

Ephemeral filesystem: /tmp is cleared between deployments but not between requests. Safe to use for job temp files. Do not store anything you need to persist.

Build time: Installing MediaPipe takes 2-4 minutes during Docker build. Railway caches layers — changes to requirements.txt bust the Python layer cache. Keep Python deps stable to avoid slow rebuilds.

Networking: If you do go with separate services, use Railway's private networking (${{service.RAILWAY_PRIVATE_DOMAIN}}) not the public URLs — it's faster and doesn't count against your bandwidth quota.

Monitoring Python Failures

encodeWorker.on('failed', (job, err) => {
  if (err.message.includes('Python script failed')) {
    // Extract and log the Python traceback
    const tracebackMatch = err.message.match(/Traceback[\s\S]+?(?=\n\n|$)/);
    logger.error('Python subprocess failure', {
      jobId: job?.id,
      traceback: tracebackMatch?.[0] || err.message
    });
  }
});

The Railway deployment running ClipSpeedAI uses this exact pattern. Python and Node.js in a single container, subprocess bridge for ML inference, health checks to catch environment drift early. It's not the most elegant architecture, but it's operationally simple and reliable under real workloads.

If building and maintaining this Python/Node.js hybrid isn't something you want to take on, ClipSpeedAI offers the complete pipeline — face detection, transcription, AI scoring, and vertical encoding — as a hosted service.