Docker Deployment

Relevant source files

• python/narrative_stack/Dockerfile.simd-r-drive-ci-server
• python/narrative_stack/us_gaap_store_integration_test.sh

Purpose and Scope

This document describes the Docker containerization strategy for the rust-sec-fetcher system, focusing on the simd-r-drive-ws-server deployment and the Docker Compose infrastructure used for integration testing. The Dockerfile builds a lightweight container that runs the WebSocket server, while Docker Compose orchestrates multi-container test environments with MySQL and the WebSocket server.

For information about the CI/CD workflows that use these Docker configurations, see CI/CD Pipeline. For details about the simd-r-drive WebSocket server functionality, see Database & Storage Integration.

Docker Image Architecture

The system uses a single-stage Dockerfile to build and deploy the simd-r-drive-ws-server binary in a minimal Rust container. This approach prioritizes simplicity and reproducibility for CI/CD environments.

graph TB
    BaseImage["rust:1.87-slim\nBase Image"]
CargoInstall["cargo install\nsimd-r-drive-ws-server@0.10.0-alpha"]
EnvSetup["Environment Setup\nSERVER_ARGS variable\ndefault: 'data.bin --host 127.0.0.1 --port 8080'"]
Expose["EXPOSE 8080\nWebSocket port"]
Entrypoint["ENTRYPOINT\n/bin/sh -c 'simd-r-drive-ws-server ${SERVER_ARGS}'"]
BaseImage --> CargoInstall
 
   CargoInstall --> EnvSetup
 
   EnvSetup --> Expose
 
   Expose --> Entrypoint

Dockerfile Structure

Sources: python/narrative_stack/Dockerfile.simd-r-drive-ci-server:1-34

Build Configuration

The Dockerfile uses build arguments and environment variables to provide flexible configuration:

Build-time customization:

Runtime customization:

Sources: python/narrative_stack/Dockerfile.simd-r-drive-ci-server:9-33

Image Composition

Sources: python/narrative_stack/Dockerfile.simd-r-drive-ci-server:10-26

Docker Compose Test Infrastructure

The integration testing environment uses Docker Compose to orchestrate multiple services with isolated networking and lifecycle management.

graph TB
    subgraph "Docker Compose Project: us_gaap_it"
        
        subgraph "db_test Service"
            MySQLContainer["Container: us_gaap_test_db\nImage: mysql\nPort: 3306\nRoot password: onlylocal"]
Database["Database: us_gaap_test\nSchema: us_gaap_schema_2025.sql"]
end
        
        subgraph "simd_r_drive_ws_server_test Service"
            WSServer["Container: simd-r-drive-ws-server\nPort: 8080\nData file: data.bin"]
end
        
        subgraph "Test Runner (Host)"
            TestScript["us_gaap_store_integration_test.sh\npytest integration tests"]
end
    end
    
 
   MySQLContainer --> Database
 
   TestScript -->|docker exec mysql commands| MySQLContainer
 
   TestScript -->|pytest via network| WSServer
 
   TestScript -->|pytest via network| Database

Service Architecture

Sources: python/narrative_stack/us_gaap_store_integration_test.sh:1-39

sequenceDiagram
    participant Script as us_gaap_store_integration_test.sh
    participant Compose as docker compose
    participant MySQL as us_gaap_test_db
    participant WSServer as simd-r-drive-ws-server
    participant Pytest as pytest
    
    Script->>Script: Set PROJECT_NAME="us_gaap_it"
    Script->>Script: Register cleanup trap
    
    Script->>Compose: up -d (profile: test)
    Compose->>MySQL: Start container
    Compose->>WSServer: Start container
    
    loop Wait for MySQL
        Script->>MySQL: mysqladmin ping
        MySQL-->>Script: Connection status
    end
    
    Script->>MySQL: CREATE DATABASE us_gaap_test
    Script->>MySQL: Load us_gaap_schema_2025.sql
    
    Script->>Pytest: Run tests/integration/test_us_gaap_store.py
    Pytest->>MySQL: Query database
    Pytest->>WSServer: WebSocket operations
    Pytest-->>Script: Test results
    
    Script->>Compose: down --volumes --remove-orphans
    Compose->>MySQL: Stop and remove
    Compose->>WSServer: Stop and remove

Test Execution Flow

The integration test script implements a robust lifecycle with automatic cleanup:

Sources: python/narrative_stack/us_gaap_store_integration_test.sh:7-39

Docker Compose Configuration Pattern

The test script uses the following Docker Compose invocation pattern:

Key commands:

Sources: python/narrative_stack/us_gaap_store_integration_test.sh:8-18

graph TB
    Start["Container Start\nImage: mysql\nRoot password: onlylocal"]
WaitReady["Health Check Loop\nmysqladmin ping -h127.0.0.1"]
CreateDB["docker exec\nCREATE DATABASE IF NOT EXISTS us_gaap_test"]
LoadSchema["docker exec\nmysql < us_gaap_schema_2025.sql"]
Ready["Database Ready\nSchema applied"]
Start --> WaitReady
 
   WaitReady -->|Ping successful| CreateDB
 
   WaitReady -->|Still initializing| WaitReady
 
   CreateDB --> LoadSchema
 
   LoadSchema --> Ready

Database Container Setup

The MySQL container requires specific initialization steps to prepare the test database:

Initialization Sequence

Sources: python/narrative_stack/us_gaap_store_integration_test.sh:25-35

MySQL Container Configuration

The script executes the following commands against the us_gaap_test_db container:

1. Health check: Polls MySQL readiness using mysqladmin ping

2. Database creation: Creates the test database if it doesn't exist

3. Schema loading: Applies the SQL schema from the repository

Sources: python/narrative_stack/us_gaap_store_integration_test.sh:25-35

Environment Variables and Configuration

The Docker deployment uses environment variables for runtime configuration of both the WebSocket server and test infrastructure.

simd-r-drive-ws-server Configuration

The SERVER_ARGS environment variable controls:

• Data file: Path to the persistent storage file (e.g., data.bin)
• Host binding: IP address for the WebSocket server (127.0.0.1 for localhost, 0.0.0.0 for all interfaces)
• Port: TCP port for the WebSocket endpoint (default 8080)

Sources: python/narrative_stack/Dockerfile.simd-r-drive-ci-server:18-32

Test Environment Variables

The integration test script sets the following environment variables:

This ensures Python can locate the narrative_stack module when running tests.

Sources: python/narrative_stack/us_gaap_store_integration_test.sh37

Deployment Considerations

Port Exposure and Network Configuration

The Dockerfile exposes port 8080 for WebSocket connections:

When deploying, map this port to the host system:

For production deployments, consider:

• Using a different external port (e.g., -p 9000:8080)
• Binding to specific interfaces with --host 0.0.0.0 in SERVER_ARGS
• Placing the container behind a reverse proxy (nginx, traefik)

Sources: python/narrative_stack/Dockerfile.simd-r-drive-ci-server25

Data Persistence

The server expects a data file specified in SERVER_ARGS (default: data.bin). For persistent storage across container restarts:

Sources: python/narrative_stack/Dockerfile.simd-r-drive-ci-server:22-23

graph TB
    Setup["Setup Phase\ndocker compose up -d"]
Trap["Register Trap\ntrap 'cleanup' EXIT"]
Execute["Test Execution\npytest integration tests"]
Cleanup["Cleanup Phase\ndocker compose down\n--volumes --remove-orphans"]
Setup --> Trap
 
   Trap --> Execute
 
   Execute -->|Success or Failure| Cleanup

Container Lifecycle Management

The integration test script demonstrates proper container lifecycle management:

The trap command ensures cleanup runs even if tests fail or the script is interrupted:

Sources: python/narrative_stack/us_gaap_store_integration_test.sh:14-19

Build and Deployment Commands

Build the Docker image:

Run the container:

Run integration tests:

Sources: python/narrative_stack/Dockerfile.simd-r-drive-ci-server:1-3 python/narrative_stack/us_gaap_store_integration_test.sh:1-39

Integration with CI/CD

The Docker configuration integrates with GitHub Actions for automated testing. The CI pipeline:

1. Builds the simd-r-drive-ci-server image
2. Spins up Docker Compose services with the test profile
3. Initializes the MySQL database with the test schema
4. Runs pytest integration tests
5. Tears down all containers and volumes

This ensures reproducible test environments across local development and CI systems.

Sources: python/narrative_stack/us_gaap_store_integration_test.sh:1-2 python/narrative_stack/Dockerfile.simd-r-drive-ci-server:1-8