Configuration System

Relevant source files

• examples/config.rs
• src/config/app_config.rs
• src/config/config_manager.rs
• tests/config_manager_tests.rs

This document describes the configuration management system in the Rust sec-fetcher application. The configuration system provides a flexible, multi-source approach to managing application settings including SEC API credentials, request throttling parameters, and cache directories.

For information about how the configuration integrates with the caching system, see Caching & Storage System. For details on credential storage mechanisms, see the credential management section below.

Overview

The configuration system consists of three primary components:

The system supports configuration from three sources with increasing priority:

1. Default values - Hard-coded defaults in AppConfig::default()
2. TOML configuration file - User-specified settings loaded from disk
3. Interactive prompts - Credential collection when running in interactive mode

Sources: src/config/app_config.rs:15-54 src/config/config_manager.rs:11-120

Configuration Structure

AppConfig Fields

All fields in AppConfig are wrapped in Option<T> to support partial configuration and merging strategies. The #[merge(strategy = overwrite_option)] attribute ensures that non-None values from user configuration always replace default values.

Sources: src/config/app_config.rs:15-54 examples/config.rs:13-16

Configuration Loading Flow

ConfigManager Initialization

Sources: src/config/config_manager.rs:19-86 src/config/config_manager.rs:107-119

Configuration File Format

TOML Structure

The configuration file uses TOML format with strict schema validation. Any unrecognized keys will cause a descriptive error listing all valid keys and their types.

Example configuration file (sec_fetcher_config.toml):

Configuration File Locations

The system searches for configuration files in the following order:

Sources: src/config/config_manager.rs:107-119 tests/config_manager_tests.rs:36-57

Credential Management

Email Credential Requirement

The SEC EDGAR API requires a valid email address in the HTTP User-Agent header. The configuration system ensures this credential is available through multiple strategies:

Interactive Mode Detection:

graph TD
    Start["ConfigManager Initialization"]
CheckEmail{"Email in\nAppConfig?"}
CheckInteractive{"is_interactive_mode()?"}
PromptUser["CredentialManager::from_prompt()"]
ReadKeyring["Read from system keyring"]
PromptInput["Prompt user for email"]
SaveKeyring["Save to keyring (optional)"]
SetEmail["Set email in user_settings"]
Error["Return Error:\nCould not obtain email credential"]
MergeConfig["Merge configurations"]
Start --> CheckEmail
 
   CheckEmail -->|Yes| MergeConfig
 
   CheckEmail -->|No| CheckInteractive
 
   CheckInteractive -->|Yes| PromptUser
 
   CheckInteractive -->|No| Error
 
   PromptUser --> ReadKeyring
 
   ReadKeyring -->|Found| SetEmail
 
   ReadKeyring -->|Not found| PromptInput
 
   PromptInput --> SaveKeyring
 
   SaveKeyring --> SetEmail
 
   SetEmail --> MergeConfig

• Returns true if stdout is a terminal (TTY)
• Can be overridden via set_interactive_mode_override() for testing

Sources: src/config/config_manager.rs:64-77 tests/config_manager_tests.rs:19-33

Configuration Merging Strategy

Merge Behavior

The AppConfig struct uses the merge crate with a custom overwrite_option strategy:

Merge Rules:

1. Some(new_value) always replaces Some(old_value)
2. Some(new_value) always replaces None
3. None never replaces Some(old_value)

This ensures user-provided values take absolute precedence over defaults while allowing partial configuration.

Sources: src/config/app_config.rs:8-13 src/config/config_manager.rs79

Schema Validation and Error Handling

graph LR
    TOML["TOML File with\ninvalid_key"]
Deserialize["config.try_deserialize()"]
ExtractSchema["AppConfig::get_valid_keys()"]
Schema["schemars::schema_for!(AppConfig)"]
FormatError["Format error message with\nvalid keys and types"]
TOML --> Deserialize
 
   Deserialize -->|Error| ExtractSchema
 
   ExtractSchema --> Schema
 
   Schema --> FormatError
 
   FormatError --> Error["Return descriptive error:\n- email (String /Null - max_concurrent Integer/ Null)\n- min_delay_ms (Integer /Null - max_retries Integer/ Null)\n- cache_base_dir (String / Null)"]

Invalid Key Detection

The configuration system uses serde(deny_unknown_fields) to reject unknown keys in TOML files. When an invalid key is detected, the error message includes a complete list of valid keys with their types:

Example error output:

unknown field `invalid_key`, expected one of `email`, `max_concurrent`, `min_delay_ms`, `max_retries`, `cache_base_dir`

Valid configuration keys are:
  - email (String | Null)
  - max_concurrent (Integer | Null)
  - min_delay_ms (Integer | Null)
  - max_retries (Integer | Null)
  - cache_base_dir (String | Null)

Sources: src/config/app_config.rs:62-77 src/config/config_manager.rs:45-62 tests/config_manager_tests.rs:67-94

Usage Examples

Loading Configuration

Default configuration path:

Custom configuration path:

Direct AppConfig construction:

Sources: examples/config.rs:1-17 tests/config_manager_tests.rs:36-57

graph LR
    CM["ConfigManager::from_config()"]
InitCaches["init_caches()"]
CachesInit["Caches::init(self)"]
HTTPCache["Initialize HTTP cache\nwith cache_base_dir"]
PreprocCache["Initialize preprocessor cache"]
OnceLock["Store in OnceLock statics"]
CM --> InitCaches
 
   InitCaches --> CachesInit
 
   CachesInit --> HTTPCache
 
   CachesInit --> PreprocCache
 
   HTTPCache --> OnceLock
 
   PreprocCache --> OnceLock

Cache Initialization

The ConfigManager automatically initializes the global Caches module during construction:

The cache_base_dir field from AppConfig is passed to the Caches module to determine where HTTP responses and preprocessed data are stored. For detailed information on cache policies and storage mechanisms, see Caching & Storage System.

Sources: src/config/config_manager.rs:83-100

Testing

Test Coverage

The configuration system includes comprehensive unit tests:

Test Utilities:

• create_temp_config(contents: &str) - Creates temporary TOML files for testing
• set_interactive_mode_override(Some(false)) - Disables interactive prompts during tests

Sources: tests/config_manager_tests.rs:1-95

Configuration System Components

Complete Component Diagram

Sources: src/config/app_config.rs:1-159 src/config/config_manager.rs:1-121