Configuration System

Loading…

Configuration System

Relevant source files

• src/config.rs
• src/config/app_config.rs
• src/config/config_manager.rs
• src/config/credential_manager.rs
• src/models/cik.rs
• src/models/investment_company.rs
• src/parsers/parse_nport_xml.rs
• tests/config_manager_tests.rs

This document describes the configuration management system in the Rust sec-fetcher application. The configuration system provides a flexible, multi-source approach to managing application settings including SEC API credentials, request throttling parameters, and cache directories.

For information about how the configuration integrates with the caching system, see Caching & Storage System. For details on credential storage mechanisms, see the credential management section below.

Overview

The configuration system consists of three primary components:

The system supports configuration from four sources with increasing priority:

1. Default values - Hard-coded defaults in AppConfig::default() src/config/app_config.rs:43-62
2. Environment Variables - SEC_FETCHER_EMAIL, SEC_FETCHER_APP_NAME, etc. src/config/config_manager.rs:56-88
3. TOML configuration file - User-specified settings loaded from disk src/config/config_manager.rs:168-171
4. Interactive prompts - Credential collection when running in interactive mode src/config/credential_manager.rs:81-127

Sources: src/config/app_config.rs:16-62 src/config/config_manager.rs:56-155

Configuration Structure

AppConfig Fields

All fields in AppConfig are wrapped in Option<T> to support partial configuration and merging strategies. The #[merge(strategy = overwrite_option)] attribute ensures that non-None values from user configuration always replace default values.

Sources: src/config/app_config.rs:16-41 src/config/app_config.rs:43-62

Configuration Loading Flow

ConfigManager Initialization

Sources: src/config/config_manager.rs:107-182 src/config/credential_manager.rs:81-127

Configuration File Format

TOML Structure

The configuration file uses TOML format with strict schema validation. Any unrecognized keys will cause a descriptive error listing all valid keys and their types using schemars integration.

Example configuration file (sec_fetcher_config.toml):

Configuration File Locations

The system searches for configuration files in the following order:

Sources: src/config/config_manager.rs:103-104 src/config/config_manager.rs:129-131 src/config/config_manager.rs:166-170

Credential Management

Email Credential Requirement

The SEC EDGAR API mandates a valid email address in the HTTP User-Agent header for identification. The configuration system resolves this using the following precedence:

Interactive Mode Detection:

graph TD
    Start["ConfigManager Resolution"]
CheckArg["App Identity Override?"]
CheckFile["Email in TOML Config?"]
CheckEnv["SEC_FETCHER_EMAIL Env Var?"]
CheckInteractive{"is_interactive_mode()?"}
PromptUser["CredentialManager::from_prompt()"]
ReadKeyring["Read from system keyring"]
PromptInput["Prompt user for email"]
SaveKeyring["Save to keyring"]
SetEmail["Set email in ConfigManager"]
Error["Return Error:\nCould not obtain email credential"]
Start --> CheckArg
 
   CheckArg -->|No| CheckFile
 
   CheckFile -->|No| CheckEnv
 
   CheckEnv -->|No| CheckInteractive
 
   CheckInteractive -->|Yes| PromptUser
 
   CheckInteractive -->|No| Error
 
   PromptUser --> ReadKeyring
 
   ReadKeyring -->|Found| SetEmail
 
   ReadKeyring -->|Not found| PromptInput
 
   PromptInput --> SaveKeyring
 
   SaveKeyring --> SetEmail

• Returns true if stdin/stdout are a terminal (TTY) src/config/credential_manager.rs:82-84
• Can be overridden via set_interactive_mode_override() for testing tests/config_manager_tests.rs81

Sources: src/config/config_manager.rs:41-56 src/config/credential_manager.rs:32-76

Configuration Merging Strategy

Merge Behavior

The AppConfig struct uses the merge crate with a custom overwrite_option strategy defined in src/config/app_config.rs:8-12:

Merge Rules:

1. Some(new_value) always replaces Some(old_value) src/config/app_config.rs:9-11
2. Some(new_value) always replaces None
3. None never replaces Some(old_value)

This ensures user-provided values take absolute precedence over defaults while allowing partial configuration.

Sources: src/config/app_config.rs:8-12 src/config/config_manager.rs:172-182

Schema Validation and Error Handling

Invalid Key Detection

The configuration system uses #[serde(deny_unknown_fields)] to reject unknown keys in TOML files src/config/app_config.rs15 When an invalid key is detected, the error message includes a complete list of valid keys with their types extracted via schemars:

Example error output:

graph LR
    TOML["TOML File with\ninvalid_key"]
Deserialize["config.try_deserialize()"]
ExtractSchema["AppConfig::get_valid_keys()"]
Schema["schemars::schema_for!(AppConfig)"]
FormatError["Format error message with\nvalid keys and types"]
TOML --> Deserialize
 
   Deserialize -->|Error| ExtractSchema
 
   ExtractSchema --> Schema
 
   Schema --> FormatError
 
   FormatError --> Error["Return descriptive error"]

unknown field `invalid_key`, expected one of `email`, `app_name`, `app_version`, `max_concurrent`, `min_delay_ms`, `max_retries`, `cache_base_dir`

Valid configuration keys are:
  - email (String | Null)
  - app_name (String | Null)
  - app_version (String | Null)
  - max_concurrent (Integer | Null)
  - min_delay_ms (Integer | Null)
  - max_retries (Integer | Null)
  - cache_base_dir (String | Null)

Sources: src/config/app_config.rs:69-83 src/config/config_manager.rs:175-182 tests/config_manager_tests.rs:154-181

Usage Examples

Loading Configuration

Default configuration path:

Custom configuration path:

Overriding App Identity:

Sources: src/config/config_manager.rs:108-110 src/config/config_manager.rs:129-131 src/config/config_manager.rs:151-155

Cache Isolation

The ConfigManager manages the lifetime of the cache directory. If cache_base_dir is not provided in the configuration, it creates a unique tempfile::TempDir src/config/app_config.rs:45-48

This ensures that tests and ephemeral runs have fully isolated cache storage that is cleaned up when the ConfigManager is dropped src/config/app_config.rs:47-48

Sources: src/config/app_config.rs:43-62 src/config/config_manager.rs:98-100

Testing

Test Coverage

The configuration system includes unit tests in tests/config_manager_tests.rs covering:

Sources: tests/config_manager_tests.rs:1-207

Dismiss

Refresh this wiki

Enter email to refresh