Testing Strategy

Loading…

Testing Strategy

Relevant source files

• src/bin/refresh_test_fixtures.rs
• src/network/fetch_cik_submissions.rs
• src/parsers/parse_us_gaap_fundamentals.rs
• src/utils.rs
• tests/cik_submissions_tests.rs
• tests/edgar_feed_tests.rs
• tests/parse_13f_xml_tests.rs
• tests/parse_nport_xml_tests.rs
• tests/sec_client_tests.rs
• tests/thirteenf_era_fixture_tests.rs
• tests/us_gaap_parser_accuracy_tests.rs

This page documents the testing approach for the rust-sec-fetcher codebase. The strategy emphasizes high-fidelity parsing verification using compressed EDGAR fixtures, HTTP layer isolation via mockito, and exhaustive mapping tests for US GAAP concept normalization.

Test Architecture Overview

The testing infrastructure is designed to handle the “dirty” nature of SEC EDGAR data (e.g., changing scale conventions in 13F filings or inconsistent XBRL tagging) by using real-world data snapshots as the primary source of truth.

graph TB
    subgraph "Rust Unit & Integration Tests"
        SecClientTest["sec_client_tests.rs\nTest SecClient HTTP & Retries"]
CikSubTest["cik_submissions_tests.rs\nTest Submissions JSON Parsing"]
ThirteenFTest["parse_13f_xml_tests.rs\nTest 13F Scaling & Sums"]
EraTest["thirteenf_era_fixture_tests.rs\nTest 2023 Schema Crossover"]
UsGaapAccTest["us_gaap_parser_accuracy_tests.rs\nVerify DF against JSON Source"]
end
    
    subgraph "Test Infrastructure & Fixtures"
        MockitoServer["mockito::Server\nHTTP mock server"]
FixtureFiles["tests/fixtures/*.gz\nCompressed SEC Snapshots"]
RefreshBin["refresh_test_fixtures.rs\nUtility to update snapshots"]
end
    
 
   SecClientTest --> MockitoServer
 
   CikSubTest --> FixtureFiles
 
   ThirteenFTest --> FixtureFiles
 
   EraTest --> FixtureFiles
 
   UsGaapAccTest --> FixtureFiles
 
   RefreshBin --> FixtureFiles

Test Component Relationships

Sources: tests/sec_client_tests.rs:1-132 tests/cik_submissions_tests.rs:1-39 tests/thirteenf_era_fixture_tests.rs:1-30 src/bin/refresh_test_fixtures.rs:1-30

Fixture-Based Testing Strategy

The project relies on a “Fixture-First” approach for data parsers. Instead of mocking complex nested JSON/XML structures by hand, the refresh_test_fixtures binary src/bin/refresh_test_fixtures.rs:1-173 downloads real filings from EDGAR and stores them as Gzip-compressed files in tests/fixtures/.

graph LR
    subgraph "Development Space"
        RefreshBin["bin/refresh_test_fixtures.rs"]
FixturesDir["tests/fixtures/"]
end

    subgraph "Code Entity Space"
        LoadFixture["load_fixture()"]
GzDecoder["flate2::read::GzDecoder"]
Parser["parse_cik_submissions_json()"]
end

    RefreshBin -- "Fetch & Compress" --> FixturesDir
    FixturesDir -- "Read .gz" --> LoadFixture
    LoadFixture -- "Decompress" --> GzDecoder
    GzDecoder -- "Stream JSON/XML" --> Parser

The Fixture Lifecycle

Sources: src/bin/refresh_test_fixtures.rs:31-173 tests/cik_submissions_tests.rs:16-30 tests/us_gaap_parser_accuracy_tests.rs:9-23

Parser Accuracy Verification

The us_gaap_parser_accuracy_tests.rs implements a deep-validation logic that ensures the Polars DataFrame produced by parse_us_gaap_fundamentals src/parsers/parse_us_gaap_fundamentals.rs:41-103 is 100% traceable to the source JSON.

• Deduplication Logic : It replicates the “Last-in Wins” logic where later amendments (e.g., 10-K/A) overwrite earlier filings for the same fiscal period src/parsers/parse_us_gaap_fundamentals.rs:27-40
• Validation : For every cell in the DataFrame, the test searches the source JSON facts object tests/us_gaap_parser_accuracy_tests.rs:75-82 applies the same fiscal year (FY) derivation logic tests/us_gaap_parser_accuracy_tests.rs:101-114 and asserts the values match within a small epsilon tests/us_gaap_parser_accuracy_tests.rs:148-150

Sources: tests/us_gaap_parser_accuracy_tests.rs:31-160 src/parsers/parse_us_gaap_fundamentals.rs:25-103

Specific Domain Testing

13F Era Crossover Testing

The SEC changed the <value> field in 13F-HR filings from “thousands of USD” to “actual USD” on 2023-01-01. The thirteenf_era_fixture_tests.rs uses three specific Berkshire Hathaway (BRK-B) fixtures to verify the normalization logic tests/thirteenf_era_fixture_tests.rs:1-12

Sources: tests/thirteenf_era_fixture_tests.rs:1-162 src/bin/refresh_test_fixtures.rs:147-172

EDGAR Atom Feed Testing

The edgar_feed_tests.rs validates the polling logic for the live EDGAR “Latest Filings” feed.

• Delta Filtering : Tests the take_while logic that stops fetching when it hits a “high-water mark” (the timestamp of the last processed filing) tests/edgar_feed_tests.rs:43-51
• High-Water Mark : Ensures the FeedDelta correctly identifies the newest entry’s timestamp as the next mark tests/edgar_feed_tests.rs:179-194

Sources: tests/edgar_feed_tests.rs:11-194

Network & Client Testing

The SecClient is tested using mockito to simulate SEC server responses, ensuring the crate respects the SEC’s strict User-Agent and rate-limiting requirements.

sequenceDiagram
    participant Test as test_fetch_json_with_retry_failure
    participant Client as SecClient
    participant Mock as mockito::Server
    
    Test->>Mock: Expect(3) calls to /path
    Test->>Client: fetch_json(mock_url)
    
    Client->>Mock: Attempt 1
    Mock-->>Client: 500 Internal Server Error
    Note over Client: Backoff Delay
    
    Client->>Mock: Attempt 2 (Retry)
    Mock-->>Client: 500 Internal Server Error
    
    Client->>Mock: Attempt 3 (Final Retry)
    Mock-->>Client: 500 Internal Server Error
    
    Client-->>Test: Err("Max retries exceeded")
    Test->>Test: assert!(result.is_err())

HTTP Mocking Pattern

Sources: tests/sec_client_tests.rs:194-222

User-Agent Compliance

The SEC requires a User-Agent in the format AppName/Version (+Email). Tests verify that SecClient correctly falls back to crate defaults or uses custom overrides provided in AppConfig tests/sec_client_tests.rs:7-82

• Default : sec-fetcher/0.1.0 (+test@example.com) tests/sec_client_tests.rs:16-22
• Custom : my-custom-app/2.0.0 (+test@example.com) tests/sec_client_tests.rs:58-61

Sources: tests/sec_client_tests.rs:7-82

Summary of Test Modules

Sources: tests/sec_client_tests.rs:1-132 tests/cik_submissions_tests.rs:1-176 tests/parse_13f_xml_tests.rs:1-187 tests/parse_nport_xml_tests.rs:1-177 tests/edgar_feed_tests.rs:1-195 tests/us_gaap_parser_accuracy_tests.rs:1-160

Dismiss

Refresh this wiki

Enter email to refresh