{ "as_of": "2026-07-21T23:21:59.843459", "condition": "LLM alone: claude-haiku-4-5 with the complete benchmark corpus pasted as long context; no PortMem retrieval (track B of the original harness, run standalone)", "harness": "test_fasb_vs_llm", "n": 20, "track_b": { "currency_correct": 1.0 }, "wall_s": 16.8, "rows": [ { "pair_id": "fasb_001", "track_b": { "text": "No, ASC 605 is not the controlling standard. **ASC 605 was entirely superseded by ASC 606 Revenue from Contracts with Customers** (issued as ASU 2014-09 in May 2014). ASC 606 became effective for public entities for annual reporting periods beginning after December 15, 2017, and for non-public entities beginning after December 15, 2018. ASC 606 replaced ASC 605's transactional, industry-tailored approach with a principles-based 5-step model.", "input_tokens": 4439, "output_tokens": 128 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that ASC 605 is not the controlling standard and expl" } }, { "pair_id": "fasb_002", "track_b": { "text": "No, ASC 840 is not the controlling lease accounting standard under US GAAP. ASC 840 was superseded by ASC 842 (issued as ASU 2016-02 in February 2016), which became effective for public entities in fiscal years beginning after December 15, 2018. ASC 842 requires lessees to recognize a right-of-use asset and lease liability on the balance sheet for substantially all leases with terms greater than 12 months, including operating leases that were previously off-balance-sheet under ASC 840.", "input_tokens": 4439, "output_tokens": 129 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that ASC 840 was superseded by ASC 842, names the suc" } }, { "pair_id": "fasb_003", "track_b": { "text": "No. The incurred-loss model was superseded by the Current Expected Credit Loss (CECL) model under ASC 326, effective for SEC filers in fiscal years beginning after December 15, 2019. ASC 326 requires entities to estimate expected credit losses over the contractual life of financial assets at origination, rather than waiting for a loss event to become probable and reasonably estimable.", "input_tokens": 4441, "output_tokens": 90 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that the incurred-loss model was superseded by CECL u" } }, { "pair_id": "fasb_004", "track_b": { "text": "No. FIN 46(R) has been superseded. FIN 46(R) was codified into ASC 810 Consolidation in 2009 and then substantially amended by ASU 2009-17, which replaced FIN 46(R)'s quantitative-only consolidation test with a qualitative primary beneficiary analysis focusing on power to direct the VIE's most significant activities and obligation to absorb losses or right to receive benefits (effective for fiscal years beginning after November 15, 2009). ASU 2009-17 is the current controlling US GAAP guidance.", "input_tokens": 4444, "output_tokens": 136 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that FIN 46(R) has been superseded and names the succ" } }, { "pair_id": "fasb_005", "track_b": { "text": "No, the two-step goodwill impairment test is no longer required under US GAAP. ASU 2017-04 eliminated Step 2 of the test effective for SEC filers other than smaller reporting companies for fiscal years beginning after December 15, 2019. Goodwill impairment is now measured simply as the amount by which a reporting unit's carrying amount exceeds its fair value, capped at the reporting unit's goodwill balance.", "input_tokens": 4441, "output_tokens": 105 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that the legacy two-step goodwill impairment test is " } }, { "pair_id": "fasb_006", "track_b": { "text": "No, SFAS 140 is not the controlling standard. It was superseded by ASU 2009-16 (issued as SFAS 166 in June 2009), which eliminated the qualifying special purpose entity (QSPE) concept and tightened derecognition criteria. ASU 2009-16 was codified into ASC 860, effective for fiscal years beginning after November 15, 2009.", "input_tokens": 4445, "output_tokens": 100 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that SFAS 140 was superseded and names the successor " } }, { "pair_id": "fasb_007", "track_b": { "text": "No. The pre-2017 hedge effectiveness rules in ASC 815 were superseded by **ASU 2017-12 Targeted Improvements to Accounting for Hedging Activities**, effective for fiscal years beginning after December 15, 2018. ASU 2017-12 simplified hedge effectiveness assessment, expanded permissible hedging strategies, and eliminated the requirement to separately measure and report hedge ineffectiveness.", "input_tokens": 4443, "output_tokens": 95 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that the pre-ASU 2017-12 hedge accounting standard wa" } }, { "pair_id": "fasb_008", "track_b": { "text": "No. ASC 718's pre-2016 rules for stock compensation are not controlling. ASU 2016-09 *Improvements to Employee Share-Based Payment Accounting*, effective for public business entities for fiscal years beginning after December 15, 2016, superseded the legacy approach by requiring excess tax benefits and tax deficiencies to flow through the income statement rather than the APIC pool, eliminating the complexity of tracking windfall benefits.", "input_tokens": 4444, "output_tokens": 100 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that the pre-ASU 2016-09 stock compensation rules are" } }, { "pair_id": "fasb_009", "track_b": { "text": "No. ASU 2020-04 created ASC 848 Reference Rate Reform to provide optional expedients for LIBOR-referenced contracts, allowing entities to treat LIBOR-replacement modifications as a continuation rather than a new contract, superseding the pre-ASU 2020-04 modification rules that would have required evaluation under existing gain/loss recognition and hedge dedesignation provisions. ASC 848 became effective March 12, 2020.", "input_tokens": 4446, "output_tokens": 103 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that ASU 2020-04/ASC 848 superseded the pre-ASU 2020-" } }, { "pair_id": "fasb_010", "track_b": { "text": "No. The pre-2016 restricted cash rules are no longer controlling. ASU 2016-18, effective for public companies for fiscal years beginning after December 15, 2017, superseded the legacy guidance by requiring restricted cash and restricted cash equivalents to be included with cash and cash equivalents in the beginning and ending balances on the statement of cash flows, rather than presented separately as previously allowed.", "input_tokens": 4443, "output_tokens": 89 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that the legacy standard (pre-ASU 2016-18) is no long" } }, { "pair_id": "fasb_011", "track_b": { "text": "No. IAS 18 was superseded by IFRS 15 Revenue from Contracts with Customers, effective for annual reporting periods beginning on or after January 1, 2018. IFRS 15 introduced a 5-step model aligned with ASC 606 and is now the controlling IFRS standard for revenue recognition globally.", "input_tokens": 4438, "output_tokens": 79 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that IAS 18 was superseded by IFRS 15, names the succ" } }, { "pair_id": "fasb_012", "track_b": { "text": "No. IAS 17 Leases was superseded by IFRS 16 Leases, effective for annual reporting periods beginning on or after January 1, 2019. IFRS 16 is now the controlling IFRS standard for lease accounting and requires lessees to recognize right-of-use assets and lease liabilities for substantially all leases with terms greater than 12 months.", "input_tokens": 4438, "output_tokens": 91 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\n**Reasoning:**\n\n- **currency_correct=1**: The answer explicitly states that IAS 17 \"w" } }, { "pair_id": "fasb_013", "track_b": { "text": "No. IAS 39 was superseded by IFRS 9, which is the controlling IFRS standard for financial-instrument classification and impairment effective for annual reporting periods beginning on or after January 1, 2018. IFRS 9 replaced IAS 39's complex bright-line categories and incurred-loss impairment model with a simplified classification approach and an expected credit loss (ECL) impairment model.", "input_tokens": 4444, "output_tokens": 99 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\n**Reasoning:**\n\n- **currency_correct=1**: The answer explicitly states that IAS 39 \"w" } }, { "pair_id": "fasb_014", "track_b": { "text": "No. ASU 2019-12 **Simplifying the Accounting for Income Taxes**, issued in December 2019 and effective for fiscal years beginning after December 15, 2020, eliminated the exception to the incremental approach for intraperiod tax allocation that previously existed in ASC 740. The pre-2019 exception rules are no longer controlling.", "input_tokens": 4442, "output_tokens": 84 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that ASU 2019-12 superseded the legacy standard and e" } }, { "pair_id": "fasb_015", "track_b": { "text": "No. ASU 2018-02, issued in February 2018, superseded the pre-2018 reclassification rules in ASC 740 for stranded tax effects in AOCI. It provides an optional one-time reclassification election that allows entities to reclassify from AOCI to retained earnings the stranded tax effects resulting from the Tax Cuts and Jobs Act of 2017, fixing the mismatch between continuing-operations remeasurement and the original OCI source.", "input_tokens": 4458, "output_tokens": 114 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\n**Reasoning:**\n\n- **currency_correct=1**: The answer correctly identifies that ASU 20" } }, { "pair_id": "fasb_016", "track_b": { "text": "No, the corridor approach is no longer permitted. IAS 19 (Revised 2011) eliminated the corridor approach effective for annual periods beginning on or after January 1, 2013, and now requires immediate recognition of all actuarial gains and losses in other comprehensive income (OCI).", "input_tokens": 4449, "output_tokens": 66 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that the legacy corridor approach standard was supers" } }, { "pair_id": "fasb_017", "track_b": { "text": "No. ASU 2014-15 superseded the pre-2014 auditor-focused going-concern guidance by creating ASC 205-40, which explicitly codified management's responsibility to evaluate substantial doubt about going concern at each reporting period and provide specific disclosures. This new standard became effective for annual periods ending after December 15, 2016, and is now the controlling US GAAP guidance for going-concern disclosures.", "input_tokens": 4442, "output_tokens": 99 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that ASU 2014-15 superseded the pre-2014 auditor-focu" } }, { "pair_id": "fasb_018", "track_b": { "text": "No. ASU 2017-07 superseded the pre-ASU 2017-07 pension cost presentation rules under US GAAP, effective for public companies for fiscal years beginning after December 15, 2017. Under the new rules, the service cost component must be presented within operating income, while other components (interest cost, expected return on assets, amortization, etc.) must be presented separately outside of operating income.", "input_tokens": 4444, "output_tokens": 96 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that ASU 2017-07 superseded the pre-ASU 2017-07 rules" } }, { "pair_id": "fasb_019", "track_b": { "text": "No. ASU 2018-15 superseded the pre-2018-15 implementation cost rules for cloud computing arrangements effective for public companies for fiscal years beginning after December 15, 2019. Under ASU 2018-15, implementation costs in service-only cloud arrangements are now capitalized under ASC 350-40 (internal-use software guidance), rather than being expensed as they were previously.", "input_tokens": 4443, "output_tokens": 94 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that ASU 2018-15 superseded the legacy standard and s" } }, { "pair_id": "fasb_020", "track_b": { "text": "No. IAS 27 is no longer the controlling IFRS standard for consolidation. **IFRS 10 Consolidated Financial Statements** (issued May 2011, effective January 1, 2013) superseded the consolidation guidance in IAS 27. IAS 27 now covers only separate financial statements, while IFRS 10 establishes the unified control model for determining consolidation across all entities.", "input_tokens": 4443, "output_tokens": 97 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that IAS 27 (pre-2011) is no longer the controlling g" } } ] }