{ "as_of": "2026-07-21T23:25:11.258255", "condition": "Knowledge-only: claude-haiku-4-5 answering with NO documents (parametric knowledge), graded by each harness's own judge", "harness": "test_fasb_vs_llm", "n": 20, "knowledge_only": { "currency_correct": 0.95 }, "wall_s": 10.4, "rows": [ { "pair_id": "fasb_001", "track_b": { "text": "No, ASC 605 has been superseded. ASC 606 (Revenue from Contracts with Customers) became the controlling US GAAP standard for revenue recognition and is effective for most public companies for fiscal years beginning after December 15, 2017, and for nonpublic entities after December 15, 2018. ASC 605 was withdrawn upon adoption of ASC 606.", "input_tokens": 78, "output_tokens": 91 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that ASC 605 has been superseded by ASC 606 and names" } }, { "pair_id": "fasb_002", "track_b": { "text": "No, ASC 840 has been superseded. **ASC 842** (Leases) became the controlling lease accounting standard under US GAAP, effective for public companies for fiscal years beginning after December 15, 2018, and for private companies after December 15, 2021. ASC 842 fundamentally changed lease accounting by requiring lessees to recognize right-of-use assets and lease liabilities for most leases on the balance sheet.", "input_tokens": 78, "output_tokens": 103 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that ASC 840 has been superseded by ASC 842, names th" } }, { "pair_id": "fasb_003", "track_b": { "text": "No, the incurred-loss model is no longer the controlling US GAAP standard. It was superseded by the **Current Expected Credit Loss (CECL) model**, which was adopted in ASC 326 and became effective for most public companies in 2020 and for private companies in 2023. The CECL model requires entities to recognize credit losses based on expected losses over the asset's life, rather than only recognizing losses that have already been incurred.", "input_tokens": 80, "output_tokens": 104 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that the incurred-loss model is no longer the control" } }, { "pair_id": "fasb_004", "track_b": { "text": "No, FIN 46(R) has been superseded. **ASC 810 (Consolidation)**, finalized in 2009 as part of the FASB's Codification project, is now the controlling US GAAP guidance for variable interest entity (VIE) consolidation. ASC 810-10-25 specifically addresses the consolidation requirements for VIEs.", "input_tokens": 83, "output_tokens": 90 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that FIN 46(R) has been superseded and explicitly nam" } }, { "pair_id": "fasb_005", "track_b": { "text": "No, the two-step goodwill impairment test is no longer required under US GAAP. In January 2017, the FASB issued ASU 2017-04, which simplified goodwill impairment testing by eliminating Step 2 (the implied fair value calculation of goodwill). Under the new standard, entities now perform a one-step impairment test by comparing the fair value of a reporting unit directly to its carrying amount, with an impairment charge recognized for any excess of carrying amount over fair value.", "input_tokens": 80, "output_tokens": 120 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly states that the legacy two-step goodwill impairment test is no l" } }, { "pair_id": "fasb_006", "track_b": { "text": "No, SFAS 140 has been superseded. It was replaced by **ASC 860 (Transfers and Servicing)** when the FASB codified US GAAP in 2009. ASC 860 is now the controlling standard for accounting for transfers and servicing of financial assets.", "input_tokens": 84, "output_tokens": 72 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that SFAS 140 has been superseded and names the succe" } }, { "pair_id": "fasb_007", "track_b": { "text": "No. The pre-2017 hedge effectiveness rules have been superseded. ASC 815 was amended by ASU 2017-12 (issued August 2017, effective for most entities in 2019), which simplified hedge accounting by replacing the quantitative 80-125% effectiveness threshold with a qualitative assessment of hedge effectiveness based on the economic relationship between the hedged item and hedging instrument. The 2017 update is the current controlling US GAAP standard for hedge accounting.", "input_tokens": 82, "output_tokens": 109 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\n**Explanation:**\n- **currency_correct=1**: The answer correctly identifies that the p" } }, { "pair_id": "fasb_008", "track_b": { "text": "No, the pre-2016 ASC 718 rules for windfall tax benefits have been superseded. In December 2016, the FASB issued ASU 2016-09, which substantially changed how companies account for the income tax effects of stock-based compensation, including eliminating the requirement to recognize windfall tax benefits through additional paid-in capital and instead requiring them to flow through the income statement as a discrete tax benefit or expense.", "input_tokens": 83, "output_tokens": 98 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that the legacy standard (pre-ASU 2016-09 ASC 718) ha" } }, { "pair_id": "fasb_009", "track_b": { "text": "No. ASC 848 (issued in 2020) established expedited guidance specifically for LIBOR transition contract modifications and remains the controlling US GAAP standard\u2014it has not been superseded. However, ASC 848 was originally temporary and required entities to reassess the accounting for modifications made after December 31, 2022; the FASB extended certain provisions through December 31, 2024, so practitioners should verify current applicability based on their transition timeline. Pre-ASC 848 guidance under ASC 842 and ASC 805 is no longer the primary standard for LIBOR-related modifications.", "input_tokens": 85, "output_tokens": 139 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\n**Reasoning:**\n\n- **currency_correct = 1**: The answer correctly identifies that ASC " } }, { "pair_id": "fasb_010", "track_b": { "text": "No. ASU 2016-18 (issued in August 2016) superseded the pre-2016 rules by requiring that changes in restricted cash be excluded from cash flow statement activities and instead be presented as part of the beginning and ending cash balances. This fundamentally changed how restricted cash is classified and presented on the cash flow statement.", "input_tokens": 82, "output_tokens": 76 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that ASU 2016-18 superseded the legacy standard and n" } }, { "pair_id": "fasb_011", "track_b": { "text": "No. IAS 18 has been superseded by **IFRS 15 Revenue from Contracts with Customers**, which became effective on January 1, 2018. IFRS 15 establishes a comprehensive five-step model for revenue recognition and replaced IAS 18 entirely.", "input_tokens": 77, "output_tokens": 68 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that IAS 18 has been superseded by IFRS 15 and names " } }, { "pair_id": "fasb_012", "track_b": { "text": "No, IAS 17 has been superseded. **IFRS 16 Leases** (effective January 1, 2019) replaced IAS 17 and fundamentally changed lease accounting by requiring lessees to recognize most leases on the balance sheet as right-of-use assets with corresponding lease liabilities. IAS 17 is no longer the controlling standard.", "input_tokens": 77, "output_tokens": 86 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that IAS 17 has been superseded by IFRS 16, names the" } }, { "pair_id": "fasb_013", "track_b": { "text": "No. IAS 39 has been superseded by **IFRS 9 (Financial Instruments)**, which became effective January 1, 2018. IFRS 9 provides a new classification and measurement model for financial assets, a forward-looking impairment approach (Expected Credit Loss model), and revised hedge accounting rules, replacing the corresponding provisions in IAS 39.", "input_tokens": 83, "output_tokens": 86 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that IAS 39 has been superseded by IFRS 9, names the " } }, { "pair_id": "fasb_014", "track_b": { "text": "No, the pre-2019 income tax exception rules are not still controlling. ASU 2016-09 (adopted for most entities in 2017) and subsequently ASU 2018-02 (effective 2019) significantly modified intraperiod tax allocation guidance under ASC 740. The current rules now require companies to allocate the income tax expense/benefit among continuing operations, discontinued operations, other comprehensive income, and equity based on a more nuanced approach, with special provisions for certain items like stock-based compensation and unrealized gains/losses.", "input_tokens": 81, "output_tokens": 123 }, "track_b_score": { "currency_correct": 0, "anchor_leak": 1, "judge_text": "currency_correct=0 anchor_leak=1\n\n**Reasoning:**\n\n- **currency_correct = 0**: The answer does not acknowledge ASU 2019-" } }, { "pair_id": "fasb_015", "track_b": { "text": "No. ASU 2018-02 (issued February 2018) modified the reclassification rules to allow companies to **reclassify stranded tax effects from AOCI to retained earnings** rather than through earnings. This guidance superseded the previous requirement to recognize such amounts in net income and became the controlling standard for tax effects related to the Tax Cuts and Jobs Act. ASU 2018-02 remains in effect unless subsequently amended.", "input_tokens": 97, "output_tokens": 100 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that ASU 2018-02 superseded the previous guidance (th" } }, { "pair_id": "fasb_016", "track_b": { "text": "No, the pre-2011 IAS 19 corridor approach is no longer permitted. IAS 19 (Employee Benefits) was revised effective January 1, 2013, eliminating the corridor method and requiring companies to recognize all actuarial gains and losses immediately in other comprehensive income (OCI) as they occur. This change applies to defined benefit pension plans.", "input_tokens": 88, "output_tokens": 82 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that the legacy standard (pre-2011 IAS 19 corridor ap" } }, { "pair_id": "fasb_017", "track_b": { "text": "No. The pre-2014 management going-concern disclosure rules have been superseded. In August 2014, the FASB issued ASU 2014-15 (Disclosure of Uncertainties about an Entity's Ability to Continue as a Going Concern), which became effective for annual periods ending after December 15, 2016, and replaced the prior guidance. The new standard requires management to assess whether there is substantial doubt about the entity's ability to continue as a going concern within one year of the financial statement date and to disclose that assessment and any relevant mitigating factors.", "input_tokens": 81, "output_tokens": 129 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that the legacy standard was superseded by ASU 2014-1" } }, { "pair_id": "fasb_018", "track_b": { "text": "No. ASU 2017-07 (issued in March 2017) amended the presentation of net periodic pension and postretirement benefit costs, and those rules became the controlling standard under US GAAP. The ASU requires that only the service cost component be presented in operating income, while remeasurement gains/losses and other components be presented below operating income\u2014superseding the previous presentation guidance that allowed different treatments.", "input_tokens": 83, "output_tokens": 91 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\nThe answer correctly identifies that ASU 2017-07 superseded the legacy standard and b" } }, { "pair_id": "fasb_019", "track_b": { "text": "No. ASU 2018-15 (issued in August 2018) amended the implementation cost guidance for cloud computing arrangements under Topic 350 (Intangibles\u2014Goodwill and Other). The ASU requires that implementation costs be capitalized as an asset if they result in or enhance resources to be used in the cloud computing arrangement; otherwise, they are expensed. This standard became effective for public companies in 2020 and for private companies in 2021, and it superseded the prior guidance that generally required expensing most implementation costs for cloud arrangements.", "input_tokens": 82, "output_tokens": 126 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0" } }, { "pair_id": "fasb_020", "track_b": { "text": "No, IAS 27 is no longer the controlling standard for defining control in consolidation. **IFRS 10 (Consolidated Financial Statements)**, issued in 2011 and effective from 2013, superseded IAS 27 and now contains the primary definition of control and consolidation requirements. IAS 27 was retained only for equity accounting of investments in subsidiaries.", "input_tokens": 82, "output_tokens": 88 }, "track_b_score": { "currency_correct": 1, "anchor_leak": 0, "judge_text": "currency_correct=1 anchor_leak=0\n\n**Reasoning:**\n\n- **currency_correct=1**: The answer correctly states that IAS 27 is " } } ] }