Propose a part

Notebook #087 · published

Coding, Haiku 5.5 with thinking off

Exact means the function returned the right answer on every hidden case. Each part ran against the mean of two unchanged runs of the same agent, on 432 paired tasks drawn by a public beacon after the design was hashed. The unchanged agent was exact on 93.4% of them. Rule v2: every interval is widened for the 7 arms this notebook compares, so luck flags any of them at most 5% of the time.

4 parts did nothing measurable. 2 parts are not proven either way yet. OPSIN called 4 of 6 before the beacon drew the tasks. Its foresight, committed before the beacon, gave what came back 32% on average; a guess gives 25%.

Preregistered
sha256 0c0718701963f8a0204038083e3bd593d089bd1a8496e5292fab6662a2bbfe48
the full design, as JSON
Tasks drawn by
drand round 6,625,451
seed drand:6625451:4d71396dd06ed7564b594da21f225915ab9ed11fc156c4c421b44cf85b61c395
Signed by
0xFf8E1E8ebF21064Ef781B75CdC8DFbd969399EbD
EIP-191 over sha256 of each verdict’s canonical JSON. Verify one yourself
Every run
Each run of every arm, with its task’s family and whether it was exact: CSV or JSON. With the seed, every verdict above can be recomputed from them.
Recorded on Elysium
transaction 0x65f197179a9b86677a44a4f781e282113653642fbdffec17f902ca857b92c8df
Written once, by the lab, and nothing can change it. The contract
  1. Control check (A/A)the unchanged agent against itselfsigned verdict 529
    INERT +0.7−1.6 to +3.0
  2. PlaceboThe office plants were watered this morning.signed verdict 530
    INERT +1.0−1.7 to +3.7 ×0.99
  3. Keep exact fractionsline · OPSIN's pick · OPSIN predicted INERT at 0.45 · foresight EXCITES 66%, INERT 26% · re-measures verdict 520 (EXCITES +8.3): it did not hold · against the placebo +1.9signed verdict 531
    UNRESOLVED +2.9−0.5 to +6.4 ×1.17
  4. Important to my careerline · OPSIN's second look at a surprise · OPSIN predicted INERT at 0.50 · foresight INERT 58%, EXCITES 24%, UNRESOLVED 17% · against the placebo −1.2signed verdict 532
    INERT −0.1−2.8 to +2.5 ×1.07
  5. Check examples by handline · OPSIN's replication · OPSIN predicted INERT at 0.50 · foresight INERT 42%, EXCITES 40%, UNRESOLVED 18% · against the placebo +0.2signed verdict 533
    INERT +1.3−1.3 to +3.7 ×1.38
  6. No commentsline · OPSIN's pick · a common rule in coding-agent prompts · OPSIN predicted INERT at 0.85 · foresight INERT 69%, EXCITES 16%, UNRESOLVED 12% · against the placebo −3.2signed verdict 534
    UNRESOLVED −2.2−5.8 to +1.0 ×0.85
  7. Think step by stepline · OPSIN's pick · Notebook #001: no effect that matters on DeFi maths with thinking off · OPSIN predicted INERT at 0.60 · foresight INERT 60%, EXCITES 19%, UNRESOLVED 14% · against the placebo −0.5signed verdict 535
    INERT +0.6−2.1 to +3.2 ×1.08
  8. Only what was askedline · OPSIN's pick · YAGNI rules in AGENTS.md files · OPSIN predicted INERT at 0.78 · foresight INERT 67%, EXCITES 17%, UNRESOLVED 12% · against the placebo −2.1signed verdict 536
    INERT −1.0−4.0 to +1.7 ×0.91