Propose a part

Notebook #088 · published

Coding, Haiku 5.5 with thinking off

Exact means the function returned the right answer on every hidden case. Each part ran against the mean of two unchanged runs of the same agent, on 432 paired tasks drawn by a public beacon after the design was hashed. The unchanged agent was exact on 94.1% of them. Rule v2: every interval is widened for the 7 arms this notebook compares, so luck flags any of them at most 5% of the time.

No comments hurt. 4 parts did nothing measurable. 1 part is not proven either way yet. OPSIN called 4 of 6 before the beacon drew the tasks. Its foresight, committed before the beacon, gave what came back 42% on average; a guess gives 25%.

Preregistered
sha256 2ee05a353294cbdddd9948374ef1c5817924761bf9b8b50a9fd619af05ea88af
the full design, as JSON
Tasks drawn by
drand round 6,625,646
seed drand:6625646:d49c724747971930f2b948289e31fc49bf58ba40d9e10b06c01d63a83c12a923
Signed by
0xFf8E1E8ebF21064Ef781B75CdC8DFbd969399EbD
EIP-191 over sha256 of each verdict’s canonical JSON. Verify one yourself
Every run
Each run of every arm, with its task’s family and whether it was exact: CSV or JSON. With the seed, every verdict above can be recomputed from them.
Recorded on Elysium
transaction 0xa0130f8db7120b70eacfc104e113cda8d9dbd4cf6d9685aac6a3245735df84ea
Written once, by the lab, and nothing can change it. The contract
  1. Control check (A/A)the unchanged agent against itselfsigned verdict 537
    INERT −0.7−3.2 to +1.6
  2. PlaceboThe office plants were watered this morning.signed verdict 538
    INERT −1.5−4.5 to +1.3 ×0.97
  3. Keep exact fractionsline · OPSIN's second look at a surprise · OPSIN predicted INERT at 0.55 · foresight EXCITES 48%, INERT 33%, UNRESOLVED 18% · against the placebo +3.5signed verdict 539
    UNRESOLVED +2.0−1.3 to +5.2 ×1.15
  4. No commentsline · OPSIN's replication · OPSIN predicted INERT at 0.70 · foresight INERT 69%, UNRESOLVED 16%, INHIBITS 14% · against the placebo −3.9signed verdict 540
    INHIBITS −5.4−9.5 to −1.6 ×0.82
  5. Exact integersline · OPSIN's pick · money-handling rules in agent rules files · OPSIN predicted INERT at 0.65 · foresight INERT 64%, EXCITES 16%, UNRESOLVED 14% · against the placebo +1.4signed verdict 541
    INERT −0.1−2.7 to +2.4 ×1.14
  6. Keep it simpleline · OPSIN's pick · KISS, in countless CLAUDE.md files · OPSIN predicted INERT at 0.78 · foresight INERT 68%, EXCITES 15%, UNRESOLVED 13% · against the placebo −0.2signed verdict 542
    INERT −1.7−4.9 to +1.2 ×1.07
  7. Plan firstline · OPSIN's pick · plan-then-code rules in coding-agent prompts · OPSIN predicted INERT at 0.70 · foresight INERT 66%, EXCITES 15%, UNRESOLVED 13% · against the placebo +1.2signed verdict 543
    INERT −0.3−2.9 to +2.1 ×1.33
  8. A $200 tipline · OPSIN's pick · the viral tipping test, December 2023 · OPSIN predicted INERT at 0.75 · foresight INERT 68%, EXCITES 14%, UNRESOLVED 13% · against the placebo +0.9signed verdict 544
    INERT −0.6−2.8 to +1.6 ×1.02