Propose a part

Notebook #057 · published

Coding, Haiku 5.5 with thinking off

Exact means the function returned the right answer on every hidden case. Each part ran against the mean of two unchanged runs of the same agent, on 432 paired tasks drawn by a public beacon after the design was hashed. The unchanged agent was exact on 93.8% of them. Rule v2: every interval is widened for the 7 arms this notebook compares, so luck flags any of them at most 5% of the time.

Thinking on helped. Write tests first hurt. 4 parts did nothing measurable. OPSIN called 5 of 6 before the beacon drew the tasks.

Preregistered
sha256 15232dce40b7c1d3ccd0891d636c285e61c9b1cea919b8137026450ae36750ef
the full design, as JSON
Tasks drawn by
drand round 6,621,603
seed drand:6621603:6d29d07efc48c34b109c6a4c5704942ac55cf5343136da57f6015802832a8060
Signed by
0xFf8E1E8ebF21064Ef781B75CdC8DFbd969399EbD
EIP-191 over sha256 of each verdict’s canonical JSON. Verify one yourself
Every run
Each run of every arm, with its task’s family and whether it was exact: CSV or JSON. With the seed, every verdict above can be recomputed from them.
Recorded on Elysium
transaction 0xfd67e077a79caf25fa9e8f7e8e21e9ca1a2b7d8cd07ab1341e117dbeb9016e5d
Written once, by the lab, and nothing can change it. The contract
  1. Control check (A/A)the unchanged agent against itselfsigned verdict 356
    INERT +0.5−1.4 to +2.3
  2. PlaceboThe weather outside is mild today.signed verdict 357
    INERT −1.4−4.6 to +1.7 ×0.96
  3. Thinking onsetting · OPSIN's pick · OPSIN predicted EXCITES at 0.50 · stopped early at 192 · provisional: stopped early, confirmed only at full size · against the placebo +4.7signed verdict 358
    EXCITES +4.2+1.3 to +7.5 ×2.17
  4. The test-driven-development skillskill · OPSIN's pick · OPSIN predicted INERT at 0.45 · against the placebo +1.4signed verdict 359
    INERT +0.0−2.8 to +2.7 ×2.72
  5. Check the examplesline · OPSIN's pick · OPSIN predicted INERT at 0.65 · against the placebo +1.9signed verdict 360
    INERT +0.5−1.9 to +2.7 ×1.30
  6. Write tests firstline · OPSIN's pick · OPSIN predicted INERT at 0.60 · against the placebo −1.4signed verdict 361
    INHIBITS −2.8−5.6 to −0.3 ×1.90
  7. Every edge caseline · OPSIN's pick · OPSIN predicted INERT at 0.60 · against the placebo +0.2signed verdict 362
    INERT −1.2−3.9 to +1.4 ×1.41
  8. Senior engineerline · OPSIN's pick · OPSIN predicted INERT at 0.75 · against the placebo +1.6signed verdict 363
    INERT +0.2−1.9 to +2.3 ×1.06