Propose a part

Notebook #058 · published

Coding, Haiku 5.5 with thinking off

Exact means the function returned the right answer on every hidden case. Each part ran against the mean of two unchanged runs of the same agent, on 432 paired tasks drawn by a public beacon after the design was hashed. The unchanged agent was exact on 92.4% of them. Rule v2: every interval is widened for the 7 arms this notebook compares, so luck flags any of them at most 5% of the time.

Thinking on helped. 5 parts did nothing measurable. The placebo, a line that should do nothing, moved +3.5 points, so small effects here deserve less trust. OPSIN called 5 of 6 before the beacon drew the tasks.

Preregistered
sha256 c03838a6738a482006917282d0e8c982d30aa8bfaed9987982463f4e90f45830
the full design, as JSON
Tasks drawn by
drand round 6,622,019
seed drand:6622019:c0cee22cda9ad968542f07bd2a2f764614baa1fab7d5edec2eab4ded8d2cee70
Signed by
0xFf8E1E8ebF21064Ef781B75CdC8DFbd969399EbD
EIP-191 over sha256 of each verdict’s canonical JSON. Verify one yourself
Every run
Each run of every arm, with its task’s family and whether it was exact: CSV or JSON. With the seed, every verdict above can be recomputed from them.
Recorded on Elysium
transaction 0x5c1992edd6d979a8a298d5bd6e79ac12f524414a5876b2f612b57395aaec66a1
Written once, by the lab, and nothing can change it. The contract

The placebo moved. A line that should do nothing came out EXCITES (+3.5 points), which luck alone does in about one notebook in 140. Read this notebook’s small effects with less trust; each part is also compared with the placebo on the same tasks, beside its verdict.

  1. Control check (A/A)the unchanged agent against itselfsigned verdict 364
    INERT +0.5−1.9 to +2.8
  2. PlaceboThis session uses the default display settings.signed verdict 365
    EXCITES +3.5+0.8 to +6.3 ×0.96
  3. Thinking onsetting · OPSIN's pick · OPSIN predicted INERT at 0.55 · re-measures verdict 358 (EXCITES +4.2): it held · against the placebo +1.2signed verdict 366
    EXCITES +4.6+2.0 to +7.5 ×2.17
  4. Write tests firstline · OPSIN's replication · OPSIN predicted INERT at 0.60 · against the placebo −4.6signed verdict 367
    INERT −1.2−4.2 to +1.6 ×1.90
  5. Trust user notes anywhereline · OPSIN's follow-up · OPSIN predicted INERT at 0.80 · against the placebo −3.9signed verdict 368
    INERT −0.5−3.7 to +2.7 ×0.98
  6. Untrusted-data ruleline · OPSIN's follow-up · OPSIN predicted INERT at 0.82 · against the placebo −1.9signed verdict 369
    INERT +1.6−1.4 to +4.5 ×1.03
  7. Shouted warningline · OPSIN's follow-up · OPSIN predicted INERT at 0.80 · against the placebo −3.7signed verdict 370
    INERT −0.2−2.9 to +2.4 ×1.01
  8. Take a deep breathline · OPSIN's follow-up · OPSIN predicted INERT at 0.82 · against the placebo −2.3signed verdict 371
    INERT +1.2−1.7 to +4.0 ×1.11