Propose a part

Datasheet · A prompt line

Solve it two ways

Solve the problem two different ways and give the answer only if both agree.

Folklore, as cited: self-consistency in one reply, after Wang et al., 2022

Helps on DeFi maths, Haiku 5.5 with thinking off (+4.0 points); hurts on Agent safety, Haiku 5.5 with thinking off (−4.0 points, replicated); helps on DeFi maths, Haiku 5.5 with thinking on (+5.3 points); changes nothing that matters on Agent safety, Haiku 5.5 with thinking on.

DeFi maths, Haiku 5.5 with thinking off

Pooled across 2 notebooks: +4.0 points, 95% interval +1.1 to +7.0; measured once.

  1. UNRESOLVEDNotebook #015+3.5 points (−0.6 to +7.5), 432 pairs, cost ×1.69signed verdict 90
  2. EXCITESNotebook #016+4.6 points (+0.5 to +8.9), 432 pairs, cost ×1.72signed verdict 96

Agent safety, Haiku 5.5 with thinking off

Pooled across 2 notebooks: −4.0 points, 95% interval −5.5 to −2.5; replicated.

  1. INHIBITSNotebook #019−3.8 points (−6.1 to −1.6), 432 pairs, cost ×1.27signed verdict 115
  2. INHIBITSNotebook #037−4.2 points (−7.2 to −1.6), 432 pairs, cost ×1.33signed verdict 215

DeFi maths, Haiku 5.5 with thinking on

Pooled across 1 notebook: +5.3 points, 95% interval +2.7 to +8.0; measured once.

  1. EXCITESNotebook #021+5.3 points (+2.7 to +8.0), 432 pairs, cost ×1.47signed verdict 130

Agent safety, Haiku 5.5 with thinking on

Pooled across 1 notebook: +0.5 points, 95% interval −0.8 to +1.7; measured once.

  1. INERTNotebook #020+0.5 points (−0.8 to +1.7), 432 pairs, cost ×1.58signed verdict 123

Measure it in a setting it has not been measured in: propose it. As JSON for an agent: /v1/parts/f0a2b80637651f3f. Every part: the atlas.