Propose a part

Datasheet · A prompt line

Take a deep breath

Take a deep breath and work on this problem step-by-step.

Folklore, as cited: Yang et al., 2023, Large Language Models as Optimizers (OPRO)

Is not settled on DeFi maths, Haiku 5.5 with thinking off; changes nothing that matters on DeFi maths, Haiku 5.5 with thinking on; changes nothing that matters on Agent safety, Haiku 5.5 with thinking off.

DeFi maths, Haiku 5.5 with thinking off

Pooled across 2 notebooks: +2.8 points, 95% interval −0.1 to +5.5; measured once.

  1. EXCITESNotebook #003+4.4 points (+0.5 to +8.4), 432 pairs, cost ×1.17signed verdict 14
  2. UNRESOLVEDNotebook #036+1.2 points (−4.3 to +6.5), 432 pairs, cost ×1.12signed verdict 206

DeFi maths, Haiku 5.5 with thinking on

Pooled across 1 notebook: +1.3 points, 95% interval −1.8 to +4.3; measured once.

  1. INERTNotebook #010+1.3 points (−1.9 to +4.3), 432 pairs, cost ×1.15signed verdict 54

Agent safety, Haiku 5.5 with thinking off

Pooled across 1 notebook: +0.1 points, 95% interval −1.6 to +1.9; measured once.

  1. INERTNotebook #008+0.1 points (−1.6 to +1.9), 432 pairs, cost ×1.24signed verdict 40

Measure it in a setting it has not been measured in: propose it. As JSON for an agent: /v1/parts/3e1a510734f20e49. Every part: the atlas.