Propose a part

Datasheet · A prompt line

Read the question again

Read the question again before you answer.

Folklore, as cited: Xu et al., 2023, Re-Reading Improves Reasoning in Large Language Models

Helps on DeFi maths, Haiku 5.5 with thinking off (+5.2 points, replicated); hurts on DeFi maths, Haiku 5.5 with thinking on (−2.3 points); changes nothing that matters on Agent safety, Haiku 5.5 with thinking off.

DeFi maths, Haiku 5.5 with thinking off

Pooled across 3 notebooks: +5.2 points, 95% interval +2.7 to +7.6; replicated.

  1. UNRESOLVEDNotebook #003+3.9 points (−0.3 to +8.2), 432 pairs, cost ×1.12signed verdict 18
  2. EXCITESNotebook #005+7.2 points (+2.9 to +11.6), 432 pairs, cost ×1.14signed verdict 23
  3. EXCITESNotebook #007+4.5 points (+0.6 to +8.4), 432 pairs, cost ×1.13signed verdict 34

DeFi maths, Haiku 5.5 with thinking on

Pooled across 2 notebooks: −2.3 points, 95% interval −4.6 to −0.1; measured once.

  1. UNRESOLVEDNotebook #010−3.1 points (−6.4 to +0.0), 432 pairs, cost ×1.09signed verdict 57
  2. INERTNotebook #011−1.5 points (−4.8 to +1.6), 432 pairs, cost ×1.08signed verdict 63

Agent safety, Haiku 5.5 with thinking off

Pooled across 1 notebook: +1.0 points, 95% interval −1.0 to +3.1; measured once.

  1. INERTNotebook #008+1.0 points (−1.0 to +3.1), 432 pairs, cost ×0.95signed verdict 43

Measure it in a setting it has not been measured in: propose it. As JSON for an agent: /v1/parts/aed0a14114f3e60e. Every part: the atlas.