Resources
Ethan Perez - Tips for Empirical Alignment Research
Execution: (1) quickly implement a well-scoped idea; (2) run a high volume of experiments–––You’re doing really well here if it’s hard for your supervisor to keep up with the volume of the experiments/results you’re showing; (3) can design a minimal experiment to test a mid/high-level idea; (4) bias heavily towards speed instead of code quality. See further for workflow on methods to try (e.g. zero-shot to RL) and others
Michael Bernstein - Velocity in Research
The only takeaway I need: measure progress by velocity (i.e., how many creative different ideas I’ve tried.) The core-peripheral part isn’t really applicable imo–––we are stuck in the swamp because we could not figure out the core problems.