@Richard I agree that hands-on experimentation is essential. One challenge I've repeatedly encountered while evaluating frontier AI models is that many interesting failures are observed once but aren't preserved in a way that enables independent verification or follow-up research. I'm currently developing an evidence-based methodology focused on documenting, preserving, and reproducing AI evaluation evidence. Since you mentioned information aggregation and verification as an interest, I'd be interested to know whether you think standardized evidence preservation could become a useful part of AI safety evaluation infrastructure.

@Austin Thank you for building Manifund. As an independent AI evaluation researcher from India, I've found the platform uniquely accessible compared with traditional funding routes. One question I've been thinking about is how grantmakers can better evaluate the quality and reproducibility of AI evaluation work before funding it. I'm exploring an evidence-based methodology around this problem and would be interested in your perspective on whether stronger evidence standards could improve grantmaking over time.

@gleech I agree that research infrastructure is an underrated bottleneck. One issue I've repeatedly encountered while evaluating frontier AI models is that many evaluation claims aren't accompanied by enough preserved evidence for independent verification or reproduction. Improving evidence preservation alongside benchmarking could make follow-up research much more reliable. I'm currently exploring this problem through an open methodology and would be interested in your thoughts if you think this is a worthwhile direction.