Cloud Hosting

One of the main takeaways is transparency. The checklist calls on researchers to provide detailed descriptions of each machine learning model, including the code, the data used to train and test the model, the hardware specifications used to produce the results, the experimental design, the projects goals and any limitations of the studys findings. The standards are flexible enough to accommodate a wide range of nuance, including private datasets and complex hardware configurations, according to the authors.

While the increased rigor of these new standards might slow the publication of any given study, the authors believe wide adoption of these standards would increase the overall rate of discovery and innovation, potentially by a significant amount.

What we ultimately care about is the pace of scientific progress, said sociologist Emily Cantrell, one of the lead authors, who is pursuing her Ph.D. at Princeton. By making sure the papers that get published are of high quality and that theyre a solid base for future papers to build on, that potentially then speeds up the pace of scientific progress. Focusing on scientific progress itself and not just getting papers out the door is really where our emphasis should be.

Kapoor concurred. The errors hurt. At the collective level, its just a major time sink, he said. That time costs money. And that money, once wasted, could have catastrophic downstream effects, limiting the kinds of science that attract funding and investment, tanking ventures that are inadvertently built on faulty science and discouraging countless numbers of young researchers.

In working toward a consensus about what should be included in the guidelines, the authors said they aimed to strike a balance: simple enough to be widely adopted, comprehensive enough to catch as many common mistakes as possible.

They say researchers could adopt the standards to improve their own work; peer reviewers could use the checklist to assess papers; and journals could adopt the standards as a requirement for publication.

The scientific literature, especially in applied machine learning research, is full of avoidable errors, Narayanan said. And we want to help people. We want to keep honest people honest.

The paper, Consensus-based recommendations formachine-learning-based science, published on May 1 in Science Advances, included the following authors: Sayash Kapoor, Princeton University; Emily Cantrell, Princeton University; Kenny Peng, Cornell University; Thanh Hien (Hien) Pham, Princeton University; Christopher A. Bail, Duke University; Odd Erik Gundersen, Norwegian University of Science and Technology; Jake M. Hofman, Microsoft Research; Jessica Hullman, Northwestern University; Michael A. Lones, Heriot-Watt University; Momin M. Malik, Center for Digital Health, Mayo Clinic; Priyanka Nanayakkara, Northwestern; Russell A. Poldrack, Stanford University; Inioluwa Deborah Raji, University of California-Berkeley; Michael Roberts, University of Cambridge; Matthew J. Salganik, Princeton University; Marta Serra-Garcia, University of California-San Diego; Brandon M. Stewart, Princeton University; Gilles Vandewiele, Ghent University; and Arvind Narayanan, Princeton University.

Adapted from a Princeton University release

Learn more about research and education at UC San Diego in: Artificial Intelligence

Follow this link:
Science has an AI problem. This group says they can fix it. - University of California San Diego