OpenAI Scholars 2020: Final Projects

These projects investigated problems such as analyzing how GPT-2 represents grammar, measuring the interpretability of models trained on Coinrun, and predicting epileptic seizures using brain recordings. More information about the next class of Scholars and how to apply will be announced this fall.

The OpenAI Scholars program provides stipends and mentorship to individuals from underrepresented groups to study deep learning and open-source a project.

Our Scholars have demonstrated core technical skills across various expert domains and self-motivation—critical competencies for a self-directed program like this one. They each entered the field of machine learning as relative newcomers, and we hope their progress shows how accessible machine learning is.

Demo Day introductions by Sam Altman and Greg Brockman

Learn more about our Scholars program.

Looking for Grammar in All The Right Places
Alethea Power

Mentor: Christine Payne
Previous Roles: B.S. in Applied Mathematics, MSc in Philosophy of Mind from Ediburgh, Software and Site Reliability Engineer at Facebook

I’m fascinated by neural network interpretability. Understanding how networks of various architectures represent information can help us build simpler and more efficient networks, as well as predict how the networks we’ve built will behave, and perhaps even give us some insight into how human beings think. Along these lines, I analyzed how GPT-2 represents English grammar, and found smaller sub-networks that seem to correspond to various grammatical structures. I will present my methodology and results.

Next, I want to work on understanding how neural networks represent information, and use that understanding to better predict how deep learning systems behave. I believe this work will make such systems safer and more beneficial to humanity, as well as making them simpler, faster, and more computationally efficient.

Blog<!– GitHub Repo –>

Semantic Parsing English to GraphQL
Andre Carerra

Mentor: Melanie Subbiah
Previous Roles: CTO at Droplii, Founder at Lambdo

My scholars program project is semantic parsing English-to-GraphQL. Given an English prompt such as “How many employees do we have?”, find a corresponding GraphQL query to return the information. The project involved creating a dataset, training models, and creating an interaction tool to see results.

I wanted to have a say in how AI is shaped—the Scholars program has been a great opportunity to learn and participate.

Blog<!– GitHub Repo –>

Long Term Credit Assignment with Temporal Reward Transport
Cathy Yeh

Mentor: Jerry Tworek
Previous Roles: Data Scientist at Square and Driver

Standard reinforcement learning algorithms struggle with poor sample efficiency in the presence of sparse rewards with long temporal delays between action and effect. To address the long term credit assignment problem, we use “temporal reward transport” (TRT) to augment the immediate rewards of significant state-action pairs with rewards from the distant future, using an attention mechanism to identify candidates for TRT. A series of gridworld experiments show clear improvements in learning when TRT is used in conjunction with a standard advantage actor critic algorithm.

I appreciate that this program gave me the freedom to learn deeply and flex my creativity.

Blog<!– GitHub Repo –>

Quantifying Interpretability of Models Trained on Coinrun
Jorge Orbay

Mentor: Karl Cobbe
Previous Roles: CS Engineering at Columbia, Research at the Creative Machines Lab, Software Engineer at Autonomic

This project’s purpose is to create a scalar that measures the interpretability of an A2C model trained on Procgen’s Coinrun. The scalar is generated using a combination of attribution on the model and masks of Coinrun’s assets. The scalar is used to test the validity of the diversity hypothesis.

This program, and specifically my mentor, has fostered a self-confidence in me to dive into a field I don’t understand and break down problems until I can solve them. I’m hoping to take the self-confidence I’ve learned from this program to continue breaking down problems in and with AI.

Blog<!– GitHub Repo –>

Social Learning in Independent Multi-Agent Reinforcement Learning
Kamal Ndousse

Mentor: Natasha Jaques
Previous Roles: Math and Physics at MIT, Algorithms Research Scientist at Fitbit, Independent Algorithms/ML consultant, ML Engineer at Coinbase

My project has explored the social transfer of expertise among completely independent RL agents trained in shared environments. The motivating question is whether novice agents can learn to mimic expert behavior to solve hard-exploration tasks that they couldn’t master in isolation. I’ll discuss my observations as well as the environments I developed to experiment with social skill transfer.

I joined the Scholars program in order to learn from the brilliant folks at OpenAI and to immerse myself in AI research. I’m grateful to have had the opportunity to explore state of the art research with the support of such talented researchers (special thanks to my mentor Natasha Jaques!)

Blog<!– GitHub Repo –>



Towards Epileptic Seizure Prediction with Deep Network
Kata Slama

Mentor: Johannes Otterbach
Previous Roles: PhD in Neuroscience at UC Berkeley, Behavioral Research at Harvard and Brown

I have been working on a project to predict epileptic seizures using brain recordings. I framed it as an image classification problem based on the spectrogram representation of the brain data. My most successful model so far has been a ResNet18. In my post-Scholars life, I plan to continue working on this project, and make my way to interpretability of spectrogram classification networks.

I wanted to learn how to apply deep learning for solving scientific and real-world problems. The OpenAI Scholars program was this magical opportunity to get started by learning from the very best minds in the field.

Blog<!– GitHub Repo –>



Universal Adversarial Perturbations and Language Models
Pamela Mishkin

Mentor: Alec Radford
Previous Roles: Math and CS at Williams College, Research Analyst at the Federal Reserve Bank of NY, Herchel Smith Scholar at Cambridge, Product Manager at The Whistle, Researcher at Lumi Labs

Adversarial perturbations are well-understood for images but less so for language. My presentation will review the literature on how universal adversarial examples can inform understanding of generative models, replicating results generating universal adversarial triggers for GPT-2 and for attacking NLI models.

This program strengthened my technical basis in machine learning and helped me understand how AI researchers understand policy implications of their work.

Blog<!– GitHub Repo –>

Diversity is core to AI having a positive effect on the world—it’s necessary to ensure the advanced AI systems in the future are built to benefit everyone.

If you’re excited to begin your own journey into ML, check out some of our educational materials. More information about the next class of scholars and how to apply will be announced this fall. Stay tuned!

Huge thanks to Microsoft for providing Azure compute credits to scholars, to our mentors for their time and commitment, and to all the supporters that made this program possible.