A Quick Update

I’ve recently completed the scholars program at OpenAI and it’s been an amazing place to journey into deep learning. I’ve explored so many ideas and learned all about PyTorch, transformers, tuning and training models, as well as some of the implications that need to be considered when looking at releasing powerful models that can change society.

A lot of my experiences occurred through my scholars project, which I am still wrapping up. I looked at how different tokenization approaches affects how well a model performs. I got lost in the fine-tuning sauce and didn’t get the results I wanted by the end of the program! Fortunately I still have a bit of compute leftover to run a few more experiments. Looking forward to sharing the results and my code!