- This event has passed.
CIS Special Industry Seminar: “Improving Foundation Models Using Human Data”
October 8 at 3:30 PM - 4:30 PM
Foundation models including LLMs and multi-modal models released by OpenAI (GPT), Anthropic (Claude), Google (Gemini), Meta (Llama), and others have shown very impressive capabilities across a range of tasks. Some key drivers of this performance — such as investments in GPUs/compute, model size, and pre-training data — are relatively well understood.
This presentation will focus on a less understood, yet extremely powerful lever that creates significant differentiation and competitive advantage among state-of-the-art models: the use of expert human data for Evaluations (“Evals”), Supervised Fine Tuning (“SFT”), Reinforcement Learning with Human Feedback (“RLHF”), and Direct Preference Optimization (“DPO”).
The talk will also outline some best practices for maximizing returns on financial investments in human data to achieve optimal model performance. This includes effective strategies for sourcing, vetting, hiring, and managing expert human data teams, as well as task design for Evals, SFT, RLHF, and DPO, along with processes and tooling to optimize team performance, data quality and throughput.
This talk is open to Penn students, COLM attendees, and anyone in the Philadelphia area interested in AI, data science, or machine learning. You don’t need to be affiliated with the conference or the university — if you’re part of the DS/ML/AI community, you’re welcome to join us.
Vijay Krishnan
Founder and CTO of Turing
Vijay Krishnan is Founder and CTO of Turing.com and leads its AI and machine learning initiatives. With over 20 years of experience across academia and industry, he has spearheaded large-scale machine learning projects addressing a wide range of industry challenges. Before founding Turing, Krishnan served as CTO of Rover (acquired by Revcontent), was a scientist at Yahoo, and has authored widely cited papers on ML and NLP. He holds degrees in Computer Science from Stanford University and IIT Bombay.
About Turing:
Turing.com enables foundation model companies like Open AI, Anthropic, Google, Nvidia, Meta, Microsoft, and many others to improve their capabilities in areas such as reasoning, coding, function calling, multimodality, factuality, safety, and more through expert human data for Evals, SFT, RLHF, and DPO. Turing also helps large enterprises like Pepsi, Disney, and J&J strategize and execute various AI solutions to create outsized business value. Turing achieves both these through its talent pool of over 3 million software developers, data scientists, and other knowledge workers spanning 100+ countries, all automatically vetted through 100+ tests of skills and expertise. Founded in 2018, Turing became a unicorn in 2021.