Brad Cao
Brad Cao
Math & CS at Columbia

I'm an undergrad at Columbia studying mathematics and computer science. I'm a research engineering intern at Sampura Research and a research fellow under the Supervised Program for Alignment Research, and previously worked at Mercor, Duke University, and the University of Maryland, Baltimore.

I'm always excited to meet ambitious and smart people.

Now
Research Engineering Intern, Sampura Research; Research Fellow, SPAR
Studying
B.A. Mathematics & Computer Science, Columbia '30
Based in
New York, NY

Education

Columbia UniversityExpected May 2030
B.A. in Mathematics & Computer Science · New York, NY
Relevant coursework: Data Structures in Java, Probability Theory, Natural Language Processing, Honors Mathematics A & B.
Activities: Columbia Alpha Partners (Global Markets Division), Columbia Artificial Intelligence Alignment Technical Fellowship, Columbia Poker Club.

Experience

Research Engineering InternOct 2026 – Present
Sampura Research · Remote
  • Onboard models and datasets onto Sampura’s scalable-oversight evaluation infrastructure end to end, so new judge models and new eval sets can be scored against each other in production.
  • Vet ~200 public AI safety and alignment datasets for threat-model fit, label provenance, ground-truth reliability, and contamination risk; request private or held-out sets from external research groups so judges are tested on data they cannot have memorized.
  • Deliver monthly summary reports and a final report on datasets reviewed, integrated, and outstanding.
Research FellowSep 2026 – Present
  • Build evaluation datasets that test whether humans working with AI “judges” can reliably oversee other AI systems, as part of a 100+ dataset leaderboard covering deception, jailbreak robustness, cultural bias, and computer-control mistakes.
  • Audit existing judge datasets for coverage gaps and create new ones to fill them, using sandwiching for ground truth.
  • Measure how easily judges are fooled by reward hacking under adversarial RL optimization.
Data Science InternApr 2026 – Sep 2026
Mercor · San Francisco, CA
  • Developed 10+ rubric-based scoring taxonomies that turn expert judgment on professional tasks into consistent criteria for evaluating and training LLMs (RLHF annotation schemas).
  • Drafted pass/fail criteria with acceptable and unacceptable example responses so expert reviewers score consistently.
  • Flagged edge cases (ambiguous, incomplete, or conflicting inputs) that ordinary prompts miss and added them to the rubrics.
  • Audited labeled data for reviewer disagreements and suggested guideline updates to raise inter-annotator agreement.
Research AssistantApr 2026 – Sep 2026
Duke University · Durham, NC
  • Designed and built an LLM pipeline (GPT-5.5, OpenAI Batch API) that analyzed 18.7M words across 3,543 cable news transcripts (CNN, Fox, MSNBC; 2011–2025), one batch job per transcript.
  • Refined a 20-page coding protocol over 18 versions to reach ~99.4% accuracy on flagged cases with no missed cases; cleaned the data.
  • Built an auditable review workflow: 12,892 flagged candidates, 1,584 ambiguous (12.3%) sent to human review; found Fox disparages media at 10× CNN’s rate, 4× MSNBC’s.
Leadership Seminar ScholarJun 2025 – Aug 2025
University of Notre Dame · South Bend, IN
  • Won a full-ride scholarship to Notre Dame’s “Inequality in America” seminar; conducted original meta-research on how the pharmaceutical industry shapes health policy, reviewing 15+ studies.
  • Ran regression analysis and visualizations in R linking industry practices to poverty outcomes, connecting to the seminar’s focus on economic inequality; presented at a symposium.
Research InternJun 2023 – Sep 2025
University of Maryland, Baltimore · Baltimore, MD
  • First-authored 2 studies, presented at the ASCPT Annual Meetings (2024, 2025); the 2025 poster won the Top Poster Award.
  • Built ML models (logistic regression, random forests) on 10 years of FDA adverse-event reports to predict serious outcomes in ADHD patients on stimulants; tuned with cross-validation; best model reached 0.75 AUC and 88% recall.
  • Modeled tacrolimus dose requirements and transplant outcomes in lung transplant recipients taking azole antifungals.

Skills & Interests

  • Languages: Python, JavaScript, Java, SQL (PostgreSQL), R, HTML/CSS
  • Tools & ML: PyTorch, TransformerLens, NumPy, pandas, scikit-learn, Django, Linux; mech interp, LLM fine-tuning, RLHF
  • Interests: Fencing (USA Fencing First Team All-American), Competitive Programming (USACO), History Olympiads, Open-Source
ThresholdSolo project
Python, NumPy, SciPy, PyTorch
  • Reproduced double descent on MNIST with random-feature regression (2,000 samples, 20% label noise): error peaks at 88.8% when parameters = samples, vs. 24.3% at 5× more.
  • Identified the cause: the solution's norm explodes at the peak (811 vs. 2.6) and 90% of the peak error is variance; tuned ridge regularization cut the peak from 88.8% to 12.3%.
  • Showed more data can hurt (error 34.8% to 89.1% as samples grew 250 to 2,000); replicated on Fashion-MNIST; wrote 85 tests at 94% coverage.
MyCureHopeCo-developer
Django, PostgreSQL, JavaScript, TailwindCSS
  • Co-developed an open-source database of treatments, doctors, and hospitals used by 33,000+ users across 78 countries.
  • Built the search feature, own the technical development, and deployed the full stack on Nginx/Gunicorn/Ubuntu.
Leveraging Supervised Machine Learning to Identify Predictors of Serious Clinical Outcome with Stimulant Treatment from Real-World Data
ASCPT Annual Meeting, 2025 · Washington, DC · Top Poster Award
Cao, B., Lee, S., Naga, A., Gobburu, J., Dunn, A.
Predictive Modeling of Tacrolimus Dose Requirements and Transplantation Outcomes in Lung Transplant Recipients on Systemic Azole Antifungals
ASCPT Annual Meeting, 2024 · Colorado Springs, CO
Cao, B., Gupta, P., Dunn, A.