Kaggle Grandmasters Unveil Winning Strategies for Data Science Superpowers

Kaggle Grandmasters David Austin and Chris Deotte from NVIDIA and Ruchi Bhatia from HP joined Brenda Flynn from Kaggle at this year’s Google Cloud Next conference in Las Vegas. They shared a bit about who they are, what motivates them to compete, and how they contribute to and win competitions on the world’s largest data science competition platform. 

This blog post captures a glimpse of their discussion, including best practices for tackling machine learning problems, tips and tricks, and their favorite tools.

Meet the Kaggle Grandmasters

David Austin is a member of the NVIDIA Kaggle Grandmaster team. He’s also a Principal AI Software Engineer at NVIDIA and a technical lead for open-source LLM application development. Austin has degrees in chemical engineering, and through his work as a data scientist, he’s contributed to various sectors including semiconductor manufacturing, industrial AI, and advanced LLM applications. His work emphasizes the intersection of technology and practical solutions, aiming to push the boundaries of AI and its real-world applications. Austin is a Competitions Kaggle Grandmaster. 

Ruchi Bhatia is a computer engineer with a Master’s in Information Systems Management from Carnegie Mellon University. Currently, she’s a Product Marketing Manager for Data Science and AI at HP. Through mentorship and sharing her journey, she inspires and guides aspiring technologists to navigate their paths in the ever-evolving world of technology. She is the youngest triple Kaggle Grandmaster across the Notebooks, Datasets, and Discussion categories.

Chris Deotte is also a member of the NVIDIA Kaggle Grandmaster team and a senior data scientist at NVIDIA. After earning a B.A. in mathematics, Deotte has held careers in a variety of fields, including as a graphic artist, photographer, carpenter, and teacher. He also earned a Ph.D. in computational science and mathematics with a thesis on optimizing parallel processing and is a quadruple Kaggle Grandmaster.

Kaggle Grandmaster reflections

What fuels you to compete, and what led you to become a Grandmaster?

Deotte: Originally, it wasn’t my goal to become a Grandmaster. I joined Kaggle in 2018 and became addicted to the wonderful community, the intellectual puzzles, and the opportunities to learn. With my background as a teacher, I quickly became a Discussions and Notebooks Grandmaster through sharing. Then I became more serious and earned Competitions and Datasets Grandmaster. I think my diverse background helps me find unconventional solutions to achieve top ranks.

Bhatia: Competing on Kaggle started as a way to sharpen my skills beyond theory, taking what I learned in class or at work and applying it to messy, real-world problems. But over time, it became something more: a habit of iteration, a rush of creativity under pressure, and a constant push to get 1% better every day.

What led me to becoming a Grandmaster in Datasets, Notebooks, and Discussions wasn’t a single big win, but rather consistency, collaboration, and an obsession with learning. I studied top solutions, asked “why did this work,” and rebuilt models from scratch until I truly understood them. And I surrounded myself with people who were better than me. I didn’t try to win. I tried to improve. The ranks followed.

Looking back on your most successful competitions, what was the single most counterintuitive or unconventional approach you took that significantly impacted your results? 

Austin: It may only be unconventional in the sense that most people don’t do it often, but an approach I always take in competitions where data is provided, is to spend the first few days of a competition visualizing and understanding the data. In image competitions, I’ll render all images onto a page and scroll through and try to understand what they’re telling me. 

There are many things our brains understand and lead us to that black box [that] algorithms cannot. In NLP competitions, I’ll do the same thing with visualizing embeddings in a lower-dimensional space. This technique was of significant help in a satellite image competition and gave insight into the different distribution of images, which turned out to be real versus synthetically generated.

What’s your process for tackling a new machine learning problem? Where do you start? What sort of information or details do you look for to inform your strategies?

Deotte: Beginning a project is easy. First, we explore the data, next we build the standard baseline model for the data type. Finally, we create a local validation scheme and compute the baseline’s validation metric score. That’s our baseline score. 

Determining how to beat the baseline is the difficult task. This requires us to understand the data deeply through EDA and understand the data by observing how our baseline model behaves. This understanding will suggest new ideas to try. Then the results of implementing these new ideas will promote more understanding, which will suggest even more new ideas. We repeat this process as fast as possible!

What does your development setup look like? What tools do you find especially useful?

Austin: I’m very particular and passionate about my development setup because I love computer hardware. I’ve built every PC I’ve owned in the last 20 years, and I like to do things like overclock specific components for specific workloads, watercool components that I run heavily, and have a good networking setup to keep heavy hardware out of my home office!  

For software, I like to use environments and containers that work well out of the box so I can spend more time building solutions than setting up software and managing dependencies. I regularly use NVIDIA CUDA-X data science libraries, like NVIDIA cuML for tasks like data visualization through plotting embeddings in a lower-dimensional space like t-SNE and UMAP.

What strategy or method from your own competition experience consistently takes you to the top ranks?

Bhatia: The biggest missing link is usually problem formulation and iterative intuition. Knowing the tools is one thing. Knowing how and when to use them is what separates top competitors. On Kaggle, success often comes from:

• Deep data storytelling, understanding what the dataset is really about.
• Smart CV strategies, especially when dealing with leakage, time splits, or imbalanced data.
• Creative feature engineering or target transformations.

One technique I rely on is simulating the public/private LB split using multiple CV folds to estimate potential leaderboard shakeups. It’s a sanity check against overfitting to public scores. Also, I treat competitions like product cycles: MVP first (simple baseline), then iterate with feedback loops, not perfectionism.

What are some Kaggle tricks or tips that you find especially useful? 

Deotte: Four things. First, I suggest setting up a robust local validation. Second, accelerate your experimental pipeline with GPU using accelerators like cuML or NVIDIA cuDF in order to run experiments as fast as possible. Third, I suggest analyzing and addressing differences between train and test data. And lastly, I suggest understanding the competition metric and optimizing your solution for it. 

Bhatia: Kaggle’s strength lies in structured learning by doing. One underrated tip: read discussion forums like research papers. Top competitors often leave gold in those threads, data leakage hints, alternative metrics, or creative feature ideas.

Another key is notebook reproducibility and runtime awareness. Use lightweight scripts for debugging, heavy models only for final runs. Save intermediate features and predictions. Treat it like a production ML pipeline.

How can AI system design or prompt engineering be optimized to improve energy efficiency, computational performance, and reduce environmental impact, while maintaining output quality?

Bhatia: In competitions, we rarely talk about energy efficiency explicitly, but we intuitively design for it by pruning, optimizing inference time, or using model distillation. I believe energy-aware modeling will become a competitive edge soon, especially in enterprise and product environments. In prompt engineering and LLM pipelines, I think about latency-budgeted reasoning, balancing depth of reasoning with speed. One strategy I use is chain-of-thought prompting only when needed, and fallback to simple prompts for common cases. Same goes for token usage and memory, minimize unnecessary context where possible.

Whether it’s AI for PCs or cloud systems, energy-efficient AI = better UX and lower TCO.

How do you pick which model to use? 

Bhatia: My approach depends on the context of the problem. If it’s a Kaggle competition, I care about squeezing out performance. If it’s product-facing, I balance performance with latency, interpretability, and deployment ease. I don’t blindly chase the “best” model; I look at the data. If I see strong tabular patterns, I start with tree-based models like XGBoost or LightGBM. If it’s text-heavy, I test both classic NLP and transformer-based approaches. For image tasks, I consider pre-trained CNNs before fine-tuning larger vision models.

I always build quick benchmarks, a linear model, and a basic tree, to get a feel for the signal. These give me a mental map of how complex the solution needs to be. From there, I iterate. I test a few architectures or frameworks, compare cross-validation scores, and track generalization.

I don’t commit too early. I treat models like hypotheses: test, learn, and pivot if needed. The key is not just picking the right model; it’s knowing when to move on from the wrong one.

NVIDIA has a whole team of Kaggle Grandmasters. What does that team do? 

Deotte: The team is a dream job. As part of our work responsibilities, we can compete on Kaggle. We use our observations and learnings from competing to help develop NVIDIA products, improve NVIDIA internal projects, and help NVIDIA customers. Our specialty is improving model accuracy, and there are many opportunities requiring our expertise at NVIDIA, whether it be building better tools or building better models. 

Learn more

The NVIDIA Kaggle Grandmaster team works on many competitions; these posts highlight a couple of their recent wins, Competition with Feature Engineering and AI Mathematical Olympiad.

To accelerate your machine learning models, learn about NVIDIA cuML and test drive it in this notebook.  

You can read more about cuML in our post, NVIDIA cuML Brings Zero Code Change Acceleration to scikit-learn, and from NVIDIA experts by enrolling in a hands-on course from our Accelerated Data Science Learning Path. 

For questions on CUDA-X libraries like cuML or to leave feedback, join us on Slack at #RAPIDS-GoAi.