I recently had the privilege of listening to a talk by Chen Tianqi, the creator of well-known open-source projects like TVM, XGBoost, and MXNet. His story is not just a chronicle of a tech leader’s growth, but a profound exploration of choice, perseverance, and courage.
The Early Years: A Self-Taught ACMer
Chen’s journey into research began in high school, where he taught himself ACM (International Collegiate Programming Contest) and even wrote his own compiler. This experience taught him the immense power of learning and collaboration through online forums. At Shanghai Jiao Tong University, he joined the ACM class, where he learned to “do things low-key, but be a person high-key” and to balance ambition with a cooperative spirit. He also credits the class’s mandatory student presentations for significantly improving his learning and public speaking skills.
A Breakthrough in Research: Persisting with the Right Problems
During his third year of university, Chen joined Microsoft Research Asia and was introduced to machine learning. He recalls buying a GPU and hand-coding CUDA to accelerate inference. In his third year of graduate school, he participated in the KDD Cup, using Restricted Boltzmann Machines for a recommendation system, which earned him a chance to study in North America.
Reflecting on that period, he shared a valuable lesson: “I persisted with the wrong method for the right problem for too long, and I couldn’t persist on two things at once.” This realization taught him that he couldn’t choose both the problem and the method simultaneously. However, he also learned to embrace challenges and accept failure, which gave him the courage to take on a wide variety of tasks.
He explained his unique “research taste,” identifying two types of researchers: those who apply a single method to various problems, and those who focus on one type of problem and explore different methods to solve it. Chen chose the latter, which constantly pushed him to break through technical bottlenecks, especially in the realm of systems.
The Rise of Open Source: The Story of XGBoost
Chen attributes the success of XGBoost to several key factors:
- Taking it to the Extreme: Focusing on one thing and doing it exceptionally well.
- Algorithm-System Co-design: Going beyond just the algorithm to focus on the synergy between the algorithm and the system. For instance, XGBoost’s built-in capability to automatically handle missing values makes it ready to use out-of-the-box.
- Community Power: A strong community is vital to a project’s success.
- Focus: Instead of building a comprehensive, all-encompassing system, he chose to focus on a single algorithm.
Although neural networks are now dominant, he believes that XGBoost remains highly effective for tabular data and time series analysis. He admits that he didn’t foresee the composability of neural network blocks.
MXNet and TVM: Seeing the Future
After the release of AlexNet, Chen recognized the immense potential of deep learning and collaborated with others to create the deep learning framework MXNet. Drawing on lessons from previous frameworks, they incorporated concepts like “Python-first,” automatic forwarding, and automatic differentiation.
However, MXNet eventually faded out. Chen concluded that the main reasons were a lack of focus on user experience and insufficient support for infrastructure. This experience led him to develop TVM, a deep learning compiler aimed at reducing the complexity of ML systems. This “don’t know your own strength” mindset has always been with him. As he says, “It takes the same amount of time to work on a mediocre problem as it does on an interesting one.”
The Path of an Entrepreneur: Beyond Technology
Chen shared his experiences at his startup, OctoML, which is based on the TVM open-source project. He noted that the company’s ultimate direction often differs from its initial plans and that people skills—communication, collaboration, and management—are far more critical than just technical expertise. He stated that he wouldn’t change any of his decisions, viewing every challenge as a valuable lesson and a part of the journey.
The Long Game: Embracing Failure and Staying True to Your Passion
At the end of his talk, Chen discussed the importance of long-term thinking. He believes that setbacks are an inevitable part of the journey, especially when stepping out of one’s comfort zone. The key is to accept failure and stay true to your original passion. He emphasized that no matter how successful you become, maintaining your passion for technology is what matters most.
Chen Tianqi’s journey shows that research and life are full of uncertainties and challenges. But by daring to try and learning from every experience, each setback becomes a stepping stone toward a greater destination.