Dr. Bonnie Kruft currently serves as the Senior Director at Microsoft’s AI for Science, and in our conversation, we cover everything from quantum chemistry to data science. Tuning in you’ll hear Dr. Kruft unpack how she went from earning a Ph.D. focused on quantum chemistry, to working in AI and machine learning.
Dr. Kruft unpacks how she went from earning a Ph.D. focused on quantum chemistry, to working in AI and machine learning. She shares how she first discovered her love of data science, and how her Ph.D. equipped her with the skills she needed to transition into this new and exciting field. We also discuss the data science approach to problem-solving, deep learning emulators, and the impact that machine learning could have on the natural sciences.
Key Points From This Episode:
Tweetables:
“Although I wasn't really working on machine learning, or data science during my Ph.D., there's a lot of transferable skills that I picked up along the way while I was working on quantum chemistry.” — Bonnie Kruft [0:03:00]
“We believe that deep learning could have a really transformational impact on the natural sciences.” — Bonnie Kruft [0:13:02]
“The idea is that deep learning emulators will be used for the things that are going to make the most impact on the world. Solving healthcare challenges, combating disease, combating climate change, and sustainability. Things like that.” — Bonnie Kruft [0:21:29]
Links Mentioned in Today’s Episode:
[INTRODUCTION]
[0:00:04.5] RS: Welcome to How AI Happens, a podcast where experts explain their work at the cutting-edge of artificial intelligence. You’ll hear from AI researchers, data scientists and machine learning engineers as they get technical about the most exciting developments in their field and the challenges they’re facing along the way. I’m your host, Rob Stevenson, and we’re about to learn How AI Happens.
[INTERVIEW]
[0:00:32.4] RS: Here with me today on How AI Happens is the Senior Director over at Microsoft’s AI for Science, Dr. Bonnie Kruft. Bonnie, welcome to the podcast, how are you today?
[0:00:41.0] BK: Hi, I’m great, thanks so much for having me.
[0:00:43.7] RS: Really pleased to have you, you are dialing in from London where it is a somber state after the queen has passed. We just got some color commentary on the Royals and the mood over there, which I appreciated but at a certain point here, we had to start talking about AI. So we’ll see if our listeners from that, you know, there’s a lot of other podcast people can listen to if they want takes on the monarchy.
But in the meantime, Bonnie, really interested in learning more about you. I guess before we get too far along, would you share a little bit about your background and your research and then, we’ll get into AI for science and the goals of that campaign?
[0:01:17.2] BK: Certainly. So I did my PhD in quantum chemistry and there, I applied density functional theory to explore the properties of transition metal complexes. So quantum chemistry is a branch of physical chemistry and it’s focused on applying quantum mechanics. So the fundamental theory and physics that describe the properties of molecules, materials and solutions at the atomic level.
So quantum chemistry is that, applied to chemical systems. Intensity functional theory is one such approximation and it’s a tool for exploring molecular properties. It is really based on a theory that the exact energy can be determined from the knowledge of the electron density and can drive new and important insights into the way that molecules work.
[0:02:02.0] RS: Got it and then how did that connect into AII? Why was that the next logical step for your career?
[0:02:07.6] BK: Yeah, great question. So during my PhD, I grew and interest in data science and machine learning and I was also involved with a pharmaceutical contract research organization that was working on clinical studies at the time and after my PHD, I went directly into the pharmaceutical industry.
So I started with GSK, played a variety of roles there in data science and machine learning and that eventually ended up at AstraZeneca where I was leading a global data and AI team. So I had a team of about 150 data and machine learning engineers and we were delivering all of the data and AI services for bio farm and oncology research.
[0:02:48.3] RS: Got it. So I miss like – so data science then was the bridge for you. What kind of – in what ways I guess was the quantum chemistry background contributing to the pharmaceutical research and production?
[0:02:59.5] BK: Yeah, that’s a good question. So although I wasn’t really working on machine learning or data science during my PHD, there’s a lot of transferrable skills that I’ve picked up along the way while I was working on quantum chemistry. So it’s really about how you use molecular simulation, computation simulation, alongside experimental data and computational power to drive some insights into something that’s particularly really complex.
And that’s pretty familiar, pretty similar to the modeling of data, massive quantities of data, massive volumes of data alongside computational powers to thrive some insights and to say, you know, the correlation between a gene and a disease or the design of a small molecule that’s going to modulate a disease, a disease target and so on and so forth.
So there is a lot of transferrable skills that I’ve picked up during my PHD that were easy to kind of move and transition into the machine learning world and data science world. What I started at GSK, it was sort of the beginning stages of most pharma companies entering that sort of data driven drug discovery world. So I was part of a small team that kind of acted like a startup within a big company.
We were both trying to build something, top down, bottom up, bring silo data sources together while at the same time trying to build something on top of it and it was kind of that brute force of, how do we try something out, how do we build something at the same time, we’re not really sure where we’re going but we’re going to experiment and see where we can get to and inspire people to start to think about how we use data in a different way.
[0:04:32.1] RS: Would you say that in your experience, the data science approach is now a driving factor or is it merely a tool in the toolbelt? How impactful is it, how top of mind and prioritized in driving innovation at least in the pharmaceuticals?
[0:04:45.9] BK: I think most pharmaceutical companies are making some sort of effort towards data driven drug discovery. So data science being sort of a broader branch of, I am collecting data and I’ve been driving insights into many components of the pipeline, machine learning being a bit more robust in looking at how we can actually build systems that learn and impact and influenced different components of the R&D pipeline too.
So, most companies are looking at how we can accelerate or augment or transform some step of the drug discovery process. Whether that’s doing something a little bit faster, whether that’s reducing the number of failed experiments that you have in the lab or really just increasing the probability of success that you're going to find the right target, the right molecule, the right patient that’s actually going to end up becoming a medicine that makes it to the market and is successful.
[0:05:40.6] RS: It’s interesting those approaches you outlined, you rattled off a handful there. Those aren’t necessarily pharmaceutical industry dependent, like, that just seems like standard, sort of data science approach to problem solving, is that fair to say?
[0:05:55.1] BK: I think you’re probably right. It’s really about making better decisions, which is probably what you're trying to do in every industry with data science and machine learning. “How do get to a better decision faster?” or “How do I eliminate the wrong decision faster?” which is often times, what we’re trying to do. It’s the concept of failing faster.
[0:06:16.0] RS: How do you fail faster Bonnie?
[0:06:18.2] BK: So, you fail faster by identifying what things aren’t going to work, right? So, in drug discovery, it’s what molecule is going to have potentially off target effects or what synthetic root is going to be wasteful or expensive or end up failing in the long run or not being able to scale.
[0:06:39.6] RS: I see. I want to go back a little bit just because we’ve maybe jumped in the deep end here. I want to ask you why quantum chemistry, for you personally?
[0:06:49.5] BK: Why I chose quantum chemistry, that’s a good question. So I think my early career was I kind of bounced around quite a little habit. So I study chemistry for my undergraduate degree, and I didn’t really know what I want to do with my life. So, someone told me, “Oh, you have a degree in chemistry? You can go make wine, you know?” I said, “Oh, okay” I actually ended up going to Australia and New Zealand for about two years working in a lab making wine and then I kind of thought, “You know what? I really do love chemistry and I’d like to go back and get my PHD.”
So I initially sought out to get my PHD in organic chemistry and I found a professor that I wanted to work with in New York in the organic chemistry lab doing photochemistry and there happened to be a project there that was slightly more computational and it was collaborating with the quantum chemistry professor in the same building and so I started working with that person and I just – I guess I’d say, I just fell in love with quantum chemistry at that point and I couldn’t get enough of it.
It was, I just, I enjoyed kind of thinking about something a little bit more complex and a little bit more abstract and how quantum mechanics can be applied to the world around us and how we can use the concepts of physics and theoretical physicalized to real life molecules and reactions and drive insights into how they’re actually working. So I worked on a couple of different projects during my PhD. As I said, they were all about transition metal complexes.
One was on an enzyme that’s used in the bacteria that causes tuberculosis. So I was really looking at sort of the finer detail of that enzyme and looking at certain mutations that had happened across the active site and using quantum mechanics to really learn what was happening there, what was the mutation actually doing, what was the outcome of that and I guess, yeah, I just really enjoyed quantum chemistry from that and then lucky for me, there was a lot of transferrable skills that came for working in that field, which led me where I am today.
[0:08:53.8] RS: Whether it’s quantum computing, quantum physics, quantum as a word is just something you can slap on any sci-fi story and make it sufficiently advanced and futuristic. Would you mind, outline just a little bit of what you refer to as the abstraction and the theoretical nature of what we’re talking about when we qualify something like chemistry or computing or physics with the word quantum?
[0:09:16.0] BK: Yeah. So anything that’s referring hack to quantum is most of the time, referring back to quantum mechanics and quantum physics. So the laws that govern small things, the atomic level. So that applied to chemistry is how do you apply the laws of quantum physics to molecules, right? The contrast to that will be something like, classical physics, right?
You are either modeling everything in terms of a ball and a spring or you’re modeling it in terms of the way that the electrons interact with each other. So it’s a different scale. So we’re talking about quantum chemistry versus something like quantum computing. Quantum computing is then applying the same sorts off principles, the theory of quantum physics, the theory of quantum mechanics to computing.
[0:10:04.9] RS: I see, thank you for giving that context for me the next time I want to watch an Avengers movie, I’ll have a little more understanding. What were the things about your PhD that you said, we’re transferrable that made you well-prepared for a career in data science, machine learning and ultimately, AI?
[0:10:22.3] BK: I think the number one thing was, being able to explain something somewhat complex and abstract to your stakeholders. During my PhD, I was working with, so as I was applying my quantum chemistry calculations to this enzyme that’s applied to the bacteria, I also had collaborators that were doing the experimental part and I would take data from the experiment and use it in my calculations.
Often times, I had to explain what I was doing and how it worked and why, to the experimentalist and it didn’t always kind of land the way that I wanted it to. So I had to really learn how to communicate the findings to different people coming from different backgrounds. I think it’s the same thing when you go into machine learning, not everyone may understand how a neural network works.
But the same time, you need people to, in a way, you need to build trust with people, so that the application that you’re building actually makes an impact to where you’re trying to make an impact, right? So if me and my team build machine learning pipeline that helped predict the design of a small molecule or predict the binding energy of a molecule to a target protein, that really only makes a positive impact on pharma R&D if someone goes and uses it.
So it’s really important that I think that you're able to communicate what you’ve done and why and why it can be trusted or maybe even more importantly, what needs to be done to ensure that we can trust it.
[0:12:02.8] RS: That is an ever-green professional skill I think, the ability to tell story of your work and loop in, take holders and explain it to them in a way they understand because if you are for example, a podcaster, it’s very clear to you why your work is valuable.
But to other people in the organization, it may not be and so it’s easy to take the import of your work for granted or the understanding of the import of your work for granted and particularly, when you’re dealing with really advanced technical things, you just cannot expect the people who control your budget or the people who determine the direction of your product to understand an infant level, what it is you do.
So being able to loop in stake holder is as crucial. Can we talk about AI for Science? I want to talk about this new initiative. I guess, what is the log line? Let’s start at the very beginning because there’s this exciting new research initiative over at Microsoft, you’re involved in the early stages. What is going on over there at AI for Science?
[0:13:00.1] BK: Yeah. So really what it comes down to is that we believe that deep learning could have really transformational impact on the natural sciences. So that being biology, physics and chemistry and the impact could really dramatically improve our ability to model and predict the natural world and that would go well beyond their traditional use of machine learning for statistical model of observational data and we think this could represent a new paradigm of scientific discovery.
[0:13:31.1] RS: Could you give an example of some of the research that you think is pursuing that end?
[0:13:35.1] BK: Yeah, so I think it really comes down to, I guess a new sort of frontier of scientific discovery that combines AI with quantum chemistry, physics, molecular biology and I guess to understand that, you have to kind of start with, what scientific discovery is today, right? So historically, science has evolved through what’s been called for paradigm.
So the first is purely empirical and it is based directly on the observation of nature. So for example, I eat this plant, I feel better and apple falls on my head and you know, I come with the concept of gravity. The second paradigm is theoretical, so you can start thinking about now we have equations that come into play, so laws of motions, Maxwell’s equations, things like that and these are typically differential equations and they describe the world around you.
The third is when digital computes come into play in the 20th century. So now, we can start to solve these equations for our more complex systems like forecasting the weather and other applications like that and then the fourth paradigm is about data intensive scientific discovery. So being able to collect, to store, to process large volumes of data and that’s where machine learning comes into a play a very important role there.
So modeling and analyzing large volumes of experimental data and those four paradigms so far are complimentary and coexist together. However, those fundamental equations for sciences are well-known but their computation is extremely demanding and most of the time you have to make a proclamation to them. So for example in quantum chemistry, we use the Schrödinger equation.
It is a partial differential equation that describes the behavior of molecules at the subatomic level with really high precision but the solution with hierocracy is only possible for really, really small systems with just a few atoms. So each two hydrogen atoms bonded together for example and then scaling to a larger system requires you to approximate, which leads to this tradeoff between scale and accuracy.
But over the last few years, we are seeing a new way to exploit machine learning and particularly deep learning as a tool to address that speed versus accuracy tradeoff for scientific discovery. So it’s a different use of machine learning from collecting, storing and then modeling observational data that we’re familiar with because the idea is that the data that’s used to train a neural network would come from the solution to the equation of science rather than from observation.
So then we’re starting to think about, you can think of the traditional solver for a scientific equation or the stimulator to the natural world. So that can be used to do things like forecast the weather or calculate the biding affinity of a candidate, of a drug molecule to target protein but those simulations are computationally very, very expensive. So for example, it can take months on a super computer to really simulate all the dynamics of a protein for just the equivalent of like a microsecond of real time.
They also kind of represent something very fixed that don’t necessarily improve with experience but from a machine learning perspective, the details of that simulation can be thought of as training data that can be then used to train something like a deep learning emulator. So in other words, you can use the inputs and the outputs and the simulation as training data for a new machine learning deep [inaudible 0:17:19.7] which can then learn to replicate or emulate that simulator.
So it’s kind of this input and output thing that we are trying to do and then if you use the emulator many, many times, you are reducing the cost of generate that train of data and the cost of training and hopefully now, you have created this fairly general purpose emulator that can run several orders of magnitude faster than a simulation itself.
[0:17:19.7] RS: So as you feed iterative simulations with the input and output from previous ones, how do you ensure accuracy? I am afraid there’s like a copy of a copy of a copy thing going on there or is that a reduction point to look at it?
[0:17:56.6] BK: So I think because it is not looking at observational data, so training data would be perfectly labelled and the quantity of a data is only going to be limited by the computational budget. So once that’s trained, the emulator can perform a new calculation with much higher efficiency and hopefully improve in speed compared to something like a direct simulation.
[0:18:21.4] RS: Okay, got it. To bring it into the real world a little bit, could you give an example of one of these emulators and then what sort of impact it has?
[0:18:28.4] BK: Yeah, so as I said, the idea is that we’re kind of opening up this new door into actually very deep research and that itself has huge potential to have important real world impact. So for example, the number of small molecule of drug candidates alone is estimated to be around 10 to the power of 60, while the total number of stable materials is around 10 to the power of 180.
So finding more efficient ways to explore those spaces with emulators would really transform our ability to discover new molecules and materials. So for example, better drugs to treat diseases that are sub-straights for capturing carbon dioxide, new materials for batteries, electrodes for fuel cells and many more. So some of the things we’re working on right now, where we have gotten a number of different partnerships.
We’re collaborating for example with another team here at Microsoft research that is working on materials engineering for carbon negative feature. So they are applying the computational tools that we’re building here in AI for science to design new materials. So for example, they are looking at say, carbon removal. So how can we use something like a metallic organic framework optimize using our machine learning algorithms for carbon utilization?
So let’s do something with the atmospheric carbon dioxide that we have captured rather than putting it into the ground. So for example, making a new material for IT systems or anything similar to that like biodegradable plastics or building materials.
[0:20:09.8] RS: Whoa, so it is sort of like it can optimize the user creation of essentially anything that requires natural resources or matter more accurately?
[0:20:19.4] BK: Yes. So the idea is that we’re finding new ways to discover new molecules, right? Through these emulations and then those molecules can go on to have impactful applications and a number of different industries. So that can be in environmental sustainability, it could be in catalysis, it could be in drug discovery. So we’ve got a collaboration with Novartis, where we are working on building a pipeline of generative and predictive models. So that’s really looking at how we speed up drug design and things like that.
[0:20:49.9] RS: It seems like this emulator approach could really be utilized anywhere data science is being done. Is that accurate?
[0:20:57.4] BK: Yeah, I mean I think it’s hard to think of an industry that is not actually using data science and machine learning at this point. It’s really just made an impact across the globe, right? But now we are thinking about, “Okay, how do we apply machine learning to things at a more atomic scale?” so actually looking at molecules to how we apply deep learning, machine learning to actually be able to explore chemical space more effectively and then that being able to build new applications in a number of different industries.
The idea is that it is used for, you know, the things that are going to make the most impact on the world, so solving healthcare challenges, combating disease, combating climate change, sustainability, things like that.
[0:21:43.1] RS: So while the possibilities are really endless for where you might point this approach, what does Microsoft want? Because it doesn’t sound like its new formulas for Excel, right? I guess what I am asking is what is kind of the directive of AI for science, what does the outcome look like because are they sort of giving you carte blanche to like, “Hey, go do science and think of some cool science-y things” or what do you think is the ideal outcome of AI for science?
[0:22:06.1] BK: Yeah, I think that is a really good question. So there is kind of three main goals, so number one, Microsoft research is here to do research, intellectually deep impactful research and so AI for science is about building that portfolio and peer reviewed publications are going to be a key deliverable there. Second, as we are part of Microsoft, Microsoft is a company whose mission is really about empowering others to be successful.
So we’ll also be looking at how we can turn some of these research into cloud based services that can then be used commercially or by academia and the applications as we said is kind of enormous. So like your simulation, it could be medicines, it could be fuel cells, it could be carbon capture, it could be catalysis and many more and then the third goal is really that real world impact.
So being able to find the metric that is tied back to a research. So healthcare, how have our emulators help discover new medicines and sustainability, so how has our research discovered new molecules and materials that can be used for carbon negative future and so on.
[0:23:21.1] RS: So what is your day like? I am curious like how do you spend your time? What is the actual work look like when you crack open your laptop to participate in this toil? What are you kind of busying yourself with?
[0:23:31.9] BK: Oh that’s a good question. Okay, so I think one of the reasons why I’m at Microsoft research is because my background and my experience and I guess part of my strengths are building and growing data and machine learning teams. So our team here are AI for Science is incredibly smart. So we’ve got a lot of really big players in the deep learning world and people that come from a variety of different backgrounds.
Some people coming from directly from academia, people coming from quantum physics, people coming from really strong in the deep learning world, people coming from biology, healthcare but I think what I bring to the table is really how do we make this a team, how do we make this an organization that has really clear goals in what we want to do and how we’re going to want to build it and the culture that we want to build too.
So it’s all brand new, right? So I joined in July and we just have this amazing opportunity to build this new team and to work with this blank canvass in a way and write the book on how we want to run this thing and that’s kind of my favorite place to be, it is a lot of fun and kind of thinking about how do we want to build this team, how do we want to grow it, what is the culture that we want to build, how do we want to set out goals and make everyone feel a part of it.
So that’s kind of the idea of where we are now. So my typical day is really trying to build this thing, working with the people around me at all different levels, all different backgrounds to kind of understand, you know, what is the sort of the top down bottom up approach we want to take to this, so we want to encourage freedom of research but also think about the bigger picture of where we’re going and why it’s important and our mission and how we’re actually going to get there and how we are going to measure success along the way too.
[0:25:31.3] RS: I see. Well Bonnie, you are in an exciting opportunity because most people are either in academia or in the private sector or they are in the private sector and they moonlight in the academic research world and you get to do both at the same time. So it’s just exciting hearing about your role and the awesome work you are doing over there and as we creep up on optimal podcast length here, I would just say thank you so much for being with me today Bonnie, I’d loved learning from you today.
[0:25:58.0] BK: Yeah, thanks so much. It was great to be here.
[END OF INTERVIEW]
[0:26:01.7] RS: How AI Happens is brought to you by Sama. Sama provides accurate data for ambitious AI, specializing in image, video and sensor data annotation and validation for machine learning algorithms and industries, such as transportation, retail, ecommerce, media, medtech, robotics and agriculture. For more information, head to sama.com.
[END]