LeanTaaS CEO Mohan Giridharadas explains how his team is solving the Supply & Demand challenge within healthcare, how the algorithms are stress tested in order to handle extreme edge cases, as well as how drift is defined, detected, and resolved in a customer-centric fashion.
LeanTaas CEO Mohan Giridharadas explains how his team is solving the Supply & Demand challenge within healthcare, how the algorithms are stress tested in order to handle extreme edge cases, as well as how drift is defined, detected, and resolved in a customer-centric fashion.
[music]
0:00:04.5 Rob Stevenson: Welcome to How AI Happens, a podcast where experts explain their work at the cutting edge of artificial intelligence. You'll hear from AI researchers, data scientists and machine learning engineers, as they get technical about the most exciting developments in their field and the challenges they're facing along the way. I'm your host, Rob Stevenson, and we are about to learn how AI happens. Joining me today on How AI Happens from LeanTaaS, is Mohan Giridharadas. Mohan, welcome to the podcast. How are you?
0:00:39.2 Mohan Giridharadas: Good, thank you Rob. It's good to be here.
0:00:41.1 RS: I'm so pleased to have you on. I have a million questions for you. Before we get into the weeds about the technology you're deploying at LeanTaaS and the opportunity and the product and all of that, I would love to know just a little bit about your background and how you came to LeanTaaS, and then why perhaps AI was the approach to this problem you're solving.
0:01:00.3 MG: Terrific. Well my background is, I have a undergraduate in Electrical Engineering from IIT Bombay. I came to the US a long time ago, to go attend graduate school at Georgia Tech, got a Master's in Computer Science, specializing in computer graphics, and then I spent four years leading a graphic software team, doing electronic design automation at Mentor Graphics. Decided to switch careers and go to business school, went to business school at Stanford, and then spent 18 years at McKinsey, doing operational excellence initiatives, Lean manufacturing, Lean service operations, optimization of service deliveries, and then decided to start LeanTaaS because the opportunity to apply Lean thinking with sophisticated mathematics and emerging AI and ML to drive improvements in operational performance seemed like an excellent thing to do, and so I started LeanTaaS, early in 2010.
0:01:53.6 RS: I'm curious to, what was your experience of this chief problem that a lot of individuals, maybe perhaps even yourself, struggled with, that led you to say, "You know what? I can solve this problem, I can build a company to attack it."
0:02:06.4 MG: There were two core problems I went after. First was, having led operational improvement engagements in many companies, many industries, many countries around the world, I felt that they were all done on the backs of Excel spreadsheets. Companies would form opportunities and assessments based on Excel, Excel-level math. And to me, Excel was middle school math. And if you use middle school math, you'll get middle school answers. And so the whole idea was, can we significantly upgrade the quality of the mathematics, by doing optimization, simulation, machine learning, AI, sophisticated predictions, prescriptive recommendation engines, etcetera. So that was one core idea. And the second core idea was, companies that relied on external consultants or even internal consultants were very reliant on a manual, labor-intensive way of driving performance. And I felt that you could drive it over technology, drive it on a software as a service platform, so that the software could come up with smart analytics, intelligent recommendations to drive impact without necessarily needing Feet on the street, day in and day out, month in and month out. And so it was on those two premises that it felt like it was an excellent idea to start a company around it.
0:03:25.3 RS: So what did the solution look like over the last several years? I love how you said that if you use middle school math, you get middle school results. What was the up-leveling of that processing look like?
0:03:37.0 MG: For the first five or six years, we were industry agnostic. So we were in everything possible. We worked with Home Depot, we worked with Flextronics, we worked with Clorox, and we worked with companies across many different industries. In each case, we were solving a complex operational problem. How do you optimize the rental tool inventory at a Home Depot? How do you optimize the supply chain of parts into a particular factory line at Flextronics, those sorts of things. And so in each case, the up-leveling of the map was building a sophisticated prediction model or a sophisticated algorithm, or a sophisticated pricing estimator around it. And so, in every case, the common elements were, a data ingestion method to pull in data from many different systems, a sophisticated set of algorithms, and a visualization layer that displayed the results and allowed the user to interact. So that was the common thread across everything we did. One of the problems we happened to solve was chemotherapy scheduling at Stanford, because it met the same criteria. It was a complex problem, it had a continuous data flow, it had a need to provide an output, and when we solved that, we suddenly said, "Every infusion centre in the country and possibly the world has this problem, it's worth pivoting the entire company to do nothing but healthcare," initially infusion, but then of course, we've added more products since then.
0:05:05.8 RS: It's a little bit like a documentarian. Every really great documentary starts out about one thing, and then about a third of the way through the filmmaker realizes that they're making a different movie, and a much more interesting movie, and it's so interesting that you sort of realized that the application was so useful in this particular area. What technology persisted through this pivot, when you were like, "You know what, let's stop being industry agnostic, let's focus on this one problem," what were you able to keep and continue applying?
0:05:32.6 MG: So the algorithms that did the supply-demand balance for infusion chairs persisted, because that was a very complex optimization algorithm to level load chairs. That certainly persisted. Now, we had built methods on data injection and we had built methods on data outputs, we ported those. So when you think about, once we pivoted to healthcare that core algorithm remained but around it, we needed to build a different kind of infrastructure, we needed to build a HIPAA compliant, a SOC 2 Type 2 compliant infrastructure for healthcare, and we needed to modernize the user interfaces. And so we upgraded all the front-end look and feel of the applications, we upgraded the infrastructure, we persisted in the underlying algorithms, and of course we've refactored and improved them continuously over the years. The algorithm was first written in 2013-2014, so it's now seven or eight years old, it's been through three or four rewrites since then. But once we created that infrastructure and the front-end around it, we were able to add more products. So we've added two more products, iQueue for operating rooms and iQueue for inpatient beds, on top of the original iQueue for... Or alongside the original iQueue for infusion centres.
0:06:45.6 RS: So looking at the particular use case for the infusion centers, was this a matter of matching availability to patients with care or what was the output?
0:06:57.3 MG: The output is an intelligent template that tells you rather than just looking at a calendar and saying, Rob, Wednesday at 10 o'clock. What is the right slot to give you? It may be Wednesday at 9:40, it maybe Wednesday at 10:20, etcetera. At the simplest level, it's an intelligent way to give every patient the right appointment slot. But if you step back a minute and say, what creates this problem in the first place? Matching of supply and demand, at the core is a very, very hard problem. Why is that? The demand is essentially hard to predict, it's very volatile, the number of patients, the type of treatment, will they show up late, how long will their treatment run, what ancillary services do they need to their treatment, all of that goes into the demand vector. The supply vector is complex, it's interdependent, you need the pump, the nurse, the chair and the drug to deliver chemo. If any one of those four is missing, chemo doesn't happen. So it's an interconnected supply, it's very constrained, there're only so many pumps, so many nurses and so many chairs, and they are also hard to predict. You can't predict when a nurse is gonna call in sick, you can't predict when a pump's gonna break.
0:08:04.5 MG: So you've got a hard to predict demand signal and a hard to predict interconnected constrained supply, and you're trying to match it. Now, let's look at other industries that have this characteristic. UPS has this characteristic or FedEx. Neither of them know that later on this afternoon I'm gonna ship something to my daughter in LA, so that demand, they don't even know about it yet, and it's gonna show up on their doorstep and it needs to be delivered in LA tomorrow. So that demand is hard to predict and yet their supply is somewhat constrained, they can't move their trucks and vans and planes in the flash of an eye. So that's one issue. Similarly with airlines, I'm planning a trip to New York, none of the airlines know that yet. Sometime this afternoon, I'll get online and I'll buy a ticket. And now suddenly that demand has happened. They can't reposition or make planes bigger, but what they do is they don't rely on simple math, they've got sophisticated yield management and demand forecasting algorithms that power these things.
0:09:00.6 MG: Healthcare has pretended this problem doesn't exist. Healthcare looks at a calendar and schedules. That's great for tennis courts, it's great for spa treatments, it's great for conference rooms, it does not work for complicated things. Ignoring the problem doesn't make it go away. I can choose to ignore gravity and keep throwing my phone up in the air, it's gonna hit the ground every single time because gravity is there all the time. So this complex math is there all the time, ignoring it doesn't make it go away, which is why healthcare has this difficulty. So that's the core problem we solve. We solve a bit differently for each of our products because at the highest level, yes, supply and demand matching, but the devil's in the details, and each product has got its own characteristics of how exactly you solve the supply-demand imbalance.
0:09:47.8 RS: It strikes me that in the example of the chemotherapy, one of the variables may be, is the drug available to be administered? Yes or no. Is there enough? Is it there? Or Is it not? I like how you gave the example though with FedEx, that they don't know when or if you are going to go to one of their stores and ship a package to your daughter in LA. Was the process for you then identifying some of those unknowns? For example, in the supply and demand challenge within operating rooms or within hospitals and forecasting them? What are those sort of unknowns? Do they exist in your example as well?
0:10:24.2 MG: Sure, sure. You can't predict that someone's gonna come in today to the clinic and get a cancer diagnosis and start their infusion regimen tomorrow. You cannot possibly predict that. So for each thing, the demand vector has to be understood very carefully and you have to model and predict what the demand vector is gonna be. So we can look at historical data, historical data is quite a good indicator, so take it product by product. For infusion, the historical data of infusion volumes is a good indicator. Why? Because infusion is a day of week game, different oncologists practice on different days of the week. A lung oncologist who practices on Mondays and Tuesdays, and does surgeries on Wednesdays and Thursdays and teaches on Fridays has a different pattern of when his or her patients show up for infusion. So when you look at them by day of week, you can start to predict the volume of patients you're gonna get because you've seen what the volume is on Mondays versus Tuesdays versus Wednesdays. You can then start to predict the duration. So once you start predicting the volume and the duration with a fair degree of accuracy, you've got a good read on the demand signal. Now on the supply side, you've got to understand, many, many, many constraints.
0:11:34.4 MG: This is where the algorithm gets complicated. You have to understand the nursing shift, you have to understand the roster, how many nurses there are at a time, what is the safety protocol, how many patients can a nurse support, how much time does it take for a nurse to get a patient started. You have to understand the chair configurations, how long does it take to flip a chair, are the chairs organized in parts. You have to understand the pharmacy, is it close by, is it remote, how does it mix, how long does it take to mix. You have to understand the clinic, how do people flow from the clinic here. So we take these hundreds and hundreds and hundreds of constraints and build Boolean vectors out of it, a bunch of ones and zeros where the constraint triggers or doesn't trigger. And from all of that, we try and find a solution space that offers the flattest profile of chair occupancy. A flat profile of chair occupancy is the smoothest profile, so you don't get the peak effect of it.
0:12:28.0 MG: And so having done that, the answer has to meet a lot of constraints, it has to meet all of those constraints I described, it also has to spread out the appointments, you can't have all your two-hour appointments happen between eight and nine in the morning, and then say, "Oops sorry, we don't have any more two-hour appointments." So you have to spread your appointments throughout the day, you cannot start long appointments late in the afternoon because you will run beyond the closing hours.
0:12:51.9 MG: So there are literally thousands and thousands of these mathematical constraints that come in, and our solution finds optimal template. There are many possible solutions by the way. The solution space is a number with more than a 100 zeros behind it, meaning it's the number of possible templates you could find. So we find a good template and we bake that into the EHR, so your scheduler in a seamless manner is making very intelligent decisions. They're offering Rob the 9:10 slot and not the 10:10 slot because it is smarter to offer you for your two-hour appointment, the 9:10 slot. So that's kind of how the product manifests, in a series of intelligent, invisible templates. To me, the best technology is the one that's invisible, that's just... It's just happening and good stuff is happening automatically and we're able to provide that.
0:13:38.0 RS: When you start weighing all of these constraints, some of them very nuanced, probably different from hospital to hospital, an example being the arrangement of the chairs in a pod. How do you solve for that? How do you access this data when it's probably different all over the place, and the data itself may not exist?
0:14:00.8 MG: Right, so let me be very clear about the difference between custom software and configurable software. So we do not customize our code, it's one body of code running in the cloud that's supporting all 400 infusion centers, all 8500 chairs. It's one body of code, it's one algorithm, but it's very configurable. So we've understood all the parameters that are configurable, the number of nurses, the shifts, the configuration of pods, etcetera, and we can flip switches in the configuration unit without changing a line of code. So our deployment teams in the initial... And by the way, we can set up in six to eight weeks, and we can set it up all remote without even showing up, which is valuable in a COVID world. And so what our deployment people can do is talk to the leaders at the infusion center, understand these parameters and make the right flips of the switches, and so that the algorithm then runs correctly.
0:14:55.7 RS: The process of building a configurable approach here, it seems like it would be a huge relief to the AI practitioner. Perhaps they don't have to solve for everything, they just have to allow individuals to customize the output for themselves.
0:15:11.6 MG: Right. But to know what is configurable and what switches need to be flipped, you have to really, really solve the problem. And so, first you solve the entire problem, then you understand what is common versus what is gonna be configurable, find a way to expose the configurable portions into admin consoles etcetera. And that's part of the beauty of how we designed and build our products.
0:15:38.2 RS: Got it. You mentioned a moment ago how the template has a 100 decimal points after. I mean there's this many possible iterations. How are you detecting and ensuring accuracy?
0:15:49.5 MG: Well, we can tell what the wait times are because if the template works, we should be able to see a 30%, 40%, 50% reduction in the wait times at peak. And so we know the wait time, so we're measuring all the metrics, we know how many patients were being seen before versus how many are being seen now, we know how many patient hours of infusion were delivered, so we can see that the volume metrics are going up while the hours of wait time, and the hours of running over the scheduled close of clinic are going down, which means we're getting more done with less, we're packing our chairs better and serving patients better, so we know that. Over a period of time, the algorithms drift and the reason they drift is the patient mix can change. Remember chemo is only a six or eight-week regimen, so at any point in time, someone's either in their first week of chemo or their last week of chemo, and so the patient mix being served on any given day is under constant flux.
0:16:47.5 MG: So imagine for instance, just theoretically, if suddenly an infusion center started to handle a lot more complex, long duration chemos. Three months from now, their average appointment length might be four hours instead of two hours in which case our templates will no longer be correct. The templates would have drifted off. If they suddenly hire five more oncologists and don't change the number of chairs, the volume might go up because these five more oncologists are showing up every day and recommending 20 patients a day to get chemo, so suddenly you've got a 100 more patients showing up on any given day. So the volume can change, the mix can change, the center may change its practices, the number of nurses may change, so lots of things can change. So what we do is we know we can predict at the start what we think is gonna happen based on our algorithm. Well, at the end of the day, we know what actually happened, and so we are able to tell whether our accuracy was worth it. It's a bit like you forecast what the temperature is gonna be today, and at night, you know what it actually was, you can tell whether you predicted close or not. So we can tell on a bunch of metrics how close we were, and we monitor that constantly.
0:17:54.6 MG: So if over a period of time we feel like we're systematically missing, we're guessing Wednesdays wrong, every Wednesday for the last five Wednesdays in a row, we go, ooh, time to fix the Wednesday template. So we understand when and where to tweak the templates. Now, we are guarded about it because we don't wanna be a nuisance to our customers and bug them every day about tweaking their templates. So we keep track and we see whether the drift is a fatal drift or just a minor inconvenience, and we batch it, in three or four times a year, we will give them their little template refresh and upgrade the templates. That philosophy is true in everything. It's true in how operating room blocks are allocated, it's true in how bed placement decisions are made. So the general model of predict the supply demand balance, figure out the optimization algorithm, intervene actively and continuously to keep the supply demand dynamic in check and then constantly retrain and tweak is where the AI and ML come in as well.
0:18:52.2 RS: Drift would result when there is a change in the assumed constraints, right?
0:18:57.4 MG: That's one way the drift could result, the others could just be the underlying volumes change. The volume story changes either because of number of providers or more people in the catchment area, or more people getting cancer, who knows. The volume could change. If the volume changes your demand prediction algorithms are off. If your demand prediction algorithms are off, the supply demand balancing algorithm is matching to an incorrect demand forecast, and therefore that algorithm won't be right, which means the template won't be right. So if anything deviates from what you thought it would be either on the demand side or the supply side, by definition, the algorithm is not working perfectly. And we build our algorithms to be resilient, they are not fragile, we run them through simulation, and we subject them to enormous shocks on the volume side, on the mix side, etcetera. We run millions and millions of runs of a simulation algorithm on everything, so that they are resilient, and if they're resilient they can handle shocks, which means it's okay to refresh the templates, not every day, but every few months.
0:20:00.7 MG: Think of it like a shock absorber, if you designed your car with such a fragile suspension that the first pothole wiped out the axle, it's not a very useful car, so the car has to have a shock absorber. Now, at some point, the shock may be so big that the shock absorber couldn't take it, and that could happen to a template too if five practices merge and send all their patients to one center for instance, that's too big a shock, but we certainly design our algorithms to be resilient in that manner.
0:20:27.5 RS: Resilience seems to be crucial because of how high the stakes are for an individual receiving chemotherapy, for example. If this were being applied to my local hair salon, for example, and you booked me for a hair salon and it didn't work, I'll be like, "You know, whatever. I'll get a haircut tomorrow." So the higher the stakes, the more important it is to stress test these technologies, right? So when you are stress testing, when you are subjecting the algorithm to extreme situations, is this just a matter of changing variables and seeing how it reacts or are you putting new data into it? How are you stress testing this?
0:21:09.7 MG: Well, we're stress testing by taking various permutations and combinations of volume and mix and so on, so that's how we're stress testing it. What's important is we're doing operational things, not clinical things. So in some sense, we are not subjecting the patient to clinical risk by our decisions, because we are not trying to mess with the regimens or whether they got a certain drug or a certain other drug. So we're not doing any of those things. And so the ultimate safety valve on this is, let's say our template is wrong. What happens is yes, some patients wait a little longer. Yes, the center instead of closing at 7:00 PM, closed at 9:30 PM on a given day. So the front-line of the health system can recover. Now, if that happens all the time, they're obviously not gonna be happy with our algorithm because it's not working correctly, but on any given patient encounter, our algorithm is not in the critical path of making a risky patient decision, because the health system is carrying the burden of treating each patient right from a clinical perspective.
0:22:12.9 RS: I suspect this stress testing will need to occur for any product attempting to solve supply and demand.
0:22:20.7 MG: Absolutely.
0:22:21.5 RS: If for no other reason than it's a better product. Right? In your case the stakes are very high, so it's absolutely compulsory. For something like the salon example or the tennis court example, like you mentioned, it would just be a better experience for the user, and thus would be smart to get it done but in your case it's compulsory.
0:22:38.0 MG: Right. For sure.
0:22:39.1 RS: I'm gonna ask you to prognosticate a little bit here, Mohan.
0:22:42.5 MG: Okay.
0:22:42.6 RS: When you think of the future of AI in the healthcare space and in medicine, I like how you compared the logistics put in place by airlines or FedEx. Is the goal to model the sophistication we see in other businesses and parallel that? Are there more applications? Where do you think is the harmony between pointing advanced technologies like these at really challenging problems within the healthcare system? What is the end goal?
0:23:10.2 MG: Well, the end goal is the highest and best use of every person's skill, so to the extent you can equip every person to make the best possible decisions they could make without wasting their time pulling numbers and counting noses and beds and so on, which they do today, then you are allowing each person to deliver at their full potential. So what you start with is simple robotic process automation can take care of routine filling out of forms and coding procedures correctly, and making sure the billings work and that the right payer is charged for the right procedure, the right amount, etcetera, etcetera. So all of that you can tick with automation. When you start getting into clinical judgment, it starts to get hard, because healthcare is a very personalized thing. The same patient with the same provider could have two wildly different encounters because their blood results have changed in between. So it's a little difficult to just assume that a black box can come up with all the answers on it.
0:24:07.5 MG: We luckily are focused on the operational part, so we don't have to opine on clinical matters, and so we don't try and offer clinical judgment, but we do have to offer operational judgment. Even there, we don't want to make the final decision. We want to enable the right decision to be made, we want to take it to the 99 yard line, but let the frontline decision maker walk it the last yard. Why? Because there's a lot of nuance we don't understand. I can look at the operations, but I don't know that this patient has a comorbidity with diabetes and therefore something different should be done, and therefore maybe you should schedule a longer slot than that, maybe a special nurse should be assigned, maybe they should be given a special room or a special piece of equipment. Our algorithms will never know that. And so what we wanna do is tee up the best possible recommendation and let the frontline make the final call, because they can bring in the clinical nuance and so on. So that's kind of how I think of it, as opposed to a black box that just flies the plane for you. I don't think that is a satisfying answer, which... And I don't think clinicians would trust it in any event.
0:25:13.1 RS: Mohan, this has been fascinating learning from you today. Thank you so much for being on the show and for your candor and for sharing your expertise with us.
0:25:19.5 MG: Great. Thanks, Rob. It was good being here.
[music]
0:25:27.7 RS: How AI Happens is brought to you by Sama. Sama provides accurate data for ambitious AI, specializing in image, video, and sensor data annotation and validation for machine learning algorithms in industries such as transportation, retail, E-commerce, media, MedTech, robotics and agriculture. For more information, head to sama.com.