Tulia offers insight into Wayfair’s ML-driven decision-making processes, how they implement AI and ML for preventative problem-solving and predictive maintenance, and how they use data enrichment and customization to help customers navigate the inspirational (and sometimes overwhelming) world of home decor.
Wayfair uses AI and machine learning (ML) technology to interpret what its customers want, connect them with products nearby, and ensure that the products they see online look and feel the same as the ones that ultimately arrive in their homes. With a background in engineering and a passion for all things STEM, Wayfair’s Director of Machine Learning, Tulia Plumettaz, is an innate problem-solver. In this episode, she offers some insight into Wayfair’s ML-driven decision-making processes, how they implement AI and ML for preventative problem-solving and predictive maintenance, and how they use data enrichment and customization to help customers navigate the inspirational (and sometimes overwhelming) world of home decor. We also discuss the culture of experimentation at Wayfair and Tulia’s advice for those looking to build a career in machine learning.
Key Points From This Episode:
Tweetables:
“[Operations research is] a very broad field at the intersection between mathematics, computer science, and economics that [applies these toolkits] to solve real-life applications.” — Tulia Plumettaz [0:03:42]
“All the decision making, from which channel should I bring you in [with] to how do I bring you back if you’re taking your sweet time to make a decision to what we show you when you [visit our site], it’s all [machine learning]-driven.” — Tulia Plumettaz [0:09:58]
“We want to be in a place [where], as early as possible, before problems are even exposed to our customers, we’re able to detect them.” — Tulia Plumettaz [0:18:26]
“We have the challenge of making you buy something that you would traditionally feel, sit [on], and touch virtually, from the comfort of your sofa. How do we do that? [Through the] enrichment of information.” — Tulia Plumettaz [0:29:05]
“We knew that making it easier to navigate this very inspirational space was going to require customization.” — Tulia Plumettaz [0:29:39]
“At its core, it’s an exploit-and-explore process with a lot of hypothesis testing. Testing is at the core of [Wayfair] being able to say: this new version is better than [the previous] version.” — Tulia Plumettaz [0:31:53]
Links Mentioned in Today’s Episode:
Tulia Plummetaz 0:00
Give me unstructured information, give me imagery, and how much can I take out of that? How much information that is going to make Rob feel a lot more confident when he's making the decision. It's also powered by machine learning.
Rob Stevenson 0:15
Welcome to how AI happens, a podcast where experts explain their work at the cutting edge of artificial intelligence. You'll hear from AI researchers, data scientists, and machine learning engineers, as they get technical about the most exciting developments in their field and the challenges they're facing along the way. I'm your host, Rob Stevenson. And we're about to learn how AI happens. Here with me today on how AI happens is director of machine learning over at Wayfair Tulia Plummetaz, Tulia. Welcome to the podcast. How the heck are you today?
Tulia Plummetaz 0:52
Thank you, Rob. I'm doing quite well. Very excited to be today talking to you about machine learning and how we think about it in at wafer with power use cases leveraging this powerful tool?
Rob Stevenson 1:04
Yeah, of course, I'm really pleased to speak with you. Because I know Wayfair decently well as a consumer within the reach with my arm are at least one probably more. If I really thought about Wayfair items, it's really been a great tool for me and fantastically complicated. From a logistics point of view. You're shipping these huge items all over the world, I suppose. So we're gonna get into all of that. Really excited to chat about it with you. Before we get too deep in the weeds. Would you mind sharing a little bit about your background and how you wound up in this role at Wayfair?
Tulia Plummetaz 1:34
Fantastic. So I'm an electrical engineer by training. So in the very, very traditional sense, I started my career building transmission lines back in my home country, Columbia filled engineer, so really, boots and engineering had in the ground. And I really enjoyed the tools. But I realized that my interest was a lot broader, like how do I bring? I am my big passion has always been the public sector, like how do I bring this mathematical tools into better decision making policy things that affect millions of people, and that you want to do them in the most possible efficient way. So that brought me to New York. So take a plain Columbia Land at Columbia University, and then pursue a PhD in operations research. And then in a way that I see operations research offer just such a solid foundation for what today we think about that broader machine learning science field. So from the theoretical toolkit today, you think about problems in literally any space and try to apply the toolkit to solve those problems, efficiency, etc. So, from there, the path is pretty straightforward. So at that point, I really I am building like I am in a POC that is offering the toolkit for that today called machine learning field, worker Research Lab fracture, my PhD jumped to the other side startup, I think those two things give me the kind of like, the tools to really jump away for that I would call it incentive match that happened, you know, right at that moment in time, that wafer was kind of like really scaling really thinking about how do we take this brand to the next level. And then from there, it's been just, I would call it a perfect match, like I have seen the team grow from, you know, I was one of the first hires on the science organization to have a class organization of scientists really power and experiences with a mouth in across our marketplace.
Rob Stevenson 3:28
Gotcha. So forgive my naivete here. Tulia and forgive my naivete could be the subtitle of this podcast. But what is operations? Research how that study takes shape for you?
Tulia Plummetaz 3:38
I have a lot of funny anecdotes with that. Because you many times have folks saying I want to study operations research and like, can you tell me what that means? So the way that I think about it, it's I mean, it's not gonna go the long winded story on but just think about it as a very broad field in the intersection between mathematics computer science, economics that is trying to solve a top like that, that toolkit to solve real life applications from how to get from A to B in the fastest possible way from how do we think about allocating with if we think about people needing kidness and donors in being willing to do it, how do you do those allocations in a way that is efficient that serve a general welfare objective school matching, you need to send kids to actually eight schools? How do you do that? Just think about it, you have any problem with you have an objective, and you are subject to a lot of constraints, and you're trying to figure out how to do that. That is to that's a way of describing the broader set of problems that they filled up operational research is trying to tackle so pretty broad like it literally think about as a toolkit that you can pull in finance, healthcare, education, business and to look for efficiencies.
Rob Stevenson 4:50
Got it. So sort of like a stem approach to efficiency, logistics, that sort of thing.
Tulia Plummetaz 4:55
That's, that's a way of summarizing it. Yeah.
Rob Stevenson 4:57
What was your thesis about
Tulia Plummetaz 4:58
ha! And I for use as one of my examples, my thesis was about kidney changes. So how can we build algorithms that allow us to make those decisions in a way that strategy proof and strategy proof literally just mean, Rob has no incentive to make information that is not accurate, like try to trick the system, because there's no way to win by trying to trick the system. So that's what we call and you want that property when you're making you know, especially public policy decisions, you want people to have no way to find loopholes and exploit those loopholes. So that's a strategy proof nests of algorithmic solutions become really important in those type of applications. That's building algorithms that have those properties was what I did during my PhD.
Rob Stevenson 5:44
Is there a term for that when there's no incentive to mislead or trying to game a system?
Tulia Plummetaz 5:48
Strategy proven it? Yeah,
Rob Stevenson 5:50
that's fantastic. Yeah. But as a user, for me what I necessarily know that what I know that I ought to just, you know, I feel like that when you're talking to your doctor, tell them the truth, they're trying to keep you like, there's, you have no incentive to lie to your doctor, probably as long as your intentions are your own health, right. But there are people who would disagree with that, right, or who may not tell the truth, their doctor does strategy proof must rely on the user agreeing that it's strategy proof,
Speaker 1 6:16
no strategy proof, let's just basically ensure that if you opt to not be truthful, you are not going to get a better outcome than if you were truthful. So in overall, it's just think about it as when you go to read it. And you see people saying, Hey, these are ways of tricking this particular, you know, there is opportunity to take advantage or to, you know, some policy abuse, that it's the thing that you want to avoid. So it's not something that to openly go saying, you know, this is strategy proof, but rather, well, there is no way that someone can put a playbook on how to track the system, because they're not going to get the outcomes that they're expected from it. So that's the beauty of strategy provenance is you can guaranteed, you don't need to worry about Mega phone in it, you just know that mathematically, there's no way to get a better outcome.
Rob Stevenson 7:06
Okay, that makes sense, it would rely on the user because say, I didn't believe in the strategy purpose of a system and thought I could game it. If it were strategy, I would lose. So it's like, whatever, like you can try it, it's not gonna work.
Tulia Plummetaz 7:19
Got it, you got it, you got it, you will lose or you will be the same that you were. So that's exactly the beauty of it.
Rob Stevenson 7:25
What made you focus on like kidneys in particular.
Tulia Plummetaz 7:29
So I've always been passionate about public sector. So my parents were bought, you know, public sector employees, I saw my mom come in growing through, you know, the run the rights and education sector in my my state. So it's anything in that space, especially when it comes to health, transportation, education, has been something that I am always interested in, that only happened to be a problem that there is that interest. But there is also that, like, what is the problem at hand at that point in time, and it was a pretty hard problem at that point in time, just matching, you know, you have multiple criteria that you're optimizing for. And that happened to be a problem that match a lot of my criteria got super interested in it. And that's basically a story like not a lot of over engineering, there's a lot of like, love factor learning there.
Rob Stevenson 8:13
Okay, got it. When you said a moment ago, Wayfair was a perfect match. As you explain your research and your studies, I can see why that's the case, right? You are applying this mathematic and engineering approach to fantastically complicated problems. Wayfair just as a base of their service, is a fantastically complicated problem, the logistics of sending a couch halfway across the country or around the world, etc. Anyone who's ever bought furniture will understand how difficult and complicated that can be. So I would love to get under the hood, a little bit of what's happening when I for example, ordered this bookcase that I know came from Wayfair. It's just off camera here in my office. How is ML and AI being integrated throughout Wayfair?
Tulia Plummetaz 8:53
Okay, I love the question. So let me get started with sprawling even before you probably know, it's even before you end up deciding to purchase that. So from the moment of Rob, start exploring bookcases, like I'm pretty sure you're not I found exactly this one. It took me one minute to make the decision. Now there is a lot of graphic MySpace, looking at the dementia, looking at the style, and then there's wait
Rob Stevenson 9:21
That's my wife's permission. Yeah.
Tulia Plummetaz 9:23
You know, all these things from the moment in which we are deciding, I'm gonna go after like, I'm going to put marketing on Rob, like I want Rob to come to wafer to the moment and in which you land and we show you things like what should we show Rob? Well, how much do we know about raw? Has he been surfing for a while that I have? I'm starting to get some information on what he could be looking for and what should we be showing him that is going to meet his needs, that is going to be the thing that he's looking for that is going to fit his space. All of that is powered by for by machine learning a lot of proprietary algorithms around it. It we also have, you know, our partners but like all that decision making from which channel should I bring you in? How do I bring you back if you're kind of like taking your sweet time to make a decision with us to really what we show you when you're with us, it's a lot of ml driven. Then you mentioned a bookcase. And I say it gotta have the right dimensions, it can be bigger than the space that you have at home. So that's another side on how do we enable, or suppliers to actually get that bookcase into wafer in the ECS. simpler, but also accurate way that when that bookcase get to your house, it's a material that you're respecting is a dimension that you're respecting. We also bring machine learning to that journey, give me unstructured information, give me imagery, and how much can I take out of that how much information that is going to make Rob feel a lot more confident when he's making the decision. It's also powered by machine learning. And then they pretty nitty gritty of logistics. So from Hey, I have these three different bookcases, Rob live in Denver, which one is the closest to you know where he is, should I put that one a bit more on top. So he get it faster. It's better unit economics for Wayfair. So it's a win win for our customer to get what they're looking for the Get it as fast as possible, and we can keep it cheaper, which means we can actually offer more competitive prices to our customers. So all of this machinery, it's powered by machine learning.
Rob Stevenson 11:34
This is by the way, the process you're outlining, this is like the the platonic ideal of like cookies and tracking and like data sharing for the consumer. It's like, Look, if my data is being used in such a way that it makes me more likely to get the bookcase I want. If it makes it easier, then I'm okay with that. It's like this other stuff about like, oh, well, you're harvesting my information and then selling it to black rock or something like then people start to object. But in you know, there's lots of consternation about the value of the data. I'm creating and sharing my own personal data. However, in this case, I'm okay with it. Because guess what, I got this great bookcase that I love. So if it does help me, it's exchange for a service. Right? And so they created this value for me. So I wanted to call that out. That's sort of the the process that I'd like to hear about, because it does help me as a consumer. You mentioned that there's this problem of unstructured data, unstructured imagery, which is interesting, because that is like a universal problem and industry agnostic problem in machine learning. What do we do with this unstructured data? So could you give examples of what that means when when Wayfarer is experiencing that and then how are you kind of ironing it out
Tulia Plummetaz 12:38
so far away. So when I think about selling batteries, and this is just to pick an example, a commodity is a product that is like pretty easy to describe, if it gets to your home, and it happened to be that you were looking for a double A and by mistake, we ended up keeping you at a battery returning it's really cheap, like it's the small good. Now I have a sofa, I have to even describe the sofa that you're looking for even half an idea, but it's a space that is not grantor now the first thing that comes to mind, it's very inspiration driven, it's very much the thing that fit a bunch of other things in your space. So they trying to get that to you with the you're touching that sofa, will you be able to sit on it feeling it, it's one of the big challenges that we have. And at the same time having that super large assortment, we want to make sure that for our suppliers when they are trying to get their product, we're not adding that complexity to that like imagine, you know telling them like just tell me everything about the sofar sociala RA or that bookcase Sasha Rob feel that he knows everything about it patterns, texture, rounded ages, the material, the type of how the wood plates are actually put together. And that's where the machines come in the middle trying to help offering you that nuance rich information that will make you feel it without physically feeling the good without adding complexity and actually inconsistency on how a supplier will be describing those products. So that's an example in which to say it's a universal problem, you're selling something you need to be able to characterize it. But it's particularly nuanced for us because of that bulky item that if I get it to your home and you don't like it, I'm in a pickle trying to get that out of your home, the costs, etc, etc. So it's particularly important that like, it's really hard to describe, it's really hard for you to feel confident about it and I'm trying to surpass all those barriers and like guarantee that with that get at your home. It is what you're looking for.
Rob Stevenson 14:46
Gotcha. So in that example from the original like producer or the seller Wayfarer has all this information about for example a couch right? Most of which I don't need to know to make a decision about buying a couch right so is Is that part of the the unstructured data problem is like, let's take this corpus of data from about furniture and you know, home goods, and sift out the things that are that someone like Rob needs to make a decision.
Tulia Plummetaz 15:14
That's just one aspect of it. So you're right, I have a notion of information, some things are just really relevant for dropped to make the decision. But there's also that ocean of information lacks some nuance on what is needed for right. So imagine, I'm gonna give you a very simple example, you find out about this color called thoroughly. And beloved, it's some sort of blue, it's a trend the blue, so you are looking for a civilian self, rather than asking my supplier to know the nitty gritty about that blue that they're producing also been called thoroughly. And we have the machines power in that. So when Rob is looking for thoroughly and unable to show them, or rich thoroughly and offering of sofas, rather than relying on did that supplier knew that this is called thoroughly and so they actually put it there. And then that's how I ended up showing it to rob. So that's actually a pretty neat example, on the enrichment of information sink, also patterns. Let me show you all the Chevron's that we have. Do I need to rely on the supplier knowing that this is Chevron and like, you know, making sure that word, it's exactly, you know, stated in information? No, I can use the image to actually extract those patterns and offer it, Rob, Hey, you want to Chevron sofa, you got to care, we have super rich assortment of those.
Rob Stevenson 16:36
So the problem is not merely let's cut down the information available so that Rob is not overwhelmed when he's buying a couch. The problem is also like maybe we didn't get enough. And we don't like the one thing that Rob needs to know whether it's really in or not is not in there. The problem is, there's too much data, it can also be there's not enough data with the enriching. So your job is to kind of get to the Goldilocks zone of the information about some of this materials.
Tulia Plummetaz 17:00
Exactly. And if any of the information is there, at the top, you could tell me this is one meter long table, and then just three lines below in your instructor information. It's a 1.1. And then that difference of 10 centimeters, it's actually a big deal. If you're living in a shared apartment, like that's the thing that made the table fit or not. So even just coming back and saying, hey, something seems off, I have two pieces of information that are contradicting each other, help me to sort it out. So like there is there is a validation component, there is an enrichment component. And there's actually an extraction component. That's how I will put together that food surface.
Rob Stevenson 17:39
That's a great example. The other description says one meter, but it's actually 1.1. And maybe the real is like the specific dimensions are listed somewhere else. So you have both examples of the data. You may not it might just be wrong. But that's a great example. Because that strikes me is the kind of thing that's like, maybe you only became aware of it once it became a problem for someone that's a lot of software development is like, oh, like it broke for the user. Now we can go back and fix it. We didn't see that when we were in testing first and then it gets into production. Now there's always problems, right? So I'm curious, is that a process of things breaking going back and fixing them? Or are you able to train your technology to look for these kinds of problems before they end up, sending me a table that doesn't fit.
Tulia Plummetaz 18:24
So the ladder, that's the Northstar, that's where we want to go think about it as a journey. So we want to be in a place in which as early as possible, before problems actually been exposed to our customers, were able to detect them. So you are adding the product, I extract all that information, oh, something, it's actually not quite fit in between the dimensions you're giving me and the weight of the table. So imagine that, you know, you make one zero, and instead of something being 100 grand, it appeared to be 10 grand, and that make no sense, given the type of good that you're trying to add, then being able to quickly interact with you, as you know, our supplier or partner and say, hey, something's off here. Let's just check it like, let's just correct it. Because something comes up before it gets to a customer that actually end up having to return an item or to be unhappy because they didn't we didn't meet their expectations. So that preventive nature, it will actually a male play a big role. When things happen, then you end up into corrective measures and identity with a lot of training data. But success means we are able to prevent those.
Rob Stevenson 19:34
Could you speak more about that preventative approach? How are you? How are you teaching the machines to see around those corners?
Tulia Plummetaz 19:40
Great. So there is we need feedback loops into reality. It's gonna go to that example. You have the bad outcomes. So the outcomes in which you know for sure there was a problem. Those become examples. So call it positive example of a problem. And then the majority of the cases is going to be you don't have a problem. Or if you have it you're in I'd actually aware of it. So that data, it's a fundamental piece and even starting to think about, we can build something that is able to detect, like, you need to be able to say good versus bad in, you know, in simple terms. So once we are and so there was a lot of efforts in how, you know, we jumpstart ourselves with training data, but also, you know, how do we actually enrich it, evolve it over time, etc, etc. And then you start integrating that into your process. So just going back to our example of dimensions, you build the first version, I call it, the crawler is like a baby that is, can't walk yet it can run, definitely, but it start to crawl. And, and that you say, Well, I'm gonna expose it to a small number of my suppliers, or I'm gonna expose it to a small number of classes, the ones in which I am good at detecting when dimensions are wrong. So I'm gonna give you an example, for things like table thinks that like, the shape of the code is like pretty boxy, like three dimensional, and standard, it's a well behaved class to be able to say, a table that has a meter versus 10 centimeters, detecting outliers, there seems relatively easy, you have the density of the material, you have to you have a lot of information that allow you to start making inference on the beginning, you are gonna start deciding, do I go really safe, and only flat things that I feel are like, absolutely wrong? If you do that, you feel really good. Because you're like, Oh, I'm doing really well, I'm very accurate, but then you leave a lot on the table. So then you start, you know, going deeper in that spectrum and saying, I am gonna flag things that are like, potentially, right, but I'm going to collect training data. So my supplier is going to tell me, there's nothing wrong with my dimensions, they're fine. And then I think that, you know what, this is a positive example, like I, you know, there was nothing wrong with like, this is a negative example, there was nothing wrong with this particular table, you start enriching the training data. So you start moving from that crawling to that kind of like styling to walk and become a multi quarter, multi version, you know, evolution of a lot of these capabilities. So the idea of like, 02, it's working, it's working really well. Isn't that how it happens?
Rob Stevenson 22:11
Right, right. Right. I'd love for you to give an example of some of the things that are a little more nuanced when it comes to outlier detection. You said that, oh, if we detect the things that are more cut and dried, like weight and dimensions, that's great, but you leave things on the table, what are the things you might leave on the table?
Tulia Plummetaz 22:27
Yeah, so there is always edge cases. So think about one of a kind items. So items that are like really nonconventional, like an area rug, that it's like extremely skinny and long. And it's used for a particular purpose, particular corridors.
Rob Stevenson 22:44
Here's a lamp that shaped like a unicorn.
Tulia Plummetaz 22:47
Oh, don't get me going on x. axis is the perfect example in which access is actually one of those columns, group of products that like detecting outliers, x is really hard period. Like it's such a narrow genius space. And I think that's where the more homogeneous is a particular space called tables, etc, the easier we're able to say, you don't look like the math. Yeah, they're more heterogeneous. It's a particular accents like figurines, in bases, all these things that you're like, the whole point is that this feels like one of a kind pieces. That's where it's less suitable to actually bring machine learning to the picture. Like, we rely on patterns, patterns that are repeatable, detectable. So the homogeneity of things, it helps you to say, if you are out of the norm, those are that gives me an I need a lot of data, say, what looks in the norm, and what looks outside the norm to be able to detect those outliers. So one of a kind items are some of those, some are actually just funny situations, like a sofa that is designed for kids. So happened that it enter or system as a sofa is indeed a sofa. But the dimensions are, exactly. It's tiny. It's tiny. And if you can look at the dimensions and you bought it, or if we got it wrong, for some reason, it shouldn't have been solved. It should have been kicked off as those are actually typical examples that are we flag and we flagged by mistake, because indeed, those are the right dimensions, or we don't flag and it ends up being things that should have been flagged. It's not in the right, yeah,
Rob Stevenson 24:27
that makes sense. I have a similar experience with that a friend of mine bought a tennis set, and it was like a tennis racket and like a cute tennis skirt, a tennis top and headband. And she's like, can you believe this deal on this tennis that is only $70 For all those things. And so she buys it, it shows up and it was an outfit for an American Girl doll. So she could just I could maybe wear the skirt as a hat, but the rest of it's not gonna work.
Tulia Plummetaz 24:50
Yeah, those are like the anecdotal ones. I think. I think you're the X an example. It's probably the one that I would say, at scale, it become really hard. So then that's where you start making trade off, is this problem suitable for animal application? And is the value that we will get justifying the investment or making it work. So let's make it work for tables and sofas and you know, those those bulky items in which returning becomes really hard. But you know, if x lens is not only hard to make it work, but actually the risk or the exposure of getting it wrong, it's lower than you start making those trade off on, where did you do your investments?
Rob Stevenson 25:32
Right, right. Yeah, by its very definition and accent would be an outlying piece of data, right? That's kind of the whole point of it is like, it's a one off thing. So that makes sense. And your internal term for like tables, for example, was is that I think I caught that well behaved class.
Tulia Plummetaz 25:47
This is me using that to create some intuition. I think about the more as like more homogeneous classes versus more heterogeneous classes, like classes, just think about it as a group of product, I want to refer to a thing that people used to see it and roughly is in this category does a sofa. If it's only to fit then it's a subgroup of that mega class and that we call it a lawsuit. So it's more about how do we create common understanding or taxonomy or referring to this very broad space? That is the Home Goods category. So the well behaved was me to saying, Hey, we all know how table for dining set look like and like, you can get created. But there's a lot of boundaries, it's pretty well ring fence. Meanwhile, with something like accents, you have a lot more of the variability a third to Neary in how it can come in which
Rob Stevenson 26:36
Sure, I was tickled by that the well behaved class because like you as opposed to what like a naughty class like the accent, but it's like well behaved you know, who's to say it's well behaved for truly as needs anyway,
Tulia Plummetaz 26:48
I, here's a scientist in me, it's similar to you know, decide these are actually in mathematics, there was a lot of properties have or certain problems have certain properties, and there is the, you know, behavour, warping around describing, you know, when something a problem fit really nicely, certain type of properties versus not. So that was just the nerd me coming out.
Rob Stevenson 27:08
Yeah, I like that. To me, I'm realizing a moment ago, when you were sharing about what happens when I go looking for a bookcase. One example was, oh, we know Rob lives in Denver, let's rank bookcases that are closer to Denver, he'll have a better experience, it'd be more likely to buy a book, it's closer to him. That's one example. But that seems like a pretty sophisticated search ranking, a lot more sophisticated than what I'm seeing in other kind of search experiences. So clearly, Wayfair has this bespoke search tool. I'm curious to hear more about that. And maybe we can start with when did it become clear that Wayfarer was going to need its own approach to search?
Tulia Plummetaz 27:45
Perfect. So let me start with confirming that we build a lot of customization on our space, and a lot has to do with testing very generic around, if you're looking in general purpose e commerce site, we sell all sorts of categories, they need level of specialization that you need when you are trying to get deeper into a particular category. And they then the bespoken is then once they what matter, for someone who land on our side, they know we are all about home, we are not selling batteries, or selling food, like it's all about home, the ability to offer that bespoke nests, that nuance, that deep understanding of your space becomes fundamental. So that's where you know, customizing our algorithms to actually show that we have a deep understanding of that space. And when you look for thoroughly and going back to the example earlier on that we actually are able to provide, you know, we show that we have expertise on color and colors that are used to describe you know, the home category, or just styles that are popping up trending and that you come and you're able to see that our store man easily, you know, in front of you. That's that's basically the nature of why we need customization. When did we realize that that's that's just pretty much in the in the bread and butter and of how you know, we have all this thought about making a better product, it sounds like, exactly, it's at the core of how we thought about our spaces. Like we have this challenge of making you buy something that you will traditionally feel see touch, virtually, from the comfort of your sofa. How do I do that and richness of information and easiness of finding that not only search, just filtering, exploring, I want, you know, modern, but I also want it in this color. But I also want became at the core of how we conceptualize our product and the ability like making it easier for customers to find. So I would say just add the core of our space, we knew that making easier to navigate is very inspirational space was going to require customization.
Rob Stevenson 29:48
Right, right. Totally. I keep bringing up my bookcase. It's now a recurring character on this on this podcast. I just set up a microphone for it but the reason I keep doing it is because I love it. It showed up on time. It's just a testimonial of Wayfarer working, and specifically the search algorithm working. So I'm curious when you are tweaking this when you're working on the bespoke search tool that you have over there, what are you looking at to kind of measure its success and improve it over time?
Tulia Plummetaz 30:14
I'll start by saying we have a pretty chord and strong culture around experimentation. So a lot of it is around, you know, we test things, making something work or not, it's there, like high cardinality problems. So like, so many things can go into making this experience better than the experience that we had before and the algorithms that we have before. So basically, we go in a culture of saying, Okay, this is what we're seeing out of this particular algorithm. This is what we're seeing. So classes that are working really well, we're seeing, you know, customers are finding things easier. These are some that are not, how can we address a particular tab that we have, let's design a new algorithm, let's contest it, because at the end of the day, we just have a hypothesis on you know, what we have seen through the data, and then you start this is we build little things for generally, one generalization like that's a beautiful property, one thing that worked for a lot of things are fewer things that work for for a lot of things so that it become that trade off of, well, this new algorithm actually improved this particular you know, smaller class that actually came at the true mental of the broader, how do we move forward? How do we start maintaining or general objective of wafer, you know, unit economics profitability, and at the same time balancing all the order constraints that you know, that you're looking to have, which is, when we have happy customers, we have the right unit economics, we offer you the right diversity, at ECC for you to find products. And that's, that's that's a lot of the fun on the journey in which it's very multidimensional what we're trying to do. So testing allows us to actually keep ourselves very rigorous around, how are we trading things off between those different dimensions. So it's really at the core, it's a very exploit unexplored type of process with a lot of hypothesis testing, testing, it's at the core of us being able to say this new version, it's better than this other version in this dimension. And we're taking a trade off on this other dimensions.
Rob Stevenson 32:09
Got to explain and explore. I love that and every every mark gets a little bit better. That's fantastic. Talia, we are cruising well past optimal podcasts, LinkedIn, but it's just because I've had a really lovely time chatting with you. Before I let you go, I would just ask you to to speak to some of the folks out there in podcast land, who are forging their own careers in machine learning. What advice would you give folks as they continue on in this space?
Tulia Plummetaz 32:32
Here's got the bite that I generally say it's going to be pretty generic, but like, I actually think it's very first principle land, it's thinking about the problem you're trying to solve, like, it's really easy to get distracted and attracted by the toolkit. And going back to first principles, like what is the problem? How do you break it in components, machine learning is one of the tools, there's so many other things in the toolkit that you can use to solve these different components. Think about what is success in solving the problem in this particular component, and then the system, you know, comes all together. So that that will be I think the most common misconception that I see out there is that it's all about the techniques, the risk phase for that, you know, if you are doing research in a particular field, like it's absolutely about the particular topic, it could be very specific on a technique or but when it comes to really practitioner going in and you know, applying ML into industry use cases, I think they are not losing sight of what is the problem that you're trying to solve? How does it fit in the business that you are in and the the peculiarities and nuances of that? I think it's been it's been something that like, I rarely see that been kind of nice playbook for delivering the value that you're expecting.
Rob Stevenson 33:46
That's great advice. Tulia. I have loved chatting with you today. This has been a fantastic conversation. Thank you for being here today.
Tulia Plummetaz 33:51
Thank you for inviting me, Rob. This has been a pleasure.
Rob Stevenson 33:55
How AI happens is brought to you by sama. Sama provides accurate data for ambitious AI specializing in image video and sensor data annotation and validation for machine learning algorithms in industries such as transportation, retail, e commerce, media, medtech, robotics and agriculture. For more information, head to sama.com