How is AI disrupting the retail landscape? In this episode, we welcome Nitzan Mekel-Bobrov to explore the transformative role of AI in reshaping online marketplaces. Nitzan is a visionary leader and AI expert driving innovation at the intersection of technology and commerce. As Chief AI Officer at eBay, he spearheads cutting-edge AI initiatives that transform the online marketplace experience. Committed to empowering people and creating economic opportunity, eBay uses advanced technology to deliver a seamless, secure, personalized shopping experience for all users.
We hear about Nitzan’s AI expertise, motivation for joining eBay, and approach to implementing AI into eBay's business model. Gain insights into the impacts of centralizing and federating AI, leveraging generative AI to create personalized content, and why patience is essential to AI development. We also unpack eBay's approach to LLM development, tailoring AI tools for eBay sellers, the pitfalls of generic marketing content, and the future of AI in retail. Join us to discover how AI is revolutionizing e-commerce and disrupting the retail sector with Nitzan Mekel-Bobrov!
Key Points From This Episode:
Quotes:
“It’s tricky to balance the short-term wins with the long-term transformation.” — Nitzan Mekel-Bobrov [0:06:50]
“An experiment is only a failure if you haven’t learned anything yourself and – generated institutional knowledge from it.” — Nitzan Mekel-Bobrov [0:09:36]
“What's nice about [eBay's] business model — is that our incentive is to enable each seller to maintain their own uniqueness.” — Nitzan Mekel-Bobrov [0:27:33]
“The companies that will thrive in this AI transformation are the ones that can figure out how to marry parts of their current culture and what all of their talent brings with what the AI delivers.” — Nitzan Mekel-Bobrov [0:33:58]
Links Mentioned in Today’s Episode:
Nitzan Mekel-Bobrov on LinkedIn
Nitzan Mekel-Bobrov: Our incentive is to enable each seller to maintain their own uniqueness. Our business model won't work if we're genericizing all the sellers.
Rob Stevenson: Welcome to How AI Happens, a podcast where experts explain their work at the cutting edge of artificial intelligence. You'll hear from AI researchers, data scientists and machine learning engineers as they get technical about the most exciting developments in their field and m the challenge challenges they're facing along the way. I'm your host, Rob Stevenson, and we're about to learn How AI Happens. Okay, hello again to all of you wonderful AI ML M data science practitioners out there in podcast land. It's me, Rob, back here again with another classic installment, I'm sure, of How AI Happens. I have a wonderful guest for you today. I know I say that every week, but this time I mean it because our guest today served as the Managing VP of Machine Learning at Capital One, was the Global VP for AI and Enterprise Data Management at Booking.com. he holds a PhD in Computational Genomics and currently he serves as a chief AI officer at eBay. Nitzan Mekel-Bobrov Welcome to the podcast. How are you today?
Nitzan Mekel-Bobrov: I'm great. Thanks so much, Rob. Thanks for having me.
Rob Stevenson: I'm really pleased to have you. And there's a couple more things in your curriculum via I forgot to add, you are also an advisor to the World Wildlife Fun and, I understand, quite a sneakerhead. Is this true?
Nitzan Mekel-Bobrov: Well, I wouldn't call myself a sneakerhead. Real sneakerheads would make fun of me if they heard me say that. But I do like sneakers as collectibles and yes, I avid supporter of World Wildlife Fun because of the need for all of us to do more sustainable practices.
Rob Stevenson: Certainly. And I like you, would not call myself a sneakerh head, but I think other people who aren't sneakerh heads would look at the amount of shoes I have and probably call me one. Which maybe is that where you live to.
Nitzan Mekel-Bobrov: I will say you can judge for yourself. But I do have a number of custom eBay AI. Not just eBay, but eBay Aai Nikes. Ah, Jordan. So I'm pretty geeky about that.
Rob Stevenson: He would send me a photo of those. I would just kill to see those. And maybe I'll even put it in the show links if my listeners behave themselves. but, ebay also, I mean there's just an alignment of interest because ebay for my money is really competing and it is my money, thank you very much. Competing with StockX for like official sneaker recognition and this kind of thing. I've seen ebay compete in lots of new and exciting areas over the last few years and I'm sure you have a hand in that because AI I think has such an awesome use case in a retail. So I'm interested in hearing about that more from you today and before we get too far into those weeks. So I would love it if you would just kind of explain a little bit about kind of your remit and how you view your role and how you've sort of busied yourself in the last few years you've been serving at ebay.
Nitzan Mekel-Bobrov: Yeah, so I've been at ebay for a little over three years now and I joined ebay because our business model and our coverage of sort of cross category coverage is just so insanely broad and deep that it really lends itself to use AI for almost everything, both customer facing but also internally. Obviously with the explosion in generative models. The last couple of years have really been about how do we overhaul our tech stack end to end as well as our AI platform to enable us to build as well as run some of these very large foundation models and then how do we deliver value to customers as quickly as possible.
Rob Stevenson: This dual approach of how do we deploy AI in the product for the benefit of customers and then also how are we outfitting our organization to benefit from using it. That feels like it could probably be two separate jobs, but I'm sure it all bubbles up to you. So how do you balance the need to be just putting AI? Frankly, everywhere I'm sure is the requests you're getting from the folks in the business.
Nitzan Mekel-Bobrov: I haven't solved it yet. I've made progress, but I won't say that I've solved it because it's something that all of my peers at other companies are struggling with as well. But I think the biggest impact we've had in moving this strategy forward, this dual part strategy, is by both centralizing and federating the impact of AI. And the way we're doing that is really having the centralized platform rolled out to every developer, scientist and even in some cases non technical folks as a sandbox to play around to their heart content, so to speak, but really explore ways in which use cases and impact for customers can be developed. But then at the same time, before rolling these out to customers or before scaling it up, having the right guardrails in place, both from a financial perspective, but also from a perspective of responsible AI, safety, security, etc. To really prioritize the absolute most impactful And I don't necessarily always mean commercially impactful, sometimes it's in other ways, but most impactful use cases.
Rob Stevenson: I'm glad you brought up impact because every organization is dealing with the pressure to deploy AI considering where we exist in a given hype cycle. But also ebay as you called out, is uniquely positioned to do so given the breadth of data and the breadth of products that you see. So when you are thinking about impact and where these tools can have the most impact, how do you narrow that down?
Nitzan Mekel-Bobrov: We try to balance between incremental improvements to existing features, experiences, et cetera that just benefit from AI. Or maybe it's from the latest innovations in AI primarily on the back end, but show up to customers often in a similar way, with what we can think of as more transformative or transformational changes in both how we deliver value, but even the business model itself in some cases it's tricky to balance the short term wins with the long term transformation. But the nice thing about working at a larger company, we do have the ability to have more patience to invest in sometimes longer term impact. I have a example that really came up literally this week where in our a key part of our strategic AI pillars this year has been leapfrogging personalization to individualization. So really how to use generative models to create individualized content to each user versus assigning you to an audience or a segment and you and everyone else like you gets the same experience because you can see how the latder is always going to be somewhat limited because in reality Rob is Robb and you're not the same like any other person, right? So if you can be custom, totally individualized, you'd be able to deliver the highest relevance. And so we've been focusing both on site and in our marketing channels, but in our marketing channels specifically we have been working on marketing campaign individualization all year and really we had one, I wouldn't say negative but neutral experiment after another where one a B test after another was not showing ly the reason I bring this up is it was really important for us to have the patience to let the team continue to iterate, to continue to figure out where the sweet spot is before we expect those impacts to land. Had we not waited, had not had that patience for now almost a year, right, because we're in December, we would have thought that oh, there's no value here in using LLMs for individualized content, at least not in marketing channels. But clearly that would have been the.
Rob Stevenson: Wrong conclusion certainly and perhaps a mistake to say that a neutral experiment is an unsuccessful one.
Nitzan Mekel-Bobrov: Absolutely. Even a negative experiment can be a good learning. And in fact, that's what I tell my team always. An experiment is only a failure if you haven't learned anything yourself and haven't generated institutional knowledge from it because it's not enough for that individual to learn. But you sort of want to graduate it to somewhat institutional learning, which is what we're seeing. We're starting to learn on a more for the whole organization on how you leverage LLMs towards this textual individualized content.
Rob Stevenson: Can you share any specifics about what was responsible for that shift?
Nitzan Mekel-Bobrov: Yeah, so I think that the keyith obviously we've gone through the data a lot to try to distill thematic learnings, but what seems to be really key is it's relatively trivial now to use an LLM to generate some piece of content, whether it's in this case copy, that is unique to that person. But what signals from the customer should the LLM anchor on versus sort of randomly whatever it thinks. And also the continued pre training of the LLMs, in this case open source, to understand which elements of specific products really capture the imagination of buyers in that community. Because it is really different. Like so the attributes of a product, let's say like a, collectible sneaker that would really draw and engage a customer, are very different from attributes that you'd want to highlight in the context of a sweater or a jacket or let alone the entire barrage of categories that ebay has. And the information that you'd want to leverage from that user are often very different too. So you can't just feed the LLM, here's all the s customer signals and here's all the product signals and you just figure out what copy to show. It's a lot more nuanced than that.
Rob Stevenson: Yeah, that makes sense that retail has perhaps used the same buying signals across entire categories of product. Maybe they've narrowed it down to like, price. You know, people don't think about a sweater as much as they do a car, but as you say, very, very different. So you must then train the LLM to index on, for example, specific features of a sneaker that it would serve up to me as someone interested in.
Nitzan Mekel-Bobrov: Sneakers rather than train in some pres specified way, we're using the signals that we've seen. I couldn't even articulate to you in human terms what it is. It's more about feeding the behavioral information and to train the LLM to pick up on those signals. It's not necessarily that I could even tell you exactly what those signals are because the scale of this is so large that what the world is moving to in general, right, is not a human looking at data and then trying to interpret signals or segments or audiences, but rather have that information go into a model that is identifying the best pieces of information to use.
Rob Stevenson: How important is it to come to understand those signals and represent them in human terms? If we know that the machine can identify an index on it, how clear do we have to be as humans?
Nitzan Mekel-Bobrov: We don't need to be clear on an individual level. We just have to have the pipelines and setup of the architecture flow in a way that would enable the model to learn. I'll give you another example that maybe would bring this to life, a little more clearly. You may take the exact same product, for example, a shirt with LeBron James photo on it, and for one user, what the reason they might be interested in it is because they like athletic shirts or they're fans of LeBron James or etc. For another user it may be because it's a Nike shirt and they just like Nike brand. And for another user it may be that they're collecting LeBron paraphernalia. They don't even care about shirts really. They're collecting LeBron James paraphernalia in general. So it's so nuanced what would engage that specific user that when you're trying to get down to the individualized level, you want to try to capture that nuance as much as possible. Which is why I'm really excited about some of the work that's still mostly in research phases in academia as well as industry and work that we're doing in connecting knowledge graphs with LLMs versus RAG based on just vector search. Because the complexity of relationships that could be captured is so much greater. And it'll be really exciting to be able to do knowledge retrieval as well as even training based on knowledge graph structures.
Rob Stevenson: Can you share more about why the knowledge graph structure represents a bigger opportunity?
Nitzan Mekel-Bobrov: They're good for different things, but there's just a lot of lossiness in the traditional RAG approach. A traditional hasn't been long around long enough.
Rob Stevenson: Isn't your grandparents RAG approaches?
Nitzan Mekel-Bobrov: I know things are moving so fast that it's something from a year ago can seem like traditionally. But let's just say the vector retrieval approach is first of all with embeddings there's a lot of lossiness already. And then with the chunking of information there's really no preservation of relationships between one item to another and how, a user in the past is navigated from one item to another. You can try to feed that as context into the LLM, but that really explodes quickly and becomes, well, it would just grind the whole system. Like the latency would just be so slow that there's no way of doing that. So the question is, how do you preserve the complex relationships between nodes that a customer has navigated between without making the system overly complex? The way it's been done to date usually is looking at the behavior of a customer as sort of a sequence of products that they've looked at and having maybe you have an LSTM or a sequence model that captures that sequence. Sometimes you can even introduce a time element to it. What's usually done is actually an embedding is generated from that. And sometimes a order isn't even maintained. But the complexity of understanding all of the information you have about each product in order to try to infer why they jumped from product A to B to C is something that really a graph can bring to life in a way that also can be computationally very efficient if you think of the traversal across the graph.
Rob Stevenson: It gets more challenging too, when you consider how much of ebay's products are user generated. And walmart maybe doesn't have this problem where all of their products are manufactured, purchased, uploaded by Walmart employees catalog Bas ye. Yes. Catalog based. Thank you. I'm new to retail, but you have this challenge then of like, the lebron James example is a good one, because what is the segmented thing about this shirt? Someone might not write Nike on it. They might just write LeBron James t shirt. And that is interesting to someone who likes Nike shirts. But there's nothing on there that suggests nothing in text anyway. So then are you having to segment this, Are you having to. To run computer vision on products to try and figure out what is it about these products that someone cares about or might care about?
Nitzan Mekel-Bobrov: The unstructured nature of ebay's data, which is fundamental to our business model, because our motto from the beginning is enable anyone to sell anything. So, I mean, ebay is a fascinating place if you can look at strange things that I think the best one was some a cheese curds someone sold that looked like, I forget which celebrity and sold for, you know, $50,000 or something. There's always these crazy things, but literally you could sell anything. There's not a catalog for that. Right. So it has been a huge hurdle for us for many years. But what large language models have done for us is really turn it on its head where it doesn't need to be a hurdle anymore. It's actually an advantage because think of it, when you're, you have a catalog, this specific product has been listed 10 million times in the exact same way. So you don't really learn that much about what matters to customers. If on the other hand it was listed in 10 million different ways, literally there's so much more opportunity. Now you do need to your point, cluster we use computer vision is we use really now multimodal approaches to generate some clustering of listings that are similar to each other. But nonetheless the amount of variance that's just in the ecosystem is so huge that there's all these learnings. Plus our sellers are teaching us a lot too. Rightuse the sellers, they have a ton of experience. They're not just randomly entering titles and descriptions, et cetera. They're actually highlighting things that matter to their customers. A lot of our sellers also have physical retail places. A lot of them have decades of experience interacting with, with avid enthusiast customers in their space and so's they themselves are extremely knowledgeable about what matters to their community of customers. And we can help them scale up that experience through AI.
Rob Stevenson: They're also coached too. Even in the event where they're not, they're not like an experienced seller. Like I've played around with that a little bit myself. The generative, tool that is helping people write descriptions. That was fant as for someone like me who's not a seller and who like very rarely sells something on ebay, that was fantastic. Now we're just talking how great the product is. But that is a use case. I'm sure it comes from the learnings of other the people who are those experienced sellers. Now I am able to be coached into writing a description that they would write.
Nitzan Mekel-Bobrov: I think the value that ebay and the marketplace in general, a managed marketplace, brings to customers, particular sellers, but buyers too is how do we take the learnings that ebay has because of the scale we have and help you as an individual build your business through the best performance you can on the platform through that sort of collective learnings. Yet at the same time, and this is where I'll say a little plug for what will come with that description generation in the new year. Yet at the same time also allow each experience seller preserve their, I don't say secret sauce, but their unique sort of touch on it. And right now we do that because a seller can and they do edit their descriptions and add information to it. It's not just the learning of the crowd, but what we will do is now build that also into our AI so they can customize much more in terms of the kinds, of description, tone, information, et cetera. That e being plugged in.
Rob Stevenson: That is important because it feels like the opportunity to distinguish oneself as content becomes easier to generate. We're already seeing it like I feel like I can read. I read one right before this where I was reading this blog post and I was pretty sure that this was chat GPT, not, not much else. And guess what? I'll never go back to that blog 100%. And when in a world where it's easy to generate that someone who generates that and sprinkles in a little fairy dust of their own style, that's the one I want to read. And so is it a matter of identifying someone's style and adding it to the generative ability or is it prompting someone, hey, now, add in your paprika?
Nitzan Mekel-Bobrov: Both. But ideally, once you have enough experience with a specific user, you don't need to proactively prompt them because you want, you don't want to, you want to minimize the amount of time you're taking, but instead you can learn directly from them. I mean my vision here is really enabling each customer to have their own agent, basically, right? Their own LLM, we can even say their own version of the LLM that is tailored to them specifically and helps them scale their own taste, knowledge, expertise, et cetera, et cetera in a way that they just couldn't do if they were having to manually do each one. But it doesn't normalize it. It doesn't sort of average everything out because I agree like a lot of even the content rich sites, but even some of the photo heavy ones, which I love, I'm seeing a lot more AI generated content that kind of, it just all looks the same and you're like, it becomes so easy to generate content that it all starts being the same. So now what I'd love to see my vision for ebay and what I'd love to see in general and it move towards injecting that secret creative spark that each person has.
Rob Stevenson: I share that hope and for me it is a hope. But you are in a position where you can help shepherd in that reality. So'm well, that's why I'm saying care.
Nitzan Mekel-Bobrov: About ebay's a vision, not just a hope. Because in a strategy, because we have some of the most Interesting people on our platform. Right. With just such esoteric but in the best way possible view of their little pocket of the world. And again this is sellers and buyers enabling that community with through AI and actually putting these AI tools into their hands versus us just doing stuff that they passively have to absorb is really exciting. So if you think of some of the features around image, not just description jud, but image editing etc. What we are doing there, what really the vision that we have is to put as many of these AI tools into the hands of our sellers so they can use them to create content and experiences for their buyers directly versus us just creating a super normalized kind of generic view of everything on ebay.
Rob Stevenson: Yeah. Yes. This excites me to hear that this is something that is important to you. And when you see the AI generated content and it looks similar, this is not only the purview of AI, this is common in content generation even when it's 100% human generated. Look at YouTube and look at like the YouTube thumbnails. Like'they're all the same. It's the creator's face in the home alone pose with just like wide eyed and wide mouth. There's like the title text looks exactly the same, the coloring is always the same. Somebody figured out that this was what people would click on and now everyone does it that way. And it'll probably keep working for a while but eventually people get bored of it S There's nothing new or unique there. So it's not differentiating. And so the differentiating part is the interesting part. And that part can stay human and can and maybe even should stay human. Right?
Nitzan Mekel-Bobrov: I agree, it should. And what's nice about our business model because ultimately it allows me to do the job in a way that is win, win, win, win for sellers, wins for buyers, win for ebay is that our incentive is to enable each seller to maintain their own uniqueness. Like our business model won't work if we're genericizing all the sellers and genericizing all the experiences. And it's also about not genericizing the experiences across categories. If you're an avid auto parts seller and unlike sneakers, I know nothing about aut parts. I won't even try to give an example. But the experience you want is very different from the avid fashionista experience. Right? And so what's great about AI, we can have the same backbone, the same platform and even capabilities on the back end, but different teams. Our parts and accessories team can use it in very different ways for Those experiences than our fashion team. Right. That does Shop uses computer vision for shop the look and those types of experiences. Well, parts and accessories doesn't care about shop the look. But what, what do they really care about? They care about fitment. They care about if you're buying this exhaust that it will. It's the same as the one that you currently have and therefore would be a perfect fit. Right. It would work for your car. So. But underneath the hood, the capabilities are very much the same, but they can be tailored into different experiences.
Rob Stevenson: Yes. Okay. Yeah, that makes sense. And even the compulsion for what is the shop the look? What is the fitment? There is something that to anchor on. There is like what does this person care about the most in this product? And that as a variable is different, but exists probably for every purchase. I guess that's the whole thesis behind this.
Nitzan Mekel-Bobrov: Yeah. By the way, a little plug to our design team because what I love about how we do AI, it's very cross functionally. We have AI designers and AI engineers and SC scientists, Etter, et cetera. Because we think about it holistically. Right. Sort of end to end. What I love about our design team is, for example, when we started doing using diffusion models for image generation, they slowed us down in terms of speed, time to market, but in a good way. They said we don't want all of these to just look sort of generic, to kind of all just be this wash of the same type of imagery. We needed a to reflect our consumer base. We need it to be relevant to an individual consumer. We want to elevate the polish of the imagery that our site generally has, but maintain the esoteric sort of unique diversity of it.
Rob Stevenson: Yeah, I'm glad you brought up your team because even just in doing this show, I've noticed the various different backgrounds of folks I get to speak with. Physics PhDs, much more common than I expected, but a good mix of academics, researchers in house, corporate types. And particularly considering the various use cases you have and product, the bre of products you have, I suspected that there was a lot of diversity amongst your team. So I was hoping you could share more a little bit about who these folks are on the AI side and their backgrounds.
Nitzan Mekel-Bobrov: Yeah. So I do think I'm pro now. It's a popular title, but I think I might be one of the very first chief AI officers. I take no credit for it. It's really a vision ebay had because I'm the first one that they created the role for with the Vision before They had ACH chief scientist with the vision of really where AI would be going, which is a place that converges across the entire experience from not just the algorithmic science, but also the engineering and the front end and mobile engineering as well as the back end and of course the UX research and design, responsible AI, etc. What we've done at ebay, it's not like I've just built a thiefdom of my own sort of AI army of that has every of person, but rather when I say AI, it's really kind of the ecosystem of people across the company that are working together towards this broader strategy. Similarly, I get to work with our head of infrastructure. EBay operates one of the largest private clouds in North America. So I get to work with our hardware engineering VP and stand up a team that's really focused specifically on the hardware engineering for AI applications. It's not like I went and built an army of hardware AI hardware engineers that it wouldn't scale that way. AI is, it's kind of like the mobile revolution or digital revolution, right? It started as like, oh, you had a digital organization or something. But very quickly it was like, okay, that great. But what we really need is to have digital everywhere. It's the same with AI now what we want is really to have AI throughout and bring everyone in rather than having walls around us.
Rob Stevenson: If you believe that AI is going to disrupt everything, which most people on this show and I assume listening to the show do, then that is the only way to structure AI in an organization. It cannot just be like, oh, this is going to be another software engineering siloed thing that takes requests or not, that boxes for resources or not, and works kind of on their own as a business within the business. It has to be deployed everywhere and it has to be in the hands of everyone, whether they can know what a stable diffusion model is or not.
Nitzan Mekel-Bobrov: That's why I would say for any organization looking to build out or even hire an AI leader, you need to be part diplomat, part engineer, part scientist, part peace broker. There's people skills. Ironically, people skills in the EQ are equally important because ultimately the companies that will thrive in this AI transformations are the ones that can figure out how to marry the parts of their current culture and what all of their talent brings with what the AI delivers and figure out how AI can actually, actually break down silos, not create silos.
Rob Stevenson: It's great advice, Nitsan. And normally at the stage in the episode I try and have some hackney way to slide into home but I think you just did it beautifully. I don't think we're going to find a better way to the conversation than that. It's great advice about how you can actually have true impact throughout an organization. So at the end of an episode full of awesome advice and insight and so I'm really pleased that we got to spend this time together in nan. So at this point I would say thank you so much for for being yourself and for sharing all this with me. I've loved chatting with you today.
Nitzan Mekel-Bobrov: Likewise. Thanks so much, Rob.
Rob Stevenson: How AI Happens is brought to you by Sama. Sama's Agile Data Labeling and model Evaluation solutions help enterprise companies maximize the return on investment for generative AI, LLM and computer vision models across retail, finance, automotive, and many other indust. For more information, head to Sama.com