We are joined today by Veritone's own Head of Engineering & Product, Chris Doe, to discuss AI applications in enterprise and what is top of mind for companies when it comes to incorporating these emerging technologies. In our conversation, Chris breaks down how Veritone is helping its clients navigate the expansive landscape of AI cognitive engines and why now is the ideal time to test and define proof of concept for generative AI.
Creating AI workflows can be a challenging process. And while purchasing these types of technologies may be straightforward, implementing them across multiple teams is often anything but. That’s where a company like Veritone can offer unparalleled support. With over 400 AI engines on their platform, they’ve created a unique operating system that helps companies orchestrate AI workflows with ease and efficacy. Chris discusses the differences between legacy and generative AI, how LLMs have transformed chatbots, and what you can do to identify potential AI use cases within an organization. AI innovations are taking place at a remarkable pace and companies are feeling the pressure to innovate or be left behind, so tune in to learn more about AI applications in business and how you can revolutionize your workflow!
Key Points From This Episode:
Quotes:
“Anybody who's writing text can leverage generative AI models to make their output better.” — @chris_doe [0:05:32]
“With large language models, they've basically given these chatbots a whole new life.” — @chris_doe [0:12:38]
“I can foresee a scenario where most enterprise applications will have an LLM power chatbot in their UI.” — @chris_doe [0:13:31]
“It's easy to buy technology, it's hard to get it adopted across multiple teams that are all moving in different directions and speeds.” — @chris_doe [0:21:16]
“People can start new companies and innovate very quickly these days. And the same has to be true for large companies. They can't just sit on their existing product set. They always have to be innovating.” — @chris_doe [0:23:05]
“We just have to identify the most problematic part of that workflow and then solve it.” — @chris_doe [0:26:20]
Links Mentioned in Today’s Episode:
Chris Doe 0:00
If you look at your organization, if there's manual work that's being done multiple times, I promise you AI can help solve it.
Rob Stevenson 0:11
Welcome to how AI happens. A podcast where experts explain their work at the cutting edge of artificial intelligence. You'll hear from AI researchers, data scientists, and machine learning engineers, as they get technical about the most exciting developments in their field, and the challenges they're facing along the way. I'm your host, Rob Stevenson. And we're about to learn how AI happens. Here with me today on how AI happens is a man with a ton of experience in our space. He has held SVP roles in product management at companies like xumo and viands. Technology. Currently, he is the head of product and engineering for commercial enterprise over at veritone. Chris doe, welcome to the podcast. How the heck are you today?
Chris Doe 0:57
I'm doing great. Thanks for having me today.
Rob Stevenson 0:59
I am really thrilled to have you because veritone is doing so many amazing things in AI and ML. Sorry, this is like a way too big of a question. But maybe you can kind of give us a SparkNotes. Could you kind of sum up how Verizon is participating in the space and some of the exciting technologies that are going on over there right now.
Chris Doe 1:14
So yeah, veritone has been in the AI space for about eight years, and they've innovated with an operating system that allows AI vendors and companies seeking AI transformation to easily interact, we have over 400 ai engines on our platform. And we allow our clients to come in and basically orchestrate any type of AI workflow that they can imagine. We also supplement this with tech enabled services, as well as a variety of SaaS applications for our clients. And there's three business units within veritone. Our HR solutions are commercial enterprise and our government and legal compliance groups.
Rob Stevenson 1:55
And you are within the commercial enterprise leg of that tripod. Right? How would you characterize your role?
Chris Doe 2:01
Yeah, so I oversee the product and engineering functions within the team. And we basically specialize in four verticals, the media, entertainment, sports, and advertising are our four main verticals. And so this is big, large film studios. This is broadcast television. This is sports clients like the masters and the US Open, where we're on site, ingesting all of their live footage in real time during the events and distributing it to 1000s of media outlets. And we also have a variety of advertising solutions where we can detect real time advertisements on radio airwaves for our radio broadcast clients. And so we participate in those four verticals. And in my role we oversee, we basically build SaaS applications. We build software, and workflows for our clients that become mission critical over time. And all of the software we build is AI infused as I like to call it they have, we're not just building an application or a UI, we're building a solution that has AI infused in it from the beginning to the end.
Rob Stevenson 3:10
Pray tell Chris, what kind of AI are you infusing?
Chris Doe 3:12
Well, the blocking and tackling of a lot of our solutions is transcription, translation, logo detection, object detection, sentiment analysis. So these type of AI cognitive engines have been around for a while, and there's 1000s of them, right. And each one can perform under different circumstances, some are better for real time use cases, some are better for financial terminology, right. And so a lot of what we do is help our clients navigate this landscape of AI, cognitive engines, right. And then in the last year or so, we've also introduced a support for all the generative AI models out there. It started with open AI, obviously, in November of 22. But since then, all the big tech Titans have come up with their own solutions. There's now dozens of large language models out there providing all sorts of generative AI solutions. And we've just added those additional options and additional engines into our lexicon of solutions.
Rob Stevenson 4:14
Generative seems like the kind of thing where I can imagine your clients coming to you and saying, how do we use generative in our solution? It's like this magic wand, you can wave over any process to as you say, infuse AI or make it better. So I guess first, is that happening? Are clients begging for generative to be implemented in some way?
Chris Doe 4:32
Of course, I mean, you can't open your news, your phone, on your tablet or whatever and not see headlines on it. So the good thing is we were uniquely positioned to already get those inquiries, inbound inquiries from our clients because they already look at us as their trusted AI partner. Right? And so they want us to guide them through this generative AI hype cycle that we're going through right now. And in doing that, we kind of break down the opportunities into, I would say two categories, either operational efficiency gains or innovation, right? This year with the economy, there's a lot of operational efficiency aspirations out there. And so just like legacy AI models can help them automate certain functions and certain roles in their company, Jen AI has just made that more scope even bigger, right. So at this point in time, if you can, even if there's somebody in your company that's writing text, right, whether it's a description, a summary, ad copy unit, a product description, anybody who's even writing text can leverage generative AI models to make their output better, right? Along with programmers, right programmers can even use these generative AI models to generate test cases, regression tests to augment their programming function, right. And so every company right now is coming to us with a variety of use cases. And that's coming from our existing clients, and even brand new clients that we would argue that aren't traditional AI or digital, competent companies, companies in agriculture or manufacturing, there's tremendous interest in the art of the possible right now with AI. And so we're getting our existing clients knocking on our door, as well as a lot of newcomers as well.
Rob Stevenson 6:24
So for those cases, where an LLM could really be helpful for folks, surely, it's not as simple as like, Alright, here's your business license of open AI, right. Here's your here's your chat GPT instance, with your logo in the top right, I imagine there's gonna be some kind of domain specialization bespoke versions of LLM is going on. Is that the case? Are you kind of pointing clients toward developing their own niche, basically, like bespoke chat versions of chat GPT, but their own companies?
Speaker 1 6:51
Yeah, it really comes down to their requirements, we can do both, we can build a knowledge graph and put a large language model on top of it, and build a proprietary chat bot off a client's data in a secure on premise environment if they so choose, right. Or we can basically tap into the public large language models out there and fine tune those to be use case specific. It comes down to a dozen or two qualifying questions of what they are comfortable with, right, in terms of data, compliance, regulation, all sorts of things come into place. And these are all kind of similar questions that we've been asking our clients for years, right. It's about security and price and vendor relationships, right? Some of these partners already have long standing relationships with one of the big tech Titans, right. And so they have to use those tools. veritone is cloud agnostic, we support all of the different cloud environments. And so we ultimately have to understand all of the options and then tailor the solution to that client and their needs. I hate to say it, but a lot of these engagements are like a custom cocktail. Right? They all look little different. But when you do enough of them, you start to see similarities, whether it's onboarding qualification, success criteria, and whatnot. And so that's kind of our specialty is that we can see across these industries across the cloud environments, and kind of hand select the best recipe for that engagement.
Rob Stevenson 8:27
That is the very unsexy but crucial piece of implementing this kind of technology, right? Whereas generative is the the tool du jour that everyone wants, getting it up and running is the same as getting any technology up and running for in a lot of cases, right? Like, it's silly to It feels weird to refer to something as legacy AI as if there's like really old, ancient, outdated AI. But don't use suppose that generative has a lot of the same challenges as when it comes to implementing it as does lots of other AI technologies.
Chris Doe 8:57
It's exactly the same challenges, I would say 90% of it's the same, but there's about 10%. That's different. I think the biggest thing going on right now is privacy. Right? And how are the models trained? Am I going to get sued six months from now? And so I think actually, you've seen some of the adoption slowed, because some people are a little bit wary of, hey, let's just wait a year and let it sort itself out before I go all in. That said, there has been rapid adoption by a lot of clients. There's 1000s of point solutions, little Google Chrome solutions, all these types of little extensions out there. Companies have been formed and 60 days, 30 days out there and tons of startups and all the tech Titans and large fortune 100 companies, Salesforce, things like that. They've all basically introduced generative AI solutions in their flagship product across their whole portfolio, right. And so everyone's pushing forward very fast right now. There is still some unknown, right of course, and uncertainty and some lawsuits are getting filed. But I think that stuff will sort itself over time because the value that these technologies provide far outweighs the liabilities that they add.
Rob Stevenson 10:13
So when you hear people being standoffish for compliance reasons, or just what we'll swing back around in a year, when some of this gets stuff gets ironed out, are you like, you're a Luddite, or you're just too risk averse? Or Should these people not be waiting? It sounds like that's kind of your position that the benefits outweigh that risk,
Chris Doe 10:30
I think you need to be testing right now we do a lot of proof of concepts. In our industry, it's all about tangibility, we need to be able to prove a value and prove the solution out. And that starts with a proof of concept, right, we need to get something to the client where they can touch and feel our tech and services with their data, right? And so that it happens in almost all of our engagements they want to try before they buy, right and so And once they get that they can actually touch it, they can kind of internalize it, because it's off their dataset, or their video footage, or whatever. And it makes it a lot more easy to consume and easy to digest and also easy to champion within their organization. So I think this is the time to be testing, and experimenting and defining those use cases and defining the success criteria and finding the goals of what you want to do with more legacy AI and generative AI. And then guess what, once you know how those POCs work, and how much they cost and how much value they bring, then you can budget and scale the ultimate solution accordingly.
Rob Stevenson 11:39
That makes sense. If you're trepidations about this, you shouldn't just do nothing. Right? You can go a little more slowly, you can verify you there's a way to roll this thing out in a more reliable fashion. Surely correct. So do you kind of foresee a future where companies who are using generative optimally, one example would be we have our own internal bespoke version of chat GPT that's trained with our own data and is designed to have output that's useful for our customers and our employees?
Chris Doe 12:06
Yes, so this is probably the area that I see the biggest impact coming in the next couple of years, is around a concept of co pilot or a kind of a renaissance of the chatbots. Right. So chat bots have been around for a long time. But they were hard to train and they were limited in value. Right? They were very hyper focused on questions that were fed into the training model, and potentially some utterances or different variations of those questions. But if you asked anything outside of that, the answer was I do not understand please ask again, right. With as large language models, they've basically give given these chatbots a whole new life because now you can interact with a chatbot and ask any question under the sun, and they will always carry that conversation with you. Right. And it will provide a blanket conversational AI framework for all of your clients, no matter what the context or the topic is. And then the hyper trained domain specific kind of chat bot angle is okay, now you take your company's data, your company's security policies, marketing position, all of the documentation you have for your company. And then you can train that chatbot to be domain specific, right. And I think we've done a couple of these internally, and a couple of them are in POC stages. And when we get to this, from what my perspective, when I see the value it brings, I can foresee a scenario where most enterprise applications will have a LLM powered chat bot and their UI in the coming years. Right, I'm talking about some of the applications you see will get half the screen will be a chat bot, the left side is the legacy application. And the right side is your new Chatbot. And it can ask you how to do things it can you can ask it to search for things, it can ask you to build things, it is coming right. And these large language models have empowered the chatbots to be so good at understanding conversation now that it's enabled a complete, like I said, real renaissance of the Chatbot experience. And again, there's variations of the chat bot, whether you want to use some of the embeddings functionality, or remotely call your data. There's all sorts of ways you can hook in your data and even real time data feeds into these chat bots so that they can take live scores or whether in any type of live data feeds. And so you can create these very interactive self learning chat bots on your prior data stack. And I think it's going to augment the client success support any type of I would say docket Help Desk kind of functionality. It's almost like the old Microsoft Clippy right? How do I create a file? How do I save this? How do I do all that? That's 25 years old. But I think you're gonna start to see a lot more of that. In fact, Salesforce introduced their version of it like two or three weeks ago, right? I think you're gonna start to see that on at scale across all applications and coming years.
Rob Stevenson 15:16
CLIPPY asking you, it looks like you're trying to 3d print this singularity. Would you like some help?
Chris Doe 15:21
Yeah, exactly. It's funny, but it's a good idea. Like why just imagine going to a form. And you have to fill out all these filters and all these conditions and all this type of stuff. Or you could just say what you want, right? It's just going to be so much more native and so much more easy for people not to have to learn these complex UIs and just say what they want, right?
Rob Stevenson 15:45
Yes, I'm already using voice as much as possible. I hate typing on a phone, I always mess it up. So I'm using the like voice to text I'm using voice to search on like my Apple TV. I'm already trying to use as many places as possible. And it will be the same like this is the difference between speaking Google ease, write a search query in Google is different than how I would speak to a friend, right? I wouldn't ask a friend restaurants near me, they'd be like you're having a stroke. But I could say that to Google. However, if I were speaking to a chatbot, or speaking somewhere else, I would could just be like, where the restaurants around me that that are good. I could ask in normal English, our last episode. And I'm Patterson from gradients said that English is the next programming language. And what she means that you can speak to these machines as you would another person and they'll understand you.
Chris Doe 16:29
Yeah. And I would also say, that's also a kind of a signal that prompt engineering is kind of the new next frontier, right? You need to be very specific on how you send these questions into the computer right into these large language models. And the more descriptive you are in terms of expressing the tone you're looking for, and the desired output, and any type of things to exclude, your prompts get pretty verbose really quickly, right? And prompt engineering is the new English is kind of that train of thought, which is you don't have to be a programmer in C++ or Python, or any of the programming languages, you just need to know how to ask a good question.
Rob Stevenson 17:13
Exactly that. Here's kind of a fundamental question. I hate chatbots. And whenever I'm on a website, I hate that It's intrusive, I hate that it pops up, I hate that it makes the sound, I immediately close it. And that's partially because I know that it's about I know, it's not like there's a photo of a person there. But I know it's not a human I'm speaking to, and I know that I'm going to be faster at navigating this, this website, this web app than the chat bot is going to help me is that just me or chatbots? Still, like really, really used I think for people making technology, we probably don't use chatbots. But for people who aren't making technology, are they like really, really popular still, even in their current iteration?
Chris Doe 17:48
Yeah, those are mostly support bots. And that I would consider those legacy right there to basically drive a conversion, right sign up for this website, or whatever they tend to be on landing pages or, or whatnot, what I'm talking about is I'm navigating a very complex enterprise application, like Salesforce, or Oracle, or name, your enterprise application, those things are 1525 years old, in terms of complexity, and options. And, and the amount of stuff you can do on these big applications is, is very large, right. And so the amount of documentation that would go into that is also very expansive, right? The number of webinars and tutorials and walkthroughs, it just never ends, right the amount of documentation you could have for one of these big, large enterprise applications. That's the type of stuff I'm talking about when we go to a large language model powered chat bot, right? Because now you can say, Look, I'm having issues with X, I'm having challenges with y. And it can really guide you through that. And in some scenarios, it will actually allow you to say, Hey, do you want to do this and you click the button, and it will do it for you. Right? So your interface to these large, these very complex enterprise applications is going to be more voice and conversational powered than you having to find the documentation and repeat it yourself within a bunch of clicks. And I think that's going to be the big difference.
Rob Stevenson 19:16
Okay, yeah, that makes sense. So it's a logged in Chatbot. It's not like I go to salesforce.com. And I get their marketing page. And it's like, hey, what kind of business do you have? Let's ask you some questions to see what Salesforce would be right for you. It's more like, I'm logged in. And I need to figure out how to pass these contacts into leads into opportunities. And I'm not to do that workflow. The chatbot inside will do it for me or help me do it.
Chris Doe 19:36
Yeah. And the ones that you're talking about those are, let's say they're there to drive conversions and whatnot. So if you ask them a question about Santa Claus, they would come back and say, I do not know what you're saying. Can you please rephrase? Right? And but if you ask a large language model, you can ask any question about Santa Claus, and actually get a bunch of responses, even before you ask your actual intended question, it's just a lot easier for the user to talk about what they want in natural language. Right, then ultimately stumbling through the UI.
Rob Stevenson 20:10
Santa Claus conversations within Salesforce Man, what a what a time. Right? So it's a weird time to be alive, that's for sure. Totally. So I'm curious, you get to speak to the lots of different clients who are trying to solve various different problems. And I'm kind of curious, what are the commonalities across them in terms of the challenges they face, what is really commonly coming up that folks are finding heart,
Chris Doe 20:30
one of the common problems I see is that there's AI impacts the business so much that it touches almost all of the company's operations. And so there needs to be ultimately multiple sponsors within the company to actually adopt this technology, right. So if we're going to change something, if we're going to automate something, if we're going to add something to accompany multiple teams within the company have to buy in, and they have to be champions, right? We could be touching their marketing department, their operations department, their IT department, and to get all of those teams on board, and figure out all the different funding models and different success criteria, it's sometimes challenging. So a lot of it is just organizational behavior within these companies. And I think that applies to a lot of technology adoption, right? It's easy to buy technology, it's hard to get it adopted across multiple teams that are all moving in different directions and speeds, right. And then with the generative AI stuff, a lot of people are still a little, I would say, scared or nervous about what the future may hold, right? We do a lot of stuff in the entertainment industry, there's all sorts of issues there with concerns around privacy. And so this and that, that are being sorted out, a lot of times a lot of people think of the technology as a threat, artificial intelligence may be a threat to my job, or my team, right. And so there's a little bit of, I would say, bias there towards of being open to these potential news technologies, as well as embracing them as much as they need to be. Right. So I think a little championing this, ultimately, when we work with these companies, we tend to identify a couple of champions that can kind of smooth out those concerns across the teams and handle all the adoption concerns. And that goes a long way
Rob Stevenson 22:26
for those champions to feel like they might be the right people to intuit some of the needs of the business and where this tech could be applied. So I'm curious what advice you would give for folks who are maybe sitting in that kind of position in their org. And they're thinking about ways that they could inject some generative to solve their problems? What advice would you give them so that they can kind of take stock of the organization and figure out where would it make sense to apply it,
Chris Doe 22:50
I would look at all the companies that didn't innovate, right? You've heard the stories of Xerox being the first people to actually come up with a digital camera, but they didn't do anything with it, right? We got to innovate or die. Companies are innovating at a record clip. Now, with all the cloud environments and all the different technologies out there, people can start new companies and innovate very quickly these days. And the same has to be true for large company has to, they can't just sit on their existing product set. They always have to be innovating. It's the innovators dilemma book in a nutshell, right? And so I would say what happens if you don't test new technologies? What happens if you don't try new things? What happens if you don't embrace AI? Guess what your competitors are going to? This is a freight train, it's not slowing down. It's only accelerating. And so I would use a little bit of fear in that tactic. Right? What happens if you don't? And the reality is everyone's kind of, again, some have already launched products, some are testing products and building products right now, you don't want to be the company just sitting and saying, Oh, I'll worry about it in five years.
Rob Stevenson 24:01
What about specifically in terms of like use cases in their organization? How can people identify those kinds of things?
Chris Doe 24:07
It's really simple. I have not talked to a company that doesn't have operational efficiencies to be realized, right? Everybody is always so busy, right? And there's a lot of mundane tasks, I would encourage you to sit down with your team and just ask what are the things that they're doing multiple times, right? If there's a task that you're doing multiple times a day, a week, a month, that's that's a candidate for automation. And if you whether you have an internal dev team or not, it comes down to doing multiple things multiple times with AI in general, when you talk about AI transformation, it's a journey of reducing the amount of people in the loop right. So before AI, everything was human right the human started the process the human finished the process the entire The journey was human LED. And now we've introduced computers and AI. And now there's things that can be automated or sped up or whatnot. But there's still humans somewhat in the loop right there, they are proving this or starting this process or kicking off this button. And so there's a lot of humans still in the loop. Eventually, our goal is to remove as much of the human from the loop and let the machine do most of the work, right, let the AI do what it's best at, which is scaling cognitive functions, right? And then let the human only do the kind of administration part of it, right. And so if you look at your organization, if there's manual work that's been done multiple times, I promise you AI can help solve it. Right? It just comes down to the business model and the cost and the priority for that company. Right. But I would look at that what's what's repetitive? And then the other thing I would say is, what have you been thinking of building that you just haven't been able to start? Right? A lot of the stuff is just you just gotta get started, just got to do that first experiment, you got to build that first MVP. And then from there, it can snowball and go into a lot of different ways. And so I kind of group people into two buckets, operational efficiency or innovation, right? The Innovators are a little bit more experimental, of course, they're just trying things out, seeing what sticks, the operational efficiency gained, those can provide value to a company immediately, right, we just got to identify the most problematic part of that workflow and then solve it.
Rob Stevenson 26:32
That is tremendous advice, Chris, I hope the folks out there are going to take that into their orgs. And look around at the places where, where they can apply this stuff. And such a great point with the innovation idea, too. I feel like any, any like really high performer has ideas like pie in the sky campaigns, they would love to execute, if they had enough time, if they had enough money, they had enough developer, bandwidth, etc. Maybe those are the things that you can, you can get done just with like a company like veritone, or with just injecting AI.
Chris Doe 26:58
I mean, just imagine the ideas we've heard right with Gen AI, and avatars and conversational AI. And we've seen almost every variation of potential user experiences out there, right. And we've entertained a lot of engagements, and some are very futuristic, and some are not that far off. I mean, some of the ideas you've seen in Black Mirror episodes, if you're familiar with that show, but it's fun. It's a great company to be working for, because we sit at that intersection of operational efficiency gains with high profile clients, but we also are asked to innovate for them and with them. And so it's very exciting and very interesting to see what companies are thinking about for their company and their brand for the next couple years.
Rob Stevenson 27:48
Well, Chris, we are creeping up on optimal podcast length here. So this has been a fantastic conversation, man, thank you for walking me through some of this stuff. I've loved chatting with you today and to see I really appreciate you being here. Man. This has been a great episode.
Chris Doe 27:59
Yeah, my pleasure. Rob, thank you for the time again.
Rob Stevenson 28:03
How AI happens is brought to you by sama. Sama provides accurate data for ambitious AI, specializing in image video and sensor data annotation and validation for machine learning algorithms in industries such as transportation, retail, e commerce, media, med tech, robotics, and agriculture. For more information, head to sama.com