Hyperscience co-founder and CEO Peter Brodsky explains why standards are fundamentally at odds with innovation, and how making technology that is backwards compatible with reality is Hyperscience's approach.
Hyperscience co-founder and CEO Peter Brodsky explains why standards are fundamentally at odds with innovation, and how making technology that is backwards compatible with reality is Hyperscience's approach.
Key topics:
0:00:00.0 Peter Brodsky: I think whoever really solves data entry automation will ultimately own the automation space as a whole.
[music]
0:00:10.2 Rob Stevenson: Welcome to How AI Happens, a podcast where experts explain their work at the cutting edge of artificial intelligence. You'll hear from AI researchers, data scientists and machine learning engineers as they get technical about the most exciting developments in their field and the challenges they're facing along the way. I'm your host, Rob Stevenson, and we are about to learn how AI happens. This week on how AI happens, we are looking at the push and pull between standards and innovation. How do you layer advanced tools that themselves grow and change on top of the stagnant good enough fundamentals of yesteryear? And when you find yourself using one of these standards like for example, human language as your foundation, how do you know if the standard is good enough to inform the technology? For Hyperscience, a company that enables the automation of data-centric processes, this question is at the center of their approach. I sat down with their CEO and co-founder, Peter Brodsky, to learn more.
0:01:17.6 PB: I became fascinated with artificial intelligence when I was a kid, was hoping to learn something about how to build what in my mind at the time was real AI. I've now come to appreciate that that's really the wrong framing of it, of course, but I went to school with the hope of learning something about it, and then made the decision to stay for grad school with really the same intent. I dropped out of grad school, started my first machine learning company, and that really led me to where I am now, there's one big continuation of my hope to do something with machine learning.
0:01:47.3 RS: How did you come to Hyperscience? What was sort of the problem you were looking to solve?
0:01:51.0 PB: Well, with Hyperscience, my co-founders and I had been at that point machine learning practitioners for almost a decade, and what we had experienced was that a very large part of our time was spent manipulating data, translating it from one format into another, but not really doing anything else to it, just translating it from one format into another so that it could work with different pieces of software. And being machine learning engineers, we wanted to automate that, it turns out that A, we didn't know how to automate it, and B, it wasn't the problem that we thought it was. It was really about this strange and perhaps arcane artifact of information science or information theory, which states that any single piece of information can be represented in a literally infinite number of ways. Where you can take the word "hello" and represent it with a sequence of letters or take a picture of the word "hello", or take a recording of someone saying "hello", or use the number one to mean "hello". And number two would be "goodbye". Number three would be some other word. Just an infinite, absolutely infinite number of ways it can represent any single piece of information.
0:02:57.8 PB: And what this means is that any two pieces of software have a one divided by infinity chance of being compatible with each other, of representing information in the same way, and that's really the crux of the problem, and the way that large organizations solve it is through data entry, they publish documents in the one universal language that seems to work for everybody, which is English or French, or Chinese or whatever the case may be. And you use that, you use natural language, something that could be read by a human, and oftentimes that human is connecting the output of one piece of software, and basically just translating it verbatim such that it could be compatible with the input requirements of another piece of software. And when we saw that, we thought, "Well, we have a little bit of insight into this. We should maybe go after this problem." Standards in general are sort of the antithesis of innovation. And so if you did suddenly get everyone to agree on a standard that would prevent any further innovation, and you see that, you see that with something as simple as like electrical outlets, there are different standards in different countries. They're all pretty old. None of them are particularly great.
0:04:00.0 PB: And all we're talking about is two holes in the wall, and yet we're never gonna see a better set of holes in our walls or anywhere else, because the standard would break everything if you were to upgrade it, and so standards create this very sticky thing that it's hard to get rid of, once you put it in place, is the opposite of innovation.
0:04:18.2 RS: This challenge of standard versus innovation is one that's coming up for many of my guests. How do you, as Peter says, build advanced technology that is backwards compatible with reality?
0:04:30.0 PB: Well, we really felt that it was important to look at how things are done today, and the way that things are done today is by defaulting to human-readable documents. I'll give you a very tangible example of this, if a lender wants to give you a mortgage, they will look at things like a bank statement, and that bank statement is produced not surprisingly by your bank, which is a totally different organization than the organization lending you the money. And that bank statement is human-readable, it's produced by a piece of software, it's fully digital, never touches paper, it's on PDF. It's human-readable, but it's not machine-readable, and so what happens is a person then receives that bank statement, takes a look at it and then manually types in the relevant information into the lenders piece of software, because you can reliably count on people to process documents they've never seen before. That's the reality today. And I think being backwards compatible with reality is really important, if we wanna get machines to talk to each other directly.
0:05:30.0 PB: You could imagine a scenario in which both the bank and the lender buy Hyperscience and there's some magical machine format, interstitial format that allows the two pieces of software to communicate with each other, but that would require both the bank and the lender to upgrade to Hyperscience at the same time, and that's not how reality works. The reality is that one of them will buy Hyperscience and the other one will continue to operate the old fashioned way, and so we need to be backwards compatible with that. And what that is, again, is language, and so we need a universal machine to machine language that is backwards compatibles with human, and that turns out to just be language, and that's what we've built. We are building software that can take human-readable documents and make them machine-readable, and the benefits of that, I think are rather surprising, we really think of data entry automation as the keys to the automation kingdom, because not only have you now solved this particularly thankless, soulless, mind-numbing, back-breaking, soul-crushing kind of work.
0:06:35.4 PB: You have also done something different, you have now made it possible for machines to work on exactly the same kinds of documents as people do, which means that when the machine gives up and eventually machines do give up, no automation has a 100% success rate. The machines seamlessly hands off the human-readable document to a human where he knows what to do with it because they just do what they would have ordinarily done with it, had there been no automation in the first place. And that backwards compatibility with the present, with reality, 'cause I think what is going to enable broad spectrum automation to take hold. Technically, and not surprisingly, a lot of the success that we enjoy today comes to us as a result of advances in deep learning, we use a tremendous amount of different and at times, novel and a time standard deep learning techniques, and that really is what powers the whole show.
0:07:31.1 RS: Where is the data coming from so that you can process this deep learning?
0:07:35.8 PB: That is the complicated question, and there's a certain amount of trade secret there that I don't wanna get too much into, but I will say that some of the data is synthetic. And I'll tell you a short story about it. At one point, we felt it was really critical that we get good at recognizing handwriting. Well, most of our documents at this point are increasingly digital, there's like a surprising amount of paper oddly in the world, and so we needed to build a model that was really good at doing handwriting, and for that, we always needed handwriting data, and some of it was synthetic, which is not an uncommon approach, we had a lot of fun generating the synthetic data. At one point, we built a model that was really good at forging someone else's handwriting, so you would give it a couple of words of someone's handwriting, and then it would just forge the rest of everything very convincingly. I wanted to use it for something nefarious as some kind of prank. But every prank idea was shot down by the board and so we never did anything too funny with it and just used it to read handwriting.
0:08:41.2 RS: So with synthetic data then you mean as opposed to taking the Declaration of Independence, you were handwriting your own notes and feeding it?
0:08:48.7 PB: Yeah, we would look at small samples of handwriting because that's typically all you get, there isn't like a large bank of handwriting that you could go to, and so you'd feed it small samples and then it would generate lots and lots of more text, you could basically make it write anything you want in that person's hand, and we looked at a lot of different small samples and created larger data sets from those samples with the handwriting synthesizer. And so pretty much all of our data representation is focused on things that are human-readable. A lot of it is language, but obviously there are other modalities that we need to consider as well, but we are very committed to being perpetually human, backwards compatible, and the idea there is that for as long as humans will work and we think humans are going to work indefinitely, you will need to be backwards compatible with humans. And if you try and create this parallel universe where machines operate on data that humans can't read, then I think you wind up with the situation that we're in today where there are parts of organizations that are very efficient from one perspective, but people can't really get involved to them.
0:09:54.1 PB: And so change becomes hard and you need specialized engineers, and the people who are involved in the process on the human side don't really know how to troubleshoot a system that isn't working as advertised, then the result is you can't cancel your gym membership or any number of other things that might have happened to you when interacting with the large organization. And so we wanna make sure that we break that barrier between mechanical and human labor by just making everything human-readable all the way through.
0:10:25.1 RS: I'm finding this reluctance on the part of AI practitioners to pursue full automation to be exceedingly common, and it doesn't seem to be based on the limitations of technology. There are various schools of thought here, and for Peter, it's less about traditional human in the loop, because as he explains, custom human in-the-loop applications present plenty of challenges and he believes the metaphor of the loop to be outdated.
0:10:48.7 PB: And the reality is that there are a lot of things that machines are phenomenally good at doing, but some of the time they fail. What you do when the machines fail? Today, you have to build a custom human loop application, it's the only thing you can do, and in fact, we don't necessarily have a better solution, except that it's not a custom human loop application. It's a human-in-the-loop that has nothing custom around them, because you don't need that custom application to translate out of the machine-readable universe and into the human-readable universe. Our machines, when you build on Hyperscience, when you build your business process on Hyperscience, they hand over the human-readable document without any transformation necessary because the machine can read the human-readable document just as well as the person can. And so you really get this interesting level of collaboration where machines and people work side by side, whereas today, most of the time you have a machine do its job and then a person takes over or a person does the job and then a machine takes over. And the two don't really mix.
0:11:45.1 PB: And when they do, it is a custom application you have to build to enable that mix, which means for every single step of every single business process, of every single company in every single industry, you need a custom human-in-the-loop application if you want collaboration between humans and machines. Now, there aren't enough engineering hours left in the universe to cover the entire world with that many custom human-in-the-loop applications. Never mind that. It's probably a bad idea. They're brittle, they don't change. You have to maintain them. It's a tremendous amount of effort. And so we sidestep that problem by ensuring that the custom human-in-the-loop application is actually just a document, and the document is the data and the document is the user interface, and that is really, I think the big insight that we have here, and why we automate what is seemingly one of the world's most fantastically uninteresting things you've ever heard of, data entry, but it ends up being, we think phenomenally important to the automation story and probably the high ground, I think whoever really solves data entry automation will ultimately own the automation space as a whole.
0:12:53.1 RS: So rather than the traditional approach to human-in-the loop, Peter prescribes and forces a relationship where the human user works in parallel with AI in a manner less like a babysitter and more closely resembling team-based work.
0:13:05.7 PB: What really happens is that you let the machine get better and take over the human's job, and then the human goes on and does the next thing the machine is not good at, and so it suddenly allows us to focus our human energy on only the things that machines aren't good at. Today, the vast majority of things that can and should be automated, both enough front-end and a back office environment aren't, and the reason they aren't automated is because there aren't enough people, there aren't enough machine learning engineers or just engineers in general to build all the little custom human-in-the-loop applications that are necessary to benefit from the partial work that a machine does, but work that would need to be completed by a person. And so you wind up with situations where people do all the work, even though they could only be doing 40% of it or 5% of it, this is often the case with our customers. And that then frees them up to do the next highest order of business, and the result is that organizations... Our customers with the exact same number of people that they had before, engaging Hyperscience can now do way more than they did before, because we're able to automate a lot of the stuff they were doing by hand, and that frees us people up to do other things and they do indeed do other things.
0:14:20.2 PB: And so organizations suddenly accelerate what it is that they can deliver to customers, never mind the usual benefits of automation, it's better, faster, cheaper. I think eventually, people are not going to see it as a human-in-the-loop for the very reason that you point it to, which is going to be seen as collaboration, it's not human-in-the-loop, it's human at work, it's just going to be work and they're going to be machines that are going to be doing the things that are particularly rouge, mundane, repetitive, uninteresting and soul-sucking, and that work is going to be taken off of our plates, and we'll all be able to handle more things as a result, because each of us will have this jet pack of automation, co-worker sitting alongside us, and so I think we're just gonna stop thinking of it that way. The collaboration is going to be pretty seamless. And in fact, our human-in-the-loop platform is called supervision and is our attempt to get out of the human-in-the-loop branding. And the idea is that when the machine raises its hand for help, there's some of there who could jump in and help it, but increasingly even that metaphor is making less and less sense as the person and the machine work in a much more collegial manner than one supervising the other.
0:15:35.1 PB: And some of the consequences of that are also rather surprising, you get a fundamentally different way that people end up collaborating with each other as well, which is their collaboration is now aided by machines as opposed to necessarily always by people. And so you have a much more collaborative organization where many more people can work together on a problem than you would expect possible today.
0:16:00.7 RS: I'm seeing a bit of a poetic parallel here, which is that that approach you're describing, this collegial approach, starts to look a little bit more like the way humans have done work with each other, are clever with each other, in the case of an employee manager or an employee, slightly more skilled, senior employee, I do this task and when I get to something I can't quite figure out, I take it to a podcaster with two more years of experience, and then they help me out, and it's based on language, this is based on existing systems, that we can kind of default to as we process, right?
0:16:36.9 PB: I think that is a really good description of it. So the lines between mechanical and human labor really start blurring, and it really does begin to look much more familiar than I think it might have in its earlier incarnations.
0:16:50.4 RS: Peter mentioned earlier that data entry may represent the most fantastically uninteresting thing you've ever heard of. The implications of streamlining this process; however, are massive, and in the case of some of Hyperscience's customers could be life or death. To put a bow on this episode about the importance of translating human-readable documents, the future of human-in-the-loop, and the challenge of building technology on top of existing standards, I wanted to make sure the opportunity of solving data entry wasn't lost.
0:17:19.7 PB: One of the things that machines are famously good at is speed, they're able to add numbers together far faster than any person can, they're able to process massive amounts of information, again, much faster than any person or team of people can. And there are many situations in which speed, again, quite literally, saves lives. A use case that we're always very excited to see, and we've seen it in a large government entities, we've seen it in the US and abroad, but a common use case that we really, really feel proud to be involved in is around things like disabilities, disability claims in particular where something very substantial has happened to you, you've been hit by a truck, you're still alive, but you're incapacitated, you're the sole provider for your family, you're behind on rent, you don't have money for food. What you need as the medical bills are mounting, what you need is a disability check, and in many cases, the complexity of these cases where there is the attending physician, there is the lab report, there's the police report, there is the nature of the person's job, tremendous complexity, case files that are thousands of pages long, there's a backlog, and that backlog means that people who urgently need a disability check can be stuck waiting for a very, very long time.
0:18:42.5 PB: And in many cases, those people aren't around to hear the good news that their claim is finally been approved, so there's a real level of urgency there, and when you put machines on that kind of thing, you can reduce that backlog materially, and once again, the surprising consequence of making bureaucracy more efficient has a massive impact on people's lives, the quality or life itself. And I think that that is also part of the thing that really inspires us, bureaucracy is this thing that underpins so much of our daily lives, it's right beneath our feet. We don't always see it, we don't always think about it, but it is how the things that we consume are actually made, both goods and services, and so when you can make that layer of the world better, you have a very far reaching kind of impact, and that's what we're focused on at Hyperscience.
0:19:39.4 RS: To end on a high note, I had to hear from Peter, what in our industry has it most excited?
0:19:44.3 PB: I think understanding the relationships between systems, the individual components of the systems continues to be the largest opportunity we have to make our lives a little bit better than probably the larger source of pitfalls we have that if we're not careful, could potentially make things worse as complexity increases. By ensuring that that complexity is human-readable just by doing data entry automation, just by taking the document that a machine can process and populating a system of record, populating a database, wherever that data needs to go, making sure that it gets in there. By making sure that the endpoints of that system are accessible to people, we really give ourselves the best chance, I think, to make sure that we can stay in touch with the complexity. But we are, and you alluded to this earlier, we are part of a very complicated global system of businesses and pieces of software and standards and infrastructure that links the world together to make it what it is, and it is a world in which there is no obvious center. There are lots of different pieces. It is very much a web, the internet itself is a world wide web, making sure that that continues to be something that people can be involved, and I think it's absolutely critical, but it is the key question, I think, how do we understand that complexity? And I think we make it easier, but that's where the work begins.
0:21:24.4 RS: How AI Happens is off next week. If you're enjoying the show, don't forget to subscribe via your favorite podcast listening app, more subscribers means more discoverability, which means more content and more guests. We'll see you in two weeks. How AI Happens is brought to you by Sama. Sama provides accurate data for ambitious AI, specializing in image, video and sensor data annotation and validation for machine learning algorithms and industries such as transportation, retail, E-commerce, media, MedTech, robotics and agriculture. More information, head to sama.com.
[music]