Cryptography, AI and the ZKML Prover w/ Daniel Shorr (Modulus Labs)

**Speaker A:** Hello and welcome back to the Strange Water Podcast. Thank you for joining us for another fantastic episode. For those of us interested in the bleeding edge of computer science and advanced mathematical theory, there are two categories of innovation that are aggressively competing for time, attention and talent. Cryptography, which I'll call blockchain, or distributed computing, and machine learning. Now, there's so much to say about these two technologies. For example, what makes them incredibly different? Or what kind of impact can both have in the real world? Or even what kinds of research problems do both technologies have and therefore share? But the big question that we all need to ask ourselves at some point, what is the intersection between crypto and AI? Now look, while I don't really have an answer yet, I could not be more confident that crypto and AI are two sides of the same coin, or maybe two chapters in the same book. There are definitely two paradigms that ultimately form just a part of modern computing. AI is about using technology to create abundance, to lower barriers to entry, and to overwhelm an analog world with more and more machine generated content. Crypto is the polar opposite. Our core principles are built around scarcity, identity and verifiability. I don't know how these two pieces. **Speaker B:** Of the puzzle will fit together. **Speaker A:** So it's time to find people that are much smarter than me who have already seen this feature and have become so obsessed with it that they are compelled to form a company. People like today's guest, Daniel Shore from Modulus Labs. Modulus is a young startup looking to build the tools that we need to complete this vision and to bring together these two powerful technologies with their ZKML Prover. Modulus is bringing the infallible verifiability of cryptography to machine learning, while at the same time bringing the computational power and creativity of AI to blockchain. Today, Daniel and the team at Modulus are working on relatively simple models and they're developing concrete short term use cases for the ZKML Prover. **Speaker B:** But I've got a dedicated portion of. **Speaker A:** My undivided attention fixed on Modulus, because while today we are talking about games and apps, tomorrow we will be talking about AGI identity and the dawn of the age of artificial intelligence. One more thing before we begin. Please do not take financial advice from this or any other podcast. Ethereum, crypto and machine learning will change the world one day, but you can easily lose all of your money between now and then. **Speaker B:** All right, time to start the show. Daniel, thank. Thank you so much for joining me on the Strange Water podcast. **Speaker C:** Yeah, happy to be Here. Thanks for having me, Rex. **Speaker B:** For sure. So, as always, like, how we like to start this podcast is like, really to start with like, who you are, your background and like, how do you find ethereum and then how do you like, fall so far down the. The black hole that you were willing to like, sacrifice your. Your standing in like, your good standing in society for it? **Speaker C:** Yeah, well, I always joke that I look like I'm 12, so I didn't really have much standing to fall from anyways. And I'm so charm, Rex, that you're kind of curious about me. Normally people are just like, what's going on in modulus? But yeah, I mean, I guess my story is actually a pretty straightforward one. It's probably one that if you have like an internal LLM, it can probably do a better job than I can. It's like the most stereotypical stuff. But, you know, love science, loved engineering since I was young, like everyone else nowadays, did quite a bit of math and at some point realized that, you know, we can actually apply mathematics as well as all these concepts that I was really excited about to have like a real world impact right. Right around the same time anyways, as I was reading a lot of sci fi where it's like, oh, this is all the heights that we can get to. So I ended up studying, you know, engineering, computer science, the rest at Stanford for a while, got into AI research along with my kind of dorm mates. And just as we were kind of wrapping up our time as undergrads, getting ready for grad school, Covid happened. So we ended up quarantining, taking a beat before presumably committing the next decade of our lives to academia, if not more. And we were just kind of bored and trying to figure out what people were excited about. And at this point, this is like 2122, the year of 2021 22. You know, folks were really excited about crypto. I guess they still are, but even more so back then, and specifically given that we had no skill set when it came to discerning signal from noise on Twitter. We just ended up like sitting down, me and my buds, reading a ton of papers. And Rex, I'm sure you've probably been in a similar place where you're like reading academic paper after academic paper and very quickly it's like some of these papers are a lot better than others. And very quickly for us anyways, it's like, wow, the ZK stuff, the zero knowledge cryptography specific research is getting so much better, so much faster, and it's clearly like this weird incidental application blockchain has caused ZK research to advance massively. So just as a quick recap, at this point, we were like, yeah, not so sure about the PhD or grad school. We knew AI from the work that we had done as undergrads, and we're just learning all about cryptography. So just purely as a science experiment, ended up trying to prove AI models using ZK proofs. And I'm sure we'll get into what that means. But anyways, that's the origin, that's the starting point of modulus. But you're all caught up on myself anyways. **Speaker B:** No, man, I mean, come on. You're definitely selling yourself short because, look, like it's easy to say, like, oh, you know this, like, technology is, like, super cool and interesting and like, it is like, anyone at this point, like, can see that, but it's one, I think, like, there's not a lot of people who are able to grapple with this stuff, like, at an academic and real research level that also, like, pivot away from, like, continuing down that rabbit trail of, like, academia. And I think, like, we both recognize that what's special in crypto and machine learning as well is that academy. The line between academia and, like, industry is becoming so blurred because the technology is moving so quick. But despite that, like, latest development in the cutting edge of technology, like, what specifically? Like, as you walked up to the preface of like, a PhD, like, why did you do the smart thing and not jump off the cliff? **Speaker C:** Yeah, I would say the verdict still out on if it's a smart thing. But I guess, you know, one thing to note is I would say there's definitely a real, like, personality difference between AI and cryptography anyways. And, you know, we were kind of aware of what the air world was. Was going to look like or had been just. Just the kind of identities at play there. I think it's very hard to talk to a handful of AI researchers and not at least stumble across a couple odd complexities. Just creating intelligence, something about that. And it makes sense, right? Just like AI is very expressive. The underlying technology itself lends, I guess, itself to a certain type of people. But cryptography is radically different. It's actually a very humble discipline that's very rigorous. Everything's super discreet. Like, this is exactly the cryptographic statement I can make and no further. And that was like, really attractive for us as somebody who. Or as a group who really had just been used to the former. And so I think for us, you know, this was a fun opportunity to marry these two tech stacks or academic disciplines that really, from a personality standpoint, again, just totally incompatible right from the get go. Um, but also, I think more earnestly maybe, or maybe more aspirationally actually we wanted to do something that like, really could have an impact that was like very concrete. And it felt like this was actually a very cool way to marry cryptography and AI. So, yeah, I guess maybe more motivated by like, what's cool and what's kind of, you know, like having us like, just be excited about the work that we were doing and less about like form factor. **Speaker D:** Right? **Speaker B:** Yeah, no, for sure. And like, look, I don't mean to like, belittle anyone that like, is. It wants to get a PhD, but I do think that, like, what is magic about our industry is like, because the technology is the product and the technology is just straight up math. And all of it's so new, is that we can push like this whole massive being together from these like, different standpoints, like whether it's going straight into a PhD or academia or building a company or doing research or even eventually once this industry grows up, like doing public policy stuff. Right. But like, why I think it would have been the wrong idea for you to do a PhD is because, like, you're clearly a builder. You don't need me to tell you that. The VCs have already told you that. We'll get to that in a minute. But let's turn our attention to this moment before Modulus Labs. When you're kind of have this background in AI, you're seeing them really incredible stuff that's going on in zk and as you point out, that's both from God. Like, once you actually unpack, like what a KZG commitment is, I mean, it actually literally feels like magic, you know, like you're evaluating numbers that no one will ever know. I mean, it's crazy, right? And then on top of that, like, you're saying the culture. So first of all, at this moment, like, how do you holistically understand what like, is going on in machine learning? And how do you holistically understand, like, what's going on in zk? **Speaker D:** And. **Speaker B:** And what do you. I guess this is a big question, but at this time, what do you think these technologies are for? **Speaker C:** It's like the question of the moment in some sense. **Speaker D:** Right? **Speaker C:** I think not to hammer the whole personality point too much, but they're actually just really distinct. And I mean all the way from the research to the eventual applications. **Speaker D:** Right. **Speaker C:** I would say in the case of AI, the applications are oftentimes like super obvious or super sticky, just inherently like, it's very clear that like a product after some AI integration is just like 10x better or stickier or whatever, right? And you can like, you know, call your family and say, hey, this is like what's happened because of the world AI, you can use ChatGPT and it's like super clear what the value add is. I think with crypto it's a lot more kind of subtle and nuanced, right? Like it's not entirely clear that a tech stack which is like more robust to man in the middle attacks, like, how does that really impact my day to day? **Speaker D:** Right. **Speaker C:** But we also, I think from an instinct standpoint understand that when we have networks which are just fundamentally more robust and you know, less dependent on these like intermediary authorities, which I guess is like a very high fluent way of talking about like crypto networks, we should at least in theory be able to build like really, really large scale applications that just like don't break as much. **Speaker D:** Right. **Speaker C:** Because we don't have to default to some like faulty trust assumption. And so I think when it comes to like, what are the best ways of marrying these two technologies, there are a lot of different answers. But the one that we tend to lean on is trying to bring the robustness and the concrete security guarantees that you get with all these cryptographic techniques to this wildly expressive wild animal that is AI compute. **Speaker D:** Right. **Speaker C:** And I think if you can walk that balance, right, what you end up with is both kind of the certainty and the accountability of crypto as well as just the sheer horsepower of AI compute. Yeah, but that's obviously just one interpretation. **Speaker B:** No, for sure. I mean, I think there's definitely at a super high level, like at a vibes level, right. There's something about AI that is generative and abundant and wild, as you're saying. And then there's something about like crypto, right, or cryptography or distributed computing that is like slow and scarce and like on purpose and like they, they seem. I, I feel like we're two cycles away from understanding truly. Actually you're ahead of us, but we're understanding truly, like how they marry. But to me it seems so clear that like in this world that's becoming like more and more chaotic, like crypto is an answer that is like almost opposite of the way that like the world is trending. **Speaker C:** Yeah, yeah. I think that a lot of that is very salient and certainly like we're figuring this out like everyone else. **Speaker D:** Right. **Speaker C:** I think, you know, there are lots of amazing academics, founders, operators who are trying to figure out how to like, plug one into the other. **Speaker D:** Right. **Speaker C:** Whether it's a content attribution, using crypto in the AI, like, like just mass scale, like scraping of the web world that we're in now, or I guess in our case, using crypto to make AI more accountable, more discreet, more dependable. Yeah, so definitely. Ideally, it's like the marriage that benefits both sides. **Speaker D:** Right. **Speaker C:** That's all we can hope for from the best marriages, I think. **Speaker B:** Never been married myself, but yeah, yeah, there's probably a lot to be said about marriages. But anyway, so before, again, before we get into modulus, like, let's talk about like, what a lot of people talk about. Crypto AI, right? And most of the time it's just some scammy token that is like, really actually a scam. But before we get into like, what's the right way to think about it and what modulus is trying to like, blaze the trail towards what are the way that you hear people talking about these two that like, really either doesn't make sense or just seems gimmicky or like, what's the wrong way to do crypto across AI in 2023? **Speaker C:** Yeah, I mean, definitely, for me, anyways, the moment that really crystallized what hype can do to a very nascent technology anyways is when I got a note on Twitter via Twitter Demon. It's like, hey, what's going on with the modulus token? I'm seeing it pump, I'm seeing it pop, it's dropping, it's whatever. And we've never launched a token. There's no modulus official sanctioned token. There's just like a bunch of proxy coins or shitcoins, and obviously it's a dime a dozen. There's like a ton of these out there. And I have no idea what is the underlying kind of value function that's driving the utility behind any of them besides speculation. So that is maybe the obvious poor intersection of the two that is only in ning. But I guess, you know, it's a funny thing where in some sense this is like an open. It's like chess, you know, like we can see all the pieces, we understand the moves, right? We understand, like, what AI is capable of and where cryptography is as an academic discipline. They're both like, massively improving and, you know, becoming more performant in their own ways. But also, like, underneath everything is like the laws of physics. Right? And so whenever People make claims that are like, to me anyways, really, really outlandish. Obviously I get very suspicious because, like we are looking at the cryptography every day. We are looking at the kind of things that we can, that can be done in both regimes. And I think from that standpoint, from first principles, it's actually very easy to figure out if something looks correct or a little funky. **Speaker D:** Right? **Speaker C:** So maybe as a point of example, there's a company called Jensen that we're a big fan of, which is using cryptography in the regime of AI training. **Speaker D:** Right? **Speaker C:** So trying to like build this like alternative marketplace of latent or taking advantage of like latent GPUs so that training massive models can be a lot cheaper, and then using cryptography to like check that all the training was done correctly, that is like a very sensible application to me. From a conceptual level, the challenge is making that economically reasonable and competitive with the centralized players that are out there. **Speaker D:** Right. **Speaker C:** And so I think, I guess the good news is the proof is in the pudding, right? Even for ourselves, can we actually make the math work out and make the economics something that's reasonable for end customers? But I guess that's how we shift from just an R and D world to businesses that actually deliver real value and move needles for everyday people. **Speaker B:** Yeah, sorry, just to tighten up what you said as an answer to the question, just to see if I got you, because I think what you just said is that essentially like, there seems to be a lot of ways to like possibly tie crypto and A. Or yeah, crypto and AI together. But like, if you actually like sketch it out and look at like the costs or the requirements or whatever, like the things are just like so unrealistic that like maybe like entire categories just don't make sense. But I think the second point you're making is that, and you didn't directly say this, but one of my favorite things to point out is that when you make order of magnitude improvements on technology, you open up entire new use cases. And so something today that, let's say in order to do the ChatGPT training data, it takes a proof that runs for four days and costs like $30,000. If you do two order of magnitude improvement, $30,000 becomes $300. And like, what did I say seven days becomes. I'm not gonna like couple hours, right? Yeah. **Speaker C:** Yes. Right. There you go. **Speaker B:** And so like now that suddenly becomes something that like you can build a business around. And so again, I think what you're basically saying is that everything out there that Combines crypto and AI seems a little like moonshotty to you, but like the point is to like get your hands in now, get dirty, like figure out how any of this would work and then work on just like driving down costs. **Speaker C:** Yeah, yeah, exactly. I think cost is like the, it's certainly the metric that we obsess over because it's very clear from even like early customers that we have that if we can bring the. And you know, cost, like very concretely for us just means like how many sips from the wall do you take as a computer? So electricity from the wall. **Speaker D:** Right. **Speaker C:** Like if we can bring that consumption down however much these use cases would be attainable, these use cases would be alive and well, then it becomes very linear. **Speaker D:** Right. **Speaker C:** And so I think, yeah, maybe to help myself clean up the answer a little bit, I'm always looking for how do you take this multi dimensional optimization problem into a very concrete. Actually it's just about cost. If we can get here, then this marriage makes sense. Then for me it's like, okay, at least it's a known challenge that we can meaningfully map data onto. Then I would feel a lot more confident about any specific approach. **Speaker B:** That makes a lot of sense. And I guess, like, final category before we go to like, what is modulus is like, how much do you see AI being an important factor in like, basically like the, the most credible like, use of AI on blockchain that I hear people say is like, oh, it's going to be important in forensics and tracking the chain. And is that something that you think of as a particularly interesting use case or is that something to you that is just like, yeah, it needs to happen. It seems like something that AI is going to be good at and it is what it is. **Speaker C:** It's a good question. Yeah, I guess in terms of the application of AI to blockchain, honestly, it just depends on where in the, in the stack you want to kind of slice that. Right. Like in some sense, like we can have AI models, including LLMs to help us write our solidity contracts. **Speaker D:** Right? **Speaker C:** Like our little smart contracts. But then like, ah, but you know, there might be like security bugs that the training Data inherits from GitHub. But then we can get a second LLM to like audit the code and then go back and forth between formal verification. But like, okay, that's like already like, that's like the lowest level intersection. **Speaker D:** Right? **Speaker C:** And then go a bit further up. Okay, what about smart contracts having access to AI outputs both centrally just by querying like OpenAI and potentially using ZK or in a decentralized fashion through someone like modulus. Okay, then like maybe it's a UX improvement. Let's go even further up the stack. Then let's use AI model to intersect with not just smart contracts, but literal applications. **Speaker D:** Right? **Speaker C:** Can we pull in a bunch of, you know, defi service data or data defi services and sit together some like new composite application which is know, smarter analytics for the chain and forensics and you know, AML and fraud detection and so forth. So in the same way that I think AI will be a big deal for or already is a big deal for the rest of the world, I would be very surprised if there isn't a good intersection in crypto. But that for me anyways is like, that's easy to say in terms of laying down like concrete steps to get to that future where like crypto is better because AI, like that's why modulus exists and that's why like our peers exist. **Speaker D:** Right? **Speaker C:** Like we need to lay down the rails and actually push that train up the hill until we, you know, we get it to a point where it's self sustaining and the value add is just like so obvious. It's like, oh, what reps your smart contracts don't have AI in it? That's like, you know, you're totally lame for not doing that. Like it's so clearly secure and working and creating value. You know, let's, let's, let's catch you up. **Speaker D:** Right? **Speaker C:** So we're still on the other side of the flywheel. We got to like push over that friction. **Speaker B:** Yeah, yeah, for sure. Yeah, I think. Yeah. Well, we can keep going on that forever, but like, let's start to ground this conversation around what you guys are actually building. So tell me, like, what is modulus? Like, I guess if you have an interesting origin story, let's hear it. But like more specifically, like, what is the problem that you see in this space? And like, what are you guys doing to if you know, again, this is a fast moving space, if not solve it today. Lay the ground rails to build the future. **Speaker C:** Sure. Yeah. And actually I think maybe like telling like literally what happened at the original modulus is a good way to unpack what is oftentimes like a very confusing topic. **Speaker D:** Right. **Speaker C:** So I guess back to wherever I parked us, we're reading all these crypto papers, we're falling in love with zk. Like the math nerd in us is like screaming with joy. And we have this moment where we realize that actually this incidental property of zero knowledge cryptography, which we call succinctness. But basically it's faster to check a Sudoku puzzle than it is to solve a Sudoku puzzle, if you will. Makes it so that you can bring arbitrary off chain logic into smart contract embedded in there. But where what's in the smart contract is just basically the check of the Sudoku puzzle. Like, oh yeah, the math was done correctly. Let's duplicate that compute around all the nodes. And this succinctness property, maybe, I guess the insight we potentially had at that point anyways was maybe this process is good enough where we can start to prove off chain AI compute. In other words, imagine you're running AI computation, so AI features directly in the smart contract. So this is just like no one had done this yet. Folks had been talking about it for years and years. We were talking to folks at ef, even at Solana about whether this is possible or not. And generally the consensus that we got anyway was just maybe it seems borderline. So we thought, okay, school's about to start again, we're winding down from quarantine, let's just try as a science experiment, what we ended up doing was training a very simple model. This neural network. It had I think about 70,000 parameters, which by AI standards is very tiny. It was trained on historic ETH prices and all it was trying to do was to classify or make a prediction rather of whether ETH prices would go up or down in the next hour. We then converted this model into ZK circuits and improved it and put the proof on Ethereum mainnet so that it could interact with the uniswap contract on L1 and Trade ETH. And it's worth noting, this entire process I've mentioned, which is a neural network making a decision about or making a prediction about what will happen to each prices and and then trading against an L1 contract is fully ZK proven, fully tamper resistant, and better yet, we put it in an account where there wasn't even a withdrawal function, right? So all you could do was just let this robot decide on what it wanted to do and execute on that forever until it ran out of gas. And we put like 500 bucks in there just to like, you know, wish it good luck and have it go and do that. So we thought that was kind of funny and we put it on Twitter and the rest of the. And what we didn't expect, which is what happened next, was that a ton of people just online started donating to the Rockefeller bot, Rocky Bot we called it just to keep it alive. And you know, we're talking like thousands of dollars in and out. And to us it's like totally wild, right? Because there's no upside. Not only is this model definitely not performing, if we solve the crypto markets, believe me, like I'd be, you know, enjoying the fruits of my labor. But not only was it not performant, there was literally no withdrawal function. Folks are purely donating in the truest sense of the word, just to keep this thing alive, I guess, because we are all excited by this idea of like autonomous AI agents just like executing without any fear of manipulation, let's say. So, you know, Rocky Bot did that. Eventually it ran out of gas and died, but it traded for over a month. And definitely it planted the seed for, for modulars, which is just, you know, I guess all the work that we think might be necessary to, to actually have performant massive expressive models in smart contracts without breaking blockchain security at all using zk. So at the most basic level, that is kind of what we're doing still, which is just like, how do we take this idea and magnify it, right? Bring it into production, bring the cost down so that it's not just like silly toy examples, but value generating, value driving features that actually make a difference for Web3 clients. So yes, so that's like the origin story, quote unquote. You know, we've been around for just about a year now and you know, that year has been filled with like research and papers and other projects. **Speaker B:** So let's hover over just like what modulus is from, you know, using the Rocky Bot as the example. So like taking a step back, like the important thing here is like, look, it was always possible before to like create a little smart contract that and like, let's just be really specific how this is working. And I'm assuming, so correct me if I'm wrong, but some computation is happening off chain. Like that could be on a laptop at your, in your guys's office. It could be from aws, like wherever, something entirely unrelated to Ethereum. And then whatever's happening there comes a, like just a very simple command, buy or sell. And then that gets sent in a Ethereum transaction into the evm. And then your smart contract will say, okay, I just got this command. And before modulus, before zk, you're basically saying, I trust that this message not only came from who it's supposed to come from, but the computation they were supposed to run was actually running. And so I don't know if it's worth really unpacking why that trust is a pretty powerful trust assumption to have and a pretty vulnerable situation to be in. But what rockybot did and what is the start of modulus is just add in one little piece of magic, right? Which is a ZK verification that, that yes buy or no sell or whatever that command like actually ran through the machine learning model that you know, the smart contract, like was expecting it to run through. So first of all, like correct me if I was wrong on any of that but like, yeah, I guess like can you just like very briefly like talk through like why it's so important to do that ZKVerify as opposed to just like basically accepting the message or creating some sort of like you know, signature password scheme or like what is the magic that ZK is bringing into that bot? **Speaker C:** Yeah, for sure. And Rex, you're totally right. Like the, I guess the final part, the on chain verifier, it's just like a gatekeeper, right? And it's just checking the receipts, if you will. **Speaker D:** Right. **Speaker C:** You know, what was modulus or whoever honest we call this process, this idea like faithful AI in the sense of like AI that can I guess cheat on you in some sense. **Speaker D:** Right. **Speaker C:** And I do think it's totally worth unpacking. I guess obviously I would say this, but the kind of counterparty risk, right? Where like what's at stake here? And certainly like if the output is pretty frivolous, maybe it's not so important. But insofar as this output potentially impacting a lot of money, like a lot of dollars are at stake or a really important decision is at stake or you know, even more importantly a super autonomous system where every piece of software just automatically goes to the next piece that is happening. I think it's going to be absolutely essential that we have this gatekeeper that goes. Hang on a minute, let me see if anything like any hanky panky is. **Speaker D:** Happening. **Speaker C:** It goes beyond just oh, maybe there will be 1% or 2% shift in prices. I think it's stuff like hey, is my credit score fair? **Speaker D:** Right? **Speaker C:** Like, like whether on chain or not, right. Even Web2 like am I being discriminated against because I have a weird name, quote unquote weird or because I live in a specific geography, shouldn't really say anything about like my ability to be a good borrower or something like that, right? All the way to, you know, is is this NFT price fair? **Speaker D:** Right? **Speaker C:** Like, like is is there like an appraisal process that is actually Neutral about like, assigning value to my assets or assets that I'm doing financial transactions on top of. And of course, you know, zooming out from crypto specific financial use cases, even stuff like, did this AI artists actually create the piece of art that I'm getting? Or is it from some other model or some other person? Or if I'm querying a model for financial recommendation, medical recommendation, any type of legal recommendation, I really want to know that there's no way in the world that whoever's giving me that output, flip a specific number, hammer the weight, or totally just swapped out the AI result entirely and fed me some falsified result. **Speaker D:** Right? **Speaker C:** So the most succinct way I've thought of anyways to capture this idea is at least you have to go back a few months. But it's like a Twitter verified check mark of old, but for your AI outputs, right? And I think in a future where the world is like really autonomous and like expressive, compute in general becomes a huge slice of our compute diet just as a society. And having these gatekeepers, having these mathematical guarantees is how we make sure that the world continues to stay very robust and we're not like, subject to some massive vulnerability or some terrible attack. **Speaker B:** Yeah. And I think just to like, really drive home the point, it's not just about like the verified check mark, like per se, right? Which is some, like, you don't even have to, like, you're trusting that actual steps were taken according to like the code that you expect, right? And so like, you're, you're not even dealing with this like, first level of like, verification. Like, make sure it's like the right person at the right time. Like, you don't even have to trust that like essentially that Jack Dorsey and his public literary company will ever get sold to like a megalomaniac because, like, it doesn't really matter because, you know, the actual things that are being done under the hood. And like, that's kind of what ZK provides. And I guess, like, you know, for, for people who already like, live and breathe this stuff, this is such kind of a basic concept. But for people who haven't made the jump yet, like, the reason why we, we so breathlessly like, switch between crypto and blockchain is because, like, it stems from the idea that, at least tell me, Daniel, if you agree with this, that like, blockchain should not be understand as like cryptocurrency as currency or as financial or like, whatever. What it should be understood is like the base layer that one like the property layer of the Internet, but more importantly, it's like the base layer for distributed compute. Right. And I think that the piece to understanding that is like what ZK cryptography allows us to do is take computer from centralized, from like dark sort, from wherever, literally wherever, and then verify it in this like, kind of pristine, like untouchable space. And then you know that a lot of it's social, a lot of it's actually not. But like through that kind of magic, we're able to say, okay, like we're able to transform anything we want to into trustless or into almost decentralized is a loaded word. But yeah, I don't know. I mean, any reflections on that before we move on? I just think we don't really talk about what ZK actually is and allows very often. **Speaker C:** Yeah, I mean, in some sense ZK was invented for privacy. This is a very strange unintended kind of consequence of that original conception. **Speaker D:** Right. **Speaker C:** Um, but I guess on the point about, you know, like what blockchains are good for and what it means to, you know, like this for me anyways, is like the ultimate upgrade to the blockchain. **Speaker D:** Right? **Speaker C:** Like, this is like a superpower. Like we can just like plug shit into the blockchain and have the thing that we plugged in be like, elevated the same security standards as the underlying, like, robust network that we spent a lot of energy getting just. Right, right. I mean, switching from proof of work to proof of stake being like one of the many milestones that we as a community within crypto has been working for years and years now on to make sure that it is that pristine, wonderful place where the robustness of the network is never in question, where it's always top priority. So, yeah, I think I'm totally following, obviously preaching to the choir a little bit, but I guess maybe just to take the skeptic or the cynic's perspective and to speak to that mindset a little bit, the goal here, like, make no, no, no mistake about it, the goal here, at least for me, is always to build more powerful applications, like services that are just fundamental, like truly like 10x better or new as compared to what's existed before. **Speaker D:** Right? **Speaker C:** Like, I think this is like we live or die as a community on this hill. Can we, like make just products that all else being equal, can outperform what exists today? **Speaker D:** Right. **Speaker C:** And for me anyways, like, bringing this, like, AI upgrade to blockchain networks is one of the key ingredients of how we do that, how we build truly unique, truly beyond what's already been done, which is also obviously very impressive. But how we truly be competitive in these just marketplace of ideas and again have an impact. So the goal is to ultimately products with capital P make something that's just like dynamite. **Speaker B:** Yeah, for sure. Yeah. I very much resonate and like that idea of what ZK is the ultimate upgrade that allows plug and play into you know what into crypto. Like if we don't want to be like too maxi about it. But at least this audience knows like how I feel that like there's crypto is. Yeah, it's Ethereum, right. Like that the purpose is to have one shared space. And I don't know, I think like for me, well, you know, I, I won't wax poetic too much on this. Like let's, let's continue talking about. So, so we kind of walk through the Rockefeller bot and why it's so important to have this ZK proof. And really the ZK verification on chain that as you mentioned is a pretty light lift, that's essentially a couple of math operations using a commitment and you're basically checking some equivalents and if it verifies, then you take an action. If it doesn't verify, you basically halt the. What is challenging is creating? Well, you, you tell me what's challenging. But like obviously like what all the hard work is on is on the computational step and like creating systems that like are verifiable. So can you talk a little bit about one, like what, what are like the broad strokes of the challenges and like how do you make a system basically generate ZK proofs that can be verified? And then two, what are the specific problems around ZK or sorry around machine learning that make the challenge even harder than just general purpose computing? **Speaker C:** Yeah, totally. And Rex, I think these are exactly the right questions to ask in some sense because they were the ones that we were asking after rockybot. It's like, okay, yeah, like this, this idea is kind of cool, but like what do we need to do next if we want to actually productionize this, this whole process? **Speaker D:** Right. **Speaker C:** And so I guess the first thing we did back then, but also maybe the first question that we just felt like we really needed to have a concrete answer around is like how hard is this actually? **Speaker D:** Right. **Speaker C:** Like as we scale the size of these models into something that's actually useful and not just like basically guessing at if prices would go up or down, like what are the challenges that we're going to bump into? And so what we did was we assembled six of the Major kind of popular proof systems that were widely used today. From different schemas to different techniques, all different flavors. And we tested the same kind of AI models across different sizes on all six proof systems. And very quickly we realized that first of all, the performance varies a lot, sure. But also maybe more importantly, the variance in performance. Actually doing this was a little bit painful. And not just because the software is bizarre and unique and we had to get used to stuff, but also because these ZK proof systems as they exist today were all built to prove the EVM or some VM generally, which makes sense, right? Because we have blockchains and we want to make them cheaper. So we just want to prove blockchains but locally and then put that proof in the expensive computer. Which is a little strange because AI compute is both simpler and much bigger than VM compute in some sense. You can consider the compute load of an AI model basically to be mostly matrix multiplication. This one operation, a bucket of non linearities and then times a bazillion times like just that simple set of operations at a massive scale. We don't really have to worry about memory or registries or all the other complicated VM stuff. And so it was like, it was like, you know, you have like a, you have like coal or something like just like train fulls of coal and you're dragging this thing with a school bus. And like school bus are awesome. They can go like off roading, I guess they can like turn and you know, like fit kids and adults and all the rest. But like it's not designed to like be pulling like you know, car carts and carts of coal or shipping containers let's say. So that kind of brought us to the core insight that has been the driving technical mode for the company for most of this past year, which is what if we just built a specialized proving system? Let's just from the silica up build a ZK system which was tailor made for the kind of mathematical operations that AI problem statements use. Basically a lot of linear transformations. Because theoretically like if you rex just imagine like a ZK proving system which is only slightly better at every single matrix multiplication. When this one operation happens like you know, tens of thousands of times, it should just conceptually be a lot faster. And so that's been like the big I think insight for us and what we've been working at for the past however many months. **Speaker B:** So when you say, the thing you just said at the end is basically that if you can using ZK create a. This is probably too specific. But let's say like a matrix multiplication that is like slightly more performant because that matrix multiplication happens so many times, you're going to save a lot of time. And I think what you mean by more performant is like a little weird. It's in ZK world, right? Because it's not necessarily the actual operation itself is more performant. It's actually probably going to be a lot less performant. But it's that the verification is much faster than actually running it yourself. Is that what you're saying or am I kind of. **Speaker C:** Yeah, yeah. I mean, here, let me make it super concrete, right? So the project we built after Rocky and after the paper, where we did the comparisons, the benchmarks is. It's a game, it's called Leela versus the World. Won't get into exactly how it works. That's for maybe later. But what's important is that the statement of work, the mission here is to bring a 4 million parameter AI model into Ethereum, right? Whatever it takes. And also don't break security. Now if you were to run this 4 million param model, which is much larger than Rocky, but still relatively small by AI standards. But if you were to run this on L1 on Ethereum mainnet, a single time one forward pass would cost and obviously gas varies a lot, but on the order of about $30,000. Never mind the fact that you couldn't actually do it because block space and the rest. But that is obviously crazy. Like, like, you know, it would have to be a tremendously value generating AI decision to justify that. Now if we pick up this model and we put on L2, that is one of these ZK proving systems, these ZK L2s which were made to emulate the EVM, it would be about a thousand bucks, not too bad, right? Like enormous cost savings compared to L1. But still this game better be like amazing, right? To justify that. If you then take it and put it into a proving system which has been like tinkered with and tailored for the specific kind of math that this AI model uses, which is what we did, it ended up being about $2 per decision, right? So, you know, in terms of the cost curve, went from 30k to $2 for the proving and then, you know, a couple more for the settlement on chain. The problem, even though that sounds like a massive improvement, is to just run this model once on AWS or even on your laptop, costs in electricity a fraction of a fraction of a fraction of a cent, right? And so fundamentally it's still the ZK Proving is still very expensive. And so for us, the question was, okay, can we keep going down that cost curve until adding AI models into your smart contracts is barely a lift at all? **Speaker B:** Makes a lot of sense. And so how much are you guys concerned about the process of adding ZK into ML in terms of like both performance and like, like speed of execution, but then also like cost? Is that on the ML side? Is that actually like something that's going to become a problem or is it really more about like getting all these pieces to fit together? **Speaker C:** Yeah, it's a great question. It's a question that honestly we obsess over. **Speaker D:** Right. **Speaker C:** Because in some sense, if you look at things from the blockchain standpoint, it's like, oh my God, this is great. You're massively reducing the cost of running these models. But if you're looking at things from the AI standpoint, I. E. No proving, let's just run it once on AWS and that's done. It's like, oh my God, you're multiplying still because you're so often through the AI compute, you don't save any costs there. But, but then you also have to do these like, sophisticated cryptographic operations on top of the AI compute. So from the AI standpoint, it's like, man, you're going to like take my overhead and blow it up by like a thousand x. Right. So if it used to cost me a dollar to run some, like, some a day worth of AI, I need to pay $1,000 at the end of the day to show people that I didn't manipulate the results. **Speaker D:** Right. **Speaker C:** That's a, like, that's a heavy lift. And so a lot of the focus from the modular side and by the way, that's, that's about the, the blowup right now, like the state of the art blowup. Even optimistically, it's about like a thousand X on top of just doing the AI once, naively. **Speaker D:** Right. **Speaker B:** And sorry, that's a, that's like a cost blow up. But is there also a, like, does that increase the time to, let's say like, do a full pass of the AI model? **Speaker C:** Yeah, yeah. In fact, that is in some sense how the cost is reflected. I need to keep the computer running for longer and so I got to pay the Columbia River, I guess, for hydroelectric generators. Right. **Speaker B:** So it's not only that you have problems on the cost side that this whole construction might blow out your latency so much that applications might not be buildable. **Speaker C:** Exactly, yeah. I mean, this is for us after we built Rocky, we did the paper and then we did Leela the second project, we were like, okay, if we want to keep going down this, this train, right, and actually get to a point where AI models are useful, we need like a step function change. We can't just like incrementally improve things anymore. And you know, the good news is of course like, like it seems like there's this opportunity by specializing approving system. But even independent of like the, the approach, just from a statement of work standpoint and a cost profile standpoint, we need this change, right? We need, I mentioned earlier about like school bus, like towing a lot of weight. We, we need a bullet train, right? Something that just runs on a single track but goes like blazingly fast. So that's been the motivation behind just modulus really. Let's massively cut down on that cost by hyper specializing. **Speaker B:** So okay, is it fair to say that the big bet and the thesis of modulus is like we need to. And somebody will build it. We're going to be the ones to find that step change? **Speaker C:** Exactly. Yeah. Like leave no stones unturned and explore every like forgotten idea in the, in the legacy of cryptography and ZK until we can prove AI in a really like efficient, low weight, low latency. That's a great point. We didn't even, I'd even mention rex, but yeah, low latency kind of way so that we can empower smart contracts with AI models, but also just like in general, like in the world introduce this idea of verifiable AI, faithful AI. **Speaker D:** Yeah. **Speaker B:** So when you're like looking at it today and, and it's like a big hairy problem, like you wouldn't be able to raise money if you hadn't answered this question. Right. But as you look at it today, where do you think that that step change is going to come from? Obviously the answer is going to be everything that I just say right now. But you know, if like you could really just like in your gut and like what you and your co founders are like, really see it. Is it really about silicon level, like you know, ASIC level, silicon specialization? Is it about integrated systems so that you can handle like, you know, latency and like transfer problems like down to, you know, a microns of a second, is it, I mean, is it about like just creating more performant models both on the machine learning side, but also on like developing just the number theory and the cryptography side? Is it something that I'm not even aware of? Like where do you see the step change like really, like what's your secret sauce? **Speaker C:** Yeah, yeah, no, this is the depending on how optimistic you are, multimillion to multi, hundreds of millions, multibillion dollar question, right? And actually I think I'm going to answer this question in a weird like kind of annoying way, which is to, to like very briefly visit like, like super recent history, right, which is, you know, like my whole team, the founding team, we came from like the AI world, right? And it's easy to think that like we've been doing AI forever, right? But it's not true. Like deep learning, basically the modern paradigm of AI didn't really start to like happen until 2009, right? And so like a very, and you know, the theory has been around for like 40 plus years, right? And so, you know, the reasonable question is like, what happened in 2009, right? I know there was a big financial collapse in 2008, but then what happened, right, like, like out of the wreckage of the housing crisis, like how did AI, you know, just like deep learning show up? And the answer is GPUs. Just like GPUs became widely accessible. And you know, it's worth noting, like GPUs is just a different way of arranging silica than CPUs. Like you can build a server farm GPUs using CPUs, right? Like theoretically that's totally doable. It's just like much more expensive. It turns out that instead of having a couple really powerful cores, when you have thousands of not very powerful cores, you can just turn through matrix multiplication as an example much more efficiently. And when you do this at massive scale, these theoretical techniques which, you know, power modern AI, which used to be not very performant, just have like magical emergent properties, right? Like if you do, if you took ChatGPT, you know, four or whatever, and you, you reduce the pram count by a thousand, it's not a thousand times worse, it doesn't work at all. But somewhere along the line, you know, boom, it starts to be able to perform, right? **Speaker B:** Well, and just sorry to take us back to earlier part of our conversation, like when you make order of magnitude improvements like new capabilities are possible. And like this is like the most high tech but like most like perfect example of like you could calculate by hand a neural net of like, I don't know, let's say like a hundred. Like you'll see like it's dumb, like it doesn't do anything. It's like bubbles and char, you know. But yeah, once you like crank it up to like 100 billion or like 3.5 trillion or whatever it is. Like your computer talks to you. **Speaker C:** Yeah, right. Yeah. And that's like a super, like I like, I lament being like the VC who like missed that, I guess, quote unquote because it's like not at all obvious. Right? Like what? How does that make any sense? **Speaker D:** Right. **Speaker C:** But I guess my point is that if you'll kind of tolerate my perspective here, this is a cost story, right? Like we hit a point in 2009 ish, where suddenly the fundamental cost thesis of compute change and boom, AI popped out, right? And I guess from the modulus worldview, this is available to us in the ZK world as well. If instead of just optimizing for EVM and or EPM operations, we built specialized ZK provers that took advantage of all the other cores, virtual or literal, in a compute cluster or a compute node, we can start to prove really, really like diverse and expressive kinds of computation that previously was just like totally irresponsible. **Speaker D:** Right. **Speaker C:** Now we happen to call that compute AI, but you can call it other things as well. **Speaker D:** Right? **Speaker C:** So that is the kind of like the technical insight and the motion anyways of technical insight that we're trying to, I think, bring into the world of ZK proving. **Speaker B:** Yeah, very. And so like, I think that's a super good way to frame it because you contrast yourself very well with one of my previous guests with the guys from Risk Zero. And like what, what they're doing is like they have the same insight that you and I and basically like everyone at the cutting edge of the next bubble has, which is like what you would ZK cryptography allows you to do is computation outside of Ethereum and then project it into the evm. But what they're doing is like creating a general purpose VM and then like basically saying we're going to create environments that are accessible to developers like as they understand them today. And then they can focus on building on application building applications as opposed to like, you know, any of this crazy stuff. You're taking the same approach, but instead of saying like we're going to just build like general purpose tools to see what you guys come up with. It sounds like you're saying we know what the real professionals, the real cutting edge guys need and we're just going to skate ahead and build the tools that they need today. One, is that a fair characterization? But two, what are the next customers or use cases or how do you see the work that you're doing for machine learning branch out into other Aspects of computation. **Speaker C:** Yeah, totally great question. And certainly the Risk zero guys are awesome. No disrespect to what they do at all, which is quite frankly much harder than what we're doing. VMs are complicated as you can imagine. And I think that obviously we're all kind of on the same team here which is like, hey, you have smart contracts, let's just add more logic to it. Then you can build cooler stuff. And we all benefit from a world where services aren't just super compute limited. I think Rex, you're exactly right in that we're just like a bit more opinionated. We say, hey, it's not just about expanding compute, the surface area or the design area for compute, which obviously would be great. But it's specifically about adding AI features because we think AI is the right beachhead, if you will, for how a dev should maybe think about oh, how do I improve my smart contract? I don't know, try decision trees or random forest or try a recommender model to give your users more personalized results to maybe like generative results. Maybe that will make the smart contract based service just much more friendly, personal, powerful. So yeah, we're just like on the same team, just like from different angles. But yeah, so I guess that's like the high level but on a functional level as well. We just think like a more tailor made tech stack makes sense for AI while just because like if you'll permit me again we're going to do a graph. If this is compute, like more compute and this is like utility at zero compute there's no utility. Sure. And a little bit of compute is like a massive amount of utility and it comes down again, I would say this part is like L1, L2 level compute. And I think the risk 0 guys and everyone else, us included, is saying hey, as you go more there's even more value modulus. Just believes that at the very end here, as you go all the way up, there's another spike which is like the AI, you know, like the treasure trove of value that can be unlocked with AI level compute. And actually we're going to build just for that because we think that's like a great way to improve your smart contracts. **Speaker B:** Yeah, for sure. And I don't mean to like put you against Risk zero as a negative thing. If anything it's like yeah, like the more people that have like the right ideas and the same ideas means like we're onto something here. But I mean I think just like specific specific, specifically bringing them up to like point out that like they're doing general purpose, you guys are doing machine learning, but like you're not like the purpose of modulus is not to live and die on machine learning, right? It's to conquer machine learning and then using all of the like specific applications or silica or like whatever you built in machine learning to apply it to other parts of compute that are like super, super intensive, super super specific but like need these step changes in order to like actually become like ZK if I. And so just to ask it again, like what, what are the other kind of genres or computer compute applications that you think that modulus really has a chance to change after you guys are in conjunction of changing machine learning? **Speaker C:** Yeah, it's a great question Rex. And I guess the most direct way to answer is to just ask what is an AI model underneath a label and if you pop open the hood of the car and you look at the engine, an AI model is just a ton of linear transformations. It's just a lot of statistical operations turns out like stats at scale. Like if you stack enough stats together, eventually you get AI, right? And so when we say that, you know, our, our prover and our whole stack is oriented towards AI like like compute or AI level compute, what we're really saying is any kind of big data esque statistical operation should be just much faster at production scale. When you, you ha. You take this like Taylor May prover approach, right? So in terms of the early use cases, which is kind of like the next gear shift we're getting into, right? Like we did the early experimentation, we wrote the paper, then we did all this R and D on a specialized prover to see if it's like meaningfully faster. And now we're throwing it into third gear, which for us is what are the first big production level applications, right? And it kind of follows a little bit, kind of maybe what you'd predict, right? It's folks who want to do, you know, statistical operations within the defi regime. It's folks who want to do generative art for NFTs, it's different services who want to prove your identity using facial scanning models or iris decryption. In the case of worldcoin, which is, you know, one of the folks where we're working with, it spans the gambit, you know, everywhere where AI seems to be making a difference in the quote unquote rest of the world that is Web2, it seems to be, you know, following a similar, at least exploratory trend within crypto. And it's not Always AI right, sometimes just like massive stacks. But we still think that this specialized prover will fundamentally just be much more attractive from a cost standpoint for those applications. **Speaker B:** Yeah, for sure. Well, I can like continue to go on this like conversation forever, but let's bring it into a close here. So what like, are the most exciting things that are coming up on modulus, you know, like roadmap or radar? Like how do you. Yeah, I guess when you're at home talking to your family or talking to your investors, what are the next milestones that you see that you guys are skating towards? **Speaker C:** Yeah, definitely. It's a busy time of year for us, I think the next big thing. So we'll be at ZK Summit 10, I think, where we'll be debuting basically, I guess the theoretical underpinnings or foundations of our specialized proving system. So that will be this month, around October, November. Ish. We expect both the audit on our first code base to be complete as well as the first production implementation of that code base. And as I guess maybe a metric here, we'll go from the current world record held by us, which is 12 AI proofs per day, to about 10,000 AI proofs per day within the next month. So that will happen. We'll also be open, sourcing our code base. We think that will be really meaningful when it comes to completing the security story around ZK for us as well as just like, hey folks, you know, like, we think that specialized proving is cool. You should look under the hood and see if you agree. **Speaker D:** Right. **Speaker C:** And then by the end of the year, we expect to be onboarding our first handful of, you know, let's call them like more White Glove customers right. In the early days before dev tooling matures and be kind of paving the way to a self serve platform. So that's the rest of the year for us. Hopefully we get to at least most of those things. But we're still a small team, so there's more work to be done than people could do it. **Speaker B:** Yeah, for sure. And so hopefully by the end of the year, these White Glove initial applications, you said that these are things that are essentially, how do we use machine learning to process both, like basically data related to blockchain activity. **Speaker C:** Exactly. Yeah, you got it. You got it. **Speaker B:** Very cool, very cool. Well, all right, like again, before I let you go, first of all, just thank you, man. Like this is such an incredible conversation and really. Yeah, man, I think like you're so clearly right at like the beachhead of like two technologies that we all Understand are moving so fast and are we're just behind the curtain of like the real specific ways that not the theoretical applications, but the actual transformations that our world is going to see. And so, man, I'm just so impressed and excited for you that what you guys are building with modulus and man, I hope to just continue to stay in touch and have you back on the show as you guys continue to hit milestones and show us the intersection of these two incredible technologies. **Speaker C:** Rex, the pleasure is all mine. Thank you so much for having me. I mean, the fact that you care about this stuff or your audience does, I'm charmed just like on that premise alone, never mind, like the excellent conversation. So thank you for just genuinely thank you for having me for sure. **Speaker B:** And before I let you go, where can people find you? Like, if they want to learn more about modulus or get involved, like, what do they do? **Speaker C:** Yeah, I mean, honestly, the. We're not really like Twitter, slash X animals, but the easiest way is probably to just follow the modulus Twitter account. And if you're like, man, Daniel looks like he really is in desperate need of an ego boost. You can also follow me personally. I'll just be posting like, maybe behind the scenes stuff. But yeah, Twitter is the easiest or X or, you know, we got to move to like, Rex, what's the name of your. Where you moved your stuff to? **Speaker B:** I don't know. The outside? **Speaker C:** I don't know. **Speaker B:** Yeah, yeah, I don't know, man. **Speaker D:** We're. **Speaker B:** We're all struggling here and all I can say is I stopped putting effort into X because, because, like, I'm pretty sure that it will be gone before the end of the year. But until then, that's where we're all coordinating. **Speaker C:** So, yeah, maybe. Maybe our medium blocks. Yeah. As a, as a backup. And yeah, maybe mirror after that. But anyways. Yes. **Speaker B:** Well, again, Daniel, thank you so much. It's been a pleasure and hope to talk to you soon. **Speaker C:** Thank you, Rex. Really appreciate your time. **Speaker D:** Sa.