Auditing, Security and Safety Inside the EVM w/ 0xTaiga (Consensys Diligence)

**Speaker A:** Hello and welcome back to another episode of the Strange Water podcast. I'm so happy you're joining us for another great conversation. Thank you for tuning in. Smart contract auditing is one of those background topics that very few people really think about. The basics are straightforward. Developers will hire specialized auditing firms to painstakingly look through their code, looking for errors, bugs and other vulnerabilities. The auditors work with the developers to harden their protocol, hopefully making it safe and ready for production. And then when devs are ready to ship, they make sure to loudly and repeatedly tell everyone that their code is audited. Now, the final part is only implied, but most of us just do the work for them. Because their code is audited, we assume that the protocol is safe. But just like all things in this world, things are much less simple when you look under the hood. It's one thing to say an auditor is going to look at your code, but what is the auditor actually going to do? Are they going to read it line by line, Throw a bunch of test cases, try to hack the pre built binaries? Today's guest is 0x Taiga, an auditor and a security engineer at Consensus Diligence, which is Consensus auditing firm. During the next hour you're going to hear Ty walk us through the ins and outs of smart contract auditing, including the basics of how an audit works, how smart contract programming differs from more traditional programming, and even some valuable alpha on how to keep your auditor happy. When you finish this conversation, I hope you walk away with two main points. First, if you are developing on chain, the standard that you need to be working towards is absolutely perfect bug free code. This is just not a place where you can cut corners. Second, you are rarely going to hear about audit companies or audits unless something catastrophic happens. But the hard work of auditing happens every single day. For a job that we all implicitly and often explicitly trust with our digital net worth, we don't give it nearly as much attention as it deserves. One more thing before we begin. Please do not take financial advice from this or any podcast. Ethereum will change the world one day, but you can easily lose all of your money between now and then. Alright, let's get started. Hi, thank you so much for joining us on the Strange Water podcast. **Speaker B:** Well, thank you for inviting me. Thank you Rex. **Speaker A:** No man, awesome. I'm super excited to have this conversation. But before we kind of get into like the more interesting things that are happening in today's crypto and today's defi, can we just hear a Little bit about your background and like what I, what I love to hear is like, who were you before you like had heard of crypto or understood the power of crypto? And like what was your moment where you realized like, oh my God, I need to be involved? **Speaker B:** Right? Yeah, that's a good question. So basically before crypto I was a software engineer at a major game engine company that most of you most likely know unreal. But the other one that most of you must like know, it was Unity. So it's more like less AAA but still a rather popular gaming engine. And what I was doing there is I was in their AI department. So, so we were basically working on reinforcement learning, being added to different in game objects for developers basically to breathe in. Some live into their game objects or industrial applications. Right. So it was a hot topic at the time. It was really cool. And for me it was a hot topic up until I learned about crypto. So then the first day I learned about Flash loans and arbitrages and basically any sort of on chain activity where you can make money without risk of losing money was fascinating. Right. So I feel like a lot of people are actually going into crypto that route. And for me it was the same. I started writing bots, some bots that were just centralized exchange bots like on Binance and things like that. But after that I just wanted to learn a little better what the smart contracts are about. And I felt like whatever I'm just watching on YouTube and whatever I'm trying to do myself is just very basic. So I started seeing if I can contribute to other protocols or anything like that. So did contribute to protocols, write some smart contracts and eventually I just wanted to convert to crypto full time. So I just went and start working for Metamask. So I did work at Metamask for some time, but unfortunately at Metamask I didn't get to write smart contracts or read smart contracts. So I just went to auditing position and consensus diligence later. And yeah, that's where I am at full time currently at consensus diligence. I'm also contributing to vendor finance. But the diligence part is also extremely entertaining and I know they did speak about vendors, so we can talk a little bit more about auditing as well today. But yeah, but this is like in a nutshell how I ended up in crypto. **Speaker A:** Yeah. So let's just unpack that a little bit more because like especially like now in 2023, you know, the only thing like kind of like at the more bleeding edge of technology than crypto is AI. And so, like, it sounds like you were like playing with those concepts before and like, more so than just like creating research projects or like chat bots and throwing them out there. Like you were using this technology to build real products. And so can you just like talk a little bit about like. Like that is like kind of the wet dream of a. Of a builder or technology person. And like, what. Why was this opportunity? Just how did it speak more to your soul, right? **Speaker B:** I mean, AI is fun, right? Like what? Technology to me always was an opportunity to just create something, right? It's like magic. You don't need anything except for your computer and then you create stuff, right? And that in itself is art. And past that point, AI is like a magic art where you just do something and then it does something that completely blows people away. And it's relatively accessible to developers. There's a subset of people who actually write those different models, trains them, or not necessarily trains them, but designs the models. But then training and actually deploying a model is not that difficult. So this magic is becoming more and more accessible. And I felt like there's just too much attention around it. I didn't feel like I'm still early, you know, and it was just before those, like, I guess not just, but like a year before the Transformers were introduced. So it was commonly introduced, I should say. But essentially I felt like there's a lot of competition and there's a lot less room to kind of impress someone or do something that going to be novel or you have to be at the very, very cutting edge of that and have a completely different kind of level of math and computer science knowledge that I might possess or would like to possess, right? And then the crypto kind of scene comes along and then you see that you can basically take advantage of all your data structure algorithm knowledge, right? A rather short program that has to be the perfection, right? And then have it hold a lot of money. And not a lot of people know about that just yet. And that's a whole nother level of thrill and adrenaline. And like, it's unbelievable that you can get adrenaline writing code. That was weird to me, you know, but that is what happens when you add the money into equation. And that's what you get as an auditor. You get a thrill of actually finding a way to take the money. And as a developer, you're just constantly scared that somebody will find them. So I've been on both sides and it's interesting, I kind of understand both sides at this point. So. **Speaker A:** Yeah, well, it's not just as a developer, you're constantly worried about something going wrong. Like as a user, as a researcher, like as a participant in this ecosystem, I think I'm just as afraid of as the developers. But I do like it really resonates with. So I studied computer science and then never did programming professionally. But I just like so vividly remember like the first time I got a computer science homework assignment, it was like I ran home and I did it. It was the first time I was excited about like work, you know. And I think that same feeling with you know, for every intro like program they, they make you write your own. I think it's a C compiler. Right. Like anyway, like these like kind of like complex things, especially for students. And like I do remember getting that adrenaline like when all your pieces kind of click together. But man, I can't even imagine like the rush when you put financial stakes to it. **Speaker B:** You're right. You're right because. And that's one of the things about working as an auditor that's a little bit different from regular software developer role. When you are a software developer, you're given a task and you kind of do small little steps towards that task and you get those dopamine boosts all the time, right. So every time something just slightly works, you get a dopamine boost and, and it's good that, that what keeps you going. But for auditing it's different. You can go without a dopamine boost for like a week, for two weeks and then you find one thing, right. So you have to kind of find a way to keep yourself excited because it's, it's different from just regular software engineering position. It, it's has some unique kind of aspects to it. But yeah, you're totally right. It's when something works, it's, it's a great feeling. **Speaker A:** Yeah. And so just before we move on from this, like you said that you, you started in crypto by just writing bots on centralized exchanges. I think for all the non programmers out there, like the reason I started on centralized exchanges is because you can like write in like with modern computers as opposed to like the janky evm. But. Well, when you started was this really about like the financial and the games and that piece of it and then you kind of backed your way into understanding like the purpose of crypto and Ethereum or did you come to this from a first Princip? **Speaker B:** Oh, I had no idea like what crypto can do and what like what is possible and how much useful it can be. I was just, I'll be honest, it was purely financial and I'm not, I'm embarrassed to kind of tell what kind of bots I was writing. So it was like, it was very financial driven. I was not there for the tech. But now it's, it's shifted though. **Speaker A:** Yeah, man, I totally understand. And look like at the end of the day it doesn't really matter how you get to the technology. It matters if you stay and why you stay. So who cares? I think you found centralized trading bots because it was fun and you could maybe make some money, maybe lose some money. But you're here now because you hear the call of Ethereum, right? And you see the vision and you want to push it forward. **Speaker B:** Yeah, definitely. Just see the perspective and what can be accomplished. **Speaker A:** Yeah, for sure. So let's, before we get to like audit world, like let's talk about your time at Metamask. So you said you weren't writing smart contracts. What were you doing? **Speaker B:** Well, so if you ever go on Metamask and if you go into a buy button, like buy crypto, so there is this kind of onboarding process that essentially just like offers you the best kind of quote and best route to purchase the crypto for fiat funds. So I was contributing to that routing engine that was finding out the best route for you to buy crypto. So this was purely kind of like logical and a backend kind of job. So it was not anything to do with smart contracts because it's all like Web three, not like web two kind of world where you interact with the service providers that just offer you to buy crypto and you send them real money and they send you crypto to a wallet. So there's not much Web3 going on there. So it was purely back end and that wasn't necessarily what I wanted. This is not what I was going into MetaMask, not the intent that I was going to metamask with. **Speaker A:** Yeah, well, totally makes sense. But given that you had that experience, do you think that like your perspective, like what I love about what you just said is that like while we're all like trying to be gigabrain about like financial derivatives and like all this crazy stuff on chain, like really what we all need to be focused on is like simple things like UX and like getting users on board and just like, you know, the blocking and tackling of like building technology. And so did you find any like, any interesting insights in the, you know, the, the ramp side of this industry. **Speaker B:** I face some of the challenges that they encounter and like, interesting aspects about how quote generation can work. Right? So most of them are very technical. I did realize how much we need web3 and how many. Like, because when you write this engine, right, you're essentially saying in your circumstances, here is the best route, right? And there's so many edge cases to match everyone's circumstances. If you're from that region of the world, you cannot buy crypto because some other old dude there said that they're like now banned country. Or this amount of money is not allowed in this currency, but this one's allowed in the other currency. All of that is nonsense for crypto, right? Like, you want to buy a token, just go buy a token. It's just that easy. But here it was insane how many kind of exceptions you have to think about. And it's a rather unpleasant experience, you know, and you can, you, you almost kind of end up in a situation where you can try buying crypto with one combination of parameters, it wouldn't let you, but then you potentially can just do it with a different one, like a different payment method. I'm like, what the hell? Like, why is the payment method important? But there are reasons, like for the banks to prefer one payment method or the other because they have different trustworthiness. But yeah, those kind of rules, they just made me appreciate web3 a little bit more and how accessible it. It can be. **Speaker A:** Yeah, no, it's funny, I mean, like, I'm going to reframe what you said and tell me if this resonates or not, but it sounds like you entered crypto because you thought it was like fun and had potential but like, didn't really care. And at your time at Metamask, like, while you didn't get to do the work that you actually wanted, that was like the experience that taught you like, well, the why of the technology. **Speaker B:** I think it's mostly like contributing to different crypto protocols that kind of opened my eyes to what can be built. It's almost like I was too small brain to understand what can be ran with solidity on EVM. So it was just like always you open a YouTube and they show you, oh, here's a multi sick wallet that we're going to write today. I'm like, great, what am I going to do with that wallet? How's that useful? What's cool about. I think when I started contributing to protocols, I understood the economy behind it, that those smart contracts actually interact with each other, that they're the Money Legos that if I want to interact with a protocol in some way or another, I don't even need to ask anybody's permission. And this is what kind of blew my mind is because usually you request the API access like keys and stuff, that was another thing. We had to get those API keys permissions and everything from all this. Basically little exchanges to buy crypto with that. But in, in defi, you don't even need to ask anyone. You just find a protocol you want, you want to build on top of aave, you just plug into their contracts and nobody is even going to be able to tell you anything. So understanding that whole economy and how things can get interconnected and you and how you can extend pretty much anything, that what blew my mind, that's what was unique to me and that was something that is just incredible that what makes the growth so exponential in terms of technology that pops up on chain. So that part I think got me into it. And by the time I was writing that MetaMask on ramp code, I was already deeply in love with Web3. So I was just more and more disappointed in Web2. **Speaker A:** Yeah, no, I think we can have like four podcasts on the disappointments of Web2 and then like another 10 podcasts on the disappointments of like the real world, but for the sake of brevity. So, so I think you've kind of like covered this in its entirety over like all your other answers. But I'll just ask you directly like totally understand why you moved on from MetaMask and it was essentially because like you wanted to put your hands in, into the EVM and get on chain and that just wasn't the like what you were doing at MetaMask. But from, from that standpoint like not a lot of people choose to be auditors, right? Like a lot of people choose to go be contributors or to like be founders or to do lots of different things. But how did I guess like now with, with the benefit of hindsight, like what about auditing drew you to it and like why do you find yourself just so passionate about it? **Speaker B:** Well, first of all, I really appreciate smart contracts. To me they're like art, they're short and they're capable and they must be perfect. It's kind of a standard in the industry that a lot of people say oh, you cannot write bug free code and good luck releasing a non bug free smart contract. Right. This is just blows a lot of developers minds that you cannot allow a single bug in production. Right. A lot of developers think in that way where they don't allow any bugs. But for a lot of people the risk is so minuscule. They don't care if there's a tiny bug. Nobody cares here. Even if you have a tiny bug, a lot of people care. That's your reputation. That's essentially bug bounties that you're going to have to pay out. So it has to be perfect and it has to be extremely efficient because of the gas cost. So you're literally not just writing a giant kind of system that whatever, like here I can cut the corner, here I cut the corner. As long as the giant system works, you're writing something extremely efficient and you're spending a lot of time trying to design it. Right. So that is just to me it's a top of the food chain for the developers. It's like some people are going to say AI developers are the top of the food chain. Somebody is going to say backend developers. Everyone has a different opinion. To me, writing smart contracts and auditing smart contracts is the top of the food chain because you need to carve that perfect piece of code. It has to be perfect. And that's essentially just the thrill of money being in the contracts and the fact that it has to be that perfect. One thing that I think that has a lot of beauty to me and that's, that's why I want to do it. **Speaker A:** Yeah, that makes a lot of sense. And I mean, I think like, it sounds to me like you get all of the like benefits of being like a developer but like with this also like sense of like responsibility and this sense of like last line of defense between, you know, like testing and production and like in production there's like real sharks out there that will like rip your face off. **Speaker B:** Yeah, definitely, definitely. And there's also like consensus. Diligence is writing a lot of products as well. A lot of them are very famous, like Visual Studio Developer. Like if somebody ever wrote a smart contract, they probably know about that VS code extension that just does all the highlighting for them. There's fuzzing that we're developing, which is pretty interesting too. It's rather performant. So if you, if you do want to write like a regular product code, you can do that at Diligence. But most of the time we audit obviously. **Speaker A:** Yeah. So let's like zoom in a little bit and like can you at a super, super high level, like especially for all of us that are not actually smart contract devs, but even for smart contract devs, can you talk us through like what what does the process of auditing even look like? Like, if I. If I send you a thousand lines of code right now, where do you start? **Speaker B:** Oh, well, it's rather straightforward. It's like. It's a simple approach, but it's not easy, you know. So when I was just starting, somebody told me that by the time the engagement ends, you have to be able to close the code that you were given and reproduce it pretty much line by line. Right. So you need to know exactly what's going on. So for me, what works is basically either printing it out or putting it on the iPad. I prefer I just put the code in iPad, function by function. That's not how I start, but that's how I essentially, more closer to the middle of the audit, I just go with a pen and pencil and basically look at each function individually, eliminating all the assumptions of the surrounding code, and look at specifically what each function does and what each function I believe should do, do, sort of, and compare my beliefs to what's happening. And then once you kind of look at them in isolation, you don't have any kind of any assumptions about what the code should do. You don't share them with the original developers. Then you start seeing those misalignments of where your beliefs go apart with what the code does. And yet you just go line by line and kind of look at each of them and try to understand each of them. And in the very beginning, you just basically start with the docs, and then you start with general architecture, what the files are, what's the function that's going to be triggered and by whom. Right. If I'm a user, which functions I'm going to interact with. Okay, so that's the entry point, and then everything just starts from there. You know, you just follow which function calls which function. But definitely starting with the docs, then looking over the. Generally the scope. Scope is the list of files that are in question. And then once you understand the general architecture, it's just like you really dive in into each line, each variable. Variable. And like, even if something is super straightforward during the audit, you actually need to look at it. Like, force yourself to look at it because you might feel like it's straightforward and then actually make sure that it's doing what it's supposed to do. Because the bugs are usually very frequently are in a places that people think are very straightforward because they don't think twice about them. And then there's like some weird kind of edge case in there that those are nicest to catch because you know, because usually you would miss them and they're usually like on, they're usually like one liner things but they're really hard to catch and you're like damn, I wonder how many people looked right at it and didn't see it. **Speaker A:** Yeah, I don't know if this is like directly relevant in this solidity case but like again going back to my computer science education, like it's like floating point mismatches would like are literally invisible on the human code side but like will destroy the application. **Speaker B:** Right. **Speaker A:** So speaking to that like how much of your process relies on like your human brain and eyes reading lines of code and how much relies on automated tools that like help just like blast the code with as many different variables as possible and so that you can kind of like see the outputs and then ladder backwards. **Speaker B:** It depends on the engagement time. Right. So usually it's sometimes easy to just run some tools that give you some kind of. You throw a code at them, doesn't matter if it compiles or anything. They do static analysis, they just look at what's written and give you some insight into it and give you some kind of risky places. Sometimes we do that, but most of my time is actually manual review. So I don't necessarily do fuzzing for example on every single engagement because then it's so much time trying to figure out how to deploy the client's code. Right. And a lot of sometimes it doesn't compile which happens rather rarely now. But so trying to actually like set up the whole environment is usually complex and the diligence, the code bases we look at are usually complex. So setting them up will take time. And it's always a risk whether you have the time to actually dedicate to setting up the environment and then like run a fuzzing campaign for example. So usually a fuzzing would be done not in the audit but or as a part of a separate engagement or just by the developers themselves because they have a limited amount of time and they can then rewrite the campaign each time they make any changes. Right. But I prefer just manual kind of line by line approach because then I feel comfortable that I looked at everything, I covered everything myself and then I know where everything is and how everything works just for like if, unless I do that I wouldn't be comfortable delivering because then I'd be nervous. **Speaker A:** Yeah, I mean dude, that's like genuinely surprising for me to hear but amazing, like totally amazing because I, as a non auditor I would just like imagine that even like let's imagine someone wrote a, like a function that rounds down like, you know, just does floor. Like I, I know how to read code, you know, I know the division and then like an edge cases around certain divisions based on like data types and stuff. But I don't know, I guess like the way I don't have the confidence, like I would need to have like machines like just throw everything at it in order to feel confident in myself. And so I don't know if that's the difference between you and me or experience or what, but it's super cool to hear. **Speaker B:** It's a different approach. It's a different approach, right? If you look at the rounding function, right, there's several ways you can go about it, right. If we look at this simple example, right, you can look at it line by line and try to convince yourself that it's secure. You can throw a lot of numbers at it and see if anything violates your assumptions. Or you can try to mathematically prove it, right? And say that, just write a mathematical proof where you say that under no circumstances this can generate an answer. And I feel like for something that does rounding I might be wrong, but like the mathematical approach will be the ultimate one, right? Because it's like a rather, I don't want to say simple, but it's, it's most likely fits the mathematical kind of direction very well. But for something that is a more complex, complex protocol, it becomes more and more difficult to kind of do an exhaustive mathematical proof. You're more like maybe proving a financial model at that point. But you might want to talk about that with people who actually like concentrate on mathematical proving. **Speaker A:** But yeah, no, I, you like you just opened up like a whole can of words that will just like we'll step past. But it's super interesting to think about like the different types of certainty tools that auditors use and like when they're applicable. But we'll keep walking. Something super interesting that you mentioned was that like sometimes you'll get code and it may be more rare now but like it might not compile or it's super hard to compile or it's just super hard to get it like in a like kind of up and going state. Like how much of your job as an auditor is to check to see like if the code even works the way that the developers want, like if it works at all versus just like reading it and looking for bugs and edge cases and that kind of stuff. **Speaker B:** So over time you can get a feel for whether the code is going to be good or bad. And one of the red flags is when the happy path does not work. Right. And by happy path I mean if you're a yield farm, right. Like let's consider something very simple. I should be able to deposit like LP tokens and get rewards, right. If every time I try to deposit LP tokens the contract going to revert, then I know that this has not been tested at all. Right. At all. And even if that path was not tested, what's going to happen when I think about the edge cases, right. What's going to happen when I try to actually break it? Well, usually things go really bad in those kind of cases where people didn't even test their main functionality. And at diligence we usually just say that. Sometimes we have to say that this code we don't recommend for production, it's just not ready. We don't post reports too frequently, we don't publicize them because we give people an option to publicize the report or keep it private. So a lot of pores just like don't ever see the light of day because you just don't want to publish them. But yeah, it's usually like you can tell rather quickly like in the first few days whether this, like whether you would put your money in it or not and whether should others. But a lot of the times this little bucks in a happy path. Much less common that the code doesn't compile. That's something that's I think kind of thing of the past. At least for the clients that we work with. Very common thing is people submitting one code for price estimation and then giving us the other code to audit. That's like majority of the clients like when I see a team that gives a commit hash or like the version of the code for price estimation and gives the same version of the code for auditing. I'm like, I like these guys, they have their shit together, they're good. **Speaker A:** Well let that be known listeners. There's some auditor alpha for you. It is not worth trying to shave off the extra few percent to have your auditor not be on board with you. **Speaker B:** Yeah, it's like give it a little bit of time, like finish your code right? Make sure it's done. I understand people have time constraints and everything, but it is frustrating. You get one code and it's never less code, it's always more code. And that is just a little annoying, a little frustrating. But we understand. We usually unless it's obnoxious, we kind of try to work with the Client and try not to try to accommodate. But it's just like annoying. It's a little annoying. And I personally do very much like teams who can give themselves some time to actually finish the code and just give the code that they wanted to price to the auditors to audit. It means that they have some process in place that works for them that is not rushed, that is more mature and more serious. So that is a nice thing to see. But yeah, also what else we do during the audit, we communicate to the client a lot. So by the time we generate the report, for example, the bugs in the report are not a surprise. As soon as we find them, we tell them and then report is just like an official thing that they can publish or share. But yeah, so. **Speaker A:** So in that vein with that, like I am not asking you for any specific stories to be clear, but like do you ever have situations where you are finding either bugs or like things that need to be addressed and like getting really strong pushback from the client teams? **Speaker B:** Yes, definitely. **Speaker A:** What, what are the types of scenarios in which like that happens? **Speaker B:** Just protocols trying to downplay their issues. And I feel like at diligence we are not putting major on minor bugs. We're really trying to be rather strict to ourselves and what we find. So we never change that. We already kind of side with a client internally. Like we want to make sure that we don't blow up the issues that we find to something disproportionate where it's like a non issue becomes medium or like a medium becomes critical. And with that philosophy, kind of not that difficult to say, guys, we believe what we wrote and we're not going to change that. Some people don't like it, some people just accept it. But it happens, happens frequently. But we just have one policy for that which don't change. **Speaker A:** Yeah. Again, without asking any specific questions or examples. Like have you ever had a time when. Because the way that the consensus works is that they offer the audit report privately and then it's up to the protocol to disclose it or not. Have there ever been situations where you know that there was something that needed to be addressed and wasn't sure if it was addressed and never saw the client report away and you just kind of had to like sit with that uncomfortable knowledge. **Speaker B:** So this is an interesting one because it never happened to me, but I wonder if that happened to my colleagues. So we are lucky where we work with mostly serious protocols so like they have to either fix their issues to show like the report to investors even if they don't publish it. They have to show it to someone. So most of the time, people just fix them. But I haven't yet worked with a protocol where you essentially see a major bug and they refuse to fix it and then refuse to publish the report because then what's the purpose of the audit? They still want to get their code better. What they want usually is to just downplay, like, they fix it, but they want the report to look better. So they're not opposed to fixing issues. They're opposed to seeing that they had an issue. So I haven't been in a situation where the bug that is extremely critical went unfixed. But I wonder if my colleagues were. I should actually ask that. **Speaker A:** Well, and if it didn't happen to your colleagues at Diligence, if it didn't happen at Diligence, it definitely happened at some audit. Audit a firm somewhere. And so which. Which makes me want to talk to you about just, like, the state of auditing in general. Like, how do you feel about where auditing is, like, as an industry, do you think that the way that, like, crypto, Twitter, and, like, the people that are relying on these audits, like, are appropriately understand, like, what the audit is telling you is safe and what's not, or do you think, like, we still have a lot of work to do in just, like, developing this safety layer that you contribute to? **Speaker B:** Let's unpack that question. So, first of all, I don't think people. I think often people misunderstand what the role of an auditor is. And, like, in general, they put too, too much weight on an audit. Well, audit is just one kind of piece of a puzzle. Protocol, security, you can have all the audits in the world and still get hacked. So I don't think it's right to kind of expect a stamp of an audit and just expect that it's safe. **Speaker A:** But also then, yeah, sorry, I did ask too dense of a question, but just picking off what you said there, I think, like, the exact. Exact. Right. A perfect example for this is the. Not the Curve hack, but the Viper hack. Right. Whereas, like, we can talk, like, we can say that Curve had enough audits, they needed more audits. Like, it doesn't matter. It does not matter how many people looked at that code because, like, the issue was in somewhere lower in the stack. And so I think, like, that might be what you're referring to, that, like, auditing is not this, like, stamp of approval of security. It's actually, like, this very specific thing. **Speaker B:** Yeah. And in general, like, that can happen. Right. But even then we, we kind of try to look at what like solidity version or Viper version the protocol uses. So it's usually still considered. But like, I think Euler, I forgot what exactly the, the issue there was. But like they were a serious team that kind of took security seriously. Right. And they had good auditors and several audits, I believe. But it happens. And there's like, like so many little things that people just might oversee once, twice, three times. It's just natural. And especially if the system is very complex, there's so many things that can go wrong and the more you work towards security, the more of those you eliminate. Right. And eventually you can eliminate them all. But it takes time and it takes many, many steps. Not just one step. Right. So that is something that needs to be understood. And I think the kind of problem people have with auditing is the fact that was in high demand and the market kind of dictated the prices to be rather high, at least in the beginning of the defi. So that kind of misalignment with of how much people think they should pay for an audit and what they should get is kind of what makes people think this way, I believe personally. But. But the market dictates what it is basically. But people have a little bit of a high expectation, I guess, from what an audit report offers them in terms of security. **Speaker A:** Well, that was a super interesting insight that you just dropped that at least from your perspective, which I am fully agreeing with that it seems like the market started expecting that these audit reports represented a lot more and that might have been a result of the like supply demand, which resulted in a pricing problem that may still be going on or may be relieved in the. In every bowl. Sorry, in every bear and come back in every bowl. But man, it's just like another incredible example of how like everything really does boil down to economics. **Speaker B:** Yeah. Because I've been on both sides, right. I've been hiring auditors and I've been doing audits. And it is incredible how I can change sides depending on what I need to do. Right. Because I do understand that for developers it's rather nice to be able to iterate quickly. As a startup, you want to iterate fast. You want to learn, fail, build again and get feedback, fail again, get feedback and then succeed. Right. Like you want to iterate as much as you can, but in defi, each iteration is extremely pricey. Each iteration you do is an additional audit and that is frustrating. Right. But then as an auditor, on the auditor side, you know that there's a backlog of protocols that need to be audited. There's a very short list of people who can do the job, especially if it's a defi related where you actually need to understand the like a defi Legos and like those financial pieces. So there's a very few people that do it, relatively few. At least they're looking at the demand. And then also you have to look at what happens if you make a mistake and your reputation gets bashed and you go for a really hard time and people start asking you to pay off their bug bounties and everything. So people want you to share their downside, but they never want to share your. Their upside with you. Right? So it's a very ungrateful kind of position because you're gonna, it's like, I don't want to compare it to like a hockey goalie because hockey goalies do get credit when they like when they do good saves. But it's basically like you don't get praised for doing a good job, but you do get baffed really hard for missing something. It's like you're expected to do a good job, but if you didn't, you're gonna get like destroyed. So nobody shares the upside, only the downside. So I also understand the auditors here. It's a rough. It's gonna be balanced and just like you said, it's gonna be like an economical kind of situation of who is ready to pay what and who is ready to do this job for how much. **Speaker A:** Now you're totally right to like point that out and like I sympathy from the community to you. Like, you're totally right. Like you will only ever eat if something happens and if nothing happens, like people will forget you exist. And like, yeah, depending on whatever your job is, you, you have the same joke, but for your job. It's like you don't know what a bad. Sorry, you don't know what a good IT person looks like. You only know if they were good once they leave. Right? And like that's, that's a auditing world. **Speaker B:** Exactly. It is like that. And there's certain things that you can do, right, to kind of stay relevant as an auditing firm. And like you can release products, educational content and like be praised for contributing to the environment in different ways. But you're not necessarily being loved for just doing your job, which is kind of weird, right? You said you're going to audit, you audit, but that's not necessarily enough all the time. **Speaker A:** And you know, honestly, it makes Me think of like Tradfi accounting firms which do audits, right? Like Enron, like we think of like Arthur Anderson and Enron like evil. They like the, the accounting firm like really up and. But like who's ever been like kpmg, they did it, you know. **Speaker B:** Yeah. It's only when they fuck it up. **Speaker A:** Yeah. So queuing into something you said about how it's, it not only is being a, let's say EVM auditor a specialized skill, but even more specialized than that is being a defi auditor. Like can you just like I, I am sure what you're referring to is that like when you start building with composability and with money Legos, like not only are you having to audit your specific protocol and the lines of code that you wrote, but you, you as an auditor now also need to like, if not do full audits, like really understand the dependencies of all the other protocols. And so can you just talk through a little bit like how, how far out from the protocol that you're auditing do you need to look in order to feel comfortable calling something safe or not? **Speaker B:** Yeah, so a decent amount, right? So first of all you need to understand what, what specific dangers are there on a particular chain that you're deployed to, right? So for example, like on Ethereum you can get front run sandwiched or things like that, right? So that's one thing to consider immediately. So you can't just look at the code and just try to find what logical issues are there. You also need to like kind of look into what environment it runs in. Then you need to understand what kind of assets can be deployed in there, right? Because some tokens are basing some of reentrant. So you need to understand like what are actual like risks based on what assets are going to be held there. Then you see like what kind of information the protocol gives out to other protocols, right. If there's some price view functions, those can be manipulated potentially. And if they can get manipulated, who is going to be reading those price feeds and who can get hurt by that? And then another thing that's really important is that you usually have a limited amount of time to audit a project. And what helps is knowing the common defi primitives, right? Like you need to know for example how staking usually works, how liquid staking derivatives work. You need to know what a typical AMM looks like or a typical farm looks like, how some options work it. If you don't, you can still audit, right? But you're going to spend much more Time trying to figure out what they do instead of actually how they do it and what could go wrong. So this knowledge definitely helps. So it helps keeping up with different protocols and like what kind of novel models they're introducing because it just saves you time during the audit significantly. But yeah, you do need to think like what are different protocols they can interact with, with the code that you're looking at, what assets are stored there, where it's deployed. So you go a decent amount. **Speaker A:** So let's just like without like we'll pick an example and one that I'm pretty sure you're familiar with, which is Umami Finance. Right. Their, their vault product. Let's say that they came to you and were like, ty, we need an audit. So for those, for those that aren't aware, Umami has a vault product that is essentially providing liquidity to GMX and then it's using. So it's doing proprietary stuff in its own protocol while providing liquidity to GMX and then also has a side protocol that's external but it's providing insurance in case of like a DPEG or that kind of thing. And then on top of that, this whole structure is on Arbitrum, which is a roll up on Ethereum, like with all those like different like are you looking into the code of gmx? **Speaker B:** Oh, absolutely, yeah. You definitely have to at that point when it's so deeply integrated, they might be utilizing like a function ABC of gmx. Right. But what if there's function like F that once you trigger it, something messes up in the functions that Umami would be using. So at this point you kind of have to look at GMX very deeply actually. **Speaker A:** Do you need to look at the like let's say an insurance protocol, everything. **Speaker B:** Whatever they put in the scope, you must look at. **Speaker A:** What about Arbitrum? **Speaker B:** Arbitrum probably not. You need to know like how Arbitrum operates, what kind of interesting things can be done on Arbitrum and which ones cannot. But you don't need to look at Arbitrum code specifically, but GMX for sure, usually it's just everything that the protocol wants you to look at. The file and anything that they're touching to a certain extent you have to look at. So if they're touching GMX then and this, this much, you, you kind of have to very well understand what happens behind the scenes where when Umami interacts with gmx. **Speaker A:** Got it. Yeah, that makes sense. And yeah, it sounds like you basically need to go as you need to touch, you need to look at everything within the EVM that you're audited, the company or sorry the protocol that you're auditing is touching. You need to look at everything and then you might, but probably don't need to look at the infrastructure that it is built on top of. **Speaker B:** Right. And usually you still don't look at GMX the same depth you're going to be looking at Umami code. Obviously you just like insert certain isolated things that are, that Umami would be interacting with, for example. **Speaker A:** Yeah, got it. So a super interesting thing you said was in reference to like when you're working on Ethereum Mainnet, like, like you need to worry about front running sandwich attacks, like that kind of stuff. And so like it didn't even really occur to me that not only like auditing is of course about like finding vulnerabilities, but it's not in like the static world that most computer science in. It's in this like very alive dynamic and like this MEV world. And so I guess my question to you is like, like how much do you think about the true MEV research and the block building and what's going on in there to inform the behaviors that are going to appear for the protocols that you care about? **Speaker B:** So it's interesting that you bring it up because on Chain there's much less kind of activity where it's typically called race conditions or something like that. So I feel like it's actually counterintuitive. But I feel like in a traditional hacking there's way more of those kind of situations where what if I call this function while I calling this function and then I just. This one finishes a millisecond faster and there's so much more you can do. But on Ethereum, no matter what, everything is executed sequentially. There's no multi thread. So it kind of eliminates so much complexity, like so much complexity. So when you think about it this way as a single thread, like front running and just like sandwiching is this like kind of not that difficult kind of race conditions that you can exploit. So you do think about it a lot, but once you think about it for some time it kind of comes naturally. You kind of start thinking about like every function you look at it, you're like what if I called something else before that? Or like what if, what if I can call a function in different order than they were like designed to be cold. So typically on chain thinking about those kind of scenarios is not too difficult because you don't need to worry about having multiple instances of something running or having two functions called somehow at the same time and trying to see which one executes first. It's kind of similar, but it's usually much more simple on chain, so which I love the fact that it runs kind of on a single thread. It just makes my life so much easier. And I think everyone's lives. **Speaker A:** Well, I mean threading is like obvious now that you say it. But the other one too is that in traditional hacking a DDoS attack is like literally free if you own the computers. Right. Whereas everything within the EVM costs the attacker money. And so you. **Speaker B:** Oh, you'd be surprised. DOS attacks on smart contracts are not always that difficult. **Speaker A:** So what is it? Go on. **Speaker B:** There's oftentimes like people just don't think that you can call a function with a certain parameters and then you call like five times and then you break access to someone accessing that contract. It's often where like one user can do something on behalf of the other user, where the other user is just going to break their file or something. So those kind of bugs, we don't talk about them too much because hackers don't necessarily incentivize to do those because they don't get any benefit, but they can just ruin somebody else's life. Right. And those bugs happen more often than not because people just don't. Developers don't think about them. And it's like, oh, you call this function with this address, but with that value and that user just loses access to a protocol or some shit. So those happen a lot actually. And it's kind of denial of surface. It's just not distributed denial of service, but kind of denial of service that is not distributed is actually very common. It happens a lot. **Speaker A:** Yeah, well, honestly that makes me pretty nervous just because like we're so, I, I do believe we're so early and like, I think that like right now we can't think of reasons that people would be incentive incentivized just to come to cause that kind of chaos. But like, holy, our world is like just people like setting fires for the fun of it all over the place. Like that makes me nervous to know that that's a lurking problem here. **Speaker B:** Yeah, I forgot what it was, but that was like one of the. I think it's one of the most famous bugs that every, every kind of auditor gets introduced to auditing with. It's like that. I forgot what it was, but it was a multi sig wallet where somebody just posted like, oh, I accidentally killed it. A lot of Money was lost and I forgot the name. It literally flew out of my head. Let me just Google it. It's so famous. **Speaker A:** Wait, is this the parody? **Speaker B:** Yeah, yeah, yeah, exactly. The guy clearly knew what he was doing. It's not simple experiment that he ran Oops, I accidentally killed it. Okay. He destroyed the implementation of the proxy, and then it's like. Like nobody could use their wallets anymore and all the funds got stuck. Right. He didn't profit from it at all, but he just decided to run it. And I'm like, I don't buy that you accidentally killed it. I feel like you just couldn't could control your genuine interest. It's like the Adam and Eve and that apple, but there at least there was an apple and this guy just like, it's just to mess. Mess with something. Yeah, yeah. **Speaker A:** I. Yeah. I mean, the only reason I even know about that story is because I read. I. I entered it through the drama of, like, hearing about people trying to push through Ethereum, like level consensus, like a change to the parody wallet, like, contract specifically to get the eth out. And like, I. Yeah, that's a wild story. I. It is, it is. One day we just got to do like. Like, actually, you know what, Ty? Like, I will, like, we'll coordinate, but, like, it would be really fun to just do like a history of like, the most crazy, ridiculous things that are like, hacks or just like crazy stories like this. **Speaker B:** But yeah, that would be cool. I think that one is one of the most famous one. It's because it's hilarious. The guy just like, breaked a lot of money and just like, oops, I accidentally killed it. It's hilarious. **Speaker A:** It was sad too, but yeah, definitely sad because, like, a lot of like, real people, like, that was their money that got bricked. But I don't know, I guess it's just more burned. So. Yay. Like, I don't know. All right, well, anyway, that, like, I. I don't want to take up your whole day here, so. Ty, I just, like, really appreciate your. Your thoughts. And like, I just learned so much about like, like, what auditing actually is and how it works. And man, you just. You have so much to teach us and to teach me about, like, the actual last step before deploying things to Ethereum. You know, it's so easy to talk about, like, big visions and like, to actually just like, hack things together. But, like, what you guys do and what you do is just like, make sure that we're actually building infrastructure and not just building like a carnival that's going to like fall apart and kill a bunch of people. **Speaker B:** Yeah, definitely. And it's a pleasure to talk to your ex all the time. You ask very wonderful questions and you keep it going very smoothly. So I appreciate that a lot. **Speaker A:** Thank you. Well, before I let you go any, like, first of all, where can the people find you? And then anything you want to just shout out or let the people know, like how do they get more involved? **Speaker B:** Yeah. So essentially they can always reach out to me on Twitter if there's any anyone who is trying to get an audit from consensus diligence. You can also reach out to me on Twitter and then I'll forward you to the people who do the scheduling. So that shouldn't be a problem. But yeah, just I hope people treat their smart contract seriously and also believe that it has to be a perfection that is completely bug free and then our lives are going to be a little easier, a little more stress free. **Speaker A:** Yeah, for sure. No. And yeah, stay tuned. Like make sure you subscribe to this podcast because like coming shortly, I hope it will be that our next episode where not only will we do like hilarious or sad war stories, but we will do tips and tricks to make your auditor like you. **Speaker B:** Yeah, that's. Actually there are quite a few of them or hate you, depending on what you want. **Speaker A:** Yeah. All right, man, Again, this is such a pleasure and I really appreciate it and talk to you soon. **Speaker B:** Yep. Thanks, Rex. Bye Bye.