Why Traditional Data Platforms Fail at Scale - A Quantum Grade Rethink with Arcitecta | Episode #102

Why Traditional Data Platforms Fail at Scale - A Quantum Grade Rethink with Arcitecta | GTwGT #102 - YouTube
https://www.youtube.com/watch?v=jjMFEPffNmo

Transcript:
(00:00) Most data platforms weren't built for the world we live in. They bolt on metadata, struggle past ped scale, and fall apart under cyber attack. Architecture says that's broken by design. In this episode, I talked to Jason Lori, CTO and founder and also just quietly a quantum physicist and the software sculptor behind Media Flux to explore how rethinking data from the ground up might just be the only way forward.
(00:31) So, what if your data platform could manage billions of files, scale to hundreds of pabytes, defend against ransomware, and more? This is episode 102 of Great Things with Great Tech with Jason Architecture. Hey Jason, welcome to episode 102 of Great Things with Great Tech. It's great to have you on and talk about Artekia and just talk about this company doing great things in data which probably people haven't really heard about but is doing so well in the industry.
(01:06) Before we get into what the company is and what they do and everything that you've created over the past, I guess 20 to 25 years, let's get a bit of background yourself. You're a quantum physics sort of guy, which always is intimidating. Um, but give us a bit of your background. You're from Melbourne, Australia, so great to have someone local. But again, welcome to the show and let's hear about yourself. Thanks, Anthony. Great to be here.
(01:26) Um, yeah, I uh I got my quantum physics degree at the University of Tasmanian. So, I'm actually Tasmanian. I moved to to Melbourne. Okay, we won't hold that against you. I love Tas 1980s, early 1990s. Um, uh, but don't be, uh, intimidated by quantum physics. I actually failed maths at uni, first and second year, and had to repeat.
(01:48) Uh, and so to sort of force my way through, I ended up with computers. Yeah. Sorry. Can I say I'm the same? So, we're kind of equal in that sense is that I I failed I got through the first semester, failed the second um during computer science degree and but it was back then it was all mixed together with the engineers and everything in curtain university. So, it was pretty difficult. So, I'm I'm feeling you there in terms of the maths. Oh, yes.
(02:12) Nonlinear differential equations that actually pretty hard. Um but I pushed through and ended up with those as I said a physics degree and at the end of this I thought uh where will I go and I could have gone and done masters or PhD and I would have done radio astronomy uh and I was interested in going out into the world of computing which is actually relatively easy also at the University of Tasmania because I'd failed I had a bit of extra time up my up my sleeve so I enrolled in uh painting and video in fine arts. So I've got a fine arts background as well. And that's sort of the two hemispheres of
(02:47) the brain, the creative side and the methodical side, the science and creativity. I'm very interested in the intersection of those uh those two, which is why the company embodies both creativity and uh the rigor of uh computing and maths as well.
(03:09) So when I stood there and actually I got I went and first started working with an alen smelter in Kamalco in Tasmania. So I had the most awesome boss to start with. He said, "I don't want you to make anything uh or do anything to start with. I want you to just play around." So for three months, I did art with the dot matrix printer. So making the pins go out and do this.
(03:29) But thereafter actually, I immediately got involved in the re process re-engineering for the smelting aluminium. We built systems with embedded control systems. ly changed the overall uh algorithms to reduce the energy consumption in the plant. So you've gone from playing to involved in a really critical uh project and it's that that really intrigues me how you get people to play and then get them to a point that they can go and do be involved in something very serious and I use that methodology for all of our interns that come into the company.
(03:58) Yeah. Ultimately though, I really wanted to be involved in motion picture and film, which is why I moved to Melbourne and worked for the Kodak team at uh Cineon. The Cineon team at Kodak, sorry. Uh and the objective here was to keep Kodak alive for 20 years because in the late 1980s, a number of people realized that Kodak wouldn't survive for more than 20 years, but you're going to scan analog film, silver halid, scan it into a digital form that you can then manipulate. that was the basis of the uh visual effects industry that we now
(04:29) know. So I really enjoyed that. Uh then I went overseas worked for discrete logic in Canada and then came back and set decided to set my own company up. Okay. And I can see I can see you know architectctor is based in that right in terms of the name your art and all the technology.
(04:48) So the the company names I don't need to sort of probe too deeply to find out you know how that came about. It's still it's still a really cool name by the way. But obviously it's it's it's basically all you, isn't it? It's your journey up to this point. Well, actually not not entirely. So when the company started, I actually did this with my wife as a contemporary dance choreographer.
(05:08) So we both actually were combining the idea of the architecture of computing, but the architecture of space. So choreography is all about form and function and and moving bodies and others. But I look at this in terms of choreographing software into an into a form. And so that's really what where the name architectctor came from was the conjunction of interests there.
(05:28) But that that whole philosophy is still continued for so that when the it was started as a consulting company in 1998 in 2002 it became a product company. So that started because uh my wife was invited to Arthur Boyd's place in Bundon New South Wales. Arthur Boyd a very Australian uh a famous Australian painter.
(05:49) uh she was invited for a residency and I put my hand up and asked could I go and I'd never had someone other than an artist actually asked to go on a residency. So I to I sat in the studio there for two weeks with a plain piece of paper. I took my Silicon Graphics O2, wheeled it in there, carried it in there uh and thought, "What will I make?" And and the the ability to have a clean sheet of paper to sit there with nothing in front of you and say, "What will I make?" is an most awesome place to be at. I try and encourage people to think like that. And so I thought I'm going to make something with data because I'd
(06:26) been very interested in that ever since I started work working in the late 80s early 90s to build systems that were managed data. Mhm. So in terms of that transition, so you're out there um Arthur Boyd as well like it's it's obviously a lot of Australians would know who that is through schooling and whatnot. Um but very famous.
(06:49) So it's it's really a place there where you can almost are you finding yourself effectively when you go to a place like that is and then you mentioned something about the plain sheet of paper and you know that's that's kind of dawning for majority of people right to sit there and to see something that's that's that's nothing and then try to make something out of it.
(07:08) But I guess from your perspective it's a it's a good way to challenge yourself to see like who you can become effectively. And you know now 20 25 years later obviously you you've got this quite successful company but yeah I um I give a lecture once a year to students at university and it's on the art of software construction and the thing that I actually get them to think about is if you could make anything and the thing beauty of software is you can make anything that you can conceive of there is nothing that you can't represent every abstraction you can represent. So, you're only limited by your imagination. And your imagination
(07:37) starts with being bored, staring at the ceiling, watching the clouds. And it starts with a piece of paper. And it starts with sort of stimulation, creative stimulation. So, I uh so that's how this started in the first place. And I didn't know where we'd end up.
(07:57) Uh everybody thought I had rocks in my head kicking this process off because for the first six years, I worked at home by myself. and I made a product uh that I knew we needed it the market wouldn't really intersect with where uh I was sort of heading for 20 years. I did completely the inverse. I built a product and waited for the customers to come.
(08:22) That's so different to that's so different to normally I ask a question to show what was the problem you're trying to solve and clearly you were trying to solve a problem but typically that's that's generated from something that a customer wants or sometimes people have been internal to a company they're developing something and they see something that other people haven't done and they try and sort of split it out and that becomes an idea and germinates into a company but you've kind of taken that it's the problem statement is there within yourself but you're not quite sure where it's going
(08:46) to lead That's that's quite unique in itself. Yeah, that's in retrospect that might seem daunting, but it's also interesting. So, what was the problem? Well, it's there's a few seeds to this. Uh, in 1986, I deleted 80,000 words of my aunt's manuscript.
(09:10) Why on earth I went up to the floppy drive and went dear and everything was gone? How old were you? How old were you back then? Oh, it's at uni still. Okay. Yep. Yep. Yep. It's at this point in blood drains. You should have known. You should have kn You should know that. You should have known, but it's should know. And so we tried to forensically reconstruct the the contents of that floppy drive. It's actually really hard.
(09:28) Uh yeah, a lot of file systems you can recover the data these days, but not there. So I thought, how is this possible? This shouldn't be right. And the other thing that happened is that dur during the 1990s you could see or I could see that data was going to underpin everything the humans did.
(09:48) It was going to be a very important currency of future innovation and that's transpired in the early 2000s. We saw the rise of the chief data officer uh people studying data science at university. We saw the words metadata uh in common lexicon. Yep. And uh and the rise of big data. So uh I wanted to build a platform that would manage data doesn't doesn't matter whether it's structured or unstructured.
(10:12) So your traditional approach would be you would have your database uh generally relational database and you would store your structured data there and then your unstructured stuff your video your documents your sensor acquisitions etc images they all go somewhere else but they're in two distinct systems that are not really connected with each other and so you have to manage them separately.
(10:36) So I think why is this it's all just data you want to actually combine the two of these into the one platform. So the vision was to create a platform that enabled uh the uh the storing of all forms of data and the management of that holistically and to be a protocol engine. So in one protocol out another to be able to have that data live for much longer than the technologies that it was the world which was being used.
(10:59) Can I say that that's actually really that's super forward thinking for that time because I mean the what the problem that exists still today in in droves is data gravity you know in terms of where the data lives but then silo data is still huge there's still silos of data everywhere in this world so to think about this 20 years ago plus where you want something to to be all together no matter what it is and the protocols and and whatnot it's it's that's a really forward thinking um statement to try and solve back in the early 2000s.
(11:30) Yes. Uh basically just like writing software and uh complexity is interesting and if you build things that are complex, you actually give yourself a chance to be not knocked off by someone else. So if I build something very simple, I might lose my market share because someone else creates something which is also simple but knocks me off and has but but if you build something that's complex and solves a large problem, the chances of that happening are fairly remote.
(11:54) And that's and that's how it sort of transpired. I always wanted to uh build things from fundamentals. So the other thing about our company is we we're first principles maker. We make every bit of software ourselves. People thought we were nuts to have actually built our own SMB protocol uh from the ground up, all our own NFS, all our own DICOM, all our we built our own database engine.
(12:17) Every bit is written by us. And it's come to pass that there are a couple of really good advantages to that. One is you understand how every bit of the technology works. So you can say ah imagine this could happen so I make it happen. So we have virtual nodes in the file system.
(12:35) We can restructure the file system based on metadata without moving any of the data around. They're things you can't do in a regular file system. Uh but you can also put things like multiffactor in the data path to prevent you accessing data if you don't have if you don't have access. So I JSON access a file system via NFS.
(12:54) I might have to assert my identity in order to continue with my access or or or delete. The other thing that happens is you reduce the attack surface of your platform. So there was a move in the early 2000s I think from memory towards microservices. So everything's distributed uh and also to lots of open-source software.
(13:13) I had to push against the world of open source software. Absolutely. Yeah. That that that increases the attack surface. Yeah. And again and a lot lots of software that's created today is by taking bits of other people's software the open source parts putting it together putting wrappers around it and effectively then you know licensing the open source within that.
(13:35) Um so the ideas are not unique um to my end did you feel like you you had more control in in that principle that you were actually from the ground up creating these SMB protocols and NFS protocols rather than bringing someone else's in. Did you feel like end to end this was part of the flow? I'm get I'm getting that sense right that this is more it's about control.
(13:53) Uh I spent a lot of time arguing with people about what the art of the possible with was art of the possible was with c companies that I worked in. I thought this is really silly. I don't want to argue with people. I just want to do. So if you uh if you make stuff from the ground up you are totally in control.
(14:12) But also if there's a if you have a a security flaw or there's a glitch you can fix that in hours and deploy it. If you depend on somebody else, you you don't have that level of SLA that you can adhere to. NIST has 26,000 CVEEs on its active CVES on its list. The software we have in the world is riddled like Swiss cheese with false. Uh and so I and so if I that gets propagated that that propagates, right? Yeah.
(14:39) Because everyone's using the same thing and then all of a sudden one CVA comes out and millions of systems are infected with it. Yeah. So if you depend on other systems. So for example, there was a hack recently that nearly got into the main Linux distributions for uh for uh SSH demon the X utils hack. If that had got in that's an absolute disaster. So that is the only thing that we would currently rely on. We've implemented half the SSH stack.
(15:04) But my end objective is to implement the entire SSH stack. So we have no depend we're the only thing that presents as a protocol at the boundary of the machine so so that we don't rely on anybody else other than ourselves if you do that you're completely in control but then you can do things that are creative so for example uh we have a customer that asked us to manage six billion files that is aggregating lots thousands of filers from a very well-known storage company into a single system that's going out to tape and then they 18
(15:33) months in they said oh Jason we've made a bit of a mistake There's actually 140 billion files. That's two orders of magnitude higher. Yeah. Uh and could you then uh uh deal with a trillion files? And I said, "Okay, uh we write our own database engine. Our database is 1 kilobyte per file on average.
(15:56) So if you got a trillion files, it's a one pabyte database." And they said, "Oh, no. We can't have that. Can't have a one pabyte dab." I said, "Well, you've got one pabyte or more of metadata in your iodes distributed around your file system. uh why why can't we do it? We can't have it. Okay, fair enough.
(16:17) So in that process, we actually redesigned a file system for them that has on average 75 bytes perode that is completely traversible. So you look at a standard file system, it's 1 kilobyte up to 8K perode. So the only reason we could do that is because we know all the bits that we can reassemble and make. And we've just released a product this year at the NAB, National Association of Broadcasts in Vegas for real-time streaming of data across the globe. Everybody that builds that, they build it at the protocol level, at the codec level, so video streaming across.
(16:44) And I went, you do that? Why don't you just have a file system that real time replicates? And I thought, why hasn't anybody else built this? So, we made it. And it's an absolute no-brainer in retrospect. But the reason we could do that is because we're used to making stuff from the ground up. Yeah.
(17:02) Yeah. And I think that's your core principles as and it must be said like if you go to your website it's very I guess ky is the word right in terms of in terms of what you're doing it's purposeful right it's it's got lots of art in there it's got lots of flows even your YouTube channel you go there and it's you say okay it says what is architectctor and then you see a video that's 30 seconds long it's got some moving bits and like some art it's like okay you got to dig a little bit more there but core principles of of software the sculptures so software sculptures first principle thinking like you said but also pedib
(17:30) Wranglers kind of described there as well in terms of enormous scale which is so if you buy a storage system traditionally anyone really would say that you know at some point it's going to tip over at some point it's going to fail right it's only if you buy 100 I'll say gigabytes or we're going to use gigs a very small number you're only going to be able to get 80 gigs of that and the last 20 is wasted because then the system falls over.
(18:00) So how does this and we haven't really talked about the platform media, right? So that's what I'm trying to get into here. So MediaFlux is the is the name of the platform of of the product effectively. Yes. Yeah. It's a software platform for managing data. It's essentially a database uh and a file system and an API and an orchestration engine integrated.
(18:20) So it's workflow the like. So um it's uh it runs on any platform. It's software. We also sell it as uh on hardware appliances that run on premise in the cloud or a combination of those. So you can move data between cloud and on premise and it's a distributed system. That's the other thing that interested me is building distributed systems because I we for a job I won't name the place at a distributed systems place and they said oh you need to actually what algorithm would you use to do this particular data structure? And I went oh I'd look it up in this book. they went nah gone you're gone that distributed computing uh
(18:58) entity no longer exists so we now have an all singing all dancing distributed comput system so that's sort of driven for that but data is uh is not you have to actually it's not always in the right place so for example I might acquire data in Europe somewhere and I need to operate on it but I need to have a single pane of glass that knows where that data is and I might need to take my compute to the data or I might bring the data to the compute. So we've got uh protocols that allow us to take your computation
(19:28) to where the data sits. That's part of the the the platform platform has its own database at the accord. So technology which we built, we call it XODB. So XML object database. So it's binary XML compressed. It supports uh objects uh geospatial time series and now vector all in the same thing.
(19:54) So you could you could say go and find me all of the images of uh of airplanes that were in this region in space at this point in time that have a good line of sight of the window in this building and you build complex uh queries. So geospatial we're also a geospatial engine and exporting uh to and from different uh formats and we're a protocol engine.
(20:19) We support NFS, SMB, SFTP, DICOM, S3, IoT, all these things because you don't know what vector you're going to come in. Uh S3 did not exist until 2006. Prior to that, it didn't exist. So that's recent history. What's going to exist in 2035? We don't know another protocol or more protocols, but the data often outlives that.
(20:38) We've got data that's been around for 100 years or more. We need to keep it. Y. So our platform is about being able to evolve, being able to connect people to be able to uh feed the systems that do patent analysis. We're not an AI engine, but we need to feed the AI systems. Uh look at it. Yeah. That they and we need to acquire stuff from lots of disparate systems and you need to know what you have and what the quality of that data is and that you need to make sure that what you store you can get out reliably at some point in the future. So we're a data platform.
(21:09) So being being that said so multif media flux sits on the top it's got its database the o xodb uh and then you're ingesting other storage systems or is it so where's that data actually sitting so if I had a traditional say netapp filer and I had a vnx from and then I was using s3 as an example okay what what would be the use case there okay so what we we've just a migration for this in Western Australia, University of Western Australia. Uh, and they've got their NetApp files. So, we just scan them uh and then we repro as
(21:50) uh through those protocols. So, you used to go to the directly to NetApp now go directly to us and then we'll talk to NetApp and in the background we may well be migrating data uh to another platform. So, when you come into our ecosystem, it's the last migration you'll ever have to do. So, so if you buy uh uh 10 pabytes of vendor A and then you replace that after year five and you've now got 15 pabytes of ver of vendor B, you need to migrate 10 pabytes and then in five years you got to migrate 25 pabytes and then it gets harder and harder and the mass of that increases.
(22:24) With us in Vand you just plug your new storage in and you start using it and in the background we're doing this migration. So your downtime for adding a new storage is zero. And that storage can be anything or it's it's a set it's a set um specification anything that appears to be it appears as PZIX NFS SMB.
(22:48) So we have our own native NFS clients our own native SMB clients as well as B servers for those anything that presents as S3. Uh and we've also got custom protocols. So Vers Scout AM uh Spectralics DS3 protocol brow datas extreme store uh IBM um uh Diamondback we've got all of these additions that we have. So S3 alone is not enough to drive tape.
(23:15) You need to know what tape barcodes uh data is on what takes the things are on what the barcodes of those are. You need to know where the placement information is. So we need extensions to those. We drive those. So doesn't really matter to us and it doesn't matter whether it's in the cloud or on premise or you're aggregating this into one virtual storage space.
(23:33) Understand that. So let's talk a little bit about um the state of data and where we're in because obviously this is your world. Um you know we talked a little bit about how data is growing more than exponentially, right? And you've got an interesting theory about you know where data is going to be and and how are we going to be able to store the data that the world is looking to store in the next 10 to 20 years. Yeah. Well, there's there I was asked a question recently.
(23:58) How how what's going to happen when we get 10^ the 26 or 10 to the 27 bytes of data. That's a huge number. I can't even remember what whether that's bytes or something else thereafter. I don't it's mindboggling. Um the problem is we can't keep storing all of this data. If you look at data itself, it takes energy to store it.
(24:17) Energy is mass. Mass is energy. Thanks to Einstein's equation. Um, there's only so much energy that we've got to store more and more data. We can't represent every thing in the universe on Earth as a piece of data. So, we're going to have to make decisions about what we throw away, what's important to us. And as humans, we love acquiring stuff.
(24:41) We've all got atticts. We've all got hoarders. You've got your storage things uh places that you put stuff. We can't do that indefinitely. All we have to increase the density in which information is stored and we're seeing that with tape. Uh and I think the upper band for tape is about 500 terabytes per per tape. Is that right? It's a huge number. Half a pabyte.
(25:07) More than more than I used to more than I used to back up to back in the early 2000s. That's for sure. And and but if you think about a lot of eggs in one basket. So if that tape fails, you've now lost half a pabyte of data. uh but we're also seeing more densely packed uh uh flash and other but that consumes power.
(25:29) So if you go and stand in front of a rack of uh of flash there's a lot of heat coming out of there. Mhm. So we need to find ways of reducing our energy consumption. But so essentially we can't even then we can't store everything that's in the universe. We have to cut down. So how do we discard? Well we need to be better informed about what we've got. We generally don't need everything.
(25:49) A human cannot go through an infinitely increasing set of data and make sense of it. That's why we've got systems uh comput systems and AI to make us help make sense of that. But if we've got it really worked out, we probably need better models and fewer equations that describe everything. In physics, there are four equations that describe the universe.
(26:09) We don't yet have a grand unified theory of physics, but there must be one somewhere. We just haven't found it. So that's we're seeking that and and it's similar with data too. So we we've got too much. We're going to have to cut down and but could that lead to different ways to store the data more? I always think back to Superman.
(26:30) I always think back to the crystals, right? And to a certain extent that that there was data on those crystals, right? And I think Microsoft a couple of years ago went through some sort of PC storing data and DNA and all sorts of crazy stuff, right? So if we can solve the problem of it taking up matter which there's only a finite amount and the energy that it creates and how we supply that um is that is that possible then to store more data in that sense you can get more data stored but there's still a limit to that so if you represented every atom in the universe with a piece of data and you have to
(27:02) have one for every atom so you need models of those uh interesting the interesting piece of technology I see uh there's DNA storage. One that's more interesting to me is glass. So Microsoft have their project silica which was store on glass. So met with those guys. And the other one that's really interesting to me is Sarapot in Germany.
(27:26) So they're looking actually one of the founders of that company is a ceramics artist. And he said oh I I know what uniforms are. Why don't people store data on with uniforms or ceramics roll forwards? They're now uh creating storage using ceramic in glass and fto lasers and the industry or customers are looking at this and egging them on because this is awesome.
(27:54) 5,000 years EMP resistance uh on glass and huge densities going forwards if they get that technology right. What will be after that I don't know but I think that's the next most interesting uh bit of storage technology to come. Yeah. And so to that end like moving through when you go to a customer what's unique about you guys and how you approach it is that you know you ask people you know what data do you have and what do you want to do with it so explain explain that methodology because I think to a certain extent most storage companies vendors will just basically tell you what to do with it
(28:23) and say hey you got data put it here end of story we're the best for that so how does your approach differ and how does that make your customers you know adhere to you We're very sticky with our customers. So basically it starts with a conversation. We're all human beings. I have a problem to solve.
(28:42) What is the problem? And uh how do I need to solve it? And our field is around that layer of data not the actual technology that uh necessarily whether you've got discs or cloud or whatever it doesn't really matter. We're in the layer above that. Uh and so the the conversation you want to ask open with and ask is what data do you have and what do you want to do with it? And that really generally leads to a um a conversation around oh I had this I acquired this from here I have this workflow that happens here I need to govern my data around this. It's a
(29:11) conversation around the data not around the technology and at this point you're talking to a human uh around processes that they understand. And why I have no issue with that is that we've got a piece of technology which is a chameleon that can adapt to different scenarios.
(29:31) I would never have asked that 20 years ago, but I'm happy to ask that today. And you listen and you say, "Okay, well, you need to read, you need to fold the paper in this form of origami that produces this outcome." So, we're not there to sell something that we've got. We're there to sell an outcome or a solution.
(29:51) And so either we have that uh in our kit bag over 20 years of development. We've got a lot of stuff or we go oh that's interesting idea because we're first principles maker. Next week we'll come out with this evolved bit of kit that does something that we haven't done before. And of course once we've done that we can do the same for everybody else. So I want a conversation with the people.
(30:10) Interesting. Yeah. That's right. And I think that then I think personalizes the relationship from the start. And again that outcomebased approach is something that I think's obviously taken hold over the past sort of 5 years. I've seen it in technology where we used to we used to talk about bells and whistles and knobs and dials but over the last 5 years you've kind of gone up the stack a little bit and now where else it was just the CEO that might have wanted the outcome or the CFO now it's actually going down into the CIO the CTO the the
(30:39) data admin the app admin. on outcomes as opposed to seeing what you knobs you can twist. So it's kind of an it's a good approach and I think from a data perspective a lot of people just don't know exactly number one you you've touched on the points of the growing data data has gravity silo data the fact that do you need everything that you're storing do you know where it's accessed and now we'll pivot in nicely into the security of that data right and I think interestingly enough me working in the backup space and um I know you you
(31:10) partner with a couple of our partners with Wasabi I think we got a couple of joint customers as well from a theme perspective, but I I've got a feeling or a thought that's a bit controversial from my perspective is that does a backup company need to deal at that level where we're becoming security and trying to prevent something from happening and as you pointed out the data is growing so significantly that it's almost impossible to manage at that secondary layer.
(31:39) It has to be on the device on the platform in real time. So what what are your thoughts around that and how are you guys working to try and solve that problem? Because you can't back up 100 pabytes it just doesn't work. Oh you to back up 100 pabytes your traditional backup environment would be full incremental incremental incremental incremental full etc over some interval and so the just time to scan that you're not going to get through it. no system.
(32:04) The backup systems are not big data systems that are going to move things at terabytes per hour or hundreds of terabytes per hour to back these up. It's just not it's just not feasible. And we we've entered the age where uh people used to have they used to have megabytes, but you know, you think half a 500 terabytes is a lot.
(32:23) No, no, a pabyte now is a minimum sort of entry point for a lot of people. And you're in the sort of tens and then you've got those increasingly into the hundreds of pabytes. And if if you're not there already, you're going to get there. So for backup you can't use these traditional mechanisms. You have to be in the data path.
(32:40) So what we do here is that every time we detect a change because we've written all the protocols uh we know when something's changed we can instigate an immediate uh backup of that. So our point in time capability records every structural change in the file system. Everyone creation every rename every deletion all the deletions are soft uh every every significant write that you do.
(33:04) And then because we know all of that, you can go back and uh so sorry that firstly gives us an RPO of near zero because the moment something's changed is immediately backed up. So a sort of normal backup system is going to be I don't know once every few hours, once a day. Your RPO is defined by how often it runs. And if you've got snapshotting in your file system, that's not going to run every second.
(33:27) So if I'm in a system that I had a file and I deleted it five minutes ago, I've got to actually go be and find it. So we do all of that. Yeah, you do have CD. So there is a concept of continuous data protection. Um asynchronous synchronous sandbased replication from our point of view, you know, you'd go into a a virtualization platform.
(33:45) You'd intercept the IO's and you'd kind of write those to a different platform and that would be your low RPOS. But even in that sense, we're not going to get near zero, right? It's it's seconds maybe, but then there's the overhead is huge. And then that's a secondary that's that's a secondary overhead because not only is it pressure on file systems and platforms, but you're also putting pressure on the backup and the network and the storage.
(34:04) You're doubling it up. So there's and so what I'm thinking through your answer there was you can have the best of both worlds to try and sort of make sure that the backup industry is still relevant. Um, you know, you can have the best of both worlds where you you get selective about what you want to protect, right? And to your point about to your point about you know knowing your data knowing what you really want to store for long term whatever it becomes again a conversation about the data much to your point earlier talking to a customer getting an outcome what
(34:36) exactly is important to you what data could you not live without you know not in the moment there but say in 5 10 20 years right and I think that's where you can take and the concept of a secondary location works um but if someone's deleted a file say to example of the media um the NAB thing that you guys are working on or this the platform you created.
(35:03) If someone's doing real time editing in in a live uh NBA game or whatever it is and they make a mistake and they lose something, you need you probably need to get that back real quick. I think that's the kind of balance that needs to be brought forward. You still always have backup systems. Now our systems are backed up but the but that class of data which is your big data uh holdings your archives your larger stuff that needs new mechanisms for backing that up.
(35:30) Everything else will still get get done and but ideally if this is a conversation with a customer if I turn up and say would you like an RP of zero? What customer in their right mind would say no? Yeah. Exactly. Yeah. If you turn up to a customer and say, "Would you like an RTO of zero recovery time objective with zero?" What customer in their right mind would say no? So the conversation with a customer is going to say yes to both of those.
(35:53) So if we can provide those and of course they're going to take that. So that's where the nature of the conversation goes. Yeah, that's interesting. Then the reality of what that actually means comes comes out in the wash a little bit later on, right? Um I'm I'm interested in one of the things I found out is that number one you've got a lot of users globally which again points to the fact that you know you are sort of there but not there. You're kind of hiding in the shadows a little bit as a company side of the moon guys. Yeah. Yeah. Which is great in itself but
(36:16) I kind of love that that whole part of it. I love the conversation around that. Um it's just very you know talking to you for the last hour. It's it's very you and I love it. Um R&D spend as well. I guess this is comes in your first principles situation.
(36:33) So you guys do reinvest a lot of revenue back into R&D, which in a and again speaks to the fact that from a marketing out there perspective, a lot of these startups will have a big proportion of their revenue pumped into into marketing because effectively they're pretending to be someone they're not until they finally make it, right? That's the whole point of startup startup marketing, right? And then you can kind of say what you're doing and then you can really do it maybe in five years or you fall off a cliff. So, how do how do how has that reinvestment helped you guys um over the years and
(36:59) how's it going to help you moving forward into future innovations? Strong economies are based on innovation. Innovation is based on creativity and research. Strong economies have good research sectors. We're a commercial entity also operate with re uh uh uh universities and the like.
(37:22) So, we've got a foot in both camps. It's pretty clear that if you make interesting technology and the only way you can make interesting technology is to invest in research, uh then you've uh you've derisked your future. So if you keep coming up with stuff that's interesting year after year after decade after decade, you will be relevant for decades.
(37:46) If you build something where you get seed funding and you your R&Ds in the first phase while you're doing marketing and then you've made something and then people spend the next 20 years selling it. So it just becomes an accountant's company or the sales and marketing guy. You you're not protecting your future investment because you're going to be pipped at the post by somebody else.
(38:03) My philosophy is that you keep making stuff and if somebody ever gets near us, you just innovate away. I don't really care. Yeah. Yeah. Yeah, and I think in Australia we typically there's there's a bit of a notion of we don't do a lot of this in Australia. Um it seems to be sucked out of our country.
(38:21) Um so seeing that actually happening still at certain levels is very very refreshing and you know it needs to be more right. Um our future as a as a country every country depends on innovation. We need innovation to solve issues with uh with food security, with biocurity, with climate security, with uh with the you know generating new forms of revenue etc. All these things are all driven fundamentally by research and interest.
(38:48) Occasionally people have an epiphany moment. Uh the guy in the shed or the woman in the shed down the road has an idea and that changes the nature of things. It doesn't happen that often. No, it doesn't. And it's it's great to see and I love the fact that you reinvest and and you're keeping it there in but also you're clearly um having great success overseas as well which is really great to see at the same time.
(39:11) Right. Just to finish off like where do you see you guys in when I used to ask this in the previous episodes I I'd go oh five years but now I'm kind of rethinking this to maybe two years or 18 months because we're moving so quick at this point in time. And I'm interested actually I mean your your quantum background.
(39:31) Maybe just give a little bit of a your thoughts on where quantum computing is because I've I've been obviously we had a little bit of a spike in interest last December when Google announced everything and Microsoft had a bit of announcement and then all the stocks went crazy and people started to invest massively and I missed out on those again but that's okay.
(39:51) Um but where do you see this fundamentally going? Are we close to you know quantum computing being mainstream? because this is the prevailing thought, but I'm wondering whether this is really going to happen in the next x amount of years. It's being going to nearly happen for x amount of years. So, but being the nature of quantum, it could be a quantum step change. So, tomorrow someone actually could have the aha moment that solves this.
(40:10) We have issues with error correction. Um, and we have issues with uh uh running generalized uh compute. Um that's an area that I'm not that involved with anymore, but I have a great deal of interest in uh watching what people are doing. I look at this from the point of view from a data point of view is we've got lots of data out there that's encrypted.
(40:36) How do you get quantum resilience encryption and NIST has proposed certain algorithms for this? I don't think any of them are going to be resilient. That's that's interesting. Don't tell don't tell the world that because everyone everyone's a lot of companies are are scrambling with the post quantum cryptography standards. Microsoft just announced one yesterday. Um that is really but I also but I also saw something today.
(40:55) Yeah, I saw something today where um the amount of cubits to actually break cryp cryp uh bitcoin encryption was a lot less. It was a in the millions as opposed to 20 million in terms of what they thought. So even now we're kind of seeing this breakdown quickly into what they thought is going to be the the ceiling and now it's actually being brought down interestingly.
(41:19) But yeah, so it's one of those funny things is no one actually has invented the perpetual motion machine yet, but it's possible that it might exist and someone comes up with it. We know we're all pushing towards quantum computing and we know that they'll make a huge difference and change. Uh but we don't know when that's going to happen. So we go out and we do our daily jobs.
(41:37) We know that we're going to be in trouble with our security and others when that happens, but we just ignore it because we have no no other choice. And when it comes, we're going to have to adjust to it. Yeah. And also, I mean, for people that don't know much, I mean, I don't know much. I'm starting to learn up on it.
(41:53) But the problem with error correction and lots of errors in the processing is one, but then the heat as well. And that's why I think and then I've seen photon based quantum computing seems to be a little bit more less heaty and that sort of thing. So to your point about energy power, it's the same sort of problem, right? It's with quantum computers don't use that much energy, but I may be incorrect. So all the heat all the all the heat all the heat they produce.
(42:17) One of the two. Maybe it's maybe it's in the output versus the inputs and whatnot. Anyway, you're the you're the quantum physicist, so I probably Yeah, I probably I'll probably trust you over me on that one. Just quietly. Anyway, hey, this has been a great conversation. Um, you know, you guys are doing some really cool stuff.
(42:34) I'm interested to see you're becoming more well known. I mean, you're on Blocks and Files. You're kind of up there. People do know about you, but I think the whole world needs to be a little bit more understanding about what you guys are trying to solve because I think it's really interesting and you're doing some great work. Thank you. We've got really interesting technology.
(42:51) We have something that people need. We've done very little marketing over 20 years. The last couple years I started to doing it. So, if you ask me where we've been a couple of years, I think we're going to be a lot more visible because we have something that people need. Great way to end it. Perfect end. Hey, thanks for being on episode 102 of Great Things with Great Tech. Uh, thanks Anthony.
(43:10) Hey, just as a reminder, thanks for listening to this episode. Stay tuned for more episodes where we continue to highlight companies and technology shaping our world. Don't forget to follow us on social media, gtwgtpodcast and visit gtwgt.com for more great content and all past episodes.
(43:30) If you enjoyed this episode, make sure to subscribe on your favorite podcast platform and on YouTube. Please spread the word and if you feel like it, drop a review. Thanks for joining us and we'll see you next time on Great Things with Great. [Music]