Tuesday, April 16, 2019

Decentralizing Systems: Cloud Storage, OSS and the Storj Labs Project

Storj.io and Tartigrade.io - driving decentralization of cloud storage
"Can we build the world's largest, and most performant, economical, and secure storage network without building or operating a single data center?" 
- Shawn Wilkinson, Founder - Storj Labs

Eight months ago, I had attended a conference - The Linux Foundation's Open Source Summit (OSS) North America in Vancouver [1]. It was the opening morning, with keynote presentations underway. Imad Sousou, Corporate Vice President and General Manager of the Intel Open Source Technology Center had just finished delivering his keynote talk titled "Software Defined Everything", with particular emphasis on their Clear Linux [2] and ACRN Hypervisor [3] projects (which I plan to cover in later posts). Then came a talk that really caught and riveted my attention.
Ben Golub, Executive Chairman of Storj Labs delivering the opening remarks of the keynote presentation
Ben Golub, Executive Chairman of Storj Lab, discussing the benefits and impact of OSS
Delivered by Ben Golub and Shawn Wilkinson from Storj Labs [4], it was titled "Open Source and the Decentralized Web." The first couple of slides were generically interesting, touching on the benefits and accomplishments of the Open Source Software model - the very things that had drawn, immersed and kept me in the OSS environment over the last 12+ years. However, as the talk and presentation slides became more specific in details, I sat up in my seat with the sudden, sharp intake of breath and focused vision when the eye latches onto something with intensity. I was listening to something very familiar, something that I had not only encountered, but also extensively studied, researched and presented on before, during my graduate program, - decentralised systems, with a particular focus on peer-to-peer (P2P) architecture. Now here was someone not only discussing the state-of-the-art, but making the case for a product, company and profitable business as well. Someone was talking my language.
This is where I really started listening....
By this point, Ben had handed the presentation over to Shawn, who carried it the rest of the way to its conclusion. He discussed the essential characteristics of a decentralised system and application, briefly touched on the blockchain and crytocurrency, before diving back into the essential thesis of the keynote - decentralizing the Web through services that were built on such architecture, and the inherent benefits that they bring in terms of infrastructure, scalability, security, fault-tolerance, flexibility, innovation and the actors involved.
More interesting and familar details on a long-standing passion of decentralised systems...
Laying out the benefits of decentralized applications and OSS, versus the alternative...
In this case, the vertical domain being discussed was cloud storage and the objective of decentralising the same. In particular, the cloud storage service that Storj had built, deployed, operated and tested - all based on a decentralised, peer-to-peer infrastructure. It was absolutely fascinating; I had spent 4 years of my life extensively involved in the architecture and applications of P2P systems, and it had become longstanding area of professional interest, but here was something more - a viable business case for commercial services that involved not just incorporated providers, but actively called for and welcomed the involvement of ordinary users through voluntary donation of their free storage space, with the generation and payout of revenue in return.

In a way, it reminded me of the SETI@home project that I had participated in during my years at graduate school, with the idea being that ordinary users could participate in major endeavors and projects being undertaken at research labs. This involved downloading and installing a small application that would make use of a small portion of their computer's processor time and power, as well as a tiny amount of storage space to perform processing of research data during the machine's idle/down time. For example, by dowloading the BOINC software application [5] provided by the University of California Berkley, users could participate in distributed processing of research data generated by SETI@home [6], Climateprediction.net, Rosetta@home, World Community Grid, and many other. 

The difference in this case was allocation of dedicated storage space as part of the decentralized network infrastructure, with the intention of generating revenue in return. That was something I hadn't come across before, and I was most interested - not necessarily from a commercial point of view, but from a technical and engineering standpoint. I wanted to join the project and help build a system that was very relevant to my expertise and core interests.
The core thesis of Storj Labs' product overview and USP at the OS Summit 2018 keynote.
I had a chance to meet and connect with some of the Storj Labs team after the keynotes were over, and discuss their work to date and plans moving forward. it was a most enlightening discussion, and I also came away with a very nice T-shirt to boot.
Closing remarks at Storj Labs' keynote presentation at OS Summit NA 2018
Although it's been several months since the conference, I gradually found time in between other pressing demands to systematically and thoroughly investigate the Storj website, GitHub repository, read their Executive Summary and detailed 90-page White Paper of their V3.0 platform, watch 5.5+ hours of all their past Town Halls and videos that I had missed in order to get up to speed on the technical perspective of the work done to date and roadmap of future development.

Storj in particular fascinates me not only because of its commercial business model and product strategy, but because of the invitation and opportunity to join a significant endeavour that I believe is essential to the future of the Internet, contribute the technical knowledge, skill set and experience that I had gained, while staying on the cutting edge of developments in a field that relates to my core expertise and skill set.

I will be posting more updates on Storj as I get further involved. See you in the next post!

REFERENCES:
==========
[5] BOINC

No comments:

Post a Comment