Interview: Storj’s Decentralized Cloud Storage Platform is a Competitor to Amazon S3
With the emergence of blockchain technology, the decentralized cloud storage solutions are designed to revolutionize the way we store and share documents, pictures, and data. Storj, the leader in decentralized cloud storage, aims to o bring decentralized, end-to-end encrypted cloud storage to the the average business and consumer.
In an interview with 8btc, Kevin Leffew, global BD manager of Storj shared his views on features of the decentralized cloud storage solution.
8btc: We know that compared with the replication, erasure coding still has problems such as long-distance downloading challenge, etc. Why does Storj still use erasure coding ? Will it be used for a long time?
Storj: First, systems that use replication experience the same long-distance downloading challenges as systems that use erasure coding, however, it actually is less of an issue with erasure coding. If you are downloading a 10/30 erasure coded file, you can rebuild it from any 10 of its 30 pieces. This means that the network only needs to request 13-30 pieces, with the extra 3 compensating if a transfer is delayed. Once the fastest 10 pieces are delivered, the remaining downloads are canceled and the file can be rebuilt. With a replication system, the network would need to attempt to download an extra copy (at minimum) of each piece of the file, otherwise, the entire file download would be delayed if even one file piece was delayed during download.
Using erasure codes greatly decreases the expansion factor of the data stored on the network while maximizing reliability. With our erasure code scheme, we can achieve 11 9s of durability with an expansion factor of less than 3, meaning for every GB of data stored, our network utilizes less than 3 GB of storage capacity on the network. Meanwhile, it would take an expansion factor of 16 to achieve this same reliability on a network using only replication (meanwhile storage node operators on this network would also earn 5x less). For example, assume a community of storage node operators makes $5 per TB of data uploaded to the network. On an erasure coded network with 3x expansion factor, that would be split 3 ways into $1.66 per GB of static data stored. With a network using replication, that would be split 16 ways into $.31 per person. What do you think happens when it no longer becomes economical to run a storage node? Storage node operators leave the network, driving repair costs up and risking data loss.
Erasure codes do not tie durability to the expansion factor. You can tune your durability without increasing the overall network traffic and amount of static data stored on the network! Erasure codes are widely used in both distributed, and peer-to-peer storage systems. While they are more complicated and possess trade-offs of their own, the scheme we adopt, Reed-Solomon, has been around since 1960 and is used everywhere from CDs, deep space communication, barcodes, advanced RAID-like applications–you name it.
Many products in this space (Filecoin, MaidSafe, Siacoin, GFS, Ceph, IPFS, etc.) by default use replication, which means simply having multiple copies of the data stored on different nodes. Our previous network did this as well! When you look at the math as we did while operating the largest decentralized cloud storage network in the world, it just doesn’t make sense.
8btc: Storj Labs is committed to making Storj more decentralized and has released specific technical solutions. Could you please give me more details about these solutions?
Storj: The V3 Storj network is composed of eight core components: Storage Nodes, Peer-to-Peer Communication and Discovery, Redundancy, Metadata, Encryption, Audits and Reputation, Data Repair, and Payments.
We go into each of these components in heavy detail through our V3 Whitepaper.
8btc: In the white paper, Storj Labs said it would use a collection of trusted satellites for users and storage nodes to choose from. Does Storj have a detailed governance plan for the satellite to prevent malpractices or other accidents?
Storj: While Storj is open source, and anyone can run a Satellite, Storj will be running a specific group of Satellites, called the Tardigrade Network, which have a guaranteed service level agreement (SLA) around availability and durability of data (equivalent to AWS) for customers. We do expect partners and others to run their own storage nodes, as the network is decentralized, open source, and free for anyone who wants to operate a Satellite. However, as with all open source technologies, users will be wary to run (and connect to) random Satellites that do not guarantee specific up-time.
8btc: Storj stops subsidizing storage nodes with the release of V3. Will the team suffer a loss from this move in the short term?
Storj: As we mentioned in question 1, the decrease in expansion factor found from migrating to an erasure coded model (over an erasure coded+replicated model) will greatly increase payouts for V3 storage nodes when compared to storage nodes operating on the V2 network. In addition, we have a waitlist of nearly 10,000 members who are interested in joining the network to earn STORJ tokens in exchange for sharing their hard drive capacity and bandwidth. As we bring customers on to the new V3 network, and payouts increase, we are very confident the network will grow as fast as we need.
We found that subsidizing nodes in V2 led to an adverse incentive model where storage node operators would spin up a large number of nodes to maximize baseline payout (essentially Sybil attack on the network). With V3, we have been very conscious of the inventive model and economic game theory behind our approach in order to optimize against adverse conditions.
8btc: How does Storj motivate storage nodes to make continuous contributions to the network, or prevent them from doing evil?
Storj: Unlike centralized solutions like Amazon S3, Storj operates in an untrusted environment where individual storage providers are not necessarily assumed to be trustworthy. Storj operates over the public internet, allowing anyone to sign up to become a storage provider. We adopt the Byzantine, Altruistic, Rational (BAR) model to discuss participants in the network.
- Byzantine nodes may deviate arbitrarily from the suggested protocol for any reason. Some examples include nodes that are broken or nodes that are actively trying to sabotage the protocol. In general, a Byzantine node is a bad actor, or one that optimizes for a utility function that is independent of the one given for the suggested protocol.
- Inevitable hardware failures aside, altruistic nodes are good actors and participate in a proposed protocol even if the rational choice is to deviate.
- Rational nodes are neutral actors and participate or deviate only when it is in their net-best interest.
Some distributed storage systems (e.g. data center-based cloud object storage systems) operate in an environment where all nodes are considered altruistic.
In contrast, Storj operates in an environment where every node is managed by its own independent operator. In this environment, we can expect that a majority of storage nodes are rational and a minority are Byzantine. Storj assumes no altruistic nodes. We must include incentives that encourage the network to ensure that the rational nodes on the network (the majority of operators) behave as similarly as possible to the expected behavior of altruistic nodes. Likewise, the effects of Byzantine behavior must be minimized or eliminated.
8btc: At present, storj still USES ERC20 as incentive, will it consider using its own native token as incentive in the future? If not, why?
Storj: Storj uses the ERC20 standard for a number of reasons. First, with the ERC20 model, the security model of the token is tied to the security model of the Ethereum blockchain, making it very difficult to execute a 51% attack, as opposed to other solutions with custom blockchains. Additionally, the ERC20 standard is widely adopted by wallet providers, exchanges, DeFI innovators, and others, making it easier for third parties to adopt Storj and integrate it into their platforms.
If you look at platforms in our space that operate their own blockchains, you will see that a significant portion of their development effort is spent supporting, mining and maintaining their blockchain – all while coming very far from achieving the security that is made possible through the Ethereum blockchain. We are first-and-foremost a cloud storage company.
As you may be aware, Storj primarily uses the blockchain as an efficient means to transfer value. The performance of the blockchain is not tied to the performance of the network. However, we are testing layer 2 solutions, like Raiden and micropayment channels as a way for us to reduce the transactional costs of running the network.
8btc: Storj’s white paper V3 mentions the compatibility with AmazonS3. What are the considerations, what are the benefits to the entire network, and does it risk centralization?
Storj: Compatibility with Amazon S3 simply means compatibility with the S3 open API Standard. It does not mean that we are using Amazon data centers or other infrastructure. Developers historically choose the path of least resistance. We want to make it as easy as possible for developers, DevOps teams, and other users of Amazon S3, to migrate to a decentralized backend so that they may take advantage of the security, performance, and economic gains offered by Storj and the Tardigrade network.
We are largely a competitor to Amazon S3, and by building an easy pathway to migration through our S3 Gateway and competing with them at the fundamentals (price, performance, security), we hope to make it easy and compelling for projects to migrate to our platform.
Storj: 5G will make it possible for more types of devices to become Storage Nodes. If a device is ‘always on’, has excess storage capacity, and is connected to a 5G storage node – it may be able to become a worthwhile storage node.