Flostream Project Explained: Innovative Decentralized Storage

Author: DFG Official Source: medium Translation: Shanoba, Golden Finance

Flostream is a decentralized physical infrastructure (DePIN) service provider that is building a new decentralized storage and distribution solution for Web 3.

Why is decentralized storage important for crypto and Web3?

Public layer-1 blockchains such as Bitcoin and Ethereum have long relied on the development of external storage technology, and blockchains are inherently unsuitable for storing large amounts of data for the following reasons:

• Blockchains themselves are designed for the transfer of value, not for storing large amounts of data, and it is very expensive to store data in blocks.
• If a large amount of data is stored in blocks, the network congestion will become very serious, which will trigger a gas war and lead to an increase in fees.
• Even though the blockchain accepts the deposit of large files through expansion, the data structure of the blockchain determines that the block information is mostly stored in the form of encryption and compression, which is not suitable for the retrieval and analysis of file information.

Obviously, these cost and efficiency issues cannot be solved by centralized storage service providers, whose data is censorship and mutable, which goes against the spirit of blockchain and Web3.

Only by realizing decentralized storage can we truly realize data ownership confirmation and sovereignty network, otherwise it makes little sense to develop a blockchain network at the expense of centralized efficiency. As a result, Decentralized Storage Network (DSN) has developed into a mainstream solution, which is a storage business model that uses blockchain as the underlying technology to store files or collections of files in the form of shards on the storage space provided by the supply side through distributed storage.

Ideally, a DSN must be able to store, retrieve, and maintain data while ensuring that all participants within the network are incentivized for their contributions, while maintaining the trustless nature of the decentralized system. However, in practice, each DSN project has made its own trade-offs and trade-offs. For example, Filecoin pursues decentralization but suffers from storage experience and performance due to its proof-of-space-time algorithm that is computationally intensive and slow to write, Storj pursues high performance at the expense of some decentralization, and Arweave takes a different approach to permanent storage, but it also brings higher storage costs, which is more suitable for NFTs and DApps.

Marketplace

At present, the overall market volume of the decentralized storage track is relatively small. According to a report published by Messari, the total revenue of the four leading storage protocols (Filecoin, Arweave, Storj, and Sia) in 2022 was only about $14.5 million, which is in stark contrast to the current demand for tens of billions of dollars in cloud storage market. According to a research report by Vantage Market Research, the global cloud storage market is expected to grow from $90 billion in 2022 to $330 billion by 2030, at a compound annual growth rate (CAGR) of 17.7% between 2023 and 2029.

The adoption of new technologies such as the Internet of Things (IoT), artificial intelligence (AI), and automation is driving exponential growth in data creation and storage needs. While centralized data storage service providers occupy a huge market share, they have exposed many drawbacks of data and market monopoly, and the decentralized storage field urgently needs more game-changing startups to join in order to promote the realization of more Web3 use cases with high requirements for data sovereignty.

Global Cloud Storage Market Size Forecast / Source: Vantage Market Research

Flostream Network: Decentralized Web3 Infrastructure Service Provider**

Flostream Network is trying to bring some changes to the decentralized cloud storage market. Flostream is a decentralized physical infrastructure (DePIN) service provider that is building a new decentralized storage and distribution solution for Web 3.

In addition to providing content storage services through Amazon S3-compatible APIs, the Flostream Network provides fast content retrieval, on-demand content distribution, and support for applications such as social media or video-on-demand streaming, all in the same performance as Web 2 equivalents.

Flostream is designed to provide data durability, security, and reliability to users or applications in its ecosystem. Any data loaded into Flostream is fully owned by the user, ensuring digital property ownership of the data. Flostream has a local verification service that verifies arbitrary data loaded with digital signatures and timestamps. The Flostream team wants to protect the integrity of the Internet through permanent storage and native authentication that allow users to back up copies of their social media profiles, creator content, or the entire Internet to the Flostream network, thereby protecting the truth in the era of deepfakes and AI.

Floststream Storage & Content Availability

Flostream's founder, David Rhodus, previously worked at Consensys and developed a number of Ethereum-based POC products for some of the largest companies, many of which ended up struggling with scaling. David and his team spent a lot of time experimenting with how to connect distributed systems to blockchains, and how to use blockchain to control those systems.

In the process of developing an open, decentralized wallet-to-wallet messaging meta-protocol, the team encountered a scalability problem. The large amount of communication generates massive amounts of data, and the Flostream Network team realized the limitations of existing decentralized storage solutions (IPFS, Filecoin, Crust, etc.), which prompted Flostream to seek its own decentralized storage solution.

Flostream has launched its own Amazon S3-compatible storage service, Flostream Storage, and has made a lot of innovations from the underlying technology to the storage architecture design to differentiate the competition. By introducing innovative technologies such as a proprietary metadata layer, the Flostream Data Transfer Protocol (FDTP), and a cutting-edge BLAKE3-based hashing algorithm, Flostream Storage is poised to address the scalability and low performance issues currently faced by the decentralized storage market. Autonomous storage protocols were developed with the goal of optimizing content distribution and ultimately becoming the content availability layer of Web3, enabling data to be easily stored, moved, retrieved, and distributed.

Technical Overview

Flostream Storage consists of multiple units, each containing a storage cluster and associated control and monitoring infrastructure. These units provide customers with Amazon S3-compatible storage services, ensuring the isolation and security of customer data. A cell consists of a storage node and a gateway node, which is the entrance to the cell storage cluster. Nodes in a cluster can be used as storage nodes or gateway nodes. Both types of nodes are capable of communicating with all other nodes in the cluster, maintaining a metadatabase, and providing S3 API endpoints.

Flostream storage architecture consisting of storage and gateway nodes / Source: Flostream documentation

Competitive Landscape

At present, there are obvious shortcomings in mainstream centralized storage service providers such as AWS and Google Cloud, as well as decentralized storage service providers Filecoin and Arweave. Although centralized solutions have high storage performance, they are not cheap, and there is also the possibility of a single point of failure. Decentralized solutions are relatively inexpensive (Arweave is a one-time payment, so it is slightly different), but there are many performance limitations in both storage and throughput, and cannot be applied to large-scale, high-concurrency Web3 application scenarios.

The pain point of existing distributed storage protocols such as IPFS is that data is stored on an expanding branch of nodes and relies on linear file retrieval methods, which limits scalability and speed. Flostream's goal is to achieve the performance advantages of centralized storage service providers, and at the same time, it can achieve low prices and lower network latency than the leading decentralized storage projects on the market, and seek a more balanced storage solution.

In response to these bottlenecks, Flostream Storage uses a proprietary metadata layer to enable fast data retrieval and low-cost operation. In addition, Flostream uses its own cutting-edge hashing algorithm based on BLAKE3 for data compression and network management, which is 14 times faster than the current SHA-256 used by AWS. At the same time, the high-performance data transmission protocol FDTP is introduced, which is optimized on the basis of the existing UDP protocol. Flostream also employs intelligent data placement and data migration, with AI data location migration to reduce hotspot deployments and edge storage to process data closer to the data source, enabling faster access and faster throughput during high loads. The in-house intelligent deduplication algorithm minimizes storage redundancy and optimizes the use of capacity, reducing costs and increasing data transfer speeds. Through a series of technical optimizations, Flostream storage has achieved great improvements in multiple dimensions:

• Storage and traffic prices are lower than centralized AWS and Google Cloud, and while still slightly higher than decentralized projects like Filecoin and Storj, they offer better storage and bandwidth performance at the same cost.
• The read/write throughput can reach the level of current centralized storage service providers, and in the centralized operation mode, the throughput capacity will far exceed that of decentralized storage head projects such as Filecoin, Arweave, Storj, etc.
• It still maintains high scalability and low latency similar to centralized storage service providers.

Flostream is currently serving thousands of individual users and is piloting with several enterprises to move storage workloads from AWS and Google to Flostream in the future.

Solid and experienced team

With extensive experience in blockchain, software engineering, cloud computing, data storage, and project management, Flostream team members are committed to leveraging their skills and resources to deliver the most advanced Web3 solutions in the crypto industry. The founder, David, is a serial entrepreneur with over 25 years of engineering experience. Back in 2010, he built a streaming platform that supports more than 100,000 live sports events a year, and was eventually acquired by Amazon Web Services, which is rare among Web3 entrepreneurs. He joined Consensys in 2015 and focuses on the development of blockchain solutions for large enterprises. David's background as a serial entrepreneur has not only provided a strong endorsement for the project, but has also attracted the attention of many well-known institutions. Flostream closed a $3 million funding round in February this year, with participation from the likes of DFG, Fuse, Scytale, D1 Ventures, and TRGC, and is one of the few decentralized storage projects that still has access to external funding in a crypto bear market.

Summary

At present, most of the cloud storage market share is still monopolized by the centralized solutions provided by giants such as Amazon and Google, and the decentralized storage project products do not show obvious advantages in terms of user experience, performance, and cost. These decentralized projects operate primarily in a mining mode and try to attract users through token incentives, which leads to a large loss of users after the reduction of token incentives in the crypto bear market. Lacking the attributes of Web3 native applications, it will inevitably fail to attract more users and enterprises and achieve large-scale growth in the long run.

Although decentralized storage can solve many problems, such as data privacy protection, censorship resistance, and avoiding single points of failure, the development of decentralized systems is still immature, and there are still many problems to be solved, and the industry scale of decentralized storage is far from the peak. Due to the overall sluggish market in the previous bear market, lack of content distribution capabilities, and limited application scenarios, even leading decentralized projects such as Filecoin, Arweave, and Storj cannot compete with centralized storage giants in terms of market size.

As Layer 2 technology matures, DApps become cheaper and faster. The accumulation of massive amounts of data and market demand will once again shift the industry's focus to decentralized storage. In the future, the entire DSN market will need more middleware and aggregators to carry more user traffic, and on the other hand, it will need to continuously adjust and optimize the threshold and cost of DSN applications to attract more users. Flostream's storage and data availability products are the ideal solution for the seamless integration of DApps and decentralized storage. As an early investor in Flostream, DFG is pleased to see that Flostream Network's products are bringing more practical application solutions to the broader Web3 market. David is a veteran of the cloud services industry, and his previous startup was acquired by AWS, a testament to his exceptional leadership and creativity. We are confident that David will be able to lead the team to new successes and enable Flostream to successfully lead the next wave of innovation in decentralized storage and Web3 applications."