Publication date: May 16, 2024
This article will explore the technological possibilities that are intended to enable the sub0layer. The focus is on the potential provisioning of the layer and on the approaches to its development and construction. The sub0layer will be technically realized through the Logos Network. Therefore, this article primarily focuses on explaining the network design and its potential implementation.
Let's start with what we imagine and why we believe that this approach will advance Web3. In today's world, cloud services are indispensable in modern computing. They offer flexible, scalable, and highly available solutions for businesses and individuals alike. For this reason, such services are often used to operate Web3 nodes (e.g. Ethereum Mainnet Statistics (opens in a new tab)). However, these services are associated with increasingly higher costs due to various factors. Cloud services often rely on very powerful and reliable hardware, which is continuously updated and maintained to meet the latest technologies and security standards. This hardware is not only expensive to purchase but also to operate. The continuous maintenance and management of the infrastructure require skilled personnel, which also incurs high costs. This includes not just IT experts but also security teams, support staff, and others. The energy costs for operating data centers, especially cooling the servers, are enormous.
On the other hand, many of us now have advanced hardware at home that could potentially be used for similar purposes as those managed in the cloud. This raises the question of why many still prefer the more expensive cloud solutions, despite owning powerful private devices. Cloud services are accessible from anywhere and offer constant availability, which often cannot be guaranteed with private devices. Some applications, such as running a blockchain validator node, require a static IP address, which is uncommon in public networks and often associated with additional costs. Cloud providers guarantee a certain quality of service, including security standards and regular backups. Cloud services often offer redundant systems that can immediately step in when a component fails, which is rarely the case with private hardware.
We are in a difficult situation. The use of cloud services contradicts the approach of Web3, and we pay a lot for resources while placing our trust in profit-oriented companies. If we do not use them, other adverse factors come into play, like we described before. A key point is that we cannot determine where the blockchain nodes are operated, as ultimately the community, who runs these nodes, makes these decisions. Since cloud services offer a user-friendly solution that brings many advantages, they are often chosen by the community because often participants have 'no other choice'. While there is always a choice, it is not as user-friendly as that of cloud services, which represents a general problem in the Web3 world that everyone is struggling with. Generally speaking, the Web3 infrastructure should be operated by the community (through running blockchain nodes such as Validator, Full-Node, Archive-Node, etc.). Currently, most of those who provide computing and storage (not just computing and storage) in a blockchain network are skilled technicians who know what they are doing. This significantly limits the participation in providing computing and storage capacity for Web3. Providing pure computing power and storage resources is simpler than the expertise required to operate a validator.
We are facing another important point regarding Web3 infrastructure. Typically, it is operated by those who have the necessary knowledge and skills. Most users and developers want nothing to do with the underlying infrastructure; they simply want to develop, deploy, and trust that the infrastructure is secure. For the average developer and user, not much should change, except that the application is operated decentrally and secured through blockchain technology. Projects like Ethereum, Cardano, Astar, Moonbeam etc. aim to make this possible. A developer does not want to worry about the underlying infrastructure or to provide a complete blockchain just to operate their application. For the average user, it will merely be noticeable that the service they are using is now secured by blockchain technology and that it is a Web3 application. It is crucial to understand that in Web3 nowdays, the infrastructure layer is clearly separated from the application layer. This allows developments, even if they are to be decentralized, to be handled similarly to the development of Web2 applications. This separation of layers offers a flexible architecture that allows developers to design and implement their applications independently of the underlying infrastructure. As a result, the development of decentralized applications can become both accessible and seamless, comparable to traditional methods from the Web2 area, but with the enhanced security and trust benefits that the blockchain technology offers.
Nevertheless, we need a secure and efficient infrastructure, and someone has to provide it. Polkadot 1.0 demonstrates a very good approach to operating a Web3 infrastructure. Where most Web3 projects struggle, however, is in ensuring truly isolated operating (core computing and storing), as it cannot be guaranteed on which hardware the blockchain nodes are run. It could be a PC or a server in a data center. There are several methods attempting to solve this problem: Technologies like Docker allow applications and their dependencies to be encapsulated in containers that are isolated from the host operating system and other containers. Containers share the kernel of the host operating system but still provide some level of process isolation. Trusted Execution Environments (TEEs) offer a secure execution environment where code and data are protected from external access, even if the operating system has been compromised. In some cases, particularly with highly sensitive data, physical isolation may be necessary. This means using dedicated hardware that is not shared with other participants. Many cloud providers offer such dedicated instances or specially reserved hardware. Although there is no 100% guarantee for the security and proper functioning of compilers and the hardware on which they run, a combination of techniques, tools, and best practices help minimize risks and strengthen trust in these systems. In an environment where absolute security is unattainable, these measures aim to reduce the risk to an acceptable level and ensure the integrity and confidentiality of the systems as much as possible. The challenges of security and trustworthiness of compilers and execution environments are not unique to blockchain or Web 3.0 technologies but affect all areas of development, including traditional Web 2.0 applications.
To create an isolated computing/storage environment suitable for any blockchain technology, designed exclusively for core computing/storage tasks, several approaches can be considered. The fundamental idea, however, is to establish a layer that serves as an abstract interface between the physical hardware and the blockchain components to be executed. This layer would handle the necessary computations and storage as directed by the higher layers, such as the validation and security layers. The sub0layer should be a pure computational and storage layer for hosting Web3 nodes, independent of their technology, in an isolated core environment, provided and governed by the community. This separation could enhance efficiency and enable different blockchain technologies to utilize the same core infrastructure.
Vision for the sub0layer.
Can the aspects of, computation and storage provision, and general node provisioning, be considered separately at all? This is the challenge that the provisioning of the Logos Network aims to address. The underlying idea is to create a separate, universally applicable Web3 layer (sub0layer) dedicated solely to isolated computation and storage of data. Fundamentally, a slot (a secure isolated environment for a Web3 node) should enable core computation, storage, and network connectivity (not interoperability). The management of the broader blockchain network is left to the operator of these networks. The approach is to establish core security and decentralization layer for the Web3 infrastructure governed by the community.
It is important to emphasize that the sub0layer is not intended to host blockchains in the traditional sense, as is the case with Polkadot, but rather just their core infrastructure. By core infrastructure, we mean computing, storage, and base network connectivity (without interoperability). A blockchain node acts as a software unit within a distributed network, specialized in hosting and managing this blockchain infrastructure. These nodes perform numerous tasks, including network management, data storage, computing processes, consensus formation, transaction processing, block production, and verification, etc.. Technically, these nodes are not independent machines like virtual machines (VMs). Rather, they are seen as executable 'applications' that are translated into machine code by a compiler and typically run within an isolated container. These containers create a shielded environment that runs on a host system and shares resources with other containers or servers in a data center. Even though blockchain 'applications' run in containers, their performance in terms of storage and computing still heavily depends on the underlying hardware. Containers do provide an isolated and portable environment for execution, but they cannot completely detach from the hardware on which they operate.
Containers share the resources of the host system on which they are executed. This means that the CPU performance, available memory (RAM), and storage capacities (such as hard drives) directly affect the performance of the applications running in the containers. Although containers have their own isolated storage space, the overall performance and availability of storage still depend on the storage resources of the host system. During intensive I/O operations, the performance of storage subsystems, such as SSDs or HDDs, can be crucial. The network performance of a container also depends on the underlying infrastructure. This includes not just the physical network hardware, but also the configuration of the network on the host system and the available bandwidth. Containers are run on an operating system that may itself be operating within a virtual machine inside a hypervisor. The performance and security of these layers can also have a significant impact on the containers. A poorly configured hypervisor or an overloaded host operating system can impair the performance of the containers. Although blockchain technology offers inherent security and protection within its own layers or environments, such as in individual "node environments", this does not mean that these nodes are immune to problems that arise from poorly configured underlying infrastructure. If the fundamental infrastructure, such as the virtual machines or containers that host these nodes, is not well configured, it can affect the performance and security of the blockchain nodes operating on them. From this, it can be concluded that the approach of providing a decentralized core infrastructure, in which each blockchain node can operate in an isolated environment that is provided and verified by the use of blockchain technology, is crucial.
Before we delve into the possible technical implementation of the sub0layer through the Logos Network, it is important to understand a fundamental aspect. The requirements mentioned so far are well met by current cloud providers, at least in the context of Web2 solutions. However, these do not align with the ideals of Web3, and they are often associated with high costs. We probably do not need to go into detail about the cost issue, as it becomes apparent as soon as one looks at the cost breakdown. 'Not in the Web3 sense' means that it often involves centralized parties that are more profit-oriented than technology-oriented. In addition, there is a lack of complete transparency. What happens within data centers and the actual state of the infrastructure remains unknown to us. Most security incidents and technical difficulties are not made public. In contrast, what is documented on the blockchain remains permanent and immutable on the blockchain. Transparency plays a very important role in the Logos Project and generally in Web3. The lack of transparency from major cloud providers and their tendency to overrate their services highlight the importance of transparency. In many cases, the advertised performance and security do not match reality, underscoring the need for genuine transparency. Web3 aims to create an environment where users can verify the quality and reliability of services themselves, leading to a more informed and trustworthy user experience.
The Logos Project aims to create an Isolated CSN Environment (computing, storage, network) for the execution of blockchain infrastructure nodes. These isolated environments are to be defined on the blockchain and automatically provisioned by the execution of transactions. To better understand this, the Logos network will be detailed more granularly. Fundamentally, the Logos Network consists of two main components: the Distributed Virtual Computing Infrastructure (DVCI) and the Logos Chain. It is important to understand that the idea for the DVCI is to be defined on the blockchain and also operated there. In other words, the DVCI is the manifestation of what is defined and executed on the blockchain. Technically, the DVCI is fundamentally a single component or approach, but the implementation is intended to be distributed across a series of DVCIs globally for scalability and efficiency of the network. Before we look at the individual components separately, it's also important to mention the Network Gatekeeper and the Distributed Computation Regulator (W3bI), as these two components will likely be Layer-2 or Layer-3 solutions on the Logos Chain. The concept for the Gatekeeper is crucial for smooth communication and security between the DVCI and the blockchain. W3bI is intended to be a Layer-3 solution that manages user interaction with the network.
It is important to mention that most of the aspects described here are still concepts that can change. Some of these concepts still need to be validated, which we are addressing step by step through research and development. It is crucial to understand our development approach, as we aim to build the network after providing the complete technical documentation. Each research step includes developments and tests to ensure that all components are detailed and theoretically sound before the network is implemented. Currently, we are focusing on research into integrating private elements into a public blockchain network and developing a security model that supports the 'no human interaction necessary' approach, as well as Zero Trust and Zero Knowledge. This is one of the fundamental factors for the provision of a restricted core computing/storage/network infrastructure over a public blockchain network. In the next few weeks, the first results of this research and the security model are expected to be published. In this article, we would like to explain the basic technical approaches that we are evaluating and likely to pursue. This is important, as we rely on the support of the community and want the community to participate in the research and construction. The project is intended to be an open-source community project that provides decentralized, isolated computing and storage. This is all easier said than done, okay, let's see how all this should work.
Basic representation of the vision for the Logos Network.
The DVCI will be implemented on the Logos Blockchain and is intended to operate entirely on-chain. The primary purpose of the DVCI is to function as a decentralized computing center. This will be achieved by utilizing community hardware, supported by a small computing and data infrastructure, which we refer to as "Nano Data Centers". The strategy involves distributing multiple DVCIs globally, with each individual DVCI including a Nano Data Center. These consist of several servers and community hardware distributed within a defined radius around the Nano Data Centers. The design allows us to establish decentralized units at a global level that adhere to Web3 principles and support efficient operations. Nano Data Centers serve as a fallback in case computational processes need to be moved to a geographically closer region. Such a case could occur if a DVCI does not have sufficient community resources available in a specific global region. The network is intended to function automatically and therefore must have temporary hardware fallbacks. Originally, we planned to use OpenStack, as mentioned in our concept paper (opens in a new tab). However, it is important to emphasize that we intend to continue pursuing this approach in a modified form. OpenStack's methods and possibly the source code could be used, but not the solution in its traditional form. The solution must operate on-chain to ensure network security. This means that the code and configurations of the DVCI will be implemented through our Smart Contract approach, with a clear separation between execution and configuration logic. The transaction execution time is secondary here, as our goal is cumulative security. Specific environments will be defined on the blockchain, which will be provisioned when certain conditions are met, similar to the principle of Smart Contracts. The provisioning and isolation verification of these environments are carried out through the execution of contracts. This means not every computing task will be individually verified, but it will be ensured that the computations occur in an isolated environment. These isolated environments will be monitored by security agents (zero-tolerance policy), who are part of the Network Gatekeeper solution, to maintain ongoing integrity.
The blockchain component in the Logos Network is intended to be a stand-alone Layer-1 solution that primarily serves the provisioning and operation of the DVCIs. The Logos Chain is being developed with the Substrate framework and is meant to be a public network with private elements. As it is a community network, it is important to keep the network as transparent and public as possible. However, in an implementation like the DVCI, not everything can be public for security reasons. This is also the first approach we are already working on: How can a network be public and yet remain highly confidential? There are several approaches such as Smart Contracts, Zero-Knowledge Proofs (ZKPs), private channels, etc., that address and attempt to solve this issue. However, we aim to provide a complete security solution that not only enables private transactions but also includes a strictly regulated access control system, the management of private keys, and the implementation of technical accounts that are to be provided and managed automatically. Only with a complete solution can we ensure that an infrastructure like the DVCI can operate securely and automatically on the blockchain. As we will be publishing our results on security and privacy in the coming weeks, I will not delve deeper into the topic here.
The second approach to be addressed is the "Definition" and the "Operations" of the DVCI on the blockchain. This is also why we want to pursue and implement a different kind of Smart Contract approach. Fundamentally, a Smart Contract is a self-executing contract protocol that is stored and executed on a blockchain, with the contract conditions written directly into code. This technology enables transactions and agreements between different parties to be carried out without the need for a central authority, legal system, or external enforcement mechanisms. The conditions of the Smart Contract are automatically executed as soon as the specified rules are met, thereby increasing transparency, security, and efficiency. Although blockchain technology and Smart Contracts are ideal for transaction management and security, executing code that operates outside of their own environment is not directly feasible on the blockchain. However, the desired goal can be achieved with a combination of On-Chain and Off-Chain processes. We want to create a process where every action in the provisioning chain of an environment is controlled and monitored for correctness by "Smart Contracts" on the blockchain, forming a "Terraform-like" system. This system uses blockchain transactions to validate and control every step, ensuring the integrity and traceability of the entire process. Unlike traditional Oracle approaches that serve as external data intermediaries for blockchains, this system relies on a complete integration of the process logic within the blockchain itself. By implementing all steps and validations directly in Smart Contracts, the need for external data sources is eliminated. This approach guarantees higher security and independence, as each operation and its verification are based solely on the blockchain and secured by its immutable nature. This is the subject that will be addressed next to enable the foundational elements of the network.
Everything provided by the Logos Project is intended to be governed by the community. This means that we also want to use the Logos Chain to establish a community governance system, where decisions about the entire ecosystem are managed. It is important to emphasize again that, after the complete establishment of the network, the sudo rights will be relinquished, and the network will be fully managed through the governance system. The entire ecosystem refers not only to the governance of the Logos Chain but also to the entire Logos Network, such as setting the prices of computational resources within the ecosystem. This creates a stable base and facilitates an overview of long-term costs. The Logos Network is designed by the community for the community and should also be governed by them.
As we can see, the Logos Chain is indeed the brain of the network, which should offer a variety of functionalities. To achieve this, our blockchain implementation must function flawlessly. It is important to mention that after the deployment of the network, the Logos Blockchain nodes, which form our blockchain infrastructure, are to be operated on the sub0layer. This means that the Logos nodes will be executed in isolated environments. A significant factor for the Logos Chain is the processing of transactions, as executing transactions on the chain can be computationally intensive. An interesting approach is the sharding principle, which processes the load quickly and relieves the main chain. With the publication of the JAM Protocol Grey Paper, new ideas and approaches have also emerged in our team, but it is too early to discuss them as they require detailed research. Similarly, the ZKP-Binius approach, introduced a few weeks ago by Vitalik Buterin, is very interesting and has captured our attention. Originally, our approach was to implement zk-STARKs and then adapt the BABE algorithm so that it could handle ZKPs. Recent publications might change our approach, but we will only know this after a detailed review of the new approaches. It is important to understand that our blockchain implementation, like any other, requires consensus, scalability, security, etc., but not up to the core operations level, as we want to provide this basic isolation with the development of the sub0layer. The difference is that we do not want to resolve this "isolation problem" directly in the same implementation as traditionally known in Web3, but rather this layer should be solved through blockchain, albeit with a different approach.
The DVCI exemplifies the infrastructure approach we intend to implement. Essentially, this means that the blockchain should provide an isolated computing and storage infrastructure that does not run directly on it. Simply put: The hardware provided for computing and storage is on-chain, yet not directly. This hardware will perform the basic computing and storage operations but is not directly part of the blockchain network, though it is secured by the blockchain. The Logos Chain is responsible for integrating the provided hardware into a virtual infrastructure and operating it. Fundamentally, the community hardware is used for computing and storage operations, which will eventually undertake the execution of blockchain operations at a lower level, but technically, they are not part of the Logos Chain since they do not influence the operation of the Logos Chain itself. To clarify: The computations, for example, that occur in an isolated environment will not be carried out by any of our validators. Comparatively, the task of the Logos validators is to provision and secure the Logos Chain itself as well as the infrastructure or isolated environments that result from it.
To provide a distributed computing infrastructure, factors such as virtualization, storage, networking, and security must be considered. It involves how exactly community resources can be used efficiently and securely, regardless of whether they are provided over a blockchain network or not. The first crucial step is to ensure secure access to the hardware or its virtualization through Type-1 or Type-2 hypervisors. Subsequently, this environment must be isolated and an efficient and seamless network communication must be facilitated. In our concept paper (opens in a new tab), we have considered approaches like KVM, QEMU, SD-Networks, and OpenStack as possible solutions, which, however, must still undergo benchmarking to find the most efficient implementation.
Fundamentally, this is a type of decentralized cloud infrastructure that aims to provide isolated environments for the operation of blockchain nodes. This provisioning and security are to be carried out via blockchain protocols executed on the Logos Chain. It is not about an infrastructure that is intended to host services in the traditional sense but about isolated base environments for the secure execution of computing and storage tasks. On one hand, this could be compared to AWS or Azure; on the other hand, the Logos Project pursues a completely different approach and technical implementation and serves a different purpose, namely hosting Web3. In conclusion, this implies that a DVCI or the sub0layer generally assumes fewer operational responsibilities compared to AWS or Azure since the overlying blockchain infrastructures handels a part of it.
In summary, an important point with the DVCI is to ensure that access and use of community devices are secure, isolated, and efficient. The network between the community hardware must be efficient enough to meet the requirements of distributed computing, which should be handled with an SD-Network implementation. Furthermore, it is important to mention that the DVCI should be provided in the spirit of IaC and automation. There will be several globally distributed DVCIs, managed by the blockchain, to enable more efficient computing.
The Network Gatekeeper (NG) will encompass a complex array of tasks and will comprise multiple integral components. The intercommunication between the DVCI and the blockchain will be mediated exclusively by the NG, which plays a pivotal role in regulating node behavior. This includes monitoring deviations from standard node operations within distributed resource networks, isolating non-compliant participants, and determining subsequent actions concerning those participants. Each component within the DVCI will interact with the blockchain through distinct NG modules. The entire framework of the Network Gatekeeper will be conceptualized, developed, and operationalized on the Logos Chain. The primary motivation behind developing the Network Gatekeeper is to facilitate an automated infrastructure that leverages cloud-native approaches. For instance, the system can be designed so that native requests (e.g., 'kubeadm join...') are interpreted by the blockchain through the gatekeeper, enabling the automatic triggering of transactions to fulfill these requests. This design ensures that while the implementation may draw parallels to existing technologies like Kubernetes, it is not a direct application of Kubernetes on the blockchain. Instead, it allows for the seamless integration of similar methodologies without substantial modifications to the underlying technology, simplifying the implementation process when such approaches are required. The "middleware" in the Logos Network, termed as Network Gatekeeper, will be developed under an open-source license. LogosLabs is focusing on technological approaches that utilize aproaches and methods like OpenStack, Terraform, and Ansible for enhancing the Logos network. Once the middleware is fully developed and operational, it will serve as a foundational codebase, enabling the deployment of additional solutions via its implementation in a suite of reference libraries.
The final component is the Computation Distribution Regulator (W3bI). This is intended to be a Layer-3 solution aimed at providing the sub0layer adequately. By adequately, we mean user-friendly. The idea is for W3bI to become a platform that allows users to easily use the sub0layer. It will be a platform where users can share their resources and deploy nodes in an isolated environment. Since a user-friendly provision is ultimately achieved in the end, and we need to focus on more critical aspects for now, we will not be dedicating much time to this topic in the near future. It is important to understand what the end result could look like.
The project and development of the Logos network are in the very early stages, and it will take some time before a clear technical specification for the entire network will be available. Each approach needs to be reviewed, tested, and clearly specified one by one. Currently, we are doing this with the initial approach of a complete provision of a security model to be implemented within the Substrate framework. It is important for us to communicate the core idea and our objectives. The implementation and the provision will proceed step by step, in a transparent and "discussable" manner.
Do you have questions or want to discuss more? Join our discussion on Discord (opens in a new tab)
References:
- Binius: highly efficient proofs over binary fields - https://vitalik.eth.limo/general/2024/04/29/binius.html (opens in a new tab)
- Container Isolation is not Safety https://cloudnativenow.com/features/container-isolation-is-not-safety (opens in a new tab)
- Container isolation gone wrong - https://sysdig.com/blog/container-isolation-gone-wrong (opens in a new tab)
- How secure is data stored in the cloud? - https://thesciencebehindit.org/how-secure-is-data-stored-in-the-cloud (opens in a new tab)
- Join-Accumulate Machine: A Semi-Coherent Scalable Trustless VM - https://graypaper.com (opens in a new tab)
- Making Containers More Isolated: An Overview of Sandboxed Container Technologies - https://unit42.paloaltonetworks.com/making-containers-more-isolated-an-overview-of-sandboxed-container-technologies (opens in a new tab)
- Polkadot: Vision for a heterogeneous multi-chain framework - https://assets.polkadot.network/Polkadot-whitepaper.pdf (opens in a new tab)
- Polkadot's JAM Chain https://wiki.polkadot.network/docs/learn-jam-chain (opens in a new tab)
- The Big Cloud Exit FAQ - https://world.hey.com/dhh/the-big-cloud-exit-faq-20274010 (opens in a new tab)
- The Father of Web3 Wants You to Trust Less - https://www.wired.com/story/web3-gavin-wood-interview (opens in a new tab)
- Trusted Execution Environment (TEE) https://learn.microsoft.com/en-us/azure/confidential-computing/trusted-execution-environment (opens in a new tab)
- What is a Trusted Execution Environment (TEE)? - https://www.halborn.com/blog/post/what-is-a-trusted-execution-environment-tee (opens in a new tab)