Speed, Always: Sharding with Shardeum 

The issue of scalability is one many blockchains have been suffering from for years now. A solution called ‘sharding’ has recently gained traction in the crypto world. Creating shards of blockchains and letting them work on adding a separate group of transactions each to the main blockchain are becoming popular. There are different sharding types, and learning about them will give you an idea of how to scale

What is Blockchain Sharding?

Sharding is a relatively new approach taken by the developers of various blockchains to facilitate better scalability. The concept here is to make a team of nodes, called shards, to process several transactions.

Let’s understand it this way, what if your computer is put through multiple rigorous tasks? Wouldn’t it be better if these tasks were assigned to multiple threads in the processor? Similarly, scaling a blockchain means increasing the number of blocks, which means more transactions.

Multiple types of sharding blockchain focus on adding transactions to a ledger efficiently and faster. Shards synchronize with each other to create a unified ledger which provides a stronghold to the blockchain. With the help of multiple sharded blockchain types, the true goal of making blockchain technology global and processing large amounts of transactions can be achieved. There are multiple sharding types involved in the crypto world, and understanding them will hopefully help you better conceptualize blockchain structures as a whole.

Horizontal and Vertical Sharding

Horizontal Sharding

To execute blockchain sharding, larger databases and their rows and columns are split into the smaller ones. Similarly, larger data tables are broken down to small ones and added to sharded tables. Whenever each added table has the same schema but every row is unique, it is called horizontal sharding.  More machines are added to the existing stack to increase the processing speed, share the load and support more traffic.

Vertical Sharding

As opposed to horizontal sharding, vertical sharding includes addition of different schema to each new table. This new schema is a faithful subset of the original one. Vertical sharding is done when queries return only a subset of the columns of the data.

Types of Sharding

Network Sharding

Network sharding is one of the types of sharded blockchains that creates better communication between the nodes. This means that the shards process transactions while properly syncing with each other. Even though consensus is performed in each shard with parallel processing of transactions, nodes still store a local copy of the whole ledger. It improves the ledger’s throughput but it runs into the typical problem of vertical scaling over time where the cost of operating the network is still high resulting in increasing gas fees once maximum throughput is throttled. Also to be noted, with vertical scaling, a sybil attack by a bad actor increases.

Transaction Sharding

Amongst various types of sharding blockchain, transaction sharding means that the ledger is stored in all the nodes but what gets distributed in shards is the addresses structure. This helps in improving the transaction speeds in the same shard. Practically, it is more or less the same as that of network sharding in terms of achieving increased throughput or TPS without sharding the global state. Further, transaction sharding mostly allows for atomic and cross shard composability which is beneficial for developers deploying smart contracts and building applications on the sharded platforms for a great UX and UI.

State Sharding

Amongst all the types of sharded blockchain, the state-sharded one was considered the most challenging until recently. So let’s spend a bit of time understanding it. Here the complete state is divided into different shards. It helps in making the verification and processing of transactions faster, secure and more accurate compared to network sharding. State sharding, in a nutshell, helps to utilize network resources way more efficiently reducing the storage overhead for node operators. State sharding, in general, allows for higher scalability and reasonable bandwidth requirement.

However, it requires the shards to constantly stay in touch and be faster with their functions because each shard here will store only a portion of the state ledger. A constant rotation of validator nodes mitigates sybil or 51% attack however rotating the nodes results in relatively high latency as new nodes introduced in the transaction validation process would take some time to download and sync to the state of the shards they are involved in.

This also means transactions are processed sequentially as the blockchain will have wait for minimum number of nodes to create and join a new shard. Further, state sharding, in and itself, does not guarantee atomic and cross shard composability making it less reliable for developers building their applications on such blockchain networks unless there are specific atomic and cross-shard composability protocols integrated to address those incapabilities.

Static State Sharding

Static state sharding is similar to state sharding. It is a type of sharding where the partitioning of the state into shards, is done in a static, predefined manner. It evidently does not change dynamically based on the current state or load of the network. Although static state sharding is less complex to implement when compared to dynamic state sharding, it also means the overall scalability and efficiency are limited since available resources cannot be adjusted in response to changes in network conditions.

Transactions, in this type of sharding, can only be processed sequentially after a minimum number of nodes join the network to create a new shard. Further, if a blockchain network using static state sharding does consensus at a block level, sequential processing of transactions is the most likely outcome. Like state sharding, static state sharding also does not guarantee atomic and cross-shard composability.

Dynamic State Sharding

Dynamic state sharding and the first network to introduce the same is explained below for your further information. It will also help you understand how dynamic state sharding will enable linear scalability for the first time in a blockchain network.

Examples Of Sharding

When it comes to operations using a blockchain, every user wants their transactions to be completed faster and in the most efficient manner. But due to high traffic congestion, bandwidth and network latency can take a serious hit. This is where different blockchain sharding types help the users. The scalability of a blockchain network improves with the help of sharding. 

Partitioned sharding and state sharding are two great examples of sharding currently. Partitioned sharding is applicable where the shards do not communicate directly. On the other hand, state sharding brings direct communication between the shards with a central or state relay.

1. NEAR Sharding

NEAR sharding is a good example of a real time sharding experience for the clients. It focused on developing the first ever experience to run real-time cross shard transactions without increasing the load on the clients. This type of sharded blockchain focused on enabling any low end device to work as a node for the network. Proof of validity is another feature of the NEAR sharded blockchain network. Blocks must have a proof of validity, and the network validators are responsible for ensuring the proof if available.

2. Ethereum Beacon Chain

For the new Casper proof of stake system of the Ethereum network, the Ethereum beacon chain turns out to be the master chain. It is another example of blockchain sharding types. Beacon chain is an integral part of the roadmap to Ethereum 2.0. Tasks like processing crosslinks, adding storage to the network, and the maintenance of validators are given to the Beacon chain. The whole chain supports the processing of Ethereum block by block consensus right beside the finality gadget. It also manages other small sharded chains that support the Ethereum network.

3. Dynamic State Sharding by Shardeum

Shardeum’s auto-scaling feature allows the network to automatically adjust the number and size of shards based on the current workload. This allows the system to optimize performance and maintain high levels of scalability as it grows and evolves.

Static state sharding only enables a blockchain network to have static/pre-defined group of shards and transactions can only be processed sequentially after a minimum number of nodes join the network to create a new shard. There are two issues at play here. For one, this does not allow for growing dynamically in proportion to the demand in the network. Two, sequential processing slows down the network as a result of high latency arising from the time taken for new nodes added to sync to the latest state of shards they are involved in.

Dynamic state sharding, on the other hand, allows validator nodes on Shardeum to be assigned dynamic address ranges across multiple shards. Unlike static state sharding where all the nodes in a shard cover the same address range of an account, dynamic state sharding requires each node to hold a different address range, but there is significant overlap between the addresses covered by nodes in respective shards. Since consensus is done at the transaction level, a transaction that affects multiple shards will be processed simultaneously by these shards rather than sequentially as with block level consensus. This not only reduces the time to process the transaction even if it affects multiple shards, but also ensures atomic processing. Shardeum will further ensure cross shard composability.

Through dynamic state sharding, every node added to the network will increase the transaction throughput instantly. So basically, by simply adding more nodes from the network’s ‘standby’ validator pool during peak demand, the TPS will increase proportionally making Shardeum the first Web3 network to scale linearly. And this is the main X factor that impacts every other outcome on a blockchain network favorably, including throughput, decentralization, security and constant transaction fees irrespective of the demand in the network.

That’s how Shardeum’s underlying protocol, Shardus, was able to demonstrate 500 TPS with 100 nodes over the last 3 years. Shardeum is aiming to enable up to 1 TPS or more per node which will be a huge breakthrough for the Web3 ecosystem. Even with 2k active nodes on existing blockchain networks, we can only see them process an average of 350 TPS at best when Web2 peers like PayPal and Visa process an average of 5000 TPS every day. Shardeum’s idea is to mobilize millions of nodes eventually which can process over 1 million TPS to realistically host a ton of dApps providing products and services to billions of users thereby eliminating middlemen taking advantage of our data/privacy.

Shardeum Maintains Low Gas Fees Forever

One key advantage with Shardeum is that the consensus and processing are done at transaction level and not at the block level. And, through dynamic state sharding, the network will shard its state by evenly and dynamically distributing compute workload, storage, and bandwidth among all the nodes. This not only allows for parallel processing of transactions but also very low overhead for validator nodes as they will store only the state data of transactions they are involved in.

This is how Shardeum will get to maintain low transaction fees for developers and end users perpetually. Just to be clear, dynamic state sharding is the most advanced version of state, transaction/network and static state sharding employed by more recent sharded chains which run into both inter-related and standalone problems such as high latency, vertical scaling (as opposed to linear scaling), sybil attack, weak finality and lack of cross shard composability. That said, dynamic state sharding is also the most complex way to shard the state of a network.

Shardeum: A Unique Blockchain for the Future

Shardeum pioneers a new approach with sharding. With dynamic state sharding on top of its EVM compatibility, Shardeum will be truly capable of handling multitude of DApps and layer 2 solutions across industries and consumers with optimum performance. It will indeed maneuver in unchartered territories in the blockchain space. Otherwise, it will not be aiming for 1 million TPS with high decentralization and solid security. 

However, that only adds to the excitement of finally seeing another blockchain that builds on the great architecture of Ethereum and its EVM.