Chapter 01

What Is Blockchain Technology?

Blockchain is a type of ledger technology that stores and records data.

Blockchain is the buzzword that seems to dominate any conversation about the future of technology, from the power of cryptocurrencies to new forms of cybersecurity. While the applications for the technology seem endless, not many people are entirely sure what the blockchain is. 

In the old days, transactions were tracked in written ledgers and stored in financial institutions. Traditional ledgers could be audited, but only by those with privileged access. Blockchain took these concepts and democratized them by removing the secrecy around how information – namely transaction data – was handled. 

In its simplest form, a blockchain is a distributed list of transactions that is constantly updated and reviewed. Also known as distributed ledger technology (DLT), it can be programmed to record and track anything of value across a network spread around multiple locations and entities. This creates a sort of worldwide spider web of connected computers. 

While often associated with cryptocurrencies, blockchain technology is not exclusive to the digital asset market. Thanks to its unique ability to add and store data, it can serve many other functions across a range of industries. 

What does a blockchain look like?

A blockchain can be broken down into two components: the block and the chain

A block is a collection of data that is linked to other blocks chronologically in a virtual chain. You can think of a blockchain as a train consisting of multiple carriages connected in a line, where each carriage contains an amount of data. Just like with passengers in a real-life train carriage, blocks can fit only a certain amount of data before they’re full.

Each block also contains a timestamp, and so it’s clear when the data was recorded and stored – something that’s vital for things like transaction or supply chain data where knowing exactly when a payment or package was processed is important.

How many copies are there?

There is not a single master copy of a blockchain. Instead, every person who runs a computer that contributes to the network – also known as a “node” – maintains their own copy of the blockchain, and constantly checks with other nodes to make sure everyone has the same record of data. By having each individual contributor store their own copy, it means there is no single point of failure. This impressive layer of security also means it’s virtually impossible for malicious agents to tamper with the data stored on blockchains. 

If a hacker group wanted to manipulate any transaction on a blockchain, they would have to break into the device of every single network contributor around the world and change all records to show the same thing.

Map of the United States
( Kat72/iStock/Getty Images Plus)

Unlike a database of financial records stored by traditional institutions, the blockchain is completely transparent and aims to be distributed, shared across networks, and in many cases, fully public. By prioritizing transparency around transactions and how the information is stored, the blockchain can act as a single source of truth.

How is data added to a blockchain?

Beyond being transparent with data, the blockchain is also a secure way to store it. Using Bitcoin as an example, here’s how a transaction is added to a new block:

When a bitcoin user sends a transaction, a message is created with both the sender’s and the receiver’s public addresses and the amount being transacted. The sender takes this data, adds their private key to the mix and then creates a hash of it (turns it into a fixed-length code.) This creates a digital signature to confirm the person who owns the amount of bitcoin intends to send it to the receiver.

The sender then packages this digital signature with the message and their own public key and broadcasts it to the network. It’s kind of like saying, “Hey, everyone! I want to send this person bitcoin.”

(Note: For most wallets and other applications, all this happens “under the hood” and users don’t have to actually deal with the processes themselves.)

The packaged transaction joins a waiting room filled with other unconfirmed transactions looking to be added to the blockchain, known as a “mempool.”

In the case of the Bitcoin network, miners who have successfully discovered new blocks through proof-of-work then take a batch of transactions from the mempool (usually based on which ones have the highest fees attached), verify each transaction to make sure each sender actually has the amount of bitcoin in their wallets they want to send, run it through software to make sure the packaged data (digital signatures, messages and public keys) are legitimate, add it to the new block and finally broadcast the proposed new block to the network so that other miners can double-check everything is correct.

This is similar to the process used in proof-of-stake blockchains, except instead of mining nodes discovering and verifying transactions, users who have locked away an amount of cryptocurrency – known as “stakers” or “validators” – carry out the process.

Nodes can perform a variety of tasks. These include keeping a historical record of all transaction data, verifying transactions, and, in the case of mining nodes or validator nodes, adding new blocks to the blockchain. Once a transaction has been approved and added, the information cannot be altered or rewritten. That is why data stored on a blockchain network is described as “immutable.”

The blockchain simply records every transaction that has ever taken place on its network. For example, the Ethereum blockchain is a record of all ether transactions that have ever taken place. So if there are updates that need to be made around a previous transaction, rather than going back to the initial data, a new record is made about the change. 

Other blockchain technology use cases

The blockchain removes the need for intermediaries like banks. The peer-to-peer network cuts out the middleman and allows transactions to be secure, cutting down on costs, and can be reviewed by anyone.

Beyond being used for finances, blockchain technology has many other functions. Hospitals are integrating the blockchain to help track medical record data and improve their accuracy. Agricultural firms use it logistically to track the supply chain of food. Smart contracts rely on it to keep a record of all agreements and state changes. More recently, it has become a means to trade, sell and authenticate original digital pieces of art.

Blockchains are becoming an increasingly important part of how we live, work and interact with our digital information. Like with every other new, revolutionary technology, there is no one set of standards, and the overall impact is still being discovered. But there’s no doubt it is here to stay.

Disclosure
The leader in news and information on cryptocurrency, digital assets and the future of money, CoinDesk is a media outlet that strives for the highest journalistic standards and abides by a strict set of editorial policies. CoinDesk is an independent operating subsidiary of Digital Currency Group, which invests in cryptocurrencies and blockchain startups.

Chapter 02

How Does Blockchain Technology Work?

As stated in our guide “What is Blockchain Technology?”, there are three principal technologies that combine to create a blockchain. None of them are new. Rather, it is their orchestration and application that is new.

These technologies are: 1) private key cryptography, 2) a distributed network with a shared ledger and 3) an incentive to service the network’s transactions, record-keeping and security.

The following is an explanation of how these technologies work together to secure digital relationships.

Cryptographic keys

Two people wish to transact over the internet.

Each of them holds a private key and a public key.

The main purpose of this component of blockchain technology is to create a secure digital identity reference. Identity is based on possession of a combination of private and public cryptographic keys.

The combination of these keys can be seen as a dexterous form of consent, creating an extremely useful digital signature.

In turn, this digital signature provides strong control of ownership.

But strong control of ownership is not enough to secure digital relationships. While authentication is solved, it must be combined with a means of approving transactions and permissions (authorisation).

For blockchains, this begins with a distributed network.

A Distributed Network

The benefit and need for a distributed network can be understood by the ‘if a tree falls in the forest’ thought experiment.

If a tree falls in a forest, with cameras to record its fall, we can be pretty certain that the tree fell. We have visual evidence, even if the particulars (why or how) may be unclear.

Much of the value of the bitcoin blockchain is that it is a large network where validators, like the cameras in the analogy, reach a consensus that they witnessed the same thing at the same time. Instead of cameras, they use mathematical verification.

In short, the size of the network is important to secure the network.

That is one of the bitcoin blockchain’s most attractive qualities — it is so large and has amassed so much computing power. At time of writing, bitcoin is secured by 3,500,000 TH/s, more than the 10,000 largest banks in the world combined. Ethereum, which is still more immature, is secured by about 12.5 TH/s, more than Google and it is only two years old and still basically in test mode.

System of record

When cryptographic keys are combined with this network, a super useful form of digital interactions emerges. The process begins with A taking their private key, making an announcement of some sort — in the case of bitcoin, that you are sending a sum of the cryptocurrency — and attach it to B’s public key.

Protocol

A block – containing a digital signature, timestamp and relevant information – is then broadcast to all nodes in the network.

A realist might challenge the tree falling in the forest thought experiment with the following question: Why would there be a million computers with cameras waiting to record whether a tree fell? In other words, how do you attract computing power to service the network to make it secure?

For open, public blockchains, this involves mining. Mining is built off a unique approach to an ancient question of economics — the tragedy of the commons.

With blockchains, by offering your computer processing power to service the network, there is a reward available for one of the computers. A person’s self-interest is being used to help service the public need.

With bitcoin, the goal of the protocol is to eliminate the possibility that the same bitcoin is used in separate transactions at the same time, in such a way that this would be difficult to detect.

This is how bitcoin seeks to act as gold, as property. Bitcoins and their base units (satoshis) must be unique to be owned and have value. To achieve this, the nodes serving the network create and maintain a history of transactions for each bitcoin by working to solve proof-of-work mathematical problems.

They basically vote with their CPU power, expressing their agreement about new blocks or rejecting invalid blocks. When a majority of the miners arrive at the same solution, they add a new block to the chain. This block is timestamped, and can also contain data or messages.

Here’s a chain of blocks:

The type, amount and verification can be different for each blockchain. It is a matter of the blockchain’s protocol – or rules for what is and is not a valid transaction, or a valid creation of a new block. The process of verification can be tailored for each blockchain. Any needed rules and incentives can be created when enough nodes arrive at a consensus on how transactions ought to be verified.

It’s a taster’s choice situation, and people are only starting to experiment.

We are currently in a period of blockchain development where many such experiments are being run. The only conclusions drawn so far are that we are yet to fully understand the dexterity of blockchain protocols.

More on this point in our guides “What are Applications and Use Cases for Blockchain Technology?” and “What is the Difference Between Open and Permissioned Blockchains?”

Authored by Nolan Bauerle; images by Maria Kuznetsov

Disclosure
The leader in news and information on cryptocurrency, digital assets and the future of money, CoinDesk is a media outlet that strives for the highest journalistic standards and abides by a strict set of editorial policies. CoinDesk is an independent operating subsidiary of Digital Currency Group, which invests in cryptocurrencies and blockchain startups.

Chapter 03

What Can a Blockchain Do?

Financial institutions have financed the disruption of countless industries over the last 30 years; they have an idea of what a revolutionary technology can do to static incumbents.

So, to stay ahead of change, banks have been proactive in setting up R&D labs, building test centers and establishing partnerships with blockchain developers to fully understand the revolutionary potential of the technology.

Financial institutions were the first to dip their feet in, but academia, governments and consulting firms have also studied the technology.

All of this work is, of course, in addition to what the entrepreneurs and developers are doing, either by finding new ways to use the bitcoin or ethereum blockchains, or else creating entirely new blockchains.

This has been going on for over three years now, and the results are starting to come in.

While some of the waters are still murky, this is what we know a blockchain can do:

Establish digital identity

As discussed in our guide “How Does Blockchain Technology Work?”, the identity component of blockchain technology is fulfilled through the use of cryptographic keys. Combining a public and private key creates a strong digital identity reference based on possession.

A public key is how you are identified in the crowd (like an email address), a private key is how you express consent to digital interactions. Cryptography is an important force behind the blockchain revolution.

As stated in our guide “What is a Distributed Ledger?”, blockchains are an innovation in information registration and distribution. They are good for recording both static data (a registry) or dynamic data (transactions), making it an evolution in systems of record.

In the case of a registry, data can be stored on blockchains in any combination of three ways:

  • Unencrypted data – can be read by every blockchain participant in the blockchain and is fully transparent.
  • Encrypted data –can be read by participants with a decryption key. The key provides access to the data on the blockchain and can prove who added the data and when it was added.
  • Hashed data – can be presented alongside the function that created it to show the data wasn’t tampered with.

Blockchain hashes are generally done in combination with the original data stored off-chain. Digital ‘fingerprints’, for example, are often hashed into the blockchain, while the main body of information can be stored offline.

Such a shared system of record can change the way disparate organizations work together.

Currently, with data siloed in private servers, there is an enormous cost for inter-company transactions involving processes, procedures and cross-checking of records.

Read more on this in our guide “What are the Applications and Use Cases of Blockchains?”.

Prove immutability

A feature of a blockchain database is that is has a history of itself. Because of this, they are often called immutable. In other words, it would be a huge effort to change an entry in the database, because it would require changing all of the data that comes afterwards, on every single node. In this way, it is more a system of record than a database.

Read more on this in our guide “What is the Difference Between a Blockchain and a Database?”.

Serve as a platform

Cryptocurrencies were the first platform developed using blockchain technology. Now, people have moved from the idea of a platform to exchange cryptocurrencies to a platform for smart contracts.

The term ‘smart contracts’ has become somewhat of a catch-all phrase, but the idea can actually be divided into several categories:

There are the ‘vending machine’ smart contracts coined in the 1990s by Nick Szabo. This is where machines engage after receiving an external input (a cryptocurrency), or else send a signal that triggers a blockchain activity.

There are also smart legal contracts, or Ricardian contracts. Much of this application is based on the idea that a contract is a meeting of the minds, and that it is the result of whatever the consenting parties to the contract agree to. So, a contract can be a mix of a verbal agreement, a written agreement, and now also some of the useful aspects of blockchains like timestamps, tokens, auditing, document coordination or business logic.

Finally, there are the ethereum smart contracts. These are programs which control blockchain assets, executed over interactions on the ethereum blockchain. Ethereum itself is a platform for smart contract code.

Blockchains are not built from a new technology. They are built from a unique orchestration of three existing technologies.

Read more on this in our guide “What are the Applications and Use Cases of Blockchain Technology?”.

Authored by Nolan Bauerle; images by Maria Kuznetsov

Disclosure
The leader in news and information on cryptocurrency, digital assets and the future of money, CoinDesk is a media outlet that strives for the highest journalistic standards and abides by a strict set of editorial policies. CoinDesk is an independent operating subsidiary of Digital Currency Group, which invests in cryptocurrencies and blockchain startups.

Chapter 04

What is a Distributed Ledger?

Ledgers, the foundation of accounting, are as ancient as writing and money.

Their medium has been clay, wooden tally sticks (that were a fire hazard), stone, papyrus and paper. Once computers became normalized in the 1980s and ’90s, paper records were digitized, often by manual data entry.

Distributed Ledger

These early digital ledgers mimicked the cataloguing and accounting of the paper-based world, and it could be said that digitization has been applied more to the logistics of paper documents rather than their creation. Paper-based institutions remain the backbone of our society: money, seals, written signatures, bills, certificates and the use of double-entry bookkeeping.

Computing power and breakthroughs in cryptography, along with the discovery and use of some new and interesting algorithms, have allowed the creation of distributed ledgers.

In its simplest form, a distributed ledger is a database held and updated independently by each participant (or node) in a large network. The distribution is unique: records are not communicated to various nodes by a central authority, but are instead independently constructed and held by every node. That is, every single node on the network processes every transaction, coming to its own conclusions and then voting on those conclusions to make certain the majority agree with the conclusions.

Once there is this consensus, the distributed ledger has been updated, and all nodes maintain their own identical copy of the ledger. This architecture allows for a new dexterity as a system of record that goes beyond being a simple database.

Distributed Ledgers are a dynamic form of media and have properties and capabilities that go far beyond static paper-based ledgers. For more on this, please read our guide “What Can a Blockchain Do?” For now, the short version is they enable us to formalize and secure new kinds of relationships in the digital world.

The gist of these new kinds of relationships is that the cost of trust (heretofore provided by notaries, lawyers, banks, regulatory compliance officers, governments, etc…) is avoided by the architecture and qualities of distributed ledgers.

Our Wikipedia analogy in our guide “What is Blockchain Technology?” hints at the power of these new kinds of relationships.

The invention of distributed ledgers represents a revolution in how information is gathered and communicated. It applies to both static data (a registry), and dynamic data (transactions). Distributed ledgers allow users to move beyond the simple custodianship of a database and divert energy to how we use, manipulate and extract value from databases — less about maintaining a database, more about managing a system of record.

Authored by Nolan Bauerle; images by Maria Kuznetsov

Disclosure
The leader in news and information on cryptocurrency, digital assets and the future of money, CoinDesk is a media outlet that strives for the highest journalistic standards and abides by a strict set of editorial policies. CoinDesk is an independent operating subsidiary of Digital Currency Group, which invests in cryptocurrencies and blockchain startups.

Chapter 05

Why Use a Blockchain?

As the implications of the invention of have become understood, a certain hype has sprung up around blockchain technology.

This is, perhaps, because it is so easy to imagine high-level use cases. But, the technology has also been closely examined: millions of dollars have been spent researching blockchain technology over the past few years, and numerous tests for whether or not blockchain technology is appropriate in various scenarios have been conducted.

Blockchain technology offers new tools for authentication and authorization in the digital world that preclude the need for many centralized administrators.

Blockchain technology offers new tools for authentication and authorization in the digital world that preclude the need for many centralized administrators. As a result, it enables the creation of new digital relationships.

By formalizing and securing new digital relationships, the blockchain revolution is posed to create the backbone of a layer of the internet for transactions and interactions of value (often called the ‘Internet of Value’, as opposed to the ‘Internet of Information’ which uses the client-server, accounts and master copy databases we’ve been using for over the past 20 years.)

But, with all the talk of building the digital backbone of a new transactional layer to the internet, sometimes blockchains, private cryptographic keys and cryptocurrencies are simply not the right way to go.

Many groups have created flowcharts to help a person or entity decide between a blockchain or master copy, client-server database. The following factors are a distillation of much of what has been previously done:

Is the data dynamic with an auditable history?

Paper can be hard to counterfeit because of the complexity of physical seals or appearances. Like etching something in stone, paper documents have certain permanence.

But, if the data is in constant flux, if it is transactions occurring regularly and frequently, then paper as a medium may not be able to keep up the system of record. Manual data entry also has human limitations.

So, if the data and its history are important to the digital relationships they are helping to establish, then blockchains offer a flexible capacity by enabling many parties to write new entries into a system of record that is also held by many custodians.

Should or can the data be controlled by a central authority?

There remain many reasons why a third party should be in charge of some authentications and authorizations. There are times when third-party control is totally appropriate and desirable. If privacy of the data is the most important consideration, there are ways to secure data by not even connecting it to a network.

But if existing IT infrastructure featuring accounts and log-ins is not sufficient for the security of digital identity, then the problem might be solved by blockchain technology.

As Satoshi Nakamoto wrote in his (or her) seminal work, “Bitcoin: A Peer-to-Peer Electronic Cash System”: “Merchants must be wary of their customers, hassling them for more information than they would otherwise need. A certain percentage of fraud is accepted as unavoidable.”

Private key cryptography enables push transactions, which don’t require centralized systems and the elaborate accounts used to establish digital relationships. If this database requires millions of dollars to secure lightweight financial transactions, then there’s a chance blockchains are the solution.

Is the speed of the transaction the most important consideration?

Does this database require high-performance millisecond transactions? (There is more on this point in our guide: “What is the Difference Between a Blockchain and a Database?”).

If high performance, millisecond transactions are what is required, then it’s best to stick with a traditional-model centralized system. Blockchains as databases are slow and there is a cost to storing the data – the processing (or ‘mining’) of every block in a chain. Centralized data systems based on the client-server model are faster and less expensive… for now.

In short, while we still don’t know the full limits and possibilities of blockchains, we can at least say the use cases which have passed inspection have all been about managing and securing digital relationships as part of a system of record.

Authored by Nolan Bauerle; images by Maria Kuznetsov

Disclosure
The leader in news and information on cryptocurrency, digital assets and the future of money, CoinDesk is a media outlet that strives for the highest journalistic standards and abides by a strict set of editorial policies. CoinDesk is an independent operating subsidiary of Digital Currency Group, which invests in cryptocurrencies and blockchain startups.

Chapter 06

How Could Blockchain Technology Change Finance?

This question has been asked by every futurist research lab in many of the largest banks, central banks, financial institutions, think tanks, consulting firms and government committees around the world.

R3CEV, a consortium effort financed by some of the world’s largest banks, is busy trying to answer this question. Goldman Sachs, McKinsey Consulting and Consumers’ Research have all written excellent reports on this question. The UK Government, the Senates of the US, Canada, Australia and the EU have all made inquiries along these lines.

Many startups also produce white papers concerning their particular innovation or use of blockchain technology, and often include the larger social question: “How this will change things?”

Much of this research underlines four major areas of change:

Infrastructure for cross-border transactions

The digital revolution has totally transformed media, as we all know. It’s had an effect in the finance industry as well. Of course, financial institutions use computers. They used them for databases in the 1970s and 1980s, they made web pages in the 1990s and they migrated to mobile apps in the new millennium.

But the digital revolution has not yet revolutionized cross-border transactions. Western Union remains a big name, running much the same business they always have. Banks continue to use a complex infrastructure for simple transactions, like sending money abroad.

The following infographic, prepared by Richard Gendal Brown, shows the infrastructure and intermediaries in cross-border banking that have been in place since the ’70s.

This architecture is the result of the finance industry using highly secured private databases. Digitization has meant we merely sort information into private databases much faster.

Blockchain technology allows for financial institutions to create direct links between each other, avoiding correspondent banking. R3’s principal product to date, Corda, aims at correspondent banking. Corda is a play on words incorporating ‘accord’ (agreement) and ‘cord’ (the straightest line between two points in a circle).

In Corda’s case, the circle is made up of banks who would use a shared ledger for transactions, contracts and important documents.

Brown used to work on IBM’s blockchain products, but has since moved over to work at R3CEV.

Competing financial institutions could use this common database to keep track of the execution, clearing and settlement of transactions without the need to involve any central database or management system. In short, the banks will be able to formalize and secure digital relationships between themselves in ways they could not before.

In the above representation, that means correspondent banking agreements and the RTGS could both be shortcutted.

Transactions can occur directly between two parties on a frictionless P2P basis. Ripple, a permissioned blockchain, is built to solve many of these problems.  

Digital assets as a class

Bitcoin created something unique: digital property.

Before bitcoin, ‘digital’ was not synonymous with scarcity. Anything digital could be copied with the click of a button. A quick look at the music industry and album sales tells this story convincingly.

But bitcoin did something new: it created uncopyable digital code.

So, for the first time since bits and bytes were invented, there was a way to own something digital that couldn’t be copied. This gave the digital code value. To this day, bitcoin’s value is based on the capacity of its blockchain to prevent double-spending and the creation of counterfeit coins.

With this in mind, bitcoin developers have pioneered coloured coins that can act as stock in a company. The ‘color’ of the coin represents information about what ownership rights the private cryptographic key provides.

After receiving SEC permission, online retail giant Overstock announced it would issue public shares of company stock on its tØ blockchain platform. We’ve also seen the advent of ‘initial coin offerings’ (ICOs) and ‘appcoins’ (cryptocurrencies native to an app that help fund development of the project).

These examples are only part of the story for blockchains in digital assets. They can be the asset, but blockchains can also be used to run the market itself.

Basically, these efforts are treating digital assets as a bearer instrument, which is a wide and dexterous application.

Governance and markets

This ability, however, extends beyond just recording transactions. Nasdaq, for example, was one of the first to build a platform enabling private companies to issue and trade shares using a blockchain.

Other developers are coding financial instruments that can be pre-programed to carry out corporate actions and business logic.

In 2016, a blockchain project called The DAO, running on the ethereum blockchain, was launched with the aim of emulating a crowdfunding market. Your percentage of contribution to the fund represented the percentage vote in how the total fund would be spent.

Regulatory reporting and compliance

Blockchains can serve as a fully transparent and accessible system of record for regulators. The can also be coded to authorize transactions which comply with regulatory reporting.

For example, banks have severe reporting obligations to agencies such as FinCEN. Every single time they authorize a transaction of more than $10,000, they must report the information to FinCEN, who stores it for use as an anti-money laundering database.

Clearing and Settlement

With paper-world trading, the time frame for clearing and settlement of a transaction is generally referred to as ‘T+3’ – that is, three days after the trade (T), the transaction is settled.

With blockchain technology, the entire lifecycle of a trade – execution, clearing and settlement – occurs at the trade stage. With a digital asset, trade is settlement, and the cryptographic keys and digital ownership they control can lower post-trade latency and counterparty risk.

Accounting and auditing

Whereas most databases are snapshots of a moment in time, blockchain databases are built from their own transaction history. They are a database with context, a history of itself, a self-contained system of record.

The implications for auditing and accounting are profound.

Authored by Nolan Bauerle

Disclosure
The leader in news and information on cryptocurrency, digital assets and the future of money, CoinDesk is a media outlet that strives for the highest journalistic standards and abides by a strict set of editorial policies. CoinDesk is an independent operating subsidiary of Digital Currency Group, which invests in cryptocurrencies and blockchain startups.

Chapter 07

What are Blockchain's Main Limitations?

Blockchain technology provides a robust method for building a decentralized database, but it’s not without its problems. 

Compared with traditional server-based systems where data is centrally managed, public blockchains – the most common type of blockchain used by cryptocurrencies – rely on a distributed network of volunteer contributors to store information, add new blocks, verify transactions and secure the ledger. 

This communal approach to recording data benefits transparency, decentralization and accessibility, but it also requires a huge amount of coordination, energy and complex technology to work as intended. Not to mention, the technology is still developing and very much in its infancy, and constantly being tested by an ever-increasing number of users.

Network size and scalability

One major challenge that impacts certain blockchains is scaling to meet the demand of new users, especially for public blockchains (blockchains that are openly accessible to anyone.) 

As of mid-2021, Bitcoin can only process 2.4 transactions per second (down from 7 in 2016), and each transaction costs around $2.46 (up from $0.20 in 2016). Due to the 1-megabyte limit on block size, the Bitcoin ledger is inherently difficult to scale. This is why secondary scaling solutions – ways to improve efficiency outside the blockchain – are being implemented, namely the Lightning Network and Segregated Witness (SegWit.)

Processing large quantities of transactions is another key component of the blockchain, but unlike their centralized counterparts, decentralized systems rely on the processing power of distributed users. Depending on how many volunteer contributors there are running their computers, known as “nodes,” it can drastically impact how quickly the network can operate. 

Hands placing blocks
(Shutterstock)

The Ethereum network was tested in late 2017 when CryptoKitties – digital cats belonging to a blockchain-based NFT game of the same name – became viral commodities. At their peak, these unique digital assets built on the Ethereum blockchain accounted for around 10% of network traffic. At that time, the network could only handle approximately 15 transactions per second, causing it to become backlogged and miners requiring more “gas” – the fee charged to conduct a successful transaction on Ethereum’s blockchain. This laid bare blockchain’s limitations to scale first-hand. 

A blockchain’s security is also only as strong as its network of distributed voluntary contributors who run their computers to maintain the network – known as “nodes.” If a blockchain does not have a robust network with a widely distributed grid of nodes, it becomes far more vulnerable to malicious activity, such as 51% attacks (see below.)

Transaction costs and network speed

The underlying cost of using a blockchain-based platform is also another pain point for the technology. Some cryptocurrencies, such as ether and bitcoin, can require users to pay exorbitant fees in order to have transactions processed in a timely manner. In February 2021, the average ether transaction cost $20 to process. This is a far cry from the fees charged by modern platforms such as Solana, which are typically $0.0000012 per transaction.

Even if there weren’t high costs of using some blockchains, due to their complexity and distributed nature, blockchains can often be slow during busy times. When there are more users sending transactions over the network, it becomes congested and payments can take longer to process than normal. With blockchains that rely on miners to validate transactions, high congestion usually leads to higher fees as people pay more to get their transactions processed faster.

Unavoidable security flaw

Many blockchain advocates preach the technology is practically uncompromisable. But there is one notable security flaw in bitcoin and other blockchains: if more than half of the computers working as network nodes tell a lie, the lie will become the truth. This is called a ‘51% attack’ – when malicious attackers gain control of more than 50% of a network’s blockchain; a problem foreshadowed by Satoshi Nakamoto when they published the Bitcoin white paper in 2008.

Bitcoin mining machines in a former steel mill in the midwest.
(Luxor Technologies)

To limit the possibility of this occurring, mining pools are closely monitored by the community to ensure no one gains outsize influence over the network. This means blockchains with fewer participants are more significantly more vulnerable to these types of attacks. Bitcoin SV – a fork of Bitcoin – was recently the victim of a 51% attack that saw over 100 blocks get removed from the blockchain, affecting over 570,000 transactions.

Blockchain politics

Because many blockchains allow contributors to vote on the direction and development of their protocols, and because miners and other types of validators form their own groups, there have been ample opportunities for public disagreements between different communities. For example, as Ethereum migrates from a mining-based proof-of-work blockchain to a proof-of-stake one, part of its 2.0 upgrade involves phasing out miners. However, since miners still make up a significant portion of Ethereum’s community many have tried to stall or block efforts to do so.

These disagreements are a notable feature of the blockchain industry and are expressed most clearly around the question or event of “forking” a blockchain; a process that involves splitting a blockchain into two separate blockchains. This usually occurs when a major update isn’t backward-compatible, or when developers fundamentally disagree on how to go forward, leading to the project forking into two separate entities. A classic example of this was when Bitcoin split in 2017 due to internal conflict between developers, leading to the creation of Bitcoin Cash.

Energy consumption

Recently, awareness has been spreading about how much energy is used to power blockchains. Elon Musk, CEO of electric car manufacturer Tesla, voiced his concerns in May 2021 about the impact bitcoin mining was having on the environment, stating Tesla would no longer accept payment in bitcoin until the industry becomes sufficiently greener. Around the same time, China began aggressively enforcing a ban on crypto mining, forcing hundreds of network validators to relocate to other countries. The nation cited climate change concerns as one of its primary motives for the renewed crackdown.

Most blockchains require large amounts of energy – most notably proof-of-work (PoW) blockchains, including Bitcoin and many other first-generation blockchains. This system requires electricity-guzzling computers to compete against each other to discover blocks and add new batches of transaction data to the blockchain, which other computers can then verify. The computers necessary to run these computations are often powerful, and with that power comes energy consumption. 

As more people invest in cryptocurrencies and as businesses adopt blockchain technology, the corresponding energy use has become an environmental and political issue. In fact, if Bitcoin were a country, it would rank as the 38th largest consumer of electricity in the world, ranking higher than Chile (as of Aug. 6, 2021). 

There are more efficient blockchain verification methods like proof-of-stake (PoS) that involve randomly assigning the task of validation to a node, and users locking away a portion of crypto assets in the network. But this would mean a large restructuring of many established blockchains.

Disclosure
The leader in news and information on cryptocurrency, digital assets and the future of money, CoinDesk is a media outlet that strives for the highest journalistic standards and abides by a strict set of editorial policies. CoinDesk is an independent operating subsidiary of Digital Currency Group, which invests in cryptocurrencies and blockchain startups.

Chapter 08

Hard Fork vs Soft Fork

Last updated: 16 March, 2018

Forks, or the threat of them, seem to be an established feature of the cryptocurrency landscape. But what are they? Why are they such a big deal? And what is the difference between a hard fork and a soft fork?

A “fork,” in programming terms, is an open-source code modification. Usually the forked code is similar to the original, but with important modifications, and the two “prongs” comfortably co-exist. Sometimes a fork is used to test a process, but with cryptocurrencies, it is more often used to implement a fundamental change, or to create a new asset with similar (but not equal) characteristics as the original.

Not all forks are intentional. With a widely distributed open-source codebase, a fork can happen accidentally when not all nodes are replicating the same information. Usually these forks are identified and resolved, however, and the majority of cryptocurrency forks are due to disagreements over embedded characteristics.

One thing to bear in mind with forks is that they have a “shared history.” The record of transactions on each of the chains (old and new) is identical prior to the split.

Hard forks

There are two main types of programming fork: hard and soft.

A hard fork is a change to a protocol that renders older versions invalid. If older versions continue running, they will end up with a different protocol and with different data than the newer version. This can lead to significant confusion and possible error.

With bitcoin, a hard fork would be necessary to change defining parameters such as the block size, the difficulty of the cryptographic puzzle that needs to be solved, limits to additional information that can be added, etc. A change to any of these rules would cause blocks to be accepted by the new protocol but rejected by older versions and could lead to serious problems – possibly even a loss of funds.

For instance, if the block size limit were to be increased from 1MB to 4MB, a 2MB block would be accepted by nodes running the new version, but rejected by nodes running the older version.

Let’s say that this 2MB block is validated by an updated node and added on to the blockchain. What if the next block is validated by a node running an older version of the protocol? It will try to add its block to the blockchain, but it will detect that the latest block is not valid. So, it will ignore that block and attach its new validation to the previous one. Suddenly you have two blockchains, one with both older and newer version blocks, and another with only older version blocks. Which chain grows faster will depend on which nodes get the next blocks validated, and there could end up being additional splits. It is feasible that the two (or more) chains could grow in parallel indefinitely.

This is a hard fork, and it’s potentially messy. It’s also risky, as it’s possible that bitcoins spent in a new block could then be spent again on an old block (since merchants, wallets and users running the previous code would not detect the spending on the new code, which they deem invalid).

The only solution is for one branch to be abandoned in favor of the other, which involves some miners losing out (the transactions themselves would not be lost, they’d just be re-allocated). Or, all nodes would need to switch to the newer version at the same time, which is difficult to achieve in a decentralized, widely spread system.

Or, bitcoin splits, which has happened (hello, bitcoin cash).

Soft fork

A soft fork can still work with older versions.

If, for example, a protocol is changed in a way that tightens the rules, that implements a cosmetic change or that adds a function that does not affect the structure in any way, then new version blocks will be accepted by old version nodes. Not the other way around, though: the newer, “tighter” version would reject old version blocks.

In bitcoin, ideally old-version miners would realize that their blocks were rejected, and would upgrade. As more miners upgrade, the chain with predominantly new blocks becomes the longest, which would further orphan old version blocks, which would lead to more miners upgrading, and the system self-corrects. Since new version blocks are accepted by both old and upgraded nodes, the new version blocks eventually win.

For instance, say the community decided to reduce the block size to 0.5MB from the current limit of 1MB. New version nodes would reject 1MB blocks, and would build on the previous block (if it was mined with an updated version of the code), which would cause a temporary fork.

This is a soft fork, and it’s already happened several times. Initially, Bitcoin didn’t have a block size limit. Introducing the limit of 1MB was done through a soft fork, since the new rule was “stricter” than the old one. The pay-to-script-hash function, which enhances the code without changing the structure, was also successfully added through a soft fork. This type of amendment generally requires only the majority of miners to upgrade, which makes it more feasible and less disruptive.

Soft forks do not carry the double-spend risk that plagues hard forks, since merchants and users running old nodes will read both new and old version blocks.

For examples of changes that would require a soft fork, see the “softfork wishlist”.

You can follow CoinDesk’s news and updates on cryptocurrency forks here.

Authored by Noelle Acheson; Fork image via Shutterstock

Disclosure
The leader in news and information on cryptocurrency, digital assets and the future of money, CoinDesk is a media outlet that strives for the highest journalistic standards and abides by a strict set of editorial policies. CoinDesk is an independent operating subsidiary of Digital Currency Group, which invests in cryptocurrencies and blockchain startups.

Chapter 09

What is SegWit?

In order for Bitcoin to achieve global adoption, it has to be able to accommodate billions of users. SegWit is part of the solution.

SegWit (short for Segregated Witness) is a protocol upgrade known as a “secondary scaling solution” designed to solve Bitcoin’s scalability issues. Scalability, in this sense, refers to how well a blockchain protocol can adapt to an increase in the number of users. 

On its own, the Bitcoin network is greatly limited to how many transactions it can process per second. What SegWit does is change the way data is stored on the Bitcoin blockchain, opening the doors to a number of improvements.

Who developed SegWit?

The concept behind SegWit – removing witness data from the main transaction block – had been around a number of years before the feature was eventually implemented in Bitcoin’s source code. In 2012, a group of Bitcoin Core developers – people working on the leading client software used to connect with Bitcoin’s network – discussed the benefits of such a feature but couldn’t see a way to effectively implement it at the time.

In 2013, developer Greg Maxwell and cryptographer Adam Back revisited the feature and continued to work on a solution. A year later, they founded a new for-profit Bitcoin development company called Blockstream, with several other Bitcoin Core developers. In 2015, Blockstream released the first version of SegWit on a sidechain project called Elements.

Pieter Wuille, one of the Bitcoin Core developers to co-found Blockstream, went on to present the idea at the Scaling Bitcoin conference in December of the same year. However, it took a another two years and an intense debate before SegWit was activated on Litecoin, followed eventually by Bitcoin in August 2017.

What are the benefits of SegWit?

Transaction malleability flaw

The motivation for SegWit was to fix a long-standing bug in Bitcoin’s code called “transaction malleability.” 

Transaction malleability was a flaw that allowed anyone to tamper with a transaction’s unique ID (and subsequent hash) before it was confirmed on the Bitcoin network. As a result, it made it possible for someone to pretend that transactions didn’t happen. While it was not a critical problem for Bitcoin per se, it prevented the development of more complex features such as second-layer protocols and more robust smart contracts.

SegWit solves transaction malleability by removing the signature information (otherwise known as the “witness” information) and storing it outside the main transaction block (also known as the base transaction block). Because signature information accounts for about 65% of a transaction’s data, it also significantly streamlined how much information could fit into a block. 

Cheaper bitcoin transaction fees

A side benefit of separating the signature information from the main transaction block is that bitcoin transactions “weigh less,” because there is less data to store. That means as many as 2,700 transactions can fit into a single block, up from around 1,650 before. As a result, SegWit bitcoin transaction fees are much lower and bitcoin miners also benefit from being able to process more transactions per block.

Part of the SegWit upgrade also included the introduction of a concept called “block weight.” This technically increased Bitcoin’s 1 megabyte block size limit to a theoretical 4 MB, comprising 1 MB of transaction data and 3 MB of signature data. But because signature data isn’t stored in the main base block and the space for transactions is still the same (1 MB), it meant SegWit was seamlessly compatible with the existing protocol and did not require a hard fork (when a non-backward-compatible upgrade requires all users to upgrade their software to a new version in order to interact with the main chain).

In order to benefit from SegWit’s lower fees, users need to make sure they’re using a SegWit-enabled wallet and address. An easy way to see if your current wallet address is a SegWit one or a legacy one is by looking at the first digit. Legacy addresses (non-SegWit-enabled addresses) start with “1,” while SegWit-enabled addresses start with either “bc1” or “3.”

When choosing a bitcoin wallet, some providers such as BitPay, Electrum and Trezor allow users to pick between creating a “SegWit” or “Simple/Legacy” wallet. 

Ledger, a hardware wallet provider, has its address format set to SegWit by default, meaning that unless users specifically opt for a legacy wallet, their Ledger addresses will support SegWit payments.

Second-layer protocols

Another big step forward made possible by SegWit is that it supported the development of second-layer protocols – new platforms built on top of a blockchain – such as the Lightning Network. 

The Lightning Network was designed specifically to allow cheap, fast and private microtransactions through peer-to-peer payment channels. When two people want to transact with each other using the Lightning Network, they open a bi-directional payment channel and both deposit bitcoin into a “2-of-2” multisig address (a wallet that requires both parties’ private keys to authorize payments). The initial payment to open the payment channel takes place on-chain; that is, it is validated by miners and stored on the Bitcoin blockchain, as usual.

Once the channel is open, however, any transactions sent between the two users are recorded on a private ledger maintained by both parties. Because the transactions happen off-chain (away from the main Bitcoin blockchain), transactions don’t need to be validated every time by miners. That means transactions are nearly instant and free. Each time a transaction is sent via the payment channel, the state of both users’ ledgers is updated to reflect the new balances. When both parties agree to close the channel, they sign a “settlement transaction,” which contains the final balance of both addresses. That single transaction is then recorded on Bitcoin’s blockchain. 

But it wasn’t just the Lightning Network that SegWit activation paved the way for. Taproot, a Bitcoin upgrade that improves bitcoin transaction privacy through the use of Schnorr signatures, was also another development made possible through the implementation of SegWit. 

Schnorr signatures are faster and smaller than Bitcoin’s current signature algorithm, known as the “elliptic curve digital signature algorithm” (ECDSA), and can support more complex smart contracts (pieces of self-executing computer code that perform certain functions when predefined conditions are satisfied). Because the signatures are smaller, complex transactions involving multiple signatures will be optimized, meaning transaction fees will be reduced further.

In July 2021, Taproot received sufficient mining support to initiate its activation, with over 99% of blocks mined, signaling approval during the upgrade’s three-month “Speedy Trial” phase.

Why was SegWit controversial?

Not everyone in the Bitcoin community agreed SegWit was the proper solution for Bitcoin. Some believe it was a case of “kicking the can down the road,” and at best, a temporary fix. 

Roger Ver, the former CEO of Bitcoin.com, was an avid believer that Bitcoin block sizes needed to be increased from 1 MB to enable people to do things like send microtransactions without incurring high fees and allow for higher transactions per second to make it more viable as a global electronic currency system.

In fact, the debate over SegWit was one of the driving factors behind the development of Bitcoin Cash, a fork of the Bitcoin network that implemented a larger block size limit rather than relying on a new transaction structure. Ver, formerly a loyal advocate of Bitcoin, retracted his support for Bitcoin in favor of the newly forked project.

Bitcoin miners also initially refused to activate the upgrade, for reasons still not entirely clear. Some believe it was because many were covertly using “ASICboost” – a method of improving mining efficiency by as much as 20% – which was not compatible with SegWit. 

To bypass the miners’ roadblock, an idea was put forward by a Litecoin developer known as “Shaolinfry” called a “User Activated Soft Fork” (UASF). That involved other nodes on the network activating SegWit without the miners’ support, threatening to split the network into two versions – one that supported SegWit and one that didn’t. 

During the same week the UASF was scheduled to take place, miners decided to finally activate SegWit.

Where are we now?

As of mid-August 2021, 77% of Bitcoin transactions use SegWit, and although the upgrade has not been adopted fully, it continues to grow as the standard for Bitcoin transactions and opens the door for future developments. 

With the main Bitcoin code also making SegWit-friendly adjustments, this could lead to an uptick in both usage and additional functionality experimentation. Bitcoin Core maintains a list on its website of businesses and projects working on SegWit integration.

Disclosure
The leader in news and information on cryptocurrency, digital assets and the future of money, CoinDesk is a media outlet that strives for the highest journalistic standards and abides by a strict set of editorial policies. CoinDesk is an independent operating subsidiary of Digital Currency Group, which invests in cryptocurrencies and blockchain startups.