How does Bitcoin mining work? I don't get it.

** takes deep breath **

Bitcoin mining is often simplified to the point where the definition is basically meaningless. I will try to explain it as simply as possible but first we need to unlearn some unhelpful things.

The goal of Bitcoin mining is NOT to earn bitcoin

If we start with the premise that the goal of mining is to earn bitcoin, it makes everything 100x harder to explain. It's a nuanced point however, miners DO mine to earn bitcoin, but in the grand scheme of things, the main goal of mining is not to create or earn bitcoin.

I like to think of Bitcoin as a complex clock or watch, there are many small moving parts all coming together to keep the system working flawlessly. Take out one small part and the whole thing might fall apart. The goal of mining is to provide "immutability" and "finality" to all the events that occur on the Bitcoin network.

Understanding the anatomy of Bitcoin

Before we can understand the purpose of Bitcoin mining, we must first understand a little about the anatomy of Bitcoin and some of the properties it must uphold to be useful.

Consider your bank account; there is a balance associated with your name, but what gives that data integrity? Usually it is the fact that the data is coming from an authoritative source, a trusted third party. If I could get a copy of the bank's database, would you trust that the same data sourced from my machine was correct? Could I have modified the records before showing it to you?

Bitcoin does not rely on any authority. This is a very important part of its design and it is what allows Bitcoin to remain independent of any company or government. To achieve this, it relies on checks and balances to be embedded within the data that it contains. Bitcoin is fully transparent in order to ensure that anyone can audit its data. The most important thing to check, of course, is that the books are balanced, but beyond that, Bitcoin introduced personal digital signatures to ensure that records are authorised by individuals directly, and it introduced an ordering system.

Digital signatures not only prove that a record was created by the owner of the bitcoin being spent, but if the record is ever altered, the signature no longer matches; this means that we don't need to trust anyone to execute our requests faithfully.

Order is important to ensure that everybody can agree with which payments are valid and which are not. Imagine I write two identical cheques to give all of my bank balance to two different people. One will settle, the other will bounce. With a traditional bank, one banker will decide which came first and will decide which settles and which bounces. In a decentralised (not centralised) system like Bitcoin, we need to be able to introduce a robust and reliable ordering system that does not rely on a trusted authority. Beyond just ordering, there also needs to be some integrity to the data so that not only an order can be agreed upon, but also there can be no re-ordering. Determining an order without preventing re-ordering is basically useless.

Ordering records

You may have heard of the "blockchain". This technology is just a data format, but Bitcoin uses this technology to help lock in an order into the record system. In a blockchain, records are grouped into blocks, and each block must have a link back to its previous block, creating a chain of blocks; pretty self explanatory in hindsight.

Imagine somebody deciding to become part of the Bitcoin network, they wish to get a copy of all the bitcoin records, but there is a problem:

1. One person shows a series of records that fully balances, has all the other checks and balances, and it shows a payment from me to you.
2. Another person provides a similar series of records that also fully balances but it does not show that payment from me to you, it shows a payment from me to me in its place.

Which one is correct? Both pass all checks, but if I choose the first set of records, then the payment from me to me becomes invalid because it attempts to spend money that has already been spent, while if I take the second set of records, the inverse becomes true.

The blockchain allows us to declare an order, but we still need a way to lock that order in, such that if someone provides an alternative set of records with a different order of events, we can all use some robust and reliable set of rules to choose the same record set without needing to coordinate with one another or rely on some trusted authoritative source.

This solution also needs to be resistant to being gamed; one elephant in the room is the idea of relying on dates and times, but computers are terrible at agreeing on time and although data can travel at the speed of electrons, there are still bottlenecks that can lead to data arriving in a different order between different computers.

Using a game of chance to prevent changes

The most fair way to make an arbitrary decision such as this, where there is no real right or wrong answer (remember "date sent" can be gamed, so we can't base any decisions on that), but a decision still needs to be made is by introducing something that is statistically hard to undo.

You may know the game of Boggle. This is a game where you have a set of dice inside a tray with letters on the sides instead of numbers; you place a box cover over it, shake it, and then allow the dice to fall into a mould at the bottom so that they are all sitting alongside each other in a square formation (usually 4x4) with one letter facing up per die. Let's re-imagine that the goal of the game was simply to shake the box, align the dice and remove the cover and if there was a 4 letter word on each row, you win, and if not you must shake again.

This is analogous to Bitcoin mining. It is a pure game of chance, and if we had a bunch of them, we could turn the game into a race: who can create the longest chain of winning boggle shakes in a row!

To illustrate how the blockchain creates links between blocks, we can add an additional rule that the first die in a Boggle tray (we will call them blocks going forward) must match the last die in the previous block. This way, we know that the order of blocks cannot change as the link is being built up.

Here is an illustration of a chain from this game:

Finally, let's make it more interesting: the goal is to have as many blocks as you can on the longest chain. If you notice that someone else has a pretty long chain, you can abandon your own and start to build blocks for their chain instead. If you create a block that satisfies the rules and your first letter matches the last letter of their last block, your block can go on top.

We will start to notice the following things:

1. People will quite quickly start to abandon their own chains and work on top of one faster growing chain.
2. The chain with the most people working on it will naturally grow at a faster pace.
3. Those competing too hard to maximise the number of blocks they have in a chain will see their chain fall behind the fastest growing chain.
4. It takes a non-negligable time between rounds and usually one person wins at a time, there are rarely any moments where two people create a valid block for the same chain within moments of one another.

Using forks to change history

It gets interesting when someone notices something like an offensive word in the chain, let's call it "DUCK". A lot of people don't care, but a large group of individuals do, so they choose to "fix" the rules, making that block and all the blocks after it invalid too (because they all eventually link back to the invalid block).

They create a new block that continues off from the block before the offensive one, creating what we call a "fork". This new chain will inherit the same past as the original chain, but it will still be shorter than the original chain because during the time it took to create this alternative block to replace the offensive one, the rest of the players continued to build blocks on the original chain as normal.

Since this game is mostly based on chance, the chain that grows the fastest will be the one that has the most players contributing to it. Even if the smaller group appears faster for some time, over a long enough time, statistics will prevail; it is the same as how over enough coin flips, we will observe that there really is a 50/50 chance of heads over tails, even if the first few flips were mostly all heads.

If enough players are motivated to contribute to the shorter, profanity-free chain, then that chain will grow faster and eventually become, and remain, the longest chain. The other indifferent players will then jump ship and join the new longest chain in order to maximise the number of their blocks in the new longest and faster growing chain.

Fortunately, the more likely scenario is that the offended group is a minority and the original chain remains the longest and fastest growing chain. This natural phenomenon ensures that the history of events cannot easily be changed for simply arbitrary reasons. On a global scale, this phenomenon is ever more present, because not even laws, politics, nor what is considered moral can be agreed across the world.

Cheating

In a game like the one above, there are opportunities to cheat. Maybe you could choose not to shake your box, but place the dice by hand. Unless someone is constantly watching you, no one would ever know.

With Bitcoin mining, there are simply no such shortcuts. The fastest and cheapest way to produce a block is to play fairly. Let's dive into what Bitcoin mining really is.

What exactly is Bitcoin mining?

People often use analogies to describe mining; "it's like a lottery", "miners solve complex mathematical problems", "miners guess random numbers". The problem with analogies are that they tend to omit key details that ultimately mislead the reader.

Miners perform a series of tasks repetitively, simplified:

1. Collect the records that users create and compose them into a file.
2. Add a reference to the previous block.
3. Add a random number to the end.
4. Check if the file satisfies a certain special criteria.
5. If not, replace the random number with a new random number and repeat from step 3.

To check if a file satisfies our special criteria, we pass the file through a special program called "SHA256" and the program reveals a large unpredictable unique number. We check to see if that number is below a pre-determined target and if so, it has satisfied the criteria (i.e. given SHA256(FILE1) = 123 and TARGET = 20, is 123 < 20?).

SHA256 is a known as a mathematical "hash function". Although its output is unpredictable, it is also "deterministic", which means that given the same input file, it will always produce the same number. It is unpredictable because given the same file with even a small change will produce a completely different number altogether. We use these type of algorithms to give our files identifiers that make them resistant to tampering; it is very useful when downloading from torrents, where files can be downloaded in parallel, from multiple places, be reconstructed, and the file is only accepted if it resolves to the same identifier, ensuring that you get exactly what you asked for.

These files containing records are traditionally called "blocks", and once an acceptable block has been produced (aka mined), it is shared with all the online Bitcoin nodes and we all move on to mining the next block. As we observed before, attempting to ignore the latest mined block is counterproductive and would result in a fork and put you on a path of creating blocks for a new chain that no one would care about and that everyone would simply discard as invalid data.

As you can imagine, this process is very repetitive. One round is hardly power intensive, but the chances of success are so low, that it takes many cycles to find a successful block. The process of mining is very power intensive due to the nature of the process being a repetitive process with a low chance of success.

The incentives

Many people who focus on the "number guessing" and "repetitive nature" side of things tend to consider this process to be pretty arbitrary, however this repetitive chance-based system is key to ensure that it cannot be cheated. As we have discussed, it introduces order, and solidifies it to bring robustness and reliability to Bitcoin.

This process also incurs a cost. Computers draw electrical power to perform their operations, and when people are competing with one another, this power draw becomes non-negligible very quickly.

Miners do earn bitcoin when they mine. As they create blocks, they include an entry allocating bitcoin to themselves. This is their incentive to add the most number of blocks to the chain, as the more blocks they add, the more bitcoin they can earn.

The amount that a miner can allocate to themselves is based on two things:

1. Transaction fees (aka fee market): users can see what transactions are waiting to be mined, what fees they are offering, and can offer a competitive fee to get their transactions prioritised. Miners will often pick the transactions that offer the highest fees to include into their blocks; this ensures that they are earning the most amount of Bitcoin that they have available to them at all times.
2. Block reward: Every block can issue a fixed amount of new bitcoin which the miner can allocate to themselves. Around every four years, the amount of new bitcoin that can be issued is reduced in a controlled manner, and in the year 2140, there will be no more bitcoin left to issue, meaning that miners will need to remain profitable using transaction fees alone.

The block reward satisfies two purposes:

1. Bitcoin must be distributed fairly; there are no licenses or authority based limitations that restrict who can mine bitcoin. Anyone with access to energy, a computer and internet can participate, and therefore new bitcoin can land in the hands of almost absolutely anyone around the world, based purely on fair chance.
2. People must be incentivised to participate in the running of the Bitcoin network, Bitcoin is nothing without a diverse and distributed mass of people running it. Early adopters earn more, while as we get closer to the year 2140, we expect that a critical mass of people will rely on Bitcoin and a plethora of tools, technologies and infrastructure have been built to support it.

These incentives are why people think the goal of mining is to produce more Bitcoin. While for a miner, the main goal is to earn bitcoin, for the system as a whole, mining serves a very different, important and distinct role and the rewards are there merely to ensure people are motivated to do it.

The difficulty adjustment

To get a complete view of Bitcoin mining, the "difficulty adjustment" feature of Bitcoin addresses the elephant in the room: as more people compete to mine bitcoin, wouldn't blocks get produced more quickly?

Bitcoin blocks are produced every 10 minutes on average. Every 2 weeks, or more precisely, every 2016 blocks, a "difficulty target" is redetermined based on the historical production rate of the previous 2016 blocks. All the participants who help to run the Bitcoin network are able to follow this well defined rule automatically on their own computers and will come to the exact same result.

The difficulty target is a way to change the odds of the chance based game, such that we can maintain the average of 10 minutes per solve. Doing this on a regular basis, ensures that even if more miners enter, or if the hardware becomes more efficient, or if they get access to even more energy and computing power, or even if many miners exit, the network can adjust itself and ensure that the network runs smoothly and at a constant pace.

There are also many extra rules that ensure that even this part of the system cannot be gamed. Many developers, mathematicians and philosophers have evaluated Bitcoin deeply and have not found ways to easily game the system.

The 10 minute pace of the system is important for a number of reasons:

1. The issuance of new bitcoin as a reward should not happen too quickly; people need time to discover and adopt bitcoin in order to ensure that bitcoin can be distributed as fairly as possible.
2. Chance is unpredictable and there are cases where multiple miners do find blocks within moments of one another, and this creates forks in the system. The chances of this happening consecutively is rare, and this allows the network to very quickly settle on a single chain and continue operating without issue. Reducing the 10 minute average interval would amplify this situation, causing problems with the smooth running of the network.
3. Considering that blocks can be up to 4mb in size, 10 minutes is ample time for it to be distributed to all bitcoin computers across the world wide web, even over slow network infrastructure and old hardware. 10 minute block times ensure that all computers can remain up to date and not be left eternally catching up as more blocks are being produced faster than their computers can download and validate.

With the difficulty property, the Bitcoin network can grow at a constant pace, and yet everyone can still tell which chain has had the most effort contributed towards it. When a computer comes across two chains, two versions of history, it does not look to find the "longest" chain, but it looks for the one that has the solved for the most amount of difficulty. The result is the same, the chain of data that is considered official is the one that has the most amount of man and electrical power behind it.