Blockchain and cryptocurrency wallet software come with a steep learning curve. In this article we will demystify hierarchical deterministic wallets (also referred to as HD wallets), as introduced in Bitcoin Improvement Proposal #32.
The following topics will be discussed in this article:
Table of contents
- What does hierarchical deterministic mean?
- Using hierarchical deterministic wallets (HD wallets)
- Understanding account derivation paths
- Creating extended keys and wallets
- How to use HD Wallets in my software?
What does hierarchical deterministic mean?
In the context of blockchain, this type of algorithm is used to facilitate the access to multiple millions of cryptocurrency wallets, with just one pass phrase.
What is a cryptocurrency wallet?
A cryptocurrency wallet is a software program or a hardware device which facilitates the management of cryptocurrency and transactions. Typically, a wallet is an interface that makes cryptocurrency accessible to end-users.
Now, let’s have a look at the definitions for the words hierarchical and deterministic as such as to understand the utility of algorithms said to be hierarchically deterministic.
Definition for hierarchical
Hierarchical is a variant of the word hierarchic which points to a subject that is relating to or arranged in a hierarchy. Example: This operation takes a minimal tree as input and returns the mirror image of that tree as output, preserving hierarchical relations.
Definition for deterministic
A deterministic algorithm is one that is relating to a process or model in which the output is determined solely by the input and initial conditions, thereby always returning the same results (as opposed to a stochastic algorithm).
Hierarchical determinism
Now, combining these two words finds us in a place where our algorithm(s) produce results that: (1) are arranged in hierarchies and (2) that are always the same given input(s) and initial condition(s).
Whether it is good practise for any type of sensitive information to be deterministic, is probably a topic for another post. We believe that with hierarchical deterministic wallets, access to Cryptocurrency is highly improved and made more secure.
Using hierarchical deterministic wallets (HD wallets)
As you may have read before, blockchain makes use of public key cryptography. Usually, a pair of public key and private key is used to create digital signatures and unlock tokens. This pair of keys is commonly referred to as “a keypair”.
With the growth of blockchain networks, private key management is a sensitive topic because only one copy of a private key could be the reason for a loss of funds. As such, it is preferred to never store private keys in a database or file.
In production, you should use more resilient solutions for key management, but let’s not drift away from our topic, here. Hierarchical deterministic wallets – or HD Wallets – make it easier for end-users to create multiple millions of keypairs and apply a simple enough structured path.
Understanding account derivation paths
As we mentioned in previous section, many millions of keypairs can be created out of just one 24-words mnemonic phrase. To make the process for end-users easier, Bitcoin Improvement Proposal #44 introduces a derivation path notation that we will use in this section.
An example derivation path is that of your (usually) default Bitcoin address: m/44/0/0/0/0. Another one is your default Bitcoin change address: m/44/0/0/1/0. For my part – I learned about derivation paths the hard way.
In fact, I had to generate multiple thousands of keypairs, before I really grasped the concept of multi-account hierarchies for Bitcoin Wallets. But a couple of years of experience in the space taught me more about HD Wallets. After some research and long nights of coding, I authored NEM Improvement Proposal #6 to enable hierarchical deterministic wallets with the upcoming Symbol from NEM.
So, let’s try an easier way to learn about ledger derivation paths, today.
// – (1) Uses `symbol-hd-wallets` package from NPM
import {MnemonicPassPhrase} from ‘symbol-hd-wallets’
// – (2) Creates a random 24-words mnemonic pass phrase
const mnemonic = MnemonicPassPhrase.createRandom()
// – Displays 24-words (careful, this is sensitive information)
console.log(mnemonic.plain)
Simple enough, for the first. All this source code does is (1) to import the symbol-hd-wallets package – which you can download from NPM. And (2) it will then generate a random mnemonic phrase.
Executing the above snippet shall give you a 24-words pass phrase. Keep this pass phrase somewhere secure – our recommendation is to not make digital copies of such pass phrases on any regular device. Use correctly encrypted storage disks, instead.
With this first random mnemonic phrase generated, we can now derive keys by combining the mnemonic and a derivation path.
Your first derivation path
Want to try out a custom derivation path? We’ll take on, as an example, my birth date which is 29.08.1988.
const path = ‘m/44/29/08/19/88’
Note that, I have not modified the 44 in the beginning of the path. This is because it identifies the derivation path as one that complies with Bitcoin Improvement Proposal #44.
It is not mandatory to keep compliance, but our recommendation is that you always use BIP44 derivation paths as they permit for an increased interoperability with other projects.
Now, is this example derivation path from above valid at all? Yes. As tells us BIP44, we use the following notation:
m / purpose / coin_type / account / change / address_index
There are six (6) parts to this type of notation for derivation paths. The m in the first place, represents the master key – or root key – of the tree. It is, in fact, the node at the top of our hierarchy of keys.
Continuing along, comes the purpose field which as explained above, we recommend to put 44 in order to keep compliant with the Bitcoin Standard and it allows to recognize keys more easily.
Next is the coin type field, which identifies the cryptocurrency. As in our example above, we used 29 as the value for this field, the key would map to the NXT blockchain project. A list of compatible blockchain networks is kept up by SatoshiLabs in an annex to BIP-44, here.
The next three fields of the derivation path identify accounts. As such if we have 08 there, it would give us the ninth (9th) account of the hierarchy (starts with 0, instead of 1).
The same rule applies for the change address and address index fields. As such, our value of 19 in the change address field, would give us the twentieth (20th) change address of the hierarchy and our value of 88 in the address index field would give us the eighty ninth (89th) address of the hierarchy.
Standards around derivation paths
The only open standard that matters when creating derivation paths is BIP44, as this is where the notation is described and explained. Yet, software wallets sometimes have – annoyingly enough – taken a different route and created custom derivation paths that do not comply with BIP44, nor with BIP32.
During my adventure with the Omni Protocol, I had to learn about derivation paths that were selected randomly by project owners – Note that the birthday derivation path from above is somewhat non-standard in that it is known only by me. Mind that this can result in frustrating experiences for system integrators.
Instead, it is best to use numbered accounts. Such that the derivation path m/44/0/0/0/0 actually maps to your first account address.
Note: Sometimes, when referring to BIP32 instead of BIP44, the “first account” is mapped to a derivation path of m/0/0/0. The reason behind this difference is that BIP32 predates BIP44 and the initial open standard did not plan on integrating with other blockchain networks and purposes.
Creating extended keys and wallets
Now that we talked about derivation paths, we can actually apply those and begin creating extended keys.
The keys we will generate are called extended keys because they permit to generate a whole subtree structure of accounts, versus normal keys which map to only one account.
Thus, we need to generate an extended key to be able to generate the keys of our multi-accounts hierarchy. Here is an example:
// – Uses `symbol-hd-wallets` package from NPM
import {MnemonicPassPhrase, ExtendedKey} from ‘symbol-hd-wallets’
// – Creates a random 24-words mnemonic pass phrase
const mnemonic = MnemonicPassPhrase.createRandom()
// – (1) Creates the underlying mnemonic seed
const encSeed = mnemonic.toSeed(‘easypassword’).toString(‘hex’)
// – (2) Creates our hierarchy’s root key
const xkey = ExtendedKey.createFromSeed(encSeed, Network.Bitcoin)
In the above, we re-use the mnemonic creation from the first snippet. Then in (1), we generate a password-protected mnemonic seed. It is not obligatory to set a password here, but it is preferred.
Adding a password in the seed creation steps makes sure that the knowledge of the 24-words phrase is not all that is needed to unlock your account. For attackers to steal your funds, they would have to find out your password as well.
Then, in (2), we generate an extended key out of the mnemonic seed. This is our hierarchy’s root key / master key. You should never share this key with anyone because it permits to re-create all children keys.
With this extended key, it is now possible to derive child accounts and denote their derivation paths as illustrated earlier.
Child account derivation
A child account is any account that is generated from an extended key. The useful bit about child accounts in hierarchical deterministic wallets, is that by combining your 24-words phrase (mnemonic), your password and given a derivation path, it will always produce the same account keys.
Creating a child account from an extended key is just one method call away:
// – Using the `xkey` from previous snippet
// – (1) Creates a “wallet” from an extended key
const wallet = new Wallet(xkey);
// – (2) Gets the child account’s private key
const child1 = wallet.getChildAccountPrivateKey(“m/44/0/0/0/0”)
There are two important parts to the above snippet, in (1) we create a wallet out of an extended key. As we explained above, an extended key lets you create a multitude of accounts. A wallet is a high-level storage for keys which are generated from an extended key – a digital wallet.
In (2), we then use the previously mentioned first Bitcoin account’s derivation path. The resulting child1 variable holds a 32 byte private key that can be used to hold Bitcoin. Again, please never share such information with others.
How to use HD Wallets in my software?
As an introductory note, we recommend that any usage form of cryptocurrency in relation with mnemonic phrases should be thoroughly planned and safely managed. As such, your mnemonic phrases should not land inside your database, or on your filesystem.
A great way to use HD wallets – or digital wallets – in software is to define persona and map numbered accounts (i.e. derivation paths) to a specific persona. An example of this is shown in our article “How to use blockchain for your business?”, where we define a governor persona that is responsible for our distributed governance scheme.
Much complicated, you say? Let’s see for a better example here:
// – Uses `symbol-hd-wallets` package from NPM
import {MnemonicPassPhrase, ExtendedKey, Wallet} from ‘symbol-hd-wallets’
// – Creates a random 24-words mnemonic pass phrase
const mnemonic = MnemonicPassPhrase.createRandom()
// – Creates the underlying mnemonic seed
const encSeed = mnemonic.toSeed(‘easypassword’).toString(‘hex’)
// – Creates our hierarchy’s root key
const xkey = ExtendedKey.createFromSeed(encSeed, Network.Bitcoin)
// – (1) Creates a wallet for our “smart home”
const smart_home_wallet = new Wallet(xkey);
// – (2) Gets the child account for “Alice”
const alice = smart_home_wallet.getChildAccountPrivateKey(“m/44/0/0/0/0”)
// – (3) Gets the child account for “Bob”
const bob = smart_home_wallet.getChildAccountPrivateKey(“m/44/0/1/0/0”)
As shown above, Alice and Bob created one digital wallet for their smart home. Each of them is then assigned an account. As we defined here, each of them is identified with a different keypair.
Note the difference in derivation paths. Alice’s and Bob’s accounts are effectively mapped to two different derivation paths. You could go further and also generate child accounts for Alice and Bob’s children, provided they had any ;).
What is done above, is only really useful when combined to Bitcoin, or digital signature algorithms. As the above would give Alice a different private key than Bob’s, they can identify themselves by digitally signing messages.
The fact that all keys are generated from the same mnemonic pass phrase, is what makes it easily automatable. Alice and Bob can generate many of such digital wallets, as to keep ahead with funds for school excursions, or general electronics repair, or other purposes.
Alice and Bob can also share the management duties over an account – using multi-signature accounts, such that they both have to confirm anything that happens on said account(s). There is indeed many different ways to organize your wallets. Hierarchical deterministic wallets make it practicable and make accounts more accessible.
Mind that you should not share mnemonic phrases with others.
We hope that this article was insightful for you and are looking forward to any feedback and messages. Please share your thoughts in the comments section below!
Disclaimer
This website may contain information about financial firms, employees of such firms, and/or their products and services such as real estate, stocks, bonds, and other types of investments. While this website may intend - as the author deem necessary - to provide information on financial matters and investments, such information or references should not be construed or interpreted as investment advice or viewed as an endorsement.
