Segregated Witness Wallet Development Guide
Overview
Most contents of this document could be found in the BIPs related to segregated witness, including BIP141, BIP143, BIP144, and BIP145. Please consider this as the first point of reference to other related documents, and as a checklist for what should and should not be done.
Basic segregated witness support
A wallet MUST implement all the features in this section, in order to be considered as segwit-compatible at a basic level:
Sending to P2SH
- A segwit-compatible wallet MUST support pay-to-script-hash (BIP16) and its address format (BIP13).
- For making payments, the wallet must be able to correctly transform a given P2SH address to a
scriptPubKey
, and create a transaction. - For receiving payments, the wallet must be able to create a P2SH address based on a P2WPKH script (defined hereinafter), and be able to recognize payment to such addresses.
- This is a mandatory requirement, even if the wallet accepts only single-signature payments.
Creation of P2SH-P2WPKH Address
- A P2SH-P2WPKH address is comparable to Bitcoin’s original single-signature P2PKH address (address with prefix 1).
- Like any other P2SH address, P2SH-P2WPKH address has prefix 3.
- Until a P2SH-P2WPKH UTXO is spent and the
redeemScript
is exposed, a P2SH-P2WPKH address is indistinguishable from a non-segwit P2SH address (such as a non-segwit multi-signature address) - P2SH-P2WPKH addresses should be used when only 1 public key is used to receive payment (like P2PKH)
- P2SH-P2WPKH uses the same public key format as P2PKH, with a very important exception: the public key used in P2SH-P2WPKH MUST be compressed, i.e. 33 bytes in size, and starting with a
0x02
or0x03
. Using any other format such as uncompressed public key may lead to irrevocable fund loss. - To create a P2SH-P2WPKH address:
- Calculate the RIPEMD160 of the SHA256 of a public key (
keyhash
). Despite thekeyhash
formula is same as P2PKH, reuse ofkeyhash
should be avoided for better privacy and prevention of accidental use of uncompressed key - The P2SH
redeemScript
is always 22 bytes. It starts with aOP_0
, followed by a canonical push of thekeyhash
(i.e.0x0014{20-byte keyhash}
) - Same as any other P2SH, the
scriptPubKey
isOP_HASH160 hash160(redeemScript) OP_EQUAL
, and the address is the corresponding P2SH address with prefix 3.
- Calculate the RIPEMD160 of the SHA256 of a public key (
Transaction Serialization
- A segwit-compatible wallet MUST support the original transaction format, as
nVersion|txins|txouts|nLockTime
. - A segwit-compatible wallet MUST also support the new serialization format, as
nVersion|marker|flag|txins|txouts|witness|nLockTime
- Format of
nVersion
,txins
,txouts
, andnLockTime
are same as the original format - The
marker
MUST be0x00
- The
flag
MUST be0x01
- The
witness
is a serialization of all witness data of the transaction.- Each txin is associated with a witness field. As a result, there is no indication of number of witness fields, as it is implied by the number of
txins
- Each witness field starts with a
compactSize
integer to indicate the number of stack items for the correspondingtxin
. It is then followed by witness stack item(s) for the correspondingtxin
, if any. - Each witness stack item starts with a
compactSize
integer to indicate the number of bytes of the item. - If a
txin
is not associated with any witness data, its corresponding witness field is an exact0x00
, indicating that the number of witness stack items is zero.
- Each txin is associated with a witness field. As a result, there is no indication of number of witness fields, as it is implied by the number of
- Format of
- If all
txins
in a transaction are not associated with any witness data, the transaction MUST be serialized in the original transaction format, withoutmarker
,flag
, andwitness
. For example, if none of thetxins
are coming from segwit UTXO, it MUST be serialized in the original transaction format. (exception: coinbase transaction) - Examples of the transaction serialization can be found under the example section of BIP143. Wallet developers may use the examples to test if their implementations correctly parse the new serialization format.
Transaction ID
- Under segwit, each transaction will have 2 IDs.
- The definition of
txid
remains unchanged: the double SHA256 of the original serialization format. - A new
wtxid
is defined, which is the double SHA256 of the new serialization format with witness data. - If a transaction does not have any witness data, its
wtxid
is same as thetxid
. - The
txid
remains the primary identifier of a transaction:- It MUST be used in the
txin
when referring to a previous output. - If a wallet or service is currently using
txid
to identify transactions, it is expected to continue using this with segwit.
- It MUST be used in the
Signature Generation and Verification for P2SH-P2WPKH
- For spending of non-segwit UTXO, the signature generation algorithm is unchanged.
- For spending of P2SH-P2WPKH:
- The
scriptSig
MUST ONLY contain a push of theredeemScript
- The corresponding witness field MUST contain exactly 2 items, a signature followed by the public key
- There is a new signature generation algorithm described in BIP143 for segwit scripts. Developers should follow the instructions carefully, and make use of the P2SH-P2WPKH example in BIP143 to make sure they are able to reproduce the
sighash
. - The BIP143 signature generating algorithm covers the value of the input being spent, which simplifies the design of air-gapped light-weight wallets and hardware wallets.
- Please note that for a P2SH-P2WPKH, the
scriptCode
is always 26 bytes including the leading size byte, as0x1976a914{20-byte keyhash}88ac
, NOT theredeemScript
norscriptPubKey
- Example
- The
Network Services (optional)
- Network services are needed if the wallet would send and receive transactions through the peer-to-peer network
- Segwit-capable nodes will advertise that they can provide witnesses through the service bit:
NODE_WITNESS = (1 << 3)
- Transactions without any witness data (and therefore serialized in original format) may be sent to nodes with or without
NODE_WITNESS
support - Transactions which spend segwit UTXOs (and therefore is serialized in the new format) MUST ONLY be sent to nodes with
NODE_WITNESS
support - Transactions which spend segwit UTXOs but with witness data stripped (and therefore serialized in original format) may be sent to nodes without
NODE_WITNESS
support. Such transactions, however, are invalid after activation of segwit and would not be accepted in a block. - Details of the network services could be found in BIP144.
User Privacy
- Until segwit transactions are commonplace, this transaction type may make Bitcoin tracking easier.
- Using P2SH-P2WPKH as default change output may also have an impact on privacy.
Transaction Fee Estimation
- Instead of transaction size, a new metric is defined, called “virtual size” (
vsize
) vsize
of a transaction equals to 3 times of the size with original serialization, plus the size with new serialization, divide the result by 4 and round up to the next integer. For example, if a transaction is 200 bytes with new serialization, and becomes 99 bytes withmarker
,flag
, andwitness
removed, thevsize
is (99 * 3 + 200) / 4 = 125 with round up.vsize
of a non-segwit transaction is simply its size- Transaction fee should be estimated by comparing the
vsize
with other transactions, not the size. - Developers should be careful not to make an off-by-4-times mistake in fee estimation.
Activation
- As of block height 481824, all SegWit ready nodes started enforcing the new SegWit consensus rules.
Backward Compatibility
- Sending and receiving legacy P2PKH payment (address with prefix 1) should continue to be supported.
Complex script support
If a wallet supports script types other than just single signature, such as multi-signature, it has to fulfill more than the basic requirements:
Creation of P2SH-P2WSH Address
- A P2SH-P2WSH address is comparable to Bitcoin’s original P2SH address, which allows representation of arbitrarily complex scripts with a fixed size address.
- Like any other P2SH and P2SH-P2WPKH address, P2SH-P2WSH address has prefix 3. They are indistinguishable until the UTXO is spent
- To create a P2SH-P2WSH address:
- Define a script, called (
witnessScript
) - Calculate the SHA256 of the
witnessScript
(scripthash
). Please pay attention that a single SHA256 is used, not double SHA256 nor RIPEMD160(SHA256) - The P2SH
redeemScript
is always 34 bytes. It starts with aOP_0
, followed by a canonical push of thescripthash
(i.e.0x0020{32-byte scripthash}
) - Same as any other P2SH, the
scriptPubKey
isOP_HASH160 hash160(redeemScript) OP_EQUAL
, and the address is the corresponding P2SH address with prefix 3.
- Define a script, called (
- Restrictions on the script
- The script evaluation must not fail, and MUST leave one and only one TRUE stack item after evaluation. Otherwise, the evaluation is failed.
- Any public key inside P2SH-P2WSH scripts MUST be compressed key, or fund may be lost permanently.
- If OP_IF or OP_NOTIF is used, it argument MUST be either an empty vector (for false) or
0x01
(for true). Use of other value may lead to permanent fund loss. (BIP draft) - If an OP_CHECKSIG or OP_CHECKMULTISIG is returning a fail, all signature(s) must be empty vector(s). Otherwise, fund may be lost permanently. (BIP146)
- There is a default policy limit for the
witnessScript
at 3600 bytes. Except thewitnessScript
, there could be at most 100 witness stack items, with at most 80 bytes each. Transactions excessing these limits may not be relayed nor included in a block - Many of the original scripts consensus limitations, such as 10000 bytes script size, 201
nOpCount
, are still applied to P2SH-P2WSH - The 520 bytes script size limit for P2SH is not applicable to P2SH-P2WSH. It is replaced by the 3600 bytes policy limit and 10000 bytes consensus limit.
Signature Generation and Verification for P2SH-P2WSH
- For spending of P2SH-P2WSH:
- The
scriptSig
MUST ONLY contain a push of theredeemScript
- The last witness item of the corresponding witness field MUST be the
witnessScript
- The new BIP143 signature generation algorithm is applied:
- Without using any OP_CODESEPARATOR, the
scriptCode
iswitnessScript
preceeded by acompactSize
integer for the size ofwitnessScript
. For example, if the script is OP_1 (0x51
), thescriptCode
being serialized is (0x0151
) - For any unusual scripts containing OP_CODESEPARATOR, please refer to BIP143 for the exact semantics
- Without using any OP_CODESEPARATOR, the
- Any witness stack items before the
witnessScript
are used as the input stack for script evaluation. The input stack is not interpreted as script. For example, there is no need to use a0x4c
(OP_PUSHDATA1) to “push” a big item. - To verify the correctness of signature generation and stack serialization, please always test against the examples in BIP143
- Example
- The
Segwit native addresses (optional)
The following functions are not required for initial segwit support.
Native Pay-to-Witness-Public-Key-Hash (P2WPKH)
- Native P2WPKH is a
scriptPubKey
of 22 bytes. It starts with aOP_0
, followed by a canonical push of thekeyhash
(i.e.0x0014{20-byte keyhash}
) - Same as P2SH-P2WPKH,
keyhash
is RIPEMD160(SHA256) of a compressed public key. - When spending a native P2WPKH, the
scriptSig
MUST be empty, and the witness stack format and signature generating rules are same as P2SH-P2WPKH (including the requirement of using compressed public key) - Example
Native Pay-to-Witness-Script-Hash (P2WSH)
- Native P2WSH is a
scriptPubKey
of 34 bytes. It starts with aOP_0
, followed by a canonical push of thescripthash
(i.e.0x0020{32-byte scripthash}
) - Same as P2SH-P2WSH,
scripthash
is SHA256 of thewitnessScript
. - When spending a native P2WSH, the
scriptSig
MUST be empty, and the witness stack format and signature generating rules are same as P2SH-P2WSH (including the requirement of using compressed public key) - Example
Why and How to Use Native (Bech32) P2WPKH and P2WSH?
- BIP173 proposes a checksummed base32 format (Bech32) for native P2WPKH and P2WSH addresses.
- Support for Bech32 addresses was included in Bitcoin Core v0.16.0
- Comparing with the P2SH versions, the transaction
vsize
of native versions is smaller in most cases, and therefore less fee may be required - Native P2WPKH and P2WSH may be used with raw
scriptPubKey
protocols, such as the Payment Protocol (BIP70). However, it may affect the privacy of the payer and recipient (see below). - Native P2WPKH and P2WSH may be used as default change address, but this may allow other people to identify the change easily (see below)
- Until native P2WPKH and P2WSH are widely used, these address types may cause privacy concerns among users.
Scripts and Transactions Examples
- Examples of different witness transaction types and transaction validity checking tool
- BIP141
- BIP143
- BIP173
- Script tests
- Valid transaction tests
- Invalid transaction tests