20 MINS INTRO TO CONSENSUS*

john.burwell

BiR8BH

20 MINS INTRO TO CONSENSUS*

DOMINIC WILLIAMS @ ETHEREUM DEVCON, LONDON 2015

*TRADITIONAL BYZANTINE FAULT TOLERANT CONSENSUS


WE HAVE BLOCKCHAINS: WHY BOTHER COMPLEX CONSENSUS?

Useful for scale-out crypto ledgers that can support

limitless “decentralized clouds”

✦ Open decentralized system apps e.g. on Ethereum 2.0/3.0

- Gmail, Uber, AirBnB, Twitter, Facebook, search…

- High volume financial exchanges and derivative webs

- IoT services layer for 50 billion devices

✦ Federated decentralized systems e.g. used by industries

to run exchanges or settlement and clearing in financial

sectors

✦ Enterprise decentralized databases e.g. for protecting

organizational data using MPC (prevent Target hack, Office of

Personnel hack…)


DFINITY SCALE-OUT ARCHITECTURE

Blockchain-like structure provides master record of consensus. Can fork. Proof of Work is poor protocol choice though.

BLOCK

BLOCK

BLOCK

BLOCK

BLOCK

MASTER

CONSENSUS

Watch dfinity.io for papers, or contact Dominic

Merkle Root of Roots

TREE OF

“VALIDATION TOWERS”

(see other papers)

RANDOM HEART BEAT

Organizes network

SCALABLE

VALIDATION

< ΔS, tx >

STORAGE

SHARDS

SHARDi

SHARDi+1

SHARDi+2

BFT consensus instance. Cannot fork i.e. is strongly consistent


BYZANTINE CONSENSUS CHALLENGE

P2

P3

P1

XYZ

DEF

ABC

P4

NMO

P7

P5

XYZ

NMO

P6

XYZ

Processes connected by a network must reach agreement on values they

wish to propose. Some processes are “correct” while others are “faulty”…


WHAT IS “BYZANTINE” CONSENSUS?

Byzantine consensus is a game…

Faulty

Correct processes run a protocol to try and reach

agreement by sending and receiving messages.

Faulty processes controlled by an “adversary” try

to (i) prevent consensus being reached, or (ii)

cause correct processes to decide invalid values

Faulty

Faulty processes are “Byzantine” and may adopt

arbitrary tactics:

✦ Decide crash/fail

✦ Corrupt data and messages

Depending model, adversary can also:

✦ Control message scheduling

✦ Has full information about process states

✦ Can corrupt correct nodes at will…

✦ Subtly subvert the communications protocol

✦ Coordinate their actions…


CONSENSUS IS: AGREEMENT

P1

P2

P1

P2

ABC

XYZ

ABC

XYZ

CONSENSUS

SRT

NMO

XYZ

XYZ

P4

P3

P4

P3

All decisions by correct processes must be the same e.g. they must decide

the same value on conclusion of the consensus protocol


CONSENSUS IS: VALIDITY

P1

P2

P1

P2

NMO

NMO

XYZ

NMO

CONSENSUS

NMO

NMO

NMO

NMO

P4

P3

P4

P3

If all correct processes start with some value, then they must decide that value.

Weak validity requires all inputs be same. Strong requires only correct inputs be same.


CONSENSUS IS: TERMINATION

P1

P2

P1

P2

NMO

NMO

XYZ

NMO

CONSENSUS

NMO

NMO

NMO

NMO

P4

P3

P4

P3

Eventually, the protocol must ensure every correct process decides a value,

irrespective of the actions faulty processes take.


ASSUMED MODEL IS CRITICAL PROTOCOL DESIGN

No timing assumptions

No leader processes

Synchronous

Asynchronous

Faulty nodes behave arbitrarily

NETWORK BEHAVIOR

Crash Failure

Byzantine

Failure “detectors” based on timeouts

Leaders typically elected

FAULT TYPE

This is simplified. There are actually numerous assumptions and variations between

models. If it seems complicated, sadly it is (just remember asynchrony better!)


CHALLENGE: KNOW MESSAGES NOT NETWORK

Faulty

Current state of network is unknowable.

Pi can only make inferences about current state

based upon messages received.

NETWORK IS BLACK BOX

M

SEND MESSAGES

Faulty

M

M

M

M

RECEIVE MESSAGES

M

Pi


CHALLENGE: EQUIVOCATION BY FAULTY NODES

P1

ABC

ABC

NMO

XYZ

NMO

P2

P3

Faulty nodes can “equivocate” and pass different versions messages.

Message echoing necessary to catch, which results in O(n 2 )


CHALLENGE: NETWORK ASYNCHRONY

How to detect if this process

is correct or faulty?

Continue waiting?

?

?

___

Expected message not

received…

XYZ

P1

P2

P1 might be a correct process being DDoS’d by the adversary.

P1 might also be a faulty process trying to make P2 wait forever for the message.

Asynchronous protocols never wait for messages that might be withheld..


RESULT: MAXIMUM FAULTY PROCESSES

If there are f faulty processes, then

correct process cannot wait for last f

messages. But, oops, then they may

collect f faulty messages, and not wait

for f correct messages…

Byzantine Quorum

Two Byzantine quorums must always

intersect in at least 1 correct

process allowing operations to be

chained. We know max quorum is n - f

=> max faulty procs. with async!!

n ≥ 3f + 1


READING: BEST OF BREED PROTOCOLS

Partially Synchronous Approach

Asynchronous Byzantine Randomized

Leader-free Binary Consensus

✦ E.g. PBFT

✦ Elect a leader to efficiently direct/conduct

consensus

✦ Re-elect leader if it fails (this process is

expensive and complex)

✦ “Detect” failure in a leader using timeouts

✦ Vulnerable constant leader election due to

- Adversary attacking process

✦ Leader-free

(since no way to detect if leader has failed)

✦ Proceed only on receipt of messages

✦ Proceeds predictably on message deliver

- nothing bad happens if they are delayed

✦ Need enrich model using a common coin

(shared source randomness)

- Network asynchrony


READING: BEST OF BREED PROTOCOLS

Practical Byzantine Fault Tolerance

1999 Castro, Liskov

Asynchronous Byzantine Randomized

Leader-free Binary Consensus

2014 Mostefaoui, Hamouna, Raynal

MODEL : Safe under asynchrony but needs

periods of synchrony to proceed (relies

upon message timeouts).

MODEL : Completely asynchronous (no

timeouts at all). Requires a “common coin”

mechanism to enrich model with

PROS : Uses a leader, and is fast while

randomness. No signatures on messages.

network synchronous and leader is correct.

Complete system sequential writes.

PROS : Has no leader to become Byzantine

or be attacked. Thus DDoS on leader or

CONS : Leader reelected on difficulties. A

Byzantine leader can fwd messages on

network irregularities cannot disrupt

progress. No sigs computationally efficient.

edge timeout window and cause massive

slow down. The adversary can attack the

network to make correct leader be seen as

faulty and cause endless leader election.

CONS : Slower than PBFT in normal

operation (although much more predictable

and faster if leader breaks).


READING: BEST OF BREED PROTOCOLS

Random Oracles in Constantinople:

Practical Asynchronous Byzantine

Agreement using Cryptography

2000 Cachin, Kursawe, Shoup

MODEL : Completely asynchronous (no

timeouts at all). Requires a “common coin”

mechanism to enrich model with

randomness. Signatures on messages.

PROS : Transmits smaller number of

messages

CONS : Messages must be signed, which is

expensive. Makes creation parallel versions

difficult. Trusted dealer for common coin.


EXAMPLE CODE: RANDOMIZED ASYNCHRONOUS BINARY CONSENSUS

common coin is shared random value

Byzantine quorum intersect, 2f+1 accepted votes

Generate common coin e.g threshold sig system

2f+1 (quorum) votes for single accepted value v…

Single quorum value equals coin, DECIDE!

Converge on single quorum value…

No single quorum value. Converge on coin…


Note innovative use of value “amplification”. This algorithm determines values that

have been voted for by at least 1 correct process. After first value returned, no way to

know if second value will be selected.

Paper https://hal.inria.fr/hal-00944019/file/RR-2016-Consensus-optimal-V5.pdf

Contact Dominic for parallel version


FUTURE: PROBABILISTIC CONSENUS PROTOCOLS

DFINITY

Please watch dfinity.io for new

consensus techniques

Thanks, contact

dominic@string.technology

Similar magazines