mirror/prysm-pulse
1
0
Fork 0
mirror of https://gitlab.com/pulsechaincom/prysm-pulse.git synced 2024-12-31 23:41:22 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

rem old docs (#5446)

Browse Source
This commit is contained in:
Raul Jordan 2020-04-15 13:53:02 -05:00 committed by GitHub
parent 5eb6485e14
commit 21f51993c5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 0 additions and 468 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

58
docs/DEMO_INFORMATION.md
View File

@ -1,58 +0,0 @@
# Ethereum Serenity Prysm Demo
**IMPORTANT: This document describes our v0.0.0 demo release from October 2018, which is no longer relevant to our current work and is only here for historical purposes**
## Overview & Research Background
At Prysmatic Labs, we started working on Ethereum Serenity all the way back since Vitalik first had a Sharding FAQ as the only reference for the system at the start of 2018. A lot has happened, with the specification evolving from a series of ETHResearch blog posts into a minimal viable blueprint for how to design a scalable, secure Ethereum blockchain using Casper Proof of Stake and Sharding at its core.
Now, the plan is to deploy ETH2.0 as a sidechain of Ethereum known as a beacon chain, where validators can stake their Ether and participate in consensus to vote on occurrences on shards known as cross-links.
## Version 0.0.0: Beacon Chain+Validator Demo
We call our Ethereum Serenity project Prysm, which will serve as the scaffold for a production-oriented release fully written in Go. We have been working hard to ensure we have a robust implementation of the Ethereum Serenity specification as created by the Ethereum Research Team along with industry standard approaches to building large scale applications.
We are proud to announce our very first release of Prysm, v0.0.0, which will serve as the building block for all future releases as we get to production. We want to show the community we have a project we have put a lot of work into through careful thought and design decisions that we hope will set a standard for future ETH2.0 developments.
## What This Release Encompasses
Version 0.0.0 includes a basic beacon-chain+validator demo which can do the following:
- Spin up a beacon chain node from a genesis configuration
- Connect a validator client via RPC
- The validator client gets shuffled into a specific shard at a given slot
- Validators propose/attest to canonical beacon blocks during their assigned slot
- Validators get reshuffled into shards every new cycle transition
- Casper FFG rewards/penalties are included in this release even though they are a constant area of research
- Basic, locally networked p2p via libp2p and the mDNS discovery protocol
- Beacon chain block sync through p2p (listening for incoming blocks + syncing to a latest head from scratch)
- A useful simulator of beacon blocks (this allows us to simulate other beacon nodes relaying info to our node locally)
- Storing blocks/attestations/states to a write-optimized key-value backend known as BoltDB
- gRPC public API client/server for querying a beacon node for canonical blocks, states, and latest validator assignments
- A robust, scalable build system known as Bazel used in production at Google, Pinterest, Dropbox, and other industry giants
- A Web3 Subscription service to listen to latest mainchain blocks and validator registrations
## Not Included in the Release
- Although a Validator Registration Contract is included, validator rotation, withdrawals, and dynasty transitions are not yet included
- Shards, their associated design, and cross-shard transactions are not included in this release
- Fork-choice rule for ETH2.0 is not yet included, we use a naive fork choice rule for this release
- Signature aggregation and verification are not included in this release
- Randomness via RANDAO and VDF are not included in this release as they are an active area of research - we use basic block hashes as a stub for now
- Serialization format for ETH2.0 is still an active area of research and is not included here
- Shardp2p and beacon node p2p peer discovery have not yet been designed beyond mDNS
- State execution is not included as it depends on shard functionality
## How to Run the Demo
Curious to see the system working and running a validator client locally? We have put together comprehensive instructions on running our demo in our Github repository! Try it out and see for yourself :).
[Running Instructions](https://github.com/prysmaticlabs/prysm/blob/master/README.md#instructions)
You’ll be able to spin up a beacon node, connect a validator client, and start getting assigned to shards where you will then create beacon blocks or vote on beacon blocks through structures called “attestations”. If you are not quite as familiar with the Ethereum Serenity Roadmap and Spec, check out the following links:
- [Ethereum Serenity Devs Handbook](https://notes.ethereum.org/s/BkSZAJNwX#)
- [Ethereum Serenity Casper+Sharding Specification](https://github.com/ethereum/eth2.0-specs/tree/master/specs)
Even though canonical blocks are created in the demo due to your activity as a validator, you’ll quickly see not much can be done with these blocks until real, state execution comes into play. However, this beacon chain is a critical piece of consensus and coordination for all actors participating in Ethereum Serenity and will as the foundation for a full-fledged, sharding system.

113
docs/PRIOR_RESEARCH.md
View File

@ -1,113 +0,0 @@
# Beacon Chain Research Synopsis
This doc will summarize some historical discussions and roadmap updates around integrating Casper/Sharding through a beacon chain and what it means for Ethereum's end-game.
### Research Notes Leading Up To This
- [Offchain Collation Headers](https://ethresear.ch/t/offchain-collation-headers/1679)
- [RANDAO Beacon Exploitability Part. 1](https://ethresear.ch/t/rng-exploitability-analysis-assuming-pure-randao-based-main-chain/1825/9)
- [Leaderless k-of-n Random Beacon](https://ethresear.ch/t/leaderless-k-of-n-random-beacon/2046)
- [Two Ways to do Cross Links](https://ethresear.ch/t/two-ways-to-do-cross-links/2074)
- [Registrations, Shard Count, and Shuffling](https://ethresear.ch/t/registrations-shard-count-and-shuffling/2129)
- [Committee Based Sharded Casper](https://ethresear.ch/t/committee-based-sharded-casper/2197)
- [Attestation Committee Based Full PoS Chains](https://ethresear.ch/t/attestation-committee-based-full-pos-chains/2259)
The beacon chain idea emerged from the research around notarization of shard information by a committee of notaries, similar to a committee of validators in Casper. The key difference in sharding, however, is the pseudorandomness generation required for fast reshuffling of actors across shards within the system.
Piggybacking off the VRF (verifiable random function) research alongside the ideas of BLS signatures put forth by DFINITY, there are many elements that can be taken to create a sidechain that would potentially merge sharding/casper :heart:.
## The Beacon Chain
### A Sidechain Instead of a Sharding Manager Contract
When handling the sharded system via smart contract on the Ethereum mainchain, we were able to derive implicit finality via the transactions that submit a collation header into the contract that would then be mined onto a block in the mainchain. However, we were _bounded_ by gas and the current functioning of the EVM 1.0. That is, the number of shards realistically could only grow as much as the sharding manager contract could handle as a load of incoming transactions.
This system is also more affected by hard-forks and changes occurring in the main Ethereum network. Moreover, the incoming integration of a hybrid Casper PoS system would create a complicated co-existence of two types of validators: namely Casper and sharding validators. Signature verification, in particular, is an extremely expensive operation if done entirely through a smart contract on the mainchain, creating inherent bottlenecks in a hybrid system.
Instead, we propose the creation of a _sidechain_ known as a **beacon chain** that has links to the mainchain by containing hashes of canonical mainchain blocks within its own block construction.
### Desiderata
There are a few important traits we will include in this sidechain construct that are particularly important for sharding.
- Block references to the main chain
- Full Proof of Stake via Casper FFG
- RANDAO for random selection of committees
- One-Way Deposits
- Ability to store verifiable metadata of occurrences across shards (we refer to this information as a cross-link)
In Vitalik Buterin's [words](https://notes.ethereum.org/SCIg8AH5SA-O4C1G1LYZHQ?both):
> A cross-link is a special type of transaction that says “here is the hash of some recent block on shard X. Here are signatures from at least 2/3 of some randomly selected sample of M validators (eg. M = 1024) that attest to the validity of the cross-link”. Every shard (eg. there might be 4000 shards total) is itself a PoS chain, and the shard chains are where the transactions and accounts will be stored. The cross-links serve to “confirm” segments of the shard chains into the main chain, and are also the primary way through which the different shards will be able to talk to each other.
The main purpose of beacon chain is to handle these shard cross-links as well as the set of validators that are locked into the system. This initial set is seeded by users burning 32ETH into a contract on the mainchain and specifying their public key, which can then be verified by the beacon chain which interacts with this contract.
The exact specification of the beacon chain is a work-in-progress being written [here](https://notes.ethereum.org/SCIg8AH5SA-O4C1G1LYZHQ?both). Instead of paraphrasing its spec, we will elaborate on the research that led to this point and what it means moving forward.
## Research History
Recall that the **Sharding Manager Contract** was originally going to be used for management of the sharded system.
> Whole point of this is to give a background and better understanding of the system...I think I won't follow the structure below in the final writing but will help as an outline.
### Limitations of the Sharding Manager Contract
Using a Sharding Manager Contract, albeit attractive from a development standpoint, poses the following challenges:
- miners on the main chain can censor transactions
- the number of shards is limited by gas costs of writes to the storage of this contract
- any upgrades to this contract/sharding system would require a hard fork on the main chain
#### Offchain Collation Headers
[ETHResearch Link](https://ethresear.ch/t/offchain-collation-headers/1679)
One idea that naturally arose was to store collation headers offchain, allowing for more shards to exist as there would be no processing bottleneck and faster finality guarantees. Reminiscent of DFINITY's chain, this process could be done through a construct known as a **random beacon side chain** that is pegged to the main chain via checkpoints on main chain blocks.
Reminiscent of plasma chains without the exit mechanism according to Justin Drake, this construct would provide a solid ground for experimentation of shard designs without modifications to the main chain. The beacon chain would provide the pseudorandomness required for committee selection of the sharding system, through BLS distributed key generation, as well as better finality given that shards only care about finalized deposits from the main chain.
Management and submission of collation headers can now be done within each respective shard, with the beacon chain only serving as a coordination device for summarizing what was voted on within each shard. That is, the beacon chain would be responsible for handling what we call a **cross-link**, which is a piece of metadata summarizing which collations were voted on as canonical within shards, who voted on these collations, and what are their blob merkle roots.
#### Randomness in Committee Selections
[ETHResearch Link](https://ethresear.ch/t/rng-exploitability-analysis-assuming-pure-randao-based-main-chain/1825/9)
While DFINITY's beacon chain uses the BLS Signature scheme integrated into something called Threshold Relay for distributed randomness, this is not needed for the random beacon chain construct we are building for sharding. That is, there are other satisfactory ways to achieve randomness that make more sense for sharding.
Ethereum's sharding beacon chain will use a system called **RANDAO** which is a Decentralized Autonomous Organization (a DAO) where randomness is generated by participants contributing a value to a "hash-onion" in the system
$$H(H(H(.....S.....)))$$
where participants creating a block have to reveal the pre-image (the value before the hash) of their commitment value and update the current commitment value to this pre-image. In a recursive manner, the next participants will have to do the same and reveal their pre-images when creating a block. This system updates a random value $R$ during each iteration by taking the $XOR$ of it with the revealed pre-image. This value, $R$, is then used for randomization of committee selection. For sharding in particular, Justin mentioned the source of randomness can be this global $R$ value also $XOR$ with the proposer's pre-image commitment for that particular shard, restricting the visibility of $R\_{shard}$ to only proposers participating in that shard.
#### Leaderless Random Beacon - Alternative to RANDAO
[ETHResearch Link](https://ethresear.ch/t/leaderless-k-of-n-random-beacon/2046)
The downside of RANDAO is the need for a leader in each step of the hash pre-image reveal step for determining the next value of $R$. In an alternative approach, we have a "committee of size $n$ generate random numbers if $k$ participants generate correctly" (Justin Drake). In this approach, we have every participant commit a temporary secret key, public key pair and form a polynomial via point interpolation. At a reveal phase of the protocol, k-of-n polynomial encrypted shares. At a reveal phase, $k$ participants reveal their private keys and it is then easy to check which ones did not reveal correctly. Even then, we can reconstruct an appropriate candidate polynomial using the revealed keys. Then, the randomness becomes random output to be the "sum of the secret keys for which the corresponding participants committed correctly" as explained by Justin.
### Shard Metadata & Finality
[ETHResearch Link](https://ethresear.ch/t/two-ways-to-do-cross-links/2074)
The main idea behind sharding is to leverage this beacon chain construct for consensus on **cross-links** which are the heart of the sharding spec. In a way, cross-links are metadata that summarize the latest occurrences on shards. That is, they compress the results of PoS consensus from each shard into a simple message that uses proposer signatures to confirm this information. They are called **cross-links** because they will eventually be the way shards can communicate between each other, as they are linked to the finality of the main chain.
#### Registrations and Committee Reshuffling
[ETHResearch Link](https://ethresear.ch/t/registrations-shard-count-and-shuffling/2129)
Leading up to the beacon chain spec, a few proposals for structuring committees were being explored, including this one by Justin Drake. We refer to participants in the Ethereum protocol 2.0 (Casper + Sharding) as validators which can be in one of three states: either pending_registration, registered, or pending_deregistration (Justin Drake).
The main idea behind structuring committees of validators is that shards will be empty and uninitialized until the beacon chain reaches a certain minimum number of registered validators.
In the post, it is mentioned that
> Proposers and notaries are shuffled (via pseudo-random permutations) across shards in a staggered fashion and at a constant rate. Proposers are assigned to shards for 2^19 periods (~30 days) and the oldest proposer from each shard are shuffled every 2^(19 - 10) periods. Notaries are assigned to a shard for 2^7 periods (~10 minutes) and the oldest 2^(10 - 7) notaries from each shard are shuffled every period.
>
> However, the current spec for the beacon chain mentions a fixed number for the SHARD_COUNT set to 1024.
The entire reshuffling mechanism was revamped given this fixed shard_count number. In the current beacon chain spec, it is mentioned that
> For shard crosslinks, the process is somewhat more complicated. First, we choose the set of shards active during each epoch. We want to target some fixed number of notaries per crosslink, but this means that since there is a fixed number of shards and a variable number of validators, we won’t be able to go through every shard in every epoch. Hence, we first need to select which shards we will be crosslinking in some given epoch
Additionally, the current spec forces casper validators to also be sharding validators, which enforces greater security and takes advantage of the enshrined randomness + full PoS properties of the beacon chain.

149
docs/READINGS.md
View File

@ -1,149 +0,0 @@
# Required Readings
This document serves as an outline of the required readings to catch up with and contribute to the implementation for of the Ethereum 2.0 (Serenity) protocol. Whether you are an expert at Ethereum and want to contribute, or are a beginner and want to take the time to learn the concepts, this will serve as a comprehensive set of readings and links to help you up to speed.
## Required Blockchain & Ethereum Knowledge
**Blockchain Fundamentals**
- [What is Blockchain Technology? A Step by Step Guide for Beginners](https://blockgeeks.com/guides/what-is-blockchain-technology/)
- [What is Bitcoin? A Step by Step Guide for Beginners](https://blockgeeks.com/guides/what-is-bitcoin/)
- [The Science Behind Cryptocurrencies' Cryptography](https://blockgeeks.com/guides/cryptocurrencies-cryptography/)
- [The Ins and Outs of Cryptographic Hash Functions](https://blockgeeks.com/guides/cryptographic-hash-functions/)
- [Blockchain Glossary from A-Z](https://blockgeeks.com/guides/blockchain-glossary-from-a-z/)
- [Blockchain Addresses 101: What Are They?](https://blockgeeks.com/guides/blockchain-address-101/)
**Ethereum**
- [What is Ethereum?](https://www.ethereum.org/beginners/)
- [How Does Ethereum Work Anyway?](https://medium.com/@preethikasireddy/how-does-ethereum-work-anyway-22d1df506369)
- [Ethereum Introduction](https://github.com/ethereum/wiki/wiki/Ethereum-introduction)
- [Ethereum Frequently Asked Questions](https://github.com/ethereum/wiki/wiki/FAQs)
- [What is Hashing?](https://blockgeeks.com/guides/what-is-hashing/)
- [Hashing Algorithms and Security](https://www.youtube.com/watch?v=b4b8ktEV4Bg)
- [Understanding Merkle Trees](https://www.codeproject.com/Articles/1176140/Understanding-Merkle-Trees-Why-use-them-who-uses-t)
- [Ethereum White Paper](https://github.com/ethereum/wiki/wiki/White-Paper)
- [Ethereum Block Architecture](https://ethereum.stackexchange.com/questions/268/ethereum-block-architecture/6413#6413)
- [Ethereum Beige Paper](https://github.com/chronaeon/beigepaper/blob/master/beigepaper.pdf)
- [What is an Ethereum Token?](https://blockgeeks.com/guides/ethereum-token/)
- [What is Ethereum Gas?](https://blockgeeks.com/guides/ethereum-gas-step-by-step-guide/)
**Ethereum Client Implementations**
- [Geth](https://github.com/ethereum/go-ethereum) (known also as go-ethereum) is the Golang implementation of the Ethereum protocol
- [Parity](https://github.com/paritytech/parity) the fastest and most performant implementation - written in Rust
- [Trinity](https://github.com/ethereum/py-evm/tree/master/trinity) new project implements Ethereum in Python
- [Cpp-Ethereum](https://github.com/ethereum/cpp-ethereum) a C++ implementation of Ethereum
**Consensus**
- [Bitcoin Original White Paper](https://bitcoin.org/bitcoin.pdf)
- [Basic Primer: Blockchain Consensus](https://blockgeeks.com/guides/blockchain-consensus/)
- [Understanding Blockchain Fundamentals: Byzantine Fault Tolerance](https://medium.com/loom-network/understanding-blockchain-fundamentals-part-1-byzantine-fault-tolerance-245f46fe8419)
- [Understanding Blockchain Fundamentals: Proof of Work vs. Proof of Stake](https://medium.com/loom-network/understanding-blockchain-fundamentals-part-2-proof-of-work-proof-of-stake-b6ae907c7edb)
- [Proof of Work vs. Proof of Stake](https://blockgeeks.com/guides/proof-of-work-vs-proof-of-stake/)
- [Proof of Stake FAQ](https://github.com/ethereum/wiki/wiki/Proof-of-Stake-FAQ)
- [How Does Ethereum Mining Work?](https://www.coindesk.com/information/ethereum-mining-works/)
- [ETHash Algorithm](https://github.com/ethereum/wiki/wiki/Ethash)
**Ethereum Virtual Machine**
- [What is the Ethereum Virtual Machine?](https://themerkle.com/what-is-the-ethereum-virtual-machine/)
- [Ethereum VM](https://medium.com/@jeff.ethereum/go-ethereums-jit-evm-27ef88277520)
- [Ethereum Protocol Subtleties](https://github.com/ethereum/wiki/wiki/Subtleties)
- [Awesome Ethereum Virtual Machine](https://github.com/ethereum/wiki/wiki/Ethereum-Virtual-Machine-(EVM)-Awesome-List)
**Ethereum-flavored WebAssembly**
- [Ewasm background, motivation, goals, and design](https://github.com/ewasm/design)
- [The current Ewasm spec](https://github.com/ewasm/design/blob/master/eth_interface.md)
- [Latest Ewasm community call including live demo of the testnet](https://www.youtube.com/watch?v=apIHpBSdBio)
- [Why eWASM? by Alex Beregszaszi](https://www.youtube.com/watch?v=VF7f_s2P3U0)
- [Panel: entire eWASM team discussion and Q&A](https://youtu.be/ThvForkdPyc?t=119)
- [Ewasm community meetup at ETHBuenosAires](https://www.youtube.com/watch?v=qDzrbj7dtyU)
**Smart Contracts, dApps, and Cryptoeconomics**
- [What are dApps? The New Decentralized Future](https://blockgeeks.com/guides/dapps/)
- [How to Learn Solidity](https://blockgeeks.com/guides/solidity/)
- [Ethereum Development Tutorial](https://github.com/ethereum/wiki/wiki/Ethereum-Development-Tutorial)
- [What is Cryptocurrency Game Theory?](https://blockgeeks.com/guides/cryptocurrency-game-theory/)
- [What is Cryptoeconomics?](https://blockgeeks.com/guides/what-is-cryptoeconomics/)
- [Mechanism Design for Cryptoeconomic Applications](https://medium.com/blockchannel/a-crash-course-in-mechanism-design-for-cryptoeconomic-applications-a9f06ab6a976)
- [Cryptoeconomics: An Introduction](https://cryptoeconomics.study/)
**Peer-to-Peer Networking**
- [Ethereum Peer to Peer Networking](https://github.com/ethereum/go-ethereum/wiki/Peer-to-Peer)
- [How Does the P2P on Ethereum Work?](https://www.reddit.com/r/ethereum/comments/3918u0/how_does_the_p2p_network_on_ethereum_work/)
- [How Does Kademlia Work?](http://gleamly.com/article/introduction-kademlia-dht-how-it-works)
- [Kademlia Protocol](http://www.divms.uiowa.edu/~ghosh/kademlia.pdf)
## Required Sharding Knowledge
With respect to knowing enough about sharding, we will cover the requirements for both part-time and core contributors to Prysmatic Labs.
### For Part Time Contributors
- [Blockchain Scalability: Why?](https://blockgeeks.com/guides/blockchain-scalability/)
- [What Are Ethereum Nodes and Sharding](https://blockgeeks.com/guides/what-are-ethereum-nodes-and-sharding/)
- [How to Scale Ethereum: Sharding Explained](https://medium.com/prysmatic-labs/how-to-scale-ethereum-sharding-explained-ba2e283b7fce)
- [Sharding FAQ](https://github.com/ethereum/wiki/wiki/Sharding-FAQ)
- [Sharding Introduction: R&D Compendium](https://github.com/ethereum/wiki/wiki/Sharding-introduction-R&D-compendium)
### For Core Contributors
**Fundamentals**
- [Blockchain Scalability: Why?](https://blockgeeks.com/guides/blockchain-scalability/)
- [What Are Ethereum Nodes and Sharding](https://blockgeeks.com/guides/what-are-ethereum-nodes-and-sharding/)
- [How to Scale Ethereum: Sharding Explained](https://medium.com/prysmatic-labs/how-to-scale-ethereum-sharding-explained-ba2e283b7fce)
- [Sharding FAQ](https://github.com/ethereum/wiki/wiki/Sharding-FAQ)
- [Sharding Introduction: R&D Compendium](https://github.com/ethereum/wiki/wiki/Sharding-introduction-R&D-compendium)
After reading the Sharding FAQ, it is important to understand the minimal implementation notes and the research that went into writing it up.
**Serenity Concepts and Notes**
- [Sharding Concepts Mental Map](https://www.mindomo.com/zh/mindmap/sharding-d7cf8b6dee714d01a77388cb5d9d2a01)
- [Taiwan Sharding Workshop Notes](https://hackmd.io/s/HJ_BbgCFz#%E2%9F%A0-General-Introduction)
- [Sharding Research Compendium](http://notes.ethereum.org/s/BJc_eGVFM)
- [Torus Shaped Sharding Network](https://ethresear.ch/t/torus-shaped-sharding-network/1720/8)
- [General Theory of Sharding](https://ethresear.ch/t/a-general-theory-of-what-quadratically-sharded-validation-is/1730/10)
- [Sharding Design Compendium](https://ethresear.ch/t/sharding-designs-compendium/1888/25)
**Serenity Research Posts in Order**
- [Sharding v2.1 Spec](https://notes.ethereum.org/SCIg8AH5SA-O4C1G1LYZHQ)
- [Casper/Sharding/Beacon Chain FAQs](https://notes.ethereum.org/9MMuzWeFTTSg-3Tz_YeiBA?view)
- [RETIRED! Sharding Phase 1 Spec](https://ethresear.ch/t/sharding-phase-1-spec-retired/1407/92)
- [Exploring the Proposer/Collator Spec and Why it Was Retired](https://ethresear.ch/t/exploring-the-proposer-collator-split/1632/24)
- [The Stateless Client Concept](https://ethresear.ch/t/the-stateless-client-concept/172/4)
- [Shard Chain Blocks vs. Collators](https://ethresear.ch/t/shard-chain-blocks-vs-collators/429)
- [Ethereum Concurrency Actors and Per Contract Sharding](https://ethresear.ch/t/ethereum-concurrency-actors-and-per-contract-sharding/375)
- [Future Compatibility for Sharding](https://ethresear.ch/t/future-compatibility-for-sharding/386)
- [Fork Choice Rule for Collation Proposal Mechanisms](https://ethresear.ch/t/fork-choice-rule-for-collation-proposal-mechanisms/922/8)
- [State Execution](https://ethresear.ch/t/state-execution-scalability-and-cost-under-dos-attacks/1048)
- [Fast Shard Chains With Notarization](https://ethresear.ch/t/as-fast-as-possible-shard-chains-with-notarization/1806/2)
- [RANDAO Notary Committees](https://ethresear.ch/t/fork-free-randao/1835/3)
- [Safe Notary Pool Size](https://ethresear.ch/t/safe-notary-pool-size/1728/3)
- [Cross Links Between Main and Shard Chains](https://ethresear.ch/t/cross-links-between-main-chain-and-shards/1860/2)
**Serenity Talk from Conferences & Meet Ups in Order**
- [Sharding Presentation by Vitalik from IC3-ETH Bootcamp](https://vod.video.cornell.edu/media/Sharding+-+Vitalik+Buterin/1_1xezsfb4/97851101)
- [Latest Research and Sharding by Justin Drake from Tech Crunch](https://www.youtube.com/watch?v=J6xO7DH20Js)
- [Beacon Casper Chain by Vitalik and Justin Drake](https://www.youtube.com/watch?v=GAywmwGToUI)
- [Proofs of Custody by Vitalik and Justin Drake](https://www.youtube.com/watch?v=jRcS9D_gw_o)
- [So You Want To Be a Casper Validator by Vitalik](https://www.youtube.com/watch?v=rl63S6kCKbA)
- [Ethereum Sharding from EDCon by Justin Drake](https://www.youtube.com/watch?v=J4rylD6w2S4)
- [Casper CBC and Sharding by Vlad Zamfir](https://www.youtube.com/watch?v=qDa4xjQq1RE&t=1951s)
- [Casper FFG in Depth by Carl](https://www.youtube.com/watch?v=uQ3IqLDf-oo)
- [Ethereum & Scalability Technology from Asia Pacific ETH meet up by Hsiao Wei](https://www.youtube.com/watch?v=GhuWWShfqBI)
## Necessary Go Knowledge & Readings
- [The Go Programming Language (Only Recommended Book)](https://www.amazon.com/Programming-Language-Addison-Wesley-Professional-Computing/dp/0134190440)
- [Ethereum Development with Go](https://goethereumbook.org)
- [How to Write Go Code](http://golang.org/doc/code.html)
- [The Go Programming Language Tour](http://tour.golang.org/)
- [Getting Started With Go](http://www.youtube.com/watch?v=2KmHtgtEZ1s)
- [Go Official Website](https://golang.org/)

148
docs/ROADMAP.md
View File

@ -1,148 +0,0 @@
# Prysmatic Labs Roadmap Reference
This document serves as a main reference for Prysmatic Labs' sharding and beacon chain implementation in Go, along with our roadmap and compilation of active research.
# Table of Contents
- [Sharding Introduction](#sharding-introduction)
- [Basic Sharding Idea and Design](#basic-sharding-idea-and-design)
- [Roadmap Phases](#roadmap-phases)
- [The Ruby Release: Local Network](#the-ruby-release-local-network)
- [The Sapphire Release: Goerli Testnet](#the-sapphire-release-goerli-testnet)
- [The Diamond Release: Ethereum Mainnet](#the-diamond-release-ethereum-mainnet)
- [System Architecture](#system-architecture)
- [Acknowledgements](#acknowledgements)
- [References](#references)
# Sharding Introduction
Currently, every single node running the Ethereum network has to process every single transaction that goes through the network. This gives the blockchain a high amount of security because of how much validation goes into each block, but at the same time it means that an entire blockchain is only as fast as its individual nodes and not the sum of its parts. Currently, transactions on the EVM are non-parallelizable, and every transaction is executed in sequence globally. The scalability problem then has to do with the idea that a blockchain can have at most 2 of these 3 properties: decentralization, security, and scalability.
If we have scalability and security, it would mean that our blockchain is centralized and that would allow it to have a faster throughput. Right now, Ethereum is decentralized and secure, but not scalable.
An approach to solving the scalability trilemma is the idea of blockchain sharding, where we split the entire state of the network into partitions called shards that contain their own independent piece of state and transaction history. In this system, certain nodes would process transactions only for certain shards, allowing the throughput of transactions processed in total across all shards to be much higher than having a single shard do all the work as the main chain does now.
## Basic Sharding Idea and Design
A sharded blockchain system is made possible by having nodes store “signed metadata” in the main chain of latest changes within each shard chain. Through this, we manage to create a layer of abstraction that tells us enough information about the global, synced state of parallel shard chains. These messages are called **cross-links**, which are specific structures that encompass important information about blocks in shards across the network. We coordinate these crosslinks on a chain known as a **beacon-chain**. In this chain, blocks are created by actors known as **proposers** that are tasked with information about cross-links together. These blocks are then voted on by a party of actors known as **attesters**. These attesters are randomly selected for particular periods of time in certain shards and are then tasked into reaching consensus on these chains and on beacon chain blocks via a **proof of stake** system. Attesters and proposers are the names of two different responsibilities a participant in Ethereum 2.0 consensus can have. These participants are known in general as **validators**.
Cross-links are stored in blocks on a full proof of stake chain known as a **beacon chain**, which will be implemented as a sidechain to the Ethereum main chain initially.
We still keep the Ethereum main chain and deploy a smart contract into it known as the **Validator Deposit Contract**, where users can deposit and burn 32 ETH. Beacon chain nodes would listen to deposits in this contract and consequently queue up a user with the associated address as a validator in the beacon chain PoS system. Validators then become part of a registered validator set in the beacon chain, and are committees of validators are selected to become attesters or proposers for a certain period of time on shards until they are ventually reshuffled into different shards.
Given that we are splitting up the global state of the Ethereum blockchain into shards, new types of attacks arise because fewer resources are required to completely dominate a shard. This is why a **source of randomness** and periods are critical components to ensuring the integrity of the system.
The Ethereum Wiki’s [Sharding FAQ](https://github.com/ethereum/wiki/wiki/Sharding-FAQ) suggests pseudorandom sampling of validators on each shard. The goal is so that these attesters will not know which shard they will get in advance. Otherwise, malicious actors could concentrate resources into a single shard and try to overtake it (See: [1% Attack](https://medium.com/@icebearhww/ethereum-sharding-and-finality-65248951f649)).
Sharding revolves around being able to store shard metadata in a full proof of stake chain known as a beacon chain. For pseudorandomness generation, a mechanism known as RANDAO can be used in the beacon chain to shuffle validators securely.
# Roadmap Phases
Prysmatic Labs will implement the official beacon chain specification and the full roadmap for Ethereum 2.0 as written in its official repository [here](https://github.com/ethereum/eth2.0-specs) by the community of core researchers and developers in Ethereum.
To roll out these phases, we will be releasing our implementation in a series of steps:
## The Ruby Release: Local Network
Our current work is focused on creating a localized version of a beacon chain that would include the following:
- A minimal, **beacon chain node** that will interact with an Ethereum 1.0 node via JSON-RPC
- A **Validator Deposit Contract** deployed on an Ethereum 1.0 chain where a beacon node can read logs to check for registered validators
- A minimal, floodsub p2p network
- Ability for proposers/attesters to be selected by the beacon chain's randomness into committees that work on specific shards
- Ability to execute state transitions, apply a fork choice rule, and advance a full proof of stake blockchain through distributed consensus
We released our Ruby release on October 2018
## The Sapphire Release: Goerli Testnet
Part 1 of the **Sapphire Release** will focus around getting the **Ruby Release** polished enough to be live on an Ethereum testnet and manage a a beacon chain system.
Part 2 of the **Sapphire Release** will focus on implementing state execution and defining the State Transition Function for sharding on a local testnet (as outlined in [Beyond Phase 1](#beyond-phase-1)) as an extenstion to the Ruby Release.
ETA: End of Q1 2019
## The Diamond Release: Ethereum Mainnet
The **Diamond Release** will reconcile the best parts of the previous releases and deploy a full-featured, cross-shard transaction system through a beacon chain, casper FFG-enabled, sharding release. As expected, this is the most difficult and time consuming release on the horizon for Prysmatic Labs. We plan on growing our community effort significantly over the first few releases to get all hands-on deck preparing for this.
The Diamond Release should be considered the production release candidate for sharding Ethereum on the mainnet.
ETA: To Be determined
# Beacon Chain Phase 0 Implementation
Prysmatic Labs will begin by focusing its implementation entirely on Phase 0 from the Ethereum 2.0 roadmap. We plan on being as pragmatic as possible to create something that can be locally run by any developer as soon as possible. Our initial deliverable will center around a command line tool that will serve as an entry point into a beacon chain node that allows for users to become validators and earn rewrads for participating in Proof of Stake consensus.
Here is a reference spec explaining how our initial system will function:
## System Architecture
Our implementation revolves around the following core components:
- A **beacon chain** that connects to this main chain node via JSON-RPC
- A **validator client** that connects to a beacon node and allows for users to earn rewards for staking 32ETH to secure the protocol
A basic, end-to-end example of the system is as follows:
1. _**User deposits 32 ETH into a Validator Deposit Contract on the Eth 1.0 chain:**_ the beacon chain listens for the logs in the chain to queue that validator into the beacon chain chain's main event loop
2. _**Registered validator begins PoS process to propose blocks:**_ the PoS validator has the resposibility to participate in the addition of new blocks to the beacon chain
3. _**RANDAO mechanism selects committees of proposers/attesters for shards:**_ the beacon chain node will use its RANDAO mechanism to select committees of proposers, attesters that each have responsibilities within the system.
4. _**Beacon Chain State Advances, Committees are Reshuffled:**_ upon completing responsibilities, the different actors of the system are them reshuffled into new committees on different shards.
## The EVM: What You Need to Know
As an important aside, we’ll take a brief detour into the EVM and what we need to understand before we modify it for a sharded blockchain. At its core, the functionality of the EVM optimizes for _security_ and not for computational power with the following restrictions:
- Every single step must be paid for upfront with gas costs to prevent DDoS
- Programs can't interact with each other without a single byte array
- This also means programs can't access other programs' state
- Sandboxed Execution - the EVM can only modify its internal state and nothing else
- Deterministic execution guarantees
So what exactly is the EVM? The EVM was purposely designed to be a stack based machine with memory-byte arrays and key-value stores that are kept on a trie
- Every single keys and storage values are 32 bytes
- There are 100 total opcodes in the EVM
- The EVM comes with a temporary memory byte-array and storage trie to hold persistent memory.
Cryptographic operations are done using pre-compiled contracts. Aside from that, the EVM provides a bunch of blockchain access-level context that allows certain opcodes to fetch useful information from the external system. For example, LOG opcodes store useful information in the log bloom filter that can be synced with light clients. This can be used as a low-gas form of storage, since LOG does not modify the state.
Additionally, the EVM contains a call-depth limit such that recursive invocations or chains of calls will eventually halt, preventing a drastic use of resources.
It is important to note that the merkle root of an Ethereum account is updated any time an `SSTORE` opcode is executed successfully by a program on the EVM that results in a key or value changing in the state merklix (merkle radix) tree.
How is this relevant to sharding? It is important to note the importance of certain opcodes in our implementation and how we will need to introduce and modify several of them for both security and scalability considerations in a sharded chain.
# Acknowledgements
A special thanks for entire [Prysmatic Labs](https://discord.gg/che9auJ) team for helping put this together and to Ethereum Research (Hsiao-Wei Wang, Vitalik, Justin Drake) for the help and guidance in our approach.
# References
[Sharding FAQ](https://github.com/ethereum/wiki/wiki/Sharding-FAQ)
[Sharding Reference Spec](https://github.com/ethereum/sharding/blob/develop/docs/doc.md)
[Data Availability and Erasure Coding](https://github.com/ethereum/research/wiki/A-note-on-data-availability-and-erasure-coding)
[Proof of Visibility for Data Availability](https://ethresear.ch/t/proof-of-visibility-for-data-availability/1073)
[History, State, and Asynchronous Accumulators in the Stateless Model](https://ethresear.ch/t/history-state-and-asynchronous-accumulators-in-the-stateless-model/287)
[Torus Shaped Sharding Network](https://ethresear.ch/t/torus-shaped-sharding-network/1720)
[Data Availability Proof-friendly State Tree Transitions](https://ethresear.ch/t/data-availability-proof-friendly-state-tree-transitions/1453)
[Safety Attester Pool Size](https://ethresear.ch/t/safe-attester-pool-size/1728)
[Fixed Size Deposits and Rewards Penalties Quadleak](https://ethresear.ch/t/fixed-size-deposits-and-rewards-penalties-quad-leak/2073/7)
[Two Ways To Do Cross Links](https://ethresear.ch/t/two-ways-to-do-cross-links/2074/2)
[Extending Minimal Sharding with Cross Links](https://ethresear.ch/t/two-ways-to-do-cross-links/2074/2)
[Leaderless K of N Random Beacon](https://ethresear.ch/t/leaderless-k-of-n-random-beacon/2046/3)