Skip to content

Commit e9cc3d1

Browse files
tjaneztjanez
authored
Merge pull request #33 from oasisprotocol/tjanez/node-hardware-requirements
run-a-node: Refactor Hardware Requirements page
2 parents 9b8c3d6 + 5942baf commit e9cc3d1

1 file changed

Lines changed: 142 additions & 19 deletions

File tree

‎docs/general/run-a-node/prerequisites/hardware-recommendations.md‎

Lines changed: 142 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,92 @@
11
# Hardware Requirements
22

3-
:::info
3+
The Oasis Network is composed of multiple classes of nodes that participate in
4+
different committees.
45

5-
The hardware requirements listed on this page are the suggested **minimum requirements**. It might be possible to configure a system with less resources, but you run the risk of being underprovisioned and thereby prone to loss of stake.
6+
This page describes the **minimum** and **recommended** system hardware
7+
requirements for running different types of nodes on the Oasis Network.
68

7-
:::
9+
:::caution
810

9-
The Oasis Network is composed of multiple classes of nodes that participate in different committees. The majority of committees have common system configurations for the participant nodes.
11+
If you configure a system with less resources than the recommended values, you
12+
run the risk of being underprovisioned and causing proposer node timeouts.
13+
This could result in losing stake.
14+
15+
:::
1016

1117
## Consensus Nodes <a id="suggested-minimum-configurations"></a>
1218

13-
To run a non-validator or a validator consensus node, your system should meet the following minimum system requirements:
19+
To run a non-validator or a validator consensus node, your system should meet
20+
the following requirements:
21+
22+
### CPU
1423

15-
* 2.0 GHz x86-64 CPU
24+
* Minimum: 2.0 GHz x86-64 CPU with [AES instruction set] support
25+
* Recommended: 2.0 GHz+ x86-64 CPU with 2 cores or 2 such virtual CPUs with
26+
[AES instruction set] and [AVX2] support
1627

1728
:::caution
1829

19-
The CPU must have [AES-NI](https://en.wikipedia.org/wiki/AES_instruction_set) support.
30+
The [AES instruction set] support is required by [Deoxys-II-256-128], a
31+
Misuse-Resistant Authenticated Encryption (MRAE) algorithm, which is used for
32+
encrypting ParaTime's state.
33+
34+
:::
35+
36+
:::info
37+
38+
The [Advanced Vector Extensions 2 (AVX2)][AVX2] support enables faster Ed25519
39+
signature verification which in turn makes a node sync faster.
2040

2141
:::
2242

23-
* 4 GB ECC RAM
43+
[AES instruction set]: https://en.wikipedia.org/wiki/AES_instruction_set
44+
[Deoxys-II-256-128]: https://sites.google.com/view/deoxyscipher
45+
[AVX2]:
46+
https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Advanced_Vector_Extensions_2
47+
48+
### Memory
49+
50+
* Minimum: 4 GB of ECC RAM
51+
* Recommended: 8 GB of ECC RAM
2452

2553
:::info
2654

27-
Ordinary node operation can work with 2 GB of RAM.
55+
Ordinary node operation can work with less memory, e.g. 4 GB of RAM.
2856

29-
However, at certain time points, it absolutely requires at least 4 GB of RAM. Examples of such more resource intensive time points are the initial state sync, BadgerDB migration when upgrading a node to Oasis Core 21.2.x, ...
57+
However, at certain time points, the node will absolutely require more memory.
58+
Examples of such more resource intensive time points are the initial state sync,
59+
BadgerDB migration when upgrading a node to Oasis Core 21.2.x, generating
60+
storage checkpoints with BadgerDB, periodic BadgerDB compactions...
3061

31-
If you can dynamically change the amount of RAM a node is provisioned, then feel free to downscale/upscale it as you see fit.
62+
If the system will not have enough memory, that will result in the Oasis node
63+
process being killed forcefully by the OOM process.
64+
Oasis node being killed forcefully could lead to BadgerDB state corruption
65+
and/or losing stake.
3266

3367
:::
3468

69+
### Storage
70+
71+
* Minimum: 300+ GB of SSD or NVMe fast storage
72+
* Recommended: 500+ GB of SSD or NVMe fast storage
3573

74+
:::caution
75+
76+
Consensus state is stored in an embedded [BadgerDB](https://dgraph.io/docs/badger/)
77+
database which was [designed to run on SSDs](https://dgraph.io/docs/badger/design/).
3678

37-
* 100+ GB High Speed Storage
79+
Hence, we **strongly discourage** trying to run a node that stores data
80+
**on classical HDDs**.
81+
82+
:::
3883

3984
:::info
4085

4186
The network accumulates state over time. The speed at which the state grows depends on the network's usage.
4287

43-
For example, the Mainnet accumulated over 80 GB of state in 5+ months between [Mainnet launch](../../mainnet/previous-upgrades/mainnet-upgrade.md) (Nov 18, 2020) and [Cobalt upgrade](../../mainnet/cobalt-upgrade.md) (Apr 28, 2021).
88+
For example, the Mainnet accumulated over 250 GB of state in ~11 months between
89+
the [Cobalt upgrade](../../mainnet/cobalt-upgrade.md) (Apr 28, 2021) and Mar 15, 2022.
4490

4591
:::
4692

@@ -60,22 +106,99 @@ To do that, set the **`consensus.tendermint.abci.prune.strategy`** and **`consen
60106

61107
## ParaTime Nodes
62108

63-
To run a ParaTime node, your system should meet the following minimum system requirements:
109+
To run a ParaTime node, your system should meet the following requirements:
64110

65-
* 2.0 GHz x86-64 CPU
111+
### CPU
112+
113+
* Minimum: 2.0 GHz x86-64 CPU with [AES instruction set] support
114+
* Recommended: 2.0 GHz+ x86-64 CPU with 2 cores or 2 such virtual CPUs with
115+
[AES instruction set] and [AVX2] support
66116

67117
:::caution
68118

69-
The CPU must have [AES-NI](https://en.wikipedia.org/wiki/AES_instruction_set) support.
119+
The [AES instruction set] support is required by [Deoxys-II-256-128], a
120+
Misuse-Resistant Authenticated Encryption (MRAE) algorithm, which is used for
121+
encrypting ParaTime's state.
122+
123+
:::
124+
125+
:::info
126+
127+
The [Advanced Vector Extensions 2 (AVX2)][AVX2] support enables faster Ed25519
128+
signature verification which in turn makes a node sync faster.
129+
130+
:::
131+
132+
[AES instruction set]: https://en.wikipedia.org/wiki/AES_instruction_set
133+
[Deoxys-II-256-128]: https://sites.google.com/view/deoxyscipher
134+
[AVX2]:
135+
https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Advanced_Vector_Extensions_2
136+
137+
:::info
138+
139+
If you want to run ParaTimes which require the use of a Trusted Execution
140+
Environment (TEE), the CPU also needs to support [Intel SGX].
141+
142+
:::
143+
144+
[Intel SGX]:
145+
https://www.intel.com/content/www/us/en/architecture-and-technology/software-guard-extensions.html
146+
147+
### Memory
148+
149+
* Minimum: 8 GB of ECC RAM
150+
* Recommended: 16 GB of ECC RAM
151+
152+
:::info
153+
154+
Ordinary node operation can work with less memory, e.g. 8 GB of RAM.
155+
156+
However, at certain time points, the node will absolutely require more memory.
157+
Examples of such more resource intensive time points are the initial state sync,
158+
generating storage checkpoints with BadgerDB, periodic BadgerDB compactions...
159+
160+
If the system will not have enough memory, that will result in the Oasis node
161+
process being killed forcefully by the OOM process.
162+
Oasis node being killed forcefully could lead to BadgerDB state corruption
163+
and/or losing stake.
70164

71165
:::
72166

167+
### Storage
168+
169+
* Minimum: 500+ GB of SSD or NVMe fast storage
170+
* Recommended: 800+ GB of SSD or NVMe fast storage
171+
73172
:::caution
74173

75-
If you want to be able to run ParaTimes which require the use of a Trusted Execution Environment (TEE), the CPU needs to support Intel SGX.
174+
Consensus state is stored in an embedded [BadgerDB](https://dgraph.io/docs/badger/)
175+
database which was [designed to run on SSDs](https://dgraph.io/docs/badger/design/).
176+
177+
Hence, we **strongly discourage** trying to run a node that stores data
178+
**on classical HDDs**.
179+
180+
:::
181+
182+
:::info
183+
184+
The consensus layer and the ParaTimes accumulate state over time.
185+
The speed at which the state grows depends on the network's and ParaTimes' usage.
186+
187+
For example, a node running the Emerald ParaTime on the Mainnet would currently
188+
(Mar 15, 2022)
189+
need to store:
190+
191+
- over 250 GBs of consensus state accumulated in ~11 months since the
192+
[Cobalt upgrade](../../mainnet/cobalt-upgrade.md) (Apr 28, 2021).
193+
- over 125 GBs of Emerald ParaTime state accumulated in ~4 months since the
194+
[Emerald Mainnet launch](https://medium.com/oasis-protocol-project/oasis-emerald-evm-paratime-is-live-on-mainnet-13afe953a4c9) (Nov 18, 2021).
76195

77196
:::
78197

79-
* 8 GB ECC RAM
80-
* 100+ GB High Speed Storage
198+
:::info
199+
200+
It is also possible to configure the Node to _not_ keep all the state from the genesis onward, reducing the amount of storage needed to keep the network's state.
81201

202+
To do that, set the **`consensus.tendermint.abci.prune.strategy`** and **`consensus.tendermint.abci.prune.num_kept`** parameters appropriately in your [Node's configuration](../set-up-your-node/run-validator.md#configuring-the-oasis-node).
203+
204+
:::

0 commit comments

Comments
 (0)