You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: benchmarks/README.md
+93-35Lines changed: 93 additions & 35 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,60 +4,63 @@ If you care only about numbers, you can find them in the results dir.
4
4
5
5
Some background and details:
6
6
* all benchmarks were run on [DigitalOcean](https://www.digitalocean.com/) infrastructure
7
-
* benchmarks were run with both single Postgres instance serving as events backend as well as multiple (sharding)
8
-
*there are the following components:
9
-
*`app` - simple Spring Boot that uses EventSQL to consume events
10
-
*`runner` - script that uses EventSQL to publish events with set per second rate and amount and waits for consumers to finish consumption, gathering relevant stats (it's running benchmarks)
11
-
*`events-db` - Postgres serving as a backend for EventSQL; events are published and consumed from it.
12
-
Depending on the benchmark, we run it in one or a few (3) instances
13
-
* most of the setup to run benchmarks is automated and described below, so it's fairly easy to reproduce
7
+
* benchmarks were run with both single Postgres instance serving as the events backend as well as multiple (sharding)
8
+
*we have the following components:
9
+
*`app` - simple Spring Boot that uses *EventSQL* to consume events
10
+
*`runner` - script that uses *EventSQL* to publish events with set per second rate and amount, waits for consumers to finish consumption and gathers relevant stats (it's running benchmarks)
11
+
*`events-db` - Postgres serving as a backend for *EventSQL* - events are published to and consumed from it;
12
+
depending on the benchmark, it's run in one or a few (3) instances
13
+
* most of the setup to run benchmarks is automated and described below; it's fairly straightforward to reproduce
14
14
15
15
## Infrastructure
16
16
17
-
Defined in `prepare_infra.py`; sometimes resources are limited by `docker run`, but essentially:
18
-
* app (consumer) runs on 2 GB and with 2 CPUs (AMD) machine
19
-
* each events-db runs on 8 GB and with 4 CPUs (AMD) machine
20
-
* each benchmarks-runner runs alongside events-db, but is throttled to 2 GB memory and 2 CPUs
21
-
* there is a basic firewall and virtual private network (vpc) setup, so that nobody is bothering us during tests
17
+
Defined in the `prepare_infra.py` script; sometimes resources are limited by `docker run` command, but essentially:
18
+
**benchmarks-app (consumer)* runs on 2 GB and 2 CPUs (AMD) machine
19
+
* each *events-db* runs on 8 GB and 4 CPUs (AMD) machine
20
+
* each *benchmarks-runner* runs alongside *events-db*, but is throttled to 2 GB memory and 2 CPUs
21
+
* there is a basic firewall and virtual private network (vpc) setup (`prepare_infra.py`), so that nobody is bothering us during benchmarks
22
22
23
23
## Requirements
24
24
25
-
* DigitalOcean account - you might also use different infrastructure provider, but will need to adjust `prepare_infra.py` script accordingly or write your own
25
+
* DigitalOcean account - you might also use a different infrastructure provider but will need to adjust `prepare_infra.py` script accordingly or write your own setup from scratch
26
26
* Python 3 & Bash for scripts
27
27
* Java 21 + compatible Maven version to build apps
28
-
* Docker to dockerize them and run various command (scripts assume non-root, current user, access)
28
+
* Docker to dockerize them and run various commands (scripts assume non-root, current user, access)
29
29
30
30
## Preparation
31
31
32
32
### Infra
33
33
34
-
From scripts, dir, Python env setup:
34
+
From scripts dir, Python env setup:
35
35
```
36
36
bash init_python_env.bash
37
37
source venv/bin/activate
38
38
```
39
39
40
-
Prepare infra; this can take a while, since we are creating a few machines - one for the consumer app and three for multiple Postgres instances.
40
+
The following might take a while, since we are creating a few machines - one for the consumer app and three for multiple Postgres instances:
41
41
```
42
42
export DO_API_TOKEN=<your DigitalOcean API key>
43
-
export SSH_KEY_FINGERPRINT=<fingerprint of your ssh key, uploaded to DigitalOcean, giving you ssh access to machines>
43
+
export SSH_KEY_FINGERPRINT=<fingerprint of your ssh key, uploaded to DigitalOcean, giving you ssh access to created machines>
44
44
45
45
python prepare_infra.py
46
46
```
47
47
48
-
We right now have 4 machines connected with each other by the vpc.
48
+
After it finishes, on the DigitalOcean UI we should see something like this:
49
+
50
+
51
+
We right now have four machines connected to each other by the vpc.
49
52
To each we have access, using ssh public key authentication, as the `eventsql` user.
Private ips can be taken from DigitalOcean UI - only they will work, public ips will not, since we have set up a firewall blocking traffic of this kind.
72
+
Private ips can be taken from the DigitalOcean UI - only they will work, public ips will not, since we have set up a firewall blocking traffic of this kind.
70
73
71
74
Finally, let's build `runner`:
72
75
```
73
76
export APP=runner
74
77
bash build_and_package.bash
75
78
```
76
79
77
-
### Deploy apps
80
+
### Deployment
78
81
79
-
As all apps are now ready, let's deploy them!
82
+
As all apps are now packaged and ready, let's deploy them!
80
83
81
-
We deploy by copying gzipped Docker images + load and run scripts to the target machines.
84
+
We deploy by copying gzipped Docker images alongside with load and run scripts to the target machines.
82
85
83
-
Three events-dbs:
86
+
Three `events-dbs`:
84
87
```
85
88
export EVENTS_DB0_HOST=<ip of events-db-0 machine"
86
89
export EVENTS_DB1_HOST=<ip of events-db-1 machine"
87
90
export EVENTS_DB2_HOST=<ip of events-db-2 machine"
88
91
bash deploy_events_dbs.bash
89
92
```
90
93
91
-
App:
94
+
`app`:
92
95
```
93
96
export APP_HOST=<ip of consumer app machine>
94
97
bash deploy_app.bash
95
98
```
96
99
97
100
All dbs and app are running now.
98
-
With runners it is slightly different - we will copy them to target machines, but not run them just yet.
99
-
They will run on the same machines dbs are hosted; each db has a corresponding benchmarksrunner:
101
+
With `benchmark-runners` it is slightly different - we will copy them to target machines but not run just yet.
102
+
They will run on the same machines dbs are hosted; each db has a corresponding benchmarks-runner:
100
103
```
101
104
export EVENTS_DB0_HOST=<ip of events-db-0 machine"
102
105
export EVENTS_DB1_HOST=<ip of events-db-1 machine"
@@ -112,16 +115,25 @@ Everything is now ready to run various benchmarks.
112
115
113
116
Let's start with single db cases.
114
117
115
-
First, copy and run `collect_docker_stats.bash` script to one of the events dbs machine and start collecting them:
118
+
First, copy and run `collect_docker_stats.bash` script to one of the events dbs machine and start collecting stats:
0 commit comments