Skip to content

Commit 95eacc6

Browse files
author
Harsh
committed
update the README
1 parent 7390847 commit 95eacc6

3 files changed

Lines changed: 11630 additions & 1115 deletions

File tree

‎README.md‎

Lines changed: 98 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
## Benchmark
2-
**Benchmark** is a simple benchmarking tool for GPU.js. This tool works both in JavaScript and in CLI.
3-
This tool runs two benchmarks:
2+
**Benchmark** is a simple benchmarking tool for GPU.js. This tool works both in JavaScript and CLI.
3+
This tool runs three benchmarks:
44
1. [Matrix Multiplication](#matrix-multiplication)
55
2. [Matrix Convolution](#matrix-convolution)
66
3. [Pipelining](#pipelining)
@@ -9,6 +9,7 @@ This tool runs two benchmarks:
99
- [Installation](#installation)
1010
- [Browser Usage](#browser-usage)
1111
- [Usage](#usage)
12+
- [Multiple Benchmarks](#multiple-benchmarks)
1213
- [Output](#output)
1314
- [Options](#options)
1415
- [Stats](#stats)
@@ -117,9 +118,64 @@ yarn start '{"num_benchmarks": 4}'
117118
```sh
118119
yarn start --multiple options
119120
```
120-
[options](#options) to the CLI are stored in stringified JSON object passed as an arguement.
121+
[options](#options) to the CLI are stored in a stringified JSON object passed as an argument.
121122
More about [Multiple Benchmarks](#multiple-benchmarks).
122123

124+
#### Multiple Benchmarks
125+
**Benchmark** allows you to run a sequence of benchmarks each with different custom options or each having number options like matrix size changed by a fixed amount.
126+
127+
```js
128+
benchmark.multipleBenchmark(options);
129+
```
130+
Where options is an object with the following properties:
131+
- `commonOptions`(*Object*): Options common to every benchmark in a sequence. (default: `{cpu_benchmark: false}`)
132+
- `range`(*Object*): Define a range of option(number type) values, one for each benchmark in the sequence. *e.g.*: matrix_size: 512, 1024, 1536... or matrix_size: 512, 1024, 2048 ...
133+
Here, the specified option can either be incremented by a fixed number(common difference) or multiplied by a fixed number(common factor).
134+
- - `optionName`(*String*): The name of the option for which the range is to be set. *e.g.*: matrix_size (Default: `matrix_size`)
135+
- - `interval`(*Array*): An array with upper and lower limits for the range. *e.g.*: [512, 2048] (Default: `[128, 1024]`)
136+
- - `step`(*Number*): The fixed number which is to be added(common difference). (Default: `100`)
137+
- - `commonRatio`(*Number*): The fixed number to be multiplied. (Default: none)
138+
###### NOTE: Only one of `step` and `commonRatio` can be used
139+
- `fullOptions`(*Array*): An array of objects specifying separate set of options for each benchmark in the sequence(commonOptions properties can be overridden here). (Default: none)
140+
###### NOTE: Only one of `range` and `fullOptions` can be used
141+
142+
##### Examples
143+
1. Range:
144+
```js
145+
benchmark.multipleBenchmark({
146+
commonOptions: {
147+
cpu_benchmark: false,
148+
logs: false
149+
},
150+
range: {
151+
optionName: 'matrix_size',
152+
interval: [128, 2048],
153+
commonRatio: 2
154+
}
155+
})
156+
```
157+
The above code runs a separate benchmark for the matrix sizes 128, 256, 512, 1024, 2048 which are in GP.
158+
159+
2. fullOptions:
160+
```js
161+
benchmark.multipleBenchmark({
162+
commonOptions: {
163+
logs: false,
164+
cpu_benchmark: false
165+
},
166+
fullOptions: [
167+
{
168+
logs: true, // override
169+
matrix_size: 2048
170+
},
171+
{
172+
cpu_benchmark: true, //override
173+
matrix_size: 128
174+
}
175+
]
176+
})
177+
```
178+
123179
#### Saving Graphs as JSON
124180
1. **Plotly Style JSON**
125181
```sh
@@ -148,7 +204,7 @@ This will log to the console, [chartist.js](https://gionkunz.github.io/chartist-
148204
```sh
149205
yarn start --multiple --saveChartistJSONToFile=path/to/file.json
150206
```
151-
This saves the [[chartist.js](https://gionkunz.github.io/chartist-js/) style JSON data for:
207+
This saves the [chartist.js](https://gionkunz.github.io/chartist-js/) style JSON data for:
152208
- GPU score v/s matrix size
153209
- GPU matrix multiplication run time v/s matrix size
154210
- CPU score v/s matrix size
@@ -201,5 +257,42 @@ The following options can be passed on to the `benchmark` or `multipleBenchmark`
201257
- `interval`(*Array*): The upper and lower limits for the option.
202258
- `step`(*Number*): The common difference between each option value. All the options will be in an AP. (only one of `step` or `commonRatio` can be used, preference given to `step`)
203259
- `commonRatio`(*Number*): The common ratio between each option value. All the options will be in a GP. (only one of `step` or `commonRatio` can be used, preference given to `step`)
204-
- `fullOptions`(*Array*): An array of options object, each one corresponding to one benchmark. Each Objects is the same as `benchmark` options. (only one of range or fullOptions can be used)
260+
- `fullOptions`(*Array*): An array of options object, each one corresponding to one benchmark. Each object is the same as `benchmark` options. (only one of range or fullOptions can be used)
261+
262+
#### Stats
263+
The [output](#output) contains a `stats` property which shows the overall stats of the benchmark:
264+
- `run_time`: The run time stats
265+
- - `mat_mult`, `mat_conv`, `pipe`(*Object*): These three objects contain the stats for each type of benchmark.
266+
- - - `diff`: Has a single property which cotains performance comparison scores between CPU and GPU.
267+
- - - - `cpu_gpu`:
268+
- - - - - `min`, `max`, `avg`: The minimum, maximum and average time taken stats
269+
- - - - - - `winner`(`gpu` | `cpu`): The better performer among the two.
270+
- - - - - - `percentage`(*Number*): By how much percentage it is better.
271+
272+
- `build_time`: The build time stats
273+
- - `mat_mult`, `mat_conv`: Built time stats for each benchmark.
274+
- - - `diff`: Same as the diff object in `run_time` except that it compares GPU v/s GPU(pipeline mode) in the property `gpu_pipe`.
275+
276+
- `overall`: The overall stats
277+
- - `mat_mult`, `mat_conv`: Overall stats for each benchmark
278+
- - - `best_performer`(`gpu` | `cpu`): The best overall performer.
279+
- - - `worst_performer`(`gpu` | `cpu`): The worst overall performer.
280+
- - - `diff`: Same as the diff object in `run_time`
281+
282+
#### Benchmarks
283+
284+
##### Matrix Multiplication
285+
This benchmark [multiplies](https://www.mathsisfun.com/algebra/matrix-multiplying.html) two randomly generated uniform-sized matrices and benchmarks the GPU and CPU against the time taken by each.
286+
287+
##### Matrix Convolution
288+
This benchmark [convolves](https://en.wikipedia.org/wiki/Kernel_(image_processing)#Convolution) a 3x3 [kernel](https://en.wikipedia.org/wiki/Kernel_(image_processing)) over a randomly generated uniform sized matrix.
289+
The convolution kernel is
290+
```
291+
1 2 1
292+
2 1 2
293+
1 2 1
294+
```
205295

296+
##### Pipelining
297+
GPU.js supports a feature called [Pipelining](https://github.com/gpujs/gpu.js#pipelining) and this benchmark benchmarks this feature.
298+
It runs three matrix multiplication benchmarks in a sequence while pipelining the output of the earlier benchmark to be used as an input to the next one. The benchmark is run both on the GPU and the CPU(without pipelining) and the time taken is compared.

0 commit comments

Comments
 (0)