You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: tutorials/hobgoblin-closeloop.md
+54-8Lines changed: 54 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ The exercises below will help you become familiar with using the [Harp Hobgoblin
4
4
5
5
## Prerequisites
6
6
7
-
- Install the `Bonsai.Dsp` package from the [Bonsai package manager](https://bonsai-rx.org/docs/articles/packages.html).
7
+
- Install the `Bonsai.Dsp`, `Bonsai.Video` and `Bonsai.Vision` packages from the [Bonsai package manager](https://bonsai-rx.org/docs/articles/packages.html).
8
8
9
9
## Close-loop latency
10
10
In a closed-loop experiment, we want the behaviour data to generate feedback in real-time into the external world, establishing a relationship where the output of the system depends on detected sensory input. Many behavioural experiments in neuroscience require some kind of closed-loop interaction between the subject and the experimental setup.
@@ -19,10 +19,6 @@ Before beginning, set up the `Hobgoblin` with the following `device pattern` tha
19
19
20
20
- Set the `DumpRegisters` property in the `Hobgoblin`[`Device`] operator to `False`. This is to avoid triggering the command loop in the next exercise.
- Connect one of the LED modules to digital output channel `GP15` on the `Hobgoblin`.
25
-
26
22
### Exercise 1: Measuring serial port communication latency
27
23
28
24
We will take advantage of the device's ability to echo back a timestamped message upon command execution to create a simple loop where each echo re-triggers the same command (toggling the digital output channel `HIGH` and `LOW`). The time interval between the echoes will give us the total closed-loop latency of the system, also known as the round-trip time.
@@ -41,27 +37,76 @@ In order to measure the difference between the timestamps:
41
37
- Right-click on the [`Parse`] operator, select the `Output (Bonsai.Harp.Timestamped<Harp.Hobgoblin.DigitalOutputs>)` > `Seconds`.
42
38
- Disconnect the `Seconds` node from the [`CreateMessage`] operator.
43
39
- Reconnect the [`Parse`] and [`CreateMessage`] operator.
44
-
- After the `Seconds` node, insert a [`Difference`] operator from the `Bonsai.Dsp` package.
40
+
- After the `Seconds` node, insert a [`Difference`] operator.
45
41
46
-
Lastly, we will use this sequence to initialize the command loop:
42
+
Lastly, we will use this sequence to toggle the digital output and initialize the command loop:
47
43
48
44
- Insert a [`KeyDown`] operator and set the `Filter` property to the key `A`.
49
45
- Insert a [`Parse`] operator and select [`DigitalOutputTogglePayload`] from the `Register` property dropdown menu. Set the `DigitalOutputToggle` property to `GP15`
50
46
- Insert a [`MulticastSubject`] operator and configure the `Name` property to `Hobgoblin Commands`.
51
47
- Run the workflow, and open the visualizer for the [`Difference`] operator.
52
48
**What do you observe?**
53
49
50
+
### Exercise 2: Measuring video acquisition latency
- Run the workflow and set the `RegionOfInterest` property to a small area around the LED.
62
+
63
+
> [!Tip]
64
+
> You can use the visual editor for an easier calibration. While the workflow is running, right-click on the [`Crop`] transform and select `Show Default Editor` from the context menu or click in the small button with ellipsis that appears when you select the `RegionOfInterest` property.
65
+
66
+
- Insert a [`Sum`] transform and select the `Val0` field from the output.
67
+
68
+
> [!Note]
69
+
> The [`Sum`] operator adds the value of all the pixels in the image together, across all the color channels. Assuming the default BGR format, the result of summing all the pixels in the `Blue` channel of the image will be stored in `Val0`. `Val1` and `Val2` would store the `Green` and `Red` values, respectively. If you are using an LED with a color other than blue, please select the output field accordingly.
70
+
71
+
- Insert a [`GreaterThan`] transform.
72
+
- Insert a [`BitwiseNot`] transform.
73
+
- Insert a [`CreateMessage`] operator, select [`DigitalOutputTogglePayload`] for the `Payload`, and `GP15` for the [`DigitalOutputToggle`] property.
74
+
- Insert a [`MulticastSubject`] operator and configure the `Name` property to `Hobgoblin Commands`.
75
+
- Run the workflow and use the visualizer of the [`Sum`] operator to choose an appropriate threshold for [`GreaterThan`]. You can use the [`KeyDown`] toggle snippet from the previous exercise to manually toggle the LED.
76
+
- Insert a [`DistinctUntilChanged`] operator after the [`BitwiseNot`] transform.
77
+
78
+
> [!Note]
79
+
> The [`DistinctUntilChanged`] operator filters consecutive duplicate items from an observable sequence. In this case, we want to change the value of the LED only when the threshold output changes from `LOW` to `HIGH`, or vice-versa. This will let us measure correctly the latency between detecting a change in the input and measuring the response to that change.
80
+
81
+
In order to measure the round-trip time between the LED toggle:
82
+
83
+
:::workflow
84
+
85
+
:::
86
+
87
+
- Insert a [`SubscribeSubject`] operator and configure the `Name` property to `Hobgoblin Events`.
88
+
- Insert a [`Parse`] operator and select [`TimestampedDigitalOutputTogglePayload`] from the `Register` property dropdown menu.
89
+
- Right-click on the [`Parse`] operator, select `Output (Bonsai.Harp.Timestamped<Harp.Hobgoblin.DigitalOutputs>)` > `Seconds` from the context menu.
90
+
- Insert a [`Difference`] operator.
91
+
- Run the workflow and open the visualizer for the [`Difference`] operator.
92
+
93
+
_Given the measurements obtained in Exercise 2, what would you estimate is the **input** latency for video acquisition?_
0 commit comments