Skip to content

Commit c1f8ee8

Browse files
banchan86banchan86
committed
Add 4th exercise on close loop ROI
1 parent aec622a commit c1f8ee8

3 files changed

Lines changed: 223 additions & 2 deletions

File tree

‎tutorials/hobgoblin-closeloop.md‎

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ Lastly, we will use this sequence to toggle the digital output and initialize th
5656
![Hobgoblin Closed-Loop Latency Video](../workflows/hobgoblin-closeloop-latency-video.bonsai)
5757
:::
5858

59-
- Insert a [`VideoCaptureDevice`] operator.
59+
- Insert a [`VideoCaptureDevice`] operator. Set the `Index` property to the right camera.
6060
- Insert a [`Crop`] transform.
6161
- Run the workflow and set the `RegionOfInterest` property to a small area around the LED.
6262

@@ -124,10 +124,31 @@ To better understand what each parameter controls, try the following modificatio
124124
>[!TIP]
125125
> If your cameras support external triggering, you can use pulse trains to trigger frame capture. Recording the same pulse train on a digital input with the `Hobgoblin` allows you to have hardware timestamped images that are automatically aligned with other acquired data.
126126
127+
### Exercise 4: Triggering a digital line based on region of interest activity
128+
129+
:::workflow
130+
![Hobgoblin Pulse Train](../workflows/hobgoblin-closeloop-roi.bonsai)
131+
:::
132+
133+
- Insert a [`VideoCaptureDevice`] operator. Set the `Index` property to the right camera.
134+
- Insert a [`Crop`] transform.
135+
- Run the workflow and set the `RegionOfInterest` property to specify the desired area.
136+
- Insert a [`Grayscale`] and a [`Threshold`] transform (or the color segmentation operators).
137+
- Insert a [`Sum`] transform, and select the `Val0` field from the output.
138+
- Insert a [`GreaterThan`] transform and configure the `Value` property to an appropriate threshold. Remember you can use the visualizers to see what values are coming through the [`Sum`] and what the result of the [`GreaterThan`] operator is.
139+
- Insert a [`CreateMessage`] operator, select [`DigitalOutputSetPayload`] for the `Payload`, and `GP15` for the [`DigitalOutputSet`] property.
140+
- Insert a [`MulticastSubject`] operator and configure the `Name` property to `Hobgoblin Commands`.
141+
- Run the workflow and verify that entering the region of interest turns on the LED.
142+
- **Optional:** Replace the [`Crop`] transform by a [`CropPolygon`] to allow for non-rectangular regions.
143+
144+
> [!Note]
145+
> The [`CropPolygon`] operator uses the `Regions` property to define multiple, possibly non-rectangular regions. The visual editor is similar to [`Crop`], where you draw a rectangular box. However, in [`CropPolygon`] you can move the corners of the box by right-clicking _inside_ the box and dragging the cursor to the new position. You can add new points by double-clicking with the left mouse button, and delete points by double-clicking with the right mouse button. You can delete regions by pressing the `Del` key and cycle through selected regions by pressing the `Tab` key.
146+
127147
<!--Reference Style Links -->
128148
[`BitwiseNot`]: xref:Bonsai.Expressions.BitwiseNotBuilder
129149
[`CreateMessage`]: xref:Harp.Hobgoblin.CreateMessage
130150
[`Crop`]: xref:Bonsai.Vision.Crop
151+
[`CropPolygon`]: xref:Bonsai.Vision.CropPolygon
131152
[`Device`]: xref:Harp.Hobgoblin.Device
132153
[`Difference`]: xref:Bonsai.Dsp.Difference
133154
[`DigitalOutputToggle`]: xref:Harp.Hobgoblin.DigitalOutputToggle
@@ -136,8 +157,9 @@ To better understand what each parameter controls, try the following modificatio
136157
[`DigitalOutputSet`]: xref:Harp.Hobgoblin.DigitalOutputSet
137158
[`DigitalOutputClear`]: xref:Harp.Hobgoblin.DigitalOutputClear
138159
<!-- [`DigitalOutputClearPayload`]: xref:Harp.Hobgoblin.CreateDigitalOutputSetPayload -->
139-
<!-- [`DigitalOutputSetPayload`]: xref:Harp.Hobgoblin.CreateDigitalOutputClearPayload -->
160+
[`DigitalOutputSetPayload`]: xref:Harp.Hobgoblin.CreateDigitalOutputClearPayload
140161
[`GreaterThan`]: xref:Bonsai.Expressions.GreaterThanBuilder
162+
[`Grayscale`]: xref:Bonsai.Vision.Grayscale
141163
<!-- [`HarpMessage`]: xref:Bonsai.Harp.HarpMessage -->
142164
[`KeyDown`]: xref:Bonsai.Windows.Input.KeyDown
143165
<!-- [`Merge`]: xref:Bonsai.Reactive.Merge -->
@@ -149,6 +171,7 @@ To better understand what each parameter controls, try the following modificatio
149171
[`StopPulseTrainPayload`]: xref:Harp.Hobgoblin.CreateStopPulseTrainPayload
150172
[`SubscribeSubject`]: xref:Bonsai.Expressions.SubscribeSubject
151173
[`Sum`]: xref:Bonsai.Dsp.Sum
174+
[`Threshold`]: xref:Bonsai.Vision.Threshold
152175
[`TimestampedDigitalOutputTogglePayload`]: xref:Harp.Hobgoblin.TimestampedDigitalOutputToggle
153176
<!-- [`TimestampedDigitalOutputSet`]: xref:Harp.Hobgoblin.TimestampedDigitalOutputSet -->
154177
<!-- [`TimestampedDigitalOutputClear`]: xref:Harp.Hobgoblin.TimestampedDigitalOutputClear -->

‎workflows/hobgoblin-closeloop-roi.bonsai‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
<?xml version="1.0" encoding="utf-8"?>
2+
<WorkflowBuilder Version="2.9.0"
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xmlns:vid="clr-namespace:Bonsai.Video;assembly=Bonsai.Video"
5+
xmlns:cv="clr-namespace:Bonsai.Vision;assembly=Bonsai.Vision"
6+
xmlns:dsp="clr-namespace:Bonsai.Dsp;assembly=Bonsai.Dsp"
7+
xmlns:p1="clr-namespace:Harp.Hobgoblin;assembly=Harp.Hobgoblin"
8+
xmlns:harp="clr-namespace:Bonsai.Harp;assembly=Bonsai.Harp"
9+
xmlns="https://bonsai-rx.org/2018/workflow">
10+
<Workflow>
11+
<Nodes>
12+
<Expression xsi:type="Combinator">
13+
<Combinator xsi:type="vid:VideoCaptureDevice">
14+
<vid:Index>2</vid:Index>
15+
<vid:CaptureProperties />
16+
</Combinator>
17+
</Expression>
18+
<Expression xsi:type="Combinator">
19+
<Combinator xsi:type="cv:Crop">
20+
<cv:RegionOfInterest>
21+
<cv:X>0</cv:X>
22+
<cv:Y>0</cv:Y>
23+
<cv:Width>0</cv:Width>
24+
<cv:Height>0</cv:Height>
25+
</cv:RegionOfInterest>
26+
</Combinator>
27+
</Expression>
28+
<Expression xsi:type="Combinator">
29+
<Combinator xsi:type="cv:Grayscale" />
30+
</Expression>
31+
<Expression xsi:type="Combinator">
32+
<Combinator xsi:type="cv:Threshold">
33+
<cv:ThresholdValue>128</cv:ThresholdValue>
34+
<cv:MaxValue>255</cv:MaxValue>
35+
<cv:ThresholdType>Binary</cv:ThresholdType>
36+
</Combinator>
37+
</Expression>
38+
<Expression xsi:type="Combinator">
39+
<Combinator xsi:type="dsp:Sum" />
40+
</Expression>
41+
<Expression xsi:type="MemberSelector">
42+
<Selector>Val0</Selector>
43+
</Expression>
44+
<Expression xsi:type="GreaterThan">
45+
<Operand xsi:type="DoubleProperty">
46+
<Value>0</Value>
47+
</Operand>
48+
</Expression>
49+
<Expression xsi:type="p1:CreateMessage">
50+
<harp:MessageType>Write</harp:MessageType>
51+
<harp:Payload xsi:type="p1:CreateDigitalOutputSetPayload">
52+
<p1:DigitalOutputSet>GP15</p1:DigitalOutputSet>
53+
</harp:Payload>
54+
</Expression>
55+
<Expression xsi:type="MulticastSubject">
56+
<Name>Hobgoblin Commands</Name>
57+
</Expression>
58+
</Nodes>
59+
<Edges>
60+
<Edge From="0" To="1" Label="Source1" />
61+
<Edge From="1" To="2" Label="Source1" />
62+
<Edge From="2" To="3" Label="Source1" />
63+
<Edge From="3" To="4" Label="Source1" />
64+
<Edge From="4" To="5" Label="Source1" />
65+
<Edge From="5" To="6" Label="Source1" />
66+
<Edge From="6" To="7" Label="Source1" />
67+
<Edge From="7" To="8" Label="Source1" />
68+
</Edges>
69+
</Workflow>
70+
</WorkflowBuilder>

0 commit comments

Comments
 (0)