You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: project/index.html
+59-16Lines changed: 59 additions & 16 deletions
Original file line number
Diff line number
Diff line change
@@ -171,7 +171,7 @@ <h4>📊 Model Family Comparison</h4>
171
171
<divclass="box">
172
172
<divclass="content">
173
173
<h4>🎨 Feature Space Control</h4>
174
-
<p>We demonstrate that orthogonal rotations in feature space control color encoding, enabling predictable image manipulations and revealing the structured nature of the feature representations.</p>
174
+
<p>We demonstrate that linear transformations in feature space control color encoding of reconstructed images on <strong>three different tasks</strong>: colorization, red-blue channel swap, and blue channel suppression.</p>
175
175
</div>
176
176
</div>
177
177
</div>
@@ -258,51 +258,94 @@ <h3 class="title is-4">Q Matrix: A Tool for Feature Manipulation</h3>
Through our Q matrix framework, we demonstrate precise control over color attributes in the feature space. Our experiments reveal that color information is encoded through orthogonal rotations rather than spatial transformations.
261
+
Using simple linear transformations in feature space, we demonstrate precise control over color attributes of reconstructed image. Our color manipulation studies serve as a validation of the feature interpretation hypothesis, supported by an image reconstruction approach.
Colorization task - transforming grayscale images to their color counterparts. This problem requires following properties:</p>
269
+
270
+
<p>1. <strong>Semantic Requirement</strong>: Successful colorization necessitates that the feature space geometry encodes real-world knowledge about plausible color distributions for objects and scenes.</p>
271
+
272
+
<p>2. <strong>Non-algorithmic Nature</strong>: Colorization cannot be achieved through simple pixel-wise transformations but requires understanding of image semantics.
<p>Properties we would expect from Red-Blue Channel Swap operator:</p>
287
+
<ul>
288
+
<li><strong>Orthogonal</strong> — the operator should be orthogonal, meaning it should preserve the norm of the vector.</li>
289
+
<li><strong>Self-inverse</strong> — double application of Red Blue color swapping is Identity transformation in image space.</li>
290
+
</ul>
291
+
<p><strong>Eigenvalues of the operator</strong> will be close to +1 and -1.</p>
292
+
<p>As we will see further all this properties are somehow preserved even for Linear operator with no strict constraints on this properties.</p>
293
+
<p></p>
294
+
295
+
<p>We trained three different operators:</p>
296
+
<ol>
297
+
<li><strong>Orthogonal self-conjugated</strong> — as a Procrustes solution with a long-range projection of the operator onto the space of self-conjugated operators.</li>
298
+
<li><strong>Orthogonal</strong> — as a Procrustes solution.</li>
299
+
<li><strong>Linear</strong> — as a regression problem. <em>(Note that this solution cannot be directly used with the reconstructor, as it fails to preserve vector norms. Since the reconstructor was trained exclusively on normalized vectors, we first normalize the resulting outputs before feeding them to the reconstructor.)</em></li>
300
+
</ol>
301
+
302
+
<p>As shown in <strong>Figure 7</strong>, the eigenvalues of all operators cluster along the real axis, indicating they primarily represent either eigenvector preservation (near +1) or inversion (near -1). While small deviations from these ideal values exist — revealing noise in the feature space — these perturbations remain relatively weak. Consequently, the feature space geometry largely preserves the properties expected from the pixel-space channel permutation operator.</p>
<p><strong>Blue Channel Suppression operator</strong> will gradually suppress the blue channel of the image multiplying blue channel by some factor less than 1.</p>
325
+
326
+
<p>Properties we would expect from <strong>Blue Channel Suppression operator</strong>:</p>
327
+
<ul>
328
+
<li>Asymptotically this operator approaches a projection operator</li>
329
+
</ul>
330
+
</p>
331
+
<p><strong>Eigenvalues of the operator</strong> are either 1 or complex values with magnitude strictly less than 1.</p>
0 commit comments