Skip to content

Commit 57d7e3d

Browse files
author
Dmitrii Tarasov
committed
Merge remote-tracking branch 'upstream/main'
2 parents 889e06b + a4f3996 commit 57d7e3d

41 files changed

Lines changed: 2556 additions & 4 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 56 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -113,11 +113,13 @@ Example output:
113113
<div align="center">
114114
<figure>
115115
<img src="resources/v1_vs_v2.png" width="600">
116-
<figcaption>SigLIP vs SigLIP2 Feature Space Comparison</figcaption>
116+
<div>
117+
<figcaption>SigLIP vs SigLIP2 Feature Space Comparison</figcaption>
118+
</div>
117119
</figure>
118120
</div>
119121

120-
## Orthogonal Transformation Learning
122+
## Orthogonal Transformation Learning For R and B channels Swap
121123
To study orthogonal transformations in feature space:
122124

123125
1. Generate dataset for `google/siglip2-base-patch16-512`
@@ -135,7 +137,57 @@ Example output:
135137
<div align="center">
136138
<figure>
137139
<img src="resources/rb_swap.png" width="600">
138-
<figcaption>RGB Channel Swap in Feature Space</figcaption>
140+
<div>
141+
<figcaption>RGB Channel Swap in Feature Space</figcaption>
142+
</div>
143+
</figure>
144+
</div>
145+
146+
## Linear Transformation Learning For B Channel Suppression
147+
To study linear transformations in feature space:
148+
149+
1. Generate dataset for `google/siglip2-base-patch16-512`
150+
2. Train reconstructor or use precomputed [weights](https://drive.google.com/file/d/1i-B-5yBpSwcZL3_Z2Dz53jfxiY9T-fkb/view?usp=drive_link)
151+
3. Place weights at:
152+
```bash
153+
metrics_calculation/precalculated_weights/models--google--siglip2-base-patch16-512.pt
154+
```
155+
4. Run the analysis notebook:
156+
```
157+
metrics_calculation/b_channel_suppression/understanding_b_suppression.ipynb
158+
```
159+
160+
Example output:
161+
<div align="center">
162+
<figure>
163+
<img src="resources/b_suppression_all_transformations.png" width="600">
164+
<div>
165+
<figcaption>B Channel Suppression in Feature Space</figcaption>
166+
</div>
167+
</figure>
168+
</div>
169+
170+
## Linear Transformation Learning For Colorization
171+
To study linear transformations in feature space:
172+
173+
1. Generate dataset for `google/siglip2-base-patch16-512`
174+
2. Train reconstructor or use precomputed [weights](https://drive.google.com/file/d/1i-B-5yBpSwcZL3_Z2Dz53jfxiY9T-fkb/view?usp=drive_link)
175+
3. Place weights at:
176+
```bash
177+
metrics_calculation/precalculated_weights/models--google--siglip2-base-patch16-512.pt
178+
```
179+
4. Run the analysis notebook:
180+
```
181+
metrics_calculation/colorization/understanding_colorization.ipynb
182+
```
183+
184+
Example output:
185+
<div align="center">
186+
<figure>
187+
<img src="resources/colorized_all_examples.png" width="600">
188+
<div>
189+
<figcaption>Colorization in Feature Space</figcaption>
190+
</div>
139191
</figure>
140192
</div>
141193

@@ -153,4 +205,4 @@ If you find this work useful, please cite it as follows:
153205
primaryClass={cs.CV},
154206
url={https://arxiv.org/abs/2506.07803},
155207
}
156-
```
208+
```

0 commit comments

Comments
 (0)