@@ -87,7 +87,7 @@ <h1 class="title is-1 publication-title">Image Reconstruction as a Tool for Feat
8787 </ span >
8888
8989 <!-- ArXiv abstract Link -->
90- <!-- TODO: add arXiv link -->
90+ <!-- TODO: add arXiv link -->
9191 < span class ="link-block ">
9292 < a href ="https://arxiv.org/abs/<ARXIV PAPER ID> " target ="_blank "
9393 class ="external-link button is-normal is-rounded is-dark ">
@@ -134,111 +134,139 @@ <h2 class="title is-3">Abstract</h2>
134134 <!-- Прописать явно контрибушны: -->
135135
136136 <!-- (1) interpretability metric -->
137- <!-- Текстовое объяснение -->
138- <!-- Базовые результаты: siglip vs siglip2 -->
139- <!-- Нужно четко подчеркнуть какие различия между модельками -->
140- <!-- И как это влияет на реконструкцию -->
141-
142- < div class ="columns is-centered has-text-centered ">
143- < div class ="column is-four-fifths ">
144- < h2 class ="title is-3 " style ="white-space: nowrap; "> Reconstruct images from feature space</ h2 >
145- < div class ="content has-text-justified ">
146- < div style ="text-align: center; ">
147- < img src ="./static/images/features_reconstruction.drawio.png " alt ="features_reconstruction " width ="900 ">
148- < p class ="caption " style ="width: 100%; text-align: center; ">
149- < b > Figure 1. Image reconstructor training.</ b > For pretrained model we train a reconstructor model that restores the image from the feature space.
150- </ p >
151- < img src ="./static/images/reconstruction_metrics.jpg " alt ="reconstruction_metrics " width ="900 ">
152- < p class ="caption " style ="width: 100%; text-align: center; ">
153- < b > Figure 2. Reconstruction Metrics.</ b > We show the results of the reconstruction for SigLip and SigLip2 for different image resultions.
154- </ p >
155- </ div >
156- </ div >
157- < br > < br >
158- </ div >
159- </ div >
160-
161- <!-- (2) Feature-space transformations -->
162- <!-- Текстовое объяснение -->
163- <!-- Визуализация фреймворка: обобщил оператор в пр-ве картинок и в пр-ве фичей -->
164- <!-- Примеры работы с RGB -->
165- <!-- Примеры работы с отключением одного канала (ожелтением) -->
166- <!-- Примеры Спектра такой м-цы, показать, что только небольшое кол-во каналов меняется -->
167- <!-- -->
168-
169- < div class ="columns is-centered has-text-centered ">
170- < div class ="column is-four-fifths ">
171- < h2 class ="title is-3 " style ="white-space: nowrap; "> Feature-space transformations. Q matrix Calculation and Application.</ h2 >
172- < div class ="content has-text-justified ">
173- < img src ="./static/images/features_reconstruction_manipulation_train_Q.drawio.png " alt ="features_reconstruction_manipulation_train_Q " width ="900 ">
137+ <!-- Текстовое объяснение -->
138+ <!-- Базовые результаты: siglip vs siglip2 -->
139+ <!-- Нужно четко подчеркнуть какие различия между модельками -->
140+ <!-- И как это влияет на реконструкцию -->
141+
142+ < div class ="columns is-centered has-text-centered ">
143+ < div class ="column is-four-fifths ">
144+ < h2 class ="title is-3 " style ="white-space: nowrap; "> Reconstruct images from feature space</ h2 >
145+ < div class ="content has-text-justified ">
146+ < div class ="columns is-centered ">
147+ < div class ="column is-half ">
148+ < img src ="./static/images/features_reconstruction.drawio.png " alt ="features_reconstruction " width ="100% ">
174149 < p class ="caption " style ="width: 100%; text-align: center; ">
175- < b > Figure 3. Feature-space transformations. Q matrix Calculation.</ b > We then calculate Q matrix for feature-space manupulation.
150+ < b > Figure 1. Image reconstructor training.</ b > For pretrained model we train a reconstructor model that
151+ restores the image from the feature space.
176152 </ p >
177- < div style ="text-align: center; ">
178- < img src ="./static/images/features_reconstruction_manipulation_eval_Q.drawio.png " alt ="features_reconstruction_manipulation_eval_Q " width ="900 ">
153+ </ div >
154+ < div class ="column is-half ">
155+ < img src ="./static/images/reconstruction_metrics.jpg " alt ="reconstruction_metrics " width ="100% ">
179156 < p class ="caption " style ="width: 100%; text-align: center; ">
180- < b > Figure 4. Feature-space transformations. Q matrix Application.</ b > After Q matrix is calculated, we apply it to the feature space. For each patch embedding.
157+ < b > Figure 2. Reconstruction Metrics.</ b > We show the results of the reconstruction for SigLip and SigLip2
158+ for different image resultions.
181159 </ p >
182160 </ div >
183161 </ div >
184- < br > < br >
185162 </ div >
163+ < br > < br >
164+ </ div >
165+ </ div >
166+
167+ <!-- (2) Feature-space transformations -->
168+ <!-- Текстовое объяснение -->
169+ <!-- Визуализация фреймворка: обобщил оператор в пр-ве картинок и в пр-ве фичей -->
170+ <!-- Примеры работы с RGB -->
171+ <!-- Примеры работы с отключением одного канала (ожелтением) -->
172+ <!-- Примеры Спектра такой м-цы, показать, что только небольшое кол-во каналов меняется -->
173+ <!-- -->
174+
175+ < div class ="columns is-centered has-text-centered ">
176+ < div class ="column is-four-fifths ">
177+ < h2 class ="title is-3 " style ="white-space: nowrap; "> Feature-space transformations. Q matrix Calculation and
178+ Application.</ h2 >
179+ < div class ="content has-text-justified ">
180+ < div style ="text-align: center; ">
181+ < img src ="./static/images/features_reconstruction_manipulation_train_Q.drawio.png "
182+ alt ="features_reconstruction_manipulation_train_Q " width ="900 ">
183+ < p class ="caption " style ="width: 100%; text-align: center; ">
184+ < b > Figure 3. Feature-space transformations. Q matrix Calculation.</ b > We then calculate Q matrix for
185+ feature-space manupulation.
186+ </ p >
187+ </ div >
188+ < div style ="text-align: center; ">
189+ < img src ="./static/images/features_reconstruction_manipulation_eval_Q.drawio.png "
190+ alt ="features_reconstruction_manipulation_eval_Q " width ="900 ">
191+ < p class ="caption " style ="width: 100%; text-align: center; ">
192+ < b > Figure 4. Feature-space transformations. Q matrix Application.</ b > After Q matrix is calculated, we apply
193+ it to the feature space. For each patch embedding.
194+ </ p >
195+ </ div >
196+ </ div >
197+ < br > < br >
186198 </ div >
199+ </ div >
187200
188201
189- < div class ="columns is-centered has-text-centered ">
190- < div class ="column is-four-fifths ">
191- < h2 class ="title is-3 " style ="white-space: nowrap; "> Feature-space transformations. Color Swap Examples.</ h2 >
192- < div class ="content has-text-justified ">
202+ < div class ="columns is-centered has-text-centered ">
203+ < div class ="column is-four-fifths ">
204+ < h2 class ="title is-3 " style ="white-space: nowrap; "> Feature-space transformations. Color Swap Examples.</ h2 >
205+ < div class ="content has-text-justified ">
193206
194- < img src ="./static/images/rb_swap.png " alt ="rb_swap " width ="900 ">
195- < p class ="caption " style ="width: 100%; text-align: center; ">
196- < b > Figure 5. Red-blue channel swap samples.</ b >
197- </ p >
198- < img src ="./static/images/rb_swap_eigenvalues.png " alt ="rb_swap_eigenvalues " width ="900 ">
199- < p class ="caption " style ="width: 100%; text-align: center; ">
200- < b > Figure 6. Eigenvalues for red-blue channel swap matrix.</ b > Majority of eigenvalues are close to 1, which means that the transformation is close to an identity matrix. While the other cluster of eigenvalues are close to -1, which means that for these channels direction is changed to the opposite.
201- </ p >
202- </ div >
203- </ div >
204- < br > < br >
207+ < div style ="text-align: center; ">
208+ < img src ="./static/images/rb_swap.png " alt ="rb_swap " width ="900 ">
209+ < p class ="caption " style ="width: 100%; text-align: center; ">
210+ < b > Figure 5. Red-blue channel swap samples.</ b >
211+ </ p >
212+ </ div >
213+
214+ < div style ="text-align: center; ">
215+ < img src ="./static/images/rb_swap_eigenvalues.png " alt ="rb_swap_eigenvalues " width ="900 ">
216+ < p class ="caption " style ="width: 100%; text-align: center; ">
217+ < b > Figure 6. Eigenvalues for red-blue channel swap matrix.</ b > Majority of eigenvalues are close to 1, which
218+ means that the transformation is close to an identity matrix. While the other cluster of eigenvalues are
219+ close to -1, which means that for these channels direction is changed to the opposite.
220+ </ p >
205221 </ div >
206222 </ div >
223+ </ div >
224+ < br > < br >
225+ </ div >
226+ </ div >
207227
208228
209- < div class ="columns is-centered has-text-centered ">
210- < div class ="column is-four-fifths ">
211- < h2 class ="title is-3 " style ="white-space: nowrap; "> Feature-space transformations. Blue Channel Suppression.</ h2 >
212- < div class ="content has-text-justified ">
229+ < div class ="columns is-centered has-text-centered ">
230+ < div class ="column is-four-fifths ">
231+ < h2 class ="title is-3 " style ="white-space: nowrap; "> Feature-space transformations. Blue Channel Suppression.</ h2 >
232+ < div class ="content has-text-justified ">
213233
214- < img src ="./static/images/b_suppression_all_transformations.png " alt ="b_suppression_all_transformations " width ="900 ">
234+ < div style ="text-align: center; ">
235+ < img src ="./static/images/b_suppression_all_transformations.png " alt ="b_suppression_all_transformations "
236+ width ="900 ">
215237 < p class ="caption " style ="width: 100%; text-align: center; ">
216238 < b > Figure 7. Blue channel suppression samples.</ b >
217239 </ p >
218- < img src ="./static/images/b_suppression_all_eigen_values.png " alt ="b_suppression_all_eigen_values " width ="900 ">
240+ </ div >
241+ < div style ="text-align: center; ">
242+ < img src ="./static/images/b_suppression_all_eigen_values.png " alt ="b_suppression_all_eigen_values "
243+ width ="900 ">
219244 < p class ="caption " style ="width: 100%; text-align: center; ">
220245 < b > Figure 8. Eigenvalues for blue channel suppression matrix.</ b >
221246 </ p >
222247 </ div >
223- </ div >
224- < br > < br >
225248 </ div >
226249 </ div >
250+ < br > < br >
251+ </ div >
252+ </ div >
227253
228254
229- < div class ="columns is-centered has-text-centered ">
230- < div class ="column is-four-fifths ">
231- < h2 class ="title is-3 " style ="white-space: nowrap; "> Feature-space transformations. Colorization.</ h2 >
232- < div class ="content has-text-justified ">
255+ < div class ="columns is-centered has-text-centered ">
256+ < div class ="column is-four-fifths ">
257+ < h2 class ="title is-3 " style ="white-space: nowrap; "> Feature-space transformations. Colorization.</ h2 >
258+ < div class ="content has-text-justified ">
233259
234- < img src ="./static/images/colorized_examples.png " alt ="colorization_all_transformations " width ="900 ">
235- < p class ="caption " style ="width: 100%; text-align: center; ">
236- < b > Figure 9. Colorization samples.</ b >
237- </ p >
238- </ div >
260+ < div style ="text-align: center; ">
261+ < img src ="./static/images/colorized_examples.png " alt ="colorization_all_transformations " width ="900 ">
262+ < p class ="caption " style ="width: 100%; text-align: center; ">
263+ < b > Figure 9. Colorization samples.</ b >
264+ </ p >
265+ </ div >
239266 </ div >
240- < br > < br >
241267 </ div >
268+ < br > < br >
269+ </ div >
242270 </ div >
243271
244272
0 commit comments