This repo contains the files related to the WCNC 2025 ML Workshop Demo.
These experiments aim to validate model's integrity in two different forms:
- behavioral integrity: assuming a golden dataset, described as a dataset of ground-through border points, the model is evaluated against it to determine if the categorization falls in error or not
- structural integrity: the model structure is evaluated to determine if there are substantial changes
for each category the model is fine-tuned two times with a different percentage of fine-tune examples, then the fine-tuned versions are compared and the integrity is calculated.
Specifications:
- model: Resnet-50 (huggingface)
- dataset used for fine-tuning: animals (huggingface)
Experiment execution:
- download the fine-tune dataset:
git submodule update --init - generate two different fine-tuned versions of model (we assumed one-trained with 50% of fine-tuning dataset and one with 100%)
- calculate integrity:
- CLI version:
python check_model_integrity.py - Minimal Flask UI:
python main.py
- CLI version: