Multispectral Beetroots

Segmentaiton of betteroorts and weeds utilising multispectral images, focused on reducing number of manual annotations required.

This study demonstrates the effectiveness of multispectral images in reducing the need for extensive manual annotations in crop segmentation tasks, as they bring models more information and allow them to learn faster. Through semi-supervised learning techniques, the benefits are visible even when the final model operates solely on RGB images. A teacher model trained on multispectral data can generate synthetic ground truth data for training an RGB model. This approach enabled a significant improvement in the IoU metric from 0.8137 to 0.8501, utilising the same amount of labelled data alongside a larger unlabelled dataset. The experiments showed nearly a tenfold reduction in necessary labelled data to achieve comparable metrics for the RGB model.

Example image from the dataset, in different modalities:

Training with Additional Unlabelled Data

Diagram below shows the idea of the pipeline utilisng multispectral images only during the training phase:

To run a single training iteration see:

• trainign script: text
• config file: text

To run a set of experiments:

tmux
. venv/bin/activate
PYTHONPATH=$PYTHONPATH:`pwd`
python3 src/main.py

Foundation models impact

Segment Anything Module can be utilised to generate segmentation masks, achieving better results while operating on a false-color image with NIR band, than with RGB images.

See script text

Data download

Download and unpack the unload script:

wget https://www.ipb.uni-bonn.de/datasets_IJRR2017/ijrr_download_scripts.zip
unzip ijrr_download_scripts.zip
mkdir dataset
cd ijrr_download_scripts/
./download_ijrr_sugar_beet_2016_raw_data.sh ../dataset/

Or optionally dwonload only part of the data (160523):

https://www.ipb.uni-bonn.de/datasets_IJRR2017/raw_data/160523/
# for dates from 10:37 to 11:12

And unpack

unzip "*.zip"

Download annotations and put them in ./data/dataset:

https://www.ipb.uni-bonn.de/datasets_IJRR2017/annotations/

Citation

This repository is a supportive code for resarch paper:

@inproceedings{aszkowski2024streamlining,
  title={Streamlining Crop Segmentation with Multispectral Imaging and Foundation Models: Minimizing Manual Annotation},
  author={Aszkowski, Przemys{\l}aw and Kraft, Marek},
  booktitle={2024 IEEE 20th International Conference on Intelligent Computer Communication and Processing (ICCP)},
  pages={1--8},
  year={2024},
  organization={IEEE}
}