|
| 1 | +# SISAP 2025 Challenge: Working example in Python |
| 2 | + |
| 3 | +This repository is a working example for the SISAP 2025 Indexing Challenge <https://sisap-challenges.github.io/>, working with Python and GitHub Actions. |
| 4 | + |
| 5 | +## Steps for running |
| 6 | +It requires a working installation of Python 3.10 with conda support, for example using Anaconda. |
| 7 | + |
| 8 | +The steps are the following: |
| 9 | + |
| 10 | +1. Clone the example repository |
| 11 | +1. Run |
| 12 | +1. Evaluate |
| 13 | + |
| 14 | +The full set of installation instructions are listed in the GitHub Actions workflow |
| 15 | + |
| 16 | +<https://github.com/sisap-challenges/sisap25-example-python/blob/main/.github/workflows/ci.yml> |
| 17 | + |
| 18 | +Note that you will need to adjust your scripts to hold the correct hyperparameters for any benchmark you use, in particular, `pubmed23` and `gooaq` which will be used in the testing stage. |
| 19 | + |
| 20 | +### Clone this repository |
| 21 | +```base |
| 22 | +git clone https://github.com/sisap-challenges/sisap25-example-python |
| 23 | +cd sisap25-example-python |
| 24 | +``` |
| 25 | + |
| 26 | +### Run |
| 27 | +Run the tasks on an example input using |
| 28 | + |
| 29 | +```bash |
| 30 | +python search.py --task {task1, task2} |
| 31 | +``` |
| 32 | + |
| 33 | +It will automatically take care of downloading the necessary example dataset. |
| 34 | + |
| 35 | + |
| 36 | + |
| 37 | +### Evaluation |
| 38 | + |
| 39 | +```bash |
| 40 | +python eval.py results.csv |
| 41 | +``` |
| 42 | +will produce a summary file of the results with the computed recall against the ground truth data. |
| 43 | + |
| 44 | +This csv file can be further processed to create plots (using `python plot.py --task {task1, task2}`) and show the fastest solutions above a certain recall threshold (using `python show_operating_points.py`). |
| 45 | + |
| 46 | +## How to take this to create my own system |
| 47 | +You can fork this repository and polish it to create your solution. Please also take care of the ci workflow (see below). |
| 48 | + |
| 49 | +## GitHub Actions: Continuous integration |
| 50 | + |
| 51 | +You can monitor your runnings in the "Actions" tab of the GitHub panel: for instance, you can see some runs of this repository: |
| 52 | +<https://github.com/sisap-challenges/sisap25-example-python/actions> |
| 53 | + |
| 54 | + |
0 commit comments