[Bug]: Hard-Deleted listings do not reappear on next scan

[Joly0]

Bug Description

I tried to setup a simple job for my home town and surrounding area. I run the job and it found a few listings. I saw that some of the listings are "trade apartments", so i wanted to exclude them from the search and added that to the blacklist in the job (added "Tauschwohnung").
To remove the "old" listings, i cliked on the small button on the job to delete all listings and selected "hard-remove". All listings were gone. I tried running the job again, but nothing was found. So to rule out a wrong blacklist, i removed the blacklist entry i added earlier and tried to run the job again, but it didnt found anything either. I tried doing the hard-remove again and run the job another time, but still nothing was found.

Steps to Reproduce

1. Create job
2. Run job
3. Maybe optional steps adding additional things like blacklisted words or so (might be optional ,cannot test now)
4. Hard-remove all listings from job
5. Run job again
6. No listings found anymore

Expected Behavior

Should just show the listings again that it already found.

Actual Behavior

Doesnt show any listings anymore.

Screenshots / Logs

2026-06-21-FredyDebug-22.9.2.zip

Environment

Unraid Community Apps

Additional Context

No response

[Joly0]

I see the problem. Although it says, that it deletes the listings from the database, it actually doesnt. They are still there:

[Joly0]

For some reason "hard-deleted" listings are "soft-deleted" ones

[Joly0]

Hm, its not getting better.
So when i hard-delete them, they actually disappear from the overview, even when hidden:

but as soon as i scan again, they reappear, but as hidden or soft-deleted, which is why by default on the "all" filter they do not appear. So thats the actual bug. Listings that are hard-deleted stay in the database and are on next scan marked as soft-deleted. Instead of being completly removed or without any tag

[Joly0]

Ok, when i run the hard-delete all listings from the job are removed from the ui, but stay in the database. They are only properly deleted when i restart the container. So looks kinda like a caching problem with the database

[Joly0]

Ooookey... soo. I dug into this and found the root cause. Hard-delete itself works, the row really does get DELETEd from the listings table. The problem is the in-memory similarity cache is never told about the deletion.
What happens on the next scan after a hard-delete:

1. _findNew correctly treats the listing as new (its hash is gone from the DB), so it gets re-inserted by _save.
2. But _filterBySimilarListings still finds the listing's title|price|address hash sitting in the in-memory similarity cache (which only reloads from the DB once an hour, or on restart). It treats the listing as a duplicate and immediately soft-deletes the row it just re-inserted (manually_deleted = 1).

That's why the listings "disappear": theyre actually re-found and re-inserted on every run, but instantly soft-deleted again by the stale cache. You can confirm this, after a hard-delete and re-run, the listings show up under the "hidden" filter (which only shows manually_deleted = 1 rows), even though the "hidden" view is meant for soft-deleted entries only.

It only works again after a container restart, because on startup the cache is rebuilt from the (now clean) DB, so the deleted hashes are gone.

The fix is to evict the entry from the similarity cache when a listing is hard-deleted (the cache currently only has add/init/reload, no removal), so a restart isn't needed.

[orangecoding]

Thanks for the deep dive. Fixed here, please test.
https://github.com/orangecoding/fredy/releases/tag/22.9.3