Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .github/ISSUE_TEMPLATE/documentation-improvement.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
name: Documentation improvement
about: Suggest a clearer explanation, example or learning resource
title: "[Docs] "
labels: documentation
assignees: ""
---

## Area to improve

Link to the notebook, document or section that could be clearer.

## Current difficulty

Describe what is confusing, incomplete or difficult for a learner to reproduce.

## Suggested improvement

Explain the change that would make the material easier to understand.

## Learner context

Mention whether you are following the Udemy course, studying independently or using the repository for a project.

Please avoid including copyrighted course material, private data or confidential examples.
40 changes: 40 additions & 0 deletions .github/ISSUE_TEMPLATE/notebook-problem.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
name: Notebook problem
about: Report an error or unexpected result in a course notebook
title: "[Notebook] "
labels: bug
assignees: ""
---

## Notebook

Provide the exact notebook file name and, where possible, the section or cell number.

## What happened?

Describe the error or unexpected result clearly.

## Expected behaviour

Explain what you expected to happen.

## Environment

- Operating system:
- Python version:
- Jupyter Notebook or JupyterLab version:
- Relevant package versions:

## Error output

Paste the complete traceback inside a code block.

```text
Paste traceback here
```

## Steps already tried

Mention whether you restarted the kernel, ran the notebook from the first cell, recreated the environment or checked the troubleshooting guide.

Do not include private, confidential or regulated data in the issue.
27 changes: 27 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
## Summary

Describe the learning, correctness or reproducibility improvement made by this pull request.

## Type of change

- [ ] Notebook correction
- [ ] Documentation improvement
- [ ] Exercise or example
- [ ] Reusable code change
- [ ] Test or CI improvement

## Validation

- [ ] I ran the affected notebook or example from a clean start.
- [ ] I added or updated tests where reusable code changed.
- [ ] I checked that existing Udemy lecture references still work.
- [ ] I did not include private, confidential or regulated data.
- [ ] I cited external datasets, papers or code where applicable.

## Learner impact

Explain how this change helps a student understand, reproduce or apply the material.

## Additional notes

Include relevant screenshots, output summaries or limitations. Avoid committing large generated files unless they are essential.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ The goal is to help learners move from understanding an algorithm in a notebook
| Learn the subject independently | Follow the [student guide](docs/student-guide.md) and [learning roadmap](docs/course-roadmap.md) |
| Try a working example quickly | Run [`examples/quickstart.py`](examples/quickstart.py) |
| Compare detectors | Run the [benchmark](#reproducible-benchmark) |
| Practise and build a portfolio project | Complete the [exercises and capstone](exercises/README.md) |
| Practise and build a portfolio project | Complete the [exercises and capstone](exercises/README.md), then use the [project checklist](docs/project-checklist.md) |
| Reuse the implementation | Explore [`src/outlier_detection/`](src/outlier_detection/) |
| Fix an environment or notebook error | Read the [troubleshooting guide](docs/troubleshooting.md) |

Expand Down Expand Up @@ -230,7 +230,7 @@ High-stakes use cases require human review, subgroup analysis, privacy safeguard

## Student practice

The [`exercises`](exercises/README.md) progress from basic statistical detection to multivariate modelling and a complete capstone. For every exercise, learners should document:
The [`exercises`](exercises/README.md) progress from basic statistical detection to multivariate modelling and a complete capstone. Use the [`project checklist`](docs/project-checklist.md) to review the final work before sharing it. For every exercise, learners should document:

- the business or analytical objective;
- preprocessing choices;
Expand Down Expand Up @@ -319,4 +319,4 @@ Repository source code is released under the [MIT License](LICENSE). Course vide

**Learn the assumptions. Run the experiment. Inspect the mistakes. Document the decision.**

</div>
</div>
27 changes: 27 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Documentation

Use this page as the starting point for the learning and project documentation in this repository.

## Course learning path

- [Course syllabus](course-syllabus.md) — maps the original Udemy lectures to the repository notebooks.
- [Student guide](student-guide.md) — explains setup, notebook usage and a practical study workflow.
- [Course roadmap](course-roadmap.md) — provides an extended path beyond the original course.
- [Troubleshooting](troubleshooting.md) — covers common installation, dependency and notebook problems.

## Project work

- [Project checklist](project-checklist.md) — reviews problem definition, data preparation, modelling, evaluation, interpretation and reproducibility before a project is shared.
- [Notebook quality guide](notebook-quality-guide.md) — provides standards for clear, reproducible and course-compatible notebooks.
- [Exercises and capstone](../exercises/README.md) — provides progressive practice tasks and a portfolio-oriented capstone.

## Recommended order

1. Set up the environment using the student guide.
2. Follow the course syllabus alongside the lecture notebooks.
3. Complete the exercises after each algorithm family.
4. Run the benchmark to compare methods consistently.
5. Use the notebook quality guide when updating an example.
6. Use the project checklist before publishing a capstone or portfolio project.

The original lecture notebooks remain unchanged where possible so that existing Udemy students can continue to follow the recorded course.
60 changes: 60 additions & 0 deletions docs/notebook-quality-guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Notebook quality guide

Use these checks when updating a lecture notebook or adding a new educational example.

## Before running

- Use a clean Python environment supported by the repository.
- Restart the kernel and run all cells from top to bottom.
- Fix random seeds where the library supports them.
- Keep datasets public, synthetic or properly attributed.

## Explanations

A useful notebook should explain:

1. the learning objective;
2. the dataset and feature meaning;
3. preprocessing choices;
4. the detector assumptions;
5. score direction and label convention;
6. threshold or contamination selection;
7. evaluation metrics;
8. observed failure cases;
9. the practical conclusion.

Avoid presenting one detector as universally best. Results depend on the anomaly definition, feature space, scaling, class balance and review cost.

## Code

- Keep cells focused and executable in sequence.
- Avoid hidden state from cells run out of order.
- Use clear variable names and brief comments for non-obvious steps.
- Separate data preparation, model fitting, scoring and evaluation.
- Do not suppress warnings without explaining why.
- Avoid absolute local file paths.

## Results

- Show more than accuracy for rare-event problems.
- Prefer precision, recall, F1, average precision and operational measures such as precision at k.
- Explain whether higher scores mean more or less anomalous.
- Report runtime when comparing computationally different methods.
- Discuss false positives and false negatives rather than showing only an aggregate score.

## Visuals

Every chart should have a descriptive title, labelled axes and a short interpretation. Use visualisations to support a conclusion, not only for decoration.

## Final review

Before committing a notebook:

- restart and run all cells;
- remove accidental debug output;
- confirm that links and paths work;
- check that no private data or credentials are present;
- record any important version-dependent behaviour;
- keep compatibility with the original Udemy lecture flow where relevant.

For a complete student project, also use the [project checklist](project-checklist.md).
91 changes: 91 additions & 0 deletions docs/project-checklist.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Anomaly Detection Project Checklist

Use this checklist when completing a course exercise, adapting a notebook, or presenting an anomaly-detection project. It is designed to make the analysis reproducible and easier to review.

## 1. Problem definition

- [ ] State what an anomaly means in the problem domain.
- [ ] Explain whether the task is outlier detection, novelty detection, or supervised rare-event classification.
- [ ] Describe the practical cost of a false positive and a false negative.
- [ ] Identify who will review or act on an alert.

## 2. Data understanding

- [ ] Record the data source, collection period, unit of observation, and target population.
- [ ] Check missing values, duplicates, impossible values, and inconsistent units.
- [ ] Examine skew, heavy tails, seasonality, and subgroup differences.
- [ ] Confirm that no confidential, personal, or regulated data is committed to the repository.

## 3. Experimental design

- [ ] Separate training, validation, and test data before fitting preprocessing steps.
- [ ] Use a time-based split when future observations must not influence the past.
- [ ] Fix random seeds where the implementation supports them.
- [ ] Record Python and package versions.
- [ ] Establish a simple baseline before trying complex methods.

## 4. Preprocessing

- [ ] Explain how missing values are handled.
- [ ] Fit scaling, encoding, and feature selection only on training data.
- [ ] Justify any logarithmic or power transformation.
- [ ] Review whether distance-based methods are being distorted by feature scale.
- [ ] Keep an auditable mapping from transformed features to their original meaning.

## 5. Detector selection

- [ ] Match each selected detector to the structure of the data.
- [ ] Document the main assumptions and important hyperparameters.
- [ ] Compare at least two credible approaches when practical.
- [ ] Confirm whether larger or smaller scores represent more anomalous observations.
- [ ] Avoid treating a library default contamination value as a known anomaly rate.

## 6. Threshold selection

- [ ] Select the threshold using validation evidence, domain prevalence, or an alert-review budget.
- [ ] Report how results change under nearby threshold values.
- [ ] State the final expected alert volume.
- [ ] Keep score generation separate from the business decision threshold.

## 7. Evaluation

- [ ] Do not rely on accuracy when anomalies are rare.
- [ ] Report precision, recall, F1, and average precision where labels are available.
- [ ] Include precision at k when investigation capacity is limited.
- [ ] Measure runtime and memory when scalability matters.
- [ ] Inspect representative false positives and false negatives.
- [ ] Compare performance across meaningful subgroups when appropriate.

## 8. Interpretation

- [ ] Explain why selected observations received high anomaly scores.
- [ ] Distinguish data errors from valid but unusual observations.
- [ ] Avoid automatically deleting detected outliers.
- [ ] Document limitations and cases where the method is likely to fail.

## 9. Reproducibility

- [ ] Run the notebook from the first cell in a clean kernel.
- [ ] Remove hidden state and machine-specific file paths.
- [ ] Keep dependencies in `pyproject.toml` or an environment file.
- [ ] Save only outputs that help the learner understand the result.
- [ ] Verify that another person can follow the setup instructions.

## 10. Production considerations

- [ ] Define monitoring for data drift, score drift, and alert volume.
- [ ] Record retraining or recalibration triggers.
- [ ] Add human review for high-stakes decisions.
- [ ] Define fallback behaviour when data or model checks fail.
- [ ] Establish privacy, security, access-control, and incident-response requirements.

## Suggested project conclusion

A strong conclusion should answer four questions:

1. Which observations were flagged and how many?
2. Why was the selected detector and threshold appropriate?
3. What did the error analysis reveal?
4. What should a decision-maker do next?

Keep the conclusion understandable to a reader who has not seen the notebook code.
5 changes: 4 additions & 1 deletion examples/quickstart.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,10 @@ def build_dataset():
random_state=7,
)
x = __import__("numpy").vstack([normal, anomalies])
y = __import__("numpy").r_[__import__("numpy").zeros(len(normal)), __import__("numpy").ones(len(anomalies))]
y = __import__("numpy").r_[
__import__("numpy").zeros(len(normal)),
__import__("numpy").ones(len(anomalies)),
]
return train_test_split(x, y, test_size=0.35, stratify=y, random_state=42)


Expand Down
Loading