|
| 1 | +<div align="center"> |
1 | 2 |
|
2 | | -#### **A comprehensive implementation of various Matrix Decomposition Techniques from the lens of Linear Algebra to produce efficient computing of SVD, PCA, Feature Selection & Data Analysis in Python.** |
3 | | -______________________________________________________________________ |
| 3 | +# Matrix Decompositions Implemenations |
4 | 4 |
|
5 | | -To gain a deeper understanding of how Orthogonalization & Matrices Decomposition works in real-life applications, & how they save bunch of time through an approach of vectorization, you'll find such techniques used in; |
| 5 | +**A hands-on, math-first implementation of Matrix Decomposition techniques —** |
| 6 | +**from Gram-Schmidt Orthogonalization to SVD, PCA & beyond.** |
6 | 7 |
|
7 | | -- **📡 Signal Processing** |
8 | | -- **🤖 Control Systems and Robotics** |
9 | | -- **🖼️ Image Processing** |
10 | | -- **➗ Solving Linear Systems i.e. *AX = B*** |
| 8 | +<br/> |
11 | 9 |
|
12 | | -With certain mathematical intuitions (*having visual introspections*),this project simplifies most of the abstract concepts and becomes easier to grasp and connect with practical applications. |
| 10 | +[](https://molab.marimo.io/notebooks/nb_TAVLehyiE58b5RDzjxFxSW) |
| 11 | +[](https://huggingface.co/spaces/your-hf-spaces-link-here) |
| 12 | +[](https://www.python.org/) |
| 13 | +[](https://numpy.org/) |
| 14 | +[](LICENSE) |
| 15 | +[](https://pragyntiwari.github.io/Matrix-Decompositions-Implementation-for-SVD-PCA) |
13 | 16 |
|
14 | | - |
| 17 | +<img src=".assets/01.gif" alt="Matrix Decompositions Demo" width="620"/> |
15 | 18 |
|
16 | | -> You'll yet to see more implementations—such as **Householder Reflection**, **Bidiagonalization**, **LU Decomposition**, on this repo, and others—*these will be added soon*. |
| 19 | +</div> |
17 | 20 |
|
18 | | -## What's Inside |
| 21 | +--- |
19 | 22 |
|
20 | | -*By latest ✨,* |
| 23 | +## Overview |
21 | 24 |
|
22 | | -The **Gram-Schmidt Orthogonalization** is one of the fundamental process in Linear Algebra to achieve *Orthonormal Vectors* for a given vector space. The Orthonormal Basis are produced by iteratively removing vector projections — also known as the *Vector Projection Elimination method*. |
| 25 | +A curated set of [marimo](https://marimo.io) notebooks based on **Matrix Decomposition** functions, written in Python — each pairing a rigorous mathematical derivation with annotated Python and an interactive visualization, in a single reactive environment. |
23 | 26 |
|
24 | | -**Terms like Orthogonality, QR Decomposition are being discussed in the — [🗨️Discussion section](https://github.com/PragyanTiwari/Matrix-Decompositions-Implementation-for-SVD-PCA/discussions).** |
| 27 | +The series is a progressive build, starting from orthogonalization fundamentals and working toward full matrix factorizations and applications: |
25 | 28 |
|
26 | | -Here's a snippet; |
| 29 | +`Gram-Schmidt` → `QR` → `LU` → `Householder` → `SVD` → `PCA` |
27 | 30 |
|
28 | | -```bash |
29 | | -def gs_Orthogonalization(X:np.ndarray)->np.ndarray: |
| 31 | +> **Main Motive —** Matrix decompositions reduce computationally expensive operations — inversion, least squares, eigensolving — into sequences of simpler, numerically stable factors. This project builds that machinery from the ground up, reducing the computational overhead at each step. |
30 | 32 |
|
31 | | - Q = np.copy(X).astype("float64") |
32 | | - n_vecs = Q.shape[1] |
| 33 | +>> Applications such as **noise reduction, signal processing, image compression** and more will be covered as the series progresses. |
33 | 34 |
|
34 | | - # defining a function to compute the L2-norm |
35 | | - length = lambda x: np.linalg.norm(x) |
| 35 | +--- |
36 | 36 |
|
37 | | - # iteration with each vector in the matrix X |
38 | | - for nth_vec in range(n_vecs): |
| 37 | +## Marimo Apps |
39 | 38 |
|
40 | | - # iteratively removing each preceding projection from nth vector |
41 | | - for k_proj in range(nth_vec): |
| 39 | +| Notebook | Open in molab | Open in HF Spaces | |
| 40 | +|---|:---:|:---:| |
| 41 | +| **Gram-Schmidt Orthogonalization** | [](https://molab.marimo.io/notebooks/nb_TAVLehyiE58b5RDzjxFxSW/app) | [](https://huggingface.co/spaces/PragyanTiwari/Gram-Schmidt-Orthonormal-Basis) | |
| 42 | +| **QR Decomposition** | 🔜 | 🔜 | |
| 43 | +| **Householder Reflection & Bidiagonalization** | 🔜 | 🔜 | |
42 | 44 |
|
43 | | - # the dot product would be the scaler coefficient |
44 | | - scaler = Q[:,nth_vec] @ Q[:,k_proj] |
45 | | - projection = scaler * Q[:,k_proj] |
46 | | - Q[:,nth_vec] -= projection # removing the Kth projection |
| 45 | +--- |
47 | 46 |
|
48 | | - norm = length(Q[:,nth_vec]) |
| 47 | +## Quickstart |
49 | 48 |
|
50 | | - # handling the case if the loop encounters linearly dependent vectors. |
51 | | - # Since, they come already under the span of vector space, hence their value will be 0. |
52 | | - if np.isclose(norm,0, rtol=1e-15, atol=1e-14, equal_nan=False): |
53 | | - Q[:,nth_vec] = 0 |
54 | | - else: |
55 | | - # making orthogonal vectors -> orthonormal |
56 | | - Q[:,nth_vec] = Q[:,nth_vec] / norm |
| 49 | +Requires Python `>= 3.12` and [`uv`](https://docs.astral.sh/uv/). |
57 | 50 |
|
58 | | - return Q |
| 51 | +**1. Clone and install dependencies** |
| 52 | + |
| 53 | +```bash |
| 54 | +git clone https://github.com/PragyanTiwari/Matrix-Decompositions-Implementation-for-SVD-PCA.git |
| 55 | +cd Matrix-Decompositions-Implementation-for-SVD-PCA |
| 56 | +uv sync |
59 | 57 | ``` |
60 | 58 |
|
61 | | -To run the notebook in a sandbox environment; |
| 59 | +**2. Run a marimo app** |
62 | 60 |
|
63 | 61 | ```bash |
| 62 | +uvx marimo run apps/gs_process.py |
| 63 | +``` |
| 64 | + |
| 65 | +**3. Optional — run any notebook in a sandboxed environment** |
| 66 | + |
| 67 | +```bash |
| 68 | +# No global installs — dependencies are inlined in the notebook |
64 | 69 | uvx marimo run --sandbox notebooks/Gram_Schmidt_QR_Decomposition.py |
| 70 | + |
| 71 | +# Or open for editing |
| 72 | +uvx marimo edit --sandbox notebooks/Gram_Schmidt_QR_Decomposition.py |
65 | 73 | ``` |
66 | 74 |
|
67 | | -## 🧪 Testing |
| 75 | +--- |
68 | 76 |
|
69 | | -The updates made on this project, can be tested for deployment, (and for personal experimentation) by the following; |
| 77 | +## Implementation Notes |
70 | 78 |
|
71 | | -- Fork the repository. |
| 79 | +<details> |
| 80 | +<summary><strong>Gram-Schmidt Orthogonalization</strong></summary> |
72 | 81 |
|
73 | | -- Run uv sync to install dependencies (*uv lockfile will help*) |
| 82 | +<br/> |
74 | 83 |
|
75 | | -```bash |
76 | | -uv sync |
77 | | -``` |
| 84 | +```python |
| 85 | +import numpy as np |
78 | 86 |
|
79 | | -- To test the export process, we'll run `.github/scripts/build.py` from the root directory through a symlink. |
| 87 | +def gram_schmidt(X: np.ndarray) -> np.ndarray: |
| 88 | + """ |
| 89 | + Transforms linearly independent column vectors into an orthonormal matrix Q. |
| 90 | + Q.T @ Q = I (verified below) |
| 91 | + """ |
| 92 | + Q = np.copy(X).astype("float64") |
| 93 | + for i in range(Q.shape[1]): |
| 94 | + for j in range(i): |
| 95 | + Q[:, i] -= (Q[:, i] @ Q[:, j]) * Q[:, j] # remove j-th projection |
| 96 | + norm = np.linalg.norm(Q[:, i]) |
| 97 | + Q[:, i] = 0 if np.isclose(norm, 0, atol=1e-14) else Q[:, i] / norm |
| 98 | + return Q |
| 99 | +``` |
80 | 100 |
|
81 | | -```bash |
82 | | -uv run .github/scripts/build.py |
| 101 | +```python |
| 102 | +# Verification: Q.T @ Q ≈ I |
| 103 | +A = np.array([[1, 0, 0], [2, 0, 3], [4, 5, 6]]).T |
| 104 | +assert np.allclose(gram_schmidt(A).T @ gram_schmidt(A), np.eye(3)) # ✓ |
83 | 105 | ``` |
84 | 106 |
|
85 | | -This will export all notebooks in a folder called `_site/` in the root directory |
| 107 | +> 💬 Questions on implementation or numerical stability? Start a thread in [Discussions](https://github.com/PragyanTiwari/Matrix-Decompositions-Implementation-for-SVD-PCA/discussions). |
| 108 | +
|
| 109 | +</details> |
| 110 | + |
| 111 | +*Implementation notes for each technique will be added as notebooks are released.* |
| 112 | + |
| 113 | +--- |
| 114 | + |
| 115 | +## Contributing |
| 116 | + |
| 117 | +Contributions are welcome — whether it's a bug report, a new decomposition technique, or a clearer explanation of the math. |
| 118 | + |
| 119 | +1. **Fork** the repository |
| 120 | +2. **Sync** dependencies: `uv sync` |
| 121 | +3. **Create a branch** for your changes |
| 122 | +4. **Open a Pull Request** — maintainers will review it |
| 123 | + |
| 124 | +For questions, suggestions, or discussion of the mathematics: |
| 125 | + |
| 126 | +- 💬 [Discussion Board](https://github.com/PragyanTiwari/Matrix-Decompositions-Implementation-for-SVD-PCA/discussions) |
| 127 | +- 🐛 [Open an Issue](https://github.com/PragyanTiwari/Matrix-Decompositions-Implementation-for-SVD-PCA/issues) |
| 128 | + |
| 129 | +--- |
| 130 | + |
| 131 | +## Resources & Acknowledgements |
| 132 | + |
| 133 | +- [**Wikipedia** — Gram-Schmidt Process](https://en.wikipedia.org/wiki/Gram%E2%80%93Schmidt_process) — foundational definitions and mathematical references |
| 134 | +- [**DataCamp** — Orthogonal Matrices](https://www.datacamp.com/tutorial/orthogonal-matrix) — accessible article on orthogonality |
| 135 | +- [**MIT OpenCourseWare** — Lecture 17](https://ocw.mit.edu/courses/18-06-linear-algebra-spring-2010/resources/lecture-17-orthogonal-matrices-and-gram-schmidt/) — in-depth treatment by *Prof. Gilbert Strang* |
| 136 | +- [**Steve Brunton**](https://www.youtube.com/@Eigensteve) — original spark for this project; exceptional intuition on engineering applications of linear algebra |
| 137 | +- [**Graphical Linear Algebra**](https://graphicallinearalgebra.net/2017/08/09/orthogonality-and-projections/) — visual treatment of orthogonality and projections |
| 138 | + |
| 139 | +--- |
86 | 140 |
|
87 | | -## 🌱 Contribution Guide |
| 141 | +<div align="center"> |
88 | 142 |
|
89 | | -- If you find a bug or have a feature request, please open an [Issue](https://github.com/PragyanTiwari/Matrix-Decompositions-Implementation-for-SVD-PCA/issues). |
| 143 | +**Try it without any setup →** [](https://molab.marimo.io/notebooks/nb_TAVLehyiE58b5RDzjxFxSW/app) |
90 | 144 |
|
91 | | -- PR will be reviewed by the maintainers. |
| 145 | +<br/> |
92 | 146 |
|
93 | | -- Questions & Suggestions can be queried on the [Discussion section](https://github.com/PragyanTiwari/Matrix-Decompositions-Implementation-for-SVD-PCA/discussions). |
| 147 | +[⭐ Star this repo](https://github.com/prgyn8/Matrix-Decomposition-Implementations/stargazers) · |
| 148 | +[💬 Join the Discussion](https://github.com/prgyn8/Matrix-Decomposition-Implementations/discussions) · |
| 149 | +[Made by Pragyan →](https://your-personal-link-here) |
94 | 150 |
|
95 | | -______________________________________________________________________ |
| 151 | +</div> |
0 commit comments