Skip to content

Commit 7686d10

Browse files
brown9804brown9804
authored
ovevriew
1 parent 8a61b4b commit 7686d10

1 file changed

Lines changed: 121 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ Last updated: 2025-08-20
1313
<details>
1414
<summary><b>List of References</b> (Click to expand)</summary>
1515

16+
- [voyage-3-large: the new state-of-the-art general-purpose embedding model](https://statics.teams.cdn.office.net/evergreen-assets/safelinks/2/atp-safelinks.html)
17+
1618
</details>
1719

1820

@@ -22,6 +24,125 @@ Last updated: 2025-08-20
2224
</details>
2325

2426

27+
## Recommendations for Alternative & Upcoming Embedding Models in Azure
28+
29+
> Current Options:
30+
31+
| **Model** | **Platform** | **Strengths** | **Limitations** | **Best Use Cases** |
32+
|-----------|--------------|---------------|------------------|---------------------|
33+
| `voyage-3` | Azure Marketplace | Balanced performance, tuned for enterprise domains, cost-efficient | No native integration with Azure AI Foundry, limited multilingual support | Internal search, domain-specific retrieval, enterprise RAG |
34+
| `voyage-3-lite` | Azure Marketplace | Lightweight, fast, low latency and compute cost | Slightly reduced semantic precision, less robust for long-form text | Mobile apps, real-time classification, edge deployments |
35+
| `voyage-finance-2` | Azure Marketplace | Specialized for financial language and jargon, tuned for structured documents | Poor generalization outside finance, narrow scope | Financial document analysis, compliance, financial RAG |
36+
| `voyage-law-2` | Azure Marketplace | Legal-specific, optimized for contracts and statutes | Limited cross-domain utility, narrow vocabulary | Legal search, contract parsing, regulatory compliance |
37+
| `voyage-multilingual-2` | Azure Marketplace | Multilingual support, tuned for global corpora | Slightly lower precision in English, not ideal for domain-specific tasks | Multilingual semantic search, global content indexing |
38+
| `text-embedding-ada-002` | Azure OpenAI / AI Foundry | Legacy model, fast and inexpensive, widely supported | Lower semantic quality, outdated compared to newer models | Lightweight search, prototyping, low-cost RAG |
39+
| `text-embedding-3-small` | Azure OpenAI / AI Foundry | Fast, cost-effective, configurable, good for short texts | Lower accuracy on nuanced or complex queries | Chat summarization, real-time applications, indexing |
40+
| `text-embedding-3-large` | Azure OpenAI / AI Foundry | High semantic fidelity, multilingual, robust across domains | Higher latency and cost, batch quirks | Semantic search, multilingual corpora, recommendation systems |
41+
| `embed-v-4-0` | Azure AI Foundry | High-performance, optimized for retrieval and RAG, scalable | Newer model with limited public benchmarks | Advanced RAG pipelines, enterprise search, hybrid retrieval |
42+
| `Cohere-embed-v3-multilingual` | Azure AI Foundry | Strong multilingual alignment, semantic robustness | Slightly slower, less tuned for English-only tasks | Multilingual indexing, global search, translation-aware retrieval |
43+
| `Cohere-embed-v3-english` | Azure AI Foundry | High precision for English, optimized for semantic tasks | Not suitable for multilingual content | English-centric semantic search, document clustering |
44+
45+
> [!NOTE]
46+
> `Upcoming Models to Watch:`
47+
- **Voyage-3-Large**: Expected to outperform OpenAI v3-large with flexible dimensions and quantization. [voyage-3-large: the new state-of-the-art general-purpose embedding model](https://statics.teams.cdn.office.net/evergreen-assets/safelinks/2/atp-safelinks.html)
48+
49+
> Azure OpenAI
50+
> - `text-embedding-ada-002`
51+
> - `text-embedding-3-large`
52+
> - `text-embedding-3-small`
53+
54+
1. Go to your Azure OpenAI Platform:
55+
56+
<img width="1916" height="840" alt="image" src="https://github.com/user-attachments/assets/424f2d85-37e4-4e74-b303-2e8c5a23a864" />
57+
58+
2. Under `Model catalog`, filter by `Inference task` > `Embeddings`:
59+
60+
<img width="1905" height="848" alt="image" src="https://github.com/user-attachments/assets/8747ad65-bf2b-4fdd-bdb8-6ad505b4ebdf" />
61+
62+
<img width="1893" height="695" alt="image" src="https://github.com/user-attachments/assets/8d822c1d-04fd-49ba-a514-1a3f74e24cdc" />
63+
64+
> Azure AI Foundry
65+
> - `embed-v-4-0`
66+
> - `Cohere-embed-v3-multilingual`
67+
> - `Cohere-embed-v3-english`
68+
> - `text-embedding-ada-002`
69+
> - `text-embedding-3-large`
70+
> - `text-embedding-3-small`
71+
72+
1. Go to your Azure AI Foundry Platform:
73+
74+
<img width="1897" height="791" alt="image" src="https://github.com/user-attachments/assets/f56fc29b-2276-4998-9f0c-226e4329ca78" />
75+
76+
2. Under `Model catalog`, filter by `Inference task` > `Embeddings`:
77+
78+
<img width="1907" height="847" alt="image" src="https://github.com/user-attachments/assets/45b1676c-8c5b-4084-be19-4eeb56e8f961" />
79+
80+
> [Azure Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=embedding&page=1):
81+
> - `voyage-3 Embedding Model`
82+
> - `voyage-3-lite Embedding Model`
83+
> - `voyage-finance-2 Embedding Model`
84+
> - `voyage-law-2 Embedding Model`
85+
> - `voyage-multilingual-2 Embedding Model`
86+
87+
<img width="1900" height="837" alt="image" src="https://github.com/user-attachments/assets/0eac2f90-bf75-45cf-aa06-5e4420dc8be5" />
88+
89+
## How Azure AI Search Can Help Optimize Embedding Performance
90+
91+
> Azure AI Search doesn’t generate embeddings from third-party models, but it **amplifies their value** through advanced indexing, retrieval, and hybrid search capabilities.
92+
93+
> [!TIP]
94+
> Use Azure AI Foundry to experiment with multiple embedding models and benchmark their performance before committing to production.
95+
96+
<details>
97+
<summary><b>Vector Search Capabilities</b> (Click to expand)</summary>
98+
99+
> Azure AI Search supports robust vector search features that allow you to fully leverage embeddings generated externally (e.g., from OpenAI, Voyage, Cohere):
100+
101+
- Accepts **custom embeddings** from any model, making it agnostic and flexible across providers.
102+
- Supports **semantic similarity search** using metrics like cosine similarity and dot product, enabling nuanced matching beyond keyword overlap.
103+
- Handles **large-scale indexing** and **low-latency retrieval**, ideal for enterprise-grade applications with millions of documents.
104+
- Integrates with **hybrid search pipelines**, combining vector and keyword search to improve both precision and recall.
105+
106+
**Example Use Cases**:
107+
- Intelligent document retrieval for legal or financial archives.
108+
- FAQ matching and chatbot grounding using semantic similarity.
109+
- Product recommendation systems based on user intent embeddings.
110+
111+
</details>
112+
113+
<details>
114+
<summary><b>Performance Optimization Tips</b> (Click to expand)</summary>
115+
116+
> To maximize efficiency and relevance in embedding-based search, consider these strategies:
117+
118+
1. **Choose embedding size wisely**: Smaller embeddings (e.g., 512–1024) reduce latency and storage costs, ideal for mobile or real-time apps.
119+
2. **Batch embedding generation**: Pre-process documents in bulk to reduce API calls and improve throughput.
120+
3. **Use domain-specific models**: Models like `voyage-finance-2` or `voyage-law-2` yield better semantic relevance in specialized contexts.
121+
4. **Monitor vector DB costs**: Larger embeddings increase storage and query costs—balance precision with efficiency.
122+
5. **Leverage hybrid search**: Combine keyword and vector search to handle both exact and fuzzy matches, especially in noisy datasets.
123+
6. **Normalize and deduplicate embeddings**: Ensure consistent vector quality and avoid redundant indexing.
124+
125+
</details>
126+
127+
<details>
128+
<summary><b>Integration Strategy</b> (Click to expand)</summary>
129+
130+
> Azure AI Search is designed to integrate seamlessly with external embedding pipelines. Here's how to build a modular and scalable setup:
131+
132+
- **Generate embeddings externally** using models from Azure OpenAI (`text-embedding-3-large`, `text-embedding-3-small`), Voyage AI (`voyage-3`, `voyage-multilingual-2`), or Cohere (`embed-v3` series).
133+
- **Store embeddings** in Azure AI Search vector fields, which are optimized for fast similarity search and scalable indexing.
134+
- **Query using embedded vectors** to perform semantic matching, enabling intelligent document retrieval, contextual search, and RAG workflows.
135+
- **Automate updates**: Use Azure Functions or Logic Apps to refresh embeddings when documents change, keeping your index up-to-date.
136+
137+
**Example Workflow**:
138+
1. Use `text-embedding-3-large` to embed support tickets.
139+
2. Store vectors in Azure AI Search.
140+
3. Query with user questions to retrieve semantically similar tickets.
141+
4. Combine with keyword filters for precision.
142+
143+
</details>
144+
145+
25146
<!-- START BADGE -->
26147
<div align="center">
27148
<img src="https://img.shields.io/badge/Total%20views-1420-limegreen" alt="Total views">

0 commit comments

Comments
 (0)