Skip to content

Commit 544addf

Browse files
authored
IBX-11401: Describe Gemini embeddings provider (#3120)
* IBX-11401: Describe Gemini embeddings provider
1 parent b3bfe14 commit 544addf

3 files changed

Lines changed: 87 additions & 36 deletions

File tree

docs/ai_actions/ai_actions_guide.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -163,9 +163,14 @@ With some customization, administrators could use the API to run a batch process
163163

164164
### Suggesting taxonomy entries
165165

166-
Content editors and product managers can use [taxonomy suggestions](taxonomy.md#taxonomy-suggestions) when assigning tags or product categorie to content items and products.
166+
Content editors and product managers can use [taxonomy suggestions](taxonomy.md#taxonomy-suggestions) when assigning tags or product categories to content items and products.
167167
Instead of manually browsing through extensive taxonomy trees, editors can request suggestions based on the content's text fields, such as name and description.
168168

169+
!!! note "Alternative suggestion provider"
170+
171+
By default, embeddings used by the taxonomy suggestions feature are generated with OpenAI.
172+
If you install and configure the [Google Gemini connector](configure_ai_actions.md#install-google-gemini-connector), you can modify the [taxonomy suggestions settings](taxonomy.md#change-embeddings-provider-to-google-gemini) and use Google Gemini as an alternative embeddings provider.
173+
169174
### Performing advanced image to text analysis
170175

171176
With some additional customization, store managers could benefit from automating part of product management by integrating their [[= product_name =]] with Google Cloud Vision and [PIM](pim_guide.md) by using [[= product_name_connect =]].

docs/ai_actions/configure_ai_actions.md

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,13 +8,17 @@ month_change: false
88
AI Actions are available in [[= product_name =]] regardless of its edition.
99
To use this feature you must first configure the built-in service connectors or build your own ones.
1010

11-
!!! note "Next steps"
11+
Once the framework is configured, before you can start using AI Actions, you can configure access to [[= product_name_base =]]-made service connectors by following the instructions below, or [create your own](extend_ai_actions.md#create-custom-action-handler).
1212

13-
Once the framework is configured, before you can start using AI Actions, you can configure access to [[= product_name_base =]]-made service connectors by following the instructions below, or [create your own](extend_ai_actions.md#create-custom-action-handler).
13+
Only then you can restart you application and start [working with the AI Actions feature]([[= user_doc =]]/ai_actions/work_with_ai_actions/).
1414

15-
Only then you can restart you application and start [working with the AI Actions feature]([[= user_doc =]]/ai_actions/work_with_ai_actions/).
15+
!!! note "Taxonomy suggestions"
16+
17+
The default OpenAI or the optional Google Gemini connectors can used by the [Taxonomy suggestions](taxonomy.md#taxonomy-suggestions) feature to generate embeddings for suggesting tags and product categories.
18+
After you configure the OpenAI connector, or set up the optional Google Gemini connector and [modify the default taxonomy suggestions settings](taxonomy.md#change-embeddings-provider-to-google-gemini), you can [create AI actions that use the Text to Taxonomy action type]([[= user_doc =]]/ai_actions/work_with_ai_actions/#create-ai-actions-that-control-taxonomy-suggestions).
19+
You can also create [your own embedding provider](taxonomy.md#replace-the-embedding-provider).
1620

17-
## Configure access to OpenAI (optional)
21+
## Configure access to OpenAI
1822

1923
To use the built-in connector with the OpenAI service, you need to create an OpenAI account, [get an API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key), and make sure that you [set up a billing method](https://help.openai.com/en/articles/9038407-how-can-i-set-up-billing-for-my-account).
2024

@@ -32,12 +36,6 @@ The AI actions come with sample AI action configurations to quickly get you star
3236

3337
Based on these examples, which reflect the most common use cases, you can learn to configure your own AI actions with greater ease.
3438

35-
!!! note "Taxonomy suggestions"
36-
37-
OpenAI connector is also used by the [Taxonomy suggestions](taxonomy.md#taxonomy-suggestions) feature to generate embeddings for suggesting tags and product categories.
38-
After you configure the connector, you can [create AI actions that use the Text to Taxonomy action type]([[= user_doc =]]/ai_actions/work_with_ai_actions/#create-ai-actions-that-control-taxonomy-suggestions).
39-
You can also create [your own embedding provider](taxonomy.md#replace-the-embedding-provider).
40-
4139
## Install Anthropic connector [[% include 'snippets/lts-update_badge.md' %]]
4240

4341
Run the following command to install the package:
@@ -187,6 +185,10 @@ When setting up models, make sure that you follow these rules:
187185
- `default_max_tokens` must not exceed the model’s limit
188186
- If you use the same model for different action types, settings must be consistent
189187

188+
!!! note "Google Gemini and taxonomy suggestions"
189+
190+
To use Google Gemini for generating taxonomy suggestions, ensure that you [change the embeddings provider and model setting accordingly](taxonomy.md#change-embeddings-provider-to-google-gemini).
191+
190192
You can now use the Gemini connector in your project.
191193

192194
For more information, see [Extend Gemini connector](extend_ai_actions.md#extend-google-gemini-connector).

docs/content_management/taxonomy/taxonomy.md

Lines changed: 69 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,7 @@ ibexa:
100100

101101
## Remove orphaned content items
102102

103-
In some rare case, especially in [[= product_name =]] v4.2 and older, when deleting parent of huge subtrees, some Taxonomy entries aren't properly deleted, leaving content items that point to a non-existing parent.
103+
In some rare case, especially in [[= product_name =]] v4.2 and older, when deleting parent of huge subtrees, some taxonomy entries aren't properly deleted, leaving content items that point to a non-existing parent.
104104
The command `ibexa:taxonomy:remove-orphaned-content` deletes those orphaned content item.
105105
It works on a taxonomy passed as an argument, and has two options that act as a protective measure against deleting data by mistake:
106106

@@ -132,7 +132,7 @@ When it happens, the `Ibexa\Taxonomy\ActionHandler\TextToTaxonomyActionHandler`
132132

133133
!!! note "Field selection"
134134

135-
You select the actual text fields, whose values are used as source for the embedding generation, when you create an [AI action](https://doc.ibexa.co/projects/userguide/en/latest/ai_actions/work_with_ai_actions/#create-ai-actions-that-use-ibexa-connect) that uses the `openai-text-to-taxonomy-entries` handler.
135+
You select the actual text fields, whose values are used as source for the embedding generation, when you create an [AI action](https://doc.ibexa.co/projects/userguide/en/latest/ai_actions/work_with_ai_actions/#create-ai-actions-that-use-ibexa-connect) that uses the `text-to-taxonomy` handler.
136136

137137
The search engine then compares the generated embedding with the taxonomy path embeddings stored in its index.
138138
By default, it selects the three best-matching taxonomy paths and presents them to the editor as suggestions.
@@ -145,23 +145,27 @@ However, before you can enable it, make sure the following prerequisites have be
145145

146146
- [Search engine](search_engines.md): Taxonomy suggestions require a search engine that supports vector search.
147147
The feature has been tested to work with Elasticsearch or Solr 9.8.1+.
148-
- [AI Actions](ai_actions.md): To be able to process embeddings, Taxonomy suggestions require that you have the [AI Actions configured](configure_ai_actions.md#configure-access-to-openai-optional) to support the OpenAI service.
148+
- [AI Actions](ai_actions.md): To be able to process embeddings, Taxonomy suggestions require that you have the AI Actions configured to support the default [OpenAI](configure_ai_actions.md#configure-access-to-openai) or the optional [Google Gemini](configure_ai_actions.md#install-google-gemini-connector) service.
149+
150+
!!! note "Alternative embeddings provider"
151+
152+
To use Google Gemini as an alternative embeddings provider, you must also modify the default [taxonomy suggestions settings](taxonomy.md#change-embeddings-provider-to-google-gemini).
149153

150154
#### Enable taxonomy embedding indexing
151155

152-
Enable embedding indexing for taxonomy branches by changing the default setting from `false` to `true`:
156+
Enable embedding indexing for taxonomy branches by changing the default setting from `false` to `true`.
157+
Toggle this setting at any time to enable or disable indexing of taxonomy embeddings.
153158

154-
```yaml
159+
```yaml hl_lines="6"
155160
ibexa:
156-
system:
157-
default:
158-
taxonomy:
159-
search:
160-
index_embeddings: true
161+
system:
162+
default:
163+
taxonomy:
164+
search:
165+
index_embeddings: true
166+
default_embedding_model: 'text-embedding-ada-002'
161167
```
162168

163-
Toggle this setting at any time to enable or disable indexing of taxonomy embeddings.
164-
165169
If you are happy with the default settings, clear the cache and reindex the search engine.
166170

167171
``` shell
@@ -220,34 +224,74 @@ Like in the case of the number of suggestions, you can override this setting per
220224
When selecting the input data for embedding creation, it's recommended to include only the essential information and limit the number of tokens sent.
221225
Otherwise, the embedding models can generate values that don't correspond closely to the actual meaning of the input.
222226

223-
### Change the embedding generation model
227+
### Change embedding generation models or embedding provider
224228

225-
By default, the system comes with a set of OpenAI models listed in its configuration, and a setting that allows you to choose the default model that should be used with the Taxonomy suggestions feature.
229+
By default, the system comes with a set of OpenAI models that can be used for embedding generation.
230+
The following example shows these models listed in system configuration, together with a setting that controls what model is used when the editor requests taxonomy suggestions for an item.
231+
232+
Also, here is where you can change the name of the model used by the provider, the embedding's dimensions, and other settings.
226233

227234
```yaml hl_lines="20"
228235
ibexa:
229236
system:
230237
default:
231238
embedding_models:
232239
text-embedding-3-small:
233-
name: text-embedding-3-small
240+
name: 'text-embedding-3-small'
234241
dimensions: 1536
235-
field_suffix: 3small
236-
embedding_provider: ibexa_openai
242+
field_suffix: '3small'
243+
embedding_provider: 'ibexa_openai'
237244
text-embedding-3-large:
238-
name: text-embedding-3-large
245+
name: 'text-embedding-3-large'
239246
dimensions: 3072
240-
field_suffix: 3large
241-
embedding_provider: ibexa_openai
247+
field_suffix: '3large'
248+
embedding_provider: 'ibexa_openai'
242249
text-embedding-ada-002:
243-
name: text-embedding-ada-002
250+
name: 'text-embedding-ada-002'
244251
dimensions: 1536
245-
field_suffix: ada002
246-
embedding_provider: ibexa_openai
247-
default_embedding_model: text-embedding-ada-002
252+
field_suffix: 'ada002'
253+
embedding_provider: 'ibexa_openai'
254+
default_embedding_model: 'text-embedding-ada-002'
248255
```
249256

250-
Also, here is where you can change the name of the model used by the provider, the embedding's dimensions, and other settings.
257+
!!! warning "Change both embedding generation models"
258+
259+
When you change the default suggestions generation model, ensure that you update the `ibexa.system.default.taxonomy.search.default_embedding_model` setting that is used for taxonomy indexing purposes.
260+
Otherwise the taxonomy suggestions feature fails to find matching entries.
261+
262+
#### Change embeddings provider to Google Gemini [[% include 'snippets/lts-update_badge.md' %]]
263+
264+
Once you have installed and configured the [Google Gemini connector](configure_ai_actions.md#install-google-gemini-connector), you can modify the default configuration to use the `ibexa_gemini` embedding provider and one of the [supported models](https://ai.google.dev/gemini-api/docs/embeddings):
265+
266+
```yaml hl_lines="15 22"
267+
ibexa:
268+
system:
269+
default:
270+
embedding_models:
271+
gemini_embedding_001_1536:
272+
name: 'gemini-embedding-001'
273+
dimensions: 1536
274+
field_suffix: 'gemini_embedding_001_1536_dv'
275+
embedding_provider: 'ibexa_gemini'
276+
gemini_embedding_001_3072:
277+
name: 'gemini-embedding-001'
278+
dimensions: 3072
279+
field_suffix: 'gemini_embedding_001_3072_dv'
280+
embedding_provider: 'ibexa_gemini'
281+
default_embedding_model: 'gemini_embedding_001_1536'
282+
283+
# ...
284+
285+
taxonomy:
286+
search:
287+
index_embeddings: true
288+
default_embedding_model: 'gemini_embedding_001_1536'
289+
```
290+
291+
After you make the change:
292+
293+
- Update the [Solr schema](field_type_search.md#configuring-solr) or [Elasticsearch mappings](configure_elasticsearch.md#fine-tune-the-search-results) by adding dynamic field definitions. Ensure that they match the dimensions (for example, 1536 or 3072) and suffixes that you defined above
294+
- Clear the cache and reindex the search engine
251295

252296
### Extending Taxonomy suggestions
253297

0 commit comments

Comments
 (0)