Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 29 additions & 4 deletions packages/components/nodes/vectorstores/Redis/utils.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,37 @@
import { isNil } from 'lodash'

/*
* Escapes all '-' characters.
* Redis Search considers '-' as a negative operator, hence we need
* to escape it
* Escapes all RediSearch special characters.
* Redis Search considers these characters as operators or special syntax,
* so they must be escaped to be treated as literals.
*/
export const escapeSpecialChars = (str: string) => {
return str.replaceAll('-', '\\-')
.replaceAll(',', '\\,')
.replaceAll('.', '\\.')
.replaceAll('<', '\\<')
.replaceAll('>', '\\>')
.replaceAll('{', '\\{')
.replaceAll('}', '\\}')
.replaceAll('[', '\\[')
.replaceAll(']', '\\]')
.replaceAll('"', '\\"')
.replaceAll("'", "\\'")
.replaceAll(':', '\\:')
.replaceAll(';', '\\;')
.replaceAll('!', '\\!')
.replaceAll('@', '\\@')
.replaceAll('#', '\\#')
.replaceAll('$', '\\$')
.replaceAll('%', '\\%')
.replaceAll('^', '\\^')
.replaceAll('&', '\\&')
.replaceAll('*', '\\*')
.replaceAll('(', '\\(')
.replaceAll(')', '\\)')
.replaceAll('+', '\\+')
.replaceAll('=', '\\=')
.replaceAll('~', '\\~')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I think we’re still missing one Redis query separator: |.

Since buildQuery uses | as the OR operator to join metadata filters, a literal value like foo | bar will be incorrectly parsed as two separate alternatives instead of a single metadata value. As noted in the Redis documentation, | is a special character that needs to be escaped in tag filters.

Could you, please, update both escapeSpecialChars and unEscapeSpecialChars to include |, and add a regression test for a metadata filter value containing a pipe? I think it'll be really useful!

}

export const escapeAllStrings = (obj: object) => {
Expand All @@ -27,5 +52,5 @@ export const escapeAllStrings = (obj: object) => {
}

export const unEscapeSpecialChars = (str: string) => {
return str.replaceAll('\\-', '-')
return str.replace(/\\([-,.<>{}[\]"':;!@#$%^&*()\-+=~])/g, '$1')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The character class contains a redundant escaped hyphen \- in addition to the leading hyphen -. We can simplify the regex by removing the redundant hyphen.

Suggested change
return str.replace(/\\([-,.<>{}[\]"':;!@#$%^&*()\-+=~])/g, '$1')
return str.replace(/\\([-,.<>{}[\]"':;!@#$%^&*()+=~])/g, '$1')

}
Comment on lines 54 to 56