Fix Mass Assignment in Dataset and DatasetRow Operations#6051
Fix Mass Assignment in Dataset and DatasetRow Operations#6051christopherholland-workday merged 40 commits intomainfrom
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a critical security vulnerability related to mass assignment in dataset and dataset row operations. By replacing generic Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request refactors dataset and dataset row creation and update operations to use explicit property assignments instead of Object.assign, aiming to prevent mass assignment vulnerabilities. However, the review highlights several critical and high-severity security vulnerabilities related to Insecure Direct Object Reference (IDOR) and missing authorization checks. Specifically, in updateDatasetRow, the authorization check occurs after the update, allowing unauthorized modifications. In createDataset, passing workspaceId directly from the request body is identified as a potential IDOR risk. Furthermore, in addDatasetRow, a missing authorization check before row creation could allow attackers to add data to unauthorized datasets.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request refactors the creation and update logic for datasets and dataset rows. It moves from using Object.assign with the entire request body to explicitly assigning specific properties such as name, description, input, output, workspaceId, and datasetId. This change improves data integrity and security by preventing unintended property assignments. Additionally, the PR introduces workspaceId checks for addDatasetRow and updateDatasetRow operations, enhancing access control by ensuring dataset rows are only manipulated within their designated workspaces. A review comment suggests improving the error message in updateDatasetRow to more accurately reflect that a dataset might not be found due to workspace access issues, rather than the row itself being missing.
| id: item.datasetId, | ||
| workspaceId: body.workspaceId | ||
| }) | ||
| if (!dataset) throw new InternalFlowiseError(StatusCodes.NOT_FOUND, `Dataset Row ${id} not found`) |
There was a problem hiding this comment.
The error message here (Dataset Row ${id} not found) might be misleading. If dataset is null, it means the dataset associated with the row (item.datasetId) was not found for the provided workspaceId. This indicates an access issue or a data inconsistency, not that the Dataset Row ${id} itself is missing (as item was successfully retrieved earlier). Consider a more precise message like Dataset ${item.datasetId} not found for workspace or Access denied to Dataset Row ${id}.
- Reintroduced NodeInfoDialog component with improved structure and functionality. - Added new properties (badge, tags, documentation) to NodeData interface for better metadata handling. - Updated flowExport and nodeFactory utilities to preserve new metadata during data processing. - Implemented tests for NodeInfoDialog to ensure proper rendering and functionality. - Adjusted Jest configuration to include coverage for the new NodeInfoDialog component.
* Initial flow * Fix styling to look the same as v2 * Add new atoms - SwitxhInput, Dropdown and ToolTipWithParser similar to v2 * Add test cases for all changes * Refactor input default value handling and consolidate utility functions * Allow imports from utils for atoms * Fix gemini comments * Fixed buold issues * FIx styling and create credentiall behavior * Add support for edit credentials * Fix bug to display tool and vector embeddign parameters * Add CredentialTypeSelector for HTTP node * Fix to add falback icon in case of error * Move inputDefaults under core/primitives * Added fix to check if all mandatory fields have value before submittign a credential * Fix new credential not being selected on creating * Fixes made to address comments * Gemini comment fix * Fix test error by adding theme * Refactor getDefaultValueForType to remove null/undefined handling * Fixed getDefaultValueForType to be generic * Fixed getDefaultValueForType to take single destructured object parameter
- Added a new `useMarkdown` prop to RichInput for toggling between Markdown and HTML content. - Updated the `extensions` function to conditionally configure link support based on the `useMarkdown` prop. - Modified the content update logic to handle Markdown conversion when `useMarkdown` is enabled. - Adjusted initial content loading to respect the `useMarkdown` setting for proper content type detection.
* Fix Mass Assignment in Variables Endpoints * Update packages/server/src/services/variables/index.ts Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Fix Mass Assignment in Variables Endpoints --------- Co-authored-by: christopherholland-workday <christopher.holland+evisort@workday.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: yau-wd <yau.ong@workday.com>
Co-authored-by: christopherholland-workday <christopher.holland+evisort@workday.com> Co-authored-by: yau-wd <yau.ong@workday.com>
* fix: unsubscribe redis pubsub when connection closes during prediction or after successful response * update: add heartbeat to keep connection alive and start/stop heartbeat on server start/stop * refactor: redis pub/sub methods, add unsubscribe, await subscribe * fix: redundant unsubscribe * fix: use map to store clients list ---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…tion and server expectation (#6092) * Intial changes * Test changes * Fix gemini suggestion * revert changes to RichTextEditor * Changes to create a common base class for NodeData and NodeDataBase * Fix TS error * Revert all changes to RichTextEditor * Fixed comments in nodeFactory.ts * Remove migration code * Removed unwanted change in behaviour * Remove all changes to RichTextEditor and test * Rename NodeDataBase to NodeDataSchema
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* feat: add Pipedream MCP support * fix: applied reviewer suggestions * chore: unwanted console log cleanup
…fig (#6069) The sonar-deep-research entry under chatPerplexity had its name field incorrectly set to 'sonar' instead of 'sonar-deep-research'. This caused two issues: 1. Users selecting sonar-deep-research in the UI would hit the regular sonar API endpoint instead of the deep research endpoint 2. A duplicate name collision with the existing sonar entry, causing incorrect cost lookups
fix(models): correct o4-mini pricing to fix 86% underbilling bug
* page sizes for agent execution view, agentflow view, chatflow view and docstore view are now stored in local storage. * Code improvements.
* Mass Assignment in Assistant Update Endpoint Allows Cross-Workspace Resource Reassignment * Mass Assignment in Assistant Update Endpoint Allows Cross-Workspace Resource Reassignment * Mass Assignment in Assistant Update Endpoint Allows Cross-Workspace Resource Reassignment * Mass Assignment in Assistant Update Endpoint --------- Co-authored-by: christopherholland-workday <christopher.holland+evisort@workday.com>
* Initial flow * Add tiptap - mention dependency similar to v2 * Add functionalities similiar to V2 * update to tiptap to 3.20.4 * Fix lint error * Added missiing test cases for new files * Styling changes * InputMessage should be visible on initial load * Fix gemini comment * Remove outdated SelectVariable * Add icon support similar to v2 * Fix gemini comments * Remove input folder and utils folder under atoms * Fix test error * Fix initlalDefualts not being set * Fix after merge with main * Fixes for review comments * Fix review comments * New test cases added for fixes * Remove additional space
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Fix Mass Assignment in Chatflow Endpoints * Fix Mass Assignment in Tools Endpoint * Revert "Fix Mass Assignment in Tools Endpoint" This reverts commit ed42f04. * Fix Mass Assignment in Chatflow Endpoints * chore(controllers/chatflows): replace explicit field assignment with stripProtectedFields --------- Co-authored-by: christopherholland-workday <christopher.holland+evisort@workday.com> Co-authored-by: yau-wd <yau.ong@workday.com>
…ck) (#6104) * fix(models): correct multiple model label typos in dropdown menus * review resolved
0xi4o
left a comment
There was a problem hiding this comment.
Just one nitpick. Good to go otherwise.
| try { | ||
| const appServer = getRunningExpressApp() | ||
| const newDs = new Dataset() | ||
| Object.assign(newDs, body) |
There was a problem hiding this comment.
Can overwrite workspaceId without having to reassign every field:
Object.assign(newDs, body, {
workspaceId,
})There was a problem hiding this comment.
However, there are other fields as well that we don't want to allow update for through the request, e.g. updatedTime. I think it could get more complicated with this?
There was a problem hiding this comment.
We can use the stripProtectedFields in packages/server/src/utils/ for this. This function removes all server-managed fields from request body.
* Fix IDOR in Evaluators and Evaluations Endpoints * chore(services/evaluations & EvaluatorDTO): replace explicit field assignment with stripProtectedFields --------- Co-authored-by: christopherholland-workday <christopher.holland+evisort@workday.com> Co-authored-by: yau-wd <yau.ong@workday.com>
* fix: update stripe email when changing email in account settings * fix: review feedback * fix: circular dependency, destroy all sessions on email change * add: email change confirmation flow + smtp available check + enable email flows for oss and enterprise * add: confirm email change page. update: change email behavior in account settings --------- Co-authored-by: yau-wd <yau.ong@workday.com>
* fix(gmail): require draft id in update_draft schema * fix(gmail): validate update_draft id format
…ore error (#5976) * fix: skip empty metadata filter in RetrieverTool to prevent vector store error When a user adds an "Additional Metadata Filter" then removes it, the empty filter object {} was still passed to the vector store, causing Pinecone (and potentially others) to reject it with "You must enter a filter object with at least one key-value pair." Now checks that the resolved filter has at least one key before assigning it. Fixes #4900 Signed-off-by: majiayu000 <1835304752@qq.com> * fix: rename metadatafilter to metadataFilter for camelCase consistency Signed-off-by: majiayu000 <1835304752@qq.com> * Delete packages/components/nodes/tools/RetrieverTool/RetrieverTool.test.ts --------- Signed-off-by: majiayu000 <1835304752@qq.com> Co-authored-by: Henry Heng <henryheng@flowiseai.com>
* Add Permission Checks to Vector Store API's * Update packages/server/src/routes/openai-assistants-vector-store/index.ts Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: christopherholland-workday <christopher.holland+evisort@workday.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: yau-wd <yau.ong@workday.com>
* fix: preserve sourceDocuments and metadata in ExecuteFlow node The Execute Flow node in AgentFlow V2 was only extracting the text content from chatflow responses, discarding sourceDocuments, usedTools, artifacts, and fileAnnotations metadata. This fix preserves all metadata fields in the return output and streams them via SSE when the node is the last in the flow. Fixes #5949 Signed-off-by: majiayu000 <1835304752@qq.com> * fix: address review feedback on PR #5972 - Use object destructuring for response.data metadata extraction - Extend SSE streaming test to cover artifacts and fileAnnotations Signed-off-by: majiayu000 <1835304752@qq.com> * Delete packages/components/nodes/agentflow/ExecuteFlow/ExecuteFlow.test.ts * Update ExecuteFlow.ts * Update ExecuteFlow.ts --------- Signed-off-by: majiayu000 <1835304752@qq.com> Co-authored-by: Henry Heng <henryheng@flowiseai.com>
…OutputParser (#6086) fix: restore token-by-token streaming for chains using createTextOnlyOutputParser
…JSON parsing (#6072) * fix: normalize conditionagent response content before parsing * fix(components): reuse normalized response content in ConditionAgent --------- Co-authored-by: joonlim <joonlim@joonlims-Mac-Studio.local>
… better handling of connection issues (#6114) * fix: unsubscribe redis pubsub when connection closes during prediction or after successful response * update: add heartbeat to keep connection alive and start/stop heartbeat on server start/stop * refactor: redis pub/sub methods, add unsubscribe, await subscribe * fix: redundant unsubscribe * fix: use map to store clients list * fix: update how thinking event is handled in redis pub/sub and sse * fix: review feedback
* feat(audit): add emitEvent and file-based audit logging * fix(sanitize.util.ts): correct IPv6 sanitization for audit log IP masking * feat(account): add account deletion * fix(sanitize.util.ts): add recursive sanitization for nested metadata * fix(MainLayout/Header/Workspace): refresh workspace list when clicking dropdown * feat(views/account): require typed confirmation for account deletion
* fix(ui): allow XML tag for rich input - Introduced utilities for escaping and unescaping non-standard XML/HTML tags to preserve them during markdown processing. - Updated RichInput and ExpandRichInputDialog components to utilize the new XML tag utilities for handling custom tags. - Added Jest configuration for the UI package and created tests for the XML tag utilities to ensure functionality. - Updated package.json to include Jest as a dependency and added a test script. * address code review comments from Gemini * allow html tag in markdown editor * add test case for markdown format * update dependency in ui
…owise into flowise-323-dataset
Flowise-323 (See testing section for Dataset and DatasetRow)