feat(olostep): add Olostep service bubble for web scraping and AI con…#223
Open
ZeeshanAdilButt wants to merge 5 commits intobubblelabai:mainfrom
Open
feat(olostep): add Olostep service bubble for web scraping and AI con…#223ZeeshanAdilButt wants to merge 5 commits intobubblelabai:mainfrom
ZeeshanAdilButt wants to merge 5 commits intobubblelabai:mainfrom
Conversation
Contributor
Suggested PR title from PearlTitle: Body: SummaryAdds a new service bubble integration for Olostep API, enabling web scraping and AI-powered content extraction capabilities. FeaturesImplements 5 core operations:
Changes
Use Cases
Technical Details
|
ZeeshanAdilButt
force-pushed
the
feat/olostep-integration
branch
from
e69c95c to
a67e345
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds comprehensive integration for Olostep, a web scraping and AI-powered content extraction service, to the BubbleLab platform. The implementation follows established patterns in the codebase and includes proper credential management, schema definitions, factory registration, and test coverage.
Key Changes:
- Adds OlostepBubble service with 5 operations: scrape, batch, crawl, map, and answer
- Introduces OLOSTEP_API_KEY credential type with full configuration across packages
- Registers the bubble in the factory and exports it from the core package
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/bubble-shared-schemas/src/types.ts | Adds OLOSTEP_API_KEY credential type and 'olostep' to BubbleName union type |
| packages/bubble-shared-schemas/src/credential-schema.ts | Registers OLOSTEP_API_KEY in credential environment mapping and bubble credential options |
| packages/bubble-shared-schemas/src/bubble-definition-schema.ts | Adds empty credential configuration map entry for OLOSTEP_API_KEY |
| packages/bubble-core/src/bubbles/service-bubble/olostep.ts | Implements OlostepBubble class with discriminated union schemas for 5 operations, API client methods, and error handling |
| packages/bubble-core/src/bubbles/service-bubble/olostep.test.ts | Provides comprehensive unit tests covering registration, schemas, metadata, credentials, and all 5 operations |
| packages/bubble-core/src/bubble-factory.ts | Registers OlostepBubble in factory and includes it in code generator bubble list |
| packages/bubble-core/src/index.ts | Exports OlostepBubble class and OlostepParamsInput type from package |
| apps/bubble-studio/src/pages/CredentialsPage.tsx | Adds UI configuration for OLOSTEP_API_KEY credential with label, description, and placeholder |
Comment on lines
+27
to
+354
| describe('OlostepBubble', () => { | ||
| // | ||
| // REGISTRATION & SCHEMA | ||
| // | ||
| describe('Registration & Schema', () => { | ||
| it('should be registered in BubbleRegistry', async () => { | ||
| const bubbleClass = factory.get('olostep'); | ||
| expect(bubbleClass).toBeDefined(); | ||
| expect(bubbleClass).toBe(OlostepBubble); | ||
| }); | ||
|
|
||
| it('schema should be a Zod discriminated union based on "operation"', () => { | ||
| const schema = OlostepBubble.schema; | ||
| expect(schema).toBeDefined(); | ||
|
|
||
| // Validate ZodDiscriminatedUnion | ||
| expect(schema instanceof ZodDiscriminatedUnion).toBe(true); | ||
|
|
||
| const du = schema as ZodDiscriminatedUnion< | ||
| 'operation', | ||
| readonly ZodObject<any>[] | ||
| >; | ||
| expect(du.discriminator).toBe('operation'); | ||
|
|
||
| const operationValues = du.options.map((o) => o.shape.operation.value); | ||
|
|
||
| expect(operationValues).toContain('scrape'); | ||
| expect(operationValues).toContain('batch'); | ||
| expect(operationValues).toContain('crawl'); | ||
| expect(operationValues).toContain('map'); | ||
| expect(operationValues).toContain('answer'); | ||
| }); | ||
|
|
||
| it('result schema should validate a sample scrape result', () => { | ||
| const sample = { | ||
| operation: 'scrape', | ||
| success: true, | ||
| error: '', | ||
| markdown_content: '# Hello World', | ||
| }; | ||
|
|
||
| const parsed = OlostepBubble.resultSchema.safeParse(sample); | ||
| expect(parsed.success).toBe(true); | ||
| }); | ||
| }); | ||
|
|
||
| // | ||
| // METADATA | ||
| // | ||
| describe('Metadata Tests', () => { | ||
| it('should have correct metadata', () => { | ||
| const metadata = factory.getMetadata('olostep'); | ||
|
|
||
| expect(metadata).toBeDefined(); | ||
| expect(metadata?.name).toBe('olostep'); | ||
| expect(metadata?.alias).toBe('web-scraper'); | ||
| expect(metadata?.schema).toBeDefined(); | ||
| expect(metadata?.resultSchema).toBeDefined(); | ||
| expect(metadata?.shortDescription).toContain('Web scraping'); | ||
| expect(metadata?.longDescription).toContain('Olostep'); | ||
| }); | ||
|
|
||
| it('static properties are correct', () => { | ||
| expect(OlostepBubble.bubbleName).toBe('olostep'); | ||
| expect(OlostepBubble.alias).toBe('web-scraper'); | ||
| expect(OlostepBubble.service).toBe('olostep'); | ||
| expect(OlostepBubble.authType).toBe('apikey'); | ||
| expect(OlostepBubble.type).toBe('service'); | ||
| expect(OlostepBubble.schema).toBeDefined(); | ||
| expect(OlostepBubble.resultSchema).toBeDefined(); | ||
| expect(OlostepBubble.shortDescription).toContain('Web scraping'); | ||
| expect(OlostepBubble.longDescription).toContain('Scrape'); | ||
| }); | ||
| }); | ||
|
|
||
| // | ||
| // CREDENTIAL VALIDATION | ||
| // | ||
| describe('Credential Validation', () => { | ||
| it('should fail testCredential() with missing credentials', async () => { | ||
| const bubble = new OlostepBubble({ | ||
| operation: 'scrape', | ||
| url: 'https://example.com', | ||
| }); | ||
|
|
||
| const result = await bubble.testCredential(); | ||
| expect(result).toBe(false); | ||
| }); | ||
|
|
||
| it('should pass testCredential() with valid credentials', async () => { | ||
| mockFetch.mockResolvedValueOnce({ | ||
| ok: true, | ||
| text: async () => JSON.stringify({ markdown_content: '# Test' }), | ||
| }); | ||
|
|
||
| const bubble = new OlostepBubble({ | ||
| operation: 'scrape', | ||
| url: 'https://example.com', | ||
| credentials: createTestCredentials(), | ||
| }); | ||
|
|
||
| const result = await bubble.testCredential(); | ||
| expect(result).toBe(true); | ||
| }); | ||
| }); | ||
|
|
||
| // | ||
| // SCRAPE OPERATION | ||
| // | ||
| describe('Scrape Operation', () => { | ||
| it('should create bubble with scrape operation', () => { | ||
| const params: OlostepParamsInput = { | ||
| operation: 'scrape', | ||
| url: 'https://example.com', | ||
| formats: ['markdown'], | ||
| }; | ||
|
|
||
| const bubble = new OlostepBubble(params); | ||
| expect((bubble as any).params.operation).toBe('scrape'); | ||
| expect((bubble as any).params.url).toBe('https://example.com'); | ||
| }); | ||
|
|
||
| it('should accept all scrape optional parameters', () => { | ||
| const params: OlostepParamsInput = { | ||
| operation: 'scrape', | ||
| url: 'https://example.com', | ||
| formats: ['markdown', 'html'], | ||
| country: 'US', | ||
| wait_before_scraping: 2000, | ||
| parser: '@olostep/product-page', | ||
| }; | ||
|
|
||
| const bubble = new OlostepBubble(params); | ||
| expect((bubble as any).params.formats).toEqual(['markdown', 'html']); | ||
| expect((bubble as any).params.country).toBe('US'); | ||
| expect((bubble as any).params.wait_before_scraping).toBe(2000); | ||
| expect((bubble as any).params.parser).toBe('@olostep/product-page'); | ||
| }); | ||
|
|
||
| it('should return success for scrape', async () => { | ||
| mockFetch.mockResolvedValueOnce({ | ||
| ok: true, | ||
| text: async () => | ||
| JSON.stringify({ | ||
| markdown_content: '# Test Page', | ||
| metadata: { title: 'Test', url: 'https://example.com' }, | ||
| }), | ||
| }); | ||
|
|
||
| const bubble = new OlostepBubble({ | ||
| operation: 'scrape', | ||
| url: 'https://example.com', | ||
| credentials: createTestCredentials(), | ||
| }); | ||
|
|
||
| const res = await bubble.action(); | ||
|
|
||
| expect(res.data.operation).toBe('scrape'); | ||
| expect(res.success).toBe(true); | ||
| expect(res.error).toBe(''); | ||
| expect(res.data.markdown_content).toBeDefined(); | ||
| }); | ||
| }); | ||
|
|
||
| // | ||
| // BATCH OPERATION | ||
| // | ||
| describe('Batch Operation', () => { | ||
| it('should create bubble with batch operation', () => { | ||
| const params: OlostepParamsInput = { | ||
| operation: 'batch', | ||
| urls: ['https://example.com/1', 'https://example.com/2'], | ||
| }; | ||
|
|
||
| const bubble = new OlostepBubble(params); | ||
| expect((bubble as any).params.operation).toBe('batch'); | ||
| expect((bubble as any).params.urls).toHaveLength(2); | ||
| }); | ||
|
|
||
| it('should return success for batch', async () => { | ||
| mockFetch.mockResolvedValueOnce({ | ||
| ok: true, | ||
| text: async () => | ||
| JSON.stringify({ | ||
| batch_id: 'batch_123', | ||
| status: 'processing', | ||
| }), | ||
| }); | ||
|
|
||
| const bubble = new OlostepBubble({ | ||
| operation: 'batch', | ||
| urls: ['https://example.com/1', 'https://example.com/2'], | ||
| credentials: createTestCredentials(), | ||
| }); | ||
|
|
||
| const res = await bubble.action(); | ||
|
|
||
| expect(res.data.operation).toBe('batch'); | ||
| expect(res.success).toBe(true); | ||
| expect(res.data.batch_id).toBeDefined(); | ||
| }); | ||
| }); | ||
|
|
||
| // | ||
| // CRAWL OPERATION | ||
| // | ||
| describe('Crawl Operation', () => { | ||
| it('should create bubble with crawl operation', () => { | ||
| const params: OlostepParamsInput = { | ||
| operation: 'crawl', | ||
| start_url: 'https://example.com', | ||
| max_pages: 50, | ||
| }; | ||
|
|
||
| const bubble = new OlostepBubble(params); | ||
| expect((bubble as any).params.operation).toBe('crawl'); | ||
| expect((bubble as any).params.start_url).toBe('https://example.com'); | ||
| expect((bubble as any).params.max_pages).toBe(50); | ||
| }); | ||
|
|
||
| it('should return success for crawl', async () => { | ||
| mockFetch.mockResolvedValueOnce({ | ||
| ok: true, | ||
| text: async () => | ||
| JSON.stringify({ | ||
| crawl_id: 'crawl_123', | ||
| status: 'completed', | ||
| pages_crawled: 10, | ||
| }), | ||
| }); | ||
|
|
||
| const bubble = new OlostepBubble({ | ||
| operation: 'crawl', | ||
| start_url: 'https://example.com', | ||
| credentials: createTestCredentials(), | ||
| }); | ||
|
|
||
| const res = await bubble.action(); | ||
|
|
||
| expect(res.data.operation).toBe('crawl'); | ||
| expect(res.success).toBe(true); | ||
| expect(res.data.crawl_id).toBeDefined(); | ||
| }); | ||
| }); | ||
|
|
||
| // | ||
| // MAP OPERATION | ||
| // | ||
| describe('Map Operation', () => { | ||
| it('should create bubble with map operation', () => { | ||
| const params: OlostepParamsInput = { | ||
| operation: 'map', | ||
| url: 'https://example.com', | ||
| top_n: 200, | ||
| }; | ||
|
|
||
| const bubble = new OlostepBubble(params); | ||
| expect((bubble as any).params.operation).toBe('map'); | ||
| expect((bubble as any).params.url).toBe('https://example.com'); | ||
| expect((bubble as any).params.top_n).toBe(200); | ||
| }); | ||
|
|
||
| it('should return success for map', async () => { | ||
| mockFetch.mockResolvedValueOnce({ | ||
| ok: true, | ||
| text: async () => | ||
| JSON.stringify({ | ||
| urls: ['https://example.com/page1', 'https://example.com/page2'], | ||
| total_urls: 2, | ||
| }), | ||
| }); | ||
|
|
||
| const bubble = new OlostepBubble({ | ||
| operation: 'map', | ||
| url: 'https://example.com', | ||
| credentials: createTestCredentials(), | ||
| }); | ||
|
|
||
| const res = await bubble.action(); | ||
|
|
||
| expect(res.data.operation).toBe('map'); | ||
| expect(res.success).toBe(true); | ||
| expect(res.data.urls).toBeDefined(); | ||
| expect(res.data.urls!.length).toBeGreaterThan(0); | ||
| }); | ||
| }); | ||
|
|
||
| // | ||
| // ANSWER OPERATION | ||
| // | ||
| describe('Answer Operation', () => { | ||
| it('should create bubble with answer operation', () => { | ||
| const params: OlostepParamsInput = { | ||
| operation: 'answer', | ||
| task: 'What is the main topic of this website?', | ||
| context_urls: ['https://example.com'], | ||
| }; | ||
|
|
||
| const bubble = new OlostepBubble(params); | ||
| expect((bubble as any).params.operation).toBe('answer'); | ||
| expect((bubble as any).params.task).toContain('main topic'); | ||
| }); | ||
|
|
||
| it('should return success for answer', async () => { | ||
| mockFetch.mockResolvedValueOnce({ | ||
| ok: true, | ||
| text: async () => | ||
| JSON.stringify({ | ||
| answer: 'This website is about example content.', | ||
| citations: [{ url: 'https://example.com', title: 'Example' }], | ||
| sources_used: 1, | ||
| }), | ||
| }); | ||
|
|
||
| const bubble = new OlostepBubble({ | ||
| operation: 'answer', | ||
| task: 'What is this website about?', | ||
| credentials: createTestCredentials(), | ||
| }); | ||
|
|
||
| const res = await bubble.action(); | ||
|
|
||
| expect(res.data.operation).toBe('answer'); | ||
| expect(res.success).toBe(true); | ||
| expect(res.data.answer).toBeDefined(); | ||
| }); | ||
| }); | ||
| }); |
There was a problem hiding this comment.
The test suite is missing error handling tests. Consider adding tests for:
- API request failures (non-2xx responses)
- Network errors (fetch throws exception)
- Invalid API key responses
- Malformed response data
Other service bubbles like ElevenLabsBubble include these types of tests to ensure robust error handling.
Comment on lines
+320
to
+341
| const apiKey = this.chooseCredential(); | ||
| if (!apiKey) return false; | ||
|
|
||
| try { | ||
| // Simple health check with minimal scrape | ||
| const response = await fetch(`${OLOSTEP_API_URL}/scrapes`, { | ||
| method: 'POST', | ||
| headers: { | ||
| Authorization: `Bearer ${apiKey}`, | ||
| 'Content-Type': 'application/json', | ||
| }, | ||
| body: JSON.stringify({ | ||
| url_to_scrape: 'https://example.com', | ||
| formats: ['text'], | ||
| }), | ||
| }); | ||
| return response.ok; | ||
| } catch { | ||
| return false; | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
The testCredential method makes a full scrape request to example.com, which could be slow and might consume API credits. Consider using a lighter-weight endpoint for credential validation, such as a health check or user info endpoint if available from the Olostep API. If no such endpoint exists, this approach is acceptable but may result in slower credential validation and unnecessary API usage.
Comment on lines
+575
to
+577
| return { operation: 'scrape', ...base }; | ||
| } | ||
| } |
There was a problem hiding this comment.
The default case returns an operation type of 'scrape' which could be misleading if an unknown operation somehow reaches this point. While TypeScript's type system should prevent this, consider either throwing an error or using a type assertion to indicate this is unreachable. For example: throw new Error('Unreachable: invalid operation type') or using operation as 'scrape' with a comment explaining this is a fallback that should never occur.
iqbalbhatti49
approved these changes
Dec 16, 2025
ZeeshanAdilButt
force-pushed
the
feat/olostep-integration
branch
from
a67e345 to
a29b5d3
Compare
…tent extraction - Add OlostepBubble with 5 operations: scrape, batch, crawl, map, answer - Add OLOSTEP_API_KEY credential type and configuration - Add 'olostep' to BubbleName type - Register bubble in BubbleFactory - Export from bubble-core index - Add comprehensive unit tests - Add credential UI configuration in bubble-studio
ZeeshanAdilButt
force-pushed
the
feat/olostep-integration
branch
from
39dceca to
b3759f3
Compare
📝 WalkthroughWalkthroughAdded a new Olostep service bubble with input/result schemas and handlers for five operations, registered the bubble and exports in core, introduced a credential type and UI mapping for OLOSTEP_API_KEY, updated schema/credential maps, and added comprehensive tests covering success and failure modes. Changes
Sequence Diagram(s)sequenceDiagram
participant Client as User/Client
participant Factory as BubbleFactory
participant Bubble as OlostepBubble
participant API as Olostep API
Client->>Factory: request bubble registration / instantiate
Factory->>Bubble: dynamic import/register 'olostep'
Client->>Bubble: testCredential()
Bubble->>API: POST /v1/scrapes (Bearer OLOSTEP_API_KEY)
API-->>Bubble: 2xx / non-2xx / error
Bubble-->>Client: boolean (credential valid/invalid)
Client->>Bubble: action(params)
Bubble->>Bubble: performAction() (route by operation)
Bubble->>API: POST /v1/{scrapes|batches|crawls|maps|answers} (Bearer)
API-->>Bubble: JSON or text response
Bubble->>Bubble: parse/normalize result or create error
Bubble-->>Client: { success, error?, result? }
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Author
|
@iqbalbhatti49 I refreshed this PR on top of the current If you get a moment, could you please take another look? |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/bubble-core/src/bubbles/service-bubble/olostep.test.ts`:
- Around line 13-14: The file stubs fetch at module scope using mockFetch and
vi.stubGlobal('fetch', mockFetch) but never restores it; add cleanup to undo the
global stub after tests by calling the appropriate Vitest restore/un-stub
function (e.g., vi.unstubAllGlobals() or vi.restoreAllMocks()) in an
afterEach/afterAll hook and reset the mock (mockFetch.mockReset() or
mockFetch.mockRestore()) so the global fetch stub does not leak into other test
files; target the module-scope mockFetch and the vi.stubGlobal('fetch',
mockFetch) usage when adding the teardown hook.
In `@packages/bubble-core/src/bubbles/service-bubble/olostep.ts`:
- Around line 319-339: testCredential makes a fetch to OLOSTEP with no timeout;
add an AbortController-based timeout: create an AbortController, pass
controller.signal into fetch, set a timer (e.g., configurable ms) that calls
controller.abort(), and clear the timer after fetch completes; catch AbortError
and return false as currently done. Apply the same pattern to the normal
execution path that issues POST /scrapes (the method that performs scrapes
during runtime) so all Olostep HTTP calls have an abort/timeout.
- Around line 550-553: The response error construction in olostep.ts currently
discards string-valued API errors and falls back to a generic `HTTP
${response.status}` message; update the error extraction in the response
handling (the block that currently throws new Error(json?.error?.message ||
json?.message || `HTTP ${response.status}`)) to detect when json?.error is a
string (e.g., typeof json.error === 'string') and prefer that string, then fall
back to json.error.message, json.message, and finally the HTTP status; keep this
change localized to the error-throwing logic in the olostep response handling so
auth and rate-limit string messages are preserved.
In `@packages/bubble-core/src/index.ts`:
- Around line 167-168: The barrel export is missing the new Olostep public types
— besides OlostepBubble and OlostepParamsInput, the module
'./bubbles/service-bubble/olostep.js' now exports additional operation-specific
param types and/or other public types that must be surfaced; update
packages/bubble-core/src/index.ts to re-export all remaining public symbols from
that module (add export type { /* the operation-specific param types and any
other exported types from olostep.js */ } from
'./bubbles/service-bubble/olostep.js') so consumers of `@bubblelab/bubble-core`
can import the new Olostep types.
In `@packages/bubble-shared-schemas/src/bubble-definition-schema.ts`:
- Line 52: You added CredentialType.OLOSTEP_API_KEY to the shared schema but
didn't rebuild the dependent package; run pnpm build in the packages/bubble-core
package to regenerate the bundle so downstream types (bubble-bundle.d.ts) pick
up the new CredentialType entry (CredentialType.OLOSTEP_API_KEY) and then commit
the updated bundle output.
In `@packages/bubble-shared-schemas/src/credential-schema.ts`:
- Line 2740: BUBBLE_CREDENTIAL_OPTIONS added an "olostep" entry mapped to
CredentialType.OLOSTEP_API_KEY but the switch in
apps/bubblelab-api/src/services/credential-validator.ts lacks a case for
CredentialType.OLOSTEP_API_KEY; add a case branch for
CredentialType.OLOSTEP_API_KEY in the validator (the same switch that handles
other API-key-like types) that checks the olostep payload fields required by the
olostep entry (e.g., the API key field present and non-empty), returns or throws
the same validation result shape as the other cases, and ensure the new branch
uses the same error messages/behavior as similar credential types so
create/update flows don’t fall through to the default no-op case.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 9edade0f-863b-4026-a0c6-a4ad155e8623
📒 Files selected for processing (8)
apps/bubble-studio/src/pages/CredentialsPage.tsxpackages/bubble-core/src/bubble-factory.tspackages/bubble-core/src/bubbles/service-bubble/olostep.test.tspackages/bubble-core/src/bubbles/service-bubble/olostep.tspackages/bubble-core/src/index.tspackages/bubble-shared-schemas/src/bubble-definition-schema.tspackages/bubble-shared-schemas/src/credential-schema.tspackages/bubble-shared-schemas/src/types.ts
| [CredentialType.NOTION_API]: {}, | ||
| [CredentialType.INSFORGE_BASE_URL]: {}, | ||
| [CredentialType.INSFORGE_API_KEY]: {}, | ||
| [CredentialType.OLOSTEP_API_KEY]: {}, |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Check for changes in bubble-core dist files and other generated outputs
git diff --stat HEAD~1..HEAD packages/bubble-core/dist/ packages/bubble-core/src/index.ts apps/bubble-studio/public/bubble-types.txt 2>/dev/null || echo "No git diff available"
# Check current state of dist files
ls -la packages/bubble-core/dist/ 2>/dev/null | head -20 || echo "dist directory not found or empty"
# Look for recent modifications to generated files
find packages/bubble-core/dist -type f -name "*.d.ts" -o -name "bubble-bundle*" 2>/dev/null | xargs ls -lt 2>/dev/null | head -10 || echo "No generated d.ts files found"Repository: bubblelabai/BubbleLab
Length of output: 662
🏁 Script executed:
# Check if bubble-core dist directory exists
ls -la packages/bubble-core/dist/ 2>/dev/null || echo "dist directory does not exist or is empty"
# Check the shared schema file that was changed
cat packages/bubble-shared-schemas/src/bubble-definition-schema.ts | grep -A 5 -B 5 "OLOSTEP_API_KEY" || echo "Pattern not found"
# List all files in packages/bubble-core/dist/
find packages/bubble-core/dist -type f 2>/dev/null | head -20 || echo "No files in dist"
# Check if bubble-bundle.d.ts exists
test -f packages/bubble-core/dist/bubble-bundle.d.ts && echo "bubble-bundle.d.ts EXISTS" || echo "bubble-bundle.d.ts DOES NOT EXIST"
# Check the package.json build scripts to confirm what build:core does
cat packages/bubble-core/package.json | grep -A 20 '"scripts"' || echo "Cannot find scripts in package.json"Repository: bubblelabai/BubbleLab
Length of output: 1533
Regenerate bubble-core after this shared-schema change.
This addition updates a shared package consumed by bubble-core. Please run pnpm build in the packages/bubble-core directory to regenerate the bundle (bubble-bundle.d.ts) and ensure the updated types are available to downstream consumers.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/bubble-shared-schemas/src/bubble-definition-schema.ts` at line 52,
You added CredentialType.OLOSTEP_API_KEY to the shared schema but didn't rebuild
the dependent package; run pnpm build in the packages/bubble-core package to
regenerate the bundle so downstream types (bubble-bundle.d.ts) pick up the new
CredentialType entry (CredentialType.OLOSTEP_API_KEY) and then commit the
updated bundle output.
| metabase: [CredentialType.METABASE_CRED], | ||
| clerk: [CredentialType.CLERK_CRED], | ||
| granola: [CredentialType.GRANOLA_API_KEY], | ||
| olostep: [CredentialType.OLOSTEP_API_KEY], |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
validator_file="$(fd '^credential-validator\.ts$' | head -n 1)"
test -n "$validator_file"
echo "Inspecting: $validator_file"
sed -n '1,260p' "$validator_file" | rg -nC2 'case CredentialType\.|OLOSTEP_API_KEY|default:'Repository: bubblelabai/BubbleLab
Length of output: 2280
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find the validator file
validator_file="$(fd '^credential-validator\.ts$' | head -n 1)"
# Count total lines and check if OLOSTEP appears anywhere
echo "=== Full file check for OLOSTEP_API_KEY ==="
rg -c 'OLOSTEP_API_KEY' "$validator_file" || echo "No matches found"
# Get line count
total_lines=$(wc -l < "$validator_file")
echo "Total lines in validator: $total_lines"
# Check the full switch statement (look for all cases)
echo "=== Full switch statement ==="
sed -n '145,210p' "$validator_file"Repository: bubblelabai/BubbleLab
Length of output: 2273
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Verify that olostep was added to BUBBLE_CREDENTIAL_OPTIONS in the shared schema file
schema_file="packages/bubble-shared-schemas/src/credential-schema.ts"
echo "=== Checking for BUBBLE_CREDENTIAL_OPTIONS.olostep ==="
rg -n 'olostep.*CredentialType\.OLOSTEP_API_KEY|OLOSTEP_API_KEY' "$schema_file"
echo ""
echo "=== Checking CredentialType enum for OLOSTEP_API_KEY ==="
rg -n 'OLOSTEP_API_KEY' "$schema_file" | head -20Repository: bubblelabai/BubbleLab
Length of output: 483
Add OLOSTEP_API_KEY handling to the credential validator.
olostep is now exposed through BUBBLE_CREDENTIAL_OPTIONS (line 2740), but apps/bubblelab-api/src/services/credential-validator.ts has no case CredentialType.OLOSTEP_API_KEY: branch. New Olostep credentials will fall through to the default case, which performs no validation and will likely fail during create/update operations.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/bubble-shared-schemas/src/credential-schema.ts` at line 2740,
BUBBLE_CREDENTIAL_OPTIONS added an "olostep" entry mapped to
CredentialType.OLOSTEP_API_KEY but the switch in
apps/bubblelab-api/src/services/credential-validator.ts lacks a case for
CredentialType.OLOSTEP_API_KEY; add a case branch for
CredentialType.OLOSTEP_API_KEY in the validator (the same switch that handles
other API-key-like types) that checks the olostep payload fields required by the
olostep entry (e.g., the API key field present and non-empty), returns or throws
the same validation result shape as the other cases, and ensure the new branch
uses the same error messages/behavior as similar credential types so
create/update flows don’t fall through to the default no-op case.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
packages/bubble-core/src/bubbles/service-bubble/olostep.ts (2)
550-553:⚠️ Potential issue | 🟡 MinorPreserve string-valued API errors.
If Olostep returns
{ error: "..." }, this path falls back toHTTP ${response.status}and hides the actual auth/rate-limit failure.Suggested fix
if (!response.ok) { + const errorMessage = + typeof json?.error === 'string' + ? json.error + : json?.error?.message || json?.message; throw new Error( - json?.error?.message || json?.message || `HTTP ${response.status}` + errorMessage || `HTTP ${response.status}` ); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/bubble-core/src/bubbles/service-bubble/olostep.ts` around lines 550 - 553, The error construction in the response failure path currently uses json?.error?.message which drops string-valued errors; update the throw in the response handling block that creates new Error(...) (the one referencing response and json) to first check if typeof json?.error === 'string' and use that, otherwise fall back to json?.error?.message || json?.message || `HTTP ${response.status}` so string error bodies like { error: "..." } are preserved.
323-335:⚠️ Potential issue | 🟠 MajorAdd a timeout/abort path to Olostep HTTP calls.
Both credential validation and runtime execution still make blocking
fetch()calls with no timeout. A stalled Olostep request can hang the worker indefinitely.Suggested fix
+ private async fetchWithTimeout( + input: string, + init: RequestInit, + timeoutMs = 30000 + ): Promise<Response> { + const controller = new AbortController(); + const timeout = setTimeout(() => controller.abort(), timeoutMs); + + try { + return await fetch(input, { ...init, signal: controller.signal }); + } finally { + clearTimeout(timeout); + } + } + public async testCredential(): Promise<boolean> { @@ - const response = await fetch(`${OLOSTEP_API_URL}/scrapes`, { + const response = await this.fetchWithTimeout(`${OLOSTEP_API_URL}/scrapes`, { method: 'POST', headers: { Authorization: `Bearer ${apiKey}`, @@ - const response = await fetch(`${OLOSTEP_API_URL}${endpoint}`, { + const response = await this.fetchWithTimeout(`${OLOSTEP_API_URL}${endpoint}`, { method: 'POST', headers: { Authorization: `Bearer ${apiKey}`,Also applies to: 527-540
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/bubble-core/src/bubbles/service-bubble/olostep.ts` around lines 323 - 335, The fetch calls in olostep.ts currently have no timeout and can hang; wrap each Olostep HTTP call (the health-check/scrape POST near the shown fetch and the other fetch at the second location) with an AbortController, pass controller.signal into fetch, start a setTimeout that calls controller.abort() after a reasonable timeout (e.g., 5–10s), and clear the timeout once fetch resolves; also catch the abort/DOMException and handle it (return a timeout error or throw a controlled exception) so credential validation and runtime execution do not block indefinitely.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/bubble-core/src/bubbles/service-bubble/olostep.ts`:
- Around line 386-387: The payload assignment currently skips explicit zero
because it uses a truthy check; replace the truthy guard with an explicit
undefined/null check so 0 is preserved—e.g., change the condition around
wait_before_scraping to check that wait_before_scraping is not undefined (or not
null) before setting payload.wait_before_scraping, and apply the same change to
the second occurrence (the other check around wait_before_scraping in the same
file).
---
Duplicate comments:
In `@packages/bubble-core/src/bubbles/service-bubble/olostep.ts`:
- Around line 550-553: The error construction in the response failure path
currently uses json?.error?.message which drops string-valued errors; update the
throw in the response handling block that creates new Error(...) (the one
referencing response and json) to first check if typeof json?.error === 'string'
and use that, otherwise fall back to json?.error?.message || json?.message ||
`HTTP ${response.status}` so string error bodies like { error: "..." } are
preserved.
- Around line 323-335: The fetch calls in olostep.ts currently have no timeout
and can hang; wrap each Olostep HTTP call (the health-check/scrape POST near the
shown fetch and the other fetch at the second location) with an AbortController,
pass controller.signal into fetch, start a setTimeout that calls
controller.abort() after a reasonable timeout (e.g., 5–10s), and clear the
timeout once fetch resolves; also catch the abort/DOMException and handle it
(return a timeout error or throw a controlled exception) so credential
validation and runtime execution do not block indefinitely.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 6640e94a-e3bf-4347-9ba2-ef91ef0b3e4d
📒 Files selected for processing (1)
packages/bubble-core/src/bubbles/service-bubble/olostep.ts
iqbalbhatti49
approved these changes
Apr 13, 2026
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
packages/bubble-core/src/bubbles/service-bubble/olostep.ts (1)
43-46: Extract duplicated credentials schema into a shared constant.The same credentials schema is repeated in every operation branch. Centralizing it will reduce drift and make future credential changes safer.
♻️ Refactor sketch
+const CredentialsSchema = z + .record(z.nativeEnum(CredentialType), z.string()) + .optional() + .describe('Credentials (injected at runtime)'); + const OlostepParamsSchema = z.discriminatedUnion('operation', [ z.object({ @@ - credentials: z - .record(z.nativeEnum(CredentialType), z.string()) - .optional() - .describe('Credentials (injected at runtime)'), + credentials: CredentialsSchema, }),Also applies to: 80-83, 119-122, 149-152, 187-190
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/bubble-core/src/bubbles/service-bubble/olostep.ts` around lines 43 - 46, Multiple operation branches duplicate the same credentials zod schema; extract that z.record(z.nativeEnum(CredentialType), z.string()).optional().describe(...) into a single exported constant (e.g., CREDENTIALS_SCHEMA or credentialsSchema) and replace each inlined occurrence (the ones around the z.record(...) expression used in the operation branches) with a reference to that constant so all schemas reuse the same definition and future changes are centralized.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/bubble-core/src/bubbles/service-bubble/olostep.ts`:
- Around line 559-570: The error builder ignores the plain-text fallback stored
in json.raw when JSON.parse fails; update the errorMessage construction in
olostep.ts to include json.raw (or append it when present) as a final fallback
so non-JSON error bodies are preserved in the thrown Error (use the existing
variables json, text, response, errorMessage and the existing throw new
Error(...) site to include json.raw alongside
json.error/json.message/response.status).
---
Nitpick comments:
In `@packages/bubble-core/src/bubbles/service-bubble/olostep.ts`:
- Around line 43-46: Multiple operation branches duplicate the same credentials
zod schema; extract that z.record(z.nativeEnum(CredentialType),
z.string()).optional().describe(...) into a single exported constant (e.g.,
CREDENTIALS_SCHEMA or credentialsSchema) and replace each inlined occurrence
(the ones around the z.record(...) expression used in the operation branches)
with a reference to that constant so all schemas reuse the same definition and
future changes are centralized.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: e350d410-0b5d-4745-9a5c-46b671c43ffb
📒 Files selected for processing (3)
packages/bubble-core/src/bubbles/service-bubble/olostep.test.tspackages/bubble-core/src/bubbles/service-bubble/olostep.tspackages/bubble-core/src/index.ts
✅ Files skipped from review due to trivial changes (1)
- packages/bubble-core/src/index.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/bubble-core/src/bubbles/service-bubble/olostep.test.ts
Comment on lines
+559
to
+570
| try { | ||
| json = text ? JSON.parse(text) : {}; | ||
| } catch { | ||
| json = { raw: text }; | ||
| } | ||
|
|
||
| if (!response.ok) { | ||
| const errorMessage = | ||
| typeof json?.error === 'string' | ||
| ? json.error | ||
| : json?.error?.message || json?.message; | ||
| throw new Error(errorMessage || `HTTP ${response.status}`); |
There was a problem hiding this comment.
Preserve plain-text error bodies on non-JSON failures.
Lines 561-563 store fallback text in json.raw, but Lines 566-570 never read it. This can hide useful API error details.
💡 Proposed fix
if (!response.ok) {
const errorMessage =
typeof json?.error === 'string'
? json.error
- : json?.error?.message || json?.message;
+ : json?.error?.message ||
+ json?.message ||
+ (typeof json?.raw === 'string' ? json.raw : undefined);
throw new Error(errorMessage || `HTTP ${response.status}`);
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| try { | |
| json = text ? JSON.parse(text) : {}; | |
| } catch { | |
| json = { raw: text }; | |
| } | |
| if (!response.ok) { | |
| const errorMessage = | |
| typeof json?.error === 'string' | |
| ? json.error | |
| : json?.error?.message || json?.message; | |
| throw new Error(errorMessage || `HTTP ${response.status}`); | |
| try { | |
| json = text ? JSON.parse(text) : {}; | |
| } catch { | |
| json = { raw: text }; | |
| } | |
| if (!response.ok) { | |
| const errorMessage = | |
| typeof json?.error === 'string' | |
| ? json.error | |
| : json?.error?.message || | |
| json?.message || | |
| (typeof json?.raw === 'string' ? json.raw : undefined); | |
| throw new Error(errorMessage || `HTTP ${response.status}`); |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/bubble-core/src/bubbles/service-bubble/olostep.ts` around lines 559
- 570, The error builder ignores the plain-text fallback stored in json.raw when
JSON.parse fails; update the errorMessage construction in olostep.ts to include
json.raw (or append it when present) as a final fallback so non-JSON error
bodies are preserved in the thrown Error (use the existing variables json, text,
response, errorMessage and the existing throw new Error(...) site to include
json.raw alongside json.error/json.message/response.status).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…tent extraction
Summary
Adds Olostep integration to BubbleLab - web scraping and AI-powered content extraction with 5 operations: scrape, batch, crawl, map, and answer.
Type of Change
Checklist
pnpm checkand all tests pass (pre-existing lint errors in bubblelab-api unrelated to this PR)Screenshots (if applicable)
Additional Context
Summary by CodeRabbit
New Features
Tests