Merge remote-tracking branch 'origin/main' into fix/truncation_strategy

Author: philipph-askui

README.md

@@ -129,11 +129,37 @@ Ready to build your first agent? Check out our documentation:
 
 ## Quick Install
 
+```bash
+pip install askui
+```
+
+**Requires Python >=3.10*
+
+The default install includes core dependencies (Agent OS client, HTTP, common model APIs, Android tooling, etc.). Add **optional extras** only when you need the features below.
+
 ```bash
 pip install askui[all]
 ```
 
-**Requires Python >=3.10, <3.14**
+`all` pulls in every optional extra in one command (larger install). Prefer picking individual extras when you know what you need. Or start using AskUI with a minimal install and add extras as you need them.
+
+### Optional install extras
+
+| Extra | When to use it |
+| --- | --- |
+| *(none)* | Everyday automation. Smallest footprint. |
+| `all` | You want every optional integration below without choosing extras (CI images, demos, or “install everything once”). |
+| `office-document` | Reading or converting **Office files** (Excel `.xls`/`.xlsx`, Word `.docx`) via MarkItDown—e.g. extracting content from spreadsheets or documents in workflows. To be used in `get()` methods. |
+| `bedrock` | Running **Anthropic Claude through AWS Bedrock** (`anthropic[bedrock]`). Use when your org routes Claude via Bedrock, not the direct Anthropic API. |
+| `vertex` | Running **Anthropic Claude on Google Vertex AI** plus Vertex client libraries. Use when models are hosted on Vertex, not only the public Anthropic API. |
+| `otel` | **OpenTelemetry** export and instrumentation you enable in code: OTLP over HTTP, plus optional instrumentation for `httpx` and SQLAlchemy. Use for production tracing/metrics pipelines—not required for local scripts. |
+| `web` | **Browser automation with Playwright** (e.g. web-focused agents and Playwright-backed flows). Core desktop control still uses Agent OS; this extra is for the Playwright stack. |
+
+Combine extras as needed, for example:
+
+```bash
+pip install askui[bedrock,otel]
+```
 
 You'll also need to install AskUI Agent OS for device control. See [Setup Guide](docs/01_setup.md) for detailed instructions.
 

docs/01_setup.md

@@ -4,7 +4,7 @@ This guide will get AskUI Vision Agent installed and configured on your system.
 
 ## Python Package Installation
 
-AskUI Vision Agent requires Python >=3.10, and <3.14.
+AskUI Vision Agent requires Python >=3.10 and <3.14.
 
 For installing the python-sdk please run
 

docs/09_observability_telemetry_tracing.md

@@ -92,7 +92,6 @@ Set the following environment variables to configure tracing. All variables use
 | `ASKUI__OTEL_ENDPOINT` | OTLP HTTP endpoint URL | — |
 | `ASKUI__OTEL_SERVICE_NAME` | Service name reported in traces | `askui-python-sdk` |
 | `ASKUI__OTEL_SERVICE_VERSION` | Service version reported in traces | Current package version |
-| `ASKUI__OTEL_CLUSTER_NAME` | Cluster name reported in traces | `askui-dev` |
 
 #### Linux & MacOS
 ```bash
@@ -132,7 +131,6 @@ from askui.telemetry.otel import OtelSettings
 
 settings = OtelSettings(
     enabled=True,
-    service_name="my-custom-service",
-    cluster_name="production",
+    service_name="my-custom-service"
 )
 ```

docs/10_extracting_data.md

@@ -43,6 +43,12 @@ The AskUI Python SDK supports the use of various file formats.
 - Word Files (.docx, .doc)
 - CSV Files (.csv)
 
+**Office documents (Excel and Word)** are loaded as `OfficeDocumentSource` and converted to Markdown with MarkItDown. That stack is optional: the base `askui` install does not include it. Install AskUI with the **office-document** extra when you use Excel or Word as `get()` sources:
+
+```bash
+pip install "askui[office-document]"
+```
+
 **Model Compatibility Matrix**
 
 | File Format         | AskUI Gemini | Anthropic Claude | Google Gemini

mypy.ini

@@ -24,9 +24,6 @@ namespace_packages = true
 [mypy-jsonref.*]
 ignore_missing_imports = true
 
-[mypy-bson.*]
-ignore_missing_imports = true
-
 [mypy-alembic.*]
 ignore_missing_imports = true
 

pdm.lock

@@ -28,17 +28,16 @@ dependencies = [
     "protobuf>=6.31.1",
     "google-genai>=1.20.0",
     "filetype>=1.2.0",
-    "markitdown[xls,xlsx,docx]>=0.1.2",
     "asyncer==0.0.8",
-    "bson>=0.5.10",
     "aiofiles>=24.1.0",
     "anyio==4.10.0",  # We need to pin this version otherwise listing mcp tools using fastmcp within runner fails
     "sqlalchemy[mypy]>=2.0.44",
     "apscheduler==4.0.0a6",
     "opentelemetry-api>=1.38.0",
-    "imagehash>=4.3.0",
+    "imagehash>=4.3.0", # requires python <3.14 because of dependency on scipy
+    "pure-python-adb>=0.3.0.dev0"
 ]
-requires-python = ">=3.10,<3.14"
+requires-python = ">=3.10, <3.14"
 readme = "README.md"
 license = {text = "MIT"}
 dynamic = ["version"]
@@ -217,9 +216,9 @@ known-first-party = ["askui"]
 known-third-party = ["pytest", "mypy"]
 
 [project.optional-dependencies]
-all = ["askui[android,bedrock,otel,vertex,web]"]
-android = [
-    "pure-python-adb>=0.3.0.dev0"
+all = ["askui[office-document,bedrock,otel,vertex,web]"]
+office-document = [
+    "markitdown[xls,xlsx,docx]>=0.1.2"
 ]
 bedrock = [
     "anthropic[bedrock]>=0.72.0"
@@ -228,7 +227,6 @@ otel = [
     "opentelemetry-api>=1.38.0",
     "opentelemetry-exporter-otlp-proto-http>=1.38.0",
     "opentelemetry-instrumentation-httpx>=0.59b0",
-    "opentelemetry-instrumentation-sqlalchemy>=0.59b0",
 ]
 vertex = [
     "google-cloud-aiplatform>=1.122.0",

pyproject.toml

@@ -1,6 +1,6 @@
 """AskUI Python SDK"""
 
-__version__ = "0.27.0"
+__version__ = "0.28.0"
 
 import logging
 import os

src/askui/__init__.py

@@ -293,12 +293,7 @@ def _execute_step(self) -> bool:
         self._switch_speaker_if_needed()
 
         # 2. Get next message(s) from speaker and add to history
-        logger.debug("Executing step with speaker: %s", self.current_speaker.name)
-        result: SpeakerResult = self.current_speaker.handle_step(
-            self, self.cache_manager
-        )
-        for message in result.messages_to_add:
-            self._add_message(message)
+        result = self._get_next_message()
 
         # 3. Execute tool calls if applicable
         continue_loop = False
@@ -320,6 +315,16 @@ def _execute_step(self) -> bool:
 
         return continue_loop
 
+    @tracer.start_as_current_span("_get_next_message")
+    def _get_next_message(self) -> SpeakerResult:
+        logger.debug("Executing step with speaker: %s", self.current_speaker.name)
+        result: SpeakerResult = self.current_speaker.handle_step(
+            self, self.cache_manager
+        )
+        for message in result.messages_to_add:
+            self._add_message(message)
+        return result
+
     @tracer.start_as_current_span("_execute_tools_if_present")
     def _execute_tools_if_present(self, message: MessageParam) -> MessageParam | None:
         """Execute tools if the message contains tool use blocks.

src/askui/models/shared/conversation.py

@@ -37,26 +37,34 @@ def normalize_to_pil_images(
     return [image]
 
 
-def truncate_content(
-    content: Any,
-    max_string_length: int = 100000,
-) -> Any:
-    """Filter out long strings (i.e. the base64 image data) to keep reports readable."""
-    if isinstance(content, str):
-        if len(content) > max_string_length:
-            return f"[truncated: {len(content)} characters]"
-        return content
-
+def truncate_base64_images(content: Any) -> Any:
+    """Replace base64 image data with a placeholder to keep reports readable.
+
+    Walks the message content recursively and replaces the ``data`` field of
+    any base64 image source block (matching the schema of
+    ``Base64ImageSourceParam``, i.e. ``{"type": "base64", "data": "...",
+    "media_type": "image/..."}``) with a placeholder string. All other
+    content (including regular long strings like prompts or tool outputs)
+    is left untouched.
+    """
     if isinstance(content, dict):
+        content_dict: dict[Any, Any] = content
+        media_type = content_dict.get("media_type")
+        if (
+            isinstance(media_type, str)
+            and media_type.startswith("image/")
+            and content_dict.get("type") == "base64"
+            and "data" in content_dict
+        ):
+            return {**content_dict, "data": "[Base64 image data truncated]"}
         return {
-            key: truncate_content(value, max_string_length)
-            for key, value in content.items()
+            key: truncate_base64_images(value) for key, value in content_dict.items()
         }
 
     if isinstance(content, list):
-        return [truncate_content(item, max_string_length) for item in content]
+        content_list: list[Any] = content
+        return [truncate_base64_images(item) for item in content_list]
 
-    # For other types (int, float, bool, None), return as-is
     return content
 
 
@@ -257,7 +265,7 @@ def add_message(
             self._start_time = datetime.now(tz=timezone.utc)
 
         _images = normalize_to_pil_images(image)
-        _content = truncate_content(content)
+        _content = truncate_base64_images(content)
 
         message = {
             "timestamp": datetime.now(tz=timezone.utc),