Relocating TSFM-Specifc CSV Files to CouchDB

[jellyfishing2346]

Description

This PR addresses issue #297 by relocating TSFM-specific CSV files from local storage to CouchDB.Previously, TSFM-related utterances used local CSV files, which has been updated to load data
from the CouchDB tsfm collection instead. This change standardizes data access patterns across
the benchmark suite and ensures all data is managed through the CouchDB infrastructure.

Fix Details

CouchDB Infrastructure:

• Added tsfm collection configuration to collections.json with CSV format and Timestamp as
primary key
• Configured proper ID prefix (tsfm) for document identification
• Set up collection with no additional indexes (optimized for time-series data access)

Data Files:

• Added 3 TSFM CSV files to src/couchdb/scenarios_data/shared/tsfm/:
• chiller9_annotated_small_test.csv (192 records for forecasting inference)
• chiller9_finetuning_small.csv (192 records for model fine-tuning)
• chiller9_tsad.csv (192 records for time-series anomaly detection)
• All files contain timestamped sensor data for Chiller 9 Condenser Water Flow
• Data spans from 2020-04-27 to 2020-04-28 with 15-minute intervals

Configuration Updates:

• Updated all scenario manifests (default, scenario_1, scenario_2) to include tsfm data sources
• Updated .allowed_datafiles to include the new tsfm CSV files for security checks
• Ensured proper path resolution through the shared data directory structure

Documentation Updates:

• case_study_industrial_asset_management.md: Updated forecasting and anomaly detection examples
to reference CouchDB collection 'tsfm'
• ground_truth_design_guideline.md: Updated ground truth patterns to use CouchDB paths and
corrected agent actions (changed from jsonreader to csvreader, updated file paths)
• data.md: Added tsfm directory to the shared data directory structure documentation

Impact on Benchmarking

• No change to baselines: This fix only improves stability/performance.
• Baseline change: This fix corrects a scoring error. (Please provide "Before vs. After"
  results).

This change is a data infrastructure improvement that does not affect benchmark scoring or
evaluation metrics. It standardizes how TSFM-related utterances access data, making the system
more maintainable and consistent with other data sources.

Related Issues

• Fixes: #297

Verification Steps

1. Run the following command: uv run pytest tests/integration
2. Describe any manual verification performed:
   • CSV Parsing Validation: Tested parsing of all 3 CSV files using the loader's CSV parser
   with tsfm collection config. Successfully parsed 192 documents from each file with proper
   field mapping.
   • Manifest Loading: Tested collection loading from the default manifest. Successfully loaded
   576 total documents (192 from each of the 3 CSV files) through the manifest-based loader.
   • Document Normalization: Verified document normalization properly adds dataset: 'tsfm' field
   and generates correct _id fields using the Timestamp primary key (e.g., tsfm:2020-04-27T00:0
   0:00).
   • Path Resolution: Confirmed that relative paths in manifests (shared/tsfm/*.csv) properly
   resolve against the scenario directory and shared data structure.
   • Schema Validation: Verified that CSV files have correct structure with Timestamp and Chille
   r_9_Condenser_Water_Flow columns matching the expected schema.

Checklist

• I have added tests that prove my fix is effective.
• My code follows the project's Ruff formatting and linting rules.
• I have signed off my commits (DCO).

[DhavalRepo18]

We are considering the dataset as a part of IoT (as it's just data), so the core file should go inside IoT.

Please also check other pending PR for the same problem.

[jellyfishing2346]

I've moved the TSFM dataset files from shared/tsfm/ to shared/iot/ as requested, since the data
should be considered part of IoT (as it's sensor/telemetry data).

Changes made:

• Moved chiller9_annotated_small_test.csv, chiller9_finetuning_small.csv, and chiller9_tsad.csv
from shared/tsfm/ to shared/iot/
• Removed the tsfm collection from collections.json
• Updated all manifest.json files to reference the iot collection instead of tsfm
• Updated .allowed_datafiles with the new IoT paths
• Updated documentation references from tsfm to iot collection throughout the codebase
• Updated data.md directory structure to reflect that IoT now contains both JSON and CSV files

Regarding other pending PRs: I checked PR #343 and found it has the same issue - the reviewer there
mentioned they should only take JSON files and delete CSV files. That PR may need similar cleanup toalign with this IoT consolidation approach.