testing_guide.md

Medical Assistant Testing Guide

Overview

The Medical Assistant application has a comprehensive test suite with over 80% code coverage on critical modules. This guide explains the testing infrastructure, how to run tests, and how to add new tests.

Test Coverage Status

As of the latest implementation:

• Overall Coverage: 80.68% (core modules)
• Total Tests: 402 tests (327 unit + 25 PyQt5 UI + 50 tkinter UI)
• Key Module Coverage:
  • database.py: 96.17%
  • ai_processor.py: 80.13%
  • audio.py: 81.12%
  • recording_manager.py: 90.76%
  • security.py: 71.14%
  • validation.py: 83.25%
  • STT Providers: 85-98% average
  • UI Components: 75 tests (25 PyQt5 demo + 50 tkinter/ttkbootstrap)

Test Structure

tests/
├── conftest.py              # Shared fixtures and configuration
├── test_setup.py            # Basic setup verification
├── run_ui_tests.py          # UI test runner script
├── unit/                    # Unit tests
│   ├── test_ai_processor.py
│   ├── test_audio.py
│   ├── test_audio_extended.py
│   ├── test_database.py
│   ├── test_recording_manager.py
│   ├── test_security.py
│   ├── test_validation.py
│   ├── test_ui_basic.py     # Basic PyQt5 UI tests (demo)
│   ├── test_ui_medical_assistant.py  # Medical Assistant PyQt5 tests (demo)
│   ├── tkinter_test_utils.py # Tkinter testing utilities
│   ├── test_tkinter_ui_basic.py # Basic tkinter/ttkbootstrap tests
│   ├── test_tkinter_ui_medical_assistant.py # Medical Assistant tkinter tests
│   ├── test_tkinter_workflow_tabs.py # Workflow tab tkinter tests
│   ├── test_tkinter_chat_and_editors.py # Chat and editor tkinter tests
│   └── test_stt_providers/  # STT provider tests
│       ├── test_base.py
│       ├── test_deepgram.py
│       ├── test_elevenlabs.py
│       ├── test_groq.py
│       ├── test_whisper.py
│       └── test_all_providers.py
└── integration/             # Integration tests
    └── test_recording_pipeline.py

Running Tests

Prerequisites

Install development dependencies:

pip install -r requirements-dev.txt

Running All Tests

# Run all tests
python -m pytest

# Run with coverage report
python -m pytest --cov=. --cov-report=term-missing

# Run with HTML coverage report
python -m pytest --cov=. --cov-report=html

Running Specific Tests

# Run unit tests only
python -m pytest tests/unit/

# Run integration tests only
python -m pytest tests/integration/

# Run a specific test file
python -m pytest tests/unit/test_database.py

# Run a specific test method
python -m pytest tests/unit/test_database.py::TestDatabase::test_add_recording_minimal

# Run tests matching a pattern
python -m pytest -k "test_transcribe"

Using the Test Runner Script

A convenience script is provided for common testing scenarios:

# Run all tests with coverage
python run_tests.py --cov

# Run only unit tests
python run_tests.py --unit

# Run tests with HTML coverage report
python run_tests.py --cov-html

# Run tests in parallel
python run_tests.py -n auto

# Run previously failed tests
python run_tests.py --failed

Test Markers

Tests are marked with categories for selective execution:

• @pytest.mark.slow - Long-running tests
• @pytest.mark.integration - Integration tests
• @pytest.mark.ui - UI tests requiring Qt

Run tests excluding certain markers:

pytest -m "not slow"
pytest -m "not ui"

Running UI Tests

The Medical Assistant uses tkinter/ttkbootstrap for its UI. We have both tkinter tests (actual) and PyQt5 tests (demonstration):

Tkinter/TTKBootstrap Tests (Actual UI Framework)

# Run tkinter UI tests
python tests/run_tkinter_ui_tests.py

# Run with coverage
python tests/run_tkinter_ui_tests.py --coverage

# Run headless (Linux)
python tests/run_tkinter_ui_tests.py --headless

# Run specific tkinter tests
pytest tests/unit/test_tkinter_*.py -v

PyQt5 Tests (Demonstration Only)

# Install PyQt5 test dependencies
pip install PyQt5 pytest-qt

# Run PyQt5 demo tests
pytest tests/unit/test_ui_*.py -v

# Run headless (Linux)
xvfb-run -a pytest tests/unit/test_ui_*.py

# Use the PyQt5 test runner
python tests/run_ui_tests.py

Note: The tkinter tests are the actual UI tests for the application. The PyQt5 tests are kept as examples of UI testing patterns.

Writing Tests

Test Structure Example

import pytest
from unittest.mock import Mock, patch

class TestMyModule:
    """Test cases for my_module."""
    
    @pytest.fixture
    def my_fixture(self):
        """Create a test fixture."""
        return MyClass()
    
    def test_basic_functionality(self, my_fixture):
        """Test basic functionality."""
        result = my_fixture.do_something()
        assert result == expected_value
    
    @patch('my_module.external_api')
    def test_with_mock(self, mock_api, my_fixture):
        """Test with mocked external dependency."""
        mock_api.return_value = {"status": "success"}
        result = my_fixture.call_api()
        assert result["status"] == "success"
        mock_api.assert_called_once()

Common Test Patterns

1. Testing Error Handling:

def test_error_handling(self):
    with pytest.raises(ValueError, match="Invalid input"):
        function_that_should_raise("bad input")

2. Testing File Operations:

def test_file_operations(self, tmp_path):
    test_file = tmp_path / "test.txt"
    test_file.write_text("content")
    result = read_file(str(test_file))
    assert result == "content"

3. Testing Async Code:

@pytest.mark.asyncio
async def test_async_function(self):
    result = await async_function()
    assert result == expected

Coverage Guidelines

Target Coverage

• Critical business logic: 90%+
• API integrations: 80%+
• Utility functions: 70%+
• UI code: 50%+ (where feasible)

Checking Coverage

# Generate coverage report
pytest --cov=module_name --cov-report=term-missing

# View HTML coverage report
pytest --cov=. --cov-report=html
open htmlcov/index.html

Improving Coverage

1. Identify uncovered lines:

   • Look for red lines in HTML coverage report
   • Check "Missing" column in terminal report

2. Write tests for:

   • Error paths
   • Edge cases
   • Different input types
   • Configuration variations

CI/CD Integration

Tests run automatically on:

• Push to main/development branches
• Pull requests
• Manual workflow dispatch

The CI pipeline:

1. Runs on multiple OS (Ubuntu, Windows, macOS)
2. Tests against Python 3.10, 3.11, 3.12
3. Runs linting checks
4. Generates coverage reports
5. Performs security scans

Pre-commit Hooks

Enable pre-commit hooks for automatic code quality checks:

pre-commit install

This will run:

• Black (code formatting)
• isort (import sorting)
• Flake8 (linting)
• MyPy (type checking)
• Basic pytest smoke tests

Debugging Failed Tests

Verbose Output

pytest -vv tests/unit/test_failing.py

Show print statements

pytest -s tests/unit/test_failing.py

Drop into debugger on failure

pytest --pdb tests/unit/test_failing.py

Run specific test with debugging

python -m pdb -m pytest tests/unit/test_specific.py::test_method

Mock Best Practices

1. Mock at boundaries: Mock external services, not internal methods
2. Use specific assertions: Verify mock calls with exact arguments
3. Reset mocks: Use mock.reset_mock() between test cases
4. Patch locations: Patch where the object is used, not where it's defined

Example:

# Good
@patch('module_using_api.requests.post')
def test_api_call(self, mock_post):
    mock_post.return_value.json.return_value = {"status": "ok"}
    
# Bad - patching at definition
@patch('requests.post')

Troubleshooting

Common Issues

1. Import Errors:

   • Ensure test file is in correct location
   • Check PYTHONPATH includes project root
   • Verify __init__.py files exist

2. Fixture Not Found:

   • Check fixture is in conftest.py or same file
   • Verify fixture scope matches usage

3. Mock Not Working:

   • Verify patch target path
   • Check import style (from X import Y vs import X)
   • Use patch.object() for instance methods

4. Async Test Issues:

   • Add @pytest.mark.asyncio decorator
   • Ensure pytest-asyncio is installed

Contributing Tests

When adding new features:

1. Write tests first (TDD approach)
2. Ensure tests cover happy path and error cases
3. Mock external dependencies
4. Add appropriate test markers
5. Verify coverage meets targets
6. Update this documentation if needed

Resources

• Pytest Documentation
• Python Mock Documentation
• Coverage.py Documentation
• Testing Best Practices