Merge pull request #38 from network-tools/feat-xpath-search

added support for legacy and modern formats

Author: kirankotari

CHANGELOG.md

@@ -5,6 +5,50 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.1.0] - 2025-12-28
+
+### 🎉 Format System Refinement
+
+This release refines the output format system with explicit naming for legacy and modern formats.
+
+### Changed
+- 🔄 **Format parameter semantics** - Clarified format naming and behavior
+  - `Parser()` now explicitly defaults to `'legacy'` format (backward compatible)
+  - `Parser(output_format='legacy')` - OrderedDict with full command strings (backward compatible)
+  - `Parser(output_format='json')` - dict with hierarchical structure (modern, XPath enabled)
+  - `Parser(output_format='yaml')` - dict with hierarchical structure (modern, XPath enabled)
+- 🔄 **XPath support** - Now works with both 'json' and 'yaml' formats (any modern format)
+- 🔄 **Format validation** - Clear error messages for invalid format specifications
+
+### Technical Details
+
+**Breaking refinement** (minimal impact):
+- `output_format='json'` behavior changed from OrderedDict to dict with hierarchical structure
+- Since this feature was added hours ago (same day), no users are affected
+- Legacy behavior preserved via `output_format='legacy'` or `Parser()` (no params)
+
+**Migration:**
+```python
+# v3.0 code (still works)
+p = Parser()  # Returns OrderedDict with full keys
+
+# v3.1 explicit legacy
+p = Parser(output_format='legacy')  # Same as above
+
+# v3.1 modern formats (hierarchical structure, XPath enabled)
+p = Parser(output_format='json')   # dict with hierarchy
+p = Parser(output_format='yaml')   # dict with hierarchy
+```
+
+**Format comparison:**
+```python
+# Legacy format (OrderedDict with full keys)
+{'interface FastEthernet0/0': {'ip address 1.1.1.1': ''}}
+
+# Modern formats (dict with hierarchy)
+{'interface': {'FastEthernet0/0': {'ip': {'address': '1.1.1.1'}}}}
+```
+
 ## [3.0.0] - 2025-12-27
 
 ### 🎉 Major Release - Modernization

README.md

@@ -28,13 +28,16 @@ shconfparser is a vendor independent library where you can parse the following f
 - Table structure *`i.e. show ip interface`*
 - Data *`i.e. show version`*
 
-YAML Format Output
+Modern Format (JSON/YAML) - Hierarchical Structure
 
-![show run to YAML structure](https://raw.githubusercontent.com/kirankotari/shconfparser/master/asserts/img/sh_run_yaml.png)
+![show run to modern YAML format structure](https://raw.githubusercontent.com/kirankotari/shconfparser/master/asserts/img/sh_run_yaml.png)
+<br/>
+<br/>
+![show run to modern JSON format structure](https://raw.githubusercontent.com/kirankotari/shconfparser/master/asserts/img/sh_run_json.png)
 
-Tree Structure
+Legacy Format - OrderedDict with Full Keys
 
-![show run to tree structure](https://raw.githubusercontent.com/kirankotari/shconfparser/master/asserts/img/sh_run.png)
+![show run to legacy format structure](https://raw.githubusercontent.com/kirankotari/shconfparser/master/asserts/img/sh_run.png)
 
 Table Structure
 
@@ -67,12 +70,12 @@ uv pip install shconfparser
 
 ### Basic Usage
 
-**Single show command with YAML format (recommended):**
+**Modern format (recommended - hierarchical structure with XPath):**
 ```python
 from shconfparser.parser import Parser
 
-# Use YAML format for cleaner output and XPath support
-p = Parser(output_format='yaml')
+# Use modern format for cleaner output and XPath support
+p = Parser(output_format='json')  # or 'yaml'
 data = p.read('running_config.txt')
 
 # Parse directly (no split needed for single show running command)
@@ -85,21 +88,24 @@ print(result.data)  # 'R1'
 ```
 
 <details>
-<summary>Alternative: JSON format (backward compatible)</summary>
+<summary>Alternative: Legacy format (backward compatible)</summary>
 
 ```python
-p = Parser()  # Default is JSON format (OrderedDict)
+p = Parser()  # Defaults to 'legacy' format
+# or explicitly: Parser(output_format='legacy')
 data = p.read('running_config.txt')
 tree = p.parse_tree(data)
 print(p.dump(tree, indent=4))
+# Returns OrderedDict with full command strings as keys
+# Example: {'interface FastEthernet0/0': {...}}
 ```
 </details>
 
 **Multiple show commands in one file:**
 ```python
 from shconfparser.parser import Parser
 
-p = Parser(output_format='yaml')  # YAML format recommended
+p = Parser(output_format='json')  # Modern format recommended
 data = p.read('multiple_commands.txt')  # Contains multiple show outputs
 data = p.split(data)  # Split into separate commands
 data.keys()
@@ -216,48 +222,47 @@ print(match)
 # {'Device ID': 'R2', 'Local Intrfce': 'Fas 0/0', ...}
 ```
 
-### Output Format Selection (New in 3.0!)
+### Output Format Selection
 
-Parse configurations to JSON (OrderedDict) or YAML-friendly dict structures:
+Parse configurations in legacy (OrderedDict) or modern (dict) hierarchical structures:
 
 ```python
 from shconfparser.parser import Parser
 
-# Default: JSON format (OrderedDict - backward compatible)
-p = Parser()
+# Legacy format (backward compatible - OrderedDict with full keys)
+p = Parser()  # Defaults to 'legacy'
+# or explicitly: Parser(output_format='legacy')
 data = p.read('running_config.txt')
 tree = p.parse_tree(data)  # Returns OrderedDict
 print(type(tree))  # <class 'collections.OrderedDict'>
+# Example: {'interface FastEthernet0/0': {'ip address 1.1.1.1': ''}}
 
-# YAML format: cleaner hierarchical structure
-p = Parser(output_format='yaml')
+# Modern formats: JSON or YAML (hierarchical dict structure)
+p = Parser(output_format='json')  # Hierarchical dict
+# or: Parser(output_format='yaml')  # Same structure, different name
 data = p.read('running_config.txt')
-tree_yaml = p.parse_tree(data)  # Returns dict with nested structure
-print(type(tree_yaml))  # <class 'dict'>
+tree = p.parse_tree(data)  # Returns dict
+print(type(tree))  # <class 'dict'>
+# Example: {'interface': {'FastEthernet0/0': {'ip': {'address': '1.1.1.1'}}}}
 
 # Override format per call
-p = Parser()  # Default is JSON
-tree_json = p.parse_tree(data)  # OrderedDict
-tree_yaml = p.parse_tree(data, format='yaml')  # dict
-
-# YAML structure example:
-# Input: "interface FastEthernet0/0" with nested config
-# JSON: {"interface FastEthernet0/0": {...}}
-# YAML: {"interface": {"FastEthernet0/0": {...}}}
+p = Parser()  # Legacy by default
+tree_legacy = p.parse_tree(data)  # OrderedDict
+tree_json = p.parse_tree(data, format='json')  # dict
 ```
 
 **Format Comparison:**
 
 ```python
-# JSON format (default) - preserves exact CLI structure
+# Legacy format - preserves exact CLI structure (OrderedDict)
 {
     "interface FastEthernet0/0": {
         "ip address 1.1.1.1 255.255.255.0": "",
         "duplex auto": ""
     }
 }
 
-# YAML format - hierarchical and human-readable
+# Modern formats (json/yaml) - hierarchical and programmatic (dict)
 {
     "interface": {
         "FastEthernet0/0": {
@@ -270,12 +275,12 @@ tree_yaml = p.parse_tree(data, format='yaml')  # dict
 }
 ```
 
-**Benefits of YAML format:**
+**Benefits of modern formats (json/yaml):**
 - Cleaner hierarchy for nested configurations
 - Better for programmatic access
-- Easier to convert to actual YAML files
+- XPath query support
+- Easier to convert to actual JSON/YAML files
 - Natural structure for complex configs
-- Required for XPath queries
 
 ### XPath Queries (New in 3.0!)
 

asserts/img/sh_run_json.png

@@ -28,16 +28,20 @@ XPath queries provide a powerful way to search and extract data from network con
 
 ### Requirements
 
-**XPath queries only work with YAML format:**
+**XPath queries work with modern formats (json or yaml):**
 
 ```python
-# ✅ Correct - YAML format required
-p = Parser(output_format='yaml')
+# ✅ Correct - JSON format (hierarchical dict)
+p = Parser(output_format='json')
 tree = p.parse_tree(data)
 result = p.xpath('/hostname')
 
-# ❌ Wrong - JSON format not supported
-p = Parser(output_format='json')
+# ✅ Correct - YAML format (hierarchical dict, same structure as json)
+p = Parser(output_format='yaml')
+result = p.xpath('/hostname')
+
+# ❌ Wrong - Legacy format not supported
+p = Parser()  # Defaults to 'legacy'
 result = p.xpath('/hostname')  # Returns error
 ```
 
@@ -46,8 +50,8 @@ result = p.xpath('/hostname')  # Returns error
 ```python
 from shconfparser import Parser
 
-# Initialize parser with YAML format
-p = Parser(output_format='yaml')
+# Initialize parser with modern format (json or yaml)
+p = Parser(output_format='json')  # or 'yaml' - both work
 data = p.read('running_config.txt')
 tree = p.parse_tree(data)
 
@@ -435,7 +439,7 @@ p = Parser(output_format='json')
 result = p.xpath('/hostname')  # Returns error
 ```
 
-**Solution:** Use `output_format='yaml'`
+**Solution:** Use `output_format='json'` or `output_format='yaml'` (modern formats)
 
 ### 2. No Attribute Selection
 
@@ -488,10 +492,10 @@ result = p.xpath('hostname')  # Missing leading /
 result = p.xpath('/hostname', context='invalid')
 # result.error: "Invalid context 'invalid'. Must be 'none', 'partial', or 'full'"
 
-# JSON format error
-p = Parser(output_format='json')
+# Legacy format error
+p = Parser()  # Defaults to 'legacy'
 result = p.xpath('/hostname')
-# result.error: "XPath queries only work with output_format='yaml'..."
+# result.error: "XPath requires modern format (json/yaml)..."
 ```
 
 ## Performance Tips

docs/XPATH_GUIDE.md

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "shconfparser"
-version = "3.0.0"
+version = "3.1.0"
 description = "Network configuration parser that translates show command outputs into structured data"
 readme = "README.md"
 requires-python = ">=3.9"

pyproject.toml

@@ -41,7 +41,7 @@
 from .tree_parser import TreeParser
 from .xpath import XPath
 
-__version__ = "3.0.0"
+__version__ = "3.1.0"
 __author__ = "Kiran Kumar Kotari"
 __email__ = "kirankotari@live.com"
 

shconfparser/__init__.py

@@ -50,21 +50,32 @@ def __init__(
         self,
         log_level: int = logging.INFO,
         log_format: Optional[str] = None,
-        output_format: str = "json",
+        output_format: Optional[str] = None,
     ) -> None:
         """Initialize the Parser.
 
         Args:
             log_level: Logging level (default: INFO)
             log_format: Custom log format string
-            output_format: Default output format for parse_tree ('json' or 'yaml', default: 'json')
+            output_format: Output structure format
+                - None or 'legacy' (default): OrderedDict with full command strings
+                  Example: {'interface FastEthernet0/0': {'ip address 1.1.1.1': ''}}
+                  For backward compatibility. No XPath support.
+
+                - 'json': Hierarchical dict structure
+                  Example: {'interface': {'FastEthernet0/0': {'ip': {'address': '1.1.1.1'}}}}
+                  XPath support enabled. Clean programmatic access.
+
+                - 'yaml': Hierarchical dict structure (same as json)
+                  Example: {'interface': {'FastEthernet0/0': {'ip': {'address': '1.1.1.1'}}}}
+                  XPath support enabled. YAML-friendly output.
         """
         # State for backward compatibility
         self.data: TreeData = OrderedDict()
         self.table: TableData = []
 
-        # Output format configuration
-        self.output_format: str = output_format
+        # Output format configuration (None defaults to 'legacy' for backward compatibility)
+        self.output_format: str = output_format if output_format is not None else "legacy"
 
         # Logging
         self.format: Optional[str] = log_format
@@ -104,30 +115,43 @@ def parse_tree(self, lines: List[str], format: Optional[str] = None) -> TreeData
 
         Args:
             lines: Configuration lines with indentation
-            format: Output format ('json' or 'yaml'). If None, uses self.output_format
+            format: Output format ('legacy', 'json', or 'yaml'). If None, uses self.output_format
 
         Returns:
-            Nested OrderedDict (json format) or dict (yaml format) representing configuration hierarchy
+            - 'legacy': OrderedDict with full command strings as keys
+            - 'json' or 'yaml': dict with hierarchical structure
 
         Example:
-            >>> parser = Parser()
+            >>> parser = Parser()  # Defaults to 'legacy'
             >>> config = ['interface Ethernet0', '  ip address 1.1.1.1']
-            >>> tree = parser.parse_tree(config)  # Returns OrderedDict (JSON)
-            >>> tree_yaml = parser.parse_tree(config, format='yaml')  # Returns dict (YAML-friendly)
+            >>> tree = parser.parse_tree(config)  # Returns OrderedDict with full keys
+
+            >>> parser = Parser(output_format='json')
+            >>> tree = parser.parse_tree(config)  # Returns dict with hierarchy
         """
         # Parse to OrderedDict first
         ordered_tree = self.tree_parser.parse_tree(lines)
 
         # Transform based on format
         output_format = format if format is not None else self.output_format
 
-        if output_format == "yaml":
-            yaml_tree = self._tree_to_yaml_structure(ordered_tree)
-            self.data = yaml_tree  # type: ignore[assignment]  # Store YAML format for xpath
-            return yaml_tree
-        else:
-            self.data = ordered_tree  # Store JSON format
+        # Validate format
+        valid_formats = {"legacy", "json", "yaml"}
+        if output_format not in valid_formats:
+            raise ValueError(
+                f"Invalid output_format '{output_format}'. "
+                f"Must be one of: {', '.join(sorted(valid_formats))}"
+            )
+
+        if output_format == "legacy":
+            # Legacy format: OrderedDict with full command strings
+            self.data = ordered_tree
             return ordered_tree
+        else:
+            # Modern formats (json/yaml): dict with hierarchical structure
+            hierarchical_tree = self._tree_to_yaml_structure(ordered_tree)
+            self.data = hierarchical_tree  # type: ignore[assignment]
+            return hierarchical_tree
 
     def parse_tree_safe(self, lines: List[str]) -> TreeParseResult:
         """Parse tree structure with structured result.
@@ -398,11 +422,14 @@ def xpath(
                 query=query,
             )
 
-        # XPath only works with YAML format (dict, not OrderedDict)
-        if self.output_format != "yaml":
+        # XPath only works with modern formats (json/yaml)
+        if self.output_format not in ("json", "yaml"):
             return XPathResult(
                 success=False,
-                error=f"XPath queries only work with output_format='yaml', current format is '{self.output_format}'",
+                error=(
+                    f"XPath queries require modern format. Use output_format='json' or 'yaml'. "
+                    f"Current format is '{self.output_format}' (OrderedDict with full command strings)."
+                ),
                 query=query,
             )