feature Yaml + XPath search

Author: kirankotari

CHANGELOG.md

@@ -22,6 +22,10 @@ This is a major release focused on modernizing the project infrastructure and to
 - ✨ **Type hints** - Improved IDE support and type safety
 - ✨ **py.typed** marker - PEP 561 compliance
 - ✨ **Modern CI/CD** - GitHub Actions with uv
+- ✨ **YAML output format** - Cleaner hierarchical structure with two-level split optimization
+- ✨ **XPath queries** - NSO-style queries for YAML-formatted configurations
+- ✨ **XPath context tracking** - Three context options (none/partial/full) to identify wildcard match sources
+- ✨ **XPath path tracking** - XPathResult.paths shows path components to each match
 - ✨ **MODERNIZATION_GUIDE.md** - Comprehensive migration guide
 - ✨ **BUSINESS_STANDARDS.md** - Enterprise compliance documentation
 - ✨ **PYTHON_COMPATIBILITY.md** - Version support documentation

README.md

@@ -28,6 +28,10 @@ shconfparser is a vendor independent library where you can parse the following f
 - Table structure *`i.e. show ip interface`*
 - Data *`i.e. show version`*
 
+YAML Format Output
+
+![show run to YAML structure](https://raw.githubusercontent.com/kirankotari/shconfparser/master/asserts/img/sh_run_yaml.png)
+
 Tree Structure
 
 ![show run to tree structure](https://raw.githubusercontent.com/kirankotari/shconfparser/master/asserts/img/sh_run.png)
@@ -43,7 +47,8 @@ Table Structure
 🔒 **Type Safe** - Full type hints and py.typed marker  
 🎯 **Vendor Independent** - Works with any network device configuration  
 📊 **Multiple Formats** - Parse trees, tables, and unstructured data  
-🔍 **XPath Queries** - Search configs with XPath-style syntax (NEW!)  
+📄 **Format Flexibility** - Output as JSON or YAML structures  
+🔍 **XPath Queries** - NSO-style queries with context tracking (NEW!)  
 🧪 **Well Tested** - 80%+ code coverage, tested on Python 3.8-3.13  
 
 ## Quick Start
@@ -62,27 +67,30 @@ uv pip install shconfparser
 
 ### Basic Usage
 
-**Single show command:**
+**Single show command with YAML format (recommended):**
 ```python
 from shconfparser.parser import Parser
 
-p = Parser()
+# Use YAML format for cleaner output and XPath support
+p = Parser(output_format='yaml')
 data = p.read('running_config.txt')
 
 # Parse directly (no split needed for single show running command)
 tree = p.parse_tree(data)
 print(p.dump(tree, indent=2))
+
+# Query with XPath
+result = p.xpath('/hostname')
+print(result.data)  # 'R1'
 ```
 
 <details>
-<summary>Alternative: Access internal properties</summary>
+<summary>Alternative: JSON format (backward compatible)</summary>
 
 ```python
-p = Parser()
-p.read('running_config.txt')
-
-# Access reader data directly
-tree = p.parse_tree(p.r.data)
+p = Parser()  # Default is JSON format (OrderedDict)
+data = p.read('running_config.txt')
+tree = p.parse_tree(data)
 print(p.dump(tree, indent=4))
 ```
 </details>
@@ -91,7 +99,7 @@ print(p.dump(tree, indent=4))
 ```python
 from shconfparser.parser import Parser
 
-p = Parser()
+p = Parser(output_format='yaml')  # YAML format recommended
 data = p.read('multiple_commands.txt')  # Contains multiple show outputs
 data = p.split(data)  # Split into separate commands
 data.keys()
@@ -208,60 +216,157 @@ print(match)
 # {'Device ID': 'R2', 'Local Intrfce': 'Fas 0/0', ...}
 ```
 
-### XPath-Style Queries (New in 3.0!)
+### Output Format Selection (New in 3.0!)
 
-Query parsed configuration trees using XPath-style syntax:
+Parse configurations to JSON (OrderedDict) or YAML-friendly dict structures:
 
 ```python
 from shconfparser.parser import Parser
 
+# Default: JSON format (OrderedDict - backward compatible)
 p = Parser()
-data = p.read('multiple_commands.txt')
-data = p.split(data)
+data = p.read('running_config.txt')
+tree = p.parse_tree(data)  # Returns OrderedDict
+print(type(tree))  # <class 'collections.OrderedDict'>
 
-# Parse the running config into a tree
-data['running'] = p.parse_tree(data['running'])
+# YAML format: cleaner hierarchical structure
+p = Parser(output_format='yaml')
+data = p.read('running_config.txt')
+tree_yaml = p.parse_tree(data)  # Returns dict with nested structure
+print(type(tree_yaml))  # <class 'dict'>
+
+# Override format per call
+p = Parser()  # Default is JSON
+tree_json = p.parse_tree(data)  # OrderedDict
+tree_yaml = p.parse_tree(data, format='yaml')  # dict
+
+# YAML structure example:
+# Input: "interface FastEthernet0/0" with nested config
+# JSON: {"interface FastEthernet0/0": {...}}
+# YAML: {"interface": {"FastEthernet0/0": {...}}}
+```
 
-# Now use XPath queries on the parsed tree
-# Find all IP addresses anywhere in config
-result = p.xpath('//ip/address', tree=data['running'])
-print(f"Found {result.count} IP addresses")
+**Format Comparison:**
 
-# Get specific interface configuration
-result = p.xpath('/interface/FastEthernet0-0/ip/address', tree=data['running'])
-if result:
-    print(f"IP: {result.data}")
+```python
+# JSON format (default) - preserves exact CLI structure
+{
+    "interface FastEthernet0/0": {
+        "ip address 1.1.1.1 255.255.255.0": "",
+        "duplex auto": ""
+    }
+}
+
+# YAML format - hierarchical and human-readable
+{
+    "interface": {
+        "FastEthernet0/0": {
+            "ip": {
+                "address": "1.1.1.1 255.255.255.0"
+            },
+            "duplex": "auto"
+        }
+    }
+}
+```
 
-# List all interface names using wildcard
-result = p.xpath('/interface/*', tree=data['running'])
-print(f"Interfaces: {', '.join(result.matches)}")
+**Benefits of YAML format:**
+- Cleaner hierarchy for nested configurations
+- Better for programmatic access
+- Easier to convert to actual YAML files
+- Natural structure for complex configs
+- Required for XPath queries
 
-# Filter interfaces by pattern
-result = p.xpath('/interface[FastEthernet*]/description', tree=data['running'])
-for desc in result.matches:
-    print(f"  - {desc}")
-```
+### XPath Queries (New in 3.0!)
 
-**Supported XPath Features:**
-- **Absolute paths**: `/interface/FastEthernet0-0/ip/address`
-- **Recursive search**: `//ip/address` (find anywhere)
-- **Wildcards**: `/interface/*/description` (all interfaces)
-- **Predicates**: `/interface[GigabitEthernet*]` (pattern matching)
-- **Root access**: `/` (entire tree)
+Query YAML-formatted configurations using NSO-style XPath with optional context tracking:
 
-**Or use stored tree directly:**
 ```python
-p = Parser()
+from shconfparser.parser import Parser
+
+# XPath requires YAML format
+p = Parser(output_format='yaml')
 data = p.read('running_config.txt')
-p.parse_tree(data)  # Stores in p.data
+tree = p.parse_tree(data)
+
+# Simple queries
+result = p.xpath('/hostname')
+print(result.data)  # 'R1'
 
-# Query uses stored tree automatically
-result = p.xpath('//ip/address')
-for addr in result.matches:
-    print(addr)
+# Wildcards - find all interface duplex settings
+result = p.xpath('/interface/*/duplex')
+print(result.matches)  # ['auto', 'auto']
+print(result.count)    # 2
+
+# Predicates with slashes (network interface names)
+result = p.xpath('/interface[FastEthernet0/0]/duplex')
+print(result.data)  # 'auto'
+
+# Recursive search - find anywhere in tree
+result = p.xpath('//duplex')
+print(result.matches)  # ['auto', 'auto']
+
+# Predicate wildcards
+result = p.xpath('/interface[FastEthernet*]/ip/address')
+print(result.data)  # '1.1.1.1 255.255.255.0'
+```
+
+**Context Options** - Solve the "which interface?" problem:
+
+```python
+# Problem: Can't identify source with wildcards
+result = p.xpath('/interface/*/duplex')
+print(result.matches)  # ['auto', 'auto'] - Which interface?
+
+# Solution 1: context='none' (default - just values)
+result = p.xpath('/interface/*/duplex', context='none')
+print(result.matches)  # ['auto', 'auto']
+
+# Solution 2: context='partial' (from wildcard match point)
+result = p.xpath('/interface/*/duplex', context='partial')
+print(result.matches)
+# [{'FastEthernet0/0': {'duplex': 'auto'}},
+#  {'FastEthernet0/1': {'duplex': 'auto'}}]
+
+# Solution 3: context='full' (complete tree hierarchy)
+result = p.xpath('/interface/*/duplex', context='full')
+print(result.matches)
+# [{'interface': {'FastEthernet0/0': {'duplex': 'auto'}}},
+#  {'interface': {'FastEthernet0/1': {'duplex': 'auto'}}}]
+
+# Path tracking (always available)
+result = p.xpath('/interface/*/speed')
+print(result.paths)
+# [['interface', 'FastEthernet0/0', 'speed'],
+#  ['interface', 'FastEthernet0/1', 'speed']]
+```
+
+**XPath Features:**
+- ✅ Absolute paths: `/interface/FastEthernet0/0/duplex`
+- ✅ Recursive search: `//duplex` (find anywhere)
+- ✅ Wildcards: `/interface/*/duplex`
+- ✅ Predicates: `/interface[FastEthernet0/0]`
+- ✅ Predicate wildcards: `/interface[FastEthernet*]`
+- ✅ Context tracking: See which match came from where
+- ✅ Path tracking: `result.paths` shows path components
+
+**XPathResult Structure:**
+```python
+result = p.xpath('//duplex')
+print(result.success)   # True
+print(result.data)      # First match: 'auto'
+print(result.matches)   # All matches: ['auto', 'auto']
+print(result.count)     # Number of matches: 2
+print(result.query)     # Original query: '//duplex'
+print(result.paths)     # Path to each match
+print(result.error)     # Error message if failed
+
+# Boolean check
+if result:
+    print(f"Found {result.count} matches")
 ```
 
-See [docs/XPATH_GUIDE.md](docs/XPATH_GUIDE.md) for complete XPath documentation.
+**Note:** XPath queries only work with `output_format='yaml'`. JSON format (OrderedDict) preserves exact CLI structure and should use traditional dict navigation.
 
 ### Alternative: Using Individual Components
 

asserts/img/sh_run_yaml.png

@@ -91,4 +91,24 @@ make check-all
 3. **Write tests** - For any new features or bug fixes
 4. **Keep it simple** - Small, focused commits are easier to review
 
+## Quick Feature Test
+
+Try the XPath feature:
+```python
+from shconfparser import Parser
+
+p = Parser(output_format='yaml')
+data = p.read('data/shrun.txt')
+tree = p.parse_tree(data)
+
+# Simple query
+result = p.xpath('/hostname')
+print(f"Hostname: {result.data}")
+
+# Query with context
+result = p.xpath('/interface/*/duplex', context='partial')
+for match in result.matches:
+    print(match)  # Shows which interface
+```
+
 Happy coding! 🚀

docs/QUICKSTART.md

@@ -14,6 +14,7 @@ Welcome to the shconfparser documentation!
 
 | Document | Description | Audience |
 |----------|-------------|----------|
+| [XPath Guide](XPATH_GUIDE.md) | Complete XPath query documentation | All users |
 | [Modernization Guide](MODERNIZATION_GUIDE.md) | Complete v2.x to v3.0 migration guide | Existing users, developers |
 | [Architecture](ARCHITECTURE.md) | Visual architecture diagrams and structure | Developers, architects |
 | [Business Standards](BUSINESS_STANDARDS.md) | Enterprise compliance documentation | Maintainers, enterprises |
@@ -31,6 +32,7 @@ Welcome to the shconfparser documentation!
 1. Start with the main [README.md](../README.md)
 2. Follow [Installation instructions](../README.md#installation-and-downloads)
 3. Check [Usage Examples](../README.md#usage-examples)
+4. Try [XPath Queries](../README.md#xpath-queries-new-in-30) for advanced querying
 
 ### For Contributors
 1. Read [QUICKSTART.md](QUICKSTART.md) (5 minutes)
@@ -95,6 +97,7 @@ Detailed summary of the modernization project:
 ### I want to...
 
 - **Get started contributing** → [QUICKSTART.md](QUICKSTART.md)
+- **Learn XPath queries** → [XPATH_GUIDE.md](XPATH_GUIDE.md)
 - **Understand v3.0 changes** → [MODERNIZATION_GUIDE.md](MODERNIZATION_GUIDE.md)
 - **Migrate from v2.x** → [MODERNIZATION_GUIDE.md](MODERNIZATION_GUIDE.md#migration-from-v2x-to-v30)
 - **Learn about the architecture** → [ARCHITECTURE.md](ARCHITECTURE.md)