Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .jules/bolt.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,7 @@
## 2024-05-24 - [Thread Pool Size for Concurrent I/O]
**Learning:** Hardcoded, small thread pool limits (like `max_workers=50`) act as severe bottlenecks for highly I/O bound concurrent network tasks like ping sweeping an entire subnet. Because pings spend most of their time waiting on network timeouts, artificially restricting concurrency forces the pool to process timeouts in batches, drastically increasing total scan time.
**Action:** When using `concurrent.futures.ThreadPoolExecutor` for pure I/O or network tasks where the operation is mostly waiting, dynamically size `max_workers` to handle the full workload concurrently (e.g., `min(total_tasks, 256)`) to complete all timeouts in parallel.

## 2026-03-23 - [Subprocess PATH lookup overhead]
**Learning:** Calling `subprocess.call(["ping", ...])` without the absolute path causes the OS/Python interpreter to repeatedly scan through all directories listed in the `PATH` environment variable to locate the executable file for *every single* invocation. In highly concurrent or iterative loops (like a network sweep using `ThreadPoolExecutor`), this redundant lookup creates a measurable performance bottleneck.
**Action:** When invoking external commands repetitively via `subprocess` in a tight loop or concurrently, cache the absolute path of the executable once at module initialization using `shutil.which("command") or "command"` to eliminate `PATH` traversal overhead.
6 changes: 4 additions & 2 deletions test_testping1.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,8 @@ def test_is_reachable_prevents_log_injection(self, mock_call):
@patch('testping1.subprocess.call')
def test_is_reachable_subprocess_timeout(self, mock_call):
"""Test is_reachable handles subprocess.TimeoutExpired securely."""
mock_call.side_effect = subprocess.TimeoutExpired(cmd='ping', timeout=7)
from testping1 import PING_PATH
mock_call.side_effect = subprocess.TimeoutExpired(cmd=PING_PATH, timeout=7)
with self.assertLogs(level='ERROR') as log:
self.assertFalse(is_reachable('127.0.0.1', timeout=5))
self.assertIn("Ping command timed out unexpectedly.", log.output[0])
Expand All @@ -98,12 +99,13 @@ def test_is_reachable_subprocess_timeout(self, mock_call):
@patch('testping1.subprocess.call')
def test_is_reachable_calls_ping_correctly(self, mock_call):
"""Test is_reachable calls the ping command with correct arguments."""
from testping1 import PING_PATH
mock_call.return_value = 0

is_reachable('192.168.1.1', timeout=5)
# Verify that subprocess.call was called with the correct arguments, including the timeout
mock_call.assert_called_once_with(
['ping', '-n', '-c', '1', '-W', '5', '192.168.1.1'],
[PING_PATH, '-n', '-c', '1', '-W', '5', '192.168.1.1'],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=7
)

Expand Down
9 changes: 8 additions & 1 deletion testping1.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,15 @@
import concurrent.futures
import ipaddress
import logging
import shutil
from tqdm import tqdm # Install with `pip install tqdm`

# ⚑ Bolt: Cache the absolute path of the ping executable.
# Calling shutil.which() once at module load avoids the overhead of traversing
# the system PATH environment variable during every subprocess.call() execution.
# This yields a measurable speedup when firing thousands of concurrent pings.
PING_PATH = shutil.which("ping") or "ping"

def is_reachable(ip, timeout=1):
"""Checks if a device at the given IP address is reachable with a ping.

Expand Down Expand Up @@ -43,7 +50,7 @@ def is_reachable(ip, timeout=1):
# The `-n` flag skips reverse DNS resolution. Without it, ping attempts to
# resolve the hostname for every IP, which can cause multi-second delays
# (even with a 1s timeout) if the IP lacks a PTR record or DNS is unresponsive.
command = ["ping", "-n", "-c", "1", "-W", str(timeout_val), str(ip_obj)] # -W for timeout in seconds (Linux)
command = [PING_PATH, "-n", "-c", "1", "-W", str(timeout_val), str(ip_obj)] # -W for timeout in seconds (Linux)

# ⚑ Bolt: Optimized ping execution by using subprocess.call and redirecting
# output to DEVNULL instead of using Popen with PIPE.
Expand Down
Loading