Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 44 additions & 51 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,16 @@ Execute blocks of nushell code within markdown documents, write results back to

```nushell no-run
# this block won't run as it has the option `no-run` in its code fence
> git clone https://github.com/nushell-prophet/numd; cd numd
> nupm install --force --path . # optionally you can install this module via nupm
> use numd
> numd run README.md --no-save

git clone https://github.com/nushell-prophet/numd; cd numd

# optionally you can install this module via nupm
nupm install --force --path .

use numd

# run it on any file to check
numd run z_examples/1_simple_markdown/simple_markdown.md --no-save
```

## How it works
Expand All @@ -21,14 +27,17 @@ Experienced nushell users can understand the logic better by looking at [example
### Details on parsing code blocks and displaying the output

1. `numd` looks for code blocks marked with ` ```nushell ` or ` ```nu `.
2. In code blocks that do not contain any lines starting with the `>` symbol, `numd` executes the entire code block as is. If the code produces any output, the output is added next to the code block after an empty line, a line with the word `Output:`, and another empty line. The output is enclosed in code fences without a language identifier.
3. In code blocks that contain one or more lines starting with the `>` symbol, `numd` filters only lines that start with the `>` or `#` symbol. It executes or prints those lines one by one, and outputs the results immediately after the executed line.
2. Code blocks are split into command groups by blank lines (double newlines). Each command group is executed separately.
3. Output from each command group is displayed inline with `# =>` prefix immediately after the command.
4. Multiline commands (pipelines split across lines without blank lines) are treated as a single command group.
5. Plain `#` comments are preserved; `# =>` output lines are regenerated on each run.
6. Use the `separate-block` fence option to output results in a separate code block instead of inline.

### `numd run` flags and params

```nushell
> use numd
> numd run --help
use numd
numd run --help
# => Run Nushell code blocks in a markdown file, output results back to the `.md`, and optionally to terminal
# =>
# => Usage:
Expand Down Expand Up @@ -66,29 +75,30 @@ Experienced nushell users can understand the logic better by looking at [example
`numd` understands the following block options. Several comma-separated block options will be combined together. The block options should be in the [infostring](https://github.github.com/gfm/#info-string) of the opening code fence like the example: ` ```nushell try, new-instance `

```nushell
> numd list-code-options --list
# => ╭─────long─────┬─short─┬───────────────────────────description───────────────────────────╮
# => │ no-output │ O │ execute code without outputting results │
# => │ no-run │ N │ do not execute code in block │
# => │ try │ t │ execute block inside `try {}` for error handling │
# => │ new-instance │ n │ execute block in new Nushell instance (useful with `try` block) │
# => ╰─────long─────┴─short─┴───────────────────────────description───────────────────────────╯
numd list-code-options --list
# => ╭──────long──────┬─short─┬───────────────────────────description────────────────────────────╮
# => │ no-output │ O │ execute code without outputting results │
# => │ no-run │ N │ do not execute code in block │
# => │ try │ t │ execute block inside `try {}` for error handling │
# => │ new-instance │ n │ execute block in new Nushell instance (useful with `try` block) │
# => │ separate-block │ s │ output results in a separate code block instead of inline `# =>` │
# => ╰──────long──────┴─short─┴───────────────────────────description────────────────────────────╯
```

### Stats of changes

By default, `numd` provides basic stats on changes made.

```nushell
> let path = [z_examples 1_simple_markdown simple_markdown_with_no_output.md] | path join
> numd run --no-save $path
let path = [z_examples 1_simple_markdown simple_markdown_with_no_output.md] | path join
numd run --no-save $path
# => ╭──────────────────┬───────────────────────────────────╮
# => │ filename │ simple_markdown_with_no_output.md │
# => │ nushell_blocks │ 3 │
# => │ levenshtein_dist │ 53
# => │ diff_lines │ +9 (30.0%) │
# => │ diff_words │ +6 (8.7%) │
# => │ diff_chars │ +53 (12.1%) │
# => │ levenshtein_dist │ 52
# => │ diff_lines │ +8 (25.8%) │
# => │ diff_words │ +6 (8.5%) │
# => │ diff_chars │ +52 (11.6%) │
# => ╰──────────────────┴───────────────────────────────────╯
```

Expand All @@ -109,18 +119,8 @@ numd run $path --echo --no-save --no-stats --prepend-code "
$env.config.table.index_mode = 'never'
$env.config.table.mode = 'basic_compact'
"
```

Output:

```
# => ```nushell
# => [[a b c]; [1 2 3]]
# => ```
# =>
# => Output:
# =>
# => ```
# => # => +---+---+---+
# => # => | a | b | c |
# => # => | 1 | 2 | 3 |
Expand All @@ -131,7 +131,7 @@ Output:
### `numd clear-outputs`

```nu
> numd clear-outputs --help
numd clear-outputs --help
# => Remove numd execution outputs from the file
# =>
# => Usage:
Expand Down Expand Up @@ -159,15 +159,15 @@ Output:
`numd` can use the `display_output` hook to write the current session prompts together with their output into a specified markdown file. There are corresponding commands `numd capture start` and `numd capture stop`.

```nushell
> numd capture start --help
numd capture start --help
# => start capturing commands and their outputs into a file
# =>
# => Usage:
# => > capture start {flags} (file)
# =>
# => Flags:
# => -h, --help: Display the help message for this command
# => --separate: don't use `>` notation, create separate blocks for each pipeline
# => --separate-blocks: create separate code blocks for each pipeline instead of inline `# =>` output
# =>
# => Parameters:
# => file <path>: (optional, default: 'numd_capture.md')
Expand All @@ -180,7 +180,7 @@ Output:
```

```nushell
> numd capture stop --help
numd capture stop --help
# => stop capturing commands and their outputs
# =>
# => Usage:
Expand All @@ -199,11 +199,10 @@ Output:
### Some random familiar examples

```nushell
> ls z_examples | sort-by name | reject modified size
ls z_examples | sort-by name | reject modified size
# => ╭──────────────────name───────────────────┬─type─╮
# => │ z_examples/1_simple_markdown │ dir │
# => │ z_examples/2_numd_commands_explanations │ dir │
# => │ z_examples/3_book_types_of_data │ dir │
# => │ z_examples/4_book_working_with_lists │ dir │
# => │ z_examples/5_simple_nu_table │ dir │
# => │ z_examples/6_edge_cases │ dir │
Expand All @@ -213,13 +212,13 @@ Output:
# => │ z_examples/9_other │ dir │
# => ╰──────────────────name───────────────────┴─type─╯

> sys host | get boot_time
# => Fri Dec 5 01:08:37 2025
sys host | get boot_time
# => Fri Dec 5 03:49:33 2025

> 2 + 2
2 + 2
# => 4

> git tag | lines | sort -n | last
git tag | lines | sort -n | last
# => 0.1.21
```

Expand All @@ -230,12 +229,6 @@ Output:
[z_examples 1_simple_markdown simple_markdown.md]
| path join
| numd run $in --echo --no-save

# run examples in the `types_of_data.md` file,
# save intermed nushell script to `types_of_data.md_intermed_from_readme.nu`
[z_examples 3_book_types_of_data types_of_data.md]
| path join
| numd run $in --no-backup --save-intermed-script $'($in)_intermed_from_readme.nu'
```

## Development and testing
Expand All @@ -246,13 +239,13 @@ Testing of the `numd` module is done via `toolkit.nu`:

```nushell no-run
# Run all tests (unit + integration)
> nu toolkit.nu testing
nu toolkit.nu testing

# Run only unit tests (uses nutest framework)
> nu toolkit.nu testing-unit
nu toolkit.nu testing-unit

# Run only integration tests (executes example markdown files)
> nu toolkit.nu testing-integration
nu toolkit.nu testing-integration
```

### Unit tests
Expand All @@ -264,7 +257,7 @@ Unit tests in `tests/` use the [nutest](https://github.com/vyadh/nutest) framewo
Integration tests run all example files in `z_examples/` through numd and report changes via Levenshtein distance. Whatever changes are made in the module - it can be easily seen if they break anything (both by the Levenshtein distance metric or by `git diff` of the updated example files versus their initial versions).

```nushell no-run
> nu toolkit.nu testing-integration
nu toolkit.nu testing-integration
# => ╭───────────────────────────────────────────────┬─────────────────┬───────────────────┬────────────┬──────────────┬─────╮
# => │ filename │ nushell_blocks │ levenshtein_dist │ diff_lines │ diff_words │ ... │
# => ├───────────────────────────────────────────────┼─────────────────┼───────────────────┼────────────┼──────────────┼─────┤
Expand Down
64 changes: 35 additions & 29 deletions numd/commands.nu
Original file line number Diff line number Diff line change
Expand Up @@ -93,10 +93,7 @@ export def clear-outputs [
| group-by block_index
| items {|block_index block_lines|
$block_lines.line.0
| where $it !~ '^# => ?'
| if ($in | where $it =~ '^>' | is-empty) { } else {
where $it =~ '^(>|#|```)'
}
| where $it !~ '^# => ?' # strip `# =>` output lines, preserve plain `#` comments
| prepend (mark-code-block $block_index)
}
| flatten
Expand All @@ -107,7 +104,6 @@ export def clear-outputs [
lines
| update 0 { $'(char nl) # ($in)' } # keep infostring
| drop
| str replace --all --regex '^>\s*' ''
| str join (char nl)
| str replace -r '\s*$' (char nl)
}
Expand All @@ -123,7 +119,7 @@ export def clear-outputs [
# start capturing commands and their outputs into a file
export def --env 'capture start' [
file: path = 'numd_capture.md'
--separate # don't use `>` notation, create separate blocks for each pipeline
--separate-blocks # create separate code blocks for each pipeline instead of inline `# =>` output
]: nothing -> nothing {
cprint $'numd commands capture has been started.
Commands and their outputs of the current nushell instance
Expand All @@ -134,9 +130,9 @@ export def --env 'capture start' [

$env.numd.status = 'running'
$env.numd.path = ($file | path expand)
$env.numd.separate-blocks = $separate
$env.numd.separate-blocks = $separate_blocks

if not $separate { "```nushell\n" | save -a $env.numd.path }
if not $separate_blocks { "```nushell\n" | save -a $env.numd.path }

$env.backup.hooks.display_output = (
$env.config.hooks?.display_output?
Expand All @@ -159,7 +155,9 @@ export def --env 'capture start' [
$"```nushell\n($command)\n```\n```output-numd\n($in)\n```\n\n"
| str replace --regex --all "[\n\r ]+```\n" "\n```\n"
} else {
$"> ($command)\n($in)\n\n"
# inline output format: command followed by `# =>` prefixed output
let output_lines = $in | lines | each {$'# => ($in)'} | str join (char nl)
$"($command)\n($output_lines)\n\n"
}
| str replace --regex "\n{3,}$" "\n\n"
| if ($in !~ 'numd capture') {
Expand Down Expand Up @@ -340,13 +338,17 @@ export def create-execution-code [
}
} else { }
| if 'no-output' in $fence_options { } else {
if $whole_block { create-fence-output } else { }
# separate-block: output goes to a separate ```output-numd``` fence
# default: output is inline with `# =>` prefix
if 'separate-block' in $fence_options { create-fence-output } else { }
| if (check-print-append $in) {
create-indented-output
| generate-print-statement
} else { }
}
| $in + (char nl)
# Always print a blank line after each command group to preserve visual separation
| $in + "print ''"

$highlighted_command + $code_execution
}
Expand Down Expand Up @@ -383,31 +385,35 @@ export def generate-intermediate-script []: table<block_index: int, row_type: st
| str replace -r "\\s*$" "\n"
}

# Split code block content by blank lines into command groups, execute each, insert `# =>` output.
export def execute-block-lines [
fence_options: list<string>
]: list<string> -> list<string> {
skip | drop # skip code fences
| if ($in | where $it =~ '^>' | is-empty) {
# find blocks with no `>` symbol to execute them entirely
str join (char nl)
| create-execution-code $fence_options --whole_block
| [$in] # quick fix so the `execute-block-lines` would always output lists. Should be refactored.
} else {
each {
# define what to do with each line of the current block one by one
if $in starts-with '>' {
# if a line starts with `>`, execute it
create-execution-code $fence_options
} else if $in starts-with '#' {
if $in !~ '# =>' {
# if a line starts with `#`, print it
create-highlight-command
}
}
| where not ($it =~ '^# =>') # strip existing `# =>` output lines (keep plain `#` comments)
| str join (char nl)
| split-by-blank-lines
| each {|group|
if ($group | str trim | is-empty) {
# preserve blank line separators
''
} else if ($group | str trim | str starts-with '#') and ($group | str trim | lines | all {|line| $line =~ '^#'}) {
# pure comment group - just highlight it
$group | create-highlight-command
} else {
# executable command group
$group | create-execution-code $fence_options --whole_block
}
}
}

# Split string by blank lines (double newlines) into command groups.
# Preserves multiline commands that don't have blank lines between them.
export def split-by-blank-lines []: string -> list<string> {
split row "\n\n"
| each { str trim -c "\n" }
}

# Parse block indices from Nushell output lines and return a table with the original markdown line numbers.
export def extract-block-index []: list<string> -> table<block_index: int, line: string> {
let clean_lines = skip until {|x| $x =~ (mark-code-block) }
Expand Down Expand Up @@ -527,6 +533,7 @@ export def list-code-options [
["no-run" "N" "do not execute code in block"]
["try" "t" "execute block inside `try {}` for error handling"]
["new-instance" "n" "execute block in new Nushell instance (useful with `try` block)"]
["separate-block" "s" "output results in a separate code block instead of inline `# =>`"]
# ["picture-output" "p" "capture output as picture and place after block"]
]
| if $list { } else {
Expand Down Expand Up @@ -609,8 +616,7 @@ export def create-highlight-command []: string -> string {

# Trim comments and extra whitespaces from code blocks for use in the generated script.
export def remove-comments-plus []: string -> string {
str replace -r '^[>\s]+' '' # trim starting `>`
| str replace -r '[\s\n]+$' '' # trim newlines and spaces from the end of a line
str replace -r '[\s\n]+$' '' # trim newlines and spaces from the end of a line
| str replace -r '\s+#.*$' '' # remove comments from the last line. Might spoil code blocks with the # symbol, used not for commenting
}

Expand Down
Loading
Loading