You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+22-21Lines changed: 22 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,20 +1,19 @@
1
1
# CheckTestOutput
2
2
3
-
A library for semi-manual tests. Run a function, manually check the output. But only if it is different than last run. Built on git - stage the new version to accept it.
3
+
**A library for semi-manual tests. Run a function, manually check the output - only if it is different than last run. Built on git - stage the new version to accept it.**
4
4
5
-
Although it's a nice idea that tests should verify if the results are correct by some smart logic, it not always possible/practical. For example, when testing a transpiler, it would come in handy to solve the halting problem. In that cases, you may end up with an assertion that simply checks if the result is equal to one of the valid outputs and that is annoying to maintain. This project just makes the long `Assert.Equal("....", generatedCode)` less annoying.
6
-
7
-
The library simply checks that the test output is the same as last time.
8
-
It the test output is **compared with its version from git index** and throws an **exception if it does not match**, prints a diff and writes a new version to the working tree.
9
-
To accept the new version, you stage the changed file.
10
-
Or, to inspect the differences, you can use your favorite diff tool.
5
+
Although it's a nice idea that tests should verify if the results are correct by some smart logic, it not always possible/practical. For example, when testing a transpiler, it would come in handy to solve the halting problem. In such cases, you will end up with an `Assert.Equal("some very long code including many \" \" and \n \n, super fun to read and maintain", generatedCode)`. This project just makes the long asserts less annoying.
11
6
7
+
CheckTestOutput simply compares the test output and the last "accepted" test output. When it differs, you get an error.
8
+
The test output is **compared with its version from git index** and throws an **exception if it does not match**, prints a diff and writes a new version to the working tree.
9
+
To accept the new version, you simply stage the changed file (`git add ...`).
10
+
To inspect the differences, you use your favorite diff tool.
12
11
13
12
## Usage
14
13
15
14
It requires your project to be in [git version control system](https://git-scm.com/) (it works without git, but does not offer the simple stage-to-accept workflow).
16
15
You can use any test framework you want, this thing just throws exceptions -- we'll use XUnit in the examples here.
17
-
The `OutputChecker` constructor parameter specifies where are the files with the expected output going to be stored relative to the test file location (it uses [C#/F# caller info attributes](https://docs.microsoft.com/cs-cz/dotnet/csharp/programming-guide/concepts/caller-information)).
16
+
The `OutputChecker` constructor parameter specifies where are the output files (relative to the test file location - it uses [C#/F# caller info attributes](https://docs.microsoft.com/cs-cz/dotnet/csharp/programming-guide/concepts/caller-information)).
18
17
In this case, it's `./testoutputs`.
19
18
The file name will be equal to the caller method name, in this case, `SomeTest.TestString.txt`.
20
19
@@ -44,7 +43,7 @@ public void TestLines()
44
43
}
45
44
```
46
45
47
-
You can check if the object matches when it is serialized to JSON (using Newtonsoft.Json)
46
+
Check if the object matches when it is serialized to JSON (using Newtonsoft.Json)
48
47
49
48
```csharp
50
49
[Fact]
@@ -55,7 +54,7 @@ public void TestObject()
55
54
}
56
55
```
57
56
58
-
In case you want to use more that one check in one test, you can give them names (so they don't end up overriding themselves):
57
+
To use more that one check in one test, you need to give them names (so they don't end up overriding themselves):
59
58
60
59
```csharp
61
60
[Fact]
@@ -67,7 +66,7 @@ public void TestWithMultipleChecks()
67
66
}
68
67
```
69
68
70
-
Or you can combine them into one anonymous JSON object (this is preferable when it's short - you don't end up with so many tiny files):
69
+
Alternatively, combine them into one anonymous JSON object. It's generally preferable when the string are short - too many tiny files are annoying:
71
70
72
71
```csharp
73
72
[Fact]
@@ -82,7 +81,7 @@ public void TestObject()
82
81
}
83
82
```
84
83
85
-
The `checkName` parameter is also useful for tests with parameters (Theory in XUnit).
84
+
The `checkName` parameter is also useful for tests with parameters (`[Theory]` in XUnit).
86
85
For example, this way we could test a regular expression:
87
86
88
87
```csharp
@@ -101,7 +100,7 @@ public void IncrementNumbers(string checkName, string testString)
101
100
}
102
101
```
103
102
104
-
The text files have `.txt` file extension by default, if that annoys you because your strings have some specific format and syntax highlighting does not work in text files...
103
+
The text files have `.txt` file extension by default, but it's easy to change:
105
104
106
105
```csharp
107
106
[Fact]
@@ -116,27 +115,29 @@ Just keep in mind that dotnet is going to treat these `.cs` files as part of sou
116
115
117
116
### F#
118
117
119
-
This library is F# friendly, although it's written in C#:
118
+
CheckTestOutput is reasonably F# friendly, although it's written in C#:
120
119
121
120
```fsharp
122
121
open CheckTestOutput
123
122
124
123
let check = OutputChecker "testoutputs"
125
124
126
125
[<Fact>]
127
-
let ``Simple object processing - UseGenericUnion`` () = task {
126
+
let ``Simple object processing - UseGenericUnion`` () =
If you test string, for example, contains some randomly generated GUIDs it's not possible to test that the output is always the same. You could either make sure that the logic you are testing is fully deterministic, or you can fix it later. CheckTestOutput has a helper functionality which allows you to replace random GUIDs with deterministically generated ones.
140
+
If the test output contains some randomly generated UUIDs it isn't possible to test that the output is always the same. To fix the problem, you would either have to use a seeded random generator, or replace the UUIDs after the fact. CheckTestOutput has a helper functionality which allows you to replace random UUIDs with deterministically generated ones.
140
141
141
142
You can enable it by setting `sanitizeGuids: true` when creating `OutputChecker` (or `sanitizeQuotedGuids` to sanitize GUIDs in quotes):
142
143
@@ -153,7 +154,7 @@ public void CheckGuidsJson()
153
154
}
154
155
```
155
156
156
-
The sanitization preserves equality - it replaces different GUIDs with different stub string and same GUID with the same string. In this case, the checked JSON will be this:
157
+
The sanitization preserves equality - it replaces different UUIDs with different stub string and same UUID with the same string. In this case, the checked JSON will be this:
157
158
158
159
```json
159
160
{
@@ -163,7 +164,7 @@ The sanitization preserves equality - it replaces different GUIDs with different
163
164
}
164
165
```
165
166
166
-
You can replace anything that can be found by a regular expression, just specify the regexes in the `nonDeterminismSanitizers` parameter.
167
+
While mostly used for UUIDs, we can replace anything that can be found by a regular expression - just specify a list of regular expressions in the `nonDeterminismSanitizers` parameter.
167
168
168
169
### Custom checks
169
170
@@ -189,11 +190,11 @@ For more inspiration, have a look at [CheckExtensions class in the Coberec proje
189
190
190
191
## Installation
191
192
192
-
Just install [CheckTestOutput NuGet package](https://www.nuget.org/packages/CheckTestOutput).
Alternatively, you can just grab the source codes from `src` folder and copy them into your project (it's MIT licensed, so just keep a link to this project in the copied code or something). This project has a dependency on [MedallionShell](https://github.com/madelson/MedallionShell) - an amazing library for executing processes without the glitches of the `Process` class. Also, it has a dependency on [Newtonsoft.Json](https://github.com/JamesNK/Newtonsoft.Json) for the JSON serialization. If you are copying the code you'll probably install these. You can exclude `JsonChecks.cs` file to get rid of Newtonsoft.Json dependency.
199
+
Alternatively, you can just grab the source codes from `src` folder and copy them into your project (it's MIT licensed, so just keep a link to this project in the copied code). This library does not have any other dependencies.
0 commit comments