Skip to content

Commit b82ea18

Browse files
machowmachow
committed
Doc squash - check_function, has_equal_ast, call tests
commit d1efd6a25a51d29292a211580236ff0c6da5be59 Merge: cb2836b ef3cf4a Author: Michael Chow <machow@princeton.edu> Date: Thu Jan 5 10:39:46 2017 -0500 Merge branch 'master' into doc-update commit cb2836b Author: Michael Chow <machow@princeton.edu> Date: Thu Jan 5 10:20:54 2017 -0500 Update has_equal_ast.md commit 869d1e9 Author: Michael Chow <machow@princeton.edu> Date: Thu Jan 5 10:20:33 2017 -0500 Update has_equal_ast.md commit 70eaab9 Author: Michael Chow <machow@princeton.edu> Date: Thu Jan 5 10:10:11 2017 -0500 Create has_equal_ast.md commit 9bf24d2 Author: Michael Chow <machow@princeton.edu> Date: Thu Jan 5 10:09:07 2017 -0500 Update index.rst commit 1ecc95a Author: Michael Chow <machow@princeton.edu> Date: Thu Jan 5 10:07:04 2017 -0500 Update part_checks.rst commit 74be751 Author: Michael Chow <machow@princeton.edu> Date: Thu Jan 5 10:06:36 2017 -0500 Update part_checks.rst commit 570dab0 Author: Michael Chow <machow@princeton.edu> Date: Thu Jan 5 09:51:39 2017 -0500 Update expression_tests.md commit 919cafe Author: Michael Chow <machow@princeton.edu> Date: Thu Jan 5 09:50:01 2017 -0500 Update expression_tests.md commit 21da399 Author: Michael Chow <machow@princeton.edu> Date: Thu Jan 5 09:44:00 2017 -0500 Update expression_tests.md commit f3b4fe2 Author: Michael Chow <machow@princeton.edu> Date: Thu Jan 5 09:28:19 2017 -0500 Update part_checks.rst commit e886696 Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 18:00:30 2017 -0500 Update spec2_summary.rst commit b49560d Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 17:57:23 2017 -0500 Update part_checks.rst commit 16fd7e1 Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 17:50:03 2017 -0500 Update part_checks.rst commit 581f8ab Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 17:47:46 2017 -0500 Update part_checks.rst commit 6f7e0a7 Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 17:47:10 2017 -0500 Update part_checks.rst commit 83ba826 Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 17:40:54 2017 -0500 Update part_checks.rst commit a917161 Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 17:40:23 2017 -0500 Update part_checks.rst commit 018cd38 Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 17:39:58 2017 -0500 Update part_checks.rst commit 6b07998 Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 17:34:59 2017 -0500 Update part_checks.rst commit 243ad0c Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 17:14:15 2017 -0500 Update part_checks.rst commit 15daa43 Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 16:55:40 2017 -0500 Update quickstart_guide.md commit 3adf522 Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 16:54:48 2017 -0500 Update quickstart_guide.md commit f649b36 Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 16:52:25 2017 -0500 Update quickstart_guide.md commit 72ebbe7 Author: Michael Chow <machow@princeton.edu> Date: Wed Jan 4 16:47:01 2017 -0500 doc - fix indexing to be 0-based
1 parent ef3cf4a commit b82ea18

6 files changed

Lines changed: 235 additions & 30 deletions

File tree

‎docs/source/expression_tests.md‎

Lines changed: 39 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -281,10 +281,45 @@ For example, the following SCT simply runs `len(x)` in the solution and student
281281
`call` Syntax
282282
-------------
283283

284-
* almost identical to has_equal_\*.
285-
* special call syntax
286-
- string form
287-
- dict form
284+
Testing a function definition or lambda may require calling it with some arguments.
285+
In order to do this, use the `call()` SCT.
286+
There are two ways to tell it what arguments to pass to the function/lambda,
287+
288+
* `call("f (1, 2, x = 3)")` - as a string, where `"f"` gets substituted with the function's name.
289+
* `call([1,2,3])` - as a list of positional arguments.
290+
291+
Below, two alternative ways of specifying the arguments to pass are shown.
292+
293+
*** =solution
294+
```{python}
295+
def my_fun(x, y = 4, z = ('a', 'b'), *args, **kwargs):
296+
return [x, y, *z, *args]
297+
```
298+
299+
*** =sct
300+
```{python}
301+
Ex().check_function_def('my_fun').call("f(1, 2, (3,4), 5, kw_arg='ok')") # as string
302+
Ex().check_function_def('my_fun').call([1, 2, (3,4), 5]) # as list
303+
```
304+
305+
```eval_rst
306+
.. note::
307+
308+
Technically, you can get crazy and replace the list approach with a dictionary of the form ``{'args': [POSARG1, POSARG2], 'kwargs': {KWARGS}}``.
309+
```
310+
311+
### Additional Parameters
312+
313+
In addition to its first argument, `call()` accepts all the parameters that the expression tests above can (i.e. `has_equal_value`, `has_equal_error`, `has_equal_output`).
314+
The function call is run at the point where these functions would evaluate an expression.
315+
Moreover, setting the argument `test` to either "value", "output", or "error" controls which expression test it behaves like.
316+
317+
For example, the SCT below shows how to run some `pre_code`, and then evaluate the output of a call.
318+
319+
```
320+
Ex().check_function_def('my_fun').call("f(1, 2)", test="output", pre_code="x = 1")
321+
```
322+
288323

289324
Managing Processes
290325
-----------------

‎docs/source/part_checks.rst‎

Lines changed: 146 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ For example, in order to test the body of the comprehension above, we could crea
2929
3030
*** =sct
3131
```{python}
32-
(Ex().check_list_comp(1) # focus on first list comp
32+
(Ex().check_list_comp(0) # focus on first list comp
3333
.check_body().test_student_typed('i\*2') # focus on its body for test
3434
)
3535
```
@@ -52,10 +52,10 @@ This section expands the above example to run tests on each part: body, iter, an
5252
5353
*** =sct
5454
```{python}
55-
list_comp = Ex().check_list_comp(1, missing_msg="Did you include a list comprehension?")
55+
list_comp = Ex().check_list_comp(0, missing_msg="Did you include a list comprehension?")
5656
list_comp.check_body().test_student_typed('i\*2')
5757
list_comp.check_iter().has_equal_value()
58-
list_comp.check_ifs(1).multi([has_equal_value(context_vals=[i]) for i in range(0,10)])
58+
list_comp.check_ifs(0).multi([has_equal_value(context_vals=[i]) for i in range(0,10)])
5959
```
6060
6161
In this SCT, the first line focuses on the first list comprehension, and assigns it to ``list_comp``, so we can test each part in turn. As a reminder, the code corresponding to each part in the solution code is..
@@ -87,14 +87,14 @@ The line
8787

8888
.. code-block:: python
8989
90-
list_comp.check_ifs(1).multi([has_equal_value(context_vals=[i] for i in range(0,10))])
90+
list_comp.check_ifs(0).multi([has_equal_value(context_vals=[i] for i in range(0,10))])
9191
9292
is a doozy, but can be broken down into
9393

9494
.. code-block:: python
9595
9696
equal_tests = [has_equal_value(context_vals=[i] for i in range(0,10))] # collection of has_equal_tests
97-
list_comp.check_ifs(1).multi(equal_tests) # focus on IFS run equal_tests`
97+
list_comp.check_ifs(0).multi(equal_tests) # focus on IFS run equal_tests`
9898
9999
In this case ``equal_tests`` is a list of ``has_equal_value`` tests that we'll want to perform. ``check_ifs(1)`` grabs the first IFS part, and ``multi(equal_tests)`` runs each ``has_equal_value`` test on that part.
100100

@@ -133,9 +133,9 @@ in order to test running the inline if expression we could go from list_comp =>
133133
134134
*** =sct
135135
```{python}
136-
(Ex().check_list_comp(1) # first comprehension
136+
(Ex().check_list_comp(0) # first comprehension
137137
.check_body().set_context(i=6) # comp's body
138-
.check_if_exp(1).has_equal_value() # body's inline IFS
138+
.check_if_exp(0).has_equal_value() # body's inline IFS
139139
)
140140
```
141141
@@ -148,7 +148,7 @@ If we left out the ``check_if_exp`` above, the resulting SCT,
148148

149149
.. code:: python
150150
151-
(Ex().check_list_comp(1).check_body().set_context(i=6)
151+
(Ex().check_list_comp(0).check_body().set_context(i=6)
152152
#.check_if_exp(1)
153153
.has_equal_value()
154154
)
@@ -326,7 +326,7 @@ but its BODY does.
326326
+------------------------+------------------------------------------------------+-------------------+
327327
| check | parts | target variables |
328328
+========================+======================================================+===================+
329-
|check_if_else | .. code:: | |
329+
|check_if_else(0) | .. code:: | |
330330
| | | |
331331
| | if TEST: | |
332332
| | BODY | |
@@ -335,38 +335,38 @@ but its BODY does.
335335
| | | |
336336
| | | |
337337
+------------------------+------------------------------------------------------+-------------------+
338-
|check_while | .. code:: python | |
338+
|check_while(0) | .. code:: python | |
339339
| | | |
340340
| | while TEST: | |
341341
| | BODY | |
342342
| | else: | |
343343
| | ORELSE | |
344344
| | | |
345345
+------------------------+------------------------------------------------------+-------------------+
346-
|check_list_comp | .. code:: | ``i`` |
346+
|check_list_comp(0) | .. code:: | ``i`` |
347347
| | | |
348348
| | [BODY for i in ITER if IFS[0] if IFS[1]] | |
349349
| | | |
350350
+------------------------+------------------------------------------------------+-------------------+
351-
|check_generator_exp | .. code:: | ``i`` |
351+
|check_generator_exp(0) | .. code:: | ``i`` |
352352
| | | |
353353
| | (BODY for i in ITER if IFS[0] if IFS[1]) | |
354354
| | | |
355355
+------------------------+------------------------------------------------------+-------------------+
356-
|check_dict_comp | .. code:: | ``k``, ``v`` |
356+
|check_dict_comp(0) | .. code:: | ``k``, ``v`` |
357357
| | | |
358358
| | {KEY : VALUE for k, v in ITER if IFS[0]} | |
359359
| | | |
360360
+------------------------+------------------------------------------------------+-------------------+
361-
|check_for_loop | .. code:: | ``i`` |
361+
|check_for_loop(0) | .. code:: | ``i`` |
362362
| | | |
363363
| | for i in ITER: | |
364364
| | BODY | |
365365
| | else: | |
366366
| | ORELSE | |
367367
| | | |
368368
+------------------------+------------------------------------------------------+-------------------+
369-
|check_try_except | .. code:: python | ``e`` |
369+
|check_try_except(0) | .. code:: python | ``e`` |
370370
| | | |
371371
| | try: | |
372372
| | BODY | |
@@ -380,24 +380,30 @@ but its BODY does.
380380
| | FINALBODY | |
381381
| | | |
382382
+------------------------+------------------------------------------------------+-------------------+
383-
|check_with | .. code:: python | `f`` |
383+
|check_with(0) | .. code:: python | `f`` |
384384
| | | |
385385
| | with CONTEXT_TEST as f: | |
386386
| | BODY | |
387387
| | | |
388388
+------------------------+------------------------------------------------------+-------------------+
389-
|check_function_def | .. code:: python | argument names |
389+
|check_function_def('f') | .. code:: python | argument names |
390390
| | | |
391391
| | def f(ARGS[0], ARGS[1]): | |
392392
| | BODY | |
393393
| | | |
394394
+------------------------+------------------------------------------------------+-------------------+
395-
|check_lambda | .. code:: | argument names |
395+
|check_lambda(0) | .. code:: | argument names |
396396
| | | |
397397
| | lambda ARGS[0], ARGS[1]: BODY | |
398398
| | | |
399399
| | | |
400400
+------------------------+------------------------------------------------------+-------------------+
401+
|check_function('f', 0) | .. code:: | argument names |
402+
| | | |
403+
| | f(ARGS[0], ARGS[1]) | |
404+
| | | |
405+
| | | |
406+
+------------------------+------------------------------------------------------+-------------------+
401407

402408
More
403409
------
@@ -425,22 +431,138 @@ can be checked with the following SCT
425431

426432
.. code:: python
427433
428-
(Ex().check_if_else(1) # lines 1-3
429-
.check_orelse().check_if_else(1) # lines 2-3
434+
(Ex().check_if_else(0) # lines 1-3
435+
.check_orelse().check_if_else(0) # lines 2-3
430436
.check_orelse().has_equal_output() # line 3
431437
)
432438
433439
434440
function definition / lambda args
435441
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
436442

443+
the ARGS part in function definitions and lambdas may be selected by position or keyword.
444+
For example, the arguments `a` and `b` below,
445+
437446
.. code:: python
438447
439-
def f(a, b):
448+
def f(a, b=2, *some_name):
440449
BODY
441450
442-
*args and **kwargs
451+
Could be tested using,
452+
453+
.. code:: python
454+
455+
Ex().check_function_def('f').multi(
456+
check_args('a').is_default(),
457+
check_args('b').is_default().has_equal_value(),
458+
check_args('*args', 'missing a starred argument!')
459+
)
460+
461+
Note that ``check_args('*args')`` and ``check_args('**kwargs')`` may be used to test *args, and **kwargs style parameters, regardless of their name in the function definition.
462+
463+
function call args
464+
~~~~~~~~~~~~~~~~~~~
465+
466+
Behind the scenes, ``check_function`` uses the same logic for matching arguments to function signatures as `test_function_v2 <pythonwhat.wiki/test_function_v2.html>`__.
467+
It also has a ``signature`` argument that accepts a custom signature.
468+
469+
Matching Signatures
470+
^^^^^^^^^^^^^^^^^^^^
471+
472+
By default, ``check_function`` tries to match each argument in the function call with the appropriate parameters in that function's call signature.
473+
For example, all the calls to ``f`` below use ``a = 1`` and ``b = 2``.
474+
475+
.. code::
476+
477+
def f(a, b): pass
478+
479+
f(1, 2) # by position
480+
f(a = 1, b = 2) # by keyword
481+
f(1, b = 2) # mixed
482+
483+
However, when testing a submission, we may not care how the argument was specified.
484+
485+
.. code::
486+
487+
*** =pre_exercise_code
488+
```{python}
489+
def f(a, b): pass
490+
```
491+
492+
*** =solution
493+
```{python}
494+
f(1, b=2)
495+
```
496+
497+
*** =sct
498+
```{python}
499+
Ex().check_function('f', 0).check_args('a').has_equal_value()
500+
```
501+
502+
will pass for all the ways of calling ``f`` listed above.
503+
504+
signature = False
443505
^^^^^^^^^^^^^^^^^^
444506

445-
testing default values
446-
^^^^^^^^^^^^^^^^^^^^^^
507+
Setting signature to false, as below, only allows you to check an argument by name, if the name was explicitly specified in the function call.
508+
For example,
509+
510+
.. code::
511+
512+
*** =solution
513+
```{python}
514+
dict( [('a', 1)], c = 2)
515+
```
516+
517+
*** =sct
518+
```{python}
519+
Ex().check_function('dict', 0, signature=False)\
520+
.multi(
521+
check_args(0), # can only select by position
522+
check_args('c') # could use check_args(1)
523+
)
524+
```
525+
526+
Note that here, an argument's position is referring to its position in the function call (not its signature).
527+
528+
Example: testing a list passed as an argument
529+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
530+
531+
Suppose you want to test the first argument passed to `sum`.
532+
Below, we show how this can be down, using `has_equal_ast()` to check that the abstract syntax trees for the 1st argument match.
533+
534+
.. code:: python
535+
536+
*** =solution
537+
```{python}
538+
sum([1, 2, 3])
539+
```
540+
541+
*** =sct
542+
```{python}
543+
(Ex().check_function('sum', 0)
544+
.check_args(0)
545+
.has_equal_ast("ast fail") # compares abstract representations
546+
.test_student_typed("\[1, 2, 3\]", "typed fail") # alternative, more rigid test
547+
)
548+
```
549+
550+
Notice that testing the argument is similar to testing, say, the body of an if statement.
551+
In this sense, we could even do deeper checks into an argument.
552+
Below, the SCT verifies that the first argument passed to sum is a list comprehension.
553+
554+
.. code:: python
555+
556+
*** =solution
557+
```{python}
558+
sum([i for i in range(10)])
559+
```
560+
561+
*** =sct
562+
```{python}
563+
(Ex().check_function('sum', 0)
564+
.check_args(0)
565+
.check_list_comp(0)
566+
.has_equal_ast()
567+
)
568+
```

‎docs/source/quickstart_guide.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,4 +55,12 @@ the automatic messages for when `x` is undefined or incorrect are replaced with
5555

5656
The same holds for `test_output_contains()`: you can use the `no_output_msg` argument to specify a custom message. For more information on all the different arguments you can set in the different `pythonwhat` functions, have a look at the articles in this wiki, describing them in detail.
5757

58-
[TODO: quick outline of next steps]
58+
Next Steps
59+
----------
60+
61+
Test functions in pythonwhat are broken into 4 groups:
62+
63+
* [Simple tests](simple_tests/index.rst): look at, e.g., the output produced by an entire code submission.
64+
* [Part checks](part_checks.rst): focus on specific pieces of code, like a particular for loop.
65+
* [Expression tests](expression_tests.md): combined with part checks, these run pieces of code and evaluate the outcome.
66+
* [Logic tests](logic_tests/index.rst): these allow logic like an or statement to be used with SCTs.

‎docs/source/simple_tests/has_equal_ast.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
has_equal_ast
2+
--------------
3+
4+
```eval_rst
5+
.. automodule:: pythonwhat.check_funcs.has_equal_ast
6+
:members:
7+
```
8+
9+
An abstract syntax tree (AST) is a way of representing the high-level structure of python code.
10+
11+
### Example: quotes
12+
13+
Whether you use the concrete syntax `x = "1"` or `x = '1'`, the abstract syntax is the same: x is being assigned to the string "1".
14+
15+
### Example: parenthesis
16+
17+
Grouping by parentheses produces the same AST, when the same statement would work the same without them.
18+
For example, `(True or False) and True`, and `True or False and True`, are the same due to operator precedence.
19+
20+
### Example: spacing
21+
22+
The same holds for different types of spacing that essentially specify the same statement: `x = 1` or `x = 1`.
23+
24+
### Caveat: evaluating
25+
26+
What the AST doesn't represent is values that are found through evaluation. For example, the first item in the list in
27+
28+
```python
29+
x = 1
30+
[x, 2, 3]
31+
```
32+
33+
and
34+
35+
```python
36+
[1, 2, 3]
37+
```
38+
39+
Is not the same. In the first case, the AST represents that a variable `x` needs to be evaluated in order to find out what its value is. In the second case, it just represents the value `1`.

0 commit comments

Comments
 (0)