add ks test to docs, add support for items sketches in ks test

Author: jmalkin

docs/source/index.rst

@@ -76,6 +76,7 @@ Helper Classes
    kernel
    jaccard
    tuple_policy
+   ks_test
 
 
 These classes are required for certain sketches or specific functionality withing sketches.

docs/source/kll.rst

@@ -103,6 +103,8 @@ Additionally, the interval may be quite large for certain distributions.
 - Let `q_hi = estimated quantile of rank (r + eps)`.
 - Then `q_lo ≤ q ≤ q_hi`, with 99% confidence.
 
+.. note::
+    For the :class:`kll_items_sketch`, objects must be comparable with ``__lt__``.
 
 .. autoclass:: _datasketches.kll_ints_sketch
     :members:

docs/source/ks_test.rst

@@ -0,0 +1,14 @@
+Kolmogorov-Smirnov Test
+#######################
+
+.. currentmodule:: datasketches
+
+A `Kolmogorov-Smirnov Test <https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test>`_` is
+a test of equality of two distributions to determine if they are likely to have come from the same
+underlying distribution.
+The DataSketches library provides a modified form of the test that takes into account the error
+in each underlying sketch in the analysis.
+
+Currently, the test assumes both input sketches are of the same family and data type.
+
+.. autofunction:: ks_test

docs/source/quantiles_depr.rst

@@ -39,6 +39,9 @@ For example, the median item returned from `get_quantile(0.5)` will be between t
 from the hypothetically sorted array of input items at normalized ranks of 0.483 and 0.517, with
 a confidence of about 99%.
 
+.. note::
+    For the :class:`quantiles_items_sketch`, objects must be comparable with ``__lt__``.
+
 .. autoclass:: _datasketches.quantiles_ints_sketch
     :members:
     :undoc-members:

docs/source/req.rst

@@ -27,6 +27,9 @@ This implementation allows the user to use both the `INCLUSIVE` criterion and th
 This implementation provides extensive debug visibility into the operation of the sketch with two levels of detail output. 
 This is not only useful for debugging, but is a powerful tool to help users understand how the sketch works.
 
+.. note::
+    For the :class:`req_items_sketch`, objects must be comparable with ``__lt__``.
+
 .. autoclass:: _datasketches.req_ints_sketch
     :members:
     :undoc-members:

src/ks_wrapper.cpp

@@ -20,6 +20,7 @@
 #include "kolmogorov_smirnov.hpp"
 #include "kll_sketch.hpp"
 #include "quantiles_sketch.hpp"
+#include "py_object_lt.hpp"
 
 #include <nanobind/nanobind.h>
 
@@ -29,38 +30,50 @@ void init_kolmogorov_smirnov(nb::module_ &m) {
   using namespace datasketches;
 
   m.def("ks_test", &kolmogorov_smirnov::test<kll_sketch<int>>, nb::arg("sk_1"), nb::arg("sk_2"), nb::arg("p"),
-    "Performs the Kolmogorov-Smirnov Test between kll_ints_sketches.\n"
+    "Performs the Kolmogorov-Smirnov Test for :code:`kll_ints_sketch` pairs.\n"
     "Note: if the given sketches have insufficient data or if the sketch sizes are too small, "
     "this will return false.\n"
     "Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying "
     "distribution) using the provided p-value, otherwise False.");
   m.def("ks_test", &kolmogorov_smirnov::test<kll_sketch<float>>, nb::arg("sk_1"), nb::arg("sk_2"), nb::arg("p"),
-    "Performs the Kolmogorov-Smirnov Test between kll_floats_sketches.\n"
+    "Performs the Kolmogorov-Smirnov Test for :code:`kll_floats_sketch` pairs.\n"
     "Note: if the given sketches have insufficient data or if the sketch sizes are too small, "
     "this will return false.\n"
     "Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying "
     "distribution) using the provided p-value, otherwise False.");
   m.def("ks_test", &kolmogorov_smirnov::test<kll_sketch<double>>, nb::arg("sk_1"), nb::arg("sk_2"), nb::arg("p"),
-    "Performs the Kolmogorov-Smirnov Test between kll_doubles_sketches.\n"
+    "Performs the Kolmogorov-Smirnov Test for :code:`kll_doubles_sketch` pairs.\n"
+    "Note: if the given sketches have insufficient data or if the sketch sizes are too small, "
+    "this will return false.\n"
+    "Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying "
+    "distribution) using the provided p-value, otherwise False.");
+  m.def("ks_test", &kolmogorov_smirnov::test<kll_sketch<nb::object, py_object_lt>>, nb::arg("sk_1"), nb::arg("sk_2"), nb::arg("p"),
+    "Performs the Kolmogorov-Smirnov Test for :code:`kll_items_sketch` pairs.\n"
     "Note: if the given sketches have insufficient data or if the sketch sizes are too small, "
     "this will return false.\n"
     "Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying "
     "distribution) using the provided p-value, otherwise False.");
 
   m.def("ks_test", &kolmogorov_smirnov::test<quantiles_sketch<int>>, nb::arg("sk_1"), nb::arg("sk_2"), nb::arg("p"),
-    "Performs the Kolmogorov-Smirnov Test between quantiles_ints_sketches.\n"
+    "Performs the Kolmogorov-Smirnov Test for :code:`quantiles_ints_sketch` pairs.\n"
     "Note: if the given sketches have insufficient data or if the sketch sizes are too small, "
     "this will return false.\n"
     "Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying "
     "distribution) using the provided p-value, otherwise False.");
   m.def("ks_test", &kolmogorov_smirnov::test<quantiles_sketch<float>>, nb::arg("sk_1"), nb::arg("sk_2"), nb::arg("p"),
-    "Performs the Kolmogorov-Smirnov Test between quantiles_floats_sketches.\n"
+    "Performs the Kolmogorov-Smirnov Test for :code:`quantiles_floats_sketch` pairs.\n"
     "Note: if the given sketches have insufficient data or if the sketch sizes are too small, "
     "this will return false.\n"
     ":Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying "
     "distribution) using the provided p-value, otherwise False.");
   m.def("ks_test", &kolmogorov_smirnov::test<quantiles_sketch<double>>, nb::arg("sk_1"), nb::arg("sk_2"), nb::arg("p"),
-    "Performs the Kolmogorov-Smirnov Test between quantiles_doubles_sketches.\n"
+    "Performs the Kolmogorov-Smirnov Test for :code:`quantiles_doubles_sketch` pairs.\n"
+    "Note: if the given sketches have insufficient data or if the sketch sizes are too small, "
+    "this will return false.\n"
+    "Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying "
+    "distribution) using the provided p-value, otherwise False.");
+  m.def("ks_test", &kolmogorov_smirnov::test<quantiles_sketch<nb::object, py_object_lt>>, nb::arg("sk_1"), nb::arg("sk_2"), nb::arg("p"),
+    "Performs the Kolmogorov-Smirnov Test for :code:`quantiles_items_sketch` pairs.\n"
     "Note: if the given sketches have insufficient data or if the sketch sizes are too small, "
     "this will return false.\n"
     "Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying "

tests/kll_test.py

@@ -163,6 +163,10 @@ def test_kll_items_sketch(self):
       self.assertEqual(kll.get_quantile(0.7), new_kll.get_quantile(0.7))
       self.assertEqual(kll.get_rank(str(n/4)), new_kll.get_rank(str(n/4)))
 
+      # The sketches are identical so we cannot reject the null hypothesis that they
+      # reflect the same underlying distribtion
+      self.assertFalse(ks_test(kll, new_kll, 0.001))
+
 
 if __name__ == '__main__':
     unittest.main()

tests/quantiles_test.py

@@ -163,6 +163,10 @@ def test_quantiles_items_sketch(self):
       self.assertEqual(quantiles.get_quantile(0.7), new_quantiles.get_quantile(0.7))
       self.assertEqual(quantiles.get_rank(str(n/4)), new_quantiles.get_rank(str(n/4)))
 
+      # The sketches are identical so we cannot reject the null hypothesis that they
+      # reflect the same underlying distribtion
+      self.assertFalse(ks_test(quantiles, new_quantiles, 0.001))
+
 
 if __name__ == '__main__':
     unittest.main()