diff --git a/CHANGELOG.md b/CHANGELOG.md index e184c2b..041b394 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,13 @@ See [Conventional Commits](Https://conventionalcommits.org) for commit guideline +## Unreleased + +### Improvements: + +* use the shared `ash_sql` grouped aggregate strategy for aggregate loading, filtering, sorting, calculations, and root query aggregates +* add grouped aggregate regression coverage for list defaults and root query aggregate kinds + ## [v0.2.17](https://github.com/ash-project/ash_sqlite/compare/v0.2.16...v0.2.17) (2026-04-22) diff --git a/README.md b/README.md index 1f3d18f..d8d68a3 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,7 @@ Welcome! `AshSqlite` is the SQLite data layer for [Ash Framework](https://hexdoc ### Resources +- [Aggregates](documentation/topics/resources/aggregates.md) - [References](documentation/topics/resources/references.md) - [Polymorphic Resources](documentation/topics/resources/polymorphic-resources.md) diff --git a/documentation/topics/about-ash-sqlite/what-is-ash-sqlite.md b/documentation/topics/about-ash-sqlite/what-is-ash-sqlite.md index 08abe09..394b8c1 100644 --- a/documentation/topics/about-ash-sqlite/what-is-ash-sqlite.md +++ b/documentation/topics/about-ash-sqlite/what-is-ash-sqlite.md @@ -6,7 +6,7 @@ SPDX-License-Identifier: MIT # What is AshSqlite? -AshSqlite is the SQLite `Ash.DataLayer` for [Ash Framework](https://hexdocs.pm/ash). This doesn't have all of the features of [AshPostgres](https://hexdocs.pm/ash_postgres), but it does support most of the features of Ash data layers. The main feature missing is Aggregate support. +AshSqlite is the SQLite `Ash.DataLayer` for [Ash Framework](https://hexdocs.pm/ash). This doesn't have all of the features of [AshPostgres](https://hexdocs.pm/ash_postgres), but it does support most of the features of Ash data layers. AshSqlite supports related aggregates, filters, sorts, and expression calculations for common SQLite-backed applications. See the [AshSqlite aggregates guide](../resources/aggregates.md) for supported aggregate cases and SQLite-specific limitations. Use this to persist records in a SQLite table. For example, the resource below would be persisted in a table called `tweets`: diff --git a/documentation/topics/resources/aggregates.md b/documentation/topics/resources/aggregates.md new file mode 100644 index 0000000..8655ecf --- /dev/null +++ b/documentation/topics/resources/aggregates.md @@ -0,0 +1,240 @@ + + +# Aggregates + +AshSqlite supports resource aggregates that can be loaded, filtered, sorted, and used in expression calculations. For general Ash aggregate usage, see the [Ash aggregates guide](https://hexdocs.pm/ash/aggregates.html). + +## Supported Aggregates + +AshSqlite supports related `count`, `sum`, `avg`, `min`, `max`, `exists`, `first`, `list`, and `custom` aggregates over normal relationship paths. + +```elixir +aggregates do + count :total_tickets, :tickets + exists :has_open_tickets, :tickets do + filter expr(status == :open) + end + + first :first_ticket_subject, :tickets, :subject do + sort subject: :asc_nils_last + end + + list :ticket_subjects, :tickets, :subject do + sort subject: :asc_nils_last + end +end +``` + +Aggregates are translated to SQL and can be used in queries. + +```elixir +require Ash.Query + +Helpdesk.Support.Representative +|> Ash.Query.filter(total_tickets > 2) +|> Ash.Query.sort(total_tickets: :desc) +|> Ash.Query.load([:total_tickets, :first_ticket_subject]) +|> Ash.read!() +``` + +Aggregates can also be loaded on records that have already been read. + +```elixir +representatives = Helpdesk.Support.read!(Helpdesk.Support.Representative) + +Ash.load!(representatives, [:total_tickets, :ticket_subjects]) +``` + +## Query Aggregates + +AshSqlite supports root query aggregates for `count`, `sum`, `avg`, `min`, `max`, +`first`, and `exists`. + +```elixir +Helpdesk.Support.Representative +|> Ash.Query.filter(active == true) +|> Ash.aggregate!(count: :count) +``` + +Relationship query aggregates are not supported by the grouped aggregate query +path. Define a resource aggregate and load, filter, or sort on that aggregate +instead. + +## Calculations + +Expression calculations can reference aggregates and be pushed down to SQLite. + +```elixir +aggregates do + count :total_tickets, :tickets + + count :open_tickets, :tickets do + filter expr(status == :open) + end +end + +calculations do + calculate :percent_open, :float, expr(open_tickets / total_tickets) +end +``` + +Calculations that reference aggregates can be loaded, filtered, and sorted in the same way. + +```elixir +require Ash.Query + +Helpdesk.Support.Representative +|> Ash.Query.filter(percent_open > 0.25) +|> Ash.Query.sort(:percent_open) +|> Ash.Query.load(:percent_open) +|> Ash.read!() +``` + +## Relationship Paths + +Aggregates are supported over normal relationship paths, including multi-hop paths. + +```elixir +aggregates do + count :comment_count, [:posts, :comments] + sum :paid_total, [:orders, :payments], :amount +end +``` + +One-hop many-to-many relationship aggregates are supported. Scalar aggregates are also supported when a multi-hop path ends in a many-to-many relationship. + +```elixir +aggregates do + count :linked_post_count, :linked_posts + count :linked_post_count_through_posts, [:posts, :linked_posts] + + first :first_linked_post_title, :linked_posts, :title do + sort title: :asc_nils_last + end +end +``` + +Parent-independent unrelated aggregates are supported when the aggregate query does not need values from the parent row. + +```elixir +aggregates do + count :published_post_count, Post do + filter expr(published == true) + end +end +``` + +## Aggregate Filters + +Aggregate filters and aggregate `join_filter`s are supported for normal paths and one-hop many-to-many paths when they do not depend on parent row values. + +```elixir +aggregates do + count :open_ticket_count, :tickets do + filter expr(status == :open) + end + + count :matching_ticket_count, :tickets do + join_filter :tickets, expr(priority == :high) + end +end +``` + +For many-to-many aggregates, a `join_filter` on the many-to-many relationship applies to the destination resource side of the aggregate. Put through-resource filtering on the relationship's configured join relationship/filter. + +For filters that need to test a to-many relationship without multiplying the aggregate rows, prefer `exists/2`. + +```elixir +aggregates do + sum :liked_comment_total, :comments, :likes do + filter expr(exists(ratings, score > 5)) + end +end +``` + +Multi-hop aggregates use each relationship's configured read action. If an intermediate hop needs scoped rows, define the read action on that relationship rather than trying to override it per aggregate. + +## SQLite Requirements + +`first` and `list` aggregates require SQLite 3.30.0 or later with JSON functions enabled. Window functions were added in SQLite 3.25.0, but AshSqlite's generated SQL also uses aggregate `FILTER` clauses and explicit `NULLS FIRST`/`NULLS LAST` ordering, which require SQLite 3.30.0 or later. + +- window functions +- aggregate `FILTER` +- JSON aggregation +- explicit null ordering + +JSON functions are built into SQLite by default as of SQLite 3.38.0. Older SQLite builds need the JSON1 extension enabled. Check the SQLite library used by your application, which may not be the same binary as the `sqlite3` command: + +```elixir +MyApp.Repo.query!("select sqlite_version()") +MyApp.Repo.query!("select json_group_array(1)") +``` + +`list` aggregates return lists through SQLite JSON aggregation. `custom` aggregates require a SQLite-compatible aggregate expression or function. + +## Custom Aggregates + +Custom aggregates should use both `Ash.Resource.Aggregate.CustomAggregate` and `AshSqlite.CustomAggregate`. + +```elixir +defmodule MyApp.StringAgg do + use Ash.Resource.Aggregate.CustomAggregate + use AshSqlite.CustomAggregate + + require Ecto.Query + + def dynamic(opts, binding) do + Ecto.Query.dynamic( + [], + fragment("group_concat(?, ?)", field(as(^binding), ^opts[:field]), ^opts[:delimiter]) + ) + end +end +``` + +Then use that implementation from a resource aggregate. + +```elixir +aggregates do + custom :ticket_subjects_joined, :tickets, :string do + implementation {MyApp.StringAgg, field: :subject, delimiter: ", "} + end +end +``` + +`AshSqlite.CustomAggregate` only defines the `dynamic/2` contract. It does not install SQLite extensions or register user-defined functions. If your custom aggregate uses a function that is not built into SQLite, register it with the SQLite connection yourself and make sure it is available in every environment. + +## Performance + +AshSqlite builds aggregate queries as grouped subqueries or windowed subqueries and joins those results back to the parent query. Add indexes for the relationship keys used by those subqueries. + +Useful indexes usually include: + +- child foreign keys, like `tickets.representative_id` +- many-to-many join resource key pairs +- fields used by aggregate filters +- fields used by `first` and `list` aggregate sorts + +## Unsupported Cases + +Full aggregate parity with [AshPostgres](https://hexdocs.pm/ash_postgres) is not available. Unsupported cases include: + +- inline query-level `list` and `custom` aggregate expressions +- unrelated aggregates that reference the parent row +- manual relationships +- `no_attributes?` relationships +- multi-hop paths that include many-to-many relationships before the final hop +- non-scalar aggregates over multi-hop paths that include many-to-many relationships +- parent-dependent relationship filters +- parent-dependent aggregate filters +- parent-dependent `join_filter`s +- aggregate filters that reference other aggregates +- expression sorts on `first` and `list` aggregates +- `uniq` list aggregates sorted by fields other than the listed field +- fanout-prone `sum`, `avg`, `list`, `custom`, or field-based `count` aggregate filters over to-many relationship references + +A fanout-prone aggregate filter is one where filtering joins another to-many relationship and can duplicate the rows being aggregated. For example, a `sum` of comment likes filtered by `popular_ratings.id` could count the same comment once per matching rating. AshSqlite rejects these shapes instead of returning an over-counted result. Use `exists/2` when you only need to test that related rows exist. diff --git a/documentation/tutorials/getting-started-with-ash-sqlite.md b/documentation/tutorials/getting-started-with-ash-sqlite.md index 7007b93..c4c6e30 100644 --- a/documentation/tutorials/getting-started-with-ash-sqlite.md +++ b/documentation/tutorials/getting-started-with-ash-sqlite.md @@ -326,11 +326,71 @@ Helpdesk.Support.Ticket ### Aggregates -As stated in [what-is-ash-sqlite](https://hexdocs.pm/ash_sqlite/getting-started-with-ash-sqlite.html#steps), -**The main feature missing is Aggregate support.**. +Aggregates include grouped data about relationships. You can read more about them in the [Ash aggregates guide](https://hexdocs.pm/ash/aggregates.html) and the [AshSqlite aggregates guide](../topics/resources/aggregates.md). -In order to use these consider using [ash_postgres](https://github.com/ash-project/ash_postgres) or -provide a patch. +Lets add aggregates to the representative resource so we can query how many tickets are assigned to a representative, how many are open, and the first ticket subject. + +```elixir +# in lib/helpdesk/support/resources/representative.ex + + aggregates do + count :total_tickets, :tickets + + count :open_tickets, :tickets do + filter expr(status == :open) + end + + exists :has_closed_tickets, :tickets do + filter expr(status == :closed) + end + + first :first_ticket_subject, :tickets, :subject do + sort subject: :asc_nils_last + end + end +``` + +Aggregates are translated to SQL and can be used in filters and sorts. + +```elixir +require Ash.Query + +Helpdesk.Support.Representative +|> Ash.Query.filter(open_tickets > 0) +|> Ash.Query.sort(total_tickets: :desc) +|> Ash.Query.load([:total_tickets, :open_tickets, :first_ticket_subject]) +|> Ash.read!() +``` + +You can also load individual aggregates after records have already been read. + +```elixir +representatives = Helpdesk.Support.read!(Helpdesk.Support.Representative) + +Ash.load!(representatives, [:open_tickets, :has_closed_tickets]) +``` + +Calculations can refer to aggregates, and those calculations can also be filtered, sorted, and loaded. + +```elixir +# in lib/helpdesk/support/resources/representative.ex + + calculations do + calculate :percent_open, :float, expr(open_tickets / total_tickets) + end +``` + +```elixir +require Ash.Query + +Helpdesk.Support.Representative +|> Ash.Query.filter(percent_open > 0.25) +|> Ash.Query.sort(:percent_open) +|> Ash.Query.load(:percent_open) +|> Ash.read!() +``` + +AshSqlite supports related `count`, `sum`, `avg`, `min`, `max`, `exists`, `first`, `list`, and `custom` aggregates over normal relationship paths, one-hop many-to-many relationship aggregates, scalar aggregates over multi-hop paths that end in a many-to-many relationship, and parent-independent unrelated aggregates. `first` and `list` aggregates require SQLite 3.30.0 or later with JSON functions enabled. ### Rich Configuration Options diff --git a/lib/custom_aggregate.ex b/lib/custom_aggregate.ex new file mode 100644 index 0000000..7a92114 --- /dev/null +++ b/lib/custom_aggregate.ex @@ -0,0 +1,30 @@ +# SPDX-FileCopyrightText: 2023 ash_sqlite contributors +# +# SPDX-License-Identifier: MIT + +defmodule AshSqlite.CustomAggregate do + @moduledoc """ + A custom aggregate implementation for Ecto queries against SQLite. + """ + + @doc """ + The dynamic expression to create the aggregate. + + The binding refers to the resource being aggregated. Use `as(^binding)` to + reference it. + + For example: + + Ecto.Query.dynamic( + [], + fragment("group_concat(?, ?)", field(as(^binding), ^opts[:field]), ^opts[:delimiter]) + ) + """ + @callback dynamic(opts :: Keyword.t(), binding :: integer) :: Ecto.Query.dynamic_expr() + + defmacro __using__(_) do + quote do + @behaviour AshSqlite.CustomAggregate + end + end +end diff --git a/lib/data_layer.ex b/lib/data_layer.ex index e3c2b6f..97e4436 100644 --- a/lib/data_layer.ex +++ b/lib/data_layer.ex @@ -467,12 +467,17 @@ defmodule AshSqlite.DataLayer do false end + def can?(_, {:aggregate, :unrelated}), do: true + def can?(_, {:exists, :unrelated}), do: true + def can?(_, :boolean_filter), do: true - def can?(_, {:aggregate, _type}), do: false + def can?(_, {:aggregate, type}) + when type in [:count, :sum, :avg, :max, :min, :exists, :first, :list, :custom], + do: true - def can?(_, :aggregate_filter), do: false - def can?(_, :aggregate_sort), do: false + def can?(_, :aggregate_filter), do: true + def can?(_, :aggregate_sort), do: true def can?(_, :expression_calculation), do: true def can?(_, :expression_calculation_sort), do: true def can?(_, :create), do: true @@ -496,7 +501,30 @@ defmodule AshSqlite.DataLayer do def can?(_, {:filter_relationship, _}), do: true - def can?(_, {:aggregate_relationship, _}), do: false + def can?(_, {:aggregate_relationship, %{manual: {_, _}}}), do: false + + def can?(_, {:aggregate_relationship, %{type: :many_to_many} = relationship}) do + join_relationship = + Ash.Resource.Info.relationship(relationship.source, relationship.join_relationship) + + not is_nil(join_relationship) && + not AshSql.Aggregate.Grouped.relationship_filter_uses_parent?(relationship) && + not AshSql.Aggregate.Grouped.relationship_filter_uses_parent?(join_relationship) && + can?(relationship.source, {:join, relationship.through}) && + can?(relationship.through, {:join, relationship.destination}) + end + + def can?(_, {:aggregate_relationship, %{no_attributes?: true}}), do: false + + def can?(_, {:aggregate_relationship, relationship}) + when not is_nil(relationship.filter) do + not AshSql.Aggregate.Grouped.relationship_filter_uses_parent?(relationship) && + can?(relationship.source, {:join, relationship.destination}) + end + + def can?(resource, {:aggregate_relationship, relationship}) do + can?(resource, {:join, relationship.destination}) + end def can?(_, :timeout), do: true def can?(_, {:filter_expr, %Ash.Query.Function.StringJoin{}}), do: false @@ -560,6 +588,37 @@ defmodule AshSqlite.DataLayer do {:ok, from(row in query, offset: ^offset)} end + @impl true + def return_query(query, resource) do + # AshSql.Query.return_query/2 also normalizes bindings. Do it here first so + # aggregate prebinding can inspect sort/load metadata before return_query + # consumes it. + query = + query + |> AshSql.Bindings.default_bindings(resource, AshSqlite.SqlImplementation) + + load_aggregates = query.__ash_bindings__[:load_aggregates] || [] + + query_without_aggregates = + Map.update!(query, :__ash_bindings__, &Map.put(&1, :load_aggregates, [])) + + with {:ok, query_without_aggregates} <- + AshSql.Aggregate.Grouped.add_sort_aggregates( + query_without_aggregates, + query_without_aggregates.__ash_bindings__[:sort], + resource + ), + {:ok, query} <- AshSql.Query.return_query(query_without_aggregates, resource) do + AshSql.Aggregate.add_aggregates( + query, + load_aggregates, + resource, + true, + query.__ash_bindings__.root_binding + ) + end + end + @impl true def run_aggregate_query(query, aggregates, resource) do AshSql.AggregateQuery.run_aggregate_query( @@ -576,7 +635,14 @@ defmodule AshSqlite.DataLayer do if query.__ash_bindings__[:sort_applied?] do {:ok, query} else - AshSql.Sort.apply_sort(query, query.__ash_bindings__[:sort], resource) + with {:ok, query} <- + AshSql.Aggregate.Grouped.add_sort_aggregates( + query, + query.__ash_bindings__[:sort], + resource + ) do + AshSql.Sort.apply_sort(query, query.__ash_bindings__[:sort], resource) + end end case with_sort_applied do @@ -609,7 +675,8 @@ defmodule AshSqlite.DataLayer do repo.all( query, opts - )} + ) + |> AshSql.Query.remap_mapped_fields(query)} end end rescue @@ -1981,20 +2048,63 @@ defmodule AshSqlite.DataLayer do @impl true def filter(query, filter, _resource, opts \\ []) do + used_aggregates = Ash.Filter.used_aggregates(filter, []) + query |> AshSql.Join.join_all_relationships(filter, opts) |> case do {:ok, query} -> - {:ok, AshSql.Filter.add_filter_expression(query, filter)} + query + |> AshSql.Aggregate.add_aggregates( + used_aggregates, + query.__ash_bindings__.resource, + false, + query.__ash_bindings__.root_binding + ) + |> case do + {:ok, query} -> + {:ok, AshSql.Filter.add_filter_expression(query, filter)} + + {:error, error} -> + {:error, error} + end {:error, error} -> {:error, error} end end + @impl true + def add_aggregates(query, aggregates, _resource) do + {:ok, + Map.update!(query, :__ash_bindings__, fn bindings -> + Map.put(bindings, :load_aggregates, aggregates) + end)} + end + @impl true def add_calculations(query, calculations, resource) do - AshSql.Calculation.add_calculations(query, calculations, resource, 0, true) + aggregates = + calculations + |> Enum.flat_map(fn {calculation, expression} -> + expression + |> Ash.Filter.used_aggregates([]) + |> Enum.map(&Map.put(&1, :context, calculation.context)) + end) + # Preserve context before deduping: identical calculation contexts share + # one aggregate binding, different contexts stay isolated. + |> Enum.uniq() + + with {:ok, query} <- + AshSql.Aggregate.add_aggregates( + query, + aggregates, + resource, + false, + query.__ash_bindings__.root_binding + ) do + AshSql.Calculation.add_calculations(query, calculations, resource, 0, true) + end end @doc false diff --git a/lib/sql_implementation.ex b/lib/sql_implementation.ex index 4029474..f4cd7da 100644 --- a/lib/sql_implementation.ex +++ b/lib/sql_implementation.ex @@ -9,6 +9,9 @@ defmodule AshSqlite.SqlImplementation do require Ecto.Query require Ash.Expr + @impl true + def aggregate_strategy(_resource), do: :grouped + @impl true def manual_relationship_function, do: :ash_sqlite_join diff --git a/mix.exs b/mix.exs index 5ba5752..569b916 100644 --- a/mix.exs +++ b/mix.exs @@ -88,6 +88,7 @@ defmodule AshSqlite.MixProject do "documentation/tutorials/getting-started-with-ash-sqlite.md", "documentation/topics/about-ash-sqlite/what-is-ash-sqlite.md", "documentation/topics/about-ash-sqlite/transactions.md", + "documentation/topics/resources/aggregates.md", "documentation/topics/resources/references.md", "documentation/topics/resources/polymorphic-resources.md", "documentation/topics/development/migrations-and-tasks.md", @@ -131,6 +132,9 @@ defmodule AshSqlite.MixProject do Types: [ AshSqlite.Type ], + "Custom Aggregates": [ + AshSqlite.CustomAggregate + ], Expressions: [ AshSqlite.Functions.Fragment, AshSqlite.Functions.Like diff --git a/test/aggregate_test.exs b/test/aggregate_test.exs index 6593479..2d3e2fa 100644 --- a/test/aggregate_test.exs +++ b/test/aggregate_test.exs @@ -6,7 +6,7 @@ defmodule AshSqlite.AggregatesTest do use AshSqlite.RepoCase, async: false require Ash.Query - alias AshSqlite.Test.Post + alias AshSqlite.Test.{Author, Comment, Post, PostLink, Profile, Rating} test "a count with a filter returns the appropriate value" do Ash.Seed.seed!(%Post{title: "foo"}) @@ -21,6 +21,39 @@ defmodule AshSqlite.AggregatesTest do assert count == 2 end + test "query aggregate kinds work" do + create_post!("query aggregate b", %{score: 1}) + create_post!("query aggregate a", %{score: 3}) + create_post!("query aggregate c", %{score: 5}) + + assert %{ + count: 3, + sum_score: 9, + max_score: 5, + min_score: 1, + avg_score: 3.0, + first_title: "query aggregate a", + exists_any: true + } = + Ash.aggregate!(Post, [ + {:count, :count}, + {:sum_score, :sum, field: :score}, + {:max_score, :max, field: :score}, + {:min_score, :min, field: :score}, + {:avg_score, :avg, field: :score}, + {:first_title, :first, field: :title, query: [sort: [title: :asc]]}, + {:exists_any, :exists} + ]) + + assert Post + |> Ash.Query.filter(title == "query aggregate b") + |> Ash.exists?() + + refute Post + |> Ash.Query.filter(title == "missing query aggregate") + |> Ash.exists?() + end + test "pagination returns the count" do Ash.Seed.seed!(%Post{title: "foo"}) Ash.Seed.seed!(%Post{title: "foo"}) @@ -31,4 +64,1090 @@ defmodule AshSqlite.AggregatesTest do |> Ash.Query.for_read(:paginated) |> Ash.read!() end + + test "paginated reads can count and load scalar aggregates" do + create_post!("paged aggregate a") + page_post = create_post!("paged aggregate b") + create_post!("paged aggregate c") + + create_comment!(page_post, "first", 1) + create_comment!(page_post, "second", 1) + + assert %Ash.Page.Offset{ + count: 3, + limit: 1, + offset: 1, + results: [ + %Post{title: "paged aggregate b", count_of_comments: 2} + ] + } = + Post + |> Ash.Query.for_read(:paginated) + |> Ash.Query.load(:count_of_comments) + |> Ash.Query.sort(:title) + |> Ash.Query.page(offset: 1, limit: 1, count: true) + |> Ash.read!() + end + + test "related scalar aggregates can be loaded" do + post = create_post!("loaded") + empty_post = create_post!("empty") + + create_comment!(post, "match", 1) + create_comment!(post, "other", 4) + create_comment!(post, "other", 10) + + loaded_post = + post + |> Ash.load!([ + :count_of_comments, + :count_of_popular_comments, + :count_of_comments_called_match, + :sum_of_comment_likes, + :sum_of_comment_likes_called_match, + :avg_comment_likes, + :min_comment_likes, + :max_comment_likes, + :has_comment_called_match + ]) + + assert loaded_post.count_of_comments == 3 + assert loaded_post.count_of_popular_comments == 0 + assert loaded_post.count_of_comments_called_match == 1 + assert loaded_post.sum_of_comment_likes == 15 + assert loaded_post.sum_of_comment_likes_called_match == 1 + assert loaded_post.avg_comment_likes == 5.0 + assert loaded_post.min_comment_likes == 1 + assert loaded_post.max_comment_likes == 10 + assert loaded_post.has_comment_called_match == true + + empty_post = + empty_post + |> Ash.load!([ + :count_of_comments, + :sum_of_comment_likes, + :avg_comment_likes, + :has_comment_called_match + ]) + + assert empty_post.count_of_comments == 0 + assert empty_post.sum_of_comment_likes == nil + assert empty_post.avg_comment_likes == nil + assert empty_post.has_comment_called_match == false + + assert [ + %Post{title: "empty", count_of_comments: 0}, + %Post{title: "loaded", count_of_comments: 3} + ] = + Post + |> Ash.Query.load(:count_of_comments) + |> Ash.Query.sort(:title) + |> Ash.read!() + end + + test "fieldless count aggregates use SQL count star" do + {:ok, query} = + Post + |> Ash.Query.load(:count_of_comments) + |> Ash.Query.data_layer_query() + + {sql, _params} = Ecto.Adapters.SQL.to_sql(:all, AshSqlite.TestRepo, query) + + assert sql =~ "count(*)" + end + + test "relationship filters are applied to loaded aggregates" do + post = create_post!("relationship filter") + + create_comment!(post, "quiet", 1) + create_comment!(post, "popular", 11) + + assert %{count_of_popular_comments: 1} = + Ash.load!(post, :count_of_popular_comments) + end + + test "resource queries can sort by related aggregates" do + one_comment = create_post!("one comment") + two_comments = create_post!("two comments") + no_comments = create_post!("no comments") + + create_comment!(one_comment, "only", 1) + create_comment!(two_comments, "first", 1) + create_comment!(two_comments, "second", 1) + + assert [ + %Post{id: two_comments_id, count_of_comments: 2}, + %Post{id: one_comment_id, count_of_comments: 1}, + %Post{id: no_comments_id, count_of_comments: 0} + ] = + Post + |> Ash.Query.load(:count_of_comments) + |> Ash.Query.sort(count_of_comments: :desc) + |> Ash.read!() + + assert two_comments_id == two_comments.id + assert one_comment_id == one_comment.id + assert no_comments_id == no_comments.id + end + + test "aggregate sorting works with pagination and aggregate filters" do + one_comment = create_post!("one comment") + two_comments = create_post!("two comments") + three_comments = create_post!("three comments") + create_post!("no comments") + + create_comment!(one_comment, "only", 1) + create_comment!(two_comments, "first", 1) + create_comment!(two_comments, "second", 1) + create_comment!(three_comments, "first", 1) + create_comment!(three_comments, "second", 1) + create_comment!(three_comments, "third", 1) + + assert [%Post{id: two_comments_id, count_of_comments: 2}] = + Post + |> Ash.Query.load(:count_of_comments) + |> Ash.Query.filter(count_of_comments > 0) + |> Ash.Query.sort(count_of_comments: :desc) + |> Ash.Query.limit(1) + |> Ash.Query.offset(1) + |> Ash.read!() + + assert two_comments_id == two_comments.id + end + + test "resource queries can filter on related aggregates" do + post = create_post!("with comments") + create_comment!(post, "match", 1) + create_comment!(post, "other", 1) + + create_post!("without comments") + + assert [%Post{id: post_id, count_of_comments: 2}] = + Post + |> Ash.Query.load(:count_of_comments) + |> Ash.Query.filter(count_of_comments > 1) + |> Ash.read!() + + assert post_id == post.id + end + + test "resource queries can filter and sort on related aggregates without loading them" do + one_comment = create_post!("one unloaded comment") + two_comments = create_post!("two unloaded comments") + create_post!("no unloaded comments") + + create_comment!(one_comment, "only", 1) + create_comment!(two_comments, "first", 1) + create_comment!(two_comments, "second", 1) + + assert [%Post{id: two_comments_id}, %Post{id: one_comment_id}] = + Post + |> Ash.Query.filter(count_of_comments > 0) + |> Ash.Query.sort(count_of_comments: :desc) + |> Ash.read!() + + assert two_comments_id == two_comments.id + assert one_comment_id == one_comment.id + end + + test "list loads related aggregates" do + post = create_post!("list load") + empty_post = create_post!("list load empty") + + create_comment!(post, "first", 1) + create_comment!(post, "second", 1) + + assert [ + %Post{id: post_id, count_of_comments: 2}, + %Post{id: empty_post_id, count_of_comments: 0} + ] = Ash.load!([post, empty_post], :count_of_comments) + + assert post_id == post.id + assert empty_post_id == empty_post.id + end + + test "aggregate join filters are applied on one-hop relationships" do + post = create_post!("join filter") + + create_comment!(post, "match", 1) + create_comment!(post, "other", 1) + + assert %{count_of_comments_with_join_filter: 1} = + Ash.load!(post, :count_of_comments_with_join_filter) + end + + test "same-path aggregates can use different read action filters" do + post = create_post!("read action aggregate") + + create_comment!(post, "low", 1) + create_comment!(post, "high", 10) + + assert %{count_of_comments: 2, count_of_liked_comments: 1} = + Ash.load!(post, [:count_of_comments, :count_of_liked_comments]) + end + + test "aggregate filters can reference relationships" do + post = create_post!("related aggregate filter") + + create_comment!(post, "first", 1) + create_comment!(post, "second", 1) + + assert %{count_of_comments_with_related_filter: 2} = + Ash.load!(post, :count_of_comments_with_related_filter) + end + + test "aggregate filters can reference related exists expressions" do + post = create_post!("related aggregate exists filter") + + create_comment!(post, "first", 1) + create_comment!(post, "second", 1) + + assert %{count_of_comments_with_related_exists_filter: 2} = + Ash.load!(post, :count_of_comments_with_related_exists_filter) + end + + test "aggregate filters over filtered to-many relationship refs do not corrupt siblings" do + post = create_post!("filtered related aggregate filter") + popular_comment = create_comment!(post, "popular", 1) + unpopular_comment = create_comment!(post, "unpopular", 1) + + create_comment_rating!(popular_comment, 10) + create_comment_rating!(popular_comment, 11) + create_comment_rating!(unpopular_comment, 1) + + assert %{ + count_of_comments: 2, + sum_of_comment_likes: 2, + count_of_comments_with_popular_ratings: 1 + } = + Ash.load!(post, [ + :count_of_comments, + :sum_of_comment_likes, + :count_of_comments_with_popular_ratings + ]) + end + + test "fieldless count filters over to-many refs count distinct aggregate rows" do + post = create_post!("distinct related aggregate filter") + popular_comment = create_comment!(post, "popular", 1) + unpopular_comment = create_comment!(post, "unpopular", 1) + + create_comment_rating!(popular_comment, 10) + create_comment_rating!(popular_comment, 11) + create_comment_rating!(unpopular_comment, 1) + + assert %{count_of_comments_with_popular_ratings: 1} = + Ash.load!(post, :count_of_comments_with_popular_ratings) + end + + test "exists filters avoid to-many fanout for sum aggregates" do + post = create_post!("exists fanout aggregate filter") + popular_comment = create_comment!(post, "popular", 4) + unpopular_comment = create_comment!(post, "unpopular", 6) + + create_comment_rating!(popular_comment, 10) + create_comment_rating!(popular_comment, 11) + create_comment_rating!(unpopular_comment, 1) + + assert %{sum_of_comment_likes_with_popular_ratings_exists: 4} = + Ash.load!(post, :sum_of_comment_likes_with_popular_ratings_exists) + end + + test "fanout-prone aggregate filters return stable unsupported errors" do + post = create_post!("fanout aggregate filter") + comment = create_comment!(post, "popular", 1) + create_comment_rating!(comment, 10) + create_comment_rating!(comment, 11) + + assert_raise Ash.Error.Unknown, ~r/sum, avg, list, custom, or field-based count/, fn -> + Ash.load!(post, :sum_of_comment_likes_with_popular_ratings) + end + + assert_raise Ash.Error.Unknown, ~r/sum, avg, list, custom, or field-based count/, fn -> + Ash.load!(post, :avg_comment_likes_with_popular_ratings) + end + + assert_raise Ash.Error.Unknown, ~r/list, custom, or field-based count aggregates/, fn -> + Ash.load!(post, :comment_titles_with_popular_ratings) + end + + assert_raise Ash.Error.Unknown, ~r/list, custom, or field-based count aggregates/, fn -> + Ash.load!(post, :comment_titles_joined_with_popular_ratings) + end + + assert_raise Ash.Error.Unknown, ~r/list, custom, or field-based count aggregates/, fn -> + Ash.load!(post, :count_comment_titles_with_popular_ratings) + end + end + + test "aggregate filters using parent expressions return a stable unsupported error" do + post = create_post!("same") + create_comment!(post, "same", 1) + + assert_raise Ash.Error.Unknown, ~r/parent-dependent aggregate filters/, fn -> + Ash.load!(post, :count_of_comments_matching_post_title) + end + end + + test "parent-dependent aggregate join filters return a stable unsupported error" do + post = create_post!("parent join") + create_comment!(post, "parent join", 1) + + assert_raise Ash.Error.Unknown, ~r/parent-dependent join filters/, fn -> + Ash.load!(post, :count_of_comments_with_parent_join_filter) + end + end + + test "aggregate filters that reference aggregates return a stable unsupported error" do + post = create_post!("aggregate filter") + create_comment!(post, "comment", 1) + + assert_raise Ash.Error.Unknown, ~r/filters that reference other aggregates/, fn -> + Ash.load!(post, :count_of_comments_with_aggregate_filter) + end + end + + test "multi-hop aggregate relationships can be loaded through normal paths" do + post = create_post!("post multi-hop") + comment = create_comment!(post, "comment", 1) + create_comment_rating!(comment, 7) + + assert %{count_of_comment_ratings: 1} = + Ash.load!(post, :count_of_comment_ratings) + end + + test "first aggregates can be loaded" do + post = create_post!("first aggregate") + empty_post = create_post!("first aggregate empty") + + create_comment!(post, nil, 1) + create_comment!(post, "bbb", 1) + create_comment!(post, "aaa", 1) + create_comment!(post, "stuff", 1) + + loaded_post = + Ash.load!(post, [ + :first_comment, + :first_comment_nils_first, + :first_comment_nils_first_called_stuff, + :first_comment_nils_first_include_nil + ]) + + assert loaded_post.first_comment == "aaa" + assert loaded_post.first_comment_nils_first == "aaa" + assert loaded_post.first_comment_nils_first_called_stuff == "stuff" + assert loaded_post.first_comment_nils_first_include_nil == nil + + assert %{first_comment: nil} = Ash.load!(empty_post, :first_comment) + end + + test "first aggregates can be sorted and used over belongs_to and multi-hop paths" do + author = create_author!("Belongs", "To") + author_post = create_post_for_author!(author, "belongs to first") + + low = create_post!("low first") + high = create_post!("high first") + + create_comment!(low, "aaa", 1) + create_comment!(high, "zzz", 1) + + assert [ + %Post{id: low_id, first_comment: "aaa"}, + %Post{id: high_id, first_comment: "zzz"} + ] = + Post + |> Ash.Query.load(:first_comment) + |> Ash.Query.filter(count_of_comments > 0) + |> Ash.Query.sort(first_comment: :asc) + |> Ash.read!() + + assert low_id == low.id + assert high_id == high.id + + assert %{author_first_name: "Belongs"} = Ash.load!(author_post, :author_first_name) + + comment = create_comment!(high, "rated", 1) + create_comment_rating!(comment, 3) + create_comment_rating!(comment, 10) + + assert %{highest_rating: 10} = Ash.load!(high, :highest_rating) + end + + test "list aggregates can be loaded" do + post = create_post!("list aggregate") + empty_post = create_post!("list aggregate empty") + + first = create_comment!(post, "bbb", 1) + create_comment!(post, nil, 1) + create_comment!(post, "aaa", 7) + create_comment!(post, "aaa", 9) + + loaded_post = + Ash.load!(post, [ + :comment_titles, + :comment_titles_with_empty_default, + :comment_titles_with_string_default, + :comment_titles_with_nils, + :uniq_comment_titles, + :comment_titles_with_5_likes, + :comment_ids, + :comment_ids_with_default, + :comment_likes_with_integer_default + ]) + + assert loaded_post.comment_titles == ["aaa", "aaa", "bbb"] + assert loaded_post.comment_titles_with_empty_default == ["aaa", "aaa", "bbb"] + assert loaded_post.comment_titles_with_string_default == ["aaa", "aaa", "bbb"] + assert loaded_post.comment_titles_with_nils == ["aaa", "aaa", "bbb", nil] + assert loaded_post.uniq_comment_titles == ["aaa", "bbb"] + assert loaded_post.comment_titles_with_5_likes == ["aaa", "aaa"] + assert first.id in loaded_post.comment_ids + assert first.id in loaded_post.comment_ids_with_default + assert loaded_post.comment_likes_with_integer_default == [1, 1, 7, 9] + + assert %{ + comment_titles: [], + comment_titles_with_empty_default: [], + comment_titles_with_string_default: ["fallback"], + comment_ids_with_default: ["11111111-1111-1111-1111-111111111111"], + comment_likes_with_integer_default: [42] + } = + Ash.load!(empty_post, [ + :comment_titles, + :comment_titles_with_empty_default, + :comment_titles_with_string_default, + :comment_ids_with_default, + :comment_likes_with_integer_default + ]) + end + + test "custom aggregates can use sqlite-specific implementations" do + post = create_post!("custom aggregate") + + create_comment!(post, "aaa", 2) + create_comment!(post, "bbb", 3) + + assert %{comment_titles_joined: joined, total_comment_likes_custom: total} = + Ash.load!(post, [:comment_titles_joined, :total_comment_likes_custom]) + + assert joined |> String.split(",") |> Enum.sort() == ["aaa", "bbb"] + assert total == 5.0 + assert is_float(total) + end + + test "unrelated aggregates without parent filters can be loaded" do + first_author = create_author!("first", "author") + second_author = create_author!("second", "author") + + create_profile!("bbb") + create_profile!("aaa") + create_profile!(nil) + + create_post!("scored one", %{score: 2}) + create_post!("scored two", %{score: 3}) + + loaded_authors = + [first_author, second_author] + |> Ash.load!([ + :total_profiles, + :total_profiles_plus_one, + :total_post_score, + :avg_post_score, + :min_post_score, + :max_post_score, + :has_any_profile, + :first_profile_description, + :profile_descriptions, + :post_titles_joined + ]) + + assert [ + %Author{ + id: first_author_id, + total_profiles: 3, + total_profiles_plus_one: 4, + total_post_score: 5, + avg_post_score: 2.5, + min_post_score: 2, + max_post_score: 3, + has_any_profile: true, + first_profile_description: "aaa", + profile_descriptions: ["aaa", "bbb"] + } = loaded_first_author, + %Author{ + id: second_author_id, + total_profiles: 3, + total_profiles_plus_one: 4, + total_post_score: 5, + avg_post_score: 2.5, + min_post_score: 2, + max_post_score: 3, + has_any_profile: true, + first_profile_description: "aaa", + profile_descriptions: ["aaa", "bbb"] + } = loaded_second_author + ] = loaded_authors + + assert first_author_id == first_author.id + assert second_author_id == second_author.id + + assert loaded_first_author.post_titles_joined |> String.split(",") |> Enum.sort() == [ + "scored one", + "scored two" + ] + + assert loaded_second_author.post_titles_joined |> String.split(",") |> Enum.sort() == [ + "scored one", + "scored two" + ] + end + + test "unsupported aggregate relationship shapes return stable errors" do + manual_relationship = Ash.Resource.Info.relationship(Post, :comments_containing_title) + no_attributes_relationship = Ash.Resource.Info.relationship(Post, :posts_with_matching_title) + + parent_filter_relationship = + Ash.Resource.Info.relationship(Post, :comments_matching_post_title) + + refute AshSqlite.DataLayer.can?(Post, {:aggregate_relationship, manual_relationship}) + refute AshSqlite.DataLayer.can?(Post, {:aggregate_relationship, no_attributes_relationship}) + refute AshSqlite.DataLayer.can?(Post, {:aggregate_relationship, parent_filter_relationship}) + end + + test "parent-dependent unrelated aggregate filters return a stable unsupported error" do + author = create_author!("parent", "unrelated") + create_profile!("parent") + + assert_raise Ash.Error.Unknown, ~r/parent-dependent aggregate filters/, fn -> + Ash.load!(author, :profiles_matching_first_name) + end + end + + test "calculations can reference related aggregates" do + post = create_post!("with aggregate calculation", %{score: 3}) + empty_post = create_post!("without aggregate calculation", %{score: 7}) + + create_comment!(post, "first", 4) + create_comment!(post, "second", 6) + + assert [ + %Post{ + id: post_id, + has_comments: true, + comment_likes_with_score: 13 + }, + %Post{ + id: empty_post_id, + has_comments: false, + comment_likes_with_score: 7 + } + ] = + Post + |> Ash.Query.load([:has_comments, :comment_likes_with_score]) + |> Ash.Query.sort(comment_likes_with_score: :desc) + |> Ash.read!() + + assert post_id == post.id + assert empty_post_id == empty_post.id + end + + test "many_to_many scalar aggregates can be loaded" do + source = create_post!("source", %{score: 5}) + match = create_post!("match", %{score: 2}) + other = create_post!("other", %{score: 6}) + archived = create_post!("archived", %{score: 20}) + empty = create_post!("empty", %{score: 1}) + + link_posts!(source, [match, other]) + create_post_link!(source, archived, :archived) + + loaded_source = + Ash.load!(source, [ + :count_of_linked_posts, + :sum_of_linked_post_scores, + :avg_linked_post_score, + :min_linked_post_score, + :max_linked_post_score, + :has_linked_post_called_match + ]) + + assert loaded_source.count_of_linked_posts == 2 + assert loaded_source.sum_of_linked_post_scores == 8 + assert loaded_source.avg_linked_post_score == 4.0 + assert loaded_source.min_linked_post_score == 2 + assert loaded_source.max_linked_post_score == 6 + assert loaded_source.has_linked_post_called_match == true + + loaded_empty = + Ash.load!(empty, [ + :count_of_linked_posts, + :sum_of_linked_post_scores, + :avg_linked_post_score, + :has_linked_post_called_match + ]) + + assert loaded_empty.count_of_linked_posts == 0 + assert loaded_empty.sum_of_linked_post_scores == nil + assert loaded_empty.avg_linked_post_score == nil + assert loaded_empty.has_linked_post_called_match == false + end + + test "many_to_many aggregates with filters that require joins can be loaded" do + source = create_post!("source") + author = create_author!("John", "Doe") + linked = create_post_for_author!(author, "linked") + + link_posts!(source, [linked]) + + assert %{count_of_linked_posts_with_author: 1} = + Ash.load!(source, :count_of_linked_posts_with_author) + end + + test "many_to_many aggregate filters that require joins work in parent queries" do + first_source = create_post!("first source") + second_source = create_post!("second source") + create_post!("no links") + + author = create_author!("Jane", "Doe") + linked_with_author = create_post_for_author!(author, "linked with author") + linked_without_author = create_post!("linked without author") + + link_posts!(first_source, [linked_with_author, linked_without_author]) + link_posts!(second_source, [linked_with_author]) + + assert [ + %Post{id: first_source_id, count_of_linked_posts_with_author: 1}, + %Post{id: second_source_id, count_of_linked_posts_with_author: 1} + ] = + Post + |> Ash.Query.load(:count_of_linked_posts_with_author) + |> Ash.Query.filter(count_of_linked_posts_with_author > 0) + |> Ash.Query.sort(title: :asc) + |> Ash.read!() + + assert first_source_id == first_source.id + assert second_source_id == second_source.id + end + + test "many_to_many first and list aggregates can be loaded" do + source = create_post!("m2m window source") + empty = create_post!("m2m window empty") + first = create_post!("bbb") + second = create_post!("ccc") + archived = create_post!("aaa") + + link_posts!(source, [second, first]) + create_post_link!(source, archived, :archived) + + assert %{ + first_linked_post_title: "bbb", + linked_post_titles: ["bbb", "ccc"] + } = + Ash.load!(source, [ + :first_linked_post_title, + :linked_post_titles + ]) + + assert %{ + first_linked_post_title: nil, + linked_post_titles: [] + } = + Ash.load!(empty, [ + :first_linked_post_title, + :linked_post_titles + ]) + end + + test "many_to_many first and list aggregates with joined filters can be loaded" do + source = create_post!("m2m joined window source") + author = create_author!("Window", "Author") + without_author = create_post!("aaa") + with_author = create_post_for_author!(author, "bbb") + with_author_later = create_post_for_author!(author, "ccc") + + link_posts!(source, [without_author, with_author_later, with_author]) + + assert %{ + first_linked_post_title_with_author: "bbb", + linked_post_titles_with_author: ["bbb", "ccc"], + first_linked_post_title_with_author_join_filter: "bbb", + linked_post_titles_with_author_join_filter: ["bbb", "ccc"] + } = + Ash.load!(source, [ + :first_linked_post_title_with_author, + :linked_post_titles_with_author, + :first_linked_post_title_with_author_join_filter, + :linked_post_titles_with_author_join_filter + ]) + end + + test "many_to_many custom aggregates can be loaded" do + source = create_post!("m2m custom source") + empty = create_post!("m2m custom empty") + first = create_post!("aaa") + second = create_post!("bbb") + archived = create_post!("ccc") + + link_posts!(source, [second, first]) + create_post_link!(source, archived, :archived) + + assert %{linked_post_titles_joined: joined} = + Ash.load!(source, :linked_post_titles_joined) + + assert joined |> String.split(",") |> Enum.sort() == ["aaa", "bbb"] + + assert %{linked_post_titles_joined: nil} = + Ash.load!(empty, :linked_post_titles_joined) + end + + test "many_to_many aggregates can be filtered, sorted and used in calculations" do + one_link = create_post!("one link", %{score: 1}) + two_links = create_post!("two links", %{score: 2}) + no_links = create_post!("no links", %{score: 3}) + + linked_a = create_post!("linked a", %{score: 4}) + linked_b = create_post!("linked b", %{score: 5}) + + link_posts!(one_link, [linked_a]) + link_posts!(two_links, [linked_a, linked_b]) + + assert [ + %Post{ + id: two_links_id, + count_of_linked_posts: 2, + linked_post_score_with_score: 11 + }, + %Post{ + id: one_link_id, + count_of_linked_posts: 1, + linked_post_score_with_score: 5 + } + ] = + Post + |> Ash.Query.load([ + :count_of_linked_posts, + :linked_post_score_with_score + ]) + |> Ash.Query.filter(count_of_linked_posts > 0) + |> Ash.Query.sort(count_of_linked_posts: :desc) + |> Ash.read!() + + assert two_links_id == two_links.id + assert one_link_id == one_link.id + + assert %{linked_post_score_with_score: 3} = + Ash.load!(no_links, :linked_post_score_with_score) + end + + test "aggregate join filters are applied on many_to_many relationships" do + source = create_post!("m2m join filter source") + match = create_post!("match") + other = create_post!("other") + + link_posts!(source, [match, other]) + + assert %{count_of_linked_posts_with_join_filter: 1} = + Ash.load!(source, :count_of_linked_posts_with_join_filter) + end + + test "multi-hop scalar aggregates can be loaded" do + author = create_author!("multi", "hop") + empty_author = create_author!("empty", "author") + + first_post = create_post_for_author!(author, "first post") + second_post = create_post_for_author!(author, "second post") + + create_comment!(first_post, "match", 1) + create_comment!(first_post, "other", 4) + create_comment!(second_post, "other", 10) + + loaded_author = + Ash.load!(author, [ + :count_of_comments_through_posts, + :sum_of_comment_likes_through_posts, + :avg_comment_likes_through_posts, + :min_comment_likes_through_posts, + :max_comment_likes_through_posts, + :has_comment_called_match_through_posts + ]) + + assert loaded_author.count_of_comments_through_posts == 3 + assert loaded_author.sum_of_comment_likes_through_posts == 15 + assert loaded_author.avg_comment_likes_through_posts == 5.0 + assert loaded_author.min_comment_likes_through_posts == 1 + assert loaded_author.max_comment_likes_through_posts == 10 + assert loaded_author.has_comment_called_match_through_posts == true + + loaded_empty = + Ash.load!(empty_author, [ + :count_of_comments_through_posts, + :sum_of_comment_likes_through_posts, + :avg_comment_likes_through_posts, + :has_comment_called_match_through_posts + ]) + + assert loaded_empty.count_of_comments_through_posts == 0 + assert loaded_empty.sum_of_comment_likes_through_posts == nil + assert loaded_empty.avg_comment_likes_through_posts == nil + assert loaded_empty.has_comment_called_match_through_posts == false + end + + test "multi-hop list and custom aggregates can be loaded" do + author = create_author!("multi", "list") + empty_author = create_author!("multi", "list empty") + + first_post = create_post_for_author!(author, "first post") + second_post = create_post_for_author!(author, "second post") + + create_comment!(first_post, "bbb", 1) + create_comment!(second_post, "aaa", 1) + + assert %{ + comment_titles_through_posts: ["aaa", "bbb"], + comment_titles_joined_through_posts: joined + } = + Ash.load!(author, [ + :comment_titles_through_posts, + :comment_titles_joined_through_posts + ]) + + assert joined |> String.split(",") |> Enum.sort() == ["aaa", "bbb"] + + assert %{ + comment_titles_through_posts: [], + comment_titles_joined_through_posts: nil + } = + Ash.load!(empty_author, [ + :comment_titles_through_posts, + :comment_titles_joined_through_posts + ]) + end + + test "multi-hop aggregates can be filtered, sorted and used in calculations" do + one_comment = create_author!("one", "comment") + two_comments = create_author!("two", "comments") + no_comments = create_author!("no", "comments") + + one_post = create_post_for_author!(one_comment, "one post") + two_post = create_post_for_author!(two_comments, "two post") + + create_comment!(one_post, "only", 4) + create_comment!(two_post, "first", 5) + create_comment!(two_post, "second", 6) + + assert [ + %Author{ + id: two_comments_id, + count_of_comments_through_posts: 2, + comment_likes_through_posts_plus_one: 12 + }, + %Author{ + id: one_comment_id, + count_of_comments_through_posts: 1, + comment_likes_through_posts_plus_one: 5 + } + ] = + Author + |> Ash.Query.load([ + :count_of_comments_through_posts, + :comment_likes_through_posts_plus_one + ]) + |> Ash.Query.filter(count_of_comments_through_posts > 0) + |> Ash.Query.sort(count_of_comments_through_posts: :desc) + |> Ash.read!() + + assert two_comments_id == two_comments.id + assert one_comment_id == one_comment.id + + assert %{comment_likes_through_posts_plus_one: 1} = + Ash.load!(no_comments, :comment_likes_through_posts_plus_one) + end + + test "aggregate join filters are applied on multi-hop relationships" do + author = create_author!("multi", "join filter") + public_post = create_post_for_author!(author, "public post", %{public: true}) + private_post = create_post_for_author!(author, "private post", %{public: false}) + + create_comment!(public_post, "match", 1) + create_comment!(public_post, "other", 1) + create_comment!(private_post, "match", 1) + + loaded_author = + Ash.load!(author, [ + :count_of_comments_on_public_posts, + :count_of_comments_called_match_with_join_filter + ]) + + assert loaded_author.count_of_comments_on_public_posts == 2 + assert loaded_author.count_of_comments_called_match_with_join_filter == 2 + end + + test "intermediate read action filters are applied on multi-hop aggregates" do + author = create_author!("multi", "read action") + public_post = create_post_for_author!(author, "public action post", %{public: true}) + private_post = create_post_for_author!(author, "private action post", %{public: false}) + + create_comment!(public_post, "public", 1) + create_comment!(private_post, "private", 1) + + assert %{count_of_comments_through_public_posts: 1} = + Ash.load!(author, :count_of_comments_through_public_posts) + end + + test "multi-hop scalar aggregates ending in many_to_many relationships can be loaded" do + author = create_author!("multi", "m2m") + empty_author = create_author!("empty", "m2m") + + public_post = create_post_for_author!(author, "public post", %{public: true}) + private_post = create_post_for_author!(author, "private post", %{public: false}) + + match = create_post!("match", %{score: 2}) + other = create_post!("other", %{score: 6}) + private = create_post!("private", %{score: 10}) + archived = create_post!("archived", %{score: 20}) + + link_posts!(public_post, [match, other]) + link_posts!(private_post, [private]) + create_post_link!(private_post, archived, :archived) + + loaded_author = + Ash.load!(author, [ + :count_of_linked_posts_through_posts, + :sum_of_linked_post_scores_through_posts, + :avg_linked_post_score_through_posts, + :min_linked_post_score_through_posts, + :max_linked_post_score_through_posts, + :has_linked_post_called_match_through_posts + ]) + + assert loaded_author.count_of_linked_posts_through_posts == 3 + assert loaded_author.sum_of_linked_post_scores_through_posts == 18 + assert loaded_author.avg_linked_post_score_through_posts == 6.0 + assert loaded_author.min_linked_post_score_through_posts == 2 + assert loaded_author.max_linked_post_score_through_posts == 10 + assert loaded_author.has_linked_post_called_match_through_posts == true + + loaded_empty = + Ash.load!(empty_author, [ + :count_of_linked_posts_through_posts, + :sum_of_linked_post_scores_through_posts, + :avg_linked_post_score_through_posts, + :has_linked_post_called_match_through_posts + ]) + + assert loaded_empty.count_of_linked_posts_through_posts == 0 + assert loaded_empty.sum_of_linked_post_scores_through_posts == nil + assert loaded_empty.avg_linked_post_score_through_posts == nil + assert loaded_empty.has_linked_post_called_match_through_posts == false + end + + test "multi-hop many_to_many scalar aggregates work in parent queries" do + one_link = create_author!("one", "m2m") + two_links = create_author!("two", "m2m") + create_author!("none", "m2m") + + one_post = create_post_for_author!(one_link, "one post") + two_post = create_post_for_author!(two_links, "two post") + + linked_a = create_post!("linked a", %{score: 4}) + linked_b = create_post!("linked b", %{score: 5}) + + link_posts!(one_post, [linked_a]) + link_posts!(two_post, [linked_a, linked_b]) + + assert [ + %Author{ + id: two_links_id, + count_of_linked_posts_through_posts: 2, + linked_post_score_through_posts_plus_one: 10 + }, + %Author{ + id: one_link_id, + count_of_linked_posts_through_posts: 1, + linked_post_score_through_posts_plus_one: 5 + } + ] = + Author + |> Ash.Query.load([ + :count_of_linked_posts_through_posts, + :linked_post_score_through_posts_plus_one + ]) + |> Ash.Query.filter(count_of_linked_posts_through_posts > 0) + |> Ash.Query.sort(count_of_linked_posts_through_posts: :desc) + |> Ash.read!() + + assert two_links_id == two_links.id + assert one_link_id == one_link.id + end + + test "unsupported multi-hop many_to_many aggregate shapes return stable errors" do + author = create_author!("multi", "m2m unsupported") + post = create_post_for_author!(author, "post") + linked_post = create_post!("linked") + + link_posts!(post, [linked_post]) + + assert_raise Ash.Error.Unknown, ~r/multi-hop paths that include many_to_many/, fn -> + Ash.load!(post, :count_of_comments_through_linked_posts) + end + + assert_raise Ash.Error.Unknown, ~r/multi-hop paths that include many_to_many/, fn -> + Ash.load!(author, :linked_post_titles_through_posts) + end + end + + defp create_post!(title, attrs \\ %{}) do + Post + |> Ash.Changeset.for_create(:create, Map.put(attrs, :title, title)) + |> Ash.create!() + end + + defp create_author!(first_name, last_name) do + Author + |> Ash.Changeset.for_create(:create, %{first_name: first_name, last_name: last_name}) + |> Ash.create!() + end + + defp create_post_for_author!(author, title, attrs \\ %{}) do + Post + |> Ash.Changeset.for_create(:create, Map.put(attrs, :title, title)) + |> Ash.Changeset.manage_relationship(:author, author, type: :append_and_remove) + |> Ash.create!() + end + + defp create_profile!(description) do + Profile + |> Ash.Changeset.for_create(:create, %{description: description}) + |> Ash.create!() + end + + defp create_comment!(post, title, likes, attrs \\ %{}) do + Comment + |> Ash.Changeset.for_create(:create, Map.merge(attrs, %{title: title, likes: likes})) + |> Ash.Changeset.manage_relationship(:post, post, type: :append_and_remove) + |> Ash.create!() + end + + defp create_comment_rating!(comment, score) do + Rating + |> Ash.Changeset.for_create(:create, %{score: score, resource_id: comment.id}) + |> Ash.Changeset.set_context(%{data_layer: %{table: "comment_ratings"}}) + |> Ash.create!() + end + + defp link_posts!(source, destinations) do + source + |> Ash.Changeset.new() + |> Ash.Changeset.manage_relationship(:linked_posts, destinations, type: :append_and_remove) + |> Ash.update!() + end + + defp create_post_link!(source, destination, state) do + PostLink + |> Ash.Changeset.new() + |> Ash.Changeset.change_attribute(:state, state) + |> Ash.Changeset.manage_relationship(:source_post, source, type: :append) + |> Ash.Changeset.manage_relationship(:destination_post, destination, type: :append) + |> Ash.create!() + end end diff --git a/test/calculation_test.exs b/test/calculation_test.exs index 48d6a06..c08dfc8 100644 --- a/test/calculation_test.exs +++ b/test/calculation_test.exs @@ -357,7 +357,7 @@ defmodule AshSqlite.CalculationTest do |> Ash.create!() end) - assert_raise Ash.Error.Invalid, ~r/does not support using aggregates/, fn -> + assert_raise Ash.Error.Unknown, ~r/only supports loading related/, fn -> Ash.load!(author, :post_titles) end end diff --git a/test/support/resources/author.ex b/test/support/resources/author.ex index fc83280..ad6c30c 100644 --- a/test/support/resources/author.ex +++ b/test/support/resources/author.ex @@ -29,6 +29,74 @@ defmodule AshSqlite.Test.Author do relationships do has_one(:profile, AshSqlite.Test.Profile, public?: true) has_many(:posts, AshSqlite.Test.Post, public?: true) + has_many(:public_posts, AshSqlite.Test.Post, public?: true, read_action: :public) + end + + aggregates do + count(:count_of_comments_through_posts, [:posts, :comments]) + count(:count_of_comments_through_public_posts, [:public_posts, :comments]) + count(:count_of_linked_posts_through_posts, [:posts, :linked_posts]) + count(:total_profiles, AshSqlite.Test.Profile) + sum(:total_post_score, AshSqlite.Test.Post, :score) + avg(:avg_post_score, AshSqlite.Test.Post, :score) + min(:min_post_score, AshSqlite.Test.Post, :score) + max(:max_post_score, AshSqlite.Test.Post, :score) + sum(:sum_of_comment_likes_through_posts, [:posts, :comments], :likes) + avg(:avg_comment_likes_through_posts, [:posts, :comments], :likes) + min(:min_comment_likes_through_posts, [:posts, :comments], :likes) + max(:max_comment_likes_through_posts, [:posts, :comments], :likes) + sum(:sum_of_linked_post_scores_through_posts, [:posts, :linked_posts], :score) + avg(:avg_linked_post_score_through_posts, [:posts, :linked_posts], :score) + min(:min_linked_post_score_through_posts, [:posts, :linked_posts], :score) + max(:max_linked_post_score_through_posts, [:posts, :linked_posts], :score) + + count :count_of_comments_on_public_posts, [:posts, :comments] do + join_filter(:posts, expr(public == true)) + end + + count :count_of_comments_called_match_with_join_filter, [:posts, :comments] do + join_filter([:posts, :comments], expr(title == "match")) + end + + exists :has_comment_called_match_through_posts, [:posts, :comments] do + filter(expr(title == "match")) + end + + exists :has_linked_post_called_match_through_posts, [:posts, :linked_posts] do + filter(expr(title == "match")) + end + + exists :has_any_profile, AshSqlite.Test.Profile do + filter(expr(not is_nil(description))) + end + + count :profiles_matching_first_name, AshSqlite.Test.Profile do + filter(expr(description == parent(first_name))) + end + + first :first_profile_description, AshSqlite.Test.Profile, :description do + sort(description: :asc_nils_last) + end + + list :profile_descriptions, AshSqlite.Test.Profile, :description do + sort(description: :asc_nils_last) + end + + list :comment_titles_through_posts, [:posts, :comments], :title do + sort(title: :asc_nils_last) + end + + list :linked_post_titles_through_posts, [:posts, :linked_posts], :title do + sort(title: :asc_nils_last) + end + + custom(:post_titles_joined, AshSqlite.Test.Post, :string) do + implementation({AshSqlite.Test.StringAgg, field: :title, delimiter: ","}) + end + + custom(:comment_titles_joined_through_posts, [:posts, :comments], :string) do + implementation({AshSqlite.Test.StringAgg, field: :title, delimiter: ","}) + end end calculations do @@ -76,5 +144,19 @@ defmodule AshSqlite.Test.Author do end calculate(:post_titles, {:array, :string}, expr(list(posts, field: :title))) + + calculate( + :comment_likes_through_posts_plus_one, + :integer, + expr((sum_of_comment_likes_through_posts || 0) + 1) + ) + + calculate( + :linked_post_score_through_posts_plus_one, + :integer, + expr((sum_of_linked_post_scores_through_posts || 0) + 1) + ) + + calculate(:total_profiles_plus_one, :integer, expr(total_profiles + 1)) end end diff --git a/test/support/resources/comment.ex b/test/support/resources/comment.ex index ca2dc6b..25374ae 100644 --- a/test/support/resources/comment.ex +++ b/test/support/resources/comment.ex @@ -31,6 +31,12 @@ defmodule AshSqlite.Test.Comment do default_accept(:*) defaults([:read, :update, :destroy]) + read :liked do + filter(expr(likes > 5)) + end + + read(:public) + create :create do primary?(true) argument(:rating, :map) @@ -64,4 +70,8 @@ defmodule AshSqlite.Test.Comment do filter: expr(score > 5) ) end + + aggregates do + count(:count_of_ratings, :ratings) + end end diff --git a/test/support/resources/post.ex b/test/support/resources/post.ex index 6b283cb..5dbe616 100644 --- a/test/support/resources/post.ex +++ b/test/support/resources/post.ex @@ -43,6 +43,10 @@ defmodule AshSqlite.Test.Post do pagination(offset?: true, required?: true) end + read :public do + filter(expr(public == true)) + end + create :create do primary?(true) argument(:rating, :map) @@ -172,9 +176,229 @@ defmodule AshSqlite.Test.Post do validate(attribute_does_not_equal(:title, "not allowed")) end + aggregates do + count(:count_of_comments, :comments) + count(:count_of_popular_comments, :popular_comments) + count(:count_of_linked_posts, :linked_posts) + count(:count_of_comments_through_linked_posts, [:linked_posts, :comments]) + count(:count_of_liked_comments, :comments, read_action: :liked) + count(:count_of_comment_ratings, [:comments, :ratings]) + sum(:sum_of_comment_likes, :comments, :likes) + sum(:sum_of_comment_likes_called_match, :comments, :likes, filter: expr(title == "match")) + + sum(:sum_of_comment_likes_with_popular_ratings, :comments, :likes) do + filter(expr(not is_nil(popular_ratings.id))) + end + + sum(:sum_of_comment_likes_with_popular_ratings_exists, :comments, :likes) do + filter(expr(exists(popular_ratings, score > 5))) + end + + sum(:sum_of_linked_post_scores, :linked_posts, :score) + avg(:avg_comment_likes, :comments, :likes) + + avg(:avg_comment_likes_with_popular_ratings, :comments, :likes) do + filter(expr(not is_nil(popular_ratings.id))) + end + + avg(:avg_linked_post_score, :linked_posts, :score) + min(:min_comment_likes, :comments, :likes) + min(:min_linked_post_score, :linked_posts, :score) + max(:max_comment_likes, :comments, :likes) + max(:max_linked_post_score, :linked_posts, :score) + + first :first_comment, :comments, :title do + sort(title: :asc_nils_last) + end + + first :first_comment_nils_first, :comments, :title do + sort(title: :asc_nils_first) + end + + first :first_comment_nils_first_called_stuff, :comments, :title do + sort(title: :asc_nils_first) + filter(expr(title == "stuff")) + end + + first :first_comment_nils_first_include_nil, :comments, :title do + include_nil?(true) + sort(title: :asc_nils_first) + end + + first :last_comment, :comments, :title do + sort(title: :desc) + end + + first :latest_comment_created_at, :comments, :created_at do + sort(created_at: :desc) + end + + first :highest_rating, [:comments, :ratings], :score do + sort(score: :desc) + end + + first(:author_first_name, :author, :first_name) + + first :first_linked_post_title, :linked_posts, :title do + sort(title: :asc_nils_last) + end + + first :first_linked_post_title_with_author, :linked_posts, :title do + sort(title: :asc_nils_last) + filter(expr(not is_nil(author.id))) + end + + first :first_linked_post_title_with_author_join_filter, :linked_posts, :title do + sort(title: :asc_nils_last) + join_filter(:linked_posts, expr(not is_nil(author.id))) + end + + list :comment_titles, :comments, :title do + sort(title: :asc_nils_last) + end + + list :comment_titles_with_empty_default, :comments, :title do + default([]) + sort(title: :asc_nils_last) + end + + list :comment_titles_with_string_default, :comments, :title do + default(["fallback"]) + sort(title: :asc_nils_last) + end + + list :comment_titles_with_nils, :comments, :title do + sort(title: :asc_nils_last) + include_nil?(true) + end + + list :uniq_comment_titles, :comments, :title do + uniq?(true) + sort(title: :asc_nils_last) + end + + list :comment_titles_with_5_likes, :comments, :title do + sort(title: :asc_nils_last) + filter(expr(likes >= 5)) + end + + list :comment_titles_with_popular_ratings, :comments, :title do + sort(title: :asc_nils_last) + filter(expr(not is_nil(popular_ratings.id))) + end + + list(:comment_ids, :comments, :id) + + list :comment_ids_with_default, :comments, :id do + default(["11111111-1111-1111-1111-111111111111"]) + end + + list :comment_likes_with_integer_default, :comments, :likes do + default([42]) + end + + list :linked_post_titles, :linked_posts, :title do + sort(title: :asc_nils_last) + end + + list :linked_post_titles_with_author, :linked_posts, :title do + sort(title: :asc_nils_last) + filter(expr(not is_nil(author.id))) + end + + list :linked_post_titles_with_author_join_filter, :linked_posts, :title do + sort(title: :asc_nils_last) + join_filter(:linked_posts, expr(not is_nil(author.id))) + end + + custom(:comment_titles_joined, :comments, :string) do + implementation({AshSqlite.Test.StringAgg, field: :title, delimiter: ","}) + end + + custom(:total_comment_likes_custom, :comments, :float) do + implementation({AshSqlite.Test.TotalAgg, field: :likes}) + end + + custom(:comment_titles_joined_with_popular_ratings, :comments, :string) do + filter(expr(not is_nil(popular_ratings.id))) + implementation({AshSqlite.Test.StringAgg, field: :title, delimiter: ","}) + end + + custom(:linked_post_titles_joined, :linked_posts, :string) do + implementation({AshSqlite.Test.StringAgg, field: :title, delimiter: ","}) + end + + count :count_of_comments_called_match, :comments do + filter(expr(title == "match")) + end + + count :count_of_comments_with_join_filter, :comments do + join_filter(:comments, expr(title == "match")) + end + + count :count_of_comments_with_related_filter, :comments do + filter(expr(not is_nil(post.id))) + end + + count :count_of_comments_with_related_exists_filter, :comments do + filter(expr(exists(post, not is_nil(id)))) + end + + count :count_of_comments_with_popular_ratings, :comments do + filter(expr(not is_nil(popular_ratings.id))) + end + + count :count_comment_titles_with_popular_ratings, :comments do + field(:title) + filter(expr(not is_nil(popular_ratings.id))) + end + + count :count_of_comments_with_aggregate_filter, :comments do + filter(expr(count_of_ratings > 0)) + end + + count :count_of_comments_matching_post_title, :comments do + filter(expr(title == parent(title))) + end + + count :count_of_comments_with_parent_join_filter, :comments do + join_filter(:comments, expr(title == parent(title))) + end + + exists :has_comment_called_match, :comments do + filter(expr(title == "match")) + end + + exists :has_linked_post_called_match, :linked_posts do + filter(expr(title == "match")) + end + + count :count_of_linked_posts_with_join_filter, :linked_posts do + join_filter(:linked_posts, expr(title == "match")) + end + + count :count_of_linked_posts_with_author, :linked_posts do + filter(expr(not is_nil(author.id))) + end + end + calculations do calculate(:score_after_winning, :integer, expr((score || 0) + 1)) calculate(:negative_score, :integer, expr(-score)) + calculate(:has_comments, :boolean, expr(count_of_comments > 0)) + + calculate( + :comment_likes_with_score, + :integer, + expr((sum_of_comment_likes || 0) + (score || 0)) + ) + + calculate( + :linked_post_score_with_score, + :integer, + expr((sum_of_linked_post_scores || 0) + (score || 0)) + ) + calculate(:category_label, :string, expr("(" <> category <> ")")) calculate(:score_with_score, :string, expr(score <> score)) calculate(:foo_bar_from_stuff, :string, expr(stuff[:foo][:bar])) diff --git a/test/support/string_agg.ex b/test/support/string_agg.ex new file mode 100644 index 0000000..9cc6519 --- /dev/null +++ b/test/support/string_agg.ex @@ -0,0 +1,19 @@ +# SPDX-FileCopyrightText: 2023 ash_sqlite contributors +# +# SPDX-License-Identifier: MIT + +defmodule AshSqlite.Test.StringAgg do + @moduledoc false + + use Ash.Resource.Aggregate.CustomAggregate + use AshSqlite.CustomAggregate + + import Ecto.Query + + def dynamic(opts, binding) do + field = Keyword.fetch!(opts, :field) + delimiter = Keyword.get(opts, :delimiter, ",") + + dynamic(fragment("group_concat(?, ?)", field(as(^binding), ^field), ^delimiter)) + end +end diff --git a/test/support/total_agg.ex b/test/support/total_agg.ex new file mode 100644 index 0000000..af3133d --- /dev/null +++ b/test/support/total_agg.ex @@ -0,0 +1,18 @@ +# SPDX-FileCopyrightText: 2023 ash_sqlite contributors +# +# SPDX-License-Identifier: MIT + +defmodule AshSqlite.Test.TotalAgg do + @moduledoc false + + use Ash.Resource.Aggregate.CustomAggregate + use AshSqlite.CustomAggregate + + import Ecto.Query + + def dynamic(opts, binding) do + field = Keyword.fetch!(opts, :field) + + dynamic(fragment("total(?)", field(as(^binding), ^field))) + end +end