Add support for streaming and cursor query execution in SQL templates

[zZHorizonZz]

Motivation:

• Added a support for streaming as discussed in Support streaming in SqlTemplate #1624 to handle large result sets via streaming or cursor queries trough vertx-sql-client-templates

Changes:

• Added forStream and forCursor methods to SqlTemplate.
• Added SqlTemplateStreamImpl, CursorSqlTemplateImpl, and MappingRowStream
• Added documentation to index.adoc with examples

Closes: #1624

[tsegismont]

At first glance, I'm wondering why do we need a separate SqlTemplateStream.

In the original issue we talked about adding a couple methods to SqlTemplate and I assumed they would return a Cursor or a RowStream.

[zZHorizonZz]

So, mostly, I think the most handy part of SqlTemplate is the mapTo methods. The issue here is that mapTo methods return RowSet<U>, but we need to return RowStream. This is not an issue for Cursor, I added that only as a simplification for if you want to use a cursor, I initially had it in SqlTemplateStream, so that you would have streaming template and current template but I moved that one to SqlTemplate, I'm not sure about that one that needs to be discussed, more I think i f we want that, where we want that, I think it is usefull, but you might have different opinion. The problem is just the stream, because I didn't want to change the API of the SqlTemplate class.

[tsegismont]

Thank you for the updates @zZHorizonZz

I took some time to review the proposal and limitations, and I think we're not going in the right direction. Indeed, we have new APIs that look like the original SqlClientTemplate, but not all methods can be supported because it doesn't fit with the usage model of cursor/rowstream.

I think that, instead, we should make it possible for users to parse a template query and map objects to parameters. Then they can use the existing preparedQuery, cursor and rowstream APIs like they usually do.

The parser could look like this (just the basics):

@VertxGen
public interface SqlTemplateParser {

  /**
   * Create a parser by extracting driver info from a SqlClient.
   * The client reference is NOT retained after this call.
   *
   * @param client the SQL client to extract driver information from
   * @param template the SQL template string with named parameters (e.g., "SELECT * FROM users WHERE id = #{id}")
   * @return the template parser
   */
  static SqlTemplateParser create(SqlClient client, String template) {
    SqlClientInternal internal = (SqlClientInternal) client;
    return SqlTemplateParserImpl.create(internal.driver(), template);
  }

  /**
   * Get the parsed SQL query with driver-specific placeholders.
   *
   * @return the SQL query string
   */
  String sql();

  /**
   * Convert parameters to Tuple using the provided TupleMapper.
   *
   * @param params the parameters object
   * @param mapper the tuple mapper
   * @param <T> the parameters type
   * @return the tuple
   */
  @GenIgnore(GenIgnore.PERMITTED_TYPE)
  <T> Tuple mapToTuple(T params, TupleMapper<T> mapper);
}

  /**
   * Convert Map parameters directly to Tuple.
   *
   * @param params the parameters map
   * @return the tuple
   */
  default Tuple mapToTuple(Map<String, Object> params) {
    return mapToTuple(params, TupleMapper.mapper(Function.identity()));
  }
}

Then for prepared queries:

class User {
  public long id;
  public String firstName;
  public String lastName;
}

// Define row mapper
RowMapper<User> ROW_USER_MAPPER = row -> {
  User user = new User();
  user.id = row.getInteger("id");
  user.firstName = row.getString("firstName");
  user.lastName = row.getString("lastName");
  return user;
};

// Parse template
SqlTemplateParser parser = SqlTemplateParser.create(client,
  "SELECT * FROM users WHERE id=#{id}");

// Execute with row mapping
Map<String, Object> params = Collections.singletonMap("id", 1);
Tuple tuple = parser.mapToTuple(params);

client.preparedQuery(parser.sql())
  .mapping(ROW_USER_MAPPER::map)
  .execute(tuple)
  .onSuccess(users -> {
    users.forEach(user -> {
      System.out.println(user.firstName + " " + user.lastName);
    });
  });

And since this is generic, you could leverage the SqlTemplateParser for cursor/rowstreams

[zZHorizonZz]

Thank you for the time @tsegismont

This is an interesting approach and actually seems cleaner, but on the other hand I'm not sure how this would work in the case of streaming, because one of the things SqlTemplateStreamImpl does is simplify that interaction. Right now there's nothing like client.stream("select ....").mapping, instead you have to do something like:

connection.prepare(sqlTemplate.getSql()).map(ps -> {
    RowStream<Row> stream = ps.createStream(fetchSize, tuple);
    stream.handler(row -> handler.handle(mapper.map(row)));
    //etc
}

I.e. there's no easier way to map rows at the moment (or at least I wasn't able to find one, happy to be corrected if I'm missing something).

TLDR - I like the idea of making SQL more or less agnostic to the operation, but I think we'd then have a separate issue of making streams easier to use.