Skip to content

Commit f3f344d

Browse files
Move thread-local connections from examples to recipes (#104)
* Rename the "connection pool" example to "thread-local connection" and move it to the PostgreSQL recipes. * Instead of selecting from nothing make inserts into tab_bar. * Document the recipe.
1 parent 59638af commit f3f344d

10 files changed

Lines changed: 170 additions & 151 deletions

File tree

docs/recipes.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,9 @@ sqlpp23 can be extended to make it more powerful or easy to use in a given conte
77
This section contains some recipes for doing so.
88

99
* [Add a custom SQL function](/docs/recipes/custom_function.md)
10-
* [Optimistic concurrency control](/docs/recipes/optimistic_concurrency_control.md)
1110
* [Mapping to/from key-value store](/docs/recipes/key_value_store.md)
11+
* [Optimistic concurrency control](/docs/recipes/optimistic_concurrency_control.md)
12+
* [Thread-local connection](/docs/recipes/thread_local_connection.md)
1213

1314
Additional ideas are welcome, of course. Please file issues or pull requests.
1415

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
[**\< Recipes**](/docs/recipes.md)
2+
3+
# Thread-local database connections
4+
5+
In this document we describe a simple pattern that allows us to make thread-safe database queries using thread-unsafe database connections.
6+
7+
## Thread safety in the database connectors
8+
9+
At the time of this writing, database connections created by the three main databases supported by sqlpp23 (MySQL, PostgreSQL, and SQLite3) are not thread-safe. The sqlpp23 library is thread-agnostic, which means that it does not add any requirements or guarantees to the thread safety of the underlying database objects and operations. So it is up to the library user to ensure the thread safety of the database operations performed through these database connections.
10+
11+
## Making thread-safe queries using thread-unsafe connectors
12+
13+
The pattern is based on the idea that each user thread is given its own database connection. When a thread wants to execute a database query, it uses its own database connection to execute the query, thus avoiding the need to implement complex and potentially expensive thread synchronization.
14+
15+
We define a database connection class called `lazy_connection`, which mimics the regular database connections provided by sqlpp23 and lets the user execute database queries, pretty much like a regular sqlpp connection does. In fact, our lazy connection creates an underlying sqlpp database connection and forwards all database queries to the sqlpp connection, but that sqlpp connection is not
16+
created immediately in the constructor of the lazy connection. Instead, its creation is postponed until the moment when the user tries to execute their first query through our lazy connection object, which is why our connection class is called "lazy".
17+
18+
Then we create a thread-local variable of type `lazy_connection` called g_dbc at global scope:
19+
```
20+
thread_local lazy_connection g_dbc{g_pool};
21+
```
22+
23+
As you can see from the definition of g_dbc, it is defined as `thread_local`, which means that each user thread gets its own copy of `g_dbc`, stored in the thread's TLS (Thread Local Storage). By using the `thread_local` keyword, we offload all the thread-related chores to the C++ compiler and runtime. When a user thread tries to execute a query through `g_dbc`, the C++ runtime automatically gets a thread-local lazy connection, creating it if necessary. The lazy connection in turn gets a new sqlpp connection from the connection pool and uses it to execute the query. The connection pool is merely an implementation detail; strictly speaking, we could skip the connection pool and create a new connection inside `lazy_connection`, but we use the connection pool for performance reasons.
24+
25+
## Why not make the connection object local?
26+
27+
One might be tempted to make our instance of `lazy_connection` local; after all, a local variable can also be declared as `thread_local`. So why did we make `g_dbc` local? It is because making it local does not work the way one might expect. Let's say that we try to define and use the thread-local lazy connection in block scope:
28+
```
29+
sqlpp::postgresql::connection_pool g_pool{...};
30+
31+
int main()
32+
{
33+
thread_local dbc{&g_pool};
34+
std::thread t{[&] {
35+
dbc(...);
36+
}};
37+
t.join ();
38+
}
39+
```
40+
41+
Attempting to use our lazy connection in this fashion will cause a runtime error, because the newly spawned thread uses an uninitialized copy of the lazy connection. While global thread-local variables are guaranteed to be initialized the moment when a thread tries to use them, the local thread-local variables are only initialized when execution passes through their definition. The newly spawned thread never actually entered the `main()` function, so its thread-local copy of the database connection was never initialized, and the attempt to use the uninitialized lazy connection caused the runtime error.
42+
43+
## Sample code
44+
45+
The sample source code, implementing this pattern, is available [here](/tests/postgresql/recipes/thread_local_connection.cpp).
46+
47+
[**\< Recipes**](/docs/recipes.md)

examples/connection_pool/CMakeLists.txt

Lines changed: 0 additions & 48 deletions
This file was deleted.

examples/connection_pool/src/db_connection.cpp

Lines changed: 0 additions & 15 deletions
This file was deleted.

examples/connection_pool/src/db_connection.h

Lines changed: 0 additions & 31 deletions
This file was deleted.

examples/connection_pool/src/db_global.cpp

Lines changed: 0 additions & 12 deletions
This file was deleted.

examples/connection_pool/src/db_global.h

Lines changed: 0 additions & 7 deletions
This file was deleted.

examples/connection_pool/src/main.cpp

Lines changed: 0 additions & 37 deletions
This file was deleted.

tests/postgresql/recipes/CMakeLists.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,3 +30,4 @@ function(create_test name)
3030
endfunction()
3131

3232
create_test(optimistic_concurrency_control)
33+
create_test(thread_local_connection)
Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
/*
2+
* Copyright (c) 2023-2026, Vesselin Atanasov
3+
* All rights reserved.
4+
*
5+
* Redistribution and use in source and binary forms, with or without
6+
* modification, are permitted provided that the following conditions are met:
7+
*
8+
* * Redistributions of source code must retain the above copyright notice,
9+
* this list of conditions and the following disclaimer.
10+
* * Redistributions in binary form must reproduce the above copyright notice,
11+
* this list of conditions and the following disclaimer in the documentation
12+
* and/or other materials provided with the distribution.
13+
*
14+
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
15+
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
16+
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
17+
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
18+
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
19+
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
20+
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
21+
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
22+
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
23+
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
24+
* POSSIBILITY OF SUCH DAMAGE.
25+
*/
26+
27+
// A sample program demonstrating how to create and use a thread-safe connection
28+
// using a lazy database connection, connection pool and thread-local storage.
29+
//
30+
// For details on the actual pattern see
31+
// /docs/recipes/thread_local_connection.md
32+
33+
#include <optional>
34+
#include <thread>
35+
#include <vector>
36+
37+
#include <sqlpp23/tests/postgresql/all.h>
38+
39+
namespace sql = ::sqlpp::postgresql;
40+
41+
// This is our main database connection class. It mimics a regular database
42+
// connection, while delegating the execution of SQL queries to an underlying
43+
// sqlpp23 database connection. This underlying database connections is not
44+
// created immediately upon construction of our lazy connection. Instead the
45+
// underlying database connection is created the first time when the user
46+
// tries to execute a database query through operator().
47+
//
48+
// The class constructor received a reference to a connection pool, which later
49+
// is used to get the underlying database connection. When the constructor is
50+
// called, the connection pool does not have to be fully initialized, because
51+
// the constructor does not use the connection pool and just stores the
52+
// reference to it for later use.
53+
//
54+
class lazy_connection {
55+
private:
56+
sql::connection_pool& _pool;
57+
std::optional<sql::pooled_connection> _dbc;
58+
59+
public:
60+
lazy_connection(sql::connection_pool& pool) : _pool{pool}, _dbc{} {}
61+
lazy_connection(const lazy_connection&) = delete;
62+
lazy_connection(lazy_connection&&) = delete;
63+
64+
lazy_connection& operator=(const lazy_connection&) = delete;
65+
lazy_connection& operator=(lazy_connection&&) = delete;
66+
67+
// Delegate to _dbc any methods of sql::connection that you may need
68+
// In our example the only delegated method is operator()
69+
70+
template <typename T>
71+
auto operator()(const T& t) {
72+
if (!_dbc) {
73+
_dbc = _pool.get();
74+
}
75+
return (*_dbc)(t);
76+
}
77+
};
78+
79+
sql::connection_pool g_pool{};
80+
81+
// This is our lazy connection object, which we use to execute SQL queries.
82+
// It is marked with the thread_local storage class specifier, which means
83+
// that the C++ runtime creates one instance of the object per thread, each
84+
// instance having its own underlying database connection.
85+
//
86+
// We don't really care about the order in which the connection pool and
87+
// the global connection object are initialized because, as described above,
88+
// the constructor of the lazy connection only stores a reference to the pool
89+
// without actually using it.
90+
//
91+
thread_local lazy_connection g_dbc{g_pool};
92+
93+
int main() {
94+
const int num_threads = 5;
95+
const int num_queries = 10;
96+
97+
// Initialize the global connection pool
98+
g_pool.initialize(sql::make_test_config(), num_threads);
99+
100+
test::createTabBar(g_dbc);
101+
test::TabBar tb{};
102+
// Spawn the threads and make each thread execute multiple SQL queries
103+
std::vector<std::thread> threads{};
104+
for (int i = 0; i < num_threads; ++i) {
105+
threads.push_back(std::thread([&]() {
106+
for (int j = 0; j < num_queries; ++j) {
107+
// For simplicity we don't begin/commit a transaction explicitly.
108+
// Instead we use the database autocommit mode in which the database
109+
// engine wraps each quety in its own transaction.
110+
//
111+
g_dbc(insert_into(tb).set(tb.intN = i * j));
112+
}
113+
}));
114+
}
115+
for (auto&& t : threads) {
116+
t.join();
117+
}
118+
119+
return 0;
120+
}

0 commit comments

Comments
 (0)