Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 5 additions & 10 deletions docs/site/content/docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,17 +54,12 @@ looks out-of-date, please open an issue or pull request.
- The initial thread should be the one to call `splinterdb_close()` when
the `splinterdb` instance is no longer needed.

- Threads (other than the initial thread) that will use the `splinterdb`
must be registered before use and unregistered before exiting:
- Threads are registered automatically the first time they call a SplinterDB
public API, and are deregistered automatically when they exit. Internally,
SplinterDB allocates per-thread scratch space at registration time.

- From a non-initial thread, call `splinterdb_register_thread()`.
Internally, SplinterDB will allocate scratch space for use by that thread.

- To avoid leaking memory, a non-initial thread should call
`splinterdb_deregister_thread()` before it exits.

- Known issue: In a pinch, non-initial, registered threads may call
`splinterdb_close()`, but their scratch memory would be leaked.
- The low-level thread registration APIs remain available for tests and
callers that want to reserve or release a thread ID explicitly.

- Note these rules apply to system threads, not [runtime-managed threading](https://en.wikipedia.org/wiki/Green_threads)
available in higher-level languages.
Expand Down
15 changes: 5 additions & 10 deletions docs/site/content/docs/v0.0.1/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,17 +54,12 @@ looks out-of-date, please open an issue or pull request.
- The initial thread should be the one to call `splinterdb_close()` when
the `splinterdb` instance is no longer needed.

- Threads (other than the initial thread) that will use the `splinterdb`
must be registered before use and unregistered before exiting:
- Threads are registered automatically the first time they call a SplinterDB
public API, and are deregistered automatically when they exit. Internally,
SplinterDB allocates per-thread scratch space at registration time.

- From a non-initial thread, call `splinterdb_register_thread()`.
Internally, SplinterDB will allocate scratch space for use by that thread.

- To avoid leaking memory, a non-initial thread should call
`splinterdb_deregister_thread()` before it exits.

- Known issue: In a pinch, non-initial, registered threads may call
`splinterdb_close()`, but their scratch memory would be leaked.
- The low-level thread registration APIs remain available for tests and
callers that want to reserve or release a thread ID explicitly.

- Note these rules apply to system threads, not [runtime-managed threading](https://en.wikipedia.org/wiki/Green_threads)
available in higher-level languages.
Expand Down
15 changes: 5 additions & 10 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,17 +54,12 @@ looks out-of-date, please open an issue or pull request.
- The initial thread should be the one to call `splinterdb_close()` when
the `splinterdb` instance is no longer needed.

- Threads (other than the initial thread) that will use the `splinterdb`
must be registered before use and unregistered before exiting:
- Threads are registered automatically the first time they call a SplinterDB
public API, and are deregistered automatically when they exit. Internally,
SplinterDB allocates per-thread scratch space at registration time.

- From a non-initial thread, call `splinterdb_register_thread()`.
Internally, SplinterDB will allocate scratch space for use by that thread.

- To avoid leaking memory, a non-initial thread should call
`splinterdb_deregister_thread()` before it exits.

- Known issue: In a pinch, non-initial, registered threads may call
`splinterdb_close()`, but their scratch memory would be leaked.
- The low-level thread registration APIs remain available for tests and
callers that want to reserve or release a thread ID explicitly.

- Note these rules apply to system threads, not [runtime-managed threading](https://en.wikipedia.org/wiki/Green_threads)
available in higher-level languages.
Expand Down
11 changes: 6 additions & 5 deletions include/splinterdb/platform_linux/public_platform.h
Original file line number Diff line number Diff line change
Expand Up @@ -82,10 +82,9 @@ platform_set_log_streams(platform_log_handle *info_stream,

// Register the current thread so that it can be used with splinterdb.
//
// Any thread that uses a splinterdb must first be registered with it.
//
// The only exception is the initial thread which called create or open,
// as that thread is implicitly registered. Re-registering it is an error.
// SplinterDB public APIs register threads automatically on first use, so most
// callers do not need this function. It remains available for lower-level tests
// and callers that want to reserve a thread ID before using lower-level APIs.
//
// A thread should not be registered more than once; that is an error.
//
Expand All @@ -97,7 +96,9 @@ platform_register_thread(void);

// Deregister the current thread.
//
// Call this function before exiting a registered thread.
// Registered threads are deregistered automatically when they exit. This
// function remains available for callers that want to release a thread ID
// early.
void
platform_deregister_thread(void);

Expand Down
5 changes: 2 additions & 3 deletions include/splinterdb/splinterdb.h
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,8 @@
* A data_config must be provided at the time of create/open.
* See default_data_config.h for a basic reference implementation.
*
* Each thread must call splinterdb_register_thread() before making any calls to
* SplinterDB. Each thread must call splinterdb_deregister_thread() before
* exiting.
* Threads are registered with SplinterDB automatically on first use and are
* deregistered automatically when they exit.
*/

#pragma once
Expand Down
128 changes: 112 additions & 16 deletions src/platform_linux/platform_threads.c
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ __thread threadid xxxtid = INVALID_TID;
threadid xxxpid;
pid_t ospid;

static pthread_key_t thread_registration_key;
static pthread_once_t thread_registration_key_once = PTHREAD_ONCE_INIT;
static char thread_registration_key_value;


/****************************************
* Thread ID allocation and management *
Expand Down Expand Up @@ -53,6 +57,67 @@ typedef struct id_allocator {
// processes.
static id_allocator *id_alloc = NULL;

static void
thread_registration_after_fork_child(void)
{
/*
* The child inherits TLS values, but the parent's thread ID still belongs to
* the parent in the shared allocator. Force the child through registration.
*/
xxxtid = INVALID_TID;
xxxpid = INVALID_TID;
ospid = 0;
}

static void
thread_registration_destructor(void *arg)
{
(void)arg;

if (xxxtid != INVALID_TID) {
platform_deregister_thread();
}
}

static void
thread_registration_key_init(void)
{
int ret = pthread_key_create(&thread_registration_key,
thread_registration_destructor);
platform_assert(ret == 0);

ret = pthread_atfork(NULL, NULL, thread_registration_after_fork_child);
platform_assert(ret == 0);
}

static platform_status
thread_registration_cleanup_set(void *value)
{
int ret;

ret =
pthread_once(&thread_registration_key_once, thread_registration_key_init);
if (ret != 0) {
return CONST_STATUS(ret);
}

ret = pthread_setspecific(thread_registration_key, value);
return CONST_STATUS(ret);
}

static platform_status
thread_registration_cleanup_arm(void)
{
return thread_registration_cleanup_set(&thread_registration_key_value);
}

static void
thread_registration_cleanup_disarm(void)
{
platform_status rc = thread_registration_cleanup_set(NULL);
platform_assert_status_ok(rc);
}

/*
* task_init_tid_bitmask() - Initialize the global bitmask of active threads
* in the task system structure to indicate that no threads are currently
Expand Down Expand Up @@ -304,13 +369,16 @@ platform_deregister_thread()
platform_assert(tid != INVALID_TID,
"Error! Attempt to deregister unregistered thread.\n");

thread_registration_cleanup_disarm();
deallocate_threadid(tid);
decref_xxxpid();
}

static void
thread_registration_cleanup_function(void *arg)
{
(void)arg;

if (xxxtid == INVALID_TID) {
platform_error_log("Thread registration cleanup function called for "
"unregistered thread %lu",
Expand All @@ -320,6 +388,38 @@ thread_registration_cleanup_function(void *arg)
}
}

static platform_status
register_thread_common(void)
{
platform_status status;
threadid thread_tid;

id_allocator_init_if_needed();

status = ensure_xxxpid_is_setup();
if (!SUCCESS(status)) {
return status;
}

thread_tid = allocate_threadid();
// Unavailable threads is a temporary state that could go away.
if (thread_tid == INVALID_TID) {
decref_xxxpid();
return STATUS_BUSY;
}

platform_assert(thread_tid < MAX_THREADS);
xxxtid = thread_tid;

status = thread_registration_cleanup_arm();
if (!SUCCESS(status)) {
deallocate_threadid(thread_tid);
decref_xxxpid();
return status;
}

return STATUS_OK;
}

/*
* platform_register_thread(): Register this new thread.
Expand All @@ -330,13 +430,6 @@ thread_registration_cleanup_function(void *arg)
int
platform_register_thread(void)
{
id_allocator_init_if_needed();

platform_status status = ensure_xxxpid_is_setup();
if (!SUCCESS(status)) {
return -1;
}

threadid thread_tid = xxxtid;

// Before registration, all SplinterDB threads' tid will be its default
Expand All @@ -346,17 +439,17 @@ platform_register_thread(void)
"registered as thread %lu\n",
thread_tid);

thread_tid = allocate_threadid();
// Unavailable threads is a temporary state that could go away.
if (thread_tid == INVALID_TID) {
decref_xxxpid();
return -1;
}
return SUCCESS(register_thread_common()) ? 0 : -1;
}

platform_assert(thread_tid < MAX_THREADS);
xxxtid = thread_tid;
platform_status
platform_register_thread_auto(void)
{
if (xxxtid != INVALID_TID) {
return STATUS_OK;
}

return 0;
return register_thread_common();
}


Expand All @@ -371,6 +464,8 @@ thread_worker_function(void *arg)
thread_invocation *thread_inv = (thread_invocation *)arg;
threadid tid = thread_inv - id_alloc->thread_invocations;
xxxtid = tid;
platform_status rc = thread_registration_cleanup_arm();
platform_assert_status_ok(rc);
thread_inv->worker(thread_inv->arg);
pthread_cleanup_pop(1);
return NULL;
Expand Down Expand Up @@ -398,6 +493,7 @@ platform_thread_create(platform_thread *thread,
// so that we can report an error if the threadid allocation fails.
threadid tid = allocate_threadid();
if (tid == INVALID_TID) {
decref_xxxpid();
return STATUS_BUSY;
}
thread_invocation *thread_inv = &id_alloc->thread_invocations[tid];
Expand Down
16 changes: 16 additions & 0 deletions src/platform_linux/platform_threads.h
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
#include "splinterdb/platform_linux/public_platform.h"
#include "platform_status.h"
#include "platform_heap.h"
#include "platform_util.h"
#include <unistd.h>
#include <pthread.h>

Expand All @@ -27,6 +28,21 @@ platform_get_tid()
return xxxtid;
}

platform_status
platform_register_thread_auto(void);

static inline platform_status
platform_ensure_thread_registered()
{
extern __thread threadid xxxtid;

if (LIKELY(xxxtid != INVALID_TID)) {
return STATUS_OK;
}

return platform_register_thread_auto();
}

/* This is not part of the platform API. It is used internally to this platform
* implementation. Specifically, it is used in laio.c. */
static inline threadid
Expand Down
Loading
Loading