Skip to content

Adfa 3802 make documentation bookshelf dynamic #1353

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jimturner-adfa wants to merge 5 commits into
base: stage
Choose a base branch
from
Open

Adfa 3802 make documentation bookshelf dynamic #1353

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
399e28d
WIP added code for dynamic bookshelf
jimturner-adfa May 28, 2026
ccfea34
first working version of dynamic bookshelf
jimturner-adfa May 29, 2026
b32a684
📝 Add docstrings to `ADFA-3802-Make-documentation-bookshelf-dynamic`
coderabbitai[bot] Jun 1, 2026
e6745a1
Merge branch 'stage' into ADFA-3802-Make-documentation-bookshelf-dynamic
davidschachterADFA Jun 1, 2026
e865057
Merge branch 'stage' into ADFA-3802-Make-documentation-bookshelf-dynamic
davidschachterADFA Jun 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
193 changes: 144 additions & 49 deletions app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ import java.util.concurrent.ConcurrentHashMap
import io.pebbletemplates.pebble.template.PebbleTemplate
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import kotlinx.serialization.builtins.UByteArraySerializer
import okio.ByteString.Companion.toByteString


data class ServerConfig(
Expand Down Expand Up @@ -62,8 +64,9 @@ class WebServer(private val config: ServerConfig) {
private val experimentsEnabled : Boolean = File(config.experimentsEnablePath).exists() // Frozen at startup. Restart server if needed.
private val encodingHeader : String = "Accept-Encoding"
private val brotliCompression : String = "br"
private val pebbleEngine = PebbleEngine.Builder().loader(StringLoader()).build()
private val templateCache = ConcurrentHashMap<Int, PebbleTemplate>()
private val pebbleEngine = PebbleEngine.Builder().loader(StringLoader()).build()
private val templateCache = ConcurrentHashMap<Int, PebbleTemplate>()
private var bookshelfTemplateId : Int = -1;
Comment on lines +67 to +69
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Invalidate these caches when the database handle changes.

Lines 308-313 can reopen database, but templateCache and bookshelfTemplateId survive that swap. After a reload, /pr/bs can keep using a template ID or compiled template from the previous database.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt` around
lines 67 - 69, When the DB handle is reopened/rotated (the block that reassigns
the database variable), clear the template caches so old templates from the
previous DB aren’t reused: call templateCache.clear() and reset
bookshelfTemplateId = -1 immediately after the database swap (the same place
where the code reopens/assigns database). This ensures PebbleTemplate entries in
templateCache and the cached bookshelfTemplateId are invalidated when the
database changes.


private val contentChunkSize = 1024 * 1024

Expand Down Expand Up @@ -243,6 +246,19 @@ clientSocket and the catch block logic are updated accordingly.
return String(bytes, 0, len, Charsets.ISO_8859_1)
}

/**
* Handles a single HTTP request on the given client socket and writes the corresponding HTTP response.
*
* Parses the request line and headers, enforces GET-only for normal routes, and routes high-priority
* "pr/" endpoints (including bookshelf, database table views, projects, and experiments). For other
* paths it reads content and metadata from the server's SQLite database, reassembles multi-fragment
* content, handles Brotli compression negotiation and decompression, optionally renders Pebble
* templates, and writes appropriate HTTP response headers and body. If a newer debug database is
* present on disk, swaps the server database to that file before serving the request. On errors
* sends an HTTP error response and logs the failure.
*
* @param clientSocket Connected client socket used for reading the request and writing the response.
*/
private fun handleClient(clientSocket: Socket) {
if (debugEnabled) log.debug("In handleClient(), socket is {}.", clientSocket)

Expand Down Expand Up @@ -397,6 +413,17 @@ clientSocket and the catch block logic are updated accordingly.
}
}

/**
* Renders a Pebble template identified by `templateId` using the provided JSON data and returns the rendered output as bytes.
*
* @param templateId The database ID of the Pebble template to load and compile.
* @param dbContent JSON bytes that will be parsed and supplied as the template context.
* @param path The request/content path associated with this template (used for diagnostic/logging purposes).
* @param dbMimeType The MIME type of the stored content (used for diagnostic/logging purposes).
* @param compression The compression label of the stored content (e.g., "br", "none") (used for diagnostic/logging purposes).
* @return The rendered template encoded as UTF-8 bytes.
* @throws Exception If the template ID is not found, is duplicated in the database, or if template lookup/instantiation fails.
*/
private fun instantiatePebbleTemplate(templateId: Int, dbContent: ByteArray, path: String, dbMimeType: String, compression: String): ByteArray {
if (debugEnabled) log.debug("Processing template for templateId={}", templateId)

Expand Down Expand Up @@ -446,6 +473,7 @@ clientSocket and the catch block logic are updated accordingly.
}
else -> {
val templateBlob = cursor.getBlob(0)
if (debugEnabled) log.debug("templateBlob = '${String(templateBlob)}'")
pebbleEngine.getTemplate(templateBlob.toString(Charsets.UTF_8))
}
}
Expand All @@ -456,13 +484,25 @@ clientSocket and the catch block logic are updated accordingly.
val mapper = ObjectMapper()
val context: Map<String, Any> = mapper.readValue(dbContent.toString(Charsets.UTF_8), object : TypeReference<Map<String, Any>>() {})

/*******************DEBUGGING ONLY*******************/
log.debug("context = ${context}")
/*******************DEBUGGING ONLY*******************/

// Evaluate template with loaded data and return the output
val sw = StringWriter()
compiledTemplate.evaluate(sw, context)
return sw.toString().toByteArray()
}


/**
* Serve an HTML page showing the 20 most recent rows of the `LastChange` table.
*
* Queries the table schema to determine column names, selects the latest 20 rows
* ordered by `changeTime`, escapes cell values for HTML, assembles an HTML table,
* and writes a normal 200 HTML response to the client. On database or rendering
* errors a 500 error response is sent. All database cursors are closed before returning.
*/
private fun handleDbEndpoint(writer: PrintWriter, output: java.io.OutputStream) {
if (debugEnabled) log.debug("Entering handleDbEndpoint().")

Expand Down Expand Up @@ -556,31 +596,38 @@ clientSocket and the catch block logic are updated accordingly.
}
}

/**
* Handles the /pr/bs endpoint by invoking the bookshelf generator and sending a 500 error if generation fails.
*
* Calls realHandleBsEndpoint to produce and write the response body; if an exception occurs, sends an HTTP 500
* error using the reported output-start state so no additional headers/body are written after output has begun.
*
* @param writer PrintWriter used for writing textual HTTP response headers.
* @param output Raw OutputStream used for writing the response body bytes.
*/
private fun handleBsEndpoint(writer: PrintWriter, output: java.io.OutputStream) {
if (debugEnabled) log.debug("Entering handleBsEndpoint().")

var projectDatabase : SQLiteDatabase? = null
var outputStarted = false

try {
projectDatabase = SQLiteDatabase.openDatabase(config.projectDatabasePath,
null,
SQLiteDatabase.OPEN_READONLY)

outputStarted = realHandleBsEndpoint(writer, output, projectDatabase)
outputStarted = realHandleBsEndpoint(writer, output)

} catch (e: Exception) {
log.error("Error handling /pr/bs endpoint: {}", e.message)
sendError(writer, output, 500, "Internal Server Error 6", "Error generating database table.", outputStarted)

} finally {
projectDatabase?.close()
sendError(writer, output, 500, "Internal Server Error 6", "Error generating bookshelf HTML.", outputStarted)
Comment on lines +608 to +619
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Set outputStarted before the first write.

outputStarted only becomes true after realHandleBsEndpoint() returns. If writeNormalToClient() throws after sending headers or part of the body, this catch will still append a second 500 because the flag is still false.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt` around
lines 567 - 578, The catch block can append a second 500 because outputStarted
is only set true after realHandleBsEndpoint returns; modify the flow so
outputStarted is set to true as soon as any headers/body are written (i.e.,
inside the write path) rather than only on return. Concretely, update
realHandleBsEndpoint (or the helper that calls writeNormalToClient) to accept a
mutable flag (e.g., AtomicBoolean or a setter lambda) or return a result object
so it flips outputStarted to true immediately when the first write/flush
happens, then keep the existing try/catch in handleBsEndpoint to rely on that
flag. Ensure references: handleBsEndpoint, realHandleBsEndpoint, and the write
method (writeNormalToClient) are changed accordingly so the catch sees the
correct outputStarted state.

Copy link
Copy Markdown
Collaborator

@jatezzz jatezzz Jun 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could create a constant to centralize the 500 value.

}

if (debugEnabled) log.debug("Leaving handleBsEndpoint().")
}


/**
* Writes a small CSS response that shows or hides elements with the
* `.code_on_the_go_experiment` class depending on the server's
* `experimentsEnabled` flag.
*/
private fun handleExEndpoint(writer: PrintWriter, output: java.io.OutputStream) {
val flag = if (experimentsEnabled) "{}" else "{display: none;}"

Expand All @@ -589,6 +636,12 @@ clientSocket and the catch block logic are updated accordingly.
sendCSS(writer, output, ".code_on_the_go_experiment $flag")
}

/**
* Handle the /pr/pr endpoint by opening the project database, delegating page generation to realHandlePrEndpoint, and sending an HTTP 500 error if generation fails.
*
* @param writer PrintWriter used to write response headers.
* @param output OutputStream used to write response body bytes.
*/
private fun handlePrEndpoint(writer: PrintWriter, output: java.io.OutputStream) {
if (debugEnabled) log.debug("Entering handlePrEndpoint().")

Expand Down Expand Up @@ -644,63 +697,105 @@ second response.
if (debugEnabled) log.debug("Leaving handlePrEndpoint().")
}

private fun realHandleBsEndpoint(writer: PrintWriter, output: java.io.OutputStream, projectDatabase: SQLiteDatabase) : Boolean {
/**
* Builds the Bookshelf content, renders it with the `bookshelf` template, and sends the resulting response to the client.
*
* @param writer PrintWriter for sending HTTP headers and control output.
* @param output OutputStream for writing the response body bytes.
* @return `true` if the templated response was written to the client, `false` if an error response was sent or no output was produced.
*/
private fun realHandleBsEndpoint(writer: PrintWriter, output: java.io.OutputStream) : Boolean {
if (debugEnabled) log.debug("Entering realHandleBsEndpoint().")

val query = """
SELECT id,
name,
DATETIME(create_at / 1000, 'unixepoch'),
DATETIME(last_modified / 1000, 'unixepoch'),
location,
template_name,
language
FROM recent_project_table
ORDER BY last_modified DESC"""
// Database fetch
val sql_query =
"""
SELECT '{"result" : [' || group_concat(Item) || ']}' FROM (
SELECT
JSON_OBJECT(
'category', IFNULL(BC.category, 'General'),
'description', BC.description,
'books', JSON_GROUP_ARRAY(JSON_OBJECT(
'title', IFNULL(B.title, C.path),
'description', B.description,
'link', C.path)
)
) AS Item
FROM Content AS C,
Bookshelf AS B,
BookCategories AS BC
WHERE C.id = B.contentID
AND B.bookCategoryID = BC.id
GROUP BY BC.category
ORDER BY BC.category,
B.title
);
""".trimIndent()

var cursor = database.rawQuery(sql_query, arrayOf())
lateinit var jsonText : ByteArray

var html = getTableHtml("Projects", "Projects") + """
<tr>
<th>Id</th>
<th>Name</th>
<th>Created</th>
<th>Modified &nbsp;&nbsp;<span style="font-family: sans-serif">V</span></th>
<th>Directory</th>
<th>Template</th>
<th>Language</th>
</tr>"""
// Process database fetch
try {
if (cursor.count != 1) {
if (cursor.count == 0)
sendError(writer, output, 404, "Not Found")
else
sendError(writer, output, 500, "Corrupt database - ${cursor.count} bookshelf results found when one was expected.")
return false
}

val cursor = projectDatabase.rawQuery(query, arrayOf())
//get the JSON from the bookshelf table
cursor.moveToFirst()
jsonText = cursor.getBlob(0)
if (debugEnabled) log.debug("json content = '${String(jsonText)}'.")
if (debugEnabled) log.debug("before fetch bookshelf template ID = '${bookshelfTemplateId}'")

//Have we already fetched the template
if (bookshelfTemplateId == -1) {
Copy link
Copy Markdown
Collaborator

@jatezzz jatezzz Jun 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you avoid the nested structure?

if () {
  if () {
    if () {

    }
  }
}

cursor = database.rawQuery("SELECT id FROM Templates WHERE name = 'bookshelf'", arrayOf())

if (cursor.count != 1) {
if (cursor.count == 0)
sendError(writer, output, 404, "Not Found")
else
sendError(writer, output, 500, "Corrupt database - ${cursor.count} Bookshelf templates found when 1 was expected.")
return false
}

try {
if (debugEnabled) log.debug("Retrieved {} rows.", cursor.count)
cursor.moveToFirst()
bookshelfTemplateId = cursor.getInt(0);
Comment on lines +755 to +767
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Don't overwrite the first cursor before closing it.

The cursor from rawQuery(sql_query, ...) is replaced by the template-ID query, so the bookshelf-result cursor never gets closed on the initial /pr/bs request. Use a second local cursor or nested use {} blocks here.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt` around
lines 696 - 708, The code in WebServer.kt overwrites the existing cursor when
running the template-ID query (affecting bookshelfTemplateId) and never closes
the original cursor; fix by ensuring each Cursor is closed — either use a second
local variable for the template query (e.g., templateCursor) or wrap each
rawQuery result in a Kotlin use { } block so cursor.close() is called
automatically before assigning or returning; update the block around the
rawQuery/cursor.moveToFirst()/getInt(0) logic that sets bookshelfTemplateId and
any sendError branches to close the appropriate cursor(s).

if (debugEnabled) log.debug("after the fetch bookshelf template ID = '${bookshelfTemplateId}'")

while (cursor.moveToNext()) {
html += """<tr>
<td>${escapeHtml(cursor.getString(0) ?: "")}</td>
<td>${escapeHtml(cursor.getString(1) ?: "")}</td>
<td>${escapeHtml(cursor.getString(2) ?: "")}</td>
<td>${escapeHtml(cursor.getString(3) ?: "")}</td>
<td>${escapeHtml(cursor.getString(4) ?: "")}</td>
<td>${escapeHtml(cursor.getString(5) ?: "")}</td>
<td>${escapeHtml(cursor.getString(6) ?: "")}</td>
</tr>"""
}

html += "</table></body></html>"
} catch (e: Exception) {
log.error("Error processing request: {}", e.message)
sendError(writer, output, 500, "Internal Server Error", e.message ?: "")

} finally {
cursor.close()
}

if (debugEnabled) log.debug("html is '{}'.", html) // May output a lot of stuff but better too much than too little. --DS, 23-Feb-2026
val result = instantiatePebbleTemplate(bookshelfTemplateId, jsonText, "/bookshelf", "application/json", "none")
Comment on lines +772 to +780
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Return immediately after sending the fetch-time error.

If anything in the fetch block fails before jsonText or bookshelfTemplateId is initialized, this catch writes an error and then continues into instantiatePebbleTemplate(...). That turns one failure into a second exception and can trigger another response write.

Suggested fix 
         } catch (e: Exception) {
             log.error("Error processing request: {}", e.message)
             sendError(writer, output, 500, "Internal Server Error", e.message ?: "")
+            return false
 
         } finally {
             cursor.close()
         }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
} catch (e: Exception) {
log.error("Error processing request: {}", e.message)
sendError(writer, output, 500, "Internal Server Error", e.message ?: "")
} finally {
cursor.close()
}
val result = instantiatePebbleTemplate(bookshelfTemplateId, jsonText, "/bookshelf", "application/json", "none")
} catch (e: Exception) {
log.error("Error processing request: {}", e.message)
sendError(writer, output, 500, "Internal Server Error", e.message ?: "")
return false
} finally {
cursor.close()
}
val result = instantiatePebbleTemplate(bookshelfTemplateId, jsonText, "/bookshelf", "application/json", "none")
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt` around
lines 713 - 721, The catch block that logs the fetch error and calls sendError
must stop further processing so instantiatePebbleTemplate(...) is not called
with uninitialized jsonText or bookshelfTemplateId; modify the catch (Exception
e) handling around the fetch to, after logging and calling sendError(writer,
output, 500, ...), immediately return (or otherwise short-circuit) from the
surrounding method so cursor.close() remains in finally and
instantiatePebbleTemplate(...) is never reached when an exception occurred.


writeNormalToClient(writer, output, html)
if (debugEnabled) log.debug("Bookshelf result is '{}'.", String(result))

if (debugEnabled) log.debug("Leaving realHandlePrEndpoint().")
writeNormalToClient(writer, output, String(result))

if (debugEnabled) log.debug("Leaving realHandleBsEndpoint().")

return true
}

/**
* Builds an HTML table of recent projects from the provided project database and writes it to the client.
*
* @param writer PrintWriter used for writing HTTP response headers.
* @param output OutputStream used for writing the HTTP response body.
* @param projectDatabase Read-only SQLiteDatabase containing the `recent_project_table`.
* @return `true` if an HTML response was written to the client.
*/
private fun realHandlePrEndpoint(writer: PrintWriter, output: java.io.OutputStream, projectDatabase: SQLiteDatabase) : Boolean {
if (debugEnabled) log.debug("Entering realHandlePrEndpoint().")

Expand Down
Loading