Skip to content

Adfa facility to disable webserver's pebble cache #1357

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jimturner-adfa wants to merge 5 commits into
base: stage
Choose a base branch
from
+148 −49
Open

Adfa facility to disable webserver's pebble cache #1357

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
399e28d
WIP added code for dynamic bookshelf
jimturner-adfa May 28, 2026
ccfea34
first working version of dynamic bookshelf
jimturner-adfa May 29, 2026
b32a684
📝 Add docstrings to `ADFA-3802-Make-documentation-bookshelf-dynamic`
coderabbitai[bot] Jun 1, 2026
e6745a1
Merge branch 'stage' into ADFA-3802-Make-documentation-bookshelf-dynamic
davidschachterADFA Jun 1, 2026
ed5ab9c
clear the pebble cache if CodeOnTheCode.webserver.cs0 is in /sdcard/D…
jimturner-adfa Jun 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
197 changes: 148 additions & 49 deletions app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ import java.util.concurrent.ConcurrentHashMap
import io.pebbletemplates.pebble.template.PebbleTemplate
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import kotlinx.serialization.builtins.UByteArraySerializer
import okio.ByteString.Companion.toByteString


data class ServerConfig(
Expand All @@ -39,6 +41,8 @@ data class ServerConfig(
"/Download/CodeOnTheGo.webserver.debug",
val experimentsEnablePath: String = android.os.Environment.getExternalStorageDirectory().toString() +
"/Download/CodeOnTheGo.exp", // TODO: Centralize this concept. --DS, 9-Feb-2026
val clearCacheEnablePath: String = android.os.Environment.getExternalStorageDirectory().toString() +
"/Download/CodeOnTheGo.webserver.cs0",

// Yes, this is hack code.
val projectDatabasePath: String = "/data/data/com.itsaky.androidide/databases/RecentProject_database"
Expand All @@ -60,10 +64,12 @@ class WebServer(private val config: ServerConfig) {
private val debugEnabled : Boolean = File(config.debugEnablePath).exists()
// TODO: Use the centralized experiments flag instead of this ad-hoc check. --DS, 10-Feb-2026
private val experimentsEnabled : Boolean = File(config.experimentsEnablePath).exists() // Frozen at startup. Restart server if needed.
private val clearCacheEnabled : Boolean = File(config.clearCacheEnablePath).exists() // Frozen at startup. Restart server if needed.
private val encodingHeader : String = "Accept-Encoding"
private val brotliCompression : String = "br"
private val pebbleEngine = PebbleEngine.Builder().loader(StringLoader()).build()
private val templateCache = ConcurrentHashMap<Int, PebbleTemplate>()
private val pebbleEngine = PebbleEngine.Builder().loader(StringLoader()).build()
private val templateCache = ConcurrentHashMap<Int, PebbleTemplate>()
private var bookshelfTemplateId : Int = -1;

private val contentChunkSize = 1024 * 1024

Expand Down Expand Up @@ -243,6 +249,19 @@ clientSocket and the catch block logic are updated accordingly.
return String(bytes, 0, len, Charsets.ISO_8859_1)
}

/**
Copy link
Copy Markdown
Contributor

@dara-abijo-adfa dara-abijo-adfa Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment/documentation and some other new ones in this file seem to have been added despite the functions not being new.
I think they can be removed or at the very least be more concise.

* Handles a single HTTP request on the given client socket and writes the corresponding HTTP response.
*
* Parses the request line and headers, enforces GET-only for normal routes, and routes high-priority
* "pr/" endpoints (including bookshelf, database table views, projects, and experiments). For other
* paths it reads content and metadata from the server's SQLite database, reassembles multi-fragment
* content, handles Brotli compression negotiation and decompression, optionally renders Pebble
* templates, and writes appropriate HTTP response headers and body. If a newer debug database is
* present on disk, swaps the server database to that file before serving the request. On errors
* sends an HTTP error response and logs the failure.
*
* @param clientSocket Connected client socket used for reading the request and writing the response.
*/
private fun handleClient(clientSocket: Socket) {
if (debugEnabled) log.debug("In handleClient(), socket is {}.", clientSocket)

Expand Down Expand Up @@ -397,6 +416,17 @@ clientSocket and the catch block logic are updated accordingly.
}
}

/**
* Renders a Pebble template identified by `templateId` using the provided JSON data and returns the rendered output as bytes.
*
* @param templateId The database ID of the Pebble template to load and compile.
* @param dbContent JSON bytes that will be parsed and supplied as the template context.
* @param path The request/content path associated with this template (used for diagnostic/logging purposes).
* @param dbMimeType The MIME type of the stored content (used for diagnostic/logging purposes).
* @param compression The compression label of the stored content (e.g., "br", "none") (used for diagnostic/logging purposes).
* @return The rendered template encoded as UTF-8 bytes.
* @throws Exception If the template ID is not found, is duplicated in the database, or if template lookup/instantiation fails.
*/
private fun instantiatePebbleTemplate(templateId: Int, dbContent: ByteArray, path: String, dbMimeType: String, compression: String): ByteArray {
if (debugEnabled) log.debug("Processing template for templateId={}", templateId)

Expand Down Expand Up @@ -446,6 +476,7 @@ clientSocket and the catch block logic are updated accordingly.
}
else -> {
val templateBlob = cursor.getBlob(0)
if (debugEnabled) log.debug("templateBlob = '${String(templateBlob)}'")
pebbleEngine.getTemplate(templateBlob.toString(Charsets.UTF_8))
}
}
Expand All @@ -456,13 +487,25 @@ clientSocket and the catch block logic are updated accordingly.
val mapper = ObjectMapper()
val context: Map<String, Any> = mapper.readValue(dbContent.toString(Charsets.UTF_8), object : TypeReference<Map<String, Any>>() {})

/*******************DEBUGGING ONLY*******************/
Copy link
Copy Markdown
Contributor

@dara-abijo-adfa dara-abijo-adfa Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This log should be safe to remove

log.debug("context = ${context}")
/*******************DEBUGGING ONLY*******************/
Comment on lines +490 to +492
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Jun 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Unconditional debug logging should be gated.

This log.debug call is not guarded by debugEnabled, unlike other debug logging throughout the file. This will execute the string interpolation and logging call on every template instantiation even in production, causing unnecessary overhead.

🔧 Proposed fix 
         /*******************DEBUGGING ONLY*******************/
-        log.debug("context = ${context}")
+        if (debugEnabled) log.debug("context = {}", context)
         /*******************DEBUGGING ONLY*******************/
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt` around
lines 490 - 492, The unconditional debug call "log.debug(\"context =
${context}\")" in WebServer.kt should be gated like other debug logs to avoid
interpolation overhead; wrap it with the existing debug check (e.g., use
log.isDebugEnabled or the file's debugEnabled pattern) or use a lazy/logging API
that defers message construction, referencing the "log.debug" call and the
"context" value so the message is only constructed and emitted when debugging is
enabled.


// Evaluate template with loaded data and return the output
val sw = StringWriter()
compiledTemplate.evaluate(sw, context)
return sw.toString().toByteArray()
}


/**
* Serve an HTML page showing the 20 most recent rows of the `LastChange` table.
*
* Queries the table schema to determine column names, selects the latest 20 rows
* ordered by `changeTime`, escapes cell values for HTML, assembles an HTML table,
* and writes a normal 200 HTML response to the client. On database or rendering
* errors a 500 error response is sent. All database cursors are closed before returning.
*/
private fun handleDbEndpoint(writer: PrintWriter, output: java.io.OutputStream) {
if (debugEnabled) log.debug("Entering handleDbEndpoint().")

Expand Down Expand Up @@ -556,31 +599,39 @@ clientSocket and the catch block logic are updated accordingly.
}
}

/**
* Handles the /pr/bs endpoint by invoking the bookshelf generator and sending a 500 error if generation fails.
*
* Calls realHandleBsEndpoint to produce and write the response body; if an exception occurs, sends an HTTP 500
* error using the reported output-start state so no additional headers/body are written after output has begun.
*
* @param writer PrintWriter used for writing textual HTTP response headers.
* @param output Raw OutputStream used for writing the response body bytes.
*/
private fun handleBsEndpoint(writer: PrintWriter, output: java.io.OutputStream) {
if (debugEnabled) log.debug("Entering handleBsEndpoint().")
if(clearCacheEnabled) templateCache.clear()

var projectDatabase : SQLiteDatabase? = null
var outputStarted = false

try {
projectDatabase = SQLiteDatabase.openDatabase(config.projectDatabasePath,
null,
SQLiteDatabase.OPEN_READONLY)

outputStarted = realHandleBsEndpoint(writer, output, projectDatabase)
outputStarted = realHandleBsEndpoint(writer, output)

} catch (e: Exception) {
log.error("Error handling /pr/bs endpoint: {}", e.message)
sendError(writer, output, 500, "Internal Server Error 6", "Error generating database table.", outputStarted)

} finally {
projectDatabase?.close()
sendError(writer, output, 500, "Internal Server Error 6", "Error generating bookshelf HTML.", outputStarted)
}

if (debugEnabled) log.debug("Leaving handleBsEndpoint().")
}


/**
* Writes a small CSS response that shows or hides elements with the
* `.code_on_the_go_experiment` class depending on the server's
* `experimentsEnabled` flag.
*/
private fun handleExEndpoint(writer: PrintWriter, output: java.io.OutputStream) {
val flag = if (experimentsEnabled) "{}" else "{display: none;}"

Expand All @@ -589,6 +640,12 @@ clientSocket and the catch block logic are updated accordingly.
sendCSS(writer, output, ".code_on_the_go_experiment $flag")
}

/**
* Handle the /pr/pr endpoint by opening the project database, delegating page generation to realHandlePrEndpoint, and sending an HTTP 500 error if generation fails.
*
* @param writer PrintWriter used to write response headers.
* @param output OutputStream used to write response body bytes.
*/
private fun handlePrEndpoint(writer: PrintWriter, output: java.io.OutputStream) {
if (debugEnabled) log.debug("Entering handlePrEndpoint().")

Expand Down Expand Up @@ -644,63 +701,105 @@ second response.
if (debugEnabled) log.debug("Leaving handlePrEndpoint().")
}

private fun realHandleBsEndpoint(writer: PrintWriter, output: java.io.OutputStream, projectDatabase: SQLiteDatabase) : Boolean {
/**
* Builds the Bookshelf content, renders it with the `bookshelf` template, and sends the resulting response to the client.
*
* @param writer PrintWriter for sending HTTP headers and control output.
* @param output OutputStream for writing the response body bytes.
* @return `true` if the templated response was written to the client, `false` if an error response was sent or no output was produced.
*/
private fun realHandleBsEndpoint(writer: PrintWriter, output: java.io.OutputStream) : Boolean {
if (debugEnabled) log.debug("Entering realHandleBsEndpoint().")

val query = """
SELECT id,
name,
DATETIME(create_at / 1000, 'unixepoch'),
DATETIME(last_modified / 1000, 'unixepoch'),
location,
template_name,
language
FROM recent_project_table
ORDER BY last_modified DESC"""
// Database fetch
val sql_query =
"""
SELECT '{"result" : [' || group_concat(Item) || ']}' FROM (
SELECT
JSON_OBJECT(
'category', IFNULL(BC.category, 'General'),
'description', BC.description,
'books', JSON_GROUP_ARRAY(JSON_OBJECT(
'title', IFNULL(B.title, C.path),
'description', B.description,
'link', C.path)
)
) AS Item
FROM Content AS C,
Bookshelf AS B,
BookCategories AS BC
WHERE C.id = B.contentID
AND B.bookCategoryID = BC.id
GROUP BY BC.category
ORDER BY BC.category,
Copy link
Copy Markdown
Contributor

@dara-abijo-adfa dara-abijo-adfa Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding ORDER BY after GROUP BY will sort the resulting category group, and not the JSON_GROUP_ARRAY itself. Is that the intention?

If that's not the intention, then this subquery will sort the items in JSON_GROUP_ARRAY:

FROM (
      SELECT BC.category, BC.description as category_desc, IFNULL(B.title, C.path) as title, B.description as book_desc, C.path
      FROM Content C
      JOIN Bookshelf B ON C.id = B.contentID
      JOIN BookCategories BC ON B.bookCategoryID = BC.id
      ORDER BY BC.category, title
  )
  GROUP BY category
);

B.title
);
""".trimIndent()

var cursor = database.rawQuery(sql_query, arrayOf())
lateinit var jsonText : ByteArray

var html = getTableHtml("Projects", "Projects") + """
<tr>
<th>Id</th>
<th>Name</th>
<th>Created</th>
<th>Modified &nbsp;&nbsp;<span style="font-family: sans-serif">V</span></th>
<th>Directory</th>
<th>Template</th>
<th>Language</th>
</tr>"""
// Process database fetch
try {
if (cursor.count != 1) {
if (cursor.count == 0)
sendError(writer, output, 404, "Not Found")
else
sendError(writer, output, 500, "Corrupt database - ${cursor.count} bookshelf results found when one was expected.")
return false
}

val cursor = projectDatabase.rawQuery(query, arrayOf())
//get the JSON from the bookshelf table
cursor.moveToFirst()
jsonText = cursor.getBlob(0)
if (debugEnabled) log.debug("json content = '${String(jsonText)}'.")
if (debugEnabled) log.debug("before fetch bookshelf template ID = '${bookshelfTemplateId}'")

//Have we already fetched the template
if (bookshelfTemplateId == -1) {
cursor = database.rawQuery("SELECT id FROM Templates WHERE name = 'bookshelf'", arrayOf())
Copy link
Copy Markdown
Contributor

@dara-abijo-adfa dara-abijo-adfa Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cursor wasn't previously closed before its reassignment here. Let's ensure safe closure.


if (cursor.count != 1) {
if (cursor.count == 0)
sendError(writer, output, 404, "Not Found")
else
sendError(writer, output, 500, "Corrupt database - ${cursor.count} Bookshelf templates found when 1 was expected.")
return false
}

try {
if (debugEnabled) log.debug("Retrieved {} rows.", cursor.count)
cursor.moveToFirst()
bookshelfTemplateId = cursor.getInt(0);
Comment on lines +759 to +771
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Jun 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Cursor resource leak when template ID is fetched.

When bookshelfTemplateId == -1, the cursor variable is reassigned at line 760, losing the reference to the first cursor (from line 739). The finally block at line 781 only closes the second cursor, leaking the first.

🐛 Proposed fix - close first cursor before reassigning 
             //Have we already fetched the template
             if (bookshelfTemplateId == -1) {
+                cursor.close()  // Close the bookshelf JSON cursor before reusing variable
                 cursor = database.rawQuery("SELECT id FROM Templates WHERE name = 'bookshelf'", arrayOf())

                 if (cursor.count != 1) {

Alternatively, use a separate variable for the template cursor to make the resource management clearer.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (bookshelfTemplateId == -1) {
cursor = database.rawQuery("SELECT id FROM Templates WHERE name = 'bookshelf'", arrayOf())
if (cursor.count != 1) {
if (cursor.count == 0)
sendError(writer, output, 404, "Not Found")
else
sendError(writer, output, 500, "Corrupt database - ${cursor.count} Bookshelf templates found when 1 was expected.")
return false
}
try {
if (debugEnabled) log.debug("Retrieved {} rows.", cursor.count)
cursor.moveToFirst()
bookshelfTemplateId = cursor.getInt(0);
if (bookshelfTemplateId == -1) {
cursor.close() // Close the bookshelf JSON cursor before reusing variable
cursor = database.rawQuery("SELECT id FROM Templates WHERE name = 'bookshelf'", arrayOf())
if (cursor.count != 1) {
if (cursor.count == 0)
sendError(writer, output, 404, "Not Found")
else
sendError(writer, output, 500, "Corrupt database - ${cursor.count} Bookshelf templates found when 1 was expected.")
return false
}
cursor.moveToFirst()
bookshelfTemplateId = cursor.getInt(0);
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt` around
lines 759 - 771, The code reassigns the variable cursor when fetching the
'bookshelf' template ID, leaking the earlier cursor resource; before calling
database.rawQuery("SELECT id FROM Templates WHERE name = 'bookshelf'") reclose
the existing cursor (or use a new variable like templateCursor) so the original
cursor is closed, then proceed to check cursor.count, moveToFirst(), and set
bookshelfTemplateId; ensure the finally block still closes whichever cursor
variable(s) are used and that sendError and the rest of the logic remain
unchanged.

if (debugEnabled) log.debug("after the fetch bookshelf template ID = '${bookshelfTemplateId}'")

while (cursor.moveToNext()) {
html += """<tr>
<td>${escapeHtml(cursor.getString(0) ?: "")}</td>
<td>${escapeHtml(cursor.getString(1) ?: "")}</td>
<td>${escapeHtml(cursor.getString(2) ?: "")}</td>
<td>${escapeHtml(cursor.getString(3) ?: "")}</td>
<td>${escapeHtml(cursor.getString(4) ?: "")}</td>
<td>${escapeHtml(cursor.getString(5) ?: "")}</td>
<td>${escapeHtml(cursor.getString(6) ?: "")}</td>
</tr>"""
}

html += "</table></body></html>"
} catch (e: Exception) {
log.error("Error processing request: {}", e.message)
sendError(writer, output, 500, "Internal Server Error", e.message ?: "")

} finally {
cursor.close()
}

if (debugEnabled) log.debug("html is '{}'.", html) // May output a lot of stuff but better too much than too little. --DS, 23-Feb-2026
val result = instantiatePebbleTemplate(bookshelfTemplateId, jsonText, "/bookshelf", "application/json", "none")
Comment on lines +776 to +784
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Jun 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Missing return false after error in catch block causes use of uninitialized variable.

The catch block at lines 776-778 sends an error response but doesn't return. Execution continues to line 784 where jsonText may be uninitialized (if the exception occurred before line 754), causing UninitializedPropertyAccessException.

🐛 Proposed fix 
         } catch (e: Exception) {
             log.error("Error processing request: {}", e.message)
             sendError(writer, output, 500, "Internal Server Error", e.message ?: "")
+            return false

         } finally {
             cursor.close()
         }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
} catch (e: Exception) {
log.error("Error processing request: {}", e.message)
sendError(writer, output, 500, "Internal Server Error", e.message ?: "")
} finally {
cursor.close()
}
if (debugEnabled) log.debug("html is '{}'.", html) // May output a lot of stuff but better too much than too little. --DS, 23-Feb-2026
val result = instantiatePebbleTemplate(bookshelfTemplateId, jsonText, "/bookshelf", "application/json", "none")
} catch (e: Exception) {
log.error("Error processing request: {}", e.message)
sendError(writer, output, 500, "Internal Server Error", e.message ?: "")
return false
} finally {
cursor.close()
}
val result = instantiatePebbleTemplate(bookshelfTemplateId, jsonText, "/bookshelf", "application/json", "none")
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt` around
lines 776 - 784, The catch in WebServer.kt currently logs and sends an error
response but does not stop further processing, allowing code after the try/catch
(which uses jsonText) to run with jsonText uninitialized; modify the catch block
in the request-handling method so that after calling sendError(writer, output,
500, "Internal Server Error", e.message ?: "") it immediately returns false (or
otherwise exits the handler) to prevent reaching instantiatePebbleTemplate(...)
and using jsonText, ensuring cursor.close() still runs in finally.


writeNormalToClient(writer, output, html)
if (debugEnabled) log.debug("Bookshelf result is '{}'.", String(result))

if (debugEnabled) log.debug("Leaving realHandlePrEndpoint().")
writeNormalToClient(writer, output, String(result))

if (debugEnabled) log.debug("Leaving realHandleBsEndpoint().")

return true
}

/**
* Builds an HTML table of recent projects from the provided project database and writes it to the client.
*
* @param writer PrintWriter used for writing HTTP response headers.
* @param output OutputStream used for writing the HTTP response body.
* @param projectDatabase Read-only SQLiteDatabase containing the `recent_project_table`.
* @return `true` if an HTML response was written to the client.
*/
private fun realHandlePrEndpoint(writer: PrintWriter, output: java.io.OutputStream, projectDatabase: SQLiteDatabase) : Boolean {
if (debugEnabled) log.debug("Entering realHandlePrEndpoint().")

Expand Down
Loading