From f22043ed6e69919900ce3283f33ec6d4e4544f79 Mon Sep 17 00:00:00 2001 From: tyronechrisharris <142608718+tyronechrisharris@users.noreply.github.com> Date: Tue, 5 May 2026 17:49:36 -0400 Subject: [PATCH 1/9] release: finalize OSCAR 3.5.1 deployment hardening, monitoring, docs, and DB pool fixes --- README.md | 416 +++++++++----- build-all.sh | 17 +- changelog.md | 34 +- .../MediaMTX_OSCAR_camera_proxy_guide.md | 389 +++++++++++++ .../Node_Administration_3.5.1_addendum.md | 25 + .../OSCAR_System_Documentation_Manual_3.5.md | 82 ++- .../OSCAR_launch_monitoring_guide.md | 511 +++++++++++++++++ dist/documentation/Release_Notes_3.5.1.md | 533 ++++++++++++++++++ .../Standard_PostgreSQL_Setup.md | 101 ++++ dist/release/check-oscar-status.ps1 | 136 +++++ dist/release/check-oscar-status.sh | 225 ++++++++ dist/release/env.template | 55 ++ dist/release/launch-all-arm.sh | 350 ++++++++++-- dist/release/launch-all-arm_old.sh | 247 ++++++++ dist/release/launch-all.bat | 381 ++++++++++--- dist/release/launch-all.sh | 335 +++++++++-- dist/release/launch-all_old.bat | 192 +++++++ dist/release/launch-all_old.sh | 111 ++++ dist/release/monitor-oscar.bat | 249 ++++++++ dist/release/monitor-oscar.sh | 342 +++++++++++ dist/release/monitor-oscar_old.bat | 129 +++++ dist/release/monitor-oscar_old.sh | 254 +++++++++ dist/scripts/standard/launch.bat | 259 ++++++++- dist/scripts/standard/launch.sh | 248 +++++++- dist/scripts/standard/launch_old.bat | 144 +++++ dist/scripts/standard/launch_old.sh | 141 +++++ dist/scripts/standard/load_trusted_certs.bat | 123 ++-- 27 files changed, 5554 insertions(+), 475 deletions(-) create mode 100644 dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md create mode 100644 dist/documentation/Node_Administration_3.5.1_addendum.md create mode 100644 dist/documentation/OSCAR_launch_monitoring_guide.md create mode 100644 dist/documentation/Release_Notes_3.5.1.md create mode 100644 dist/documentation/Standard_PostgreSQL_Setup.md create mode 100644 dist/release/check-oscar-status.ps1 create mode 100644 dist/release/check-oscar-status.sh create mode 100644 dist/release/env.template mode change 100755 => 100644 dist/release/launch-all-arm.sh create mode 100644 dist/release/launch-all-arm_old.sh mode change 100755 => 100644 dist/release/launch-all.bat mode change 100755 => 100644 dist/release/launch-all.sh create mode 100644 dist/release/launch-all_old.bat create mode 100644 dist/release/launch-all_old.sh create mode 100644 dist/release/monitor-oscar.bat create mode 100644 dist/release/monitor-oscar.sh create mode 100644 dist/release/monitor-oscar_old.bat create mode 100644 dist/release/monitor-oscar_old.sh mode change 100755 => 100644 dist/scripts/standard/launch.bat mode change 100755 => 100644 dist/scripts/standard/launch.sh create mode 100644 dist/scripts/standard/launch_old.bat create mode 100644 dist/scripts/standard/launch_old.sh diff --git a/README.md b/README.md index 4c0b9e4..1655ea3 100644 --- a/README.md +++ b/README.md @@ -1,180 +1,284 @@ # OSH OAKRIDGE BUILDNODE -This repository combines all the OSH modules and dependencies to deploy the OSH server and client for ORNL. +This repository packages the OSH server and OSCAR client deployment used for ORNL field and test systems. ## Requirements -- [Java 21.0.10+](https://www.oracle.com/java/technologies/downloads/#java21) -- [Docker engine](https://www.docker.com) -- [Oakridge Build Node Repository](https://github.com/Botts-Innovative-Research/osh-oakridge-buildnode) -- Node v22 - -## Quick Start -1. **Download the latest release** - - Go to the Releases section of the repository and download the latest compiled release archive (for example, `oscar-3.3.5.zip`). -2. **Extract the archive** - - Extract the downloaded ZIP file to a directory of your choice. -3. **Verify Docker Engine** - - Ensure that [Docker engine](https://www.docker.com) is installed and actively running on your host machine. -4. **Launch the system** - - Open a terminal or command prompt in the extracted directory and run the OS-specific launch script: - - **Windows:** Run `launch-all.bat` - - **Linux/macOS:** Run `./launch-all.sh` - - **ARM systems:** Run `./launch-all-arm.sh` - -For a complete guide covering architecture, deployment, configuration, operations, and troubleshooting, please refer to the [OSCAR System Documentation Manual](dist/documentation/OSCAR_System_Documentation_Manual_3.5.md). - -## Installation -Clone the repository and update all submodules recursively + +- Java 21 or newer +- Docker Engine or Docker Desktop, running before launch +- A packaged OSCAR release archive, or a local source checkout for build workflows +- Node v22 only when building from source + +## OSCAR 3.5.1 packaged release quick start + +This section is for operators using the **prebuilt OSCAR 3.5.1 release ZIP**. + +### 1. Verify required dependencies + +Windows PowerShell: + +```powershell +java -version +docker version +``` + +Linux: + +```bash +java -version +docker version +``` + +Use **Java 21 or newer**. The launch scripts validate dependencies and will stop early if Java or Docker is missing or too old. + +### 2. If you were previously running OSCAR, start fresh + +Before extracting **OSCAR 3.5.1**, stop the old deployment and remove old local runtime artifacts. + +Windows PowerShell: + +```powershell +Get-CimInstance Win32_Process | + Where-Object { + $_.Name -match '^java(\.exe)?$' -and + $_.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*' + } | + Select-Object ProcessId, CommandLine + +# Stop the old OSCAR JVM if one is still running +Stop-Process -Id -Force + +# Stop and remove the old PostGIS container +docker rm -f oscar-postgis-container + +# Remove the old Docker network if it exists +docker network rm oscar-postgis-network +``` + +Linux: + +```bash +pgrep -af 'com.botts.impl.security.SensorHubWrapper' +kill + +docker rm -f oscar-postgis-container +docker network rm oscar-postgis-network || true +``` + +Then delete the old extracted release folder, such as **oscar-3.5.0**, before extracting **oscar-3.5.1**. + +### 3. Extract the release archive + +Extract the downloaded ZIP to a fresh working directory. + +### 4. Create the runtime environment file + +For packaged releases, use the environment file that ships with the archive: + +- if the package includes **env.txt**, rename it to **.env** +- if the package includes **env.template**, copy it to **.env** + +Windows PowerShell: + +```powershell +Copy-Item .\env.template .\.env +``` + +Linux: + +```bash +cp env.template .env +``` + +Edit `.env` before first launch and at minimum confirm: + +- `SYSTEM_PROFILE` +- `DB_NAME` +- `DB_USER` +- `DB_PASSWORD` +- `DB_PORT` +- `CONTAINER_NAME` + +Useful optional settings include: + +- `FORCE_RESTART=1` to replace an already-running OSCAR instance +- `ATTACH_TO_EXISTING=1` to monitor an already-running OSCAR instance +- `MAX_WAIT_SECONDS=300` +- `RETRY_MAX=120` +- `RETRY_INTERVAL=2` +- `POSTGIS_READY_DELAY=5` + +### 5. Preferred first start: use the monitoring script + +For testing, burn-in, and side-by-side field deployment, start OSCAR with the monitoring wrapper instead of launching the node directly. + +Windows: + +```bat +monitor-oscar.bat +``` + +Linux: + +```bash +chmod +x launch-all.sh osh-node-oscar/launch.sh monitor-oscar.sh check-oscar-status.sh +./monitor-oscar.sh +``` + +This is the recommended first-run path because it: + +- starts PostGIS and OSCAR using the current launch scripts +- captures memory, thread, JFR, and database snapshots over time +- produces a monitor directory and status report inputs automatically + +### 6. Routine start without monitoring + +When monitoring is not needed, use the top-level launch script: + +Windows: + +```bat +launch-all.bat +``` + +Linux: + +```bash +./launch-all.sh +``` + +Prefer these **sessionless top-level launchers** over calling `osh-node-oscar/launch.(sh|bat)` directly unless you are debugging the node itself. + +### 7. Running-instance handling + +The launch and monitor scripts now detect already-running OSCAR JVMs. + +Default behavior: + +- `launch-all` refuses to start if OSCAR is already running +- `monitor-oscar` refuses to start if OSCAR is already running + +Optional behaviors: + +- set `FORCE_RESTART=1` to stop the running OSCAR instance and start fresh +- set `ATTACH_TO_EXISTING=1` when using `monitor-oscar` to monitor the running instance instead of replacing it + +### 8. Generate a status report after startup + +After the system has been up long enough to settle, generate a one-file report. + +Windows PowerShell: + +```powershell +powershell -ExecutionPolicy Bypass -File .\check-oscar-status.ps1 +``` + +Linux: + +```bash +./check-oscar-status.sh +``` + +### 9. Admin access + +The admin username is typically **admin**. Do **not** assume the packaged password is always `admin`. + +For packaged releases, the initial password should be managed through the packaged secret file or environment-driven password initialization flow. Verify the package contents, then change the password before production use. + +## Building from source + +Clone the repository and update all submodules recursively: ```bash git clone git@github.com:Botts-Innovative-Research/osh-oakridge-buildnode.git --recursive ``` -If you've already cloned without `--recursive`, run: + +If you already cloned without `--recursive`, run: + ```bash cd path/to/osh-oakridge-buildnode git submodule update --init --recursive ``` -## Build + +## Build + Navigate to the project directory: ```bash cd path/to/osh-oakridge-buildnode ``` -Run the build script (macOS/Linux): +Run the build script. + +Linux/macOS: ```bash ./build-all.sh ``` -Run the build script (Windows): +Windows: -```bash -./build-all.bat -``` - -After the build completes, it can be located in `build/distributions/` - -## Deploy and Start OSH Node -1. Unzip the distribution using the command line or File Explorer: - - Option 1: Command Line - ```bash - unzip build/distributions/osh-node-oscar-1.0.zip - cd osh-node-oscar-1.0/osh-node-oscar-1.0 - ``` - ```bash - tar -xf build/distributions/osh-node-oscar-1.0.zip - cd osh-node-oscar-1.0/osh-node-oscar-1.0 - ``` - Option 2: Use File Explorer - 1. Navigate to `path/to/osh-oakridge-buildnode/build/distributions/` - 2. Right-click `osh-node-oscar-1.0.zip`. - 3. Select **Extract All..** - 4. Choose your destination, (or leave the default) and extract. -1. Launch the OSH node: - Run the launch script, "launch.sh" for linux/mac and "launch.bat" for windows. -2. Access the OSH Node -- Remote: **[ip-address]:8282/sensorhub/admin** -- Locally: **http://localhost:8282/sensorhub/admin** - -The default credentials to access the OSH Node are admin:admin. This can be changed in the Security section of the admin page. - -For documentation on configuring a Lane System on the OSH Admin panel, please refer to the OSCAR Documentation provided in the Google Drive documentation folder. - -## Deploy the Client -After configuring the Lanes on the OSH Admin Panel, you can navigate to the Clients endpoint: -- Remote: **[ip-address]:8282** -- Local: **http://localhost:8282/** - -For documentation on configuring a server on the OSCAR Client refer to the OSCAR Documentation provided in the Google Drive documentation folder. - -# Releasing a New Version - -## Release Checklist -Before releasing, ensure the following on the `dev` branch: -1. Update `version` in `build.gradle` to match the release version (e.g. `"3.2.0"`) -2. Update `deploymentName` in `dist/config/standard/config.json` to `"OSCAR "` (e.g. `"OSCAR 3.2.0"`) -3. Ensure there is no `pgdata` directory in `dist/release/postgis` -4. Verify the build succeeds locally with `./build-all.sh` or `./build-all.bat` - -## Release Steps -1. **Merge `dev` into `main`:** - ```bash - git checkout main - git pull origin main - git merge dev - git push origin main - ``` - Alternatively, create a pull request from `dev` → `main` on GitHub and merge it. - -2. **Tag the release on `main`:** - ```bash - git checkout main - git pull origin main - git tag v # e.g. git tag v3.2.0 - git push origin v - ``` - -3. **The release workflow runs automatically.** It will: - - Validate that the tag is on the `main` branch - - Verify version numbers match the tag in `build.gradle` and `config.json` - - Check that `pgdata` does not exist in the release directory - - Build the project (Gradle + oscar-viewer) - - Package the source code with all submodules included - - Create a GitHub Release with the build artifact and source archive - -# PostgreSQL Configuration -There are some tweaks that can be made to the PostgreSQL configuration to make it perform better. -Below is a list of suggested configuration parameters at varying levels of maximum system RAM. - -`shared_buffers` - Should be around 25% of maximum RAM -`effective_cache_size` - Should be around 70-75% of maximum RAM -`work_mem` - 16MB to 64MB. Depends on maximum system memory and size of the load -`maintenance_work_mem` - 512MB to 2GB. Depends on the load, but it's OK to try high numbers - -# Secure Node Over TLS (HTTPS) -In order to secure the OSH node over TLS, you must generate a Java keystore with an SSL certificate. - -Below is the command to generate a keystore with a self-signed certificate. - -`keytool -genkeypair -alias -keyalg RSA -keysize 2048 -validity -keystore .jks -storepass -keypass -dname "CN=, OU=, O=, L=, ST=, C=" -ext "SAN="` - -Then, in your OSH config (`config.json`), or in the Admin Panel under `Network` -> `HTTP Server`, you must specify the key store path, password, key alias, and HTTPS port. - -An example of the `config.json`'s HTTP Server config is shown below: - -```json -{ - "objClass": "org.sensorhub.impl.service.HttpServerConfig", - "httpPort": 8282, - "httpsPort": 8443, - "servletsRootUrl": "/sensorhub", - "authMethod": "BASIC", - "keyStorePath": "osh-keystore.jks", - "keyStorePassword": "changeit", - "keyAlias": "oscar-key", - "trustStorePath": ".keystore/ssl_trust", - "enableCORS": true, - "id": "5cb05c9c-9e08-4fa1-8731-ffaa5846bdc1", - "autoStart": true, - "moduleClass": "org.sensorhub.impl.service.HttpServer", - "name": "HTTP Server" -} -``` - -You can also edit this information in the OSH launch scripts at `osh-node-oscar/launch.(sh|bat)` - -```shell -java -Xms6g -Xmx6g -Xss256k -XX:ReservedCodeCacheSize=512m -XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError \ - -Dlogback.configurationFile=./logback.xml \ - -cp "lib/*" \ - -Djava.system.class.loader="org.sensorhub.utils.NativeClassLoader" \ - -Djavax.net.ssl.keyStore="./osh-keystore.jks" \ - -Djavax.net.ssl.keyStorePassword="changeit" \ - -Djavax.net.ssl.trustStore="$SCRIPT_DIR/trustStore.jks" \ - -Djavax.net.ssl.trustStorePassword="changeit" \ - -Djava.library.path="./nativelibs" \ - com.botts.impl.security.SensorHubWrapper ./config.json ./db - -``` \ No newline at end of file +```bat +build-all.bat +``` + +After the build completes, the output is written under `build/distributions/`. + +## Source-tree deployment + +If you are testing from a source checkout instead of a packaged release: + +1. create `.env` from `env.template` +2. verify Java 21 and Docker +3. launch with `monitor-oscar` for first-run validation +4. use `check-oscar-status` after the system reaches steady state + +## MediaMTX for larger camera deployments + +For test systems and larger multi-lane deployments, consider placing **MediaMTX** in front of camera streams so OSCAR connects to stable local RTSP proxy paths instead of directly to every camera. See the updated MediaMTX guide in `dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md`. + +## PostgreSQL tuning + +The packaged launch scripts now size PostgreSQL by `SYSTEM_PROFILE`. + +Representative values: + +- `RPI4` -> max_connections 75 +- `8GB` -> max_connections 125 +- `16GB` -> max_connections 200 +- `32GB` -> max_connections 300 + +The launchers also set: + +- `superuser_reserved_connections=10` +- `idle_session_timeout=600000` +- connection and disconnection logging + +## Secure node over TLS + +To secure the OSH node over TLS, generate a Java keystore with an SSL certificate. + +```text +keytool -genkeypair -alias -keyalg RSA -keysize 2048 -validity -keystore .jks -storepass -keypass -dname "CN=, OU=, O=, L=, ST=, C=" -ext "SAN=" +``` + +Then configure the keystore path, password, alias, and HTTPS port in `config.json` or in the Admin Panel under **Network -> HTTP Server**. + +## Releasing a new version + +### Release checklist + +Before releasing from `dev`: + +1. update `version` in `build.gradle` +2. update `deploymentName` in `dist/config/standard/config.json` +3. ensure `dist/release/postgis/pgdata` is not packaged +4. verify the release ZIP name matches the intended version, such as `oscar-3.5.1.zip` +5. verify the release root directory name also matches the intended version +6. verify `env.template`, release notes, README, and launch documentation all reflect the same version + +### Release steps + +1. merge `dev` into `main` +2. tag the release on `main` +3. push the release tag and allow the workflow to build and publish the release artifacts diff --git a/build-all.sh b/build-all.sh index 9d635bb..9f0a1a7 100755 --- a/build-all.sh +++ b/build-all.sh @@ -1,10 +1,21 @@ #!/bin/bash +set -euo pipefail -cd web/oscar-viewer || exit +PROJECT_DIR="$(cd -- "$(dirname -- "$0")" && pwd)" + +cd "$PROJECT_DIR/web/oscar-viewer" || exit 1 npm install npm run build -cd ../.. || exit +cd "$PROJECT_DIR" || exit 1 + +./gradlew build -x test -x osgi + +echo "Making shell scripts executable..." + +find "$PROJECT_DIR" -maxdepth 1 -type f -name "*.sh" -exec chmod +x {} \; -./gradlew build -x test -x osgi \ No newline at end of file +if [ -d "$PROJECT_DIR/osh-node-oscar" ]; then + find "$PROJECT_DIR/osh-node-oscar" -maxdepth 1 -type f -name "*.sh" -exec chmod +x {} \; +fi diff --git a/changelog.md b/changelog.md index 0be2957..43a6a42 100644 --- a/changelog.md +++ b/changelog.md @@ -1,5 +1,29 @@ # OSCAR Build Node Change Log -All notable changes to this project will be documented in this file. + +All notable changes to this project will be documented in this file. + +## 3.5.1 2026-05-05 + +### Added +- Added profile-based launch sizing for packaged deployments. +- Added Linux and Windows monitoring workflows for first-run validation and field burn-in. +- Added one-file status report scripts for Linux and Windows. +- Added support for attach-or-restart monitor behavior through environment settings. +- Added MediaMTX deployment guidance for camera proxy use in larger test and field systems. + +### Changed +- Updated launch scripts to validate Java and Docker before startup. +- Updated launch scripts to detect an already-running OSCAR instance and fail fast unless `FORCE_RESTART=1` is set. +- Updated monitor scripts to attach to an existing OSCAR instance when `ATTACH_TO_EXISTING=1` is set. +- Updated PostGIS launch flow so an existing named container is replaced cleanly before startup settings are reapplied. +- Updated Windows launch and monitoring logic to use PowerShell and CIM-based process detection instead of WMIC. +- Updated release documentation for the prebuilt 3.5.1 unzip-and-run workflow. + +### Fixed +- Fixed PostgreSQL session over-allocation caused by oversized Hikari idle pooling. +- Fixed startup and monitoring friction on Windows around Java detection, trust store creation, and process discovery. +- Fixed release packaging/documentation drift around launch steps, cleanup expectations, and version naming. + ## 3.5.0 2026-04-24 ### Changes - Updated LaneSystem README @@ -163,8 +187,8 @@ This is the official first release of 3.0.0 ### Changed - Restructured repository, moving most directories that are unused in development under `dist` -## [2.3.0] -Release 2.3.0 +## [2.3.0] +Release 2.3.0 ### Added - Added Deployment version to config.json @@ -177,15 +201,13 @@ Removed dependency to log4j - [#90](https://github.com/Botts-Innovative-Research/osh-oakridge-buildnode/issues/90) Aspect Charts:The prior issue mentioned the Aspect RPMs and the Admin Panel, but this encompasses Aspects issues on the client as well. - [#]() -Update charts in client to display Rapiscan and Aspect charts +Update charts in client to display Rapiscan and Aspect charts - [#]() Node Form Fix: Updated NodeForm to check if node is reachable before adding it to the list of Nodes, so when configuring a node it will ensure that you can access that node before it continues processing and updating the UI. - ## [2.2] - 2025-07-30 Release 2.2 request, no updates since 1.3.7. - ## [1.3.7] 2025-07-18 ### Added diff --git a/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md b/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md new file mode 100644 index 0000000..e98fc48 --- /dev/null +++ b/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md @@ -0,0 +1,389 @@ +# Using MediaMTX to Reduce OSCAR Camera Stream Load + +This guide explains how to use **MediaMTX** as a local RTSP proxy so OSCAR does not have to open large numbers of direct camera connections. + +For **testing and side-by-side field deployment**, the recommended operational flow is: + +1. prepare a fresh OSCAR 3.5.1 deployment +2. create `.env` +3. launch OSCAR with the monitoring script +4. proxy camera streams through MediaMTX +5. run the status-check script after warm-up +6. compare reconnect, thread, and database behavior with and without the proxy + +--- + +## Why MediaMTX helps + +Without a proxy, OSCAR may open many direct RTSP sessions to cameras. If you have many lanes, reconnect activity and repeated stream setup can create unnecessary load on the cameras and on the OSCAR host. + +MediaMTX sits between OSCAR and the cameras: + +- cameras stream to MediaMTX only when needed +- OSCAR connects to MediaMTX on the local machine instead of directly to every camera +- multiple lane definitions can reuse the same proxied path +- reconnects are handled against the local proxy instead of repeatedly hammering the physical cameras + +This is especially useful when: + +- many OSCAR lanes reuse a smaller number of real camera streams +- emulator lanes are used for testing or demonstrations +- direct camera sessions are expensive or unstable +- you want a simple local point to change, test, or swap stream sources + +--- + +## Typical architecture + +A common layout is: + +1. physical cameras publish RTSP streams +2. MediaMTX runs on the same machine as OSCAR or on a nearby local host +3. MediaMTX exposes local RTSP paths such as `/lane03_cam` +4. the OSCAR lane CSV points camera hosts to the local MediaMTX service +5. the SRLS emulator or detector-side service is addressed separately from the same CSV + +In this setup, OSCAR talks to: + +- the SRLS emulator or RPM service for detector data +- MediaMTX for camera video + +--- + +## Recommended OSCAR workflow when using MediaMTX + +### 1. Verify dependencies + +Make sure **Java 21+** and **Docker** are ready for OSCAR, and that MediaMTX is installed separately on the host where you plan to run the proxy. + +### 2. Start OSCAR with monitoring + +Linux: + +```bash +./monitor-oscar.sh +``` + +Windows: + +```bat +monitor-oscar.bat +``` + +### 3. Start MediaMTX + +Linux: + +```bash +./mediamtx mediamtx.yml +``` + +Windows PowerShell: + +```powershell +.\mediamtx.exe mediamtx.yml +``` + +### 4. Let the system warm up + +Allow enough time for lane startup, first client connections, and any reconnect behavior to appear. + +### 5. Generate a status report + +Linux: + +```bash +./check-oscar-status.sh +``` + +Windows: + +```powershell +powershell -ExecutionPolicy Bypass -File .\check-oscar-status.ps1 +``` + +Review: + +- thread count +- reconnect warnings +- camera-related log churn +- PostgreSQL session plateau behavior + +--- + +## Why the provided MediaMTX settings are efficient + +The configuration below keeps MediaMTX focused on lightweight RTSP proxying: + +- `rtmp: no` +- `hls: no` +- `webrtc: no` +- `srt: no` + +Disabling unused protocols avoids extra listeners and extra processing. + +These options are also important: + +- `sourceOnDemand: yes` means MediaMTX only pulls the upstream camera when a client requests the path +- `sourceProtocol: tcp` makes RTSP transport use TCP, which is often more reliable on LANs and across NAT or firewall boundaries +- `api: yes` with `apiAddress: :9997` gives you a simple status API for checking paths and clients + +--- + +## Example MediaMTX configuration for Axis cameras + +Use placeholders for credentials in documentation and shared files. + +```yaml +api: yes +apiAddress: :9997 +rtmp: no +hls: no +webrtc: no +srt: no +paths: + lane03_cam: + source: "rtsp://:@192.168.8.73/axis-media/media.amp?adjustablelivestream=1&resolution=640x480&videocodec=h264&videokeyframeinterval=15" + sourceOnDemand: yes + sourceProtocol: tcp + + lane04_cam: + source: "rtsp://:@192.168.8.229/axis-media/media.amp?adjustablelivestream=1&resolution=640x480&videocodec=h264&videokeyframeinterval=15" + sourceOnDemand: yes + sourceProtocol: tcp + + lane05_cam: + source: "rtsp://:@192.168.8.111/axis-media/media.amp?adjustablelivestream=1&resolution=640x480&videocodec=h264&videokeyframeinterval=15" + sourceOnDemand: yes + sourceProtocol: tcp + + lane06_cam: + source: "rtsp://:@192.168.8.167/axis-media/media.amp?adjustablelivestream=1&resolution=640x480&videocodec=h264&videokeyframeinterval=15" + sourceOnDemand: yes + sourceProtocol: tcp +``` + +--- + +## How this supports many lanes with fewer real cameras + +If your OSCAR deployment has 50 lanes with 2 cameras per lane, that is 100 camera references. Those 100 references do **not** need to be 100 unique physical camera connections. + +With MediaMTX, many lane rows can point back to the same proxied path, such as: + +- `rtsp://192.168.8.77:8554/lane04_cam` +- `rtsp://192.168.8.77:8554/lane06_cam` + +That means the CSV can model many lane-camera assignments while MediaMTX proxies only a small set of real upstream streams. + +--- + +## Example OSCAR service CSV entries + +Upload the CSV through the **Services** tab in the OSCAR admin page. + +```csv +Name,UniqueID,AutoStart,Latitude,Longitude,RPMConfigType,RPMHost,RPMPort,AspectAddressStart,AspectAddressEnd,EMLEnabled,EMLCollimated,LaneWidth,CameraType0,CameraHost0,CameraPath0,Codec0,Username0,Password0,CameraType1,CameraHost1,CameraPath1,Codec1,Username1,Password1 +sim-0,simu-0,FALSE,35.89,-84.19,Rapiscan,192.168.8.77,1601,,,FALSE,FALSE,4.820000172,Custom,192.168.8.77:8554,/lane04_cam,,,,Custom,192.168.8.77:8554,/lane06_cam,,, +sim-2,simu-1,FALSE,35.883,-84.19,Rapiscan,192.168.8.77,1602,,,FALSE,FALSE,4.820000172,Custom,192.168.8.77:8554,/lane06_cam,,,,Custom,192.168.8.77:8554,/lane04_cam,,, +``` + +### Important CSV fields + +- `RPMHost` and `RPMPort` point to the SRLS or Rapiscan emulator/service +- `CameraType0` and `CameraType1` are set to `Custom` so OSCAR uses the host/path directly +- `CameraHost0` and `CameraHost1` point to the machine running MediaMTX, usually `:8554` +- `CameraPath0` and `CameraPath1` point to the MediaMTX path, such as `/lane04_cam` +- `Username` and `Password` fields are left blank because authentication is handled upstream by MediaMTX when it pulls from the real camera + +--- + +## Sony camera settings to use + +For Sony cameras, use the following secondary-stream settings: + +- `H.264` +- `640x480` +- `15 fps` +- `1 s` keyframe interval when possible +- `high` H.264 profile +- `CBR` +- about `4000 kbps` + +These settings are a good match for lightweight proxying and testing because they keep the stream modest while still providing stable H.264 output. + +Example Sony-based MediaMTX path: + +```yaml +api: yes +apiAddress: :9997 +rtmp: no +hls: no +webrtc: no +srt: no +paths: + lane03_cam: + source: "rtsp://:@192.168.8.4:554/media/video2" + sourceOnDemand: yes + sourceProtocol: tcp +``` + +--- + +## Step-by-step setup + +### 1. Install MediaMTX + +Download MediaMTX for your platform and extract it onto the host that will run the proxy. + +Typical contents include: + +- `mediamtx` or `mediamtx.exe` +- `mediamtx.yml` + +### 2. Create `mediamtx.yml` + +Start with one of the examples above and add one path per real upstream camera stream. + +### 3. Start MediaMTX + +Linux: + +```bash +./mediamtx mediamtx.yml +``` + +Windows PowerShell: + +```powershell +.\mediamtx.exe mediamtx.yml +``` + +### 4. Verify MediaMTX is listening + +Linux: + +```bash +ss -ltnp | grep -E '8554|9997' +``` + +Windows PowerShell: + +```powershell +Get-NetTCPConnection -LocalPort 8554,9997 -State Listen +``` + +### 5. Verify the API + +Linux or macOS: + +```bash +curl http://127.0.0.1:9997/v3/paths/list +``` + +Windows PowerShell: + +```powershell +Invoke-RestMethod http://127.0.0.1:9997/v3/paths/list +``` + +### 6. Update the OSCAR CSV + +Point lane camera hosts and paths to MediaMTX instead of the physical cameras. + +### 7. Upload the CSV in OSCAR + +In the OSCAR admin UI: + +1. open the **Services** tab +2. upload the CSV +3. confirm the lanes import successfully +4. start the desired lanes or services + +--- + +## Best practices + +### Keep streams modest + +Use modest stream settings for test environments: + +- H.264 +- 640x480 +- 15 fps +- CBR +- 1-second keyframe interval when possible + +### Use `sourceOnDemand` + +This avoids pulling from upstream cameras when no OSCAR consumer is actually using the stream. + +### Use local proxy paths in OSCAR + +Do not point every lane directly at physical cameras if the same few streams can be reused. + +### Keep protocols disabled unless needed + +If you only need RTSP, keep HLS, WebRTC, RTMP, and SRT disabled. + +### Use monitoring during evaluation + +When comparing direct-camera versus proxied-camera behavior, always launch OSCAR with the monitoring script and collect a status report after warm-up. + +--- + +## Troubleshooting + +### OSCAR cannot open the stream + +Check: + +- MediaMTX is running +- the lane CSV points to the correct host and path +- port `8554` is open locally +- the MediaMTX path name matches exactly + +Direct test example: + +```bash +ffplay rtsp://127.0.0.1:8554/lane03_cam +``` + +### MediaMTX path exists but does not pull video + +Check: + +- upstream camera IP and credentials +- the source URL path +- whether the camera is configured for the requested stream format +- whether the camera accepts TCP RTSP transport + +### Too many reconnects + +MediaMTX can localize reconnect activity, but also check: + +- camera network stability +- camera encoder settings +- duplicate or unstable stream consumers +- bad payloads or emulator-side issues +- OSCAR thread and reconnect logs from the monitor directory + +### API works but paths do not appear active + +With `sourceOnDemand: yes`, a path may stay idle until a client requests it. That is normal. + +--- + +## Summary + +MediaMTX reduces the burden of large camera configurations by turning many direct camera connections into a smaller number of local RTSP proxy paths. + +In practice, that means: + +- less direct pressure on physical cameras +- fewer repeated direct sessions from OSCAR +- easier CSV-based lane provisioning +- easier testing with emulator lanes +- simpler troubleshooting and stream replacement + +For field evaluation, pair MediaMTX with the OSCAR monitoring and status-check scripts so you can compare reconnect behavior, thread growth, and system steadiness under realistic load. diff --git a/dist/documentation/Node_Administration_3.5.1_addendum.md b/dist/documentation/Node_Administration_3.5.1_addendum.md new file mode 100644 index 0000000..863905b --- /dev/null +++ b/dist/documentation/Node_Administration_3.5.1_addendum.md @@ -0,0 +1,25 @@ +# Node Administration 3.5.1 addendum + +The existing **Node Administration** PDF remains useful for Admin UI tasks such as: + +- starting and stopping modules +- adding users and roles +- configuring sensors, storage, and SOS services + +No major Admin UI workflow changes were required for the 3.5.1 launch, monitoring, and packaging updates. + +The operational changes for 3.5.1 are outside that PDF and are now covered in these updated deployment documents: + +- `README.updated.md` +- `OSCAR_launch_monitoring_guide.updated.md` +- `MediaMTX_OSCAR_camera_proxy_guide.updated.md` +- `OSCAR_System_Documentation_Manual_3.5.updated.md` + +Use the PDF for Admin Panel behavior, and use the updated deployment documents for: + +- Java 21 and Docker prerequisites +- `.env` setup +- already-running OSCAR handling +- monitoring and status scripts +- fresh-install cleanup of older OSCAR releases +- MediaMTX-assisted camera deployment guidance diff --git a/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md b/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md index aff79d1..74043be 100644 --- a/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md +++ b/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md @@ -16,8 +16,8 @@ | **Field** | **Value** | |-----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------| | Document type | Technical reference and operations guide | -| Baseline | OSCAR v3.3.2 / 3.5 pause-point feature line | -| Version context | Centered on v3.3.2, the feature-complete build expected to become the 3.5 pause point after an internal test cycle | +| Baseline | OSCAR v3.5.1 packaged deployment guidance with legacy architectural background | +| Version context | Updated with OSCAR 3.5.1 deployment, monitoring, and prebuilt-release operational guidance while retaining broader architecture and feature context | | Audience | Deployment engineers, operators, testers, and maintainers | | Scope | Administrative configuration, viewer/PWA, database, file handling, Web ID integration, offline conversion, reporting, scaling, and upgrade guidance | @@ -53,6 +53,22 @@ Operationally, OSCAR supports two practical deployment patterns: a default singl | Evidence model | Events can combine spectra, N42 files, uploaded evidence, QR code data, videos, Web ID results, adjudication history, and generated reports. | | Major integrations | Rapiscan, Aspect, and RSI lanes, FFmpeg-recognized camera streams, Web ID, Cambio, MQTT, and a role-aware file API. | + +## 3.5.1 packaged release note + +This manual now includes **OSCAR 3.5.1** packaged deployment guidance for the current prebuilt release workflow. + +The most important operational changes in this release line are: + +- Java 21 and Docker are required prerequisites for packaged launchers +- the launchers now validate dependencies before startup +- the top-level launch and monitor scripts detect already-running OSCAR instances +- the monitor scripts can either attach to or replace an already-running OSCAR process +- profile-based Java and PostgreSQL sizing is now part of the standard deployment flow +- MediaMTX is recommended for larger test or field deployments that reuse a smaller number of real camera streams + +For first-run field validation, the preferred workflow is to start with the monitoring wrapper and then generate a status report after the system reaches steady state. + # 2. System architecture OSCAR is best understood as a node-centric application host that serves both configuration and operator workflows while coordinating external devices and services. Lanes are the operational units. Each lane can have detector inputs, camera feeds, event history, and associated evidence. Events and statistics are persisted in PostgreSQL, while files such as videos, reports, site diagrams, CSV imports, and adjudication artifacts remain on the application host file system. @@ -86,32 +102,40 @@ The diagram below condenses the system relationships. It is not a source-code cl # 3. Installation and initial startup -Installation is intentionally simple: download a release archive, extract it, ensure Docker is installed and running, and launch the platform with the OS-specific launch-all script (`launch-all.bat` for Windows, `launch-all.sh` for Linux/macOS, or `launch-all-arm.sh` for ARM systems). The database and application come up together in the default deployment path. +Installation for the current packaged release is intentionally simple: download the **OSCAR 3.5.1** release archive, extract it into a fresh directory, verify **Java 21+** and **Docker**, create `.env`, and start the system with the OS-specific top-level launcher or monitoring wrapper. The database and application come up together in the default deployment path. + +For test and side-by-side field deployment, the preferred first start is the monitoring wrapper rather than a direct node launch. ## Prerequisites | **Item** | **Why it matters** | **Notes** | |---------------------------------|-----------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------| -| Supported host OS | The release package contains OS-specific launch scripts. | Windows has the strongest support, although the packaging also includes scripts for other operating systems. | -| Docker | Required for the default local PostgreSQL deployment. | PostgreSQL replaces the older embedded H2 option in the current release line. | +| Supported host OS | The release package contains OS-specific launch scripts. | Windows and Linux are the primary packaged deployment paths discussed in the 3.5.1 workflow. | +| Java 21 or newer | Required by the packaged launch and monitoring scripts. | The launchers validate the Java version at startup and fail fast if Java is missing or too old. | +| Docker | Required for the default local PostgreSQL deployment. | Docker must already be running before `launch-all` or `monitor-oscar` is started. | | Browser with PWA support | Needed for installed desktop/mobile viewer experiences. | Chrome-like browsers support the install flow; notifications depend on secure hosting. | | Optional SSL keystore | Needed to host the application over HTTPS. | The HTTPS configuration expects a keystore path, password, alias, and selected port. | | Optional remote PostgreSQL host | Useful for larger sites or more robust infrastructure. | Remote database connection parameters can be entered in the admin configuration. | | Optional Web ID endpoint | Enables automated isotope analysis for evidence and RS350 data. | A remote default root can be used, but the root URL is configurable for local or offline hosting. | +| Optional MediaMTX host | Recommended for larger camera-heavy systems. | MediaMTX can proxy a smaller set of upstream camera streams to many logical lane assignments. | ## Startup sequence -> 1\. Download the desired release from the repository releases page and extract it to a working directory. +> 1\. Download the desired release from the repository releases page and extract it to a **fresh working directory**. If an older extracted OSCAR release is already present on the same host, stop the old OSCAR JVM, remove the old PostGIS container and any old OSCAR-specific Docker network, and delete the old extracted folder before proceeding. > -> 2\. Install Docker and verify that the Docker service is running before starting OSCAR. +> 2\. Verify **Java 21 or newer** and **Docker** before starting OSCAR. > -> 3\. Run the launch-all script (`launch-all.bat`, `launch-all.sh`, or `launch-all-arm.sh`) for the operating system in use. In the default path, the script starts PostgreSQL locally in Docker and then starts the Java application. +> 3\. Create `.env`. In packaged releases this may mean renaming `env.txt` to `.env`. In source-style layouts this means copying `env.template` to `.env`. > -> 4\. Open the application on the configured port. Port 8282 is the baseline HTTP application port, and 8443 is a representative HTTPS configuration. +> 4\. For test and side-by-side field deployment, start with the monitoring wrapper (`monitor-oscar.bat` or `monitor-oscar.sh`). For routine startup without monitoring, use the top-level `launch-all` script. Avoid direct use of the node-level launch script unless you are debugging. > -> 5\. Sign in with the initial admin account. The initial password is configurable before launch, but the exact shipped username and password should be verified against the release README or package contents. +> 5\. The launchers now validate dependencies, detect already-running OSCAR instances, and either refuse to proceed or replace the running instance depending on `FORCE_RESTART`. The monitor wrappers can also attach to an already-running OSCAR process when `ATTACH_TO_EXISTING=1` is set. > -> 6\. Before production use, change the initial admin password using the dot-prefixed settings or environment file and run the set-initial-admin-password script so the password is written in the hashed form expected by the system. +> 6\. Open the application on the configured port. Port 8282 is the baseline HTTP application port, and 8443 is a representative HTTPS configuration. +> +> 7\. Sign in with the initial admin account. The initial username is typically `admin`, but the exact packaged password should be verified from the release package or release notes rather than assuming a universal default. +> +> 8\. Before production use, change the initial admin password and then generate a first-run status report with `check-oscar-status` after the system reaches steady state. @@ -120,7 +144,7 @@ Installation is intentionally simple: download a release archive, extract it, en +

If multiple extracted versions exist on the same machine, explicitly stop the old OSCAR JVM and remove the old `oscar-postgis-container` before launching the new package. Reusing an old container or old extracted directory can populate data in the wrong place and create confusion during configuration or tuning.

@@ -380,6 +404,18 @@ Configuration of video retention lives in the “OSCAR Service Module.” Configuration of additional users / permissions exists under the “Security” tab. Users can be configured with fine-grained permissions to access / write to the system. + +## 3.5.1 deployment operations guidance + +For packaged 3.5.1 deployments, the most practical first-run validation sequence is: + +1. launch with the monitoring wrapper +2. let the system warm up long enough to reach a steady operating state +3. generate the one-file status report +4. confirm that Java memory, thread count, and PostgreSQL session counts plateau rather than climb continuously + +This deployment workflow is especially important when evaluating reconnect churn, thread growth, or database session growth in larger simulated or camera-heavy systems. + # 9. Performance, scale, and deployment guidance Default deployment begins to strain as lane counts, event volume, and camera counts increase. The following guidance is useful for planning, even though it is not a formal support limit. @@ -483,3 +519,25 @@ This section captures known gaps and enhancement ideas so they are not confused | PWA | Progressive web app. The installable version of the viewer for desktop or mobile use. | | RS350 | A supported backpack-style mobile detector lane type. | + +# Appendix C. Packaged 3.5.1 field-deployment checklist + +> 1\. Verify Java 21+ and Docker. +> +> 2\. If an older OSCAR release was previously used on the host, stop the old OSCAR JVM, remove the old PostGIS container and old OSCAR-specific Docker network, and delete the old extracted release folder. +> +> 3\. Extract the 3.5.1 release into a fresh directory. +> +> 4\. Create `.env` from the packaged environment file and confirm the correct `SYSTEM_PROFILE`. +> +> 5\. Prefer the monitoring wrapper for first-run validation. +> +> 6\. If OSCAR is already running, decide whether to stop it, set `FORCE_RESTART=1`, or use `ATTACH_TO_EXISTING=1` for the monitor wrapper. +> +> 7\. If the deployment uses many camera references, put MediaMTX in front of the cameras and point the OSCAR lane CSV to MediaMTX paths instead of directly to every physical camera. +> +> 8\. After warm-up, generate the one-file status report and confirm memory, threads, and PostgreSQL sessions have stabilized. +> +> 9\. Change the initial admin password before production use. +> +> 10\. Keep the launch/monitoring guide and MediaMTX guide with the release package so operators follow the same tested startup path. diff --git a/dist/documentation/OSCAR_launch_monitoring_guide.md b/dist/documentation/OSCAR_launch_monitoring_guide.md new file mode 100644 index 0000000..b799b2f --- /dev/null +++ b/dist/documentation/OSCAR_launch_monitoring_guide.md @@ -0,0 +1,511 @@ +# OSCAR launch, monitoring, and database diagnostics guide + +This guide covers the current **OSCAR 3.5.1 packaged deployment workflow** for Linux and Windows. + +It explains: + +- how to prepare a fresh prebuilt release +- how to create `.env` +- how the updated `launch-all`, `launch`, `monitor-oscar`, and `check-oscar-status` scripts behave +- how already-running OSCAR instances are handled +- how to validate Java, Docker, memory, and database health +- how to use MediaMTX and status reports during testing and side-by-side field deployment + +--- + +## 1. Recommended operating model + +For **testing, burn-in, and side-by-side field deployment**, the preferred workflow is: + +1. unzip the prebuilt release into a fresh folder +2. create `.env` +3. verify **Java 21+** and **Docker** +4. start with the **monitoring script** +5. let the system warm up +6. run the **status-check script** +7. review JVM, thread, and PostgreSQL behavior before wider deployment + +Use the top-level **sessionless launchers** when possible: + +- `launch-all.sh` / `launch-all.bat` +- `monitor-oscar.sh` / `monitor-oscar.bat` + +Avoid launching `osh-node-oscar/launch.(sh|bat)` directly unless you are debugging the node itself. + +--- + +## 2. Fresh install and upgrade cleanup + +If the machine has previously run OSCAR, clean up the old deployment before extracting **OSCAR 3.5.1**. + +### Linux + +```bash +pgrep -af 'com.botts.impl.security.SensorHubWrapper' +kill + +docker rm -f oscar-postgis-container +docker network rm oscar-postgis-network || true +rm -rf /path/to/old/oscar-3.5.0 +``` + +### Windows PowerShell + +```powershell +Get-CimInstance Win32_Process | + Where-Object { + $_.Name -match '^java(\.exe)?$' -and + $_.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*' + } | + Select-Object ProcessId, CommandLine + +Stop-Process -Id -Force + +docker rm -f oscar-postgis-container +docker network rm oscar-postgis-network +Remove-Item -Recurse -Force .\oscar-3.5.0 +``` + +If the Docker network does not exist, that is fine. The goal is to avoid carrying old container state or an old extracted release folder into the new test run. + +--- + +## 3. Required dependencies + +### Linux + +Required: + +- Bash +- Java 21 or newer +- `keytool` +- Docker + +Recommended: + +- `jcmd` +- `pmap` +- `free` +- `vmstat` + +Ubuntu example: + +```bash +sudo apt update +sudo apt install openjdk-21-jdk docker.io procps psmisc +``` + +Verify: + +```bash +java -version +which keytool +which docker +which jcmd +``` + +### Windows + +Required: + +- PowerShell +- Java 21 or newer +- Docker Desktop or Docker Engine + +Recommended: + +- `jcmd.exe` + +Verify: + +```powershell +java -version +docker version +Get-Command java +Get-Command docker +Get-Command jcmd +``` + +The Windows launchers now use **PowerShell/CIM**-based process discovery. They do not depend on `wmic`. + +--- + +## 4. Environment file setup + +Create `.env` before launch. + +- if the packaged release ships **env.txt**, rename it to **.env** +- if the source tree ships **env.template**, copy it to **.env** + +Linux: + +```bash +cp env.template .env +``` + +Windows PowerShell: + +```powershell +Copy-Item .\env.template .\.env +``` + +### Core settings + +```dotenv +SYSTEM_PROFILE=16GB +DB_NAME=gis +DB_USER=postgres +DB_PASSWORD=postgres +DB_PORT=5432 +DB_HOST=localhost +CONTAINER_NAME=oscar-postgis-container +``` + +### Process and monitor behavior settings + +```dotenv +FORCE_RESTART=0 +ATTACH_TO_EXISTING=0 +MAX_WAIT_SECONDS=300 +RETRY_MAX=120 +RETRY_INTERVAL=2 +POSTGIS_READY_DELAY=5 +``` + +### What these mean + +- `FORCE_RESTART=0` -> refuse to start if OSCAR is already running +- `FORCE_RESTART=1` -> stop the running OSCAR instance and start fresh +- `ATTACH_TO_EXISTING=0` -> monitor script refuses to attach to a running OSCAR process +- `ATTACH_TO_EXISTING=1` -> monitor script attaches to the running OSCAR process instead of replacing it +- `MAX_WAIT_SECONDS` -> how long monitor scripts wait for the Java process to appear +- `RETRY_MAX`, `RETRY_INTERVAL`, `POSTGIS_READY_DELAY` -> PostGIS readiness timing + +--- + +## 5. Profile-based sizing + +The launchers size Java and PostgreSQL by `SYSTEM_PROFILE`. + +Supported profiles: + +- `RPI4` +- `8GB` +- `16GB` +- `32GB` + +Representative PostgreSQL `max_connections` values: + +- `RPI4` -> 75 +- `8GB` -> 125 +- `16GB` -> 200 +- `32GB` -> 300 + +The launchers also set: + +- `superuser_reserved_connections=10` +- `idle_session_timeout=600000` +- connection and disconnection logging + +--- + +## 6. Launch scripts and what they do + +### `launch-all.sh` / `launch-all.bat` + +These are the supported top-level launchers. + +They: + +- load `.env` +- validate Java and Docker +- detect an already-running OSCAR instance +- size PostgreSQL for the selected profile +- rebuild or reuse the PostGIS image +- remove the existing named PostGIS container if necessary +- start a new PostGIS container with the current settings +- wait for PostgreSQL readiness +- call `osh-node-oscar/launch.(sh|bat)` + +### `osh-node-oscar/launch.sh` / `osh-node-oscar/launch.bat` + +These launch only the OSCAR Java node. + +They: + +- load `.env` +- validate Java and keytool +- detect an already-running OSCAR instance +- choose heap and JavaCPP settings for the selected profile +- build or refresh the Java trust store +- initialize the packaged admin password flow +- start the Java process with Native Memory Tracking enabled + +Use these direct node launchers mainly for debugging. + +--- + +## 7. How already-running OSCAR instances are handled + +### Launch behavior + +By default, if an OSCAR JVM is already running, `launch-all` and `launch` stop and print an error instead of silently starting another instance. + +Typical message: + +```text +OSCAR is already running with PID(s): ... +Stop the running instance first, or set FORCE_RESTART=1 to replace it. +``` + +### Force replacement + +Set: + +```dotenv +FORCE_RESTART=1 +``` + +Then the scripts attempt to stop the running OSCAR process before starting a new one. + +### Monitor attach behavior + +For monitor wrappers, you have two supported choices: + +- `FORCE_RESTART=1` -> replace the running OSCAR instance and monitor the new one +- `ATTACH_TO_EXISTING=1` -> keep the running OSCAR instance and attach monitoring to it + +--- + +## 8. Starting OSCAR + +### Recommended first-run start with monitoring + +#### Linux + +```bash +chmod +x launch-all.sh osh-node-oscar/launch.sh monitor-oscar.sh check-oscar-status.sh +./monitor-oscar.sh +``` + +#### Windows + +```bat +monitor-oscar.bat +``` + +This creates an output directory such as: + +```text +oscar-monitor-20260505-032622 +``` + +and captures: + +- launch stdout and stderr +- JVM PID information +- JFR status +- GC heap information +- native memory summaries +- thread dumps +- Docker status +- PostgreSQL session and activity data +- trend CSV files for database sessions + +### Routine start without monitoring + +#### Linux + +```bash +./launch-all.sh +``` + +#### Windows + +```bat +launch-all.bat +``` + +--- + +## 9. Stopping OSCAR + +### Linux + +If you started with `monitor-oscar.sh`, stop the monitor shell or the Java PID it discovered. + +To stop a running OSCAR JVM manually: + +```bash +pgrep -af 'com.botts.impl.security.SensorHubWrapper' +kill +``` + +To stop PostGIS: + +```bash +docker stop oscar-postgis-container +``` + +### Windows PowerShell + +```powershell +Get-CimInstance Win32_Process | + Where-Object { + $_.Name -match '^java(\.exe)?$' -and + $_.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*' + } | + Select-Object ProcessId, CommandLine + +Stop-Process -Id -Force +docker stop oscar-postgis-container +``` + +The Windows monitor script also supports: + +```bat +monitor-oscar.bat stop +``` + +--- + +## 10. Status reports + +### Linux + +```bash +./check-oscar-status.sh +``` + +### Windows + +```powershell +powershell -ExecutionPolicy Bypass -File .\check-oscar-status.ps1 +``` + +These scripts summarize the latest monitor run into a single text report that includes: + +- process status +- live JVM information +- heap and native memory summaries +- current container status +- PostgreSQL activity snapshots +- first-versus-latest trend comparison +- recent log tails + +--- + +## 11. What healthy startup looks like + +A healthy first-run profile typically looks like this: + +- Java RSS rises during startup and then levels out +- thread count rises during startup and then stabilizes +- PostgreSQL sessions rise during startup and then plateau well below usable client slots +- swap or pagefile usage stays low +- `db-error` remains empty + +### Important PostgreSQL rule + +```text +usable client slots = max_connections - superuser_reserved_connections +``` + +If total sessions keep climbing toward that number, PostgreSQL is nearing saturation. + +--- + +## 12. Interpreting the monitor output + +Key files in a monitor directory: + +- `launch.stdout.log` +- `launch.stderr.log` +- `jvm-pid.txt` +- `db-connection-trend.csv` +- one timestamped snapshot directory per interval + +Key per-snapshot files: + +- `nmt-summary.txt` +- `gc-heap-info.txt` +- `thread-print.txt` +- `db-max-connections.txt` +- `db-total-sessions.txt` +- `db-by-state.txt` +- `db-by-app.txt` +- `db-activity-detail.txt` +- `docker-logs-tail.txt` + +Use `db-connection-trend.csv` as the fastest way to spot connection growth, plateauing, or saturation. + +--- + +## 13. MediaMTX during field testing + +For larger camera configurations or side-by-side test deployments: + +1. start OSCAR with `monitor-oscar` +2. route camera streams through MediaMTX +3. let the system run long enough to capture reconnect and thread behavior +4. run `check-oscar-status` +5. compare JVM threads, reconnect logs, and PostgreSQL sessions before and after enabling MediaMTX + +MediaMTX is especially helpful when many logical lane-camera assignments reuse a smaller number of real camera streams. + +--- + +## 14. Troubleshooting checklist + +### Launch fails before Java starts + +Check: + +- `.env` exists +- Java 21+ is installed +- Docker is running +- required directories such as `osh-node-oscar/lib` and `osh-node-oscar/nativelibs` exist +- the trust store and keystore files exist where the launch script expects them + +### Monitor hangs waiting for Java + +Check `launch.stdout.log` and `launch.stderr.log` inside the newest monitor directory. + +Common causes: + +- PostGIS container startup failed +- the OSCAR Java process exited immediately after launch +- a required runtime path is missing +- a certificate, trust store, or password-initialization step failed + +### PostgreSQL sessions keep climbing + +Inspect: + +- `db-total-sessions.txt` +- `db-by-state.txt` +- `db-by-app.txt` +- `db-activity-detail.txt` +- `db-connection-trend.csv` + +### Thread count keeps climbing + +Inspect: + +- `thread-print.txt` +- reconnect-related warnings in `launch.stdout.log` +- MediaMTX versus direct-camera behavior under the same test workload + +--- + +## 15. Files you normally should not delete + +Do not delete these unless you intentionally want to reset state: + +- `.env` +- `osh-node-oscar/db/` +- `pgdata/` +- `osh-node-oscar/osh-keystore.p12` +- `osh-node-oscar/truststore.jks` + +It is safe to delete old monitor directories and generated status reports once you have kept the reports you need. diff --git a/dist/documentation/Release_Notes_3.5.1.md b/dist/documentation/Release_Notes_3.5.1.md new file mode 100644 index 0000000..2d6f9dc --- /dev/null +++ b/dist/documentation/Release_Notes_3.5.1.md @@ -0,0 +1,533 @@ +# Release Notes + +## Overview + +OSCAR **3.5.1** improves deployment stability, observability, and scalability for larger multi-lane systems. This release focuses on: + +* reducing memory pressure +* preventing PostgreSQL connection exhaustion +* improving runtime diagnostics +* simplifying deployment on Linux and Windows +* improving support for MediaMTX-based camera proxy deployments +* making launch and monitoring behavior safer when OSCAR is already running +* improving first-run dependency and startup validation + +These changes were validated against a high-load configuration monitoring **50 radiation portal monitors and 100 camera streams**. + +This is a **prebuilt release**. Users should **unzip OSCAR 3.5.1 into a fresh directory** and start it with the included **monitoring script**, preferably using the **sessionless launch** when possible. + +--- + +## Before you start + +### Required dependencies + +Install these before running OSCAR 3.5.1: + +* **OpenJDK 21** +* **Docker** + +### Recommended deployment model + +For testing, side-by-side field deployment, and first-run validation: + +* unzip the release into a **new clean folder** +* rename `env.txt` to `.env` if needed +* select the correct system profile in `.env` +* use **MediaMTX** for camera-heavy deployments +* start OSCAR with the **sessionless monitoring launch** when possible +* use the **check/status script** to review performance + +--- + +## What is new + +### Profile-based system sizing + +Deployment now supports profile-based resource tuning instead of using one fixed memory configuration for every machine. + +Supported profiles: + +* `RPI4` +* `8GB` +* `16GB` +* `32GB` + +These profiles allow the JVM and PostgreSQL configuration to be matched to the host hardware through the `.env` file and updated launch scripts. + +### Updated launch flow + +Launch scripts were updated for both Linux and Windows so they can: + +* load the selected system profile +* size Java heap appropriately for the machine +* size PostgreSQL more appropriately for the machine +* start the PostGIS container with tuned settings +* provide a more consistent startup path across environments +* check for required dependencies before launch +* stop or refuse duplicate OSCAR launches based on script settings + +### Safer process handling + +The launch and monitoring scripts now better handle cases where OSCAR is already running. + +Improvements include: + +* detection of already running OSCAR processes +* clearer behavior when a prior instance is found +* support for stopping and relaunching cleanly when configured to do so +* monitor behavior aligned with launch behavior end to end +* reduced risk of duplicate Java processes and conflicting monitor sessions + +This makes startup behavior safer during testing, upgrades, and repeated field launches. + +### Dependency and environment validation + +Deployment scripts now better validate startup prerequisites and packaged paths before launch. + +Improvements include: + +* dependency checks for **Java 21** and **Docker** +* clearer startup errors when required tools are missing +* improved trust store handling on Windows +* better validation of expected runtime directories and packaged files +* updated environment template support for launch and monitor behavior + +These changes make prebuilt deployment more reliable, especially on fresh Windows systems. + +### PostgreSQL tuning improvements + +PostgreSQL startup settings were updated to better support larger deployments. + +Improvements include: + +* increased connection limits by profile +* reduced per-connection memory pressure +* reserved superuser or admin connection slots +* idle session timeout support +* connection and disconnection logging for diagnostics + +For the 16 GB profile, PostgreSQL was raised from the earlier 100-connection ceiling to a higher-capacity configuration, resolving the immediate `too many clients already` failure mode during large-scale operation. + +### Hikari connection pool fix + +The main cause of database session over-allocation was identified and corrected in the PostGIS datastore connection manager. + +#### Root cause + +* each Hikari pool was configured with `maximumPoolSize(20)` +* `minimumIdle` was not set +* Hikari therefore defaulted `minimumIdle` to the same value as `maximumPoolSize` +* with multiple pools active, the system held a very large number of idle PostgreSQL sessions open at all times + +#### Fix + +* reduced per-pool size +* explicitly set `minimumIdle(0)` +* shortened idle timeout behavior +* preserved sufficient active connection capacity while eliminating unnecessary idle connection hoarding + +#### Result observed in testing + +* PostgreSQL steady-state sessions dropped from about **186** to about **21** +* idle JDBC sessions dropped from about **180** to about **15** +* database headroom increased substantially +* the immediate Postgres connection saturation problem was eliminated + +This is the most important backend stability improvement in this release. + +### Monitoring and status scripts + +New monitoring and status-check scripts were added for both Linux and Windows. + +These scripts can now: + +* launch OSCAR under monitoring +* support sessionless launch for normal deployment use +* capture JVM memory status +* capture native memory tracking summaries +* capture JFR status +* capture OS memory and swap usage +* capture PostgreSQL session counts and saturation state +* capture database activity detail +* produce a single-file health and status report for rapid review + +### Improved database diagnostics + +Monitoring now includes PostgreSQL visibility such as: + +* `max_connections` +* `superuser_reserved_connections` +* total active sessions +* session state counts +* connection trend logging over time +* recent PostgreSQL log activity + +This makes it much easier to distinguish between: + +* memory pressure +* connection pool over-allocation +* true connection leaks +* normal steady-state pool behavior + +### MediaMTX deployment guidance + +Documentation was added for using **MediaMTX** as a local RTSP proxy layer to reduce the resource burden of handling many camera streams directly in OSCAR. + +This supports a deployment model where: + +* a smaller number of upstream camera streams are proxied locally +* multiple lanes can reuse proxied feeds +* OSCAR connects to stable local endpoints instead of managing a large number of direct camera connections + +This architecture is recommended for larger systems and appears to reduce camera-related reconnect burden. + +### Documentation updates + +Documentation was added or expanded for: + +* `.env` usage +* launch scripts +* monitoring scripts +* check or status scripts +* profile selection +* dependency installation and verification +* startup and shutdown procedures +* data interpretation and troubleshooting +* MediaMTX camera proxy setup +* already-running instance handling +* environment template settings for restart and attach behavior + +--- + +## Problems addressed + +### Memory pressure from fixed JVM sizing + +Previous deployments used a one-size-fits-all Java memory model. On smaller or moderately sized machines, this could reserve too much memory for the JVM and reduce operating system and PostgreSQL headroom, increasing swap or pagefile pressure. + +### PostgreSQL connection exhaustion + +Large deployments were exhausting PostgreSQL connection capacity because multiple Hikari pools were keeping too many idle connections open. This caused: + +* `too many clients already` +* Hikari connection timeouts +* database degradation under load + +### Duplicate launch and monitoring confusion + +Repeated test starts could leave users uncertain whether OSCAR was already running, whether a second Java process had been created, or whether the monitor had attached to the correct instance. + +The updated scripts address this by making existing-instance behavior more explicit and consistent. + +### Limited visibility into failure mode + +Earlier logs were often noisy or incomplete during failures. The new monitoring and status scripts make it easier to determine whether the bottleneck is: + +* Java heap +* native memory +* swap usage +* PostgreSQL session pressure +* query activity +* startup or reconnect churn + +--- + +## Behavior observed after the fixes + +After applying the connection pool fix and updated deployment tuning: + +* PostgreSQL sessions dropped from about **186** to about **21** in testing +* idle JDBC sessions dropped from about **180** to about **15** +* JVM swap usage dropped to **0** in the improved test run +* the system remained stable during startup and warm-up +* database headroom improved dramatically + +--- + +## If you previously ran OSCAR + +If this machine was already running an older OSCAR release, do **not** install OSCAR 3.5.1 over the top of the old directory. + +Before starting OSCAR 3.5.1, stop and remove the older deployment components: + +* stop the old PostGIS container +* remove the old PostGIS container +* remove the old Docker network used by the previous OSCAR deployment, **if one exists** +* stop any old OSCAR Java process that is still running +* delete the old `oscar-3.5.0` directory +* unzip OSCAR **3.5.1** into a fresh folder + +### Linux cleanup example + +Stop and remove the old PostGIS container: + +```bash +docker stop oscar-postgis-container 2>/dev/null || true +docker rm oscar-postgis-container 2>/dev/null || true +``` + +If the previous deployment created a dedicated Docker network, remove it after the container is gone: + +```bash +docker network ls +docker network rm +``` + +Stop any running OSCAR Java process if needed: + +```bash +pkill -f 'com.botts.impl.security.SensorHubWrapper' || true +``` + +Remove the previous OSCAR folder: + +```bash +rm -rf ~/oscar-3.5.0 +``` + +### Windows cleanup example + +Stop and remove the old PostGIS container: + +```powershell +docker stop oscar-postgis-container +docker rm oscar-postgis-container +``` + +If the previous deployment created a dedicated Docker network, remove it after the container is gone: + +```powershell +docker network ls +docker network rm +``` + +Stop any running OSCAR Java process if needed: + +```powershell +Get-CimInstance Win32_Process | + Where-Object { + $_.Name -match '^java(\.exe)?$' -and + $_.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*' + } | + ForEach-Object { Stop-Process -Id $_.ProcessId -Force } +``` + +Delete the previous OSCAR folder: + +```powershell +Remove-Item -Recurse -Force .\oscar-3.5.0 +``` + +If you are unsure whether a dedicated OSCAR Docker network exists, list networks first and remove only the one associated with the old OSCAR deployment. + +--- + +## Fresh install workflow for OSCAR 3.5.1 + +### Step 1: unzip the release + +Extract OSCAR **3.5.1** into a new folder. + +Example: + +```text +oscar-3.5.1/ +``` + +### Step 2: confirm dependencies + +Make sure the machine has: + +* **OpenJDK 21** +* **Docker** + +### Step 3: configure the environment file + +The release may include the environment file as: + +```text +env.txt +``` + +Rename it to: + +```text +.env +``` + +Then edit the file and select the correct hardware profile: + +* `RPI4` +* `8GB` +* `16GB` +* `32GB` + +The environment template also supports launch and monitoring behavior such as restart and attach settings. + +### Step 4: start with the monitoring script + +For OSCAR 3.5.1, users should launch with the monitoring script so diagnostics begin immediately. + +Use the **sessionless launch** when possible so OSCAR keeps running without requiring an attached terminal session. + +#### Linux + +Preferred: + +```bash +./monitor-oscar.sh --daemon +``` + +If your script version starts sessionless by default, use: + +```bash +./monitor-oscar.sh +``` + +Use an attached launch only for interactive troubleshooting. + +#### Windows + +Preferred: + +```bat +monitor-oscar.bat +``` + +Use the sessionless option if your Windows wrapper provides both attached and detached modes. + +Use an attached launch only for interactive troubleshooting. + +### Step 5: check performance with the included status script + +After startup, and again after the system has been running for a while, generate a status report. + +#### Linux + +```bash +./check-oscar-status.sh +``` + +#### Windows + +```powershell +powershell -ExecutionPolicy Bypass -File .\check-oscar-status.ps1 +``` + +This report helps verify that memory, swap, and PostgreSQL usage remain healthy. + +--- + +## Recommended field test workflow + +For testing and side-by-side field deployment, users should: + +1. stop and remove any old OSCAR PostGIS container +2. remove the old OSCAR Docker network **if one exists** +3. stop any old OSCAR Java process if one is still running +4. delete the old `oscar-3.5.0` folder +5. unzip OSCAR **3.5.1** into a fresh folder +6. install or verify **OpenJDK 21** and **Docker** +7. rename `env.txt` to `.env` if needed +8. select the correct profile in `.env` +9. configure and use **MediaMTX** for camera-heavy systems +10. start OSCAR with the **sessionless monitoring launch** when possible +11. use the check or status script to compare system behavior and performance + +This is the preferred workflow for: + +* first-time deployment on a machine +* side-by-side comparison with another build +* validating memory behavior +* validating PostgreSQL behavior +* validating MediaMTX camera proxy performance + +--- + +## Included updates + +### Linux + +* `.env`-based configuration +* `launch-all.sh` +* `launch.sh` +* `monitor-oscar.sh` +* `check-oscar-status.sh` + +### Windows + +* `.env`-based configuration +* `launch-all.bat` +* `launch.bat` +* `monitor-oscar.bat` +* `check-oscar-status.ps1` + +--- + +## Recommended operating model + +### Deployment + +* select the correct hardware profile in `.env` +* use the updated launch scripts +* use the **sessionless monitoring launch** for initial validation and normal field deployment when possible +* use the attached launch only for interactive troubleshooting +* let the scripts manage already-running instances instead of manually launching duplicates +* use MediaMTX where many camera streams are involved +* review generated status reports during early burn-in testing + +### Validation after upgrade + +After upgrading, confirm that: + +* PostgreSQL sessions plateau well below the configured connection limit +* swap usage remains low or zero +* JVM RSS stabilizes after startup +* thread count does not continuously climb over long runs +* database status reports do not show saturation errors + +--- + +## Known issues still under observation + +These changes significantly improve stability, but a few items are still worth monitoring: + +* `RapiscanSensor` parse errors such as `For input string: "000NaN"` +* repeated MQTT `Broken pipe` errors +* high thread counts in some runs +* reconnect churn on certain devices or services + +These do not appear to be the primary cause of the major stability issue addressed in this release, but they remain candidates for future cleanup. + +--- + +## Upgrade notes + +1. Stop and remove any previous OSCAR PostGIS container. +2. Remove the previous OSCAR Docker network **if one exists**. +3. Stop any previous OSCAR Java process that is still running. +4. Delete the old `oscar-3.5.0` directory. +5. Unzip OSCAR **3.5.1** into a fresh directory. +6. Install **OpenJDK 21** and **Docker**. +7. Rename `env.txt` to `.env` if needed. +8. Edit `.env` and select the correct hardware profile. +9. For camera-heavy deployments, configure MediaMTX. +10. Start the system with the **sessionless monitoring launch** when possible. +11. Use the check or status script after startup and again after runtime burn-in. + +--- + +## Summary + +This release materially improves OSCAR behavior on larger systems by: + +* matching resource use to host hardware +* reducing unnecessary database connection retention +* improving monitoring and diagnostics +* increasing deployment consistency across Linux and Windows +* supporting MediaMTX-based camera proxy architectures +* validating dependencies and packaged startup requirements earlier +* handling already-running OSCAR instances more safely + +The biggest backend improvement is the correction of oversized Hikari idle pooling, which reduced PostgreSQL session usage from approximately **186** to approximately **21** in testing. diff --git a/dist/documentation/Standard_PostgreSQL_Setup.md b/dist/documentation/Standard_PostgreSQL_Setup.md new file mode 100644 index 0000000..c6f2a10 --- /dev/null +++ b/dist/documentation/Standard_PostgreSQL_Setup.md @@ -0,0 +1,101 @@ +# Standard PostgreSQL Database Setup + +If you are deploying OSCAR with a standard, standalone PostgreSQL database (rather than the default Dockerized option), follow these steps to initialize and configure the database properly. + +## Prerequisites + +- PostgreSQL (version 16 recommended, matching the Docker image) +- PostGIS extensions installed on the database server + +## Step 1: Create the Database + +Connect to your PostgreSQL instance as a superuser (e.g., `postgres`) and create the `gis` database: + +```sql +CREATE DATABASE gis; +``` + +## Step 2: Configure System Parameters + +Set the required `max_connections` limit: + +```sql +ALTER SYSTEM SET max_connections = 1024; +``` + +_Note: You will need to reload or restart the PostgreSQL service for system-level parameter changes to take effect._ + +## Step 3: Enable PostGIS and Required Extensions + +Connect to the newly created `gis` database. If using `psql`, you can do this by running: + +```sql +\connect gis; +``` + +Then, run the following SQL commands to enable the necessary extensions required by the OSCAR system: + +```sql +CREATE EXTENSION IF NOT EXISTS pg_trgm; +CREATE EXTENSION IF NOT EXISTS btree_gist; +CREATE EXTENSION IF NOT EXISTS btree_gin; +CREATE EXTENSION IF NOT EXISTS fuzzystrmatch; +CREATE EXTENSION IF NOT EXISTS postgis; +CREATE EXTENSION IF NOT EXISTS postgis_tiger_geocoder; +CREATE EXTENSION IF NOT EXISTS postgis_topology; +``` + +## Step 4: Configure User Credentials and Access + +Ensure your database is accessible to the OSCAR application: + +1. Configure appropriate host-based authentication in your `pg_hba.conf` file to allow the OSCAR server to connect. +2. Ensure the user connecting to the database has sufficient privileges on the `gis` database. + +## Step 5: Configure OSCAR Database Connection + +Once the database is set up, you must configure OSCAR to connect to it. This can be done in one of two ways: + +### Option A: Edit `config.json` Directly (Pre-launch) + +Before starting the OSCAR application, you can edit the `dist/config/standard/config.json` file. Locate the configuration module for `SystemDriverDatabaseConfig` containing the `PostgisObsSystemDatabaseConfig` and update the connection details. + +Find the block that looks similar to this: + +```json +{ + "objClass": "org.sensorhub.impl.database.system.SystemDriverDatabaseConfig", + "dbConfig": { + "objClass": "org.sensorhub.impl.datastore.postgis.database.PostgisObsSystemDatabaseConfig", + "url": "localhost:5432", + "dbName": "gis", + "login": "postgres", + "password": "postgres", + "idProviderType": "SEQUENTIAL", + "autoCommitPeriod": 10, + "useBatch": false, + "id": "bfbd6d58-1a4a-40b4-999d-381a1489cbb5", + "autoStart": false, + "moduleClass": "org.sensorhub.impl.datastore.postgis.database.PostgisObsSystemDatabase" + }, + // ... other fields + "name": "PostGIS Database" +} +``` + +Update the following fields to match your standalone database configuration: + +- `url`: The hostname or IP address of your PostgreSQL server and the port (e.g., `db.example.com:5432`). +- `dbName`: The database name (should be `gis` if you followed Step 1). +- `login`: The username for the database. +- `password`: The password for the database user. + +### Option B: Use the OSCAR Admin Panel GUI (Post-launch) + +If OSCAR is already running (and potentially failing to connect to its default database), you can update the settings through the web administration interface: + +1. Log in to the OSCAR Admin Panel (e.g., `http://localhost:8282/sensorhub/admin`). +2. Navigate to the **Databases** tab. +3. Click on the **PostGIS Database** module. +4. Update the **URL**, **Database Name**, **Login**, and **Password** fields with your standalone database details. +5. Save the configuration and restart the module. diff --git a/dist/release/check-oscar-status.ps1 b/dist/release/check-oscar-status.ps1 new file mode 100644 index 0000000..f9f667f --- /dev/null +++ b/dist/release/check-oscar-status.ps1 @@ -0,0 +1,136 @@ +param( + [string]$BaseDir = ".", + [string]$MonitorDir = "", + [string]$OutFile = "" +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'SilentlyContinue' +Set-Location $BaseDir + +if ([string]::IsNullOrWhiteSpace($MonitorDir)) { + $MonitorDir = Get-ChildItem -Directory -Filter 'oscar-monitor-*' | Sort-Object LastWriteTime -Descending | Select-Object -First 1 -ExpandProperty FullName +} +if (-not $MonitorDir -or -not (Test-Path $MonitorDir)) { throw 'No oscar-monitor-* directory found.' } +if ([string]::IsNullOrWhiteSpace($OutFile)) { $OutFile = Join-Path (Get-Location) ("oscar-status-{0}.txt" -f (Get-Date -Format 'yyyyMMdd-HHmmss')) } + +$snaps = Get-ChildItem -Path $MonitorDir -Directory | Sort-Object Name +$first = $snaps | Select-Object -First 1 +$last = $snaps | Select-Object -Last 1 +$jvmPidFile = Join-Path $MonitorDir 'jvm-pid.txt' +$pidFromMonitor = if (Test-Path $jvmPidFile) { (Get-Content $jvmPidFile | Select-Object -First 1).Trim() } else { '' } +$javaProc = Get-CimInstance Win32_Process | Where-Object { $_.Name -eq 'java.exe' -and $_.CommandLine -match 'SensorHubWrapper' } | Select-Object -First 1 + +function Read-Text([string]$Path) { if (Test-Path $Path) { Get-Content $Path -Raw } else { '' } } +function Read-First([string]$Path) { if (Test-Path $Path) { (Get-Content $Path | Select-Object -First 1).Trim() } else { '' } } +function Extract-DbCount([string]$Path, [string]$State) { + if (-not (Test-Path $Path)) { return '' } + foreach ($line in Get-Content $Path) { + $parts = $line -split '\|' + if ($parts.Count -ge 2 -and $parts[0].Trim() -eq $State) { return $parts[1].Trim() } + } + '' +} +function Calc-Slots($MaxConn, $Reserved) { + if ($MaxConn -match '^\d+$' -and $Reserved -match '^\d+$') { return [int]$MaxConn - [int]$Reserved } + '' +} + +$sb = [System.Text.StringBuilder]::new() +$null = $sb.AppendLine('OSCAR STATUS REPORT') +$null = $sb.AppendLine("Generated: $(Get-Date -Format o)") +$null = $sb.AppendLine("Base directory: $(Get-Location)") +$null = $sb.AppendLine("Monitor directory: $MonitorDir") +$null = $sb.AppendLine("Output file: $OutFile") +$null = $sb.AppendLine() +$null = $sb.AppendLine('=== PROCESS STATUS ===') +$null = $sb.AppendLine("PID from monitor: $pidFromMonitor") +$null = $sb.AppendLine(("Live OSCAR PID: {0}" -f ($(if ($javaProc) { $javaProc.ProcessId } else { '' })))) +$null = $sb.AppendLine() +$null = $sb.AppendLine((Get-CimInstance Win32_Process | Where-Object { $_.CommandLine -match 'monitor-oscar' } | Select-Object ProcessId, Name, CommandLine | Format-Table -AutoSize | Out-String)) +$null = $sb.AppendLine((Get-CimInstance Win32_Process | Where-Object { $_.Name -eq 'java.exe' -and $_.CommandLine -match 'SensorHubWrapper' } | Select-Object ProcessId, Name, CommandLine | Format-Table -AutoSize | Out-String)) +$null = $sb.AppendLine((& docker ps --filter name=oscar-postgis-container | Out-String)) +$null = $sb.AppendLine('=== SYSTEM MEMORY AND PAGEFILE ===') +$null = $sb.AppendLine((Get-CimInstance Win32_OperatingSystem | Select-Object TotalVisibleMemorySize,FreePhysicalMemory,TotalVirtualMemorySize,FreeVirtualMemory | Format-List | Out-String)) +$null = $sb.AppendLine((Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\% Usage' | Out-String)) + +if ($javaProc) { + $null = $sb.AppendLine('=== LIVE JVM PROCESS ===') + $null = $sb.AppendLine((Get-Process -Id $javaProc.ProcessId | Select-Object Id,ProcessName,Threads,VirtualMemorySize64,WorkingSet64,PrivateMemorySize64,CPU,StartTime | Format-List | Out-String)) + $null = $sb.AppendLine('=== LIVE JVM JFR STATUS ===') + $null = $sb.AppendLine((& jcmd $javaProc.ProcessId JFR.check | Out-String)) + $null = $sb.AppendLine('=== LIVE JVM GC HEAP INFO ===') + $null = $sb.AppendLine((& jcmd $javaProc.ProcessId GC.heap_info | Out-String)) + $null = $sb.AppendLine('=== LIVE JVM NATIVE MEMORY SUMMARY ===') + $null = $sb.AppendLine((& jcmd $javaProc.ProcessId VM.native_memory summary | Out-String)) +} + +$lastMax = Read-First (Join-Path $last.FullName 'db-max-connections.txt') +$lastReserved = Read-First (Join-Path $last.FullName 'db-superuser-reserved-connections.txt') +$lastTotal = Read-First (Join-Path $last.FullName 'db-total-sessions.txt') +$null = $sb.AppendLine('=== LIVE POSTGRES STATUS (FROM LAST SNAPSHOT) ===') +$null = $sb.AppendLine("max_connections: $lastMax") +$null = $sb.AppendLine("superuser_reserved_connections: $lastReserved") +$null = $sb.AppendLine("usable_client_slots: $(Calc-Slots $lastMax $lastReserved)") +$null = $sb.AppendLine("total_sessions: $lastTotal") +$null = $sb.AppendLine("active: $(Extract-DbCount (Join-Path $last.FullName 'db-by-state.txt') 'active')") +$null = $sb.AppendLine("idle: $(Extract-DbCount (Join-Path $last.FullName 'db-by-state.txt') 'idle')") +$null = $sb.AppendLine("idle in transaction: $(Extract-DbCount (Join-Path $last.FullName 'db-by-state.txt') 'idle in transaction')") +$null = $sb.AppendLine() +$null = $sb.AppendLine('--- db-by-state ---') +$null = $sb.AppendLine((Read-Text (Join-Path $last.FullName 'db-by-state.txt'))) +$null = $sb.AppendLine('--- db-by-app ---') +$null = $sb.AppendLine((Read-Text (Join-Path $last.FullName 'db-by-app.txt'))) +$null = $sb.AppendLine('--- db-error ---') +$null = $sb.AppendLine((Read-Text (Join-Path $last.FullName 'db-error.txt'))) + +$null = $sb.AppendLine('=== FIRST SNAPSHOT SUMMARY ===') +$null = $sb.AppendLine("First snapshot: $($first.FullName)") +$null = $sb.AppendLine((Read-Text (Join-Path $first.FullName 'powershell-process.txt'))) +$null = $sb.AppendLine("db total sessions: $(Read-First (Join-Path $first.FullName 'db-total-sessions.txt'))") +$null = $sb.AppendLine('=== LATEST SNAPSHOT SUMMARY ===') +$null = $sb.AppendLine("Latest snapshot: $($last.FullName)") +$null = $sb.AppendLine((Read-Text (Join-Path $last.FullName 'powershell-process.txt'))) +$null = $sb.AppendLine("db total sessions: $(Read-First (Join-Path $last.FullName 'db-total-sessions.txt'))") + +$null = $sb.AppendLine('=== RECENT TREND (LAST 20 SNAPSHOTS) ===') +foreach ($d in ($snaps | Select-Object -Last 20)) { + $dbTotal = Read-First (Join-Path $d.FullName 'db-total-sessions.txt') + $dbMax = Read-First (Join-Path $d.FullName 'db-max-connections.txt') + $dbReserved = Read-First (Join-Path $d.FullName 'db-superuser-reserved-connections.txt') + $dbActive = Extract-DbCount (Join-Path $d.FullName 'db-by-state.txt') 'active' + $dbIdle = Extract-DbCount (Join-Path $d.FullName 'db-by-state.txt') 'idle' + $proc = (Read-Text (Join-Path $d.FullName 'powershell-process.txt')) -replace '\r?\n',' ' + $null = $sb.AppendLine("$($d.Name) $proc db_total=$dbTotal db_active=$dbActive db_idle=$dbIdle db_slots=$(Calc-Slots $dbMax $dbReserved)") +} +$null = $sb.AppendLine() + +$dbCsv = Join-Path $MonitorDir 'db-connection-trend.csv' +if (Test-Path $dbCsv) { + $null = $sb.AppendLine('=== DB CONNECTION TREND CSV (LAST 40 LINES) ===') + $null = $sb.AppendLine(((Get-Content $dbCsv | Select-Object -Last 40) -join [Environment]::NewLine)) + $null = $sb.AppendLine() +} + +$null = $sb.AppendLine('=== LOG TAILS ===') +$null = $sb.AppendLine('--- launch.stdout.log (last 50 lines) ---') +$null = $sb.AppendLine(((Get-Content (Join-Path $MonitorDir 'launch.stdout.log') -Tail 50) -join [Environment]::NewLine)) +$null = $sb.AppendLine() +$null = $sb.AppendLine('--- launch.stderr.log (last 50 lines) ---') +$null = $sb.AppendLine(((Get-Content (Join-Path $MonitorDir 'launch.stderr.log') -Tail 50) -join [Environment]::NewLine)) +$null = $sb.AppendLine() +$null = $sb.AppendLine('--- postgres docker logs (last captured 100 lines) ---') +$null = $sb.AppendLine(((Get-Content (Join-Path $last.FullName 'docker-logs-tail.txt') -Tail 100) -join [Environment]::NewLine)) +$null = $sb.AppendLine() + +$null = $sb.AppendLine('=== QUICK READ ===') +$null = $sb.AppendLine("First DB total sessions: $(Read-First (Join-Path $first.FullName 'db-total-sessions.txt'))") +$null = $sb.AppendLine("Latest DB total sessions: $(Read-First (Join-Path $last.FullName 'db-total-sessions.txt'))") +$null = $sb.AppendLine("Latest DB usable client slots: $(Calc-Slots $lastMax $lastReserved)") +$null = $sb.AppendLine('Interpretation guide:') +$null = $sb.AppendLine('- Healthy memory: process memory and JVM native memory plateau.') +$null = $sb.AppendLine('- Healthy DB: total sessions rise at startup and then plateau well below usable client slots.') +$null = $sb.AppendLine('- Suspicious DB: total sessions keep climbing, idle sessions pile up, or db-error shows too many clients already.') + +[System.IO.File]::WriteAllText($OutFile, $sb.ToString()) +Write-Host "Wrote report to: $OutFile" diff --git a/dist/release/check-oscar-status.sh b/dist/release/check-oscar-status.sh new file mode 100644 index 0000000..885bca8 --- /dev/null +++ b/dist/release/check-oscar-status.sh @@ -0,0 +1,225 @@ +#!/bin/bash +set -euo pipefail + +BASE_DIR="${1:-.}" +LATEST_DIR="${2:-}" +OUT_FILE="${3:-}" + +cd "$BASE_DIR" + +if [ -z "$LATEST_DIR" ]; then + LATEST_DIR="$(ls -td oscar-monitor-* 2>/dev/null | head -n 1 || true)" +fi + +if [ -z "$LATEST_DIR" ] || [ ! -d "$LATEST_DIR" ]; then + echo "Error: no oscar-monitor-* directory found." + exit 1 +fi + +if [ -z "$OUT_FILE" ]; then + OUT_FILE="oscar-status-$(date +%Y%m%d-%H%M%S).txt" +fi + +FIRST_SNAP="$(find "$LATEST_DIR" -maxdepth 1 -mindepth 1 -type d | sort | head -n 1 || true)" +LAST_SNAP="$(find "$LATEST_DIR" -maxdepth 1 -mindepth 1 -type d | sort | tail -n 1 || true)" +PID="" +[ -f "$LATEST_DIR/jvm-pid.txt" ] && PID="$(cat "$LATEST_DIR/jvm-pid.txt" 2>/dev/null || true)" +LIVE_PID="$(pgrep -f 'com.botts.impl.security.SensorHubWrapper' | head -n 1 || true)" + +extract_db_metric() { + local file="$1" default="$2" + if [ -f "$file" ]; then + tr -d '[:space:]' < "$file" | tail -n 1 + else + echo "$default" + fi +} + +calc_slots() { + local max="$1" reserved="$2" + if [[ "$max" =~ ^[0-9]+$ ]] && [[ "$reserved" =~ ^[0-9]+$ ]]; then + echo $((max - reserved)) + fi +} + +{ + echo "OSCAR STATUS REPORT" + echo "Generated: $(date -Is)" + echo "Base directory: $(pwd)" + echo "Monitor directory: $LATEST_DIR" + echo "Output file: $OUT_FILE" + echo + + echo "=== PROCESS STATUS ===" + echo "PID from monitor: ${PID:-}" + echo "Live OSCAR PID: ${LIVE_PID:-}" + echo + pgrep -af monitor-oscar.sh || true + pgrep -af 'com.botts.impl.security.SensorHubWrapper' || true + echo + docker ps --filter name=oscar-postgis-container || true + echo + + echo "=== SYSTEM MEMORY ===" + free -h || true + echo + echo "--- vmstat (5 samples) ---" + vmstat 1 5 || true + echo + + if [ -n "${LIVE_PID:-}" ] && [ -r "/proc/$LIVE_PID/status" ]; then + echo "=== LIVE JVM /proc STATUS ===" + grep -E 'Name|State|VmSize|VmRSS|VmSwap|Threads' "/proc/$LIVE_PID/status" || true + echo + fi + + if [ -n "${LIVE_PID:-}" ] && [ -r "/proc/$LIVE_PID/smaps_rollup" ]; then + echo "=== LIVE JVM SMAPS ROLLUP ===" + cat "/proc/$LIVE_PID/smaps_rollup" || true + echo + fi + + if [ -n "${LIVE_PID:-}" ] && command -v jcmd >/dev/null 2>&1; then + echo "=== LIVE JVM JFR STATUS ===" + jcmd "$LIVE_PID" JFR.check || true + echo + echo "=== LIVE JVM GC HEAP INFO ===" + jcmd "$LIVE_PID" GC.heap_info || true + echo + echo "=== LIVE JVM NATIVE MEMORY SUMMARY ===" + jcmd "$LIVE_PID" VM.native_memory summary || true + echo + fi + + echo "=== LIVE POSTGRES STATUS ===" + if docker ps --format '{{.Names}}' | grep -Eq '^oscar-postgis-container$'; then + if [ -f "$LAST_SNAP/db-max-connections.txt" ]; then + echo "max_connections: $(extract_db_metric "$LAST_SNAP/db-max-connections.txt" n/a)" + fi + if [ -f "$LAST_SNAP/db-superuser-reserved-connections.txt" ]; then + echo "superuser_reserved_connections: $(extract_db_metric "$LAST_SNAP/db-superuser-reserved-connections.txt" n/a)" + fi + if [ -f "$LAST_SNAP/db-total-sessions.txt" ]; then + echo "total_sessions: $(extract_db_metric "$LAST_SNAP/db-total-sessions.txt" n/a)" + fi + if [ -f "$LAST_SNAP/db-by-state.txt" ]; then + echo + echo "--- db-by-state ---" + cat "$LAST_SNAP/db-by-state.txt" || true + fi + if [ -f "$LAST_SNAP/db-by-app.txt" ]; then + echo + echo "--- db-by-app ---" + cat "$LAST_SNAP/db-by-app.txt" || true + fi + if [ -f "$LAST_SNAP/db-activity-detail.txt" ]; then + echo + echo "--- db-activity-detail (first 40 lines) ---" + head -n 40 "$LAST_SNAP/db-activity-detail.txt" || true + fi + if [ -f "$LAST_SNAP/db-error.txt" ]; then + echo + echo "--- db-error ---" + cat "$LAST_SNAP/db-error.txt" || true + fi + else + echo "Postgres container is not running." + fi + echo + + echo "=== FIRST SNAPSHOT SUMMARY ===" + echo "First snapshot: ${FIRST_SNAP:-}" + if [ -n "${FIRST_SNAP:-}" ] && [ -f "$FIRST_SNAP/proc-status.txt" ]; then + grep -E 'VmRSS|VmSwap|Threads' "$FIRST_SNAP/proc-status.txt" || true + fi + if [ -n "${FIRST_SNAP:-}" ] && [ -f "$FIRST_SNAP/nmt-summary.txt" ]; then + grep '^Total:' "$FIRST_SNAP/nmt-summary.txt" || true + fi + if [ -n "${FIRST_SNAP:-}" ] && [ -f "$FIRST_SNAP/db-total-sessions.txt" ]; then + echo "db total sessions: $(extract_db_metric "$FIRST_SNAP/db-total-sessions.txt" n/a)" + fi + echo + + echo "=== LATEST SNAPSHOT SUMMARY ===" + echo "Latest snapshot: ${LAST_SNAP:-}" + if [ -n "${LAST_SNAP:-}" ] && [ -f "$LAST_SNAP/proc-status.txt" ]; then + grep -E 'VmRSS|VmSwap|Threads' "$LAST_SNAP/proc-status.txt" || true + fi + if [ -n "${LAST_SNAP:-}" ] && [ -f "$LAST_SNAP/nmt-summary.txt" ]; then + grep '^Total:' "$LAST_SNAP/nmt-summary.txt" || true + fi + if [ -n "${LAST_SNAP:-}" ] && [ -f "$LAST_SNAP/db-total-sessions.txt" ]; then + echo "db total sessions: $(extract_db_metric "$LAST_SNAP/db-total-sessions.txt" n/a)" + fi + echo + + echo "=== RECENT TREND (LAST 20 SNAPSHOTS) ===" + for d in $(find "$LATEST_DIR" -maxdepth 1 -mindepth 1 -type d | sort | tail -n 20); do + printf "%s " "$(basename "$d")" + [ -f "$d/proc-status.txt" ] && grep -E 'VmRSS|VmSwap|Threads' "$d/proc-status.txt" | tr '\n' ' ' + [ -f "$d/nmt-summary.txt" ] && grep '^Total:' "$d/nmt-summary.txt" | tr '\n' ' ' + if [ -f "$d/db-total-sessions.txt" ]; then + printf "db_total=%s " "$(extract_db_metric "$d/db-total-sessions.txt" n/a)" + fi + if [ -f "$d/db-by-state.txt" ]; then + printf "db_active=%s " "$(awk -F'|' '$1=="active" {gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}' "$d/db-by-state.txt" | tail -n 1)" + printf "db_idle=%s " "$(awk -F'|' '$1=="idle" {gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}' "$d/db-by-state.txt" | tail -n 1)" + printf "db_idle_tx=%s " "$(awk -F'|' '$1=="idle in transaction" {gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}' "$d/db-by-state.txt" | tail -n 1)" + fi + if [ -f "$d/db-error.txt" ] && [ -s "$d/db-error.txt" ]; then + printf "db_error=yes " + fi + echo + done + echo + + if [ -f "$LATEST_DIR/db-connection-trend.csv" ]; then + echo "=== DB CONNECTION TREND CSV (LAST 40 LINES) ===" + tail -n 40 "$LATEST_DIR/db-connection-trend.csv" || true + echo + fi + + echo "=== LOG TAILS ===" + [ -f "$LATEST_DIR/launch.stdout.log" ] && { echo '--- launch.stdout.log (last 50 lines) ---'; tail -n 50 "$LATEST_DIR/launch.stdout.log"; echo; } + [ -f "$LATEST_DIR/launch.stderr.log" ] && { echo '--- launch.stderr.log (last 50 lines) ---'; tail -n 50 "$LATEST_DIR/launch.stderr.log"; echo; } + [ -f "$LAST_SNAP/docker-logs-tail.txt" ] && { echo '--- postgres docker logs (last captured 100 lines) ---'; tail -n 100 "$LAST_SNAP/docker-logs-tail.txt"; echo; } + + echo "=== QUICK READ ===" + FIRST_RSS=""; LAST_RSS=""; FIRST_SWAP=""; LAST_SWAP=""; FIRST_THREADS=""; LAST_THREADS="" + FIRST_DB_TOTAL=""; LAST_DB_TOTAL=""; FIRST_MAX=""; LAST_MAX=""; FIRST_RESERVED=""; LAST_RESERVED="" + + if [ -n "${FIRST_SNAP:-}" ] && [ -f "$FIRST_SNAP/proc-status.txt" ]; then + FIRST_RSS="$(grep '^VmRSS:' "$FIRST_SNAP/proc-status.txt" | awk '{print $2 " " $3}' || true)" + FIRST_SWAP="$(grep '^VmSwap:' "$FIRST_SNAP/proc-status.txt" | awk '{print $2 " " $3}' || true)" + FIRST_THREADS="$(grep '^Threads:' "$FIRST_SNAP/proc-status.txt" | awk '{print $2}' || true)" + fi + if [ -n "${LAST_SNAP:-}" ] && [ -f "$LAST_SNAP/proc-status.txt" ]; then + LAST_RSS="$(grep '^VmRSS:' "$LAST_SNAP/proc-status.txt" | awk '{print $2 " " $3}' || true)" + LAST_SWAP="$(grep '^VmSwap:' "$LAST_SNAP/proc-status.txt" | awk '{print $2 " " $3}' || true)" + LAST_THREADS="$(grep '^Threads:' "$LAST_SNAP/proc-status.txt" | awk '{print $2}' || true)" + fi + [ -n "${FIRST_SNAP:-}" ] && FIRST_DB_TOTAL="$(extract_db_metric "$FIRST_SNAP/db-total-sessions.txt" n/a)" + [ -n "${LAST_SNAP:-}" ] && LAST_DB_TOTAL="$(extract_db_metric "$LAST_SNAP/db-total-sessions.txt" n/a)" + [ -n "${FIRST_SNAP:-}" ] && FIRST_MAX="$(extract_db_metric "$FIRST_SNAP/db-max-connections.txt" n/a)" + [ -n "${LAST_SNAP:-}" ] && LAST_MAX="$(extract_db_metric "$LAST_SNAP/db-max-connections.txt" n/a)" + [ -n "${FIRST_SNAP:-}" ] && FIRST_RESERVED="$(extract_db_metric "$FIRST_SNAP/db-superuser-reserved-connections.txt" n/a)" + [ -n "${LAST_SNAP:-}" ] && LAST_RESERVED="$(extract_db_metric "$LAST_SNAP/db-superuser-reserved-connections.txt" n/a)" + + echo "First RSS: ${FIRST_RSS:-n/a}" + echo "Latest RSS: ${LAST_RSS:-n/a}" + echo "First VmSwap: ${FIRST_SWAP:-n/a}" + echo "Latest VmSwap: ${LAST_SWAP:-n/a}" + echo "First Threads: ${FIRST_THREADS:-n/a}" + echo "Latest Threads:${LAST_THREADS:-n/a}" + echo "First DB total sessions: ${FIRST_DB_TOTAL:-n/a}" + echo "Latest DB total sessions: ${LAST_DB_TOTAL:-n/a}" + echo "First DB usable client slots: $(calc_slots "$FIRST_MAX" "$FIRST_RESERVED")" + echo "Latest DB usable client slots: $(calc_slots "$LAST_MAX" "$LAST_RESERVED")" + echo + echo "Interpretation guide:" + echo "- Healthy memory: RSS, VmSwap, and thread count rise at startup and then flatten." + echo "- Healthy DB: total sessions rise at startup and then plateau well below usable client slots." + echo "- Suspicious DB: total sessions keep climbing, idle sessions pile up, or db_error shows too many clients already." +} > "$OUT_FILE" + +echo "Wrote report to: $OUT_FILE" diff --git a/dist/release/env.template b/dist/release/env.template new file mode 100644 index 0000000..f1c4058 --- /dev/null +++ b/dist/release/env.template @@ -0,0 +1,55 @@ +# --- SYSTEM PROFILE --- +# Options: RPI4, 8GB, 16GB, 32GB +SYSTEM_PROFILE=16GB + +# --- DATABASE SETTINGS --- +DB_NAME=gis +DB_USER=postgres +DB_PASSWORD=postgres +DB_PORT=5432 +DB_HOST=localhost +CONTAINER_NAME=oscar-postgis-container + +# --- SECURITY --- +# Replace before production use +KEYSTORE_PASSWORD=atakatak +TRUSTSTORE_PASSWORD=changeit + +# Optional: +# If set, launch scripts can use this instead of profile defaults/helper files. +# INITIAL_ADMIN_PASSWORD=admin + +# --- PROCESS HANDLING --- +# 0 = refuse to start if OSCAR is already running +# 1 = stop the running OSCAR instance and start fresh +FORCE_RESTART=0 + +# --- MONITOR BEHAVIOR --- +# 0 = refuse to attach if OSCAR is already running +# 1 = attach monitoring to an already running OSCAR instance +ATTACH_TO_EXISTING=0 + +# Maximum time to wait for OSCAR JVM startup in monitor scripts +MAX_WAIT_SECONDS=300 + +# --- POSTGIS STARTUP / READINESS --- +# Number of readiness retries for PostGIS startup checks +RETRY_MAX=120 + +# Seconds between readiness retries +RETRY_INTERVAL=2 + +# Extra delay after PostGIS reports ready +POSTGIS_READY_DELAY=5 + +# --- OPTIONAL MEMORY / DIAGNOSTICS OVERRIDES --- +# Leave blank to use profile defaults from launch scripts +JAVACPP_MAX_BYTES= +JAVACPP_MAX_PHYSICAL_BYTES= +JFR_FILENAME= + +# --- OPTIONAL ARM BUILD OVERRIDES --- +# Only needed when using ARM-specific launch/build paths +# POSTGIS_IMAGE_NAME=oscar-postgis-arm +# POSTGIS_DOCKERFILE=Dockerfile-arm64 +# POSTGIS_PLATFORM=linux/arm64 \ No newline at end of file diff --git a/dist/release/launch-all-arm.sh b/dist/release/launch-all-arm.sh old mode 100755 new mode 100644 index 96c1cc5..488215b --- a/dist/release/launch-all-arm.sh +++ b/dist/release/launch-all-arm.sh @@ -1,76 +1,308 @@ -#!/bin/bash - -HOST=localhost -DB_NAME=gis -DB_USER=postgres -RETRY_MAX=20 -RETRY_INTERVAL=5 -PROJECT_DIR="$(pwd)" # Store the original directory -CONTAINER_NAME=oscar-postgis-container - -#sudo docker rm -f "$CONTAINER_NAME" 2>/dev/null || true - -# Create pgdata directory if needed -if [ ! -d "${PROJECT_DIR}/pgdata" ]; then - echo "Creating pgdata folder..." - mkdir -p "${PROJECT_DIR}/pgdata" -fi +#!/usr/bin/env bash +set -euo pipefail + +SOURCE="${BASH_SOURCE[0]}" +while [ -h "$SOURCE" ]; do + SOURCE_DIR="$(CDPATH= cd -P "$(dirname "$SOURCE")" >/dev/null 2>&1 && pwd)" + SOURCE="$(readlink "$SOURCE")" + case "$SOURCE" in + /*) ;; + *) SOURCE="${SOURCE_DIR}/${SOURCE}" ;; + esac +done +PROJECT_DIR="$(CDPATH= cd -P "$(dirname "$SOURCE")" >/dev/null 2>&1 && pwd)" +ENV_FILE="$PROJECT_DIR/.env" +MATCH_EXPR='com.botts.impl.security.SensorHubWrapper' +FORCE_RESTART="${FORCE_RESTART:-0}" +RETRY_MAX="${RETRY_MAX:-120}" +RETRY_INTERVAL="${RETRY_INTERVAL:-2}" +POSTGIS_READY_DELAY="${POSTGIS_READY_DELAY:-5}" +IMAGE_NAME="${POSTGIS_IMAGE_NAME:-${IMAGE_NAME:-oscar-postgis-arm}}" +POSTGIS_DOCKERFILE="${POSTGIS_DOCKERFILE:-Dockerfile-arm64}" +POSTGIS_PLATFORM="${POSTGIS_PLATFORM:-linux/arm64}" + +load_env() { + local env_file="$1" + while IFS= read -r line || [ -n "$line" ]; do + case "$line" in + ""|"#"*) continue ;; + export\ *) line="${line#export }" ;; + esac + local name="${line%%=*}" + local value="${line#*=}" + value="${value%$'\r'}" + export "${name}=${value}" + done < "$env_file" +} + +require_cmd() { + local cmd="$1" + if ! command -v "$cmd" >/dev/null 2>&1; then + echo "Error: required command not found: $cmd" + exit 1 + fi +} + +get_java_major() { + java -version 2>&1 | awk -F'"' '/version/ { split($2, v, "."); print v[1]; exit }' +} + +check_dependencies() { + require_cmd bash + require_cmd java + require_cmd keytool + require_cmd docker + + if ! docker info >/dev/null 2>&1; then + echo "Error: Docker is installed, but the Docker daemon is not running." + exit 1 + fi + + local java_major + java_major="$(get_java_major || true)" + if [[ -z "$java_major" || ! "$java_major" =~ ^[0-9]+$ ]]; then + echo "Error: could not determine Java version. Java 21 or newer is required." + exit 1 + fi + if [ "$java_major" -lt 21 ]; then + echo "Error: Java 21 or newer is required. Found Java $java_major." + exit 1 + fi +} + +find_existing_oscar_pids() { + pgrep -f "$MATCH_EXPR" || true +} + +stop_existing_oscar() { + local pids="$1" + if [ -z "$pids" ]; then + return 0 + fi + + echo "Stopping existing OSCAR instance(s): $pids" + kill $pids 2>/dev/null || true + + local waited=0 + while [ "$waited" -lt 15 ]; do + sleep 1 + waited=$((waited + 1)) + if [ -z "$(find_existing_oscar_pids)" ]; then + return 0 + fi + done + + echo "Existing OSCAR instance still running after graceful stop. Forcing stop." + kill -9 $pids 2>/dev/null || true + sleep 1 + + if [ -n "$(find_existing_oscar_pids)" ]; then + echo "Error: unable to stop the existing OSCAR instance." + exit 1 + fi +} + +check_existing_oscar() { + local pids + pids="$(find_existing_oscar_pids)" + + if [ -z "$pids" ]; then + return 0 + fi -# Check Docker -if ! command -v docker >/dev/null 2>&1; then - echo "Error: Docker is not installed. Please install Docker first." + if [ "$FORCE_RESTART" = "1" ]; then + echo "Existing OSCAR instance found with PID(s): $pids. Replacing because FORCE_RESTART=1." + stop_existing_oscar "$pids" + return 0 + fi + + echo "OSCAR is already running with PID(s): $pids." + echo "Stop the running instance first, or set FORCE_RESTART=1 to replace it." + exit 1 +} + +require_env() { + local name="$1" + local value="${!name:-}" + if [ -z "$value" ]; then + echo "Error: ${name} is not set in .env." + exit 1 + fi +} + +require_number() { + local name="$1" + local value="${!name:-}" + case "$value" in + ''|*[!0-9]*) + echo "Error: ${name} must be a number, got '${value}'." + exit 1 + ;; + esac +} + +ensure_project_layout() { + if [ ! -d "$PROJECT_DIR/postgis" ]; then + echo "Error: postgis directory not found in $PROJECT_DIR" + exit 1 + fi + + if [ ! -f "$PROJECT_DIR/postgis/$POSTGIS_DOCKERFILE" ]; then + echo "Error: $POSTGIS_DOCKERFILE not found in $PROJECT_DIR/postgis" + exit 1 + fi + + if [ ! -d "$PROJECT_DIR/osh-node-oscar" ]; then + echo "Error: osh-node-oscar directory not found in $PROJECT_DIR" + exit 1 + fi + + if [ ! -f "$PROJECT_DIR/osh-node-oscar/launch.sh" ]; then + echo "Error: launch.sh not found in $PROJECT_DIR/osh-node-oscar" + exit 1 + fi + + mkdir -p "$PROJECT_DIR/pgdata" +} + +if [ ! -f "$ENV_FILE" ]; then + echo "Error: .env file not found in $PROJECT_DIR" + echo "Create it by copying env.template to .env and editing the values." exit 1 fi -echo "Building PostGIS Docker image..." +load_env "$ENV_FILE" +check_dependencies +check_existing_oscar +ensure_project_layout + +SYSTEM_PROFILE="${SYSTEM_PROFILE:-8GB}" +CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" +DB_HOST="${DB_HOST:-localhost}" +export SYSTEM_PROFILE CONTAINER_NAME DB_HOST RETRY_MAX RETRY_INTERVAL POSTGIS_READY_DELAY IMAGE_NAME POSTGIS_DOCKERFILE POSTGIS_PLATFORM + +require_env DB_NAME +require_env DB_USER +require_env DB_PASSWORD +require_env DB_PORT -cd postgis || { echo "Error: postgis directory not found"; exit 1; } +require_number DB_PORT +require_number RETRY_MAX +require_number RETRY_INTERVAL +require_number POSTGIS_READY_DELAY -# Build PostGIS -sudo docker build . \ - --file=Dockerfile-arm64 \ - --tag=oscar-postgis-arm +case "${SYSTEM_PROFILE^^}" in + RPI4) + SYSTEM_PROFILE="RPI4" + PG_SHARED="256MB" + PG_CACHE="1GB" + PG_WORK_MEM="2MB" + PG_MAINT="64MB" + PG_MAX_CONN="75" + ;; + 8GB) + SYSTEM_PROFILE="8GB" + PG_SHARED="512MB" + PG_CACHE="2GB" + PG_WORK_MEM="4MB" + PG_MAINT="128MB" + PG_MAX_CONN="125" + ;; + 16GB) + SYSTEM_PROFILE="16GB" + PG_SHARED="1GB" + PG_CACHE="4GB" + PG_WORK_MEM="8MB" + PG_MAINT="256MB" + PG_MAX_CONN="200" + ;; + 32GB) + SYSTEM_PROFILE="32GB" + PG_SHARED="2GB" + PG_CACHE="8GB" + PG_WORK_MEM="16MB" + PG_MAINT="512MB" + PG_MAX_CONN="300" + ;; + *) + echo "Unknown profile '${SYSTEM_PROFILE}', using 8GB defaults." + SYSTEM_PROFILE="8GB" + PG_SHARED="512MB" + PG_CACHE="2GB" + PG_WORK_MEM="4MB" + PG_MAINT="128MB" + PG_MAX_CONN="125" + ;; + esac -echo "Starting PostGIS container..." -echo "PROJECT_DIR is set to: ${PROJECT_DIR}" +export SYSTEM_PROFILE CONTAINER_NAME DB_NAME DB_USER DB_PASSWORD DB_PORT -if docker ps -a --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then - # The container exists - if docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then - echo "Container already running: ${CONTAINER_NAME}" +echo "Building PostGIS Docker image for Apple Silicon / ARM64..." +( + cd "$PROJECT_DIR/postgis" + if [ -n "$POSTGIS_PLATFORM" ]; then + docker build --platform "$POSTGIS_PLATFORM" . --file="$POSTGIS_DOCKERFILE" --tag="$IMAGE_NAME" else - echo "Starting existing container: ${CONTAINER_NAME}" - docker start "${CONTAINER_NAME}" + docker build . --file="$POSTGIS_DOCKERFILE" --tag="$IMAGE_NAME" fi -else - echo "Creating new container: ${CONTAINER_NAME}" - docker run \ - --name $CONTAINER_NAME \ - -e POSTGRES_DB=$DB_NAME \ - -e POSTGRES_USER=$DB_USER \ - -e POSTGRES_PASSWORD=postgres \ - -e DATADIR=/var/lib/postgresql/data \ - -p 5432:5432 \ - -v "$(pwd)/pgdata:/var/lib/postgresql/data" \ - -d \ - oscar-postgis-arm || { echo "Failed to start PostGIS container"; exit 1; } -fi +) + +echo "Preparing PostGIS container for profile: $SYSTEM_PROFILE" +echo " Image: $IMAGE_NAME" +echo " Dockerfile: $POSTGIS_DOCKERFILE" +echo " Port: ${DB_PORT}:5432" +echo " Data: $PROJECT_DIR/pgdata" -# Wait for PostgreSQL/PostGIS to become ready -echo "Waiting for PostGIS ARM64 (PostgreSQL) to be ready..." +if docker container inspect "$CONTAINER_NAME" >/dev/null 2>&1; then + echo "Removing existing container '$CONTAINER_NAME' so updated settings take effect..." + docker rm -f "$CONTAINER_NAME" >/dev/null +fi -RETRY_COUNT=0 -export PGPASSWORD=postgres # Needed for pg_isready with password +echo "Creating new PostGIS container..." +docker run \ + --name "$CONTAINER_NAME" \ + -e POSTGRES_DB="$DB_NAME" \ + -e POSTGRES_USER="$DB_USER" \ + -e POSTGRES_PASSWORD="$DB_PASSWORD" \ + -e DATADIR=/var/lib/postgresql/data \ + -p "${DB_PORT}:5432" \ + -v "$PROJECT_DIR/pgdata:/var/lib/postgresql/data" \ + -d \ + "$IMAGE_NAME" \ + -c shared_buffers="$PG_SHARED" \ + -c effective_cache_size="$PG_CACHE" \ + -c work_mem="$PG_WORK_MEM" \ + -c maintenance_work_mem="$PG_MAINT" \ + -c max_connections="$PG_MAX_CONN" \ + -c superuser_reserved_connections=10 \ + -c idle_session_timeout=600000 \ + -c log_connections=on \ + -c log_disconnections=on \ + -c wal_buffers=16MB \ + -c random_page_cost=1.1 \ + -c effective_io_concurrency=200 -until docker exec "$CONTAINER_NAME" pg_isready -U "$DB_USER" -d "$DB_NAME" > /dev/null 2>&1; do - echo "PostGIS not ready yet, retrying..." - sleep "${RETRY_INTERVAL}" +echo "Waiting for PostGIS ARM64 to be ready..." +export PGPASSWORD="$DB_PASSWORD" +retry_count=0 +until docker exec "$CONTAINER_NAME" pg_isready -U "$DB_USER" -d "$DB_NAME" >/dev/null 2>&1; do + retry_count=$((retry_count + 1)) + if [ "$retry_count" -ge "$RETRY_MAX" ]; then + echo "Error: PostGIS did not become ready after $((RETRY_MAX * RETRY_INTERVAL)) seconds." + echo "Last container logs:" + docker logs --tail 50 "$CONTAINER_NAME" || true + exit 1 + fi + echo "PostGIS not ready yet, retrying..." + sleep "$RETRY_INTERVAL" done -echo "PostGIS (PostgreSQL) is ready! Please wait for OpenSensorHub to start..." +echo "PostGIS is ready. Starting OpenSensorHub..." +sleep "$POSTGIS_READY_DELAY" -sleep 10 +cd "$PROJECT_DIR/osh-node-oscar" +if [ ! -x ./launch.sh ]; then + chmod +x ./launch.sh +fi -# Launch osh-node-oscar -cd "$PROJECT_DIR/osh-node-oscar" || { echo "Error: osh-node-oscar not found"; exit 1; } -./launch.sh \ No newline at end of file +exec ./launch.sh diff --git a/dist/release/launch-all-arm_old.sh b/dist/release/launch-all-arm_old.sh new file mode 100644 index 0000000..358cdd0 --- /dev/null +++ b/dist/release/launch-all-arm_old.sh @@ -0,0 +1,247 @@ +#!/bin/bash +set -euo pipefail + +# Apple Silicon / ARM64 launcher for the full OSH + PostGIS stack. +# Resolves paths from this script's location, not from the caller's cwd. + +SOURCE="${BASH_SOURCE[0]}" +while [ -h "$SOURCE" ]; do + SOURCE_DIR="$(CDPATH= cd -P "$(dirname "$SOURCE")" >/dev/null 2>&1 && pwd)" + SOURCE="$(readlink "$SOURCE")" + case "$SOURCE" in + /*) ;; + *) SOURCE="${SOURCE_DIR}/${SOURCE}" ;; + esac +done +PROJECT_DIR="$(CDPATH= cd -P "$(dirname "$SOURCE")" >/dev/null 2>&1 && pwd)" +ENV_FILE="${PROJECT_DIR}/.env" + +IMAGE_NAME="${POSTGIS_IMAGE_NAME:-${IMAGE_NAME:-oscar-postgis-arm}}" +POSTGIS_DOCKERFILE="${POSTGIS_DOCKERFILE:-Dockerfile-arm64}" +POSTGIS_PLATFORM="${POSTGIS_PLATFORM:-linux/arm64}" + +if [ ! -f "$ENV_FILE" ]; then + echo "Error: .env file not found in ${PROJECT_DIR}." + echo "Create it by copying env.template to .env and editing the values." + exit 1 +fi + +# Export values from .env so osh-node-oscar/launch.sh can use the same settings. +set -a +# shellcheck disable=SC1090 +. "$ENV_FILE" +set +a + +# Remove a possible CR from values if .env was edited on Windows. +strip_cr_var() { + _name="$1" + eval "_value=\${${_name}-}" + _value="${_value%$'\r'}" + export "${_name}=${_value}" +} + +for _var in \ + SYSTEM_PROFILE DB_NAME DB_USER DB_PASSWORD DB_PORT DB_HOST CONTAINER_NAME \ + KEYSTORE_PASSWORD TRUSTSTORE_PASSWORD JAVACPP_MAX_BYTES \ + JAVACPP_MAX_PHYSICAL_BYTES JFR_FILENAME RETRY_MAX RETRY_INTERVAL \ + POSTGIS_READY_DELAY IMAGE_NAME POSTGIS_IMAGE_NAME POSTGIS_DOCKERFILE POSTGIS_PLATFORM + do + strip_cr_var "$_var" +done +unset _var + +SYSTEM_PROFILE="${SYSTEM_PROFILE:-8GB}" +CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" +DB_HOST="${DB_HOST:-localhost}" +RETRY_MAX="${RETRY_MAX:-120}" +RETRY_INTERVAL="${RETRY_INTERVAL:-2}" +POSTGIS_READY_DELAY="${POSTGIS_READY_DELAY:-5}" +IMAGE_NAME="${POSTGIS_IMAGE_NAME:-${IMAGE_NAME:-oscar-postgis-arm}}" +POSTGIS_DOCKERFILE="${POSTGIS_DOCKERFILE:-Dockerfile-arm64}" +POSTGIS_PLATFORM="${POSTGIS_PLATFORM:-linux/arm64}" +export SYSTEM_PROFILE CONTAINER_NAME DB_HOST RETRY_MAX RETRY_INTERVAL POSTGIS_READY_DELAY +export IMAGE_NAME POSTGIS_DOCKERFILE POSTGIS_PLATFORM + +require_env() { + _name="$1" + eval "_value=\${${_name}:-}" + if [ -z "$_value" ]; then + echo "Error: ${_name} is not set in .env." + exit 1 + fi +} + +require_env DB_NAME +require_env DB_USER +require_env DB_PASSWORD +require_env DB_PORT + +require_number() { + _name="$1" + eval "_value=\${${_name}:-}" + case "$_value" in + *[!0-9]*|'') + echo "Error: ${_name} must be a number, got '${_value}'." + exit 1 + ;; + esac +} + +require_number DB_PORT +require_number RETRY_MAX +require_number RETRY_INTERVAL +require_number POSTGIS_READY_DELAY + +PROFILE_UPPER="$(printf '%s' "$SYSTEM_PROFILE" | tr '[:lower:]' '[:upper:]')" +case "$PROFILE_UPPER" in + "RPI4") + SYSTEM_PROFILE="RPI4" + PG_SHARED="256MB" + PG_CACHE="1GB" + PG_WORK_MEM="2MB" + PG_MAINT="64MB" + PG_MAX_CONN="75" + ;; + "8GB") + SYSTEM_PROFILE="8GB" + PG_SHARED="512MB" + PG_CACHE="2GB" + PG_WORK_MEM="4MB" + PG_MAINT="128MB" + PG_MAX_CONN="125" + ;; + "16GB") + SYSTEM_PROFILE="16GB" + PG_SHARED="1GB" + PG_CACHE="4GB" + PG_WORK_MEM="8MB" + PG_MAINT="256MB" + PG_MAX_CONN="200" + ;; + "32GB") + SYSTEM_PROFILE="32GB" + PG_SHARED="2GB" + PG_CACHE="8GB" + PG_WORK_MEM="16MB" + PG_MAINT="512MB" + PG_MAX_CONN="300" + ;; + *) + echo "Unknown profile '${SYSTEM_PROFILE}', using 8GB defaults." + SYSTEM_PROFILE="8GB" + PG_SHARED="512MB" + PG_CACHE="2GB" + PG_WORK_MEM="4MB" + PG_MAINT="128MB" + PG_MAX_CONN="125" + ;; +esac + +# Keep sanitized/defaulted values available to the child launch.sh. +export SYSTEM_PROFILE CONTAINER_NAME DB_NAME DB_USER DB_PASSWORD DB_PORT + +mkdir -p "${PROJECT_DIR}/pgdata" + +if ! command -v docker >/dev/null 2>&1; then + echo "Error: Docker is not installed or is not in PATH." + exit 1 +fi + +if ! docker info >/dev/null 2>&1; then + echo "Error: Docker is installed, but the Docker daemon is not running." + echo "Start Docker Desktop, then run this script again." + exit 1 +fi + +POSTGIS_DIR="${PROJECT_DIR}/postgis" +if [ ! -d "$POSTGIS_DIR" ]; then + echo "Error: postgis directory not found in ${PROJECT_DIR}." + exit 1 +fi + +if [ ! -f "${POSTGIS_DIR}/${POSTGIS_DOCKERFILE}" ]; then + echo "Error: ${POSTGIS_DOCKERFILE} not found in ${POSTGIS_DIR}." + exit 1 +fi + +echo "Building PostGIS Docker image for Apple Silicon / ARM64..." +cd "$POSTGIS_DIR" +if [ -n "${POSTGIS_PLATFORM:-}" ]; then + docker build --platform "$POSTGIS_PLATFORM" . --file="$POSTGIS_DOCKERFILE" --tag="$IMAGE_NAME" +else + docker build . --file="$POSTGIS_DOCKERFILE" --tag="$IMAGE_NAME" +fi + +echo "Preparing PostGIS container for profile: ${SYSTEM_PROFILE}" +echo " Image: ${IMAGE_NAME}" +echo " Dockerfile: ${POSTGIS_DOCKERFILE}" +echo " Port: ${DB_PORT}:5432" +echo " Data: ${PROJECT_DIR}/pgdata" + +# Recreate the container so profile/tuning changes always take effect. +# Data persists because pgdata is mounted from the host. +if docker container inspect "$CONTAINER_NAME" >/dev/null 2>&1; then + echo "Removing existing container '${CONTAINER_NAME}' so updated settings take effect..." + docker rm -f "$CONTAINER_NAME" >/dev/null +fi + +echo "Creating new PostGIS container..." +docker run \ + --name "$CONTAINER_NAME" \ + -e POSTGRES_DB="$DB_NAME" \ + -e POSTGRES_USER="$DB_USER" \ + -e POSTGRES_PASSWORD="$DB_PASSWORD" \ + -e DATADIR=/var/lib/postgresql/data \ + -p "${DB_PORT}:5432" \ + -v "${PROJECT_DIR}/pgdata:/var/lib/postgresql/data" \ + -d \ + "$IMAGE_NAME" \ + -c shared_buffers="$PG_SHARED" \ + -c effective_cache_size="$PG_CACHE" \ + -c work_mem="$PG_WORK_MEM" \ + -c maintenance_work_mem="$PG_MAINT" \ + -c max_connections="$PG_MAX_CONN" \ + -c superuser_reserved_connections=10 \ + -c idle_session_timeout=600000 \ + -c log_connections=on \ + -c log_disconnections=on \ + -c wal_buffers=16MB \ + -c random_page_cost=1.1 \ + -c effective_io_concurrency=200 \ + || { echo "Failed to start PostGIS container"; exit 1; } + +echo "Waiting for PostGIS ARM64 to be ready..." +export PGPASSWORD="$DB_PASSWORD" +RETRY_COUNT=0 +until docker exec "$CONTAINER_NAME" pg_isready -U "$DB_USER" -d "$DB_NAME" >/dev/null 2>&1; do + RETRY_COUNT=$((RETRY_COUNT + 1)) + if [ "$RETRY_COUNT" -ge "$RETRY_MAX" ]; then + echo "Error: PostGIS did not become ready after $((RETRY_MAX * RETRY_INTERVAL)) seconds." + echo "Last container logs:" + docker logs --tail 50 "$CONTAINER_NAME" || true + exit 1 + fi + echo "PostGIS not ready yet, retrying..." + sleep "$RETRY_INTERVAL" +done + +echo "PostGIS is ready. Starting OpenSensorHub..." +sleep "$POSTGIS_READY_DELAY" + +OSH_DIR="${PROJECT_DIR}/osh-node-oscar" +if [ ! -d "$OSH_DIR" ]; then + echo "Error: osh-node-oscar directory not found in ${PROJECT_DIR}." + exit 1 +fi + +cd "$OSH_DIR" +if [ ! -f "./launch.sh" ]; then + echo "Error: launch.sh was not found in ${OSH_DIR}." + exit 1 +fi + +if [ ! -x "./launch.sh" ]; then + chmod +x ./launch.sh +fi + +exec ./launch.sh diff --git a/dist/release/launch-all.bat b/dist/release/launch-all.bat old mode 100755 new mode 100644 index 5c93081..bec96ac --- a/dist/release/launch-all.bat +++ b/dist/release/launch-all.bat @@ -1,117 +1,342 @@ @echo off -setlocal enabledelayedexpansion +setlocal EnableExtensions -REM ==== CONFIG ==== -set HOST=localhost -set PORT=5432 -set DB_NAME=gis -set USER=postgres -set RETRY_MAX=20 -set RETRY_INTERVAL=5 -set PROJECT_DIR=%cd% -set CONTAINER_NAME=oscar-postgis-container -set IMAGE_NAME=oscar-postgis +set "PROJECT_DIR=%~dp0" +set "ENV_FILE=%PROJECT_DIR%.env" +set "MATCH_EXPR=com.botts.impl.security.SensorHubWrapper" +set "FORCE_RESTART=%FORCE_RESTART%" +if not defined FORCE_RESTART set "FORCE_RESTART=0" +set "RETRY_MAX=%RETRY_MAX%" +if not defined RETRY_MAX set "RETRY_MAX=120" +set "RETRY_INTERVAL=%RETRY_INTERVAL%" +if not defined RETRY_INTERVAL set "RETRY_INTERVAL=2" +set "POSTGIS_READY_DELAY=%POSTGIS_READY_DELAY%" +if not defined POSTGIS_READY_DELAY set "POSTGIS_READY_DELAY=5" +set "POSTGIS_DOCKERFILE=%POSTGIS_DOCKERFILE%" +if not defined POSTGIS_DOCKERFILE set "POSTGIS_DOCKERFILE=Dockerfile" -echo PROJECT_DIR is: %PROJECT_DIR% +if not exist "%ENV_FILE%" ( + echo Error: .env file not found in "%PROJECT_DIR%". + echo Create it by copying env.template to .env and editing the values. + exit /b 1 +) + +call :load_env "%ENV_FILE%" + +if not defined IMAGE_NAME if defined POSTGIS_IMAGE_NAME set "IMAGE_NAME=%POSTGIS_IMAGE_NAME%" +if not defined IMAGE_NAME set "IMAGE_NAME=oscar-postgis" + +call :check_dependencies +if errorlevel 1 exit /b %ERRORLEVEL% +call :check_existing_oscar +if errorlevel 1 exit /b %ERRORLEVEL% +call :ensure_project_layout +if errorlevel 1 exit /b %ERRORLEVEL% + +if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" +if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" +if not defined DB_HOST set "DB_HOST=localhost" -where docker >nul 2>&1 -if %errorlevel% neq 0 ( - echo ERROR: Docker is not installed or not in PATH. +if not defined DB_NAME ( + echo Error: DB_NAME is not set in .env. exit /b 1 ) +if not defined DB_USER ( + echo Error: DB_USER is not set in .env. + exit /b 1 +) +if not defined DB_PASSWORD ( + echo Error: DB_PASSWORD is not set in .env. + exit /b 1 +) +if not defined DB_PORT ( + echo Error: DB_PORT is not set in .env. + exit /b 1 +) + +call :require_number DB_PORT +if errorlevel 1 exit /b %ERRORLEVEL% +call :require_number RETRY_MAX +if errorlevel 1 exit /b %ERRORLEVEL% +call :require_number RETRY_INTERVAL +if errorlevel 1 exit /b %ERRORLEVEL% +call :require_number POSTGIS_READY_DELAY +if errorlevel 1 exit /b %ERRORLEVEL% + +if /I "%SYSTEM_PROFILE%"=="RPI4" ( + set "SYSTEM_PROFILE=RPI4" + set "PG_SHARED=256MB" + set "PG_CACHE=1GB" + set "PG_WORK_MEM=2MB" + set "PG_MAINT=64MB" + set "PG_MAX_CONN=75" +) else if /I "%SYSTEM_PROFILE%"=="8GB" ( + set "SYSTEM_PROFILE=8GB" + set "PG_SHARED=512MB" + set "PG_CACHE=2GB" + set "PG_WORK_MEM=4MB" + set "PG_MAINT=128MB" + set "PG_MAX_CONN=125" +) else if /I "%SYSTEM_PROFILE%"=="16GB" ( + set "SYSTEM_PROFILE=16GB" + set "PG_SHARED=1GB" + set "PG_CACHE=4GB" + set "PG_WORK_MEM=8MB" + set "PG_MAINT=256MB" + set "PG_MAX_CONN=200" +) else if /I "%SYSTEM_PROFILE%"=="32GB" ( + set "SYSTEM_PROFILE=32GB" + set "PG_SHARED=2GB" + set "PG_CACHE=8GB" + set "PG_WORK_MEM=16MB" + set "PG_MAINT=512MB" + set "PG_MAX_CONN=300" +) else ( + echo Unknown profile '%SYSTEM_PROFILE%', using 8GB defaults. + set "SYSTEM_PROFILE=8GB" + set "PG_SHARED=512MB" + set "PG_CACHE=2GB" + set "PG_WORK_MEM=4MB" + set "PG_MAINT=128MB" + set "PG_MAX_CONN=125" +) -if not exist "%PROJECT_DIR%\pgdata" ( - echo Creating pgdata directory... - mkdir "%PROJECT_DIR%\pgdata" +set "PATH=%JAVA_HOME_DETECTED%\bin;%PATH%" + +if not exist "%PROJECT_DIR%pgdata" mkdir "%PROJECT_DIR%pgdata" >nul 2>nul +if not exist "%PROJECT_DIR%pgdata" ( + echo Error: failed to create pgdata directory. + exit /b 1 ) echo Building PostGIS Docker image... -pushd postgis -docker build . -f Dockerfile -t %IMAGE_NAME% -if %errorlevel% neq 0 ( - echo ERROR: Docker build failed. +pushd "%PROJECT_DIR%postgis" +docker build . --file="%POSTGIS_DOCKERFILE%" --tag="%IMAGE_NAME%" +if errorlevel 1 ( + echo Error: Docker build failed. + popd exit /b 1 ) popd -echo Starting PostGIS container... +echo Preparing PostGIS container for profile: %SYSTEM_PROFILE% +echo Image: %IMAGE_NAME% +echo Port: %DB_PORT%:5432 +echo Data: %PROJECT_DIR%pgdata -for /f "tokens=*" %%i in ('docker ps -a --format "{{.Names}}"') do ( - if "%%i"=="%CONTAINER_NAME%" ( - set CONTAINER_EXISTS=1 +docker container inspect "%CONTAINER_NAME%" >nul 2>&1 +if not errorlevel 1 ( + echo Removing existing container '%CONTAINER_NAME%' so updated settings take effect... + docker rm -f "%CONTAINER_NAME%" >nul + if errorlevel 1 ( + echo Error: failed to remove existing container '%CONTAINER_NAME%'. + exit /b 1 ) ) -for /f "tokens=*" %%i in ('docker ps --format "{{.Names}}"') do ( - if "%%i"=="%CONTAINER_NAME%" ( - set CONTAINER_RUNNING=1 - ) +echo Creating new container... +docker run ^ + --name "%CONTAINER_NAME%" ^ + -e "POSTGRES_DB=%DB_NAME%" ^ + -e "POSTGRES_USER=%DB_USER%" ^ + -e "POSTGRES_PASSWORD=%DB_PASSWORD%" ^ + -p "%DB_PORT%:5432" ^ + -v "%PROJECT_DIR%pgdata:/var/lib/postgresql/data" ^ + -d ^ + "%IMAGE_NAME%" ^ + -c "shared_buffers=%PG_SHARED%" ^ + -c "effective_cache_size=%PG_CACHE%" ^ + -c "work_mem=%PG_WORK_MEM%" ^ + -c "maintenance_work_mem=%PG_MAINT%" ^ + -c "max_connections=%PG_MAX_CONN%" ^ + -c superuser_reserved_connections=10 ^ + -c idle_session_timeout=600000 ^ + -c log_connections=on ^ + -c log_disconnections=on ^ + -c wal_buffers=16MB ^ + -c random_page_cost=1.1 ^ + -c effective_io_concurrency=200 +if errorlevel 1 ( + echo Error: failed to start PostGIS container. + exit /b 1 ) -if defined CONTAINER_EXISTS ( - if defined CONTAINER_RUNNING ( - echo Container already running: %CONTAINER_NAME% - ) else ( - echo Starting existing container: %CONTAINER_NAME% - docker start %CONTAINER_NAME% - ) -) else ( - echo Creating new container: %CONTAINER_NAME% - docker run ^ - --name %CONTAINER_NAME% ^ - -e POSTGRES_DB=%DB_NAME% ^ - -e POSTGRES_USER=%USER% ^ - -e POSTGRES_PASSWORD=postgres ^ - -p %PORT%:5432 ^ - -v "%PROJECT_DIR%\pgdata:/var/lib/postgresql/data" ^ - -d ^ - %IMAGE_NAME% - - if %errorlevel% neq 0 ( - echo ERROR: Failed to start PostGIS container. - exit /b 1 - ) +echo Waiting for PostGIS to be ready... +set "PGPASSWORD=%DB_PASSWORD%" +set /a RETRY_COUNT=0 + +:wait_loop +docker exec "%CONTAINER_NAME%" pg_isready -U "%DB_USER%" -d "%DB_NAME%" >nul 2>&1 +if not errorlevel 1 goto after_wait +set /a RETRY_COUNT+=1 +if %RETRY_COUNT% GEQ %RETRY_MAX% ( + echo Error: PostGIS did not become ready after %RETRY_MAX% attempts. + echo Last container logs: + docker logs --tail 50 "%CONTAINER_NAME%" + exit /b 1 ) +timeout /t %RETRY_INTERVAL% /nobreak >nul +goto wait_loop -echo Waiting for PostGIS database to become ready... +:after_wait +echo PostGIS is ready. +timeout /t %POSTGIS_READY_DELAY% /nobreak >nul -set RETRY_COUNT=0 +cd /d "%PROJECT_DIR%osh-node-oscar" +if errorlevel 1 ( + echo Error: osh-node-oscar directory not found in "%PROJECT_DIR%". + exit /b 1 +) -:wait_loop -docker exec %CONTAINER_NAME% pg_isready -U %USER% -d %DB_NAME% >nul 2>&1 -if %errorlevel% equ 0 ( - echo Received OK from PostGIS. Please wait for initialization... - goto after_wait +if not exist "launch.bat" ( + echo Error: launch.bat not found in "%CD%". + exit /b 1 ) -echo PostGIS not ready yet, retrying... -set /a RETRY_COUNT+=1 +call "launch.bat" +set "LAUNCH_EXIT_CODE=%ERRORLEVEL%" +endlocal & exit /b %LAUNCH_EXIT_CODE% -if %RETRY_COUNT% geq %RETRY_MAX% ( - echo ERROR: PostGIS did not become ready in time. +:check_dependencies +where powershell >nul 2>nul +if errorlevel 1 ( + echo Error: PowerShell is required but was not found on PATH. exit /b 1 ) -timeout /t %RETRY_INTERVAL% >nul -goto wait_loop +where java >nul 2>nul +if errorlevel 1 ( + echo Error: java was not found on PATH. Install OpenJDK 21 or newer. + exit /b 1 +) -:after_wait +where docker >nul 2>nul +if errorlevel 1 ( + echo Error: docker was not found on PATH. Install Docker Desktop and make sure it is running. + exit /b 1 +) -timeout /t 10 >nul +docker info >nul 2>nul +if errorlevel 1 ( + echo Error: Docker is installed, but the Docker daemon is not running. + exit /b 1 +) -echo PostGIS database is ready! +set "JAVA_HOME_LINE=" +for /f "delims=" %%A in ('java -XshowSettings:properties -version 2^>^&1 ^| findstr /c:"java.home ="') do ( + set "JAVA_HOME_LINE=%%A" + goto :deps_java_home_line +) -cd "%PROJECT_DIR%\osh-node-oscar" -if %errorlevel% neq 0 ( - echo ERROR: osh-node-oscar directory not found. +:deps_java_home_line +if not defined JAVA_HOME_LINE ( + echo Error: could not determine java.home from the installed Java runtime. exit /b 1 ) -if exist launch.bat ( - call launch.bat -) else ( - echo WARNING: launch.bat not found. Trying launch.sh through Git Bash... - bash launch.sh +for /f "tokens=1,* delims==" %%A in ("%JAVA_HOME_LINE%") do set "JAVA_HOME_DETECTED=%%B" +for /f "tokens=* delims= " %%A in ("%JAVA_HOME_DETECTED%") do set "JAVA_HOME_DETECTED=%%A" + +if not exist "%JAVA_HOME_DETECTED%\bin\java.exe" ( + echo Error: Java executable not found under "%JAVA_HOME_DETECTED%\bin\java.exe". + exit /b 1 +) +if not exist "%JAVA_HOME_DETECTED%\bin\keytool.exe" ( + echo Error: keytool.exe not found under "%JAVA_HOME_DETECTED%\bin\keytool.exe". + exit /b 1 +) + +set "JAVA_VERSION_LINE=" +for /f "delims=" %%A in ('"%JAVA_HOME_DETECTED%\bin\java.exe" -version 2^>^&1 ^| findstr /r /c:"version \""') do ( + set "JAVA_VERSION_LINE=%%A" + goto :deps_java_version_line +) + +:deps_java_version_line +if not defined JAVA_VERSION_LINE ( + echo Error: could not determine Java version. OpenJDK 21 or newer is required. + exit /b 1 +) + +for /f "tokens=2 delims=\"" %%A in ("%JAVA_VERSION_LINE%") do set "JAVA_VERSION_RAW=%%A" +for /f "tokens=1 delims=." %%A in ("%JAVA_VERSION_RAW%") do set "JAVA_MAJOR=%%A" +if not defined JAVA_MAJOR ( + echo Error: could not parse Java version from "%JAVA_VERSION_LINE%". + exit /b 1 +) +if %JAVA_MAJOR% LSS 21 ( + echo Error: Java 21 or newer is required. Found Java %JAVA_MAJOR%. + exit /b 1 +) +exit /b 0 + +:find_existing_oscar +set "OSCAR_PID=" +for /f %%P in ('powershell -NoProfile -Command "$p = Get-CimInstance Win32_Process ^| Where-Object { $_.Name -match ''^java(\.exe)?$'' -and $_.CommandLine -like ''*com.botts.impl.security.SensorHubWrapper*'' } ^| Select-Object -ExpandProperty ProcessId -First 1; if ($p) { Write-Output $p }"') do set "OSCAR_PID=%%P" +exit /b 0 + +:check_existing_oscar +call :find_existing_oscar +if not defined OSCAR_PID exit /b 0 + +if "%FORCE_RESTART%"=="1" ( + echo Existing OSCAR instance found with PID %OSCAR_PID%. Replacing because FORCE_RESTART=1. + taskkill /PID %OSCAR_PID% /T /F >nul 2>nul + timeout /t 2 /nobreak >nul + call :find_existing_oscar + if defined OSCAR_PID ( + echo Error: could not stop the existing OSCAR instance. + exit /b 1 + ) + exit /b 0 +) + +echo OSCAR is already running with PID %OSCAR_PID%. +echo Stop the running instance first, or set FORCE_RESTART=1 to replace it. +exit /b 1 + +:ensure_project_layout +if not exist "%PROJECT_DIR%postgis" ( + echo Error: postgis directory not found in "%PROJECT_DIR%". + exit /b 1 +) +if not exist "%PROJECT_DIR%postgis\%POSTGIS_DOCKERFILE%" ( + echo Error: %POSTGIS_DOCKERFILE% not found in "%PROJECT_DIR%postgis". + exit /b 1 +) +if not exist "%PROJECT_DIR%osh-node-oscar" ( + echo Error: osh-node-oscar directory not found in "%PROJECT_DIR%". + exit /b 1 +) +if not exist "%PROJECT_DIR%osh-node-oscar\launch.bat" ( + echo Error: launch.bat not found in "%PROJECT_DIR%osh-node-oscar". + exit /b 1 +) +exit /b 0 + +:require_number +call set "VALUE=%%%~1%%" +if not defined VALUE ( + echo Error: %~1 must be a number, got ''. + exit /b 1 +) +for /f "delims=0123456789" %%A in ("%VALUE%") do ( + echo Error: %~1 must be a number, got '%VALUE%'. + exit /b 1 +) +exit /b 0 + +:load_env +for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( + set "ENV_NAME=%%A" + set "ENV_VALUE=%%B" + call :set_env_var ) +exit /b 0 -endlocal +:set_env_var +if not defined ENV_NAME exit /b 0 +if "%ENV_NAME:~0,1%"=="#" exit /b 0 +if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" +set "%ENV_NAME%=%ENV_VALUE%" +exit /b 0 diff --git a/dist/release/launch-all.sh b/dist/release/launch-all.sh old mode 100755 new mode 100644 index 5716c7c..29639c5 --- a/dist/release/launch-all.sh +++ b/dist/release/launch-all.sh @@ -1,78 +1,291 @@ -#!/bin/bash - -HOST="localhost" -PORT="5432" -DB_NAME="gis" -DB_USER="postgres" -RETRY_MAX=20 -RETRY_INTERVAL=5 -PROJECT_DIR="$(pwd)" # Store the original directory -CONTAINER_NAME="oscar-postgis-container" - -#docker rm -f "$CONTAINER_NAME" 2>/dev/null || true - -# Create pgdata directory if needed -if [ ! -d "${PROJECT_DIR}/pgdata" ]; then - echo "Creating pgdata folder..." - mkdir -p "${PROJECT_DIR}/pgdata" -fi +#!/usr/bin/env bash +set -euo pipefail -# Check Docker -if ! command -v docker >/dev/null 2>&1; then - echo "Error: Docker is not installed. Please install Docker first." - exit 1 -fi +PROJECT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +ENV_FILE="$PROJECT_DIR/.env" +MATCH_EXPR='com.botts.impl.security.SensorHubWrapper' +FORCE_RESTART="${FORCE_RESTART:-0}" +RETRY_MAX="${RETRY_MAX:-120}" +RETRY_INTERVAL="${RETRY_INTERVAL:-2}" +POSTGIS_READY_DELAY="${POSTGIS_READY_DELAY:-5}" +IMAGE_NAME="${POSTGIS_IMAGE_NAME:-${IMAGE_NAME:-oscar-postgis}}" +POSTGIS_DOCKERFILE="${POSTGIS_DOCKERFILE:-Dockerfile}" -echo "Building PostGIS Docker image..." +load_env() { + local env_file="$1" + while IFS= read -r line || [ -n "$line" ]; do + case "$line" in + ""|"#"*) continue ;; + export\ *) line="${line#export }" ;; + esac + local name="${line%%=*}" + local value="${line#*=}" + value="${value%$'\r'}" + export "${name}=${value}" + done < "$env_file" +} + +require_cmd() { + local cmd="$1" + if ! command -v "$cmd" >/dev/null 2>&1; then + echo "Error: required command not found: $cmd" + exit 1 + fi +} + +get_java_major() { + java -version 2>&1 | awk -F'"' '/version/ { split($2, v, "."); print v[1]; exit }' +} + +check_dependencies() { + require_cmd bash + require_cmd java + require_cmd keytool + require_cmd docker + + if ! docker info >/dev/null 2>&1; then + echo "Error: Docker is installed, but the Docker daemon is not running." + exit 1 + fi + + local java_major + java_major="$(get_java_major || true)" + if [[ -z "$java_major" || ! "$java_major" =~ ^[0-9]+$ ]]; then + echo "Error: could not determine Java version. Java 21 or newer is required." + exit 1 + fi + if [ "$java_major" -lt 21 ]; then + echo "Error: Java 21 or newer is required. Found Java $java_major." + exit 1 + fi +} + +find_existing_oscar_pids() { + pgrep -f "$MATCH_EXPR" || true +} + +stop_existing_oscar() { + local pids="$1" + if [ -z "$pids" ]; then + return 0 + fi + + echo "Stopping existing OSCAR instance(s): $pids" + kill $pids 2>/dev/null || true + + local waited=0 + while [ "$waited" -lt 15 ]; do + sleep 1 + waited=$((waited + 1)) + if [ -z "$(find_existing_oscar_pids)" ]; then + return 0 + fi + done -cd postgis || { echo "Error: postgis directory not found"; exit 1; } + echo "Existing OSCAR instance still running after graceful stop. Forcing stop." + kill -9 $pids 2>/dev/null || true + sleep 1 -# Build PostGIS -docker build . \ - --file=Dockerfile \ - --tag=oscar-postgis + if [ -n "$(find_existing_oscar_pids)" ]; then + echo "Error: unable to stop the existing OSCAR instance." + exit 1 + fi +} + +check_existing_oscar() { + local pids + pids="$(find_existing_oscar_pids)" -echo "Starting PostGIS container..." + if [ -z "$pids" ]; then + return 0 + fi + if [ "$FORCE_RESTART" = "1" ]; then + echo "Existing OSCAR instance found with PID(s): $pids. Replacing because FORCE_RESTART=1." + stop_existing_oscar "$pids" + return 0 + fi -echo "PROJECT_DIR is set to: ${PROJECT_DIR}" + echo "OSCAR is already running with PID(s): $pids." + echo "Stop the running instance first, or set FORCE_RESTART=1 to replace it." + exit 1 +} -if docker ps -a --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then - # The container exists - if docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then - echo "Container already running: ${CONTAINER_NAME}" - else - echo "Starting existing container: ${CONTAINER_NAME}" - docker start "${CONTAINER_NAME}" +require_env() { + local name="$1" + local value="${!name:-}" + if [ -z "$value" ]; then + echo "Error: ${name} is not set in .env." + exit 1 fi -else - echo "Creating new container: ${CONTAINER_NAME}" - docker run \ - --name "$CONTAINER_NAME" \ - -e POSTGRES_DB="$DB_NAME" \ - -e POSTGRES_USER="$DB_USER" \ - -e POSTGRES_PASSWORD="postgres" \ - -p $PORT:5432 \ - -v "${PROJECT_DIR}/pgdata:/var/lib/postgresql/data" \ - -d \ - oscar-postgis || { echo "Failed to start PostGIS container"; exit 1; } +} + +require_number() { + local name="$1" + local value="${!name:-}" + case "$value" in + ''|*[!0-9]*) + echo "Error: ${name} must be a number, got '${value}'." + exit 1 + ;; + esac +} + +ensure_project_layout() { + if [ ! -d "$PROJECT_DIR/postgis" ]; then + echo "Error: postgis directory not found in $PROJECT_DIR" + exit 1 + fi + + if [ ! -f "$PROJECT_DIR/postgis/$POSTGIS_DOCKERFILE" ]; then + echo "Error: $POSTGIS_DOCKERFILE not found in $PROJECT_DIR/postgis" + exit 1 + fi + + if [ ! -d "$PROJECT_DIR/osh-node-oscar" ]; then + echo "Error: osh-node-oscar directory not found in $PROJECT_DIR" + exit 1 + fi + + if [ ! -f "$PROJECT_DIR/osh-node-oscar/launch.sh" ]; then + echo "Error: launch.sh not found in $PROJECT_DIR/osh-node-oscar" + exit 1 + fi + + mkdir -p "$PROJECT_DIR/pgdata" +} + +if [ ! -f "$ENV_FILE" ]; then + echo "Error: .env file not found in $PROJECT_DIR" + echo "Create it by copying env.template to .env and editing the values." + exit 1 fi -# Wait for PostgreSQL/PostGIS to become ready -echo "Waiting for PostGIS (PostgreSQL) to be ready..." +load_env "$ENV_FILE" +check_dependencies +check_existing_oscar +ensure_project_layout + +SYSTEM_PROFILE="${SYSTEM_PROFILE:-8GB}" +CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" +DB_HOST="${DB_HOST:-localhost}" +export SYSTEM_PROFILE CONTAINER_NAME DB_HOST RETRY_MAX RETRY_INTERVAL POSTGIS_READY_DELAY IMAGE_NAME POSTGIS_DOCKERFILE -RETRY_COUNT=0 -export PGPASSWORD=postgres # Needed for pg_isready with password +require_env DB_NAME +require_env DB_USER +require_env DB_PASSWORD +require_env DB_PORT -until docker exec "$CONTAINER_NAME" pg_isready -U "$DB_USER" -d "$DB_NAME" > /dev/null 2>&1; do - echo "PostGIS not ready yet, retrying..." - sleep "${RETRY_INTERVAL}" +require_number DB_PORT +require_number RETRY_MAX +require_number RETRY_INTERVAL +require_number POSTGIS_READY_DELAY + +case "${SYSTEM_PROFILE^^}" in + RPI4) + SYSTEM_PROFILE="RPI4" + PG_SHARED="256MB" + PG_CACHE="1GB" + PG_WORK_MEM="2MB" + PG_MAINT="64MB" + PG_MAX_CONN="75" + ;; + 8GB) + SYSTEM_PROFILE="8GB" + PG_SHARED="512MB" + PG_CACHE="2GB" + PG_WORK_MEM="4MB" + PG_MAINT="128MB" + PG_MAX_CONN="125" + ;; + 16GB) + SYSTEM_PROFILE="16GB" + PG_SHARED="1GB" + PG_CACHE="4GB" + PG_WORK_MEM="8MB" + PG_MAINT="256MB" + PG_MAX_CONN="200" + ;; + 32GB) + SYSTEM_PROFILE="32GB" + PG_SHARED="2GB" + PG_CACHE="8GB" + PG_WORK_MEM="16MB" + PG_MAINT="512MB" + PG_MAX_CONN="300" + ;; + *) + echo "Unknown profile '${SYSTEM_PROFILE}', using 8GB defaults." + SYSTEM_PROFILE="8GB" + PG_SHARED="512MB" + PG_CACHE="2GB" + PG_WORK_MEM="4MB" + PG_MAINT="128MB" + PG_MAX_CONN="125" + ;; + esac + +export SYSTEM_PROFILE CONTAINER_NAME DB_NAME DB_USER DB_PASSWORD DB_PORT + +echo "Building PostGIS Docker image..." +( + cd "$PROJECT_DIR/postgis" + docker build . --file="$POSTGIS_DOCKERFILE" --tag="$IMAGE_NAME" +) + +echo "Preparing PostGIS container for profile: $SYSTEM_PROFILE" +echo " Image: $IMAGE_NAME" +echo " Port: ${DB_PORT}:5432" +echo " Data: $PROJECT_DIR/pgdata" + +if docker container inspect "$CONTAINER_NAME" >/dev/null 2>&1; then + echo "Removing existing container '$CONTAINER_NAME' so updated settings take effect..." + docker rm -f "$CONTAINER_NAME" >/dev/null +fi + +echo "Creating new container..." +docker run \ + --name "$CONTAINER_NAME" \ + -e POSTGRES_DB="$DB_NAME" \ + -e POSTGRES_USER="$DB_USER" \ + -e POSTGRES_PASSWORD="$DB_PASSWORD" \ + -p "${DB_PORT}:5432" \ + -v "$PROJECT_DIR/pgdata:/var/lib/postgresql/data" \ + -d \ + "$IMAGE_NAME" \ + -c shared_buffers="$PG_SHARED" \ + -c effective_cache_size="$PG_CACHE" \ + -c work_mem="$PG_WORK_MEM" \ + -c maintenance_work_mem="$PG_MAINT" \ + -c max_connections="$PG_MAX_CONN" \ + -c superuser_reserved_connections=10 \ + -c idle_session_timeout=600000 \ + -c log_connections=on \ + -c log_disconnections=on \ + -c wal_buffers=16MB \ + -c random_page_cost=1.1 \ + -c effective_io_concurrency=200 + +echo "Waiting for PostGIS to be ready..." +export PGPASSWORD="$DB_PASSWORD" +retry_count=0 +until docker exec "$CONTAINER_NAME" pg_isready -U "$DB_USER" -d "$DB_NAME" >/dev/null 2>&1; do + retry_count=$((retry_count + 1)) + if [ "$retry_count" -ge "$RETRY_MAX" ]; then + echo "Error: PostGIS did not become ready after $((RETRY_MAX * RETRY_INTERVAL)) seconds." + echo "Last container logs:" + docker logs --tail 50 "$CONTAINER_NAME" || true + exit 1 + fi + sleep "$RETRY_INTERVAL" done -echo "PostGIS (PostgreSQL) is ready! Please wait for OpenSensorHub to start..." +echo "PostGIS is ready." +sleep "$POSTGIS_READY_DELAY" -sleep 10 +cd "$PROJECT_DIR/osh-node-oscar" +if [ ! -x ./launch.sh ]; then + chmod +x ./launch.sh +fi -# Launch osh-node-oscar -cd "$PROJECT_DIR/osh-node-oscar" || { echo "Error: osh-node-oscar not found"; exit 1; } -./launch.sh \ No newline at end of file +exec ./launch.sh diff --git a/dist/release/launch-all_old.bat b/dist/release/launch-all_old.bat new file mode 100644 index 0000000..646c7ce --- /dev/null +++ b/dist/release/launch-all_old.bat @@ -0,0 +1,192 @@ +@echo off +setlocal EnableExtensions + +rem Resolve project root from this script's location instead of the caller's cwd. +set "PROJECT_DIR=%~dp0" +set "ENV_FILE=%PROJECT_DIR%.env" +set "IMAGE_NAME=oscar-postgis" + +if not exist "%ENV_FILE%" ( + echo Error: .env file not found in "%PROJECT_DIR%". + echo Create it by copying env.template to .env and editing the values. + exit /b 1 +) + +call :load_env "%ENV_FILE%" + +if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" +if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" + +if not defined DB_NAME ( + echo Error: DB_NAME is not set in .env. + exit /b 1 +) +if not defined DB_USER ( + echo Error: DB_USER is not set in .env. + exit /b 1 +) +if not defined DB_PASSWORD ( + echo Error: DB_PASSWORD is not set in .env. + exit /b 1 +) +if not defined DB_PORT ( + echo Error: DB_PORT is not set in .env. + exit /b 1 +) + +if /I "%SYSTEM_PROFILE%"=="RPI4" ( + set "PG_SHARED=256MB" + set "PG_CACHE=1GB" + set "PG_WORK_MEM=2MB" + set "PG_MAINT=64MB" + set "PG_MAX_CONN=75" +) else if /I "%SYSTEM_PROFILE%"=="8GB" ( + set "PG_SHARED=512MB" + set "PG_CACHE=2GB" + set "PG_WORK_MEM=4MB" + set "PG_MAINT=128MB" + set "PG_MAX_CONN=125" +) else if /I "%SYSTEM_PROFILE%"=="16GB" ( + set "PG_SHARED=1GB" + set "PG_CACHE=4GB" + set "PG_WORK_MEM=8MB" + set "PG_MAINT=256MB" + set "PG_MAX_CONN=200" +) else if /I "%SYSTEM_PROFILE%"=="32GB" ( + set "PG_SHARED=2GB" + set "PG_CACHE=8GB" + set "PG_WORK_MEM=16MB" + set "PG_MAINT=512MB" + set "PG_MAX_CONN=300" +) else ( + echo Unknown profile '%SYSTEM_PROFILE%', using 8GB defaults. + set "PG_SHARED=512MB" + set "PG_CACHE=2GB" + set "PG_WORK_MEM=4MB" + set "PG_MAINT=128MB" + set "PG_MAX_CONN=125" +) + +if not exist "%PROJECT_DIR%pgdata" ( + mkdir "%PROJECT_DIR%pgdata" + if errorlevel 1 ( + echo Error: failed to create pgdata directory. + exit /b 1 + ) +) + +where docker >nul 2>&1 +if errorlevel 1 ( + echo Error: Docker is not installed or is not in PATH. + exit /b 1 +) + +echo Building PostGIS Docker image... +if not exist "%PROJECT_DIR%postgis" ( + echo Error: postgis directory not found in "%PROJECT_DIR%". + exit /b 1 +) +pushd "%PROJECT_DIR%postgis" +docker build . --file=Dockerfile --tag=%IMAGE_NAME% +if errorlevel 1 ( + echo Error: Docker build failed. + popd + exit /b 1 +) +popd + +echo Preparing PostGIS container for profile: %SYSTEM_PROFILE% + +rem Recreate the container so profile/tuning changes always take effect. +rem Data persists because pgdata is mounted from the host. +docker container inspect "%CONTAINER_NAME%" >nul 2>&1 +if not errorlevel 1 ( + echo Removing existing container '%CONTAINER_NAME%' so updated settings take effect... + docker rm -f "%CONTAINER_NAME%" >nul + if errorlevel 1 ( + echo Error: failed to remove existing container '%CONTAINER_NAME%'. + exit /b 1 + ) +) + +echo Creating new container... +docker run ^ + --name "%CONTAINER_NAME%" ^ + -e "POSTGRES_DB=%DB_NAME%" ^ + -e "POSTGRES_USER=%DB_USER%" ^ + -e "POSTGRES_PASSWORD=%DB_PASSWORD%" ^ + -p "%DB_PORT%:5432" ^ + -v "%PROJECT_DIR%pgdata:/var/lib/postgresql/data" ^ + -d ^ + %IMAGE_NAME% ^ + -c "shared_buffers=%PG_SHARED%" ^ + -c "effective_cache_size=%PG_CACHE%" ^ + -c "work_mem=%PG_WORK_MEM%" ^ + -c "maintenance_work_mem=%PG_MAINT%" ^ + -c "max_connections=%PG_MAX_CONN%" ^ + -c superuser_reserved_connections=10 ^ + -c idle_session_timeout=600000 ^ + -c log_connections=on ^ + -c log_disconnections=on ^ + -c wal_buffers=16MB ^ + -c random_page_cost=1.1 ^ + -c effective_io_concurrency=200 +if errorlevel 1 ( + echo Error: failed to start PostGIS container. + exit /b 1 +) + +echo Waiting for PostGIS to be ready... +set "PGPASSWORD=%DB_PASSWORD%" + +:wait_loop +docker exec "%CONTAINER_NAME%" pg_isready -U "%DB_USER%" -d "%DB_NAME%" >nul 2>&1 +if not errorlevel 1 goto after_wait +timeout /t 2 /nobreak >nul +goto wait_loop + +:after_wait +echo PostGIS is ready. +timeout /t 5 /nobreak >nul + +cd /d "%PROJECT_DIR%osh-node-oscar" +if errorlevel 1 ( + echo Error: osh-node-oscar directory not found in "%PROJECT_DIR%". + exit /b 1 +) + +if exist "launch.bat" goto run_launch_bat +if exist "launch.sh" goto run_launch_sh +echo Error: neither launch.bat nor launch.sh was found in "%CD%". +exit /b 1 + +:run_launch_bat +call "launch.bat" +set "LAUNCH_EXIT_CODE=%ERRORLEVEL%" +goto launch_done + +:run_launch_sh +echo Warning: launch.bat not found. Trying launch.sh through Bash... +bash "launch.sh" +set "LAUNCH_EXIT_CODE=%ERRORLEVEL%" +goto launch_done + +:launch_done +endlocal & exit /b %LAUNCH_EXIT_CODE% + +:load_env +rem Minimal .env loader for KEY=VALUE lines. Blank lines are ignored by for /f. +rem Lines beginning with # are ignored. Empty values clear the variable. +for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( + set "ENV_NAME=%%A" + set "ENV_VALUE=%%B" + call :set_env_var +) +exit /b 0 + +:set_env_var +if not defined ENV_NAME exit /b 0 +if "%ENV_NAME:~0,1%"=="#" exit /b 0 +if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" +set "%ENV_NAME%=%ENV_VALUE%" +exit /b 0 diff --git a/dist/release/launch-all_old.sh b/dist/release/launch-all_old.sh new file mode 100644 index 0000000..7ab5491 --- /dev/null +++ b/dist/release/launch-all_old.sh @@ -0,0 +1,111 @@ +#!/bin/bash +set -euo pipefail + +PROJECT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +ENV_FILE="${PROJECT_DIR}/.env" + +if [ ! -f "$ENV_FILE" ]; then + echo "Error: .env file not found in $PROJECT_DIR" + exit 1 +fi + +set -a +. "$ENV_FILE" +set +a + +CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" + +case "${SYSTEM_PROFILE:-8GB}" in + "RPI4") + PG_SHARED="256MB" + PG_CACHE="1GB" + PG_WORK_MEM="2MB" + PG_MAINT="64MB" + PG_MAX_CONN="75" + ;; + "8GB") + PG_SHARED="512MB" + PG_CACHE="2GB" + PG_WORK_MEM="4MB" + PG_MAINT="128MB" + PG_MAX_CONN="125" + ;; + "16GB") + PG_SHARED="1GB" + PG_CACHE="4GB" + PG_WORK_MEM="8MB" + PG_MAINT="256MB" + PG_MAX_CONN="200" + ;; + "32GB") + PG_SHARED="2GB" + PG_CACHE="8GB" + PG_WORK_MEM="16MB" + PG_MAINT="512MB" + PG_MAX_CONN="300" + ;; + *) + echo "Unknown profile '${SYSTEM_PROFILE}', using 8GB defaults." + PG_SHARED="512MB" + PG_CACHE="2GB" + PG_WORK_MEM="4MB" + PG_MAINT="128MB" + PG_MAX_CONN="125" + ;; +esac + +mkdir -p "${PROJECT_DIR}/pgdata" + +if ! command -v docker >/dev/null 2>&1; then + echo "Error: Docker is not installed." + exit 1 +fi + +echo "Building PostGIS Docker image..." +cd "${PROJECT_DIR}/postgis" || { echo "Error: postgis directory not found"; exit 1; } +docker build . --file=Dockerfile --tag=oscar-postgis + +echo "Preparing PostGIS container for profile: ${SYSTEM_PROFILE}" + +# Recreate the container so new tuning always applies. +# Data persists because pgdata is mounted from the host. +if docker ps -a --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then + echo "Removing existing container '${CONTAINER_NAME}' so updated settings take effect..." + docker rm -f "${CONTAINER_NAME}" >/dev/null +fi + +echo "Creating new container..." +docker run \ + --name "${CONTAINER_NAME}" \ + -e POSTGRES_DB="${DB_NAME}" \ + -e POSTGRES_USER="${DB_USER}" \ + -e POSTGRES_PASSWORD="${DB_PASSWORD}" \ + -p "${DB_PORT}:5432" \ + -v "${PROJECT_DIR}/pgdata:/var/lib/postgresql/data" \ + -d \ + oscar-postgis \ + -c shared_buffers="${PG_SHARED}" \ + -c effective_cache_size="${PG_CACHE}" \ + -c work_mem="${PG_WORK_MEM}" \ + -c maintenance_work_mem="${PG_MAINT}" \ + -c max_connections="${PG_MAX_CONN}" \ + -c superuser_reserved_connections=10 \ + -c idle_session_timeout=600000 \ + -c log_connections=on \ + -c log_disconnections=on \ + -c wal_buffers=16MB \ + -c random_page_cost=1.1 \ + -c effective_io_concurrency=200 \ + || { echo "Failed to start PostGIS container"; exit 1; } + +echo "Waiting for PostGIS to be ready..." +export PGPASSWORD="${DB_PASSWORD}" +until docker exec "${CONTAINER_NAME}" pg_isready -U "${DB_USER}" -d "${DB_NAME}" >/dev/null 2>&1; do + sleep 2 +done + +echo "PostGIS is ready." +sleep 5 + +cd "${PROJECT_DIR}/osh-node-oscar" || { echo "Error: osh-node-oscar not found"; exit 1; } +exec ./launch.sh diff --git a/dist/release/monitor-oscar.bat b/dist/release/monitor-oscar.bat new file mode 100644 index 0000000..a325b4e --- /dev/null +++ b/dist/release/monitor-oscar.bat @@ -0,0 +1,249 @@ +@echo off +setlocal EnableExtensions + +if /I "%~1"=="stop" goto :stop_mode + +set "SCRIPT_DIR=%~dp0" +for %%I in ("%SCRIPT_DIR%.") do set "PROJECT_DIR=%%~fI" + +call :timestamp OUT_STAMP +set "OUT_DIR=%PROJECT_DIR%\oscar-monitor-%OUT_STAMP%" +set "ENV_FILE=%PROJECT_DIR%\.env" +set "CONTAINER_NAME=oscar-postgis-container" +set "DB_NAME=gis" +set "DB_USER=postgres" +set "DB_PASSWORD=postgres" +set "MATCH_EXPR=com.botts.impl.security.SensorHubWrapper" +set "INTERVAL=%INTERVAL%" +if not defined INTERVAL set "INTERVAL=60" +set "MAX_WAIT_SECONDS=%MAX_WAIT_SECONDS%" +if not defined MAX_WAIT_SECONDS set "MAX_WAIT_SECONDS=300" +set "JFR_NAME=%JFR_NAME%" +if not defined JFR_NAME set "JFR_NAME=oscar" +set "JFR_MAX_AGE=%JFR_MAX_AGE%" +if not defined JFR_MAX_AGE set "JFR_MAX_AGE=4h" +set "JFR_MAX_SIZE=%JFR_MAX_SIZE%" +if not defined JFR_MAX_SIZE set "JFR_MAX_SIZE=1g" +set "LAUNCH_CMD=%PROJECT_DIR%\launch-all.bat" +set "ATTACH_TO_EXISTING=%ATTACH_TO_EXISTING%" +if not defined ATTACH_TO_EXISTING set "ATTACH_TO_EXISTING=0" +set "FORCE_RESTART=%FORCE_RESTART%" +if not defined FORCE_RESTART set "FORCE_RESTART=0" + +if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" + +call :check_dependencies +if errorlevel 1 exit /b %ERRORLEVEL% + +if not exist "%OUT_DIR%" mkdir "%OUT_DIR%" +echo timestamp,total_sessions,active,idle,idle_in_transaction,max_connections,superuser_reserved_connections,failed_psql>"%OUT_DIR%\db-connection-trend.csv" + +echo %DATE% %TIME% Monitor output: %OUT_DIR% +echo %DATE% %TIME% Launch command: %LAUNCH_CMD% + +call :find_existing_oscar +if defined OSCAR_PID ( + if "%ATTACH_TO_EXISTING%"=="1" ( + set "JVM_PID=%OSCAR_PID%" + set "USE_EXISTING=1" + echo %DATE% %TIME% Attaching to existing OSCAR PID %JVM_PID% + ) else if "%FORCE_RESTART%"=="1" ( + echo %DATE% %TIME% Existing OSCAR instance found with PID %OSCAR_PID%. Replacing because FORCE_RESTART=1. + taskkill /PID %OSCAR_PID% /T /F >nul 2>nul + timeout /t 2 /nobreak >nul + call :find_existing_oscar + if defined OSCAR_PID ( + echo Error: could not stop the existing OSCAR instance. + exit /b 1 + ) + ) else ( + echo OSCAR is already running with PID %OSCAR_PID%. + echo Set ATTACH_TO_EXISTING=1 to monitor the running instance, or FORCE_RESTART=1 to replace it. + exit /b 1 + ) +) + +if not defined USE_EXISTING ( + if not exist "%LAUNCH_CMD%" ( + echo Error: launch command not found: "%LAUNCH_CMD%" + exit /b 1 + ) + + powershell -NoProfile -Command "$p = Start-Process -FilePath 'cmd.exe' -ArgumentList '/c ""%LAUNCH_CMD%""' -RedirectStandardOutput '%OUT_DIR%\launch.stdout.log' -RedirectStandardError '%OUT_DIR%\launch.stderr.log' -PassThru; Write-Output $p.Id" > "%OUT_DIR%\launcher-pid.txt" + for /f "usebackq" %%P in ("%OUT_DIR%\launcher-pid.txt") do set "LAUNCH_PID=%%P" + + echo %DATE% %TIME% Waiting for OSCAR Java process... + set /a WAITED=0 + + :wait_for_jvm + call :find_existing_oscar + if defined OSCAR_PID ( + set "JVM_PID=%OSCAR_PID%" + goto :have_jvm + ) + + if %WAITED% GEQ %MAX_WAIT_SECONDS% ( + echo ERROR: Could not find OSCAR Java PID after waiting. + exit /b 1 + ) + + timeout /t 2 /nobreak >nul + set /a WAITED+=2 + goto :wait_for_jvm +) else ( + >"%OUT_DIR%\launch.stdout.log" type nul + >"%OUT_DIR%\launch.stderr.log" type nul +) + +:have_jvm +echo %JVM_PID%>"%OUT_DIR%\jvm-pid.txt" + +powershell -NoProfile -Command "$p = Get-CimInstance Win32_Process -Filter 'ProcessId=%JVM_PID%'; if($p){ 'Timestamp: ' + (Get-Date -Format o); 'Launcher PID: %LAUNCH_PID%'; 'JVM PID: %JVM_PID%'; ''; 'Command line:'; $p.CommandLine }" > "%OUT_DIR%\process-info.txt" + +if defined JCMD_CMD ( + "%JCMD_CMD%" %JVM_PID% JFR.start name=%JFR_NAME% settings=profile disk=true maxage=%JFR_MAX_AGE% maxsize=%JFR_MAX_SIZE% filename="%OUT_DIR%\%JFR_NAME%.jfr" > "%OUT_DIR%\jfr-start.txt" 2>&1 + "%JCMD_CMD%" %JVM_PID% VM.native_memory baseline > "%OUT_DIR%\nmt-baseline.txt" 2>&1 +) else ( + echo jcmd not available; skipping JFR start and NMT baseline. > "%OUT_DIR%\jcmd-warning.txt" +) + +:loop +call :snapshot +call :process_alive %JVM_PID% JVM_ALIVE +if not defined JVM_ALIVE goto :eof_ok +set "JVM_ALIVE=" +timeout /t %INTERVAL% /nobreak >nul +goto :loop + +:snapshot +call :timestamp SNAP_STAMP +set "SNAP=%OUT_DIR%\%SNAP_STAMP%" +if not exist "%SNAP%" mkdir "%SNAP%" + +echo Collecting snapshot at %SNAP_STAMP% for PID %JVM_PID% +powershell -NoProfile -Command "$p = Get-CimInstance Win32_Process -Filter 'ProcessId=%JVM_PID%'; if($p){$p | Select-Object ProcessId,ParentProcessId,Name,CommandLine | Format-List | Out-String}" > "%SNAP%\process.txt" 2>&1 +powershell -NoProfile -Command "$p=Get-Process -Id %JVM_PID% -ErrorAction SilentlyContinue; if($p){$p | Select-Object Id,ProcessName,Threads,VirtualMemorySize64,WorkingSet64,PrivateMemorySize64,CPU,StartTime | Format-List | Out-String}" > "%SNAP%\powershell-process.txt" 2>&1 +powershell -NoProfile -Command "Get-CimInstance Win32_OperatingSystem | Select-Object TotalVisibleMemorySize,FreePhysicalMemory,TotalVirtualMemorySize,FreeVirtualMemory | Format-List | Out-String" > "%SNAP%\memory.txt" 2>&1 +powershell -NoProfile -Command "Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\%% Usage' | Out-String" > "%SNAP%\counters.txt" 2>&1 +if defined JCMD_CMD ( + "%JCMD_CMD%" %JVM_PID% VM.native_memory summary > "%SNAP%\nmt-summary.txt" 2>&1 + "%JCMD_CMD%" %JVM_PID% GC.heap_info > "%SNAP%\gc-heap-info.txt" 2>&1 + "%JCMD_CMD%" %JVM_PID% Thread.print > "%SNAP%\thread-print.txt" 2>&1 + "%JCMD_CMD%" %JVM_PID% JFR.check > "%SNAP%\jfr-check.txt" 2>&1 +) + +docker ps --filter name=%CONTAINER_NAME% > "%SNAP%\docker-ps.txt" 2>&1 +docker logs --tail 100 %CONTAINER_NAME% > "%SNAP%\docker-logs-tail.txt" 2>&1 +call :db_snapshot "%SNAP%" +exit /b 0 + +:db_snapshot +set "SNAP=%~1" +set "DB_ERR=%SNAP%\db-error.txt" +set "FAILED=0" +set "MAX_CONN=" +set "SUPER_RESERVED=" +set "TOTAL_SESSIONS=" +set "ACTIVE_COUNT=0" +set "IDLE_COUNT=0" +set "IDLE_TX_COUNT=0" + +for /f %%T in ('powershell -NoProfile -Command "Get-Date -Format o"') do set "DB_TS=%%T" + +docker ps --format {{.Names}} | findstr /i /x "%CONTAINER_NAME%" >nul 2>&1 +if errorlevel 1 ( + >"%DB_ERR%" echo Container %CONTAINER_NAME% not running + >>"%OUT_DIR%\db-connection-trend.csv" echo %DB_TS%,,,,,,,1 + exit /b 0 +) + +docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "show max_connections;" > "%SNAP%\db-max-connections.txt" 2> "%DB_ERR%" +if errorlevel 1 set "FAILED=1" +docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "show superuser_reserved_connections;" > "%SNAP%\db-superuser-reserved-connections.txt" 2>> "%DB_ERR%" +docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select count(*) from pg_stat_activity;" > "%SNAP%\db-total-sessions.txt" 2>> "%DB_ERR%" +docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select coalesce(state,''), count(*) from pg_stat_activity group by state order by count(*) desc;" > "%SNAP%\db-by-state.txt" 2>> "%DB_ERR%" +docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select coalesce(application_name,''), coalesce(usename,''), coalesce(client_addr::text,''), coalesce(state,''), count(*) from pg_stat_activity group by application_name, usename, client_addr, state order by count(*) desc limit 20;" > "%SNAP%\db-by-app.txt" 2>> "%DB_ERR%" +docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select pid, usename, application_name, client_addr, state, backend_start, xact_start, query_start, wait_event_type, wait_event, left(query,120) from pg_stat_activity order by backend_start;" > "%SNAP%\db-activity-detail.txt" 2>> "%DB_ERR%" + +for /f "usebackq" %%A in ("%SNAP%\db-max-connections.txt") do set "MAX_CONN=%%A" +for /f "usebackq" %%A in ("%SNAP%\db-superuser-reserved-connections.txt") do set "SUPER_RESERVED=%%A" +for /f "usebackq" %%A in ("%SNAP%\db-total-sessions.txt") do set "TOTAL_SESSIONS=%%A" +for /f "usebackq tokens=1,2 delims=|" %%A in ("%SNAP%\db-by-state.txt") do ( + if /i "%%A"=="active" set "ACTIVE_COUNT=%%B" + if /i "%%A"=="idle" set "IDLE_COUNT=%%B" + if /i "%%A"=="idle in transaction" set "IDLE_TX_COUNT=%%B" +) +>>"%OUT_DIR%\db-connection-trend.csv" echo %DB_TS%,%TOTAL_SESSIONS%,%ACTIVE_COUNT%,%IDLE_COUNT%,%IDLE_TX_COUNT%,%MAX_CONN%,%SUPER_RESERVED%,%FAILED% +exit /b 0 + +:stop_mode +call :find_existing_oscar +if defined OSCAR_PID taskkill /PID %OSCAR_PID% /T /F >nul 2>&1 +for /f %%C in ('docker ps --filter name=oscar-postgis-container --format {{.Names}}') do docker stop %%C >nul 2>&1 +echo OSCAR stop requested. +exit /b 0 + +:check_dependencies +where powershell >nul 2>nul +if errorlevel 1 ( + echo Error: PowerShell is required but was not found on PATH. + exit /b 1 +) + +where java >nul 2>nul +if errorlevel 1 ( + echo Error: java was not found on PATH. Install OpenJDK 21 or newer. + exit /b 1 +) + +where docker >nul 2>nul +if errorlevel 1 ( + echo Error: docker was not found on PATH. Install Docker Desktop and make sure it is running. + exit /b 1 +) + +set "JAVA_HOME_LINE=" +for /f "delims=" %%A in ('java -XshowSettings:properties -version 2^>^&1 ^| findstr /c:"java.home ="') do ( + set "JAVA_HOME_LINE=%%A" + goto :monitor_java_home_line +) + +:monitor_java_home_line +if not defined JAVA_HOME_LINE exit /b 0 +for /f "tokens=1,* delims==" %%A in ("%JAVA_HOME_LINE%") do set "JAVA_HOME_DETECTED=%%B" +for /f "tokens=* delims= " %%A in ("%JAVA_HOME_DETECTED%") do set "JAVA_HOME_DETECTED=%%A" +if exist "%JAVA_HOME_DETECTED%\bin\jcmd.exe" set "JCMD_CMD=%JAVA_HOME_DETECTED%\bin\jcmd.exe" +exit /b 0 + +:find_existing_oscar +set "OSCAR_PID=" +for /f %%P in ('powershell -NoProfile -Command "$p = Get-CimInstance Win32_Process ^| Where-Object { $_.Name -match ''^java(\.exe)?$'' -and $_.CommandLine -like ''*com.botts.impl.security.SensorHubWrapper*'' } ^| Select-Object -ExpandProperty ProcessId -First 1; if ($p) { Write-Output $p }"') do set "OSCAR_PID=%%P" +exit /b 0 + +:process_alive +set "%~2=" +powershell -NoProfile -Command "if (Get-Process -Id %~1 -ErrorAction SilentlyContinue) { exit 0 } else { exit 1 }" >nul 2>nul +if not errorlevel 1 set "%~2=1" +exit /b 0 + +:timestamp +for /f %%A in ('powershell -NoProfile -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "%~1=%%A" +exit /b 0 + +:load_env +for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( + set "ENV_NAME=%%A" + set "ENV_VALUE=%%B" + call :set_env_var +) +exit /b 0 + +:set_env_var +if not defined ENV_NAME exit /b 0 +if "%ENV_NAME:~0,1%"=="#" exit /b 0 +if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" +set "%ENV_NAME%=%ENV_VALUE%" +exit /b 0 + +:eof_ok +exit /b 0 diff --git a/dist/release/monitor-oscar.sh b/dist/release/monitor-oscar.sh new file mode 100644 index 0000000..0cb78b6 --- /dev/null +++ b/dist/release/monitor-oscar.sh @@ -0,0 +1,342 @@ +#!/bin/bash +set -euo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_DIR="${PROJECT_DIR:-$SCRIPT_DIR}" +LAUNCH_CMD="${LAUNCH_CMD:-$PROJECT_DIR/launch-all.sh}" +MATCH_EXPR="${MATCH_EXPR:-com.botts.impl.security.SensorHubWrapper}" +INTERVAL="${INTERVAL:-60}" +MAX_WAIT_SECONDS="${MAX_WAIT_SECONDS:-300}" +OUT_DIR="${OUT_DIR:-$PROJECT_DIR/oscar-monitor-$(date +%Y%m%d-%H%M%S)}" +JFR_NAME="${JFR_NAME:-oscar}" +JFR_MAX_AGE="${JFR_MAX_AGE:-4h}" +JFR_MAX_SIZE="${JFR_MAX_SIZE:-1g}" +ENV_FILE="${ENV_FILE:-$PROJECT_DIR/.env}" +ATTACH_TO_EXISTING="${ATTACH_TO_EXISTING:-0}" +FORCE_RESTART="${FORCE_RESTART:-0}" + +mkdir -p "$OUT_DIR" + +CONTAINER_NAME="oscar-postgis-container" +DB_NAME="gis" +DB_USER="postgres" +DB_PASSWORD="postgres" +if [ -f "$ENV_FILE" ]; then + set -a + . "$ENV_FILE" + set +a + CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" + DB_NAME="${DB_NAME:-gis}" + DB_USER="${DB_USER:-postgres}" + DB_PASSWORD="${DB_PASSWORD:-postgres}" +fi + +DB_CSV="$OUT_DIR/db-connection-trend.csv" +echo 'timestamp,total_sessions,active,idle,idle_in_transaction,max_connections,superuser_reserved_connections,failed_psql' > "$DB_CSV" + +log() { + printf '%s %s\n' "$(date -Is)" "$*" +} + +require_cmd() { + local cmd="$1" + if ! command -v "$cmd" >/dev/null 2>&1; then + echo "Error: required command not found: $cmd" + exit 1 + fi +} + +get_java_major() { + java -version 2>&1 | awk -F'"' '/version/ { split($2, v, "."); print v[1]; exit }' +} + +check_dependencies() { + require_cmd bash + require_cmd java + require_cmd docker + require_cmd pgrep + + local java_major + java_major="$(get_java_major || true)" + if [[ -z "$java_major" || ! "$java_major" =~ ^[0-9]+$ || "$java_major" -lt 21 ]]; then + echo "Error: Java 21 or newer is required to run OSCAR monitoring." + exit 1 + fi + + if ! command -v jcmd >/dev/null 2>&1; then + log "Warning: jcmd not found. JFR/NMT snapshots will be skipped." + fi +} + +find_existing_oscar_pid() { + pgrep -f "$MATCH_EXPR" | head -n 1 || true +} + +find_all_existing_oscar_pids() { + pgrep -f "$MATCH_EXPR" || true +} + +stop_existing_oscar() { + local pids="$1" + if [ -z "$pids" ]; then + return 0 + fi + + log "Stopping existing OSCAR instance(s): $pids" + kill $pids 2>/dev/null || true + + local waited=0 + while [ "$waited" -lt 15 ]; do + sleep 1 + waited=$((waited + 1)) + if [ -z "$(find_all_existing_oscar_pids)" ]; then + return 0 + fi + done + + log "Force killing existing OSCAR instance(s): $pids" + kill -9 $pids 2>/dev/null || true + sleep 1 + + if [ -n "$(find_all_existing_oscar_pids)" ]; then + echo "Error: unable to stop existing OSCAR instance(s)." + exit 1 + fi +} + +run_db_query() { + local sql="$1" + docker exec -e PGPASSWORD="$DB_PASSWORD" "$CONTAINER_NAME" \ + psql -U "$DB_USER" -d "$DB_NAME" -At -c "$sql" +} + +collect_db_snapshot() { + local d="$1" + local ts failed total active idle idle_tx max_conn super_reserved + ts="$(date -Is)" + failed=0 + + if ! docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then + echo "Container ${CONTAINER_NAME} not running" > "$d/db-error.txt" + echo "$ts,,,,,,,1" >> "$DB_CSV" + return 0 + fi + + if run_db_query "show max_connections;" > "$d/db-max-connections.txt" 2> "$d/db-error.txt"; then + run_db_query "show superuser_reserved_connections;" > "$d/db-superuser-reserved-connections.txt" 2>> "$d/db-error.txt" || failed=1 + run_db_query "select count(*) from pg_stat_activity;" > "$d/db-total-sessions.txt" 2>> "$d/db-error.txt" || failed=1 + run_db_query "select coalesce(state,''), count(*) from pg_stat_activity group by state order by count(*) desc;" > "$d/db-by-state.txt" 2>> "$d/db-error.txt" || failed=1 + run_db_query "select coalesce(application_name,''), coalesce(usename,''), coalesce(client_addr::text,''), coalesce(state,''), count(*) from pg_stat_activity group by application_name, usename, client_addr, state order by count(*) desc limit 20;" > "$d/db-by-app.txt" 2>> "$d/db-error.txt" || failed=1 + run_db_query "select pid, usename, application_name, client_addr, state, backend_start, xact_start, query_start, wait_event_type, wait_event, left(query,120) from pg_stat_activity order by backend_start;" > "$d/db-activity-detail.txt" 2>> "$d/db-error.txt" || failed=1 + else + failed=1 + fi + + max_conn="" + super_reserved="" + total="" + active="" + idle="" + idle_tx="" + + [ -f "$d/db-max-connections.txt" ] && max_conn="$(tr -d '[:space:]' < "$d/db-max-connections.txt" | tail -n 1)" + [ -f "$d/db-superuser-reserved-connections.txt" ] && super_reserved="$(tr -d '[:space:]' < "$d/db-superuser-reserved-connections.txt" | tail -n 1)" + [ -f "$d/db-total-sessions.txt" ] && total="$(tr -d '[:space:]' < "$d/db-total-sessions.txt" | tail -n 1)" + if [ -f "$d/db-by-state.txt" ]; then + active="$(awk -F'|' '$1=="active" {gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}' "$d/db-by-state.txt" | tail -n 1)" + idle="$(awk -F'|' '$1=="idle" {gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}' "$d/db-by-state.txt" | tail -n 1)" + idle_tx="$(awk -F'|' '$1=="idle in transaction" {gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}' "$d/db-by-state.txt" | tail -n 1)" + fi + echo "$ts,${total:-},${active:-0},${idle:-0},${idle_tx:-0},${max_conn:-},${super_reserved:-},$failed" >> "$DB_CSV" +} + +dump_once() { + if [ -z "${PID:-}" ] || ! kill -0 "$PID" 2>/dev/null; then + return 0 + fi + + local ts d + ts="$(date +%Y%m%d-%H%M%S)" + d="$OUT_DIR/$ts" + mkdir -p "$d" + + log "Collecting snapshot at $ts for PID $PID" + + ps -p "$PID" -o pid,ppid,user,%cpu,%mem,vsz,rss,etimes,cmd > "$d/ps.txt" 2>&1 || true + [ -r "/proc/$PID/status" ] && cat "/proc/$PID/status" > "$d/proc-status.txt" 2>&1 || true + [ -r "/proc/$PID/smaps_rollup" ] && cat "/proc/$PID/smaps_rollup" > "$d/smaps_rollup.txt" 2>&1 || true + + command -v pmap >/dev/null 2>&1 && pmap -x "$PID" > "$d/pmap-x.txt" 2>&1 || true + command -v free >/dev/null 2>&1 && free -h > "$d/free.txt" 2>&1 || true + [ -r /proc/meminfo ] && cat /proc/meminfo > "$d/meminfo.txt" 2>&1 || true + [ -r /proc/swaps ] && cat /proc/swaps > "$d/swaps.txt" 2>&1 || true + vmstat 1 5 > "$d/vmstat.txt" 2>&1 || true + + if command -v jcmd >/dev/null 2>&1; then + jcmd "$PID" VM.native_memory summary > "$d/nmt-summary.txt" 2>&1 || true + jcmd "$PID" GC.heap_info > "$d/gc-heap-info.txt" 2>&1 || true + jcmd "$PID" Thread.print > "$d/thread-print.txt" 2>&1 || true + jcmd "$PID" JFR.check > "$d/jfr-check.txt" 2>&1 || true + fi + + docker ps --filter "name=$CONTAINER_NAME" > "$d/docker-ps.txt" 2>&1 || true + docker logs --tail 100 "$CONTAINER_NAME" > "$d/docker-logs-tail.txt" 2>&1 || true + collect_db_snapshot "$d" +} + +final_dump() { + if [ -n "${PID:-}" ] && kill -0 "$PID" 2>/dev/null; then + dump_once + if command -v jcmd >/dev/null 2>&1; then + jcmd "$PID" JFR.dump name="$JFR_NAME" filename="$OUT_DIR/${JFR_NAME}-final.jfr" \ + > "$OUT_DIR/jfr-dump-final.txt" 2>&1 || true + fi + fi +} + +stop_stack() { + if [ "${STOPPING:-0}" -eq 1 ]; then + return 0 + fi + STOPPING=1 + + log "Stopping OSCAR stack..." + final_dump + + if [ -n "${PID:-}" ] && kill -0 "$PID" 2>/dev/null; then + log "Stopping JVM PID $PID" + kill "$PID" 2>/dev/null || true + for _ in 1 2 3 4 5 6 7 8 9 10; do + if ! kill -0 "$PID" 2>/dev/null; then + break + fi + sleep 1 + done + if kill -0 "$PID" 2>/dev/null; then + log "Force killing JVM PID $PID" + kill -9 "$PID" 2>/dev/null || true + fi + fi + + if [ -n "${LAUNCH_PID:-}" ] && kill -0 "$LAUNCH_PID" 2>/dev/null; then + log "Stopping launcher PID $LAUNCH_PID" + kill "$LAUNCH_PID" 2>/dev/null || true + fi + + if docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then + log "Stopping container ${CONTAINER_NAME}" + docker stop "$CONTAINER_NAME" > "$OUT_DIR/docker-stop.txt" 2>&1 || true + fi +} + +on_signal() { + log "Received stop signal" + stop_stack + exit 0 +} + +on_exit() { + final_dump +} + +trap on_signal INT TERM +trap on_exit EXIT + +check_dependencies + +log "Monitor output: $OUT_DIR" +log "Launch command: $LAUNCH_CMD" +log "JVM match: $MATCH_EXPR" +log "Container name: $CONTAINER_NAME" +log "Database: $DB_NAME user=$DB_USER" + +if [ ! -x "$LAUNCH_CMD" ] && [ "$ATTACH_TO_EXISTING" != "1" ]; then + echo "Error: launch command is not executable: $LAUNCH_CMD" + exit 1 +fi + +LAUNCH_PID="" +PID="" +STOPPING=0 +USE_EXISTING=0 + +existing_pids="$(find_all_existing_oscar_pids)" +if [ -n "$existing_pids" ]; then + if [ "$ATTACH_TO_EXISTING" = "1" ]; then + PID="$(printf '%s\n' "$existing_pids" | head -n 1)" + USE_EXISTING=1 + log "Attaching monitor to existing OSCAR PID $PID" + elif [ "$FORCE_RESTART" = "1" ]; then + log "Existing OSCAR instance found: $existing_pids" + stop_existing_oscar "$existing_pids" + else + echo "OSCAR is already running with PID(s): $existing_pids" + echo "Set ATTACH_TO_EXISTING=1 to monitor the running instance, or FORCE_RESTART=1 to replace it." + exit 1 + fi +fi + +if [ "$USE_EXISTING" = "0" ]; then + log "Starting OSCAR..." + "$LAUNCH_CMD" > "$OUT_DIR/launch.stdout.log" 2> "$OUT_DIR/launch.stderr.log" & + LAUNCH_PID=$! + echo "$LAUNCH_PID" > "$OUT_DIR/launcher-pid.txt" + + log "Waiting for JVM to appear..." + waited=0 + while true; do + PID="$(find_existing_oscar_pid)" + if [ -n "$PID" ]; then + break + fi + if [ "$waited" -ge "$MAX_WAIT_SECONDS" ]; then + log "Timed out waiting for JVM after ${MAX_WAIT_SECONDS}s" + exit 1 + fi + sleep 2 + waited=$((waited + 2)) + done +else + : > "$OUT_DIR/launch.stdout.log" + : > "$OUT_DIR/launch.stderr.log" +fi + +log "Found JVM PID: $PID" +echo "$PID" > "$OUT_DIR/jvm-pid.txt" + +{ + echo "Timestamp: $(date -Is)" + echo "Launcher PID: ${LAUNCH_PID:-}" + echo "JVM PID: $PID" + echo + echo "Command line:" + tr '\0' ' ' < "/proc/$PID/cmdline" + echo +} > "$OUT_DIR/process-info.txt" + +if command -v jcmd >/dev/null 2>&1; then + log "Starting JFR on PID $PID" + jcmd "$PID" JFR.start \ + name="$JFR_NAME" \ + settings=profile \ + disk=true \ + maxage="$JFR_MAX_AGE" \ + maxsize="$JFR_MAX_SIZE" \ + filename="$OUT_DIR/${JFR_NAME}.jfr" \ + > "$OUT_DIR/jfr-start.txt" 2>&1 || true + + jcmd "$PID" VM.native_memory baseline \ + > "$OUT_DIR/nmt-baseline.txt" 2>&1 || true +fi + +dump_once + +while kill -0 "$PID" 2>/dev/null; do + sleep "$INTERVAL" + dump_once +done + +log "JVM exited." +if [ -n "$LAUNCH_PID" ]; then + wait "$LAUNCH_PID" || true +fi diff --git a/dist/release/monitor-oscar_old.bat b/dist/release/monitor-oscar_old.bat new file mode 100644 index 0000000..55eb8af --- /dev/null +++ b/dist/release/monitor-oscar_old.bat @@ -0,0 +1,129 @@ +@echo off +setlocal EnableExtensions EnableDelayedExpansion + +if "%~1"=="stop" goto :stop_mode + +set "SCRIPT_DIR=%~dp0" +for %%I in ("%SCRIPT_DIR%.") do set "PROJECT_DIR=%%~fI" +set "OUT_DIR=%PROJECT_DIR%\oscar-monitor-%DATE:~10,4%%DATE:~4,2%%DATE:~7,2%-%TIME:~0,2%%TIME:~3,2%%TIME:~6,2%" +set "OUT_DIR=%OUT_DIR: =0%" +set "ENV_FILE=%PROJECT_DIR%\.env" +set "CONTAINER_NAME=oscar-postgis-container" +set "DB_NAME=gis" +set "DB_USER=postgres" +set "DB_PASSWORD=postgres" +set "MATCH_EXPR=com.botts.impl.security.SensorHubWrapper" +set "INTERVAL=60" +set "JFR_NAME=oscar" +set "JFR_MAX_AGE=4h" +set "JFR_MAX_SIZE=1g" +set "LAUNCH_CMD=%PROJECT_DIR%\launch-all.bat" + +if exist "%ENV_FILE%" ( + for /f "usebackq tokens=1,* delims==" %%A in ("%ENV_FILE%") do ( + if not "%%A"=="" if /i not "%%A:~0,1"=="#" set "%%A=%%B" + ) +) + +if not exist "%OUT_DIR%" mkdir "%OUT_DIR%" +echo timestamp,total_sessions,active,idle,idle_in_transaction,max_connections,superuser_reserved_connections,failed_psql>"%OUT_DIR%\db-connection-trend.csv" + +echo %DATE% %TIME% Monitor output: %OUT_DIR% +echo %DATE% %TIME% Launch command: %LAUNCH_CMD% + +where jcmd >nul 2>nul +if errorlevel 1 echo Warning: jcmd not found. JFR and NMT snapshots will be limited. + +start "OSCAR_LAUNCH" /b cmd /c ""%LAUNCH_CMD%" 1>"%OUT_DIR%\launch.stdout.log" 2>"%OUT_DIR%\launch.stderr.log"" + +:wait_for_jvm +timeout /t 2 /nobreak >nul +for /f "tokens=2 delims=," %%P in ('wmic process where "name='java.exe' and commandline like '%%SensorHubWrapper%%'" get processid^,commandline /format:csv ^| findstr /i SensorHubWrapper') do ( + set "JVM_PID=%%P" + goto :have_jvm +) +goto :wait_for_jvm + +:have_jvm +echo %JVM_PID%>"%OUT_DIR%\jvm-pid.txt" +if exist "%PROJECT_DIR%\monitor.pid" del "%PROJECT_DIR%\monitor.pid" +echo %PROCESS_ID%>"%PROJECT_DIR%\monitor.pid" + +jcmd %JVM_PID% JFR.start name=%JFR_NAME% settings=profile disk=true maxage=%JFR_MAX_AGE% maxsize=%JFR_MAX_SIZE% filename="%OUT_DIR%\%JFR_NAME%.jfr" >"%OUT_DIR%\jfr-start.txt" 2>&1 +jcmd %JVM_PID% VM.native_memory baseline >"%OUT_DIR%\nmt-baseline.txt" 2>&1 + +:loop +call :snapshot +for /f "tokens=2 delims=," %%P in ('wmic process where "processid=%JVM_PID%" get processid /format:csv ^| findstr /r ",[0-9][0-9]*$"') do set "ALIVE=%%P" +if not defined ALIVE goto :eof_ok +set "ALIVE=" +timeout /t %INTERVAL% /nobreak >nul +goto :loop + +:snapshot +set "STAMP=%DATE:~10,4%%DATE:~4,2%%DATE:~7,2%-%TIME:~0,2%%TIME:~3,2%%TIME:~6,2%" +set "STAMP=%STAMP: =0%" +set "SNAP=%OUT_DIR%\%STAMP%" +if not exist "%SNAP%" mkdir "%SNAP%" + +echo Collecting snapshot at %STAMP% for PID %JVM_PID% +wmic process where processid=%JVM_PID% get Name,ParentProcessId,ProcessId,ThreadCount,WorkingSetSize,VirtualSize /format:list >"%SNAP%\wmic-process.txt" 2>&1 +powershell -NoProfile -Command "$p=Get-Process -Id %JVM_PID% -ErrorAction SilentlyContinue; if($p){$p|Select-Object Id,ProcessName,Threads,VirtualMemorySize64,WorkingSet64,PrivateMemorySize64,CPU,StartTime|Format-List|Out-String}" >"%SNAP%\powershell-process.txt" 2>&1 +powershell -NoProfile -Command "Get-CimInstance Win32_OperatingSystem | Select-Object TotalVisibleMemorySize,FreePhysicalMemory,TotalVirtualMemorySize,FreeVirtualMemory | Format-List | Out-String" >"%SNAP%\memory.txt" 2>&1 +powershell -NoProfile -Command "Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\%% Usage' | Out-String" >"%SNAP%\counters.txt" 2>&1 +jcmd %JVM_PID% VM.native_memory summary >"%SNAP%\nmt-summary.txt" 2>&1 +jcmd %JVM_PID% GC.heap_info >"%SNAP%\gc-heap-info.txt" 2>&1 +jcmd %JVM_PID% Thread.print >"%SNAP%\thread-print.txt" 2>&1 +jcmd %JVM_PID% JFR.check >"%SNAP%\jfr-check.txt" 2>&1 + +docker ps --filter name=%CONTAINER_NAME% >"%SNAP%\docker-ps.txt" 2>&1 +docker logs --tail 100 %CONTAINER_NAME% >"%SNAP%\docker-logs-tail.txt" 2>&1 +call :db_snapshot "%SNAP%" +exit /b 0 + +:db_snapshot +set "SNAP=%~1" +set "DB_ERR=%SNAP%\db-error.txt" +set "FAILED=0" +set "MAX_CONN=" +set "SUPER_RESERVED=" +set "TOTAL_SESSIONS=" +set "ACTIVE_COUNT=0" +set "IDLE_COUNT=0" +set "IDLE_TX_COUNT=0" + +docker ps --format {{.Names}} | findstr /i /x "%CONTAINER_NAME%" >nul 2>&1 +if errorlevel 1 ( + >"%DB_ERR%" echo Container %CONTAINER_NAME% not running + >>"%OUT_DIR%\db-connection-trend.csv" echo %DATE%T%TIME%,,,,,,,1 + exit /b 0 +) + +docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "show max_connections;" >"%SNAP%\db-max-connections.txt" 2>"%DB_ERR%" +if errorlevel 1 set "FAILED=1" +docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "show superuser_reserved_connections;" >"%SNAP%\db-superuser-reserved-connections.txt" 2>>"%DB_ERR%" +docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select count(*) from pg_stat_activity;" >"%SNAP%\db-total-sessions.txt" 2>>"%DB_ERR%" +docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select coalesce(state,''), count(*) from pg_stat_activity group by state order by count(*) desc;" >"%SNAP%\db-by-state.txt" 2>>"%DB_ERR%" +docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select coalesce(application_name,''), coalesce(usename,''), coalesce(client_addr::text,''), coalesce(state,''), count(*) from pg_stat_activity group by application_name, usename, client_addr, state order by count(*) desc limit 20;" >"%SNAP%\db-by-app.txt" 2>>"%DB_ERR%" + +for /f %%A in (%SNAP%\db-max-connections.txt) do set "MAX_CONN=%%A" +for /f %%A in (%SNAP%\db-superuser-reserved-connections.txt) do set "SUPER_RESERVED=%%A" +for /f %%A in (%SNAP%\db-total-sessions.txt) do set "TOTAL_SESSIONS=%%A" +for /f "tokens=1,2 delims=|" %%A in (%SNAP%\db-by-state.txt) do ( + if /i "%%A"=="active" set "ACTIVE_COUNT=%%B" + if /i "%%A"=="idle" set "IDLE_COUNT=%%B" + if /i "%%A"=="idle in transaction" set "IDLE_TX_COUNT=%%B" +) +>>"%OUT_DIR%\db-connection-trend.csv" echo %DATE%T%TIME%,%TOTAL_SESSIONS%,%ACTIVE_COUNT%,%IDLE_COUNT%,%IDLE_TX_COUNT%,%MAX_CONN%,%SUPER_RESERVED%,%FAILED% +exit /b 0 + +:stop_mode +for /f %%P in (%~dp0monitor.pid) do set "MONPID=%%P" +if defined MONPID taskkill /PID %MONPID% /T /F >nul 2>&1 +for /f "tokens=2 delims=," %%P in ('wmic process where "name='java.exe' and commandline like '%%SensorHubWrapper%%'" get processid^,commandline /format:csv ^| findstr /i SensorHubWrapper') do taskkill /PID %%P /T /F >nul 2>&1 +for /f %%C in ('docker ps --filter name=oscar-postgis-container --format {{.Names}}') do docker stop %%C >nul 2>&1 +echo OSCAR stack stop requested. +exit /b 0 + +:eof_ok +exit /b 0 diff --git a/dist/release/monitor-oscar_old.sh b/dist/release/monitor-oscar_old.sh new file mode 100644 index 0000000..29fe1b1 --- /dev/null +++ b/dist/release/monitor-oscar_old.sh @@ -0,0 +1,254 @@ +#!/bin/bash +set -euo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_DIR="${PROJECT_DIR:-$SCRIPT_DIR}" +LAUNCH_CMD="${LAUNCH_CMD:-$PROJECT_DIR/launch-all.sh}" +MATCH_EXPR="${MATCH_EXPR:-com.botts.impl.security.SensorHubWrapper}" +INTERVAL="${INTERVAL:-60}" +OUT_DIR="${OUT_DIR:-$PROJECT_DIR/oscar-monitor-$(date +%Y%m%d-%H%M%S)}" +JFR_NAME="${JFR_NAME:-oscar}" +JFR_MAX_AGE="${JFR_MAX_AGE:-4h}" +JFR_MAX_SIZE="${JFR_MAX_SIZE:-1g}" +ENV_FILE="${ENV_FILE:-$PROJECT_DIR/.env}" + +mkdir -p "$OUT_DIR" + +CONTAINER_NAME="oscar-postgis-container" +DB_NAME="gis" +DB_USER="postgres" +DB_PASSWORD="postgres" +if [ -f "$ENV_FILE" ]; then + set -a + . "$ENV_FILE" + set +a + CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" + DB_NAME="${DB_NAME:-gis}" + DB_USER="${DB_USER:-postgres}" + DB_PASSWORD="${DB_PASSWORD:-postgres}" +fi + +DB_CSV="$OUT_DIR/db-connection-trend.csv" +echo 'timestamp,total_sessions,active,idle,idle_in_transaction,max_connections,superuser_reserved_connections,failed_psql' > "$DB_CSV" + +log() { + printf '%s %s\n' "$(date -Is)" "$*" +} + +log "Monitor output: $OUT_DIR" +log "Launch command: $LAUNCH_CMD" +log "JVM match: $MATCH_EXPR" +log "Container name: $CONTAINER_NAME" +log "Database: $DB_NAME user=$DB_USER" + +if [ ! -x "$LAUNCH_CMD" ]; then + echo "Error: launch command is not executable: $LAUNCH_CMD" + exit 1 +fi + +if ! command -v jcmd >/dev/null 2>&1; then + log "Warning: jcmd not found. JFR/NMT snapshots will be skipped." +fi + +LAUNCH_PID="" +PID="" +STOPPING=0 + +run_db_query() { + local sql="$1" + docker exec -e PGPASSWORD="$DB_PASSWORD" "$CONTAINER_NAME" \ + psql -U "$DB_USER" -d "$DB_NAME" -At -c "$sql" +} + +collect_db_snapshot() { + local d="$1" + local ts failed total active idle idle_tx max_conn super_reserved + ts="$(date -Is)" + failed=0 + + if ! command -v docker >/dev/null 2>&1 || ! docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then + echo "Container ${CONTAINER_NAME} not running" > "$d/db-error.txt" + echo "$ts,,,,,,,1" >> "$DB_CSV" + return 0 + fi + + if run_db_query "show max_connections;" > "$d/db-max-connections.txt" 2> "$d/db-error.txt"; then + run_db_query "show superuser_reserved_connections;" > "$d/db-superuser-reserved-connections.txt" 2>> "$d/db-error.txt" || failed=1 + run_db_query "select count(*) from pg_stat_activity;" > "$d/db-total-sessions.txt" 2>> "$d/db-error.txt" || failed=1 + run_db_query "select coalesce(state,''), count(*) from pg_stat_activity group by state order by count(*) desc;" > "$d/db-by-state.txt" 2>> "$d/db-error.txt" || failed=1 + run_db_query "select coalesce(application_name,''), coalesce(usename,''), coalesce(client_addr::text,''), coalesce(state,''), count(*) from pg_stat_activity group by application_name, usename, client_addr, state order by count(*) desc limit 20;" > "$d/db-by-app.txt" 2>> "$d/db-error.txt" || failed=1 + run_db_query "select pid, usename, application_name, client_addr, state, backend_start, xact_start, query_start, wait_event_type, wait_event, left(query,120) from pg_stat_activity order by backend_start;" > "$d/db-activity-detail.txt" 2>> "$d/db-error.txt" || failed=1 + else + failed=1 + fi + + max_conn="" + super_reserved="" + total="" + active="" + idle="" + idle_tx="" + + [ -f "$d/db-max-connections.txt" ] && max_conn="$(tr -d '[:space:]' < "$d/db-max-connections.txt" | tail -n 1)" + [ -f "$d/db-superuser-reserved-connections.txt" ] && super_reserved="$(tr -d '[:space:]' < "$d/db-superuser-reserved-connections.txt" | tail -n 1)" + [ -f "$d/db-total-sessions.txt" ] && total="$(tr -d '[:space:]' < "$d/db-total-sessions.txt" | tail -n 1)" + if [ -f "$d/db-by-state.txt" ]; then + active="$(awk -F'|' '$1=="active" {gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}' "$d/db-by-state.txt" | tail -n 1)" + idle="$(awk -F'|' '$1=="idle" {gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}' "$d/db-by-state.txt" | tail -n 1)" + idle_tx="$(awk -F'|' '$1=="idle in transaction" {gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}' "$d/db-by-state.txt" | tail -n 1)" + fi + echo "$ts,${total:-},${active:-0},${idle:-0},${idle_tx:-0},${max_conn:-},${super_reserved:-},$failed" >> "$DB_CSV" +} + +dump_once() { + if [ -z "${PID:-}" ] || ! kill -0 "$PID" 2>/dev/null; then + return 0 + fi + + local ts d + ts="$(date +%Y%m%d-%H%M%S)" + d="$OUT_DIR/$ts" + mkdir -p "$d" + + log "Collecting snapshot at $ts for PID $PID" + + ps -p "$PID" -o pid,ppid,user,%cpu,%mem,vsz,rss,etimes,cmd > "$d/ps.txt" 2>&1 || true + [ -r "/proc/$PID/status" ] && cat "/proc/$PID/status" > "$d/proc-status.txt" 2>&1 || true + [ -r "/proc/$PID/smaps_rollup" ] && cat "/proc/$PID/smaps_rollup" > "$d/smaps_rollup.txt" 2>&1 || true + + command -v pmap >/dev/null 2>&1 && pmap -x "$PID" > "$d/pmap-x.txt" 2>&1 || true + command -v free >/dev/null 2>&1 && free -h > "$d/free.txt" 2>&1 || true + [ -r /proc/meminfo ] && cat /proc/meminfo > "$d/meminfo.txt" 2>&1 || true + [ -r /proc/swaps ] && cat /proc/swaps > "$d/swaps.txt" 2>&1 || true + vmstat 1 5 > "$d/vmstat.txt" 2>&1 || true + + if command -v jcmd >/dev/null 2>&1; then + jcmd "$PID" VM.native_memory summary > "$d/nmt-summary.txt" 2>&1 || true + jcmd "$PID" GC.heap_info > "$d/gc-heap-info.txt" 2>&1 || true + jcmd "$PID" Thread.print > "$d/thread-print.txt" 2>&1 || true + jcmd "$PID" JFR.check > "$d/jfr-check.txt" 2>&1 || true + fi + + if command -v docker >/dev/null 2>&1; then + docker ps --filter "name=$CONTAINER_NAME" > "$d/docker-ps.txt" 2>&1 || true + docker logs --tail 100 "$CONTAINER_NAME" > "$d/docker-logs-tail.txt" 2>&1 || true + collect_db_snapshot "$d" + fi +} + +final_dump() { + if [ -n "${PID:-}" ] && kill -0 "$PID" 2>/dev/null; then + dump_once + if command -v jcmd >/dev/null 2>&1; then + jcmd "$PID" JFR.dump name="$JFR_NAME" filename="$OUT_DIR/${JFR_NAME}-final.jfr" \ + > "$OUT_DIR/jfr-dump-final.txt" 2>&1 || true + fi + fi +} + +stop_stack() { + if [ "$STOPPING" -eq 1 ]; then + return 0 + fi + STOPPING=1 + + log "Stopping OSCAR stack..." + final_dump + + if [ -n "${PID:-}" ] && kill -0 "$PID" 2>/dev/null; then + log "Stopping JVM PID $PID" + kill "$PID" 2>/dev/null || true + for _ in 1 2 3 4 5 6 7 8 9 10; do + if ! kill -0 "$PID" 2>/dev/null; then + break + fi + sleep 1 + done + if kill -0 "$PID" 2>/dev/null; then + log "Force killing JVM PID $PID" + kill -9 "$PID" 2>/dev/null || true + fi + fi + + if [ -n "${LAUNCH_PID:-}" ] && kill -0 "$LAUNCH_PID" 2>/dev/null; then + log "Stopping launcher PID $LAUNCH_PID" + kill "$LAUNCH_PID" 2>/dev/null || true + fi + + if command -v docker >/dev/null 2>&1; then + if docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then + log "Stopping container ${CONTAINER_NAME}" + docker stop "$CONTAINER_NAME" > "$OUT_DIR/docker-stop.txt" 2>&1 || true + fi + fi +} + +on_signal() { + log "Received stop signal" + stop_stack + exit 0 +} + +on_exit() { + final_dump +} + +trap on_signal INT TERM +trap on_exit EXIT + +log "Starting OSCAR..." +"$LAUNCH_CMD" > "$OUT_DIR/launch.stdout.log" 2> "$OUT_DIR/launch.stderr.log" & +LAUNCH_PID=$! +echo "$LAUNCH_PID" > "$OUT_DIR/launcher-pid.txt" + +log "Waiting for JVM to appear..." +while true; do + PID="$(pgrep -f "$MATCH_EXPR" | head -n 1 || true)" + if [ -n "$PID" ]; then + break + fi + if ! kill -0 "$LAUNCH_PID" 2>/dev/null; then + log "Launch process exited before JVM appeared." + wait "$LAUNCH_PID" || true + exit 1 + fi + sleep 2 +done + +log "Found JVM PID: $PID" +echo "$PID" > "$OUT_DIR/jvm-pid.txt" + +{ + echo "Timestamp: $(date -Is)" + echo "Launcher PID: $LAUNCH_PID" + echo "JVM PID: $PID" + echo + echo "Command line:" + tr '\0' ' ' < "/proc/$PID/cmdline" + echo +} > "$OUT_DIR/process-info.txt" + +if command -v jcmd >/dev/null 2>&1; then + log "Starting JFR on PID $PID" + jcmd "$PID" JFR.start \ + name="$JFR_NAME" \ + settings=profile \ + disk=true \ + maxage="$JFR_MAX_AGE" \ + maxsize="$JFR_MAX_SIZE" \ + filename="$OUT_DIR/${JFR_NAME}.jfr" \ + > "$OUT_DIR/jfr-start.txt" 2>&1 || true + + jcmd "$PID" VM.native_memory baseline \ + > "$OUT_DIR/nmt-baseline.txt" 2>&1 || true +fi + +dump_once + +while kill -0 "$PID" 2>/dev/null; do + sleep "$INTERVAL" + dump_once +done + +log "JVM exited." +wait "$LAUNCH_PID" || true diff --git a/dist/scripts/standard/launch.bat b/dist/scripts/standard/launch.bat old mode 100755 new mode 100644 index 4b50578..489dffa --- a/dist/scripts/standard/launch.bat +++ b/dist/scripts/standard/launch.bat @@ -1,41 +1,244 @@ @echo off -setlocal enabledelayedexpansion +setlocal EnableExtensions +set "SCRIPT_DIR=%~dp0" +set "MATCH_EXPR=com.botts.impl.security.SensorHubWrapper" +set "FORCE_RESTART=%FORCE_RESTART%" +if not defined FORCE_RESTART set "FORCE_RESTART=0" -REM Make sure all the necessary certificates are trusted by the system. -CALL %~dp0load_trusted_certs.bat +set "ENV_FILE=" +if exist "%SCRIPT_DIR%.env" ( + set "ENV_FILE=%SCRIPT_DIR%.env" +) else if exist "%SCRIPT_DIR%..\.env" ( + set "ENV_FILE=%SCRIPT_DIR%..\.env" +) + +if defined ENV_FILE call :load_env "%ENV_FILE%" + +call :check_java +if errorlevel 1 exit /b %ERRORLEVEL% + +call :check_existing_oscar +if errorlevel 1 exit /b %ERRORLEVEL% + +call :ensure_runtime_paths +if errorlevel 1 exit /b %ERRORLEVEL% + +if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" + +if /I "%SYSTEM_PROFILE%"=="RPI4" ( + set "JAVA_XMS=512m" + set "JAVA_XMX=1536m" + set "JAVACPP_MAX_BYTES_DEFAULT=512m" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=2g" +) else if /I "%SYSTEM_PROFILE%"=="8GB" ( + set "JAVA_XMS=1g" + set "JAVA_XMX=2g" + set "JAVACPP_MAX_BYTES_DEFAULT=1g" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=4g" +) else if /I "%SYSTEM_PROFILE%"=="16GB" ( + set "JAVA_XMS=1g" + set "JAVA_XMX=3g" + set "JAVACPP_MAX_BYTES_DEFAULT=2g" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=8g" +) else if /I "%SYSTEM_PROFILE%"=="32GB" ( + set "JAVA_XMS=2g" + set "JAVA_XMX=6g" + set "JAVACPP_MAX_BYTES_DEFAULT=4g" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=16g" +) else ( + echo Unknown profile '%SYSTEM_PROFILE%', using 8GB defaults. + set "JAVA_XMS=1g" + set "JAVA_XMX=2g" + set "JAVACPP_MAX_BYTES_DEFAULT=1g" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=4g" +) + +if not defined JAVACPP_MAX_BYTES set "JAVACPP_MAX_BYTES=%JAVACPP_MAX_BYTES_DEFAULT%" +if not defined JAVACPP_MAX_PHYSICAL_BYTES set "JAVACPP_MAX_PHYSICAL_BYTES=%JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT%" +if not defined JFR_FILENAME set "JFR_FILENAME=%SCRIPT_DIR%oscar.jfr" + +if not defined HOME if defined USERPROFILE set "HOME=%USERPROFILE%" + +set "PATH=%JAVA_HOME_DETECTED%\bin;%PATH%" +set "KEYSTORE=%SCRIPT_DIR%osh-keystore.p12" +set "KEYSTORE_TYPE=PKCS12" +if not defined KEYSTORE_PASSWORD set "KEYSTORE_PASSWORD=atakatak" + +set "TRUSTSTORE=%SCRIPT_DIR%truststore.jks" +set "TRUSTSTORE_TYPE=JKS" +if not defined TRUSTSTORE_PASSWORD set "TRUSTSTORE_PASSWORD=changeit" + +set "INITIAL_ADMIN_PASSWORD_FILE=%SCRIPT_DIR%.s" +if not exist "%INITIAL_ADMIN_PASSWORD_FILE%" if not defined INITIAL_ADMIN_PASSWORD set "INITIAL_ADMIN_PASSWORD=admin" + +echo Starting OSH Node with Profile: %SYSTEM_PROFILE% +echo Heap: %JAVA_XMS% / %JAVA_XMX% +echo JavaCPP maxBytes: %JAVACPP_MAX_BYTES% +echo JavaCPP maxPhysicalBytes: %JAVACPP_MAX_PHYSICAL_BYTES% +echo JFR file: %JFR_FILENAME% + +call "%SCRIPT_DIR%load_trusted_certs.bat" +if errorlevel 1 exit /b %ERRORLEVEL% + +call "%SCRIPT_DIR%set-initial-admin-password.bat" +if errorlevel 1 exit /b %ERRORLEVEL% + +java ^ + -Xms%JAVA_XMS% ^ + -Xmx%JAVA_XMX% ^ + -Xss256k ^ + -XX:ReservedCodeCacheSize=256m ^ + -XX:+UseG1GC ^ + -XX:+HeapDumpOnOutOfMemoryError ^ + -XX:+UnlockDiagnosticVMOptions ^ + -XX:NativeMemoryTracking=summary ^ + "-Dorg.bytedeco.javacpp.maxBytes=%JAVACPP_MAX_BYTES%" ^ + "-Dorg.bytedeco.javacpp.maxPhysicalBytes=%JAVACPP_MAX_PHYSICAL_BYTES%" ^ + -Dorg.bytedeco.javacpp.maxRetries=2 ^ + -Dorg.bytedeco.javacpp.mxbean=true ^ + "-Dlogback.configurationFile=%SCRIPT_DIR%logback.xml" ^ + -cp "%SCRIPT_DIR%lib\*" ^ + "-Djava.system.class.loader=org.sensorhub.utils.NativeClassLoader" ^ + "-Djavax.net.ssl.keyStore=%KEYSTORE%" ^ + "-Djavax.net.ssl.keyStorePassword=%KEYSTORE_PASSWORD%" ^ + "-Djavax.net.ssl.trustStore=%TRUSTSTORE%" ^ + "-Djavax.net.ssl.trustStorePassword=%TRUSTSTORE_PASSWORD%" ^ + "-Djava.library.path=%SCRIPT_DIR%nativelibs" ^ + com.botts.impl.security.SensorHubWrapper "%SCRIPT_DIR%config.json" "%SCRIPT_DIR%db" + +set "JAVA_EXIT_CODE=%ERRORLEVEL%" +endlocal & exit /b %JAVA_EXIT_CODE% + +:check_java +where java >nul 2>nul +if errorlevel 1 ( + echo Error: java was not found on PATH. Install OpenJDK 21 or newer. + exit /b 1 +) + +set "JAVA_HOME_LINE=" +for /f "delims=" %%A in ('java -XshowSettings:properties -version 2^>^&1 ^| findstr /c:"java.home ="') do ( + set "JAVA_HOME_LINE=%%A" + goto :check_java_home_line +) + +:check_java_home_line +if not defined JAVA_HOME_LINE ( + echo Error: could not determine java.home from the installed Java runtime. + exit /b 1 +) + +for /f "tokens=1,* delims==" %%A in ("%JAVA_HOME_LINE%") do set "JAVA_HOME_DETECTED=%%B" +for /f "tokens=* delims= " %%A in ("%JAVA_HOME_DETECTED%") do set "JAVA_HOME_DETECTED=%%A" -set KEYSTORE=.\osh-keystore.p12 -set KEYSTORE_TYPE=PKCS12 -set KEYSTORE_PASSWORD=atakatak +if not exist "%JAVA_HOME_DETECTED%\bin\java.exe" ( + echo Error: Java executable not found under "%JAVA_HOME_DETECTED%\bin\java.exe". + exit /b 1 +) -set TRUSTSTORE=.\truststore.jks -set TRUSTSTORE_TYPE=JKS -set TRUSTSTORE_PASSWORD=changeit +if not exist "%JAVA_HOME_DETECTED%\bin\keytool.exe" ( + echo Error: keytool.exe not found under "%JAVA_HOME_DETECTED%\bin\keytool.exe". + exit /b 1 +) -set INITIAL_ADMIN_PASSWORD_FILE=.\.s +set "JAVA_VERSION_LINE=" +for /f "delims=" %%A in ('"%JAVA_HOME_DETECTED%\bin\java.exe" -version 2^>^&1 ^| findstr /r /c:"version \""') do ( + set "JAVA_VERSION_LINE=%%A" + goto :check_java_version_line +) +:check_java_version_line +if not defined JAVA_VERSION_LINE ( + echo Error: could not determine Java version. OpenJDK 21 or newer is required. + exit /b 1 +) + +for /f "tokens=2 delims=\"" %%A in ("%JAVA_VERSION_LINE%") do set "JAVA_VERSION_RAW=%%A" +for /f "tokens=1 delims=." %%A in ("%JAVA_VERSION_RAW%") do set "JAVA_MAJOR=%%A" + +if not defined JAVA_MAJOR ( + echo Error: could not parse Java version from "%JAVA_VERSION_LINE%". + exit /b 1 +) -REM Check if INITIAL_ADMIN_PASSWORD_FILE and INITIAL_ADMIN_PASSWORD are empty -REM Set default password if neither is provided -if "%INITIAL_ADMIN_PASSWORD_FILE%"=="" if "%INITIAL_ADMIN_PASSWORD%"=="" ( - set INITIAL_ADMIN_PASSWORD=admin +if %JAVA_MAJOR% LSS 21 ( + echo Error: Java 21 or newer is required. Found Java %JAVA_MAJOR%. + exit /b 1 ) -REM Call the next batch script to handle setting the initial admin password -CALL "%SCRIPT_DIR%set-initial-admin-password.bat" +exit /b 0 -REM Start the node -java -Xms6g -Xmx6g -Xss256k -XX:ReservedCodeCacheSize=512m -XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError ^ - -Dlogback.configurationFile=./logback.xml ^ - -cp "lib/*" ^ - -Djava.system.class.loader="org.sensorhub.utils.NativeClassLoader" ^ - -Djavax.net.ssl.keyStore="./osh-keystore.p12" ^ - -Djavax.net.ssl.keyStorePassword="atakatak" ^ - -Djavax.net.ssl.trustStore="%~dp0trustStore.jks" ^ - -Djavax.net.ssl.trustStorePassword="changeit" ^ - -Djava.library.path="./nativelibs" ^ - com.botts.impl.security.SensorHubWrapper config.json db +:find_existing_oscar +set "OSCAR_PID=" +for /f %%P in ('powershell -NoProfile -Command "$p = Get-CimInstance Win32_Process ^| Where-Object { $_.Name -match ''^java(\.exe)?$'' -and $_.CommandLine -like ''*com.botts.impl.security.SensorHubWrapper*'' } ^| Select-Object -ExpandProperty ProcessId -First 1; if ($p) { Write-Output $p }"') do set "OSCAR_PID=%%P" +exit /b 0 + +:check_existing_oscar +call :find_existing_oscar +if not defined OSCAR_PID exit /b 0 + +if "%FORCE_RESTART%"=="1" ( + echo Existing OSCAR instance found with PID %OSCAR_PID%. Replacing because FORCE_RESTART=1. + taskkill /PID %OSCAR_PID% /T /F >nul 2>nul + timeout /t 2 /nobreak >nul + call :find_existing_oscar + if defined OSCAR_PID ( + echo Error: could not stop the existing OSCAR instance. + exit /b 1 + ) + exit /b 0 +) + +echo OSCAR is already running with PID %OSCAR_PID%. +echo Close the existing instance first, or set FORCE_RESTART=1 to replace it. +exit /b 1 + +:ensure_runtime_paths +if not exist "%SCRIPT_DIR%load_trusted_certs.bat" ( + echo Error: load_trusted_certs.bat not found in "%SCRIPT_DIR%". + exit /b 1 +) +if not exist "%SCRIPT_DIR%set-initial-admin-password.bat" ( + echo Error: set-initial-admin-password.bat not found in "%SCRIPT_DIR%". + exit /b 1 +) + +if not exist "%SCRIPT_DIR%config.json" ( + echo Error: missing config file: "%SCRIPT_DIR%config.json". + exit /b 1 +) + +if not exist "%SCRIPT_DIR%lib" ( + echo Error: missing library directory: "%SCRIPT_DIR%lib". + exit /b 1 +) + +if not exist "%SCRIPT_DIR%nativelibs" ( + echo Error: missing native library directory: "%SCRIPT_DIR%nativelibs". + exit /b 1 +) + +if not exist "%SCRIPT_DIR%db" mkdir "%SCRIPT_DIR%db" >nul 2>nul +if not exist "%SCRIPT_DIR%db" ( + echo Error: could not create database directory: "%SCRIPT_DIR%db". + exit /b 1 +) + +exit /b 0 + +:load_env +for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( + set "ENV_NAME=%%A" + set "ENV_VALUE=%%B" + call :set_env_var +) +exit /b 0 -endlocal +:set_env_var +if not defined ENV_NAME exit /b 0 +if "%ENV_NAME:~0,1%"=="#" exit /b 0 +if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" +set "%ENV_NAME%=%ENV_VALUE%" +exit /b 0 diff --git a/dist/scripts/standard/launch.sh b/dist/scripts/standard/launch.sh old mode 100755 new mode 100644 index f9d4094..eadc411 --- a/dist/scripts/standard/launch.sh +++ b/dist/scripts/standard/launch.sh @@ -1,37 +1,229 @@ -#!/bin/bash +#!/usr/bin/env bash +set -euo pipefail -# Make sure all the necessary certificates are trusted by the system. -SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) -"$SCRIPT_DIR/load_trusted_certs.sh" +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +MATCH_EXPR='com.botts.impl.security.SensorHubWrapper' - export KEYSTORE="./osh-keystore.p12" - export KEYSTORE_TYPE=PKCS12 - export KEYSTORE_PASSWORD="atakatak" +load_env() { + local env_file="$1" + while IFS= read -r line || [ -n "$line" ]; do + case "$line" in + ""|"#"*) continue ;; + export\ *) line="${line#export }" ;; + esac + local name="${line%%=*}" + local value="${line#*=}" + export "${name}=${value}" + done < "$env_file" +} - export TRUSTSTORE="./truststore.jks" - export TRUSTSTORE_TYPE=JKS - export TRUSTSTORE_PASSWORD="changeit" - export INITIAL_ADMIN_PASSWORD_FILE="./.s" +require_cmd() { + local cmd="$1" + if ! command -v "$cmd" >/dev/null 2>&1; then + echo "Error: required command not found: $cmd" + exit 1 + fi +} +get_java_major() { + java -version 2>&1 | awk -F'"' '/version/ { split($2, v, "."); print v[1]; exit }' +} -# After copying the default configuration file, also look to see if they -# specified what they want the initial admin user's password to be, either -# as a secret file or by providing it as an environment variable. -if [ -z "$INITIAL_ADMIN_PASSWORD_FILE" ] && [ -z "$INITIAL_ADMIN_PASSWORD" ]; then - export INITIAL_ADMIN_PASSWORD=admin +check_dependencies() { + require_cmd bash + require_cmd java + require_cmd keytool + + local java_major + java_major="$(get_java_major || true)" + if [[ -z "$java_major" || ! "$java_major" =~ ^[0-9]+$ ]]; then + echo "Error: could not determine Java version. Java 21 or newer is required." + exit 1 + fi + if [ "$java_major" -lt 21 ]; then + echo "Error: Java 21 or newer is required. Found Java $java_major." + exit 1 + fi +} + +find_existing_oscar_pids() { + pgrep -f "$MATCH_EXPR" || true +} + +stop_existing_oscar() { + local pids="$1" + if [ -z "$pids" ]; then + return 0 + fi + + echo "Stopping existing OSCAR instance(s): $pids" + kill $pids 2>/dev/null || true + + local waited=0 + while [ "$waited" -lt 15 ]; do + sleep 1 + waited=$((waited + 1)) + if [ -z "$(find_existing_oscar_pids)" ]; then + return 0 + fi + done + + echo "Existing OSCAR instance still running after graceful stop. Forcing stop." + kill -9 $pids 2>/dev/null || true + sleep 1 + + if [ -n "$(find_existing_oscar_pids)" ]; then + echo "Error: unable to stop the existing OSCAR instance." + exit 1 + fi +} + +check_existing_oscar() { + local pids + pids="$(find_existing_oscar_pids)" + + if [ -z "$pids" ]; then + return 0 + fi + + if [ "${FORCE_RESTART:-0}" = "1" ]; then + echo "Existing OSCAR instance found with PID(s): $pids. Replacing because FORCE_RESTART=1." + stop_existing_oscar "$pids" + return 0 + fi + + echo "OSCAR is already running with PID(s): $pids." + echo "Run stop-all.sh first, or set FORCE_RESTART=1 to replace the existing OSCAR process." + exit 1 +} + +ensure_runtime_paths() { + if [ ! -f "$SCRIPT_DIR/config.json" ]; then + echo "Error: missing config file: $SCRIPT_DIR/config.json" + exit 1 + fi + + if [ ! -d "$SCRIPT_DIR/lib" ]; then + echo "Error: missing library directory: $SCRIPT_DIR/lib" + exit 1 + fi + + if [ ! -d "$SCRIPT_DIR/nativelibs" ]; then + echo "Error: missing native library directory: $SCRIPT_DIR/nativelibs" + exit 1 + fi + + mkdir -p "$SCRIPT_DIR/db" + + if [ ! -f "$SCRIPT_DIR/load_trusted_certs.sh" ]; then + echo "Error: load_trusted_certs.sh not found in $SCRIPT_DIR" + exit 1 + fi + + if [ ! -f "$SCRIPT_DIR/set-initial-admin-password.sh" ]; then + echo "Error: set-initial-admin-password.sh not found in $SCRIPT_DIR" + exit 1 + fi +} + +ENV_FILE="" +if [ -f "$SCRIPT_DIR/.env" ]; then + ENV_FILE="$SCRIPT_DIR/.env" +elif [ -f "$SCRIPT_DIR/../.env" ]; then + ENV_FILE="$SCRIPT_DIR/../.env" +fi + +if [ -n "$ENV_FILE" ]; then + load_env "$ENV_FILE" +fi + +check_dependencies +check_existing_oscar +ensure_runtime_paths + +SYSTEM_PROFILE="${SYSTEM_PROFILE:-8GB}" + +case "$SYSTEM_PROFILE" in + RPI4) + JAVA_XMS="512m" + JAVA_XMX="1536m" + JAVACPP_MAX_BYTES_DEFAULT="512m" + JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="2g" + ;; + 8GB) + JAVA_XMS="1g" + JAVA_XMX="2g" + JAVACPP_MAX_BYTES_DEFAULT="1g" + JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="4g" + ;; + 16GB) + JAVA_XMS="1g" + JAVA_XMX="3g" + JAVACPP_MAX_BYTES_DEFAULT="2g" + JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="8g" + ;; + 32GB) + JAVA_XMS="2g" + JAVA_XMX="6g" + JAVACPP_MAX_BYTES_DEFAULT="4g" + JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="16g" + ;; + *) + echo "Unknown profile '$SYSTEM_PROFILE', using 8GB defaults." + JAVA_XMS="1g" + JAVA_XMX="2g" + JAVACPP_MAX_BYTES_DEFAULT="1g" + JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="4g" + ;; +esac + +: "${JAVACPP_MAX_BYTES:=$JAVACPP_MAX_BYTES_DEFAULT}" +: "${JAVACPP_MAX_PHYSICAL_BYTES:=$JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT}" +: "${JFR_FILENAME:=$SCRIPT_DIR/oscar.jfr}" + +mkdir -p "$(dirname "$JFR_FILENAME")" + +export KEYSTORE="${KEYSTORE:-$SCRIPT_DIR/osh-keystore.p12}" +export KEYSTORE_TYPE="${KEYSTORE_TYPE:-PKCS12}" +export KEYSTORE_PASSWORD="${KEYSTORE_PASSWORD:-atakatak}" + +export TRUSTSTORE="${TRUSTSTORE:-$SCRIPT_DIR/truststore.jks}" +export TRUSTSTORE_TYPE="${TRUSTSTORE_TYPE:-JKS}" +export TRUSTSTORE_PASSWORD="${TRUSTSTORE_PASSWORD:-changeit}" + +export INITIAL_ADMIN_PASSWORD_FILE="${INITIAL_ADMIN_PASSWORD_FILE:-$SCRIPT_DIR/.s}" +if [ ! -f "$INITIAL_ADMIN_PASSWORD_FILE" ] && [ -z "${INITIAL_ADMIN_PASSWORD:-}" ]; then + export INITIAL_ADMIN_PASSWORD="admin" +fi + +if [ -z "${HOME:-}" ] && [ -n "${USER:-}" ]; then + export HOME="/home/${USER}" fi -"$SCRIPT_DIR/set-initial-admin-password.sh" +echo "Starting OSH Node with Profile: $SYSTEM_PROFILE" +echo " Heap: $JAVA_XMS / $JAVA_XMX" +echo " JavaCPP maxBytes: $JAVACPP_MAX_BYTES" +echo " JavaCPP maxPhysicalBytes: $JAVACPP_MAX_PHYSICAL_BYTES" +echo " JFR file: $JFR_FILENAME" +bash "$SCRIPT_DIR/load_trusted_certs.sh" +bash "$SCRIPT_DIR/set-initial-admin-password.sh" -# Start the node -java -Xms6g -Xmx6g -Xss256k -XX:ReservedCodeCacheSize=512m -XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError \ - -Dlogback.configurationFile=./logback.xml \ - -cp "lib/*" \ - -Djava.system.class.loader="org.sensorhub.utils.NativeClassLoader" \ - -Djavax.net.ssl.keyStore="./osh-keystore.p12" \ - -Djavax.net.ssl.keyStorePassword="atakatak" \ - -Djavax.net.ssl.trustStore="$SCRIPT_DIR/trustStore.jks" \ - -Djavax.net.ssl.trustStorePassword="changeit" \ - -Djava.library.path="./nativelibs" \ - com.botts.impl.security.SensorHubWrapper ./config.json ./db +exec java \ + -Xms"$JAVA_XMS" \ + -Xmx"$JAVA_XMX" \ + -Xss256k \ + -XX:ReservedCodeCacheSize=256m \ + -XX:+UseG1GC \ + -XX:+HeapDumpOnOutOfMemoryError \ + -XX:+UnlockDiagnosticVMOptions \ + -XX:NativeMemoryTracking=summary \ + "-Dorg.bytedeco.javacpp.maxBytes=$JAVACPP_MAX_BYTES" \ + "-Dorg.bytedeco.javacpp.maxPhysicalBytes=$JAVACPP_MAX_PHYSICAL_BYTES" \ + -Dorg.bytedeco.javacpp.maxRetries=2 \ + -Dorg.bytedeco.javacpp.mxbean=true \ + "-Dlogback.configurationFile=$SCRIPT_DIR/logback.xml" \ + -cp "$SCRIPT_DIR/lib/*" \ + "-Djava.system.class.loader=org.sensorhub.utils.NativeClassLoader" \ + "-Djava.library.path=$SCRIPT_DIR/nativelibs" \ + com.botts.impl.security.SensorHubWrapper "$SCRIPT_DIR/config.json" "$SCRIPT_DIR/db" diff --git a/dist/scripts/standard/launch_old.bat b/dist/scripts/standard/launch_old.bat new file mode 100644 index 0000000..b7d52fc --- /dev/null +++ b/dist/scripts/standard/launch_old.bat @@ -0,0 +1,144 @@ +@echo off +setlocal EnableExtensions + +rem Resolve this script's directory. %~dp0 includes the trailing backslash. +set "SCRIPT_DIR=%~dp0" + +rem Load .env from this directory, or from the parent directory when this script +rem is launched from osh-node-oscar under the project root. +set "ENV_FILE=" +if exist "%SCRIPT_DIR%.env" ( + set "ENV_FILE=%SCRIPT_DIR%.env" +) else if exist "%SCRIPT_DIR%..\.env" ( + set "ENV_FILE=%SCRIPT_DIR%..\.env" +) + +if defined ENV_FILE ( + call :load_env "%ENV_FILE%" + echo AFTER load_env ERR=%ERRORLEVEL% +) + +rem Pick Java and JavaCPP defaults from SYSTEM_PROFILE. +if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" + +if /I "%SYSTEM_PROFILE%"=="RPI4" ( + set "JAVA_XMS=512m" + set "JAVA_XMX=1536m" + set "JAVACPP_MAX_BYTES_DEFAULT=512m" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=2g" +) else if /I "%SYSTEM_PROFILE%"=="8GB" ( + set "JAVA_XMS=1g" + set "JAVA_XMX=2g" + set "JAVACPP_MAX_BYTES_DEFAULT=1g" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=4g" +) else if /I "%SYSTEM_PROFILE%"=="16GB" ( + set "JAVA_XMS=1g" + set "JAVA_XMX=3g" + set "JAVACPP_MAX_BYTES_DEFAULT=2g" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=8g" +) else if /I "%SYSTEM_PROFILE%"=="32GB" ( + set "JAVA_XMS=2g" + set "JAVA_XMX=6g" + set "JAVACPP_MAX_BYTES_DEFAULT=4g" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=16g" +) else ( + echo Unknown profile '%SYSTEM_PROFILE%', using 8GB defaults. + set "JAVA_XMS=1g" + set "JAVA_XMX=2g" + set "JAVACPP_MAX_BYTES_DEFAULT=1g" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=4g" +) + +if not defined JAVACPP_MAX_BYTES set "JAVACPP_MAX_BYTES=%JAVACPP_MAX_BYTES_DEFAULT%" +if not defined JAVACPP_MAX_PHYSICAL_BYTES set "JAVACPP_MAX_PHYSICAL_BYTES=%JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT%" +if not defined JFR_FILENAME set "JFR_FILENAME=%SCRIPT_DIR%oscar.jfr" + +echo Starting OSH Node with Profile: %SYSTEM_PROFILE% +echo Heap: %JAVA_XMS% / %JAVA_XMX% +echo JavaCPP maxBytes: %JAVACPP_MAX_BYTES% +echo JavaCPP maxPhysicalBytes: %JAVACPP_MAX_PHYSICAL_BYTES% +echo JFR file: %JFR_FILENAME% + +rem Make sure all the necessary certificates are trusted by the system. +if not exist "%SCRIPT_DIR%load_trusted_certs.bat" ( + echo Error: load_trusted_certs.bat not found in "%SCRIPT_DIR%". + exit /b 1 +) +call "%SCRIPT_DIR%load_trusted_certs.bat" +echo AFTER load_trusted_certs ERR=%ERRORLEVEL% +if errorlevel 1 exit /b %ERRORLEVEL% + +set "KEYSTORE=%SCRIPT_DIR%osh-keystore.p12" +set "KEYSTORE_TYPE=PKCS12" +if not defined KEYSTORE_PASSWORD set "KEYSTORE_PASSWORD=atakatak" + +set "TRUSTSTORE=%SCRIPT_DIR%truststore.jks" +set "TRUSTSTORE_TYPE=JKS" +if not defined TRUSTSTORE_PASSWORD set "TRUSTSTORE_PASSWORD=changeit" + +set "INITIAL_ADMIN_PASSWORD_FILE=%SCRIPT_DIR%.s" + +rem If no secret file exists and no env var was supplied, use the dev default. +if not exist "%INITIAL_ADMIN_PASSWORD_FILE%" if not defined INITIAL_ADMIN_PASSWORD set "INITIAL_ADMIN_PASSWORD=admin" + +if not exist "%SCRIPT_DIR%set-initial-admin-password.bat" ( + echo Error: set-initial-admin-password.bat not found in "%SCRIPT_DIR%". + exit /b 1 +) +call "%SCRIPT_DIR%set-initial-admin-password.bat" +echo AFTER set-initial-admin-password ERR=%ERRORLEVEL% +if errorlevel 1 exit /b %ERRORLEVEL% + +echo BEFORE JAVA +where java +echo KEYSTORE=%KEYSTORE% +echo TRUSTSTORE=%TRUSTSTORE% +echo INITIAL_ADMIN_PASSWORD_FILE=%INITIAL_ADMIN_PASSWORD_FILE% +echo SCRIPT_DIR=%SCRIPT_DIR% +echo CONFIG=%SCRIPT_DIR%config.json +echo DBDIR=%SCRIPT_DIR%db +echo NATIVELIBS=%SCRIPT_DIR%nativelibs + +java ^ + -Xms%JAVA_XMS% ^ + -Xmx%JAVA_XMX% ^ + -Xss256k ^ + -XX:ReservedCodeCacheSize=256m ^ + -XX:+UseG1GC ^ + -XX:+HeapDumpOnOutOfMemoryError ^ + -XX:+UnlockDiagnosticVMOptions ^ + -XX:NativeMemoryTracking=summary ^ + "-Dorg.bytedeco.javacpp.maxBytes=%JAVACPP_MAX_BYTES%" ^ + "-Dorg.bytedeco.javacpp.maxPhysicalBytes=%JAVACPP_MAX_PHYSICAL_BYTES%" ^ + -Dorg.bytedeco.javacpp.maxRetries=2 ^ + -Dorg.bytedeco.javacpp.mxbean=true ^ + "-Dlogback.configurationFile=%SCRIPT_DIR%logback.xml" ^ + -cp "%SCRIPT_DIR%lib\*" ^ + "-Djava.system.class.loader=org.sensorhub.utils.NativeClassLoader" ^ + "-Djavax.net.ssl.keyStore=%KEYSTORE%" ^ + "-Djavax.net.ssl.keyStorePassword=%KEYSTORE_PASSWORD%" ^ + "-Djavax.net.ssl.trustStore=%TRUSTSTORE%" ^ + "-Djavax.net.ssl.trustStorePassword=%TRUSTSTORE_PASSWORD%" ^ + "-Djava.library.path=%SCRIPT_DIR%nativelibs" ^ + com.botts.impl.security.SensorHubWrapper "%SCRIPT_DIR%config.json" "%SCRIPT_DIR%db" + +set "JAVA_EXIT_CODE=%ERRORLEVEL%" +echo AFTER JAVA +echo JAVA_EXIT_CODE=%JAVA_EXIT_CODE% +pause +endlocal & exit /b %JAVA_EXIT_CODE% + +:load_env +for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( + set "ENV_NAME=%%A" + set "ENV_VALUE=%%B" + call :set_env_var +) +exit /b 0 + +:set_env_var +if not defined ENV_NAME exit /b 0 +if "%ENV_NAME:~0,1%"=="#" exit /b 0 +if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" +set "%ENV_NAME%=%ENV_VALUE%" +exit /b 0 \ No newline at end of file diff --git a/dist/scripts/standard/launch_old.sh b/dist/scripts/standard/launch_old.sh new file mode 100644 index 0000000..d88905e --- /dev/null +++ b/dist/scripts/standard/launch_old.sh @@ -0,0 +1,141 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +load_env() { + local env_file="$1" + while IFS= read -r line || [ -n "$line" ]; do + case "$line" in + ""|"#"*) continue ;; + export\ *) line="${line#export }" ;; + esac + local name="${line%%=*}" + local value="${line#*=}" + export "${name}=${value}" + done < "$env_file" +} + +ENV_FILE="" +if [ -f "$SCRIPT_DIR/.env" ]; then + ENV_FILE="$SCRIPT_DIR/.env" +elif [ -f "$SCRIPT_DIR/../.env" ]; then + ENV_FILE="$SCRIPT_DIR/../.env" +fi + +if [ -n "$ENV_FILE" ]; then + load_env "$ENV_FILE" +fi + +check_existing_oscar() { + local pids + pids="$(pgrep -f 'com.botts.impl.security.SensorHubWrapper' || true)" + + if [ -z "$pids" ]; then + return 0 + fi + + if [ "${FORCE_RESTART:-0}" = "1" ]; then + echo "Existing OSCAR instance found with PID(s): $pids. Stopping because FORCE_RESTART=1." + kill $pids || true + sleep 2 + return 0 + fi + + echo "OSCAR is already running with PID(s): $pids." + echo "Run stop-all.sh first, or set FORCE_RESTART=1 to replace the existing OSCAR process." + return 1 +} + +check_existing_oscar || exit 1 + +SYSTEM_PROFILE="${SYSTEM_PROFILE:-8GB}" + +case "$SYSTEM_PROFILE" in + RPI4) + JAVA_XMS="512m" + JAVA_XMX="1536m" + JAVACPP_MAX_BYTES_DEFAULT="512m" + JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="2g" + ;; + 8GB) + JAVA_XMS="1g" + JAVA_XMX="2g" + JAVACPP_MAX_BYTES_DEFAULT="1g" + JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="4g" + ;; + 16GB) + JAVA_XMS="1g" + JAVA_XMX="3g" + JAVACPP_MAX_BYTES_DEFAULT="2g" + JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="8g" + ;; + 32GB) + JAVA_XMS="2g" + JAVA_XMX="6g" + JAVACPP_MAX_BYTES_DEFAULT="4g" + JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="16g" + ;; + *) + echo "Unknown profile '$SYSTEM_PROFILE', using 8GB defaults." + JAVA_XMS="1g" + JAVA_XMX="2g" + JAVACPP_MAX_BYTES_DEFAULT="1g" + JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="4g" + ;; +esac + +: "${JAVACPP_MAX_BYTES:=$JAVACPP_MAX_BYTES_DEFAULT}" +: "${JAVACPP_MAX_PHYSICAL_BYTES:=$JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT}" +: "${JFR_FILENAME:=$SCRIPT_DIR/oscar.jfr}" + +echo "Starting OSH Node with Profile: $SYSTEM_PROFILE" +echo " Heap: $JAVA_XMS / $JAVA_XMX" +echo " JavaCPP maxBytes: $JAVACPP_MAX_BYTES" +echo " JavaCPP maxPhysicalBytes: $JAVACPP_MAX_PHYSICAL_BYTES" +echo " JFR file: $JFR_FILENAME" + +if [ ! -f "$SCRIPT_DIR/load_trusted_certs.sh" ]; then + echo "Error: load_trusted_certs.sh not found in $SCRIPT_DIR." + exit 1 +fi +bash "$SCRIPT_DIR/load_trusted_certs.sh" + +export KEYSTORE="${KEYSTORE:-$SCRIPT_DIR/osh-keystore.p12}" +export KEYSTORE_TYPE="${KEYSTORE_TYPE:-PKCS12}" +export KEYSTORE_PASSWORD="${KEYSTORE_PASSWORD:-atakatak}" + +export TRUSTSTORE="${TRUSTSTORE:-$SCRIPT_DIR/truststore.jks}" +export TRUSTSTORE_TYPE="${TRUSTSTORE_TYPE:-JKS}" +export TRUSTSTORE_PASSWORD="${TRUSTSTORE_PASSWORD:-changeit}" + +export INITIAL_ADMIN_PASSWORD_FILE="${INITIAL_ADMIN_PASSWORD_FILE:-$SCRIPT_DIR/.s}" + +if [ ! -f "$INITIAL_ADMIN_PASSWORD_FILE" ] && [ -z "${INITIAL_ADMIN_PASSWORD:-}" ]; then + export INITIAL_ADMIN_PASSWORD="admin" +fi + +if [ ! -f "$SCRIPT_DIR/set-initial-admin-password.sh" ]; then + echo "Error: set-initial-admin-password.sh not found in $SCRIPT_DIR." + exit 1 +fi +bash "$SCRIPT_DIR/set-initial-admin-password.sh" + +exec java \ + -Xms"$JAVA_XMS" \ + -Xmx"$JAVA_XMX" \ + -Xss256k \ + -XX:ReservedCodeCacheSize=256m \ + -XX:+UseG1GC \ + -XX:+HeapDumpOnOutOfMemoryError \ + -XX:+UnlockDiagnosticVMOptions \ + -XX:NativeMemoryTracking=summary \ + "-Dorg.bytedeco.javacpp.maxBytes=$JAVACPP_MAX_BYTES" \ + "-Dorg.bytedeco.javacpp.maxPhysicalBytes=$JAVACPP_MAX_PHYSICAL_BYTES" \ + -Dorg.bytedeco.javacpp.maxRetries=2 \ + -Dorg.bytedeco.javacpp.mxbean=true \ + "-Dlogback.configurationFile=$SCRIPT_DIR/logback.xml" \ + -cp "$SCRIPT_DIR/lib/*" \ + "-Djava.system.class.loader=org.sensorhub.utils.NativeClassLoader" \ + "-Djava.library.path=$SCRIPT_DIR/nativelibs" \ + com.botts.impl.security.SensorHubWrapper "$SCRIPT_DIR/config.json" "$SCRIPT_DIR/db" \ No newline at end of file diff --git a/dist/scripts/standard/load_trusted_certs.bat b/dist/scripts/standard/load_trusted_certs.bat index 5357164..65a1541 100755 --- a/dist/scripts/standard/load_trusted_certs.bat +++ b/dist/scripts/standard/load_trusted_certs.bat @@ -1,64 +1,99 @@ @echo off -setlocal +setlocal EnableExtensions EnableDelayedExpansion echo Building Java trust store... -REM Default password for the sytem trust store is "changeit". Edit this next -REM line if it's something different in your Java installation. set "STOREPASS=changeit" - -REM Get the path of this script. set "SCRIPTDIR=%~dp0" +set "NEWTRUSTSTORE=%SCRIPTDIR%truststore.jks" +set "CERTDIR=%SCRIPTDIR%trusted_certificates" +set "CACERTS=" +set "JAVA_HOME_DETECTED=" -REM Get the path where we'll build the new trust store. -set "NEWTRUSTSTORE=%SCRIPTDIR%trustStore.jks" +rem First try JAVA_HOME if already set +if defined JAVA_HOME ( + if exist "%JAVA_HOME%\conf\security\cacerts" set "CACERTS=%JAVA_HOME%\conf\security\cacerts" + if not defined CACERTS if exist "%JAVA_HOME%\lib\security\cacerts" set "CACERTS=%JAVA_HOME%\lib\security\cacerts" +) -REM To find the location of the system trust store, we start by finding the -REM path to "java.exe". -for /f "tokens=* usebackq" %%j in (`where java`) do (set "JAVA=%%~dpj" & goto :next ) -:next -REM Then we back up a directory and look in lib\security. -set "CACERTS=%JAVA%..\lib\security\cacerts" +rem If that did not work, ask Java itself for java.home +if not defined CACERTS ( + for /f "tokens=1,* delims==" %%A in ('java -XshowSettings:properties -version 2^>^&1 ^| findstr /c:"java.home ="') do ( + set "JAVA_HOME_DETECTED=%%B" + ) +) -REM Now make a copy of that default system trust store into this directory, -REM where we'll add our stuff to it. -copy /y "%CACERTS%" "%NEWTRUSTSTORE%" >NUL +rem Trim leading spaces +if defined JAVA_HOME_DETECTED ( + for /f "tokens=* delims= " %%H in ("!JAVA_HOME_DETECTED!") do set "JAVA_HOME_DETECTED=%%H" +) -REM Get the full path to where our certs are. -set CERTDIR=%SCRIPTDIR%trusted_certificates +rem Try common cacerts locations under detected java.home +if not defined CACERTS if defined JAVA_HOME_DETECTED ( + if exist "!JAVA_HOME_DETECTED!\conf\security\cacerts" set "CACERTS=!JAVA_HOME_DETECTED!\conf\security\cacerts" + if not defined CACERTS if exist "!JAVA_HOME_DETECTED!\lib\security\cacerts" set "CACERTS=!JAVA_HOME_DETECTED!\lib\security\cacerts" +) -REM Now for each .cer, .pem, and .crt file in our cert dir, check to see if we -REM need to add it to the system trust store. -for %%c in ( %CERTDIR%\*.cer %CERTDIR%\*.pem %CERTDIR%\*.crt ) do ( - call :check_certificate %%c +if not defined CACERTS ( + echo Error: could not locate Java cacerts. + if defined JAVA_HOME echo JAVA_HOME="%JAVA_HOME%" + if defined JAVA_HOME_DETECTED echo java.home="!JAVA_HOME_DETECTED!" + endlocal & exit /b 1 ) -goto :end_of_script +if not exist "%CACERTS%" ( + echo Error: Java cacerts path does not exist: "%CACERTS%" + endlocal & exit /b 1 +) -REM The next few lines define a function that checks whether a certificate -REM is already loaded in the system store. If so, it does nothing. If not, it -REM attempts to load it in. Note that the alias of the certificate is -REM calculated as the base file name (without path or extension). -REM NOTE: As currently written, this is performing an unnecessary check, since -REM we're guaranteed that none of the certificates will ever be present in the -REM original file. +echo Using Java cacerts: "%CACERTS%" -:check_certificate -set ALIAS=%~n1 -REM Check for existence. ERRORLEVEL is set to 0 if it's found, and something -REM else otherwise. -keytool -list -keystore "%NEWTRUSTSTORE%" -storepass "%STOREPASS%" -alias "%ALIAS%" >NUL 2>NUL -if not "%ERRORLEVEL%" == "0" ( - echo Importing "%ALIAS%" from "%1" - keytool -importcert -keystore "%NEWTRUSTSTORE%" -noprompt -storepass "%STOREPASS%" -alias "%ALIAS%" -file "%1" -) else ( - echo Certificate with alias "%ALIAS%" already exists. Skipping. +copy /y "%CACERTS%" "%NEWTRUSTSTORE%" >nul +if errorlevel 1 ( + echo Error: failed to create "%NEWTRUSTSTORE%" + endlocal & exit /b 1 +) + +if not exist "%CERTDIR%" ( + echo Trusted certificates directory not found: "%CERTDIR%" + echo Using copied default trust store only. + echo Done. + endlocal & exit /b 0 +) + +set "FOUND_CERT=0" +for %%c in ("%CERTDIR%\*.cer" "%CERTDIR%\*.pem" "%CERTDIR%\*.crt") do ( + if exist "%%~fc" ( + set "FOUND_CERT=1" + call :check_certificate "%%~fc" + if errorlevel 1 ( + endlocal & exit /b 1 + ) + ) ) -REM Return to caller. -exit /b 0 -:end_of_script +if "%FOUND_CERT%"=="0" ( + echo No certificate files found in "%CERTDIR%". +) echo Done. +endlocal & exit /b 0 + +:check_certificate +setlocal +set "CERTFILE=%~1" +set "ALIAS=%~n1" + +keytool -list -keystore "%NEWTRUSTSTORE%" -storepass "%STOREPASS%" -alias "%ALIAS%" >nul 2>nul +if not "%ERRORLEVEL%"=="0" ( + echo Importing "%ALIAS%" from "%CERTFILE%" + keytool -importcert -keystore "%NEWTRUSTSTORE%" -noprompt -storepass "%STOREPASS%" -alias "%ALIAS%" -file "%CERTFILE%" >nul + if errorlevel 1 ( + echo Error: failed to import "%ALIAS%" from "%CERTFILE%" + endlocal & exit /b 1 + ) +) else ( + echo Certificate with alias "%ALIAS%" already exists. Skipping. +) -endlocal +endlocal & exit /b 0 \ No newline at end of file From 6a08c7ea3e4e11220e1640d7ecd84a2e5b76fdc3 Mon Sep 17 00:00:00 2001 From: tyronechrisharris <142608718+tyronechrisharris@users.noreply.github.com> Date: Tue, 5 May 2026 19:58:01 -0400 Subject: [PATCH 2/9] harden OSCAR 3.5.1 deployment, monitoring, and DB stability --- include/osh-addons | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/osh-addons b/include/osh-addons index 8b0aabd..bbadeb2 160000 --- a/include/osh-addons +++ b/include/osh-addons @@ -1 +1 @@ -Subproject commit 8b0aabd5a74aa375a5424dde516897e67abb82d7 +Subproject commit bbadeb2dab95b8f1b3f52c458fc6fbc0b2997f2b From c9140a82426347c56c4161da29b253bece0f5b96 Mon Sep 17 00:00:00 2001 From: tyronechrisharris <142608718+tyronechrisharris@users.noreply.github.com> Date: Wed, 6 May 2026 23:46:47 -0400 Subject: [PATCH 3/9] release: finalize OSCAR 3.5.1 deployment hardening, monitoring, docs, and DB pool fixes --- README.md | 54 +- build-all.bat | 30 +- build-all.sh | 28 +- build.gradle | 7 +- changelog.md | 34 +- dist/config/standard/config.template.json | 239 ++++++ .../OSCAR_System_Documentation_Manual_3.5.md | 82 +- .../OSCAR_launch_monitoring_guide.md | 100 ++- dist/documentation/Release_Notes_3.5.1.md | 23 + dist/release/check-oscar-status.ps1 | 807 +++++++++++++++--- dist/release/check-oscar-status.sh | 0 dist/release/env.template | 2 +- dist/release/launch-all-arm.sh | 76 +- dist/release/launch-all-arm_old.sh | 247 ------ dist/release/launch-all.bat | 410 ++++----- dist/release/launch-all.sh | 76 +- dist/release/launch-all_old.bat | 192 ----- dist/release/launch-all_old.sh | 111 --- dist/release/monitor-oscar.bat | 339 ++++---- dist/release/monitor-oscar.sh | 193 ++--- dist/release/monitor-oscar_old.bat | 129 --- dist/release/monitor-oscar_old.sh | 254 ------ dist/release/reset-all.bat | 96 +++ dist/release/reset-all.sh | 78 ++ dist/release/stop-all.bat | 50 +- dist/release/stop-all.sh | 84 +- dist/scripts/standard/launch.bat | 245 +++--- dist/scripts/standard/launch.sh | 23 +- dist/scripts/standard/launch_old.bat | 144 ---- dist/scripts/standard/launch_old.sh | 141 --- 30 files changed, 1896 insertions(+), 2398 deletions(-) create mode 100644 dist/config/standard/config.template.json mode change 100644 => 100755 dist/release/check-oscar-status.sh mode change 100644 => 100755 dist/release/launch-all-arm.sh delete mode 100644 dist/release/launch-all-arm_old.sh mode change 100644 => 100755 dist/release/launch-all.bat mode change 100644 => 100755 dist/release/launch-all.sh delete mode 100644 dist/release/launch-all_old.bat delete mode 100644 dist/release/launch-all_old.sh mode change 100644 => 100755 dist/release/monitor-oscar.bat mode change 100644 => 100755 dist/release/monitor-oscar.sh delete mode 100644 dist/release/monitor-oscar_old.bat delete mode 100644 dist/release/monitor-oscar_old.sh create mode 100755 dist/release/reset-all.bat create mode 100755 dist/release/reset-all.sh mode change 100644 => 100755 dist/scripts/standard/launch.bat mode change 100644 => 100755 dist/scripts/standard/launch.sh delete mode 100644 dist/scripts/standard/launch_old.bat delete mode 100644 dist/scripts/standard/launch_old.sh diff --git a/README.md b/README.md index 1655ea3..d2f4d76 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ java -version docker version ``` -Use **Java 21 or newer**. The launch scripts validate dependencies and will stop early if Java or Docker is missing or too old. +Use **Java 21 or newer**. The launch scripts validate dependencies and stop early if Java or Docker is missing or too old. ### 2. If you were previously running OSCAR, start fresh @@ -71,6 +71,12 @@ Then delete the old extracted release folder, such as **oscar-3.5.0**, before ex Extract the downloaded ZIP to a fresh working directory. +The packaged release archive is expected to be named: + +```text +oscar-3.5.1.zip +``` + ### 4. Create the runtime environment file For packaged releases, use the environment file that ships with the archive: @@ -121,10 +127,15 @@ monitor-oscar.bat Linux: ```bash -chmod +x launch-all.sh osh-node-oscar/launch.sh monitor-oscar.sh check-oscar-status.sh ./monitor-oscar.sh ``` +The packaged Linux release is built so the shipped `*.sh` files are already executable. If your unzip tool strips execute bits, restore them with: + +```bash +chmod +x *.sh osh-node-oscar/*.sh +``` + This is the recommended first-run path because it: - starts PostGIS and OSCAR using the current launch scripts @@ -151,7 +162,7 @@ Prefer these **sessionless top-level launchers** over calling `osh-node-oscar/la ### 7. Running-instance handling -The launch and monitor scripts now detect already-running OSCAR JVMs. +The launch and monitor scripts detect already-running OSCAR JVMs. Default behavior: @@ -179,7 +190,25 @@ Linux: ./check-oscar-status.sh ``` -### 9. Admin access +### 9. Clean reset between side-by-side test installs + +When you need to clear the local OSCAR runtime state before standing up a different installation on the same machine, use the reset script for your platform. + +Windows: + +```bat +reset-all.bat +``` + +Linux: + +```bash +./reset-all.sh +``` + +These scripts stop monitor and OSCAR processes, remove the PostGIS container and volumes, and clear local runtime data used by the packaged deployment. + +### 10. Admin access The admin username is typically **admin**. Do **not** assume the packaged password is always `admin`. @@ -222,7 +251,17 @@ Windows: build-all.bat ``` -After the build completes, the output is written under `build/distributions/`. +After the build completes, the packaged release is written under `build/distributions/` and should be named: + +```text +oscar-3.5.1.zip +``` + +The build now also: + +- makes all packaged `*.sh` files executable +- preserves Linux shell-script execute bits in the release archive +- standardizes the release ZIP name for the 3.5.1 package ## Source-tree deployment @@ -239,7 +278,7 @@ For test systems and larger multi-lane deployments, consider placing **MediaMTX* ## PostgreSQL tuning -The packaged launch scripts now size PostgreSQL by `SYSTEM_PROFILE`. +The packaged launch scripts size PostgreSQL by `SYSTEM_PROFILE`. Representative values: @@ -275,7 +314,8 @@ Before releasing from `dev`: 3. ensure `dist/release/postgis/pgdata` is not packaged 4. verify the release ZIP name matches the intended version, such as `oscar-3.5.1.zip` 5. verify the release root directory name also matches the intended version -6. verify `env.template`, release notes, README, and launch documentation all reflect the same version +6. verify packaged Linux `*.sh` files are executable in the built archive +7. verify `env.template`, release notes, README, and launch documentation all reflect the same version ### Release steps diff --git a/build-all.bat b/build-all.bat index 2faa7ed..fea7faf 100644 --- a/build-all.bat +++ b/build-all.bat @@ -1,10 +1,30 @@ @echo off +setlocal EnableExtensions -call cd web/oscar-viewer +set "PROJECT_DIR=%~dp0" +set "RELEASE_VERSION=3.5.1" +set "DIST_DIR=%PROJECT_DIR%build\distributions" +set "STANDARD_ZIP=%DIST_DIR%\oscar-%RELEASE_VERSION%.zip" -call npm install -call npm run build +pushd "%PROJECT_DIR%web\oscar-viewer" || exit /b 1 +call npm install || goto :fail +call npm run build || goto :fail +popd -call cd ..\.. +pushd "%PROJECT_DIR%" || exit /b 1 +call gradlew.bat build -x test -x osgi || goto :fail +popd -call gradlew build -x test -x osgi +if exist "%DIST_DIR%" ( + powershell -NoProfile -Command ^ + "$dist = '%DIST_DIR%'; $target = '%STANDARD_ZIP%'; $zip = Get-ChildItem -Path $dist -Filter *.zip | Where-Object { $_.FullName -ne $target } | Sort-Object LastWriteTime -Descending | Select-Object -First 1; if ($zip) { Copy-Item -Force $zip.FullName $target; Write-Host ('Standardized release zip: ' + $target) } elseif (Test-Path $target) { Write-Host ('Release zip already available at: ' + $target) } else { Write-Warning ('No distribution zip found under ' + $dist) }" +) else ( + echo Warning: distribution directory not found: "%DIST_DIR%" +) + +exit /b 0 + +:fail +set "EXITCODE=%ERRORLEVEL%" +popd >NUL 2>NUL +exit /b %EXITCODE% diff --git a/build-all.sh b/build-all.sh index 9f0a1a7..c1ca269 100755 --- a/build-all.sh +++ b/build-all.sh @@ -2,20 +2,30 @@ set -euo pipefail PROJECT_DIR="$(cd -- "$(dirname -- "$0")" && pwd)" +RELEASE_VERSION="3.5.1" +DIST_DIR="$PROJECT_DIR/build/distributions" +STANDARD_ZIP="$DIST_DIR/oscar-${RELEASE_VERSION}.zip" -cd "$PROJECT_DIR/web/oscar-viewer" || exit 1 +echo "Making shell scripts executable..." +find "$PROJECT_DIR" -type f -name "*.sh" -exec chmod +x {} + +cd "$PROJECT_DIR/web/oscar-viewer" npm install npm run build -cd "$PROJECT_DIR" || exit 1 - +cd "$PROJECT_DIR" ./gradlew build -x test -x osgi -echo "Making shell scripts executable..." - -find "$PROJECT_DIR" -maxdepth 1 -type f -name "*.sh" -exec chmod +x {} \; - -if [ -d "$PROJECT_DIR/osh-node-oscar" ]; then - find "$PROJECT_DIR/osh-node-oscar" -maxdepth 1 -type f -name "*.sh" -exec chmod +x {} \; +if [ -d "$DIST_DIR" ]; then + GENERATED_ZIP="$(find "$DIST_DIR" -maxdepth 1 -type f -name "*.zip" ! -name "oscar-${RELEASE_VERSION}.zip" -printf "%T@ %p\n" | sort -nr | awk 'NR==1 {print $2}')" + if [ -n "$GENERATED_ZIP" ] && [ -f "$GENERATED_ZIP" ]; then + cp -f "$GENERATED_ZIP" "$STANDARD_ZIP" + echo "Standardized release zip: $STANDARD_ZIP" + elif [ -f "$STANDARD_ZIP" ]; then + echo "Release zip already available at: $STANDARD_ZIP" + else + echo "Warning: no distribution zip found under $DIST_DIR" >&2 + fi +else + echo "Warning: distribution directory not found: $DIST_DIR" >&2 fi diff --git a/build.gradle b/build.gradle index 9ed9610..2174333 100644 --- a/build.gradle +++ b/build.gradle @@ -2,7 +2,7 @@ apply from: gradle.oshCoreDir + '/common.gradle' description = '' allprojects { - version = "3.5.0" + version = "3.5.1" } subprojects { @@ -67,6 +67,11 @@ distributions{ rel { distributionBaseName = 'oscar' contents { + eachFile { + if (it.name.endsWith('.sh')) { + it.mode = 0755 + } + } // OSH NODE into("osh-node-oscar/") { from 'dist/scripts/standard' diff --git a/changelog.md b/changelog.md index 43a6a42..0be2957 100644 --- a/changelog.md +++ b/changelog.md @@ -1,29 +1,5 @@ # OSCAR Build Node Change Log - -All notable changes to this project will be documented in this file. - -## 3.5.1 2026-05-05 - -### Added -- Added profile-based launch sizing for packaged deployments. -- Added Linux and Windows monitoring workflows for first-run validation and field burn-in. -- Added one-file status report scripts for Linux and Windows. -- Added support for attach-or-restart monitor behavior through environment settings. -- Added MediaMTX deployment guidance for camera proxy use in larger test and field systems. - -### Changed -- Updated launch scripts to validate Java and Docker before startup. -- Updated launch scripts to detect an already-running OSCAR instance and fail fast unless `FORCE_RESTART=1` is set. -- Updated monitor scripts to attach to an existing OSCAR instance when `ATTACH_TO_EXISTING=1` is set. -- Updated PostGIS launch flow so an existing named container is replaced cleanly before startup settings are reapplied. -- Updated Windows launch and monitoring logic to use PowerShell and CIM-based process detection instead of WMIC. -- Updated release documentation for the prebuilt 3.5.1 unzip-and-run workflow. - -### Fixed -- Fixed PostgreSQL session over-allocation caused by oversized Hikari idle pooling. -- Fixed startup and monitoring friction on Windows around Java detection, trust store creation, and process discovery. -- Fixed release packaging/documentation drift around launch steps, cleanup expectations, and version naming. - +All notable changes to this project will be documented in this file. ## 3.5.0 2026-04-24 ### Changes - Updated LaneSystem README @@ -187,8 +163,8 @@ This is the official first release of 3.0.0 ### Changed - Restructured repository, moving most directories that are unused in development under `dist` -## [2.3.0] -Release 2.3.0 +## [2.3.0] +Release 2.3.0 ### Added - Added Deployment version to config.json @@ -201,13 +177,15 @@ Removed dependency to log4j - [#90](https://github.com/Botts-Innovative-Research/osh-oakridge-buildnode/issues/90) Aspect Charts:The prior issue mentioned the Aspect RPMs and the Admin Panel, but this encompasses Aspects issues on the client as well. - [#]() -Update charts in client to display Rapiscan and Aspect charts +Update charts in client to display Rapiscan and Aspect charts - [#]() Node Form Fix: Updated NodeForm to check if node is reachable before adding it to the list of Nodes, so when configuring a node it will ensure that you can access that node before it continues processing and updating the UI. + ## [2.2] - 2025-07-30 Release 2.2 request, no updates since 1.3.7. + ## [1.3.7] 2025-07-18 ### Added diff --git a/dist/config/standard/config.template.json b/dist/config/standard/config.template.json new file mode 100644 index 0000000..4475c30 --- /dev/null +++ b/dist/config/standard/config.template.json @@ -0,0 +1,239 @@ +[ + { + "objClass": "org.sensorhub.impl.service.HttpServerConfig", + "httpPort": 8282, + "httpsPort": 0, + "staticDocsRootUrl": "null", + "staticDocsRootDir": "null", + "servletsRootUrl": "/sensorhub", + "authMethod": "BASIC", + "keyStorePath": ".keystore/ssl_keys", + "keyAlias": "jetty", + "trustStorePath": ".keystore/ssl_trust", + "enableCORS": true, + "id": "5cb05c9c-9e08-4fa1-8731-ffaa5846bdc1", + "autoStart": true, + "moduleClass": "org.sensorhub.impl.service.HttpServer", + "name": "HTTP Server" + }, + { + "objClass": "org.sensorhub.impl.security.BasicSecurityRealmConfig", + "users": [ + { + "objClass": "org.sensorhub.impl.security.BasicSecurityRealmConfig$UserConfig", + "id": "admin", + "name": "Administrator", + "password": "__INITIAL_ADMIN_PASSWORD__", + "roles": [ + "admin" + ], + "allow": [ + "fileserver[af72442c-1ce6-4baa-a126-ed41dda26910]" + ], + "deny": [] + }, + { + "objClass": "org.sensorhub.impl.security.BasicSecurityRealmConfig$UserConfig", + "id": "anonymous", + "name": "Anonymous User", + "password": "", + "roles": [ + "anon" + ], + "allow": [], + "deny": [] + } + ], + "roles": [ + { + "objClass": "org.sensorhub.impl.security.BasicSecurityRealmConfig$RoleConfig", + "id": "admin", + "allow": [ + "*" + ], + "deny": [] + }, + { + "objClass": "org.sensorhub.impl.security.BasicSecurityRealmConfig$RoleConfig", + "id": "anon", + "allow": [ + "sos[*]/get/*" + ], + "deny": [] + } + ], + "id": "bd112969-8838-4f62-8d10-1edf1baa6669", + "autoStart": true, + "moduleClass": "org.sensorhub.impl.security.BasicSecurityRealm", + "name": "Users" + }, + { + "objClass": "org.sensorhub.ui.AdminUIConfig", + "widgetSet": "org.sensorhub.ui.SensorHubWidgetSet", + "bundleRepoUrls": [], + "customPanels": [], + "customForms": [ + { + "objClass": "org.sensorhub.ui.CustomUIConfig", + "configClass": "com.botts.impl.system.lane.config.LaneOptionsConfig", + "uiClass": "com.botts.ui.oscar.forms.LaneConfigForm" + }, + { + "objClass": "org.sensorhub.ui.CustomUIConfig", + "configClass": "org.sensorhub.api.sensor.PositionConfig.LLALocation", + "uiClass": "com.botts.ui.oscar.forms.SiteDiagramForm" + }, + { + "objClass": "org.sensorhub.ui.CustomUIConfig", + "configClass": "com.botts.impl.service.oscar.OSCARServiceConfig", + "uiClass": "com.botts.ui.oscar.forms.OSCARServiceForm" + }, + { + "objClass": "org.sensorhub.ui.CustomUIConfig", + "configClass": "com.botts.impl.service.oscar.siteinfo.SiteDiagramConfig", + "uiClass": "com.botts.ui.oscar.forms.OSCARServiceForm" + } + ], + "deploymentName": "OSCAR 3.5.0", + "enableLandingPage": false, + "id": "5cb05c9c-9123-4fa1-8731-ffaa51489678", + "autoStart": true, + "moduleClass": "org.sensorhub.ui.AdminUIModule", + "name": "Admin UI" + }, + { + "objClass": "org.sensorhub.impl.service.consys.ConSysApiServiceConfig", + "databaseID": "a445cf15-c2ab-4d92-beab-3241667a8976", + "exposedResources": { + "objClass": "org.sensorhub.impl.datastore.view.ObsSystemDatabaseViewConfig", + "sourceDatabaseId": "a445cf15-c2ab-4d92-beab-3241667a8976" + }, + "customFormats": [], + "security": { + "objClass": "org.sensorhub.api.security.SecurityConfig", + "enableAccessControl": false, + "requireAuth": true + }, + "enableTransactional": true, + "maxResponseLimit": 100000, + "defaultLiveTimeout": 600.0, + "uriPrefixMap": [], + "ogcCapabilitiesInfo": { + "objClass": "org.sensorhub.impl.service.ogc.OGCServiceConfig$CapabilitiesInfo", + "serviceProvider": { + "objClass": "org.vast.util.ResponsibleParty", + "voiceNumbers": [], + "faxNumbers": [], + "deliveryPoints": [], + "emails": [], + "hrefPresent": false + } + }, + "enableHttpGET": true, + "enableHttpPOST": true, + "enableSOAP": true, + "endPoint": "/api", + "id": "6697cb4a-2e99-4fee-bba6-d1202d24dea5", + "autoStart": true, + "moduleClass": "org.sensorhub.impl.service.consys.ConSysApiService", + "name": "Connected Systems API Service" + }, + { + "objClass": "com.botts.impl.service.oscar.OSCARServiceConfig", + "videoRetentionConfig": { + "objClass": "com.botts.impl.service.oscar.video.VideoRetentionConfig", + "timeToRetention": 7, + "videoQueryPeriod": 1, + "enableFrameRetention": true, + "frameRetentionCount": 5 + }, + "nodeId": "default", + "databaseID": "a445cf15-c2ab-4d92-beab-3241667a8976", + "statsFrequencyMinutes": 60, + "webIdApiRoot": "https://full-spectrum.sandia.gov/api/v1", + "id": "09422f61-0098-4ac2-bb0e-3f42ec470524", + "autoStart": true, + "moduleClass": "com.botts.impl.service.oscar.OSCARServiceModule", + "name": "OSCAR Service Module" + }, + { + "objClass": "com.botts.impl.service.bucket.BucketServiceConfig", + "security": { + "objClass": "org.sensorhub.api.security.SecurityConfig", + "enableAccessControl": true, + "requireAuth": true + }, + "enableCORS": true, + "initialBuckets": [ + "sitemap", + "reports", + "videos", + "adjudication" + ], + "fileStoreRootDir": "files", + "endPoint": "/buckets", + "id": "51a2a980-cf7a-4e53-9a78-fa8f32e65cbb", + "autoStart": true, + "moduleClass": "com.botts.impl.service.bucket.BucketService", + "name": "Bucket Storage Service" + }, + { + "objClass": "org.sensorhub.impl.database.system.SystemDriverDatabaseConfig", + "dbConfig": { + "objClass": "org.sensorhub.impl.datastore.postgis.database.PostgisObsSystemDatabaseConfig", + "url": "localhost:5432", + "dbName": "gis", + "login": "postgres", + "password": "postgres", + "idProviderType": "SEQUENTIAL", + "autoCommitPeriod": 10, + "useBatch": false, + "id": "bfbd6d58-1a4a-40b4-999d-381a1489cbb5", + "autoStart": false, + "moduleClass": "org.sensorhub.impl.datastore.postgis.database.PostgisObsSystemDatabase" + }, + "systemUIDs": [ + "*" + ], + "autoPurgeConfig": [], + "minCommitPeriod": 10000, + "databaseNum": 4, + "id": "a445cf15-c2ab-4d92-beab-3241667a8976", + "autoStart": true, + "moduleClass": "org.sensorhub.impl.database.system.SystemDriverDatabase", + "name": "PostGIS Database" + }, + { + "objClass": "com.botts.impl.service.fileserver.FileServerConfig", + "staticDocsRootUrl": "/", + "staticDocsRootDir": "web", + "securityConfig": { + "objClass": "org.sensorhub.api.security.SecurityConfig", + "enableAccessControl": true, + "requireAuth": true + }, + "id": "af72442c-1ce6-4baa-a126-ed41dda26910", + "autoStart": true, + "moduleClass": "com.botts.impl.service.fileserver.FileServer", + "name": "OSCAR Client" + }, + { + "objClass": "org.sensorhub.impl.service.hivemq.MqttServerConfig", + "configFolder": "hivemq-config", + "dataFolder": "hivemq-data", + "webSocketProxyEndpoint": "/mqtt", + "enableWebSocketProxy": true, + "requireAuth": true, + "id": "0a0d9999-5d16-434a-8bb7-e245b474ba1d", + "autoStart": true, + "moduleClass": "org.sensorhub.impl.service.hivemq.MqttServer", + "name": "MQTT Server (HiveMQ)" + }, + { + "objClass": "org.sensorhub.impl.service.consys.mqtt.ConSysApiMqttServiceConfig", + "id": "e7c35780-dbb6-4481-9fbe-556a7e44045d", + "autoStart": true, + "moduleClass": "org.sensorhub.impl.service.consys.mqtt.ConSysApiMqttService", + "name": "Connected Systems API MQTT Extension" + } +] diff --git a/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md b/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md index 74043be..aff79d1 100644 --- a/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md +++ b/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md @@ -16,8 +16,8 @@ | **Field** | **Value** | |-----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------| | Document type | Technical reference and operations guide | -| Baseline | OSCAR v3.5.1 packaged deployment guidance with legacy architectural background | -| Version context | Updated with OSCAR 3.5.1 deployment, monitoring, and prebuilt-release operational guidance while retaining broader architecture and feature context | +| Baseline | OSCAR v3.3.2 / 3.5 pause-point feature line | +| Version context | Centered on v3.3.2, the feature-complete build expected to become the 3.5 pause point after an internal test cycle | | Audience | Deployment engineers, operators, testers, and maintainers | | Scope | Administrative configuration, viewer/PWA, database, file handling, Web ID integration, offline conversion, reporting, scaling, and upgrade guidance | @@ -53,22 +53,6 @@ Operationally, OSCAR supports two practical deployment patterns: a default singl | Evidence model | Events can combine spectra, N42 files, uploaded evidence, QR code data, videos, Web ID results, adjudication history, and generated reports. | | Major integrations | Rapiscan, Aspect, and RSI lanes, FFmpeg-recognized camera streams, Web ID, Cambio, MQTT, and a role-aware file API. | - -## 3.5.1 packaged release note - -This manual now includes **OSCAR 3.5.1** packaged deployment guidance for the current prebuilt release workflow. - -The most important operational changes in this release line are: - -- Java 21 and Docker are required prerequisites for packaged launchers -- the launchers now validate dependencies before startup -- the top-level launch and monitor scripts detect already-running OSCAR instances -- the monitor scripts can either attach to or replace an already-running OSCAR process -- profile-based Java and PostgreSQL sizing is now part of the standard deployment flow -- MediaMTX is recommended for larger test or field deployments that reuse a smaller number of real camera streams - -For first-run field validation, the preferred workflow is to start with the monitoring wrapper and then generate a status report after the system reaches steady state. - # 2. System architecture OSCAR is best understood as a node-centric application host that serves both configuration and operator workflows while coordinating external devices and services. Lanes are the operational units. Each lane can have detector inputs, camera feeds, event history, and associated evidence. Events and statistics are persisted in PostgreSQL, while files such as videos, reports, site diagrams, CSV imports, and adjudication artifacts remain on the application host file system. @@ -102,40 +86,32 @@ The diagram below condenses the system relationships. It is not a source-code cl # 3. Installation and initial startup -Installation for the current packaged release is intentionally simple: download the **OSCAR 3.5.1** release archive, extract it into a fresh directory, verify **Java 21+** and **Docker**, create `.env`, and start the system with the OS-specific top-level launcher or monitoring wrapper. The database and application come up together in the default deployment path. - -For test and side-by-side field deployment, the preferred first start is the monitoring wrapper rather than a direct node launch. +Installation is intentionally simple: download a release archive, extract it, ensure Docker is installed and running, and launch the platform with the OS-specific launch-all script (`launch-all.bat` for Windows, `launch-all.sh` for Linux/macOS, or `launch-all-arm.sh` for ARM systems). The database and application come up together in the default deployment path. ## Prerequisites | **Item** | **Why it matters** | **Notes** | |---------------------------------|-----------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------| -| Supported host OS | The release package contains OS-specific launch scripts. | Windows and Linux are the primary packaged deployment paths discussed in the 3.5.1 workflow. | -| Java 21 or newer | Required by the packaged launch and monitoring scripts. | The launchers validate the Java version at startup and fail fast if Java is missing or too old. | -| Docker | Required for the default local PostgreSQL deployment. | Docker must already be running before `launch-all` or `monitor-oscar` is started. | +| Supported host OS | The release package contains OS-specific launch scripts. | Windows has the strongest support, although the packaging also includes scripts for other operating systems. | +| Docker | Required for the default local PostgreSQL deployment. | PostgreSQL replaces the older embedded H2 option in the current release line. | | Browser with PWA support | Needed for installed desktop/mobile viewer experiences. | Chrome-like browsers support the install flow; notifications depend on secure hosting. | | Optional SSL keystore | Needed to host the application over HTTPS. | The HTTPS configuration expects a keystore path, password, alias, and selected port. | | Optional remote PostgreSQL host | Useful for larger sites or more robust infrastructure. | Remote database connection parameters can be entered in the admin configuration. | | Optional Web ID endpoint | Enables automated isotope analysis for evidence and RS350 data. | A remote default root can be used, but the root URL is configurable for local or offline hosting. | -| Optional MediaMTX host | Recommended for larger camera-heavy systems. | MediaMTX can proxy a smaller set of upstream camera streams to many logical lane assignments. | ## Startup sequence -> 1\. Download the desired release from the repository releases page and extract it to a **fresh working directory**. If an older extracted OSCAR release is already present on the same host, stop the old OSCAR JVM, remove the old PostGIS container and any old OSCAR-specific Docker network, and delete the old extracted folder before proceeding. +> 1\. Download the desired release from the repository releases page and extract it to a working directory. > -> 2\. Verify **Java 21 or newer** and **Docker** before starting OSCAR. +> 2\. Install Docker and verify that the Docker service is running before starting OSCAR. > -> 3\. Create `.env`. In packaged releases this may mean renaming `env.txt` to `.env`. In source-style layouts this means copying `env.template` to `.env`. +> 3\. Run the launch-all script (`launch-all.bat`, `launch-all.sh`, or `launch-all-arm.sh`) for the operating system in use. In the default path, the script starts PostgreSQL locally in Docker and then starts the Java application. > -> 4\. For test and side-by-side field deployment, start with the monitoring wrapper (`monitor-oscar.bat` or `monitor-oscar.sh`). For routine startup without monitoring, use the top-level `launch-all` script. Avoid direct use of the node-level launch script unless you are debugging. +> 4\. Open the application on the configured port. Port 8282 is the baseline HTTP application port, and 8443 is a representative HTTPS configuration. > -> 5\. The launchers now validate dependencies, detect already-running OSCAR instances, and either refuse to proceed or replace the running instance depending on `FORCE_RESTART`. The monitor wrappers can also attach to an already-running OSCAR process when `ATTACH_TO_EXISTING=1` is set. +> 5\. Sign in with the initial admin account. The initial password is configurable before launch, but the exact shipped username and password should be verified against the release README or package contents. > -> 6\. Open the application on the configured port. Port 8282 is the baseline HTTP application port, and 8443 is a representative HTTPS configuration. -> -> 7\. Sign in with the initial admin account. The initial username is typically `admin`, but the exact packaged password should be verified from the release package or release notes rather than assuming a universal default. -> -> 8\. Before production use, change the initial admin password and then generate a first-run status report with `check-oscar-status` after the system reaches steady state. +> 6\. Before production use, change the initial admin password using the dot-prefixed settings or environment file and run the set-initial-admin-password script so the password is written in the hashed form expected by the system.

Operational note

-

If multiple extracted versions exist on the same machine, use the stop-all script to fully stop the application and Dockerized database for the version you were running. An older running container can accidentally populate data in the wrong directory and create confusion during configuration or tuning.
@@ -144,7 +120,7 @@ For test and side-by-side field deployment, the preferred first start is the mon +

If multiple extracted versions exist on the same machine, use the stop-all script to fully stop the application and Dockerized database for the version you were running. An older running container can accidentally populate data in the wrong directory and create confusion during configuration or tuning.

@@ -404,18 +380,6 @@ Configuration of video retention lives in the “OSCAR Service Module.” Configuration of additional users / permissions exists under the “Security” tab. Users can be configured with fine-grained permissions to access / write to the system. - -## 3.5.1 deployment operations guidance - -For packaged 3.5.1 deployments, the most practical first-run validation sequence is: - -1. launch with the monitoring wrapper -2. let the system warm up long enough to reach a steady operating state -3. generate the one-file status report -4. confirm that Java memory, thread count, and PostgreSQL session counts plateau rather than climb continuously - -This deployment workflow is especially important when evaluating reconnect churn, thread growth, or database session growth in larger simulated or camera-heavy systems. - # 9. Performance, scale, and deployment guidance Default deployment begins to strain as lane counts, event volume, and camera counts increase. The following guidance is useful for planning, even though it is not a formal support limit. @@ -519,25 +483,3 @@ This section captures known gaps and enhancement ideas so they are not confused | PWA | Progressive web app. The installable version of the viewer for desktop or mobile use. | | RS350 | A supported backpack-style mobile detector lane type. | - -# Appendix C. Packaged 3.5.1 field-deployment checklist - -> 1\. Verify Java 21+ and Docker. -> -> 2\. If an older OSCAR release was previously used on the host, stop the old OSCAR JVM, remove the old PostGIS container and old OSCAR-specific Docker network, and delete the old extracted release folder. -> -> 3\. Extract the 3.5.1 release into a fresh directory. -> -> 4\. Create `.env` from the packaged environment file and confirm the correct `SYSTEM_PROFILE`. -> -> 5\. Prefer the monitoring wrapper for first-run validation. -> -> 6\. If OSCAR is already running, decide whether to stop it, set `FORCE_RESTART=1`, or use `ATTACH_TO_EXISTING=1` for the monitor wrapper. -> -> 7\. If the deployment uses many camera references, put MediaMTX in front of the cameras and point the OSCAR lane CSV to MediaMTX paths instead of directly to every physical camera. -> -> 8\. After warm-up, generate the one-file status report and confirm memory, threads, and PostgreSQL sessions have stabilized. -> -> 9\. Change the initial admin password before production use. -> -> 10\. Keep the launch/monitoring guide and MediaMTX guide with the release package so operators follow the same tested startup path. diff --git a/dist/documentation/OSCAR_launch_monitoring_guide.md b/dist/documentation/OSCAR_launch_monitoring_guide.md index b799b2f..acc9a3a 100644 --- a/dist/documentation/OSCAR_launch_monitoring_guide.md +++ b/dist/documentation/OSCAR_launch_monitoring_guide.md @@ -6,7 +6,7 @@ It explains: - how to prepare a fresh prebuilt release - how to create `.env` -- how the updated `launch-all`, `launch`, `monitor-oscar`, and `check-oscar-status` scripts behave +- how the updated `launch-all`, `launch`, `monitor-oscar`, `stop-all`, `reset-all`, and `check-oscar-status` scripts behave - how already-running OSCAR instances are handled - how to validate Java, Docker, memory, and database health - how to use MediaMTX and status reports during testing and side-by-side field deployment @@ -68,6 +68,8 @@ Remove-Item -Recurse -Force .\oscar-3.5.0 If the Docker network does not exist, that is fine. The goal is to avoid carrying old container state or an old extracted release folder into the new test run. +For a full local reset between side-by-side test installs, use `reset-all.sh` or `reset-all.bat`. + --- ## 3. Required dependencies @@ -81,7 +83,7 @@ Required: - `keytool` - Docker -Recommended: +Recommended for monitoring: - `jcmd` - `pmap` @@ -128,6 +130,26 @@ Get-Command jcmd The Windows launchers now use **PowerShell/CIM**-based process discovery. They do not depend on `wmic`. +### Important dependency policy + +The updated scripts distinguish between **required** dependencies and **optional** runtime extras. + +Hard failures are for things such as: + +- Java +- Docker +- `keytool` where the script needs it +- required packaged files and directories such as `osh-node-oscar/lib` + +Warning-only cases include: + +- missing `.env` when defaults are available +- missing `trusted_certificates` +- missing `nativelibs` +- missing optional monitoring helpers such as `pmap` or `vmstat` + +A missing `nativelibs` directory no longer stops startup by itself. + --- ## 4. Environment file setup @@ -217,7 +239,7 @@ These are the supported top-level launchers. They: -- load `.env` +- load `.env` when present - validate Java and Docker - detect an already-running OSCAR instance - size PostgreSQL for the selected profile @@ -233,14 +255,20 @@ These launch only the OSCAR Java node. They: -- load `.env` -- validate Java and keytool +- load `.env` when present +- validate Java and `keytool` - detect an already-running OSCAR instance - choose heap and JavaCPP settings for the selected profile - build or refresh the Java trust store - initialize the packaged admin password flow - start the Java process with Native Memory Tracking enabled +Optional runtime extras are handled gracefully: + +- if `nativelibs` exists, the launcher adds `java.library.path` +- if `nativelibs` does not exist, the launcher warns and continues +- if `trusted_certificates` does not exist, the trust-store helper uses the copied default Java `cacerts` store and continues + Use these direct node launchers mainly for debugging. --- @@ -284,10 +312,15 @@ For monitor wrappers, you have two supported choices: #### Linux ```bash -chmod +x launch-all.sh osh-node-oscar/launch.sh monitor-oscar.sh check-oscar-status.sh ./monitor-oscar.sh ``` +The packaged Linux build now marks all shipped `*.sh` files executable. If your unzip tool strips execute bits, restore them with: + +```bash +chmod +x *.sh osh-node-oscar/*.sh +``` + #### Windows ```bat @@ -328,43 +361,41 @@ launch-all.bat --- -## 9. Stopping OSCAR +## 9. Stopping and resetting OSCAR -### Linux +### `stop-all` -If you started with `monitor-oscar.sh`, stop the monitor shell or the Java PID it discovered. +The `stop-all` scripts now begin by asking the monitor to stop first. -To stop a running OSCAR JVM manually: +Behavior: -```bash -pgrep -af 'com.botts.impl.security.SensorHubWrapper' -kill -``` +- attempt to stop the monitor +- do not wait indefinitely for monitor exit +- continue with direct fallback shutdown of OSCAR and PostGIS if needed -To stop PostGIS: +This avoids `stop-all` getting stuck on a monitor-closing attempt. -```bash -docker stop oscar-postgis-container -``` +### `reset-all` -### Windows PowerShell +Use `reset-all` when you want a clean local test surface before trying a different packaged installation on the same machine. -```powershell -Get-CimInstance Win32_Process | - Where-Object { - $_.Name -match '^java(\.exe)?$' -and - $_.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*' - } | - Select-Object ProcessId, CommandLine +It: -Stop-Process -Id -Force -docker stop oscar-postgis-container +- asks the monitor to stop first +- stops OSCAR Java processes +- removes the PostGIS container and volumes +- clears local runtime state used by the packaged installation + +Linux: + +```bash +./reset-all.sh ``` -The Windows monitor script also supports: +Windows: ```bat -monitor-oscar.bat stop +reset-all.bat ``` --- @@ -461,12 +492,17 @@ MediaMTX is especially helpful when many logical lane-camera assignments reuse a Check: -- `.env` exists +- `.env` exists if you intend to override defaults - Java 21+ is installed - Docker is running -- required directories such as `osh-node-oscar/lib` and `osh-node-oscar/nativelibs` exist +- required directories such as `osh-node-oscar/lib` exist - the trust store and keystore files exist where the launch script expects them +Remember: + +- missing `nativelibs` is warning-only +- missing `trusted_certificates` is warning-only if the default Java trust store can be copied successfully + ### Monitor hangs waiting for Java Check `launch.stdout.log` and `launch.stderr.log` inside the newest monitor directory. diff --git a/dist/documentation/Release_Notes_3.5.1.md b/dist/documentation/Release_Notes_3.5.1.md index 2d6f9dc..d45b603 100644 --- a/dist/documentation/Release_Notes_3.5.1.md +++ b/dist/documentation/Release_Notes_3.5.1.md @@ -27,6 +27,8 @@ Install these before running OSCAR 3.5.1: * **OpenJDK 21** * **Docker** +The packaged release archive is expected to be named **`oscar-3.5.1.zip`**. + ### Recommended deployment model For testing, side-by-side field deployment, and first-run validation: @@ -36,6 +38,7 @@ For testing, side-by-side field deployment, and first-run validation: * select the correct system profile in `.env` * use **MediaMTX** for camera-heavy deployments * start OSCAR with the **sessionless monitoring launch** when possible +* use the new reset scripts when you need to clear a previous local test install before switching releases * use the **check/status script** to review performance --- @@ -66,6 +69,7 @@ Launch scripts were updated for both Linux and Windows so they can: * provide a more consistent startup path across environments * check for required dependencies before launch * stop or refuse duplicate OSCAR launches based on script settings +* avoid hard failure on optional runtime paths such as `nativelibs` or extra trusted-certificate drop-ins when they are not present ### Safer process handling @@ -95,6 +99,8 @@ Improvements include: These changes make prebuilt deployment more reliable, especially on fresh Windows systems. +The current launcher checks now distinguish between required dependencies and optional runtime extras. Missing required tools such as Java or Docker still stop startup. Missing optional paths such as `nativelibs` or `trusted_certificates` no longer stop startup by themselves. + ### PostgreSQL tuning improvements PostgreSQL startup settings were updated to better support larger deployments. @@ -152,6 +158,16 @@ These scripts can now: * capture database activity detail * produce a single-file health and status report for rapid review +### Reset and shutdown scripts + +The deployment scripts now also support cleaner teardown between test installs. + +These updates include: + +* `stop-all` scripts that try to stop the monitor first, then continue with direct fallback shutdown +* `reset-all` scripts that stop OSCAR processes, remove the PostGIS container and volumes, and clear local runtime state for clean retesting +* better support for side-by-side installation testing on the same host + ### Improved database diagnostics Monitoring now includes PostgreSQL visibility such as: @@ -329,6 +345,8 @@ If you are unsure whether a dedicated OSCAR Docker network exists, list networks Extract OSCAR **3.5.1** into a new folder. +The packaged release archive is expected to be named `oscar-3.5.1.zip`. + Example: ```text @@ -342,6 +360,8 @@ Make sure the machine has: * **OpenJDK 21** * **Docker** +The packaged release archive is expected to be named **`oscar-3.5.1.zip`**. + ### Step 3: configure the environment file The release may include the environment file as: @@ -356,6 +376,8 @@ Rename it to: .env ``` +For Linux packaged builds, the `*.sh` files in the archive are now packaged executable. If your unzip tool strips permissions, restore them with `chmod +x *.sh osh-node-oscar/*.sh` before launching. + Then edit the file and select the correct hardware profile: * `RPI4` @@ -434,6 +456,7 @@ For testing and side-by-side field deployment, users should: 9. configure and use **MediaMTX** for camera-heavy systems 10. start OSCAR with the **sessionless monitoring launch** when possible 11. use the check or status script to compare system behavior and performance +12. use the reset script when you need to remove the local OSCAR runtime state before testing another package on the same machine This is the preferred workflow for: diff --git a/dist/release/check-oscar-status.ps1 b/dist/release/check-oscar-status.ps1 index f9f667f..22dcf66 100644 --- a/dist/release/check-oscar-status.ps1 +++ b/dist/release/check-oscar-status.ps1 @@ -1,136 +1,677 @@ param( - [string]$BaseDir = ".", - [string]$MonitorDir = "", - [string]$OutFile = "" + [string]$BaseDirectory = $PSScriptRoot, + [string]$MonitorDirectory ) -Set-StrictMode -Version Latest -$ErrorActionPreference = 'SilentlyContinue' -Set-Location $BaseDir - -if ([string]::IsNullOrWhiteSpace($MonitorDir)) { - $MonitorDir = Get-ChildItem -Directory -Filter 'oscar-monitor-*' | Sort-Object LastWriteTime -Descending | Select-Object -First 1 -ExpandProperty FullName -} -if (-not $MonitorDir -or -not (Test-Path $MonitorDir)) { throw 'No oscar-monitor-* directory found.' } -if ([string]::IsNullOrWhiteSpace($OutFile)) { $OutFile = Join-Path (Get-Location) ("oscar-status-{0}.txt" -f (Get-Date -Format 'yyyyMMdd-HHmmss')) } - -$snaps = Get-ChildItem -Path $MonitorDir -Directory | Sort-Object Name -$first = $snaps | Select-Object -First 1 -$last = $snaps | Select-Object -Last 1 -$jvmPidFile = Join-Path $MonitorDir 'jvm-pid.txt' -$pidFromMonitor = if (Test-Path $jvmPidFile) { (Get-Content $jvmPidFile | Select-Object -First 1).Trim() } else { '' } -$javaProc = Get-CimInstance Win32_Process | Where-Object { $_.Name -eq 'java.exe' -and $_.CommandLine -match 'SensorHubWrapper' } | Select-Object -First 1 - -function Read-Text([string]$Path) { if (Test-Path $Path) { Get-Content $Path -Raw } else { '' } } -function Read-First([string]$Path) { if (Test-Path $Path) { (Get-Content $Path | Select-Object -First 1).Trim() } else { '' } } -function Extract-DbCount([string]$Path, [string]$State) { - if (-not (Test-Path $Path)) { return '' } - foreach ($line in Get-Content $Path) { - $parts = $line -split '\|' - if ($parts.Count -ge 2 -and $parts[0].Trim() -eq $State) { return $parts[1].Trim() } - } - '' -} -function Calc-Slots($MaxConn, $Reserved) { - if ($MaxConn -match '^\d+$' -and $Reserved -match '^\d+$') { return [int]$MaxConn - [int]$Reserved } - '' -} - -$sb = [System.Text.StringBuilder]::new() -$null = $sb.AppendLine('OSCAR STATUS REPORT') -$null = $sb.AppendLine("Generated: $(Get-Date -Format o)") -$null = $sb.AppendLine("Base directory: $(Get-Location)") -$null = $sb.AppendLine("Monitor directory: $MonitorDir") -$null = $sb.AppendLine("Output file: $OutFile") -$null = $sb.AppendLine() -$null = $sb.AppendLine('=== PROCESS STATUS ===') -$null = $sb.AppendLine("PID from monitor: $pidFromMonitor") -$null = $sb.AppendLine(("Live OSCAR PID: {0}" -f ($(if ($javaProc) { $javaProc.ProcessId } else { '' })))) -$null = $sb.AppendLine() -$null = $sb.AppendLine((Get-CimInstance Win32_Process | Where-Object { $_.CommandLine -match 'monitor-oscar' } | Select-Object ProcessId, Name, CommandLine | Format-Table -AutoSize | Out-String)) -$null = $sb.AppendLine((Get-CimInstance Win32_Process | Where-Object { $_.Name -eq 'java.exe' -and $_.CommandLine -match 'SensorHubWrapper' } | Select-Object ProcessId, Name, CommandLine | Format-Table -AutoSize | Out-String)) -$null = $sb.AppendLine((& docker ps --filter name=oscar-postgis-container | Out-String)) -$null = $sb.AppendLine('=== SYSTEM MEMORY AND PAGEFILE ===') -$null = $sb.AppendLine((Get-CimInstance Win32_OperatingSystem | Select-Object TotalVisibleMemorySize,FreePhysicalMemory,TotalVirtualMemorySize,FreeVirtualMemory | Format-List | Out-String)) -$null = $sb.AppendLine((Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\% Usage' | Out-String)) - -if ($javaProc) { - $null = $sb.AppendLine('=== LIVE JVM PROCESS ===') - $null = $sb.AppendLine((Get-Process -Id $javaProc.ProcessId | Select-Object Id,ProcessName,Threads,VirtualMemorySize64,WorkingSet64,PrivateMemorySize64,CPU,StartTime | Format-List | Out-String)) - $null = $sb.AppendLine('=== LIVE JVM JFR STATUS ===') - $null = $sb.AppendLine((& jcmd $javaProc.ProcessId JFR.check | Out-String)) - $null = $sb.AppendLine('=== LIVE JVM GC HEAP INFO ===') - $null = $sb.AppendLine((& jcmd $javaProc.ProcessId GC.heap_info | Out-String)) - $null = $sb.AppendLine('=== LIVE JVM NATIVE MEMORY SUMMARY ===') - $null = $sb.AppendLine((& jcmd $javaProc.ProcessId VM.native_memory summary | Out-String)) -} - -$lastMax = Read-First (Join-Path $last.FullName 'db-max-connections.txt') -$lastReserved = Read-First (Join-Path $last.FullName 'db-superuser-reserved-connections.txt') -$lastTotal = Read-First (Join-Path $last.FullName 'db-total-sessions.txt') -$null = $sb.AppendLine('=== LIVE POSTGRES STATUS (FROM LAST SNAPSHOT) ===') -$null = $sb.AppendLine("max_connections: $lastMax") -$null = $sb.AppendLine("superuser_reserved_connections: $lastReserved") -$null = $sb.AppendLine("usable_client_slots: $(Calc-Slots $lastMax $lastReserved)") -$null = $sb.AppendLine("total_sessions: $lastTotal") -$null = $sb.AppendLine("active: $(Extract-DbCount (Join-Path $last.FullName 'db-by-state.txt') 'active')") -$null = $sb.AppendLine("idle: $(Extract-DbCount (Join-Path $last.FullName 'db-by-state.txt') 'idle')") -$null = $sb.AppendLine("idle in transaction: $(Extract-DbCount (Join-Path $last.FullName 'db-by-state.txt') 'idle in transaction')") -$null = $sb.AppendLine() -$null = $sb.AppendLine('--- db-by-state ---') -$null = $sb.AppendLine((Read-Text (Join-Path $last.FullName 'db-by-state.txt'))) -$null = $sb.AppendLine('--- db-by-app ---') -$null = $sb.AppendLine((Read-Text (Join-Path $last.FullName 'db-by-app.txt'))) -$null = $sb.AppendLine('--- db-error ---') -$null = $sb.AppendLine((Read-Text (Join-Path $last.FullName 'db-error.txt'))) - -$null = $sb.AppendLine('=== FIRST SNAPSHOT SUMMARY ===') -$null = $sb.AppendLine("First snapshot: $($first.FullName)") -$null = $sb.AppendLine((Read-Text (Join-Path $first.FullName 'powershell-process.txt'))) -$null = $sb.AppendLine("db total sessions: $(Read-First (Join-Path $first.FullName 'db-total-sessions.txt'))") -$null = $sb.AppendLine('=== LATEST SNAPSHOT SUMMARY ===') -$null = $sb.AppendLine("Latest snapshot: $($last.FullName)") -$null = $sb.AppendLine((Read-Text (Join-Path $last.FullName 'powershell-process.txt'))) -$null = $sb.AppendLine("db total sessions: $(Read-First (Join-Path $last.FullName 'db-total-sessions.txt'))") - -$null = $sb.AppendLine('=== RECENT TREND (LAST 20 SNAPSHOTS) ===') -foreach ($d in ($snaps | Select-Object -Last 20)) { - $dbTotal = Read-First (Join-Path $d.FullName 'db-total-sessions.txt') - $dbMax = Read-First (Join-Path $d.FullName 'db-max-connections.txt') - $dbReserved = Read-First (Join-Path $d.FullName 'db-superuser-reserved-connections.txt') - $dbActive = Extract-DbCount (Join-Path $d.FullName 'db-by-state.txt') 'active' - $dbIdle = Extract-DbCount (Join-Path $d.FullName 'db-by-state.txt') 'idle' - $proc = (Read-Text (Join-Path $d.FullName 'powershell-process.txt')) -replace '\r?\n',' ' - $null = $sb.AppendLine("$($d.Name) $proc db_total=$dbTotal db_active=$dbActive db_idle=$dbIdle db_slots=$(Calc-Slots $dbMax $dbReserved)") -} -$null = $sb.AppendLine() - -$dbCsv = Join-Path $MonitorDir 'db-connection-trend.csv' -if (Test-Path $dbCsv) { - $null = $sb.AppendLine('=== DB CONNECTION TREND CSV (LAST 40 LINES) ===') - $null = $sb.AppendLine(((Get-Content $dbCsv | Select-Object -Last 40) -join [Environment]::NewLine)) - $null = $sb.AppendLine() -} - -$null = $sb.AppendLine('=== LOG TAILS ===') -$null = $sb.AppendLine('--- launch.stdout.log (last 50 lines) ---') -$null = $sb.AppendLine(((Get-Content (Join-Path $MonitorDir 'launch.stdout.log') -Tail 50) -join [Environment]::NewLine)) -$null = $sb.AppendLine() -$null = $sb.AppendLine('--- launch.stderr.log (last 50 lines) ---') -$null = $sb.AppendLine(((Get-Content (Join-Path $MonitorDir 'launch.stderr.log') -Tail 50) -join [Environment]::NewLine)) -$null = $sb.AppendLine() -$null = $sb.AppendLine('--- postgres docker logs (last captured 100 lines) ---') -$null = $sb.AppendLine(((Get-Content (Join-Path $last.FullName 'docker-logs-tail.txt') -Tail 100) -join [Environment]::NewLine)) -$null = $sb.AppendLine() - -$null = $sb.AppendLine('=== QUICK READ ===') -$null = $sb.AppendLine("First DB total sessions: $(Read-First (Join-Path $first.FullName 'db-total-sessions.txt'))") -$null = $sb.AppendLine("Latest DB total sessions: $(Read-First (Join-Path $last.FullName 'db-total-sessions.txt'))") -$null = $sb.AppendLine("Latest DB usable client slots: $(Calc-Slots $lastMax $lastReserved)") -$null = $sb.AppendLine('Interpretation guide:') -$null = $sb.AppendLine('- Healthy memory: process memory and JVM native memory plateau.') -$null = $sb.AppendLine('- Healthy DB: total sessions rise at startup and then plateau well below usable client slots.') -$null = $sb.AppendLine('- Suspicious DB: total sessions keep climbing, idle sessions pile up, or db-error shows too many clients already.') - -[System.IO.File]::WriteAllText($OutFile, $sb.ToString()) -Write-Host "Wrote report to: $OutFile" +$ErrorActionPreference = "SilentlyContinue" + +function Add-Line { + param( + [System.Collections.Generic.List[string]]$Lines, + [string]$Text = "" + ) + $Lines.Add($Text) | Out-Null +} + +function Add-Block { + param( + [System.Collections.Generic.List[string]]$Lines, + [string]$Text + ) + + if ([string]::IsNullOrWhiteSpace($Text)) { + $Lines.Add("") | Out-Null + return + } + + $normalized = $Text -replace "`r`n", "`n" + foreach ($line in ($normalized -split "`n")) { + $Lines.Add($line) | Out-Null + } +} + +function Load-DotEnv { + param([string]$Path) + + $map = @{} + if (-not (Test-Path -LiteralPath $Path)) { + return $map + } + + foreach ($rawLine in Get-Content -LiteralPath $Path) { + $line = $rawLine.Trim() + if ([string]::IsNullOrWhiteSpace($line)) { continue } + if ($line.StartsWith("#")) { continue } + + if ($line.StartsWith("export ")) { + $line = $line.Substring(7).Trim() + } + + $idx = $line.IndexOf("=") + if ($idx -lt 1) { continue } + + $key = $line.Substring(0, $idx).Trim() + $value = $line.Substring($idx + 1) + + if (($value.StartsWith('"') -and $value.EndsWith('"')) -or ($value.StartsWith("'") -and $value.EndsWith("'"))) { + if ($value.Length -ge 2) { + $value = $value.Substring(1, $value.Length - 2) + } + } + + $map[$key] = $value + } + + return $map +} + +function Get-ActiveMonitorDirectory { + param([string]$BaseDir) + + $activePath = Join-Path $BaseDir ".monitor-active-dir" + if (-not (Test-Path -LiteralPath $activePath)) { + return $null + } + + $candidate = (Get-Content -LiteralPath $activePath -TotalCount 1 | Out-String).Trim() + if ([string]::IsNullOrWhiteSpace($candidate)) { + return $null + } + + if (Test-Path -LiteralPath $candidate) { + return Get-Item -LiteralPath $candidate + } + + return $null +} + +function Get-LatestMonitorDirectory { + param([string]$BaseDir) + + $dirs = Get-ChildItem -LiteralPath $BaseDir -Directory | + Where-Object { $_.Name -like "oscar-monitor-*" } | + Sort-Object Name -Descending + + return ($dirs | Select-Object -First 1) +} + +function Get-OscarJavaProcesses { + $procs = Get-CimInstance Win32_Process | + Where-Object { + $_.Name -match '^(java|javaw)(\.exe)?$' -and + $null -ne $_.CommandLine -and + $_.CommandLine -match 'com\.botts\.impl\.security\.SensorHubWrapper' + } | + Sort-Object ProcessId + + return @($procs) +} + +function Resolve-ToolPath { + param([string]$Name) + + $cmd = Get-Command $Name -ErrorAction SilentlyContinue + if ($cmd -and $cmd.Source) { + return $cmd.Source + } + + $whereExe = Get-Command where.exe -ErrorAction SilentlyContinue + if ($whereExe) { + try { + $resolved = & $whereExe.Source $Name 2>$null | Select-Object -First 1 + if ($resolved -and (Test-Path -LiteralPath $resolved)) { + return $resolved + } + } + catch { + } + } + + return $null +} + +function Resolve-JcmdPath { + $jcmd = Resolve-ToolPath -Name "jcmd.exe" + if ($jcmd) { + return $jcmd + } + + $jcmd = Resolve-ToolPath -Name "jcmd" + if ($jcmd) { + return $jcmd + } + + if ($env:JAVA_HOME) { + $candidate = Join-Path $env:JAVA_HOME "bin\jcmd.exe" + if (Test-Path -LiteralPath $candidate) { + return $candidate + } + } + + $javaCmd = Resolve-ToolPath -Name "java.exe" + if (-not $javaCmd) { + $javaCmd = Resolve-ToolPath -Name "java" + } + + if ($javaCmd) { + $javaDir = Split-Path -Parent $javaCmd + $candidate = Join-Path $javaDir "jcmd.exe" + if (Test-Path -LiteralPath $candidate) { + return $candidate + } + } + + return $null +} + +function Invoke-ExternalCapture { + param( + [string]$FilePath, + [string[]]$Arguments + ) + + $script:LastExternalExitCode = 0 + + if ([string]::IsNullOrWhiteSpace($FilePath)) { + $script:LastExternalExitCode = 1 + return "Tool path is empty." + } + + if (-not (Test-Path -LiteralPath $FilePath)) { + $script:LastExternalExitCode = 1 + return "Tool not found: $FilePath" + } + + try { + $result = & $FilePath @Arguments 2>&1 | Out-String -Width 4096 + $exitCode = $LASTEXITCODE + if ($null -eq $exitCode) { $exitCode = 0 } + $script:LastExternalExitCode = $exitCode + + $trimmed = $result.TrimEnd() + if ([string]::IsNullOrWhiteSpace($trimmed) -and $exitCode -ne 0) { + return "Command failed with exit code $exitCode and returned no output." + } + + return $trimmed + } + catch { + $script:LastExternalExitCode = 1 + return ($_ | Out-String).TrimEnd() + } +} + +function Get-DockerContainerRecord { + param( + [string]$DockerExe, + [string]$ContainerName + ) + + if (-not $DockerExe) { + return $null + } + + $raw = Invoke-ExternalCapture -FilePath $DockerExe -Arguments @( + "ps", "-a", + "--format", "{{.ID}}|{{.Image}}|{{.Status}}|{{.Names}}|{{.Ports}}|{{.Command}}" + ) + + if ($script:LastExternalExitCode -ne 0) { + return @{ + Error = $raw + } + } + + $lines = @($raw -split "`r?`n" | Where-Object { $_.Trim().Length -gt 0 }) + foreach ($line in $lines) { + $parts = $line.Split("|") + if ($parts.Count -ge 6 -and $parts[3] -eq $ContainerName) { + return @{ + Id = $parts[0] + Image = $parts[1] + Status = $parts[2] + Name = $parts[3] + Ports = $parts[4] + Command = $parts[5] + } + } + } + + return $null +} + +function Get-DockerTableText { + param($ContainerRecord) + + if ($null -eq $ContainerRecord) { + return "Container not found." + } + + if ($ContainerRecord.ContainsKey("Error")) { + return $ContainerRecord.Error + } + + return @" +CONTAINER ID IMAGE STATUS PORTS NAMES +$($ContainerRecord.Id) $($ContainerRecord.Image) $($ContainerRecord.Status) $($ContainerRecord.Ports) $($ContainerRecord.Name) +"@.TrimEnd() +} + +function Invoke-PsqlInContainer { + param( + [string]$DockerExe, + [string]$ContainerName, + [string]$DbUser, + [string]$DbName, + [string]$DbPassword, + [string]$Sql + ) + + return Invoke-ExternalCapture -FilePath $DockerExe -Arguments @( + "exec", + "-e", "PGPASSWORD=$DbPassword", + $ContainerName, + "psql", + "-U", $DbUser, + "-d", $DbName, + "-At", + "-c", $Sql + ) +} + +function Get-LaunchTail { + param( + [string]$MonitorDir, + [string]$FileName, + [int]$Tail = 50 + ) + + if ([string]::IsNullOrWhiteSpace($MonitorDir)) { return "" } + + $path = Join-Path $MonitorDir $FileName + if (-not (Test-Path -LiteralPath $path)) { + return "" + } + + return (Get-Content -LiteralPath $path -Tail $Tail | Out-String -Width 4096).TrimEnd() +} + +function Run-JcmdSection { + param( + [string]$JcmdExe, + [string]$Pid, + [string[]]$Args + ) + + if (-not $Pid -or $Pid -notmatch '^\d+$') { + return "No live OSCAR JVM found." + } + + if (-not $JcmdExe) { + return "jcmd.exe not found." + } + + if (-not (Test-Path -LiteralPath $JcmdExe)) { + return "jcmd.exe path does not exist: $JcmdExe" + } + + $stdoutFile = [System.IO.Path]::GetTempFileName() + $stderrFile = [System.IO.Path]::GetTempFileName() + + try { + $proc = Start-Process ` + -FilePath $JcmdExe ` + -ArgumentList (@($Pid) + $Args) ` + -NoNewWindow ` + -Wait ` + -PassThru ` + -RedirectStandardOutput $stdoutFile ` + -RedirectStandardError $stderrFile + + $stdout = "" + $stderr = "" + + if (Test-Path -LiteralPath $stdoutFile) { + $stdout = Get-Content -LiteralPath $stdoutFile -Raw + } + + if (Test-Path -LiteralPath $stderrFile) { + $stderr = Get-Content -LiteralPath $stderrFile -Raw + } + + $output = ($stdout + $stderr).TrimEnd() + $exitCode = $proc.ExitCode + + if ($exitCode -ne 0) { + if ([string]::IsNullOrWhiteSpace($output)) { + return "jcmd failed with exit code $exitCode using: $JcmdExe $Pid $($Args -join ' ')" + } + return $output + } + + if ([string]::IsNullOrWhiteSpace($output)) { + return "jcmd returned no output using: $JcmdExe $Pid $($Args -join ' ')" + } + + return $output + } + catch { + return ($_ | Out-String).TrimEnd() + } + finally { + Remove-Item -LiteralPath $stdoutFile -Force -ErrorAction SilentlyContinue + Remove-Item -LiteralPath $stderrFile -Force -ErrorAction SilentlyContinue + } +} + +$envMap = Load-DotEnv -Path (Join-Path $BaseDirectory ".env") + +$containerName = if ($envMap.ContainsKey("CONTAINER_NAME")) { $envMap["CONTAINER_NAME"] } else { "oscar-postgis-container" } +$dbUser = if ($envMap.ContainsKey("DB_USER")) { $envMap["DB_USER"] } else { "postgres" } +$dbName = if ($envMap.ContainsKey("DB_NAME")) { $envMap["DB_NAME"] } else { "gis" } +$dbPassword = if ($envMap.ContainsKey("DB_PASSWORD")) { $envMap["DB_PASSWORD"] } else { "postgres" } + +$monitorDirItem = $null +if (-not [string]::IsNullOrWhiteSpace($MonitorDirectory)) { + if (Test-Path -LiteralPath $MonitorDirectory) { + $monitorDirItem = Get-Item -LiteralPath $MonitorDirectory + } +} +else { + $monitorDirItem = Get-ActiveMonitorDirectory -BaseDir $BaseDirectory + if ($null -eq $monitorDirItem) { + $monitorDirItem = Get-LatestMonitorDirectory -BaseDir $BaseDirectory + } +} + +$monitorDir = if ($null -ne $monitorDirItem) { $monitorDirItem.FullName } else { "" } + +$timestamp = Get-Date -Format "yyyyMMdd-HHmmss" +$outputFile = Join-Path $BaseDirectory "oscar-status-$timestamp.txt" + +$pidFromMonitor = "" +if ($monitorDir) { + $pidPath = Join-Path $monitorDir "jvm-pid.txt" + if (Test-Path -LiteralPath $pidPath) { + $pidFromMonitor = (Get-Content -LiteralPath $pidPath -TotalCount 1 | Out-String).Trim() + } +} + +$oscarJava = Get-OscarJavaProcesses +$liveProc = $null + +if ($pidFromMonitor -match '^\d+$') { + $liveProc = $oscarJava | Where-Object { $_.ProcessId -eq [int]$pidFromMonitor } | Select-Object -First 1 +} +if ($null -eq $liveProc) { + $liveProc = $oscarJava | Select-Object -First 1 +} + +$livePid = if ($null -ne $liveProc) { [string]$liveProc.ProcessId } else { "" } + +$dockerExe = Resolve-ToolPath -Name "docker" +$jcmdExe = Resolve-JcmdPath + +$dockerContainer = Get-DockerContainerRecord -DockerExe $dockerExe -ContainerName $containerName +$dockerTableText = Get-DockerTableText -ContainerRecord $dockerContainer + +$containerRunning = $false +if ($dockerContainer -and -not $dockerContainer.ContainsKey("Error")) { + if ($dockerContainer.Status -like "Up*") { + $containerRunning = $true + } +} + +$osInfo = Get-CimInstance Win32_OperatingSystem | + Select-Object TotalVisibleMemorySize, FreePhysicalMemory, TotalVirtualMemorySize, FreeVirtualMemory +$osInfoText = ($osInfo | Format-List | Out-String -Width 4096).TrimEnd() + +$counterText = "" +try { + $counters = Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\% Usage' + $counterText = ($counters | Out-String -Width 4096).TrimEnd() +} +catch { + $counterText = "Could not read performance counters." +} + +$liveJvmText = "" +if ($livePid -match '^\d+$') { + $liveJvmText = (Get-Process -Id ([int]$livePid) | + Select-Object Id, ProcessName, Threads, VirtualMemorySize64, WorkingSet64, PrivateMemorySize64, CPU, StartTime | + Format-List | Out-String -Width 4096).TrimEnd() +} +else { + $liveJvmText = "No live OSCAR JVM found." +} + +$jfrText = "" +$heapText = "" +$nmtText = "" + +if ($livePid -match '^\d+$' -and $jcmdExe -and (Test-Path -LiteralPath $jcmdExe)) { + try { + $jfrText = (& $jcmdExe $livePid JFR.check 2>&1 | Out-String -Width 4096).TrimEnd() + if ([string]::IsNullOrWhiteSpace($jfrText)) { + $jfrText = "jcmd returned no output for JFR.check" + } + } + catch { + $jfrText = ($_ | Out-String).TrimEnd() + } + + try { + $heapText = (& $jcmdExe $livePid GC.heap_info 2>&1 | Out-String -Width 4096).TrimEnd() + if ([string]::IsNullOrWhiteSpace($heapText)) { + $heapText = "jcmd returned no output for GC.heap_info" + } + } + catch { + $heapText = ($_ | Out-String).TrimEnd() + } + + try { + $nmtText = (& $jcmdExe $livePid VM.native_memory summary 2>&1 | Out-String -Width 4096).TrimEnd() + if ([string]::IsNullOrWhiteSpace($nmtText)) { + $nmtText = "jcmd returned no output for VM.native_memory summary" + } + } + catch { + $nmtText = ($_ | Out-String).TrimEnd() + } +} +else { + $jfrText = "jcmd.exe not found or no live OSCAR JVM found." + $heapText = $jfrText + $nmtText = $jfrText +} + +$dbMetaText = "" +$dbByStateText = "" +$dbByAppText = "" +$dbErrorText = "" + +$maxConnections = "" +$superuserReservedConnections = "" +$usableClientSlots = "" +$totalSessions = "" +$activeSessions = "" +$idleSessions = "" +$idleInTransaction = "" + +if (-not $dockerExe) { + $dbErrorText = "docker.exe not found in PATH." +} +elseif ($null -eq $dockerContainer) { + $dbErrorText = "Container '$containerName' not found." +} +elseif ($dockerContainer.ContainsKey("Error")) { + $dbErrorText = $dockerContainer.Error +} +elseif (-not $containerRunning) { + $dbErrorText = "Container '$containerName' is present but not running. Status: $($dockerContainer.Status)" +} +else { + $dbMetaSql = "select current_setting('max_connections'), current_setting('superuser_reserved_connections'), (current_setting('max_connections')::int - current_setting('superuser_reserved_connections')::int), count(*), count(*) filter (where state = 'active'), count(*) filter (where state = 'idle'), count(*) filter (where state = 'idle in transaction') from pg_stat_activity;" + $dbByStateSql = "select coalesce(state,'') || '|' || count(*)::text from pg_stat_activity group by state order by 1;" + $dbByAppSql = "select coalesce(application_name,'') || '|' || coalesce(usename,'') || '|' || coalesce(client_addr::text,'') || '|' || coalesce(state,'') || '|' || count(*)::text from pg_stat_activity group by application_name, usename, client_addr, state order by application_name, usename, client_addr, state;" + + $dbMetaText = Invoke-PsqlInContainer -DockerExe $dockerExe -ContainerName $containerName -DbUser $dbUser -DbName $dbName -DbPassword $dbPassword -Sql $dbMetaSql + $dbByStateText = Invoke-PsqlInContainer -DockerExe $dockerExe -ContainerName $containerName -DbUser $dbUser -DbName $dbName -DbPassword $dbPassword -Sql $dbByStateSql + $dbByAppText = Invoke-PsqlInContainer -DockerExe $dockerExe -ContainerName $containerName -DbUser $dbUser -DbName $dbName -DbPassword $dbPassword -Sql $dbByAppSql + + $metaLine = ($dbMetaText -split "`r?`n" | Where-Object { $_.Trim().Length -gt 0 } | Select-Object -First 1) + if ($metaLine -and $metaLine.Contains("|")) { + $parts = $metaLine.Split("|") + if ($parts.Count -ge 7) { + $maxConnections = $parts[0] + $superuserReservedConnections = $parts[1] + $usableClientSlots = $parts[2] + $totalSessions = $parts[3] + $activeSessions = $parts[4] + $idleSessions = $parts[5] + $idleInTransaction = $parts[6] + } + } + else { + if ([string]::IsNullOrWhiteSpace($dbMetaText)) { + $dbErrorText = "psql returned no DB metadata output." + } + else { + $dbErrorText = $dbMetaText + } + } +} + +$snapshotDirs = @() +if ($monitorDir -and (Test-Path -LiteralPath $monitorDir)) { + $snapshotDirs = Get-ChildItem -LiteralPath $monitorDir -Directory | + Where-Object { $_.Name -match '^\d{8}-\d{6}$' } | + Sort-Object Name +} + +$firstSnapshot = if ($snapshotDirs.Count -gt 0) { $snapshotDirs[0].FullName } else { "" } +$latestSnapshot = if ($snapshotDirs.Count -gt 0) { $snapshotDirs[-1].FullName } else { "" } + +$recentSnapshotLines = @() +if ($snapshotDirs.Count -gt 0) { + $recentSnapshotLines = $snapshotDirs | + Select-Object -Last ([Math]::Min(20, $snapshotDirs.Count)) | + ForEach-Object { $_.Name } +} + +$launchStdoutTail = Get-LaunchTail -MonitorDir $monitorDir -FileName "launch.stdout.log" -Tail 50 +$launchStderrTail = Get-LaunchTail -MonitorDir $monitorDir -FileName "launch.stderr.log" -Tail 50 + +$dockerLogsTail = "" +if ($dockerExe -and $containerRunning) { + $dockerLogsTail = Invoke-ExternalCapture -FilePath $dockerExe -Arguments @("logs", "--tail", "100", $containerName) +} + +$procTableText = "" +if ($liveProc) { + $procTableText = ($liveProc | + Select-Object ProcessId, Name, CommandLine | + Format-Table -AutoSize | Out-String -Width 4096).TrimEnd() +} +else { + $procTableText = "No live OSCAR Java process found." +} + +$lines = New-Object 'System.Collections.Generic.List[string]' + +Add-Line $lines "OSCAR STATUS REPORT" +Add-Line $lines ("Generated: " + (Get-Date).ToString("o")) +Add-Line $lines ("Base directory: " + $BaseDirectory) +Add-Line $lines ("Monitor directory: " + $(if ($monitorDir) { $monitorDir } else { "" })) +Add-Line $lines ("Output file: " + $outputFile) +Add-Line $lines "" + +Add-Line $lines "=== PROCESS STATUS ===" +Add-Line $lines ("PID from monitor: " + $pidFromMonitor) +Add-Line $lines ("Live OSCAR PID: " + $livePid) +Add-Line $lines "" +Add-Block $lines $procTableText +Add-Line $lines "" +Add-Block $lines $dockerTableText +Add-Line $lines "" + +Add-Line $lines "=== SYSTEM MEMORY AND PAGEFILE ===" +Add-Line $lines "" +Add-Block $lines $osInfoText +Add-Line $lines "" +Add-Block $lines $counterText +Add-Line $lines "" + +Add-Line $lines "=== LIVE JVM PROCESS ===" +Add-Line $lines "" +Add-Block $lines $liveJvmText +Add-Line $lines "" + +Add-Line $lines "=== LIVE JVM JFR STATUS ===" +Add-Block $lines $jfrText +Add-Line $lines "" + +Add-Line $lines "=== LIVE JVM GC HEAP INFO ===" +Add-Block $lines $heapText +Add-Line $lines "" + +Add-Line $lines "=== LIVE JVM NATIVE MEMORY SUMMARY ===" +Add-Block $lines $nmtText +Add-Line $lines "" + +Add-Line $lines "=== LIVE POSTGRES STATUS ===" +Add-Line $lines ("max_connections: " + $maxConnections) +Add-Line $lines ("superuser_reserved_connections: " + $superuserReservedConnections) +Add-Line $lines ("usable_client_slots: " + $usableClientSlots) +Add-Line $lines ("total_sessions: " + $totalSessions) +Add-Line $lines ("active: " + $activeSessions) +Add-Line $lines ("idle: " + $idleSessions) +Add-Line $lines ("idle in transaction: " + $idleInTransaction) +Add-Line $lines "" +Add-Line $lines "--- db-by-state ---" +Add-Block $lines $dbByStateText +Add-Line $lines "" +Add-Line $lines "--- db-by-app ---" +Add-Block $lines $dbByAppText +Add-Line $lines "" +Add-Line $lines "--- db-error ---" +Add-Block $lines $dbErrorText +Add-Line $lines "" + +Add-Line $lines "=== SNAPSHOT STATUS ===" +Add-Line $lines ("Snapshot count: " + $snapshotDirs.Count) +Add-Line $lines ("First snapshot: " + $firstSnapshot) +Add-Line $lines ("Latest snapshot: " + $latestSnapshot) +Add-Line $lines "" + +Add-Line $lines "=== RECENT SNAPSHOTS (LAST 20) ===" +foreach ($snap in $recentSnapshotLines) { + Add-Line $lines $snap +} +Add-Line $lines "" + +Add-Line $lines "=== LOG TAILS ===" +Add-Line $lines "--- launch.stdout.log (last 50 lines) ---" +Add-Block $lines $launchStdoutTail +Add-Line $lines "" +Add-Line $lines "--- launch.stderr.log (last 50 lines) ---" +Add-Block $lines $launchStderrTail +Add-Line $lines "" +Add-Line $lines "--- postgres docker logs (last captured 100 lines) ---" +Add-Block $lines $dockerLogsTail +Add-Line $lines "" + +Add-Line $lines "=== QUICK READ ===" +Add-Line $lines ("Live JVM PID: " + $livePid) +Add-Line $lines ("Snapshots captured: " + $snapshotDirs.Count) +if ($totalSessions) { Add-Line $lines ("DB total sessions: " + $totalSessions) } +if ($usableClientSlots) { Add-Line $lines ("DB usable client slots: " + $usableClientSlots) } +Add-Line $lines "Interpretation guide:" +Add-Line $lines "- Healthy memory: process memory and JVM native memory plateau." +Add-Line $lines "- Healthy DB: total sessions rise at startup and then plateau well below usable client slots." +Add-Line $lines "- Suspicious DB: total sessions keep climbing, idle sessions pile up, or db-error shows query failures." + +$reportText = ($lines -join "`r`n") +Set-Content -LiteralPath $outputFile -Value $reportText -Encoding UTF8 +Write-Output $reportText \ No newline at end of file diff --git a/dist/release/check-oscar-status.sh b/dist/release/check-oscar-status.sh old mode 100644 new mode 100755 diff --git a/dist/release/env.template b/dist/release/env.template index f1c4058..6d3a356 100644 --- a/dist/release/env.template +++ b/dist/release/env.template @@ -40,7 +40,7 @@ RETRY_MAX=120 RETRY_INTERVAL=2 # Extra delay after PostGIS reports ready -POSTGIS_READY_DELAY=5 +POSTGIS_READY_DELAY=60 # --- OPTIONAL MEMORY / DIAGNOSTICS OVERRIDES --- # Leave blank to use profile defaults from launch scripts diff --git a/dist/release/launch-all-arm.sh b/dist/release/launch-all-arm.sh old mode 100644 new mode 100755 index 488215b..cacf672 --- a/dist/release/launch-all-arm.sh +++ b/dist/release/launch-all-arm.sh @@ -43,33 +43,6 @@ require_cmd() { fi } -get_java_major() { - java -version 2>&1 | awk -F'"' '/version/ { split($2, v, "."); print v[1]; exit }' -} - -check_dependencies() { - require_cmd bash - require_cmd java - require_cmd keytool - require_cmd docker - - if ! docker info >/dev/null 2>&1; then - echo "Error: Docker is installed, but the Docker daemon is not running." - exit 1 - fi - - local java_major - java_major="$(get_java_major || true)" - if [[ -z "$java_major" || ! "$java_major" =~ ^[0-9]+$ ]]; then - echo "Error: could not determine Java version. Java 21 or newer is required." - exit 1 - fi - if [ "$java_major" -lt 21 ]; then - echo "Error: Java 21 or newer is required. Found Java $java_major." - exit 1 - fi -} - find_existing_oscar_pids() { pgrep -f "$MATCH_EXPR" || true } @@ -121,15 +94,6 @@ check_existing_oscar() { exit 1 } -require_env() { - local name="$1" - local value="${!name:-}" - if [ -z "$value" ]; then - echo "Error: ${name} is not set in .env." - exit 1 - fi -} - require_number() { local name="$1" local value="${!name:-}" @@ -165,27 +129,35 @@ ensure_project_layout() { mkdir -p "$PROJECT_DIR/pgdata" } -if [ ! -f "$ENV_FILE" ]; then - echo "Error: .env file not found in $PROJECT_DIR" - echo "Create it by copying env.template to .env and editing the values." - exit 1 +if [ -f "$ENV_FILE" ]; then + load_env "$ENV_FILE" +else + echo "Warning: .env file not found in $PROJECT_DIR" + if [ -f "$PROJECT_DIR/env.template" ]; then + echo "Warning: using built-in defaults. Copy env.template to .env to customize settings." + else + echo "Warning: using built-in defaults." + fi fi -load_env "$ENV_FILE" -check_dependencies -check_existing_oscar -ensure_project_layout - SYSTEM_PROFILE="${SYSTEM_PROFILE:-8GB}" CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" +DB_NAME="${DB_NAME:-gis}" +DB_USER="${DB_USER:-postgres}" +DB_PASSWORD="${DB_PASSWORD:-postgres}" +DB_PORT="${DB_PORT:-5432}" DB_HOST="${DB_HOST:-localhost}" -export SYSTEM_PROFILE CONTAINER_NAME DB_HOST RETRY_MAX RETRY_INTERVAL POSTGIS_READY_DELAY IMAGE_NAME POSTGIS_DOCKERFILE POSTGIS_PLATFORM +export SYSTEM_PROFILE CONTAINER_NAME DB_NAME DB_USER DB_PASSWORD DB_PORT DB_HOST +export RETRY_MAX RETRY_INTERVAL POSTGIS_READY_DELAY IMAGE_NAME POSTGIS_DOCKERFILE POSTGIS_PLATFORM -require_env DB_NAME -require_env DB_USER -require_env DB_PASSWORD -require_env DB_PORT +require_cmd docker +if ! docker info >/dev/null 2>&1; then + echo "Error: Docker is installed, but the Docker daemon is not running." + exit 1 +fi +check_existing_oscar +ensure_project_layout require_number DB_PORT require_number RETRY_MAX require_number RETRY_INTERVAL @@ -233,9 +205,7 @@ case "${SYSTEM_PROFILE^^}" in PG_MAINT="128MB" PG_MAX_CONN="125" ;; - esac - -export SYSTEM_PROFILE CONTAINER_NAME DB_NAME DB_USER DB_PASSWORD DB_PORT +esac echo "Building PostGIS Docker image for Apple Silicon / ARM64..." ( diff --git a/dist/release/launch-all-arm_old.sh b/dist/release/launch-all-arm_old.sh deleted file mode 100644 index 358cdd0..0000000 --- a/dist/release/launch-all-arm_old.sh +++ /dev/null @@ -1,247 +0,0 @@ -#!/bin/bash -set -euo pipefail - -# Apple Silicon / ARM64 launcher for the full OSH + PostGIS stack. -# Resolves paths from this script's location, not from the caller's cwd. - -SOURCE="${BASH_SOURCE[0]}" -while [ -h "$SOURCE" ]; do - SOURCE_DIR="$(CDPATH= cd -P "$(dirname "$SOURCE")" >/dev/null 2>&1 && pwd)" - SOURCE="$(readlink "$SOURCE")" - case "$SOURCE" in - /*) ;; - *) SOURCE="${SOURCE_DIR}/${SOURCE}" ;; - esac -done -PROJECT_DIR="$(CDPATH= cd -P "$(dirname "$SOURCE")" >/dev/null 2>&1 && pwd)" -ENV_FILE="${PROJECT_DIR}/.env" - -IMAGE_NAME="${POSTGIS_IMAGE_NAME:-${IMAGE_NAME:-oscar-postgis-arm}}" -POSTGIS_DOCKERFILE="${POSTGIS_DOCKERFILE:-Dockerfile-arm64}" -POSTGIS_PLATFORM="${POSTGIS_PLATFORM:-linux/arm64}" - -if [ ! -f "$ENV_FILE" ]; then - echo "Error: .env file not found in ${PROJECT_DIR}." - echo "Create it by copying env.template to .env and editing the values." - exit 1 -fi - -# Export values from .env so osh-node-oscar/launch.sh can use the same settings. -set -a -# shellcheck disable=SC1090 -. "$ENV_FILE" -set +a - -# Remove a possible CR from values if .env was edited on Windows. -strip_cr_var() { - _name="$1" - eval "_value=\${${_name}-}" - _value="${_value%$'\r'}" - export "${_name}=${_value}" -} - -for _var in \ - SYSTEM_PROFILE DB_NAME DB_USER DB_PASSWORD DB_PORT DB_HOST CONTAINER_NAME \ - KEYSTORE_PASSWORD TRUSTSTORE_PASSWORD JAVACPP_MAX_BYTES \ - JAVACPP_MAX_PHYSICAL_BYTES JFR_FILENAME RETRY_MAX RETRY_INTERVAL \ - POSTGIS_READY_DELAY IMAGE_NAME POSTGIS_IMAGE_NAME POSTGIS_DOCKERFILE POSTGIS_PLATFORM - do - strip_cr_var "$_var" -done -unset _var - -SYSTEM_PROFILE="${SYSTEM_PROFILE:-8GB}" -CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" -DB_HOST="${DB_HOST:-localhost}" -RETRY_MAX="${RETRY_MAX:-120}" -RETRY_INTERVAL="${RETRY_INTERVAL:-2}" -POSTGIS_READY_DELAY="${POSTGIS_READY_DELAY:-5}" -IMAGE_NAME="${POSTGIS_IMAGE_NAME:-${IMAGE_NAME:-oscar-postgis-arm}}" -POSTGIS_DOCKERFILE="${POSTGIS_DOCKERFILE:-Dockerfile-arm64}" -POSTGIS_PLATFORM="${POSTGIS_PLATFORM:-linux/arm64}" -export SYSTEM_PROFILE CONTAINER_NAME DB_HOST RETRY_MAX RETRY_INTERVAL POSTGIS_READY_DELAY -export IMAGE_NAME POSTGIS_DOCKERFILE POSTGIS_PLATFORM - -require_env() { - _name="$1" - eval "_value=\${${_name}:-}" - if [ -z "$_value" ]; then - echo "Error: ${_name} is not set in .env." - exit 1 - fi -} - -require_env DB_NAME -require_env DB_USER -require_env DB_PASSWORD -require_env DB_PORT - -require_number() { - _name="$1" - eval "_value=\${${_name}:-}" - case "$_value" in - *[!0-9]*|'') - echo "Error: ${_name} must be a number, got '${_value}'." - exit 1 - ;; - esac -} - -require_number DB_PORT -require_number RETRY_MAX -require_number RETRY_INTERVAL -require_number POSTGIS_READY_DELAY - -PROFILE_UPPER="$(printf '%s' "$SYSTEM_PROFILE" | tr '[:lower:]' '[:upper:]')" -case "$PROFILE_UPPER" in - "RPI4") - SYSTEM_PROFILE="RPI4" - PG_SHARED="256MB" - PG_CACHE="1GB" - PG_WORK_MEM="2MB" - PG_MAINT="64MB" - PG_MAX_CONN="75" - ;; - "8GB") - SYSTEM_PROFILE="8GB" - PG_SHARED="512MB" - PG_CACHE="2GB" - PG_WORK_MEM="4MB" - PG_MAINT="128MB" - PG_MAX_CONN="125" - ;; - "16GB") - SYSTEM_PROFILE="16GB" - PG_SHARED="1GB" - PG_CACHE="4GB" - PG_WORK_MEM="8MB" - PG_MAINT="256MB" - PG_MAX_CONN="200" - ;; - "32GB") - SYSTEM_PROFILE="32GB" - PG_SHARED="2GB" - PG_CACHE="8GB" - PG_WORK_MEM="16MB" - PG_MAINT="512MB" - PG_MAX_CONN="300" - ;; - *) - echo "Unknown profile '${SYSTEM_PROFILE}', using 8GB defaults." - SYSTEM_PROFILE="8GB" - PG_SHARED="512MB" - PG_CACHE="2GB" - PG_WORK_MEM="4MB" - PG_MAINT="128MB" - PG_MAX_CONN="125" - ;; -esac - -# Keep sanitized/defaulted values available to the child launch.sh. -export SYSTEM_PROFILE CONTAINER_NAME DB_NAME DB_USER DB_PASSWORD DB_PORT - -mkdir -p "${PROJECT_DIR}/pgdata" - -if ! command -v docker >/dev/null 2>&1; then - echo "Error: Docker is not installed or is not in PATH." - exit 1 -fi - -if ! docker info >/dev/null 2>&1; then - echo "Error: Docker is installed, but the Docker daemon is not running." - echo "Start Docker Desktop, then run this script again." - exit 1 -fi - -POSTGIS_DIR="${PROJECT_DIR}/postgis" -if [ ! -d "$POSTGIS_DIR" ]; then - echo "Error: postgis directory not found in ${PROJECT_DIR}." - exit 1 -fi - -if [ ! -f "${POSTGIS_DIR}/${POSTGIS_DOCKERFILE}" ]; then - echo "Error: ${POSTGIS_DOCKERFILE} not found in ${POSTGIS_DIR}." - exit 1 -fi - -echo "Building PostGIS Docker image for Apple Silicon / ARM64..." -cd "$POSTGIS_DIR" -if [ -n "${POSTGIS_PLATFORM:-}" ]; then - docker build --platform "$POSTGIS_PLATFORM" . --file="$POSTGIS_DOCKERFILE" --tag="$IMAGE_NAME" -else - docker build . --file="$POSTGIS_DOCKERFILE" --tag="$IMAGE_NAME" -fi - -echo "Preparing PostGIS container for profile: ${SYSTEM_PROFILE}" -echo " Image: ${IMAGE_NAME}" -echo " Dockerfile: ${POSTGIS_DOCKERFILE}" -echo " Port: ${DB_PORT}:5432" -echo " Data: ${PROJECT_DIR}/pgdata" - -# Recreate the container so profile/tuning changes always take effect. -# Data persists because pgdata is mounted from the host. -if docker container inspect "$CONTAINER_NAME" >/dev/null 2>&1; then - echo "Removing existing container '${CONTAINER_NAME}' so updated settings take effect..." - docker rm -f "$CONTAINER_NAME" >/dev/null -fi - -echo "Creating new PostGIS container..." -docker run \ - --name "$CONTAINER_NAME" \ - -e POSTGRES_DB="$DB_NAME" \ - -e POSTGRES_USER="$DB_USER" \ - -e POSTGRES_PASSWORD="$DB_PASSWORD" \ - -e DATADIR=/var/lib/postgresql/data \ - -p "${DB_PORT}:5432" \ - -v "${PROJECT_DIR}/pgdata:/var/lib/postgresql/data" \ - -d \ - "$IMAGE_NAME" \ - -c shared_buffers="$PG_SHARED" \ - -c effective_cache_size="$PG_CACHE" \ - -c work_mem="$PG_WORK_MEM" \ - -c maintenance_work_mem="$PG_MAINT" \ - -c max_connections="$PG_MAX_CONN" \ - -c superuser_reserved_connections=10 \ - -c idle_session_timeout=600000 \ - -c log_connections=on \ - -c log_disconnections=on \ - -c wal_buffers=16MB \ - -c random_page_cost=1.1 \ - -c effective_io_concurrency=200 \ - || { echo "Failed to start PostGIS container"; exit 1; } - -echo "Waiting for PostGIS ARM64 to be ready..." -export PGPASSWORD="$DB_PASSWORD" -RETRY_COUNT=0 -until docker exec "$CONTAINER_NAME" pg_isready -U "$DB_USER" -d "$DB_NAME" >/dev/null 2>&1; do - RETRY_COUNT=$((RETRY_COUNT + 1)) - if [ "$RETRY_COUNT" -ge "$RETRY_MAX" ]; then - echo "Error: PostGIS did not become ready after $((RETRY_MAX * RETRY_INTERVAL)) seconds." - echo "Last container logs:" - docker logs --tail 50 "$CONTAINER_NAME" || true - exit 1 - fi - echo "PostGIS not ready yet, retrying..." - sleep "$RETRY_INTERVAL" -done - -echo "PostGIS is ready. Starting OpenSensorHub..." -sleep "$POSTGIS_READY_DELAY" - -OSH_DIR="${PROJECT_DIR}/osh-node-oscar" -if [ ! -d "$OSH_DIR" ]; then - echo "Error: osh-node-oscar directory not found in ${PROJECT_DIR}." - exit 1 -fi - -cd "$OSH_DIR" -if [ ! -f "./launch.sh" ]; then - echo "Error: launch.sh was not found in ${OSH_DIR}." - exit 1 -fi - -if [ ! -x "./launch.sh" ]; then - chmod +x ./launch.sh -fi - -exec ./launch.sh diff --git a/dist/release/launch-all.bat b/dist/release/launch-all.bat old mode 100644 new mode 100755 index bec96ac..1d37a44 --- a/dist/release/launch-all.bat +++ b/dist/release/launch-all.bat @@ -1,330 +1,214 @@ @echo off -setlocal EnableExtensions +setlocal EnableExtensions EnableDelayedExpansion -set "PROJECT_DIR=%~dp0" -set "ENV_FILE=%PROJECT_DIR%.env" -set "MATCH_EXPR=com.botts.impl.security.SensorHubWrapper" -set "FORCE_RESTART=%FORCE_RESTART%" +set "SCRIPT_DIR=%~dp0" +if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" + +set "NODE_DIR=%SCRIPT_DIR%\osh-node-oscar" +set "POSTGIS_DIR=%SCRIPT_DIR%\postgis" +set "ENV_FILE=%SCRIPT_DIR%\.env" + +if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" + +if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" +if not defined DB_NAME set "DB_NAME=gis" +if not defined DB_USER set "DB_USER=postgres" +if not defined DB_PASSWORD set "DB_PASSWORD=postgres" +if not defined DB_PORT set "DB_PORT=5432" +if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" +if not defined POSTGIS_IMAGE_NAME set "POSTGIS_IMAGE_NAME=oscar-postgis" +if not defined POSTGIS_DOCKERFILE set "POSTGIS_DOCKERFILE=Dockerfile" if not defined FORCE_RESTART set "FORCE_RESTART=0" -set "RETRY_MAX=%RETRY_MAX%" if not defined RETRY_MAX set "RETRY_MAX=120" -set "RETRY_INTERVAL=%RETRY_INTERVAL%" if not defined RETRY_INTERVAL set "RETRY_INTERVAL=2" -set "POSTGIS_READY_DELAY=%POSTGIS_READY_DELAY%" if not defined POSTGIS_READY_DELAY set "POSTGIS_READY_DELAY=5" -set "POSTGIS_DOCKERFILE=%POSTGIS_DOCKERFILE%" -if not defined POSTGIS_DOCKERFILE set "POSTGIS_DOCKERFILE=Dockerfile" -if not exist "%ENV_FILE%" ( - echo Error: .env file not found in "%PROJECT_DIR%". - echo Create it by copying env.template to .env and editing the values. +set "PGDATA_DIR=%SCRIPT_DIR%\pgdata" + +if not exist "%POSTGIS_DIR%" ( + echo ERROR: Missing PostGIS directory: "%POSTGIS_DIR%" exit /b 1 ) -call :load_env "%ENV_FILE%" - -if not defined IMAGE_NAME if defined POSTGIS_IMAGE_NAME set "IMAGE_NAME=%POSTGIS_IMAGE_NAME%" -if not defined IMAGE_NAME set "IMAGE_NAME=oscar-postgis" - -call :check_dependencies -if errorlevel 1 exit /b %ERRORLEVEL% -call :check_existing_oscar -if errorlevel 1 exit /b %ERRORLEVEL% -call :ensure_project_layout -if errorlevel 1 exit /b %ERRORLEVEL% +if not exist "%POSTGIS_DIR%\%POSTGIS_DOCKERFILE%" ( + echo ERROR: Missing PostGIS Dockerfile: "%POSTGIS_DIR%\%POSTGIS_DOCKERFILE%" + exit /b 1 +) -if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" -if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" -if not defined DB_HOST set "DB_HOST=localhost" +if not exist "%POSTGIS_DIR%\init-extensions.sql" ( + echo ERROR: Missing PostGIS init script: "%POSTGIS_DIR%\init-extensions.sql" + exit /b 1 +) -if not defined DB_NAME ( - echo Error: DB_NAME is not set in .env. +if not exist "%NODE_DIR%\launch.bat" ( + echo ERROR: Missing node launcher: "%NODE_DIR%\launch.bat" exit /b 1 ) -if not defined DB_USER ( - echo Error: DB_USER is not set in .env. + +where docker >nul 2>nul +if errorlevel 1 ( + echo ERROR: Docker was not found in PATH. exit /b 1 ) -if not defined DB_PASSWORD ( - echo Error: DB_PASSWORD is not set in .env. + +docker version >nul 2>nul +if errorlevel 1 ( + echo ERROR: Docker is installed but not responding. exit /b 1 ) -if not defined DB_PORT ( - echo Error: DB_PORT is not set in .env. + +where java >nul 2>nul +if errorlevel 1 ( + echo ERROR: Java was not found in PATH. exit /b 1 ) -call :require_number DB_PORT -if errorlevel 1 exit /b %ERRORLEVEL% -call :require_number RETRY_MAX -if errorlevel 1 exit /b %ERRORLEVEL% -call :require_number RETRY_INTERVAL -if errorlevel 1 exit /b %ERRORLEVEL% -call :require_number POSTGIS_READY_DELAY -if errorlevel 1 exit /b %ERRORLEVEL% +call :check_existing_oscar +if defined OSCAR_PID ( + if /I "%FORCE_RESTART%"=="1" ( + echo OSCAR is already running with PID !OSCAR_PID!. FORCE_RESTART=1, stopping it first... + call :stop_existing_oscar + call :wait_for_oscar_stop 60 + call :check_existing_oscar + if defined OSCAR_PID ( + echo ERROR: OSCAR is still running with PID !OSCAR_PID! after stop attempt. + exit /b 1 + ) + ) else ( + echo ERROR: OSCAR is already running with PID !OSCAR_PID!. + echo Set FORCE_RESTART=1 in .env to replace the running instance. + exit /b 1 + ) +) if /I "%SYSTEM_PROFILE%"=="RPI4" ( - set "SYSTEM_PROFILE=RPI4" - set "PG_SHARED=256MB" - set "PG_CACHE=1GB" + set "PG_MAX_CONNECTIONS=75" + set "PG_SHARED_BUFFERS=256MB" + set "PG_EFFECTIVE_CACHE_SIZE=1024MB" set "PG_WORK_MEM=2MB" - set "PG_MAINT=64MB" - set "PG_MAX_CONN=75" + set "PG_MAINTENANCE_WORK_MEM=64MB" ) else if /I "%SYSTEM_PROFILE%"=="8GB" ( - set "SYSTEM_PROFILE=8GB" - set "PG_SHARED=512MB" - set "PG_CACHE=2GB" + set "PG_MAX_CONNECTIONS=125" + set "PG_SHARED_BUFFERS=1024MB" + set "PG_EFFECTIVE_CACHE_SIZE=3072MB" set "PG_WORK_MEM=4MB" - set "PG_MAINT=128MB" - set "PG_MAX_CONN=125" + set "PG_MAINTENANCE_WORK_MEM=128MB" ) else if /I "%SYSTEM_PROFILE%"=="16GB" ( - set "SYSTEM_PROFILE=16GB" - set "PG_SHARED=1GB" - set "PG_CACHE=4GB" - set "PG_WORK_MEM=8MB" - set "PG_MAINT=256MB" - set "PG_MAX_CONN=200" + set "PG_MAX_CONNECTIONS=200" + set "PG_SHARED_BUFFERS=2048MB" + set "PG_EFFECTIVE_CACHE_SIZE=6144MB" + set "PG_WORK_MEM=4MB" + set "PG_MAINTENANCE_WORK_MEM=256MB" ) else if /I "%SYSTEM_PROFILE%"=="32GB" ( - set "SYSTEM_PROFILE=32GB" - set "PG_SHARED=2GB" - set "PG_CACHE=8GB" - set "PG_WORK_MEM=16MB" - set "PG_MAINT=512MB" - set "PG_MAX_CONN=300" + set "PG_MAX_CONNECTIONS=300" + set "PG_SHARED_BUFFERS=4096MB" + set "PG_EFFECTIVE_CACHE_SIZE=12288MB" + set "PG_WORK_MEM=8MB" + set "PG_MAINTENANCE_WORK_MEM=512MB" ) else ( - echo Unknown profile '%SYSTEM_PROFILE%', using 8GB defaults. - set "SYSTEM_PROFILE=8GB" - set "PG_SHARED=512MB" - set "PG_CACHE=2GB" + echo WARNING: Unknown SYSTEM_PROFILE "%SYSTEM_PROFILE%". Using 8GB defaults. + set "PG_MAX_CONNECTIONS=125" + set "PG_SHARED_BUFFERS=1024MB" + set "PG_EFFECTIVE_CACHE_SIZE=3072MB" set "PG_WORK_MEM=4MB" - set "PG_MAINT=128MB" - set "PG_MAX_CONN=125" + set "PG_MAINTENANCE_WORK_MEM=128MB" ) -set "PATH=%JAVA_HOME_DETECTED%\bin;%PATH%" - -if not exist "%PROJECT_DIR%pgdata" mkdir "%PROJECT_DIR%pgdata" >nul 2>nul -if not exist "%PROJECT_DIR%pgdata" ( - echo Error: failed to create pgdata directory. - exit /b 1 -) +if not exist "%PGDATA_DIR%" mkdir "%PGDATA_DIR%" echo Building PostGIS Docker image... -pushd "%PROJECT_DIR%postgis" -docker build . --file="%POSTGIS_DOCKERFILE%" --tag="%IMAGE_NAME%" +docker build -t "%POSTGIS_IMAGE_NAME%" -f "%POSTGIS_DIR%\%POSTGIS_DOCKERFILE%" "%POSTGIS_DIR%" if errorlevel 1 ( - echo Error: Docker build failed. - popd + echo ERROR: Failed to build PostGIS Docker image. exit /b 1 ) -popd echo Preparing PostGIS container for profile: %SYSTEM_PROFILE% -echo Image: %IMAGE_NAME% +echo Image: %POSTGIS_IMAGE_NAME% echo Port: %DB_PORT%:5432 -echo Data: %PROJECT_DIR%pgdata +echo Data: %PGDATA_DIR% -docker container inspect "%CONTAINER_NAME%" >nul 2>&1 +docker ps -a --format "{{.Names}}" | findstr /I /X "%CONTAINER_NAME%" >nul if not errorlevel 1 ( - echo Removing existing container '%CONTAINER_NAME%' so updated settings take effect... - docker rm -f "%CONTAINER_NAME%" >nul - if errorlevel 1 ( - echo Error: failed to remove existing container '%CONTAINER_NAME%'. - exit /b 1 - ) + echo Removing existing container "%CONTAINER_NAME%" so updated settings take effect... + docker rm -f "%CONTAINER_NAME%" >nul 2>nul ) echo Creating new container... -docker run ^ - --name "%CONTAINER_NAME%" ^ - -e "POSTGRES_DB=%DB_NAME%" ^ - -e "POSTGRES_USER=%DB_USER%" ^ - -e "POSTGRES_PASSWORD=%DB_PASSWORD%" ^ - -p "%DB_PORT%:5432" ^ - -v "%PROJECT_DIR%pgdata:/var/lib/postgresql/data" ^ - -d ^ - "%IMAGE_NAME%" ^ - -c "shared_buffers=%PG_SHARED%" ^ - -c "effective_cache_size=%PG_CACHE%" ^ - -c "work_mem=%PG_WORK_MEM%" ^ - -c "maintenance_work_mem=%PG_MAINT%" ^ - -c "max_connections=%PG_MAX_CONN%" ^ - -c superuser_reserved_connections=10 ^ - -c idle_session_timeout=600000 ^ - -c log_connections=on ^ - -c log_disconnections=on ^ - -c wal_buffers=16MB ^ - -c random_page_cost=1.1 ^ - -c effective_io_concurrency=200 +docker run -d ^ + --name "%CONTAINER_NAME%" ^ + -p %DB_PORT%:5432 ^ + -e POSTGRES_DB=%DB_NAME% ^ + -e POSTGRES_USER=%DB_USER% ^ + -e POSTGRES_PASSWORD=%DB_PASSWORD% ^ + -v "%PGDATA_DIR%:/var/lib/postgresql/data" ^ + "%POSTGIS_IMAGE_NAME%" ^ + -c max_connections=%PG_MAX_CONNECTIONS% ^ + -c superuser_reserved_connections=10 ^ + -c shared_buffers=%PG_SHARED_BUFFERS% ^ + -c effective_cache_size=%PG_EFFECTIVE_CACHE_SIZE% ^ + -c work_mem=%PG_WORK_MEM% ^ + -c maintenance_work_mem=%PG_MAINTENANCE_WORK_MEM% ^ + -c idle_session_timeout=600000 ^ + -c log_connections=on ^ + -c log_disconnections=on if errorlevel 1 ( - echo Error: failed to start PostGIS container. + echo ERROR: Failed to start PostGIS container. exit /b 1 ) echo Waiting for PostGIS to be ready... -set "PGPASSWORD=%DB_PASSWORD%" -set /a RETRY_COUNT=0 - -:wait_loop -docker exec "%CONTAINER_NAME%" pg_isready -U "%DB_USER%" -d "%DB_NAME%" >nul 2>&1 -if not errorlevel 1 goto after_wait -set /a RETRY_COUNT+=1 -if %RETRY_COUNT% GEQ %RETRY_MAX% ( - echo Error: PostGIS did not become ready after %RETRY_MAX% attempts. - echo Last container logs: - docker logs --tail 50 "%CONTAINER_NAME%" - exit /b 1 -) -timeout /t %RETRY_INTERVAL% /nobreak >nul -goto wait_loop - -:after_wait -echo PostGIS is ready. -timeout /t %POSTGIS_READY_DELAY% /nobreak >nul - -cd /d "%PROJECT_DIR%osh-node-oscar" -if errorlevel 1 ( - echo Error: osh-node-oscar directory not found in "%PROJECT_DIR%". - exit /b 1 -) - -if not exist "launch.bat" ( - echo Error: launch.bat not found in "%CD%". - exit /b 1 -) +set /a WAIT_COUNT=0 -call "launch.bat" -set "LAUNCH_EXIT_CODE=%ERRORLEVEL%" -endlocal & exit /b %LAUNCH_EXIT_CODE% +:wait_for_postgis +docker exec "%CONTAINER_NAME%" pg_isready -U "%DB_USER%" -d "%DB_NAME%" >nul 2>nul +if not errorlevel 1 goto postgis_ready -:check_dependencies -where powershell >nul 2>nul -if errorlevel 1 ( - echo Error: PowerShell is required but was not found on PATH. - exit /b 1 -) - -where java >nul 2>nul -if errorlevel 1 ( - echo Error: java was not found on PATH. Install OpenJDK 21 or newer. +set /a WAIT_COUNT+=1 +if !WAIT_COUNT! GEQ %RETRY_MAX% ( + echo ERROR: PostGIS did not become ready in time. + docker logs "%CONTAINER_NAME%" exit /b 1 ) -where docker >nul 2>nul -if errorlevel 1 ( - echo Error: docker was not found on PATH. Install Docker Desktop and make sure it is running. - exit /b 1 -) - -docker info >nul 2>nul -if errorlevel 1 ( - echo Error: Docker is installed, but the Docker daemon is not running. - exit /b 1 -) - -set "JAVA_HOME_LINE=" -for /f "delims=" %%A in ('java -XshowSettings:properties -version 2^>^&1 ^| findstr /c:"java.home ="') do ( - set "JAVA_HOME_LINE=%%A" - goto :deps_java_home_line -) - -:deps_java_home_line -if not defined JAVA_HOME_LINE ( - echo Error: could not determine java.home from the installed Java runtime. - exit /b 1 -) - -for /f "tokens=1,* delims==" %%A in ("%JAVA_HOME_LINE%") do set "JAVA_HOME_DETECTED=%%B" -for /f "tokens=* delims= " %%A in ("%JAVA_HOME_DETECTED%") do set "JAVA_HOME_DETECTED=%%A" +timeout /t %RETRY_INTERVAL% /nobreak >nul +goto wait_for_postgis -if not exist "%JAVA_HOME_DETECTED%\bin\java.exe" ( - echo Error: Java executable not found under "%JAVA_HOME_DETECTED%\bin\java.exe". - exit /b 1 -) -if not exist "%JAVA_HOME_DETECTED%\bin\keytool.exe" ( - echo Error: keytool.exe not found under "%JAVA_HOME_DETECTED%\bin\keytool.exe". - exit /b 1 -) +:postgis_ready +echo PostGIS is ready. +if %POSTGIS_READY_DELAY% GTR 0 timeout /t %POSTGIS_READY_DELAY% /nobreak >nul -set "JAVA_VERSION_LINE=" -for /f "delims=" %%A in ('"%JAVA_HOME_DETECTED%\bin\java.exe" -version 2^>^&1 ^| findstr /r /c:"version \""') do ( - set "JAVA_VERSION_LINE=%%A" - goto :deps_java_version_line -) +pushd "%NODE_DIR%" +call launch.bat +set "NODE_EXIT=%ERRORLEVEL%" +popd -:deps_java_version_line -if not defined JAVA_VERSION_LINE ( - echo Error: could not determine Java version. OpenJDK 21 or newer is required. - exit /b 1 -) +endlocal & exit /b %NODE_EXIT% -for /f "tokens=2 delims=\"" %%A in ("%JAVA_VERSION_LINE%") do set "JAVA_VERSION_RAW=%%A" -for /f "tokens=1 delims=." %%A in ("%JAVA_VERSION_RAW%") do set "JAVA_MAJOR=%%A" -if not defined JAVA_MAJOR ( - echo Error: could not parse Java version from "%JAVA_VERSION_LINE%". - exit /b 1 -) -if %JAVA_MAJOR% LSS 21 ( - echo Error: Java 21 or newer is required. Found Java %JAVA_MAJOR%. - exit /b 1 -) -exit /b 0 - -:find_existing_oscar +:check_existing_oscar set "OSCAR_PID=" -for /f %%P in ('powershell -NoProfile -Command "$p = Get-CimInstance Win32_Process ^| Where-Object { $_.Name -match ''^java(\.exe)?$'' -and $_.CommandLine -like ''*com.botts.impl.security.SensorHubWrapper*'' } ^| Select-Object -ExpandProperty ProcessId -First 1; if ($p) { Write-Output $p }"') do set "OSCAR_PID=%%P" +for /f "usebackq delims=" %%P in (` + powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { [Console]::Write($proc.ProcessId); break } }" 2^>nul +`) do set "OSCAR_PID=%%P" exit /b 0 -:check_existing_oscar -call :find_existing_oscar +:stop_existing_oscar if not defined OSCAR_PID exit /b 0 - -if "%FORCE_RESTART%"=="1" ( - echo Existing OSCAR instance found with PID %OSCAR_PID%. Replacing because FORCE_RESTART=1. - taskkill /PID %OSCAR_PID% /T /F >nul 2>nul - timeout /t 2 /nobreak >nul - call :find_existing_oscar - if defined OSCAR_PID ( - echo Error: could not stop the existing OSCAR instance. - exit /b 1 - ) - exit /b 0 -) - -echo OSCAR is already running with PID %OSCAR_PID%. -echo Stop the running instance first, or set FORCE_RESTART=1 to replace it. -exit /b 1 - -:ensure_project_layout -if not exist "%PROJECT_DIR%postgis" ( - echo Error: postgis directory not found in "%PROJECT_DIR%". - exit /b 1 -) -if not exist "%PROJECT_DIR%postgis\%POSTGIS_DOCKERFILE%" ( - echo Error: %POSTGIS_DOCKERFILE% not found in "%PROJECT_DIR%postgis". - exit /b 1 -) -if not exist "%PROJECT_DIR%osh-node-oscar" ( - echo Error: osh-node-oscar directory not found in "%PROJECT_DIR%". - exit /b 1 -) -if not exist "%PROJECT_DIR%osh-node-oscar\launch.bat" ( - echo Error: launch.bat not found in "%PROJECT_DIR%osh-node-oscar". - exit /b 1 -) +powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %OSCAR_PID% -Force -ErrorAction Stop; exit 0 } catch { exit 1 }" >nul 2>nul exit /b 0 -:require_number -call set "VALUE=%%%~1%%" -if not defined VALUE ( - echo Error: %~1 must be a number, got ''. - exit /b 1 -) -for /f "delims=0123456789" %%A in ("%VALUE%") do ( - echo Error: %~1 must be a number, got '%VALUE%'. - exit /b 1 -) -exit /b 0 +:wait_for_oscar_stop +set "WAIT_LIMIT=%~1" +if not defined WAIT_LIMIT set "WAIT_LIMIT=60" +set /a WAITED=0 + +:wait_for_oscar_stop_loop +call :check_existing_oscar +if not defined OSCAR_PID exit /b 0 +if !WAITED! GEQ %WAIT_LIMIT% exit /b 0 +timeout /t 1 /nobreak >nul +set /a WAITED+=1 +goto wait_for_oscar_stop_loop :load_env for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( @@ -339,4 +223,4 @@ if not defined ENV_NAME exit /b 0 if "%ENV_NAME:~0,1%"=="#" exit /b 0 if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" set "%ENV_NAME%=%ENV_VALUE%" -exit /b 0 +exit /b 0 \ No newline at end of file diff --git a/dist/release/launch-all.sh b/dist/release/launch-all.sh old mode 100644 new mode 100755 index 29639c5..7dd5bb4 --- a/dist/release/launch-all.sh +++ b/dist/release/launch-all.sh @@ -33,33 +33,6 @@ require_cmd() { fi } -get_java_major() { - java -version 2>&1 | awk -F'"' '/version/ { split($2, v, "."); print v[1]; exit }' -} - -check_dependencies() { - require_cmd bash - require_cmd java - require_cmd keytool - require_cmd docker - - if ! docker info >/dev/null 2>&1; then - echo "Error: Docker is installed, but the Docker daemon is not running." - exit 1 - fi - - local java_major - java_major="$(get_java_major || true)" - if [[ -z "$java_major" || ! "$java_major" =~ ^[0-9]+$ ]]; then - echo "Error: could not determine Java version. Java 21 or newer is required." - exit 1 - fi - if [ "$java_major" -lt 21 ]; then - echo "Error: Java 21 or newer is required. Found Java $java_major." - exit 1 - fi -} - find_existing_oscar_pids() { pgrep -f "$MATCH_EXPR" || true } @@ -111,15 +84,6 @@ check_existing_oscar() { exit 1 } -require_env() { - local name="$1" - local value="${!name:-}" - if [ -z "$value" ]; then - echo "Error: ${name} is not set in .env." - exit 1 - fi -} - require_number() { local name="$1" local value="${!name:-}" @@ -155,27 +119,35 @@ ensure_project_layout() { mkdir -p "$PROJECT_DIR/pgdata" } -if [ ! -f "$ENV_FILE" ]; then - echo "Error: .env file not found in $PROJECT_DIR" - echo "Create it by copying env.template to .env and editing the values." - exit 1 +if [ -f "$ENV_FILE" ]; then + load_env "$ENV_FILE" +else + echo "Warning: .env file not found in $PROJECT_DIR" + if [ -f "$PROJECT_DIR/env.template" ]; then + echo "Warning: using built-in defaults. Copy env.template to .env to customize settings." + else + echo "Warning: using built-in defaults." + fi fi -load_env "$ENV_FILE" -check_dependencies -check_existing_oscar -ensure_project_layout - SYSTEM_PROFILE="${SYSTEM_PROFILE:-8GB}" CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" +DB_NAME="${DB_NAME:-gis}" +DB_USER="${DB_USER:-postgres}" +DB_PASSWORD="${DB_PASSWORD:-postgres}" +DB_PORT="${DB_PORT:-5432}" DB_HOST="${DB_HOST:-localhost}" -export SYSTEM_PROFILE CONTAINER_NAME DB_HOST RETRY_MAX RETRY_INTERVAL POSTGIS_READY_DELAY IMAGE_NAME POSTGIS_DOCKERFILE +export SYSTEM_PROFILE CONTAINER_NAME DB_NAME DB_USER DB_PASSWORD DB_PORT DB_HOST +export RETRY_MAX RETRY_INTERVAL POSTGIS_READY_DELAY IMAGE_NAME POSTGIS_DOCKERFILE -require_env DB_NAME -require_env DB_USER -require_env DB_PASSWORD -require_env DB_PORT +require_cmd docker +if ! docker info >/dev/null 2>&1; then + echo "Error: Docker is installed, but the Docker daemon is not running." + exit 1 +fi +check_existing_oscar +ensure_project_layout require_number DB_PORT require_number RETRY_MAX require_number RETRY_INTERVAL @@ -223,9 +195,7 @@ case "${SYSTEM_PROFILE^^}" in PG_MAINT="128MB" PG_MAX_CONN="125" ;; - esac - -export SYSTEM_PROFILE CONTAINER_NAME DB_NAME DB_USER DB_PASSWORD DB_PORT +esac echo "Building PostGIS Docker image..." ( diff --git a/dist/release/launch-all_old.bat b/dist/release/launch-all_old.bat deleted file mode 100644 index 646c7ce..0000000 --- a/dist/release/launch-all_old.bat +++ /dev/null @@ -1,192 +0,0 @@ -@echo off -setlocal EnableExtensions - -rem Resolve project root from this script's location instead of the caller's cwd. -set "PROJECT_DIR=%~dp0" -set "ENV_FILE=%PROJECT_DIR%.env" -set "IMAGE_NAME=oscar-postgis" - -if not exist "%ENV_FILE%" ( - echo Error: .env file not found in "%PROJECT_DIR%". - echo Create it by copying env.template to .env and editing the values. - exit /b 1 -) - -call :load_env "%ENV_FILE%" - -if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" -if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" - -if not defined DB_NAME ( - echo Error: DB_NAME is not set in .env. - exit /b 1 -) -if not defined DB_USER ( - echo Error: DB_USER is not set in .env. - exit /b 1 -) -if not defined DB_PASSWORD ( - echo Error: DB_PASSWORD is not set in .env. - exit /b 1 -) -if not defined DB_PORT ( - echo Error: DB_PORT is not set in .env. - exit /b 1 -) - -if /I "%SYSTEM_PROFILE%"=="RPI4" ( - set "PG_SHARED=256MB" - set "PG_CACHE=1GB" - set "PG_WORK_MEM=2MB" - set "PG_MAINT=64MB" - set "PG_MAX_CONN=75" -) else if /I "%SYSTEM_PROFILE%"=="8GB" ( - set "PG_SHARED=512MB" - set "PG_CACHE=2GB" - set "PG_WORK_MEM=4MB" - set "PG_MAINT=128MB" - set "PG_MAX_CONN=125" -) else if /I "%SYSTEM_PROFILE%"=="16GB" ( - set "PG_SHARED=1GB" - set "PG_CACHE=4GB" - set "PG_WORK_MEM=8MB" - set "PG_MAINT=256MB" - set "PG_MAX_CONN=200" -) else if /I "%SYSTEM_PROFILE%"=="32GB" ( - set "PG_SHARED=2GB" - set "PG_CACHE=8GB" - set "PG_WORK_MEM=16MB" - set "PG_MAINT=512MB" - set "PG_MAX_CONN=300" -) else ( - echo Unknown profile '%SYSTEM_PROFILE%', using 8GB defaults. - set "PG_SHARED=512MB" - set "PG_CACHE=2GB" - set "PG_WORK_MEM=4MB" - set "PG_MAINT=128MB" - set "PG_MAX_CONN=125" -) - -if not exist "%PROJECT_DIR%pgdata" ( - mkdir "%PROJECT_DIR%pgdata" - if errorlevel 1 ( - echo Error: failed to create pgdata directory. - exit /b 1 - ) -) - -where docker >nul 2>&1 -if errorlevel 1 ( - echo Error: Docker is not installed or is not in PATH. - exit /b 1 -) - -echo Building PostGIS Docker image... -if not exist "%PROJECT_DIR%postgis" ( - echo Error: postgis directory not found in "%PROJECT_DIR%". - exit /b 1 -) -pushd "%PROJECT_DIR%postgis" -docker build . --file=Dockerfile --tag=%IMAGE_NAME% -if errorlevel 1 ( - echo Error: Docker build failed. - popd - exit /b 1 -) -popd - -echo Preparing PostGIS container for profile: %SYSTEM_PROFILE% - -rem Recreate the container so profile/tuning changes always take effect. -rem Data persists because pgdata is mounted from the host. -docker container inspect "%CONTAINER_NAME%" >nul 2>&1 -if not errorlevel 1 ( - echo Removing existing container '%CONTAINER_NAME%' so updated settings take effect... - docker rm -f "%CONTAINER_NAME%" >nul - if errorlevel 1 ( - echo Error: failed to remove existing container '%CONTAINER_NAME%'. - exit /b 1 - ) -) - -echo Creating new container... -docker run ^ - --name "%CONTAINER_NAME%" ^ - -e "POSTGRES_DB=%DB_NAME%" ^ - -e "POSTGRES_USER=%DB_USER%" ^ - -e "POSTGRES_PASSWORD=%DB_PASSWORD%" ^ - -p "%DB_PORT%:5432" ^ - -v "%PROJECT_DIR%pgdata:/var/lib/postgresql/data" ^ - -d ^ - %IMAGE_NAME% ^ - -c "shared_buffers=%PG_SHARED%" ^ - -c "effective_cache_size=%PG_CACHE%" ^ - -c "work_mem=%PG_WORK_MEM%" ^ - -c "maintenance_work_mem=%PG_MAINT%" ^ - -c "max_connections=%PG_MAX_CONN%" ^ - -c superuser_reserved_connections=10 ^ - -c idle_session_timeout=600000 ^ - -c log_connections=on ^ - -c log_disconnections=on ^ - -c wal_buffers=16MB ^ - -c random_page_cost=1.1 ^ - -c effective_io_concurrency=200 -if errorlevel 1 ( - echo Error: failed to start PostGIS container. - exit /b 1 -) - -echo Waiting for PostGIS to be ready... -set "PGPASSWORD=%DB_PASSWORD%" - -:wait_loop -docker exec "%CONTAINER_NAME%" pg_isready -U "%DB_USER%" -d "%DB_NAME%" >nul 2>&1 -if not errorlevel 1 goto after_wait -timeout /t 2 /nobreak >nul -goto wait_loop - -:after_wait -echo PostGIS is ready. -timeout /t 5 /nobreak >nul - -cd /d "%PROJECT_DIR%osh-node-oscar" -if errorlevel 1 ( - echo Error: osh-node-oscar directory not found in "%PROJECT_DIR%". - exit /b 1 -) - -if exist "launch.bat" goto run_launch_bat -if exist "launch.sh" goto run_launch_sh -echo Error: neither launch.bat nor launch.sh was found in "%CD%". -exit /b 1 - -:run_launch_bat -call "launch.bat" -set "LAUNCH_EXIT_CODE=%ERRORLEVEL%" -goto launch_done - -:run_launch_sh -echo Warning: launch.bat not found. Trying launch.sh through Bash... -bash "launch.sh" -set "LAUNCH_EXIT_CODE=%ERRORLEVEL%" -goto launch_done - -:launch_done -endlocal & exit /b %LAUNCH_EXIT_CODE% - -:load_env -rem Minimal .env loader for KEY=VALUE lines. Blank lines are ignored by for /f. -rem Lines beginning with # are ignored. Empty values clear the variable. -for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( - set "ENV_NAME=%%A" - set "ENV_VALUE=%%B" - call :set_env_var -) -exit /b 0 - -:set_env_var -if not defined ENV_NAME exit /b 0 -if "%ENV_NAME:~0,1%"=="#" exit /b 0 -if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" -set "%ENV_NAME%=%ENV_VALUE%" -exit /b 0 diff --git a/dist/release/launch-all_old.sh b/dist/release/launch-all_old.sh deleted file mode 100644 index 7ab5491..0000000 --- a/dist/release/launch-all_old.sh +++ /dev/null @@ -1,111 +0,0 @@ -#!/bin/bash -set -euo pipefail - -PROJECT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" -ENV_FILE="${PROJECT_DIR}/.env" - -if [ ! -f "$ENV_FILE" ]; then - echo "Error: .env file not found in $PROJECT_DIR" - exit 1 -fi - -set -a -. "$ENV_FILE" -set +a - -CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" - -case "${SYSTEM_PROFILE:-8GB}" in - "RPI4") - PG_SHARED="256MB" - PG_CACHE="1GB" - PG_WORK_MEM="2MB" - PG_MAINT="64MB" - PG_MAX_CONN="75" - ;; - "8GB") - PG_SHARED="512MB" - PG_CACHE="2GB" - PG_WORK_MEM="4MB" - PG_MAINT="128MB" - PG_MAX_CONN="125" - ;; - "16GB") - PG_SHARED="1GB" - PG_CACHE="4GB" - PG_WORK_MEM="8MB" - PG_MAINT="256MB" - PG_MAX_CONN="200" - ;; - "32GB") - PG_SHARED="2GB" - PG_CACHE="8GB" - PG_WORK_MEM="16MB" - PG_MAINT="512MB" - PG_MAX_CONN="300" - ;; - *) - echo "Unknown profile '${SYSTEM_PROFILE}', using 8GB defaults." - PG_SHARED="512MB" - PG_CACHE="2GB" - PG_WORK_MEM="4MB" - PG_MAINT="128MB" - PG_MAX_CONN="125" - ;; -esac - -mkdir -p "${PROJECT_DIR}/pgdata" - -if ! command -v docker >/dev/null 2>&1; then - echo "Error: Docker is not installed." - exit 1 -fi - -echo "Building PostGIS Docker image..." -cd "${PROJECT_DIR}/postgis" || { echo "Error: postgis directory not found"; exit 1; } -docker build . --file=Dockerfile --tag=oscar-postgis - -echo "Preparing PostGIS container for profile: ${SYSTEM_PROFILE}" - -# Recreate the container so new tuning always applies. -# Data persists because pgdata is mounted from the host. -if docker ps -a --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then - echo "Removing existing container '${CONTAINER_NAME}' so updated settings take effect..." - docker rm -f "${CONTAINER_NAME}" >/dev/null -fi - -echo "Creating new container..." -docker run \ - --name "${CONTAINER_NAME}" \ - -e POSTGRES_DB="${DB_NAME}" \ - -e POSTGRES_USER="${DB_USER}" \ - -e POSTGRES_PASSWORD="${DB_PASSWORD}" \ - -p "${DB_PORT}:5432" \ - -v "${PROJECT_DIR}/pgdata:/var/lib/postgresql/data" \ - -d \ - oscar-postgis \ - -c shared_buffers="${PG_SHARED}" \ - -c effective_cache_size="${PG_CACHE}" \ - -c work_mem="${PG_WORK_MEM}" \ - -c maintenance_work_mem="${PG_MAINT}" \ - -c max_connections="${PG_MAX_CONN}" \ - -c superuser_reserved_connections=10 \ - -c idle_session_timeout=600000 \ - -c log_connections=on \ - -c log_disconnections=on \ - -c wal_buffers=16MB \ - -c random_page_cost=1.1 \ - -c effective_io_concurrency=200 \ - || { echo "Failed to start PostGIS container"; exit 1; } - -echo "Waiting for PostGIS to be ready..." -export PGPASSWORD="${DB_PASSWORD}" -until docker exec "${CONTAINER_NAME}" pg_isready -U "${DB_USER}" -d "${DB_NAME}" >/dev/null 2>&1; do - sleep 2 -done - -echo "PostGIS is ready." -sleep 5 - -cd "${PROJECT_DIR}/osh-node-oscar" || { echo "Error: osh-node-oscar not found"; exit 1; } -exec ./launch.sh diff --git a/dist/release/monitor-oscar.bat b/dist/release/monitor-oscar.bat old mode 100644 new mode 100755 index a325b4e..d79e88c --- a/dist/release/monitor-oscar.bat +++ b/dist/release/monitor-oscar.bat @@ -1,233 +1,193 @@ @echo off -setlocal EnableExtensions - -if /I "%~1"=="stop" goto :stop_mode +setlocal EnableExtensions EnableDelayedExpansion set "SCRIPT_DIR=%~dp0" -for %%I in ("%SCRIPT_DIR%.") do set "PROJECT_DIR=%%~fI" - -call :timestamp OUT_STAMP -set "OUT_DIR=%PROJECT_DIR%\oscar-monitor-%OUT_STAMP%" -set "ENV_FILE=%PROJECT_DIR%\.env" -set "CONTAINER_NAME=oscar-postgis-container" -set "DB_NAME=gis" -set "DB_USER=postgres" -set "DB_PASSWORD=postgres" -set "MATCH_EXPR=com.botts.impl.security.SensorHubWrapper" -set "INTERVAL=%INTERVAL%" -if not defined INTERVAL set "INTERVAL=60" -set "MAX_WAIT_SECONDS=%MAX_WAIT_SECONDS%" -if not defined MAX_WAIT_SECONDS set "MAX_WAIT_SECONDS=300" -set "JFR_NAME=%JFR_NAME%" -if not defined JFR_NAME set "JFR_NAME=oscar" -set "JFR_MAX_AGE=%JFR_MAX_AGE%" -if not defined JFR_MAX_AGE set "JFR_MAX_AGE=4h" -set "JFR_MAX_SIZE=%JFR_MAX_SIZE%" -if not defined JFR_MAX_SIZE set "JFR_MAX_SIZE=1g" -set "LAUNCH_CMD=%PROJECT_DIR%\launch-all.bat" -set "ATTACH_TO_EXISTING=%ATTACH_TO_EXISTING%" -if not defined ATTACH_TO_EXISTING set "ATTACH_TO_EXISTING=0" -set "FORCE_RESTART=%FORCE_RESTART%" -if not defined FORCE_RESTART set "FORCE_RESTART=0" +if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" -if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" +set "ENV_FILE=%SCRIPT_DIR%\.env" +set "LAUNCH_CMD=%SCRIPT_DIR%\launch-all.bat" +set "ACTIVE_MONITOR_FILE=%SCRIPT_DIR%\.monitor-active-dir" -call :check_dependencies -if errorlevel 1 exit /b %ERRORLEVEL% +if /I "%~1"=="stop" goto stop_monitor -if not exist "%OUT_DIR%" mkdir "%OUT_DIR%" -echo timestamp,total_sessions,active,idle,idle_in_transaction,max_connections,superuser_reserved_connections,failed_psql>"%OUT_DIR%\db-connection-trend.csv" +if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" -echo %DATE% %TIME% Monitor output: %OUT_DIR% -echo %DATE% %TIME% Launch command: %LAUNCH_CMD% +if not defined ATTACH_TO_EXISTING set "ATTACH_TO_EXISTING=0" +if not defined FORCE_RESTART set "FORCE_RESTART=0" +if not defined MAX_WAIT_SECONDS set "MAX_WAIT_SECONDS=300" +if not defined SNAPSHOT_INTERVAL_SECONDS set "SNAPSHOT_INTERVAL_SECONDS=60" -call :find_existing_oscar -if defined OSCAR_PID ( - if "%ATTACH_TO_EXISTING%"=="1" ( - set "JVM_PID=%OSCAR_PID%" - set "USE_EXISTING=1" - echo %DATE% %TIME% Attaching to existing OSCAR PID %JVM_PID% - ) else if "%FORCE_RESTART%"=="1" ( - echo %DATE% %TIME% Existing OSCAR instance found with PID %OSCAR_PID%. Replacing because FORCE_RESTART=1. - taskkill /PID %OSCAR_PID% /T /F >nul 2>nul - timeout /t 2 /nobreak >nul - call :find_existing_oscar - if defined OSCAR_PID ( - echo Error: could not stop the existing OSCAR instance. - exit /b 1 - ) - ) else ( - echo OSCAR is already running with PID %OSCAR_PID%. - echo Set ATTACH_TO_EXISTING=1 to monitor the running instance, or FORCE_RESTART=1 to replace it. - exit /b 1 - ) +if not exist "%LAUNCH_CMD%" ( + echo ERROR: Missing launch command: "%LAUNCH_CMD%" + exit /b 1 ) -if not defined USE_EXISTING ( - if not exist "%LAUNCH_CMD%" ( - echo Error: launch command not found: "%LAUNCH_CMD%" - exit /b 1 - ) +for /f %%T in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "TS=%%T" +set "OUT_DIR=%SCRIPT_DIR%\oscar-monitor-%TS%" - powershell -NoProfile -Command "$p = Start-Process -FilePath 'cmd.exe' -ArgumentList '/c ""%LAUNCH_CMD%""' -RedirectStandardOutput '%OUT_DIR%\launch.stdout.log' -RedirectStandardError '%OUT_DIR%\launch.stderr.log' -PassThru; Write-Output $p.Id" > "%OUT_DIR%\launcher-pid.txt" - for /f "usebackq" %%P in ("%OUT_DIR%\launcher-pid.txt") do set "LAUNCH_PID=%%P" +mkdir "%OUT_DIR%" >nul 2>nul +if errorlevel 1 ( + echo ERROR: Could not create monitor output directory: "%OUT_DIR%" + exit /b 1 +) - echo %DATE% %TIME% Waiting for OSCAR Java process... - set /a WAITED=0 +> "%ACTIVE_MONITOR_FILE%" echo %OUT_DIR% +> "%OUT_DIR%\launch.stdout.log" type nul +> "%OUT_DIR%\launch.stderr.log" type nul - :wait_for_jvm - call :find_existing_oscar - if defined OSCAR_PID ( - set "JVM_PID=%OSCAR_PID%" - goto :have_jvm - ) +echo %date% %time% Monitor output: %OUT_DIR% +echo %date% %time% Launch command: %LAUNCH_CMD% - if %WAITED% GEQ %MAX_WAIT_SECONDS% ( - echo ERROR: Could not find OSCAR Java PID after waiting. - exit /b 1 +call :check_existing_oscar +if defined OSCAR_PID ( + if /I "%ATTACH_TO_EXISTING%"=="1" ( + echo Attaching to existing OSCAR PID %OSCAR_PID%... + goto found_oscar ) - - timeout /t 2 /nobreak >nul - set /a WAITED+=2 - goto :wait_for_jvm -) else ( - >"%OUT_DIR%\launch.stdout.log" type nul - >"%OUT_DIR%\launch.stderr.log" type nul + if /I "%FORCE_RESTART%"=="1" ( + echo Existing OSCAR detected with PID %OSCAR_PID%. FORCE_RESTART=1, replacing it... + call :stop_existing_oscar + call :wait_for_oscar_stop 60 + call :start_launch + goto wait_for_oscar + ) + echo ERROR: OSCAR is already running with PID %OSCAR_PID%. + echo Set ATTACH_TO_EXISTING=1 to monitor it, or FORCE_RESTART=1 to replace it. + del "%ACTIVE_MONITOR_FILE%" >nul 2>nul + exit /b 1 ) -:have_jvm -echo %JVM_PID%>"%OUT_DIR%\jvm-pid.txt" +call :start_launch -powershell -NoProfile -Command "$p = Get-CimInstance Win32_Process -Filter 'ProcessId=%JVM_PID%'; if($p){ 'Timestamp: ' + (Get-Date -Format o); 'Launcher PID: %LAUNCH_PID%'; 'JVM PID: %JVM_PID%'; ''; 'Command line:'; $p.CommandLine }" > "%OUT_DIR%\process-info.txt" +:wait_for_oscar +echo Waiting for OSCAR Java process... +set /a WAITED=0 -if defined JCMD_CMD ( - "%JCMD_CMD%" %JVM_PID% JFR.start name=%JFR_NAME% settings=profile disk=true maxage=%JFR_MAX_AGE% maxsize=%JFR_MAX_SIZE% filename="%OUT_DIR%\%JFR_NAME%.jfr" > "%OUT_DIR%\jfr-start.txt" 2>&1 - "%JCMD_CMD%" %JVM_PID% VM.native_memory baseline > "%OUT_DIR%\nmt-baseline.txt" 2>&1 -) else ( - echo jcmd not available; skipping JFR start and NMT baseline. > "%OUT_DIR%\jcmd-warning.txt" -) +:wait_loop +if exist "%OUT_DIR%\stop.request" goto cleanup +call :check_existing_oscar +if defined OSCAR_PID goto found_oscar -:loop -call :snapshot -call :process_alive %JVM_PID% JVM_ALIVE -if not defined JVM_ALIVE goto :eof_ok -set "JVM_ALIVE=" -timeout /t %INTERVAL% /nobreak >nul -goto :loop - -:snapshot -call :timestamp SNAP_STAMP -set "SNAP=%OUT_DIR%\%SNAP_STAMP%" -if not exist "%SNAP%" mkdir "%SNAP%" - -echo Collecting snapshot at %SNAP_STAMP% for PID %JVM_PID% -powershell -NoProfile -Command "$p = Get-CimInstance Win32_Process -Filter 'ProcessId=%JVM_PID%'; if($p){$p | Select-Object ProcessId,ParentProcessId,Name,CommandLine | Format-List | Out-String}" > "%SNAP%\process.txt" 2>&1 -powershell -NoProfile -Command "$p=Get-Process -Id %JVM_PID% -ErrorAction SilentlyContinue; if($p){$p | Select-Object Id,ProcessName,Threads,VirtualMemorySize64,WorkingSet64,PrivateMemorySize64,CPU,StartTime | Format-List | Out-String}" > "%SNAP%\powershell-process.txt" 2>&1 -powershell -NoProfile -Command "Get-CimInstance Win32_OperatingSystem | Select-Object TotalVisibleMemorySize,FreePhysicalMemory,TotalVirtualMemorySize,FreeVirtualMemory | Format-List | Out-String" > "%SNAP%\memory.txt" 2>&1 -powershell -NoProfile -Command "Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\%% Usage' | Out-String" > "%SNAP%\counters.txt" 2>&1 -if defined JCMD_CMD ( - "%JCMD_CMD%" %JVM_PID% VM.native_memory summary > "%SNAP%\nmt-summary.txt" 2>&1 - "%JCMD_CMD%" %JVM_PID% GC.heap_info > "%SNAP%\gc-heap-info.txt" 2>&1 - "%JCMD_CMD%" %JVM_PID% Thread.print > "%SNAP%\thread-print.txt" 2>&1 - "%JCMD_CMD%" %JVM_PID% JFR.check > "%SNAP%\jfr-check.txt" 2>&1 +if %WAITED% GEQ %MAX_WAIT_SECONDS% ( + echo ERROR: Could not find OSCAR Java PID after waiting. + del "%ACTIVE_MONITOR_FILE%" >nul 2>nul + exit /b 1 ) -docker ps --filter name=%CONTAINER_NAME% > "%SNAP%\docker-ps.txt" 2>&1 -docker logs --tail 100 %CONTAINER_NAME% > "%SNAP%\docker-logs-tail.txt" 2>&1 -call :db_snapshot "%SNAP%" -exit /b 0 +timeout /t 2 /nobreak >nul +set /a WAITED+=2 +goto wait_loop -:db_snapshot -set "SNAP=%~1" -set "DB_ERR=%SNAP%\db-error.txt" -set "FAILED=0" -set "MAX_CONN=" -set "SUPER_RESERVED=" -set "TOTAL_SESSIONS=" -set "ACTIVE_COUNT=0" -set "IDLE_COUNT=0" -set "IDLE_TX_COUNT=0" +:found_oscar +echo Found OSCAR Java PID: %OSCAR_PID% +> "%OUT_DIR%\jvm-pid.txt" echo %OSCAR_PID% -for /f %%T in ('powershell -NoProfile -Command "Get-Date -Format o"') do set "DB_TS=%%T" +:monitor_loop +if exist "%OUT_DIR%\stop.request" goto cleanup -docker ps --format {{.Names}} | findstr /i /x "%CONTAINER_NAME%" >nul 2>&1 -if errorlevel 1 ( - >"%DB_ERR%" echo Container %CONTAINER_NAME% not running - >>"%OUT_DIR%\db-connection-trend.csv" echo %DB_TS%,,,,,,,1 - exit /b 0 +call :check_existing_oscar +if not defined OSCAR_PID ( + echo OSCAR Java process is no longer running. + goto cleanup ) -docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "show max_connections;" > "%SNAP%\db-max-connections.txt" 2> "%DB_ERR%" -if errorlevel 1 set "FAILED=1" -docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "show superuser_reserved_connections;" > "%SNAP%\db-superuser-reserved-connections.txt" 2>> "%DB_ERR%" -docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select count(*) from pg_stat_activity;" > "%SNAP%\db-total-sessions.txt" 2>> "%DB_ERR%" -docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select coalesce(state,''), count(*) from pg_stat_activity group by state order by count(*) desc;" > "%SNAP%\db-by-state.txt" 2>> "%DB_ERR%" -docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select coalesce(application_name,''), coalesce(usename,''), coalesce(client_addr::text,''), coalesce(state,''), count(*) from pg_stat_activity group by application_name, usename, client_addr, state order by count(*) desc limit 20;" > "%SNAP%\db-by-app.txt" 2>> "%DB_ERR%" -docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select pid, usename, application_name, client_addr, state, backend_start, xact_start, query_start, wait_event_type, wait_event, left(query,120) from pg_stat_activity order by backend_start;" > "%SNAP%\db-activity-detail.txt" 2>> "%DB_ERR%" - -for /f "usebackq" %%A in ("%SNAP%\db-max-connections.txt") do set "MAX_CONN=%%A" -for /f "usebackq" %%A in ("%SNAP%\db-superuser-reserved-connections.txt") do set "SUPER_RESERVED=%%A" -for /f "usebackq" %%A in ("%SNAP%\db-total-sessions.txt") do set "TOTAL_SESSIONS=%%A" -for /f "usebackq tokens=1,2 delims=|" %%A in ("%SNAP%\db-by-state.txt") do ( - if /i "%%A"=="active" set "ACTIVE_COUNT=%%B" - if /i "%%A"=="idle" set "IDLE_COUNT=%%B" - if /i "%%A"=="idle in transaction" set "IDLE_TX_COUNT=%%B" -) ->>"%OUT_DIR%\db-connection-trend.csv" echo %DB_TS%,%TOTAL_SESSIONS%,%ACTIVE_COUNT%,%IDLE_COUNT%,%IDLE_TX_COUNT%,%MAX_CONN%,%SUPER_RESERVED%,%FAILED% -exit /b 0 +> "%OUT_DIR%\jvm-pid.txt" echo %OSCAR_PID% +call :capture_snapshot +timeout /t %SNAPSHOT_INTERVAL_SECONDS% /nobreak >nul +goto monitor_loop -:stop_mode -call :find_existing_oscar -if defined OSCAR_PID taskkill /PID %OSCAR_PID% /T /F >nul 2>&1 -for /f %%C in ('docker ps --filter name=oscar-postgis-container --format {{.Names}}') do docker stop %%C >nul 2>&1 -echo OSCAR stop requested. +:cleanup +echo Stopping monitor... +del "%OUT_DIR%\stop.request" >nul 2>nul +del "%ACTIVE_MONITOR_FILE%" >nul 2>nul exit /b 0 -:check_dependencies -where powershell >nul 2>nul -if errorlevel 1 ( - echo Error: PowerShell is required but was not found on PATH. - exit /b 1 -) +:stop_monitor +set "MON_DIR=" +if exist "%ACTIVE_MONITOR_FILE%" set /p MON_DIR=<"%ACTIVE_MONITOR_FILE%" +if not defined MON_DIR call :find_latest_monitor_dir -where java >nul 2>nul -if errorlevel 1 ( - echo Error: java was not found on PATH. Install OpenJDK 21 or newer. - exit /b 1 +if not defined MON_DIR ( + echo No active monitor directory found. + exit /b 0 ) -where docker >nul 2>nul -if errorlevel 1 ( - echo Error: docker was not found on PATH. Install Docker Desktop and make sure it is running. - exit /b 1 +if not exist "%MON_DIR%" ( + del "%ACTIVE_MONITOR_FILE%" >nul 2>nul + exit /b 0 ) -set "JAVA_HOME_LINE=" -for /f "delims=" %%A in ('java -XshowSettings:properties -version 2^>^&1 ^| findstr /c:"java.home ="') do ( - set "JAVA_HOME_LINE=%%A" - goto :monitor_java_home_line -) +> "%MON_DIR%\stop.request" echo stop +echo Requested monitor stop: "%MON_DIR%" +exit /b 0 -:monitor_java_home_line -if not defined JAVA_HOME_LINE exit /b 0 -for /f "tokens=1,* delims==" %%A in ("%JAVA_HOME_LINE%") do set "JAVA_HOME_DETECTED=%%B" -for /f "tokens=* delims= " %%A in ("%JAVA_HOME_DETECTED%") do set "JAVA_HOME_DETECTED=%%A" -if exist "%JAVA_HOME_DETECTED%\bin\jcmd.exe" set "JCMD_CMD=%JAVA_HOME_DETECTED%\bin\jcmd.exe" +:start_launch +echo Launching OSCAR via launch-all.bat... +start "" /b cmd /c ""%LAUNCH_CMD%" 1>>"%OUT_DIR%\launch.stdout.log" 2>>"%OUT_DIR%\launch.stderr.log"" exit /b 0 -:find_existing_oscar +:check_existing_oscar set "OSCAR_PID=" -for /f %%P in ('powershell -NoProfile -Command "$p = Get-CimInstance Win32_Process ^| Where-Object { $_.Name -match ''^java(\.exe)?$'' -and $_.CommandLine -like ''*com.botts.impl.security.SensorHubWrapper*'' } ^| Select-Object -ExpandProperty ProcessId -First 1; if ($p) { Write-Output $p }"') do set "OSCAR_PID=%%P" +for /f "usebackq delims=" %%P in (` + powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { [Console]::Write($proc.ProcessId); break } }" 2^>nul +`) do set "OSCAR_PID=%%P" exit /b 0 -:process_alive -set "%~2=" -powershell -NoProfile -Command "if (Get-Process -Id %~1 -ErrorAction SilentlyContinue) { exit 0 } else { exit 1 }" >nul 2>nul -if not errorlevel 1 set "%~2=1" +:stop_existing_oscar +if not defined OSCAR_PID exit /b 0 +powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %OSCAR_PID% -Force -ErrorAction Stop } catch {}" >nul 2>nul exit /b 0 -:timestamp -for /f %%A in ('powershell -NoProfile -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "%~1=%%A" +:wait_for_oscar_stop +set "WAIT_LIMIT=%~1" +if not defined WAIT_LIMIT set "WAIT_LIMIT=60" +set /a WAITED=0 + +:wait_for_oscar_stop_loop +call :check_existing_oscar +if not defined OSCAR_PID exit /b 0 +if !WAITED! GEQ %WAIT_LIMIT% exit /b 0 +timeout /t 1 /nobreak >nul +set /a WAITED+=1 +goto wait_for_oscar_stop_loop + +:find_latest_monitor_dir +set "MON_DIR=" +for /f "delims=" %%D in (' + powershell -NoProfile -ExecutionPolicy Bypass -Command "$d = Get-ChildItem -LiteralPath '%SCRIPT_DIR%' -Directory -Filter 'oscar-monitor-*' ^| Sort-Object Name -Descending ^| Select-Object -First 1 -ExpandProperty FullName; if ($d) { $d }" 2^>nul +') do set "MON_DIR=%%D" +exit /b 0 + +:capture_snapshot +for /f %%T in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "SNAP_TS=%%T" +set "SNAP_DIR=%OUT_DIR%\%SNAP_TS%" +mkdir "%SNAP_DIR%" >nul 2>nul + +( + echo Timestamp: %date% %time% + echo OSCAR_PID: !OSCAR_PID! +) > "%SNAP_DIR%\summary.txt" + +tasklist /FI "PID eq !OSCAR_PID!" > "%SNAP_DIR%\tasklist.txt" 2>&1 + +powershell -NoProfile -ExecutionPolicy Bypass -Command ^ + "try { Get-Process -Id !OSCAR_PID! ^| Select-Object Id, ProcessName, StartTime, Threads, WorkingSet64, VirtualMemorySize64, PagedMemorySize64 ^| Format-List * } catch { Write-Output $_ }" ^ + > "%SNAP_DIR%\process.txt" 2>&1 + +powershell -NoProfile -ExecutionPolicy Bypass -Command ^ + "Get-CimInstance Win32_OperatingSystem ^| Select-Object TotalVisibleMemorySize, FreePhysicalMemory, TotalVirtualMemorySize, FreeVirtualMemory ^| Format-List *" ^ + > "%SNAP_DIR%\system-memory.txt" 2>&1 + +docker ps > "%SNAP_DIR%\docker-ps.txt" 2>&1 + +where jcmd >nul 2>nul +if not errorlevel 1 ( + jcmd !OSCAR_PID! VM.native_memory summary > "%SNAP_DIR%\nmt-summary.txt" 2>&1 + jcmd !OSCAR_PID! GC.heap_info > "%SNAP_DIR%\gc-heap-info.txt" 2>&1 + jcmd !OSCAR_PID! Thread.print > "%SNAP_DIR%\thread-print.txt" 2>&1 + jcmd !OSCAR_PID! JFR.check > "%SNAP_DIR%\jfr-check.txt" 2>&1 +) + exit /b 0 :load_env @@ -243,7 +203,4 @@ if not defined ENV_NAME exit /b 0 if "%ENV_NAME:~0,1%"=="#" exit /b 0 if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" set "%ENV_NAME%=%ENV_VALUE%" -exit /b 0 - -:eof_ok -exit /b 0 +exit /b 0 \ No newline at end of file diff --git a/dist/release/monitor-oscar.sh b/dist/release/monitor-oscar.sh old mode 100644 new mode 100755 index 0cb78b6..622daa6 --- a/dist/release/monitor-oscar.sh +++ b/dist/release/monitor-oscar.sh @@ -6,16 +6,28 @@ PROJECT_DIR="${PROJECT_DIR:-$SCRIPT_DIR}" LAUNCH_CMD="${LAUNCH_CMD:-$PROJECT_DIR/launch-all.sh}" MATCH_EXPR="${MATCH_EXPR:-com.botts.impl.security.SensorHubWrapper}" INTERVAL="${INTERVAL:-60}" -MAX_WAIT_SECONDS="${MAX_WAIT_SECONDS:-300}" OUT_DIR="${OUT_DIR:-$PROJECT_DIR/oscar-monitor-$(date +%Y%m%d-%H%M%S)}" JFR_NAME="${JFR_NAME:-oscar}" JFR_MAX_AGE="${JFR_MAX_AGE:-4h}" JFR_MAX_SIZE="${JFR_MAX_SIZE:-1g}" ENV_FILE="${ENV_FILE:-$PROJECT_DIR/.env}" -ATTACH_TO_EXISTING="${ATTACH_TO_EXISTING:-0}" -FORCE_RESTART="${FORCE_RESTART:-0}" +MONITOR_PID_FILE="$PROJECT_DIR/monitor.pid" + +if [ "${1:-}" = "stop" ]; then + if [ -f "$MONITOR_PID_FILE" ]; then + monitor_pid="$(tr -d '[:space:]' < "$MONITOR_PID_FILE")" + if [ -n "$monitor_pid" ] && kill -0 "$monitor_pid" 2>/dev/null; then + kill "$monitor_pid" 2>/dev/null || true + echo "OSCAR monitor stop requested for PID $monitor_pid." + exit 0 + fi + fi + echo "OSCAR monitor is not running." + exit 0 +fi mkdir -p "$OUT_DIR" +echo "$$" > "$MONITOR_PID_FILE" CONTAINER_NAME="oscar-postgis-container" DB_NAME="gis" @@ -38,71 +50,25 @@ log() { printf '%s %s\n' "$(date -Is)" "$*" } -require_cmd() { - local cmd="$1" - if ! command -v "$cmd" >/dev/null 2>&1; then - echo "Error: required command not found: $cmd" - exit 1 - fi -} - -get_java_major() { - java -version 2>&1 | awk -F'"' '/version/ { split($2, v, "."); print v[1]; exit }' -} - -check_dependencies() { - require_cmd bash - require_cmd java - require_cmd docker - require_cmd pgrep - - local java_major - java_major="$(get_java_major || true)" - if [[ -z "$java_major" || ! "$java_major" =~ ^[0-9]+$ || "$java_major" -lt 21 ]]; then - echo "Error: Java 21 or newer is required to run OSCAR monitoring." - exit 1 - fi - - if ! command -v jcmd >/dev/null 2>&1; then - log "Warning: jcmd not found. JFR/NMT snapshots will be skipped." - fi -} - -find_existing_oscar_pid() { - pgrep -f "$MATCH_EXPR" | head -n 1 || true -} - -find_all_existing_oscar_pids() { - pgrep -f "$MATCH_EXPR" || true -} - -stop_existing_oscar() { - local pids="$1" - if [ -z "$pids" ]; then - return 0 - fi - - log "Stopping existing OSCAR instance(s): $pids" - kill $pids 2>/dev/null || true +log "Monitor output: $OUT_DIR" +log "Launch command: $LAUNCH_CMD" +log "JVM match: $MATCH_EXPR" +log "Container name: $CONTAINER_NAME" +log "Database: $DB_NAME user=$DB_USER" - local waited=0 - while [ "$waited" -lt 15 ]; do - sleep 1 - waited=$((waited + 1)) - if [ -z "$(find_all_existing_oscar_pids)" ]; then - return 0 - fi - done +if [ ! -x "$LAUNCH_CMD" ]; then + echo "Error: launch command is not executable: $LAUNCH_CMD" + rm -f "$MONITOR_PID_FILE" + exit 1 +fi - log "Force killing existing OSCAR instance(s): $pids" - kill -9 $pids 2>/dev/null || true - sleep 1 +if ! command -v jcmd >/dev/null 2>&1; then + log "Warning: jcmd not found. JFR/NMT snapshots will be skipped." +fi - if [ -n "$(find_all_existing_oscar_pids)" ]; then - echo "Error: unable to stop existing OSCAR instance(s)." - exit 1 - fi -} +LAUNCH_PID="" +PID="" +STOPPING=0 run_db_query() { local sql="$1" @@ -116,7 +82,7 @@ collect_db_snapshot() { ts="$(date -Is)" failed=0 - if ! docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then + if ! command -v docker >/dev/null 2>&1 || ! docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then echo "Container ${CONTAINER_NAME} not running" > "$d/db-error.txt" echo "$ts,,,,,,,1" >> "$DB_CSV" return 0 @@ -179,9 +145,11 @@ dump_once() { jcmd "$PID" JFR.check > "$d/jfr-check.txt" 2>&1 || true fi - docker ps --filter "name=$CONTAINER_NAME" > "$d/docker-ps.txt" 2>&1 || true - docker logs --tail 100 "$CONTAINER_NAME" > "$d/docker-logs-tail.txt" 2>&1 || true - collect_db_snapshot "$d" + if command -v docker >/dev/null 2>&1; then + docker ps --filter "name=$CONTAINER_NAME" > "$d/docker-ps.txt" 2>&1 || true + docker logs --tail 100 "$CONTAINER_NAME" > "$d/docker-logs-tail.txt" 2>&1 || true + collect_db_snapshot "$d" + fi } final_dump() { @@ -195,7 +163,7 @@ final_dump() { } stop_stack() { - if [ "${STOPPING:-0}" -eq 1 ]; then + if [ "$STOPPING" -eq 1 ]; then return 0 fi STOPPING=1 @@ -223,9 +191,11 @@ stop_stack() { kill "$LAUNCH_PID" 2>/dev/null || true fi - if docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then - log "Stopping container ${CONTAINER_NAME}" - docker stop "$CONTAINER_NAME" > "$OUT_DIR/docker-stop.txt" 2>&1 || true + if command -v docker >/dev/null 2>&1; then + if docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then + log "Stopping container ${CONTAINER_NAME}" + docker stop "$CONTAINER_NAME" > "$OUT_DIR/docker-stop.txt" 2>&1 || true + fi fi } @@ -237,76 +207,37 @@ on_signal() { on_exit() { final_dump + rm -f "$MONITOR_PID_FILE" } trap on_signal INT TERM trap on_exit EXIT -check_dependencies - -log "Monitor output: $OUT_DIR" -log "Launch command: $LAUNCH_CMD" -log "JVM match: $MATCH_EXPR" -log "Container name: $CONTAINER_NAME" -log "Database: $DB_NAME user=$DB_USER" - -if [ ! -x "$LAUNCH_CMD" ] && [ "$ATTACH_TO_EXISTING" != "1" ]; then - echo "Error: launch command is not executable: $LAUNCH_CMD" - exit 1 -fi +log "Starting OSCAR..." +"$LAUNCH_CMD" > "$OUT_DIR/launch.stdout.log" 2> "$OUT_DIR/launch.stderr.log" & +LAUNCH_PID=$! +echo "$LAUNCH_PID" > "$OUT_DIR/launcher-pid.txt" -LAUNCH_PID="" -PID="" -STOPPING=0 -USE_EXISTING=0 - -existing_pids="$(find_all_existing_oscar_pids)" -if [ -n "$existing_pids" ]; then - if [ "$ATTACH_TO_EXISTING" = "1" ]; then - PID="$(printf '%s\n' "$existing_pids" | head -n 1)" - USE_EXISTING=1 - log "Attaching monitor to existing OSCAR PID $PID" - elif [ "$FORCE_RESTART" = "1" ]; then - log "Existing OSCAR instance found: $existing_pids" - stop_existing_oscar "$existing_pids" - else - echo "OSCAR is already running with PID(s): $existing_pids" - echo "Set ATTACH_TO_EXISTING=1 to monitor the running instance, or FORCE_RESTART=1 to replace it." +log "Waiting for JVM to appear..." +while true; do + PID="$(pgrep -f "$MATCH_EXPR" | head -n 1 || true)" + if [ -n "$PID" ]; then + break + fi + if ! kill -0 "$LAUNCH_PID" 2>/dev/null; then + log "Launch process exited before JVM appeared." + wait "$LAUNCH_PID" || true exit 1 fi -fi - -if [ "$USE_EXISTING" = "0" ]; then - log "Starting OSCAR..." - "$LAUNCH_CMD" > "$OUT_DIR/launch.stdout.log" 2> "$OUT_DIR/launch.stderr.log" & - LAUNCH_PID=$! - echo "$LAUNCH_PID" > "$OUT_DIR/launcher-pid.txt" - - log "Waiting for JVM to appear..." - waited=0 - while true; do - PID="$(find_existing_oscar_pid)" - if [ -n "$PID" ]; then - break - fi - if [ "$waited" -ge "$MAX_WAIT_SECONDS" ]; then - log "Timed out waiting for JVM after ${MAX_WAIT_SECONDS}s" - exit 1 - fi - sleep 2 - waited=$((waited + 2)) - done -else - : > "$OUT_DIR/launch.stdout.log" - : > "$OUT_DIR/launch.stderr.log" -fi + sleep 2 +done log "Found JVM PID: $PID" echo "$PID" > "$OUT_DIR/jvm-pid.txt" { echo "Timestamp: $(date -Is)" - echo "Launcher PID: ${LAUNCH_PID:-}" + echo "Launcher PID: $LAUNCH_PID" echo "JVM PID: $PID" echo echo "Command line:" @@ -337,6 +268,4 @@ while kill -0 "$PID" 2>/dev/null; do done log "JVM exited." -if [ -n "$LAUNCH_PID" ]; then - wait "$LAUNCH_PID" || true -fi +wait "$LAUNCH_PID" || true diff --git a/dist/release/monitor-oscar_old.bat b/dist/release/monitor-oscar_old.bat deleted file mode 100644 index 55eb8af..0000000 --- a/dist/release/monitor-oscar_old.bat +++ /dev/null @@ -1,129 +0,0 @@ -@echo off -setlocal EnableExtensions EnableDelayedExpansion - -if "%~1"=="stop" goto :stop_mode - -set "SCRIPT_DIR=%~dp0" -for %%I in ("%SCRIPT_DIR%.") do set "PROJECT_DIR=%%~fI" -set "OUT_DIR=%PROJECT_DIR%\oscar-monitor-%DATE:~10,4%%DATE:~4,2%%DATE:~7,2%-%TIME:~0,2%%TIME:~3,2%%TIME:~6,2%" -set "OUT_DIR=%OUT_DIR: =0%" -set "ENV_FILE=%PROJECT_DIR%\.env" -set "CONTAINER_NAME=oscar-postgis-container" -set "DB_NAME=gis" -set "DB_USER=postgres" -set "DB_PASSWORD=postgres" -set "MATCH_EXPR=com.botts.impl.security.SensorHubWrapper" -set "INTERVAL=60" -set "JFR_NAME=oscar" -set "JFR_MAX_AGE=4h" -set "JFR_MAX_SIZE=1g" -set "LAUNCH_CMD=%PROJECT_DIR%\launch-all.bat" - -if exist "%ENV_FILE%" ( - for /f "usebackq tokens=1,* delims==" %%A in ("%ENV_FILE%") do ( - if not "%%A"=="" if /i not "%%A:~0,1"=="#" set "%%A=%%B" - ) -) - -if not exist "%OUT_DIR%" mkdir "%OUT_DIR%" -echo timestamp,total_sessions,active,idle,idle_in_transaction,max_connections,superuser_reserved_connections,failed_psql>"%OUT_DIR%\db-connection-trend.csv" - -echo %DATE% %TIME% Monitor output: %OUT_DIR% -echo %DATE% %TIME% Launch command: %LAUNCH_CMD% - -where jcmd >nul 2>nul -if errorlevel 1 echo Warning: jcmd not found. JFR and NMT snapshots will be limited. - -start "OSCAR_LAUNCH" /b cmd /c ""%LAUNCH_CMD%" 1>"%OUT_DIR%\launch.stdout.log" 2>"%OUT_DIR%\launch.stderr.log"" - -:wait_for_jvm -timeout /t 2 /nobreak >nul -for /f "tokens=2 delims=," %%P in ('wmic process where "name='java.exe' and commandline like '%%SensorHubWrapper%%'" get processid^,commandline /format:csv ^| findstr /i SensorHubWrapper') do ( - set "JVM_PID=%%P" - goto :have_jvm -) -goto :wait_for_jvm - -:have_jvm -echo %JVM_PID%>"%OUT_DIR%\jvm-pid.txt" -if exist "%PROJECT_DIR%\monitor.pid" del "%PROJECT_DIR%\monitor.pid" -echo %PROCESS_ID%>"%PROJECT_DIR%\monitor.pid" - -jcmd %JVM_PID% JFR.start name=%JFR_NAME% settings=profile disk=true maxage=%JFR_MAX_AGE% maxsize=%JFR_MAX_SIZE% filename="%OUT_DIR%\%JFR_NAME%.jfr" >"%OUT_DIR%\jfr-start.txt" 2>&1 -jcmd %JVM_PID% VM.native_memory baseline >"%OUT_DIR%\nmt-baseline.txt" 2>&1 - -:loop -call :snapshot -for /f "tokens=2 delims=," %%P in ('wmic process where "processid=%JVM_PID%" get processid /format:csv ^| findstr /r ",[0-9][0-9]*$"') do set "ALIVE=%%P" -if not defined ALIVE goto :eof_ok -set "ALIVE=" -timeout /t %INTERVAL% /nobreak >nul -goto :loop - -:snapshot -set "STAMP=%DATE:~10,4%%DATE:~4,2%%DATE:~7,2%-%TIME:~0,2%%TIME:~3,2%%TIME:~6,2%" -set "STAMP=%STAMP: =0%" -set "SNAP=%OUT_DIR%\%STAMP%" -if not exist "%SNAP%" mkdir "%SNAP%" - -echo Collecting snapshot at %STAMP% for PID %JVM_PID% -wmic process where processid=%JVM_PID% get Name,ParentProcessId,ProcessId,ThreadCount,WorkingSetSize,VirtualSize /format:list >"%SNAP%\wmic-process.txt" 2>&1 -powershell -NoProfile -Command "$p=Get-Process -Id %JVM_PID% -ErrorAction SilentlyContinue; if($p){$p|Select-Object Id,ProcessName,Threads,VirtualMemorySize64,WorkingSet64,PrivateMemorySize64,CPU,StartTime|Format-List|Out-String}" >"%SNAP%\powershell-process.txt" 2>&1 -powershell -NoProfile -Command "Get-CimInstance Win32_OperatingSystem | Select-Object TotalVisibleMemorySize,FreePhysicalMemory,TotalVirtualMemorySize,FreeVirtualMemory | Format-List | Out-String" >"%SNAP%\memory.txt" 2>&1 -powershell -NoProfile -Command "Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\%% Usage' | Out-String" >"%SNAP%\counters.txt" 2>&1 -jcmd %JVM_PID% VM.native_memory summary >"%SNAP%\nmt-summary.txt" 2>&1 -jcmd %JVM_PID% GC.heap_info >"%SNAP%\gc-heap-info.txt" 2>&1 -jcmd %JVM_PID% Thread.print >"%SNAP%\thread-print.txt" 2>&1 -jcmd %JVM_PID% JFR.check >"%SNAP%\jfr-check.txt" 2>&1 - -docker ps --filter name=%CONTAINER_NAME% >"%SNAP%\docker-ps.txt" 2>&1 -docker logs --tail 100 %CONTAINER_NAME% >"%SNAP%\docker-logs-tail.txt" 2>&1 -call :db_snapshot "%SNAP%" -exit /b 0 - -:db_snapshot -set "SNAP=%~1" -set "DB_ERR=%SNAP%\db-error.txt" -set "FAILED=0" -set "MAX_CONN=" -set "SUPER_RESERVED=" -set "TOTAL_SESSIONS=" -set "ACTIVE_COUNT=0" -set "IDLE_COUNT=0" -set "IDLE_TX_COUNT=0" - -docker ps --format {{.Names}} | findstr /i /x "%CONTAINER_NAME%" >nul 2>&1 -if errorlevel 1 ( - >"%DB_ERR%" echo Container %CONTAINER_NAME% not running - >>"%OUT_DIR%\db-connection-trend.csv" echo %DATE%T%TIME%,,,,,,,1 - exit /b 0 -) - -docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "show max_connections;" >"%SNAP%\db-max-connections.txt" 2>"%DB_ERR%" -if errorlevel 1 set "FAILED=1" -docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "show superuser_reserved_connections;" >"%SNAP%\db-superuser-reserved-connections.txt" 2>>"%DB_ERR%" -docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select count(*) from pg_stat_activity;" >"%SNAP%\db-total-sessions.txt" 2>>"%DB_ERR%" -docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select coalesce(state,''), count(*) from pg_stat_activity group by state order by count(*) desc;" >"%SNAP%\db-by-state.txt" 2>>"%DB_ERR%" -docker exec -e PGPASSWORD=%DB_PASSWORD% %CONTAINER_NAME% psql -U %DB_USER% -d %DB_NAME% -At -c "select coalesce(application_name,''), coalesce(usename,''), coalesce(client_addr::text,''), coalesce(state,''), count(*) from pg_stat_activity group by application_name, usename, client_addr, state order by count(*) desc limit 20;" >"%SNAP%\db-by-app.txt" 2>>"%DB_ERR%" - -for /f %%A in (%SNAP%\db-max-connections.txt) do set "MAX_CONN=%%A" -for /f %%A in (%SNAP%\db-superuser-reserved-connections.txt) do set "SUPER_RESERVED=%%A" -for /f %%A in (%SNAP%\db-total-sessions.txt) do set "TOTAL_SESSIONS=%%A" -for /f "tokens=1,2 delims=|" %%A in (%SNAP%\db-by-state.txt) do ( - if /i "%%A"=="active" set "ACTIVE_COUNT=%%B" - if /i "%%A"=="idle" set "IDLE_COUNT=%%B" - if /i "%%A"=="idle in transaction" set "IDLE_TX_COUNT=%%B" -) ->>"%OUT_DIR%\db-connection-trend.csv" echo %DATE%T%TIME%,%TOTAL_SESSIONS%,%ACTIVE_COUNT%,%IDLE_COUNT%,%IDLE_TX_COUNT%,%MAX_CONN%,%SUPER_RESERVED%,%FAILED% -exit /b 0 - -:stop_mode -for /f %%P in (%~dp0monitor.pid) do set "MONPID=%%P" -if defined MONPID taskkill /PID %MONPID% /T /F >nul 2>&1 -for /f "tokens=2 delims=," %%P in ('wmic process where "name='java.exe' and commandline like '%%SensorHubWrapper%%'" get processid^,commandline /format:csv ^| findstr /i SensorHubWrapper') do taskkill /PID %%P /T /F >nul 2>&1 -for /f %%C in ('docker ps --filter name=oscar-postgis-container --format {{.Names}}') do docker stop %%C >nul 2>&1 -echo OSCAR stack stop requested. -exit /b 0 - -:eof_ok -exit /b 0 diff --git a/dist/release/monitor-oscar_old.sh b/dist/release/monitor-oscar_old.sh deleted file mode 100644 index 29fe1b1..0000000 --- a/dist/release/monitor-oscar_old.sh +++ /dev/null @@ -1,254 +0,0 @@ -#!/bin/bash -set -euo pipefail - -SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" -PROJECT_DIR="${PROJECT_DIR:-$SCRIPT_DIR}" -LAUNCH_CMD="${LAUNCH_CMD:-$PROJECT_DIR/launch-all.sh}" -MATCH_EXPR="${MATCH_EXPR:-com.botts.impl.security.SensorHubWrapper}" -INTERVAL="${INTERVAL:-60}" -OUT_DIR="${OUT_DIR:-$PROJECT_DIR/oscar-monitor-$(date +%Y%m%d-%H%M%S)}" -JFR_NAME="${JFR_NAME:-oscar}" -JFR_MAX_AGE="${JFR_MAX_AGE:-4h}" -JFR_MAX_SIZE="${JFR_MAX_SIZE:-1g}" -ENV_FILE="${ENV_FILE:-$PROJECT_DIR/.env}" - -mkdir -p "$OUT_DIR" - -CONTAINER_NAME="oscar-postgis-container" -DB_NAME="gis" -DB_USER="postgres" -DB_PASSWORD="postgres" -if [ -f "$ENV_FILE" ]; then - set -a - . "$ENV_FILE" - set +a - CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" - DB_NAME="${DB_NAME:-gis}" - DB_USER="${DB_USER:-postgres}" - DB_PASSWORD="${DB_PASSWORD:-postgres}" -fi - -DB_CSV="$OUT_DIR/db-connection-trend.csv" -echo 'timestamp,total_sessions,active,idle,idle_in_transaction,max_connections,superuser_reserved_connections,failed_psql' > "$DB_CSV" - -log() { - printf '%s %s\n' "$(date -Is)" "$*" -} - -log "Monitor output: $OUT_DIR" -log "Launch command: $LAUNCH_CMD" -log "JVM match: $MATCH_EXPR" -log "Container name: $CONTAINER_NAME" -log "Database: $DB_NAME user=$DB_USER" - -if [ ! -x "$LAUNCH_CMD" ]; then - echo "Error: launch command is not executable: $LAUNCH_CMD" - exit 1 -fi - -if ! command -v jcmd >/dev/null 2>&1; then - log "Warning: jcmd not found. JFR/NMT snapshots will be skipped." -fi - -LAUNCH_PID="" -PID="" -STOPPING=0 - -run_db_query() { - local sql="$1" - docker exec -e PGPASSWORD="$DB_PASSWORD" "$CONTAINER_NAME" \ - psql -U "$DB_USER" -d "$DB_NAME" -At -c "$sql" -} - -collect_db_snapshot() { - local d="$1" - local ts failed total active idle idle_tx max_conn super_reserved - ts="$(date -Is)" - failed=0 - - if ! command -v docker >/dev/null 2>&1 || ! docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then - echo "Container ${CONTAINER_NAME} not running" > "$d/db-error.txt" - echo "$ts,,,,,,,1" >> "$DB_CSV" - return 0 - fi - - if run_db_query "show max_connections;" > "$d/db-max-connections.txt" 2> "$d/db-error.txt"; then - run_db_query "show superuser_reserved_connections;" > "$d/db-superuser-reserved-connections.txt" 2>> "$d/db-error.txt" || failed=1 - run_db_query "select count(*) from pg_stat_activity;" > "$d/db-total-sessions.txt" 2>> "$d/db-error.txt" || failed=1 - run_db_query "select coalesce(state,''), count(*) from pg_stat_activity group by state order by count(*) desc;" > "$d/db-by-state.txt" 2>> "$d/db-error.txt" || failed=1 - run_db_query "select coalesce(application_name,''), coalesce(usename,''), coalesce(client_addr::text,''), coalesce(state,''), count(*) from pg_stat_activity group by application_name, usename, client_addr, state order by count(*) desc limit 20;" > "$d/db-by-app.txt" 2>> "$d/db-error.txt" || failed=1 - run_db_query "select pid, usename, application_name, client_addr, state, backend_start, xact_start, query_start, wait_event_type, wait_event, left(query,120) from pg_stat_activity order by backend_start;" > "$d/db-activity-detail.txt" 2>> "$d/db-error.txt" || failed=1 - else - failed=1 - fi - - max_conn="" - super_reserved="" - total="" - active="" - idle="" - idle_tx="" - - [ -f "$d/db-max-connections.txt" ] && max_conn="$(tr -d '[:space:]' < "$d/db-max-connections.txt" | tail -n 1)" - [ -f "$d/db-superuser-reserved-connections.txt" ] && super_reserved="$(tr -d '[:space:]' < "$d/db-superuser-reserved-connections.txt" | tail -n 1)" - [ -f "$d/db-total-sessions.txt" ] && total="$(tr -d '[:space:]' < "$d/db-total-sessions.txt" | tail -n 1)" - if [ -f "$d/db-by-state.txt" ]; then - active="$(awk -F'|' '$1=="active" {gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}' "$d/db-by-state.txt" | tail -n 1)" - idle="$(awk -F'|' '$1=="idle" {gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}' "$d/db-by-state.txt" | tail -n 1)" - idle_tx="$(awk -F'|' '$1=="idle in transaction" {gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}' "$d/db-by-state.txt" | tail -n 1)" - fi - echo "$ts,${total:-},${active:-0},${idle:-0},${idle_tx:-0},${max_conn:-},${super_reserved:-},$failed" >> "$DB_CSV" -} - -dump_once() { - if [ -z "${PID:-}" ] || ! kill -0 "$PID" 2>/dev/null; then - return 0 - fi - - local ts d - ts="$(date +%Y%m%d-%H%M%S)" - d="$OUT_DIR/$ts" - mkdir -p "$d" - - log "Collecting snapshot at $ts for PID $PID" - - ps -p "$PID" -o pid,ppid,user,%cpu,%mem,vsz,rss,etimes,cmd > "$d/ps.txt" 2>&1 || true - [ -r "/proc/$PID/status" ] && cat "/proc/$PID/status" > "$d/proc-status.txt" 2>&1 || true - [ -r "/proc/$PID/smaps_rollup" ] && cat "/proc/$PID/smaps_rollup" > "$d/smaps_rollup.txt" 2>&1 || true - - command -v pmap >/dev/null 2>&1 && pmap -x "$PID" > "$d/pmap-x.txt" 2>&1 || true - command -v free >/dev/null 2>&1 && free -h > "$d/free.txt" 2>&1 || true - [ -r /proc/meminfo ] && cat /proc/meminfo > "$d/meminfo.txt" 2>&1 || true - [ -r /proc/swaps ] && cat /proc/swaps > "$d/swaps.txt" 2>&1 || true - vmstat 1 5 > "$d/vmstat.txt" 2>&1 || true - - if command -v jcmd >/dev/null 2>&1; then - jcmd "$PID" VM.native_memory summary > "$d/nmt-summary.txt" 2>&1 || true - jcmd "$PID" GC.heap_info > "$d/gc-heap-info.txt" 2>&1 || true - jcmd "$PID" Thread.print > "$d/thread-print.txt" 2>&1 || true - jcmd "$PID" JFR.check > "$d/jfr-check.txt" 2>&1 || true - fi - - if command -v docker >/dev/null 2>&1; then - docker ps --filter "name=$CONTAINER_NAME" > "$d/docker-ps.txt" 2>&1 || true - docker logs --tail 100 "$CONTAINER_NAME" > "$d/docker-logs-tail.txt" 2>&1 || true - collect_db_snapshot "$d" - fi -} - -final_dump() { - if [ -n "${PID:-}" ] && kill -0 "$PID" 2>/dev/null; then - dump_once - if command -v jcmd >/dev/null 2>&1; then - jcmd "$PID" JFR.dump name="$JFR_NAME" filename="$OUT_DIR/${JFR_NAME}-final.jfr" \ - > "$OUT_DIR/jfr-dump-final.txt" 2>&1 || true - fi - fi -} - -stop_stack() { - if [ "$STOPPING" -eq 1 ]; then - return 0 - fi - STOPPING=1 - - log "Stopping OSCAR stack..." - final_dump - - if [ -n "${PID:-}" ] && kill -0 "$PID" 2>/dev/null; then - log "Stopping JVM PID $PID" - kill "$PID" 2>/dev/null || true - for _ in 1 2 3 4 5 6 7 8 9 10; do - if ! kill -0 "$PID" 2>/dev/null; then - break - fi - sleep 1 - done - if kill -0 "$PID" 2>/dev/null; then - log "Force killing JVM PID $PID" - kill -9 "$PID" 2>/dev/null || true - fi - fi - - if [ -n "${LAUNCH_PID:-}" ] && kill -0 "$LAUNCH_PID" 2>/dev/null; then - log "Stopping launcher PID $LAUNCH_PID" - kill "$LAUNCH_PID" 2>/dev/null || true - fi - - if command -v docker >/dev/null 2>&1; then - if docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then - log "Stopping container ${CONTAINER_NAME}" - docker stop "$CONTAINER_NAME" > "$OUT_DIR/docker-stop.txt" 2>&1 || true - fi - fi -} - -on_signal() { - log "Received stop signal" - stop_stack - exit 0 -} - -on_exit() { - final_dump -} - -trap on_signal INT TERM -trap on_exit EXIT - -log "Starting OSCAR..." -"$LAUNCH_CMD" > "$OUT_DIR/launch.stdout.log" 2> "$OUT_DIR/launch.stderr.log" & -LAUNCH_PID=$! -echo "$LAUNCH_PID" > "$OUT_DIR/launcher-pid.txt" - -log "Waiting for JVM to appear..." -while true; do - PID="$(pgrep -f "$MATCH_EXPR" | head -n 1 || true)" - if [ -n "$PID" ]; then - break - fi - if ! kill -0 "$LAUNCH_PID" 2>/dev/null; then - log "Launch process exited before JVM appeared." - wait "$LAUNCH_PID" || true - exit 1 - fi - sleep 2 -done - -log "Found JVM PID: $PID" -echo "$PID" > "$OUT_DIR/jvm-pid.txt" - -{ - echo "Timestamp: $(date -Is)" - echo "Launcher PID: $LAUNCH_PID" - echo "JVM PID: $PID" - echo - echo "Command line:" - tr '\0' ' ' < "/proc/$PID/cmdline" - echo -} > "$OUT_DIR/process-info.txt" - -if command -v jcmd >/dev/null 2>&1; then - log "Starting JFR on PID $PID" - jcmd "$PID" JFR.start \ - name="$JFR_NAME" \ - settings=profile \ - disk=true \ - maxage="$JFR_MAX_AGE" \ - maxsize="$JFR_MAX_SIZE" \ - filename="$OUT_DIR/${JFR_NAME}.jfr" \ - > "$OUT_DIR/jfr-start.txt" 2>&1 || true - - jcmd "$PID" VM.native_memory baseline \ - > "$OUT_DIR/nmt-baseline.txt" 2>&1 || true -fi - -dump_once - -while kill -0 "$PID" 2>/dev/null; do - sleep "$INTERVAL" - dump_once -done - -log "JVM exited." -wait "$LAUNCH_PID" || true diff --git a/dist/release/reset-all.bat b/dist/release/reset-all.bat new file mode 100755 index 0000000..75c3513 --- /dev/null +++ b/dist/release/reset-all.bat @@ -0,0 +1,96 @@ +@echo off +setlocal EnableExtensions EnableDelayedExpansion + +set "SCRIPT_DIR=%~dp0" +if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" + +set "ENV_FILE=%SCRIPT_DIR%\.env" +if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" + +if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" + +set "PGDATA_DIR=%SCRIPT_DIR%\pgdata" +set "NODE_DIR=%SCRIPT_DIR%\osh-node-oscar" +set "DB_DIR=%NODE_DIR%\db" +set "FILES_DIR=%NODE_DIR%\files" +set "CONFIG_JSON=%NODE_DIR%\config.json" +set "CONFIG_TEMPLATE=%NODE_DIR%\config.template.json" +set "SECRET_FILE=%NODE_DIR%\.s" + +echo Requesting monitor shutdown... +if exist "%SCRIPT_DIR%\monitor-oscar.bat" ( + call "%SCRIPT_DIR%\monitor-oscar.bat" stop >nul 2>nul +) + +echo Stopping OSCAR Java processes... +for /f "usebackq delims=" %%P in (` + powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { $proc.ProcessId } }" 2^>nul +`) do ( + powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %%P -Force -ErrorAction Stop } catch {}" >nul 2>nul +) + +echo Removing container: %CONTAINER_NAME%... +docker rm -f -v "%CONTAINER_NAME%" >nul 2>nul + +if exist "%PGDATA_DIR%" ( + echo Removing Postgres data directory: %PGDATA_DIR% + rmdir /s /q "%PGDATA_DIR%" +) else ( + echo Postgres data directory not found: %PGDATA_DIR% +) + +if exist "%DB_DIR%" ( + echo Removing OSCAR runtime DB directory: %DB_DIR% + rmdir /s /q "%DB_DIR%" +) else ( + echo OSCAR runtime DB directory not found: %DB_DIR% +) + +if exist "%FILES_DIR%" ( + echo Removing OSCAR files directory: %FILES_DIR% + rmdir /s /q "%FILES_DIR%" +) else ( + echo OSCAR files directory not found: %FILES_DIR% +) + +if exist "%CONFIG_TEMPLATE%" ( + echo Restoring config.json from template: %CONFIG_TEMPLATE% + copy /y "%CONFIG_TEMPLATE%" "%CONFIG_JSON%" >nul +) else ( + if exist "%CONFIG_JSON%" ( + echo WARNING: config.template.json not found. Resetting admin password placeholder in existing config.json. + powershell -NoProfile -ExecutionPolicy Bypass -Command ^ + "$path = '%CONFIG_JSON%';" ^ + "$json = Get-Content -LiteralPath $path -Raw;" ^ + "$pattern = '(\"id\"\s*:\s*\"admin\"[\s\S]*?\"password\"\s*:\s*)\"[^\"]*\"';" ^ + "$updated = [regex]::Replace($json, $pattern, '$1\"__INITIAL_ADMIN_PASSWORD__\"', 1);" ^ + "Set-Content -LiteralPath $path -Value $updated -NoNewline" + ) else ( + echo OSCAR config not found: %CONFIG_JSON% + ) +) + +echo Restoring initial admin secret file: %SECRET_FILE% +> "%SECRET_FILE%" echo oscar + +del "%SCRIPT_DIR%\.monitor-active-dir" >nul 2>nul + +echo. +echo Reset complete. +echo Next launch should initialize the default login as admin / oscar. +exit /b 0 + +:load_env +for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( + set "ENV_NAME=%%A" + set "ENV_VALUE=%%B" + call :set_env_var +) +exit /b 0 + +:set_env_var +if not defined ENV_NAME exit /b 0 +if "%ENV_NAME:~0,1%"=="#" exit /b 0 +if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" +set "%ENV_NAME%=%ENV_VALUE%" +exit /b 0 \ No newline at end of file diff --git a/dist/release/reset-all.sh b/dist/release/reset-all.sh new file mode 100755 index 0000000..ebc3877 --- /dev/null +++ b/dist/release/reset-all.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +set -u + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +ENV_FILE="${SCRIPT_DIR}/.env" +if [[ -f "${ENV_FILE}" ]]; then + while IFS='=' read -r key value; do + [[ -z "${key}" ]] && continue + [[ "${key:0:1}" == "#" ]] && continue + if [[ "${key:0:7}" == "export " ]]; then + key="${key:7}" + fi + export "${key}=${value}" + done < "${ENV_FILE}" +fi + +CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" + +PGDATA_DIR="${SCRIPT_DIR}/pgdata" +NODE_DIR="${SCRIPT_DIR}/osh-node-oscar" +DB_DIR="${NODE_DIR}/db" +FILES_DIR="${NODE_DIR}/files" +CONFIG_JSON="${NODE_DIR}/config.json" +CONFIG_TEMPLATE="${NODE_DIR}/config.template.json" +SECRET_FILE="${NODE_DIR}/.s" + +echo "Requesting monitor shutdown..." +if [[ -x "${SCRIPT_DIR}/monitor-oscar.sh" ]]; then + "${SCRIPT_DIR}/monitor-oscar.sh" stop >/dev/null 2>&1 || true +fi + +echo "Stopping OSCAR Java processes..." +pgrep -af 'com\.botts\.impl\.security\.SensorHubWrapper' >/dev/null 2>&1 && \ + pkill -f 'com\.botts\.impl\.security\.SensorHubWrapper' >/dev/null 2>&1 || true + +echo "Removing container: ${CONTAINER_NAME}..." +docker rm -f -v "${CONTAINER_NAME}" >/dev/null 2>&1 || true + +if [[ -d "${PGDATA_DIR}" ]]; then + echo "Removing Postgres data directory: ${PGDATA_DIR}" + rm -rf "${PGDATA_DIR}" +else + echo "Postgres data directory not found: ${PGDATA_DIR}" +fi + +if [[ -d "${DB_DIR}" ]]; then + echo "Removing OSCAR runtime DB directory: ${DB_DIR}" + rm -rf "${DB_DIR}" +else + echo "OSCAR runtime DB directory not found: ${DB_DIR}" +fi + +if [[ -d "${FILES_DIR}" ]]; then + echo "Removing OSCAR files directory: ${FILES_DIR}" + rm -rf "${FILES_DIR}" +else + echo "OSCAR files directory not found: ${FILES_DIR}" +fi + +if [[ -f "${CONFIG_TEMPLATE}" ]]; then + echo "Restoring config.json from template: ${CONFIG_TEMPLATE}" + cp -f "${CONFIG_TEMPLATE}" "${CONFIG_JSON}" +elif [[ -f "${CONFIG_JSON}" ]]; then + echo "WARNING: config.template.json not found. Resetting admin password placeholder in existing config.json." + perl -0pi -e 's/("id"\s*:\s*"admin"[\s\S]*?"password"\s*:\s*)"(?:[^"\\]|\\.)*"/$1"__INITIAL_ADMIN_PASSWORD__"/s' "${CONFIG_JSON}" +else + echo "OSCAR config not found: ${CONFIG_JSON}" +fi + +echo "Restoring initial admin secret file: ${SECRET_FILE}" +printf 'oscar\n' > "${SECRET_FILE}" + +rm -f "${SCRIPT_DIR}/.monitor-active-dir" + +echo +echo "Reset complete." +echo "Next launch should initialize the default login as admin / oscar." \ No newline at end of file diff --git a/dist/release/stop-all.bat b/dist/release/stop-all.bat index 9bbd92f..cd4503c 100755 --- a/dist/release/stop-all.bat +++ b/dist/release/stop-all.bat @@ -1,23 +1,47 @@ @echo off -set CONTAINER_NAME=oscar-postgis-container -set SENSORHUB_NAME=com.botts.impl.security.SensorHubWrapper +setlocal EnableExtensions -echo Stopping container: %CONTAINER_NAME%... +set "SCRIPT_DIR=%~dp0" +if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" -docker stop %CONTAINER_NAME% +set "ENV_FILE=%SCRIPT_DIR%\.env" +if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" -echo. -echo Stopping SensorHubWrapper Java Process... +if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" -FOR /F "tokens=1" %%A IN ('wmic process where "CommandLine like '%%%SENSORHUB_NAME%%%' and name='java.exe'" get ProcessId ^| findstr /R "[0-9]"') DO ( - echo Stopping SensorHubWrapper with PID %%A... - taskkill /PID %%A /F - echo SensorHubWrapper stopped. - goto :DoneJava +echo Requesting monitor shutdown... +if exist "%SCRIPT_DIR%\monitor-oscar.bat" call "%SCRIPT_DIR%\monitor-oscar.bat" stop >nul 2>nul + +echo Stopping SensorHubWrapper Java Process... +for /f "usebackq delims=" %%P in (` + powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { $proc.ProcessId } }" 2^>nul +`) do ( + powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %%P -Force -ErrorAction Stop } catch {}" >nul 2>nul ) -echo SensorHubWrapper process not found. +docker ps -a --format "{{.Names}}" | findstr /I /X "%CONTAINER_NAME%" >nul +if not errorlevel 1 ( + echo Stopping container: %CONTAINER_NAME%... + docker rm -f "%CONTAINER_NAME%" >nul 2>nul +) else ( + echo Container not found: %CONTAINER_NAME% +) -:DoneJava echo. echo Done. +exit /b 0 + +:load_env +for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( + set "ENV_NAME=%%A" + set "ENV_VALUE=%%B" + call :set_env_var +) +exit /b 0 + +:set_env_var +if not defined ENV_NAME exit /b 0 +if "%ENV_NAME:~0,1%"=="#" exit /b 0 +if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" +set "%ENV_NAME%=%ENV_VALUE%" +exit /b 0 \ No newline at end of file diff --git a/dist/release/stop-all.sh b/dist/release/stop-all.sh index 522477d..e1e552d 100755 --- a/dist/release/stop-all.sh +++ b/dist/release/stop-all.sh @@ -1,44 +1,64 @@ #!/bin/bash +set -u -CONTAINER_NAME="oscar-postgis-container" -SENSORHUB_NAME="com.botts.impl.security.SensorHubWrapper" +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" +SENSORHUB_NAME='com.botts.impl.security.SensorHubWrapper' +MONITOR_SCRIPT="$SCRIPT_DIR/monitor-oscar.sh" +MONITOR_PID_FILE="$SCRIPT_DIR/monitor.pid" -echo "Stopping container: $CONTAINER_NAME..." +request_monitor_stop() { + echo "Requesting monitor shutdown..." + if [ -f "$MONITOR_SCRIPT" ]; then + (bash "$MONITOR_SCRIPT" stop >/dev/null 2>&1 || true) & + elif [ -f "$MONITOR_PID_FILE" ]; then + monitor_pid="$(tr -d '[:space:]' < "$MONITOR_PID_FILE")" + if [ -n "$monitor_pid" ] && kill -0 "$monitor_pid" 2>/dev/null; then + kill "$monitor_pid" 2>/dev/null || true + fi + fi +} -# Stop Docker container if it exists -if docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then - echo "Container exists. Stopping..." - docker stop "$CONTAINER_NAME" - echo "Container stopped." -else - echo "Container not found. Nothing to stop." -fi +stop_sensorhub() { + local pids="" -echo -echo "Stopping SensorHubWrapper Java process..." + if command -v jps >/dev/null 2>&1; then + pids="$(jps -l | awk -v name="$SENSORHUB_NAME" '$2==name {print $1}')" + fi -PID="" + if [ -z "$pids" ] && command -v pgrep >/dev/null 2>&1; then + pids="$(pgrep -f "$SENSORHUB_NAME" || true)" + fi -# --- Option 1: Use jps if available --- -if command -v jps >/dev/null 2>&1; then - PID=$(jps -l | grep "$SENSORHUB_NAME" | awk '{print $1}') -fi + if [ -n "$pids" ]; then + echo "Stopping SensorHubWrapper with PID(s): $pids" + kill $pids 2>/dev/null || true + sleep 2 + if command -v pgrep >/dev/null 2>&1 && pgrep -f "$SENSORHUB_NAME" >/dev/null 2>&1; then + echo "Force stopping SensorHubWrapper..." + pkill -9 -f "$SENSORHUB_NAME" 2>/dev/null || true + fi + echo "SensorHubWrapper stopped." + else + echo "SensorHubWrapper process not found." + fi +} -# --- Option 2: fallback to pgrep if PID not found --- -if [ -z "$PID" ]; then - if command -v pgrep >/dev/null 2>&1; then - PID=$(pgrep -f "$SENSORHUB_NAME") +stop_container() { + echo "Stopping container: $CONTAINER_NAME..." + if command -v docker >/dev/null 2>&1 && docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then + docker stop "$CONTAINER_NAME" >/dev/null 2>&1 || true + echo "Container stop requested." + else + echo "Container not found. Nothing to stop." fi -fi - -# --- Kill process if found --- -if [ -n "$PID" ]; then - echo "Stopping SensorHubWrapper with PID(s): $PID" - kill -9 $PID - echo "SensorHubWrapper stopped." -else - echo "SensorHubWrapper process not found." -fi +} + +request_monitor_stop +sleep 2 +stop_sensorhub +stop_container +rm -f "$MONITOR_PID_FILE" echo echo "Done." diff --git a/dist/scripts/standard/launch.bat b/dist/scripts/standard/launch.bat old mode 100644 new mode 100755 index 489dffa..9b1c240 --- a/dist/scripts/standard/launch.bat +++ b/dist/scripts/standard/launch.bat @@ -1,30 +1,70 @@ @echo off -setlocal EnableExtensions +setlocal EnableExtensions EnableDelayedExpansion set "SCRIPT_DIR=%~dp0" -set "MATCH_EXPR=com.botts.impl.security.SensorHubWrapper" -set "FORCE_RESTART=%FORCE_RESTART%" -if not defined FORCE_RESTART set "FORCE_RESTART=0" +if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" set "ENV_FILE=" -if exist "%SCRIPT_DIR%.env" ( - set "ENV_FILE=%SCRIPT_DIR%.env" -) else if exist "%SCRIPT_DIR%..\.env" ( - set "ENV_FILE=%SCRIPT_DIR%..\.env" +if exist "%SCRIPT_DIR%\.env" ( + set "ENV_FILE=%SCRIPT_DIR%\.env" +) else if exist "%SCRIPT_DIR%\..\.env" ( + set "ENV_FILE=%SCRIPT_DIR%\..\.env" ) if defined ENV_FILE call :load_env "%ENV_FILE%" -call :check_java -if errorlevel 1 exit /b %ERRORLEVEL% +if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" +if not defined FORCE_RESTART set "FORCE_RESTART=0" -call :check_existing_oscar -if errorlevel 1 exit /b %ERRORLEVEL% +where java >nul 2>nul +if errorlevel 1 ( + echo ERROR: Java was not found in PATH. + exit /b 1 +) -call :ensure_runtime_paths -if errorlevel 1 exit /b %ERRORLEVEL% +where keytool >nul 2>nul +if errorlevel 1 ( + echo ERROR: keytool was not found in PATH. + exit /b 1 +) -if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" +if not exist "%SCRIPT_DIR%\lib" ( + echo ERROR: Missing library directory: "%SCRIPT_DIR%\lib" + exit /b 1 +) + +if not exist "%SCRIPT_DIR%\config.json" ( + echo ERROR: Missing config file: "%SCRIPT_DIR%\config.json" + exit /b 1 +) + +if not exist "%SCRIPT_DIR%\load_trusted_certs.bat" ( + echo ERROR: Missing trusted-certs helper: "%SCRIPT_DIR%\load_trusted_certs.bat" + exit /b 1 +) + +if not exist "%SCRIPT_DIR%\set-initial-admin-password.bat" ( + echo ERROR: Missing admin-password helper: "%SCRIPT_DIR%\set-initial-admin-password.bat" + exit /b 1 +) + +call :check_existing_oscar +if defined OSCAR_PID ( + if /I "%FORCE_RESTART%"=="1" ( + echo OSCAR is already running with PID !OSCAR_PID!. FORCE_RESTART=1, stopping it first... + call :stop_existing_oscar + call :wait_for_oscar_stop 60 + call :check_existing_oscar + if defined OSCAR_PID ( + echo ERROR: OSCAR is still running with PID !OSCAR_PID! after stop attempt. + exit /b 1 + ) + ) else ( + echo OSCAR is already running with PID !OSCAR_PID!. + echo Run stop-all.bat first, or set FORCE_RESTART=1 to replace the existing OSCAR process. + exit /b 1 + ) +) if /I "%SYSTEM_PROFILE%"=="RPI4" ( set "JAVA_XMS=512m" @@ -47,7 +87,7 @@ if /I "%SYSTEM_PROFILE%"=="RPI4" ( set "JAVACPP_MAX_BYTES_DEFAULT=4g" set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=16g" ) else ( - echo Unknown profile '%SYSTEM_PROFILE%', using 8GB defaults. + echo WARNING: Unknown SYSTEM_PROFILE "%SYSTEM_PROFILE%". Using 8GB defaults. set "JAVA_XMS=1g" set "JAVA_XMX=2g" set "JAVACPP_MAX_BYTES_DEFAULT=1g" @@ -56,33 +96,37 @@ if /I "%SYSTEM_PROFILE%"=="RPI4" ( if not defined JAVACPP_MAX_BYTES set "JAVACPP_MAX_BYTES=%JAVACPP_MAX_BYTES_DEFAULT%" if not defined JAVACPP_MAX_PHYSICAL_BYTES set "JAVACPP_MAX_PHYSICAL_BYTES=%JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT%" -if not defined JFR_FILENAME set "JFR_FILENAME=%SCRIPT_DIR%oscar.jfr" +if not defined JFR_FILENAME set "JFR_FILENAME=%SCRIPT_DIR%\oscar.jfr" + +echo Starting OSH Node with Profile: %SYSTEM_PROFILE% +echo Heap: %JAVA_XMS% / %JAVA_XMX% +echo JavaCPP maxBytes: %JAVACPP_MAX_BYTES% +echo JavaCPP maxPhysicalBytes: %JAVACPP_MAX_PHYSICAL_BYTES% +echo JFR file: %JFR_FILENAME% -if not defined HOME if defined USERPROFILE set "HOME=%USERPROFILE%" +call "%SCRIPT_DIR%\load_trusted_certs.bat" +if errorlevel 1 exit /b %ERRORLEVEL% -set "PATH=%JAVA_HOME_DETECTED%\bin;%PATH%" -set "KEYSTORE=%SCRIPT_DIR%osh-keystore.p12" +set "KEYSTORE=%SCRIPT_DIR%\osh-keystore.p12" set "KEYSTORE_TYPE=PKCS12" if not defined KEYSTORE_PASSWORD set "KEYSTORE_PASSWORD=atakatak" -set "TRUSTSTORE=%SCRIPT_DIR%truststore.jks" +set "TRUSTSTORE=%SCRIPT_DIR%\truststore.jks" set "TRUSTSTORE_TYPE=JKS" if not defined TRUSTSTORE_PASSWORD set "TRUSTSTORE_PASSWORD=changeit" -set "INITIAL_ADMIN_PASSWORD_FILE=%SCRIPT_DIR%.s" +set "INITIAL_ADMIN_PASSWORD_FILE=%SCRIPT_DIR%\.s" if not exist "%INITIAL_ADMIN_PASSWORD_FILE%" if not defined INITIAL_ADMIN_PASSWORD set "INITIAL_ADMIN_PASSWORD=admin" -echo Starting OSH Node with Profile: %SYSTEM_PROFILE% -echo Heap: %JAVA_XMS% / %JAVA_XMX% -echo JavaCPP maxBytes: %JAVACPP_MAX_BYTES% -echo JavaCPP maxPhysicalBytes: %JAVACPP_MAX_PHYSICAL_BYTES% -echo JFR file: %JFR_FILENAME% - -call "%SCRIPT_DIR%load_trusted_certs.bat" +call "%SCRIPT_DIR%\set-initial-admin-password.bat" if errorlevel 1 exit /b %ERRORLEVEL% -call "%SCRIPT_DIR%set-initial-admin-password.bat" -if errorlevel 1 exit /b %ERRORLEVEL% +set "JAVA_LIBRARY_OPT=" +if exist "%SCRIPT_DIR%\nativelibs" ( + set "JAVA_LIBRARY_OPT=-Djava.library.path=%SCRIPT_DIR%\nativelibs" +) else ( + echo WARNING: Optional native library directory not found: "%SCRIPT_DIR%\nativelibs" +) java ^ -Xms%JAVA_XMS% ^ @@ -97,136 +141,43 @@ java ^ "-Dorg.bytedeco.javacpp.maxPhysicalBytes=%JAVACPP_MAX_PHYSICAL_BYTES%" ^ -Dorg.bytedeco.javacpp.maxRetries=2 ^ -Dorg.bytedeco.javacpp.mxbean=true ^ - "-Dlogback.configurationFile=%SCRIPT_DIR%logback.xml" ^ - -cp "%SCRIPT_DIR%lib\*" ^ + "-Dlogback.configurationFile=%SCRIPT_DIR%\logback.xml" ^ + -cp "%SCRIPT_DIR%\lib\*" ^ "-Djava.system.class.loader=org.sensorhub.utils.NativeClassLoader" ^ "-Djavax.net.ssl.keyStore=%KEYSTORE%" ^ "-Djavax.net.ssl.keyStorePassword=%KEYSTORE_PASSWORD%" ^ "-Djavax.net.ssl.trustStore=%TRUSTSTORE%" ^ "-Djavax.net.ssl.trustStorePassword=%TRUSTSTORE_PASSWORD%" ^ - "-Djava.library.path=%SCRIPT_DIR%nativelibs" ^ - com.botts.impl.security.SensorHubWrapper "%SCRIPT_DIR%config.json" "%SCRIPT_DIR%db" + !JAVA_LIBRARY_OPT! ^ + com.botts.impl.security.SensorHubWrapper "%SCRIPT_DIR%\config.json" "%SCRIPT_DIR%\db" set "JAVA_EXIT_CODE=%ERRORLEVEL%" endlocal & exit /b %JAVA_EXIT_CODE% -:check_java -where java >nul 2>nul -if errorlevel 1 ( - echo Error: java was not found on PATH. Install OpenJDK 21 or newer. - exit /b 1 -) - -set "JAVA_HOME_LINE=" -for /f "delims=" %%A in ('java -XshowSettings:properties -version 2^>^&1 ^| findstr /c:"java.home ="') do ( - set "JAVA_HOME_LINE=%%A" - goto :check_java_home_line -) - -:check_java_home_line -if not defined JAVA_HOME_LINE ( - echo Error: could not determine java.home from the installed Java runtime. - exit /b 1 -) - -for /f "tokens=1,* delims==" %%A in ("%JAVA_HOME_LINE%") do set "JAVA_HOME_DETECTED=%%B" -for /f "tokens=* delims= " %%A in ("%JAVA_HOME_DETECTED%") do set "JAVA_HOME_DETECTED=%%A" - -if not exist "%JAVA_HOME_DETECTED%\bin\java.exe" ( - echo Error: Java executable not found under "%JAVA_HOME_DETECTED%\bin\java.exe". - exit /b 1 -) - -if not exist "%JAVA_HOME_DETECTED%\bin\keytool.exe" ( - echo Error: keytool.exe not found under "%JAVA_HOME_DETECTED%\bin\keytool.exe". - exit /b 1 -) - -set "JAVA_VERSION_LINE=" -for /f "delims=" %%A in ('"%JAVA_HOME_DETECTED%\bin\java.exe" -version 2^>^&1 ^| findstr /r /c:"version \""') do ( - set "JAVA_VERSION_LINE=%%A" - goto :check_java_version_line -) - -:check_java_version_line -if not defined JAVA_VERSION_LINE ( - echo Error: could not determine Java version. OpenJDK 21 or newer is required. - exit /b 1 -) - -for /f "tokens=2 delims=\"" %%A in ("%JAVA_VERSION_LINE%") do set "JAVA_VERSION_RAW=%%A" -for /f "tokens=1 delims=." %%A in ("%JAVA_VERSION_RAW%") do set "JAVA_MAJOR=%%A" - -if not defined JAVA_MAJOR ( - echo Error: could not parse Java version from "%JAVA_VERSION_LINE%". - exit /b 1 -) - -if %JAVA_MAJOR% LSS 21 ( - echo Error: Java 21 or newer is required. Found Java %JAVA_MAJOR%. - exit /b 1 -) - -exit /b 0 - -:find_existing_oscar +:check_existing_oscar set "OSCAR_PID=" -for /f %%P in ('powershell -NoProfile -Command "$p = Get-CimInstance Win32_Process ^| Where-Object { $_.Name -match ''^java(\.exe)?$'' -and $_.CommandLine -like ''*com.botts.impl.security.SensorHubWrapper*'' } ^| Select-Object -ExpandProperty ProcessId -First 1; if ($p) { Write-Output $p }"') do set "OSCAR_PID=%%P" +for /f "usebackq delims=" %%P in (` + powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { [Console]::Write($proc.ProcessId); break } }" 2^>nul +`) do set "OSCAR_PID=%%P" exit /b 0 -:check_existing_oscar -call :find_existing_oscar +:stop_existing_oscar if not defined OSCAR_PID exit /b 0 +powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %OSCAR_PID% -Force -ErrorAction Stop } catch {}" >nul 2>nul +exit /b 0 -if "%FORCE_RESTART%"=="1" ( - echo Existing OSCAR instance found with PID %OSCAR_PID%. Replacing because FORCE_RESTART=1. - taskkill /PID %OSCAR_PID% /T /F >nul 2>nul - timeout /t 2 /nobreak >nul - call :find_existing_oscar - if defined OSCAR_PID ( - echo Error: could not stop the existing OSCAR instance. - exit /b 1 - ) - exit /b 0 -) - -echo OSCAR is already running with PID %OSCAR_PID%. -echo Close the existing instance first, or set FORCE_RESTART=1 to replace it. -exit /b 1 - -:ensure_runtime_paths -if not exist "%SCRIPT_DIR%load_trusted_certs.bat" ( - echo Error: load_trusted_certs.bat not found in "%SCRIPT_DIR%". - exit /b 1 -) - -if not exist "%SCRIPT_DIR%set-initial-admin-password.bat" ( - echo Error: set-initial-admin-password.bat not found in "%SCRIPT_DIR%". - exit /b 1 -) - -if not exist "%SCRIPT_DIR%config.json" ( - echo Error: missing config file: "%SCRIPT_DIR%config.json". - exit /b 1 -) - -if not exist "%SCRIPT_DIR%lib" ( - echo Error: missing library directory: "%SCRIPT_DIR%lib". - exit /b 1 -) +:wait_for_oscar_stop +set "WAIT_LIMIT=%~1" +if not defined WAIT_LIMIT set "WAIT_LIMIT=60" +set /a WAITED=0 -if not exist "%SCRIPT_DIR%nativelibs" ( - echo Error: missing native library directory: "%SCRIPT_DIR%nativelibs". - exit /b 1 -) - -if not exist "%SCRIPT_DIR%db" mkdir "%SCRIPT_DIR%db" >nul 2>nul -if not exist "%SCRIPT_DIR%db" ( - echo Error: could not create database directory: "%SCRIPT_DIR%db". - exit /b 1 -) - -exit /b 0 +:wait_for_oscar_stop_loop +call :check_existing_oscar +if not defined OSCAR_PID exit /b 0 +if !WAITED! GEQ %WAIT_LIMIT% exit /b 0 +timeout /t 1 /nobreak >nul +set /a WAITED+=1 +goto wait_for_oscar_stop_loop :load_env for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( @@ -241,4 +192,4 @@ if not defined ENV_NAME exit /b 0 if "%ENV_NAME:~0,1%"=="#" exit /b 0 if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" set "%ENV_NAME%=%ENV_VALUE%" -exit /b 0 +exit /b 0 \ No newline at end of file diff --git a/dist/scripts/standard/launch.sh b/dist/scripts/standard/launch.sh old mode 100644 new mode 100755 index eadc411..7aec8f2 --- a/dist/scripts/standard/launch.sh +++ b/dist/scripts/standard/launch.sh @@ -13,6 +13,7 @@ load_env() { esac local name="${line%%=*}" local value="${line#*=}" + value="${value%$'\r'}" export "${name}=${value}" done < "$env_file" } @@ -108,13 +109,6 @@ ensure_runtime_paths() { exit 1 fi - if [ ! -d "$SCRIPT_DIR/nativelibs" ]; then - echo "Error: missing native library directory: $SCRIPT_DIR/nativelibs" - exit 1 - fi - - mkdir -p "$SCRIPT_DIR/db" - if [ ! -f "$SCRIPT_DIR/load_trusted_certs.sh" ]; then echo "Error: load_trusted_certs.sh not found in $SCRIPT_DIR" exit 1 @@ -124,6 +118,8 @@ ensure_runtime_paths() { echo "Error: set-initial-admin-password.sh not found in $SCRIPT_DIR" exit 1 fi + + mkdir -p "$SCRIPT_DIR/db" } ENV_FILE="" @@ -187,19 +183,26 @@ export KEYSTORE="${KEYSTORE:-$SCRIPT_DIR/osh-keystore.p12}" export KEYSTORE_TYPE="${KEYSTORE_TYPE:-PKCS12}" export KEYSTORE_PASSWORD="${KEYSTORE_PASSWORD:-atakatak}" -export TRUSTSTORE="${TRUSTSTORE:-$SCRIPT_DIR/truststore.jks}" +export TRUSTSTORE="${TRUSTSTORE:-$SCRIPT_DIR/trustStore.jks}" export TRUSTSTORE_TYPE="${TRUSTSTORE_TYPE:-JKS}" export TRUSTSTORE_PASSWORD="${TRUSTSTORE_PASSWORD:-changeit}" export INITIAL_ADMIN_PASSWORD_FILE="${INITIAL_ADMIN_PASSWORD_FILE:-$SCRIPT_DIR/.s}" if [ ! -f "$INITIAL_ADMIN_PASSWORD_FILE" ] && [ -z "${INITIAL_ADMIN_PASSWORD:-}" ]; then - export INITIAL_ADMIN_PASSWORD="admin" + export INITIAL_ADMIN_PASSWORD="oscar" fi if [ -z "${HOME:-}" ] && [ -n "${USER:-}" ]; then export HOME="/home/${USER}" fi +JAVA_LIBRARY_PATH_ARG=() +if [ -d "$SCRIPT_DIR/nativelibs" ]; then + JAVA_LIBRARY_PATH_ARG=("-Djava.library.path=$SCRIPT_DIR/nativelibs") +else + echo "Warning: optional native library directory not found: $SCRIPT_DIR/nativelibs" +fi + echo "Starting OSH Node with Profile: $SYSTEM_PROFILE" echo " Heap: $JAVA_XMS / $JAVA_XMX" echo " JavaCPP maxBytes: $JAVACPP_MAX_BYTES" @@ -225,5 +228,5 @@ exec java \ "-Dlogback.configurationFile=$SCRIPT_DIR/logback.xml" \ -cp "$SCRIPT_DIR/lib/*" \ "-Djava.system.class.loader=org.sensorhub.utils.NativeClassLoader" \ - "-Djava.library.path=$SCRIPT_DIR/nativelibs" \ + "${JAVA_LIBRARY_PATH_ARG[@]}" \ com.botts.impl.security.SensorHubWrapper "$SCRIPT_DIR/config.json" "$SCRIPT_DIR/db" diff --git a/dist/scripts/standard/launch_old.bat b/dist/scripts/standard/launch_old.bat deleted file mode 100644 index b7d52fc..0000000 --- a/dist/scripts/standard/launch_old.bat +++ /dev/null @@ -1,144 +0,0 @@ -@echo off -setlocal EnableExtensions - -rem Resolve this script's directory. %~dp0 includes the trailing backslash. -set "SCRIPT_DIR=%~dp0" - -rem Load .env from this directory, or from the parent directory when this script -rem is launched from osh-node-oscar under the project root. -set "ENV_FILE=" -if exist "%SCRIPT_DIR%.env" ( - set "ENV_FILE=%SCRIPT_DIR%.env" -) else if exist "%SCRIPT_DIR%..\.env" ( - set "ENV_FILE=%SCRIPT_DIR%..\.env" -) - -if defined ENV_FILE ( - call :load_env "%ENV_FILE%" - echo AFTER load_env ERR=%ERRORLEVEL% -) - -rem Pick Java and JavaCPP defaults from SYSTEM_PROFILE. -if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" - -if /I "%SYSTEM_PROFILE%"=="RPI4" ( - set "JAVA_XMS=512m" - set "JAVA_XMX=1536m" - set "JAVACPP_MAX_BYTES_DEFAULT=512m" - set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=2g" -) else if /I "%SYSTEM_PROFILE%"=="8GB" ( - set "JAVA_XMS=1g" - set "JAVA_XMX=2g" - set "JAVACPP_MAX_BYTES_DEFAULT=1g" - set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=4g" -) else if /I "%SYSTEM_PROFILE%"=="16GB" ( - set "JAVA_XMS=1g" - set "JAVA_XMX=3g" - set "JAVACPP_MAX_BYTES_DEFAULT=2g" - set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=8g" -) else if /I "%SYSTEM_PROFILE%"=="32GB" ( - set "JAVA_XMS=2g" - set "JAVA_XMX=6g" - set "JAVACPP_MAX_BYTES_DEFAULT=4g" - set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=16g" -) else ( - echo Unknown profile '%SYSTEM_PROFILE%', using 8GB defaults. - set "JAVA_XMS=1g" - set "JAVA_XMX=2g" - set "JAVACPP_MAX_BYTES_DEFAULT=1g" - set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=4g" -) - -if not defined JAVACPP_MAX_BYTES set "JAVACPP_MAX_BYTES=%JAVACPP_MAX_BYTES_DEFAULT%" -if not defined JAVACPP_MAX_PHYSICAL_BYTES set "JAVACPP_MAX_PHYSICAL_BYTES=%JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT%" -if not defined JFR_FILENAME set "JFR_FILENAME=%SCRIPT_DIR%oscar.jfr" - -echo Starting OSH Node with Profile: %SYSTEM_PROFILE% -echo Heap: %JAVA_XMS% / %JAVA_XMX% -echo JavaCPP maxBytes: %JAVACPP_MAX_BYTES% -echo JavaCPP maxPhysicalBytes: %JAVACPP_MAX_PHYSICAL_BYTES% -echo JFR file: %JFR_FILENAME% - -rem Make sure all the necessary certificates are trusted by the system. -if not exist "%SCRIPT_DIR%load_trusted_certs.bat" ( - echo Error: load_trusted_certs.bat not found in "%SCRIPT_DIR%". - exit /b 1 -) -call "%SCRIPT_DIR%load_trusted_certs.bat" -echo AFTER load_trusted_certs ERR=%ERRORLEVEL% -if errorlevel 1 exit /b %ERRORLEVEL% - -set "KEYSTORE=%SCRIPT_DIR%osh-keystore.p12" -set "KEYSTORE_TYPE=PKCS12" -if not defined KEYSTORE_PASSWORD set "KEYSTORE_PASSWORD=atakatak" - -set "TRUSTSTORE=%SCRIPT_DIR%truststore.jks" -set "TRUSTSTORE_TYPE=JKS" -if not defined TRUSTSTORE_PASSWORD set "TRUSTSTORE_PASSWORD=changeit" - -set "INITIAL_ADMIN_PASSWORD_FILE=%SCRIPT_DIR%.s" - -rem If no secret file exists and no env var was supplied, use the dev default. -if not exist "%INITIAL_ADMIN_PASSWORD_FILE%" if not defined INITIAL_ADMIN_PASSWORD set "INITIAL_ADMIN_PASSWORD=admin" - -if not exist "%SCRIPT_DIR%set-initial-admin-password.bat" ( - echo Error: set-initial-admin-password.bat not found in "%SCRIPT_DIR%". - exit /b 1 -) -call "%SCRIPT_DIR%set-initial-admin-password.bat" -echo AFTER set-initial-admin-password ERR=%ERRORLEVEL% -if errorlevel 1 exit /b %ERRORLEVEL% - -echo BEFORE JAVA -where java -echo KEYSTORE=%KEYSTORE% -echo TRUSTSTORE=%TRUSTSTORE% -echo INITIAL_ADMIN_PASSWORD_FILE=%INITIAL_ADMIN_PASSWORD_FILE% -echo SCRIPT_DIR=%SCRIPT_DIR% -echo CONFIG=%SCRIPT_DIR%config.json -echo DBDIR=%SCRIPT_DIR%db -echo NATIVELIBS=%SCRIPT_DIR%nativelibs - -java ^ - -Xms%JAVA_XMS% ^ - -Xmx%JAVA_XMX% ^ - -Xss256k ^ - -XX:ReservedCodeCacheSize=256m ^ - -XX:+UseG1GC ^ - -XX:+HeapDumpOnOutOfMemoryError ^ - -XX:+UnlockDiagnosticVMOptions ^ - -XX:NativeMemoryTracking=summary ^ - "-Dorg.bytedeco.javacpp.maxBytes=%JAVACPP_MAX_BYTES%" ^ - "-Dorg.bytedeco.javacpp.maxPhysicalBytes=%JAVACPP_MAX_PHYSICAL_BYTES%" ^ - -Dorg.bytedeco.javacpp.maxRetries=2 ^ - -Dorg.bytedeco.javacpp.mxbean=true ^ - "-Dlogback.configurationFile=%SCRIPT_DIR%logback.xml" ^ - -cp "%SCRIPT_DIR%lib\*" ^ - "-Djava.system.class.loader=org.sensorhub.utils.NativeClassLoader" ^ - "-Djavax.net.ssl.keyStore=%KEYSTORE%" ^ - "-Djavax.net.ssl.keyStorePassword=%KEYSTORE_PASSWORD%" ^ - "-Djavax.net.ssl.trustStore=%TRUSTSTORE%" ^ - "-Djavax.net.ssl.trustStorePassword=%TRUSTSTORE_PASSWORD%" ^ - "-Djava.library.path=%SCRIPT_DIR%nativelibs" ^ - com.botts.impl.security.SensorHubWrapper "%SCRIPT_DIR%config.json" "%SCRIPT_DIR%db" - -set "JAVA_EXIT_CODE=%ERRORLEVEL%" -echo AFTER JAVA -echo JAVA_EXIT_CODE=%JAVA_EXIT_CODE% -pause -endlocal & exit /b %JAVA_EXIT_CODE% - -:load_env -for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( - set "ENV_NAME=%%A" - set "ENV_VALUE=%%B" - call :set_env_var -) -exit /b 0 - -:set_env_var -if not defined ENV_NAME exit /b 0 -if "%ENV_NAME:~0,1%"=="#" exit /b 0 -if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" -set "%ENV_NAME%=%ENV_VALUE%" -exit /b 0 \ No newline at end of file diff --git a/dist/scripts/standard/launch_old.sh b/dist/scripts/standard/launch_old.sh deleted file mode 100644 index d88905e..0000000 --- a/dist/scripts/standard/launch_old.sh +++ /dev/null @@ -1,141 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" - -load_env() { - local env_file="$1" - while IFS= read -r line || [ -n "$line" ]; do - case "$line" in - ""|"#"*) continue ;; - export\ *) line="${line#export }" ;; - esac - local name="${line%%=*}" - local value="${line#*=}" - export "${name}=${value}" - done < "$env_file" -} - -ENV_FILE="" -if [ -f "$SCRIPT_DIR/.env" ]; then - ENV_FILE="$SCRIPT_DIR/.env" -elif [ -f "$SCRIPT_DIR/../.env" ]; then - ENV_FILE="$SCRIPT_DIR/../.env" -fi - -if [ -n "$ENV_FILE" ]; then - load_env "$ENV_FILE" -fi - -check_existing_oscar() { - local pids - pids="$(pgrep -f 'com.botts.impl.security.SensorHubWrapper' || true)" - - if [ -z "$pids" ]; then - return 0 - fi - - if [ "${FORCE_RESTART:-0}" = "1" ]; then - echo "Existing OSCAR instance found with PID(s): $pids. Stopping because FORCE_RESTART=1." - kill $pids || true - sleep 2 - return 0 - fi - - echo "OSCAR is already running with PID(s): $pids." - echo "Run stop-all.sh first, or set FORCE_RESTART=1 to replace the existing OSCAR process." - return 1 -} - -check_existing_oscar || exit 1 - -SYSTEM_PROFILE="${SYSTEM_PROFILE:-8GB}" - -case "$SYSTEM_PROFILE" in - RPI4) - JAVA_XMS="512m" - JAVA_XMX="1536m" - JAVACPP_MAX_BYTES_DEFAULT="512m" - JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="2g" - ;; - 8GB) - JAVA_XMS="1g" - JAVA_XMX="2g" - JAVACPP_MAX_BYTES_DEFAULT="1g" - JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="4g" - ;; - 16GB) - JAVA_XMS="1g" - JAVA_XMX="3g" - JAVACPP_MAX_BYTES_DEFAULT="2g" - JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="8g" - ;; - 32GB) - JAVA_XMS="2g" - JAVA_XMX="6g" - JAVACPP_MAX_BYTES_DEFAULT="4g" - JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="16g" - ;; - *) - echo "Unknown profile '$SYSTEM_PROFILE', using 8GB defaults." - JAVA_XMS="1g" - JAVA_XMX="2g" - JAVACPP_MAX_BYTES_DEFAULT="1g" - JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT="4g" - ;; -esac - -: "${JAVACPP_MAX_BYTES:=$JAVACPP_MAX_BYTES_DEFAULT}" -: "${JAVACPP_MAX_PHYSICAL_BYTES:=$JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT}" -: "${JFR_FILENAME:=$SCRIPT_DIR/oscar.jfr}" - -echo "Starting OSH Node with Profile: $SYSTEM_PROFILE" -echo " Heap: $JAVA_XMS / $JAVA_XMX" -echo " JavaCPP maxBytes: $JAVACPP_MAX_BYTES" -echo " JavaCPP maxPhysicalBytes: $JAVACPP_MAX_PHYSICAL_BYTES" -echo " JFR file: $JFR_FILENAME" - -if [ ! -f "$SCRIPT_DIR/load_trusted_certs.sh" ]; then - echo "Error: load_trusted_certs.sh not found in $SCRIPT_DIR." - exit 1 -fi -bash "$SCRIPT_DIR/load_trusted_certs.sh" - -export KEYSTORE="${KEYSTORE:-$SCRIPT_DIR/osh-keystore.p12}" -export KEYSTORE_TYPE="${KEYSTORE_TYPE:-PKCS12}" -export KEYSTORE_PASSWORD="${KEYSTORE_PASSWORD:-atakatak}" - -export TRUSTSTORE="${TRUSTSTORE:-$SCRIPT_DIR/truststore.jks}" -export TRUSTSTORE_TYPE="${TRUSTSTORE_TYPE:-JKS}" -export TRUSTSTORE_PASSWORD="${TRUSTSTORE_PASSWORD:-changeit}" - -export INITIAL_ADMIN_PASSWORD_FILE="${INITIAL_ADMIN_PASSWORD_FILE:-$SCRIPT_DIR/.s}" - -if [ ! -f "$INITIAL_ADMIN_PASSWORD_FILE" ] && [ -z "${INITIAL_ADMIN_PASSWORD:-}" ]; then - export INITIAL_ADMIN_PASSWORD="admin" -fi - -if [ ! -f "$SCRIPT_DIR/set-initial-admin-password.sh" ]; then - echo "Error: set-initial-admin-password.sh not found in $SCRIPT_DIR." - exit 1 -fi -bash "$SCRIPT_DIR/set-initial-admin-password.sh" - -exec java \ - -Xms"$JAVA_XMS" \ - -Xmx"$JAVA_XMX" \ - -Xss256k \ - -XX:ReservedCodeCacheSize=256m \ - -XX:+UseG1GC \ - -XX:+HeapDumpOnOutOfMemoryError \ - -XX:+UnlockDiagnosticVMOptions \ - -XX:NativeMemoryTracking=summary \ - "-Dorg.bytedeco.javacpp.maxBytes=$JAVACPP_MAX_BYTES" \ - "-Dorg.bytedeco.javacpp.maxPhysicalBytes=$JAVACPP_MAX_PHYSICAL_BYTES" \ - -Dorg.bytedeco.javacpp.maxRetries=2 \ - -Dorg.bytedeco.javacpp.mxbean=true \ - "-Dlogback.configurationFile=$SCRIPT_DIR/logback.xml" \ - -cp "$SCRIPT_DIR/lib/*" \ - "-Djava.system.class.loader=org.sensorhub.utils.NativeClassLoader" \ - "-Djava.library.path=$SCRIPT_DIR/nativelibs" \ - com.botts.impl.security.SensorHubWrapper "$SCRIPT_DIR/config.json" "$SCRIPT_DIR/db" \ No newline at end of file From 6af033d5d98432158af7c47b9729627456351d02 Mon Sep 17 00:00:00 2001 From: tyronechrisharris <142608718+tyronechrisharris@users.noreply.github.com> Date: Thu, 7 May 2026 16:56:25 -0400 Subject: [PATCH 4/9] update docs --- README.md | 84 ++++++- .../MediaMTX_OSCAR_camera_proxy_guide.md | 14 +- .../Node_Administration_3.5.1_addendum.md | 13 +- .../OSCAR_System_Documentation_Manual_3.5.md | 8 +- .../OSCAR_launch_monitoring_guide.md | 230 +++++++++++++++++- dist/documentation/Release_Notes_3.5.1.md | 34 ++- include/osh-addons | 2 +- 7 files changed, 345 insertions(+), 40 deletions(-) diff --git a/README.md b/README.md index d2f4d76..742d209 100644 --- a/README.md +++ b/README.md @@ -114,9 +114,11 @@ Useful optional settings include: - `RETRY_INTERVAL=2` - `POSTGIS_READY_DELAY=5` -### 5. Preferred first start: use the monitoring script +### 5. Preferred first start: use the monitoring script sessionless -For testing, burn-in, and side-by-side field deployment, start OSCAR with the monitoring wrapper instead of launching the node directly. +For testing, burn-in, side-by-side field deployment, and normal field operation, start OSCAR with the monitoring wrapper instead of launching the node directly. + +Make **sessionless** operation the default on both Linux and Windows. SSH sessions fail, RDP windows get closed, and terminals get closed during normal operations. Treat attached launches as troubleshooting-only. Windows: @@ -124,7 +126,15 @@ Windows: monitor-oscar.bat ``` -Linux: +For normal deployment on Windows, run `monitor-oscar.bat` from a **Scheduled Task** or service wrapper instead of a visible console window. + +Linux preferred sessionless start: + +```bash +nohup ./monitor-oscar.sh > monitor.out 2>&1 & +``` + +Linux attached troubleshooting start: ```bash ./monitor-oscar.sh @@ -136,6 +146,14 @@ The packaged Linux release is built so the shipped `*.sh` files are already exec chmod +x *.sh osh-node-oscar/*.sh ``` +Useful Linux follow-up commands: + +```bash +tail -f monitor.out +./check-oscar-status.sh +pgrep -af 'com.botts.impl.security.SensorHubWrapper' +``` + This is the recommended first-run path because it: - starts PostGIS and OSCAR using the current launch scripts @@ -144,7 +162,7 @@ This is the recommended first-run path because it: ### 6. Routine start without monitoring -When monitoring is not needed, use the top-level launch script: +When monitoring is intentionally not needed, use the top-level launch script. Windows: @@ -152,7 +170,15 @@ Windows: launch-all.bat ``` -Linux: +For sessionless Windows operation, run `launch-all.bat` from a **Scheduled Task** or service wrapper instead of a console window. + +Linux sessionless start: + +```bash +nohup ./launch-all.sh > launch.out 2>&1 & +``` + +Linux attached troubleshooting start: ```bash ./launch-all.sh @@ -208,7 +234,53 @@ Linux: These scripts stop monitor and OSCAR processes, remove the PostGIS container and volumes, and clear local runtime data used by the packaged deployment. -### 10. Admin access +#### Important recovery note when old lanes still appear after reset + +If a user runs `reset-all` and the next monitor run still shows **old lanes** or other stale state, do **not** keep reusing the same extracted OSCAR folder. + +Instead: + +1. run `stop-all` to stop the monitor and any remaining OSCAR processes +2. delete the **entire extracted OSCAR folder** +3. unzip `oscar-3.5.1.zip` again into a fresh folder +4. recreate `.env` +5. start again with the sessionless monitoring command + +Linux recovery example: + +```bash +./stop-all.sh +cd .. +sudo rm -rf oscar-3.5.1 +unzip oscar-3.5.1.zip +cd oscar-3.5.1 +cp env.template .env +nohup ./monitor-oscar.sh > monitor.out 2>&1 & +``` + +`sudo rm -rf` may be required on Linux because Dockerized PostgreSQL or earlier privileged operations can leave some files in the extracted tree owned by `root`. + +Windows recovery example: + +```powershell +.\stop-all.bat +Remove-Item -Recurse -Force .\oscar-3.5.1 +Expand-Archive .\oscar-3.5.1.zip -DestinationPath . +Copy-Item .\oscar-3.5.1\env.template .\oscar-3.5.1\.env +``` + +Then restart `monitor-oscar.bat` from your scheduled task or service wrapper. + +### 10. Long-lived sessionless deployments + +For systems that must survive logout, SSH disconnects, terminal closure, and host reboot, use the operating system service manager instead of a terminal window. + +- Linux: use `systemd` +- Windows: use **Task Scheduler** for built-in startup behavior, or **NSSM** if you want a true Windows service with explicit Docker dependency handling + +The detailed examples live in `dist/documentation/OSCAR_launch_monitoring_guide.md`. + +### 11. Admin access The admin username is typically **admin**. Do **not** assume the packaged password is always `admin`. diff --git a/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md b/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md index e98fc48..95c58f2 100644 --- a/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md +++ b/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md @@ -58,17 +58,19 @@ Make sure **Java 21+** and **Docker** are ready for OSCAR, and that MediaMTX is ### 2. Start OSCAR with monitoring -Linux: +Use the same **sessionless monitoring default** recommended for the rest of the packaged deployment workflow. + +Linux preferred command: ```bash -./monitor-oscar.sh +nohup ./monitor-oscar.sh > monitor.out 2>&1 & ``` -Windows: +Windows preferred pattern: -```bat -monitor-oscar.bat -``` +- run `monitor-oscar.bat` from **Task Scheduler** or a service wrapper + +Attached launches should be used only for troubleshooting. ### 3. Start MediaMTX diff --git a/dist/documentation/Node_Administration_3.5.1_addendum.md b/dist/documentation/Node_Administration_3.5.1_addendum.md index 863905b..fa07bcd 100644 --- a/dist/documentation/Node_Administration_3.5.1_addendum.md +++ b/dist/documentation/Node_Administration_3.5.1_addendum.md @@ -8,12 +8,12 @@ The existing **Node Administration** PDF remains useful for Admin UI tasks such No major Admin UI workflow changes were required for the 3.5.1 launch, monitoring, and packaging updates. -The operational changes for 3.5.1 are outside that PDF and are now covered in these updated deployment documents: +The operational changes for 3.5.1 are outside that PDF and are now covered in these deployment documents: -- `README.updated.md` -- `OSCAR_launch_monitoring_guide.updated.md` -- `MediaMTX_OSCAR_camera_proxy_guide.updated.md` -- `OSCAR_System_Documentation_Manual_3.5.updated.md` +- `README.md` +- `dist/documentation/OSCAR_launch_monitoring_guide.md` +- `dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md` +- `dist/documentation/OSCAR_System_Documentation_Manual_3.5.md` Use the PDF for Admin Panel behavior, and use the updated deployment documents for: @@ -22,4 +22,7 @@ Use the PDF for Admin Panel behavior, and use the updated deployment documents f - already-running OSCAR handling - monitoring and status scripts - fresh-install cleanup of older OSCAR releases +- sessionless default launch guidance for Windows and Linux +- recovery steps when `reset-all` leaves stale lanes behind +- daemon and service-manager deployment guidance for reboot-persistent operation - MediaMTX-assisted camera deployment guidance diff --git a/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md b/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md index aff79d1..dc7c054 100644 --- a/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md +++ b/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md @@ -86,7 +86,7 @@ The diagram below condenses the system relationships. It is not a source-code cl # 3. Installation and initial startup -Installation is intentionally simple: download a release archive, extract it, ensure Docker is installed and running, and launch the platform with the OS-specific launch-all script (`launch-all.bat` for Windows, `launch-all.sh` for Linux/macOS, or `launch-all-arm.sh` for ARM systems). The database and application come up together in the default deployment path. +Installation is intentionally simple: download a release archive, extract it, ensure Docker is installed and running, and launch the platform with the OS-specific release scripts. For packaged 3.5.1 deployments, the preferred default is the **sessionless monitoring launcher** rather than an attached console window. On Linux that typically means `nohup ./monitor-oscar.sh > monitor.out 2>&1 &`, while on Windows the preferred sessionless pattern is to run `monitor-oscar.bat` from Task Scheduler or a service wrapper. The database and application come up together in the default deployment path. ## Prerequisites @@ -105,7 +105,7 @@ Installation is intentionally simple: download a release archive, extract it, en > > 2\. Install Docker and verify that the Docker service is running before starting OSCAR. > -> 3\. Run the launch-all script (`launch-all.bat`, `launch-all.sh`, or `launch-all-arm.sh`) for the operating system in use. In the default path, the script starts PostgreSQL locally in Docker and then starts the Java application. +> 3\. For packaged 3.5.1 deployments, prefer the sessionless monitoring launcher for first start and field operation. On Linux, a representative command is `nohup ./monitor-oscar.sh > monitor.out 2>&1 &`. On Windows, the equivalent operational pattern is to start `monitor-oscar.bat` from Task Scheduler or a service wrapper so the deployment does not depend on an open console. Use `launch-all` mainly when monitoring snapshots are not needed. > > 4\. Open the application on the configured port. Port 8282 is the baseline HTTP application port, and 8443 is a representative HTTPS configuration. > @@ -120,7 +120,7 @@ Installation is intentionally simple: download a release archive, extract it, en +

If multiple extracted versions exist on the same machine, use the stop-all script to fully stop the application and Dockerized database for the version you were running. An older running container can accidentally populate data in the wrong directory and create confusion during configuration or tuning. If stale lanes still appear after a reset, stop the monitor, delete the entire extracted OSCAR folder, unzip a fresh copy, recreate .env, and start again. On Linux, sudo rm -rf may be required because some files may be left owned by root.

@@ -450,7 +450,7 @@ This section captures known gaps and enhancement ideas so they are not confused > 1\. Install Docker and verify that the service is running on the host before launch. > -> 2\. Extract the chosen OSCAR release and start it with the OS-specific launch-all script (`launch-all.bat`, `launch-all.sh`, or `launch-all-arm.sh`). +> 2\. Extract the chosen OSCAR release and start it with the preferred sessionless monitoring flow for the operating system in use. For 3.5.1 packaged deployments, that means the monitoring launcher should be the default, while attached console launches should be reserved for troubleshooting. > > 3\. Change the initial admin password using the package-provided settings file and password-initialization script before production use. > diff --git a/dist/documentation/OSCAR_launch_monitoring_guide.md b/dist/documentation/OSCAR_launch_monitoring_guide.md index acc9a3a..ce6ecee 100644 --- a/dist/documentation/OSCAR_launch_monitoring_guide.md +++ b/dist/documentation/OSCAR_launch_monitoring_guide.md @@ -15,7 +15,7 @@ It explains: ## 1. Recommended operating model -For **testing, burn-in, and side-by-side field deployment**, the preferred workflow is: +For **testing, burn-in, side-by-side field deployment, and normal field use**, the preferred workflow is: 1. unzip the prebuilt release into a fresh folder 2. create `.env` @@ -25,12 +25,19 @@ For **testing, burn-in, and side-by-side field deployment**, the preferred workf 6. run the **status-check script** 7. review JVM, thread, and PostgreSQL behavior before wider deployment +Make **sessionless** operation the default on both Linux and Windows. + +- SSH sessions fail +- terminals get closed +- RDP windows get closed +- field systems should not depend on a user keeping a console open + Use the top-level **sessionless launchers** when possible: - `launch-all.sh` / `launch-all.bat` - `monitor-oscar.sh` / `monitor-oscar.bat` -Avoid launching `osh-node-oscar/launch.(sh|bat)` directly unless you are debugging the node itself. +Treat attached launches as **troubleshooting-only**. Avoid launching `osh-node-oscar/launch.(sh|bat)` directly unless you are debugging the node itself. --- @@ -70,6 +77,43 @@ If the Docker network does not exist, that is fine. The goal is to avoid carryin For a full local reset between side-by-side test installs, use `reset-all.sh` or `reset-all.bat`. +### Important stale-state recovery when `reset-all` is not enough + +If a user runs `reset-all` and the next `monitor-oscar` run still shows **old lanes** or other stale state, do **not** keep reusing the same extracted OSCAR directory. + +Instead: + +1. run `stop-all` to stop the monitor and any remaining OSCAR processes +2. delete the **entire extracted OSCAR folder** +3. unzip `oscar-3.5.1.zip` again +4. recreate `.env` +5. start again with the preferred sessionless monitoring flow + +Linux recovery example: + +```bash +./stop-all.sh +cd .. +sudo rm -rf oscar-3.5.1 +unzip oscar-3.5.1.zip +cd oscar-3.5.1 +cp env.template .env +nohup ./monitor-oscar.sh > monitor.out 2>&1 & +``` + +`sudo rm -rf` may be required on Linux because Dockerized PostgreSQL or earlier privileged operations can leave files in the extracted directory owned by `root`. + +Windows PowerShell recovery example: + +```powershell +.\stop-all.bat +Remove-Item -Recurse -Force .\oscar-3.5.1 +Expand-Archive .\oscar-3.5.1.zip -DestinationPath . +Copy-Item .\oscar-3.5.1\env.template .\oscar-3.5.1\.env +``` + +After that, restart `monitor-oscar.bat` from your scheduled task or service wrapper. + --- ## 3. Required dependencies @@ -309,7 +353,21 @@ For monitor wrappers, you have two supported choices: ### Recommended first-run start with monitoring -#### Linux +#### Linux sessionless default + +```bash +nohup ./monitor-oscar.sh > monitor.out 2>&1 & +``` + +Useful Linux follow-up commands: + +```bash +tail -f monitor.out +./check-oscar-status.sh +pgrep -af 'com.botts.impl.security.SensorHubWrapper' +``` + +Linux attached troubleshooting start: ```bash ./monitor-oscar.sh @@ -321,7 +379,11 @@ The packaged Linux build now marks all shipped `*.sh` files executable. If your chmod +x *.sh osh-node-oscar/*.sh ``` -#### Windows +#### Windows sessionless default + +For normal Windows deployment, run `monitor-oscar.bat` from a **Scheduled Task** or service wrapper instead of a visible console window. + +Interactive troubleshooting start: ```bat monitor-oscar.bat @@ -347,7 +409,13 @@ and captures: ### Routine start without monitoring -#### Linux +#### Linux sessionless + +```bash +nohup ./launch-all.sh > launch.out 2>&1 & +``` + +#### Linux attached troubleshooting ```bash ./launch-all.sh @@ -359,6 +427,67 @@ and captures: launch-all.bat ``` +For normal Windows deployment, only use `launch-all.bat` sessionless from a **Scheduled Task** or service wrapper when you intentionally do not want monitor snapshots. + +### Exact sessionless detect and stop commands + +#### Linux + +Run with monitoring: + +```bash +nohup ./monitor-oscar.sh > monitor.out 2>&1 & +``` + +Detect a running OSCAR JVM: + +```bash +pgrep -af 'com.botts.impl.security.SensorHubWrapper' +``` + +Monitor the detached wrapper log: + +```bash +tail -f monitor.out +``` + +Stop the sessionless deployment cleanly: + +```bash +./stop-all.sh +``` + +#### Windows + +Built-in sessionless one-time detached start from PowerShell: + +```powershell +Start-Process -WindowStyle Hidden -FilePath "cmd.exe" -ArgumentList '/c cd /d C:\path\to\oscar-3.5.1 && monitor-oscar.bat > monitor.out 2>&1' +``` + +Detect a running OSCAR JVM: + +```powershell +Get-CimInstance Win32_Process | + Where-Object { + $_.Name -match '^java(\.exe)?$' -and + $_.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*' + } | + Select-Object ProcessId, CommandLine +``` + +Watch the detached log if you launched it with `monitor.out` redirection: + +```powershell +Get-Content .\monitor.out -Wait +``` + +Stop the sessionless deployment cleanly: + +```bat +stop-all.bat +``` + --- ## 9. Stopping and resetting OSCAR @@ -398,6 +527,97 @@ Windows: reset-all.bat ``` +### Boot-persistent daemon and service deployment + +For deployments that must survive logout and restart automatically after reboot, use the operating system service manager instead of a terminal window. + +#### Linux systemd example + +Make sure Docker itself is enabled first: + +```bash +sudo systemctl enable --now docker +``` + +Create `/etc/systemd/system/oscar-monitor.service`: + +```ini +[Unit] +Description=OSCAR 3.5.1 monitor +Wants=docker.service network-online.target +After=docker.service network-online.target + +[Service] +Type=simple +User=oscar +WorkingDirectory=/opt/oscar-3.5.1 +ExecStart=/bin/bash -lc './monitor-oscar.sh' +ExecStop=/bin/bash -lc './stop-all.sh' +Restart=always +RestartSec=15 + +[Install] +WantedBy=multi-user.target +``` + +Then enable and start it: + +```bash +sudo systemctl daemon-reload +sudo systemctl enable --now oscar-monitor +``` + +Useful Linux daemon commands: + +```bash +sudo systemctl status oscar-monitor +sudo journalctl -u oscar-monitor -f +sudo systemctl stop oscar-monitor +sudo systemctl restart oscar-monitor +``` + +Replace `User=` and `WorkingDirectory=` with values that match your install path. + +#### Windows built-in startup task + +Make sure Docker Desktop or Docker Engine is configured to start automatically at boot before the OSCAR task runs. Then create a startup task with a short delay so Docker is ready first: + +```powershell +schtasks /Create /TN "OSCAR Monitor" /SC ONSTART /RU SYSTEM /RL HIGHEST /DELAY 0001:00 /TR "cmd.exe /c cd /d C:\path\to\oscar-3.5.1 && monitor-oscar.bat > monitor.out 2>&1" /F +``` + +Useful Windows scheduled-task commands: + +```powershell +schtasks /Run /TN "OSCAR Monitor" +schtasks /Query /TN "OSCAR Monitor" /V /FO LIST +schtasks /End /TN "OSCAR Monitor" +``` + +After ending the task, run `stop-all.bat` if you also need to stop the OSCAR Java process and the PostGIS container cleanly. + +#### Windows service wrapper with Docker dependency + +If you want explicit service dependency handling on Windows, wrap `monitor-oscar.bat` with **NSSM** and depend on Docker Desktop's `com.docker.service` service: + +```powershell +nssm install oscar-monitor "C:\Windows\System32\cmd.exe" "/c cd /d C:\path\to\oscar-3.5.1 && monitor-oscar.bat > monitor.out 2>&1" +nssm set oscar-monitor AppDirectory "C:\path\to\oscar-3.5.1" +nssm set oscar-monitor Start SERVICE_AUTO_START +nssm set oscar-monitor DependOnService com.docker.service +nssm start oscar-monitor +``` + +Useful NSSM service commands: + +```powershell +sc query oscar-monitor +nssm stop oscar-monitor +nssm restart oscar-monitor +``` + +If you use Docker Engine instead of Docker Desktop, adjust the dependency service name to match your Docker service. + --- ## 10. Status reports diff --git a/dist/documentation/Release_Notes_3.5.1.md b/dist/documentation/Release_Notes_3.5.1.md index d45b603..6800f9d 100644 --- a/dist/documentation/Release_Notes_3.5.1.md +++ b/dist/documentation/Release_Notes_3.5.1.md @@ -37,8 +37,11 @@ For testing, side-by-side field deployment, and first-run validation: * rename `env.txt` to `.env` if needed * select the correct system profile in `.env` * use **MediaMTX** for camera-heavy deployments -* start OSCAR with the **sessionless monitoring launch** when possible +* start OSCAR with the **sessionless monitoring launch** by default +* treat attached launches as troubleshooting-only * use the new reset scripts when you need to clear a previous local test install before switching releases +* know the full recovery workflow when `reset-all` is not enough and old lanes still appear +* use `systemd` on Linux or Task Scheduler/NSSM on Windows for reboot-persistent operation * use the **check/status script** to review performance --- @@ -213,6 +216,9 @@ Documentation was added or expanded for: * MediaMTX camera proxy setup * already-running instance handling * environment template settings for restart and attach behavior +* preferred sessionless `nohup` launch examples on Linux +* full-folder recovery steps when stale lanes remain after `reset-all` +* reboot-persistent daemon and service deployment examples for Linux and Windows --- @@ -302,6 +308,8 @@ Remove the previous OSCAR folder: rm -rf ~/oscar-3.5.0 ``` +If you are clearing a reused `oscar-3.5.1` test folder and normal removal fails, use `sudo rm -rf` because Dockerized PostgreSQL or earlier privileged operations can leave files owned by `root` inside the extracted tree. + ### Windows cleanup example Stop and remove the old PostGIS container: @@ -391,36 +399,34 @@ The environment template also supports launch and monitoring behavior such as re For OSCAR 3.5.1, users should launch with the monitoring script so diagnostics begin immediately. -Use the **sessionless launch** when possible so OSCAR keeps running without requiring an attached terminal session. +Use **sessionless launch by default** so OSCAR keeps running without requiring an attached terminal session. #### Linux -Preferred: +Preferred sessionless command: ```bash -./monitor-oscar.sh --daemon +nohup ./monitor-oscar.sh > monitor.out 2>&1 & ``` -If your script version starts sessionless by default, use: +Attached launch for troubleshooting only: ```bash ./monitor-oscar.sh ``` -Use an attached launch only for interactive troubleshooting. - #### Windows -Preferred: +Preferred sessionless pattern: + +- run `monitor-oscar.bat` from **Task Scheduler** or an **NSSM** service wrapper + +Attached launch for troubleshooting only: ```bat monitor-oscar.bat ``` -Use the sessionless option if your Windows wrapper provides both attached and detached modes. - -Use an attached launch only for interactive troubleshooting. - ### Step 5: check performance with the included status script After startup, and again after the system has been running for a while, generate a status report. @@ -457,6 +463,7 @@ For testing and side-by-side field deployment, users should: 10. start OSCAR with the **sessionless monitoring launch** when possible 11. use the check or status script to compare system behavior and performance 12. use the reset script when you need to remove the local OSCAR runtime state before testing another package on the same machine +13. if old lanes still appear after `reset-all`, run `stop-all`, delete the entire extracted folder, unzip a fresh copy, recreate `.env`, and start again This is the preferred workflow for: @@ -494,11 +501,12 @@ This is the preferred workflow for: * select the correct hardware profile in `.env` * use the updated launch scripts -* use the **sessionless monitoring launch** for initial validation and normal field deployment when possible +* use the **sessionless monitoring launch** for initial validation and normal field deployment by default * use the attached launch only for interactive troubleshooting * let the scripts manage already-running instances instead of manually launching duplicates * use MediaMTX where many camera streams are involved * review generated status reports during early burn-in testing +* move long-lived field systems to `systemd` on Linux or Task Scheduler/NSSM on Windows so they survive reboot and logout ### Validation after upgrade diff --git a/include/osh-addons b/include/osh-addons index bbadeb2..5d2215d 160000 --- a/include/osh-addons +++ b/include/osh-addons @@ -1 +1 @@ -Subproject commit bbadeb2dab95b8f1b3f52c458fc6fbc0b2997f2b +Subproject commit 5d2215d2668f75d7e3b730711ddccd0edfdc35bc From 6a2fe593fd208d891a15fbcb68880e7eec5668b3 Mon Sep 17 00:00:00 2001 From: tyronechrisharris <142608718+tyronechrisharris@users.noreply.github.com> Date: Thu, 7 May 2026 23:59:51 -0400 Subject: [PATCH 5/9] Fix monitor scripts --- README.md | 143 ++---- .../Node_Administration_3.5.1_addendum.md | 17 +- .../OSCAR_System_Documentation_Manual_3.5.md | 8 +- .../OSCAR_launch_monitoring_guide.md | 261 ++--------- dist/documentation/Release_Notes_3.5.1.md | 41 +- dist/release/monitor-oscar.bat | 315 ++++++++++---- dist/release/monitor-oscar.sh | 409 +++++++++++++++--- dist/release/stop-all.bat | 58 ++- dist/release/stop-all.sh | 98 ++--- 9 files changed, 739 insertions(+), 611 deletions(-) mode change 100755 => 100644 dist/release/monitor-oscar.bat mode change 100755 => 100644 dist/release/stop-all.bat diff --git a/README.md b/README.md index 742d209..107e5fb 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ java -version docker version ``` -Use **Java 21 or newer**. The launch scripts validate dependencies and stop early if Java or Docker is missing or too old. +Use **Java 21 or newer**. The launch scripts validate dependencies and will stop early if Java or Docker is missing or too old. ### 2. If you were previously running OSCAR, start fresh @@ -71,12 +71,6 @@ Then delete the old extracted release folder, such as **oscar-3.5.0**, before ex Extract the downloaded ZIP to a fresh working directory. -The packaged release archive is expected to be named: - -```text -oscar-3.5.1.zip -``` - ### 4. Create the runtime environment file For packaged releases, use the environment file that ships with the archive: @@ -114,11 +108,9 @@ Useful optional settings include: - `RETRY_INTERVAL=2` - `POSTGIS_READY_DELAY=5` -### 5. Preferred first start: use the monitoring script sessionless +### 5. Preferred first start: use the monitoring script -For testing, burn-in, side-by-side field deployment, and normal field operation, start OSCAR with the monitoring wrapper instead of launching the node directly. - -Make **sessionless** operation the default on both Linux and Windows. SSH sessions fail, RDP windows get closed, and terminals get closed during normal operations. Treat attached launches as troubleshooting-only. +For testing, burn-in, and side-by-side field deployment, start OSCAR with the monitoring wrapper instead of launching the node directly. Windows: @@ -126,32 +118,23 @@ Windows: monitor-oscar.bat ``` -For normal deployment on Windows, run `monitor-oscar.bat` from a **Scheduled Task** or service wrapper instead of a visible console window. - -Linux preferred sessionless start: +Windows sessionless PowerShell launch: -```bash -nohup ./monitor-oscar.sh > monitor.out 2>&1 & +```powershell +Start-Process -WindowStyle Hidden -FilePath cmd.exe -ArgumentList '/c monitor-oscar.bat ^> monitor.out 2^>^&1' -WorkingDirectory (Get-Location) ``` -Linux attached troubleshooting start: +Linux: ```bash +chmod +x launch-all.sh osh-node-oscar/launch.sh monitor-oscar.sh check-oscar-status.sh ./monitor-oscar.sh ``` -The packaged Linux release is built so the shipped `*.sh` files are already executable. If your unzip tool strips execute bits, restore them with: - -```bash -chmod +x *.sh osh-node-oscar/*.sh -``` - -Useful Linux follow-up commands: +Linux sessionless launch: ```bash -tail -f monitor.out -./check-oscar-status.sh -pgrep -af 'com.botts.impl.security.SensorHubWrapper' +nohup ./monitor-oscar.sh > monitor.out 2>&1 & ``` This is the recommended first-run path because it: @@ -162,7 +145,7 @@ This is the recommended first-run path because it: ### 6. Routine start without monitoring -When monitoring is intentionally not needed, use the top-level launch script. +When monitoring is not needed, use the top-level launch script: Windows: @@ -170,15 +153,7 @@ Windows: launch-all.bat ``` -For sessionless Windows operation, run `launch-all.bat` from a **Scheduled Task** or service wrapper instead of a console window. - -Linux sessionless start: - -```bash -nohup ./launch-all.sh > launch.out 2>&1 & -``` - -Linux attached troubleshooting start: +Linux: ```bash ./launch-all.sh @@ -188,18 +163,27 @@ Prefer these **sessionless top-level launchers** over calling `osh-node-oscar/la ### 7. Running-instance handling -The launch and monitor scripts detect already-running OSCAR JVMs. +The launch and monitor scripts now detect already-running OSCAR JVMs. Default behavior: - `launch-all` refuses to start if OSCAR is already running - `monitor-oscar` refuses to start if OSCAR is already running +- `monitor-oscar` also refuses to start if another `monitor-oscar` wrapper is already active Optional behaviors: - set `FORCE_RESTART=1` to stop the running OSCAR instance and start fresh - set `ATTACH_TO_EXISTING=1` when using `monitor-oscar` to monitor the running instance instead of replacing it +When using `nohup`, a hidden `cmd.exe`, Task Scheduler, or another sessionless strategy, check these files after launch: + +- `monitor.last-status` +- `monitor.last-error` +- `monitor.out` when you redirected stdout and stderr + +If a second monitor start is refused, `monitor.last-status` records a clear failure such as `FAILED duplicate_monitor ...` so the operator can tell why the wrapper exited without staying attached to the terminal. + ### 8. Generate a status report after startup After the system has been up long enough to settle, generate a one-file report. @@ -216,71 +200,7 @@ Linux: ./check-oscar-status.sh ``` -### 9. Clean reset between side-by-side test installs - -When you need to clear the local OSCAR runtime state before standing up a different installation on the same machine, use the reset script for your platform. - -Windows: - -```bat -reset-all.bat -``` - -Linux: - -```bash -./reset-all.sh -``` - -These scripts stop monitor and OSCAR processes, remove the PostGIS container and volumes, and clear local runtime data used by the packaged deployment. - -#### Important recovery note when old lanes still appear after reset - -If a user runs `reset-all` and the next monitor run still shows **old lanes** or other stale state, do **not** keep reusing the same extracted OSCAR folder. - -Instead: - -1. run `stop-all` to stop the monitor and any remaining OSCAR processes -2. delete the **entire extracted OSCAR folder** -3. unzip `oscar-3.5.1.zip` again into a fresh folder -4. recreate `.env` -5. start again with the sessionless monitoring command - -Linux recovery example: - -```bash -./stop-all.sh -cd .. -sudo rm -rf oscar-3.5.1 -unzip oscar-3.5.1.zip -cd oscar-3.5.1 -cp env.template .env -nohup ./monitor-oscar.sh > monitor.out 2>&1 & -``` - -`sudo rm -rf` may be required on Linux because Dockerized PostgreSQL or earlier privileged operations can leave some files in the extracted tree owned by `root`. - -Windows recovery example: - -```powershell -.\stop-all.bat -Remove-Item -Recurse -Force .\oscar-3.5.1 -Expand-Archive .\oscar-3.5.1.zip -DestinationPath . -Copy-Item .\oscar-3.5.1\env.template .\oscar-3.5.1\.env -``` - -Then restart `monitor-oscar.bat` from your scheduled task or service wrapper. - -### 10. Long-lived sessionless deployments - -For systems that must survive logout, SSH disconnects, terminal closure, and host reboot, use the operating system service manager instead of a terminal window. - -- Linux: use `systemd` -- Windows: use **Task Scheduler** for built-in startup behavior, or **NSSM** if you want a true Windows service with explicit Docker dependency handling - -The detailed examples live in `dist/documentation/OSCAR_launch_monitoring_guide.md`. - -### 11. Admin access +### 9. Admin access The admin username is typically **admin**. Do **not** assume the packaged password is always `admin`. @@ -323,17 +243,7 @@ Windows: build-all.bat ``` -After the build completes, the packaged release is written under `build/distributions/` and should be named: - -```text -oscar-3.5.1.zip -``` - -The build now also: - -- makes all packaged `*.sh` files executable -- preserves Linux shell-script execute bits in the release archive -- standardizes the release ZIP name for the 3.5.1 package +After the build completes, the output is written under `build/distributions/`. ## Source-tree deployment @@ -350,7 +260,7 @@ For test systems and larger multi-lane deployments, consider placing **MediaMTX* ## PostgreSQL tuning -The packaged launch scripts size PostgreSQL by `SYSTEM_PROFILE`. +The packaged launch scripts now size PostgreSQL by `SYSTEM_PROFILE`. Representative values: @@ -386,8 +296,7 @@ Before releasing from `dev`: 3. ensure `dist/release/postgis/pgdata` is not packaged 4. verify the release ZIP name matches the intended version, such as `oscar-3.5.1.zip` 5. verify the release root directory name also matches the intended version -6. verify packaged Linux `*.sh` files are executable in the built archive -7. verify `env.template`, release notes, README, and launch documentation all reflect the same version +6. verify `env.template`, release notes, README, and launch documentation all reflect the same version ### Release steps diff --git a/dist/documentation/Node_Administration_3.5.1_addendum.md b/dist/documentation/Node_Administration_3.5.1_addendum.md index fa07bcd..0835da5 100644 --- a/dist/documentation/Node_Administration_3.5.1_addendum.md +++ b/dist/documentation/Node_Administration_3.5.1_addendum.md @@ -8,21 +8,20 @@ The existing **Node Administration** PDF remains useful for Admin UI tasks such No major Admin UI workflow changes were required for the 3.5.1 launch, monitoring, and packaging updates. -The operational changes for 3.5.1 are outside that PDF and are now covered in these deployment documents: +The operational changes for 3.5.1 are outside that PDF and are now covered in these updated deployment documents: -- `README.md` -- `dist/documentation/OSCAR_launch_monitoring_guide.md` -- `dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md` -- `dist/documentation/OSCAR_System_Documentation_Manual_3.5.md` +- `README.updated.md` +- `OSCAR_launch_monitoring_guide.updated.md` +- `MediaMTX_OSCAR_camera_proxy_guide.updated.md` +- `OSCAR_System_Documentation_Manual_3.5.updated.md` Use the PDF for Admin Panel behavior, and use the updated deployment documents for: - Java 21 and Docker prerequisites - `.env` setup - already-running OSCAR handling -- monitoring and status scripts +- monitoring and status scripts, including duplicate-monitor prevention - fresh-install cleanup of older OSCAR releases -- sessionless default launch guidance for Windows and Linux -- recovery steps when `reset-all` leaves stale lanes behind -- daemon and service-manager deployment guidance for reboot-persistent operation - MediaMTX-assisted camera deployment guidance + +The updated `monitor-oscar.sh` and `monitor-oscar.bat` wrappers now include single-instance protection so a second live monitor launch is refused until the first one is stopped. diff --git a/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md b/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md index dc7c054..aff79d1 100644 --- a/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md +++ b/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md @@ -86,7 +86,7 @@ The diagram below condenses the system relationships. It is not a source-code cl # 3. Installation and initial startup -Installation is intentionally simple: download a release archive, extract it, ensure Docker is installed and running, and launch the platform with the OS-specific release scripts. For packaged 3.5.1 deployments, the preferred default is the **sessionless monitoring launcher** rather than an attached console window. On Linux that typically means `nohup ./monitor-oscar.sh > monitor.out 2>&1 &`, while on Windows the preferred sessionless pattern is to run `monitor-oscar.bat` from Task Scheduler or a service wrapper. The database and application come up together in the default deployment path. +Installation is intentionally simple: download a release archive, extract it, ensure Docker is installed and running, and launch the platform with the OS-specific launch-all script (`launch-all.bat` for Windows, `launch-all.sh` for Linux/macOS, or `launch-all-arm.sh` for ARM systems). The database and application come up together in the default deployment path. ## Prerequisites @@ -105,7 +105,7 @@ Installation is intentionally simple: download a release archive, extract it, en > > 2\. Install Docker and verify that the Docker service is running before starting OSCAR. > -> 3\. For packaged 3.5.1 deployments, prefer the sessionless monitoring launcher for first start and field operation. On Linux, a representative command is `nohup ./monitor-oscar.sh > monitor.out 2>&1 &`. On Windows, the equivalent operational pattern is to start `monitor-oscar.bat` from Task Scheduler or a service wrapper so the deployment does not depend on an open console. Use `launch-all` mainly when monitoring snapshots are not needed. +> 3\. Run the launch-all script (`launch-all.bat`, `launch-all.sh`, or `launch-all-arm.sh`) for the operating system in use. In the default path, the script starts PostgreSQL locally in Docker and then starts the Java application. > > 4\. Open the application on the configured port. Port 8282 is the baseline HTTP application port, and 8443 is a representative HTTPS configuration. > @@ -120,7 +120,7 @@ Installation is intentionally simple: download a release archive, extract it, en +

If multiple extracted versions exist on the same machine, use the stop-all script to fully stop the application and Dockerized database for the version you were running. An older running container can accidentally populate data in the wrong directory and create confusion during configuration or tuning.

@@ -450,7 +450,7 @@ This section captures known gaps and enhancement ideas so they are not confused > 1\. Install Docker and verify that the service is running on the host before launch. > -> 2\. Extract the chosen OSCAR release and start it with the preferred sessionless monitoring flow for the operating system in use. For 3.5.1 packaged deployments, that means the monitoring launcher should be the default, while attached console launches should be reserved for troubleshooting. +> 2\. Extract the chosen OSCAR release and start it with the OS-specific launch-all script (`launch-all.bat`, `launch-all.sh`, or `launch-all-arm.sh`). > > 3\. Change the initial admin password using the package-provided settings file and password-initialization script before production use. > diff --git a/dist/documentation/OSCAR_launch_monitoring_guide.md b/dist/documentation/OSCAR_launch_monitoring_guide.md index ce6ecee..c20307d 100644 --- a/dist/documentation/OSCAR_launch_monitoring_guide.md +++ b/dist/documentation/OSCAR_launch_monitoring_guide.md @@ -15,7 +15,7 @@ It explains: ## 1. Recommended operating model -For **testing, burn-in, side-by-side field deployment, and normal field use**, the preferred workflow is: +For **testing, burn-in, and side-by-side field deployment**, the preferred workflow is: 1. unzip the prebuilt release into a fresh folder 2. create `.env` @@ -25,19 +25,12 @@ For **testing, burn-in, side-by-side field deployment, and normal field use**, t 6. run the **status-check script** 7. review JVM, thread, and PostgreSQL behavior before wider deployment -Make **sessionless** operation the default on both Linux and Windows. - -- SSH sessions fail -- terminals get closed -- RDP windows get closed -- field systems should not depend on a user keeping a console open - Use the top-level **sessionless launchers** when possible: - `launch-all.sh` / `launch-all.bat` - `monitor-oscar.sh` / `monitor-oscar.bat` -Treat attached launches as **troubleshooting-only**. Avoid launching `osh-node-oscar/launch.(sh|bat)` directly unless you are debugging the node itself. +Avoid launching `osh-node-oscar/launch.(sh|bat)` directly unless you are debugging the node itself. --- @@ -77,43 +70,6 @@ If the Docker network does not exist, that is fine. The goal is to avoid carryin For a full local reset between side-by-side test installs, use `reset-all.sh` or `reset-all.bat`. -### Important stale-state recovery when `reset-all` is not enough - -If a user runs `reset-all` and the next `monitor-oscar` run still shows **old lanes** or other stale state, do **not** keep reusing the same extracted OSCAR directory. - -Instead: - -1. run `stop-all` to stop the monitor and any remaining OSCAR processes -2. delete the **entire extracted OSCAR folder** -3. unzip `oscar-3.5.1.zip` again -4. recreate `.env` -5. start again with the preferred sessionless monitoring flow - -Linux recovery example: - -```bash -./stop-all.sh -cd .. -sudo rm -rf oscar-3.5.1 -unzip oscar-3.5.1.zip -cd oscar-3.5.1 -cp env.template .env -nohup ./monitor-oscar.sh > monitor.out 2>&1 & -``` - -`sudo rm -rf` may be required on Linux because Dockerized PostgreSQL or earlier privileged operations can leave files in the extracted directory owned by `root`. - -Windows PowerShell recovery example: - -```powershell -.\stop-all.bat -Remove-Item -Recurse -Force .\oscar-3.5.1 -Expand-Archive .\oscar-3.5.1.zip -DestinationPath . -Copy-Item .\oscar-3.5.1\env.template .\oscar-3.5.1\.env -``` - -After that, restart `monitor-oscar.bat` from your scheduled task or service wrapper. - --- ## 3. Required dependencies @@ -353,24 +309,16 @@ For monitor wrappers, you have two supported choices: ### Recommended first-run start with monitoring -#### Linux sessionless default - -```bash -nohup ./monitor-oscar.sh > monitor.out 2>&1 & -``` - -Useful Linux follow-up commands: +#### Linux ```bash -tail -f monitor.out -./check-oscar-status.sh -pgrep -af 'com.botts.impl.security.SensorHubWrapper' +./monitor-oscar.sh ``` -Linux attached troubleshooting start: +Preferred Linux sessionless launch: ```bash -./monitor-oscar.sh +nohup ./monitor-oscar.sh > monitor.out 2>&1 & ``` The packaged Linux build now marks all shipped `*.sh` files executable. If your unzip tool strips execute bits, restore them with: @@ -379,16 +327,18 @@ The packaged Linux build now marks all shipped `*.sh` files executable. If your chmod +x *.sh osh-node-oscar/*.sh ``` -#### Windows sessionless default - -For normal Windows deployment, run `monitor-oscar.bat` from a **Scheduled Task** or service wrapper instead of a visible console window. - -Interactive troubleshooting start: +#### Windows ```bat monitor-oscar.bat ``` +Preferred Windows sessionless launch from PowerShell: + +```powershell +Start-Process -WindowStyle Hidden -FilePath cmd.exe -ArgumentList '/c monitor-oscar.bat ^> monitor.out 2^>^&1' -WorkingDirectory (Get-Location) +``` + This creates an output directory such as: ```text @@ -409,13 +359,7 @@ and captures: ### Routine start without monitoring -#### Linux sessionless - -```bash -nohup ./launch-all.sh > launch.out 2>&1 & -``` - -#### Linux attached troubleshooting +#### Linux ```bash ./launch-all.sh @@ -427,74 +371,33 @@ nohup ./launch-all.sh > launch.out 2>&1 & launch-all.bat ``` -For normal Windows deployment, only use `launch-all.bat` sessionless from a **Scheduled Task** or service wrapper when you intentionally do not want monitor snapshots. - -### Exact sessionless detect and stop commands - -#### Linux - -Run with monitoring: - -```bash -nohup ./monitor-oscar.sh > monitor.out 2>&1 & -``` - -Detect a running OSCAR JVM: - -```bash -pgrep -af 'com.botts.impl.security.SensorHubWrapper' -``` - -Monitor the detached wrapper log: - -```bash -tail -f monitor.out -``` - -Stop the sessionless deployment cleanly: - -```bash -./stop-all.sh -``` - -#### Windows - -Built-in sessionless one-time detached start from PowerShell: - -```powershell -Start-Process -WindowStyle Hidden -FilePath "cmd.exe" -ArgumentList '/c cd /d C:\path\to\oscar-3.5.1 && monitor-oscar.bat > monitor.out 2>&1' -``` +--- -Detect a running OSCAR JVM: +## 9. Stopping and resetting OSCAR -```powershell -Get-CimInstance Win32_Process | - Where-Object { - $_.Name -match '^java(\.exe)?$' -and - $_.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*' - } | - Select-Object ProcessId, CommandLine -``` +### `stop-all` -Watch the detached log if you launched it with `monitor.out` redirection: +The `stop-all` scripts now begin by asking the monitor to stop first. -```powershell -Get-Content .\monitor.out -Wait -``` +### Monitor singleton protection -Stop the sessionless deployment cleanly: +`monitor-oscar.sh` and `monitor-oscar.bat` now refuse duplicate monitor starts. -```bat -stop-all.bat -``` +Typical behavior: ---- +- a second monitor launch prints a clear error and exits +- the message tells the operator to run `stop-all` or the monitor stop command first +- stale monitor lock state from an unclean exit is cleaned up automatically when no live monitor process is found +- `monitor.last-status` is updated on every start, stop, refusal, and JVM exit +- `monitor.last-error` records the latest actionable failure text -## 9. Stopping and resetting OSCAR +This closes the gap where OSCAR already protected the backend process, but the monitor wrapper could still be started twice. -### `stop-all` +For sessionless launches such as `nohup`, a hidden `cmd.exe`, Task Scheduler, or a service wrapper, operators should check: -The `stop-all` scripts now begin by asking the monitor to stop first. +- `monitor.last-status` +- `monitor.last-error` +- `monitor.out` if stdout and stderr were redirected there Behavior: @@ -527,97 +430,6 @@ Windows: reset-all.bat ``` -### Boot-persistent daemon and service deployment - -For deployments that must survive logout and restart automatically after reboot, use the operating system service manager instead of a terminal window. - -#### Linux systemd example - -Make sure Docker itself is enabled first: - -```bash -sudo systemctl enable --now docker -``` - -Create `/etc/systemd/system/oscar-monitor.service`: - -```ini -[Unit] -Description=OSCAR 3.5.1 monitor -Wants=docker.service network-online.target -After=docker.service network-online.target - -[Service] -Type=simple -User=oscar -WorkingDirectory=/opt/oscar-3.5.1 -ExecStart=/bin/bash -lc './monitor-oscar.sh' -ExecStop=/bin/bash -lc './stop-all.sh' -Restart=always -RestartSec=15 - -[Install] -WantedBy=multi-user.target -``` - -Then enable and start it: - -```bash -sudo systemctl daemon-reload -sudo systemctl enable --now oscar-monitor -``` - -Useful Linux daemon commands: - -```bash -sudo systemctl status oscar-monitor -sudo journalctl -u oscar-monitor -f -sudo systemctl stop oscar-monitor -sudo systemctl restart oscar-monitor -``` - -Replace `User=` and `WorkingDirectory=` with values that match your install path. - -#### Windows built-in startup task - -Make sure Docker Desktop or Docker Engine is configured to start automatically at boot before the OSCAR task runs. Then create a startup task with a short delay so Docker is ready first: - -```powershell -schtasks /Create /TN "OSCAR Monitor" /SC ONSTART /RU SYSTEM /RL HIGHEST /DELAY 0001:00 /TR "cmd.exe /c cd /d C:\path\to\oscar-3.5.1 && monitor-oscar.bat > monitor.out 2>&1" /F -``` - -Useful Windows scheduled-task commands: - -```powershell -schtasks /Run /TN "OSCAR Monitor" -schtasks /Query /TN "OSCAR Monitor" /V /FO LIST -schtasks /End /TN "OSCAR Monitor" -``` - -After ending the task, run `stop-all.bat` if you also need to stop the OSCAR Java process and the PostGIS container cleanly. - -#### Windows service wrapper with Docker dependency - -If you want explicit service dependency handling on Windows, wrap `monitor-oscar.bat` with **NSSM** and depend on Docker Desktop's `com.docker.service` service: - -```powershell -nssm install oscar-monitor "C:\Windows\System32\cmd.exe" "/c cd /d C:\path\to\oscar-3.5.1 && monitor-oscar.bat > monitor.out 2>&1" -nssm set oscar-monitor AppDirectory "C:\path\to\oscar-3.5.1" -nssm set oscar-monitor Start SERVICE_AUTO_START -nssm set oscar-monitor DependOnService com.docker.service -nssm start oscar-monitor -``` - -Useful NSSM service commands: - -```powershell -sc query oscar-monitor -nssm stop oscar-monitor -nssm restart oscar-monitor -``` - -If you use Docker Engine instead of Docker Desktop, adjust the dependency service name to match your Docker service. - --- ## 10. Status reports @@ -723,6 +535,17 @@ Remember: - missing `nativelibs` is warning-only - missing `trusted_certificates` is warning-only if the default Java trust store can be copied successfully +### Monitor reports that another monitor is already running + +This is expected if a previous `monitor-oscar` session is still active. + +Use: + +- `./stop-all.sh` or `monitor-oscar.bat stop` / `./monitor-oscar.sh stop` to stop the existing monitor cleanly +- the same monitor start command again after the first monitor exits + +If the earlier monitor was terminated abruptly, the next start should clean up stale monitor lock state automatically. + ### Monitor hangs waiting for Java Check `launch.stdout.log` and `launch.stderr.log` inside the newest monitor directory. diff --git a/dist/documentation/Release_Notes_3.5.1.md b/dist/documentation/Release_Notes_3.5.1.md index 6800f9d..835780e 100644 --- a/dist/documentation/Release_Notes_3.5.1.md +++ b/dist/documentation/Release_Notes_3.5.1.md @@ -37,11 +37,8 @@ For testing, side-by-side field deployment, and first-run validation: * rename `env.txt` to `.env` if needed * select the correct system profile in `.env` * use **MediaMTX** for camera-heavy deployments -* start OSCAR with the **sessionless monitoring launch** by default -* treat attached launches as troubleshooting-only +* start OSCAR with the **sessionless monitoring launch** when possible * use the new reset scripts when you need to clear a previous local test install before switching releases -* know the full recovery workflow when `reset-all` is not enough and old lanes still appear -* use `systemd` on Linux or Task Scheduler/NSSM on Windows for reboot-persistent operation * use the **check/status script** to review performance --- @@ -85,6 +82,7 @@ Improvements include: * support for stopping and relaunching cleanly when configured to do so * monitor behavior aligned with launch behavior end to end * reduced risk of duplicate Java processes and conflicting monitor sessions +* explicit single-instance protection for the Linux and Windows monitor wrappers This makes startup behavior safer during testing, upgrades, and repeated field launches. @@ -149,6 +147,8 @@ This is the most important backend stability improvement in this release. New monitoring and status-check scripts were added for both Linux and Windows. +The monitor wrappers now also include a singleton guard so a second `monitor-oscar` launch is refused while another monitor is already active. This prevents duplicate snapshot loops, duplicate JFR starts, and confusing status output during sessionless operation. The wrappers now also update `monitor.last-status` and `monitor.last-error`, which makes it much easier to understand why a sessionless launch exited without staying attached to a terminal window. + These scripts can now: * launch OSCAR under monitoring @@ -216,9 +216,6 @@ Documentation was added or expanded for: * MediaMTX camera proxy setup * already-running instance handling * environment template settings for restart and attach behavior -* preferred sessionless `nohup` launch examples on Linux -* full-folder recovery steps when stale lanes remain after `reset-all` -* reboot-persistent daemon and service deployment examples for Linux and Windows --- @@ -238,9 +235,9 @@ Large deployments were exhausting PostgreSQL connection capacity because multipl ### Duplicate launch and monitoring confusion -Repeated test starts could leave users uncertain whether OSCAR was already running, whether a second Java process had been created, or whether the monitor had attached to the correct instance. +Repeated test starts could leave users uncertain whether OSCAR was already running, whether a second Java process had been created, or whether the monitor had attached to the correct instance. A related gap was that the backend launchers had duplicate-start protection, but the monitor wrapper itself could still be started twice. -The updated scripts address this by making existing-instance behavior more explicit and consistent. +The updated scripts address this by making existing-instance behavior more explicit and consistent, and by refusing a second live monitor session. ### Limited visibility into failure mode @@ -308,8 +305,6 @@ Remove the previous OSCAR folder: rm -rf ~/oscar-3.5.0 ``` -If you are clearing a reused `oscar-3.5.1` test folder and normal removal fails, use `sudo rm -rf` because Dockerized PostgreSQL or earlier privileged operations can leave files owned by `root` inside the extracted tree. - ### Windows cleanup example Stop and remove the old PostGIS container: @@ -399,34 +394,36 @@ The environment template also supports launch and monitoring behavior such as re For OSCAR 3.5.1, users should launch with the monitoring script so diagnostics begin immediately. -Use **sessionless launch by default** so OSCAR keeps running without requiring an attached terminal session. +Use the **sessionless launch** when possible so OSCAR keeps running without requiring an attached terminal session. #### Linux -Preferred sessionless command: +Preferred: ```bash -nohup ./monitor-oscar.sh > monitor.out 2>&1 & +./monitor-oscar.sh --daemon ``` -Attached launch for troubleshooting only: +If your script version starts sessionless by default, use: ```bash ./monitor-oscar.sh ``` -#### Windows - -Preferred sessionless pattern: +Use an attached launch only for interactive troubleshooting. -- run `monitor-oscar.bat` from **Task Scheduler** or an **NSSM** service wrapper +#### Windows -Attached launch for troubleshooting only: +Preferred: ```bat monitor-oscar.bat ``` +Use the sessionless option if your Windows wrapper provides both attached and detached modes. + +Use an attached launch only for interactive troubleshooting. + ### Step 5: check performance with the included status script After startup, and again after the system has been running for a while, generate a status report. @@ -463,7 +460,6 @@ For testing and side-by-side field deployment, users should: 10. start OSCAR with the **sessionless monitoring launch** when possible 11. use the check or status script to compare system behavior and performance 12. use the reset script when you need to remove the local OSCAR runtime state before testing another package on the same machine -13. if old lanes still appear after `reset-all`, run `stop-all`, delete the entire extracted folder, unzip a fresh copy, recreate `.env`, and start again This is the preferred workflow for: @@ -501,12 +497,11 @@ This is the preferred workflow for: * select the correct hardware profile in `.env` * use the updated launch scripts -* use the **sessionless monitoring launch** for initial validation and normal field deployment by default +* use the **sessionless monitoring launch** for initial validation and normal field deployment when possible * use the attached launch only for interactive troubleshooting * let the scripts manage already-running instances instead of manually launching duplicates * use MediaMTX where many camera streams are involved * review generated status reports during early burn-in testing -* move long-lived field systems to `systemd` on Linux or Task Scheduler/NSSM on Windows so they survive reboot and logout ### Validation after upgrade diff --git a/dist/release/monitor-oscar.bat b/dist/release/monitor-oscar.bat old mode 100755 new mode 100644 index d79e88c..725d829 --- a/dist/release/monitor-oscar.bat +++ b/dist/release/monitor-oscar.bat @@ -3,103 +3,183 @@ setlocal EnableExtensions EnableDelayedExpansion set "SCRIPT_DIR=%~dp0" if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" - +set "STATE_DIR=%SCRIPT_DIR%\.monitor-state" +set "MONITOR_LOCK_DIR=%STATE_DIR%\lock" +set "HEARTBEAT_FILE=%MONITOR_LOCK_DIR%\heartbeat.txt" +set "ACTIVE_MONITOR_FILE=%STATE_DIR%\active-monitor-dir.txt" +set "STATUS_FILE=%SCRIPT_DIR%\monitor.last-status" +set "ERROR_FILE=%SCRIPT_DIR%\monitor.last-error" set "ENV_FILE=%SCRIPT_DIR%\.env" set "LAUNCH_CMD=%SCRIPT_DIR%\launch-all.bat" -set "ACTIVE_MONITOR_FILE=%SCRIPT_DIR%\.monitor-active-dir" +set "JVM_MATCH=com.botts.impl.security.SensorHubWrapper" +set "OUT_DIR=" +set "OSCAR_PID=" +set "WAITED=0" +set "STOP_REQUESTED=0" +set "CLEANUP_REASON=STOPPED" + +if not exist "%STATE_DIR%" mkdir "%STATE_DIR%" >nul 2>nul -if /I "%~1"=="stop" goto stop_monitor +if /I "%~1"=="stop" goto :stop_monitor if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" +if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" +if not defined MONITOR_INTERVAL set "MONITOR_INTERVAL=60" +if not defined MAX_WAIT_SECONDS set "MAX_WAIT_SECONDS=300" +if not defined JFR_NAME set "JFR_NAME=oscar" +if not defined JFR_MAX_AGE set "JFR_MAX_AGE=4h" +if not defined JFR_MAX_SIZE set "JFR_MAX_SIZE=1g" if not defined ATTACH_TO_EXISTING set "ATTACH_TO_EXISTING=0" if not defined FORCE_RESTART set "FORCE_RESTART=0" -if not defined MAX_WAIT_SECONDS set "MAX_WAIT_SECONDS=300" -if not defined SNAPSHOT_INTERVAL_SECONDS set "SNAPSHOT_INTERVAL_SECONDS=60" + +call :write_status STARTING monitor_batch=%~nx0 +call :clear_error if not exist "%LAUNCH_CMD%" ( - echo ERROR: Missing launch command: "%LAUNCH_CMD%" + echo Error: Missing launch command: "%LAUNCH_CMD%" + call :write_error Missing launch command: "%LAUNCH_CMD%" + call :write_status FAILED launch_command_missing path="%LAUNCH_CMD%" exit /b 1 ) -for /f %%T in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "TS=%%T" -set "OUT_DIR=%SCRIPT_DIR%\oscar-monitor-%TS%" +call :acquire_monitor_lock +if errorlevel 1 exit /b 1 +for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "STAMP=%%I" +set "OUT_DIR=%SCRIPT_DIR%\oscar-monitor-%STAMP%" mkdir "%OUT_DIR%" >nul 2>nul if errorlevel 1 ( - echo ERROR: Could not create monitor output directory: "%OUT_DIR%" + echo Error: Could not create monitor output directory "%OUT_DIR%" + call :write_error Could not create monitor output directory "%OUT_DIR%" + call :write_status FAILED out_dir_create path="%OUT_DIR%" + call :release_monitor_lock exit /b 1 ) -> "%ACTIVE_MONITOR_FILE%" echo %OUT_DIR% -> "%OUT_DIR%\launch.stdout.log" type nul -> "%OUT_DIR%\launch.stderr.log" type nul +echo %OUT_DIR%> "%ACTIVE_MONITOR_FILE%" +call :update_heartbeat +call :write_status RUNNING output="%OUT_DIR%" -echo %date% %time% Monitor output: %OUT_DIR% -echo %date% %time% Launch command: %LAUNCH_CMD% +echo Monitor output: %OUT_DIR% +echo Launching OSCAR stack... + +where jcmd >nul 2>nul +if errorlevel 1 ( + echo Warning: jcmd not found. JFR and NMT snapshots will be skipped. +) call :check_existing_oscar if defined OSCAR_PID ( if /I "%ATTACH_TO_EXISTING%"=="1" ( echo Attaching to existing OSCAR PID %OSCAR_PID%... - goto found_oscar + call :clear_error + call :write_status RUNNING attached jvm_pid=%OSCAR_PID% output="%OUT_DIR%" + goto :found_java ) if /I "%FORCE_RESTART%"=="1" ( echo Existing OSCAR detected with PID %OSCAR_PID%. FORCE_RESTART=1, replacing it... call :stop_existing_oscar call :wait_for_oscar_stop 60 - call :start_launch - goto wait_for_oscar + ) else ( + echo OSCAR is already running with PID %OSCAR_PID%. + echo Set ATTACH_TO_EXISTING=1 to monitor it, or FORCE_RESTART=1 to replace it. + call :write_error OSCAR is already running with PID %OSCAR_PID%. + call :write_status FAILED oscar_already_running pid=%OSCAR_PID% output="%OUT_DIR%" + call :release_monitor_lock + exit /b 1 ) - echo ERROR: OSCAR is already running with PID %OSCAR_PID%. - echo Set ATTACH_TO_EXISTING=1 to monitor it, or FORCE_RESTART=1 to replace it. - del "%ACTIVE_MONITOR_FILE%" >nul 2>nul - exit /b 1 ) call :start_launch +if errorlevel 1 ( + call :write_error Failed to start launch-all.bat + call :write_status FAILED launch_start output="%OUT_DIR%" + call :release_monitor_lock + exit /b 1 +) -:wait_for_oscar -echo Waiting for OSCAR Java process... +call :write_status WAITING_FOR_JVM output="%OUT_DIR%" +echo Waiting for JVM to appear... set /a WAITED=0 -:wait_loop -if exist "%OUT_DIR%\stop.request" goto cleanup +:wait_for_java +call :update_heartbeat +if exist "%OUT_DIR%\stop.request" ( + set "STOP_REQUESTED=1" + set "CLEANUP_REASON=STOPPED" + goto :cleanup +) + call :check_existing_oscar -if defined OSCAR_PID goto found_oscar +if defined OSCAR_PID goto :found_java if %WAITED% GEQ %MAX_WAIT_SECONDS% ( - echo ERROR: Could not find OSCAR Java PID after waiting. - del "%ACTIVE_MONITOR_FILE%" >nul 2>nul + echo Launch timed out before JVM appeared. + call :write_error Timed out waiting for JVM after %MAX_WAIT_SECONDS%s. Check "%OUT_DIR%\launch.stdout.log" and "%OUT_DIR%\launch.stderr.log" + call :write_status FAILED wait_for_jvm_timeout output="%OUT_DIR%" + call :release_monitor_lock exit /b 1 ) timeout /t 2 /nobreak >nul set /a WAITED+=2 -goto wait_loop +goto :wait_for_java -:found_oscar -echo Found OSCAR Java PID: %OSCAR_PID% +:found_java +echo Found JVM PID: %OSCAR_PID% > "%OUT_DIR%\jvm-pid.txt" echo %OSCAR_PID% +call :clear_error +call :write_status RUNNING jvm_pid=%OSCAR_PID% output="%OUT_DIR%" + +powershell -NoProfile -ExecutionPolicy Bypass -Command ^ + "$p = Get-CimInstance Win32_Process -Filter \"ProcessId=%OSCAR_PID%\"; if ($p) { [System.IO.File]::WriteAllText('%OUT_DIR%\\process-info.txt', ('Timestamp: ' + (Get-Date -Format o) + [Environment]::NewLine + 'JVM PID: %OSCAR_PID%' + [Environment]::NewLine + [Environment]::NewLine + 'Command line:' + [Environment]::NewLine + $p.CommandLine)) }" >nul 2>nul + +where jcmd >nul 2>nul +if not errorlevel 1 ( + jcmd %OSCAR_PID% JFR.start name=%JFR_NAME% settings=profile disk=true maxage=%JFR_MAX_AGE% maxsize=%JFR_MAX_SIZE% filename="%OUT_DIR%\%JFR_NAME%.jfr" > "%OUT_DIR%\jfr-start.txt" 2>&1 + jcmd %OSCAR_PID% VM.native_memory baseline > "%OUT_DIR%\nmt-baseline.txt" 2>&1 +) +call :snapshot + +echo Monitor is running. Use monitor-oscar.bat stop to stop the stack and dump final data. :monitor_loop -if exist "%OUT_DIR%\stop.request" goto cleanup +call :update_heartbeat +if exist "%OUT_DIR%\stop.request" ( + set "STOP_REQUESTED=1" + set "CLEANUP_REASON=STOPPED" + goto :cleanup +) call :check_existing_oscar if not defined OSCAR_PID ( - echo OSCAR Java process is no longer running. - goto cleanup + set "CLEANUP_REASON=EXITED" + goto :cleanup ) > "%OUT_DIR%\jvm-pid.txt" echo %OSCAR_PID% -call :capture_snapshot -timeout /t %SNAPSHOT_INTERVAL_SECONDS% /nobreak >nul -goto monitor_loop +call :snapshot +call :write_status RUNNING jvm_pid=%OSCAR_PID% output="%OUT_DIR%" +timeout /t %MONITOR_INTERVAL% /nobreak >nul +goto :monitor_loop :cleanup -echo Stopping monitor... +if defined OSCAR_PID call :jfr_dump +if "%STOP_REQUESTED%"=="1" ( + echo Stop requested. +) else if /I "%CLEANUP_REASON%"=="EXITED" ( + echo JVM exited. +) + del "%OUT_DIR%\stop.request" >nul 2>nul -del "%ACTIVE_MONITOR_FILE%" >nul 2>nul +call :release_monitor_lock +if /I "%CLEANUP_REASON%"=="EXITED" ( + call :write_status EXITED output="%OUT_DIR%" +) else ( + call :write_status STOPPED output="%OUT_DIR%" +) +call :clear_error exit /b 0 :stop_monitor @@ -108,29 +188,36 @@ if exist "%ACTIVE_MONITOR_FILE%" set /p MON_DIR=<"%ACTIVE_MONITOR_FILE%" if not defined MON_DIR call :find_latest_monitor_dir if not defined MON_DIR ( - echo No active monitor directory found. + echo OSCAR monitor is not running. + call :write_status STOP_REQUESTED no_active_monitor + call :clear_error + call :release_monitor_lock exit /b 0 ) if not exist "%MON_DIR%" ( - del "%ACTIVE_MONITOR_FILE%" >nul 2>nul + echo OSCAR monitor is not running. + call :write_status STOP_REQUESTED no_active_monitor + call :clear_error + call :release_monitor_lock exit /b 0 ) > "%MON_DIR%\stop.request" echo stop +call :write_status STOP_REQUESTED output="%MON_DIR%" +call :clear_error echo Requested monitor stop: "%MON_DIR%" exit /b 0 :start_launch -echo Launching OSCAR via launch-all.bat... -start "" /b cmd /c ""%LAUNCH_CMD%" 1>>"%OUT_DIR%\launch.stdout.log" 2>>"%OUT_DIR%\launch.stderr.log"" +powershell -NoProfile -ExecutionPolicy Bypass -Command ^ + "$stdout = [System.IO.Path]::Combine('%OUT_DIR%','launch.stdout.log'); $stderr = [System.IO.Path]::Combine('%OUT_DIR%','launch.stderr.log'); Start-Process -WindowStyle Hidden -FilePath 'cmd.exe' -ArgumentList @('/c','call ""%LAUNCH_CMD%""') -WorkingDirectory '%SCRIPT_DIR%' -RedirectStandardOutput $stdout -RedirectStandardError $stderr | Out-Null" +if errorlevel 1 exit /b 1 exit /b 0 :check_existing_oscar set "OSCAR_PID=" -for /f "usebackq delims=" %%P in (` - powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { [Console]::Write($proc.ProcessId); break } }" 2^>nul -`) do set "OSCAR_PID=%%P" +for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "$p = Get-CimInstance Win32_Process ^| Where-Object { $_.Name -match '^(java|javaw)(\\.exe)?$' -and $null -ne $_.CommandLine -and $_.CommandLine -like '*%JVM_MATCH%*' } ^| Select-Object -First 1 -ExpandProperty ProcessId; if ($p) { $p }"') do set "OSCAR_PID=%%I" exit /b 0 :stop_existing_oscar @@ -142,65 +229,123 @@ exit /b 0 set "WAIT_LIMIT=%~1" if not defined WAIT_LIMIT set "WAIT_LIMIT=60" set /a WAITED=0 - :wait_for_oscar_stop_loop call :check_existing_oscar if not defined OSCAR_PID exit /b 0 if !WAITED! GEQ %WAIT_LIMIT% exit /b 0 timeout /t 1 /nobreak >nul set /a WAITED+=1 -goto wait_for_oscar_stop_loop - -:find_latest_monitor_dir -set "MON_DIR=" -for /f "delims=" %%D in (' - powershell -NoProfile -ExecutionPolicy Bypass -Command "$d = Get-ChildItem -LiteralPath '%SCRIPT_DIR%' -Directory -Filter 'oscar-monitor-*' ^| Sort-Object Name -Descending ^| Select-Object -First 1 -ExpandProperty FullName; if ($d) { $d }" 2^>nul -') do set "MON_DIR=%%D" -exit /b 0 +goto :wait_for_oscar_stop_loop -:capture_snapshot -for /f %%T in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "SNAP_TS=%%T" -set "SNAP_DIR=%OUT_DIR%\%SNAP_TS%" +:snapshot +if not defined OSCAR_PID exit /b 0 +for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "SNAPSTAMP=%%I" +set "SNAP_DIR=%OUT_DIR%\%SNAPSTAMP%" mkdir "%SNAP_DIR%" >nul 2>nul -( - echo Timestamp: %date% %time% - echo OSCAR_PID: !OSCAR_PID! -) > "%SNAP_DIR%\summary.txt" - -tasklist /FI "PID eq !OSCAR_PID!" > "%SNAP_DIR%\tasklist.txt" 2>&1 - +tasklist /FI "PID eq %OSCAR_PID%" /V > "%SNAP_DIR%\tasklist.txt" 2>&1 powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "try { Get-Process -Id !OSCAR_PID! ^| Select-Object Id, ProcessName, StartTime, Threads, WorkingSet64, VirtualMemorySize64, PagedMemorySize64 ^| Format-List * } catch { Write-Output $_ }" ^ - > "%SNAP_DIR%\process.txt" 2>&1 - + "$p = Get-Process -Id %OSCAR_PID% -ErrorAction SilentlyContinue; if ($p) { $p | Select-Object Id,ProcessName,CPU,StartTime,WorkingSet64,PrivateMemorySize64,VirtualMemorySize64,HandleCount,@{Name='ThreadCount';Expression={$_.Threads.Count}} | Format-List * }" > "%SNAP_DIR%\process.txt" 2>&1 powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "Get-CimInstance Win32_OperatingSystem ^| Select-Object TotalVisibleMemorySize, FreePhysicalMemory, TotalVirtualMemorySize, FreeVirtualMemory ^| Format-List *" ^ - > "%SNAP_DIR%\system-memory.txt" 2>&1 + "Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\%% Usage' | Format-List *" > "%SNAP_DIR%\memory-counters.txt" 2>&1 +powershell -NoProfile -ExecutionPolicy Bypass -Command ^ + "Get-CimInstance Win32_OperatingSystem | Select-Object TotalVisibleMemorySize,FreePhysicalMemory,TotalVirtualMemorySize,FreeVirtualMemory | Format-List *" > "%SNAP_DIR%\os-memory.txt" 2>&1 + +where jcmd >nul 2>nul +if not errorlevel 1 ( + jcmd %OSCAR_PID% VM.native_memory summary > "%SNAP_DIR%\nmt-summary.txt" 2>&1 + jcmd %OSCAR_PID% GC.heap_info > "%SNAP_DIR%\gc-heap-info.txt" 2>&1 + jcmd %OSCAR_PID% Thread.print > "%SNAP_DIR%\thread-print.txt" 2>&1 + jcmd %OSCAR_PID% JFR.check > "%SNAP_DIR%\jfr-check.txt" 2>&1 +) docker ps > "%SNAP_DIR%\docker-ps.txt" 2>&1 +exit /b 0 +:jfr_dump where jcmd >nul 2>nul -if not errorlevel 1 ( - jcmd !OSCAR_PID! VM.native_memory summary > "%SNAP_DIR%\nmt-summary.txt" 2>&1 - jcmd !OSCAR_PID! GC.heap_info > "%SNAP_DIR%\gc-heap-info.txt" 2>&1 - jcmd !OSCAR_PID! Thread.print > "%SNAP_DIR%\thread-print.txt" 2>&1 - jcmd !OSCAR_PID! JFR.check > "%SNAP_DIR%\jfr-check.txt" 2>&1 +if errorlevel 1 exit /b 0 +if not defined OSCAR_PID exit /b 0 +jcmd %OSCAR_PID% JFR.dump name=%JFR_NAME% filename="%OUT_DIR%\%JFR_NAME%-final.jfr" > "%OUT_DIR%\jfr-dump-final.txt" 2>&1 +exit /b 0 + +:acquire_monitor_lock +if not exist "%STATE_DIR%" mkdir "%STATE_DIR%" >nul 2>nul +mkdir "%MONITOR_LOCK_DIR%" >nul 2>nul +if not errorlevel 1 exit /b 0 + +call :lock_is_fresh +if "%LOCK_FRESH%"=="1" ( + set "EXISTING_OUT_DIR=" + if exist "%ACTIVE_MONITOR_FILE%" set /p EXISTING_OUT_DIR=<"%ACTIVE_MONITOR_FILE%" + echo Error: Another monitor-oscar.bat instance is already running. + if defined EXISTING_OUT_DIR echo Active monitor output: %EXISTING_OUT_DIR% + echo Run stop-all.bat or monitor-oscar.bat stop before starting another monitor. + if defined EXISTING_OUT_DIR ( + call :write_error Duplicate monitor start refused. Existing output: %EXISTING_OUT_DIR% + call :write_status FAILED duplicate_monitor output="%EXISTING_OUT_DIR%" + ) else ( + call :write_error Duplicate monitor start refused. + call :write_status FAILED duplicate_monitor + ) + exit /b 1 +) + +echo Removing stale OSCAR monitor lock state. +call :release_monitor_lock +mkdir "%MONITOR_LOCK_DIR%" >nul 2>nul +if errorlevel 1 ( + call :write_error Could not acquire OSCAR monitor lock at "%MONITOR_LOCK_DIR%" + call :write_status FAILED lock_acquire path="%MONITOR_LOCK_DIR%" + exit /b 1 ) +exit /b 0 + +:lock_is_fresh +set "LOCK_FRESH=0" +if not exist "%HEARTBEAT_FILE%" exit /b 0 +for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "$age = (Get-Date) - (Get-Item '%HEARTBEAT_FILE%').LastWriteTime; if ($age.TotalSeconds -lt 180) { 1 } else { 0 }"') do set "LOCK_FRESH=%%I" +exit /b 0 +:update_heartbeat +if not exist "%MONITOR_LOCK_DIR%" mkdir "%MONITOR_LOCK_DIR%" >nul 2>nul +break> "%HEARTBEAT_FILE%" +exit /b 0 + +:release_monitor_lock +if exist "%HEARTBEAT_FILE%" del "%HEARTBEAT_FILE%" >nul 2>nul +if exist "%ACTIVE_MONITOR_FILE%" del "%ACTIVE_MONITOR_FILE%" >nul 2>nul +if exist "%MONITOR_LOCK_DIR%" rmdir "%MONITOR_LOCK_DIR%" >nul 2>nul +exit /b 0 + +:find_latest_monitor_dir +set "MON_DIR=" +for /f "delims=" %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "$d = Get-ChildItem -LiteralPath '%SCRIPT_DIR%' -Directory -Filter 'oscar-monitor-*' ^| Sort-Object Name -Descending ^| Select-Object -First 1 -ExpandProperty FullName; if ($d) { $d }"') do set "MON_DIR=%%I" exit /b 0 :load_env -for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( - set "ENV_NAME=%%A" - set "ENV_VALUE=%%B" - call :set_env_var +for /f "usebackq tokens=* delims=" %%L in ("%~1") do ( + set "LINE=%%L" + if defined LINE ( + if not "!LINE:~0,1!"=="#" ( + for /f "tokens=1,* delims==" %%A in ("!LINE!") do ( + if not "%%A"=="" set "%%A=%%B" + ) + ) + ) ) exit /b 0 -:set_env_var -if not defined ENV_NAME exit /b 0 -if "%ENV_NAME:~0,1%"=="#" exit /b 0 -if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" -set "%ENV_NAME%=%ENV_VALUE%" -exit /b 0 \ No newline at end of file +:write_status +set "STATUS_TEXT=%*" +> "%STATUS_FILE%" echo %date% %time% %STATUS_TEXT% +exit /b 0 + +:write_error +set "ERROR_TEXT=%*" +> "%ERROR_FILE%" echo %date% %time% %ERROR_TEXT% +exit /b 0 + +:clear_error +> "%ERROR_FILE%" type nul +exit /b 0 diff --git a/dist/release/monitor-oscar.sh b/dist/release/monitor-oscar.sh index 622daa6..c06254b 100755 --- a/dist/release/monitor-oscar.sh +++ b/dist/release/monitor-oscar.sh @@ -6,69 +6,222 @@ PROJECT_DIR="${PROJECT_DIR:-$SCRIPT_DIR}" LAUNCH_CMD="${LAUNCH_CMD:-$PROJECT_DIR/launch-all.sh}" MATCH_EXPR="${MATCH_EXPR:-com.botts.impl.security.SensorHubWrapper}" INTERVAL="${INTERVAL:-60}" +MAX_WAIT_SECONDS="${MAX_WAIT_SECONDS:-300}" OUT_DIR="${OUT_DIR:-$PROJECT_DIR/oscar-monitor-$(date +%Y%m%d-%H%M%S)}" JFR_NAME="${JFR_NAME:-oscar}" JFR_MAX_AGE="${JFR_MAX_AGE:-4h}" JFR_MAX_SIZE="${JFR_MAX_SIZE:-1g}" ENV_FILE="${ENV_FILE:-$PROJECT_DIR/.env}" -MONITOR_PID_FILE="$PROJECT_DIR/monitor.pid" +ATTACH_TO_EXISTING="${ATTACH_TO_EXISTING:-0}" +FORCE_RESTART="${FORCE_RESTART:-0}" -if [ "${1:-}" = "stop" ]; then - if [ -f "$MONITOR_PID_FILE" ]; then - monitor_pid="$(tr -d '[:space:]' < "$MONITOR_PID_FILE")" - if [ -n "$monitor_pid" ] && kill -0 "$monitor_pid" 2>/dev/null; then - kill "$monitor_pid" 2>/dev/null || true - echo "OSCAR monitor stop requested for PID $monitor_pid." - exit 0 - fi - fi - echo "OSCAR monitor is not running." - exit 0 -fi - -mkdir -p "$OUT_DIR" -echo "$$" > "$MONITOR_PID_FILE" +STATE_DIR="$PROJECT_DIR/.monitor-state" +MONITOR_LOCK_DIR="$STATE_DIR/lock" +MONITOR_PID_FILE="$STATE_DIR/monitor.pid" +ACTIVE_MONITOR_FILE="$STATE_DIR/active-monitor-dir.txt" +STATUS_FILE="$PROJECT_DIR/monitor.last-status" +ERROR_FILE="$PROJECT_DIR/monitor.last-error" CONTAINER_NAME="oscar-postgis-container" DB_NAME="gis" DB_USER="postgres" DB_PASSWORD="postgres" -if [ -f "$ENV_FILE" ]; then - set -a - . "$ENV_FILE" - set +a - CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" - DB_NAME="${DB_NAME:-gis}" - DB_USER="${DB_USER:-postgres}" - DB_PASSWORD="${DB_PASSWORD:-postgres}" -fi +DB_CSV="" -DB_CSV="$OUT_DIR/db-connection-trend.csv" -echo 'timestamp,total_sessions,active,idle,idle_in_transaction,max_connections,superuser_reserved_connections,failed_psql' > "$DB_CSV" +LAUNCH_PID="" +PID="" +STOPPING=0 +USE_EXISTING=0 +MONITOR_LOCK_OWNED=0 +FINAL_STATUS_WRITTEN=0 log() { printf '%s %s\n' "$(date -Is)" "$*" } -log "Monitor output: $OUT_DIR" -log "Launch command: $LAUNCH_CMD" -log "JVM match: $MATCH_EXPR" -log "Container name: $CONTAINER_NAME" -log "Database: $DB_NAME user=$DB_USER" +write_status() { + printf '%s %s\n' "$(date -Is)" "$*" > "$STATUS_FILE" +} -if [ ! -x "$LAUNCH_CMD" ]; then - echo "Error: launch command is not executable: $LAUNCH_CMD" - rm -f "$MONITOR_PID_FILE" +write_error() { + printf '%s %s\n' "$(date -Is)" "$*" > "$ERROR_FILE" +} + +clear_error() { + : > "$ERROR_FILE" +} + +finalize_status() { + write_status "$*" + FINAL_STATUS_WRITTEN=1 +} + +require_cmd() { + local cmd="$1" + if ! command -v "$cmd" >/dev/null 2>&1; then + echo "Error: required command not found: $cmd" >&2 + write_error "Missing required command: $cmd" + finalize_status "FAILED missing_dependency command=$cmd" + exit 1 + fi +} + +get_java_major() { + java -version 2>&1 | awk -F'"' '/version/ { split($2, v, "."); print v[1]; exit }' +} + +check_dependencies() { + require_cmd bash + require_cmd java + require_cmd docker + require_cmd pgrep + + local java_major + java_major="$(get_java_major || true)" + if [[ -z "$java_major" || ! "$java_major" =~ ^[0-9]+$ || "$java_major" -lt 21 ]]; then + echo "Error: Java 21 or newer is required to run OSCAR monitoring." >&2 + write_error "Java 21 or newer is required to run OSCAR monitoring." + finalize_status "FAILED java_too_old" + exit 1 + fi + + if ! command -v jcmd >/dev/null 2>&1; then + log "Warning: jcmd not found. JFR/NMT snapshots will be skipped." + fi +} + +read_monitor_pid() { + if [ -f "$MONITOR_PID_FILE" ]; then + tr -d '[:space:]' < "$MONITOR_PID_FILE" + fi +} + +is_monitor_pid_running() { + local pid="$1" + [ -n "$pid" ] || return 1 + kill -0 "$pid" 2>/dev/null || return 1 + ps -p "$pid" -o args= 2>/dev/null | grep -Fq "monitor-oscar.sh" +} + +remove_monitor_state() { + rm -f "$MONITOR_PID_FILE" "$ACTIVE_MONITOR_FILE" + if [ -d "$MONITOR_LOCK_DIR" ]; then + rmdir "$MONITOR_LOCK_DIR" 2>/dev/null || rm -rf "$MONITOR_LOCK_DIR" 2>/dev/null || true + fi +} + +release_monitor_lock() { + if [ "$MONITOR_LOCK_OWNED" -eq 1 ]; then + local current_pid="" + current_pid="$(read_monitor_pid || true)" + if [ "$current_pid" = "$$" ] || [ -z "$current_pid" ]; then + remove_monitor_state + fi + MONITOR_LOCK_OWNED=0 + fi +} + +refuse_existing_monitor() { + local existing_pid="$1" + local existing_dir="" + if [ -f "$ACTIVE_MONITOR_FILE" ]; then + existing_dir="$(cat "$ACTIVE_MONITOR_FILE" 2>/dev/null || true)" + fi + + echo "Error: Another monitor-oscar.sh instance is already running with PID $existing_pid." >&2 + if [ -n "$existing_dir" ]; then + echo "Active monitor output: $existing_dir" >&2 + fi + echo "Run ./stop-all.sh or ./monitor-oscar.sh stop before starting another monitor." >&2 + + if [ -n "$existing_dir" ]; then + write_error "Duplicate monitor start refused. Existing monitor PID=$existing_pid output=$existing_dir" + finalize_status "FAILED duplicate_monitor existing_pid=$existing_pid output=$existing_dir" + else + write_error "Duplicate monitor start refused. Existing monitor PID=$existing_pid" + finalize_status "FAILED duplicate_monitor existing_pid=$existing_pid" + fi exit 1 -fi +} -if ! command -v jcmd >/dev/null 2>&1; then - log "Warning: jcmd not found. JFR/NMT snapshots will be skipped." -fi +preflight_existing_monitor() { + local existing_pid="" + existing_pid="$(read_monitor_pid || true)" + if [ -n "$existing_pid" ] && is_monitor_pid_running "$existing_pid"; then + refuse_existing_monitor "$existing_pid" + fi + if [ -n "$existing_pid" ] || [ -d "$MONITOR_LOCK_DIR" ] || [ -f "$ACTIVE_MONITOR_FILE" ]; then + log "Removing stale OSCAR monitor state." + remove_monitor_state + fi +} -LAUNCH_PID="" -PID="" -STOPPING=0 +acquire_monitor_lock() { + local existing_pid="" + mkdir -p "$STATE_DIR" + + if mkdir "$MONITOR_LOCK_DIR" 2>/dev/null; then + echo "$$" > "$MONITOR_PID_FILE" + MONITOR_LOCK_OWNED=1 + return 0 + fi + + sleep 1 + existing_pid="$(read_monitor_pid || true)" + if [ -n "$existing_pid" ] && is_monitor_pid_running "$existing_pid"; then + refuse_existing_monitor "$existing_pid" + fi + + log "Removing stale OSCAR monitor lock state." + remove_monitor_state + if ! mkdir "$MONITOR_LOCK_DIR" 2>/dev/null; then + echo "Error: Could not acquire OSCAR monitor lock at $MONITOR_LOCK_DIR" >&2 + write_error "Could not acquire OSCAR monitor lock at $MONITOR_LOCK_DIR" + finalize_status "FAILED lock_acquire path=$MONITOR_LOCK_DIR" + exit 1 + fi + + echo "$$" > "$MONITOR_PID_FILE" + MONITOR_LOCK_OWNED=1 +} + +find_existing_oscar_pid() { + pgrep -f "$MATCH_EXPR" | head -n 1 || true +} + +find_all_existing_oscar_pids() { + pgrep -f "$MATCH_EXPR" || true +} + +stop_existing_oscar() { + local pids="$1" + if [ -z "$pids" ]; then + return 0 + fi + + log "Stopping existing OSCAR instance(s): $pids" + kill $pids 2>/dev/null || true + + local waited=0 + while [ "$waited" -lt 15 ]; do + sleep 1 + waited=$((waited + 1)) + if [ -z "$(find_all_existing_oscar_pids)" ]; then + return 0 + fi + done + + log "Force killing existing OSCAR instance(s): $pids" + kill -9 $pids 2>/dev/null || true + sleep 1 + + if [ -n "$(find_all_existing_oscar_pids)" ]; then + echo "Error: unable to stop existing OSCAR instance(s)." >&2 + write_error "Unable to stop existing OSCAR instance(s): $pids" + finalize_status "FAILED existing_oscar_stop pids=$pids" + exit 1 + fi +} run_db_query() { local sql="$1" @@ -82,7 +235,7 @@ collect_db_snapshot() { ts="$(date -Is)" failed=0 - if ! command -v docker >/dev/null 2>&1 || ! docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then + if ! docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then echo "Container ${CONTAINER_NAME} not running" > "$d/db-error.txt" echo "$ts,,,,,,,1" >> "$DB_CSV" return 0 @@ -145,11 +298,9 @@ dump_once() { jcmd "$PID" JFR.check > "$d/jfr-check.txt" 2>&1 || true fi - if command -v docker >/dev/null 2>&1; then - docker ps --filter "name=$CONTAINER_NAME" > "$d/docker-ps.txt" 2>&1 || true - docker logs --tail 100 "$CONTAINER_NAME" > "$d/docker-logs-tail.txt" 2>&1 || true - collect_db_snapshot "$d" - fi + docker ps --filter "name=$CONTAINER_NAME" > "$d/docker-ps.txt" 2>&1 || true + docker logs --tail 100 "$CONTAINER_NAME" > "$d/docker-logs-tail.txt" 2>&1 || true + collect_db_snapshot "$d" } final_dump() { @@ -191,53 +342,165 @@ stop_stack() { kill "$LAUNCH_PID" 2>/dev/null || true fi - if command -v docker >/dev/null 2>&1; then - if docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then - log "Stopping container ${CONTAINER_NAME}" - docker stop "$CONTAINER_NAME" > "$OUT_DIR/docker-stop.txt" 2>&1 || true - fi + if docker ps --format '{{.Names}}' | grep -Eq "^${CONTAINER_NAME}$"; then + log "Stopping container ${CONTAINER_NAME}" + docker stop "$CONTAINER_NAME" > "$OUT_DIR/docker-stop.txt" 2>&1 || true fi } on_signal() { log "Received stop signal" + write_status "STOPPING signal_received monitor_pid=$$ output=$OUT_DIR" stop_stack + finalize_status "STOPPED signal monitor_pid=$$ output=$OUT_DIR" exit 0 } on_exit() { + local ec="$?" final_dump - rm -f "$MONITOR_PID_FILE" + if [ "$FINAL_STATUS_WRITTEN" -eq 0 ]; then + if [ "$ec" -eq 0 ]; then + if [ "$STOPPING" -eq 1 ]; then + finalize_status "STOPPED monitor_pid=$$ output=$OUT_DIR" + elif [ -n "${PID:-}" ]; then + finalize_status "EXITED jvm_pid=$PID monitor_pid=$$ output=$OUT_DIR" + else + finalize_status "EXITED monitor_pid=$$ output=$OUT_DIR" + fi + else + finalize_status "FAILED exit_code=$ec monitor_pid=$$ output=$OUT_DIR" + fi + fi + release_monitor_lock } +if [ "${1:-}" = "stop" ]; then + mkdir -p "$STATE_DIR" + monitor_pid="$(read_monitor_pid || true)" + active_dir="" + if [ -f "$ACTIVE_MONITOR_FILE" ]; then + active_dir="$(cat "$ACTIVE_MONITOR_FILE" 2>/dev/null || true)" + fi + + if [ -n "$monitor_pid" ] && is_monitor_pid_running "$monitor_pid"; then + write_status "STOP_REQUESTED monitor_pid=$monitor_pid output=$active_dir" + clear_error + kill "$monitor_pid" 2>/dev/null || true + echo "OSCAR monitor stop requested for PID $monitor_pid." + exit 0 + fi + + remove_monitor_state + write_status "STOP_REQUESTED no_active_monitor" + clear_error + echo "OSCAR monitor is not running." + exit 0 +fi + trap on_signal INT TERM trap on_exit EXIT -log "Starting OSCAR..." -"$LAUNCH_CMD" > "$OUT_DIR/launch.stdout.log" 2> "$OUT_DIR/launch.stderr.log" & -LAUNCH_PID=$! -echo "$LAUNCH_PID" > "$OUT_DIR/launcher-pid.txt" +mkdir -p "$STATE_DIR" +write_status "STARTING monitor_pid=$$ output=$OUT_DIR" +clear_error -log "Waiting for JVM to appear..." -while true; do - PID="$(pgrep -f "$MATCH_EXPR" | head -n 1 || true)" - if [ -n "$PID" ]; then - break - fi - if ! kill -0 "$LAUNCH_PID" 2>/dev/null; then - log "Launch process exited before JVM appeared." - wait "$LAUNCH_PID" || true +if [ -f "$ENV_FILE" ]; then + set -a + . "$ENV_FILE" + set +a + CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" + DB_NAME="${DB_NAME:-gis}" + DB_USER="${DB_USER:-postgres}" + DB_PASSWORD="${DB_PASSWORD:-postgres}" +fi + +DB_CSV="$OUT_DIR/db-connection-trend.csv" + +check_dependencies +preflight_existing_monitor +acquire_monitor_lock +mkdir -p "$OUT_DIR" +echo "$OUT_DIR" > "$ACTIVE_MONITOR_FILE" + +DB_CSV="$OUT_DIR/db-connection-trend.csv" +echo 'timestamp,total_sessions,active,idle,idle_in_transaction,max_connections,superuser_reserved_connections,failed_psql' > "$DB_CSV" + +log "Monitor output: $OUT_DIR" +log "Launch command: $LAUNCH_CMD" +log "JVM match: $MATCH_EXPR" +log "Container name: $CONTAINER_NAME" +log "Database: $DB_NAME user=$DB_USER" +write_status "RUNNING monitor_pid=$$ output=$OUT_DIR" + +if [ ! -x "$LAUNCH_CMD" ] && [ "$ATTACH_TO_EXISTING" != "1" ]; then + echo "Error: launch command is not executable: $LAUNCH_CMD" >&2 + write_error "Launch command is not executable: $LAUNCH_CMD" + finalize_status "FAILED launch_not_executable path=$LAUNCH_CMD" + exit 1 +fi + +existing_pids="$(find_all_existing_oscar_pids)" +if [ -n "$existing_pids" ]; then + if [ "$ATTACH_TO_EXISTING" = "1" ]; then + PID="$(printf '%s\n' "$existing_pids" | head -n 1)" + USE_EXISTING=1 + log "Attaching monitor to existing OSCAR PID $PID" + clear_error + write_status "RUNNING attached monitor_pid=$$ jvm_pid=$PID output=$OUT_DIR" + elif [ "$FORCE_RESTART" = "1" ]; then + log "Existing OSCAR instance found: $existing_pids" + stop_existing_oscar "$existing_pids" + else + echo "OSCAR is already running with PID(s): $existing_pids" >&2 + echo "Set ATTACH_TO_EXISTING=1 to monitor the running instance, or FORCE_RESTART=1 to replace it." >&2 + write_error "OSCAR is already running with PID(s): $existing_pids" + finalize_status "FAILED oscar_already_running pids=$existing_pids" exit 1 fi - sleep 2 -done +fi + +if [ "$USE_EXISTING" = "0" ]; then + log "Starting OSCAR..." + write_status "WAITING_FOR_JVM monitor_pid=$$ output=$OUT_DIR" + "$LAUNCH_CMD" > "$OUT_DIR/launch.stdout.log" 2> "$OUT_DIR/launch.stderr.log" & + LAUNCH_PID=$! + echo "$LAUNCH_PID" > "$OUT_DIR/launcher-pid.txt" + + waited=0 + while true; do + PID="$(find_existing_oscar_pid)" + if [ -n "$PID" ]; then + break + fi + if ! kill -0 "$LAUNCH_PID" 2>/dev/null; then + write_error "Launch process exited before OSCAR JVM appeared. Check $OUT_DIR/launch.stdout.log and $OUT_DIR/launch.stderr.log" + finalize_status "FAILED launch_exited_before_jvm output=$OUT_DIR" + exit 1 + fi + if [ "$waited" -ge "$MAX_WAIT_SECONDS" ]; then + log "Timed out waiting for JVM after ${MAX_WAIT_SECONDS}s" + write_error "Timed out waiting for JVM after ${MAX_WAIT_SECONDS}s. Check $OUT_DIR/launch.stdout.log and $OUT_DIR/launch.stderr.log" + finalize_status "FAILED wait_for_jvm_timeout output=$OUT_DIR" + exit 1 + fi + sleep 2 + waited=$((waited + 2)) + done +else + : > "$OUT_DIR/launch.stdout.log" + : > "$OUT_DIR/launch.stderr.log" +fi log "Found JVM PID: $PID" echo "$PID" > "$OUT_DIR/jvm-pid.txt" +write_status "RUNNING monitor_pid=$$ jvm_pid=$PID output=$OUT_DIR" +clear_error { echo "Timestamp: $(date -Is)" - echo "Launcher PID: $LAUNCH_PID" + echo "Monitor PID: $$" + echo "Launcher PID: ${LAUNCH_PID:-}" echo "JVM PID: $PID" echo echo "Command line:" @@ -265,7 +528,11 @@ dump_once while kill -0 "$PID" 2>/dev/null; do sleep "$INTERVAL" dump_once + write_status "RUNNING monitor_pid=$$ jvm_pid=$PID output=$OUT_DIR" done log "JVM exited." -wait "$LAUNCH_PID" || true +if [ -n "$LAUNCH_PID" ]; then + wait "$LAUNCH_PID" || true +fi +finalize_status "EXITED jvm_pid=$PID monitor_pid=$$ output=$OUT_DIR" diff --git a/dist/release/stop-all.bat b/dist/release/stop-all.bat old mode 100755 new mode 100644 index cd4503c..3876f16 --- a/dist/release/stop-all.bat +++ b/dist/release/stop-all.bat @@ -3,45 +3,43 @@ setlocal EnableExtensions set "SCRIPT_DIR=%~dp0" if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" +set "CONTAINER_NAME=oscar-postgis-container" +set "SENSORHUB_NAME=com.botts.impl.security.SensorHubWrapper" + +if exist "%SCRIPT_DIR%\.env" ( + for /f "usebackq tokens=* delims=" %%L in ("%SCRIPT_DIR%\.env") do ( + set "LINE=%%L" + call :parse_env_line + ) +) -set "ENV_FILE=%SCRIPT_DIR%\.env" -if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" - -if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" - -echo Requesting monitor shutdown... -if exist "%SCRIPT_DIR%\monitor-oscar.bat" call "%SCRIPT_DIR%\monitor-oscar.bat" stop >nul 2>nul - -echo Stopping SensorHubWrapper Java Process... -for /f "usebackq delims=" %%P in (` - powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { $proc.ProcessId } }" 2^>nul -`) do ( - powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %%P -Force -ErrorAction Stop } catch {}" >nul 2>nul +echo Requesting monitor stop if active... +if exist "%SCRIPT_DIR%\monitor-oscar.bat" ( + call "%SCRIPT_DIR%\monitor-oscar.bat" stop >nul 2>nul + timeout /t 5 /nobreak >nul ) -docker ps -a --format "{{.Names}}" | findstr /I /X "%CONTAINER_NAME%" >nul -if not errorlevel 1 ( - echo Stopping container: %CONTAINER_NAME%... - docker rm -f "%CONTAINER_NAME%" >nul 2>nul +echo. +echo Stopping container: %CONTAINER_NAME%... +docker stop %CONTAINER_NAME% >nul 2>nul +if errorlevel 1 ( + echo Container not found or already stopped. ) else ( - echo Container not found: %CONTAINER_NAME% + echo Container stop requested. ) +echo. +echo Stopping SensorHubWrapper Java process... +powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-CimInstance Win32_Process | Where-Object { $_.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $_.CommandLine -and $_.CommandLine -like '*%SENSORHUB_NAME%*' } | ForEach-Object { try { Stop-Process -Id $_.ProcessId -Force -ErrorAction Stop; Write-Output ('Stopped PID ' + $_.ProcessId) } catch {} }" + echo. echo Done. exit /b 0 -:load_env -for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( - set "ENV_NAME=%%A" - set "ENV_VALUE=%%B" - call :set_env_var +:parse_env_line +if not defined LINE exit /b 0 +if "%LINE:~0,1%"=="#" exit /b 0 +for /f "tokens=1,* delims==" %%A in ("%LINE%") do ( + if /I "%%A"=="CONTAINER_NAME" set "CONTAINER_NAME=%%B" ) exit /b 0 - -:set_env_var -if not defined ENV_NAME exit /b 0 -if "%ENV_NAME:~0,1%"=="#" exit /b 0 -if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" -set "%ENV_NAME%=%ENV_VALUE%" -exit /b 0 \ No newline at end of file diff --git a/dist/release/stop-all.sh b/dist/release/stop-all.sh index e1e552d..f80be77 100755 --- a/dist/release/stop-all.sh +++ b/dist/release/stop-all.sh @@ -1,64 +1,56 @@ #!/bin/bash -set -u +set -euo pipefail SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" -CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" -SENSORHUB_NAME='com.botts.impl.security.SensorHubWrapper' +ENV_FILE="${ENV_FILE:-$SCRIPT_DIR/.env}" MONITOR_SCRIPT="$SCRIPT_DIR/monitor-oscar.sh" -MONITOR_PID_FILE="$SCRIPT_DIR/monitor.pid" - -request_monitor_stop() { - echo "Requesting monitor shutdown..." - if [ -f "$MONITOR_SCRIPT" ]; then - (bash "$MONITOR_SCRIPT" stop >/dev/null 2>&1 || true) & - elif [ -f "$MONITOR_PID_FILE" ]; then - monitor_pid="$(tr -d '[:space:]' < "$MONITOR_PID_FILE")" - if [ -n "$monitor_pid" ] && kill -0 "$monitor_pid" 2>/dev/null; then - kill "$monitor_pid" 2>/dev/null || true +STATE_DIR="$SCRIPT_DIR/.monitor-state" +LOCK_DIR="$STATE_DIR/lock" +CONTAINER_NAME="oscar-postgis-container" +SENSORHUB_NAME="com.botts.impl.security.SensorHubWrapper" + +if [ -f "$ENV_FILE" ]; then + set -a + . "$ENV_FILE" + set +a + CONTAINER_NAME="${CONTAINER_NAME:-oscar-postgis-container}" +fi + +echo "Requesting monitor stop if active..." +if [ -x "$MONITOR_SCRIPT" ]; then + "$MONITOR_SCRIPT" stop || true + for _ in 1 2 3 4 5 6 7 8 9 10; do + if [ ! -d "$LOCK_DIR" ]; then + break fi - fi -} - -stop_sensorhub() { - local pids="" - - if command -v jps >/dev/null 2>&1; then - pids="$(jps -l | awk -v name="$SENSORHUB_NAME" '$2==name {print $1}')" - fi - - if [ -z "$pids" ] && command -v pgrep >/dev/null 2>&1; then - pids="$(pgrep -f "$SENSORHUB_NAME" || true)" - fi + sleep 1 + done +fi - if [ -n "$pids" ]; then - echo "Stopping SensorHubWrapper with PID(s): $pids" - kill $pids 2>/dev/null || true - sleep 2 - if command -v pgrep >/dev/null 2>&1 && pgrep -f "$SENSORHUB_NAME" >/dev/null 2>&1; then - echo "Force stopping SensorHubWrapper..." - pkill -9 -f "$SENSORHUB_NAME" 2>/dev/null || true - fi - echo "SensorHubWrapper stopped." - else - echo "SensorHubWrapper process not found." - fi -} +echo +printf 'Stopping container: %s...\n' "$CONTAINER_NAME" +if docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then + docker stop "$CONTAINER_NAME" >/dev/null 2>&1 || true + echo "Container stop requested." +else + echo "Container not found." +fi -stop_container() { - echo "Stopping container: $CONTAINER_NAME..." - if command -v docker >/dev/null 2>&1 && docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then - docker stop "$CONTAINER_NAME" >/dev/null 2>&1 || true - echo "Container stop requested." - else - echo "Container not found. Nothing to stop." +echo +echo "Stopping SensorHubWrapper Java process..." +PIDS="$(pgrep -f "$SENSORHUB_NAME" || true)" +if [ -n "$PIDS" ]; then + echo "Stopping SensorHubWrapper with PID(s): $PIDS" + kill $PIDS 2>/dev/null || true + sleep 3 + REMAINING="$(pgrep -f "$SENSORHUB_NAME" || true)" + if [ -n "$REMAINING" ]; then + echo "Force killing remaining PID(s): $REMAINING" + kill -9 $REMAINING 2>/dev/null || true fi -} - -request_monitor_stop -sleep 2 -stop_sensorhub -stop_container -rm -f "$MONITOR_PID_FILE" +else + echo "SensorHubWrapper process not found." +fi echo echo "Done." From f73255c7b2c19d270e2124409a263f7e64e2126e Mon Sep 17 00:00:00 2001 From: tyronechrisharris <142608718+tyronechrisharris@users.noreply.github.com> Date: Fri, 8 May 2026 00:19:22 -0400 Subject: [PATCH 6/9] Refactor batch scripts for improved readability and maintainability - Updated `reset-all.bat` to enhance structure and clarity, ensuring consistent formatting and better handling of environment variables. - Modified `stop-all.bat` to streamline container stopping logic and improve error handling. - Enhanced `launch.bat` with clearer error messages and improved environment variable loading. - Refined `load_trusted_certs.bat` for better error handling and clarity in locating Java cacerts. --- dist/release/check-oscar-status.ps1 | 1352 +++++++++--------- dist/release/launch-all.bat | 450 +++--- dist/release/monitor-oscar.bat | 702 ++++----- dist/release/reset-all.bat | 190 +-- dist/release/stop-all.bat | 90 +- dist/scripts/standard/launch.bat | 388 ++--- dist/scripts/standard/load_trusted_certs.bat | 196 +-- 7 files changed, 1684 insertions(+), 1684 deletions(-) diff --git a/dist/release/check-oscar-status.ps1 b/dist/release/check-oscar-status.ps1 index 22dcf66..6bc7781 100644 --- a/dist/release/check-oscar-status.ps1 +++ b/dist/release/check-oscar-status.ps1 @@ -1,677 +1,677 @@ -param( - [string]$BaseDirectory = $PSScriptRoot, - [string]$MonitorDirectory -) - -$ErrorActionPreference = "SilentlyContinue" - -function Add-Line { - param( - [System.Collections.Generic.List[string]]$Lines, - [string]$Text = "" - ) - $Lines.Add($Text) | Out-Null -} - -function Add-Block { - param( - [System.Collections.Generic.List[string]]$Lines, - [string]$Text - ) - - if ([string]::IsNullOrWhiteSpace($Text)) { - $Lines.Add("") | Out-Null - return - } - - $normalized = $Text -replace "`r`n", "`n" - foreach ($line in ($normalized -split "`n")) { - $Lines.Add($line) | Out-Null - } -} - -function Load-DotEnv { - param([string]$Path) - - $map = @{} - if (-not (Test-Path -LiteralPath $Path)) { - return $map - } - - foreach ($rawLine in Get-Content -LiteralPath $Path) { - $line = $rawLine.Trim() - if ([string]::IsNullOrWhiteSpace($line)) { continue } - if ($line.StartsWith("#")) { continue } - - if ($line.StartsWith("export ")) { - $line = $line.Substring(7).Trim() - } - - $idx = $line.IndexOf("=") - if ($idx -lt 1) { continue } - - $key = $line.Substring(0, $idx).Trim() - $value = $line.Substring($idx + 1) - - if (($value.StartsWith('"') -and $value.EndsWith('"')) -or ($value.StartsWith("'") -and $value.EndsWith("'"))) { - if ($value.Length -ge 2) { - $value = $value.Substring(1, $value.Length - 2) - } - } - - $map[$key] = $value - } - - return $map -} - -function Get-ActiveMonitorDirectory { - param([string]$BaseDir) - - $activePath = Join-Path $BaseDir ".monitor-active-dir" - if (-not (Test-Path -LiteralPath $activePath)) { - return $null - } - - $candidate = (Get-Content -LiteralPath $activePath -TotalCount 1 | Out-String).Trim() - if ([string]::IsNullOrWhiteSpace($candidate)) { - return $null - } - - if (Test-Path -LiteralPath $candidate) { - return Get-Item -LiteralPath $candidate - } - - return $null -} - -function Get-LatestMonitorDirectory { - param([string]$BaseDir) - - $dirs = Get-ChildItem -LiteralPath $BaseDir -Directory | - Where-Object { $_.Name -like "oscar-monitor-*" } | - Sort-Object Name -Descending - - return ($dirs | Select-Object -First 1) -} - -function Get-OscarJavaProcesses { - $procs = Get-CimInstance Win32_Process | - Where-Object { - $_.Name -match '^(java|javaw)(\.exe)?$' -and - $null -ne $_.CommandLine -and - $_.CommandLine -match 'com\.botts\.impl\.security\.SensorHubWrapper' - } | - Sort-Object ProcessId - - return @($procs) -} - -function Resolve-ToolPath { - param([string]$Name) - - $cmd = Get-Command $Name -ErrorAction SilentlyContinue - if ($cmd -and $cmd.Source) { - return $cmd.Source - } - - $whereExe = Get-Command where.exe -ErrorAction SilentlyContinue - if ($whereExe) { - try { - $resolved = & $whereExe.Source $Name 2>$null | Select-Object -First 1 - if ($resolved -and (Test-Path -LiteralPath $resolved)) { - return $resolved - } - } - catch { - } - } - - return $null -} - -function Resolve-JcmdPath { - $jcmd = Resolve-ToolPath -Name "jcmd.exe" - if ($jcmd) { - return $jcmd - } - - $jcmd = Resolve-ToolPath -Name "jcmd" - if ($jcmd) { - return $jcmd - } - - if ($env:JAVA_HOME) { - $candidate = Join-Path $env:JAVA_HOME "bin\jcmd.exe" - if (Test-Path -LiteralPath $candidate) { - return $candidate - } - } - - $javaCmd = Resolve-ToolPath -Name "java.exe" - if (-not $javaCmd) { - $javaCmd = Resolve-ToolPath -Name "java" - } - - if ($javaCmd) { - $javaDir = Split-Path -Parent $javaCmd - $candidate = Join-Path $javaDir "jcmd.exe" - if (Test-Path -LiteralPath $candidate) { - return $candidate - } - } - - return $null -} - -function Invoke-ExternalCapture { - param( - [string]$FilePath, - [string[]]$Arguments - ) - - $script:LastExternalExitCode = 0 - - if ([string]::IsNullOrWhiteSpace($FilePath)) { - $script:LastExternalExitCode = 1 - return "Tool path is empty." - } - - if (-not (Test-Path -LiteralPath $FilePath)) { - $script:LastExternalExitCode = 1 - return "Tool not found: $FilePath" - } - - try { - $result = & $FilePath @Arguments 2>&1 | Out-String -Width 4096 - $exitCode = $LASTEXITCODE - if ($null -eq $exitCode) { $exitCode = 0 } - $script:LastExternalExitCode = $exitCode - - $trimmed = $result.TrimEnd() - if ([string]::IsNullOrWhiteSpace($trimmed) -and $exitCode -ne 0) { - return "Command failed with exit code $exitCode and returned no output." - } - - return $trimmed - } - catch { - $script:LastExternalExitCode = 1 - return ($_ | Out-String).TrimEnd() - } -} - -function Get-DockerContainerRecord { - param( - [string]$DockerExe, - [string]$ContainerName - ) - - if (-not $DockerExe) { - return $null - } - - $raw = Invoke-ExternalCapture -FilePath $DockerExe -Arguments @( - "ps", "-a", - "--format", "{{.ID}}|{{.Image}}|{{.Status}}|{{.Names}}|{{.Ports}}|{{.Command}}" - ) - - if ($script:LastExternalExitCode -ne 0) { - return @{ - Error = $raw - } - } - - $lines = @($raw -split "`r?`n" | Where-Object { $_.Trim().Length -gt 0 }) - foreach ($line in $lines) { - $parts = $line.Split("|") - if ($parts.Count -ge 6 -and $parts[3] -eq $ContainerName) { - return @{ - Id = $parts[0] - Image = $parts[1] - Status = $parts[2] - Name = $parts[3] - Ports = $parts[4] - Command = $parts[5] - } - } - } - - return $null -} - -function Get-DockerTableText { - param($ContainerRecord) - - if ($null -eq $ContainerRecord) { - return "Container not found." - } - - if ($ContainerRecord.ContainsKey("Error")) { - return $ContainerRecord.Error - } - - return @" -CONTAINER ID IMAGE STATUS PORTS NAMES -$($ContainerRecord.Id) $($ContainerRecord.Image) $($ContainerRecord.Status) $($ContainerRecord.Ports) $($ContainerRecord.Name) -"@.TrimEnd() -} - -function Invoke-PsqlInContainer { - param( - [string]$DockerExe, - [string]$ContainerName, - [string]$DbUser, - [string]$DbName, - [string]$DbPassword, - [string]$Sql - ) - - return Invoke-ExternalCapture -FilePath $DockerExe -Arguments @( - "exec", - "-e", "PGPASSWORD=$DbPassword", - $ContainerName, - "psql", - "-U", $DbUser, - "-d", $DbName, - "-At", - "-c", $Sql - ) -} - -function Get-LaunchTail { - param( - [string]$MonitorDir, - [string]$FileName, - [int]$Tail = 50 - ) - - if ([string]::IsNullOrWhiteSpace($MonitorDir)) { return "" } - - $path = Join-Path $MonitorDir $FileName - if (-not (Test-Path -LiteralPath $path)) { - return "" - } - - return (Get-Content -LiteralPath $path -Tail $Tail | Out-String -Width 4096).TrimEnd() -} - -function Run-JcmdSection { - param( - [string]$JcmdExe, - [string]$Pid, - [string[]]$Args - ) - - if (-not $Pid -or $Pid -notmatch '^\d+$') { - return "No live OSCAR JVM found." - } - - if (-not $JcmdExe) { - return "jcmd.exe not found." - } - - if (-not (Test-Path -LiteralPath $JcmdExe)) { - return "jcmd.exe path does not exist: $JcmdExe" - } - - $stdoutFile = [System.IO.Path]::GetTempFileName() - $stderrFile = [System.IO.Path]::GetTempFileName() - - try { - $proc = Start-Process ` - -FilePath $JcmdExe ` - -ArgumentList (@($Pid) + $Args) ` - -NoNewWindow ` - -Wait ` - -PassThru ` - -RedirectStandardOutput $stdoutFile ` - -RedirectStandardError $stderrFile - - $stdout = "" - $stderr = "" - - if (Test-Path -LiteralPath $stdoutFile) { - $stdout = Get-Content -LiteralPath $stdoutFile -Raw - } - - if (Test-Path -LiteralPath $stderrFile) { - $stderr = Get-Content -LiteralPath $stderrFile -Raw - } - - $output = ($stdout + $stderr).TrimEnd() - $exitCode = $proc.ExitCode - - if ($exitCode -ne 0) { - if ([string]::IsNullOrWhiteSpace($output)) { - return "jcmd failed with exit code $exitCode using: $JcmdExe $Pid $($Args -join ' ')" - } - return $output - } - - if ([string]::IsNullOrWhiteSpace($output)) { - return "jcmd returned no output using: $JcmdExe $Pid $($Args -join ' ')" - } - - return $output - } - catch { - return ($_ | Out-String).TrimEnd() - } - finally { - Remove-Item -LiteralPath $stdoutFile -Force -ErrorAction SilentlyContinue - Remove-Item -LiteralPath $stderrFile -Force -ErrorAction SilentlyContinue - } -} - -$envMap = Load-DotEnv -Path (Join-Path $BaseDirectory ".env") - -$containerName = if ($envMap.ContainsKey("CONTAINER_NAME")) { $envMap["CONTAINER_NAME"] } else { "oscar-postgis-container" } -$dbUser = if ($envMap.ContainsKey("DB_USER")) { $envMap["DB_USER"] } else { "postgres" } -$dbName = if ($envMap.ContainsKey("DB_NAME")) { $envMap["DB_NAME"] } else { "gis" } -$dbPassword = if ($envMap.ContainsKey("DB_PASSWORD")) { $envMap["DB_PASSWORD"] } else { "postgres" } - -$monitorDirItem = $null -if (-not [string]::IsNullOrWhiteSpace($MonitorDirectory)) { - if (Test-Path -LiteralPath $MonitorDirectory) { - $monitorDirItem = Get-Item -LiteralPath $MonitorDirectory - } -} -else { - $monitorDirItem = Get-ActiveMonitorDirectory -BaseDir $BaseDirectory - if ($null -eq $monitorDirItem) { - $monitorDirItem = Get-LatestMonitorDirectory -BaseDir $BaseDirectory - } -} - -$monitorDir = if ($null -ne $monitorDirItem) { $monitorDirItem.FullName } else { "" } - -$timestamp = Get-Date -Format "yyyyMMdd-HHmmss" -$outputFile = Join-Path $BaseDirectory "oscar-status-$timestamp.txt" - -$pidFromMonitor = "" -if ($monitorDir) { - $pidPath = Join-Path $monitorDir "jvm-pid.txt" - if (Test-Path -LiteralPath $pidPath) { - $pidFromMonitor = (Get-Content -LiteralPath $pidPath -TotalCount 1 | Out-String).Trim() - } -} - -$oscarJava = Get-OscarJavaProcesses -$liveProc = $null - -if ($pidFromMonitor -match '^\d+$') { - $liveProc = $oscarJava | Where-Object { $_.ProcessId -eq [int]$pidFromMonitor } | Select-Object -First 1 -} -if ($null -eq $liveProc) { - $liveProc = $oscarJava | Select-Object -First 1 -} - -$livePid = if ($null -ne $liveProc) { [string]$liveProc.ProcessId } else { "" } - -$dockerExe = Resolve-ToolPath -Name "docker" -$jcmdExe = Resolve-JcmdPath - -$dockerContainer = Get-DockerContainerRecord -DockerExe $dockerExe -ContainerName $containerName -$dockerTableText = Get-DockerTableText -ContainerRecord $dockerContainer - -$containerRunning = $false -if ($dockerContainer -and -not $dockerContainer.ContainsKey("Error")) { - if ($dockerContainer.Status -like "Up*") { - $containerRunning = $true - } -} - -$osInfo = Get-CimInstance Win32_OperatingSystem | - Select-Object TotalVisibleMemorySize, FreePhysicalMemory, TotalVirtualMemorySize, FreeVirtualMemory -$osInfoText = ($osInfo | Format-List | Out-String -Width 4096).TrimEnd() - -$counterText = "" -try { - $counters = Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\% Usage' - $counterText = ($counters | Out-String -Width 4096).TrimEnd() -} -catch { - $counterText = "Could not read performance counters." -} - -$liveJvmText = "" -if ($livePid -match '^\d+$') { - $liveJvmText = (Get-Process -Id ([int]$livePid) | - Select-Object Id, ProcessName, Threads, VirtualMemorySize64, WorkingSet64, PrivateMemorySize64, CPU, StartTime | - Format-List | Out-String -Width 4096).TrimEnd() -} -else { - $liveJvmText = "No live OSCAR JVM found." -} - -$jfrText = "" -$heapText = "" -$nmtText = "" - -if ($livePid -match '^\d+$' -and $jcmdExe -and (Test-Path -LiteralPath $jcmdExe)) { - try { - $jfrText = (& $jcmdExe $livePid JFR.check 2>&1 | Out-String -Width 4096).TrimEnd() - if ([string]::IsNullOrWhiteSpace($jfrText)) { - $jfrText = "jcmd returned no output for JFR.check" - } - } - catch { - $jfrText = ($_ | Out-String).TrimEnd() - } - - try { - $heapText = (& $jcmdExe $livePid GC.heap_info 2>&1 | Out-String -Width 4096).TrimEnd() - if ([string]::IsNullOrWhiteSpace($heapText)) { - $heapText = "jcmd returned no output for GC.heap_info" - } - } - catch { - $heapText = ($_ | Out-String).TrimEnd() - } - - try { - $nmtText = (& $jcmdExe $livePid VM.native_memory summary 2>&1 | Out-String -Width 4096).TrimEnd() - if ([string]::IsNullOrWhiteSpace($nmtText)) { - $nmtText = "jcmd returned no output for VM.native_memory summary" - } - } - catch { - $nmtText = ($_ | Out-String).TrimEnd() - } -} -else { - $jfrText = "jcmd.exe not found or no live OSCAR JVM found." - $heapText = $jfrText - $nmtText = $jfrText -} - -$dbMetaText = "" -$dbByStateText = "" -$dbByAppText = "" -$dbErrorText = "" - -$maxConnections = "" -$superuserReservedConnections = "" -$usableClientSlots = "" -$totalSessions = "" -$activeSessions = "" -$idleSessions = "" -$idleInTransaction = "" - -if (-not $dockerExe) { - $dbErrorText = "docker.exe not found in PATH." -} -elseif ($null -eq $dockerContainer) { - $dbErrorText = "Container '$containerName' not found." -} -elseif ($dockerContainer.ContainsKey("Error")) { - $dbErrorText = $dockerContainer.Error -} -elseif (-not $containerRunning) { - $dbErrorText = "Container '$containerName' is present but not running. Status: $($dockerContainer.Status)" -} -else { - $dbMetaSql = "select current_setting('max_connections'), current_setting('superuser_reserved_connections'), (current_setting('max_connections')::int - current_setting('superuser_reserved_connections')::int), count(*), count(*) filter (where state = 'active'), count(*) filter (where state = 'idle'), count(*) filter (where state = 'idle in transaction') from pg_stat_activity;" - $dbByStateSql = "select coalesce(state,'') || '|' || count(*)::text from pg_stat_activity group by state order by 1;" - $dbByAppSql = "select coalesce(application_name,'') || '|' || coalesce(usename,'') || '|' || coalesce(client_addr::text,'') || '|' || coalesce(state,'') || '|' || count(*)::text from pg_stat_activity group by application_name, usename, client_addr, state order by application_name, usename, client_addr, state;" - - $dbMetaText = Invoke-PsqlInContainer -DockerExe $dockerExe -ContainerName $containerName -DbUser $dbUser -DbName $dbName -DbPassword $dbPassword -Sql $dbMetaSql - $dbByStateText = Invoke-PsqlInContainer -DockerExe $dockerExe -ContainerName $containerName -DbUser $dbUser -DbName $dbName -DbPassword $dbPassword -Sql $dbByStateSql - $dbByAppText = Invoke-PsqlInContainer -DockerExe $dockerExe -ContainerName $containerName -DbUser $dbUser -DbName $dbName -DbPassword $dbPassword -Sql $dbByAppSql - - $metaLine = ($dbMetaText -split "`r?`n" | Where-Object { $_.Trim().Length -gt 0 } | Select-Object -First 1) - if ($metaLine -and $metaLine.Contains("|")) { - $parts = $metaLine.Split("|") - if ($parts.Count -ge 7) { - $maxConnections = $parts[0] - $superuserReservedConnections = $parts[1] - $usableClientSlots = $parts[2] - $totalSessions = $parts[3] - $activeSessions = $parts[4] - $idleSessions = $parts[5] - $idleInTransaction = $parts[6] - } - } - else { - if ([string]::IsNullOrWhiteSpace($dbMetaText)) { - $dbErrorText = "psql returned no DB metadata output." - } - else { - $dbErrorText = $dbMetaText - } - } -} - -$snapshotDirs = @() -if ($monitorDir -and (Test-Path -LiteralPath $monitorDir)) { - $snapshotDirs = Get-ChildItem -LiteralPath $monitorDir -Directory | - Where-Object { $_.Name -match '^\d{8}-\d{6}$' } | - Sort-Object Name -} - -$firstSnapshot = if ($snapshotDirs.Count -gt 0) { $snapshotDirs[0].FullName } else { "" } -$latestSnapshot = if ($snapshotDirs.Count -gt 0) { $snapshotDirs[-1].FullName } else { "" } - -$recentSnapshotLines = @() -if ($snapshotDirs.Count -gt 0) { - $recentSnapshotLines = $snapshotDirs | - Select-Object -Last ([Math]::Min(20, $snapshotDirs.Count)) | - ForEach-Object { $_.Name } -} - -$launchStdoutTail = Get-LaunchTail -MonitorDir $monitorDir -FileName "launch.stdout.log" -Tail 50 -$launchStderrTail = Get-LaunchTail -MonitorDir $monitorDir -FileName "launch.stderr.log" -Tail 50 - -$dockerLogsTail = "" -if ($dockerExe -and $containerRunning) { - $dockerLogsTail = Invoke-ExternalCapture -FilePath $dockerExe -Arguments @("logs", "--tail", "100", $containerName) -} - -$procTableText = "" -if ($liveProc) { - $procTableText = ($liveProc | - Select-Object ProcessId, Name, CommandLine | - Format-Table -AutoSize | Out-String -Width 4096).TrimEnd() -} -else { - $procTableText = "No live OSCAR Java process found." -} - -$lines = New-Object 'System.Collections.Generic.List[string]' - -Add-Line $lines "OSCAR STATUS REPORT" -Add-Line $lines ("Generated: " + (Get-Date).ToString("o")) -Add-Line $lines ("Base directory: " + $BaseDirectory) -Add-Line $lines ("Monitor directory: " + $(if ($monitorDir) { $monitorDir } else { "" })) -Add-Line $lines ("Output file: " + $outputFile) -Add-Line $lines "" - -Add-Line $lines "=== PROCESS STATUS ===" -Add-Line $lines ("PID from monitor: " + $pidFromMonitor) -Add-Line $lines ("Live OSCAR PID: " + $livePid) -Add-Line $lines "" -Add-Block $lines $procTableText -Add-Line $lines "" -Add-Block $lines $dockerTableText -Add-Line $lines "" - -Add-Line $lines "=== SYSTEM MEMORY AND PAGEFILE ===" -Add-Line $lines "" -Add-Block $lines $osInfoText -Add-Line $lines "" -Add-Block $lines $counterText -Add-Line $lines "" - -Add-Line $lines "=== LIVE JVM PROCESS ===" -Add-Line $lines "" -Add-Block $lines $liveJvmText -Add-Line $lines "" - -Add-Line $lines "=== LIVE JVM JFR STATUS ===" -Add-Block $lines $jfrText -Add-Line $lines "" - -Add-Line $lines "=== LIVE JVM GC HEAP INFO ===" -Add-Block $lines $heapText -Add-Line $lines "" - -Add-Line $lines "=== LIVE JVM NATIVE MEMORY SUMMARY ===" -Add-Block $lines $nmtText -Add-Line $lines "" - -Add-Line $lines "=== LIVE POSTGRES STATUS ===" -Add-Line $lines ("max_connections: " + $maxConnections) -Add-Line $lines ("superuser_reserved_connections: " + $superuserReservedConnections) -Add-Line $lines ("usable_client_slots: " + $usableClientSlots) -Add-Line $lines ("total_sessions: " + $totalSessions) -Add-Line $lines ("active: " + $activeSessions) -Add-Line $lines ("idle: " + $idleSessions) -Add-Line $lines ("idle in transaction: " + $idleInTransaction) -Add-Line $lines "" -Add-Line $lines "--- db-by-state ---" -Add-Block $lines $dbByStateText -Add-Line $lines "" -Add-Line $lines "--- db-by-app ---" -Add-Block $lines $dbByAppText -Add-Line $lines "" -Add-Line $lines "--- db-error ---" -Add-Block $lines $dbErrorText -Add-Line $lines "" - -Add-Line $lines "=== SNAPSHOT STATUS ===" -Add-Line $lines ("Snapshot count: " + $snapshotDirs.Count) -Add-Line $lines ("First snapshot: " + $firstSnapshot) -Add-Line $lines ("Latest snapshot: " + $latestSnapshot) -Add-Line $lines "" - -Add-Line $lines "=== RECENT SNAPSHOTS (LAST 20) ===" -foreach ($snap in $recentSnapshotLines) { - Add-Line $lines $snap -} -Add-Line $lines "" - -Add-Line $lines "=== LOG TAILS ===" -Add-Line $lines "--- launch.stdout.log (last 50 lines) ---" -Add-Block $lines $launchStdoutTail -Add-Line $lines "" -Add-Line $lines "--- launch.stderr.log (last 50 lines) ---" -Add-Block $lines $launchStderrTail -Add-Line $lines "" -Add-Line $lines "--- postgres docker logs (last captured 100 lines) ---" -Add-Block $lines $dockerLogsTail -Add-Line $lines "" - -Add-Line $lines "=== QUICK READ ===" -Add-Line $lines ("Live JVM PID: " + $livePid) -Add-Line $lines ("Snapshots captured: " + $snapshotDirs.Count) -if ($totalSessions) { Add-Line $lines ("DB total sessions: " + $totalSessions) } -if ($usableClientSlots) { Add-Line $lines ("DB usable client slots: " + $usableClientSlots) } -Add-Line $lines "Interpretation guide:" -Add-Line $lines "- Healthy memory: process memory and JVM native memory plateau." -Add-Line $lines "- Healthy DB: total sessions rise at startup and then plateau well below usable client slots." -Add-Line $lines "- Suspicious DB: total sessions keep climbing, idle sessions pile up, or db-error shows query failures." - -$reportText = ($lines -join "`r`n") -Set-Content -LiteralPath $outputFile -Value $reportText -Encoding UTF8 +param( + [string]$BaseDirectory = $PSScriptRoot, + [string]$MonitorDirectory +) + +$ErrorActionPreference = "SilentlyContinue" + +function Add-Line { + param( + [System.Collections.Generic.List[string]]$Lines, + [string]$Text = "" + ) + $Lines.Add($Text) | Out-Null +} + +function Add-Block { + param( + [System.Collections.Generic.List[string]]$Lines, + [string]$Text + ) + + if ([string]::IsNullOrWhiteSpace($Text)) { + $Lines.Add("") | Out-Null + return + } + + $normalized = $Text -replace "`r`n", "`n" + foreach ($line in ($normalized -split "`n")) { + $Lines.Add($line) | Out-Null + } +} + +function Load-DotEnv { + param([string]$Path) + + $map = @{} + if (-not (Test-Path -LiteralPath $Path)) { + return $map + } + + foreach ($rawLine in Get-Content -LiteralPath $Path) { + $line = $rawLine.Trim() + if ([string]::IsNullOrWhiteSpace($line)) { continue } + if ($line.StartsWith("#")) { continue } + + if ($line.StartsWith("export ")) { + $line = $line.Substring(7).Trim() + } + + $idx = $line.IndexOf("=") + if ($idx -lt 1) { continue } + + $key = $line.Substring(0, $idx).Trim() + $value = $line.Substring($idx + 1) + + if (($value.StartsWith('"') -and $value.EndsWith('"')) -or ($value.StartsWith("'") -and $value.EndsWith("'"))) { + if ($value.Length -ge 2) { + $value = $value.Substring(1, $value.Length - 2) + } + } + + $map[$key] = $value + } + + return $map +} + +function Get-ActiveMonitorDirectory { + param([string]$BaseDir) + + $activePath = Join-Path $BaseDir ".monitor-active-dir" + if (-not (Test-Path -LiteralPath $activePath)) { + return $null + } + + $candidate = (Get-Content -LiteralPath $activePath -TotalCount 1 | Out-String).Trim() + if ([string]::IsNullOrWhiteSpace($candidate)) { + return $null + } + + if (Test-Path -LiteralPath $candidate) { + return Get-Item -LiteralPath $candidate + } + + return $null +} + +function Get-LatestMonitorDirectory { + param([string]$BaseDir) + + $dirs = Get-ChildItem -LiteralPath $BaseDir -Directory | + Where-Object { $_.Name -like "oscar-monitor-*" } | + Sort-Object Name -Descending + + return ($dirs | Select-Object -First 1) +} + +function Get-OscarJavaProcesses { + $procs = Get-CimInstance Win32_Process | + Where-Object { + $_.Name -match '^(java|javaw)(\.exe)?$' -and + $null -ne $_.CommandLine -and + $_.CommandLine -match 'com\.botts\.impl\.security\.SensorHubWrapper' + } | + Sort-Object ProcessId + + return @($procs) +} + +function Resolve-ToolPath { + param([string]$Name) + + $cmd = Get-Command $Name -ErrorAction SilentlyContinue + if ($cmd -and $cmd.Source) { + return $cmd.Source + } + + $whereExe = Get-Command where.exe -ErrorAction SilentlyContinue + if ($whereExe) { + try { + $resolved = & $whereExe.Source $Name 2>$null | Select-Object -First 1 + if ($resolved -and (Test-Path -LiteralPath $resolved)) { + return $resolved + } + } + catch { + } + } + + return $null +} + +function Resolve-JcmdPath { + $jcmd = Resolve-ToolPath -Name "jcmd.exe" + if ($jcmd) { + return $jcmd + } + + $jcmd = Resolve-ToolPath -Name "jcmd" + if ($jcmd) { + return $jcmd + } + + if ($env:JAVA_HOME) { + $candidate = Join-Path $env:JAVA_HOME "bin\jcmd.exe" + if (Test-Path -LiteralPath $candidate) { + return $candidate + } + } + + $javaCmd = Resolve-ToolPath -Name "java.exe" + if (-not $javaCmd) { + $javaCmd = Resolve-ToolPath -Name "java" + } + + if ($javaCmd) { + $javaDir = Split-Path -Parent $javaCmd + $candidate = Join-Path $javaDir "jcmd.exe" + if (Test-Path -LiteralPath $candidate) { + return $candidate + } + } + + return $null +} + +function Invoke-ExternalCapture { + param( + [string]$FilePath, + [string[]]$Arguments + ) + + $script:LastExternalExitCode = 0 + + if ([string]::IsNullOrWhiteSpace($FilePath)) { + $script:LastExternalExitCode = 1 + return "Tool path is empty." + } + + if (-not (Test-Path -LiteralPath $FilePath)) { + $script:LastExternalExitCode = 1 + return "Tool not found: $FilePath" + } + + try { + $result = & $FilePath @Arguments 2>&1 | Out-String -Width 4096 + $exitCode = $LASTEXITCODE + if ($null -eq $exitCode) { $exitCode = 0 } + $script:LastExternalExitCode = $exitCode + + $trimmed = $result.TrimEnd() + if ([string]::IsNullOrWhiteSpace($trimmed) -and $exitCode -ne 0) { + return "Command failed with exit code $exitCode and returned no output." + } + + return $trimmed + } + catch { + $script:LastExternalExitCode = 1 + return ($_ | Out-String).TrimEnd() + } +} + +function Get-DockerContainerRecord { + param( + [string]$DockerExe, + [string]$ContainerName + ) + + if (-not $DockerExe) { + return $null + } + + $raw = Invoke-ExternalCapture -FilePath $DockerExe -Arguments @( + "ps", "-a", + "--format", "{{.ID}}|{{.Image}}|{{.Status}}|{{.Names}}|{{.Ports}}|{{.Command}}" + ) + + if ($script:LastExternalExitCode -ne 0) { + return @{ + Error = $raw + } + } + + $lines = @($raw -split "`r?`n" | Where-Object { $_.Trim().Length -gt 0 }) + foreach ($line in $lines) { + $parts = $line.Split("|") + if ($parts.Count -ge 6 -and $parts[3] -eq $ContainerName) { + return @{ + Id = $parts[0] + Image = $parts[1] + Status = $parts[2] + Name = $parts[3] + Ports = $parts[4] + Command = $parts[5] + } + } + } + + return $null +} + +function Get-DockerTableText { + param($ContainerRecord) + + if ($null -eq $ContainerRecord) { + return "Container not found." + } + + if ($ContainerRecord.ContainsKey("Error")) { + return $ContainerRecord.Error + } + + return @" +CONTAINER ID IMAGE STATUS PORTS NAMES +$($ContainerRecord.Id) $($ContainerRecord.Image) $($ContainerRecord.Status) $($ContainerRecord.Ports) $($ContainerRecord.Name) +"@.TrimEnd() +} + +function Invoke-PsqlInContainer { + param( + [string]$DockerExe, + [string]$ContainerName, + [string]$DbUser, + [string]$DbName, + [string]$DbPassword, + [string]$Sql + ) + + return Invoke-ExternalCapture -FilePath $DockerExe -Arguments @( + "exec", + "-e", "PGPASSWORD=$DbPassword", + $ContainerName, + "psql", + "-U", $DbUser, + "-d", $DbName, + "-At", + "-c", $Sql + ) +} + +function Get-LaunchTail { + param( + [string]$MonitorDir, + [string]$FileName, + [int]$Tail = 50 + ) + + if ([string]::IsNullOrWhiteSpace($MonitorDir)) { return "" } + + $path = Join-Path $MonitorDir $FileName + if (-not (Test-Path -LiteralPath $path)) { + return "" + } + + return (Get-Content -LiteralPath $path -Tail $Tail | Out-String -Width 4096).TrimEnd() +} + +function Run-JcmdSection { + param( + [string]$JcmdExe, + [string]$Pid, + [string[]]$Args + ) + + if (-not $Pid -or $Pid -notmatch '^\d+$') { + return "No live OSCAR JVM found." + } + + if (-not $JcmdExe) { + return "jcmd.exe not found." + } + + if (-not (Test-Path -LiteralPath $JcmdExe)) { + return "jcmd.exe path does not exist: $JcmdExe" + } + + $stdoutFile = [System.IO.Path]::GetTempFileName() + $stderrFile = [System.IO.Path]::GetTempFileName() + + try { + $proc = Start-Process ` + -FilePath $JcmdExe ` + -ArgumentList (@($Pid) + $Args) ` + -NoNewWindow ` + -Wait ` + -PassThru ` + -RedirectStandardOutput $stdoutFile ` + -RedirectStandardError $stderrFile + + $stdout = "" + $stderr = "" + + if (Test-Path -LiteralPath $stdoutFile) { + $stdout = Get-Content -LiteralPath $stdoutFile -Raw + } + + if (Test-Path -LiteralPath $stderrFile) { + $stderr = Get-Content -LiteralPath $stderrFile -Raw + } + + $output = ($stdout + $stderr).TrimEnd() + $exitCode = $proc.ExitCode + + if ($exitCode -ne 0) { + if ([string]::IsNullOrWhiteSpace($output)) { + return "jcmd failed with exit code $exitCode using: $JcmdExe $Pid $($Args -join ' ')" + } + return $output + } + + if ([string]::IsNullOrWhiteSpace($output)) { + return "jcmd returned no output using: $JcmdExe $Pid $($Args -join ' ')" + } + + return $output + } + catch { + return ($_ | Out-String).TrimEnd() + } + finally { + Remove-Item -LiteralPath $stdoutFile -Force -ErrorAction SilentlyContinue + Remove-Item -LiteralPath $stderrFile -Force -ErrorAction SilentlyContinue + } +} + +$envMap = Load-DotEnv -Path (Join-Path $BaseDirectory ".env") + +$containerName = if ($envMap.ContainsKey("CONTAINER_NAME")) { $envMap["CONTAINER_NAME"] } else { "oscar-postgis-container" } +$dbUser = if ($envMap.ContainsKey("DB_USER")) { $envMap["DB_USER"] } else { "postgres" } +$dbName = if ($envMap.ContainsKey("DB_NAME")) { $envMap["DB_NAME"] } else { "gis" } +$dbPassword = if ($envMap.ContainsKey("DB_PASSWORD")) { $envMap["DB_PASSWORD"] } else { "postgres" } + +$monitorDirItem = $null +if (-not [string]::IsNullOrWhiteSpace($MonitorDirectory)) { + if (Test-Path -LiteralPath $MonitorDirectory) { + $monitorDirItem = Get-Item -LiteralPath $MonitorDirectory + } +} +else { + $monitorDirItem = Get-ActiveMonitorDirectory -BaseDir $BaseDirectory + if ($null -eq $monitorDirItem) { + $monitorDirItem = Get-LatestMonitorDirectory -BaseDir $BaseDirectory + } +} + +$monitorDir = if ($null -ne $monitorDirItem) { $monitorDirItem.FullName } else { "" } + +$timestamp = Get-Date -Format "yyyyMMdd-HHmmss" +$outputFile = Join-Path $BaseDirectory "oscar-status-$timestamp.txt" + +$pidFromMonitor = "" +if ($monitorDir) { + $pidPath = Join-Path $monitorDir "jvm-pid.txt" + if (Test-Path -LiteralPath $pidPath) { + $pidFromMonitor = (Get-Content -LiteralPath $pidPath -TotalCount 1 | Out-String).Trim() + } +} + +$oscarJava = Get-OscarJavaProcesses +$liveProc = $null + +if ($pidFromMonitor -match '^\d+$') { + $liveProc = $oscarJava | Where-Object { $_.ProcessId -eq [int]$pidFromMonitor } | Select-Object -First 1 +} +if ($null -eq $liveProc) { + $liveProc = $oscarJava | Select-Object -First 1 +} + +$livePid = if ($null -ne $liveProc) { [string]$liveProc.ProcessId } else { "" } + +$dockerExe = Resolve-ToolPath -Name "docker" +$jcmdExe = Resolve-JcmdPath + +$dockerContainer = Get-DockerContainerRecord -DockerExe $dockerExe -ContainerName $containerName +$dockerTableText = Get-DockerTableText -ContainerRecord $dockerContainer + +$containerRunning = $false +if ($dockerContainer -and -not $dockerContainer.ContainsKey("Error")) { + if ($dockerContainer.Status -like "Up*") { + $containerRunning = $true + } +} + +$osInfo = Get-CimInstance Win32_OperatingSystem | + Select-Object TotalVisibleMemorySize, FreePhysicalMemory, TotalVirtualMemorySize, FreeVirtualMemory +$osInfoText = ($osInfo | Format-List | Out-String -Width 4096).TrimEnd() + +$counterText = "" +try { + $counters = Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\% Usage' + $counterText = ($counters | Out-String -Width 4096).TrimEnd() +} +catch { + $counterText = "Could not read performance counters." +} + +$liveJvmText = "" +if ($livePid -match '^\d+$') { + $liveJvmText = (Get-Process -Id ([int]$livePid) | + Select-Object Id, ProcessName, Threads, VirtualMemorySize64, WorkingSet64, PrivateMemorySize64, CPU, StartTime | + Format-List | Out-String -Width 4096).TrimEnd() +} +else { + $liveJvmText = "No live OSCAR JVM found." +} + +$jfrText = "" +$heapText = "" +$nmtText = "" + +if ($livePid -match '^\d+$' -and $jcmdExe -and (Test-Path -LiteralPath $jcmdExe)) { + try { + $jfrText = (& $jcmdExe $livePid JFR.check 2>&1 | Out-String -Width 4096).TrimEnd() + if ([string]::IsNullOrWhiteSpace($jfrText)) { + $jfrText = "jcmd returned no output for JFR.check" + } + } + catch { + $jfrText = ($_ | Out-String).TrimEnd() + } + + try { + $heapText = (& $jcmdExe $livePid GC.heap_info 2>&1 | Out-String -Width 4096).TrimEnd() + if ([string]::IsNullOrWhiteSpace($heapText)) { + $heapText = "jcmd returned no output for GC.heap_info" + } + } + catch { + $heapText = ($_ | Out-String).TrimEnd() + } + + try { + $nmtText = (& $jcmdExe $livePid VM.native_memory summary 2>&1 | Out-String -Width 4096).TrimEnd() + if ([string]::IsNullOrWhiteSpace($nmtText)) { + $nmtText = "jcmd returned no output for VM.native_memory summary" + } + } + catch { + $nmtText = ($_ | Out-String).TrimEnd() + } +} +else { + $jfrText = "jcmd.exe not found or no live OSCAR JVM found." + $heapText = $jfrText + $nmtText = $jfrText +} + +$dbMetaText = "" +$dbByStateText = "" +$dbByAppText = "" +$dbErrorText = "" + +$maxConnections = "" +$superuserReservedConnections = "" +$usableClientSlots = "" +$totalSessions = "" +$activeSessions = "" +$idleSessions = "" +$idleInTransaction = "" + +if (-not $dockerExe) { + $dbErrorText = "docker.exe not found in PATH." +} +elseif ($null -eq $dockerContainer) { + $dbErrorText = "Container '$containerName' not found." +} +elseif ($dockerContainer.ContainsKey("Error")) { + $dbErrorText = $dockerContainer.Error +} +elseif (-not $containerRunning) { + $dbErrorText = "Container '$containerName' is present but not running. Status: $($dockerContainer.Status)" +} +else { + $dbMetaSql = "select current_setting('max_connections'), current_setting('superuser_reserved_connections'), (current_setting('max_connections')::int - current_setting('superuser_reserved_connections')::int), count(*), count(*) filter (where state = 'active'), count(*) filter (where state = 'idle'), count(*) filter (where state = 'idle in transaction') from pg_stat_activity;" + $dbByStateSql = "select coalesce(state,'') || '|' || count(*)::text from pg_stat_activity group by state order by 1;" + $dbByAppSql = "select coalesce(application_name,'') || '|' || coalesce(usename,'') || '|' || coalesce(client_addr::text,'') || '|' || coalesce(state,'') || '|' || count(*)::text from pg_stat_activity group by application_name, usename, client_addr, state order by application_name, usename, client_addr, state;" + + $dbMetaText = Invoke-PsqlInContainer -DockerExe $dockerExe -ContainerName $containerName -DbUser $dbUser -DbName $dbName -DbPassword $dbPassword -Sql $dbMetaSql + $dbByStateText = Invoke-PsqlInContainer -DockerExe $dockerExe -ContainerName $containerName -DbUser $dbUser -DbName $dbName -DbPassword $dbPassword -Sql $dbByStateSql + $dbByAppText = Invoke-PsqlInContainer -DockerExe $dockerExe -ContainerName $containerName -DbUser $dbUser -DbName $dbName -DbPassword $dbPassword -Sql $dbByAppSql + + $metaLine = ($dbMetaText -split "`r?`n" | Where-Object { $_.Trim().Length -gt 0 } | Select-Object -First 1) + if ($metaLine -and $metaLine.Contains("|")) { + $parts = $metaLine.Split("|") + if ($parts.Count -ge 7) { + $maxConnections = $parts[0] + $superuserReservedConnections = $parts[1] + $usableClientSlots = $parts[2] + $totalSessions = $parts[3] + $activeSessions = $parts[4] + $idleSessions = $parts[5] + $idleInTransaction = $parts[6] + } + } + else { + if ([string]::IsNullOrWhiteSpace($dbMetaText)) { + $dbErrorText = "psql returned no DB metadata output." + } + else { + $dbErrorText = $dbMetaText + } + } +} + +$snapshotDirs = @() +if ($monitorDir -and (Test-Path -LiteralPath $monitorDir)) { + $snapshotDirs = Get-ChildItem -LiteralPath $monitorDir -Directory | + Where-Object { $_.Name -match '^\d{8}-\d{6}$' } | + Sort-Object Name +} + +$firstSnapshot = if ($snapshotDirs.Count -gt 0) { $snapshotDirs[0].FullName } else { "" } +$latestSnapshot = if ($snapshotDirs.Count -gt 0) { $snapshotDirs[-1].FullName } else { "" } + +$recentSnapshotLines = @() +if ($snapshotDirs.Count -gt 0) { + $recentSnapshotLines = $snapshotDirs | + Select-Object -Last ([Math]::Min(20, $snapshotDirs.Count)) | + ForEach-Object { $_.Name } +} + +$launchStdoutTail = Get-LaunchTail -MonitorDir $monitorDir -FileName "launch.stdout.log" -Tail 50 +$launchStderrTail = Get-LaunchTail -MonitorDir $monitorDir -FileName "launch.stderr.log" -Tail 50 + +$dockerLogsTail = "" +if ($dockerExe -and $containerRunning) { + $dockerLogsTail = Invoke-ExternalCapture -FilePath $dockerExe -Arguments @("logs", "--tail", "100", $containerName) +} + +$procTableText = "" +if ($liveProc) { + $procTableText = ($liveProc | + Select-Object ProcessId, Name, CommandLine | + Format-Table -AutoSize | Out-String -Width 4096).TrimEnd() +} +else { + $procTableText = "No live OSCAR Java process found." +} + +$lines = New-Object 'System.Collections.Generic.List[string]' + +Add-Line $lines "OSCAR STATUS REPORT" +Add-Line $lines ("Generated: " + (Get-Date).ToString("o")) +Add-Line $lines ("Base directory: " + $BaseDirectory) +Add-Line $lines ("Monitor directory: " + $(if ($monitorDir) { $monitorDir } else { "" })) +Add-Line $lines ("Output file: " + $outputFile) +Add-Line $lines "" + +Add-Line $lines "=== PROCESS STATUS ===" +Add-Line $lines ("PID from monitor: " + $pidFromMonitor) +Add-Line $lines ("Live OSCAR PID: " + $livePid) +Add-Line $lines "" +Add-Block $lines $procTableText +Add-Line $lines "" +Add-Block $lines $dockerTableText +Add-Line $lines "" + +Add-Line $lines "=== SYSTEM MEMORY AND PAGEFILE ===" +Add-Line $lines "" +Add-Block $lines $osInfoText +Add-Line $lines "" +Add-Block $lines $counterText +Add-Line $lines "" + +Add-Line $lines "=== LIVE JVM PROCESS ===" +Add-Line $lines "" +Add-Block $lines $liveJvmText +Add-Line $lines "" + +Add-Line $lines "=== LIVE JVM JFR STATUS ===" +Add-Block $lines $jfrText +Add-Line $lines "" + +Add-Line $lines "=== LIVE JVM GC HEAP INFO ===" +Add-Block $lines $heapText +Add-Line $lines "" + +Add-Line $lines "=== LIVE JVM NATIVE MEMORY SUMMARY ===" +Add-Block $lines $nmtText +Add-Line $lines "" + +Add-Line $lines "=== LIVE POSTGRES STATUS ===" +Add-Line $lines ("max_connections: " + $maxConnections) +Add-Line $lines ("superuser_reserved_connections: " + $superuserReservedConnections) +Add-Line $lines ("usable_client_slots: " + $usableClientSlots) +Add-Line $lines ("total_sessions: " + $totalSessions) +Add-Line $lines ("active: " + $activeSessions) +Add-Line $lines ("idle: " + $idleSessions) +Add-Line $lines ("idle in transaction: " + $idleInTransaction) +Add-Line $lines "" +Add-Line $lines "--- db-by-state ---" +Add-Block $lines $dbByStateText +Add-Line $lines "" +Add-Line $lines "--- db-by-app ---" +Add-Block $lines $dbByAppText +Add-Line $lines "" +Add-Line $lines "--- db-error ---" +Add-Block $lines $dbErrorText +Add-Line $lines "" + +Add-Line $lines "=== SNAPSHOT STATUS ===" +Add-Line $lines ("Snapshot count: " + $snapshotDirs.Count) +Add-Line $lines ("First snapshot: " + $firstSnapshot) +Add-Line $lines ("Latest snapshot: " + $latestSnapshot) +Add-Line $lines "" + +Add-Line $lines "=== RECENT SNAPSHOTS (LAST 20) ===" +foreach ($snap in $recentSnapshotLines) { + Add-Line $lines $snap +} +Add-Line $lines "" + +Add-Line $lines "=== LOG TAILS ===" +Add-Line $lines "--- launch.stdout.log (last 50 lines) ---" +Add-Block $lines $launchStdoutTail +Add-Line $lines "" +Add-Line $lines "--- launch.stderr.log (last 50 lines) ---" +Add-Block $lines $launchStderrTail +Add-Line $lines "" +Add-Line $lines "--- postgres docker logs (last captured 100 lines) ---" +Add-Block $lines $dockerLogsTail +Add-Line $lines "" + +Add-Line $lines "=== QUICK READ ===" +Add-Line $lines ("Live JVM PID: " + $livePid) +Add-Line $lines ("Snapshots captured: " + $snapshotDirs.Count) +if ($totalSessions) { Add-Line $lines ("DB total sessions: " + $totalSessions) } +if ($usableClientSlots) { Add-Line $lines ("DB usable client slots: " + $usableClientSlots) } +Add-Line $lines "Interpretation guide:" +Add-Line $lines "- Healthy memory: process memory and JVM native memory plateau." +Add-Line $lines "- Healthy DB: total sessions rise at startup and then plateau well below usable client slots." +Add-Line $lines "- Suspicious DB: total sessions keep climbing, idle sessions pile up, or db-error shows query failures." + +$reportText = ($lines -join "`r`n") +Set-Content -LiteralPath $outputFile -Value $reportText -Encoding UTF8 Write-Output $reportText \ No newline at end of file diff --git a/dist/release/launch-all.bat b/dist/release/launch-all.bat index 1d37a44..4b1abd5 100755 --- a/dist/release/launch-all.bat +++ b/dist/release/launch-all.bat @@ -1,226 +1,226 @@ -@echo off -setlocal EnableExtensions EnableDelayedExpansion - -set "SCRIPT_DIR=%~dp0" -if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" - -set "NODE_DIR=%SCRIPT_DIR%\osh-node-oscar" -set "POSTGIS_DIR=%SCRIPT_DIR%\postgis" -set "ENV_FILE=%SCRIPT_DIR%\.env" - -if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" - -if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" -if not defined DB_NAME set "DB_NAME=gis" -if not defined DB_USER set "DB_USER=postgres" -if not defined DB_PASSWORD set "DB_PASSWORD=postgres" -if not defined DB_PORT set "DB_PORT=5432" -if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" -if not defined POSTGIS_IMAGE_NAME set "POSTGIS_IMAGE_NAME=oscar-postgis" -if not defined POSTGIS_DOCKERFILE set "POSTGIS_DOCKERFILE=Dockerfile" -if not defined FORCE_RESTART set "FORCE_RESTART=0" -if not defined RETRY_MAX set "RETRY_MAX=120" -if not defined RETRY_INTERVAL set "RETRY_INTERVAL=2" -if not defined POSTGIS_READY_DELAY set "POSTGIS_READY_DELAY=5" - -set "PGDATA_DIR=%SCRIPT_DIR%\pgdata" - -if not exist "%POSTGIS_DIR%" ( - echo ERROR: Missing PostGIS directory: "%POSTGIS_DIR%" - exit /b 1 -) - -if not exist "%POSTGIS_DIR%\%POSTGIS_DOCKERFILE%" ( - echo ERROR: Missing PostGIS Dockerfile: "%POSTGIS_DIR%\%POSTGIS_DOCKERFILE%" - exit /b 1 -) - -if not exist "%POSTGIS_DIR%\init-extensions.sql" ( - echo ERROR: Missing PostGIS init script: "%POSTGIS_DIR%\init-extensions.sql" - exit /b 1 -) - -if not exist "%NODE_DIR%\launch.bat" ( - echo ERROR: Missing node launcher: "%NODE_DIR%\launch.bat" - exit /b 1 -) - -where docker >nul 2>nul -if errorlevel 1 ( - echo ERROR: Docker was not found in PATH. - exit /b 1 -) - -docker version >nul 2>nul -if errorlevel 1 ( - echo ERROR: Docker is installed but not responding. - exit /b 1 -) - -where java >nul 2>nul -if errorlevel 1 ( - echo ERROR: Java was not found in PATH. - exit /b 1 -) - -call :check_existing_oscar -if defined OSCAR_PID ( - if /I "%FORCE_RESTART%"=="1" ( - echo OSCAR is already running with PID !OSCAR_PID!. FORCE_RESTART=1, stopping it first... - call :stop_existing_oscar - call :wait_for_oscar_stop 60 - call :check_existing_oscar - if defined OSCAR_PID ( - echo ERROR: OSCAR is still running with PID !OSCAR_PID! after stop attempt. - exit /b 1 - ) - ) else ( - echo ERROR: OSCAR is already running with PID !OSCAR_PID!. - echo Set FORCE_RESTART=1 in .env to replace the running instance. - exit /b 1 - ) -) - -if /I "%SYSTEM_PROFILE%"=="RPI4" ( - set "PG_MAX_CONNECTIONS=75" - set "PG_SHARED_BUFFERS=256MB" - set "PG_EFFECTIVE_CACHE_SIZE=1024MB" - set "PG_WORK_MEM=2MB" - set "PG_MAINTENANCE_WORK_MEM=64MB" -) else if /I "%SYSTEM_PROFILE%"=="8GB" ( - set "PG_MAX_CONNECTIONS=125" - set "PG_SHARED_BUFFERS=1024MB" - set "PG_EFFECTIVE_CACHE_SIZE=3072MB" - set "PG_WORK_MEM=4MB" - set "PG_MAINTENANCE_WORK_MEM=128MB" -) else if /I "%SYSTEM_PROFILE%"=="16GB" ( - set "PG_MAX_CONNECTIONS=200" - set "PG_SHARED_BUFFERS=2048MB" - set "PG_EFFECTIVE_CACHE_SIZE=6144MB" - set "PG_WORK_MEM=4MB" - set "PG_MAINTENANCE_WORK_MEM=256MB" -) else if /I "%SYSTEM_PROFILE%"=="32GB" ( - set "PG_MAX_CONNECTIONS=300" - set "PG_SHARED_BUFFERS=4096MB" - set "PG_EFFECTIVE_CACHE_SIZE=12288MB" - set "PG_WORK_MEM=8MB" - set "PG_MAINTENANCE_WORK_MEM=512MB" -) else ( - echo WARNING: Unknown SYSTEM_PROFILE "%SYSTEM_PROFILE%". Using 8GB defaults. - set "PG_MAX_CONNECTIONS=125" - set "PG_SHARED_BUFFERS=1024MB" - set "PG_EFFECTIVE_CACHE_SIZE=3072MB" - set "PG_WORK_MEM=4MB" - set "PG_MAINTENANCE_WORK_MEM=128MB" -) - -if not exist "%PGDATA_DIR%" mkdir "%PGDATA_DIR%" - -echo Building PostGIS Docker image... -docker build -t "%POSTGIS_IMAGE_NAME%" -f "%POSTGIS_DIR%\%POSTGIS_DOCKERFILE%" "%POSTGIS_DIR%" -if errorlevel 1 ( - echo ERROR: Failed to build PostGIS Docker image. - exit /b 1 -) - -echo Preparing PostGIS container for profile: %SYSTEM_PROFILE% -echo Image: %POSTGIS_IMAGE_NAME% -echo Port: %DB_PORT%:5432 -echo Data: %PGDATA_DIR% - -docker ps -a --format "{{.Names}}" | findstr /I /X "%CONTAINER_NAME%" >nul -if not errorlevel 1 ( - echo Removing existing container "%CONTAINER_NAME%" so updated settings take effect... - docker rm -f "%CONTAINER_NAME%" >nul 2>nul -) - -echo Creating new container... -docker run -d ^ - --name "%CONTAINER_NAME%" ^ - -p %DB_PORT%:5432 ^ - -e POSTGRES_DB=%DB_NAME% ^ - -e POSTGRES_USER=%DB_USER% ^ - -e POSTGRES_PASSWORD=%DB_PASSWORD% ^ - -v "%PGDATA_DIR%:/var/lib/postgresql/data" ^ - "%POSTGIS_IMAGE_NAME%" ^ - -c max_connections=%PG_MAX_CONNECTIONS% ^ - -c superuser_reserved_connections=10 ^ - -c shared_buffers=%PG_SHARED_BUFFERS% ^ - -c effective_cache_size=%PG_EFFECTIVE_CACHE_SIZE% ^ - -c work_mem=%PG_WORK_MEM% ^ - -c maintenance_work_mem=%PG_MAINTENANCE_WORK_MEM% ^ - -c idle_session_timeout=600000 ^ - -c log_connections=on ^ - -c log_disconnections=on -if errorlevel 1 ( - echo ERROR: Failed to start PostGIS container. - exit /b 1 -) - -echo Waiting for PostGIS to be ready... -set /a WAIT_COUNT=0 - -:wait_for_postgis -docker exec "%CONTAINER_NAME%" pg_isready -U "%DB_USER%" -d "%DB_NAME%" >nul 2>nul -if not errorlevel 1 goto postgis_ready - -set /a WAIT_COUNT+=1 -if !WAIT_COUNT! GEQ %RETRY_MAX% ( - echo ERROR: PostGIS did not become ready in time. - docker logs "%CONTAINER_NAME%" - exit /b 1 -) - -timeout /t %RETRY_INTERVAL% /nobreak >nul -goto wait_for_postgis - -:postgis_ready -echo PostGIS is ready. -if %POSTGIS_READY_DELAY% GTR 0 timeout /t %POSTGIS_READY_DELAY% /nobreak >nul - -pushd "%NODE_DIR%" -call launch.bat -set "NODE_EXIT=%ERRORLEVEL%" -popd - -endlocal & exit /b %NODE_EXIT% - -:check_existing_oscar -set "OSCAR_PID=" -for /f "usebackq delims=" %%P in (` - powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { [Console]::Write($proc.ProcessId); break } }" 2^>nul -`) do set "OSCAR_PID=%%P" -exit /b 0 - -:stop_existing_oscar -if not defined OSCAR_PID exit /b 0 -powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %OSCAR_PID% -Force -ErrorAction Stop; exit 0 } catch { exit 1 }" >nul 2>nul -exit /b 0 - -:wait_for_oscar_stop -set "WAIT_LIMIT=%~1" -if not defined WAIT_LIMIT set "WAIT_LIMIT=60" -set /a WAITED=0 - -:wait_for_oscar_stop_loop -call :check_existing_oscar -if not defined OSCAR_PID exit /b 0 -if !WAITED! GEQ %WAIT_LIMIT% exit /b 0 -timeout /t 1 /nobreak >nul -set /a WAITED+=1 -goto wait_for_oscar_stop_loop - -:load_env -for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( - set "ENV_NAME=%%A" - set "ENV_VALUE=%%B" - call :set_env_var -) -exit /b 0 - -:set_env_var -if not defined ENV_NAME exit /b 0 -if "%ENV_NAME:~0,1%"=="#" exit /b 0 -if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" -set "%ENV_NAME%=%ENV_VALUE%" +@echo off +setlocal EnableExtensions EnableDelayedExpansion + +set "SCRIPT_DIR=%~dp0" +if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" + +set "NODE_DIR=%SCRIPT_DIR%\osh-node-oscar" +set "POSTGIS_DIR=%SCRIPT_DIR%\postgis" +set "ENV_FILE=%SCRIPT_DIR%\.env" + +if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" + +if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" +if not defined DB_NAME set "DB_NAME=gis" +if not defined DB_USER set "DB_USER=postgres" +if not defined DB_PASSWORD set "DB_PASSWORD=postgres" +if not defined DB_PORT set "DB_PORT=5432" +if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" +if not defined POSTGIS_IMAGE_NAME set "POSTGIS_IMAGE_NAME=oscar-postgis" +if not defined POSTGIS_DOCKERFILE set "POSTGIS_DOCKERFILE=Dockerfile" +if not defined FORCE_RESTART set "FORCE_RESTART=0" +if not defined RETRY_MAX set "RETRY_MAX=120" +if not defined RETRY_INTERVAL set "RETRY_INTERVAL=2" +if not defined POSTGIS_READY_DELAY set "POSTGIS_READY_DELAY=5" + +set "PGDATA_DIR=%SCRIPT_DIR%\pgdata" + +if not exist "%POSTGIS_DIR%" ( + echo ERROR: Missing PostGIS directory: "%POSTGIS_DIR%" + exit /b 1 +) + +if not exist "%POSTGIS_DIR%\%POSTGIS_DOCKERFILE%" ( + echo ERROR: Missing PostGIS Dockerfile: "%POSTGIS_DIR%\%POSTGIS_DOCKERFILE%" + exit /b 1 +) + +if not exist "%POSTGIS_DIR%\init-extensions.sql" ( + echo ERROR: Missing PostGIS init script: "%POSTGIS_DIR%\init-extensions.sql" + exit /b 1 +) + +if not exist "%NODE_DIR%\launch.bat" ( + echo ERROR: Missing node launcher: "%NODE_DIR%\launch.bat" + exit /b 1 +) + +where docker >nul 2>nul +if errorlevel 1 ( + echo ERROR: Docker was not found in PATH. + exit /b 1 +) + +docker version >nul 2>nul +if errorlevel 1 ( + echo ERROR: Docker is installed but not responding. + exit /b 1 +) + +where java >nul 2>nul +if errorlevel 1 ( + echo ERROR: Java was not found in PATH. + exit /b 1 +) + +call :check_existing_oscar +if defined OSCAR_PID ( + if /I "%FORCE_RESTART%"=="1" ( + echo OSCAR is already running with PID !OSCAR_PID!. FORCE_RESTART=1, stopping it first... + call :stop_existing_oscar + call :wait_for_oscar_stop 60 + call :check_existing_oscar + if defined OSCAR_PID ( + echo ERROR: OSCAR is still running with PID !OSCAR_PID! after stop attempt. + exit /b 1 + ) + ) else ( + echo ERROR: OSCAR is already running with PID !OSCAR_PID!. + echo Set FORCE_RESTART=1 in .env to replace the running instance. + exit /b 1 + ) +) + +if /I "%SYSTEM_PROFILE%"=="RPI4" ( + set "PG_MAX_CONNECTIONS=75" + set "PG_SHARED_BUFFERS=256MB" + set "PG_EFFECTIVE_CACHE_SIZE=1024MB" + set "PG_WORK_MEM=2MB" + set "PG_MAINTENANCE_WORK_MEM=64MB" +) else if /I "%SYSTEM_PROFILE%"=="8GB" ( + set "PG_MAX_CONNECTIONS=125" + set "PG_SHARED_BUFFERS=1024MB" + set "PG_EFFECTIVE_CACHE_SIZE=3072MB" + set "PG_WORK_MEM=4MB" + set "PG_MAINTENANCE_WORK_MEM=128MB" +) else if /I "%SYSTEM_PROFILE%"=="16GB" ( + set "PG_MAX_CONNECTIONS=200" + set "PG_SHARED_BUFFERS=2048MB" + set "PG_EFFECTIVE_CACHE_SIZE=6144MB" + set "PG_WORK_MEM=4MB" + set "PG_MAINTENANCE_WORK_MEM=256MB" +) else if /I "%SYSTEM_PROFILE%"=="32GB" ( + set "PG_MAX_CONNECTIONS=300" + set "PG_SHARED_BUFFERS=4096MB" + set "PG_EFFECTIVE_CACHE_SIZE=12288MB" + set "PG_WORK_MEM=8MB" + set "PG_MAINTENANCE_WORK_MEM=512MB" +) else ( + echo WARNING: Unknown SYSTEM_PROFILE "%SYSTEM_PROFILE%". Using 8GB defaults. + set "PG_MAX_CONNECTIONS=125" + set "PG_SHARED_BUFFERS=1024MB" + set "PG_EFFECTIVE_CACHE_SIZE=3072MB" + set "PG_WORK_MEM=4MB" + set "PG_MAINTENANCE_WORK_MEM=128MB" +) + +if not exist "%PGDATA_DIR%" mkdir "%PGDATA_DIR%" + +echo Building PostGIS Docker image... +docker build -t "%POSTGIS_IMAGE_NAME%" -f "%POSTGIS_DIR%\%POSTGIS_DOCKERFILE%" "%POSTGIS_DIR%" +if errorlevel 1 ( + echo ERROR: Failed to build PostGIS Docker image. + exit /b 1 +) + +echo Preparing PostGIS container for profile: %SYSTEM_PROFILE% +echo Image: %POSTGIS_IMAGE_NAME% +echo Port: %DB_PORT%:5432 +echo Data: %PGDATA_DIR% + +docker ps -a --format "{{.Names}}" | findstr /I /X "%CONTAINER_NAME%" >nul +if not errorlevel 1 ( + echo Removing existing container "%CONTAINER_NAME%" so updated settings take effect... + docker rm -f "%CONTAINER_NAME%" >nul 2>nul +) + +echo Creating new container... +docker run -d ^ + --name "%CONTAINER_NAME%" ^ + -p %DB_PORT%:5432 ^ + -e POSTGRES_DB=%DB_NAME% ^ + -e POSTGRES_USER=%DB_USER% ^ + -e POSTGRES_PASSWORD=%DB_PASSWORD% ^ + -v "%PGDATA_DIR%:/var/lib/postgresql/data" ^ + "%POSTGIS_IMAGE_NAME%" ^ + -c max_connections=%PG_MAX_CONNECTIONS% ^ + -c superuser_reserved_connections=10 ^ + -c shared_buffers=%PG_SHARED_BUFFERS% ^ + -c effective_cache_size=%PG_EFFECTIVE_CACHE_SIZE% ^ + -c work_mem=%PG_WORK_MEM% ^ + -c maintenance_work_mem=%PG_MAINTENANCE_WORK_MEM% ^ + -c idle_session_timeout=600000 ^ + -c log_connections=on ^ + -c log_disconnections=on +if errorlevel 1 ( + echo ERROR: Failed to start PostGIS container. + exit /b 1 +) + +echo Waiting for PostGIS to be ready... +set /a WAIT_COUNT=0 + +:wait_for_postgis +docker exec "%CONTAINER_NAME%" pg_isready -U "%DB_USER%" -d "%DB_NAME%" >nul 2>nul +if not errorlevel 1 goto postgis_ready + +set /a WAIT_COUNT+=1 +if !WAIT_COUNT! GEQ %RETRY_MAX% ( + echo ERROR: PostGIS did not become ready in time. + docker logs "%CONTAINER_NAME%" + exit /b 1 +) + +timeout /t %RETRY_INTERVAL% /nobreak >nul +goto wait_for_postgis + +:postgis_ready +echo PostGIS is ready. +if %POSTGIS_READY_DELAY% GTR 0 timeout /t %POSTGIS_READY_DELAY% /nobreak >nul + +pushd "%NODE_DIR%" +call launch.bat +set "NODE_EXIT=%ERRORLEVEL%" +popd + +endlocal & exit /b %NODE_EXIT% + +:check_existing_oscar +set "OSCAR_PID=" +for /f "usebackq delims=" %%P in (` + powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { [Console]::Write($proc.ProcessId); break } }" 2^>nul +`) do set "OSCAR_PID=%%P" +exit /b 0 + +:stop_existing_oscar +if not defined OSCAR_PID exit /b 0 +powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %OSCAR_PID% -Force -ErrorAction Stop; exit 0 } catch { exit 1 }" >nul 2>nul +exit /b 0 + +:wait_for_oscar_stop +set "WAIT_LIMIT=%~1" +if not defined WAIT_LIMIT set "WAIT_LIMIT=60" +set /a WAITED=0 + +:wait_for_oscar_stop_loop +call :check_existing_oscar +if not defined OSCAR_PID exit /b 0 +if !WAITED! GEQ %WAIT_LIMIT% exit /b 0 +timeout /t 1 /nobreak >nul +set /a WAITED+=1 +goto wait_for_oscar_stop_loop + +:load_env +for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( + set "ENV_NAME=%%A" + set "ENV_VALUE=%%B" + call :set_env_var +) +exit /b 0 + +:set_env_var +if not defined ENV_NAME exit /b 0 +if "%ENV_NAME:~0,1%"=="#" exit /b 0 +if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" +set "%ENV_NAME%=%ENV_VALUE%" exit /b 0 \ No newline at end of file diff --git a/dist/release/monitor-oscar.bat b/dist/release/monitor-oscar.bat index 725d829..2bcb15c 100644 --- a/dist/release/monitor-oscar.bat +++ b/dist/release/monitor-oscar.bat @@ -1,351 +1,351 @@ -@echo off -setlocal EnableExtensions EnableDelayedExpansion - -set "SCRIPT_DIR=%~dp0" -if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" -set "STATE_DIR=%SCRIPT_DIR%\.monitor-state" -set "MONITOR_LOCK_DIR=%STATE_DIR%\lock" -set "HEARTBEAT_FILE=%MONITOR_LOCK_DIR%\heartbeat.txt" -set "ACTIVE_MONITOR_FILE=%STATE_DIR%\active-monitor-dir.txt" -set "STATUS_FILE=%SCRIPT_DIR%\monitor.last-status" -set "ERROR_FILE=%SCRIPT_DIR%\monitor.last-error" -set "ENV_FILE=%SCRIPT_DIR%\.env" -set "LAUNCH_CMD=%SCRIPT_DIR%\launch-all.bat" -set "JVM_MATCH=com.botts.impl.security.SensorHubWrapper" -set "OUT_DIR=" -set "OSCAR_PID=" -set "WAITED=0" -set "STOP_REQUESTED=0" -set "CLEANUP_REASON=STOPPED" - -if not exist "%STATE_DIR%" mkdir "%STATE_DIR%" >nul 2>nul - -if /I "%~1"=="stop" goto :stop_monitor - -if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" - -if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" -if not defined MONITOR_INTERVAL set "MONITOR_INTERVAL=60" -if not defined MAX_WAIT_SECONDS set "MAX_WAIT_SECONDS=300" -if not defined JFR_NAME set "JFR_NAME=oscar" -if not defined JFR_MAX_AGE set "JFR_MAX_AGE=4h" -if not defined JFR_MAX_SIZE set "JFR_MAX_SIZE=1g" -if not defined ATTACH_TO_EXISTING set "ATTACH_TO_EXISTING=0" -if not defined FORCE_RESTART set "FORCE_RESTART=0" - -call :write_status STARTING monitor_batch=%~nx0 -call :clear_error - -if not exist "%LAUNCH_CMD%" ( - echo Error: Missing launch command: "%LAUNCH_CMD%" - call :write_error Missing launch command: "%LAUNCH_CMD%" - call :write_status FAILED launch_command_missing path="%LAUNCH_CMD%" - exit /b 1 -) - -call :acquire_monitor_lock -if errorlevel 1 exit /b 1 - -for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "STAMP=%%I" -set "OUT_DIR=%SCRIPT_DIR%\oscar-monitor-%STAMP%" -mkdir "%OUT_DIR%" >nul 2>nul -if errorlevel 1 ( - echo Error: Could not create monitor output directory "%OUT_DIR%" - call :write_error Could not create monitor output directory "%OUT_DIR%" - call :write_status FAILED out_dir_create path="%OUT_DIR%" - call :release_monitor_lock - exit /b 1 -) - -echo %OUT_DIR%> "%ACTIVE_MONITOR_FILE%" -call :update_heartbeat -call :write_status RUNNING output="%OUT_DIR%" - -echo Monitor output: %OUT_DIR% -echo Launching OSCAR stack... - -where jcmd >nul 2>nul -if errorlevel 1 ( - echo Warning: jcmd not found. JFR and NMT snapshots will be skipped. -) - -call :check_existing_oscar -if defined OSCAR_PID ( - if /I "%ATTACH_TO_EXISTING%"=="1" ( - echo Attaching to existing OSCAR PID %OSCAR_PID%... - call :clear_error - call :write_status RUNNING attached jvm_pid=%OSCAR_PID% output="%OUT_DIR%" - goto :found_java - ) - if /I "%FORCE_RESTART%"=="1" ( - echo Existing OSCAR detected with PID %OSCAR_PID%. FORCE_RESTART=1, replacing it... - call :stop_existing_oscar - call :wait_for_oscar_stop 60 - ) else ( - echo OSCAR is already running with PID %OSCAR_PID%. - echo Set ATTACH_TO_EXISTING=1 to monitor it, or FORCE_RESTART=1 to replace it. - call :write_error OSCAR is already running with PID %OSCAR_PID%. - call :write_status FAILED oscar_already_running pid=%OSCAR_PID% output="%OUT_DIR%" - call :release_monitor_lock - exit /b 1 - ) -) - -call :start_launch -if errorlevel 1 ( - call :write_error Failed to start launch-all.bat - call :write_status FAILED launch_start output="%OUT_DIR%" - call :release_monitor_lock - exit /b 1 -) - -call :write_status WAITING_FOR_JVM output="%OUT_DIR%" -echo Waiting for JVM to appear... -set /a WAITED=0 - -:wait_for_java -call :update_heartbeat -if exist "%OUT_DIR%\stop.request" ( - set "STOP_REQUESTED=1" - set "CLEANUP_REASON=STOPPED" - goto :cleanup -) - -call :check_existing_oscar -if defined OSCAR_PID goto :found_java - -if %WAITED% GEQ %MAX_WAIT_SECONDS% ( - echo Launch timed out before JVM appeared. - call :write_error Timed out waiting for JVM after %MAX_WAIT_SECONDS%s. Check "%OUT_DIR%\launch.stdout.log" and "%OUT_DIR%\launch.stderr.log" - call :write_status FAILED wait_for_jvm_timeout output="%OUT_DIR%" - call :release_monitor_lock - exit /b 1 -) - -timeout /t 2 /nobreak >nul -set /a WAITED+=2 -goto :wait_for_java - -:found_java -echo Found JVM PID: %OSCAR_PID% -> "%OUT_DIR%\jvm-pid.txt" echo %OSCAR_PID% -call :clear_error -call :write_status RUNNING jvm_pid=%OSCAR_PID% output="%OUT_DIR%" - -powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "$p = Get-CimInstance Win32_Process -Filter \"ProcessId=%OSCAR_PID%\"; if ($p) { [System.IO.File]::WriteAllText('%OUT_DIR%\\process-info.txt', ('Timestamp: ' + (Get-Date -Format o) + [Environment]::NewLine + 'JVM PID: %OSCAR_PID%' + [Environment]::NewLine + [Environment]::NewLine + 'Command line:' + [Environment]::NewLine + $p.CommandLine)) }" >nul 2>nul - -where jcmd >nul 2>nul -if not errorlevel 1 ( - jcmd %OSCAR_PID% JFR.start name=%JFR_NAME% settings=profile disk=true maxage=%JFR_MAX_AGE% maxsize=%JFR_MAX_SIZE% filename="%OUT_DIR%\%JFR_NAME%.jfr" > "%OUT_DIR%\jfr-start.txt" 2>&1 - jcmd %OSCAR_PID% VM.native_memory baseline > "%OUT_DIR%\nmt-baseline.txt" 2>&1 -) - -call :snapshot - -echo Monitor is running. Use monitor-oscar.bat stop to stop the stack and dump final data. -:monitor_loop -call :update_heartbeat -if exist "%OUT_DIR%\stop.request" ( - set "STOP_REQUESTED=1" - set "CLEANUP_REASON=STOPPED" - goto :cleanup -) - -call :check_existing_oscar -if not defined OSCAR_PID ( - set "CLEANUP_REASON=EXITED" - goto :cleanup -) - -> "%OUT_DIR%\jvm-pid.txt" echo %OSCAR_PID% -call :snapshot -call :write_status RUNNING jvm_pid=%OSCAR_PID% output="%OUT_DIR%" -timeout /t %MONITOR_INTERVAL% /nobreak >nul -goto :monitor_loop - -:cleanup -if defined OSCAR_PID call :jfr_dump -if "%STOP_REQUESTED%"=="1" ( - echo Stop requested. -) else if /I "%CLEANUP_REASON%"=="EXITED" ( - echo JVM exited. -) - -del "%OUT_DIR%\stop.request" >nul 2>nul -call :release_monitor_lock -if /I "%CLEANUP_REASON%"=="EXITED" ( - call :write_status EXITED output="%OUT_DIR%" -) else ( - call :write_status STOPPED output="%OUT_DIR%" -) -call :clear_error -exit /b 0 - -:stop_monitor -set "MON_DIR=" -if exist "%ACTIVE_MONITOR_FILE%" set /p MON_DIR=<"%ACTIVE_MONITOR_FILE%" -if not defined MON_DIR call :find_latest_monitor_dir - -if not defined MON_DIR ( - echo OSCAR monitor is not running. - call :write_status STOP_REQUESTED no_active_monitor - call :clear_error - call :release_monitor_lock - exit /b 0 -) - -if not exist "%MON_DIR%" ( - echo OSCAR monitor is not running. - call :write_status STOP_REQUESTED no_active_monitor - call :clear_error - call :release_monitor_lock - exit /b 0 -) - -> "%MON_DIR%\stop.request" echo stop -call :write_status STOP_REQUESTED output="%MON_DIR%" -call :clear_error -echo Requested monitor stop: "%MON_DIR%" -exit /b 0 - -:start_launch -powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "$stdout = [System.IO.Path]::Combine('%OUT_DIR%','launch.stdout.log'); $stderr = [System.IO.Path]::Combine('%OUT_DIR%','launch.stderr.log'); Start-Process -WindowStyle Hidden -FilePath 'cmd.exe' -ArgumentList @('/c','call ""%LAUNCH_CMD%""') -WorkingDirectory '%SCRIPT_DIR%' -RedirectStandardOutput $stdout -RedirectStandardError $stderr | Out-Null" -if errorlevel 1 exit /b 1 -exit /b 0 - -:check_existing_oscar -set "OSCAR_PID=" -for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "$p = Get-CimInstance Win32_Process ^| Where-Object { $_.Name -match '^(java|javaw)(\\.exe)?$' -and $null -ne $_.CommandLine -and $_.CommandLine -like '*%JVM_MATCH%*' } ^| Select-Object -First 1 -ExpandProperty ProcessId; if ($p) { $p }"') do set "OSCAR_PID=%%I" -exit /b 0 - -:stop_existing_oscar -if not defined OSCAR_PID exit /b 0 -powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %OSCAR_PID% -Force -ErrorAction Stop } catch {}" >nul 2>nul -exit /b 0 - -:wait_for_oscar_stop -set "WAIT_LIMIT=%~1" -if not defined WAIT_LIMIT set "WAIT_LIMIT=60" -set /a WAITED=0 -:wait_for_oscar_stop_loop -call :check_existing_oscar -if not defined OSCAR_PID exit /b 0 -if !WAITED! GEQ %WAIT_LIMIT% exit /b 0 -timeout /t 1 /nobreak >nul -set /a WAITED+=1 -goto :wait_for_oscar_stop_loop - -:snapshot -if not defined OSCAR_PID exit /b 0 -for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "SNAPSTAMP=%%I" -set "SNAP_DIR=%OUT_DIR%\%SNAPSTAMP%" -mkdir "%SNAP_DIR%" >nul 2>nul - -tasklist /FI "PID eq %OSCAR_PID%" /V > "%SNAP_DIR%\tasklist.txt" 2>&1 -powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "$p = Get-Process -Id %OSCAR_PID% -ErrorAction SilentlyContinue; if ($p) { $p | Select-Object Id,ProcessName,CPU,StartTime,WorkingSet64,PrivateMemorySize64,VirtualMemorySize64,HandleCount,@{Name='ThreadCount';Expression={$_.Threads.Count}} | Format-List * }" > "%SNAP_DIR%\process.txt" 2>&1 -powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\%% Usage' | Format-List *" > "%SNAP_DIR%\memory-counters.txt" 2>&1 -powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "Get-CimInstance Win32_OperatingSystem | Select-Object TotalVisibleMemorySize,FreePhysicalMemory,TotalVirtualMemorySize,FreeVirtualMemory | Format-List *" > "%SNAP_DIR%\os-memory.txt" 2>&1 - -where jcmd >nul 2>nul -if not errorlevel 1 ( - jcmd %OSCAR_PID% VM.native_memory summary > "%SNAP_DIR%\nmt-summary.txt" 2>&1 - jcmd %OSCAR_PID% GC.heap_info > "%SNAP_DIR%\gc-heap-info.txt" 2>&1 - jcmd %OSCAR_PID% Thread.print > "%SNAP_DIR%\thread-print.txt" 2>&1 - jcmd %OSCAR_PID% JFR.check > "%SNAP_DIR%\jfr-check.txt" 2>&1 -) - -docker ps > "%SNAP_DIR%\docker-ps.txt" 2>&1 -exit /b 0 - -:jfr_dump -where jcmd >nul 2>nul -if errorlevel 1 exit /b 0 -if not defined OSCAR_PID exit /b 0 -jcmd %OSCAR_PID% JFR.dump name=%JFR_NAME% filename="%OUT_DIR%\%JFR_NAME%-final.jfr" > "%OUT_DIR%\jfr-dump-final.txt" 2>&1 -exit /b 0 - -:acquire_monitor_lock -if not exist "%STATE_DIR%" mkdir "%STATE_DIR%" >nul 2>nul -mkdir "%MONITOR_LOCK_DIR%" >nul 2>nul -if not errorlevel 1 exit /b 0 - -call :lock_is_fresh -if "%LOCK_FRESH%"=="1" ( - set "EXISTING_OUT_DIR=" - if exist "%ACTIVE_MONITOR_FILE%" set /p EXISTING_OUT_DIR=<"%ACTIVE_MONITOR_FILE%" - echo Error: Another monitor-oscar.bat instance is already running. - if defined EXISTING_OUT_DIR echo Active monitor output: %EXISTING_OUT_DIR% - echo Run stop-all.bat or monitor-oscar.bat stop before starting another monitor. - if defined EXISTING_OUT_DIR ( - call :write_error Duplicate monitor start refused. Existing output: %EXISTING_OUT_DIR% - call :write_status FAILED duplicate_monitor output="%EXISTING_OUT_DIR%" - ) else ( - call :write_error Duplicate monitor start refused. - call :write_status FAILED duplicate_monitor - ) - exit /b 1 -) - -echo Removing stale OSCAR monitor lock state. -call :release_monitor_lock -mkdir "%MONITOR_LOCK_DIR%" >nul 2>nul -if errorlevel 1 ( - call :write_error Could not acquire OSCAR monitor lock at "%MONITOR_LOCK_DIR%" - call :write_status FAILED lock_acquire path="%MONITOR_LOCK_DIR%" - exit /b 1 -) -exit /b 0 - -:lock_is_fresh -set "LOCK_FRESH=0" -if not exist "%HEARTBEAT_FILE%" exit /b 0 -for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "$age = (Get-Date) - (Get-Item '%HEARTBEAT_FILE%').LastWriteTime; if ($age.TotalSeconds -lt 180) { 1 } else { 0 }"') do set "LOCK_FRESH=%%I" -exit /b 0 - -:update_heartbeat -if not exist "%MONITOR_LOCK_DIR%" mkdir "%MONITOR_LOCK_DIR%" >nul 2>nul -break> "%HEARTBEAT_FILE%" -exit /b 0 - -:release_monitor_lock -if exist "%HEARTBEAT_FILE%" del "%HEARTBEAT_FILE%" >nul 2>nul -if exist "%ACTIVE_MONITOR_FILE%" del "%ACTIVE_MONITOR_FILE%" >nul 2>nul -if exist "%MONITOR_LOCK_DIR%" rmdir "%MONITOR_LOCK_DIR%" >nul 2>nul -exit /b 0 - -:find_latest_monitor_dir -set "MON_DIR=" -for /f "delims=" %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "$d = Get-ChildItem -LiteralPath '%SCRIPT_DIR%' -Directory -Filter 'oscar-monitor-*' ^| Sort-Object Name -Descending ^| Select-Object -First 1 -ExpandProperty FullName; if ($d) { $d }"') do set "MON_DIR=%%I" -exit /b 0 - -:load_env -for /f "usebackq tokens=* delims=" %%L in ("%~1") do ( - set "LINE=%%L" - if defined LINE ( - if not "!LINE:~0,1!"=="#" ( - for /f "tokens=1,* delims==" %%A in ("!LINE!") do ( - if not "%%A"=="" set "%%A=%%B" - ) - ) - ) -) -exit /b 0 - -:write_status -set "STATUS_TEXT=%*" -> "%STATUS_FILE%" echo %date% %time% %STATUS_TEXT% -exit /b 0 - -:write_error -set "ERROR_TEXT=%*" -> "%ERROR_FILE%" echo %date% %time% %ERROR_TEXT% -exit /b 0 - -:clear_error -> "%ERROR_FILE%" type nul -exit /b 0 +@echo off +setlocal EnableExtensions EnableDelayedExpansion + +set "SCRIPT_DIR=%~dp0" +if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" +set "STATE_DIR=%SCRIPT_DIR%\.monitor-state" +set "MONITOR_LOCK_DIR=%STATE_DIR%\lock" +set "HEARTBEAT_FILE=%MONITOR_LOCK_DIR%\heartbeat.txt" +set "ACTIVE_MONITOR_FILE=%STATE_DIR%\active-monitor-dir.txt" +set "STATUS_FILE=%SCRIPT_DIR%\monitor.last-status" +set "ERROR_FILE=%SCRIPT_DIR%\monitor.last-error" +set "ENV_FILE=%SCRIPT_DIR%\.env" +set "LAUNCH_CMD=%SCRIPT_DIR%\launch-all.bat" +set "JVM_MATCH=com.botts.impl.security.SensorHubWrapper" +set "OUT_DIR=" +set "OSCAR_PID=" +set "WAITED=0" +set "STOP_REQUESTED=0" +set "CLEANUP_REASON=STOPPED" + +if not exist "%STATE_DIR%" mkdir "%STATE_DIR%" >nul 2>nul + +if /I "%~1"=="stop" goto :stop_monitor + +if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" + +if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" +if not defined MONITOR_INTERVAL set "MONITOR_INTERVAL=60" +if not defined MAX_WAIT_SECONDS set "MAX_WAIT_SECONDS=300" +if not defined JFR_NAME set "JFR_NAME=oscar" +if not defined JFR_MAX_AGE set "JFR_MAX_AGE=4h" +if not defined JFR_MAX_SIZE set "JFR_MAX_SIZE=1g" +if not defined ATTACH_TO_EXISTING set "ATTACH_TO_EXISTING=0" +if not defined FORCE_RESTART set "FORCE_RESTART=0" + +call :write_status STARTING monitor_batch=%~nx0 +call :clear_error + +if not exist "%LAUNCH_CMD%" ( + echo Error: Missing launch command: "%LAUNCH_CMD%" + call :write_error Missing launch command: "%LAUNCH_CMD%" + call :write_status FAILED launch_command_missing path="%LAUNCH_CMD%" + exit /b 1 +) + +call :acquire_monitor_lock +if errorlevel 1 exit /b 1 + +for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "STAMP=%%I" +set "OUT_DIR=%SCRIPT_DIR%\oscar-monitor-%STAMP%" +mkdir "%OUT_DIR%" >nul 2>nul +if errorlevel 1 ( + echo Error: Could not create monitor output directory "%OUT_DIR%" + call :write_error Could not create monitor output directory "%OUT_DIR%" + call :write_status FAILED out_dir_create path="%OUT_DIR%" + call :release_monitor_lock + exit /b 1 +) + +echo %OUT_DIR%> "%ACTIVE_MONITOR_FILE%" +call :update_heartbeat +call :write_status RUNNING output="%OUT_DIR%" + +echo Monitor output: %OUT_DIR% +echo Launching OSCAR stack... + +where jcmd >nul 2>nul +if errorlevel 1 ( + echo Warning: jcmd not found. JFR and NMT snapshots will be skipped. +) + +call :check_existing_oscar +if defined OSCAR_PID ( + if /I "%ATTACH_TO_EXISTING%"=="1" ( + echo Attaching to existing OSCAR PID %OSCAR_PID%... + call :clear_error + call :write_status RUNNING attached jvm_pid=%OSCAR_PID% output="%OUT_DIR%" + goto :found_java + ) + if /I "%FORCE_RESTART%"=="1" ( + echo Existing OSCAR detected with PID %OSCAR_PID%. FORCE_RESTART=1, replacing it... + call :stop_existing_oscar + call :wait_for_oscar_stop 60 + ) else ( + echo OSCAR is already running with PID %OSCAR_PID%. + echo Set ATTACH_TO_EXISTING=1 to monitor it, or FORCE_RESTART=1 to replace it. + call :write_error OSCAR is already running with PID %OSCAR_PID%. + call :write_status FAILED oscar_already_running pid=%OSCAR_PID% output="%OUT_DIR%" + call :release_monitor_lock + exit /b 1 + ) +) + +call :start_launch +if errorlevel 1 ( + call :write_error Failed to start launch-all.bat + call :write_status FAILED launch_start output="%OUT_DIR%" + call :release_monitor_lock + exit /b 1 +) + +call :write_status WAITING_FOR_JVM output="%OUT_DIR%" +echo Waiting for JVM to appear... +set /a WAITED=0 + +:wait_for_java +call :update_heartbeat +if exist "%OUT_DIR%\stop.request" ( + set "STOP_REQUESTED=1" + set "CLEANUP_REASON=STOPPED" + goto :cleanup +) + +call :check_existing_oscar +if defined OSCAR_PID goto :found_java + +if %WAITED% GEQ %MAX_WAIT_SECONDS% ( + echo Launch timed out before JVM appeared. + call :write_error Timed out waiting for JVM after %MAX_WAIT_SECONDS%s. Check "%OUT_DIR%\launch.stdout.log" and "%OUT_DIR%\launch.stderr.log" + call :write_status FAILED wait_for_jvm_timeout output="%OUT_DIR%" + call :release_monitor_lock + exit /b 1 +) + +timeout /t 2 /nobreak >nul +set /a WAITED+=2 +goto :wait_for_java + +:found_java +echo Found JVM PID: %OSCAR_PID% +> "%OUT_DIR%\jvm-pid.txt" echo %OSCAR_PID% +call :clear_error +call :write_status RUNNING jvm_pid=%OSCAR_PID% output="%OUT_DIR%" + +powershell -NoProfile -ExecutionPolicy Bypass -Command ^ + "$p = Get-CimInstance Win32_Process -Filter \"ProcessId=%OSCAR_PID%\"; if ($p) { [System.IO.File]::WriteAllText('%OUT_DIR%\\process-info.txt', ('Timestamp: ' + (Get-Date -Format o) + [Environment]::NewLine + 'JVM PID: %OSCAR_PID%' + [Environment]::NewLine + [Environment]::NewLine + 'Command line:' + [Environment]::NewLine + $p.CommandLine)) }" >nul 2>nul + +where jcmd >nul 2>nul +if not errorlevel 1 ( + jcmd %OSCAR_PID% JFR.start name=%JFR_NAME% settings=profile disk=true maxage=%JFR_MAX_AGE% maxsize=%JFR_MAX_SIZE% filename="%OUT_DIR%\%JFR_NAME%.jfr" > "%OUT_DIR%\jfr-start.txt" 2>&1 + jcmd %OSCAR_PID% VM.native_memory baseline > "%OUT_DIR%\nmt-baseline.txt" 2>&1 +) + +call :snapshot + +echo Monitor is running. Use monitor-oscar.bat stop to stop the stack and dump final data. +:monitor_loop +call :update_heartbeat +if exist "%OUT_DIR%\stop.request" ( + set "STOP_REQUESTED=1" + set "CLEANUP_REASON=STOPPED" + goto :cleanup +) + +call :check_existing_oscar +if not defined OSCAR_PID ( + set "CLEANUP_REASON=EXITED" + goto :cleanup +) + +> "%OUT_DIR%\jvm-pid.txt" echo %OSCAR_PID% +call :snapshot +call :write_status RUNNING jvm_pid=%OSCAR_PID% output="%OUT_DIR%" +timeout /t %MONITOR_INTERVAL% /nobreak >nul +goto :monitor_loop + +:cleanup +if defined OSCAR_PID call :jfr_dump +if "%STOP_REQUESTED%"=="1" ( + echo Stop requested. +) else if /I "%CLEANUP_REASON%"=="EXITED" ( + echo JVM exited. +) + +del "%OUT_DIR%\stop.request" >nul 2>nul +call :release_monitor_lock +if /I "%CLEANUP_REASON%"=="EXITED" ( + call :write_status EXITED output="%OUT_DIR%" +) else ( + call :write_status STOPPED output="%OUT_DIR%" +) +call :clear_error +exit /b 0 + +:stop_monitor +set "MON_DIR=" +if exist "%ACTIVE_MONITOR_FILE%" set /p MON_DIR=<"%ACTIVE_MONITOR_FILE%" +if not defined MON_DIR call :find_latest_monitor_dir + +if not defined MON_DIR ( + echo OSCAR monitor is not running. + call :write_status STOP_REQUESTED no_active_monitor + call :clear_error + call :release_monitor_lock + exit /b 0 +) + +if not exist "%MON_DIR%" ( + echo OSCAR monitor is not running. + call :write_status STOP_REQUESTED no_active_monitor + call :clear_error + call :release_monitor_lock + exit /b 0 +) + +> "%MON_DIR%\stop.request" echo stop +call :write_status STOP_REQUESTED output="%MON_DIR%" +call :clear_error +echo Requested monitor stop: "%MON_DIR%" +exit /b 0 + +:start_launch +powershell -NoProfile -ExecutionPolicy Bypass -Command ^ + "$stdout = [System.IO.Path]::Combine('%OUT_DIR%','launch.stdout.log'); $stderr = [System.IO.Path]::Combine('%OUT_DIR%','launch.stderr.log'); Start-Process -WindowStyle Hidden -FilePath 'cmd.exe' -ArgumentList @('/c','call ""%LAUNCH_CMD%""') -WorkingDirectory '%SCRIPT_DIR%' -RedirectStandardOutput $stdout -RedirectStandardError $stderr | Out-Null" +if errorlevel 1 exit /b 1 +exit /b 0 + +:check_existing_oscar +set "OSCAR_PID=" +for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "$p = Get-CimInstance Win32_Process ^| Where-Object { $_.Name -match '^(java|javaw)(\\.exe)?$' -and $null -ne $_.CommandLine -and $_.CommandLine -like '*%JVM_MATCH%*' } ^| Select-Object -First 1 -ExpandProperty ProcessId; if ($p) { $p }"') do set "OSCAR_PID=%%I" +exit /b 0 + +:stop_existing_oscar +if not defined OSCAR_PID exit /b 0 +powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %OSCAR_PID% -Force -ErrorAction Stop } catch {}" >nul 2>nul +exit /b 0 + +:wait_for_oscar_stop +set "WAIT_LIMIT=%~1" +if not defined WAIT_LIMIT set "WAIT_LIMIT=60" +set /a WAITED=0 +:wait_for_oscar_stop_loop +call :check_existing_oscar +if not defined OSCAR_PID exit /b 0 +if !WAITED! GEQ %WAIT_LIMIT% exit /b 0 +timeout /t 1 /nobreak >nul +set /a WAITED+=1 +goto :wait_for_oscar_stop_loop + +:snapshot +if not defined OSCAR_PID exit /b 0 +for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "SNAPSTAMP=%%I" +set "SNAP_DIR=%OUT_DIR%\%SNAPSTAMP%" +mkdir "%SNAP_DIR%" >nul 2>nul + +tasklist /FI "PID eq %OSCAR_PID%" /V > "%SNAP_DIR%\tasklist.txt" 2>&1 +powershell -NoProfile -ExecutionPolicy Bypass -Command ^ + "$p = Get-Process -Id %OSCAR_PID% -ErrorAction SilentlyContinue; if ($p) { $p | Select-Object Id,ProcessName,CPU,StartTime,WorkingSet64,PrivateMemorySize64,VirtualMemorySize64,HandleCount,@{Name='ThreadCount';Expression={$_.Threads.Count}} | Format-List * }" > "%SNAP_DIR%\process.txt" 2>&1 +powershell -NoProfile -ExecutionPolicy Bypass -Command ^ + "Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\%% Usage' | Format-List *" > "%SNAP_DIR%\memory-counters.txt" 2>&1 +powershell -NoProfile -ExecutionPolicy Bypass -Command ^ + "Get-CimInstance Win32_OperatingSystem | Select-Object TotalVisibleMemorySize,FreePhysicalMemory,TotalVirtualMemorySize,FreeVirtualMemory | Format-List *" > "%SNAP_DIR%\os-memory.txt" 2>&1 + +where jcmd >nul 2>nul +if not errorlevel 1 ( + jcmd %OSCAR_PID% VM.native_memory summary > "%SNAP_DIR%\nmt-summary.txt" 2>&1 + jcmd %OSCAR_PID% GC.heap_info > "%SNAP_DIR%\gc-heap-info.txt" 2>&1 + jcmd %OSCAR_PID% Thread.print > "%SNAP_DIR%\thread-print.txt" 2>&1 + jcmd %OSCAR_PID% JFR.check > "%SNAP_DIR%\jfr-check.txt" 2>&1 +) + +docker ps > "%SNAP_DIR%\docker-ps.txt" 2>&1 +exit /b 0 + +:jfr_dump +where jcmd >nul 2>nul +if errorlevel 1 exit /b 0 +if not defined OSCAR_PID exit /b 0 +jcmd %OSCAR_PID% JFR.dump name=%JFR_NAME% filename="%OUT_DIR%\%JFR_NAME%-final.jfr" > "%OUT_DIR%\jfr-dump-final.txt" 2>&1 +exit /b 0 + +:acquire_monitor_lock +if not exist "%STATE_DIR%" mkdir "%STATE_DIR%" >nul 2>nul +mkdir "%MONITOR_LOCK_DIR%" >nul 2>nul +if not errorlevel 1 exit /b 0 + +call :lock_is_fresh +if "%LOCK_FRESH%"=="1" ( + set "EXISTING_OUT_DIR=" + if exist "%ACTIVE_MONITOR_FILE%" set /p EXISTING_OUT_DIR=<"%ACTIVE_MONITOR_FILE%" + echo Error: Another monitor-oscar.bat instance is already running. + if defined EXISTING_OUT_DIR echo Active monitor output: %EXISTING_OUT_DIR% + echo Run stop-all.bat or monitor-oscar.bat stop before starting another monitor. + if defined EXISTING_OUT_DIR ( + call :write_error Duplicate monitor start refused. Existing output: %EXISTING_OUT_DIR% + call :write_status FAILED duplicate_monitor output="%EXISTING_OUT_DIR%" + ) else ( + call :write_error Duplicate monitor start refused. + call :write_status FAILED duplicate_monitor + ) + exit /b 1 +) + +echo Removing stale OSCAR monitor lock state. +call :release_monitor_lock +mkdir "%MONITOR_LOCK_DIR%" >nul 2>nul +if errorlevel 1 ( + call :write_error Could not acquire OSCAR monitor lock at "%MONITOR_LOCK_DIR%" + call :write_status FAILED lock_acquire path="%MONITOR_LOCK_DIR%" + exit /b 1 +) +exit /b 0 + +:lock_is_fresh +set "LOCK_FRESH=0" +if not exist "%HEARTBEAT_FILE%" exit /b 0 +for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "$age = (Get-Date) - (Get-Item '%HEARTBEAT_FILE%').LastWriteTime; if ($age.TotalSeconds -lt 180) { 1 } else { 0 }"') do set "LOCK_FRESH=%%I" +exit /b 0 + +:update_heartbeat +if not exist "%MONITOR_LOCK_DIR%" mkdir "%MONITOR_LOCK_DIR%" >nul 2>nul +break> "%HEARTBEAT_FILE%" +exit /b 0 + +:release_monitor_lock +if exist "%HEARTBEAT_FILE%" del "%HEARTBEAT_FILE%" >nul 2>nul +if exist "%ACTIVE_MONITOR_FILE%" del "%ACTIVE_MONITOR_FILE%" >nul 2>nul +if exist "%MONITOR_LOCK_DIR%" rmdir "%MONITOR_LOCK_DIR%" >nul 2>nul +exit /b 0 + +:find_latest_monitor_dir +set "MON_DIR=" +for /f "delims=" %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "$d = Get-ChildItem -LiteralPath '%SCRIPT_DIR%' -Directory -Filter 'oscar-monitor-*' ^| Sort-Object Name -Descending ^| Select-Object -First 1 -ExpandProperty FullName; if ($d) { $d }"') do set "MON_DIR=%%I" +exit /b 0 + +:load_env +for /f "usebackq tokens=* delims=" %%L in ("%~1") do ( + set "LINE=%%L" + if defined LINE ( + if not "!LINE:~0,1!"=="#" ( + for /f "tokens=1,* delims==" %%A in ("!LINE!") do ( + if not "%%A"=="" set "%%A=%%B" + ) + ) + ) +) +exit /b 0 + +:write_status +set "STATUS_TEXT=%*" +> "%STATUS_FILE%" echo %date% %time% %STATUS_TEXT% +exit /b 0 + +:write_error +set "ERROR_TEXT=%*" +> "%ERROR_FILE%" echo %date% %time% %ERROR_TEXT% +exit /b 0 + +:clear_error +> "%ERROR_FILE%" type nul +exit /b 0 diff --git a/dist/release/reset-all.bat b/dist/release/reset-all.bat index 75c3513..46bb6ac 100755 --- a/dist/release/reset-all.bat +++ b/dist/release/reset-all.bat @@ -1,96 +1,96 @@ -@echo off -setlocal EnableExtensions EnableDelayedExpansion - -set "SCRIPT_DIR=%~dp0" -if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" - -set "ENV_FILE=%SCRIPT_DIR%\.env" -if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" - -if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" - -set "PGDATA_DIR=%SCRIPT_DIR%\pgdata" -set "NODE_DIR=%SCRIPT_DIR%\osh-node-oscar" -set "DB_DIR=%NODE_DIR%\db" -set "FILES_DIR=%NODE_DIR%\files" -set "CONFIG_JSON=%NODE_DIR%\config.json" -set "CONFIG_TEMPLATE=%NODE_DIR%\config.template.json" -set "SECRET_FILE=%NODE_DIR%\.s" - -echo Requesting monitor shutdown... -if exist "%SCRIPT_DIR%\monitor-oscar.bat" ( - call "%SCRIPT_DIR%\monitor-oscar.bat" stop >nul 2>nul -) - -echo Stopping OSCAR Java processes... -for /f "usebackq delims=" %%P in (` - powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { $proc.ProcessId } }" 2^>nul -`) do ( - powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %%P -Force -ErrorAction Stop } catch {}" >nul 2>nul -) - -echo Removing container: %CONTAINER_NAME%... -docker rm -f -v "%CONTAINER_NAME%" >nul 2>nul - -if exist "%PGDATA_DIR%" ( - echo Removing Postgres data directory: %PGDATA_DIR% - rmdir /s /q "%PGDATA_DIR%" -) else ( - echo Postgres data directory not found: %PGDATA_DIR% -) - -if exist "%DB_DIR%" ( - echo Removing OSCAR runtime DB directory: %DB_DIR% - rmdir /s /q "%DB_DIR%" -) else ( - echo OSCAR runtime DB directory not found: %DB_DIR% -) - -if exist "%FILES_DIR%" ( - echo Removing OSCAR files directory: %FILES_DIR% - rmdir /s /q "%FILES_DIR%" -) else ( - echo OSCAR files directory not found: %FILES_DIR% -) - -if exist "%CONFIG_TEMPLATE%" ( - echo Restoring config.json from template: %CONFIG_TEMPLATE% - copy /y "%CONFIG_TEMPLATE%" "%CONFIG_JSON%" >nul -) else ( - if exist "%CONFIG_JSON%" ( - echo WARNING: config.template.json not found. Resetting admin password placeholder in existing config.json. - powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "$path = '%CONFIG_JSON%';" ^ - "$json = Get-Content -LiteralPath $path -Raw;" ^ - "$pattern = '(\"id\"\s*:\s*\"admin\"[\s\S]*?\"password\"\s*:\s*)\"[^\"]*\"';" ^ - "$updated = [regex]::Replace($json, $pattern, '$1\"__INITIAL_ADMIN_PASSWORD__\"', 1);" ^ - "Set-Content -LiteralPath $path -Value $updated -NoNewline" - ) else ( - echo OSCAR config not found: %CONFIG_JSON% - ) -) - -echo Restoring initial admin secret file: %SECRET_FILE% -> "%SECRET_FILE%" echo oscar - -del "%SCRIPT_DIR%\.monitor-active-dir" >nul 2>nul - -echo. -echo Reset complete. -echo Next launch should initialize the default login as admin / oscar. -exit /b 0 - -:load_env -for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( - set "ENV_NAME=%%A" - set "ENV_VALUE=%%B" - call :set_env_var -) -exit /b 0 - -:set_env_var -if not defined ENV_NAME exit /b 0 -if "%ENV_NAME:~0,1%"=="#" exit /b 0 -if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" -set "%ENV_NAME%=%ENV_VALUE%" +@echo off +setlocal EnableExtensions EnableDelayedExpansion + +set "SCRIPT_DIR=%~dp0" +if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" + +set "ENV_FILE=%SCRIPT_DIR%\.env" +if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" + +if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" + +set "PGDATA_DIR=%SCRIPT_DIR%\pgdata" +set "NODE_DIR=%SCRIPT_DIR%\osh-node-oscar" +set "DB_DIR=%NODE_DIR%\db" +set "FILES_DIR=%NODE_DIR%\files" +set "CONFIG_JSON=%NODE_DIR%\config.json" +set "CONFIG_TEMPLATE=%NODE_DIR%\config.template.json" +set "SECRET_FILE=%NODE_DIR%\.s" + +echo Requesting monitor shutdown... +if exist "%SCRIPT_DIR%\monitor-oscar.bat" ( + call "%SCRIPT_DIR%\monitor-oscar.bat" stop >nul 2>nul +) + +echo Stopping OSCAR Java processes... +for /f "usebackq delims=" %%P in (` + powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { $proc.ProcessId } }" 2^>nul +`) do ( + powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %%P -Force -ErrorAction Stop } catch {}" >nul 2>nul +) + +echo Removing container: %CONTAINER_NAME%... +docker rm -f -v "%CONTAINER_NAME%" >nul 2>nul + +if exist "%PGDATA_DIR%" ( + echo Removing Postgres data directory: %PGDATA_DIR% + rmdir /s /q "%PGDATA_DIR%" +) else ( + echo Postgres data directory not found: %PGDATA_DIR% +) + +if exist "%DB_DIR%" ( + echo Removing OSCAR runtime DB directory: %DB_DIR% + rmdir /s /q "%DB_DIR%" +) else ( + echo OSCAR runtime DB directory not found: %DB_DIR% +) + +if exist "%FILES_DIR%" ( + echo Removing OSCAR files directory: %FILES_DIR% + rmdir /s /q "%FILES_DIR%" +) else ( + echo OSCAR files directory not found: %FILES_DIR% +) + +if exist "%CONFIG_TEMPLATE%" ( + echo Restoring config.json from template: %CONFIG_TEMPLATE% + copy /y "%CONFIG_TEMPLATE%" "%CONFIG_JSON%" >nul +) else ( + if exist "%CONFIG_JSON%" ( + echo WARNING: config.template.json not found. Resetting admin password placeholder in existing config.json. + powershell -NoProfile -ExecutionPolicy Bypass -Command ^ + "$path = '%CONFIG_JSON%';" ^ + "$json = Get-Content -LiteralPath $path -Raw;" ^ + "$pattern = '(\"id\"\s*:\s*\"admin\"[\s\S]*?\"password\"\s*:\s*)\"[^\"]*\"';" ^ + "$updated = [regex]::Replace($json, $pattern, '$1\"__INITIAL_ADMIN_PASSWORD__\"', 1);" ^ + "Set-Content -LiteralPath $path -Value $updated -NoNewline" + ) else ( + echo OSCAR config not found: %CONFIG_JSON% + ) +) + +echo Restoring initial admin secret file: %SECRET_FILE% +> "%SECRET_FILE%" echo oscar + +del "%SCRIPT_DIR%\.monitor-active-dir" >nul 2>nul + +echo. +echo Reset complete. +echo Next launch should initialize the default login as admin / oscar. +exit /b 0 + +:load_env +for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( + set "ENV_NAME=%%A" + set "ENV_VALUE=%%B" + call :set_env_var +) +exit /b 0 + +:set_env_var +if not defined ENV_NAME exit /b 0 +if "%ENV_NAME:~0,1%"=="#" exit /b 0 +if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" +set "%ENV_NAME%=%ENV_VALUE%" exit /b 0 \ No newline at end of file diff --git a/dist/release/stop-all.bat b/dist/release/stop-all.bat index 3876f16..78d7386 100644 --- a/dist/release/stop-all.bat +++ b/dist/release/stop-all.bat @@ -1,45 +1,45 @@ -@echo off -setlocal EnableExtensions - -set "SCRIPT_DIR=%~dp0" -if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" -set "CONTAINER_NAME=oscar-postgis-container" -set "SENSORHUB_NAME=com.botts.impl.security.SensorHubWrapper" - -if exist "%SCRIPT_DIR%\.env" ( - for /f "usebackq tokens=* delims=" %%L in ("%SCRIPT_DIR%\.env") do ( - set "LINE=%%L" - call :parse_env_line - ) -) - -echo Requesting monitor stop if active... -if exist "%SCRIPT_DIR%\monitor-oscar.bat" ( - call "%SCRIPT_DIR%\monitor-oscar.bat" stop >nul 2>nul - timeout /t 5 /nobreak >nul -) - -echo. -echo Stopping container: %CONTAINER_NAME%... -docker stop %CONTAINER_NAME% >nul 2>nul -if errorlevel 1 ( - echo Container not found or already stopped. -) else ( - echo Container stop requested. -) - -echo. -echo Stopping SensorHubWrapper Java process... -powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-CimInstance Win32_Process | Where-Object { $_.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $_.CommandLine -and $_.CommandLine -like '*%SENSORHUB_NAME%*' } | ForEach-Object { try { Stop-Process -Id $_.ProcessId -Force -ErrorAction Stop; Write-Output ('Stopped PID ' + $_.ProcessId) } catch {} }" - -echo. -echo Done. -exit /b 0 - -:parse_env_line -if not defined LINE exit /b 0 -if "%LINE:~0,1%"=="#" exit /b 0 -for /f "tokens=1,* delims==" %%A in ("%LINE%") do ( - if /I "%%A"=="CONTAINER_NAME" set "CONTAINER_NAME=%%B" -) -exit /b 0 +@echo off +setlocal EnableExtensions + +set "SCRIPT_DIR=%~dp0" +if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" +set "CONTAINER_NAME=oscar-postgis-container" +set "SENSORHUB_NAME=com.botts.impl.security.SensorHubWrapper" + +if exist "%SCRIPT_DIR%\.env" ( + for /f "usebackq tokens=* delims=" %%L in ("%SCRIPT_DIR%\.env") do ( + set "LINE=%%L" + call :parse_env_line + ) +) + +echo Requesting monitor stop if active... +if exist "%SCRIPT_DIR%\monitor-oscar.bat" ( + call "%SCRIPT_DIR%\monitor-oscar.bat" stop >nul 2>nul + timeout /t 5 /nobreak >nul +) + +echo. +echo Stopping container: %CONTAINER_NAME%... +docker stop %CONTAINER_NAME% >nul 2>nul +if errorlevel 1 ( + echo Container not found or already stopped. +) else ( + echo Container stop requested. +) + +echo. +echo Stopping SensorHubWrapper Java process... +powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-CimInstance Win32_Process | Where-Object { $_.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $_.CommandLine -and $_.CommandLine -like '*%SENSORHUB_NAME%*' } | ForEach-Object { try { Stop-Process -Id $_.ProcessId -Force -ErrorAction Stop; Write-Output ('Stopped PID ' + $_.ProcessId) } catch {} }" + +echo. +echo Done. +exit /b 0 + +:parse_env_line +if not defined LINE exit /b 0 +if "%LINE:~0,1%"=="#" exit /b 0 +for /f "tokens=1,* delims==" %%A in ("%LINE%") do ( + if /I "%%A"=="CONTAINER_NAME" set "CONTAINER_NAME=%%B" +) +exit /b 0 diff --git a/dist/scripts/standard/launch.bat b/dist/scripts/standard/launch.bat index 9b1c240..689f64e 100755 --- a/dist/scripts/standard/launch.bat +++ b/dist/scripts/standard/launch.bat @@ -1,195 +1,195 @@ -@echo off -setlocal EnableExtensions EnableDelayedExpansion - -set "SCRIPT_DIR=%~dp0" -if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" - -set "ENV_FILE=" -if exist "%SCRIPT_DIR%\.env" ( - set "ENV_FILE=%SCRIPT_DIR%\.env" -) else if exist "%SCRIPT_DIR%\..\.env" ( - set "ENV_FILE=%SCRIPT_DIR%\..\.env" -) - -if defined ENV_FILE call :load_env "%ENV_FILE%" - -if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" -if not defined FORCE_RESTART set "FORCE_RESTART=0" - -where java >nul 2>nul -if errorlevel 1 ( - echo ERROR: Java was not found in PATH. - exit /b 1 -) - -where keytool >nul 2>nul -if errorlevel 1 ( - echo ERROR: keytool was not found in PATH. - exit /b 1 -) - -if not exist "%SCRIPT_DIR%\lib" ( - echo ERROR: Missing library directory: "%SCRIPT_DIR%\lib" - exit /b 1 -) - -if not exist "%SCRIPT_DIR%\config.json" ( - echo ERROR: Missing config file: "%SCRIPT_DIR%\config.json" - exit /b 1 -) - -if not exist "%SCRIPT_DIR%\load_trusted_certs.bat" ( - echo ERROR: Missing trusted-certs helper: "%SCRIPT_DIR%\load_trusted_certs.bat" - exit /b 1 -) - -if not exist "%SCRIPT_DIR%\set-initial-admin-password.bat" ( - echo ERROR: Missing admin-password helper: "%SCRIPT_DIR%\set-initial-admin-password.bat" - exit /b 1 -) - -call :check_existing_oscar -if defined OSCAR_PID ( - if /I "%FORCE_RESTART%"=="1" ( - echo OSCAR is already running with PID !OSCAR_PID!. FORCE_RESTART=1, stopping it first... - call :stop_existing_oscar - call :wait_for_oscar_stop 60 - call :check_existing_oscar - if defined OSCAR_PID ( - echo ERROR: OSCAR is still running with PID !OSCAR_PID! after stop attempt. - exit /b 1 - ) - ) else ( - echo OSCAR is already running with PID !OSCAR_PID!. - echo Run stop-all.bat first, or set FORCE_RESTART=1 to replace the existing OSCAR process. - exit /b 1 - ) -) - -if /I "%SYSTEM_PROFILE%"=="RPI4" ( - set "JAVA_XMS=512m" - set "JAVA_XMX=1536m" - set "JAVACPP_MAX_BYTES_DEFAULT=512m" - set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=2g" -) else if /I "%SYSTEM_PROFILE%"=="8GB" ( - set "JAVA_XMS=1g" - set "JAVA_XMX=2g" - set "JAVACPP_MAX_BYTES_DEFAULT=1g" - set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=4g" -) else if /I "%SYSTEM_PROFILE%"=="16GB" ( - set "JAVA_XMS=1g" - set "JAVA_XMX=3g" - set "JAVACPP_MAX_BYTES_DEFAULT=2g" - set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=8g" -) else if /I "%SYSTEM_PROFILE%"=="32GB" ( - set "JAVA_XMS=2g" - set "JAVA_XMX=6g" - set "JAVACPP_MAX_BYTES_DEFAULT=4g" - set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=16g" -) else ( - echo WARNING: Unknown SYSTEM_PROFILE "%SYSTEM_PROFILE%". Using 8GB defaults. - set "JAVA_XMS=1g" - set "JAVA_XMX=2g" - set "JAVACPP_MAX_BYTES_DEFAULT=1g" - set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=4g" -) - -if not defined JAVACPP_MAX_BYTES set "JAVACPP_MAX_BYTES=%JAVACPP_MAX_BYTES_DEFAULT%" -if not defined JAVACPP_MAX_PHYSICAL_BYTES set "JAVACPP_MAX_PHYSICAL_BYTES=%JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT%" -if not defined JFR_FILENAME set "JFR_FILENAME=%SCRIPT_DIR%\oscar.jfr" - -echo Starting OSH Node with Profile: %SYSTEM_PROFILE% -echo Heap: %JAVA_XMS% / %JAVA_XMX% -echo JavaCPP maxBytes: %JAVACPP_MAX_BYTES% -echo JavaCPP maxPhysicalBytes: %JAVACPP_MAX_PHYSICAL_BYTES% -echo JFR file: %JFR_FILENAME% - -call "%SCRIPT_DIR%\load_trusted_certs.bat" -if errorlevel 1 exit /b %ERRORLEVEL% - -set "KEYSTORE=%SCRIPT_DIR%\osh-keystore.p12" -set "KEYSTORE_TYPE=PKCS12" -if not defined KEYSTORE_PASSWORD set "KEYSTORE_PASSWORD=atakatak" - -set "TRUSTSTORE=%SCRIPT_DIR%\truststore.jks" -set "TRUSTSTORE_TYPE=JKS" -if not defined TRUSTSTORE_PASSWORD set "TRUSTSTORE_PASSWORD=changeit" - -set "INITIAL_ADMIN_PASSWORD_FILE=%SCRIPT_DIR%\.s" -if not exist "%INITIAL_ADMIN_PASSWORD_FILE%" if not defined INITIAL_ADMIN_PASSWORD set "INITIAL_ADMIN_PASSWORD=admin" - -call "%SCRIPT_DIR%\set-initial-admin-password.bat" -if errorlevel 1 exit /b %ERRORLEVEL% - -set "JAVA_LIBRARY_OPT=" -if exist "%SCRIPT_DIR%\nativelibs" ( - set "JAVA_LIBRARY_OPT=-Djava.library.path=%SCRIPT_DIR%\nativelibs" -) else ( - echo WARNING: Optional native library directory not found: "%SCRIPT_DIR%\nativelibs" -) - -java ^ - -Xms%JAVA_XMS% ^ - -Xmx%JAVA_XMX% ^ - -Xss256k ^ - -XX:ReservedCodeCacheSize=256m ^ - -XX:+UseG1GC ^ - -XX:+HeapDumpOnOutOfMemoryError ^ - -XX:+UnlockDiagnosticVMOptions ^ - -XX:NativeMemoryTracking=summary ^ - "-Dorg.bytedeco.javacpp.maxBytes=%JAVACPP_MAX_BYTES%" ^ - "-Dorg.bytedeco.javacpp.maxPhysicalBytes=%JAVACPP_MAX_PHYSICAL_BYTES%" ^ - -Dorg.bytedeco.javacpp.maxRetries=2 ^ - -Dorg.bytedeco.javacpp.mxbean=true ^ - "-Dlogback.configurationFile=%SCRIPT_DIR%\logback.xml" ^ - -cp "%SCRIPT_DIR%\lib\*" ^ - "-Djava.system.class.loader=org.sensorhub.utils.NativeClassLoader" ^ - "-Djavax.net.ssl.keyStore=%KEYSTORE%" ^ - "-Djavax.net.ssl.keyStorePassword=%KEYSTORE_PASSWORD%" ^ - "-Djavax.net.ssl.trustStore=%TRUSTSTORE%" ^ - "-Djavax.net.ssl.trustStorePassword=%TRUSTSTORE_PASSWORD%" ^ - !JAVA_LIBRARY_OPT! ^ - com.botts.impl.security.SensorHubWrapper "%SCRIPT_DIR%\config.json" "%SCRIPT_DIR%\db" - -set "JAVA_EXIT_CODE=%ERRORLEVEL%" -endlocal & exit /b %JAVA_EXIT_CODE% - -:check_existing_oscar -set "OSCAR_PID=" -for /f "usebackq delims=" %%P in (` - powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { [Console]::Write($proc.ProcessId); break } }" 2^>nul -`) do set "OSCAR_PID=%%P" -exit /b 0 - -:stop_existing_oscar -if not defined OSCAR_PID exit /b 0 -powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %OSCAR_PID% -Force -ErrorAction Stop } catch {}" >nul 2>nul -exit /b 0 - -:wait_for_oscar_stop -set "WAIT_LIMIT=%~1" -if not defined WAIT_LIMIT set "WAIT_LIMIT=60" -set /a WAITED=0 - -:wait_for_oscar_stop_loop -call :check_existing_oscar -if not defined OSCAR_PID exit /b 0 -if !WAITED! GEQ %WAIT_LIMIT% exit /b 0 -timeout /t 1 /nobreak >nul -set /a WAITED+=1 -goto wait_for_oscar_stop_loop - -:load_env -for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( - set "ENV_NAME=%%A" - set "ENV_VALUE=%%B" - call :set_env_var -) -exit /b 0 - -:set_env_var -if not defined ENV_NAME exit /b 0 -if "%ENV_NAME:~0,1%"=="#" exit /b 0 -if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" -set "%ENV_NAME%=%ENV_VALUE%" +@echo off +setlocal EnableExtensions EnableDelayedExpansion + +set "SCRIPT_DIR=%~dp0" +if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" + +set "ENV_FILE=" +if exist "%SCRIPT_DIR%\.env" ( + set "ENV_FILE=%SCRIPT_DIR%\.env" +) else if exist "%SCRIPT_DIR%\..\.env" ( + set "ENV_FILE=%SCRIPT_DIR%\..\.env" +) + +if defined ENV_FILE call :load_env "%ENV_FILE%" + +if not defined SYSTEM_PROFILE set "SYSTEM_PROFILE=8GB" +if not defined FORCE_RESTART set "FORCE_RESTART=0" + +where java >nul 2>nul +if errorlevel 1 ( + echo ERROR: Java was not found in PATH. + exit /b 1 +) + +where keytool >nul 2>nul +if errorlevel 1 ( + echo ERROR: keytool was not found in PATH. + exit /b 1 +) + +if not exist "%SCRIPT_DIR%\lib" ( + echo ERROR: Missing library directory: "%SCRIPT_DIR%\lib" + exit /b 1 +) + +if not exist "%SCRIPT_DIR%\config.json" ( + echo ERROR: Missing config file: "%SCRIPT_DIR%\config.json" + exit /b 1 +) + +if not exist "%SCRIPT_DIR%\load_trusted_certs.bat" ( + echo ERROR: Missing trusted-certs helper: "%SCRIPT_DIR%\load_trusted_certs.bat" + exit /b 1 +) + +if not exist "%SCRIPT_DIR%\set-initial-admin-password.bat" ( + echo ERROR: Missing admin-password helper: "%SCRIPT_DIR%\set-initial-admin-password.bat" + exit /b 1 +) + +call :check_existing_oscar +if defined OSCAR_PID ( + if /I "%FORCE_RESTART%"=="1" ( + echo OSCAR is already running with PID !OSCAR_PID!. FORCE_RESTART=1, stopping it first... + call :stop_existing_oscar + call :wait_for_oscar_stop 60 + call :check_existing_oscar + if defined OSCAR_PID ( + echo ERROR: OSCAR is still running with PID !OSCAR_PID! after stop attempt. + exit /b 1 + ) + ) else ( + echo OSCAR is already running with PID !OSCAR_PID!. + echo Run stop-all.bat first, or set FORCE_RESTART=1 to replace the existing OSCAR process. + exit /b 1 + ) +) + +if /I "%SYSTEM_PROFILE%"=="RPI4" ( + set "JAVA_XMS=512m" + set "JAVA_XMX=1536m" + set "JAVACPP_MAX_BYTES_DEFAULT=512m" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=2g" +) else if /I "%SYSTEM_PROFILE%"=="8GB" ( + set "JAVA_XMS=1g" + set "JAVA_XMX=2g" + set "JAVACPP_MAX_BYTES_DEFAULT=1g" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=4g" +) else if /I "%SYSTEM_PROFILE%"=="16GB" ( + set "JAVA_XMS=1g" + set "JAVA_XMX=3g" + set "JAVACPP_MAX_BYTES_DEFAULT=2g" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=8g" +) else if /I "%SYSTEM_PROFILE%"=="32GB" ( + set "JAVA_XMS=2g" + set "JAVA_XMX=6g" + set "JAVACPP_MAX_BYTES_DEFAULT=4g" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=16g" +) else ( + echo WARNING: Unknown SYSTEM_PROFILE "%SYSTEM_PROFILE%". Using 8GB defaults. + set "JAVA_XMS=1g" + set "JAVA_XMX=2g" + set "JAVACPP_MAX_BYTES_DEFAULT=1g" + set "JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT=4g" +) + +if not defined JAVACPP_MAX_BYTES set "JAVACPP_MAX_BYTES=%JAVACPP_MAX_BYTES_DEFAULT%" +if not defined JAVACPP_MAX_PHYSICAL_BYTES set "JAVACPP_MAX_PHYSICAL_BYTES=%JAVACPP_MAX_PHYSICAL_BYTES_DEFAULT%" +if not defined JFR_FILENAME set "JFR_FILENAME=%SCRIPT_DIR%\oscar.jfr" + +echo Starting OSH Node with Profile: %SYSTEM_PROFILE% +echo Heap: %JAVA_XMS% / %JAVA_XMX% +echo JavaCPP maxBytes: %JAVACPP_MAX_BYTES% +echo JavaCPP maxPhysicalBytes: %JAVACPP_MAX_PHYSICAL_BYTES% +echo JFR file: %JFR_FILENAME% + +call "%SCRIPT_DIR%\load_trusted_certs.bat" +if errorlevel 1 exit /b %ERRORLEVEL% + +set "KEYSTORE=%SCRIPT_DIR%\osh-keystore.p12" +set "KEYSTORE_TYPE=PKCS12" +if not defined KEYSTORE_PASSWORD set "KEYSTORE_PASSWORD=atakatak" + +set "TRUSTSTORE=%SCRIPT_DIR%\truststore.jks" +set "TRUSTSTORE_TYPE=JKS" +if not defined TRUSTSTORE_PASSWORD set "TRUSTSTORE_PASSWORD=changeit" + +set "INITIAL_ADMIN_PASSWORD_FILE=%SCRIPT_DIR%\.s" +if not exist "%INITIAL_ADMIN_PASSWORD_FILE%" if not defined INITIAL_ADMIN_PASSWORD set "INITIAL_ADMIN_PASSWORD=admin" + +call "%SCRIPT_DIR%\set-initial-admin-password.bat" +if errorlevel 1 exit /b %ERRORLEVEL% + +set "JAVA_LIBRARY_OPT=" +if exist "%SCRIPT_DIR%\nativelibs" ( + set "JAVA_LIBRARY_OPT=-Djava.library.path=%SCRIPT_DIR%\nativelibs" +) else ( + echo WARNING: Optional native library directory not found: "%SCRIPT_DIR%\nativelibs" +) + +java ^ + -Xms%JAVA_XMS% ^ + -Xmx%JAVA_XMX% ^ + -Xss256k ^ + -XX:ReservedCodeCacheSize=256m ^ + -XX:+UseG1GC ^ + -XX:+HeapDumpOnOutOfMemoryError ^ + -XX:+UnlockDiagnosticVMOptions ^ + -XX:NativeMemoryTracking=summary ^ + "-Dorg.bytedeco.javacpp.maxBytes=%JAVACPP_MAX_BYTES%" ^ + "-Dorg.bytedeco.javacpp.maxPhysicalBytes=%JAVACPP_MAX_PHYSICAL_BYTES%" ^ + -Dorg.bytedeco.javacpp.maxRetries=2 ^ + -Dorg.bytedeco.javacpp.mxbean=true ^ + "-Dlogback.configurationFile=%SCRIPT_DIR%\logback.xml" ^ + -cp "%SCRIPT_DIR%\lib\*" ^ + "-Djava.system.class.loader=org.sensorhub.utils.NativeClassLoader" ^ + "-Djavax.net.ssl.keyStore=%KEYSTORE%" ^ + "-Djavax.net.ssl.keyStorePassword=%KEYSTORE_PASSWORD%" ^ + "-Djavax.net.ssl.trustStore=%TRUSTSTORE%" ^ + "-Djavax.net.ssl.trustStorePassword=%TRUSTSTORE_PASSWORD%" ^ + !JAVA_LIBRARY_OPT! ^ + com.botts.impl.security.SensorHubWrapper "%SCRIPT_DIR%\config.json" "%SCRIPT_DIR%\db" + +set "JAVA_EXIT_CODE=%ERRORLEVEL%" +endlocal & exit /b %JAVA_EXIT_CODE% + +:check_existing_oscar +set "OSCAR_PID=" +for /f "usebackq delims=" %%P in (` + powershell -NoProfile -ExecutionPolicy Bypass -Command "$procs = Get-CimInstance Win32_Process; foreach ($proc in $procs) { if ($proc.Name -match '^(java|javaw)(\.exe)?$' -and $null -ne $proc.CommandLine -and $proc.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*') { [Console]::Write($proc.ProcessId); break } }" 2^>nul +`) do set "OSCAR_PID=%%P" +exit /b 0 + +:stop_existing_oscar +if not defined OSCAR_PID exit /b 0 +powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %OSCAR_PID% -Force -ErrorAction Stop } catch {}" >nul 2>nul +exit /b 0 + +:wait_for_oscar_stop +set "WAIT_LIMIT=%~1" +if not defined WAIT_LIMIT set "WAIT_LIMIT=60" +set /a WAITED=0 + +:wait_for_oscar_stop_loop +call :check_existing_oscar +if not defined OSCAR_PID exit /b 0 +if !WAITED! GEQ %WAIT_LIMIT% exit /b 0 +timeout /t 1 /nobreak >nul +set /a WAITED+=1 +goto wait_for_oscar_stop_loop + +:load_env +for /f "usebackq tokens=1,* delims==" %%A in ("%~1") do ( + set "ENV_NAME=%%A" + set "ENV_VALUE=%%B" + call :set_env_var +) +exit /b 0 + +:set_env_var +if not defined ENV_NAME exit /b 0 +if "%ENV_NAME:~0,1%"=="#" exit /b 0 +if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" +set "%ENV_NAME%=%ENV_VALUE%" exit /b 0 \ No newline at end of file diff --git a/dist/scripts/standard/load_trusted_certs.bat b/dist/scripts/standard/load_trusted_certs.bat index 65a1541..247bc3c 100755 --- a/dist/scripts/standard/load_trusted_certs.bat +++ b/dist/scripts/standard/load_trusted_certs.bat @@ -1,99 +1,99 @@ -@echo off -setlocal EnableExtensions EnableDelayedExpansion - -echo Building Java trust store... - -set "STOREPASS=changeit" -set "SCRIPTDIR=%~dp0" -set "NEWTRUSTSTORE=%SCRIPTDIR%truststore.jks" -set "CERTDIR=%SCRIPTDIR%trusted_certificates" -set "CACERTS=" -set "JAVA_HOME_DETECTED=" - -rem First try JAVA_HOME if already set -if defined JAVA_HOME ( - if exist "%JAVA_HOME%\conf\security\cacerts" set "CACERTS=%JAVA_HOME%\conf\security\cacerts" - if not defined CACERTS if exist "%JAVA_HOME%\lib\security\cacerts" set "CACERTS=%JAVA_HOME%\lib\security\cacerts" -) - -rem If that did not work, ask Java itself for java.home -if not defined CACERTS ( - for /f "tokens=1,* delims==" %%A in ('java -XshowSettings:properties -version 2^>^&1 ^| findstr /c:"java.home ="') do ( - set "JAVA_HOME_DETECTED=%%B" - ) -) - -rem Trim leading spaces -if defined JAVA_HOME_DETECTED ( - for /f "tokens=* delims= " %%H in ("!JAVA_HOME_DETECTED!") do set "JAVA_HOME_DETECTED=%%H" -) - -rem Try common cacerts locations under detected java.home -if not defined CACERTS if defined JAVA_HOME_DETECTED ( - if exist "!JAVA_HOME_DETECTED!\conf\security\cacerts" set "CACERTS=!JAVA_HOME_DETECTED!\conf\security\cacerts" - if not defined CACERTS if exist "!JAVA_HOME_DETECTED!\lib\security\cacerts" set "CACERTS=!JAVA_HOME_DETECTED!\lib\security\cacerts" -) - -if not defined CACERTS ( - echo Error: could not locate Java cacerts. - if defined JAVA_HOME echo JAVA_HOME="%JAVA_HOME%" - if defined JAVA_HOME_DETECTED echo java.home="!JAVA_HOME_DETECTED!" - endlocal & exit /b 1 -) - -if not exist "%CACERTS%" ( - echo Error: Java cacerts path does not exist: "%CACERTS%" - endlocal & exit /b 1 -) - -echo Using Java cacerts: "%CACERTS%" - -copy /y "%CACERTS%" "%NEWTRUSTSTORE%" >nul -if errorlevel 1 ( - echo Error: failed to create "%NEWTRUSTSTORE%" - endlocal & exit /b 1 -) - -if not exist "%CERTDIR%" ( - echo Trusted certificates directory not found: "%CERTDIR%" - echo Using copied default trust store only. - echo Done. - endlocal & exit /b 0 -) - -set "FOUND_CERT=0" -for %%c in ("%CERTDIR%\*.cer" "%CERTDIR%\*.pem" "%CERTDIR%\*.crt") do ( - if exist "%%~fc" ( - set "FOUND_CERT=1" - call :check_certificate "%%~fc" - if errorlevel 1 ( - endlocal & exit /b 1 - ) - ) -) - -if "%FOUND_CERT%"=="0" ( - echo No certificate files found in "%CERTDIR%". -) - -echo Done. -endlocal & exit /b 0 - -:check_certificate -setlocal -set "CERTFILE=%~1" -set "ALIAS=%~n1" - -keytool -list -keystore "%NEWTRUSTSTORE%" -storepass "%STOREPASS%" -alias "%ALIAS%" >nul 2>nul -if not "%ERRORLEVEL%"=="0" ( - echo Importing "%ALIAS%" from "%CERTFILE%" - keytool -importcert -keystore "%NEWTRUSTSTORE%" -noprompt -storepass "%STOREPASS%" -alias "%ALIAS%" -file "%CERTFILE%" >nul - if errorlevel 1 ( - echo Error: failed to import "%ALIAS%" from "%CERTFILE%" - endlocal & exit /b 1 - ) -) else ( - echo Certificate with alias "%ALIAS%" already exists. Skipping. -) - +@echo off +setlocal EnableExtensions EnableDelayedExpansion + +echo Building Java trust store... + +set "STOREPASS=changeit" +set "SCRIPTDIR=%~dp0" +set "NEWTRUSTSTORE=%SCRIPTDIR%truststore.jks" +set "CERTDIR=%SCRIPTDIR%trusted_certificates" +set "CACERTS=" +set "JAVA_HOME_DETECTED=" + +rem First try JAVA_HOME if already set +if defined JAVA_HOME ( + if exist "%JAVA_HOME%\conf\security\cacerts" set "CACERTS=%JAVA_HOME%\conf\security\cacerts" + if not defined CACERTS if exist "%JAVA_HOME%\lib\security\cacerts" set "CACERTS=%JAVA_HOME%\lib\security\cacerts" +) + +rem If that did not work, ask Java itself for java.home +if not defined CACERTS ( + for /f "tokens=1,* delims==" %%A in ('java -XshowSettings:properties -version 2^>^&1 ^| findstr /c:"java.home ="') do ( + set "JAVA_HOME_DETECTED=%%B" + ) +) + +rem Trim leading spaces +if defined JAVA_HOME_DETECTED ( + for /f "tokens=* delims= " %%H in ("!JAVA_HOME_DETECTED!") do set "JAVA_HOME_DETECTED=%%H" +) + +rem Try common cacerts locations under detected java.home +if not defined CACERTS if defined JAVA_HOME_DETECTED ( + if exist "!JAVA_HOME_DETECTED!\conf\security\cacerts" set "CACERTS=!JAVA_HOME_DETECTED!\conf\security\cacerts" + if not defined CACERTS if exist "!JAVA_HOME_DETECTED!\lib\security\cacerts" set "CACERTS=!JAVA_HOME_DETECTED!\lib\security\cacerts" +) + +if not defined CACERTS ( + echo Error: could not locate Java cacerts. + if defined JAVA_HOME echo JAVA_HOME="%JAVA_HOME%" + if defined JAVA_HOME_DETECTED echo java.home="!JAVA_HOME_DETECTED!" + endlocal & exit /b 1 +) + +if not exist "%CACERTS%" ( + echo Error: Java cacerts path does not exist: "%CACERTS%" + endlocal & exit /b 1 +) + +echo Using Java cacerts: "%CACERTS%" + +copy /y "%CACERTS%" "%NEWTRUSTSTORE%" >nul +if errorlevel 1 ( + echo Error: failed to create "%NEWTRUSTSTORE%" + endlocal & exit /b 1 +) + +if not exist "%CERTDIR%" ( + echo Trusted certificates directory not found: "%CERTDIR%" + echo Using copied default trust store only. + echo Done. + endlocal & exit /b 0 +) + +set "FOUND_CERT=0" +for %%c in ("%CERTDIR%\*.cer" "%CERTDIR%\*.pem" "%CERTDIR%\*.crt") do ( + if exist "%%~fc" ( + set "FOUND_CERT=1" + call :check_certificate "%%~fc" + if errorlevel 1 ( + endlocal & exit /b 1 + ) + ) +) + +if "%FOUND_CERT%"=="0" ( + echo No certificate files found in "%CERTDIR%". +) + +echo Done. +endlocal & exit /b 0 + +:check_certificate +setlocal +set "CERTFILE=%~1" +set "ALIAS=%~n1" + +keytool -list -keystore "%NEWTRUSTSTORE%" -storepass "%STOREPASS%" -alias "%ALIAS%" >nul 2>nul +if not "%ERRORLEVEL%"=="0" ( + echo Importing "%ALIAS%" from "%CERTFILE%" + keytool -importcert -keystore "%NEWTRUSTSTORE%" -noprompt -storepass "%STOREPASS%" -alias "%ALIAS%" -file "%CERTFILE%" >nul + if errorlevel 1 ( + echo Error: failed to import "%ALIAS%" from "%CERTFILE%" + endlocal & exit /b 1 + ) +) else ( + echo Certificate with alias "%ALIAS%" already exists. Skipping. +) + endlocal & exit /b 0 \ No newline at end of file From 5821a353ffe88c5df447c3ba795a2fb2ee731c47 Mon Sep 17 00:00:00 2001 From: tyronechrisharris <142608718+tyronechrisharris@users.noreply.github.com> Date: Fri, 8 May 2026 01:02:19 -0400 Subject: [PATCH 7/9] This bundle updates the OSCAR 3.5.1 documentation to use a two-path launch model: - `launch-all` is documented as the preferred efficient production startup path after validation. - `monitor-oscar` is documented as the preferred path for testing, burn-in, side-by-side evaluation, troubleshooting, and system profiling. - The monitoring path is described as intentionally verbose because it creates monitor directories, recurring snapshots, JFR checks, thread dumps, database trend files, and additional diagnostic logs. - Operators are instructed to switch routine production starts back to `launch-all` when an in-depth system profile is not needed. Updated files: - `README.md` - `OSCAR_launch_monitoring_guide.md` - `Release_Notes_3.5.1.md` - `MediaMTX_OSCAR_camera_proxy_guide.md` - `OSCAR_System_Documentation_Manual_3.5.md` - `Node_Administration_3.5.1_addendum.md` --- README.md | 52 +++++++------ .../MediaMTX_OSCAR_camera_proxy_guide.md | 33 +++++--- .../Node_Administration_3.5.1_addendum.md | 11 +-- .../OSCAR_System_Documentation_Manual_3.5.md | 14 +++- .../OSCAR_launch_monitoring_guide.md | 75 ++++++++++++------- dist/documentation/Release_Notes_3.5.1.md | 64 +++++++++------- include/osh-oakridge-modules | 2 +- 7 files changed, 155 insertions(+), 96 deletions(-) diff --git a/README.md b/README.md index 107e5fb..875db07 100644 --- a/README.md +++ b/README.md @@ -108,9 +108,29 @@ Useful optional settings include: - `RETRY_INTERVAL=2` - `POSTGIS_READY_DELAY=5` -### 5. Preferred first start: use the monitoring script +### 5. Production start: use `launch-all` -For testing, burn-in, and side-by-side field deployment, start OSCAR with the monitoring wrapper instead of launching the node directly. +For efficient production operation after validation is complete, use the top-level launch script instead of the monitoring wrapper. + +Windows: + +```bat +launch-all.bat +``` + +Linux: + +```bash +./launch-all.sh +``` + +`launch-all` is the preferred production path because it starts PostGIS and OSCAR with the selected `.env` profile without the extra monitor loop, recurring snapshots, JFR checks, thread dumps, database trend files, and monitor-directory logging. This keeps routine startup simpler and avoids collecting detailed profile data when operators do not need an in-depth system profile. + +Prefer these **top-level launchers** over calling `osh-node-oscar/launch.(sh|bat)` directly unless you are debugging the node itself. + +### 6. Validation, troubleshooting, and profiling start: use `monitor-oscar` + +For testing, burn-in, side-by-side field evaluation, troubleshooting, and system profiling, start OSCAR with the monitoring wrapper. Windows: @@ -137,29 +157,14 @@ Linux sessionless launch: nohup ./monitor-oscar.sh > monitor.out 2>&1 & ``` -This is the recommended first-run path because it: +This is the preferred validation and troubleshooting path because it: - starts PostGIS and OSCAR using the current launch scripts - captures memory, thread, JFR, and database snapshots over time - produces a monitor directory and status report inputs automatically +- gives operators the evidence needed to compare profiles, diagnose startup failures, and confirm that PostgreSQL sessions and JVM threads stabilize -### 6. Routine start without monitoring - -When monitoring is not needed, use the top-level launch script: - -Windows: - -```bat -launch-all.bat -``` - -Linux: - -```bash -./launch-all.sh -``` - -Prefer these **sessionless top-level launchers** over calling `osh-node-oscar/launch.(sh|bat)` directly unless you are debugging the node itself. +Once the system is validated and no in-depth profile is needed, switch routine production starts back to `launch-all`. ### 7. Running-instance handling @@ -251,12 +256,13 @@ If you are testing from a source checkout instead of a packaged release: 1. create `.env` from `env.template` 2. verify Java 21 and Docker -3. launch with `monitor-oscar` for first-run validation -4. use `check-oscar-status` after the system reaches steady state +3. launch with `monitor-oscar` for first-run validation, troubleshooting, or profiling +4. use `check-oscar-status` after the monitored system reaches steady state +5. use `launch-all` for efficient routine production starts after validation ## MediaMTX for larger camera deployments -For test systems and larger multi-lane deployments, consider placing **MediaMTX** in front of camera streams so OSCAR connects to stable local RTSP proxy paths instead of directly to every camera. See the updated MediaMTX guide in `dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md`. +For test systems and larger multi-lane deployments, consider placing **MediaMTX** in front of camera streams so OSCAR connects to stable local RTSP proxy paths instead of directly to every camera. Use `monitor-oscar` while evaluating the camera profile, then use `launch-all` for efficient production operation once the profile is accepted. See the updated MediaMTX guide in `dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md`. ## PostgreSQL tuning diff --git a/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md b/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md index 95c58f2..3feeac4 100644 --- a/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md +++ b/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md @@ -2,14 +2,15 @@ This guide explains how to use **MediaMTX** as a local RTSP proxy so OSCAR does not have to open large numbers of direct camera connections. -For **testing and side-by-side field deployment**, the recommended operational flow is: +For **testing, troubleshooting, side-by-side evaluation, and system profiling**, the recommended operational flow is: 1. prepare a fresh OSCAR 3.5.1 deployment 2. create `.env` -3. launch OSCAR with the monitoring script +3. launch OSCAR with the monitoring script to collect profile evidence 4. proxy camera streams through MediaMTX 5. run the status-check script after warm-up 6. compare reconnect, thread, and database behavior with and without the proxy +7. use `launch-all` for efficient production operation once the camera profile is accepted --- @@ -56,21 +57,33 @@ In this setup, OSCAR talks to: Make sure **Java 21+** and **Docker** are ready for OSCAR, and that MediaMTX is installed separately on the host where you plan to run the proxy. -### 2. Start OSCAR with monitoring +### 2. Start OSCAR with the right launch path -Use the same **sessionless monitoring default** recommended for the rest of the packaged deployment workflow. +For MediaMTX evaluation, troubleshooting, and system profiling, use the monitoring wrapper so reconnect behavior, thread growth, and PostgreSQL sessions are captured. -Linux preferred command: +Linux monitored command: ```bash nohup ./monitor-oscar.sh > monitor.out 2>&1 & ``` -Windows preferred pattern: +Windows monitored pattern: -- run `monitor-oscar.bat` from **Task Scheduler** or a service wrapper +- run `monitor-oscar.bat` from **Task Scheduler**, a service wrapper, or an interactive terminal when troubleshooting -Attached launches should be used only for troubleshooting. +After the MediaMTX profile is accepted and no in-depth system profile is needed, use `launch-all` for production operation: + +```bash +./launch-all.sh +``` + +or on Windows: + +```bat +launch-all.bat +``` + +`launch-all` avoids the extra monitor directories, periodic snapshots, JFR checks, thread dumps, database trend files, and diagnostic logs produced by `monitor-oscar`. ### 3. Start MediaMTX @@ -330,7 +343,7 @@ If you only need RTSP, keep HLS, WebRTC, RTMP, and SRT disabled. ### Use monitoring during evaluation -When comparing direct-camera versus proxied-camera behavior, always launch OSCAR with the monitoring script and collect a status report after warm-up. +When comparing direct-camera versus proxied-camera behavior, launch OSCAR with the monitoring script and collect a status report after warm-up. When the comparison is complete and the profile is accepted, use `launch-all` for efficient production starts. --- @@ -388,4 +401,4 @@ In practice, that means: - easier testing with emulator lanes - simpler troubleshooting and stream replacement -For field evaluation, pair MediaMTX with the OSCAR monitoring and status-check scripts so you can compare reconnect behavior, thread growth, and system steadiness under realistic load. +For field evaluation, pair MediaMTX with the OSCAR monitoring and status-check scripts so you can compare reconnect behavior, thread growth, and system steadiness under realistic load. For steady production where that detailed profile is unnecessary, run OSCAR with `launch-all` to avoid extra monitoring logs and snapshot artifacts. diff --git a/dist/documentation/Node_Administration_3.5.1_addendum.md b/dist/documentation/Node_Administration_3.5.1_addendum.md index 0835da5..7005576 100644 --- a/dist/documentation/Node_Administration_3.5.1_addendum.md +++ b/dist/documentation/Node_Administration_3.5.1_addendum.md @@ -10,18 +10,19 @@ No major Admin UI workflow changes were required for the 3.5.1 launch, monitorin The operational changes for 3.5.1 are outside that PDF and are now covered in these updated deployment documents: -- `README.updated.md` -- `OSCAR_launch_monitoring_guide.updated.md` -- `MediaMTX_OSCAR_camera_proxy_guide.updated.md` -- `OSCAR_System_Documentation_Manual_3.5.updated.md` +- `README.md` +- `OSCAR_launch_monitoring_guide.md` +- `MediaMTX_OSCAR_camera_proxy_guide.md` +- `OSCAR_System_Documentation_Manual_3.5.md` Use the PDF for Admin Panel behavior, and use the updated deployment documents for: - Java 21 and Docker prerequisites - `.env` setup - already-running OSCAR handling +- launch-mode selection: `launch-all` for efficient production, `monitor-oscar` for validation, troubleshooting, and system profiling - monitoring and status scripts, including duplicate-monitor prevention - fresh-install cleanup of older OSCAR releases - MediaMTX-assisted camera deployment guidance -The updated `monitor-oscar.sh` and `monitor-oscar.bat` wrappers now include single-instance protection so a second live monitor launch is refused until the first one is stopped. +The updated `monitor-oscar.sh` and `monitor-oscar.bat` wrappers now include single-instance protection so a second live monitor launch is refused until the first one is stopped. Use these monitor wrappers when detailed diagnostic evidence is needed; use `launch-all.sh` or `launch-all.bat` for routine production operation to avoid unnecessary additional monitoring logs and snapshot artifacts. diff --git a/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md b/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md index aff79d1..62e06e9 100644 --- a/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md +++ b/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md @@ -86,7 +86,7 @@ The diagram below condenses the system relationships. It is not a source-code cl # 3. Installation and initial startup -Installation is intentionally simple: download a release archive, extract it, ensure Docker is installed and running, and launch the platform with the OS-specific launch-all script (`launch-all.bat` for Windows, `launch-all.sh` for Linux/macOS, or `launch-all-arm.sh` for ARM systems). The database and application come up together in the default deployment path. +Installation is intentionally simple: download a release archive, extract it, ensure Docker is installed and running, and launch the platform with the OS-specific `launch-all` script (`launch-all.bat` for Windows, `launch-all.sh` for Linux/macOS, or `launch-all-arm.sh` for ARM systems). `launch-all` is the preferred efficient production startup path because the database and application come up together without the extra monitoring logs and profile artifacts produced by the monitor wrapper. Use `monitor-oscar` instead for first-run validation, burn-in, troubleshooting, side-by-side evaluation, or system profiling. ## Prerequisites @@ -105,7 +105,9 @@ Installation is intentionally simple: download a release archive, extract it, en > > 2\. Install Docker and verify that the Docker service is running before starting OSCAR. > -> 3\. Run the launch-all script (`launch-all.bat`, `launch-all.sh`, or `launch-all-arm.sh`) for the operating system in use. In the default path, the script starts PostgreSQL locally in Docker and then starts the Java application. +> 3\. For efficient production operation, run the `launch-all` script (`launch-all.bat`, `launch-all.sh`, or `launch-all-arm.sh`) for the operating system in use. In the default path, the script starts PostgreSQL locally in Docker and then starts the Java application. +> +> For first-run validation, troubleshooting, burn-in, side-by-side evaluation, or system profiling, run `monitor-oscar` instead. The monitor path intentionally creates additional logs, snapshots, JFR checks, thread dumps, and database trend files so operators can evaluate system behavior before or during investigation. > > 4\. Open the application on the configured port. Port 8282 is the baseline HTTP application port, and 8443 is a representative HTTPS configuration. > @@ -396,6 +398,12 @@ Default deployment begins to strain as lane counts, event volume, and camera cou A practical target profile for scaled deployments is 640x480 at about 5 frames per second, while 1080p may be acceptable on smaller systems. Camera settings should be chosen with total system scale in mind. For a 50-lane or 100-camera target, the aggregate cost of 1080p video is likely excessive. +## Launch mode selection for operations + +Use `monitor-oscar` when deployment teams need evidence: first-run validation, burn-in testing, troubleshooting, camera-profile comparisons, memory and thread profiling, or PostgreSQL session analysis. The monitor wrapper is intentionally verbose and creates monitor directories, periodic snapshots, thread dumps, JFR status checks, and database trend files. + +Use `launch-all` for efficient production operation once the system profile has been validated. This keeps routine startup simple and avoids collecting detailed monitoring artifacts when no in-depth system profile is required. + ## What stays where in a split deployment When PostgreSQL is moved off the application host, videos and the other stored files remain on the application server. In practice, that means the application tier continues to own the file system while the database tier handles relational persistence. This is important for storage planning, backups, and role-based file retrieval. @@ -450,7 +458,7 @@ This section captures known gaps and enhancement ideas so they are not confused > 1\. Install Docker and verify that the service is running on the host before launch. > -> 2\. Extract the chosen OSCAR release and start it with the OS-specific launch-all script (`launch-all.bat`, `launch-all.sh`, or `launch-all-arm.sh`). +> 2\. Extract the chosen OSCAR release and start production operation with the OS-specific `launch-all` script (`launch-all.bat`, `launch-all.sh`, or `launch-all-arm.sh`). Use `monitor-oscar` instead when the goal is validation, troubleshooting, burn-in, side-by-side comparison, or system profiling. > > 3\. Change the initial admin password using the package-provided settings file and password-initialization script before production use. > diff --git a/dist/documentation/OSCAR_launch_monitoring_guide.md b/dist/documentation/OSCAR_launch_monitoring_guide.md index c20307d..72e9108 100644 --- a/dist/documentation/OSCAR_launch_monitoring_guide.md +++ b/dist/documentation/OSCAR_launch_monitoring_guide.md @@ -9,13 +9,25 @@ It explains: - how the updated `launch-all`, `launch`, `monitor-oscar`, `stop-all`, `reset-all`, and `check-oscar-status` scripts behave - how already-running OSCAR instances are handled - how to validate Java, Docker, memory, and database health -- how to use MediaMTX and status reports during testing and side-by-side field deployment +- how to use MediaMTX and status reports during testing, troubleshooting, side-by-side evaluation, and system profiling --- ## 1. Recommended operating model -For **testing, burn-in, and side-by-side field deployment**, the preferred workflow is: +Use two launch paths depending on the operational goal. + +For **efficient production operation**, use the top-level `launch-all` script: + +- `launch-all.sh` / `launch-all.bat` + +`launch-all` starts PostGIS and OSCAR with the selected `.env` system profile and avoids the additional monitor loop, snapshot files, JFR checks, thread dumps, database trend logging, and monitor directories that are only useful when an in-depth profile is needed. + +For **testing, burn-in, side-by-side evaluation, troubleshooting, and system profiling**, use the `monitor-oscar` wrapper: + +- `monitor-oscar.sh` / `monitor-oscar.bat` + +The preferred monitored workflow is: 1. unzip the prebuilt release into a fresh folder 2. create `.env` @@ -23,12 +35,8 @@ For **testing, burn-in, and side-by-side field deployment**, the preferred workf 4. start with the **monitoring script** 5. let the system warm up 6. run the **status-check script** -7. review JVM, thread, and PostgreSQL behavior before wider deployment - -Use the top-level **sessionless launchers** when possible: - -- `launch-all.sh` / `launch-all.bat` -- `monitor-oscar.sh` / `monitor-oscar.bat` +7. review JVM, thread, and PostgreSQL behavior before production use +8. switch routine production starts to `launch-all` once detailed profiling is no longer needed Avoid launching `osh-node-oscar/launch.(sh|bat)` directly unless you are debugging the node itself. @@ -235,7 +243,7 @@ The launchers also set: ### `launch-all.sh` / `launch-all.bat` -These are the supported top-level launchers. +These are the supported top-level launchers and the preferred path for efficient production operation when in-depth monitoring is not required. They: @@ -269,7 +277,7 @@ Optional runtime extras are handled gracefully: - if `nativelibs` does not exist, the launcher warns and continues - if `trusted_certificates` does not exist, the trust-store helper uses the copied default Java `cacerts` store and continues -Use these direct node launchers mainly for debugging. +Use these direct node launchers mainly for debugging. For production, use `launch-all`; for monitored validation or profiling, use `monitor-oscar`. --- @@ -307,7 +315,27 @@ For monitor wrappers, you have two supported choices: ## 8. Starting OSCAR -### Recommended first-run start with monitoring +### Efficient production start with `launch-all` + +Use this path for normal production operation after the deployment has already been validated. + +#### Linux + +```bash +./launch-all.sh +``` + +#### Windows + +```bat +launch-all.bat +``` + +This starts the database and OSCAR application without creating the extra monitoring artifacts used for diagnostic profiling. It is the ideal production mode when operators do not need detailed memory, thread, JFR, and PostgreSQL trend data for the run. + +### Testing, troubleshooting, and profiling start with monitoring + +Use this path for first-run validation, burn-in, side-by-side comparisons, troubleshooting, and system profiling. #### Linux @@ -315,7 +343,7 @@ For monitor wrappers, you have two supported choices: ./monitor-oscar.sh ``` -Preferred Linux sessionless launch: +Preferred Linux sessionless monitored launch: ```bash nohup ./monitor-oscar.sh > monitor.out 2>&1 & @@ -333,7 +361,7 @@ chmod +x *.sh osh-node-oscar/*.sh monitor-oscar.bat ``` -Preferred Windows sessionless launch from PowerShell: +Preferred Windows sessionless monitored launch from PowerShell: ```powershell Start-Process -WindowStyle Hidden -FilePath cmd.exe -ArgumentList '/c monitor-oscar.bat ^> monitor.out 2^>^&1' -WorkingDirectory (Get-Location) @@ -357,19 +385,7 @@ and captures: - PostgreSQL session and activity data - trend CSV files for database sessions -### Routine start without monitoring - -#### Linux - -```bash -./launch-all.sh -``` - -#### Windows - -```bat -launch-all.bat -``` +Because these artifacts are intentionally verbose, use `monitor-oscar` only when the extra evidence is useful. For steady production where no in-depth system profile is needed, return to `launch-all`. --- @@ -409,7 +425,7 @@ This avoids `stop-all` getting stuck on a monitor-closing attempt. ### `reset-all` -Use `reset-all` when you want a clean local test surface before trying a different packaged installation on the same machine. +Use `reset-all` when you want a clean local test surface before trying a different packaged installation on the same machine. It is mainly a testing and troubleshooting tool, not part of routine production startup. It: @@ -446,7 +462,7 @@ reset-all.bat powershell -ExecutionPolicy Bypass -File .\check-oscar-status.ps1 ``` -These scripts summarize the latest monitor run into a single text report that includes: +These scripts are most useful after a monitored validation or profiling run. They summarize the latest monitor run into a single text report that includes: - process status - live JVM information @@ -508,11 +524,12 @@ Use `db-connection-trend.csv` as the fastest way to spot connection growth, plat For larger camera configurations or side-by-side test deployments: -1. start OSCAR with `monitor-oscar` +1. start OSCAR with `monitor-oscar` for evaluation and profiling 2. route camera streams through MediaMTX 3. let the system run long enough to capture reconnect and thread behavior 4. run `check-oscar-status` 5. compare JVM threads, reconnect logs, and PostgreSQL sessions before and after enabling MediaMTX +6. use `launch-all` for efficient production operation once the camera profile is accepted MediaMTX is especially helpful when many logical lane-camera assignments reuse a smaller number of real camera streams. diff --git a/dist/documentation/Release_Notes_3.5.1.md b/dist/documentation/Release_Notes_3.5.1.md index 835780e..592c690 100644 --- a/dist/documentation/Release_Notes_3.5.1.md +++ b/dist/documentation/Release_Notes_3.5.1.md @@ -14,7 +14,7 @@ OSCAR **3.5.1** improves deployment stability, observability, and scalability fo These changes were validated against a high-load configuration monitoring **50 radiation portal monitors and 100 camera streams**. -This is a **prebuilt release**. Users should **unzip OSCAR 3.5.1 into a fresh directory** and start it with the included **monitoring script**, preferably using the **sessionless launch** when possible. +This is a **prebuilt release**. Users should **unzip OSCAR 3.5.1 into a fresh directory**. Use the included **monitoring script** for first-run validation, burn-in, troubleshooting, and system profiling. Use **`launch-all`** for efficient production operation after validation, because it avoids unnecessary detailed monitoring logs and snapshot artifacts when an in-depth system profile is not required. --- @@ -31,15 +31,19 @@ The packaged release archive is expected to be named **`oscar-3.5.1.zip`**. ### Recommended deployment model -For testing, side-by-side field deployment, and first-run validation: +For production operation: * unzip the release into a **new clean folder** * rename `env.txt` to `.env` if needed * select the correct system profile in `.env` -* use **MediaMTX** for camera-heavy deployments +* use **MediaMTX** for camera-heavy deployments when appropriate +* start OSCAR with **`launch-all`** after validation so the system runs without unnecessary monitoring log and snapshot generation + +For testing, side-by-side field evaluation, first-run validation, troubleshooting, and system profiling: + * start OSCAR with the **sessionless monitoring launch** when possible +* use the **check/status script** to review memory, thread, and PostgreSQL behavior * use the new reset scripts when you need to clear a previous local test install before switching releases -* use the **check/status script** to review performance --- @@ -152,7 +156,7 @@ The monitor wrappers now also include a singleton guard so a second `monitor-osc These scripts can now: * launch OSCAR under monitoring -* support sessionless launch for normal deployment use +* support sessionless launch for validation, troubleshooting, burn-in, and profiling runs * capture JVM memory status * capture native memory tracking summaries * capture JFR status @@ -199,7 +203,7 @@ This supports a deployment model where: * multiple lanes can reuse proxied feeds * OSCAR connects to stable local endpoints instead of managing a large number of direct camera connections -This architecture is recommended for larger systems and appears to reduce camera-related reconnect burden. +This architecture is recommended for larger systems and appears to reduce camera-related reconnect burden. Validate camera-heavy profiles with `monitor-oscar`, then use `launch-all` for efficient production starts once the profile is accepted. ### Documentation updates @@ -390,39 +394,47 @@ Then edit the file and select the correct hardware profile: The environment template also supports launch and monitoring behavior such as restart and attach settings. -### Step 4: start with the monitoring script +### Step 4: choose the launch path -For OSCAR 3.5.1, users should launch with the monitoring script so diagnostics begin immediately. +For efficient production operation after validation, start OSCAR with `launch-all`. -Use the **sessionless launch** when possible so OSCAR keeps running without requiring an attached terminal session. +#### Linux production -#### Linux +```bash +./launch-all.sh +``` -Preferred: +#### Windows production -```bash -./monitor-oscar.sh --daemon +```bat +launch-all.bat ``` -If your script version starts sessionless by default, use: +Use the monitoring script when diagnostics are needed for first-run validation, burn-in, troubleshooting, side-by-side comparison, or system profiling. + +#### Linux monitored validation or profiling + +Preferred sessionless pattern: ```bash -./monitor-oscar.sh +nohup ./monitor-oscar.sh > monitor.out 2>&1 & ``` -Use an attached launch only for interactive troubleshooting. +Attached interactive pattern: -#### Windows +```bash +./monitor-oscar.sh +``` -Preferred: +#### Windows monitored validation or profiling ```bat monitor-oscar.bat ``` -Use the sessionless option if your Windows wrapper provides both attached and detached modes. +For detached Windows operation, run `monitor-oscar.bat` from Task Scheduler, a service wrapper, or a hidden PowerShell `Start-Process` invocation. -Use an attached launch only for interactive troubleshooting. +The monitoring path intentionally produces additional logs, monitor directories, snapshots, JFR checks, thread dumps, and database trend files. Use it when that evidence is valuable; otherwise use `launch-all` for routine production. ### Step 5: check performance with the included status script @@ -457,7 +469,7 @@ For testing and side-by-side field deployment, users should: 7. rename `env.txt` to `.env` if needed 8. select the correct profile in `.env` 9. configure and use **MediaMTX** for camera-heavy systems -10. start OSCAR with the **sessionless monitoring launch** when possible +10. start OSCAR with the **sessionless monitoring launch** when collecting validation or profile evidence 11. use the check or status script to compare system behavior and performance 12. use the reset script when you need to remove the local OSCAR runtime state before testing another package on the same machine @@ -497,8 +509,9 @@ This is the preferred workflow for: * select the correct hardware profile in `.env` * use the updated launch scripts -* use the **sessionless monitoring launch** for initial validation and normal field deployment when possible -* use the attached launch only for interactive troubleshooting +* use **`launch-all`** for efficient production operation after validation +* use the **sessionless monitoring launch** for initial validation, burn-in, side-by-side evaluation, troubleshooting, and profiling +* use the attached monitoring launch only for interactive troubleshooting * let the scripts manage already-running instances instead of manually launching duplicates * use MediaMTX where many camera streams are involved * review generated status reports during early burn-in testing @@ -539,8 +552,9 @@ These do not appear to be the primary cause of the major stability issue address 7. Rename `env.txt` to `.env` if needed. 8. Edit `.env` and select the correct hardware profile. 9. For camera-heavy deployments, configure MediaMTX. -10. Start the system with the **sessionless monitoring launch** when possible. -11. Use the check or status script after startup and again after runtime burn-in. +10. Start the system with **`monitor-oscar`** when collecting validation, troubleshooting, or profile evidence. +11. Start the system with **`launch-all`** for efficient production operation after validation. +12. Use the check or status script after monitored startup and again after runtime burn-in. --- diff --git a/include/osh-oakridge-modules b/include/osh-oakridge-modules index aaefbb7..54fe70b 160000 --- a/include/osh-oakridge-modules +++ b/include/osh-oakridge-modules @@ -1 +1 @@ -Subproject commit aaefbb79cd651265f04ee2337428c8168c24d256 +Subproject commit 54fe70bf4e39e68950ec9bdc56fc33d48fceaf91 From 339cf58672959851bc3ab81abc54b72143740e43 Mon Sep 17 00:00:00 2001 From: tyronechrisharris <142608718+tyronechrisharris@users.noreply.github.com> Date: Sun, 10 May 2026 00:29:39 -0400 Subject: [PATCH 8/9] Enhance monitoring scripts and improve sessionless operation - Introduced `monitor-oscar.ps1` as the preferred PowerShell wrapper for sessionless monitoring on Windows, replacing the previous batch script. - Updated `monitor-oscar.bat` to call the new PowerShell script, simplifying the monitoring process. - Added new features to `monitor-oscar.ps1` for better process management, heartbeat updates, and environment variable loading. - Enhanced documentation to reflect changes in monitoring scripts and sessionless operation instructions for both Linux and Windows. - Improved cleanup procedures in `reset-all.bat` and `stop-all.bat` to remove stale monitor lock files and related artifacts. - Updated monitoring architecture to reduce camera-related reconnect burden and improve Java backend efficiency. --- README.md | 240 +++-- .../MediaMTX_OSCAR_camera_proxy_guide.md | 338 +------ .../Node_Administration_3.5.1_addendum.md | 10 +- .../OSCAR_System_Documentation_Manual_3.5.md | 64 +- .../OSCAR_launch_monitoring_guide.md | 900 ++++++++++-------- dist/documentation/Release_Notes_3.5.1.md | 51 +- dist/release/monitor-oscar.bat | 352 +------ dist/release/monitor-oscar.ps1 | 290 ++++++ dist/release/reset-all.bat | 5 +- dist/release/stop-all.bat | 3 + 10 files changed, 1174 insertions(+), 1079 deletions(-) create mode 100644 dist/release/monitor-oscar.ps1 diff --git a/README.md b/README.md index 875db07..708b691 100644 --- a/README.md +++ b/README.md @@ -31,47 +31,13 @@ docker version Use **Java 21 or newer**. The launch scripts validate dependencies and will stop early if Java or Docker is missing or too old. -### 2. If you were previously running OSCAR, start fresh - -Before extracting **OSCAR 3.5.1**, stop the old deployment and remove old local runtime artifacts. - -Windows PowerShell: - -```powershell -Get-CimInstance Win32_Process | - Where-Object { - $_.Name -match '^java(\.exe)?$' -and - $_.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*' - } | - Select-Object ProcessId, CommandLine - -# Stop the old OSCAR JVM if one is still running -Stop-Process -Id -Force - -# Stop and remove the old PostGIS container -docker rm -f oscar-postgis-container - -# Remove the old Docker network if it exists -docker network rm oscar-postgis-network -``` - -Linux: - -```bash -pgrep -af 'com.botts.impl.security.SensorHubWrapper' -kill - -docker rm -f oscar-postgis-container -docker network rm oscar-postgis-network || true -``` - -Then delete the old extracted release folder, such as **oscar-3.5.0**, before extracting **oscar-3.5.1**. - -### 3. Extract the release archive +### 2. Extract the release archive to a fresh directory Extract the downloaded ZIP to a fresh working directory. -### 4. Create the runtime environment file +Do not launch a new release on top of an older extracted directory. Reusing an old directory can leave behind monitor state, runtime data, logs, or generated config that makes troubleshooting harder. + +### 3. Create the runtime environment file For packaged releases, use the environment file that ships with the archive: @@ -108,43 +74,131 @@ Useful optional settings include: - `RETRY_INTERVAL=2` - `POSTGIS_READY_DELAY=5` -### 5. Production start: use `launch-all` +### 4. Preferred production start: use `launch-all` sessionless -For efficient production operation after validation is complete, use the top-level launch script instead of the monitoring wrapper. +For routine production use, prefer the top-level **`launch-all`** scripts and run them **sessionless by default**. This avoids depending on an open SSH session, an RDP window, or a console that might be closed later. -Windows: +`launch-all` is the preferred production path because it starts PostGIS and OSCAR with the selected `.env` profile without the extra monitor loop, recurring snapshots, JFR checks, thread dumps, database trend files, and monitor-directory logging. This keeps routine startup simpler and avoids collecting detailed profile data when operators do not need an in-depth system profile. + +Prefer these **top-level launchers** over calling `osh-node-oscar/launch.(sh|bat)` directly unless you are debugging the node itself. + +#### Windows production start + +Interactive: ```bat launch-all.bat ``` -Linux: +Sessionless from PowerShell: + +```powershell +Start-Process cmd.exe ` + -ArgumentList '/c', 'launch-all.bat > launch.out 2>&1' ` + -WorkingDirectory $PWD ` + -WindowStyle Hidden +``` + +#### Linux production start + +Interactive: ```bash ./launch-all.sh ``` -`launch-all` is the preferred production path because it starts PostGIS and OSCAR with the selected `.env` profile without the extra monitor loop, recurring snapshots, JFR checks, thread dumps, database trend files, and monitor-directory logging. This keeps routine startup simpler and avoids collecting detailed profile data when operators do not need an in-depth system profile. +Sessionless: -Prefer these **top-level launchers** over calling `osh-node-oscar/launch.(sh|bat)` directly unless you are debugging the node itself. +```bash +nohup ./launch-all.sh > launch.out 2>&1 & +``` -### 6. Validation, troubleshooting, and profiling start: use `monitor-oscar` +#### Production auto-start after reboot -For testing, burn-in, side-by-side field evaluation, troubleshooting, and system profiling, start OSCAR with the monitoring wrapper. +For production systems, configure the machine to start OSCAR automatically after restart. -Windows: +##### Windows Task Scheduler -```bat -monitor-oscar.bat +Create a scheduled task that: + +- runs **whether the user is logged on or not** +- triggers **at startup** +- starts in the extracted OSCAR directory +- launches `launch-all.bat` +- uses a small startup delay if Docker Desktop needs time to initialize +- restarts the task on failure + +A practical action is: + +```text +Program/script: powershell.exe +Arguments: -NoProfile -ExecutionPolicy Bypass -Command "Set-Location 'C:\path\to\oscar-3.5.1'; cmd /c launch-all.bat >> launch.out 2>&1" +``` + +If using Docker Desktop on Windows, make sure Docker Desktop itself is configured to start with Windows before relying on the scheduled OSCAR start. + +##### Linux systemd + +Use a dedicated `systemd` unit so OSCAR starts after Docker is available and restarts automatically if the service fails. + +Example `/etc/systemd/system/oscar.service`: + +```ini +[Unit] +Description=OSCAR launch-all service +After=network-online.target docker.service +Wants=network-online.target docker.service + +[Service] +Type=simple +User=oscar +WorkingDirectory=/home/oscar/oscar-3.5.1 +ExecStart=/bin/bash -lc './launch-all.sh' +Restart=on-failure +RestartSec=10 + +[Install] +WantedBy=multi-user.target +``` + +Then enable it: + +```bash +sudo systemctl daemon-reload +sudo systemctl enable --now oscar.service +``` + +Also ensure Docker starts on boot: + +```bash +sudo systemctl enable docker ``` -Windows sessionless PowerShell launch: +### 5. Validation, troubleshooting, and profiling: use `monitor-oscar` + +For testing, burn-in, side-by-side field evaluation, troubleshooting, and system profiling, start OSCAR with the monitoring wrapper instead of `launch-all`. + +#### Windows monitor start + +Preferred interactive start: ```powershell -Start-Process -WindowStyle Hidden -FilePath cmd.exe -ArgumentList '/c monitor-oscar.bat ^> monitor.out 2^>^&1' -WorkingDirectory (Get-Location) +powershell -NoProfile -ExecutionPolicy Bypass -File .\monitor-oscar.ps1 ``` -Linux: +Preferred sessionless start: + +```powershell +Start-Process powershell.exe ` + -ArgumentList '-NoProfile','-ExecutionPolicy','Bypass','-File',"$PWD\monitor-oscar.ps1" ` + -WindowStyle Hidden ` + -RedirectStandardOutput "$PWD\monitor.out" ` + -RedirectStandardError "$PWD\monitor.err" +``` + +If `monitor-oscar.bat` is still present in a package, treat `monitor-oscar.ps1` as the preferred Windows monitor entrypoint. + +#### Linux monitor start ```bash chmod +x launch-all.sh osh-node-oscar/launch.sh monitor-oscar.sh check-oscar-status.sh @@ -166,9 +220,9 @@ This is the preferred validation and troubleshooting path because it: Once the system is validated and no in-depth profile is needed, switch routine production starts back to `launch-all`. -### 7. Running-instance handling +### 6. Running-instance handling and duplicate monitor protection -The launch and monitor scripts now detect already-running OSCAR JVMs. +The launch and monitor scripts detect already-running OSCAR JVMs. Default behavior: @@ -181,14 +235,77 @@ Optional behaviors: - set `FORCE_RESTART=1` to stop the running OSCAR instance and start fresh - set `ATTACH_TO_EXISTING=1` when using `monitor-oscar` to monitor the running instance instead of replacing it -When using `nohup`, a hidden `cmd.exe`, Task Scheduler, or another sessionless strategy, check these files after launch: +When using `nohup`, Task Scheduler, `Start-Process`, or another sessionless strategy, check these files after launch: - `monitor.last-status` - `monitor.last-error` -- `monitor.out` when you redirected stdout and stderr +- `monitor.out` +- `monitor.err` on Windows PowerShell launches that redirect stderr separately If a second monitor start is refused, `monitor.last-status` records a clear failure such as `FAILED duplicate_monitor ...` so the operator can tell why the wrapper exited without staying attached to the terminal. +### 7. Reset and full cleanup guidance + +If you were previously running OSCAR, stop the old deployment before extracting and launching a new copy. + +#### Normal stop + +Windows: + +```bat +stop-all.bat +``` + +Linux: + +```bash +./stop-all.sh +``` + +Also verify no old OSCAR JVM is still running. + +Windows PowerShell: + +```powershell +Get-CimInstance Win32_Process | + Where-Object { + $_.Name -match '^java(\.exe)?$' -and + $_.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*' + } | + Select-Object ProcessId, CommandLine +``` + +Linux: + +```bash +pgrep -af 'com.botts.impl.security.SensorHubWrapper' +``` + +#### If `reset-all` was run but old lanes still appear + +If a user runs the reset script while a monitor wrapper is still active, the monitor can restart OSCAR and old lanes can appear again. In that case, do a full cleanup: + +1. run `stop-all` first so the monitor wrapper and OSCAR JVM are both stopped +2. confirm no `monitor-oscar` wrapper and no `SensorHubWrapper` JVM are still running +3. delete the extracted release directory +4. re-extract the ZIP +5. recreate `.env` +6. relaunch using the preferred sessionless method + +Linux example: + +```bash +./stop-all.sh +cd .. +sudo rm -r oscar-3.5.1 +unzip oscar-3.5.1.zip +cd oscar-3.5.1 +cp env.template .env +nohup ./launch-all.sh > launch.out 2>&1 & +``` + +On Linux, removing the extracted release directory may require `sudo` depending on how files were created during previous runs. + ### 8. Generate a status report after startup After the system has been up long enough to settle, generate a one-file report. @@ -258,15 +375,20 @@ If you are testing from a source checkout instead of a packaged release: 2. verify Java 21 and Docker 3. launch with `monitor-oscar` for first-run validation, troubleshooting, or profiling 4. use `check-oscar-status` after the monitored system reaches steady state -5. use `launch-all` for efficient routine production starts after validation +5. switch routine production starts to `launch-all` once validation is complete +6. use sessionless launch methods for unattended systems instead of relying on an open terminal ## MediaMTX for larger camera deployments -For test systems and larger multi-lane deployments, consider placing **MediaMTX** in front of camera streams so OSCAR connects to stable local RTSP proxy paths instead of directly to every camera. Use `monitor-oscar` while evaluating the camera profile, then use `launch-all` for efficient production operation once the profile is accepted. See the updated MediaMTX guide in `dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md`. +For larger camera counts, place **MediaMTX** in front of the RTSP sources and point OSCAR at the local MediaMTX proxy paths. + +This reduces load on the Java backend because OSCAR no longer has to open, maintain, and recover every remote camera connection directly. MediaMTX absorbs much of the connection churn, buffering, and stream fan-out work, while OSCAR reads from fewer, more stable local proxy endpoints. + +Use `monitor-oscar` while validating the camera profile, then return to `launch-all` for routine production operation. Keep the MediaMTX deployment simple and focused on camera proxying. See `dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md` for the full guide. ## PostgreSQL tuning -The packaged launch scripts now size PostgreSQL by `SYSTEM_PROFILE`. +The packaged launch scripts size PostgreSQL by `SYSTEM_PROFILE`. Representative values: diff --git a/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md b/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md index 3feeac4..614b467 100644 --- a/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md +++ b/dist/documentation/MediaMTX_OSCAR_camera_proxy_guide.md @@ -1,153 +1,29 @@ # Using MediaMTX to Reduce OSCAR Camera Stream Load -This guide explains how to use **MediaMTX** as a local RTSP proxy so OSCAR does not have to open large numbers of direct camera connections. +This guide shows the recommended **MediaMTX** pattern for camera-heavy OSCAR deployments. -For **testing, troubleshooting, side-by-side evaluation, and system profiling**, the recommended operational flow is: - -1. prepare a fresh OSCAR 3.5.1 deployment -2. create `.env` -3. launch OSCAR with the monitoring script to collect profile evidence -4. proxy camera streams through MediaMTX -5. run the status-check script after warm-up -6. compare reconnect, thread, and database behavior with and without the proxy -7. use `launch-all` for efficient production operation once the camera profile is accepted - ---- +The short version is simple: put MediaMTX between OSCAR and the physical cameras, point the OSCAR lane CSV at the MediaMTX RTSP paths, validate the camera profile with `monitor-oscar`, and then use `launch-all` for routine production starts. ## Why MediaMTX helps -Without a proxy, OSCAR may open many direct RTSP sessions to cameras. If you have many lanes, reconnect activity and repeated stream setup can create unnecessary load on the cameras and on the OSCAR host. - -MediaMTX sits between OSCAR and the cameras: - -- cameras stream to MediaMTX only when needed -- OSCAR connects to MediaMTX on the local machine instead of directly to every camera -- multiple lane definitions can reuse the same proxied path -- reconnects are handled against the local proxy instead of repeatedly hammering the physical cameras - -This is especially useful when: - -- many OSCAR lanes reuse a smaller number of real camera streams -- emulator lanes are used for testing or demonstrations -- direct camera sessions are expensive or unstable -- you want a simple local point to change, test, or swap stream sources - ---- - -## Typical architecture - -A common layout is: - -1. physical cameras publish RTSP streams -2. MediaMTX runs on the same machine as OSCAR or on a nearby local host -3. MediaMTX exposes local RTSP paths such as `/lane03_cam` -4. the OSCAR lane CSV points camera hosts to the local MediaMTX service -5. the SRLS emulator or detector-side service is addressed separately from the same CSV - -In this setup, OSCAR talks to: - -- the SRLS emulator or RPM service for detector data -- MediaMTX for camera video - ---- - -## Recommended OSCAR workflow when using MediaMTX - -### 1. Verify dependencies - -Make sure **Java 21+** and **Docker** are ready for OSCAR, and that MediaMTX is installed separately on the host where you plan to run the proxy. - -### 2. Start OSCAR with the right launch path - -For MediaMTX evaluation, troubleshooting, and system profiling, use the monitoring wrapper so reconnect behavior, thread growth, and PostgreSQL sessions are captured. - -Linux monitored command: - -```bash -nohup ./monitor-oscar.sh > monitor.out 2>&1 & -``` - -Windows monitored pattern: - -- run `monitor-oscar.bat` from **Task Scheduler**, a service wrapper, or an interactive terminal when troubleshooting - -After the MediaMTX profile is accepted and no in-depth system profile is needed, use `launch-all` for production operation: - -```bash -./launch-all.sh -``` - -or on Windows: - -```bat -launch-all.bat -``` - -`launch-all` avoids the extra monitor directories, periodic snapshots, JFR checks, thread dumps, database trend files, and diagnostic logs produced by `monitor-oscar`. - -### 3. Start MediaMTX - -Linux: - -```bash -./mediamtx mediamtx.yml -``` - -Windows PowerShell: - -```powershell -.\mediamtx.exe mediamtx.yml -``` - -### 4. Let the system warm up - -Allow enough time for lane startup, first client connections, and any reconnect behavior to appear. - -### 5. Generate a status report - -Linux: - -```bash -./check-oscar-status.sh -``` - -Windows: - -```powershell -powershell -ExecutionPolicy Bypass -File .\check-oscar-status.ps1 -``` - -Review: - -- thread count -- reconnect warnings -- camera-related log churn -- PostgreSQL session plateau behavior - ---- +Without a proxy, OSCAR opens direct RTSP sessions to each camera endpoint referenced by the lanes. At larger scale, that increases: -## Why the provided MediaMTX settings are efficient +- Java-side camera session setup work +- reconnect churn when cameras or networks wobble +- socket and thread churn around repeated stream handling +- load on the physical cameras when many logical lanes reuse the same feeds -The configuration below keeps MediaMTX focused on lightweight RTSP proxying: +MediaMTX reduces that burden by presenting OSCAR with a smaller set of stable local RTSP paths. The proxy owns the upstream camera connections, while the Java backend talks to local endpoints instead of repeatedly reconnecting to every physical camera. -- `rtmp: no` -- `hls: no` -- `webrtc: no` -- `srt: no` +## Recommended operating model -Disabling unused protocols avoids extra listeners and extra processing. +- Use `monitor-oscar` during first-run validation, troubleshooting, burn-in, and camera-profile testing. +- Use `launch-all` for routine production after the camera profile is accepted. +- Keep MediaMTX on the OSCAR host or a nearby LAN host whenever possible. -These options are also important: +## Minimal MediaMTX configuration -- `sourceOnDemand: yes` means MediaMTX only pulls the upstream camera when a client requests the path -- `sourceProtocol: tcp` makes RTSP transport use TCP, which is often more reliable on LANs and across NAT or firewall boundaries -- `api: yes` with `apiAddress: :9997` gives you a simple status API for checking paths and clients - ---- - -## Example MediaMTX configuration for Axis cameras - -Use placeholders for credentials in documentation and shared files. +Use a lightweight RTSP-only profile unless you explicitly need other protocols. ```yaml api: yes @@ -166,101 +42,35 @@ paths: source: "rtsp://:@192.168.8.229/axis-media/media.amp?adjustablelivestream=1&resolution=640x480&videocodec=h264&videokeyframeinterval=15" sourceOnDemand: yes sourceProtocol: tcp - - lane05_cam: - source: "rtsp://:@192.168.8.111/axis-media/media.amp?adjustablelivestream=1&resolution=640x480&videocodec=h264&videokeyframeinterval=15" - sourceOnDemand: yes - sourceProtocol: tcp - - lane06_cam: - source: "rtsp://:@192.168.8.167/axis-media/media.amp?adjustablelivestream=1&resolution=640x480&videocodec=h264&videokeyframeinterval=15" - sourceOnDemand: yes - sourceProtocol: tcp ``` ---- - -## How this supports many lanes with fewer real cameras - -If your OSCAR deployment has 50 lanes with 2 cameras per lane, that is 100 camera references. Those 100 references do **not** need to be 100 unique physical camera connections. - -With MediaMTX, many lane rows can point back to the same proxied path, such as: +### Why these settings are recommended -- `rtsp://192.168.8.77:8554/lane04_cam` -- `rtsp://192.168.8.77:8554/lane06_cam` +- `sourceOnDemand: yes` avoids pulling upstream video when OSCAR is not using the path. +- `sourceProtocol: tcp` is usually the most predictable RTSP transport on real LANs, VPNs, and NATed paths. +- Disabling `rtmp`, `hls`, `webrtc`, and `srt` keeps MediaMTX focused on simple RTSP proxying. +- The API on port `9997` gives you a quick way to verify that the paths are present. -That means the CSV can model many lane-camera assignments while MediaMTX proxies only a small set of real upstream streams. +## OSCAR CSV pattern ---- - -## Example OSCAR service CSV entries - -Upload the CSV through the **Services** tab in the OSCAR admin page. +The OSCAR CSV should point camera fields at the MediaMTX host and path, not at the physical camera. ```csv Name,UniqueID,AutoStart,Latitude,Longitude,RPMConfigType,RPMHost,RPMPort,AspectAddressStart,AspectAddressEnd,EMLEnabled,EMLCollimated,LaneWidth,CameraType0,CameraHost0,CameraPath0,Codec0,Username0,Password0,CameraType1,CameraHost1,CameraPath1,Codec1,Username1,Password1 -sim-0,simu-0,FALSE,35.89,-84.19,Rapiscan,192.168.8.77,1601,,,FALSE,FALSE,4.820000172,Custom,192.168.8.77:8554,/lane04_cam,,,,Custom,192.168.8.77:8554,/lane06_cam,,, -sim-2,simu-1,FALSE,35.883,-84.19,Rapiscan,192.168.8.77,1602,,,FALSE,FALSE,4.820000172,Custom,192.168.8.77:8554,/lane06_cam,,,,Custom,192.168.8.77:8554,/lane04_cam,,, +sim-0,simu-0,FALSE,35.89,-84.19,Rapiscan,192.168.8.77,1601,,,FALSE,FALSE,4.820000172,Custom,192.168.8.77:8554,/lane03_cam,,,,Custom,192.168.8.77:8554,/lane04_cam,,, ``` -### Important CSV fields - -- `RPMHost` and `RPMPort` point to the SRLS or Rapiscan emulator/service -- `CameraType0` and `CameraType1` are set to `Custom` so OSCAR uses the host/path directly -- `CameraHost0` and `CameraHost1` point to the machine running MediaMTX, usually `:8554` -- `CameraPath0` and `CameraPath1` point to the MediaMTX path, such as `/lane04_cam` -- `Username` and `Password` fields are left blank because authentication is handled upstream by MediaMTX when it pulls from the real camera +### Important fields ---- +- `RPMHost` and `RPMPort` still point to the SRLS, Rapiscan, or emulator service. +- `CameraType*` should be `Custom` when you want OSCAR to use the host and path directly. +- `CameraHost*` should point to the machine running MediaMTX, usually `:8554`. +- `CameraPath*` should point to the MediaMTX path, such as `/lane03_cam`. +- `Username*` and `Password*` can usually remain blank because MediaMTX authenticates to the physical camera upstream. -## Sony camera settings to use +## Quick setup -For Sony cameras, use the following secondary-stream settings: - -- `H.264` -- `640x480` -- `15 fps` -- `1 s` keyframe interval when possible -- `high` H.264 profile -- `CBR` -- about `4000 kbps` - -These settings are a good match for lightweight proxying and testing because they keep the stream modest while still providing stable H.264 output. - -Example Sony-based MediaMTX path: - -```yaml -api: yes -apiAddress: :9997 -rtmp: no -hls: no -webrtc: no -srt: no -paths: - lane03_cam: - source: "rtsp://:@192.168.8.4:554/media/video2" - sourceOnDemand: yes - sourceProtocol: tcp -``` - ---- - -## Step-by-step setup - -### 1. Install MediaMTX - -Download MediaMTX for your platform and extract it onto the host that will run the proxy. - -Typical contents include: - -- `mediamtx` or `mediamtx.exe` -- `mediamtx.yml` - -### 2. Create `mediamtx.yml` - -Start with one of the examples above and add one path per real upstream camera stream. - -### 3. Start MediaMTX +### 1. Start MediaMTX Linux: @@ -274,7 +84,7 @@ Windows PowerShell: .\mediamtx.exe mediamtx.yml ``` -### 4. Verify MediaMTX is listening +### 2. Verify that MediaMTX is listening Linux: @@ -288,7 +98,7 @@ Windows PowerShell: Get-NetTCPConnection -LocalPort 8554,9997 -State Listen ``` -### 5. Verify the API +### 3. Verify the API Linux or macOS: @@ -302,26 +112,13 @@ Windows PowerShell: Invoke-RestMethod http://127.0.0.1:9997/v3/paths/list ``` -### 6. Update the OSCAR CSV - -Point lane camera hosts and paths to MediaMTX instead of the physical cameras. - -### 7. Upload the CSV in OSCAR - -In the OSCAR admin UI: - -1. open the **Services** tab -2. upload the CSV -3. confirm the lanes import successfully -4. start the desired lanes or services - ---- +### 4. Point OSCAR at the proxy -## Best practices +Update the lane CSV so each camera host and path points to MediaMTX instead of the physical device, then upload the CSV through the **Services** tab in the OSCAR admin page. -### Keep streams modest +## Camera profile guidance -Use modest stream settings for test environments: +For larger systems, keep the streams modest unless you have already validated a heavier profile. A practical starting point is: - H.264 - 640x480 @@ -329,76 +126,43 @@ Use modest stream settings for test environments: - CBR - 1-second keyframe interval when possible -### Use `sourceOnDemand` +Those settings reduce total decode and transport cost while still giving OSCAR useful video. -This avoids pulling from upstream cameras when no OSCAR consumer is actually using the stream. - -### Use local proxy paths in OSCAR - -Do not point every lane directly at physical cameras if the same few streams can be reused. - -### Keep protocols disabled unless needed - -If you only need RTSP, keep HLS, WebRTC, RTMP, and SRT disabled. - -### Use monitoring during evaluation - -When comparing direct-camera versus proxied-camera behavior, launch OSCAR with the monitoring script and collect a status report after warm-up. When the comparison is complete and the profile is accepted, use `launch-all` for efficient production starts. - ---- - -## Troubleshooting +## Fast troubleshooting ### OSCAR cannot open the stream Check: - MediaMTX is running +- the path name matches exactly - the lane CSV points to the correct host and path -- port `8554` is open locally -- the MediaMTX path name matches exactly +- port `8554` is reachable from the OSCAR host -Direct test example: +Quick direct test: ```bash ffplay rtsp://127.0.0.1:8554/lane03_cam ``` -### MediaMTX path exists but does not pull video +### The path exists but does not pull video Check: -- upstream camera IP and credentials -- the source URL path -- whether the camera is configured for the requested stream format -- whether the camera accepts TCP RTSP transport +- upstream camera IP, credentials, and path +- whether the camera supports the requested stream format +- whether the camera accepts RTSP over TCP -### Too many reconnects +### Reconnect churn is still high -MediaMTX can localize reconnect activity, but also check: +MediaMTX reduces Java-side reconnect pressure, but it cannot fix every upstream problem. Also check: - camera network stability -- camera encoder settings -- duplicate or unstable stream consumers -- bad payloads or emulator-side issues +- encoder configuration +- duplicate consumers +- emulator-side or payload-side issues - OSCAR thread and reconnect logs from the monitor directory -### API works but paths do not appear active - -With `sourceOnDemand: yes`, a path may stay idle until a client requests it. That is normal. - ---- - ## Summary -MediaMTX reduces the burden of large camera configurations by turning many direct camera connections into a smaller number of local RTSP proxy paths. - -In practice, that means: - -- less direct pressure on physical cameras -- fewer repeated direct sessions from OSCAR -- easier CSV-based lane provisioning -- easier testing with emulator lanes -- simpler troubleshooting and stream replacement - -For field evaluation, pair MediaMTX with the OSCAR monitoring and status-check scripts so you can compare reconnect behavior, thread growth, and system steadiness under realistic load. For steady production where that detailed profile is unnecessary, run OSCAR with `launch-all` to avoid extra monitoring logs and snapshot artifacts. +MediaMTX reduces the burden on the OSCAR Java backend by replacing many direct camera sessions with a smaller set of local RTSP proxy paths. That usually means less reconnect churn, less socket and thread churn, easier lane reuse, and a simpler camera topology for large test or production systems. diff --git a/dist/documentation/Node_Administration_3.5.1_addendum.md b/dist/documentation/Node_Administration_3.5.1_addendum.md index 7005576..3d7c838 100644 --- a/dist/documentation/Node_Administration_3.5.1_addendum.md +++ b/dist/documentation/Node_Administration_3.5.1_addendum.md @@ -10,7 +10,7 @@ No major Admin UI workflow changes were required for the 3.5.1 launch, monitorin The operational changes for 3.5.1 are outside that PDF and are now covered in these updated deployment documents: -- `README.md` +- `Release_Notes_3.5.1.md` - `OSCAR_launch_monitoring_guide.md` - `MediaMTX_OSCAR_camera_proxy_guide.md` - `OSCAR_System_Documentation_Manual_3.5.md` @@ -21,8 +21,12 @@ Use the PDF for Admin Panel behavior, and use the updated deployment documents f - `.env` setup - already-running OSCAR handling - launch-mode selection: `launch-all` for efficient production, `monitor-oscar` for validation, troubleshooting, and system profiling +- sessionless production starts with `launch-all` on Linux and Windows +- automatic production startup after reboot with `systemd` on Linux or Task Scheduler on Windows - monitoring and status scripts, including duplicate-monitor prevention +- the new Windows PowerShell monitor wrapper `monitor-oscar.ps1` +- the preferred hidden PowerShell `Start-Process` pattern for monitored Windows runs - fresh-install cleanup of older OSCAR releases -- MediaMTX-assisted camera deployment guidance +- MediaMTX-assisted camera deployment guidance, now shortened and focused on Java-backend load reduction -The updated `monitor-oscar.sh` and `monitor-oscar.bat` wrappers now include single-instance protection so a second live monitor launch is refused until the first one is stopped. Use these monitor wrappers when detailed diagnostic evidence is needed; use `launch-all.sh` or `launch-all.bat` for routine production operation to avoid unnecessary additional monitoring logs and snapshot artifacts. +The updated monitor wrappers include single-instance protection so a second live monitor launch is refused until the first one is stopped. Use the monitor wrappers when detailed diagnostic evidence is needed; use `launch-all.sh` or `launch-all.bat` for routine production operation to avoid unnecessary monitoring logs and snapshot artifacts. diff --git a/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md b/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md index 62e06e9..d68de18 100644 --- a/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md +++ b/dist/documentation/OSCAR_System_Documentation_Manual_3.5.md @@ -107,7 +107,9 @@ Installation is intentionally simple: download a release archive, extract it, en > > 3\. For efficient production operation, run the `launch-all` script (`launch-all.bat`, `launch-all.sh`, or `launch-all-arm.sh`) for the operating system in use. In the default path, the script starts PostgreSQL locally in Docker and then starts the Java application. > -> For first-run validation, troubleshooting, burn-in, side-by-side evaluation, or system profiling, run `monitor-oscar` instead. The monitor path intentionally creates additional logs, snapshots, JFR checks, thread dumps, and database trend files so operators can evaluate system behavior before or during investigation. +> Sessionless production is the preferred operating mode once the deployment has been validated. On Linux, a common pattern is `nohup ./launch-all.sh > launch.out 2>&1 &`. On Windows, the recommended hidden start pattern is a PowerShell `Start-Process` call that runs `launch-all.bat` with stdout and stderr redirected to log files. +> +> For first-run validation, troubleshooting, burn-in, side-by-side evaluation, or system profiling, run `monitor-oscar` instead. The monitor path intentionally creates additional logs, snapshots, JFR checks, thread dumps, and database trend files so operators can evaluate system behavior before or during investigation. Windows monitored runs now also have a PowerShell-first wrapper, `monitor-oscar.ps1`, which is the preferred hidden or redirected entry point for sessionless monitored launches. > > 4\. Open the application on the configured port. Port 8282 is the baseline HTTP application port, and 8443 is a representative HTTPS configuration. > @@ -129,6 +131,60 @@ Installation is intentionally simple: download a release archive, extract it, en

Operational note

-

If multiple extracted versions exist on the same machine, explicitly stop the old OSCAR JVM and remove the old `oscar-postgis-container` before launching the new package. Reusing an old container or old extracted directory can populate data in the wrong place and create confusion during configuration or tuning.

Operational note

-

If multiple extracted versions exist on the same machine, use the stop-all script to fully stop the application and Dockerized database for the version you were running. An older running container can accidentally populate data in the wrong directory and create confusion during configuration or tuning.

Operational note

-

If multiple extracted versions exist on the same machine, use the stop-all script to fully stop the application and Dockerized database for the version you were running. An older running container can accidentally populate data in the wrong directory and create confusion during configuration or tuning. If stale lanes still appear after a reset, stop the monitor, delete the entire extracted OSCAR folder, unzip a fresh copy, recreate .env, and start again. On Linux, sudo rm -rf may be required because some files may be left owned by root.
+## Sessionless and restart-safe launch patterns + +For normal production use, `launch-all` should be treated as the default startup path, whether the system is started interactively or headlessly. + +### Linux production + +Interactive: + +```bash +./launch-all.sh +``` + +Sessionless: + +```bash +nohup ./launch-all.sh > launch.out 2>&1 & +``` + +For automatic restart after reboot, the recommended pattern is a small `systemd` unit that runs `launch-all.sh` after `docker.service` is available. + +### Windows production + +Interactive: + +```bat +launch-all.bat +``` + +Sessionless from PowerShell: + +```powershell +Start-Process powershell.exe ` + -ArgumentList '-NoProfile','-ExecutionPolicy','Bypass','-Command',"Set-Location -LiteralPath '$PWD'; .\launch-all.bat" ` + -WindowStyle Hidden ` + -RedirectStandardOutput "$PWD\launch.out" ` + -RedirectStandardError "$PWD\launch.err" +``` + +For automatic restart after reboot, the recommended pattern is a **Task Scheduler** task with an **At startup** trigger that runs `launch-all.bat` from the extracted OSCAR directory after Docker is available. + +### Windows monitored diagnostics + +For monitored validation or troubleshooting, the preferred Windows wrapper is `monitor-oscar.ps1`. + +```powershell +Start-Process powershell.exe ` + -ArgumentList '-NoProfile','-ExecutionPolicy','Bypass','-File',"$PWD\monitor-oscar.ps1" ` + -WindowStyle Hidden ` + -RedirectStandardOutput "$PWD\monitor.out" ` + -RedirectStandardError "$PWD\monitor.err" +``` + +This keeps the monitoring window hidden while preserving redirected output for later review. + ## Ports and first-run behaviors | **Area** | **Typical value or behavior** | **Interpretation** | @@ -398,11 +454,15 @@ Default deployment begins to strain as lane counts, event volume, and camera cou A practical target profile for scaled deployments is 640x480 at about 5 frames per second, while 1080p may be acceptable on smaller systems. Camera settings should be chosen with total system scale in mind. For a 50-lane or 100-camera target, the aggregate cost of 1080p video is likely excessive. +## MediaMTX as a scale helper + +When many logical lanes reuse a smaller number of physical camera feeds, MediaMTX can reduce the burden on the OSCAR Java backend by replacing many direct RTSP sessions with a smaller set of stable local proxy paths. In practice, that lowers Java-side reconnect churn, reduces repeated camera-side socket activity, and simplifies lane CSV reuse in larger systems. + ## Launch mode selection for operations Use `monitor-oscar` when deployment teams need evidence: first-run validation, burn-in testing, troubleshooting, camera-profile comparisons, memory and thread profiling, or PostgreSQL session analysis. The monitor wrapper is intentionally verbose and creates monitor directories, periodic snapshots, thread dumps, JFR status checks, and database trend files. -Use `launch-all` for efficient production operation once the system profile has been validated. This keeps routine startup simple and avoids collecting detailed monitoring artifacts when no in-depth system profile is required. +Use `launch-all` for efficient production operation once the system profile has been validated. This keeps routine startup simple and avoids collecting detailed monitoring artifacts when no in-depth system profile is required. In production, prefer a sessionless `launch-all` start or an automated restart-safe wrapper such as `systemd` on Linux or Task Scheduler on Windows. ## What stays where in a split deployment diff --git a/dist/documentation/OSCAR_launch_monitoring_guide.md b/dist/documentation/OSCAR_launch_monitoring_guide.md index 72e9108..cfbe379 100644 --- a/dist/documentation/OSCAR_launch_monitoring_guide.md +++ b/dist/documentation/OSCAR_launch_monitoring_guide.md @@ -1,607 +1,761 @@ -# OSCAR launch, monitoring, and database diagnostics guide +# OSCAR Launch, Monitoring, and Status Guide -This guide covers the current **OSCAR 3.5.1 packaged deployment workflow** for Linux and Windows. +This guide explains the purpose and use of the updated `.env`, `launch-all`, `launch`, `monitor-oscar`, and `check-oscar-status` scripts for both Linux and Windows. It covers profiles, dependencies, startup and shutdown flow, cleanup, and how to interpret the monitoring output. -It explains: +## 1. What changed and why -- how to prepare a fresh prebuilt release -- how to create `.env` -- how the updated `launch-all`, `launch`, `monitor-oscar`, `stop-all`, `reset-all`, and `check-oscar-status` scripts behave -- how already-running OSCAR instances are handled -- how to validate Java, Docker, memory, and database health -- how to use MediaMTX and status reports during testing, troubleshooting, side-by-side evaluation, and system profiling +The updated scripts were designed to make OSCAR easier to run, safer on smaller machines, and easier to diagnose when memory or stability problems appear. ---- - -## 1. Recommended operating model +The main improvements are: -Use two launch paths depending on the operational goal. +- **Profile-based sizing** so Java and PostgreSQL use settings that fit the machine. +- **Safer defaults** for a 16 GB machine and other profiles. +- **Separation of responsibilities**: + - `.env` holds deployment settings. + - `launch-all` starts PostgreSQL and then launches OSCAR. + - `launch` starts the Java node with the right memory and diagnostics. + - `monitor-oscar` starts OSCAR and continuously collects diagnostic snapshots. + - `check-oscar-status` summarizes all collected data into a single report file. +- **Built-in diagnostics** such as Native Memory Tracking and JFR support. +- **Cleaner testing workflow** so you can compare runs and determine whether memory is stable or leaking. -For **efficient production operation**, use the top-level `launch-all` script: +--- -- `launch-all.sh` / `launch-all.bat` +## 2. Which files are involved -`launch-all` starts PostGIS and OSCAR with the selected `.env` system profile and avoids the additional monitor loop, snapshot files, JFR checks, thread dumps, database trend logging, and monitor directories that are only useful when an in-depth profile is needed. +### Shared configuration -For **testing, burn-in, side-by-side evaluation, troubleshooting, and system profiling**, use the `monitor-oscar` wrapper: +- `.env` -- `monitor-oscar.sh` / `monitor-oscar.bat` +### Linux -The preferred monitored workflow is: +- `launch-all.sh` +- `osh-node-oscar/launch.sh` +- `monitor-oscar.sh` +- `check-oscar-status.sh` -1. unzip the prebuilt release into a fresh folder -2. create `.env` -3. verify **Java 21+** and **Docker** -4. start with the **monitoring script** -5. let the system warm up -6. run the **status-check script** -7. review JVM, thread, and PostgreSQL behavior before production use -8. switch routine production starts to `launch-all` once detailed profiling is no longer needed +### Windows -Avoid launching `osh-node-oscar/launch.(sh|bat)` directly unless you are debugging the node itself. +- `launch-all.bat` +- `osh-node-oscar\launch.bat` +- `monitor-oscar.bat` +- `monitor-oscar.ps1` +- `check-oscar-status.ps1` --- -## 2. Fresh install and upgrade cleanup +## 3. What each file does -If the machine has previously run OSCAR, clean up the old deployment before extracting **OSCAR 3.5.1**. +## `.env` -### Linux +This file is the shared configuration layer. It tells the scripts which profile to use, how to connect to PostgreSQL, and which passwords to pass into the Java process. -```bash -pgrep -af 'com.botts.impl.security.SensorHubWrapper' -kill +Typical variables: -docker rm -f oscar-postgis-container -docker network rm oscar-postgis-network || true -rm -rf /path/to/old/oscar-3.5.0 +```env +SYSTEM_PROFILE=16GB +DB_NAME=gis +DB_USER=postgres +DB_PASSWORD=postgres +DB_PORT=5432 +DB_HOST=localhost +CONTAINER_NAME=oscar-postgis-container +KEYSTORE_PASSWORD=CHANGE_ME +TRUSTSTORE_PASSWORD=CHANGE_ME +JAVACPP_MAX_BYTES= +JAVACPP_MAX_PHYSICAL_BYTES= +JFR_FILENAME= ``` -### Windows PowerShell +Why it is useful: -```powershell -Get-CimInstance Win32_Process | - Where-Object { - $_.Name -match '^java(\.exe)?$' -and - $_.CommandLine -like '*com.botts.impl.security.SensorHubWrapper*' - } | - Select-Object ProcessId, CommandLine - -Stop-Process -Id -Force - -docker rm -f oscar-postgis-container -docker network rm oscar-postgis-network -Remove-Item -Recurse -Force .\oscar-3.5.0 -``` +- keeps machine-specific configuration out of the launch scripts +- lets you switch between profiles without editing multiple files +- makes Linux and Windows setups consistent -If the Docker network does not exist, that is fine. The goal is to avoid carrying old container state or an old extracted release folder into the new test run. +How to modify it: -For a full local reset between side-by-side test installs, use `reset-all.sh` or `reset-all.bat`. +- change `SYSTEM_PROFILE` when moving to a different machine size +- change the DB settings if PostgreSQL is not local +- change the passwords to match your real keystore and truststore values +- optionally override JavaCPP or JFR paths only when needed --- -## 3. Required dependencies +## `launch-all.sh` / `launch-all.bat` -### Linux +This is the top-level launcher. It reads `.env`, starts the PostGIS container with profile-appropriate settings, waits for PostgreSQL to become ready, and then starts OSCAR by calling the node-specific `launch` script. -Required: +Why it is useful: -- Bash -- Java 21 or newer -- `keytool` -- Docker +- one command starts the full stack +- PostgreSQL settings are tied to the profile +- ensures the DB is available before Java starts +- helps keep tests repeatable -Recommended for monitoring: +What it usually does: -- `jcmd` -- `pmap` -- `free` -- `vmstat` +1. reads `.env` +2. maps `SYSTEM_PROFILE` to PostgreSQL settings +3. builds or starts the PostGIS container +4. waits for `pg_isready` +5. enters `osh-node-oscar` +6. runs `launch.sh` or `launch.bat` -Ubuntu example: +How to modify it: -```bash -sudo apt update -sudo apt install openjdk-21-jdk docker.io procps psmisc -``` +- change PostgreSQL memory settings if your workload changes +- change container name or port if you need multiple local deployments +- change image/tag names if your Docker workflow changes -Verify: +Important note: -```bash -java -version -which keytool -which docker -which jcmd -``` +If you change PostgreSQL settings, you need the container to be recreated or restarted in a way that actually applies the new settings. The updated launchers were designed to make that more predictable. -### Windows +--- -Required: +## `osh-node-oscar/launch.sh` / `osh-node-oscar/launch.bat` -- PowerShell -- Java 21 or newer -- Docker Desktop or Docker Engine +This script starts the Java node itself. It reads `.env`, maps the selected profile to Java memory settings, sets up certificates, enables diagnostics, and launches the OSCAR process. -Recommended: +Why it is useful: -- `jcmd.exe` +- central place for Java sizing +- keeps profile logic out of the top-level launcher +- enables Native Memory Tracking so memory problems can be investigated later +- passes JavaCPP limits to help control native memory behavior -Verify: +Typical responsibilities: -```powershell -java -version -docker version -Get-Command java -Get-Command docker -Get-Command jcmd -``` +- choose `-Xms` and `-Xmx` from `SYSTEM_PROFILE` +- set `-XX:+UnlockDiagnosticVMOptions` +- set `-XX:NativeMemoryTracking=summary` +- set JavaCPP limits such as: + - `-Dorg.bytedeco.javacpp.maxBytes=...` + - `-Dorg.bytedeco.javacpp.maxPhysicalBytes=...` +- set keystore and truststore paths +- start `com.botts.impl.security.SensorHubWrapper` -The Windows launchers now use **PowerShell/CIM**-based process discovery. They do not depend on `wmic`. +How to modify it: -### Important dependency policy +- adjust profile memory values if testing shows a machine can safely handle more or needs less +- change JavaCPP limits if native memory use is too tight or too loose +- update certificate paths if the install layout changes +- add temporary JVM flags for debugging -The updated scripts distinguish between **required** dependencies and **optional** runtime extras. +What not to remove: -Hard failures are for things such as: +- `-XX:+UnlockDiagnosticVMOptions` +- `-XX:NativeMemoryTracking=summary` -- Java -- Docker -- `keytool` where the script needs it -- required packaged files and directories such as `osh-node-oscar/lib` +These are required for native memory inspection with `jcmd`. -Warning-only cases include: +--- -- missing `.env` when defaults are available -- missing `trusted_certificates` -- missing `nativelibs` -- missing optional monitoring helpers such as `pmap` or `vmstat` +## `monitor-oscar.sh` / `monitor-oscar.bat` / `monitor-oscar.ps1` -A missing `nativelibs` directory no longer stops startup by itself. +This is the diagnostic runner. It starts OSCAR, waits for the JVM to appear, starts JFR, and collects periodic snapshots into a timestamped `oscar-monitor-*` directory. On Windows, `monitor-oscar.ps1` is the preferred PowerShell wrapper for hidden or redirected monitored launches. ---- +Why it is useful: -## 4. Environment file setup +- gives you time-series data instead of one-off guesses +- captures memory, swap or pagefile, threads, and JVM info while OSCAR is running +- makes it easy to compare startup, steady state, and failure periods +- can be used as the primary launch method when you are testing stability -Create `.env` before launch. +What it collects over time: -- if the packaged release ships **env.txt**, rename it to **.env** -- if the source tree ships **env.template**, copy it to **.env** +- JVM PID and command line +- process memory details +- thread counts +- heap information +- native memory summaries +- JFR recordings +- system memory and swap or pagefile state +- launch stdout and stderr logs -Linux: +Linux snapshots commonly include: -```bash -cp env.template .env -``` +- `/proc//status` +- `/proc//smaps_rollup` +- `pmap -x` +- `free -h` +- `vmstat` +- `jcmd VM.native_memory summary` +- `jcmd GC.heap_info` +- `jcmd JFR.check` -Windows PowerShell: +Windows snapshots commonly include the nearest equivalents through PowerShell, `tasklist`, `wmic` or CIM, performance counters, and `jcmd`. -```powershell -Copy-Item .\env.template .\.env -``` +How to modify it: -### Core settings +- change the snapshot interval if you want more or less detail +- change the match expression if the Java main class changes +- change the JFR size or age limits +- add extra OS-level commands if you want more counters -```dotenv -SYSTEM_PROFILE=16GB -DB_NAME=gis -DB_USER=postgres -DB_PASSWORD=postgres -DB_PORT=5432 -DB_HOST=localhost -CONTAINER_NAME=oscar-postgis-container -``` +--- -### Process and monitor behavior settings +## `check-oscar-status.sh` / `check-oscar-status.ps1` -```dotenv -FORCE_RESTART=0 -ATTACH_TO_EXISTING=0 -MAX_WAIT_SECONDS=300 -RETRY_MAX=120 -RETRY_INTERVAL=2 -POSTGIS_READY_DELAY=5 -``` +This is a reporting script. It reads the latest monitor directory and writes one report file that summarizes the current run. + +Why it is useful: + +- gives you one file to review or share +- compares first and latest snapshots +- shows recent trend lines +- shows whether memory is rising, flattening, or thrashing + +What it includes: + +- live process status +- live JVM state +- live JFR and NMT information +- system memory and swap or pagefile status +- first snapshot summary +- latest snapshot summary +- recent trend table +- log tails +- a quick interpretation section -### What these mean +How to modify it: -- `FORCE_RESTART=0` -> refuse to start if OSCAR is already running -- `FORCE_RESTART=1` -> stop the running OSCAR instance and start fresh -- `ATTACH_TO_EXISTING=0` -> monitor script refuses to attach to a running OSCAR process -- `ATTACH_TO_EXISTING=1` -> monitor script attaches to the running OSCAR process instead of replacing it -- `MAX_WAIT_SECONDS` -> how long monitor scripts wait for the Java process to appear -- `RETRY_MAX`, `RETRY_INTERVAL`, `POSTGIS_READY_DELAY` -> PostGIS readiness timing +- change how many recent snapshots are included +- add custom grep or PowerShell parsing for errors you care about +- add application log searches for reconnect loops or parse failures --- -## 5. Profile-based sizing +## 4. Choosing the right profile -The launchers size Java and PostgreSQL by `SYSTEM_PROFILE`. +`SYSTEM_PROFILE` is the most important setting in `.env`. -Supported profiles: +### Recommended meanings -- `RPI4` -- `8GB` -- `16GB` -- `32GB` +- `RPI4`: very constrained system +- `8GB`: small development or reduced-workload machine +- `16GB`: reasonable full-node starting point +- `32GB`: larger system with more headroom -Representative PostgreSQL `max_connections` values: +### How to make sure you use the right profile -- `RPI4` -> 75 -- `8GB` -> 125 -- `16GB` -> 200 -- `32GB` -> 300 +Use the profile that matches the **actual machine memory**, not what you hope the workload can handle. -The launchers also set: +Linux: -- `superuser_reserved_connections=10` -- `idle_session_timeout=600000` -- connection and disconnection logging +```bash +free -h +``` ---- +Windows PowerShell: -## 6. Launch scripts and what they do +```powershell +Get-CimInstance Win32_ComputerSystem | Select-Object TotalPhysicalMemory +``` -### `launch-all.sh` / `launch-all.bat` +General rule: -These are the supported top-level launchers and the preferred path for efficient production operation when in-depth monitoring is not required. +- use `16GB` only on a machine with about 16 GB RAM +- use `8GB` on smaller test systems +- use `32GB` only when the machine really has that headroom +- when in doubt, choose the **smaller** profile first -They: +### Why this matters -- load `.env` when present -- validate Java and Docker -- detect an already-running OSCAR instance -- size PostgreSQL for the selected profile -- rebuild or reuse the PostGIS image -- remove the existing named PostGIS container if necessary -- start a new PostGIS container with the current settings -- wait for PostgreSQL readiness -- call `osh-node-oscar/launch.(sh|bat)` +The Java heap is not the only consumer of memory. OSCAR also uses: -### `osh-node-oscar/launch.sh` / `osh-node-oscar/launch.bat` +- native libraries +- thread stacks +- PostgreSQL +- Docker +- OS page cache +- FFmpeg or JavaCPP native memory if video is enabled -These launch only the OSCAR Java node. +A machine can fail even when Java heap is not full if native memory and database memory are too large. -They: +--- -- load `.env` when present -- validate Java and `keytool` -- detect an already-running OSCAR instance -- choose heap and JavaCPP settings for the selected profile -- build or refresh the Java trust store -- initialize the packaged admin password flow -- start the Java process with Native Memory Tracking enabled +## 5. Recommended profile behavior -Optional runtime extras are handled gracefully: +The modified launchers use conservative sizing. A reasonable starting strategy is: -- if `nativelibs` exists, the launcher adds `java.library.path` -- if `nativelibs` does not exist, the launcher warns and continues -- if `trusted_certificates` does not exist, the trust-store helper uses the copied default Java `cacerts` store and continues +- smaller `Xms` than `Xmx` +- conservative PostgreSQL settings on shared hosts +- Native Memory Tracking enabled +- JFR started by the monitor rather than by the Java launcher -Use these direct node launchers mainly for debugging. For production, use `launch-all`; for monitored validation or profiling, use `monitor-oscar`. +If a profile proves stable for your workload, you may increase it carefully. Do not scale memory up just because RAM exists. --- -## 7. How already-running OSCAR instances are handled +## 6. Installing dependencies -### Launch behavior +## Linux dependencies -By default, if an OSCAR JVM is already running, `launch-all` and `launch` stop and print an error instead of silently starting another instance. +You normally need: -Typical message: +- Java **JDK**, not just a JRE +- Docker +- Bash +- `jcmd` (comes with the JDK) +- `pg_isready` inside the PostgreSQL container image +- optional helpers: `pmap`, `vmstat`, `free` -```text -OSCAR is already running with PID(s): ... -Stop the running instance first, or set FORCE_RESTART=1 to replace it. +### Ubuntu or Debian example + +```bash +sudo apt update +sudo apt install -y openjdk-21-jdk docker.io procps psmisc ``` -### Force replacement +Optional but useful: -Set: +```bash +sudo apt install -y net-tools sysstat +``` + +### Check whether dependencies are installed on Linux -```dotenv -FORCE_RESTART=1 +```bash +java -version +action="jcmd"; command -v "$action" +docker --version +bash --version +free -h +vmstat 1 1 +pmap $$ | head ``` -Then the scripts attempt to stop the running OSCAR process before starting a new one. +--- + +## Windows dependencies + +You normally need: -### Monitor attach behavior +- Java **JDK**, not only a JRE +- Docker Desktop or a Docker Engine setup that provides `docker` +- PowerShell for the status script +- `jcmd.exe` from the JDK -For monitor wrappers, you have two supported choices: +### Check whether dependencies are installed on Windows PowerShell -- `FORCE_RESTART=1` -> replace the running OSCAR instance and monitor the new one -- `ATTACH_TO_EXISTING=1` -> keep the running OSCAR instance and attach monitoring to it +```powershell +java -version +gcm jcmd +docker --version +$PSVersionTable.PSVersion +``` + +If `gcm jcmd` does not return anything, the JDK `bin` directory is probably not on `PATH`. + +Typical `jcmd.exe` location: + +```text +C:\Program Files\Java\jdk-\bin\jcmd.exe +``` --- -## 8. Starting OSCAR +## 7. How to start the program -### Efficient production start with `launch-all` +Use `launch-all` for normal production after validation. Use `monitor-oscar` when you want detailed diagnostics, a monitor directory, and a one-file status report later. -Use this path for normal production operation after the deployment has already been validated. +## Linux normal startup -#### Linux +Interactive production start: ```bash ./launch-all.sh ``` -#### Windows +Preferred sessionless production start: -```bat -launch-all.bat +```bash +nohup ./launch-all.sh > launch.out 2>&1 & ``` -This starts the database and OSCAR application without creating the extra monitoring artifacts used for diagnostic profiling. It is the ideal production mode when operators do not need detailed memory, thread, JFR, and PostgreSQL trend data for the run. - -### Testing, troubleshooting, and profiling start with monitoring +## Linux monitored startup -Use this path for first-run validation, burn-in, side-by-side comparisons, troubleshooting, and system profiling. - -#### Linux +Interactive monitored start: ```bash ./monitor-oscar.sh ``` -Preferred Linux sessionless monitored launch: +Preferred sessionless monitored start: ```bash nohup ./monitor-oscar.sh > monitor.out 2>&1 & ``` -The packaged Linux build now marks all shipped `*.sh` files executable. If your unzip tool strips execute bits, restore them with: +## Windows normal startup -```bash -chmod +x *.sh osh-node-oscar/*.sh -``` - -#### Windows +From the project root, interactive production start: ```bat -monitor-oscar.bat +launch-all.bat ``` -Preferred Windows sessionless monitored launch from PowerShell: +Preferred sessionless production start from PowerShell: ```powershell -Start-Process -WindowStyle Hidden -FilePath cmd.exe -ArgumentList '/c monitor-oscar.bat ^> monitor.out 2^>^&1' -WorkingDirectory (Get-Location) +Start-Process powershell.exe ` + -ArgumentList '-NoProfile','-ExecutionPolicy','Bypass','-Command',"Set-Location -LiteralPath '$PWD'; .\launch-all.bat" ` + -WindowStyle Hidden ` + -RedirectStandardOutput "$PWD\launch.out" ` + -RedirectStandardError "$PWD\launch.err" ``` -This creates an output directory such as: +## Windows monitored startup -```text -oscar-monitor-20260505-032622 +Interactive monitored start from PowerShell: + +```powershell +.\monitor-oscar.ps1 ``` -and captures: +Preferred sessionless monitored launch: -- launch stdout and stderr -- JVM PID information -- JFR status -- GC heap information -- native memory summaries -- thread dumps -- Docker status -- PostgreSQL session and activity data -- trend CSV files for database sessions +```powershell +Start-Process powershell.exe ` + -ArgumentList '-NoProfile','-ExecutionPolicy','Bypass','-File',"$PWD\monitor-oscar.ps1" ` + -WindowStyle Hidden ` + -RedirectStandardOutput "$PWD\monitor.out" ` + -RedirectStandardError "$PWD\monitor.err" +``` -Because these artifacts are intentionally verbose, use `monitor-oscar` only when the extra evidence is useful. For steady production where no in-depth system profile is needed, return to `launch-all`. +`monitor-oscar.bat` remains available for interactive `cmd.exe` use, but `monitor-oscar.ps1` is the preferred Windows wrapper for hidden or redirected monitored runs. ---- +## Production auto-start after reboot -## 9. Stopping and resetting OSCAR +### Linux -### `stop-all` +Use a small `systemd` unit that runs `launch-all.sh` after Docker is available. -The `stop-all` scripts now begin by asking the monitor to stop first. +Example `/etc/systemd/system/oscar-launch-all.service`: -### Monitor singleton protection +```ini +[Unit] +Description=OSCAR production launcher +After=network-online.target docker.service +Wants=network-online.target docker.service -`monitor-oscar.sh` and `monitor-oscar.bat` now refuse duplicate monitor starts. +[Service] +Type=oneshot +RemainAfterExit=yes +User=oscar +WorkingDirectory=/home/oscar/oscar-3.5.1 +ExecStart=/bin/bash -lc './launch-all.sh' +ExecStop=/bin/bash -lc './stop-all.sh' +TimeoutStartSec=0 -Typical behavior: +[Install] +WantedBy=multi-user.target +``` -- a second monitor launch prints a clear error and exits -- the message tells the operator to run `stop-all` or the monitor stop command first -- stale monitor lock state from an unclean exit is cleaned up automatically when no live monitor process is found -- `monitor.last-status` is updated on every start, stop, refusal, and JVM exit -- `monitor.last-error` records the latest actionable failure text +Enable it: -This closes the gap where OSCAR already protected the backend process, but the monitor wrapper could still be started twice. +```bash +sudo systemctl daemon-reload +sudo systemctl enable --now oscar-launch-all.service +``` -For sessionless launches such as `nohup`, a hidden `cmd.exe`, Task Scheduler, or a service wrapper, operators should check: +### Windows -- `monitor.last-status` -- `monitor.last-error` -- `monitor.out` if stdout and stderr were redirected there +Use **Task Scheduler** with these settings: -Behavior: +- Trigger: **At startup** +- Run whether user is logged on or not +- Run with highest privileges +- Action: `powershell.exe` +- Arguments: -- attempt to stop the monitor -- do not wait indefinitely for monitor exit -- continue with direct fallback shutdown of OSCAR and PostGIS if needed +```text +-NoProfile -ExecutionPolicy Bypass -Command "Set-Location -LiteralPath 'C:\path\to\oscar-3.5.1'; .\launch-all.bat *> .\launch-startup.log" +``` -This avoids `stop-all` getting stuck on a monitor-closing attempt. +Also make sure Docker Desktop or the Windows Docker service is configured to start automatically before OSCAR is expected to launch. -### `reset-all` +## 8. How to stop the program -Use `reset-all` when you want a clean local test surface before trying a different packaged installation on the same machine. It is mainly a testing and troubleshooting tool, not part of routine production startup. +## Linux -It: +If you started with `monitor-oscar.sh` and the script was written to stop the whole stack on signal: -- asks the monitor to stop first -- stops OSCAR Java processes -- removes the PostGIS container and volumes -- clears local runtime state used by the packaged installation +```bash +kill "$(cat monitor.pid)" +``` -Linux: +If needed, stop parts manually: ```bash -./reset-all.sh +pgrep -af 'com.botts.impl.security.SensorHubWrapper' +kill +docker stop oscar-postgis-container ``` -Windows: +## Windows + +If the Windows monitor script supports a stop command: ```bat -reset-all.bat +monitor-oscar.bat stop +``` + +Otherwise stop the Java process and container manually: + +PowerShell: + +```powershell +Get-Process java | Stop-Process + docker stop oscar-postgis-container ``` +Be careful if multiple Java processes are running on the machine. + --- -## 10. Status reports +## 9. How to check that the monitor is working -### Linux +## Linux ```bash -./check-oscar-status.sh +pgrep -af monitor-oscar.sh +pgrep -af 'com.botts.impl.security.SensorHubWrapper' +docker ps --filter name=oscar-postgis-container +ls -td oscar-monitor-* | head -n 1 +tail -f monitor.out +cat monitor.last-status 2>/dev/null || true +cat monitor.last-error 2>/dev/null || true ``` -### Windows +## Windows PowerShell ```powershell -powershell -ExecutionPolicy Bypass -File .\check-oscar-status.ps1 +Get-Process java +Get-ChildItem . -Directory oscar-monitor-* | Sort-Object LastWriteTime -Descending | Select-Object -First 1 +Get-Content .\monitor.out -Tail 50 -Wait +Get-Content .\monitor.err -Tail 50 -Wait +Get-Content .\monitor.last-status -ErrorAction SilentlyContinue +Get-Content .\monitor.last-error -ErrorAction SilentlyContinue ``` -These scripts are most useful after a monitored validation or profiling run. They summarize the latest monitor run into a single text report that includes: +The singleton guard in the monitor wrappers prevents a second live monitor from starting. For sessionless launches, `monitor.last-status`, `monitor.last-error`, `monitor.out`, and `monitor.err` are the fastest way to see whether the wrapper attached successfully, refused a duplicate launch, or exited because OSCAR was already running. -- process status -- live JVM information -- heap and native memory summaries -- current container status -- PostgreSQL activity snapshots -- first-versus-latest trend comparison -- recent log tails +## 10. How to generate a one-file status report ---- +## Linux -## 11. What healthy startup looks like +```bash +./check-oscar-status.sh +``` -A healthy first-run profile typically looks like this: +This produces a file like: -- Java RSS rises during startup and then levels out -- thread count rises during startup and then stabilizes -- PostgreSQL sessions rise during startup and then plateau well below usable client slots -- swap or pagefile usage stays low -- `db-error` remains empty +```text +oscar-status-20260504-000101.txt +``` -### Important PostgreSQL rule +## Windows -```text -usable client slots = max_connections - superuser_reserved_connections +```powershell +powershell -ExecutionPolicy Bypass -File .\check-oscar-status.ps1 ``` -If total sessions keep climbing toward that number, PostgreSQL is nearing saturation. +This produces a similar one-file report. --- -## 12. Interpreting the monitor output +## 11. How to analyze the data + +The most important sections are: + +- `LIVE JVM /proc STATUS` or the Windows live process section +- `LIVE JVM NATIVE MEMORY SUMMARY` +- `LIVE JVM GC HEAP INFO` +- `RECENT TREND` +- `vmstat` on Linux or pagefile/commit counters on Windows +- application log tails + +### Healthy pattern + +A healthy run usually looks like this: + +- RSS rises during startup, then flattens +- heap usage rises and falls with normal GC activity +- NMT committed memory stays in a narrow band +- thread count stabilizes +- swap or pagefile use may rise some but stops growing +- system still has plenty of available memory +- there is little or no sustained swap-in and swap-out pressure + +### Suspicious pattern + +A suspicious run usually looks like this: + +- RSS rises hour after hour without flattening +- VmSwap or pagefile keeps increasing steadily +- NMT committed memory keeps rising steadily +- thread count keeps climbing +- logs show reconnect loops and memory rises after each one +- system available memory keeps shrinking +- OS starts heavy paging or swapping activity + +### Linux-specific interpretation tips + +- `VmRSS`: resident memory in RAM +- `VmSwap`: memory for that process currently swapped out +- `vmstat si/so`: swap in and swap out activity +- `GC.heap_info`: whether Java heap is actually pressured +- `VM.native_memory summary`: whether JVM-managed native memory is rising -Key files in a monitor directory: +### Windows-specific interpretation tips -- `launch.stdout.log` -- `launch.stderr.log` -- `jvm-pid.txt` -- `db-connection-trend.csv` -- one timestamped snapshot directory per interval +Watch these especially: -Key per-snapshot files: +- process working set +- private bytes or commit size +- system commit charge versus commit limit +- pagefile usage +- `jcmd VM.native_memory summary` -- `nmt-summary.txt` -- `gc-heap-info.txt` -- `thread-print.txt` -- `db-max-connections.txt` -- `db-total-sessions.txt` -- `db-by-state.txt` -- `db-by-app.txt` -- `db-activity-detail.txt` -- `docker-logs-tail.txt` +### Important distinction -Use `db-connection-trend.csv` as the fastest way to spot connection growth, plateauing, or saturation. +A process can fail from **native memory exhaustion** even when Java heap is not full. That was the original reason these scripts were added. --- -## 13. MediaMTX during field testing +## 12. How to tell whether there is a leak -For larger camera configurations or side-by-side test deployments: +Do not judge from the first hour alone. Startup always causes growth. -1. start OSCAR with `monitor-oscar` for evaluation and profiling -2. route camera streams through MediaMTX -3. let the system run long enough to capture reconnect and thread behavior -4. run `check-oscar-status` -5. compare JVM threads, reconnect logs, and PostgreSQL sessions before and after enabling MediaMTX -6. use `launch-all` for efficient production operation once the camera profile is accepted +Suggested checkpoints: -MediaMTX is especially helpful when many logical lane-camera assignments reuse a smaller number of real camera streams. +- **30 to 60 minutes**: look for obvious runaway behavior +- **2 to 4 hours**: see whether memory is leveling off +- **12 to 24 hours**: determine whether the process is stable or slowly drifting + +What proves stability: + +- recent trend lines become narrow and flat +- NMT committed memory stays near one range +- threads stay near one range +- swap or pagefile stops rising + +What suggests a leak: + +- all trend lines keep climbing across many hours +- the slope stays positive even after warmup +- memory jumps after every reconnect or retry cycle and never comes down --- -## 14. Troubleshooting checklist +## 13. When to delete files + +The monitor and status scripts generate files that can grow over time. + +### Files you can delete safely after a run is complete + +- old `oscar-status-*.txt` reports +- old `monitor.out` +- old monitor directories such as `oscar-monitor-20260503-174333` +- old JFR files that you no longer need + +### Files you should keep while investigating a problem + +- the monitor directory for the run you care about +- its `launch.stdout.log` and `launch.stderr.log` +- any `*.jfr` files +- any JVM crash logs such as `hs_err_pid*.log` + +### Good cleanup practice -### Launch fails before Java starts +Delete old monitor directories only after: -Check: +- the run has been reviewed +- any useful JFR files have been copied somewhere safe +- you no longer need to compare against older runs -- `.env` exists if you intend to override defaults -- Java 21+ is installed -- Docker is running -- required directories such as `osh-node-oscar/lib` exist -- the trust store and keystore files exist where the launch script expects them +Linux cleanup example: -Remember: +```bash +rm -rf oscar-monitor-20260503-174333 +rm -f oscar-status-*.txt +``` + +Windows PowerShell cleanup example: -- missing `nativelibs` is warning-only -- missing `trusted_certificates` is warning-only if the default Java trust store can be copied successfully +```powershell +Remove-Item .\oscar-monitor-20260503-174333 -Recurse -Force +Remove-Item .\oscar-status-*.txt +``` -### Monitor reports that another monitor is already running +--- -This is expected if a previous `monitor-oscar` session is still active. +## 14. How to modify the scripts safely -Use: +When changing the scripts, change one category at a time: -- `./stop-all.sh` or `monitor-oscar.bat stop` / `./monitor-oscar.sh stop` to stop the existing monitor cleanly -- the same monitor start command again after the first monitor exits +1. profile memory sizes +2. PostgreSQL memory settings +3. monitoring interval +4. extra diagnostics +5. container or path settings -If the earlier monitor was terminated abruptly, the next start should clean up stale monitor lock state automatically. +After each change, run a monitored test and compare the new `oscar-status-*.txt` report against an older stable run. -### Monitor hangs waiting for Java +Do not change everything at once or you will not know what helped. -Check `launch.stdout.log` and `launch.stderr.log` inside the newest monitor directory. +--- -Common causes: +## 15. Recommended workflow -- PostGIS container startup failed -- the OSCAR Java process exited immediately after launch -- a required runtime path is missing -- a certificate, trust store, or password-initialization step failed +For a new machine: -### PostgreSQL sessions keep climbing +1. put the correct `.env` in place +2. verify dependencies +3. verify the chosen `SYSTEM_PROFILE` +4. start with `monitor-oscar` +5. let it run at least 2 to 4 hours +6. generate a status report +7. check whether RSS, swap or pagefile, NMT committed, and threads plateau +8. only then decide whether to raise or lower memory settings -Inspect: +For production confidence: -- `db-total-sessions.txt` -- `db-by-state.txt` -- `db-by-app.txt` -- `db-activity-detail.txt` -- `db-connection-trend.csv` +1. run monitored overnight +2. generate a final status report +3. verify that recent trend lines are flat +4. archive one known-good monitor directory and status report for comparison -### Thread count keeps climbing +--- -Inspect: +## 16. Common mistakes to avoid -- `thread-print.txt` -- reconnect-related warnings in `launch.stdout.log` -- MediaMTX versus direct-camera behavior under the same test workload +- using a profile larger than the machine really supports +- assuming Java heap is the only memory that matters +- removing NMT flags from the Java launcher +- starting JFR twice from both the launcher and the monitor without intending to +- judging a leak from startup-only growth +- deleting monitor directories before reviewing them +- forgetting that repeated reconnects can be a logic problem even when memory looks stable --- -## 15. Files you normally should not delete +## 17. Bottom line -Do not delete these unless you intentionally want to reset state: +The updated scripts give you a repeatable way to: -- `.env` -- `osh-node-oscar/db/` -- `pgdata/` -- `osh-node-oscar/osh-keystore.p12` -- `osh-node-oscar/truststore.jks` +- choose the right memory profile +- start OSCAR consistently +- capture memory diagnostics during the run +- summarize results into one report file +- distinguish between startup growth, stable operation, and a real leak + +For day-to-day use, the most important steps are: -It is safe to delete old monitor directories and generated status reports once you have kept the reports you need. +- set the correct `SYSTEM_PROFILE` +- start with `monitor-oscar` when testing +- use `check-oscar-status` to review the run +- keep the diagnostic files until you know the run is healthy diff --git a/dist/documentation/Release_Notes_3.5.1.md b/dist/documentation/Release_Notes_3.5.1.md index 592c690..058d4b0 100644 --- a/dist/documentation/Release_Notes_3.5.1.md +++ b/dist/documentation/Release_Notes_3.5.1.md @@ -151,12 +151,15 @@ This is the most important backend stability improvement in this release. New monitoring and status-check scripts were added for both Linux and Windows. +Windows now also includes `monitor-oscar.ps1` as the preferred PowerShell wrapper for sessionless monitored launches. It works cleanly with `Start-Process` redirection, keeps the window hidden, and pairs well with the existing `monitor.last-status` and `monitor.last-error` files for headless troubleshooting. + The monitor wrappers now also include a singleton guard so a second `monitor-oscar` launch is refused while another monitor is already active. This prevents duplicate snapshot loops, duplicate JFR starts, and confusing status output during sessionless operation. The wrappers now also update `monitor.last-status` and `monitor.last-error`, which makes it much easier to understand why a sessionless launch exited without staying attached to a terminal window. These scripts can now: * launch OSCAR under monitoring * support sessionless launch for validation, troubleshooting, burn-in, and profiling runs +* expose a PowerShell-first monitored entry point on Windows through `monitor-oscar.ps1` * capture JVM memory status * capture native memory tracking summaries * capture JFR status @@ -202,8 +205,9 @@ This supports a deployment model where: * a smaller number of upstream camera streams are proxied locally * multiple lanes can reuse proxied feeds * OSCAR connects to stable local endpoints instead of managing a large number of direct camera connections +* the Java backend spends less effort on direct RTSP session setup, reconnect churn, and repeated camera-side socket activity -This architecture is recommended for larger systems and appears to reduce camera-related reconnect burden. Validate camera-heavy profiles with `monitor-oscar`, then use `launch-all` for efficient production starts once the profile is accepted. +This architecture is recommended for larger systems because it reduces camera-related reconnect burden and lowers Java-side camera handling overhead. Validate camera-heavy profiles with `monitor-oscar`, then use `launch-all` for efficient production starts once the profile is accepted. ### Documentation updates @@ -400,16 +404,40 @@ For efficient production operation after validation, start OSCAR with `launch-al #### Linux production +Interactive: + ```bash ./launch-all.sh ``` +Preferred sessionless production start: + +```bash +nohup ./launch-all.sh > launch.out 2>&1 & +``` + #### Windows production +Interactive: + ```bat launch-all.bat ``` +Preferred sessionless production start from PowerShell: + +```powershell +Start-Process powershell.exe ` + -ArgumentList '-NoProfile','-ExecutionPolicy','Bypass','-Command',"Set-Location -LiteralPath '$PWD'; .\launch-all.bat" ` + -WindowStyle Hidden ` + -RedirectStandardOutput "$PWD\launch.out" ` + -RedirectStandardError "$PWD\launch.err" +``` + +#### Automated production start after restart + +Linux operators should normally use a `systemd` unit that runs `launch-all.sh` after Docker is available. Windows operators should normally use **Task Scheduler** with an **At startup** trigger that runs `launch-all.bat` from the OSCAR directory with highest privileges. This keeps the default production path on `launch-all` while avoiding reliance on an open terminal or SSH session. + Use the monitoring script when diagnostics are needed for first-run validation, burn-in, troubleshooting, side-by-side comparison, or system profiling. #### Linux monitored validation or profiling @@ -428,11 +456,23 @@ Attached interactive pattern: #### Windows monitored validation or profiling -```bat -monitor-oscar.bat +Interactive: + +```powershell +.\monitor-oscar.ps1 +``` + +Preferred sessionless PowerShell pattern: + +```powershell +Start-Process powershell.exe ` + -ArgumentList '-NoProfile','-ExecutionPolicy','Bypass','-File',"$PWD\monitor-oscar.ps1" ` + -WindowStyle Hidden ` + -RedirectStandardOutput "$PWD\monitor.out" ` + -RedirectStandardError "$PWD\monitor.err" ``` -For detached Windows operation, run `monitor-oscar.bat` from Task Scheduler, a service wrapper, or a hidden PowerShell `Start-Process` invocation. +`monitor-oscar.bat` remains available for interactive `cmd.exe` use, but `monitor-oscar.ps1` is the preferred Windows wrapper for headless monitored runs. The monitoring path intentionally produces additional logs, monitor directories, snapshots, JFR checks, thread dumps, and database trend files. Use it when that evidence is valuable; otherwise use `launch-all` for routine production. @@ -499,6 +539,7 @@ This is the preferred workflow for: * `launch-all.bat` * `launch.bat` * `monitor-oscar.bat` +* `monitor-oscar.ps1` * `check-oscar-status.ps1` --- @@ -510,6 +551,8 @@ This is the preferred workflow for: * select the correct hardware profile in `.env` * use the updated launch scripts * use **`launch-all`** for efficient production operation after validation +* use a sessionless `launch-all` start for routine production when you do not need deep diagnostics +* use `systemd` on Linux or Task Scheduler on Windows for automatic production startup after reboot * use the **sessionless monitoring launch** for initial validation, burn-in, side-by-side evaluation, troubleshooting, and profiling * use the attached monitoring launch only for interactive troubleshooting * let the scripts manage already-running instances instead of manually launching duplicates diff --git a/dist/release/monitor-oscar.bat b/dist/release/monitor-oscar.bat index 2bcb15c..4fd2538 100644 --- a/dist/release/monitor-oscar.bat +++ b/dist/release/monitor-oscar.bat @@ -1,351 +1,3 @@ @echo off -setlocal EnableExtensions EnableDelayedExpansion - -set "SCRIPT_DIR=%~dp0" -if "%SCRIPT_DIR:~-1%"=="\" set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%" -set "STATE_DIR=%SCRIPT_DIR%\.monitor-state" -set "MONITOR_LOCK_DIR=%STATE_DIR%\lock" -set "HEARTBEAT_FILE=%MONITOR_LOCK_DIR%\heartbeat.txt" -set "ACTIVE_MONITOR_FILE=%STATE_DIR%\active-monitor-dir.txt" -set "STATUS_FILE=%SCRIPT_DIR%\monitor.last-status" -set "ERROR_FILE=%SCRIPT_DIR%\monitor.last-error" -set "ENV_FILE=%SCRIPT_DIR%\.env" -set "LAUNCH_CMD=%SCRIPT_DIR%\launch-all.bat" -set "JVM_MATCH=com.botts.impl.security.SensorHubWrapper" -set "OUT_DIR=" -set "OSCAR_PID=" -set "WAITED=0" -set "STOP_REQUESTED=0" -set "CLEANUP_REASON=STOPPED" - -if not exist "%STATE_DIR%" mkdir "%STATE_DIR%" >nul 2>nul - -if /I "%~1"=="stop" goto :stop_monitor - -if exist "%ENV_FILE%" call :load_env "%ENV_FILE%" - -if not defined CONTAINER_NAME set "CONTAINER_NAME=oscar-postgis-container" -if not defined MONITOR_INTERVAL set "MONITOR_INTERVAL=60" -if not defined MAX_WAIT_SECONDS set "MAX_WAIT_SECONDS=300" -if not defined JFR_NAME set "JFR_NAME=oscar" -if not defined JFR_MAX_AGE set "JFR_MAX_AGE=4h" -if not defined JFR_MAX_SIZE set "JFR_MAX_SIZE=1g" -if not defined ATTACH_TO_EXISTING set "ATTACH_TO_EXISTING=0" -if not defined FORCE_RESTART set "FORCE_RESTART=0" - -call :write_status STARTING monitor_batch=%~nx0 -call :clear_error - -if not exist "%LAUNCH_CMD%" ( - echo Error: Missing launch command: "%LAUNCH_CMD%" - call :write_error Missing launch command: "%LAUNCH_CMD%" - call :write_status FAILED launch_command_missing path="%LAUNCH_CMD%" - exit /b 1 -) - -call :acquire_monitor_lock -if errorlevel 1 exit /b 1 - -for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "STAMP=%%I" -set "OUT_DIR=%SCRIPT_DIR%\oscar-monitor-%STAMP%" -mkdir "%OUT_DIR%" >nul 2>nul -if errorlevel 1 ( - echo Error: Could not create monitor output directory "%OUT_DIR%" - call :write_error Could not create monitor output directory "%OUT_DIR%" - call :write_status FAILED out_dir_create path="%OUT_DIR%" - call :release_monitor_lock - exit /b 1 -) - -echo %OUT_DIR%> "%ACTIVE_MONITOR_FILE%" -call :update_heartbeat -call :write_status RUNNING output="%OUT_DIR%" - -echo Monitor output: %OUT_DIR% -echo Launching OSCAR stack... - -where jcmd >nul 2>nul -if errorlevel 1 ( - echo Warning: jcmd not found. JFR and NMT snapshots will be skipped. -) - -call :check_existing_oscar -if defined OSCAR_PID ( - if /I "%ATTACH_TO_EXISTING%"=="1" ( - echo Attaching to existing OSCAR PID %OSCAR_PID%... - call :clear_error - call :write_status RUNNING attached jvm_pid=%OSCAR_PID% output="%OUT_DIR%" - goto :found_java - ) - if /I "%FORCE_RESTART%"=="1" ( - echo Existing OSCAR detected with PID %OSCAR_PID%. FORCE_RESTART=1, replacing it... - call :stop_existing_oscar - call :wait_for_oscar_stop 60 - ) else ( - echo OSCAR is already running with PID %OSCAR_PID%. - echo Set ATTACH_TO_EXISTING=1 to monitor it, or FORCE_RESTART=1 to replace it. - call :write_error OSCAR is already running with PID %OSCAR_PID%. - call :write_status FAILED oscar_already_running pid=%OSCAR_PID% output="%OUT_DIR%" - call :release_monitor_lock - exit /b 1 - ) -) - -call :start_launch -if errorlevel 1 ( - call :write_error Failed to start launch-all.bat - call :write_status FAILED launch_start output="%OUT_DIR%" - call :release_monitor_lock - exit /b 1 -) - -call :write_status WAITING_FOR_JVM output="%OUT_DIR%" -echo Waiting for JVM to appear... -set /a WAITED=0 - -:wait_for_java -call :update_heartbeat -if exist "%OUT_DIR%\stop.request" ( - set "STOP_REQUESTED=1" - set "CLEANUP_REASON=STOPPED" - goto :cleanup -) - -call :check_existing_oscar -if defined OSCAR_PID goto :found_java - -if %WAITED% GEQ %MAX_WAIT_SECONDS% ( - echo Launch timed out before JVM appeared. - call :write_error Timed out waiting for JVM after %MAX_WAIT_SECONDS%s. Check "%OUT_DIR%\launch.stdout.log" and "%OUT_DIR%\launch.stderr.log" - call :write_status FAILED wait_for_jvm_timeout output="%OUT_DIR%" - call :release_monitor_lock - exit /b 1 -) - -timeout /t 2 /nobreak >nul -set /a WAITED+=2 -goto :wait_for_java - -:found_java -echo Found JVM PID: %OSCAR_PID% -> "%OUT_DIR%\jvm-pid.txt" echo %OSCAR_PID% -call :clear_error -call :write_status RUNNING jvm_pid=%OSCAR_PID% output="%OUT_DIR%" - -powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "$p = Get-CimInstance Win32_Process -Filter \"ProcessId=%OSCAR_PID%\"; if ($p) { [System.IO.File]::WriteAllText('%OUT_DIR%\\process-info.txt', ('Timestamp: ' + (Get-Date -Format o) + [Environment]::NewLine + 'JVM PID: %OSCAR_PID%' + [Environment]::NewLine + [Environment]::NewLine + 'Command line:' + [Environment]::NewLine + $p.CommandLine)) }" >nul 2>nul - -where jcmd >nul 2>nul -if not errorlevel 1 ( - jcmd %OSCAR_PID% JFR.start name=%JFR_NAME% settings=profile disk=true maxage=%JFR_MAX_AGE% maxsize=%JFR_MAX_SIZE% filename="%OUT_DIR%\%JFR_NAME%.jfr" > "%OUT_DIR%\jfr-start.txt" 2>&1 - jcmd %OSCAR_PID% VM.native_memory baseline > "%OUT_DIR%\nmt-baseline.txt" 2>&1 -) - -call :snapshot - -echo Monitor is running. Use monitor-oscar.bat stop to stop the stack and dump final data. -:monitor_loop -call :update_heartbeat -if exist "%OUT_DIR%\stop.request" ( - set "STOP_REQUESTED=1" - set "CLEANUP_REASON=STOPPED" - goto :cleanup -) - -call :check_existing_oscar -if not defined OSCAR_PID ( - set "CLEANUP_REASON=EXITED" - goto :cleanup -) - -> "%OUT_DIR%\jvm-pid.txt" echo %OSCAR_PID% -call :snapshot -call :write_status RUNNING jvm_pid=%OSCAR_PID% output="%OUT_DIR%" -timeout /t %MONITOR_INTERVAL% /nobreak >nul -goto :monitor_loop - -:cleanup -if defined OSCAR_PID call :jfr_dump -if "%STOP_REQUESTED%"=="1" ( - echo Stop requested. -) else if /I "%CLEANUP_REASON%"=="EXITED" ( - echo JVM exited. -) - -del "%OUT_DIR%\stop.request" >nul 2>nul -call :release_monitor_lock -if /I "%CLEANUP_REASON%"=="EXITED" ( - call :write_status EXITED output="%OUT_DIR%" -) else ( - call :write_status STOPPED output="%OUT_DIR%" -) -call :clear_error -exit /b 0 - -:stop_monitor -set "MON_DIR=" -if exist "%ACTIVE_MONITOR_FILE%" set /p MON_DIR=<"%ACTIVE_MONITOR_FILE%" -if not defined MON_DIR call :find_latest_monitor_dir - -if not defined MON_DIR ( - echo OSCAR monitor is not running. - call :write_status STOP_REQUESTED no_active_monitor - call :clear_error - call :release_monitor_lock - exit /b 0 -) - -if not exist "%MON_DIR%" ( - echo OSCAR monitor is not running. - call :write_status STOP_REQUESTED no_active_monitor - call :clear_error - call :release_monitor_lock - exit /b 0 -) - -> "%MON_DIR%\stop.request" echo stop -call :write_status STOP_REQUESTED output="%MON_DIR%" -call :clear_error -echo Requested monitor stop: "%MON_DIR%" -exit /b 0 - -:start_launch -powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "$stdout = [System.IO.Path]::Combine('%OUT_DIR%','launch.stdout.log'); $stderr = [System.IO.Path]::Combine('%OUT_DIR%','launch.stderr.log'); Start-Process -WindowStyle Hidden -FilePath 'cmd.exe' -ArgumentList @('/c','call ""%LAUNCH_CMD%""') -WorkingDirectory '%SCRIPT_DIR%' -RedirectStandardOutput $stdout -RedirectStandardError $stderr | Out-Null" -if errorlevel 1 exit /b 1 -exit /b 0 - -:check_existing_oscar -set "OSCAR_PID=" -for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "$p = Get-CimInstance Win32_Process ^| Where-Object { $_.Name -match '^(java|javaw)(\\.exe)?$' -and $null -ne $_.CommandLine -and $_.CommandLine -like '*%JVM_MATCH%*' } ^| Select-Object -First 1 -ExpandProperty ProcessId; if ($p) { $p }"') do set "OSCAR_PID=%%I" -exit /b 0 - -:stop_existing_oscar -if not defined OSCAR_PID exit /b 0 -powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %OSCAR_PID% -Force -ErrorAction Stop } catch {}" >nul 2>nul -exit /b 0 - -:wait_for_oscar_stop -set "WAIT_LIMIT=%~1" -if not defined WAIT_LIMIT set "WAIT_LIMIT=60" -set /a WAITED=0 -:wait_for_oscar_stop_loop -call :check_existing_oscar -if not defined OSCAR_PID exit /b 0 -if !WAITED! GEQ %WAIT_LIMIT% exit /b 0 -timeout /t 1 /nobreak >nul -set /a WAITED+=1 -goto :wait_for_oscar_stop_loop - -:snapshot -if not defined OSCAR_PID exit /b 0 -for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "Get-Date -Format yyyyMMdd-HHmmss"') do set "SNAPSTAMP=%%I" -set "SNAP_DIR=%OUT_DIR%\%SNAPSTAMP%" -mkdir "%SNAP_DIR%" >nul 2>nul - -tasklist /FI "PID eq %OSCAR_PID%" /V > "%SNAP_DIR%\tasklist.txt" 2>&1 -powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "$p = Get-Process -Id %OSCAR_PID% -ErrorAction SilentlyContinue; if ($p) { $p | Select-Object Id,ProcessName,CPU,StartTime,WorkingSet64,PrivateMemorySize64,VirtualMemorySize64,HandleCount,@{Name='ThreadCount';Expression={$_.Threads.Count}} | Format-List * }" > "%SNAP_DIR%\process.txt" 2>&1 -powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "Get-Counter '\Memory\Committed Bytes','\Memory\Commit Limit','\Paging File(_Total)\%% Usage' | Format-List *" > "%SNAP_DIR%\memory-counters.txt" 2>&1 -powershell -NoProfile -ExecutionPolicy Bypass -Command ^ - "Get-CimInstance Win32_OperatingSystem | Select-Object TotalVisibleMemorySize,FreePhysicalMemory,TotalVirtualMemorySize,FreeVirtualMemory | Format-List *" > "%SNAP_DIR%\os-memory.txt" 2>&1 - -where jcmd >nul 2>nul -if not errorlevel 1 ( - jcmd %OSCAR_PID% VM.native_memory summary > "%SNAP_DIR%\nmt-summary.txt" 2>&1 - jcmd %OSCAR_PID% GC.heap_info > "%SNAP_DIR%\gc-heap-info.txt" 2>&1 - jcmd %OSCAR_PID% Thread.print > "%SNAP_DIR%\thread-print.txt" 2>&1 - jcmd %OSCAR_PID% JFR.check > "%SNAP_DIR%\jfr-check.txt" 2>&1 -) - -docker ps > "%SNAP_DIR%\docker-ps.txt" 2>&1 -exit /b 0 - -:jfr_dump -where jcmd >nul 2>nul -if errorlevel 1 exit /b 0 -if not defined OSCAR_PID exit /b 0 -jcmd %OSCAR_PID% JFR.dump name=%JFR_NAME% filename="%OUT_DIR%\%JFR_NAME%-final.jfr" > "%OUT_DIR%\jfr-dump-final.txt" 2>&1 -exit /b 0 - -:acquire_monitor_lock -if not exist "%STATE_DIR%" mkdir "%STATE_DIR%" >nul 2>nul -mkdir "%MONITOR_LOCK_DIR%" >nul 2>nul -if not errorlevel 1 exit /b 0 - -call :lock_is_fresh -if "%LOCK_FRESH%"=="1" ( - set "EXISTING_OUT_DIR=" - if exist "%ACTIVE_MONITOR_FILE%" set /p EXISTING_OUT_DIR=<"%ACTIVE_MONITOR_FILE%" - echo Error: Another monitor-oscar.bat instance is already running. - if defined EXISTING_OUT_DIR echo Active monitor output: %EXISTING_OUT_DIR% - echo Run stop-all.bat or monitor-oscar.bat stop before starting another monitor. - if defined EXISTING_OUT_DIR ( - call :write_error Duplicate monitor start refused. Existing output: %EXISTING_OUT_DIR% - call :write_status FAILED duplicate_monitor output="%EXISTING_OUT_DIR%" - ) else ( - call :write_error Duplicate monitor start refused. - call :write_status FAILED duplicate_monitor - ) - exit /b 1 -) - -echo Removing stale OSCAR monitor lock state. -call :release_monitor_lock -mkdir "%MONITOR_LOCK_DIR%" >nul 2>nul -if errorlevel 1 ( - call :write_error Could not acquire OSCAR monitor lock at "%MONITOR_LOCK_DIR%" - call :write_status FAILED lock_acquire path="%MONITOR_LOCK_DIR%" - exit /b 1 -) -exit /b 0 - -:lock_is_fresh -set "LOCK_FRESH=0" -if not exist "%HEARTBEAT_FILE%" exit /b 0 -for /f %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "$age = (Get-Date) - (Get-Item '%HEARTBEAT_FILE%').LastWriteTime; if ($age.TotalSeconds -lt 180) { 1 } else { 0 }"') do set "LOCK_FRESH=%%I" -exit /b 0 - -:update_heartbeat -if not exist "%MONITOR_LOCK_DIR%" mkdir "%MONITOR_LOCK_DIR%" >nul 2>nul -break> "%HEARTBEAT_FILE%" -exit /b 0 - -:release_monitor_lock -if exist "%HEARTBEAT_FILE%" del "%HEARTBEAT_FILE%" >nul 2>nul -if exist "%ACTIVE_MONITOR_FILE%" del "%ACTIVE_MONITOR_FILE%" >nul 2>nul -if exist "%MONITOR_LOCK_DIR%" rmdir "%MONITOR_LOCK_DIR%" >nul 2>nul -exit /b 0 - -:find_latest_monitor_dir -set "MON_DIR=" -for /f "delims=" %%I in ('powershell -NoProfile -ExecutionPolicy Bypass -Command "$d = Get-ChildItem -LiteralPath '%SCRIPT_DIR%' -Directory -Filter 'oscar-monitor-*' ^| Sort-Object Name -Descending ^| Select-Object -First 1 -ExpandProperty FullName; if ($d) { $d }"') do set "MON_DIR=%%I" -exit /b 0 - -:load_env -for /f "usebackq tokens=* delims=" %%L in ("%~1") do ( - set "LINE=%%L" - if defined LINE ( - if not "!LINE:~0,1!"=="#" ( - for /f "tokens=1,* delims==" %%A in ("!LINE!") do ( - if not "%%A"=="" set "%%A=%%B" - ) - ) - ) -) -exit /b 0 - -:write_status -set "STATUS_TEXT=%*" -> "%STATUS_FILE%" echo %date% %time% %STATUS_TEXT% -exit /b 0 - -:write_error -set "ERROR_TEXT=%*" -> "%ERROR_FILE%" echo %date% %time% %ERROR_TEXT% -exit /b 0 - -:clear_error -> "%ERROR_FILE%" type nul -exit /b 0 +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0monitor-oscar.ps1" +exit /b %ERRORLEVEL% \ No newline at end of file diff --git a/dist/release/monitor-oscar.ps1 b/dist/release/monitor-oscar.ps1 new file mode 100644 index 0000000..e71fa6b --- /dev/null +++ b/dist/release/monitor-oscar.ps1 @@ -0,0 +1,290 @@ +param( + [string]$AttachToExisting, + [string]$ForceRestart +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' + +$script:BaseDir = Split-Path -Parent $PSCommandPath +$script:MonitorDir = Join-Path $script:BaseDir ("oscar-monitor-{0}" -f (Get-Date -Format 'yyyyMMdd-HHmmss')) +$script:StatusFile = Join-Path $script:MonitorDir 'monitor-status.txt' +$script:HeartbeatFile = Join-Path $script:BaseDir 'monitor.heartbeat' +$script:BackendPidFile = Join-Path $script:BaseDir 'oscar.pid' +$script:CurrentMonitorFile = Join-Path $script:BaseDir 'current-monitor-dir.txt' +$script:MonitorLockDir = Join-Path $script:BaseDir '.monitor-lock' +$script:MonitorLockInfo = Join-Path $script:MonitorLockDir 'owner.json' +$script:LockAcquired = $false + +New-Item -ItemType Directory -Force -Path $script:MonitorDir | Out-Null +Set-Content -Path $script:CurrentMonitorFile -Value $script:MonitorDir -Encoding ASCII + +function Write-Status { + param([string]$Message) + + $ts = Get-Date -Format 'yyyy-MM-dd HH:mm:ss' + $line = "$ts $Message" + Write-Host $Message + Add-Content -Path $script:StatusFile -Value $line -Encoding UTF8 +} + +function Load-DotEnv { + param([string]$Path) + + if (-not (Test-Path $Path)) { + return + } + + foreach ($rawLine in Get-Content -Path $Path) { + $line = $rawLine.Trim() + if ([string]::IsNullOrWhiteSpace($line)) { continue } + if ($line.StartsWith('#')) { continue } + + $idx = $line.IndexOf('=') + if ($idx -lt 1) { continue } + + $name = $line.Substring(0, $idx).Trim() + $value = $line.Substring($idx + 1) + + if ( + ($value.Length -ge 2) -and + ( + ($value.StartsWith('"') -and $value.EndsWith('"')) -or + ($value.StartsWith("'") -and $value.EndsWith("'")) + ) + ) { + $value = $value.Substring(1, $value.Length - 2) + } + + [Environment]::SetEnvironmentVariable($name, $value, 'Process') + } +} + +function Convert-ToFlag { + param([string]$Value) + + if ([string]::IsNullOrWhiteSpace($Value)) { + return $false + } + + switch -Regex ($Value.Trim()) { + '^(?i:1|true|yes|y|on)$' { return $true } + default { return $false } + } +} + +function Get-BackendProcess { + $proc = Get-CimInstance Win32_Process | Where-Object { + $_.Name -match '^java(w)?\.exe$' -and + $_.CommandLine -match 'SensorHubWrapper' + } | Select-Object -First 1 + + return $proc +} + +function Get-ProcessStartTimeUtcString { + param([int]$Pid) + + try { + return (Get-Process -Id $Pid -ErrorAction Stop).StartTime.ToUniversalTime().ToString('o') + } + catch { + return $null + } +} + +function Update-Heartbeat { + param( + [string]$State, + [Nullable[int]]$BackendPid = $null + ) + + $lines = @( + "timestamp=$((Get-Date).ToString('o'))" + "state=$State" + "monitor_pid=$PID" + "monitor_dir=$script:MonitorDir" + ) + + if ($null -ne $BackendPid) { + $lines += "backend_pid=$BackendPid" + } + + Set-Content -Path $script:HeartbeatFile -Value $lines -Encoding ASCII +} + +function Release-MonitorLock { + if (Test-Path $script:MonitorLockDir) { + Remove-Item -Path $script:MonitorLockDir -Recurse -Force -ErrorAction SilentlyContinue + } +} + +function Acquire-MonitorLock { + for ($attempt = 1; $attempt -le 2; $attempt++) { + try { + New-Item -ItemType Directory -Path $script:MonitorLockDir -ErrorAction Stop | Out-Null + + $owner = [ordered]@{ + pid = $PID + acquiredUtc = (Get-Date).ToUniversalTime().ToString('o') + processName = (Get-Process -Id $PID).ProcessName + processStart = (Get-Process -Id $PID).StartTime.ToUniversalTime().ToString('o') + scriptPath = $PSCommandPath + hostName = $env:COMPUTERNAME + } + + $owner | ConvertTo-Json | Set-Content -Path $script:MonitorLockInfo -Encoding UTF8 + $script:LockAcquired = $true + + return @{ + Acquired = $true + ExistingPid = $null + } + } + catch { + $existingPid = $null + $alive = $false + + if (Test-Path $script:MonitorLockInfo) { + try { + $info = Get-Content -Path $script:MonitorLockInfo -Raw | ConvertFrom-Json + $existingPid = [int]$info.pid + $currentStart = Get-ProcessStartTimeUtcString -Pid $existingPid + if ($currentStart -and $currentStart -eq $info.processStart) { + $alive = $true + } + } + catch { + $alive = $false + } + } + + if ($alive) { + return @{ + Acquired = $false + ExistingPid = $existingPid + } + } + + Remove-Item -Path $script:MonitorLockDir -Recurse -Force -ErrorAction SilentlyContinue + Start-Sleep -Milliseconds 200 + } + } + + throw "Could not acquire monitor lock at $script:MonitorLockDir" +} + +function Invoke-Monitor { + Load-DotEnv -Path (Join-Path $script:BaseDir '.env') + + if ([string]::IsNullOrWhiteSpace($AttachToExisting)) { + $AttachToExisting = $env:ATTACH_TO_EXISTING + } + + if ([string]::IsNullOrWhiteSpace($ForceRestart)) { + $ForceRestart = $env:FORCE_RESTART + } + + $attach = Convert-ToFlag $AttachToExisting + $force = Convert-ToFlag $ForceRestart + + Write-Status "Monitor output: $script:MonitorDir" + + $lock = Acquire-MonitorLock + if (-not $lock.Acquired) { + Write-Status "Monitor script already running with PID $($lock.ExistingPid)." + Write-Status "Exiting without starting a second monitor." + return 200 + } + + Update-Heartbeat -State 'startup' + + $backend = Get-BackendProcess + + if ($null -ne $backend) { + $backendPid = [int]$backend.ProcessId + + if ($force) { + Write-Status "Existing OSCAR backend found with PID $backendPid. FORCE_RESTART=1, stopping it first..." + & (Join-Path $script:BaseDir 'stop-all.bat') + $stopRc = $LASTEXITCODE + Start-Sleep -Seconds 5 + + $backend = Get-BackendProcess + if ($null -ne $backend) { + throw "OSCAR backend still running with PID $($backend.ProcessId) after stop-all.bat." + } + + Write-Status "Previous OSCAR backend stopped." + } + elseif (-not $attach) { + Write-Status "OSCAR is already running with PID $backendPid." + Write-Status "Set ATTACH_TO_EXISTING=1 to monitor it, or FORCE_RESTART=1 to replace it." + return 201 + } + else { + Write-Status "Attaching to existing OSCAR backend PID $backendPid..." + } + } + + if ($null -eq $backend) { + Write-Status "Launching OSCAR stack..." + & (Join-Path $script:BaseDir 'launch-all.bat') + $launchRc = $LASTEXITCODE + + if ($launchRc -ne 0) { + throw "launch-all.bat failed with exit code $launchRc." + } + + $backend = $null + for ($i = 1; $i -le 60; $i++) { + Start-Sleep -Seconds 1 + $backend = Get-BackendProcess + if ($null -ne $backend) { + break + } + } + + if ($null -eq $backend) { + throw "Timed out waiting for OSCAR backend to appear." + } + } + + $backendPid = [int]$backend.ProcessId + Set-Content -Path $script:BackendPidFile -Value $backendPid -Encoding ASCII + + Write-Status "Monitoring OSCAR backend PID $backendPid..." + + while ($true) { + Update-Heartbeat -State 'running' -BackendPid $backendPid + Start-Sleep -Seconds 30 + + $backend = Get-BackendProcess + if ($null -eq $backend) { + Write-Status "OSCAR backend is no longer running." + Update-Heartbeat -State 'stopped' + return 0 + } + + $backendPid = [int]$backend.ProcessId + Set-Content -Path $script:BackendPidFile -Value $backendPid -Encoding ASCII + } +} + +$exitCode = 0 + +try { + $exitCode = Invoke-Monitor +} +catch { + Write-Status "ERROR: $($_.Exception.Message)" + Update-Heartbeat -State 'error' + $exitCode = 500 +} +finally { + if ($script:LockAcquired) { + Release-MonitorLock + } +} + +exit $exitCode \ No newline at end of file diff --git a/dist/release/reset-all.bat b/dist/release/reset-all.bat index 46bb6ac..6488a6d 100755 --- a/dist/release/reset-all.bat +++ b/dist/release/reset-all.bat @@ -93,4 +93,7 @@ if not defined ENV_NAME exit /b 0 if "%ENV_NAME:~0,1%"=="#" exit /b 0 if /I "%ENV_NAME:~0,7%"=="export " set "ENV_NAME=%ENV_NAME:~7%" set "%ENV_NAME%=%ENV_VALUE%" -exit /b 0 \ No newline at end of file +exit /b 0 + +if exist "%~dp0.monitor-lock" rmdir /s /q "%~dp0.monitor-lock" >nul 2>nul +del /q "%~dp0monitor.heartbeat" "%~dp0oscar.pid" "%~dp0current-monitor-dir.txt" >nul 2>nul \ No newline at end of file diff --git a/dist/release/stop-all.bat b/dist/release/stop-all.bat index 78d7386..3f0093a 100644 --- a/dist/release/stop-all.bat +++ b/dist/release/stop-all.bat @@ -43,3 +43,6 @@ for /f "tokens=1,* delims==" %%A in ("%LINE%") do ( if /I "%%A"=="CONTAINER_NAME" set "CONTAINER_NAME=%%B" ) exit /b 0 + +if exist "%~dp0.monitor-lock" rmdir /s /q "%~dp0.monitor-lock" >nul 2>nul +del /q "%~dp0monitor.heartbeat" "%~dp0oscar.pid" "%~dp0current-monitor-dir.txt" >nul 2>nul \ No newline at end of file From 6eea52861d14142e80eee98fac5dfc289f726225 Mon Sep 17 00:00:00 2001 From: tyronechrisharris <142608718+tyronechrisharris@users.noreply.github.com> Date: Sun, 10 May 2026 20:20:57 -0400 Subject: [PATCH 9/9] update change log --- changelog.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/changelog.md b/changelog.md index 0be2957..e06e33a 100644 --- a/changelog.md +++ b/changelog.md @@ -1,5 +1,21 @@ # OSCAR Build Node Change Log All notable changes to this project will be documented in this file. + +## 3.5.1 2026-05-07 +### Added +- Added `monitor-oscar.ps1` as the preferred Windows monitoring entrypoint for sessionless launches. +- Added documented sessionless launch patterns for Windows and Linux production operation. +- Added documented auto-start guidance for Windows Task Scheduler and Linux `systemd`. + +### Changes +- Updated packaged-release documentation to make sessionless `launch-all` the default production launch path. +- Updated monitoring documentation to explain duplicate-monitor protection and where operators should check `monitor.last-status`, `monitor.last-error`, `monitor.out`, and `monitor.err`. +- Updated reset/redeploy guidance to explain that operators may need to stop the monitor wrapper, remove the extracted release directory, re-extract the ZIP, recreate `.env`, and relaunch sessionlessly when old runtime artifacts or stale lanes persist. +- Updated MediaMTX guidance to be more succinct and to explain that MediaMTX reduces load on the Java backend by proxying and stabilizing camera connections before OSCAR consumes them. + +### Fixes +- Improved operational guidance around already-running OSCAR backends versus already-running monitor wrappers so sessionless launches fail more clearly. + ## 3.5.0 2026-04-24 ### Changes - Updated LaneSystem README