Skip to content

Container Image creation and Github Action#12

Closed
Xieyt wants to merge 61 commits intomainfrom
action/0
Closed

Container Image creation and Github Action#12
Xieyt wants to merge 61 commits intomainfrom
action/0

Conversation

@Xieyt
Copy link
Copy Markdown
Collaborator

@Xieyt Xieyt commented Nov 27, 2025

GitHub Action & Workflow Integration:

  • Added a new composite GitHub Action in action.yml to automate Frappe deployment and image building, supporting both pull and build-image commands with configurable inputs.
  • Created a test workflow in .github/workflows/test.yml to run the build-image command using the new action and configuration file.
  • Introduced a TOML configuration file .github/configs/test.toml to define site, app repositories, and Docker build settings for Frappe and Nginx.

BuildManager Enhancements:

Image Build Orchestration:

  • Added methods to BuildManager for building Frappe base images, final images, and Nginx images, with support for multi-platform builds and Dockerfile rendering based on TOML config.

Repository Cloning:

  • Refactored clone_apps to clone repositories in parallel using ThreadPoolExecutor, improving efficiency and reliability for multiple app clones.
  • Added helper methods for parallel clone task execution and directory ownership management (chown_dir).

Bench Bake & Verification:

  • Implemented an is_baked method to verify if a bench directory is ready for image building, checking assets and app installations.

Container Execution Improvements:

  • Updated host_run to use the correct image and user based on new config structure, and improved logic for running commands inside containers, supporting both Frappe and Nginx images. [1] [2] [3]

These changes collectively provide a robust foundation for automated, configurable, and efficient Frappe deployments and image builds in CI/CD environments.

[1] [2] [3] [4] [5]

Xieyt added 30 commits November 18, 2025 18:38
- Introduce `BuildManager` methods for image building
- Add `_build_frappe_image` and `_build_nginx_image` methods
- Adjust `BuildManager` initialization for output directory handling
- Add Dockerfile rendering logic using Jinja2 templates
- Update `Config` with separate build configurations for Frappe and Nginx
- Add new `build_image` command for image building
- Include template for Frappe Dockerfile

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove unnecessary `entrypoint` script creation
- Update ENTRYPOINT to use `/usr/bin/env bash`
- Remove redundant blank lines

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove redundant print statements in build process
- Update Dockerfile rendering to include site and bench names
- Modify default names in Nginx and Frappe build configurations
- Add user field to Nginx Dockerfile template and adjust permissions
- Refactor build commands in Nginx and Frappe image build methods

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove unused Docker image existence check logic
- Clean up related commented-out code in build methods

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add `image_type` argument to `build_images` method
- Update `build_images` logic to build specific images
- Introduce `ImageType` enum for image type constraints
- Extend `build_image` command to accept `image_type` option

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove `output_dir` parameter from `BuildManager` constructor
- Automatically set `output_dir` in `BuildManager` based on `bench_path`
- Add `_build_frappe_base_image` method to separate builder stage logic
- Rename `base_image_target_name` to `builder_image_name` in config
- Adjust Docker image naming and commands to align with refactored logic

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add test workflow using GitHub Actions
- Add action.yml to define reusable deployer action
- Add deploy.sh script for pull/build-image commands
- Add helpers.sh for common helper functions
- Modify build_image.py to include site_name and github_token
- Update config.py to make site_name mandatory

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Create `.github/configs/test.toml`
- Define test settings like `site_name`, `mode`, and `uv`
- Set `releases_retain_limit` and `python_version`
- Add common site config with `socketio_port`
- Include `apps` configuration with repos and refs

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Change file mode from 100644 to 100755 to make it executable

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove redundant check for INPUT_SITENAME in build_image_command

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add creation of frappe user and group in workflow
- Update pip install to use frappe user's environment
- Update deploy command to run under frappe user

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add tmate setup step in action.yml
- Move mkdir logic to build_images method in BuildManager

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Comment out tmate session setup in action.yml

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Replace Python setup with uv installation script
- Install frappe-deployer via uv's Python virtual environment
- Update deploy command to use activated virtual environment

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove standalone uv installation step
- Add uv installation to frappe-deployer step
- Update PATH variable handling during frappe-deployer setup

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Replace frappe-deployer installation with uv installer
- Add step to handle Docker user setup with UID 1000
- Update code to reference dynamic Docker user variable
- Refactor path handling for uv executable and .venv

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add chmod command to make UV binary executable

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Replace repetitive user output logic with `TARGET_USER`
- Add workspace ownership change for the target user
- Improve clarity and maintainability of user creation script

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add chmod for UV_EXEC_PATH to ensure execute permissions
- Use "cd" for consistent workspace directory context
- Replace "source" with "." for virtual environment activation
- Ensure proper chaining of commands with "&&"

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Removed redundant `cd` command in bash scripts
- Used `-w` flag for setting working directory
- Simplified `.bin/uv` usage within scripts

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add traverse permissions for workspace parent directory
- Use variable for UV executable path
- Replace redundant `-w` option with explicit `cd` command

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove redundant directory traversal permission grant
- Use relative paths for script execution
- Add working-directory specification for relevant steps

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Replace single quotes with double quotes in strings
- Update comment for ownership change
- Add permission-setting step for new owner

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Update ownership command to use current user
- Comment out old ownership command

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add `mkdir -p` command to ensure workspace directory exists

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Fix incorrect use of quotes in chown command

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Replace $(whoami):$(whoami) with ${TARGET_USER}:${TARGET_USER}
- Maintain consistent ownership assignment logic

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add actions/checkout@v5 step in action.yml
- Ensure proper workspace setup before running commands

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add tmate action for debugging purposes
- Configure tmate to limit access to the actor

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove docker user setup logic
- Use `chown` directly to adjust ownership
- Eliminate redundant `sudo -u` commands

Signed-off-by: xieyt <alokmilenium@gmail.com>
Xieyt and others added 16 commits November 19, 2025 15:50
- Update chown_dir to include group ID for ownership

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add command to set group read/write/execute permissions
- Use `chmod` recursively on target_path

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove tmate session step from action.yml
- Simplify build process in BuildManager methods
- Improve readability of comments and key steps
- Remove unused try-finally blocks

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add CI environment variable to deployment script environment

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add conditional live line streaming based on environment
- Decode bytes in output lines with error replacement when needed
- Ensure consistent handling of printed lines across methods

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add label configuration to `test.toml`
- Remove debug `print` statement in `build_manager.py`
- Add `labels` attribute to `ImageBuildConfig` in `build.py`
- Update config merging logic to handle `build_frappe` and `build_nginx`
- Add label rendering logic to Dockerfile templates

Signed-off-by: xieyt <alokmilenium@gmail.com>
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.4 to 3.1.6.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.4...3.1.6)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-version: 3.1.6
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
- Remove unused Union from typing imports
- Remove unused random_password_generate import

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add `push` flag to `build_image` command in `build_image.py`
- Update `ImageBuildConfig` model to include `push` attribute
- Modify `BuildManager` to extend build commands with `--push`

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Update deploy script path to use github.workspace

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Update deploy script path to use `github.action_path`
- Ensure proper resolution of script location during execution

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Fix mismatched parenthesis in dictionary key "bench_path"

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add --push flag to build-image command in deploy script

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove tag property and use fixed "latest" tag
- Update builder_image_name property to include computed fields

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove redundant app-specific build message
- Adjust build command to include production flags
- Add pre/post build checks in a loop after main build
- Comment out individual app build messages for clarity

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add --github-token parameter to build-image command
- Use GITHUB_TOKEN environment variable

Signed-off-by: xieyt <alokmilenium@gmail.com>
Copilot AI review requested due to automatic review settings December 10, 2025 10:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request introduces comprehensive container image creation and GitHub Action workflow integration for automated Frappe deployment. The changes enable building Docker images for both Frappe and Nginx components with configurable TOML-based settings, and provide a reusable composite action for CI/CD pipelines.

Key Changes:

  • Added a new build-image command to create Frappe and Nginx Docker images with multi-platform support
  • Introduced GitHub Action composite action for automated deployment and image building workflows
  • Refactored app cloning to use parallel execution with ThreadPoolExecutor for improved performance

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 25 comments.

Show a summary per file
File Description
scripts/helpers.sh New bash helper script providing logging, SSH setup, and remote execution utilities
scripts/deploy.sh Main deployment script orchestrating pull and build-image commands
frappe_deployer/template/nginx.Dockerfile Jinja2 template for building Nginx container images
frappe_deployer/template/frappe.Dockerfile Jinja2 template for building Frappe container images with multi-stage build
frappe_deployer/config/config.py Updated config model to support separate Frappe and Nginx build configurations with parallel app configuration
frappe_deployer/config/build.py New build configuration models for Frappe and Nginx images with Dockerfile rendering
frappe_deployer/commands/build_image.py New CLI command for building Docker images
frappe_deployer/commands/init.py Updated to include new build_image command module
frappe_deployer/build_manager.py Enhanced with image build orchestration, parallel cloning, and bench verification methods
frappe_deployer/deployment_manager.py Refactored bench build process to run production builds twice for all apps
action.yml GitHub composite action definition for deployment and image building
.github/workflows/test.yml Test workflow demonstrating the action usage
.github/configs/test.toml Example TOML configuration for testing image builds
Comments suppressed due to low confidence (3)

frappe_deployer/config/build.py:3

  • Import of 'os' is not used.
import os

frappe_deployer/commands/build_image.py:6

  • Import of 'os' is not used.
import os

frappe_deployer/commands/build_image.py:49

  • Modification of the locals() dictionary will have no effect on the local variables.
    current_locals["build_nginx"] = {"name": "frappe-nginx", "push": push}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread frappe_deployer/build_manager.py
Comment thread frappe_deployer/build_manager.py
Comment thread action.yml
Comment thread scripts/deploy.sh Outdated
Comment thread frappe_deployer/config/config.py
Comment thread frappe_deployer/commands/build_image.py
Comment thread scripts/deploy.sh
Comment thread scripts/deploy.sh Outdated
Comment thread scripts/helpers.sh
Comment thread scripts/deploy.sh
Xieyt and others added 9 commits December 10, 2025 16:35
- Rename `github_token` to `gh_token` in action.yml
- Update scripts to use `FRAPPE_DEPLOYER_GITHUB_TOKEN` variable
- Adjust commands to reflect the new token name

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add step to set up tmate session using mxschmitt/action-tmate@v3
- Place tmate step in the action.yml workflow

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add "where is action" step for debugging deploy script
- Update environment variables for tmate session with new inputs

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add step to create a temporary directory
- Change workspace paths to temporary directory
- Remove unnecessary tmate setup and inputs
- Add cleanup step for temporary directory

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Replace `rm -rf` with a Docker `find` command for cleanup
- Add secondary `rm -rf` execution

Signed-off-by: xieyt <alokmilenium@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Enable Yarn using Corepack
- Add activation of pnpm@10.13.1 with Corepack

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add `tag` field to `ImageBuildConfig` model with default "latest"
- Add `additional_packages` field with description in `ImageBuildConfig`
- Modify Dockerfiles to install custom `additional_packages` if provided
- Remove unused label from `frappe.Dockerfile`

Signed-off-by: xieyt <alokmilenium@gmail.com>
chore(deps): Bump jinja2 from 3.1.4 to 3.1.6
@Xieyt Xieyt closed this Apr 21, 2026
@Xieyt Xieyt deleted the action/0 branch April 21, 2026 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants