diff --git a/CHANGELOG.md b/CHANGELOG.md index 8493171..8fc8c54 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- Bulk scheduled job actions — a "Run All Now" button on the Scheduled tab back-dates all scheduled executions in a single `update_all` call, causing SolidQueue's dispatcher to pick them up immediately; respects the active period filter so only jobs within the current window are promoted +- Priority filter — a `?priority=N` param on the jobs index narrows the list to a specific integer priority value; a select dropdown appears in the search bar when multiple distinct priorities exist in the current status; priority is preserved across status tab switches, period changes, and search; Discard All also respects the active priority filter - Performance analytics — a new Performance page (`/jobs/performance`) shows per-job-class statistics derived from the history table: run count, average duration, p50, p95, min, and max; rows are sorted by p95 descending so the slowest classes appear first; a period filter (1h / 24h / 7d / All) scopes the dataset; each class name links to the History page pre-filtered to that class; business logic lives in a `JobPerformanceStats` service using a single pluck query with Ruby-side aggregation for DB-agnostic percentile computation - Metrics / health endpoint — `GET /jobs/metrics.json` returns a JSON document with job counts (`ready`, `scheduled`, `claimed`, `blocked`, `failed`), throughput (`completed_1h`, `completed_24h`), per-queue depth and pause state, and process health (`total`, `healthy`, `stale`, `by_kind`); when `slow_job_threshold` is configured, a `slow_jobs` count is also included; the endpoint goes through the same authentication and `connects_to` middleware as all other routes - Recurring task "Run Now" — a "Run Now" button on the Recurring Tasks page triggers `task.enqueue(at: Time.current)` to enqueue the job immediately without waiting for its next scheduled run; SolidQueue's `RecurringExecution` deduplication prevents double-enqueuing diff --git a/README.md b/README.md index 1a2f8ae..2e78cf6 100644 --- a/README.md +++ b/README.md @@ -35,8 +35,8 @@ SolidQueueWeb surfaces all of this in a browser UI available at any route you ch - **Dashboard** — stat cards showing counts for ready, scheduled, running, blocked, and failed jobs, plus queues, recurring tasks, and processes; "Done (1h)" and "Done (24h)" throughput cards; a "Throughput — Last 12 Hours" bar chart (blue) and a "Queue Depth — Last 12 Hours" bar chart (purple) showing hourly snapshots of active job count; pure CSS, no charting library; auto-refreshes every 5 seconds - **Queues** — all queues sorted by name with size; oldest ready job latency (color-coded, with UTC timestamp tooltip); Done (24h) and Failed (24h) throughput counts; a mini 12-bar failure rate sparkline per queue showing failure % per hour over the last 12 hours; pause/resume controls -- **Jobs** — filterable by status (ready, scheduled, claimed, blocked, failed) and by queue; search by job class name with dynamic auto-submit; time-based period filter (1 h / 24 h / 7 d); discard individual or all jobs; Turbo Frame navigation so only the table updates on filter or search; auto-refreshes every 10 seconds -- **Scheduled job management** — reschedule a scheduled job to run immediately ("Run Now") or push its `scheduled_at` forward by 1 h, 24 h, or 7 d; Turbo Stream responses update the row in place +- **Jobs** — filterable by status (ready, scheduled, claimed, blocked, failed), queue, and priority; search by job class name with dynamic auto-submit; time-based period filter (1 h / 24 h / 7 d); discard individual or all jobs; Turbo Frame navigation so only the table updates on filter or search; auto-refreshes every 10 seconds +- **Scheduled job management** — reschedule a scheduled job to run immediately ("Run Now") or push its `scheduled_at` forward by 1 h, 24 h, or 7 d; Turbo Stream responses update the row in place; "Run All Now" bulk action promotes every scheduled job in the current filtered view in a single operation - **Failed jobs** — list of failed executions with error details; search by class name; filter by queue; time-based period filter; retry or discard individually or in bulk; bulk retry with configurable stagger (+5s / +10s / +30s / +1m) to avoid thundering herd on recovery - **Job detail** — full arguments, timestamps, blocked-until date, and error backtrace; action buttons based on job status - **Queue management** — pause and resume individual queues; queue-scoped job list with status filter, search, and discard @@ -205,15 +205,11 @@ When `connects_to` is `nil` (the default), no connection switching occurs and si ## Roadmap -Planned features, roughly ordered by priority: +Post-1.0 planned features: **Operations** - Admin audit log — record who retried or discarded which jobs and when (requires host-app user identity) - Failed job retry with modified arguments — edit the arguments JSON from the job detail page before retrying; useful for correcting bad payloads without redeploying -- Bulk scheduled job actions — "Run All Now" button on the Scheduled tab, mirroring the "Retry All" pattern on the Failed Jobs page - -**Observability** -- Priority filter — filter and sort the jobs list by Solid Queue job priority **Notifications** - Multiple webhook targets — support an array of `alert_webhook_url` values so alerts can fan out to Slack, PagerDuty, and custom endpoints simultaneously diff --git a/app/assets/stylesheets/solid_queue_web/_07_forms.css b/app/assets/stylesheets/solid_queue_web/_07_forms.css index 6903ae0..5f4dabc 100644 --- a/app/assets/stylesheets/solid_queue_web/_07_forms.css +++ b/app/assets/stylesheets/solid_queue_web/_07_forms.css @@ -76,6 +76,23 @@ color: #fff; } +.sqd-select { + padding: 0.35rem 0.6rem; + border: 1px solid var(--border); + border-radius: 5px; + font-size: 13px; + background: var(--surface); + color: var(--text); + line-height: 1.5; + cursor: pointer; +} + +.sqd-select:focus { + outline: 2px solid var(--primary); + outline-offset: -1px; + border-color: var(--primary); +} + .sqd-period-filter { display: flex; align-items: center; diff --git a/app/controllers/solid_queue_web/jobs_controller.rb b/app/controllers/solid_queue_web/jobs_controller.rb index f78a2e2..d824ab5 100644 --- a/app/controllers/solid_queue_web/jobs_controller.rb +++ b/app/controllers/solid_queue_web/jobs_controller.rb @@ -1,16 +1,20 @@ module SolidQueueWeb class JobsController < ApplicationController - before_action :set_status, only: [:destroy, :discard_selected] - def index - @status = params[:status].presence_in(Job::STATUSES) || "ready" - @search = params[:q].presence - @period = params[:period].presence_in(PERIOD_DURATIONS.keys) + @status = params[:status].presence_in(Job::STATUSES) || "ready" + @search = params[:q].presence + @period = params[:period].presence_in(PERIOD_DURATIONS.keys) + @priority = params[:priority].presence + scope = Job::EXECUTION_MODELS[@status].includes(:job) - scope = scope.references(:job).where("solid_queue_jobs.class_name LIKE ?", "%#{@search}%") if @search.present? + scope = scope.references(:job).where("solid_queue_jobs.class_name LIKE ?", "%#{@search}%") if @search.present? scope = scope.references(:job).where("solid_queue_jobs.created_at >= ?", PERIOD_DURATIONS[@period].ago) if @period.present? + scope = scope.references(:job).where("solid_queue_jobs.priority = ?", @priority.to_i) if @priority.present? scope = scope.order(created_at: :desc) + @priority_options = Job::EXECUTION_MODELS[@status].joins(:job) + .distinct.pluck("solid_queue_jobs.priority").sort + respond_to do |format| format.html { @pagy, @jobs = pagy(scope) } format.csv do @@ -25,11 +29,14 @@ def show @job = SolidQueue::Job .includes(:ready_execution, :scheduled_execution, :claimed_execution, :blocked_execution, :failed_execution) .find(params[:id]) - @execution_status = derive_status(@job) + @execution_status = Job.derive_status(@job) end def destroy - model = execution_model_for!(@status) + @status = params[:status] + @period = params[:period].presence_in(PERIOD_DURATIONS.keys) + @priority = params[:priority].presence + model = Job.execution_model_for!(@status) if params[:id] @execution = model.find(params[:id]) @execution.discard @@ -63,29 +70,11 @@ def jobs_csv(scope) end end - def derive_status(job) - return "failed" if job.failed_execution.present? - return "claimed" if job.claimed_execution.present? - return "blocked" if job.blocked_execution.present? - return "ready" if job.ready_execution.present? - return "scheduled" if job.scheduled_execution.present? - "finished" - end - - def set_status - @status = params[:status] - @period = params[:period].presence_in(PERIOD_DURATIONS.keys) - end - def filtered_scope(model) scope = model.includes(:job) scope = scope.references(:job).where("solid_queue_jobs.created_at >= ?", PERIOD_DURATIONS[@period].ago) if @period.present? + scope = scope.references(:job).where("solid_queue_jobs.priority = ?", @priority.to_i) if @priority.present? scope end - - def execution_model_for!(status) - raise ArgumentError, "Cannot discard #{status} jobs from this page." unless Job::DISCARDABLE.include?(status) - Job::EXECUTION_MODELS[status] - end end end diff --git a/app/controllers/solid_queue_web/scheduled_jobs_controller.rb b/app/controllers/solid_queue_web/scheduled_jobs_controller.rb index b48dc5e..14f62b8 100644 --- a/app/controllers/solid_queue_web/scheduled_jobs_controller.rb +++ b/app/controllers/solid_queue_web/scheduled_jobs_controller.rb @@ -1,19 +1,24 @@ module SolidQueueWeb class ScheduledJobsController < ApplicationController - OFFSETS = { "1h" => 1.hour, "24h" => 24.hours, "7d" => 7.days }.freeze + def create + @period = params[:period].presence_in(PERIOD_DURATIONS.keys) + job_ids = scheduled_scope.pluck("solid_queue_jobs.id") + + SolidQueue::ScheduledExecution.where(job_id: job_ids).update_all(scheduled_at: 1.second.ago) + SolidQueue::Job.where(id: job_ids).update_all(scheduled_at: 1.second.ago) + + redirect_to jobs_path(status: "scheduled", period: @period), + notice: "#{job_ids.size} #{"job".pluralize(job_ids.size)} scheduled to run immediately." + rescue => e + redirect_to jobs_path(status: "scheduled", period: @period), + alert: "Could not run jobs: #{e.message}" + end def update @execution = SolidQueue::ScheduledExecution.find(params[:id]) @period = params[:period].presence_in(PERIOD_DURATIONS.keys) @run_now = params[:offset] == "now" - - new_time = if @run_now - 1.second.ago - elsif OFFSETS.key?(params[:offset]) - @execution.scheduled_at + OFFSETS[params[:offset]] - else - raise ArgumentError, "Invalid offset." - end + new_time = resolve_new_time(@execution, params[:offset]) @execution.update!(scheduled_at: new_time) @execution.job.update!(scheduled_at: new_time) @@ -30,5 +35,20 @@ def update rescue => e redirect_to jobs_path(status: "scheduled"), alert: "Could not reschedule job: #{e.message}" end + + private + + def scheduled_scope + scope = SolidQueue::ScheduledExecution.joins(:job) + scope = scope.where("solid_queue_jobs.created_at >= ?", PERIOD_DURATIONS[@period].ago) if @period.present? + scope + end + + def resolve_new_time(execution, offset) + return 1.second.ago if offset == "now" + raise ArgumentError, "Invalid offset." unless PERIOD_DURATIONS.key?(offset) + + execution.scheduled_at + PERIOD_DURATIONS[offset] + end end end diff --git a/app/models/solid_queue_web/job.rb b/app/models/solid_queue_web/job.rb index 6cc2f57..8bde3ba 100644 --- a/app/models/solid_queue_web/job.rb +++ b/app/models/solid_queue_web/job.rb @@ -1,6 +1,6 @@ module SolidQueueWeb class Job - STATUSES = %w[ready scheduled claimed blocked failed].freeze + STATUSES = %w[ready scheduled claimed blocked failed].freeze DISCARDABLE = %w[ready scheduled blocked].freeze EXECUTION_MODELS = { "ready" => SolidQueue::ReadyExecution, @@ -9,5 +9,21 @@ class Job "blocked" => SolidQueue::BlockedExecution, "failed" => SolidQueue::FailedExecution }.freeze + + def self.execution_model_for!(status) + raise ArgumentError, "Cannot discard #{status} jobs from this page." unless DISCARDABLE.include?(status) + + EXECUTION_MODELS[status] + end + + def self.derive_status(job) + return "failed" if job.failed_execution.present? + return "claimed" if job.claimed_execution.present? + return "blocked" if job.blocked_execution.present? + return "ready" if job.ready_execution.present? + return "scheduled" if job.scheduled_execution.present? + + "finished" + end end end diff --git a/app/views/solid_queue_web/jobs/index.html.erb b/app/views/solid_queue_web/jobs/index.html.erb index e9c8f43..0b6afc5 100644 --- a/app/views/solid_queue_web/jobs/index.html.erb +++ b/app/views/solid_queue_web/jobs/index.html.erb @@ -5,22 +5,29 @@
- <%= link_to "Ready", jobs_path(status: "ready", q: @search, period: @period), class: @status == "ready" ? "active" : "" %> - <%= link_to "Scheduled", jobs_path(status: "scheduled", q: @search, period: @period), class: @status == "scheduled" ? "active" : "" %> - <%= link_to "Running", jobs_path(status: "claimed", q: @search, period: @period), class: @status == "claimed" ? "active" : "" %> - <%= link_to "Blocked", jobs_path(status: "blocked", q: @search, period: @period), class: @status == "blocked" ? "active" : "" %> - <%= link_to "Failed", jobs_path(status: "failed", q: @search, period: @period), class: @status == "failed" ? "active" : "" %> + <%= link_to "Ready", jobs_path(status: "ready", q: @search, period: @period, priority: @priority), class: @status == "ready" ? "active" : "" %> + <%= link_to "Scheduled", jobs_path(status: "scheduled", q: @search, period: @period, priority: @priority), class: @status == "scheduled" ? "active" : "" %> + <%= link_to "Running", jobs_path(status: "claimed", q: @search, period: @period, priority: @priority), class: @status == "claimed" ? "active" : "" %> + <%= link_to "Blocked", jobs_path(status: "blocked", q: @search, period: @period, priority: @priority), class: @status == "blocked" ? "active" : "" %> + <%= link_to "Failed", jobs_path(status: "failed", q: @search, period: @period, priority: @priority), class: @status == "failed" ? "active" : "" %>
<% if @jobs.any? %>
<%= link_to "Export CSV", jobs_path(format: :csv, status: @status, q: @search, period: @period), class: "sqd-btn sqd-btn--muted", data: { turbo: false } %> + <% if @status == "scheduled" %> + <%= button_to "Run All Now", run_all_now_scheduled_jobs_path, + method: :post, + params: { period: @period }, + class: "sqd-btn sqd-btn--primary", + data: { confirm: "Run all #{@pagy.count} scheduled jobs immediately?" } %> + <% end %> <% if discardable %> <%= button_to "Discard All", discard_all_jobs_path, method: :post, params: { status: @status, period: @period }, class: "sqd-btn sqd-btn--danger", - data: { confirm: "Discard all #{@jobs.size} #{@status} jobs? This cannot be undone." } %> + data: { confirm: "Discard all #{@pagy.count} #{@status} jobs? This cannot be undone." } %> <% end %>
<% end %> @@ -32,14 +39,23 @@ - <% if @search.present? %> + <% if @priority_options.size > 1 %> + + <% end %> + <% if @search.present? || @priority.present? %> <%= link_to "Clear", jobs_path(status: @status, period: @period), class: "sqd-btn sqd-btn--muted" %> <% end %>
- <%= link_to "All", jobs_path(status: @status, q: @search), class: @period.nil? ? "active" : "", aria: { current: @period.nil? ? "true" : nil }, aria_label: "All time" %> - <%= link_to "1h", jobs_path(status: @status, q: @search, period: "1h"), class: @period == "1h" ? "active" : "", aria: { current: @period == "1h" ? "true" : nil }, aria_label: "Last 1 hour" %> - <%= link_to "24h", jobs_path(status: @status, q: @search, period: "24h"), class: @period == "24h" ? "active" : "", aria: { current: @period == "24h" ? "true" : nil }, aria_label: "Last 24 hours" %> - <%= link_to "7d", jobs_path(status: @status, q: @search, period: "7d"), class: @period == "7d" ? "active" : "", aria: { current: @period == "7d" ? "true" : nil }, aria_label: "Last 7 days" %> + <%= link_to "All", jobs_path(status: @status, q: @search, priority: @priority), class: @period.nil? ? "active" : "", aria: { current: @period.nil? ? "true" : nil }, aria_label: "All time" %> + <%= link_to "1h", jobs_path(status: @status, q: @search, priority: @priority, period: "1h"), class: @period == "1h" ? "active" : "", aria: { current: @period == "1h" ? "true" : nil }, aria_label: "Last 1 hour" %> + <%= link_to "24h", jobs_path(status: @status, q: @search, priority: @priority, period: "24h"), class: @period == "24h" ? "active" : "", aria: { current: @period == "24h" ? "true" : nil }, aria_label: "Last 24 hours" %> + <%= link_to "7d", jobs_path(status: @status, q: @search, priority: @priority, period: "7d"), class: @period == "7d" ? "active" : "", aria: { current: @period == "7d" ? "true" : nil }, aria_label: "Last 7 days" %>
diff --git a/config/routes.rb b/config/routes.rb index 6af7982..86cfd55 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -22,7 +22,11 @@ # Singular selection resources must be defined before the member routes of their # parent resources, otherwise DELETE /list/selection matches /list/:id first. - resources :scheduled_jobs, only: [:update] + resources :scheduled_jobs, only: [:update] do + collection do + post :run_all_now, action: :create + end + end resource :job_selection, path: "list/selection", only: [:destroy], controller: "jobs/selections" resources :jobs, path: "list", only: [:index, :show, :destroy] do diff --git a/spec/requests/solid_queue_web/jobs_spec.rb b/spec/requests/solid_queue_web/jobs_spec.rb index b18f052..7d8ca55 100644 --- a/spec/requests/solid_queue_web/jobs_spec.rb +++ b/spec/requests/solid_queue_web/jobs_spec.rb @@ -322,4 +322,49 @@ expect(response.body).not_to include('class="sqd-row--slow"') end end + + describe "GET /jobs/list?priority= (priority filter)" do + let!(:high_priority_job) do + SolidQueue::Job.create!( + queue_name: "default", class_name: "HighPriorityJob", + arguments: {}, active_job_id: SecureRandom.uuid, priority: 0 + ) + end + + let!(:low_priority_job) do + SolidQueue::Job.create!( + queue_name: "default", class_name: "LowPriorityJob", + arguments: {}, active_job_id: SecureRandom.uuid, priority: 10 + ) + end + + it "shows all jobs when no priority filter is set" do + get "/jobs/list", params: { status: "ready" } + expect(response.body).to include("HighPriorityJob") + expect(response.body).to include("LowPriorityJob") + end + + it "filters to only jobs with the specified priority" do + get "/jobs/list", params: { status: "ready", priority: "10" } + expect(response.body).to include("LowPriorityJob") + expect(response.body).not_to include("HighPriorityJob") + end + + it "renders the priority select dropdown when multiple priorities exist" do + get "/jobs/list", params: { status: "ready" } + expect(response.body).to include("sqd-select") + expect(response.body).to include("All priorities") + end + + it "preserves priority across status tab links" do + get "/jobs/list", params: { status: "ready", priority: "0" } + expect(response.body).to include("priority=0") + end + + it "discard all respects the priority filter" do + post "/jobs/list/discard_all", params: { status: "ready", priority: "10" } + expect(SolidQueue::ReadyExecution.exists?(job: high_priority_job)).to be true + expect(SolidQueue::ReadyExecution.exists?(job: low_priority_job)).to be false + end + end end diff --git a/spec/requests/solid_queue_web/scheduled_jobs_spec.rb b/spec/requests/solid_queue_web/scheduled_jobs_spec.rb index 3a0960e..ac3fb9e 100644 --- a/spec/requests/solid_queue_web/scheduled_jobs_spec.rb +++ b/spec/requests/solid_queue_web/scheduled_jobs_spec.rb @@ -13,6 +13,75 @@ let(:execution) { job.scheduled_execution } + describe "POST /scheduled_jobs/run_all_now" do + it "back-dates all scheduled executions and redirects with a notice" do + post "/jobs/scheduled_jobs/run_all_now" + expect(response).to redirect_to("/jobs/list?status=scheduled") + follow_redirect! + expect(response.body).to include("scheduled to run immediately") + end + + it "sets scheduled_at to the past for each execution" do + post "/jobs/scheduled_jobs/run_all_now" + execution.reload + expect(execution.scheduled_at).to be <= Time.current + end + + it "also updates the underlying job's scheduled_at" do + post "/jobs/scheduled_jobs/run_all_now" + job.reload + expect(job.scheduled_at).to be <= Time.current + end + + it "includes the count in the notice" do + post "/jobs/scheduled_jobs/run_all_now" + follow_redirect! + expect(response.body).to include("1 job scheduled to run immediately") + end + + it "pluralises correctly for multiple jobs" do + SolidQueue::Job.create!( + queue_name: "default", class_name: "OtherJob", + arguments: {}, active_job_id: SecureRandom.uuid, + scheduled_at: 3.hours.from_now + ) + post "/jobs/scheduled_jobs/run_all_now" + follow_redirect! + expect(response.body).to include("2 jobs scheduled to run immediately") + end + + it "preserves period in the redirect when provided" do + post "/jobs/scheduled_jobs/run_all_now", params: { period: "24h" } + expect(response).to redirect_to("/jobs/list?period=24h&status=scheduled") + end + + it "redirects with an alert when an unexpected error occurs" do + allow(SolidQueue::ScheduledExecution).to receive(:joins).and_raise(RuntimeError, "db error") + post "/jobs/scheduled_jobs/run_all_now" + expect(response).to redirect_to("/jobs/list?status=scheduled") + expect(flash[:alert]).to include("Could not run jobs") + end + + it "only runs jobs within the period when period is provided" do + old_job = SolidQueue::Job.create!( + queue_name: "default", class_name: "OldJob", + arguments: {}, active_job_id: SecureRandom.uuid, + scheduled_at: 2.hours.from_now + ) + old_job.scheduled_execution.update_columns(created_at: 48.hours.ago) + old_job.update_columns(created_at: 48.hours.ago) + + post "/jobs/scheduled_jobs/run_all_now", params: { period: "1h" } + + execution.reload + old_job.scheduled_execution.reload + # execution was created recently (within 1h window) — runs immediately + expect(execution.scheduled_at).to be <= Time.current + # old_job was created 48h ago (outside 1h window) — untouched + expect(old_job.scheduled_execution.scheduled_at).to be > Time.current + end + end + describe "PATCH /scheduled_jobs/:id" do context "with offset=now" do it "sets scheduled_at to the past and redirects" do