diff --git a/pull-request-description.md b/pull-request-description.md new file mode 100644 index 0000000..83db2f6 --- /dev/null +++ b/pull-request-description.md @@ -0,0 +1,75 @@ +# PR: Improve rig fitting, helper controls, and weight smoothing + +## Summary + +This pull request improves the rig fitting workflow for manually positioned skeletons and softens several problem areas in generated skin weights. + +The main functional changes are: + +- added mesh-drag vertex snapping with a user-controlled snap strength slider +- added condensed main-chain visibility toggles in Position Joints mode +- centered pelvis, spine, chest, neck, and head placement against the mesh instead of the front surface +- reduced finger joint helper circle size and made helper visibility respect hidden chains +- prevented branching parent bones such as chest and pelvis from being auto-rotated when child bones are moved independently +- expanded weight smoothing around pelvis, thighs, buttocks, shoulders, clavicles, neck, and upper arms +- added focused regression tests for the new placement, chain grouping, rotation, and smoothing behavior + +## Why + +These changes target the issues seen during rig fitting and auto-weighting on humanoid avatars: + +- joint placement needed stronger assistance when dragging bones over the mesh +- main chains needed to be easier to isolate visually during joint positioning +- centerline bones were drifting too far toward the front of the character +- shoulder and hip areas needed softer, more natural blending similar to the imported DAE reference +- chest and shoulder orientation could drift when moving branching child bones + +## What changed + +### Position Joints workflow + +- Added a snap strength slider with a `0` to `20` range for mesh-drag snapping. +- Added visible-chain checkboxes for condensed main chains, with fingers grouped into the hand chain. +- Wired helper visibility changes so hidden chains are removed from the active helper display. + +### Bone placement and orientation + +- Added nearest-vertex snap blending during mesh drag. +- Added mesh-centerline targeting for primary centerline bones. +- Applied automatic centerline snapping after skeleton load for pelvis, spine, chest, neck, and head chains. +- Preserved template orientation on branching parent bones during independent child movement. + +### Weight smoothing + +- Expanded boundary smoothing to keep more multi-bone influence at seam regions. +- Added along-bone gradients so bone centers stay dominant while joints retain blended influence. +- Added stronger symmetric socket smoothing for: + - pelvis to thigh + - spine/chest/neck to shoulder chain + - clavicle to upper arm +- Added pelvis-basin smoothing to improve the glute and central pelvis region. +- Strengthened torso-to-upperarm blending in the front shoulder area. + +### Helper and utility updates + +- Added condensed chain grouping utilities for visibility and labeling. +- Added smaller helper points for finger joints. +- Ensured hidden chain roots affect helper rendering consistently. + +### Investigation support + +- Added temporary scripts under `tmp/` for comparing skeleton topology and weight profiles during debugging. + +## Testing + +- `npm run build` +- Focused regression tests were added for: + - chain grouping utilities + - mesh drag snapping and centerline helpers + - independent branching-bone movement behavior + - advanced weight smoothing boundaries + +## Notes for review + +- This PR includes temporary analysis scripts in `tmp/` because they were used to validate skeleton and weight-profile differences during the investigation. +- The changes are concentrated in the rig-editing and skin-weight smoothing flow rather than import/export behavior. \ No newline at end of file diff --git a/restore-point-2026-05-08-position-joints-topbar.md b/restore-point-2026-05-08-position-joints-topbar.md new file mode 100644 index 0000000..2396618 --- /dev/null +++ b/restore-point-2026-05-08-position-joints-topbar.md @@ -0,0 +1,57 @@ +# Restore Point - 2026-05-08 (Position Joints Topbar) + +Date: 2026-05-08 +Workspace: c:\Users\jeffa\mesh2motion-app + +## Summary + +This restore point captures the Position Joints UI refactor into a create-only topbar layout, with a dedicated stylesheet and horizontal Visible Chains layout matching the reference screenshot. + +## Key behavior + +- Position Joints UI renders as a topbar only on the Create page during Edit Skeleton. +- The right-side tool panel is hidden in Edit Skeleton on Create page. +- Visible Chains checkboxes flow horizontally across the topbar row. +- Controls row is centered with compact spacing; Back/Finish sit to the far right after undo/redo. + +## Files touched + +- src/create.html +- src/create-position-joints.css +- src/Mesh2MotionEngine.ts +- src/lib/processes/edit-skeleton/StepEditSkeleton.ts + +## Detailed notes + +### Create page topbar layout + +- Added `create-page` class on `body` in `create.html`. +- Injected a dedicated stylesheet `create-position-joints.css`. +- Moved Edit Skeleton UI (`#skeleton-step-actions`) into `#position-joints-topbar`. +- Topbar rows: + - Row 1: Selected bone label centered. + - Row 2: Visible Chains checkbox list (label removed). + - Row 3: Controls row with Position by mesh volume, Preview toggle, Vertex Snap slider, Mirror/Move options, Undo/Redo, Back/Finish. + +### Scoped CSS + +- All topbar styles scoped under `body.create-page` to avoid bleeding into other pages. +- `body.create-page.edit-skeleton-topbar` toggles topbar visibility and hides `#tool-panel`. +- Visible Chains list forced to horizontal layout via flex row and a custom fieldset class. +- Controls row centered with 10px gaps between items. + +### Process step hook + +- `Mesh2MotionEngine` toggles `edit-skeleton-topbar` on the Create page when entering/leaving `ProcessStep.EditSkeleton`. + +### Visible Chains rendering + +- `StepEditSkeleton` renders the chain checkboxes into a fieldset with class `position-joints-chain-fieldset` to enforce horizontal layout and avoid duplicate labels. + +## Verification checklist + +- Create page (Use Your Model) shows topbar only in Edit Skeleton step. +- Explore and Retarget pages are unaffected. +- Visible Chains checkboxes flow horizontally in one row. +- Back/Finish buttons appear to the right of undo/redo. +- Controls row items are centered and tightly spaced. diff --git a/restore-point-2026-05-08.md b/restore-point-2026-05-08.md new file mode 100644 index 0000000..46f2b7e --- /dev/null +++ b/restore-point-2026-05-08.md @@ -0,0 +1,35 @@ +# Restore Point - 2026-05-08 + +Date: 2026-05-08 +Workspace: c:\Users\jeffa\mesh2motion-app + +## Summary + +This restore point captures the state after tightening the Visible Chains UI and improving default spine layout for the Fox rig. + +## Visible Chains grouping changes + +- Condensed spine, head, and quadruped leg chains into fewer checkboxes by grouping chain roots. +- Spine chains now anchor to the rootmost spine bone so multiple spine segments appear under one checkbox. +- Head chains anchor to the main head bone (tips/end bones grouped under head). +- Quadruped front legs group under front leg shoulder anchors (left/right). +- Quadruped back legs group under back leg pelvis anchors (left/right). + +Files touched: +- src/lib/Utilities.ts +- src/lib/Utilities.test.ts + +## Fox spine default layout + +- Added a fox-specific spine spread on skeleton load to place spine bones horizontally. +- The spine chain spreads along the mesh horizontal axis and interpolates height between pelvis and head. +- Runs after centerline snapping during the skeletonLoaded flow. + +Files touched: +- src/lib/processes/edit-skeleton/MeshDragBonePlacement.ts +- src/lib/EventListeners.ts + +## Notes + +- Fox spine distribution uses mesh bounds and pelvis/head positions as hints. +- If rig bone names differ from spine/pelvis/head naming, adjust the matchers in MeshDragBonePlacement. diff --git a/restore-point-2026-05-09-edited-armature-skinning.md b/restore-point-2026-05-09-edited-armature-skinning.md new file mode 100644 index 0000000..e95cd9d --- /dev/null +++ b/restore-point-2026-05-09-edited-armature-skinning.md @@ -0,0 +1,12 @@ +# Restore Point - 2026-05-09 (Edited Armature Skinning) + +Date: 2026-05-09 +Workspace: c:\Users\jeffa\mesh2motion-app + +## Summary + +This restore point captures the fix that uses the edited skeleton transforms when generating the armature for skinning, preventing collapsed meshes after finishing joint placement. + +## Files touched + +- src/lib/processes/edit-skeleton/StepEditSkeleton.ts diff --git a/restore-point-2026-05-09-orbit-target-axis-gizmo.md b/restore-point-2026-05-09-orbit-target-axis-gizmo.md new file mode 100644 index 0000000..0d58913 --- /dev/null +++ b/restore-point-2026-05-09-orbit-target-axis-gizmo.md @@ -0,0 +1,15 @@ +# Restore Point - 2026-05-09 (Orbit Target Axis Gizmo) + +Date: 2026-05-09 +Workspace: c:\Users\jeffa\mesh2motion-app + +## Summary + +This restore point captures the updated work area layout with the axis gizmo snapped to the current orbit target and the view helper sized at 2x. + +## Files touched + +- src/lib/SceneEnvironmentManager.ts +- src/Mesh2MotionEngine.ts +- src/lib/CustomViewHelper.ts +- src/styles.css diff --git a/restore-point-2026-05-09-view-helper-2x.md b/restore-point-2026-05-09-view-helper-2x.md new file mode 100644 index 0000000..28dbb7f --- /dev/null +++ b/restore-point-2026-05-09-view-helper-2x.md @@ -0,0 +1,13 @@ +# Restore Point - 2026-05-09 (View Helper 2x) + +Date: 2026-05-09 +Workspace: c:\Users\jeffa\mesh2motion-app + +## Summary + +This restore point captures the view helper axis gizmo at 2x size with snap-to-axis behavior preserved. + +## Files touched + +- src/lib/CustomViewHelper.ts +- src/styles.css diff --git a/restore-point-2026-05-09.md b/restore-point-2026-05-09.md new file mode 100644 index 0000000..64f8d10 --- /dev/null +++ b/restore-point-2026-05-09.md @@ -0,0 +1,8 @@ +# Restore Point - 2026-05-09 + +Date: 2026-05-09 +Workspace: c:\Users\jeffa\mesh2motion-app + +## Summary + +This restore point captures the state before adding centerline snapping controls, rotation spinner adjustments, and additional joint placement UI updates. diff --git a/src/Mesh2MotionEngine.ts b/src/Mesh2MotionEngine.ts index e599dc4..d6e991c 100644 --- a/src/Mesh2MotionEngine.ts +++ b/src/Mesh2MotionEngine.ts @@ -214,6 +214,9 @@ export class Mesh2MotionEngine { this.skeleton_helper.setHideRightSideJoints( is_edit_skeleton_step && this.edit_skeleton_step.is_mirror_mode_enabled() ) + this.skeleton_helper.setHiddenChainRoots( + is_edit_skeleton_step ? this.edit_skeleton_step.hidden_bone_chain_root_names() : [] + ) } public update_a_pose_options_visibility (): void { @@ -251,6 +254,7 @@ export class Mesh2MotionEngine { this.is_model_gizmo_active = true this.transform_controls.attach(this.load_model_step.model_meshes()) this.transform_controls.setMode('translate') + this.transform_controls.size = 1 this.transform_controls.enabled = true } @@ -292,6 +296,11 @@ export class Mesh2MotionEngine { this.is_transform_controls_dragging = false } + public update_orbit_controls_state (allow_orbit: boolean): void { + const orbit_disabled = this.edit_skeleton_step.is_orbit_rotation_disabled() + this.enable_orbit_controls(allow_orbit && !orbit_disabled) + } + public get is_mesh_drag_mode_dragging (): boolean { return this.mesh_drag_bone_placement.is_dragging() } @@ -343,6 +352,7 @@ export class Mesh2MotionEngine { // update the current process step variable this.update_current_process_step(process_step) + this.update_position_joints_topbar_state(process_step) // clean up things related to steps in since we can navigate back and forth this.edit_skeleton_step.cleanup_on_exit_step() @@ -403,6 +413,8 @@ export class Mesh2MotionEngine { this.edit_skeleton_step.begin(this.scene, this.load_skeleton_step.skeleton_type()) this.update_edit_bone_interaction_mode() this.transform_controls.setMode(this.transform_controls_type) // 'translate', 'rotate' + this.update_transform_controls_size() + this.update_orbit_controls_state(true) this.sync_skeleton_helper_joint_visibility() @@ -451,6 +463,20 @@ export class Mesh2MotionEngine { return this.process_step } // end process_step_changed() + private update_position_joints_topbar_state (process_step: ProcessStep): void { + const body = document.body + if (!body.classList.contains('create-page')) { + return + } + + if (process_step === ProcessStep.EditSkeleton) { + body.classList.add('edit-skeleton-topbar') + return + } + + body.classList.remove('edit-skeleton-topbar') + } + private animate (): void { requestAnimationFrame(this.animate) @@ -468,6 +494,10 @@ export class Mesh2MotionEngine { this.renderer.render(this.scene, this.camera) // view helper + const orbit_target = this.scene_environment.get_orbit_target() + if (orbit_target !== null) { + this.view_helper.center.copy(orbit_target) + } this.view_helper.render(this.renderer) // updates current viewport if (this.view_helper.animating) { this.view_helper.update(delta_time) // updates animation when clicking on axis @@ -494,10 +524,12 @@ export class Mesh2MotionEngine { case 'translate': this.transform_controls_type = TransformControlType.Translation this.transform_controls.setMode('translate') + this.update_transform_controls_size() break case 'rotation': this.transform_controls_type = TransformControlType.Rotation this.transform_controls.setMode('rotate') + this.update_transform_controls_size() break default: console.warn(`Unknown transform mode selected: ${radio_button_selected}`) @@ -505,6 +537,10 @@ export class Mesh2MotionEngine { } } + private update_transform_controls_size (): void { + this.transform_controls.size = 1 + } + public changed_transform_controls_space (radio_button_selected: TransformSpace | undefined): void { if (radio_button_selected) { this.transform_space_type = radio_button_selected diff --git a/src/create-position-joints.css b/src/create-position-joints.css new file mode 100644 index 0000000..7cc8a02 --- /dev/null +++ b/src/create-position-joints.css @@ -0,0 +1,160 @@ +body.create-page #position-joints-topbar { + display: none; + position: fixed; + top: var(--navigation-height); + left: 0; + right: 0; + z-index: 2; + background: linear-gradient(180deg, var(--bg-primary) 0%, var(--bg-alternate) 100%); + border-bottom: 1px solid var(--bg-tertiary); + box-shadow: 0 4px 10px rgba(0, 0, 0, 0.2); +} + +body.create-page.edit-skeleton-topbar #position-joints-topbar { + display: block; +} + +body.create-page.edit-skeleton-topbar #tool-panel { + display: none; +} + +body.create-page .position-joints-layout { + display: flex; + flex-direction: column; + gap: 0.4rem; + padding: 0.6rem 1.2rem 0.8rem; + box-sizing: border-box; +} + +body.create-page .position-joints-row { + display: flex; + align-items: center; + gap: 0.75rem; + flex-wrap: wrap; + width: 100%; +} + +body.create-page .position-joints-row--selected { + justify-content: center; +} + +body.create-page .position-joints-row--chains { + justify-content: center; +} + +body.create-page .position-joints-row--actions { + justify-content: center; + gap: 1rem; +} + +body.create-page .position-joints-row--controls { + justify-content: center; + gap: 10px; +} + +body.create-page .position-joints-controls-left, +body.create-page .position-joints-controls-center, +body.create-page .position-joints-controls-right { + display: flex; + align-items: center; + gap: 10px; + flex-wrap: wrap; +} + +body.create-page .position-joints-controls-center { + justify-content: center; +} + +body.create-page .position-joints-controls-right { + justify-content: center; +} + +body.create-page .position-joints-actions { + display: flex; + align-items: center; + gap: 10px; +} + +body.create-page .position-joints-section-title { + font-size: 0.75rem; + letter-spacing: 0.08em; + text-transform: uppercase; + color: var(--text-alternate); +} + +body.create-page .position-joints-label { + font-size: 0.85rem; + color: var(--text-alternate); +} + +body.create-page .position-joints-undo { + display: flex; + gap: 0.25rem; + align-items: center; +} + +body.create-page .position-joints-chain-list { + display: flex; + flex-direction: row; + flex-wrap: wrap; + gap: 0.35rem 0.75rem; + max-height: none; + overflow: visible; + padding-right: 0; + justify-content: center; + flex: 1 1 auto; + min-width: 0; +} + +body.create-page .position-joints-chain-fieldset { + display: flex; + flex-direction: row; + flex-wrap: wrap; + gap: 0.35rem 0.75rem; + align-items: center; + margin: 0; + padding: 0; + border: none; +} + +body.create-page .position-joints-chain-list .styled-checkbox { + display: inline-flex; + align-items: center; + margin: 0; +} + +body.create-page .position-joints-chain-list label { + white-space: nowrap; +} + +body.create-page .position-joints-inline { + display: flex; + align-items: center; + gap: 0.5rem; +} + +body.create-page .position-joints-snap { + display: flex; + flex-direction: column; + gap: 0.35rem; +} + +body.create-page .position-joints-snap input[type="range"] { + width: 140px; +} + +body.create-page .position-joints-snap-value { + width: 2.5rem; + text-align: right; +} + +body.create-page .position-joints-preview { + display: flex; + align-items: center; + gap: 0.5rem; +} + +body.create-page #bone-selection-section { + background: transparent; + padding: 0; +} diff --git a/src/create.html b/src/create.html index ff72d1c..ff048c1 100644 --- a/src/create.html +++ b/src/create.html @@ -7,9 +7,14 @@ - + + +
+
+
+

+ Selected Bone: None +

+
+ +
+
+
+ +
+
+
+ + +
+ +
+ + +
+ +
+ + +
+ +
+ Preview +
+ + + + +
+
+
+ +
+
+
+ + help +
+
+ + 10 +
+
+
+ +
+
+ + +
+ +
+ + +
+ +
+ + +
+ +
+ + +
+
+
+
+
+
@@ -242,108 +333,6 @@
- - -
-
- - -
-
- -

- Selected Bone: None -

- -
-
- - -
-
-
- Transform -
- - - - -
-
-
- Space -
- - - - -
-
-
-
- -
- -
- Preview - -
- - - - -
- -
- -
- - -
- -
- - -
- - -
-
- - - help -
-
- - - 0.50 -
-
-
- - -
- - -
- - diff --git a/src/lib/CustomSkeletonHelper.ts b/src/lib/CustomSkeletonHelper.ts index 5d68bb3..184d092 100644 --- a/src/lib/CustomSkeletonHelper.ts +++ b/src/lib/CustomSkeletonHelper.ts @@ -5,6 +5,7 @@ import { Color, Matrix4, Vector3, Points, PointsMaterial, BufferGeometry, Float32BufferAttribute, TextureLoader, LineSegments, LineBasicMaterial, type Bone, type ColorRepresentation } from 'three' import { BoneCategory, BoneClassifier } from './solvers/BoneClassifier' +import { Utility } from './Utilities' const _vector = /*@__PURE__*/ new Vector3() const _boneMatrix = /*@__PURE__*/ new Matrix4() @@ -24,8 +25,10 @@ const bone_category_colors: Record = { class CustomSkeletonHelper extends LineSegments { private readonly joint_points: Points + private readonly small_joint_points: Points private readonly jointTexture = new TextureLoader().load('/images/skeleton-joint-point.png') private hide_right_side_joints: boolean = false + private hidden_chain_root_names: Set = new Set() constructor (object: any, options: CustomSkeletonHelperOptions = {}) { const bones = getBoneList(object) @@ -84,6 +87,21 @@ class CustomSkeletonHelper extends LineSegments { const pointPositions = new Float32BufferAttribute(bones.length * 3, 3) pointsGeometry.setAttribute('position', pointPositions) + const smallPointsGeometry = new BufferGeometry() + const smallPointsMaterial = new PointsMaterial({ + size: 10, // smaller circles for detailed finger joints + color: options.jointColor !== undefined ? joint_color : 0xffffff, + depthTest: false, + sizeAttenuation: false, + map: this.jointTexture, + transparent: true, + opacity: 0.8, + vertexColors: options.jointColor === undefined + }) + + const smallPointPositions = new Float32BufferAttribute(bones.length * 3, 3) + smallPointsGeometry.setAttribute('position', smallPointPositions) + // use bone category to color joints to help with seeing a bunch of them if (options.jointColor === undefined) { const bone_classifier = new BoneClassifier(bones as Bone[]) @@ -95,16 +113,21 @@ class CustomSkeletonHelper extends LineSegments { point_colors.push(category_color.r, category_color.g, category_color.b) }) - pointsGeometry.setAttribute('color', new Float32BufferAttribute(point_colors, 3)) + const point_color_attribute = new Float32BufferAttribute(point_colors, 3) + pointsGeometry.setAttribute('color', point_color_attribute) + smallPointsGeometry.setAttribute('color', point_color_attribute.clone()) } this.joint_points = new Points(pointsGeometry, pointsMaterial) + this.small_joint_points = new Points(smallPointsGeometry, smallPointsMaterial) this.add(this.joint_points) + this.add(this.small_joint_points) } updateMatrixWorld (force: boolean): void { const bones = this.bones const pointPositions = this.joint_points.geometry.getAttribute('position') + const smallPointPositions = this.small_joint_points.geometry.getAttribute('position') const geometry = this.geometry const positions = geometry.getAttribute('position') @@ -117,25 +140,45 @@ class CustomSkeletonHelper extends LineSegments { _boneMatrix.multiplyMatrices(_matrixWorldInv, bone.matrixWorld) _vector.setFromMatrixPosition(_boneMatrix) - if (this.hide_right_side_joints && is_right_side_bone(bone)) { + const is_hidden_by_chain = this.is_bone_hidden_by_chain(bone) + + if ((this.hide_right_side_joints && is_right_side_bone(bone)) || is_hidden_by_chain) { pointPositions.setXYZ(i, Number.NaN, Number.NaN, Number.NaN) + smallPointPositions.setXYZ(i, Number.NaN, Number.NaN, Number.NaN) + } else if (is_small_joint_bone(bone)) { + pointPositions.setXYZ(i, Number.NaN, Number.NaN, Number.NaN) + smallPointPositions.setXYZ(i, _vector.x, _vector.y, _vector.z) } else { pointPositions.setXYZ(i, _vector.x, _vector.y, _vector.z) // Update point position + smallPointPositions.setXYZ(i, Number.NaN, Number.NaN, Number.NaN) } if (bone.parent && bone.parent.isBone) { - _boneMatrix.multiplyMatrices(_matrixWorldInv, bone.parent.matrixWorld) - _vector.setFromMatrixPosition(_boneMatrix) - positions.setXYZ(lineIndex * 2, _vector.x, _vector.y, _vector.z) + const parent_bone = bone.parent as Bone + const hide_line = + (this.hide_right_side_joints && (is_right_side_bone(parent_bone) || is_right_side_bone(bone))) || + is_hidden_by_chain || + this.is_bone_hidden_by_chain(parent_bone) + + if (hide_line) { + positions.setXYZ(lineIndex * 2, Number.NaN, Number.NaN, Number.NaN) + positions.setXYZ(lineIndex * 2 + 1, Number.NaN, Number.NaN, Number.NaN) + } else { + _boneMatrix.multiplyMatrices(_matrixWorldInv, parent_bone.matrixWorld) + _vector.setFromMatrixPosition(_boneMatrix) + positions.setXYZ(lineIndex * 2, _vector.x, _vector.y, _vector.z) + + _boneMatrix.multiplyMatrices(_matrixWorldInv, bone.matrixWorld) + _vector.setFromMatrixPosition(_boneMatrix) + positions.setXYZ(lineIndex * 2 + 1, _vector.x, _vector.y, _vector.z) + } - _boneMatrix.multiplyMatrices(_matrixWorldInv, bone.matrixWorld) - _vector.setFromMatrixPosition(_boneMatrix) - positions.setXYZ(lineIndex * 2 + 1, _vector.x, _vector.y, _vector.z) lineIndex++ } } pointPositions.needsUpdate = true + smallPointPositions.needsUpdate = true positions.needsUpdate = true // Update bounding box and bounding sphere @@ -151,6 +194,8 @@ class CustomSkeletonHelper extends LineSegments { this.material.dispose() this.joint_points.geometry.dispose() this.joint_points.material.dispose() + this.small_joint_points.geometry.dispose() + this.small_joint_points.material.dispose() } public show(): void { @@ -163,11 +208,20 @@ class CustomSkeletonHelper extends LineSegments { public setJointsVisible (visible: boolean): void { this.joint_points.visible = visible + this.small_joint_points.visible = visible } public setHideRightSideJoints (value: boolean): void { this.hide_right_side_joints = value } + + public setHiddenChainRoots (chain_root_names: string[]): void { + this.hidden_chain_root_names = new Set(chain_root_names) + } + + private is_bone_hidden_by_chain (bone: Bone): boolean { + return this.hidden_chain_root_names.has(Utility.chain_root_bone_from_bone(bone).name) + } } function getBoneList (object: any): any[] { @@ -190,4 +244,9 @@ function is_right_side_bone (bone: Bone): boolean { return /(^right_|^r_|_right$|_r$|\.right$|\.r$|-right$|-r$)/.test(normalized_bone_name) } +function is_small_joint_bone (bone: Bone): boolean { + const normalized_bone_name = bone.name.toLowerCase() + return /(thumb|index|middle|ring|pinky|finger)/.test(normalized_bone_name) +} + export { CustomSkeletonHelper } diff --git a/src/lib/CustomViewHelper.ts b/src/lib/CustomViewHelper.ts index 55e1afb..b695532 100644 --- a/src/lib/CustomViewHelper.ts +++ b/src/lib/CustomViewHelper.ts @@ -60,7 +60,7 @@ export class CustomViewHelper extends Object3D { private readonly negYAxisHelper: Sprite private readonly negZAxisHelper: Sprite private readonly point: Vector3 = new Vector3() - private readonly dim: number = 128 + private readonly dim: number = 256 private readonly turnRate: number = 2 * Math.PI private readonly targetPosition: Vector3 = new Vector3() private readonly targetQuaternion: Quaternion = new Quaternion() diff --git a/src/lib/EventListeners.ts b/src/lib/EventListeners.ts index 94c24e7..167d6d3 100644 --- a/src/lib/EventListeners.ts +++ b/src/lib/EventListeners.ts @@ -5,6 +5,7 @@ import { TransformSpace } from './enums/TransformSpace' import { Utility } from './Utilities' import { ModelCleanupUtility } from './processes/load-model/ModelCleanupUtility' import { type Bone } from 'three' +import { SkeletonType } from './enums/SkeletonType' export class EventListeners { constructor (private readonly bootstrap: Mesh2MotionEngine) {} @@ -17,6 +18,11 @@ export class EventListeners { this.bootstrap.load_skeleton_step.addEventListener('skeletonLoaded', () => { this.bootstrap.edit_skeleton_step.load_original_armature_from_model(this.bootstrap.load_skeleton_step.armature()) + this.bootstrap.mesh_drag_bone_placement.snap_primary_centerline_bones_to_mesh_center() + + if (this.bootstrap.load_skeleton_step.skeleton_type() === SkeletonType.Fox) { + this.bootstrap.mesh_drag_bone_placement.spread_spine_chain_for_fox() + } this.bootstrap.process_step = this.bootstrap.process_step_changed(ProcessStep.EditSkeleton) }) @@ -43,6 +49,20 @@ export class EventListeners { this.bootstrap.update_edit_bone_interaction_mode() }) + this.bootstrap.edit_skeleton_step.addEventListener('orbit-rotation-changed', () => { + this.bootstrap.update_orbit_controls_state(true) + }) + + this.bootstrap.edit_skeleton_step.addEventListener('chainVisibilityChanged', () => { + this.bootstrap.sync_skeleton_helper_joint_visibility() + + if (this.bootstrap.transform_controls.object !== undefined && + this.bootstrap.transform_controls.object !== null && + !this.bootstrap.edit_skeleton_step.is_bone_selectable(this.bootstrap.transform_controls.object as Bone)) { + this.bootstrap.transform_controls.detach() + } + }) + // attribution link clicking brings up contributors dialog this.bootstrap.ui.dom_attribution_link?.addEventListener('click', (event: MouseEvent) => { event.preventDefault() @@ -105,7 +125,7 @@ export class EventListeners { // we can know about the "mouseup" event with this this.bootstrap.transform_controls?.addEventListener('dragging-changed', (event: any) => { this.bootstrap.is_transform_controls_dragging = event.value - this.bootstrap.enable_orbit_controls(!event.value) + this.bootstrap.update_orbit_controls_state(!event.value) // Store undo state when we start dragging (event.value = true) if (event.value && this.bootstrap.process_step === ProcessStep.EditSkeleton) { diff --git a/src/lib/SceneEnvironmentManager.ts b/src/lib/SceneEnvironmentManager.ts index 486effb..afee821 100644 --- a/src/lib/SceneEnvironmentManager.ts +++ b/src/lib/SceneEnvironmentManager.ts @@ -48,7 +48,7 @@ export class SceneEnvironmentManager { // center orbit controls around mid-section area with target change this.controls = new OrbitControls(this.camera, this.renderer.domElement) - this.controls.target.set(0, 0.9, 0) + this.controls.target.set(0, 1.2, 0) // Set zoom limits to prevent excessive zooming in or out this.controls.minDistance = 0.5 // Minimum zoom (closest to model) @@ -91,6 +91,10 @@ export class SceneEnvironmentManager { this.controls?.update() } + public get_orbit_target (): Vector3 | null { + return this.controls?.target ?? null + } + public set_zoom_limits (min_distance: number, max_distance: number): void { if (this.controls !== undefined) { this.controls.minDistance = min_distance diff --git a/src/lib/UI.ts b/src/lib/UI.ts index b6b392e..f6d2615 100644 --- a/src/lib/UI.ts +++ b/src/lib/UI.ts @@ -27,6 +27,11 @@ export class UI { dom_mirror_skeleton_checkbox: HTMLInputElement | null = null dom_independent_bone_movement_checkbox: HTMLInputElement | null = null dom_mesh_drag_placement_checkbox: HTMLInputElement | null = null + dom_mesh_drag_centerline_snap_checkbox: HTMLInputElement | null = null + dom_disable_orbit_rotation_checkbox: HTMLInputElement | null = null + dom_mesh_drag_snap_strength_input: HTMLInputElement | null = null + dom_mesh_drag_snap_strength_label: HTMLElement | null = null + dom_mesh_drag_snap_strength_container: HTMLElement | null = null dom_scale_skeleton_button: HTMLButtonElement | null = null dom_undo_button: HTMLButtonElement | null = null dom_redo_button: HTMLButtonElement | null = null @@ -42,6 +47,7 @@ export class UI { // edit skeleton UI step controls dom_selected_bone_label: HTMLElement | null = null + dom_bone_chain_visibility_container: HTMLElement | null = null dom_transform_manual_options: HTMLElement | null = null dom_transform_type_radio_group: HTMLElement | null = null dom_transform_space_radio_group: HTMLElement | null = null @@ -148,6 +154,11 @@ export class UI { this.dom_mirror_skeleton_checkbox = document.querySelector('#mirror-skeleton') this.dom_independent_bone_movement_checkbox = document.querySelector('#independent-bone-movement') this.dom_mesh_drag_placement_checkbox = document.querySelector('#mesh-drag-placement') + this.dom_mesh_drag_centerline_snap_checkbox = document.querySelector('#mesh-drag-centerline-snap') + this.dom_disable_orbit_rotation_checkbox = document.querySelector('#disable-rotation-spinner') + this.dom_mesh_drag_snap_strength_input = document.querySelector('#mesh-drag-snap-strength-input') + this.dom_mesh_drag_snap_strength_label = document.querySelector('#mesh-drag-snap-strength-label') + this.dom_mesh_drag_snap_strength_container = document.querySelector('#mesh-drag-snap-strength-container') this.dom_scale_skeleton_button = document.querySelector('#scale-skeleton-button') this.dom_reset_skeleton_scale_button = document.querySelector('#reset-skeleton-scale-button') @@ -155,6 +166,7 @@ export class UI { this.dom_redo_button = document.querySelector('#redo-button') this.dom_selected_bone_label = document.querySelector('#edit-selected-bone-label') + this.dom_bone_chain_visibility_container = document.querySelector('#bone-chain-visibility-container') this.dom_transform_manual_options = document.querySelector('#transform-manual-options') this.dom_transform_type_radio_group = document.querySelector('#transform-control-type-group') diff --git a/src/lib/Utilities.test.ts b/src/lib/Utilities.test.ts new file mode 100644 index 0000000..27bd96a --- /dev/null +++ b/src/lib/Utilities.test.ts @@ -0,0 +1,230 @@ +import { describe, expect, it } from 'vitest' +import { Bone, Skeleton } from 'three' + +import { Utility } from './Utilities' + +function build_test_skeleton (): Skeleton { + const root = new Bone() + root.name = 'root' + + const hips = new Bone() + hips.name = 'Hips' + root.add(hips) + + const spine = new Bone() + spine.name = 'Spine' + hips.add(spine) + + const left_leg = new Bone() + left_leg.name = 'LeftLeg' + hips.add(left_leg) + + const right_leg = new Bone() + right_leg.name = 'RightLeg' + hips.add(right_leg) + + const chest = new Bone() + chest.name = 'Chest' + spine.add(chest) + + const left_shoulder = new Bone() + left_shoulder.name = 'LeftShoulder' + chest.add(left_shoulder) + + const left_forearm = new Bone() + left_forearm.name = 'LeftForeArm' + left_shoulder.add(left_forearm) + + const left_hand = new Bone() + left_hand.name = 'LeftHand' + left_forearm.add(left_hand) + + const left_thumb = new Bone() + left_thumb.name = 'LeftHandThumb1' + left_hand.add(left_thumb) + + const left_index = new Bone() + left_index.name = 'LeftHandIndex1' + left_hand.add(left_index) + + const right_shoulder = new Bone() + right_shoulder.name = 'RightShoulder' + chest.add(right_shoulder) + + const head = new Bone() + head.name = 'Head' + chest.add(head) + + return new Skeleton([ + root, + hips, + spine, + left_leg, + right_leg, + chest, + left_shoulder, + left_forearm, + left_hand, + left_thumb, + left_index, + right_shoulder, + head + ]) +} + +function build_quadruped_test_skeleton (): Skeleton { + const root = new Bone() + root.name = 'root' + + const pelvis = new Bone() + pelvis.name = 'Pelvis' + root.add(pelvis) + + const spine1 = new Bone() + spine1.name = 'Spine1' + pelvis.add(spine1) + + const spine2 = new Bone() + spine2.name = 'Spine2' + spine1.add(spine2) + + const chest = new Bone() + chest.name = 'Chest' + spine2.add(chest) + + const head = new Bone() + head.name = 'Head' + chest.add(head) + + const head_tip = new Bone() + head_tip.name = 'HeadTip' + head.add(head_tip) + + const front_leg_shoulder_l = new Bone() + front_leg_shoulder_l.name = 'FrontLegShoulder_L' + pelvis.add(front_leg_shoulder_l) + + const front_leg_upper_l = new Bone() + front_leg_upper_l.name = 'FrontLegUpper_L' + front_leg_shoulder_l.add(front_leg_upper_l) + + const front_leg_foot_l = new Bone() + front_leg_foot_l.name = 'FrontLegFoot_L' + front_leg_upper_l.add(front_leg_foot_l) + + const front_leg_foot1_l = new Bone() + front_leg_foot1_l.name = 'FrontLegFoot1_L' + front_leg_foot_l.add(front_leg_foot1_l) + + const front_leg_shoulder_r = new Bone() + front_leg_shoulder_r.name = 'FrontLegShoulder_R' + pelvis.add(front_leg_shoulder_r) + + const front_leg_upper_r = new Bone() + front_leg_upper_r.name = 'FrontLegUpper_R' + front_leg_shoulder_r.add(front_leg_upper_r) + + const front_leg_foot_r = new Bone() + front_leg_foot_r.name = 'FrontLegFoot_R' + front_leg_upper_r.add(front_leg_foot_r) + + const back_leg_pelvis_l = new Bone() + back_leg_pelvis_l.name = 'BackLegPelvis_L' + pelvis.add(back_leg_pelvis_l) + + const back_leg_upper_l = new Bone() + back_leg_upper_l.name = 'BackLegUpper_L' + back_leg_pelvis_l.add(back_leg_upper_l) + + const back_leg_foot_l = new Bone() + back_leg_foot_l.name = 'BackLegFoot_L' + back_leg_upper_l.add(back_leg_foot_l) + + const back_leg_foot1_l = new Bone() + back_leg_foot1_l.name = 'BackLegFoot1_L' + back_leg_foot_l.add(back_leg_foot1_l) + + const back_leg_pelvis_r = new Bone() + back_leg_pelvis_r.name = 'BackLegPelvis_R' + pelvis.add(back_leg_pelvis_r) + + const back_leg_upper_r = new Bone() + back_leg_upper_r.name = 'BackLegUpper_R' + back_leg_pelvis_r.add(back_leg_upper_r) + + const back_leg_foot_r = new Bone() + back_leg_foot_r.name = 'BackLegFoot_R' + back_leg_upper_r.add(back_leg_foot_r) + + const back_leg_foot1_r = new Bone() + back_leg_foot1_r.name = 'BackLegFoot1_R' + back_leg_foot_r.add(back_leg_foot1_r) + + return new Skeleton([ + root, + pelvis, + spine1, + spine2, + chest, + head, + head_tip, + front_leg_shoulder_l, + front_leg_upper_l, + front_leg_foot_l, + front_leg_foot1_l, + front_leg_shoulder_r, + front_leg_upper_r, + front_leg_foot_r, + back_leg_pelvis_l, + back_leg_upper_l, + back_leg_foot_l, + back_leg_foot1_l, + back_leg_pelvis_r, + back_leg_upper_r, + back_leg_foot_r, + back_leg_foot1_r + ]) +} + +describe('Utility chain roots', () => { + it('derives condensed main chains and folds fingers into the hand chain', () => { + const skeleton = build_test_skeleton() + const chain_root_names = Utility.unique_chain_root_bones_from_skeleton(skeleton).map((bone) => bone.name) + + expect(chain_root_names).toEqual(['Hips', 'Spine', 'LeftLeg', 'RightLeg', 'LeftShoulder', 'LeftHand', 'RightShoulder', 'Head']) + }) + + it('maps descendant bones back to their condensed chain root', () => { + const skeleton = build_test_skeleton() + const chest_bone = skeleton.bones.find((bone) => bone.name === 'Chest') + const head_bone = skeleton.bones.find((bone) => bone.name === 'Head') + const thumb_bone = skeleton.bones.find((bone) => bone.name === 'LeftHandThumb1') + const hand_bone = skeleton.bones.find((bone) => bone.name === 'LeftHand') + + expect(chest_bone).toBeDefined() + expect(head_bone).toBeDefined() + expect(thumb_bone).toBeDefined() + expect(hand_bone).toBeDefined() + expect(Utility.chain_root_bone_from_bone(chest_bone!)).toBe(skeleton.bones.find((bone) => bone.name === 'Spine')) + expect(Utility.chain_root_bone_from_bone(head_bone!)).toBe(head_bone) + expect(Utility.chain_root_bone_from_bone(thumb_bone!)).toBe(hand_bone) + }) + + it('condenses spine, head, and quadruped leg chains', () => { + const skeleton = build_quadruped_test_skeleton() + const chest_bone = skeleton.bones.find((bone) => bone.name === 'Chest') + const head_tip_bone = skeleton.bones.find((bone) => bone.name === 'HeadTip') + const front_leg_foot_l = skeleton.bones.find((bone) => bone.name === 'FrontLegFoot_L') + const back_leg_foot1_r = skeleton.bones.find((bone) => bone.name === 'BackLegFoot1_R') + + expect(chest_bone).toBeDefined() + expect(head_tip_bone).toBeDefined() + expect(front_leg_foot_l).toBeDefined() + expect(back_leg_foot1_r).toBeDefined() + + expect(Utility.chain_root_bone_from_bone(chest_bone!)).toBe(skeleton.bones.find((bone) => bone.name === 'Spine1')) + expect(Utility.chain_root_bone_from_bone(head_tip_bone!)).toBe(skeleton.bones.find((bone) => bone.name === 'Head')) + expect(Utility.chain_root_bone_from_bone(front_leg_foot_l!)).toBe(skeleton.bones.find((bone) => bone.name === 'FrontLegShoulder_L')) + expect(Utility.chain_root_bone_from_bone(back_leg_foot1_r!)).toBe(skeleton.bones.find((bone) => bone.name === 'BackLegPelvis_R')) + }) +}) \ No newline at end of file diff --git a/src/lib/Utilities.ts b/src/lib/Utilities.ts index 149ae95..917eabc 100644 --- a/src/lib/Utilities.ts +++ b/src/lib/Utilities.ts @@ -121,6 +121,203 @@ export class Utility { return bones } + static child_bones_from_object (object: Object3D): Bone[] { + return object.children.filter((child): child is Bone => child instanceof Bone) + } + + static is_chain_root_bone (bone: Bone): boolean { + const grouped_chain_anchor = Utility.grouped_chain_anchor_bone_from_bone(bone) + if (grouped_chain_anchor !== null) { + return grouped_chain_anchor === bone + } + + if (bone.name === 'root') { + return false + } + + const parent = bone.parent + if (!(parent instanceof Bone)) { + return true + } + + if (parent.name === 'root') { + return true + } + + return Utility.child_bones_from_object(parent).length > 1 + } + + static chain_root_bone_from_bone (bone: Bone): Bone { + const grouped_chain_anchor = Utility.grouped_chain_anchor_bone_from_bone(bone) + if (grouped_chain_anchor !== null) { + return grouped_chain_anchor + } + + let current_bone: Bone = bone + + while (current_bone.parent instanceof Bone && !Utility.is_chain_root_bone(current_bone)) { + current_bone = current_bone.parent + } + + return current_bone + } + + static unique_chain_root_bones_from_skeleton (skeleton: Skeleton): Bone[] { + const chain_roots: Bone[] = [] + const seen_names = new Set() + + skeleton.bones.forEach((bone) => { + if (!Utility.is_chain_root_bone(bone) || seen_names.has(bone.name)) { + return + } + + seen_names.add(bone.name) + chain_roots.push(bone) + }) + + return chain_roots + } + + private static grouped_chain_anchor_bone_from_bone (bone: Bone): Bone | null { + const normalized_bone_name = bone.name.toLowerCase() + + if (Utility.is_front_leg_chain_bone_name(normalized_bone_name)) { + const front_leg_anchor = Utility.find_nearest_bone_ancestor_including_self(bone, (candidate_name) => + Utility.is_front_leg_anchor_bone_name(candidate_name)) + + if (front_leg_anchor !== null) { + return front_leg_anchor + } + } + + if (Utility.is_back_leg_chain_bone_name(normalized_bone_name)) { + const back_leg_anchor = Utility.find_nearest_bone_ancestor_including_self(bone, (candidate_name) => + Utility.is_back_leg_anchor_bone_name(candidate_name)) + + if (back_leg_anchor !== null) { + return back_leg_anchor + } + } + + if (Utility.is_spine_chain_bone_name(normalized_bone_name)) { + const spine_anchor = Utility.find_rootmost_bone_ancestor_including_self(bone, (candidate_name) => + Utility.is_spine_anchor_bone_name(candidate_name)) + + if (spine_anchor !== null) { + return spine_anchor + } + } + + if (Utility.is_head_chain_bone_name(normalized_bone_name)) { + const head_anchor = Utility.find_nearest_bone_ancestor_including_self(bone, (candidate_name) => + Utility.is_head_anchor_bone_name(candidate_name)) + + if (head_anchor !== null) { + return head_anchor + } + } + + if (Utility.is_hand_chain_bone_name(normalized_bone_name)) { + return Utility.find_nearest_bone_ancestor_including_self(bone, (candidate_name) => Utility.is_hand_anchor_bone_name(candidate_name)) + } + + if (Utility.is_foot_chain_bone_name(normalized_bone_name)) { + return Utility.find_nearest_bone_ancestor_including_self(bone, (candidate_name) => Utility.is_foot_anchor_bone_name(candidate_name)) + } + + return null + } + + private static find_nearest_bone_ancestor_including_self ( + bone: Bone, + matcher: (normalized_bone_name: string) => boolean + ): Bone | null { + let current_bone: Bone | null = bone + + while (current_bone !== null) { + if (matcher(current_bone.name.toLowerCase())) { + return current_bone + } + + current_bone = current_bone.parent instanceof Bone ? current_bone.parent : null + } + + return null + } + + private static find_rootmost_bone_ancestor_including_self ( + bone: Bone, + matcher: (normalized_bone_name: string) => boolean + ): Bone | null { + let current_bone: Bone | null = bone + let rootmost_match: Bone | null = null + + while (current_bone !== null) { + if (matcher(current_bone.name.toLowerCase())) { + rootmost_match = current_bone + } + + current_bone = current_bone.parent instanceof Bone ? current_bone.parent : null + } + + return rootmost_match + } + + private static is_hand_chain_bone_name (normalized_bone_name: string): boolean { + return /(hand|thumb|index|middle|ring|pinky|finger)/.test(normalized_bone_name) + } + + private static is_hand_anchor_bone_name (normalized_bone_name: string): boolean { + return normalized_bone_name.includes('hand') && !/(thumb|index|middle|ring|pinky|finger)/.test(normalized_bone_name) + } + + private static is_foot_chain_bone_name (normalized_bone_name: string): boolean { + return /(foot|toe|ball)/.test(normalized_bone_name) + } + + private static is_foot_anchor_bone_name (normalized_bone_name: string): boolean { + return normalized_bone_name.includes('foot') && !/(toe|ball)/.test(normalized_bone_name) + } + + private static is_head_chain_bone_name (normalized_bone_name: string): boolean { + return /(head|ear|eye|jaw|chin|mouth|teeth|tongue|horn|antler|nose|brow|lash)/.test(normalized_bone_name) + } + + private static is_head_anchor_bone_name (normalized_bone_name: string): boolean { + return normalized_bone_name.includes('head') && !/(tip|end|nub)/.test(normalized_bone_name) + } + + private static is_spine_chain_bone_name (normalized_bone_name: string): boolean { + return /(spine|chest|upperchest|torso|ribcage)/.test(normalized_bone_name) + } + + private static is_spine_anchor_bone_name (normalized_bone_name: string): boolean { + return /(spine|torso)/.test(normalized_bone_name) + } + + private static is_front_leg_chain_bone_name (normalized_bone_name: string): boolean { + return /(front|fore)[-_ ]?leg/.test(normalized_bone_name) + } + + private static is_front_leg_anchor_bone_name (normalized_bone_name: string): boolean { + return /(front|fore)[-_ ]?leg/.test(normalized_bone_name) && /(shoulder|scapula)/.test(normalized_bone_name) + } + + private static is_back_leg_chain_bone_name (normalized_bone_name: string): boolean { + return /(back|rear|hind)[-_ ]?leg/.test(normalized_bone_name) + } + + private static is_back_leg_anchor_bone_name (normalized_bone_name: string): boolean { + return /(back|rear|hind)[-_ ]?leg/.test(normalized_bone_name) && /(pelvis|hip)/.test(normalized_bone_name) + } + + static format_bone_chain_label (bone_name: string): string { + return bone_name + .replace(/[_-]+/g, ' ') + .replace(/([a-z])([A-Z])/g, '$1 $2') + .trim() + } + static intersection_points_between_positions_and_mesh (positions: BufferAttribute | InterleavedBufferAttribute, envelope_mesh: Mesh): IntersectionPointData { const vertex_positions_inside_bone_envelope: Vector3[] = [] diff --git a/src/lib/processes/edit-skeleton/IndependentBoneMovement.test.ts b/src/lib/processes/edit-skeleton/IndependentBoneMovement.test.ts new file mode 100644 index 0000000..d74ee98 --- /dev/null +++ b/src/lib/processes/edit-skeleton/IndependentBoneMovement.test.ts @@ -0,0 +1,50 @@ +import { describe, expect, it } from 'vitest' +import { Bone, Quaternion, Skeleton } from 'three' + +import { IndependentBoneMovement } from './IndependentBoneMovement.ts' + +function quaternion_angle_difference (left: Quaternion, right: Quaternion): number { + const dot = Math.min(1, Math.abs(left.dot(right))) + return 2 * Math.acos(dot) +} + +describe('IndependentBoneMovement', () => { + it('keeps a branching chest parent at its rest rotation when moving the clavicle', () => { + const root = new Bone() + root.name = 'root' + + const chest = new Bone() + chest.name = 'chest' + root.add(chest) + + const neck = new Bone() + neck.name = 'neck' + neck.position.set(0, 1, 0) + chest.add(neck) + + const clavicle = new Bone() + clavicle.name = 'leftClavicle' + clavicle.position.set(1, 0.15, 0) + chest.add(clavicle) + + const upper_arm = new Bone() + upper_arm.name = 'leftUpperArm' + upper_arm.position.set(1, 0, 0) + clavicle.add(upper_arm) + + root.updateWorldMatrix(true, true) + + const movement = new IndependentBoneMovement() + movement.set_rest_pose(new Skeleton([root, chest, neck, clavicle, upper_arm])) + + const chest_rest_rotation = chest.getWorldQuaternion(new Quaternion()).clone() + + clavicle.position.set(0.55, 0.35, -0.45) + root.updateWorldMatrix(true, true) + + movement.finalize_drop(clavicle) + + const chest_final_rotation = chest.getWorldQuaternion(new Quaternion()) + expect(quaternion_angle_difference(chest_final_rotation, chest_rest_rotation)).toBeLessThan(1e-5) + }) +}) \ No newline at end of file diff --git a/src/lib/processes/edit-skeleton/IndependentBoneMovement.ts b/src/lib/processes/edit-skeleton/IndependentBoneMovement.ts index 434b969..05c394e 100644 --- a/src/lib/processes/edit-skeleton/IndependentBoneMovement.ts +++ b/src/lib/processes/edit-skeleton/IndependentBoneMovement.ts @@ -105,7 +105,7 @@ export class IndependentBoneMovement { ? bone.parent : null - if (parent_bone !== null) { + if (parent_bone !== null && this._should_update_parent_rotation_from_child_translation(parent_bone)) { // Also snapshot siblings so they can be re-pinned after parent rotates parent_bone.children.forEach((sibling) => { if (sibling === bone || !this._is_bone(sibling)) { return } @@ -293,4 +293,12 @@ export class IndependentBoneMovement { return (value as { isBone?: boolean }).isBone === true } + + private _should_update_parent_rotation_from_child_translation (bone: Bone): boolean { + // Branching parents like chest/pelvis/clavicle hubs should preserve their + // template orientation when a single child is translated; otherwise the + // averaged child-direction solve can twist the torso/shoulder backward. + const bone_children = bone.children.filter((child): child is Bone => this._is_bone(child)) + return bone_children.length <= 1 + } } diff --git a/src/lib/processes/edit-skeleton/MeshDragBonePlacement.test.ts b/src/lib/processes/edit-skeleton/MeshDragBonePlacement.test.ts new file mode 100644 index 0000000..0e744a8 --- /dev/null +++ b/src/lib/processes/edit-skeleton/MeshDragBonePlacement.test.ts @@ -0,0 +1,41 @@ +import { describe, expect, it } from 'vitest' +import { Vector3 } from 'three' + +import { + apply_mesh_centerline_target, + blend_target_with_snap_vertex, + calculate_vertex_snap_influence, + is_centerline_mesh_snap_bone_name +} from './MeshDragBonePlacement.ts' + +describe('MeshDragBonePlacement', () => { + it('maps snap strength from 0 to 20 into a 0 to 1 blend factor', () => { + expect(calculate_vertex_snap_influence(0)).toBe(0) + expect(calculate_vertex_snap_influence(10)).toBe(0.5) + expect(calculate_vertex_snap_influence(20)).toBe(1) + }) + + it('blends the target position toward the snap vertex based on strength', () => { + const midpoint = new Vector3(0, 0, 0) + const snap_vertex = new Vector3(10, 0, 0) + + expect(blend_target_with_snap_vertex(midpoint, snap_vertex, 0).toArray()).toEqual([0, 0, 0]) + expect(blend_target_with_snap_vertex(midpoint, snap_vertex, 10).toArray()).toEqual([5, 0, 0]) + expect(blend_target_with_snap_vertex(midpoint, snap_vertex, 20).toArray()).toEqual([10, 0, 0]) + }) + + it('identifies center-line torso and head bones for mesh centering', () => { + expect(is_centerline_mesh_snap_bone_name('Pelvis')).toBe(true) + expect(is_centerline_mesh_snap_bone_name('Spine2')).toBe(true) + expect(is_centerline_mesh_snap_bone_name('Neck')).toBe(true) + expect(is_centerline_mesh_snap_bone_name('Head')).toBe(true) + expect(is_centerline_mesh_snap_bone_name('LeftHead')).toBe(false) + expect(is_centerline_mesh_snap_bone_name('RightShoulder')).toBe(false) + }) + + it('keeps the clicked height while snapping torso bones onto the mesh centerline', () => { + const target = new Vector3(1.25, 1.8, -0.35) + + expect(apply_mesh_centerline_target(target, 0.1, 0.05).toArray()).toEqual([0.1, 1.8, 0.05]) + }) +}) \ No newline at end of file diff --git a/src/lib/processes/edit-skeleton/MeshDragBonePlacement.ts b/src/lib/processes/edit-skeleton/MeshDragBonePlacement.ts index e08336e..62ee7bb 100644 --- a/src/lib/processes/edit-skeleton/MeshDragBonePlacement.ts +++ b/src/lib/processes/edit-skeleton/MeshDragBonePlacement.ts @@ -2,15 +2,63 @@ import * as THREE from 'three' import { type OrbitControls } from 'three/examples/jsm/controls/OrbitControls.js' import { type TransformControls } from 'three/examples/jsm/controls/TransformControls.js' import { ProcessStep } from '../../enums/ProcessStep.ts' +import { SkeletonType } from '../../enums/SkeletonType.ts' import { Utility } from '../../Utilities.ts' import { type StepEditSkeleton } from './StepEditSkeleton.ts' import { type StepLoadModel } from '../load-model/StepLoadModel.ts' import { type StepWeightSkin } from '../weight-skin/StepWeightSkin.ts' -import { type PerspectiveCamera, type Vector3, type Object3D, type Skeleton } from 'three' +import { Bone, type PerspectiveCamera, Vector3, type Object3D, type Skeleton, type Intersection, type BufferAttribute, type Mesh, type SkinnedMesh } from 'three' + +export function calculate_vertex_snap_influence (snap_strength: number): number { + const clamped_strength = Math.max(0, Math.min(20, snap_strength)) + return clamped_strength / 20 +} + +export function blend_target_with_snap_vertex ( + target_world_position: Vector3, + snap_vertex_world_position: Vector3 | null, + snap_strength: number +): Vector3 { + const snap_influence = calculate_vertex_snap_influence(snap_strength) + if (snap_influence <= 0 || snap_vertex_world_position === null) { + return target_world_position.clone() + } + + return target_world_position.clone().lerp(snap_vertex_world_position, snap_influence) +} + +export function is_centerline_mesh_snap_bone_name (bone_name: string): boolean { + const normalized_bone_name = bone_name.toLowerCase() + const has_side_marker = + normalized_bone_name.startsWith('left') || + normalized_bone_name.startsWith('right') || + /^l[_-]/.test(normalized_bone_name) || + /^r[_-]/.test(normalized_bone_name) || + /(^|[^a-z])(left|right)([^a-z]|$)/.test(normalized_bone_name) + + if (has_side_marker) { + return false + } + + return /(pelvis|hips|spine|chest|torso|abdomen|waist|neck|head)/.test(normalized_bone_name) +} + +export function apply_mesh_centerline_target ( + target_world_position: Vector3, + center_x: number, + center_z: number +): Vector3 { + return new Vector3(center_x, target_world_position.y, center_z) +} export class MeshDragBonePlacement { private orbit_controls: OrbitControls | undefined = undefined private is_dragging_mode_active: boolean = false + private closest_vertex_cache: { + object_uuid: string + face_key: string + closest_vertex_world_position: Vector3 | null + } | null = null constructor ( private readonly camera: PerspectiveCamera, @@ -37,16 +85,12 @@ export class MeshDragBonePlacement { if (using_mesh_drag_mode) { transform_controls.detach() - if (this.orbit_controls !== undefined) { - this.orbit_controls.enabled = true - } + this.set_orbit_controls_enabled(true) } if (this.is_dragging_mode_active && !using_mesh_drag_mode) { this.is_dragging_mode_active = false - if (this.orbit_controls !== undefined) { - this.orbit_controls.enabled = true - } + this.set_orbit_controls_enabled(true) } } @@ -56,6 +100,8 @@ export class MeshDragBonePlacement { return } + this.closest_vertex_cache = null + const skeleton_to_test: Skeleton | undefined = this.edit_skeleton_step.skeleton() if (skeleton_to_test === undefined) { return @@ -91,9 +137,7 @@ export class MeshDragBonePlacement { } this.is_dragging_mode_active = true - if (this.orbit_controls !== undefined) { - this.orbit_controls.enabled = false - } + this.set_orbit_controls_enabled(false) this.move_selected_bone_to_mesh_midpoint(mouse_event) } @@ -120,13 +164,130 @@ export class MeshDragBonePlacement { } this.is_dragging_mode_active = false - if (this.orbit_controls !== undefined) { - this.orbit_controls.enabled = true - } + this.closest_vertex_cache = null + this.set_orbit_controls_enabled(true) return true } + public snap_primary_centerline_bones_to_mesh_center (): void { + if (!this.edit_skeleton_step.is_mesh_drag_centerline_snap_enabled()) { + return + } + + const skeleton_to_snap = this.edit_skeleton_step.skeleton() + if (skeleton_to_snap === undefined) { + return + } + + const mesh_targets = this.get_centerline_mesh_targets() + const is_fox = this.edit_skeleton_step.get_skeleton_type() === SkeletonType.Fox + if (mesh_targets.length === 0 && !is_fox) { + return + } + + skeleton_to_snap.bones.forEach((bone) => { + if (!is_centerline_mesh_snap_bone_name(bone.name) || !(bone.parent instanceof Bone)) { + return + } + + const centered_world_position = this.get_centerline_target_at_world_position( + Utility.world_position_from_object(bone) + ) + + if (centered_world_position === null) { + return + } + + const centered_local_position = centered_world_position.clone() + bone.parent.worldToLocal(centered_local_position) + bone.position.copy(centered_local_position) + bone.updateWorldMatrix(true, true) + }) + } + + public spread_spine_chain_for_fox (): void { + const skeleton_to_adjust = this.edit_skeleton_step.skeleton() + if (skeleton_to_adjust === undefined) { + return + } + + const pelvis_bone = this.find_bone_by_name_match(skeleton_to_adjust, /(pelvis|hips)/) + const head_bone = this.find_bone_by_name_match(skeleton_to_adjust, /head/) + const spine_bones = this.find_spine_chain_bones(skeleton_to_adjust) + + const mesh_bounds = this.get_mesh_bounds() + if (mesh_bounds === null) { + return + } + + if (pelvis_bone === null || head_bone === null || spine_bones.length === 0) { + return + } + + const pelvis_world = Utility.world_position_from_object(pelvis_bone) + const head_world = Utility.world_position_from_object(head_bone) + + const bounds_size = mesh_bounds.getSize(new Vector3()) + const bounds_center = mesh_bounds.getCenter(new Vector3()) + const use_x_axis = bounds_size.x >= bounds_size.z + let axis_dir = use_x_axis ? new Vector3(1, 0, 0) : new Vector3(0, 0, 1) + + const head_hint = new Vector3(head_world.x, 0, head_world.z) + const pelvis_hint = new Vector3(pelvis_world.x, 0, pelvis_world.z) + const hint_axis = head_hint.clone().sub(pelvis_hint) + + if (hint_axis.lengthSq() > 0.0001) { + axis_dir = hint_axis.normalize() + } else { + const center_hint = new Vector3(bounds_center.x, 0, bounds_center.z) + if (head_hint.sub(center_hint).dot(axis_dir) < 0) { + axis_dir.multiplyScalar(-1) + } + } + + const axis_length = Math.max(use_x_axis ? bounds_size.x : bounds_size.z, 0.1) + const half_length = axis_length * 0.45 + const tail_position = bounds_center.clone().add(axis_dir.clone().multiplyScalar(-half_length)) + const head_position = bounds_center.clone().add(axis_dir.clone().multiplyScalar(half_length)) + + tail_position.y = pelvis_world.y + head_position.y = head_world.y + + const spine_axis = head_position.clone().sub(tail_position) + if (spine_axis.lengthSq() < 0.0001) { + return + } + + const spine_direction = spine_axis.clone().normalize() + spine_bones.sort((bone_a, bone_b) => { + const bone_a_pos = Utility.world_position_from_object(bone_a) + const bone_b_pos = Utility.world_position_from_object(bone_b) + const bone_a_depth = bone_a_pos.clone().sub(tail_position).dot(spine_direction) + const bone_b_depth = bone_b_pos.clone().sub(tail_position).dot(spine_direction) + + if (bone_a_depth === bone_b_depth) { + return bone_a.name.localeCompare(bone_b.name) + } + + return bone_a_depth - bone_b_depth + }) + + spine_bones.forEach((bone, index) => { + if (!(bone.parent instanceof Bone)) { + return + } + + const lerp_factor = (index + 1) / (spine_bones.length + 1) + const target_world_position = tail_position.clone().lerp(head_position, lerp_factor) + target_world_position.y = pelvis_world.y + (head_world.y - pelvis_world.y) * lerp_factor + const target_local_position = target_world_position.clone() + bone.parent.worldToLocal(target_local_position) + bone.position.copy(target_local_position) + bone.updateWorldMatrix(true, true) + }) + } + private move_selected_bone_to_mesh_midpoint (mouse_event: MouseEvent): void { const selected_bone = this.edit_skeleton_step.get_currently_selected_bone() @@ -134,12 +295,22 @@ export class MeshDragBonePlacement { return } - const intersection_segment = this.get_edit_mesh_intersection_segment(mouse_event) + const intersection_target = this.get_edit_mesh_intersection_target(mouse_event) let target_world_position: Vector3 | null = null - if (intersection_segment !== null) { - const [first_intersection, last_intersection] = intersection_segment - target_world_position = first_intersection.clone().add(last_intersection).multiplyScalar(0.5) + if (intersection_target !== null) { + if (this.edit_skeleton_step.is_mesh_drag_centerline_snap_enabled() && + is_centerline_mesh_snap_bone_name(selected_bone.name)) { + target_world_position = this.get_centerline_target_at_world_position(intersection_target.midpoint) + } + + if (target_world_position === null) { + target_world_position = blend_target_with_snap_vertex( + intersection_target.midpoint, + intersection_target.closest_vertex_world_position, + this.edit_skeleton_step.get_mesh_drag_snap_strength() + ) + } } else { target_world_position = this.get_point_on_viewport_plane_from_mouse(selected_bone, mouse_event) } @@ -166,7 +337,154 @@ export class MeshDragBonePlacement { } } - private get_edit_mesh_intersection_segment (mouse_event: MouseEvent): [Vector3, Vector3] | null { + private get_centerline_mesh_targets (): Object3D[] { + const mesh_targets: Object3D[] = [] + mesh_targets.push(this.load_model_step.model_meshes()) + + const weight_painted_mesh = this.weight_skin_step.weight_painted_mesh_group() + if (weight_painted_mesh !== null) { + mesh_targets.push(weight_painted_mesh) + } + + return mesh_targets.filter((target) => target.children.length > 0) + } + + private get_mesh_bounds (): THREE.Box3 | null { + const mesh_targets = this.get_centerline_mesh_targets() + if (mesh_targets.length === 0) { + return null + } + + const scene_bounds = new THREE.Box3() + mesh_targets.forEach((target) => { + scene_bounds.expandByObject(target) + }) + + if (scene_bounds.isEmpty()) { + return null + } + + return scene_bounds + } + + private get_centerline_target_at_world_position (target_world_position: Vector3): Vector3 | null { + if (this.edit_skeleton_step.get_skeleton_type() === SkeletonType.Fox) { + const fox_target = this.get_fox_centerline_target_at_world_position(target_world_position) + if (fox_target !== null) { + return fox_target + } + } + + return this.get_mesh_centerline_target_at_world_position(target_world_position) + } + + private get_mesh_centerline_target_at_world_position ( + target_world_position: Vector3, + mesh_targets: Object3D[] = this.get_centerline_mesh_targets() + ): Vector3 | null { + if (mesh_targets.length === 0) { + return null + } + + const scene_bounds = this.get_mesh_bounds() + if (scene_bounds === null) { + return null + } + + const scene_center = scene_bounds.getCenter(new Vector3()) + const scene_size = scene_bounds.getSize(new Vector3()) + const ray_margin = Math.max(0.25, scene_size.length() * 0.25) + const target_y = target_world_position.y + + let snapped_x = scene_center.x + let snapped_z = scene_center.z + + const initial_z_midpoint = this.get_opposing_surface_midpoint( + mesh_targets, + new Vector3(scene_center.x, target_y, scene_bounds.max.z + ray_margin), + new Vector3(0, 0, -1), + new Vector3(scene_center.x, target_y, scene_bounds.min.z - ray_margin), + new Vector3(0, 0, 1) + ) + + if (initial_z_midpoint !== null) { + snapped_z = initial_z_midpoint.z + } + + const x_midpoint = this.get_opposing_surface_midpoint( + mesh_targets, + new Vector3(scene_bounds.min.x - ray_margin, target_y, snapped_z), + new Vector3(1, 0, 0), + new Vector3(scene_bounds.max.x + ray_margin, target_y, snapped_z), + new Vector3(-1, 0, 0) + ) + + if (x_midpoint !== null) { + snapped_x = x_midpoint.x + } + + const refined_z_midpoint = this.get_opposing_surface_midpoint( + mesh_targets, + new Vector3(snapped_x, target_y, scene_bounds.max.z + ray_margin), + new Vector3(0, 0, -1), + new Vector3(snapped_x, target_y, scene_bounds.min.z - ray_margin), + new Vector3(0, 0, 1) + ) + + if (refined_z_midpoint !== null) { + snapped_z = refined_z_midpoint.z + } + + return apply_mesh_centerline_target(target_world_position, snapped_x, snapped_z) + } + + private get_fox_centerline_target_at_world_position (target_world_position: Vector3): Vector3 | null { + const mesh_bounds = this.get_mesh_bounds() + if (mesh_bounds === null) { + return null + } + + const center_x = mesh_bounds.getCenter(new Vector3()).x + return new Vector3(center_x, target_world_position.y, target_world_position.z) + } + + private find_spine_chain_bones (skeleton: Skeleton): Bone[] { + return skeleton.bones.filter((bone) => /spine/.test(bone.name.toLowerCase())) + } + + private find_bone_by_name_match (skeleton: Skeleton, matcher: RegExp): Bone | null { + const bone_match = skeleton.bones.find((bone) => matcher.test(bone.name.toLowerCase())) + return bone_match ?? null + } + + private get_opposing_surface_midpoint ( + mesh_targets: Object3D[], + forward_origin: Vector3, + forward_direction: Vector3, + reverse_origin: Vector3, + reverse_direction: Vector3 + ): Vector3 | null { + const forward_hit = this.get_axis_surface_hit(mesh_targets, forward_origin, forward_direction) + const reverse_hit = this.get_axis_surface_hit(mesh_targets, reverse_origin, reverse_direction) + + if (forward_hit !== null && reverse_hit !== null) { + return forward_hit.add(reverse_hit).multiplyScalar(0.5) + } + + return forward_hit ?? reverse_hit + } + + private get_axis_surface_hit ( + mesh_targets: Object3D[], + origin: Vector3, + direction: Vector3 + ): Vector3 | null { + const axis_raycaster = new THREE.Raycaster(origin, direction.clone().normalize()) + const intersections = axis_raycaster.intersectObjects(mesh_targets, true) + return intersections.length > 0 ? intersections[0].point.clone() : null + } + + private get_edit_mesh_intersection_target (mouse_event: MouseEvent): MeshIntersectionTarget | null { const mesh_targets: Object3D[] = [] const imported_model = this.load_model_step.model_meshes() @@ -191,7 +509,9 @@ export class MeshDragBonePlacement { return null } - const first_intersection = forward_intersections[0].point.clone() + const first_hit = forward_intersections[0] + const first_intersection = first_hit.point.clone() + const closest_vertex_world_position = this.get_closest_vertex_world_position_from_hit(first_hit) const scene_bounds = new THREE.Box3() mesh_targets.forEach((target) => { @@ -211,11 +531,71 @@ export class MeshDragBonePlacement { const reverse_intersections = reverse_raycaster.intersectObjects(mesh_targets, true) if (reverse_intersections.length === 0) { - return [first_intersection, first_intersection] + return { + midpoint: first_intersection, + closest_vertex_world_position + } } const last_intersection = reverse_intersections[0].point.clone() - return [first_intersection, last_intersection] + return { + midpoint: first_intersection.clone().add(last_intersection).multiplyScalar(0.5), + closest_vertex_world_position + } + } + + private get_closest_vertex_world_position_from_hit (intersection: Intersection): Vector3 | null { + const face = intersection.face + const object = intersection.object + const geometry = 'geometry' in object ? object.geometry : undefined + + if (face === null || geometry === undefined) { + return null + } + + const position_attribute = geometry.getAttribute('position') as BufferAttribute | undefined + if (position_attribute === undefined) { + return null + } + + const face_key = `${face.a}-${face.b}-${face.c}` + if (this.closest_vertex_cache !== null && + this.closest_vertex_cache.object_uuid === object.uuid && + this.closest_vertex_cache.face_key === face_key) { + return this.closest_vertex_cache.closest_vertex_world_position?.clone() ?? null + } + + const vertex_indices = [face.a, face.b, face.c] + let closest_vertex_world_position: Vector3 | null = null + let closest_vertex_distance = Number.POSITIVE_INFINITY + + for (const vertex_index of vertex_indices) { + const vertex_world_position = this.get_vertex_world_position(object as Mesh | SkinnedMesh, position_attribute, vertex_index) + const vertex_distance = vertex_world_position.distanceTo(intersection.point) + + if (vertex_distance < closest_vertex_distance) { + closest_vertex_distance = vertex_distance + closest_vertex_world_position = vertex_world_position + } + } + + this.closest_vertex_cache = { + object_uuid: object.uuid, + face_key, + closest_vertex_world_position: closest_vertex_world_position?.clone() ?? null + } + + return closest_vertex_world_position + } + + private get_vertex_world_position (object: Mesh | SkinnedMesh, positions: BufferAttribute, vertex_index: number): Vector3 { + const vertex_world_position = new Vector3().fromBufferAttribute(positions, vertex_index) + + if ('isSkinnedMesh' in object && object.isSkinnedMesh && typeof object.applyBoneTransform === 'function') { + object.applyBoneTransform(vertex_index, vertex_world_position) + } + + return object.localToWorld(vertex_world_position) } private get_point_on_viewport_plane_from_mouse (selected_bone: THREE.Bone, mouse_event: MouseEvent): Vector3 | null { @@ -233,4 +613,18 @@ export class MeshDragBonePlacement { const intersection_point = mouse_raycaster.ray.intersectPlane(viewport_plane, new THREE.Vector3()) return intersection_point === null ? null : intersection_point.clone() } + + private set_orbit_controls_enabled (enabled: boolean): void { + if (this.orbit_controls === undefined) { + return + } + + const orbit_allowed = !this.edit_skeleton_step.is_orbit_rotation_disabled() + this.orbit_controls.enabled = enabled && orbit_allowed + } +} + +interface MeshIntersectionTarget { + midpoint: Vector3 + closest_vertex_world_position: Vector3 | null } diff --git a/src/lib/processes/edit-skeleton/StepEditSkeleton.ts b/src/lib/processes/edit-skeleton/StepEditSkeleton.ts index 019e665..a110f72 100644 --- a/src/lib/processes/edit-skeleton/StepEditSkeleton.ts +++ b/src/lib/processes/edit-skeleton/StepEditSkeleton.ts @@ -38,10 +38,15 @@ export class StepEditSkeleton extends EventTarget { // Skeleton created from the armature that Three.js uses private threejs_skeleton: Skeleton = new Skeleton() + private skeleton_type: SkeletonType | null = null private mirror_mode_enabled: boolean = true private mesh_drag_placement_enabled: boolean = true + private mesh_drag_centerline_snap_enabled: boolean = true + private mesh_drag_snap_strength: number = 10 + private orbit_rotation_disabled: boolean = false private skinning_algorithm: string | null = null private show_debug: boolean = true + private readonly bone_chain_visibility = new Map() private currently_selected_bone: Bone | null = null @@ -118,6 +123,7 @@ export class StepEditSkeleton extends EventTarget { } public begin (main_scene: Scene, skeleton_type: SkeletonType): void { + this.skeleton_type = skeleton_type this.update_ui_options_on_begin(skeleton_type) // show UI elemnents for editing mesh @@ -153,6 +159,21 @@ export class StepEditSkeleton extends EventTarget { this.set_mesh_drag_placement_enabled(this.ui.dom_mesh_drag_placement_checkbox.checked) } + if (this.ui.dom_mesh_drag_centerline_snap_checkbox !== null) { + this.set_mesh_drag_centerline_snap_enabled(this.ui.dom_mesh_drag_centerline_snap_checkbox.checked) + } + + if (this.ui.dom_disable_orbit_rotation_checkbox !== null) { + this.set_orbit_rotation_disabled(this.ui.dom_disable_orbit_rotation_checkbox.checked) + } + + if (this.ui.dom_mesh_drag_snap_strength_input !== null) { + const initial_snap_strength = Number(this.ui.dom_mesh_drag_snap_strength_input.value) + this.set_mesh_drag_snap_strength(Number.isFinite(initial_snap_strength) ? initial_snap_strength : 10) + } + + this.render_bone_chain_visibility_options() + this.update_bind_button_text() // Don't add event listeners again if we are navigating back to this step @@ -236,16 +257,64 @@ export class StepEditSkeleton extends EventTarget { })) } + public set_mesh_drag_centerline_snap_enabled (value: boolean): void { + this.mesh_drag_centerline_snap_enabled = value + } + + public is_mesh_drag_centerline_snap_enabled (): boolean { + return this.mesh_drag_centerline_snap_enabled + } + public is_mesh_drag_placement_enabled (): boolean { return this.mesh_drag_placement_enabled } + public get_skeleton_type (): SkeletonType | null { + return this.skeleton_type + } + + public set_mesh_drag_snap_strength (value: number): void { + const clamped_value = Math.max(0, Math.min(20, Math.round(value))) + this.mesh_drag_snap_strength = clamped_value + + if (this.ui.dom_mesh_drag_snap_strength_input !== null) { + this.ui.dom_mesh_drag_snap_strength_input.value = clamped_value.toString() + } + + if (this.ui.dom_mesh_drag_snap_strength_label !== null) { + this.ui.dom_mesh_drag_snap_strength_label.textContent = clamped_value.toString() + } + } + + public get_mesh_drag_snap_strength (): number { + return this.mesh_drag_snap_strength + } + + public set_orbit_rotation_disabled (value: boolean): void { + this.orbit_rotation_disabled = value + this.dispatchEvent(new CustomEvent('orbit-rotation-changed', { + detail: { disabled: value } + })) + } + + public is_orbit_rotation_disabled (): boolean { + return this.orbit_rotation_disabled + } + + public hidden_bone_chain_root_names (): string[] { + return [...this.bone_chain_visibility.entries()] + .filter(([, is_visible]) => !is_visible) + .map(([bone_name]) => bone_name) + } + private update_manual_transform_options_visibility (): void { - if (this.ui.dom_transform_manual_options === null) { - return + if (this.ui.dom_transform_manual_options !== null) { + this.ui.dom_transform_manual_options.style.display = this.mesh_drag_placement_enabled ? 'none' : 'flex' } - this.ui.dom_transform_manual_options.style.display = this.mesh_drag_placement_enabled ? 'none' : 'flex' + if (this.ui.dom_mesh_drag_snap_strength_container !== null) { + this.ui.dom_mesh_drag_snap_strength_container.style.display = this.mesh_drag_placement_enabled ? 'flex' : 'none' + } } public is_bone_selectable (bone: Bone | null): boolean { @@ -253,6 +322,10 @@ export class StepEditSkeleton extends EventTarget { return false } + if (!this.is_bone_chain_visible(bone)) { + return false + } + if (!this.mirror_mode_enabled) { return true } @@ -361,6 +434,51 @@ export class StepEditSkeleton extends EventTarget { }) } + if (this.ui.dom_mesh_drag_centerline_snap_checkbox !== null) { + this.ui.dom_mesh_drag_centerline_snap_checkbox.addEventListener('change', (event) => { + const target = event.target as HTMLInputElement | null + + if (target === null) { + return + } + + this.set_mesh_drag_centerline_snap_enabled(target.checked) + }) + } + + if (this.ui.dom_disable_orbit_rotation_checkbox !== null) { + this.ui.dom_disable_orbit_rotation_checkbox.addEventListener('change', (event) => { + const target = event.target as HTMLInputElement | null + + if (target === null) { + return + } + + this.set_orbit_rotation_disabled(target.checked) + }) + } + + this.ui.dom_mesh_drag_snap_strength_input?.addEventListener('input', (event) => { + const target = event.target as HTMLInputElement | null + + if (target === null) { + return + } + + const snap_strength = Number(target.value) + this.set_mesh_drag_snap_strength(Number.isFinite(snap_strength) ? snap_strength : 0) + }) + + this.ui.dom_bone_chain_visibility_container?.addEventListener('change', (event) => { + const target = event.target as HTMLInputElement | null + + if (target === null || target.dataset.chainRootName === undefined) { + return + } + + this.set_bone_chain_visibility(target.dataset.chainRootName, target.checked) + }) + this.ui.dom_enable_skin_debugging?.addEventListener('change', (event) => { const target = event.target as HTMLInputElement | null @@ -487,6 +605,7 @@ export class StepEditSkeleton extends EventTarget { this.edited_armature = armature.clone() this.create_threejs_skeleton_object() + this.initialize_bone_chain_visibility_state() this.independent_bone_movement.set_rest_pose(this.threejs_skeleton) // Initialize the undo/redo system with the skeleton @@ -506,13 +625,68 @@ export class StepEditSkeleton extends EventTarget { } public armature (): Object3D { - return this.edited_armature + const skeleton_root = this.threejs_skeleton.bones[0] + const cloned_root = skeleton_root.clone(true) + const armature = new Object3D() + armature.add(cloned_root) + armature.updateMatrixWorld(true) + return armature } public skeleton (): Skeleton { return this.threejs_skeleton } + public is_bone_chain_visible (bone: Bone): boolean { + return this.bone_chain_visibility.get(Utility.chain_root_bone_from_bone(bone).name) !== false + } + + private initialize_bone_chain_visibility_state (): void { + this.bone_chain_visibility.clear() + + Utility.unique_chain_root_bones_from_skeleton(this.threejs_skeleton).forEach((bone) => { + this.bone_chain_visibility.set(bone.name, true) + }) + } + + private render_bone_chain_visibility_options (): void { + if (this.ui.dom_bone_chain_visibility_container === null) { + return + } + + const chain_root_bones = Utility.unique_chain_root_bones_from_skeleton(this.threejs_skeleton) + if (chain_root_bones.length === 0) { + this.ui.dom_bone_chain_visibility_container.style.display = 'none' + this.ui.dom_bone_chain_visibility_container.innerHTML = '' + return + } + + const checkbox_markup = chain_root_bones.map((bone) => { + const checkbox_id = `bone-chain-${bone.name.replace(/[^a-zA-Z0-9_-]/g, '-')}` + const checked_attribute = this.bone_chain_visibility.get(bone.name) === false ? '' : ' checked' + const label = Utility.format_bone_chain_label(bone.name) + return `
` + }).join('') + + this.ui.dom_bone_chain_visibility_container.style.display = 'flex' + this.ui.dom_bone_chain_visibility_container.innerHTML = `
${checkbox_markup}
` + } + + private set_bone_chain_visibility (chain_root_name: string, is_visible: boolean): void { + this.bone_chain_visibility.set(chain_root_name, is_visible) + + const selected_bone = this.get_currently_selected_bone() + if (selected_bone !== null && !this.is_bone_chain_visible(selected_bone)) { + this.set_currently_selected_bone(null) + if (this.ui.dom_selected_bone_label !== null) { + this.ui.dom_selected_bone_label.innerHTML = 'None' + } + this.update_bone_hover_point_position(null) + } + + this.dispatchEvent(new CustomEvent('chainVisibilityChanged')) + } + public apply_mirror_mode (selected_bone: Bone, transform_type: string): void { const mirror_bone = this.find_mirror_bone(selected_bone) diff --git a/src/lib/processes/weight-skin/StepWeightSkin.ts b/src/lib/processes/weight-skin/StepWeightSkin.ts index bd86d17..c2fc668 100644 --- a/src/lib/processes/weight-skin/StepWeightSkin.ts +++ b/src/lib/processes/weight-skin/StepWeightSkin.ts @@ -168,6 +168,10 @@ export class StepWeightSkin extends EventTarget { this.bone_skinning_formula!.set_geometry(geometry_data) const [final_skin_indices, final_skin_weights]: number[][] = this.calculate_weights() + if (this.ui.dom_enable_skin_debugging?.checked === true) { + this.log_stomach_weight_totals(final_skin_indices, final_skin_weights, geometry_data.name) + } + geometry_data.setAttribute('skinIndex', new Uint16BufferAttribute(final_skin_indices, 4)) geometry_data.setAttribute('skinWeight', new Float32BufferAttribute(final_skin_weights, 4)) @@ -188,4 +192,38 @@ export class StepWeightSkin extends EventTarget { console.log('Final skinned meshes:', this.skinned_meshes) console.log('Preview weight painted mesh re-generated:', this.weight_painted_mesh_preview) } + + private log_stomach_weight_totals (skin_indices: number[], skin_weights: number[], mesh_label: string): void { + if (this.binding_skeleton === undefined) { + return + } + + const stomach_bone_entries = this.binding_skeleton.bones + .map((bone, index) => ({ bone, index })) + .filter(({ bone }) => /stomach|abdomen|belly/.test(bone.name.toLowerCase())) + + if (stomach_bone_entries.length === 0) { + return + } + + const weight_totals: number[] = new Array(this.binding_skeleton.bones.length).fill(0) + for (let i = 0; i < skin_indices.length; i++) { + const bone_index = skin_indices[i] + const weight_value = skin_weights[i] ?? 0 + if (bone_index === undefined) { + continue + } + weight_totals[bone_index] = (weight_totals[bone_index] ?? 0) + weight_value + } + + stomach_bone_entries.forEach(({ bone, index }) => { + const total_weight = weight_totals[index] ?? 0 + if (total_weight <= 0.0001) { + console.warn(`Stomach weight check: ${bone.name} has near-zero weight for ${mesh_label}.`) + return + } + + console.log(`Stomach weight check: ${bone.name} total weight for ${mesh_label} = ${total_weight.toFixed(4)}`) + }) + } } diff --git a/src/lib/solvers/WeightSmoother.test.ts b/src/lib/solvers/WeightSmoother.test.ts new file mode 100644 index 0000000..219117c --- /dev/null +++ b/src/lib/solvers/WeightSmoother.test.ts @@ -0,0 +1,368 @@ +import { describe, expect, it } from 'vitest' +import { Bone, BufferGeometry, Float32BufferAttribute, Uint16BufferAttribute } from 'three' + +import { WeightSmoother } from './WeightSmoother.js' + +function count_non_zero_weights (skin_weights: number[], vertex: number): number { + const offset = vertex * 4 + return [ + skin_weights[offset], + skin_weights[offset + 1], + skin_weights[offset + 2], + skin_weights[offset + 3] + ].filter(weight => weight > 1e-6).length +} + +describe('WeightSmoother', () => { + it('accumulates influences across adjacent standard boundaries instead of collapsing back to two weights', () => { + const geometry = new BufferGeometry() + geometry.setAttribute('position', new Float32BufferAttribute([ + 0, 1, 0, + 1, 1, 0, + 2, 1, 0, + 0, 0, 0, + 1, 0, 0, + 2, 0, 0 + ], 3)) + geometry.setIndex(new Uint16BufferAttribute([ + 0, 3, 1, + 1, 3, 4, + 1, 4, 2, + 2, 4, 5 + ], 1)) + + const bones = ['bone-left', 'bone-middle', 'bone-right'].map((name) => { + const bone = new Bone() + bone.name = name + return bone + }) + + const skin_indices = [ + 0, 0, 0, 0, + 1, 0, 0, 0, + 2, 0, 0, 0, + 0, 0, 0, 0, + 1, 0, 0, 0, + 2, 0, 0, 0 + ] + const skin_weights = [ + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0 + ] + + const smoother = new WeightSmoother(geometry, bones) + smoother.smooth_bone_weight_boundaries(skin_indices, skin_weights) + + expect(count_non_zero_weights(skin_weights, 1)).toBeGreaterThanOrEqual(3) + expect(count_non_zero_weights(skin_weights, 4)).toBeGreaterThanOrEqual(3) + }) + + it('applies a joint-to-center gradient along a bone chain', () => { + const geometry = new BufferGeometry() + geometry.setAttribute('position', new Float32BufferAttribute([ + 0, 0, 0, + 1, 0, 0, + 2, 0, 0 + ], 3)) + + const root_bone = new Bone() + root_bone.name = 'root' + + const mid_bone = new Bone() + mid_bone.name = 'spine' + mid_bone.position.set(1, 0, 0) + root_bone.add(mid_bone) + + const child_bone = new Bone() + child_bone.name = 'chest' + child_bone.position.set(1, 0, 0) + mid_bone.add(child_bone) + + root_bone.updateWorldMatrix(true, true) + + const bones = [root_bone, mid_bone, child_bone] + const skin_indices = [ + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0 + ] + const skin_weights = [ + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0 + ] + + const smoother = new WeightSmoother(geometry, bones) + smoother.smooth_bone_weight_boundaries(skin_indices, skin_weights) + + expect(skin_weights[0]).toBeCloseTo(0.2, 5) + expect(skin_weights[1]).toBeCloseTo(0.8, 5) + expect(skin_weights[4]).toBeCloseTo(1.0, 5) + expect(skin_weights[8]).toBeCloseTo(0.2, 5) + expect(skin_weights[9]).toBeCloseTo(0.8, 5) + }) + + it('uses a wider smoothing band between pelvis and thigh bones', () => { + const geometry = new BufferGeometry() + geometry.setAttribute('position', new Float32BufferAttribute([ + 0, 1, 0, + 1, 1, 0, + 2, 1, 0, + 3, 1, 0, + 0, 0, 0, + 1, 0, 0, + 2, 0, 0, + 3, 0, 0 + ], 3)) + geometry.setIndex(new Uint16BufferAttribute([ + 0, 4, 1, + 1, 4, 5, + 1, 5, 2, + 2, 5, 6, + 2, 6, 3, + 3, 6, 7 + ], 1)) + + const pelvis = new Bone() + pelvis.name = 'pelvis' + + const spine = new Bone() + spine.name = 'spine' + pelvis.add(spine) + + const thigh = new Bone() + thigh.name = 'leftThigh' + pelvis.add(thigh) + + const shin = new Bone() + shin.name = 'leftShin' + thigh.add(shin) + + pelvis.updateWorldMatrix(true, true) + + const bones = [pelvis, spine, thigh, shin] + const skin_indices = [ + 0, 0, 0, 0, + 0, 0, 0, 0, + 2, 0, 0, 0, + 2, 0, 0, 0, + 0, 0, 0, 0, + 0, 0, 0, 0, + 2, 0, 0, 0, + 2, 0, 0, 0 + ] + const skin_weights = [ + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0 + ] + + const smoother = new WeightSmoother(geometry, bones) + smoother.smooth_bone_weight_boundaries(skin_indices, skin_weights) + + expect(skin_weights[1]).toBeCloseTo(0.48, 5) + expect(skin_weights[5]).toBeCloseTo(0.5, 5) + expect(skin_weights[13]).toBeCloseTo(0.48, 5) + }) + + it('adds extra pelvis-basin blending for central glute-area vertices', () => { + const geometry = new BufferGeometry() + geometry.setAttribute('position', new Float32BufferAttribute([ + 0, -0.05, 0, + 0.25, -0.15, 0, + -0.25, -0.15, 0, + 0, -0.3, 0, + 0.5, -0.45, 0, + -0.5, -0.45, 0 + ], 3)) + + const pelvis = new Bone() + pelvis.name = 'pelvis' + + const spine = new Bone() + spine.name = 'spine' + spine.position.set(0, 0.4, 0) + pelvis.add(spine) + + const left_thigh = new Bone() + left_thigh.name = 'leftThigh' + left_thigh.position.set(-0.4, -0.6, 0) + pelvis.add(left_thigh) + + const right_thigh = new Bone() + right_thigh.name = 'rightThigh' + right_thigh.position.set(0.4, -0.6, 0) + pelvis.add(right_thigh) + + pelvis.updateWorldMatrix(true, true) + + const bones = [pelvis, spine, left_thigh, right_thigh] + const skin_indices = [ + 0, 0, 0, 0, + 0, 0, 0, 0, + 0, 0, 0, 0, + 0, 0, 0, 0, + 2, 0, 0, 0, + 3, 0, 0, 0 + ] + const skin_weights = [ + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0 + ] + + const smoother = new WeightSmoother(geometry, bones) + smoother.smooth_bone_weight_boundaries(skin_indices, skin_weights) + + expect(skin_weights[4 + 1]).toBeGreaterThan(0.12) + expect([2, 3]).toContain(skin_indices[4 + 1]) + expect(skin_weights[8 + 1]).toBeGreaterThan(0.12) + expect([2, 3]).toContain(skin_indices[8 + 1]) + expect(skin_weights[12 + 1]).toBeGreaterThan(0.12) + expect([2, 3]).toContain(skin_indices[12 + 1]) + }) + + it('uses a wider smoothing band between spine-chain and shoulder-chain bones', () => { + const geometry = new BufferGeometry() + geometry.setAttribute('position', new Float32BufferAttribute([ + 0, 1, 0, + 1, 1, 0, + 2, 1, 0, + 3, 1, 0, + 0, 0, 0, + 1, 0, 0, + 2, 0, 0, + 3, 0, 0 + ], 3)) + geometry.setIndex(new Uint16BufferAttribute([ + 0, 4, 1, + 1, 4, 5, + 1, 5, 2, + 2, 5, 6, + 2, 6, 3, + 3, 6, 7 + ], 1)) + + const chest = new Bone() + chest.name = 'chest' + + const neck = new Bone() + neck.name = 'neck' + chest.add(neck) + + const clavicle = new Bone() + clavicle.name = 'leftClavicle' + chest.add(clavicle) + + const upper_arm = new Bone() + upper_arm.name = 'leftUpperArm' + clavicle.add(upper_arm) + + chest.updateWorldMatrix(true, true) + + const bones = [chest, neck, clavicle, upper_arm] + const skin_indices = [ + 0, 0, 0, 0, + 0, 0, 0, 0, + 2, 0, 0, 0, + 2, 0, 0, 0, + 0, 0, 0, 0, + 0, 0, 0, 0, + 2, 0, 0, 0, + 2, 0, 0, 0 + ] + const skin_weights = [ + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0 + ] + + const smoother = new WeightSmoother(geometry, bones) + smoother.smooth_bone_weight_boundaries(skin_indices, skin_weights) + + expect(skin_weights[1]).toBeCloseTo(0.5, 5) + expect(skin_weights[5]).toBeCloseTo(0.5, 5) + expect(skin_weights[13]).toBeCloseTo(0.5, 5) + }) + + it('uses a symmetric socket smoothing band between clavicle and upper arm bones', () => { + const geometry = new BufferGeometry() + geometry.setAttribute('position', new Float32BufferAttribute([ + 0, 1, 0, + 1, 1, 0, + 2, 1, 0, + 3, 1, 0, + 0, 0, 0, + 1, 0, 0, + 2, 0, 0, + 3, 0, 0 + ], 3)) + geometry.setIndex(new Uint16BufferAttribute([ + 0, 4, 1, + 1, 4, 5, + 1, 5, 2, + 2, 5, 6, + 2, 6, 3, + 3, 6, 7 + ], 1)) + + const clavicle = new Bone() + clavicle.name = 'leftClavicle' + + const upper_arm = new Bone() + upper_arm.name = 'leftUpperArm' + clavicle.add(upper_arm) + + const forearm = new Bone() + forearm.name = 'leftForeArm' + upper_arm.add(forearm) + + clavicle.updateWorldMatrix(true, true) + + const bones = [clavicle, upper_arm, forearm] + const skin_indices = [ + 0, 0, 0, 0, + 0, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 0, 0, 0, 0, + 0, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0 + ] + const skin_weights = [ + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0, + 1, 0, 0, 0 + ] + + const smoother = new WeightSmoother(geometry, bones) + smoother.smooth_bone_weight_boundaries(skin_indices, skin_weights) + + expect(skin_weights[1]).toBeCloseTo(0.5, 5) + expect(skin_weights[5]).toBeCloseTo(0.5, 5) + expect(skin_weights[13]).toBeCloseTo(0.5, 5) + }) +}) \ No newline at end of file diff --git a/src/lib/solvers/WeightSmoother.ts b/src/lib/solvers/WeightSmoother.ts index e635a6a..3db4c5b 100644 --- a/src/lib/solvers/WeightSmoother.ts +++ b/src/lib/solvers/WeightSmoother.ts @@ -1,10 +1,12 @@ import { - type Bone, + Bone, type BufferAttribute, - type BufferGeometry + type BufferGeometry, + Vector3 } from 'three' import { BoneClassifier } from './BoneClassifier.js' +import { Utility } from '../Utilities.js' /** * Smooths skin weight boundaries between bone influences using vertex adjacency. @@ -48,7 +50,104 @@ export class WeightSmoother { this.apply_limb_smoothing(skin_indices, skin_weights, position_to_indices, boundary_pairs) // Pass 4: Apply standard smoothing for remaining boundaries - this.apply_standard_smoothing(skin_indices, skin_weights, position_to_indices, boundary_pairs) + this.apply_standard_smoothing(skin_indices, skin_weights, adjacency, position_to_indices, boundary_pairs) + + // Pass 4.5: Add an extra blend through the central pelvis basin so + // buttocks/glute vertices stay connected to both pelvis and upper thighs. + this.apply_pelvis_basin_smoothing(skin_indices, skin_weights, position_to_indices) + + // Pass 5: Apply an along-the-bone gradient so joints stay soft, + // while the center of each bone remains dominant. + this.apply_axial_gradient_smoothing(skin_indices, skin_weights, position_to_indices) + } + + private apply_axial_gradient_smoothing ( + skin_indices: number[], + skin_weights: number[], + position_to_indices: Map + ): void { + const vertex_count = this.geometry_vertex_count() + const processed_vertices = new Set() + + for (let vertex = 0; vertex < vertex_count; vertex++) { + if (processed_vertices.has(vertex)) { + continue + } + + const offset = vertex * 4 + const primary_bone = skin_indices[offset] + if (primary_bone === undefined || primary_bone === 0 && this.bones[0]?.name === 'root') { + continue + } + + const gradient_data = this.get_axial_gradient_data(primary_bone) + if (gradient_data === null) { + continue + } + + const shared_vertices = this.get_shared_vertices(vertex, position_to_indices) + shared_vertices.forEach((shared_vertex) => processed_vertices.add(shared_vertex)) + + for (const shared_vertex of shared_vertices) { + if (!this.can_apply_axial_gradient_to_vertex(shared_vertex, skin_indices, skin_weights, gradient_data)) { + continue + } + + const target_weights = this.calculate_axial_gradient_weights(shared_vertex, gradient_data) + if (target_weights === null) { + continue + } + + this.set_vertex_weights( + skin_indices, + skin_weights, + shared_vertex, + target_weights.primary_bone, + target_weights.secondary_bone, + target_weights.primary_weight, + target_weights.secondary_weight + ) + } + } + } + + private apply_pelvis_basin_smoothing ( + skin_indices: number[], + skin_weights: number[], + position_to_indices: Map + ): void { + const pelvis_data = this.get_pelvis_basin_data() + if (pelvis_data === null) { + return + } + + const processed_vertices = new Set() + const vertex_count = this.geometry_vertex_count() + + for (let vertex = 0; vertex < vertex_count; vertex++) { + if (processed_vertices.has(vertex)) { + continue + } + + const shared_vertices = this.get_shared_vertices(vertex, position_to_indices) + shared_vertices.forEach((shared_vertex) => processed_vertices.add(shared_vertex)) + + for (const shared_vertex of shared_vertices) { + const basin_blend = this.calculate_pelvis_basin_blend(shared_vertex, skin_indices, pelvis_data) + if (basin_blend === null) { + continue + } + + this.merge_vertex_weights( + skin_indices, + skin_weights, + shared_vertex, + basin_blend.primary_bone, + basin_blend.secondary_bone, + basin_blend.secondary_weight + ) + } + } } /** @@ -80,7 +179,9 @@ export class WeightSmoother { visited.add(key) let smoothing_type: SmoothingType = SmoothingType.Standard - if (this.classifier.is_torso_boundary(bone_a, bone_b)) { + if (this.is_special_socket_boundary(bone_a, bone_b)) { + smoothing_type = SmoothingType.Socket + } else if (this.classifier.is_torso_boundary(bone_a, bone_b)) { smoothing_type = SmoothingType.Torso } else if (this.classifier.is_limb_boundary(bone_a, bone_b)) { smoothing_type = SmoothingType.Limb @@ -105,7 +206,7 @@ export class WeightSmoother { position_to_indices: Map, pairs: BoundaryPair[] ): void { - const torso_pairs = pairs.filter(p => p.smoothing_type === SmoothingType.Torso) + const torso_pairs = pairs.filter(p => p.smoothing_type === SmoothingType.Torso || p.smoothing_type === SmoothingType.Socket) if (torso_pairs.length === 0) return // Collect all boundary vertices and their bone assignments @@ -115,10 +216,6 @@ export class WeightSmoother { boundary_vertices.add(pair.vertex_b) } - // Ring 0 (boundary): 50/50 blend - // Ring 1 (one step out): 75/25 blend with neighbor bone - // Ring 2 (two steps out): 90/10 blend with neighbor bone - const ring_weights = [0.5, 0.25, 0.10] const processed = new Set() // Apply blending ring by ring outward from the boundary @@ -126,53 +223,82 @@ export class WeightSmoother { // First, blend the direct boundary pairs (ring 0) for (const pair of torso_pairs) { + const ring_weights = this.get_torso_ring_weights(pair) this.blend_vertex_pair(skin_indices, skin_weights, position_to_indices, pair.vertex_a, pair.vertex_b, pair.bone_a, pair.bone_b, ring_weights[0]) + + if (pair.smoothing_type === SmoothingType.Socket) { + this.expand_boundary_side_multiple_rings( + skin_indices, + skin_weights, + adjacency, + position_to_indices, + pair.vertex_a, + pair.bone_a, + pair.bone_b, + ring_weights.slice(1) + ) + this.expand_boundary_side_multiple_rings( + skin_indices, + skin_weights, + adjacency, + position_to_indices, + pair.vertex_b, + pair.bone_b, + pair.bone_a, + ring_weights.slice(1) + ) + } + processed.add(pair.vertex_a) processed.add(pair.vertex_b) current_ring_vertices.add(pair.vertex_a) current_ring_vertices.add(pair.vertex_b) } - // Expand outward for rings 1 and 2 - for (let ring = 1; ring < ring_weights.length; ring++) { + const max_ring_count = torso_pairs.reduce((max_count, pair) => { + return Math.max(max_count, this.get_torso_ring_weights(pair).length) + }, 0) + + for (let ring = 1; ring < max_ring_count; ring++) { const next_ring_vertices = new Set() - const secondary_weight = ring_weights[ring] - for (const vertex_idx of current_ring_vertices) { - const offset = vertex_idx * 4 - const primary_bone = skin_indices[offset] + for (const pair of torso_pairs) { + const ring_weights = this.get_torso_ring_weights(pair) + const secondary_weight = ring_weights[ring] + if (secondary_weight === undefined) { + continue + } - for (const neighbor of adjacency[vertex_idx]) { - if (processed.has(neighbor)) continue + const pair_vertices = [pair.vertex_a, pair.vertex_b] + for (const vertex_idx of pair_vertices) { + if (!current_ring_vertices.has(vertex_idx)) { + continue + } - const neighbor_offset = neighbor * 4 - const neighbor_bone = skin_indices[neighbor_offset] + const offset = vertex_idx * 4 + const primary_bone = skin_indices[offset] - // Only expand into vertices that share the same primary bone - if (neighbor_bone !== primary_bone) continue - if (skin_weights[neighbor_offset] !== 1.0) continue + for (const neighbor of adjacency[vertex_idx]) { + if (processed.has(neighbor)) continue - // Find the other bone to blend with from the boundary pair info - const other_bone = this.find_neighbor_bone_from_boundary(vertex_idx, skin_indices, primary_bone) - if (other_bone === -1) continue + const neighbor_offset = neighbor * 4 + const neighbor_bone = skin_indices[neighbor_offset] - // Apply tapering blend to this vertex and its shared-position duplicates - const shared = this.get_shared_vertices(neighbor, position_to_indices) - for (const idx of shared) { - const off = idx * 4 - skin_indices[off + 0] = neighbor_bone - skin_indices[off + 1] = other_bone - skin_weights[off + 0] = 1.0 - secondary_weight - skin_weights[off + 1] = secondary_weight - skin_indices[off + 2] = 0 - skin_indices[off + 3] = 0 - skin_weights[off + 2] = 0 - skin_weights[off + 3] = 0 - } + if (neighbor_bone !== primary_bone) continue + if (skin_weights[neighbor_offset] !== 1.0) continue + + const other_bone = this.find_neighbor_bone_from_boundary(vertex_idx, skin_indices, primary_bone) + if (other_bone === -1) continue - processed.add(neighbor) - next_ring_vertices.add(neighbor) + const shared = this.get_shared_vertices(neighbor, position_to_indices) + for (const idx of shared) { + this.merge_vertex_weights(skin_indices, skin_weights, idx, neighbor_bone, other_bone, secondary_weight) + } + + processed.add(neighbor) + next_ring_vertices.add(neighbor) + } } } @@ -222,6 +348,7 @@ export class WeightSmoother { private apply_standard_smoothing ( skin_indices: number[], skin_weights: number[], + adjacency: Array>, position_to_indices: Map, pairs: BoundaryPair[] ): void { @@ -229,6 +356,12 @@ export class WeightSmoother { for (const pair of standard_pairs) { this.blend_vertex_pair(skin_indices, skin_weights, position_to_indices, pair.vertex_a, pair.vertex_b, pair.bone_a, pair.bone_b, 0.5) + + this.expand_boundary_side(skin_indices, skin_weights, adjacency, position_to_indices, + pair.vertex_a, pair.bone_a, pair.bone_b, 0.2) + + this.expand_boundary_side(skin_indices, skin_weights, adjacency, position_to_indices, + pair.vertex_b, pair.bone_b, pair.bone_a, 0.2) } } @@ -246,32 +379,14 @@ export class WeightSmoother { bone_b: number, secondary_weight: number ): void { - const primary_weight = 1.0 - secondary_weight - const shared_a = this.get_shared_vertices(vertex_a, position_to_indices) for (const idx of shared_a) { - const off = idx * 4 - skin_indices[off + 0] = bone_a - skin_indices[off + 1] = bone_b - skin_weights[off + 0] = primary_weight - skin_weights[off + 1] = secondary_weight - skin_indices[off + 2] = 0 - skin_indices[off + 3] = 0 - skin_weights[off + 2] = 0 - skin_weights[off + 3] = 0 + this.merge_vertex_weights(skin_indices, skin_weights, idx, bone_a, bone_b, secondary_weight) } const shared_b = this.get_shared_vertices(vertex_b, position_to_indices) for (const idx of shared_b) { - const off = idx * 4 - skin_indices[off + 0] = bone_b - skin_indices[off + 1] = bone_a - skin_weights[off + 0] = primary_weight - skin_weights[off + 1] = secondary_weight - skin_indices[off + 2] = 0 - skin_indices[off + 3] = 0 - skin_weights[off + 2] = 0 - skin_weights[off + 3] = 0 + this.merge_vertex_weights(skin_indices, skin_weights, idx, bone_b, bone_a, secondary_weight) } } @@ -288,18 +403,129 @@ export class WeightSmoother { secondary_bone: number, secondary_weight: number ): void { - const primary_weight = 1.0 - secondary_weight const shared = this.get_shared_vertices(vertex, position_to_indices) for (const idx of shared) { - const off = idx * 4 - skin_indices[off + 0] = primary_bone - skin_indices[off + 1] = secondary_bone - skin_weights[off + 0] = primary_weight - skin_weights[off + 1] = secondary_weight - skin_indices[off + 2] = 0 - skin_indices[off + 3] = 0 - skin_weights[off + 2] = 0 - skin_weights[off + 3] = 0 + this.merge_vertex_weights(skin_indices, skin_weights, idx, primary_bone, secondary_bone, secondary_weight) + } + } + + private expand_boundary_side ( + skin_indices: number[], + skin_weights: number[], + adjacency: Array>, + position_to_indices: Map, + boundary_vertex: number, + primary_bone: number, + secondary_bone: number, + secondary_weight: number + ): void { + for (const neighbor of adjacency[boundary_vertex]) { + const offset = neighbor * 4 + if (skin_indices[offset] !== primary_bone) continue + if (skin_weights[offset] !== 1.0) continue + + const shared = this.get_shared_vertices(neighbor, position_to_indices) + for (const idx of shared) { + this.merge_vertex_weights(skin_indices, skin_weights, idx, primary_bone, secondary_bone, secondary_weight) + } + } + } + + private expand_boundary_side_multiple_rings ( + skin_indices: number[], + skin_weights: number[], + adjacency: Array>, + position_to_indices: Map, + boundary_vertex: number, + primary_bone: number, + secondary_bone: number, + secondary_weights: number[] + ): void { + let current_frontier = new Set([boundary_vertex]) + const visited = new Set([boundary_vertex]) + + for (const secondary_weight of secondary_weights) { + const next_frontier = new Set() + + for (const frontier_vertex of current_frontier) { + for (const neighbor of adjacency[frontier_vertex]) { + if (visited.has(neighbor)) { + continue + } + + const offset = neighbor * 4 + if (skin_indices[offset] !== primary_bone) { + continue + } + + if (skin_weights[offset] !== 1.0) { + continue + } + + const shared = this.get_shared_vertices(neighbor, position_to_indices) + for (const idx of shared) { + this.merge_vertex_weights(skin_indices, skin_weights, idx, primary_bone, secondary_bone, secondary_weight) + } + + visited.add(neighbor) + next_frontier.add(neighbor) + } + } + + current_frontier = next_frontier + if (current_frontier.size === 0) { + return + } + } + } + + private merge_vertex_weights ( + skin_indices: number[], + skin_weights: number[], + vertex: number, + primary_bone: number, + secondary_bone: number, + secondary_weight: number + ): void { + const offset = vertex * 4 + const existing_entries: Array<{ bone: number, weight: number }> = [] + + for (let slot = 0; slot < 4; slot++) { + const weight = skin_weights[offset + slot] + if (weight <= 0) continue + existing_entries.push({ bone: skin_indices[offset + slot], weight }) + } + + if (existing_entries.length === 0) { + existing_entries.push({ bone: primary_bone, weight: 1.0 }) + } + + const merged_by_bone = new Map() + for (const entry of existing_entries) { + merged_by_bone.set(entry.bone, (merged_by_bone.get(entry.bone) ?? 0) + entry.weight * (1.0 - secondary_weight)) + } + + merged_by_bone.set(secondary_bone, (merged_by_bone.get(secondary_bone) ?? 0) + secondary_weight) + + if (!merged_by_bone.has(primary_bone)) { + merged_by_bone.set(primary_bone, 0) + } + + const sorted_entries = [...merged_by_bone.entries()] + .map(([bone, weight]) => ({ bone, weight })) + .filter((entry) => entry.weight > 1e-6) + .sort((left, right) => right.weight - left.weight) + .slice(0, 4) + + const weight_sum = sorted_entries.reduce((sum, entry) => sum + entry.weight, 0) + const normalized_entries = weight_sum > 0 + ? sorted_entries.map((entry) => ({ bone: entry.bone, weight: entry.weight / weight_sum })) + : [{ bone: primary_bone, weight: 1.0 }] + + for (let slot = 0; slot < 4; slot++) { + const entry = normalized_entries[slot] + skin_indices[offset + slot] = entry?.bone ?? 0 + skin_weights[offset + slot] = entry?.weight ?? 0 } } @@ -372,6 +598,314 @@ export class WeightSmoother { } return adjacency } + + private get_axial_gradient_data (bone_index: number): AxialGradientData | null { + const bone = this.bones[bone_index] + if (bone === undefined || bone.name === 'root') { + return null + } + + const parent_bone = bone.parent instanceof Bone ? bone.parent : null + const child_bones = bone.children.filter((child): child is Bone => child instanceof Bone) + if (child_bones.length > 1) { + return null + } + + const child_bone = child_bones[0] ?? null + + if (parent_bone === null && child_bone === null) { + return null + } + + const bone_world_position = Utility.world_position_from_object(bone) + const start_point = parent_bone !== null + ? Utility.world_position_from_object(parent_bone) + : bone_world_position.clone() + + const end_point = child_bone !== null + ? Utility.world_position_from_object(child_bone) + : bone_world_position.clone() + + if (start_point.distanceToSquared(end_point) < 1e-8) { + return null + } + + return { + primary_bone: bone_index, + parent_bone: parent_bone !== null ? this.bones.indexOf(parent_bone) : null, + child_bone: child_bone !== null ? this.bones.indexOf(child_bone) : null, + start_point, + end_point + } + } + + private get_torso_ring_weights (pair: BoundaryPair): number[] { + if (this.is_pelvis_thigh_boundary(pair.bone_a, pair.bone_b)) { + return [0.5, 0.48, 0.38, 0.28, 0.18, 0.1] + } + + if (this.is_shoulder_spine_boundary(pair.bone_a, pair.bone_b)) { + return [0.5, 0.5, 0.42, 0.32, 0.22, 0.12] + } + + if (this.is_clavicle_upperarm_boundary(pair.bone_a, pair.bone_b)) { + return [0.5, 0.5, 0.42, 0.32, 0.22] + } + + return [0.5, 0.25, 0.10] + } + + private is_special_socket_boundary (bone_index_a: number, bone_index_b: number): boolean { + return this.is_pelvis_thigh_boundary(bone_index_a, bone_index_b) || + this.is_shoulder_spine_boundary(bone_index_a, bone_index_b) || + this.is_clavicle_upperarm_boundary(bone_index_a, bone_index_b) + } + + private is_pelvis_thigh_boundary (bone_index_a: number, bone_index_b: number): boolean { + const bone_a_name = this.bones[bone_index_a]?.name.toLowerCase() ?? '' + const bone_b_name = this.bones[bone_index_b]?.name.toLowerCase() ?? '' + + const is_pelvis_name = (name: string): boolean => /(pelvis|hips)/.test(name) + const is_thigh_name = (name: string): boolean => /(thigh|upleg|upperleg|leg)/.test(name) + + return (is_pelvis_name(bone_a_name) && is_thigh_name(bone_b_name)) || + (is_pelvis_name(bone_b_name) && is_thigh_name(bone_a_name)) + } + + private is_shoulder_spine_boundary (bone_index_a: number, bone_index_b: number): boolean { + const bone_a_name = this.bones[bone_index_a]?.name.toLowerCase() ?? '' + const bone_b_name = this.bones[bone_index_b]?.name.toLowerCase() ?? '' + + const is_spine_chain_name = (name: string): boolean => /(spine|chest|torso|body|neck)/.test(name) + const is_shoulder_chain_name = (name: string): boolean => /(clavicle|shoulder|upperarm|arm)/.test(name) + + return (is_spine_chain_name(bone_a_name) && is_shoulder_chain_name(bone_b_name)) || + (is_spine_chain_name(bone_b_name) && is_shoulder_chain_name(bone_a_name)) + } + + private is_clavicle_upperarm_boundary (bone_index_a: number, bone_index_b: number): boolean { + const bone_a_name = this.bones[bone_index_a]?.name.toLowerCase() ?? '' + const bone_b_name = this.bones[bone_index_b]?.name.toLowerCase() ?? '' + + const is_clavicle_name = (name: string): boolean => /(clavicle|shoulder)/.test(name) && !/(spine|chest|neck|torso|body)/.test(name) + const is_upperarm_name = (name: string): boolean => /(upperarm|arm)/.test(name) && !/(forearm|lowerarm)/.test(name) + + return (is_clavicle_name(bone_a_name) && is_upperarm_name(bone_b_name)) || + (is_clavicle_name(bone_b_name) && is_upperarm_name(bone_a_name)) + } + + private can_apply_axial_gradient_to_vertex ( + vertex: number, + skin_indices: number[], + skin_weights: number[], + gradient_data: AxialGradientData + ): boolean { + const offset = vertex * 4 + const allowed_bones = new Set([ + gradient_data.primary_bone, + gradient_data.parent_bone ?? -1, + gradient_data.child_bone ?? -1 + ]) + + let non_zero_influence_count = 0 + + for (let slot = 0; slot < 4; slot++) { + const weight = skin_weights[offset + slot] + if (weight <= 1e-6) { + continue + } + + non_zero_influence_count++ + const bone_index = skin_indices[offset + slot] + if (!allowed_bones.has(bone_index) && bone_index !== 0) { + return false + } + } + + return non_zero_influence_count <= 2 + } + + private calculate_axial_gradient_weights ( + vertex: number, + gradient_data: AxialGradientData + ): AxialGradientWeights | null { + const position = new Vector3().fromBufferAttribute(this.geometry.attributes.position, vertex) + const segment = gradient_data.end_point.clone().sub(gradient_data.start_point) + const segment_length_squared = segment.lengthSq() + if (segment_length_squared < 1e-8) { + return null + } + + const segment_t = position.clone().sub(gradient_data.start_point).dot(segment) / segment_length_squared + const clamped_t = Math.max(0, Math.min(1, segment_t)) + const center_falloff = Math.abs((clamped_t * 2) - 1) + const primary_weight = 0.2 + (0.8 * (1 - center_falloff)) + + const secondary_bone = clamped_t <= 0.5 + ? gradient_data.parent_bone ?? gradient_data.child_bone + : gradient_data.child_bone ?? gradient_data.parent_bone + + if (secondary_bone === null) { + return { + primary_bone: gradient_data.primary_bone, + secondary_bone: 0, + primary_weight: 1, + secondary_weight: 0 + } + } + + return { + primary_bone: gradient_data.primary_bone, + secondary_bone, + primary_weight, + secondary_weight: Math.max(0, 1 - primary_weight) + } + } + + private set_vertex_weights ( + skin_indices: number[], + skin_weights: number[], + vertex: number, + primary_bone: number, + secondary_bone: number, + primary_weight: number, + secondary_weight: number + ): void { + const offset = vertex * 4 + skin_indices[offset] = primary_bone + skin_weights[offset] = primary_weight + skin_indices[offset + 1] = secondary_bone + skin_weights[offset + 1] = secondary_weight + skin_indices[offset + 2] = 0 + skin_weights[offset + 2] = 0 + skin_indices[offset + 3] = 0 + skin_weights[offset + 3] = 0 + } + + private get_pelvis_basin_data (): PelvisBasinData | null { + const pelvis_bone_index = this.bones.findIndex((bone) => /(pelvis|hips)/.test(bone.name.toLowerCase())) + if (pelvis_bone_index === -1) { + return null + } + + const pelvis_bone = this.bones[pelvis_bone_index] + const thigh_bones = this.bones + .map((bone, bone_index) => ({ bone, bone_index })) + .filter(({ bone }) => /(thigh|upleg|upperleg|leg)/.test(bone.name.toLowerCase())) + + if (thigh_bones.length === 0) { + return null + } + + const pelvis_position = Utility.world_position_from_object(pelvis_bone) + const thigh_positions = thigh_bones.map(({ bone }) => Utility.world_position_from_object(bone)) + + let average_thigh_position = new Vector3() + thigh_positions.forEach((position) => { + average_thigh_position.add(position) + }) + average_thigh_position.divideScalar(thigh_positions.length) + + const pelvis_to_thigh_distance = Math.max(0.0001, pelvis_position.distanceTo(average_thigh_position)) + + return { + pelvis_bone_index, + pelvis_position, + thigh_bones, + average_thigh_position, + pelvis_to_thigh_distance + } + } + + private calculate_pelvis_basin_blend ( + vertex: number, + skin_indices: number[], + pelvis_data: PelvisBasinData + ): PelvisBasinBlend | null { + const offset = vertex * 4 + const primary_bone = skin_indices[offset] + const thigh_bone_indices = new Set(pelvis_data.thigh_bones.map(({ bone_index }) => bone_index)) + + if (primary_bone !== pelvis_data.pelvis_bone_index && !thigh_bone_indices.has(primary_bone)) { + return null + } + + const vertex_position = new Vector3().fromBufferAttribute(this.geometry.attributes.position, vertex) + const thigh_center_distance = vertex_position.distanceTo(pelvis_data.average_thigh_position) + const pelvis_distance = vertex_position.distanceTo(pelvis_data.pelvis_position) + + if (pelvis_distance > pelvis_data.pelvis_to_thigh_distance * 1.45) { + return null + } + + const vertical_min = Math.min(pelvis_data.pelvis_position.y, pelvis_data.average_thigh_position.y) - (pelvis_data.pelvis_to_thigh_distance * 0.35) + const vertical_max = pelvis_data.pelvis_position.y + (pelvis_data.pelvis_to_thigh_distance * 0.2) + if (vertex_position.y < vertical_min || vertex_position.y > vertical_max) { + return null + } + + const basin_center_distance = vertex_position.distanceTo( + pelvis_data.pelvis_position.clone().lerp(pelvis_data.average_thigh_position, 0.45) + ) + const influence_radius = pelvis_data.pelvis_to_thigh_distance * 1.15 + if (basin_center_distance > influence_radius) { + return null + } + + const normalized_distance = Math.min(1, basin_center_distance / influence_radius) + const secondary_weight = 0.12 + ((1 - normalized_distance) * 0.18) + + if (primary_bone === pelvis_data.pelvis_bone_index) { + const nearest_thigh = pelvis_data.thigh_bones.reduce((closest, candidate) => { + const candidate_distance = vertex_position.distanceTo(Utility.world_position_from_object(candidate.bone)) + if (candidate_distance < closest.distance) { + return { bone_index: candidate.bone_index, distance: candidate_distance } + } + return closest + }, { bone_index: pelvis_data.thigh_bones[0].bone_index, distance: Number.POSITIVE_INFINITY }) + + return { + primary_bone, + secondary_bone: nearest_thigh.bone_index, + secondary_weight + } + } + + return { + primary_bone, + secondary_bone: pelvis_data.pelvis_bone_index, + secondary_weight + } + } +} + +interface PelvisBasinData { + pelvis_bone_index: number + pelvis_position: Vector3 + thigh_bones: Array<{ bone: Bone, bone_index: number }> + average_thigh_position: Vector3 + pelvis_to_thigh_distance: number +} + +interface PelvisBasinBlend { + primary_bone: number + secondary_bone: number + secondary_weight: number +} + +interface AxialGradientData { + primary_bone: number + parent_bone: number | null + child_bone: number | null + start_point: Vector3 + end_point: Vector3 +} + +interface AxialGradientWeights { + primary_bone: number + secondary_bone: number + primary_weight: number + secondary_weight: number } interface BoundaryPair { @@ -383,6 +917,7 @@ interface BoundaryPair { } enum SmoothingType { + Socket = 'socket', Torso = 'torso', Limb = 'limb', Standard = 'standard' diff --git a/src/styles.css b/src/styles.css index b94cabf..34c17b2 100644 --- a/src/styles.css +++ b/src/styles.css @@ -678,8 +678,8 @@ input[type="file"] { The event listener for the click events latches onto this */ #view-control-hitbox { - height: 120px; - width: 120px; + height: 256px; + width: 256px; position: absolute; bottom: 0px; left: 0; diff --git a/tmp/compare-skeletons.mjs b/tmp/compare-skeletons.mjs new file mode 100644 index 0000000..a3a59c5 --- /dev/null +++ b/tmp/compare-skeletons.mjs @@ -0,0 +1,92 @@ +import fs from 'node:fs' +import path from 'node:path' +import { Bone, SkinnedMesh, Texture, TextureLoader, ImageLoader } from 'three' +import { FBXLoader } from 'three/examples/jsm/loaders/FBXLoader.js' +import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js' + +globalThis.window ??= { + URL: { + createObjectURL: () => 'blob:node', + revokeObjectURL: () => {} + } +} + +globalThis.self ??= globalThis + +TextureLoader.prototype.load = function load (_url, onLoad) { + const texture = new Texture() + if (onLoad) onLoad(texture) + return texture +} + +ImageLoader.prototype.load = function load (_url, onLoad) { + const image = { width: 0, height: 0 } + if (onLoad) onLoad(image) + return image +} + +const toArrayBuffer = (buffer) => buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength) +const fbxFile = 'C:/Users/jeffa/Downloads/Meshy_AI_Icebound_Dragon_Knighmix.fbx' +const glbFile = 'C:/Users/jeffa/Downloads/testmixamo.glb' + +const summarize = (root, label) => { + const bones = [] + const skinnedMeshes = [] + const objectLines = [] + + root.traverse((object) => { + if (object.isBone || object instanceof Bone) bones.push(object) + if (object.isSkinnedMesh || object instanceof SkinnedMesh) skinnedMeshes.push(object) + objectLines.push(`${object.type} ${object.name || ''}`) + }) + + const rootBones = bones.filter((bone) => !(bone.parent && (bone.parent.isBone || bone.parent instanceof Bone))) + const byName = new Map() + + for (const bone of bones) { + byName.set(bone.name, (byName.get(bone.name) ?? 0) + 1) + } + + console.log(`=== ${label} ===`) + console.log('total_bones', bones.length) + console.log('root_bones', rootBones.map((bone) => bone.name || '').join(' | ') || 'none') + console.log('skinned_meshes', skinnedMeshes.map((mesh) => mesh.name || '').join(' | ') || 'none') + console.log('object_types') + for (const line of objectLines) console.log(line) + + for (const bone of bones) { + const parentName = bone.parent && (bone.parent.isBone || bone.parent instanceof Bone) + ? bone.parent.name || '' + : '' + const childBones = bone.children + .filter((child) => child.isBone || child instanceof Bone) + .map((child) => child.name || '') + + console.log(`bone ${bone.name || ''} | parent=${parentName} | children=${childBones.join('|') || 'none'}`) + } + + console.log() + + return new Set(bones.map((bone) => bone.name)) +} + +console.log('read fbx') +const fbxBuffer = fs.readFileSync(fbxFile) +console.log('parse fbx') +const fbxRoot = new FBXLoader().parse(toArrayBuffer(fbxBuffer), `${path.dirname(fbxFile)}/`) + +console.log('read glb') +const glbBuffer = fs.readFileSync(glbFile) +console.log('parse glb') +const gltf = await new Promise((resolve, reject) => { + new GLTFLoader().parse(toArrayBuffer(glbBuffer), `${path.dirname(glbFile)}/`, resolve, reject) +}) + +const fbxNames = summarize(fbxRoot, 'fbx') +const glbNames = summarize(gltf.scene, 'glb') +const onlyInFbx = [...fbxNames].filter((name) => !glbNames.has(name)) +const onlyInGlb = [...glbNames].filter((name) => !fbxNames.has(name)) + +console.log('=== name_set_diff ===') +console.log('only_in_fbx', onlyInFbx.join(' | ') || 'none') +console.log('only_in_glb', onlyInGlb.join(' | ') || 'none') diff --git a/tmp/compare-weight-profiles.mjs b/tmp/compare-weight-profiles.mjs new file mode 100644 index 0000000..bfeea80 --- /dev/null +++ b/tmp/compare-weight-profiles.mjs @@ -0,0 +1,121 @@ +import fs from 'node:fs' +import path from 'node:path' +import { JSDOM } from 'jsdom' +import { LoadingManager, Texture, TextureLoader, ImageLoader, SkinnedMesh } from 'three' +import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js' +import { ColladaLoader } from 'three/examples/jsm/loaders/ColladaLoader.js' + +const dom = new JSDOM('', { contentType: 'text/html' }) + +globalThis.window ??= dom.window +globalThis.document ??= dom.window.document +globalThis.DOMParser ??= dom.window.DOMParser +globalThis.XMLSerializer ??= dom.window.XMLSerializer +globalThis.self ??= globalThis + +window.URL ??= { + createObjectURL: () => 'blob:node', + revokeObjectURL: () => {} +} + +TextureLoader.prototype.load = function load (_url, onLoad) { + const texture = new Texture() + if (onLoad) onLoad(texture) + return texture +} + +ImageLoader.prototype.load = function load (_url, onLoad) { + const image = { width: 0, height: 0 } + if (onLoad) onLoad(image) + return image +} + +const toArrayBuffer = (buffer) => buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength) + +const findSkinnedMesh = (root) => { + let found = null + root.traverse((object) => { + if (found === null && (object.isSkinnedMesh || object instanceof SkinnedMesh)) { + found = object + } + }) + return found +} + +const summarizeWeights = (mesh, label) => { + const skinWeight = mesh.geometry.getAttribute('skinWeight') + const vertexCount = skinWeight.count + + let activeSum = 0 + let strongestSum = 0 + let secondSum = 0 + const activeHistogram = new Map() + const strongestBuckets = { + rigid_95_plus: 0, + strong_80_95: 0, + mixed_60_80: 0, + blended_under_60: 0 + } + + for (let index = 0; index < vertexCount; index++) { + const weights = [ + skinWeight.getX(index), + skinWeight.getY(index), + skinWeight.getZ(index), + skinWeight.getW(index) + ].filter((value) => value > 1e-4).sort((a, b) => b - a) + + const activeCount = weights.length + const strongest = weights[0] ?? 0 + const second = weights[1] ?? 0 + + activeSum += activeCount + strongestSum += strongest + secondSum += second + activeHistogram.set(activeCount, (activeHistogram.get(activeCount) ?? 0) + 1) + + if (strongest >= 0.95) strongestBuckets.rigid_95_plus++ + else if (strongest >= 0.80) strongestBuckets.strong_80_95++ + else if (strongest >= 0.60) strongestBuckets.mixed_60_80++ + else strongestBuckets.blended_under_60++ + } + + console.log(`=== ${label} ===`) + console.log('mesh_name', mesh.name || '') + console.log('vertex_count', vertexCount) + console.log('avg_active_influences', (activeSum / vertexCount).toFixed(4)) + console.log('avg_strongest_weight', (strongestSum / vertexCount).toFixed(4)) + console.log('avg_second_weight', (secondSum / vertexCount).toFixed(4)) + console.log('active_histogram', JSON.stringify(Object.fromEntries([...activeHistogram.entries()].sort((a, b) => a[0] - b[0])))) + console.log('strongest_buckets', JSON.stringify(strongestBuckets)) + console.log() +} + +const loadGlb = async (file) => { + const buffer = fs.readFileSync(file) + const loader = new GLTFLoader(new LoadingManager()) + return await new Promise((resolve, reject) => { + loader.parse(toArrayBuffer(buffer), `${path.dirname(file)}/`, resolve, reject) + }) +} + +const loadDae = (file) => { + const text = fs.readFileSync(file, 'utf8') + const loader = new ColladaLoader(new LoadingManager()) + return loader.parse(text, path.dirname(file)) +} + +const glbFile = 'C:/Users/jeffa/Downloads/testmixamo.glb' +const daeFile = 'C:/Users/jeffa/Downloads/testmixamoconverted.dae' + +const gltf = await loadGlb(glbFile) +const dae = loadDae(daeFile) + +const glbMesh = findSkinnedMesh(gltf.scene) +const daeMesh = findSkinnedMesh(dae.scene) + +if (glbMesh === null) throw new Error('No skinned mesh found in GLB') +if (daeMesh === null) throw new Error('No skinned mesh found in DAE') + +summarizeWeights(glbMesh, 'glb') +summarizeWeights(daeMesh, 'dae') \ No newline at end of file diff --git a/tmp/inspect-moo-skeleton.mjs b/tmp/inspect-moo-skeleton.mjs new file mode 100644 index 0000000..7dc3fda --- /dev/null +++ b/tmp/inspect-moo-skeleton.mjs @@ -0,0 +1,60 @@ +import fs from 'node:fs' +import path from 'node:path' +import { Bone, ImageLoader, SkinnedMesh, Texture, TextureLoader } from 'three' +import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js' + +globalThis.window ??= { + URL: { + createObjectURL: () => 'blob:node', + revokeObjectURL: () => {} + } +} + +globalThis.self ??= globalThis + +TextureLoader.prototype.load = function load (_url, onLoad) { + const texture = new Texture() + if (onLoad) onLoad(texture) + return texture +} + +ImageLoader.prototype.load = function load (_url, onLoad) { + const image = { width: 0, height: 0 } + if (onLoad) onLoad(image) + return image +} + +const file = 'C:/Users/jeffa/Downloads/moosetest.glb' +const to_array_buffer = (buffer) => buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength) + +const loader = new GLTFLoader() +const buffer = fs.readFileSync(file) +const gltf = await new Promise((resolve, reject) => { + loader.parse(to_array_buffer(buffer), `${path.dirname(file)}/`, resolve, reject) +}) + +const bones = [] +const skinned_meshes = [] + +gltf.scene.traverse((object) => { + if (object.isBone || object instanceof Bone) bones.push(object) + if (object.isSkinnedMesh || object instanceof SkinnedMesh) skinned_meshes.push(object) +}) + +const root_bones = bones.filter((bone) => !(bone.parent && (bone.parent.isBone || bone.parent instanceof Bone))) + +console.log('total_bones', bones.length) +console.log('root_bones', root_bones.map((bone) => bone.name || '').join(' | ') || 'none') +console.log('skinned_meshes', skinned_meshes.map((mesh) => mesh.name || '').join(' | ') || 'none') + +for (const bone of bones) { + const parent = bone.parent && (bone.parent.isBone || bone.parent instanceof Bone) + ? bone.parent.name || '' + : '' + const children = bone.children + .filter((child) => child.isBone || child instanceof Bone) + .map((child) => child.name || '') + .join('|') + + console.log(`bone ${bone.name || ''} | parent=${parent} | children=${children || 'none'}`) +} \ No newline at end of file diff --git a/workspace-change-summary-2026-05-03.md b/workspace-change-summary-2026-05-03.md new file mode 100644 index 0000000..06207a0 --- /dev/null +++ b/workspace-change-summary-2026-05-03.md @@ -0,0 +1,124 @@ +# Workspace Change Summary - 2026-05-03 + +This file summarizes the changes made in this workspace since the previous commit. + +## Main outcomes + +- Improved skeleton editing for centerline bones so pelvis, spine, chest, neck, and head snap toward the mesh center instead of drifting toward the front surface. +- Added vertex snap strength controls for mesh-drag joint placement. +- Added condensed bone-chain visibility controls so main chains can be shown or hidden while keeping finger bones grouped under the hand chain. +- Refined skeleton helper rendering so finger joints use smaller circles and hidden chains are respected by both points and lines. +- Expanded skin-weight smoothing substantially for human shoulder, clavicle, torso, pelvis, thigh, buttocks, and glute-area transitions. +- Prevented branching parent bones such as the chest from auto-rotating during independent child movement, which helps preserve the default human rig shoulder orientation. +- Added focused regression tests for utility chain grouping, mesh-drag snapping, independent bone movement, and weight smoothing. +- Added temporary investigation scripts under `tmp/` for comparing rigs and weight profiles outside the runtime app. + +## UI and interaction changes + +### Position Joints workflow + +- Added `Visible Chains` UI in `src/create.html` and wired it through `src/lib/UI.ts` and `src/lib/processes/edit-skeleton/StepEditSkeleton.ts`. +- Added mesh-drag vertex snap slider controls in `src/create.html`, `src/lib/UI.ts`, and `src/lib/processes/edit-skeleton/StepEditSkeleton.ts`. +- Added chain visibility event handling in `src/lib/EventListeners.ts` and helper sync in `src/Mesh2MotionEngine.ts`. + +### Skeleton helper rendering + +- Updated `src/lib/CustomSkeletonHelper.ts` to: + - support hidden chain roots + - hide helper lines for hidden chains + - render finger joints with a dedicated smaller points layer + - keep small finger circles independent from mesh weighting logic + +## Bone placement and orientation changes + +### Mesh drag placement + +- Updated `src/lib/processes/edit-skeleton/MeshDragBonePlacement.ts` to: + - support configurable vertex snap influence + - find nearest hit-face vertex for snapping + - compute mesh centerline targets for centerline bones + - snap primary centerline bones to the mesh center when the skeleton loads + +### Independent bone movement + +- Updated `src/lib/processes/edit-skeleton/IndependentBoneMovement.ts` so branching parents like chest or pelvis preserve their template orientation when a child bone is translated. +- This prevents chest and shoulder hubs from twisting backward due to averaged child-direction reorientation. + +## Weight smoothing changes + +### Broader smoothing model + +- Updated `src/lib/solvers/WeightSmoother.ts` to: + - preserve and merge multiple influences instead of collapsing back to simple two-weight assignments + - expand standard boundary smoothing with extra neighbor rings + - add an axial joint-to-center gradient for simple bone chains + - add symmetric socket smoothing for pelvis-thigh, spine/neck-to-shoulder, and clavicle-to-upperarm boundaries + - widen pelvis-thigh socket smoothing further + - add dedicated pelvis-basin smoothing for glute-area vertices + - bias the front shoulder seam to keep torso-to-upperarm regions better connected + +### Areas specifically improved + +- Pelvis to thigh transition +- Buttocks and central pelvis basin +- Spine, chest, neck to shoulder transition +- Clavicle to upper arm transition +- Front torso to upper arm seam + +## Utility and grouping changes + +- Updated `src/lib/Utilities.ts` to support: + - chain root detection + - grouped hand and foot chains + - condensed chain labels + - mapping descendant bones back to grouped main chains + +## Test coverage added + +- `src/lib/Utilities.test.ts` +- `src/lib/processes/edit-skeleton/IndependentBoneMovement.test.ts` +- `src/lib/processes/edit-skeleton/MeshDragBonePlacement.test.ts` +- `src/lib/solvers/WeightSmoother.test.ts` + +These tests cover: + +- condensed chain grouping +- centerline and vertex snap logic +- shoulder/chest orientation preservation +- boundary accumulation and advanced smoothing regressions + +## Temporary investigation scripts + +Added under `tmp/`: + +- `compare-skeletons.mjs` +- `compare-weight-profiles.mjs` +- `inspect-moo-skeleton.mjs` + +These were used for one-off inspection of imported rigs and weight distributions. + +## Files touched + +Modified: + +- `src/Mesh2MotionEngine.ts` +- `src/create.html` +- `src/lib/CustomSkeletonHelper.ts` +- `src/lib/EventListeners.ts` +- `src/lib/UI.ts` +- `src/lib/Utilities.ts` +- `src/lib/processes/edit-skeleton/IndependentBoneMovement.ts` +- `src/lib/processes/edit-skeleton/MeshDragBonePlacement.ts` +- `src/lib/processes/edit-skeleton/StepEditSkeleton.ts` +- `src/lib/solvers/WeightSmoother.ts` + +Added: + +- `src/lib/Utilities.test.ts` +- `src/lib/processes/edit-skeleton/IndependentBoneMovement.test.ts` +- `src/lib/processes/edit-skeleton/MeshDragBonePlacement.test.ts` +- `src/lib/solvers/WeightSmoother.test.ts` +- `tmp/compare-skeletons.mjs` +- `tmp/compare-weight-profiles.mjs` +- `tmp/inspect-moo-skeleton.mjs` +- `workspace-change-summary-2026-05-03.md` \ No newline at end of file