Skip to content

⚡ Bolt: optimize estimator prediction and face detection#40

Open
guesswh0 wants to merge 1 commit into
masterfrom
bolt-optimize-inference-793044226827246022
Open

⚡ Bolt: optimize estimator prediction and face detection#40
guesswh0 wants to merge 1 commit into
masterfrom
bolt-optimize-inference-793044226827246022

Conversation

@guesswh0
Copy link
Copy Markdown
Owner

💡 What:

  • Vectorized BasicEstimator.predict using the squared distance expansion formula.
  • Pre-calculated fitted embedding norms in BasicEstimator.fit and ensured backward compatibility in load.
  • Vectorized bounding box area calculation in FaceEngine.find_faces.
  • Fixed a bug in FaceEngine.find_faces where extra metadata was incorrectly indexed when limit was applied (it was taking a single element instead of slicing/reordering).

🎯 Why:

  • The original predict implementation used an explicit Python loop over input embeddings, which was a significant bottleneck for batch processing.
  • find_faces used a list comprehension for area calculation which can be vectorized for better performance.
  • The limit logic in find_faces was broken, causing metadata misalignment and potential crashes when extra fields were accessed.

📊 Impact:

  • ~6.4x speedup for BasicEstimator.predict (measured from ~0.22s to ~0.035s for 500 query embeddings against 2000 fitted embeddings).
  • More efficient face detection limiting by avoiding list comprehensions.
  • Fixed a critical bug where metadata (like confidence scores) would be misaligned or lost when limiting the number of detected faces.

🔬 Measurement:

  • Verified numerical correctness against the original implementation.
  • Verified that empty input batches are handled correctly.
  • Verified that persistence and backward compatibility work as expected.
  • Verified that metadata alignment in find_faces is now correct via mocked detector tests.

PR created automatically by Jules for task 793044226827246022 started by @guesswh0

- Vectorize BasicEstimator.predict using squared distance expansion formula
- Pre-calculate fitted embedding norms for faster inference
- Vectorize bounding box area calculation in FaceEngine.find_faces
- Fix metadata indexing bug in FaceEngine.find_faces when limit is applied
- Add backward compatibility for loaded models missing pre-calculated norms
- Ensure numerical stability with np.maximum(dists_sq, 0)

Co-authored-by: guesswh0 <10531675+guesswh0@users.noreply.github.com>
@google-labs-jules
Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant