Repository navigation
Export: the audio analysis as JSON, CSV and MIDI - #857
Merged
Merged
Conversation
A new export option, off by default, writes the audio analysis next to the video. "Audio Analysis Only" writes it from an audio file without rendering video. The files: - JSON and CSV: one row per video frame. Each row has every modulation source measured from the audio, the beat grid, the chord of the current beat and the key. The JSON also has a summary, events, and chord and key sections. - MIDI: beats, kick, snare and hi-hat on channel 10, and the chords as notes, at the detected tempo. The recorder runs on its own audio engine, so the video does not change: the frame digests match with the option on and off. Tempo, beat grid, chords and key come from the whole file: - tempo from the autocorrelation of the loudness rise, weighted around 120 BPM; - a steady grid with its phase fitted to the onsets; - chords from the average pitch classes of each beat, at least two beats long; - the key from the pitch classes of the whole file. Event times are corrected for the analysis window. Also over MCP: `analysis` on sv_start_export, and a new tool, sv_export_analysis. Co-authored-by: caganturgut <271973834+caganturgut@users.noreply.github.com>
Co-authored-by: caganturgut <271973834+caganturgut@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Writes what the app measures in a song to files, for video editors, DAWs and spreadsheets. It ships with 3.1.6, as the owner asked.
What it does
analysis.*inside their folder. It is not available with a seamless loop, because that output starts mid-render.<name> - analysis.jsonand.csv: one row per video frame (row N = frame N).beat,downbeat,pitchHz,chordandkey.beatGridFit), events (beats, onsets, kick, snare and hi-hat with strength), chord and key sections, and a description of every field.<name> - analysis.mid: an SMF type 1 file with four tracks, at the detected tempo.analysis: trueonsv_start_export, and a new tool,sv_export_analysis(97 tools now).Why the summary uses the whole file
On a 20 s excerpt, the live tempo engine wandered from 172 to 80 BPM and the live chord changed 136 times. The files use the whole song instead:
beatGridFitreports how well the grid fits.The video does not change
The recorder has its own
SVAudioandModulator, fed the same frame bytes.getBarsadvances state on every call, so sharing the render engine would have changed the video. Measured with--bench-export:Tests:
npm test3573/3573.tests/analysis-export.test.jschecks known answers on synthetic audio run through the exporter's own FFT and byte scale:Checked in the running app, isolated profile, Turkish and English:
Also in this PR