Skip to content

Commit ef83177

Browse files
Danielku15Danielku15
committed
docs: Extend guide on audio sync
1 parent 33b6ce4 commit ef83177

3 files changed

Lines changed: 381 additions & 3 deletions

File tree

‎docs/guides/audio-video-sync.mdx‎

Lines changed: 178 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,6 @@
22
title: Audio & Video Sync
33
---
44

5-
> [!WARNING]
6-
> This guide is still incomplete and is currently being extended.
7-
85
alphaTab can be synchronized with an external audio or video backing track. You can either use Guitar Pro 8 files with an audio track or synchronize alphaTab with an external media using the
96
[Media Sync Editor in the Playground](/docs/playground/playground.mdx).
107

@@ -34,6 +31,15 @@ Beside that you can load Guitar Pro 8 files and directly benefit from the enhanc
3431
### Custom External Media Player
3532

3633
alphaTab can be integrated with any external media system but it requires some implementation on the integrator side. To properly synchronize alphaTab and an external media source (audio or video) the `alphaTab.synth.IExternalMediaHandler` interface has to be implemented and provided to alphaTab.
34+
Beside that, alphaTab has to be informed about the time updates on the external media (e.g. during playback and seeking). This update should happen at a rate smaller than two subsequent notes. 50ms updates have shown to work well on even fast songs, but to save power and battery you might want to handle this dynamically.
35+
36+
The following example illustrates a full integration with a HTML Media Element like `<audio />` or `<video />`. This should be a good reference on how to implement the related sync with your media player.
37+
Most things should be quite obvious like syncing volume, playback speed and providing the time and duration.
38+
39+
import { ExternalMediaSample } from '@site/src/components/ExternalMediaSample';
40+
41+
<ExternalMediaSample />
42+
3743

3844
#### YouTube
3945

@@ -46,3 +52,172 @@ Some key reasons behind this is:
4652

4753
Nevertheless we want to give you some guidance on how to link alphaTab to YouTube. The following steps show how to use the [YouTube Player API Reference for iframe Embeds](https://developers.google.com/youtube/iframe_api_reference) together with alphaTab.
4854

55+
56+
```js
57+
// assuming a <div id="youtube"></div> somewhere
58+
59+
60+
//
61+
// 1. load the YouTube IFrame API. we use promises to have a bit better control over the initialization sequence
62+
63+
const playerElement = document.getElementById('youtube');
64+
65+
const tag = document.createElement('script');
66+
tag.src = "https://www.youtube.com/player_api";
67+
playerElement.parentNode.insertBefore(tag, playerElement);
68+
69+
const youtubeApiReady = Promise.withResolvers();
70+
window.onYouTubePlayerAPIReady = youtubeApiReady.resolve;
71+
72+
//
73+
// 2. Already initialize alphaTab
74+
75+
const api = new alphaTab.AlphaTabApi('#alphaTab', {
76+
core: {
77+
file: 'my-synced-file.gp'
78+
},
79+
player: {
80+
// set the external media mode
81+
playerMode: alphaTab.PlayerMode.EnabledExternalMedia
82+
}
83+
});
84+
window.at = at;
85+
86+
//
87+
// 3. Wait for the youtube API to be ready
88+
await youtubeApiReady.promise;
89+
90+
91+
//
92+
// 4. Setup the youtube player and wait for the video to be ready for playback
93+
const youtubePlayerReady = Promise.withResolvers();
94+
let currentTimeInterval = 0;
95+
const player = new YT.Player(playerElement, {
96+
height: '360',
97+
width: '640',
98+
videoId: 'your-video-id',
99+
playerVars: { 'autoplay': 0 }, // we do not want autoplay
100+
events: {
101+
'onReady': (e) => {
102+
youtubePlayerReady.resolve();
103+
},
104+
105+
// when the player state changes we update alphatab accordingly.
106+
'onStateChange': (e) => {
107+
//
108+
switch (e.data) {
109+
case YT.PlayerState.PLAYING:
110+
currentTimeInterval = window.setInterval(() => {
111+
api.player.output.updatePosition(player.getCurrentTime() * 1000)
112+
}, 50);
113+
api.play();
114+
break;
115+
case YT.PlayerState.ENDED:
116+
window.clearInterval(currentTimeInterval);
117+
api.stop();
118+
break;
119+
case YT.PlayerState.PAUSED:
120+
window.clearInterval(currentTimeInterval);
121+
api.pause();
122+
break;
123+
default:
124+
break;
125+
}
126+
},
127+
'onPlaybackRateChange': (e) => {
128+
api.playbackSpeed = e.data;
129+
},
130+
'onError': (e) => {
131+
youtubePlayerReady.reject(e);
132+
},
133+
}
134+
});
135+
136+
await youtubePlayerReady.promise;
137+
138+
//
139+
// 5. Setup the handler to let alphaTab control the youtube player when needed
140+
141+
// Setup alphaTab with youtube handler
142+
const alphaTabYoutubeHandler = {
143+
get backingTrackDuration() {
144+
return player.getDuration() * 1000;
145+
},
146+
get playbackRate() {
147+
return player.getPlaybackRate();
148+
},
149+
set playbackRate(value) {
150+
player.setPlaybackRate(value);
151+
},
152+
get masterVolume() {
153+
return player.getVolume() / 100;
154+
},
155+
set masterVolume(value) {
156+
player.setVolume(value * 100);
157+
},
158+
seekTo(time) {
159+
player.seekTo(time / 1000);
160+
},
161+
play() {
162+
player.playVideo();
163+
},
164+
pause() {
165+
player.pauseVideo();
166+
}
167+
};
168+
api.player.output.handler = alphaTabYoutubeHandler;
169+
```
170+
171+
As you can see it is again just a matter of passing the right calls and values back and forth just like with an `<audio />` element.
172+
173+
> [!WARNING]
174+
> The YouTube IFrame API has a quite nasty behavior when it comes to seeking: For some strange reason `seekTo` will start the playback of the video
175+
> depending on the state. The docs say:
176+
>
177+
> > `player.seekTo(seconds:Number, allowSeekAhead:Boolean):Void`
178+
> >
179+
> > Seeks to a specified time in the video. If the player is paused when the function is called, it will remain paused. If the function is called from another state (playing, video cued, etc.), the player will play the video.
180+
>
181+
> alphaTab might initially do some seeking to the start position. This can cause the video to directly start playing which you might not want.
182+
> To workaround this problem you could adjust the handler to check the player state and react accordingly.
183+
184+
```js
185+
let initialSeek = -1;
186+
const alphaTabYoutubeHandler = {
187+
get backingTrackDuration() {
188+
return player.getDuration() * 1000;
189+
},
190+
get playbackRate() {
191+
return player.getPlaybackRate();
192+
},
193+
set playbackRate(value) {
194+
player.setPlaybackRate(value);
195+
},
196+
get masterVolume() {
197+
return player.getVolume() / 100;
198+
},
199+
set masterVolume(value) {
200+
player.setVolume(value * 100);
201+
},
202+
seekTo(time) {
203+
if (
204+
player.getPlayerState() !== YT.PlayerState.PAUSED &&
205+
player.getPlayerState() !== YT.PlayerState.PLAYING
206+
) {
207+
initialSeek = value / 1000;
208+
} else {
209+
player.seekTo(value / 1000);
210+
}
211+
},
212+
play() {
213+
player.playVideo();
214+
if (initialSeek >= 0) {
215+
player.seekTo(initialSeek);
216+
initialSeek = -1;
217+
}
218+
},
219+
pause() {
220+
player.pauseVideo();
221+
}
222+
};
223+
```

0 commit comments

Comments
 (0)