Fix issue where vertex colors are sometimes ignored

[bnolan]

I think it's a race condition between the material component and the voxel loader.