Hey I am on Medial • 2d
I read manga sometimes. It's a really good idea. I am here to just give out my type of version of how it could be done. You give the comics pdf or panels to the application as input, it will analyze the scene, determine the soundtracs and sound tagging is done for panel to panel. Lastly compile it together and it's ready for immersive reading. This may sound simple but implementation is somewhat hard as it will be all dynamic and generalized model. You can start with one comic first though. previously some projects has tried similar thing or you can take pointers from these projects: SoundtrackMyComic, Madefire, Unity/Unreal Visual Novels. Hope you find this reply helpful in completing your project. Good Luck
Download the medial app to read full posts, comements and news.