Download the full pack: Note that the full application is about 5GB. That is mainly because the large-size parameters of the pre-trained deep learning models. Download the whole pack contains the pre-trained models with optimized parameters and corresponding compilable codes.
Record a piece of video contains human motions. Satisfy all the prerequisite libraries stated below.
After downloading, firstly activate tensorflow environment in the terminal of anaconda.
Run OpenposeVideo.bat and follow the pop-out instructions.
Then proceed to the 3d-pose-baseline-vmd folder and run OpenposeTo3D.bat. Follow the pop-out instructions.
After that, proceed to the FCRN-DepthPrediction-vmd folder and run VideoToDepth.dat.
Finally, proceed to the VMD-3d-pose-baseline-multi folder and run 3DToVMD.bat. You will get the vmd file.
VMD files are 3D animation file used by MikuMikuDance, a program used to create dance animation movies. Open your MikuMikuDance and input the VMD file.
You will see your Miku begin acting the same motions as that in your recorded video.
Tutorial in Chinese (中文教程): Developers that can understand Chinese are encouraged to read the tutorial written by @mrzjy on Bilibili Articles: Click Here. This tutorial covers how to install and run the OpenMMD. The tutorial also introduces some common issues of OpenMMD.
Library Dependencies
pip install python-dateutil
pip install pytz
pip install pyparsing
pip install six
pip install imageio
Special Thanks I would like to give special thanks for the contribution from @zhangxinyi0106. As an expert in Photoshop and Video Processing, He offered great help in recording and processing the sample images and videos. He also offered nice ideas on how to improve video smoothing performance.
I would like to say special thanks to @miu200521358 who provides a series of detailed tutorials on OpenPose and relevant models. That really promotes my implementation progress. His versions of model implementations are also of great help. I learn a lot from his instructions on anaconda and tensorflow.
I would like to give special thanks to @mrzjy from Bilibili who writes a very detailed Chinese tutorial on how to install and run OpenMMD. This tutorial covers detailed steps and possible bugs when using the project. The quality of the tutorial is very high. 哔哩哔哩(゜-゜)つロ干杯!
Feedback This project is an open source project. Let me know if:
you find videos or images conversion does not work well.
you know how to speed up or improve the performance of 3D converting.
you have suggestions about future possible functionalities.
and anything you want to ask...
Just comment on GitHub or make a pull request and I will answer as soon as possible!
If you appreciate the project, please kindly star it. :D Feel free to download and develop your own 3D animations.
Thank you for your time!