DeepLips is an AI lip sync tool that works with almost any character, even ones that were never built for facial animation.
For more details about this asset from the Unity Asset Store, please visit: Click Here
This is a paid asset, but now you can download the “DeepLips: Lip Sync AI” for FREE. Please note that this package is provided solely for learning purposes or to test the product before purchase, and not for commercial use.
DeepLips: Lip Sync AI v1.0
“If possible, please buy the package to support the developer”
Does your character not have jaw bones, blendshapes, or visemes?
DeepLips can still make it talk.
DeepLips is an AI lip sync tool that works with almost any character, even ones that were never built for facial animation.
Just load your character + audio clip and click Generate. DeepLips creates the lip sync automatically.
No facial rigs.
No phoneme setup.
No hand animating mouths.
DeepLips is a texture based lip sync AI that works by playing a lip sync “video” on the face.
⏱ Skip Hours of Setup
Traditional lip sync workflows require:
Creating phoneme blendshapes
Mapping visemes
Animating mouth poses
Adjusting timing
… DeepLips skips all of that.
You simply:
Load your character
Select the face area
Add an audio clip
Click Generate
That’s it.
🎭 Works With Almost Any Character
You can use it with:
Stylized characters
Realistic characters
NPC crowds
Characters without facial rigs
Characters from the Mixamo, Fuse, CC3/4
Custom avatars
Basically, if the character has a texture, DeepLips can drive the speech animation.
FEATURES
🖼 Frame Speech Animation: DeepLips creates a frame sequence that can be played back in sync with audio. This gives you a simple, visual result that is easy to preview, easy to tweak, and easy to integrate into your scene.
🎨 LipMask Shader Support: Includes support for a LipMask shader workflow with controls for X Offset, Y Offset, X Size, and Y Size. This makes it easy to position and fit the mouth area onto your character.
🦴 No Facial Rig Required: Most lip sync systems require jaw bones, viseme rigs, or blendshapes. DeepLips does not. If your character has a face texture and a mesh, you can make it speak.
🧠 AI Runs Locally: Lip sync frames are generated locally inside Unity. No cloud processing required for the core workflow.
⚡ Very Easy Runtime Setup: After generating frames, just attach the LipSyncFrameSequence component, audio and frame data generated by DeepLips.
🎯 More Natural Than Basic Viseme Systems: Many viseme systems rely on simple mouth shape switching, which can look robotic, inaccurate or slightly off. DeepLips generates mouth animation directly from the audio and plays a lip sync video on the face.
API:
Is as easy as:
—————————————–
*.Play(Audio Clip, “Frames Folder”);
—————————————–
* Class Name: LipSyncFrameSequence
Dependencies:
Requires Inference Engine 2.5 from Unity 6+
Unity 6.2 – Standard Pipeline Supported
Unity 6.4 – URP Supported
Limitations
– DeepLips uses image based animation rather than mesh deformation. Because the mesh itself does not move, the face can sometimes feel slightly stiffer than full facial rigs.
– The illusion may break under harsh lighting as the mesh does not deform.
– With long sequences, storing frames can take up storage space. [New*] added JPEG encoding for dramatically smaller sizes.
– Real-time generation and playback: With fast GPUs, generating the entire frame sequence can take a few seconds. With CPU, it may take a minute or two to generate the frames for lip sync. The playback is smooth and in real-time. Pre-generate and load the frames in cache for an instant playback.
– May break or generate incorrectly if the face is too far from a human face. The AI model works with cartoon faces, but if it deviates too much, the generation may be incorrect. If you are not sure if your characters are compatible with DeepLips, please shoot us an email at [email protected]
These tradeoff allows DeepLips to work on characters that were never built for facial animation and reduces setup time.
