About the Role
This role focuses on researching and developing advanced speech synthesis and generation algorithms using LLMs and multimodal models. The position involves optimizing systems for online applications and exploring full-duplex/streaming multimodal LLM capabilities for real-time interaction.
Responsibilities
- Research and develop advanced speech synthesis and generation algorithms (e.g., TTS, voice conversion, sound/music generation) based on LLMs, multimodal/omnimodal LLMs.
- Develop and optimize speech and audio synthesis systems for online applications, improving effectiveness, efficiency, and scalability.
- Explore and advance full-duplex/streaming multimodal LLM capabilities in speech understanding, generation, and real-time spoken interaction.
- Collaborate cross-functionally with research and engineering teams from prototyping to production.
Requirements
- Ph.D. in Computer Science, Electrical Engineering, Signal Processing, or a closely related field.
- Strong foundation in speech/audio processing and modern generative models (e.g., diffusion, flow matching, autoregressive, codec-based approaches).
- Hands-on experience extending LLMs to speech/audio modalities (e.g., speech tokenizers, multimodal adapters, speech-text joint training).
- Experience with full-duplex or streaming spoken dialogue systems and real-time interaction modeling is a plus.
- Proficient in Python and deep learning frameworks (e.g., PyTorch).
- Experience with distributed training is a plus.
- Track record of publications at top-tier venues (e.g., ICML, NeurIPS, ICLR, ACL, ICASSP, Interspeech).
Skills
- Speech synthesis
- Speech generation
- LLMs
- Multimodal LLMs
- Omnimodal LLMs
- Voice conversion
- Sound generation
- Music generation
- Speech processing
- Audio processing
- Generative models
- Diffusion models
- Flow matching
- Autoregressive models
- Codec-based approaches
- Speech tokenizers
- Multimodal adapters
- Speech-text joint training
- Full-duplex spoken dialogue systems
- Streaming spoken dialogue systems
- Real-time interaction modeling
- Python
- Deep learning frameworks
- PyTorch
- Distributed training
Education Level
- Ph.D.
About the Company
- LIGHTSPEED STUDIOS is made up of passionate players who advance the art & science of game development through great stories, great gameplay, and advanced technology.
- We are focused on bringing next generation experiences to gamers who want to enjoy them anywhere, anytime, across multiple genres and devices.
- Lightspeed Tech Center is a R&D department under Lightspeed Studios which develop PUBG Mobile and other high-quality games.
- Our Tech Center leads the research, exploration, and discovery of innovative technologies and provides technical services for all games during all phases of life cycle, including engine, audio, QA, AI, next generation game, technical cooperation, etc.
Equal Opportunity
- As an equal opportunity employer, we firmly believe that diverse voices fuel our innovation and allow us to better serve our users and the community.
- We foster an environment where every employee of Tencent feels supported and inspired to achieve individual and common goals.
