Dongmei Jiang got her Ph.D degree in Computer Science and Technology from Northwestern Polytechnical University (NPU), Xi’an China in 2000. She worked as a Post Doctor in NPU from 2001.1 to 2003.11. During this period, she worked in Dept. Electronics and informatics (ETRO), Vrije Universiteit Brussel (VUB), Belgium from 2001.11 to 2002.7, for a bilateral project between the Ministry of Science and Technology (MOST), China and Flanders Government, Belgium. From 2003.11, she has been an associate professor in School of Computer Science, NPU. From 2005, Dr. Dongmei Jiang has been the spokeswoman of the Joint Research Group on Audio Visual Signal Processing (AVSP) between NPU China and VUB Belgium. From 2006.6 to 2007.10, she worked as a team leader in ETRO VUB, responsible for another international bilateral project.

     Dongmei Jiang’s research interest mainly focuses on Audio Visual Speech Processing, including: speech recognition, speech driven face animation, audio visual emotion recognition and expression. She has published about 40 papers and book chapters in English and Chinese core journals and famous international conferences. Under her supervision, 10 graduate students have obtained,and the other 10 are pursuing their Master degree.

    As the (co)promoter of research projects, Dr. Dongmei Jiang has been (and are) responsible for one National Natural Science Foundation project, one international bilateral project, two Shaanxi provincial Natural Science Foundation projects, as well an industrial project with budget exceeding 1 million CNY.

Research
 

Research Interest:

  • Audio Visual Speech Processing,
  • Audio Visual Emotion Recognition & expression,
  • Talking Face Animation,
  • Human-Machine Speech Communication..

Research Perspectives & Plans:
Building enhanced audio visual bimodal Human-Computer Interaction (HCI) systems, such that in an audio visual immersive virtual reality environment, the virtual avatar can recognize the emotion and speech of a user, and communicate with the user in corresponding face expression and emotional speech.

  • Audio Visual Emotion Recognition & Expressional Facial Animation

Under tight collaboration with Vrije Universiteit Brussel(VUB) Belgium, we are dedicating to emotion recognition through speech and facial expression, as well constructing expressive facial animation whose mouth movement matches well the speech content, and expression matches the emotion in the speech.

  • Virtual 3D Audio Effects
Depending on the sound perception mechanism of human head, we are trying to build virtual 3D audio effect using the Head Related Transform Function (HRTF), which is measured in a special audio lab. Without many speakers surrounding you, you can also feel the 3D sound effect.
Research Projects
 

National Natural Science Foundation Project. ”Research on Key Issues of Audio Visual Emotion Representation and Expression”, 2008.1-2010.12.

  • Shaanxi Provincial Natural Science Foundation Project. “Speech Driven Accurate Mouth Animation Synthesis”, 2008.1-2009.12
  • Shaanxi Provincial Natural Science Foundation Project. “Research on Hidden Markov Model (HMM) and Neural Tree Network Based Speaker Verification System”, 2003.1-2004.12.
  • Bilateral Scientific and Technological Collaboration between Flanders, Belgium and the Ministry of Science and Technology (MOST), China.”Audio Visual Speech Recognition and Synthesis: Bimodal Approach”, 2005.1-2007.12.
  • Industrial Project. “Sound Effect Processing algorithm of Audio Signal”, 2006.1-2006.5.
  • The Developing Program for Outstanding Persons—DPOP of Northwestern Polytechnical University. “Audio Visual Multi-Stream Speech Models Based on Dynamic Bayesian Networks”, 2004.7—2007.7.
Award-winning Results
 

Teaching and Research Award of Northwestern Polytechnical University for the semester year 2008-2009;

  • The paper “The Viseme Based Continous Speech Recognition System for a Talking Head" (Journal of Electronics & Information Technology, 26(3), pp.375-381, 2004) was awarded as “Shaanxi Provincial Excellent Science and Technology Publications” (second class) in 2005.

 

Selected Publications
 

09

  1. Dongmei Jiang, Ilse Ravyse, Hichem Sahli, Werner Verhelst. “Speech Driven Realistic Mouth Animation Based on Multi-modal Unit Selection”, Journal of Multi-Modal User Interfaces (Springer), OnlineFirst, DOI 10.1007/s12193-009-0015-7, 2009.
  2. Dongmei Jiang, Ilse Ravyse, Peizhen Liu, Hichem Sahli, Werner Verhelst. “Realistic Mouth Animation Based On An Articulatory DBN Model With Constrained Asynchrony”, Int. Conf. Audio, Speech and Signal Processing (ICASSP2010), Submitted.
  3. Dongmei Jiang, Peizhen Liu, Ilse Ravyse, Hichem Sahli, Werner Verhelst. “Video Realistic Mouth Animation Based on an Audio Visual DBN Model with Articulatory Features and Constrained Asynchrony”, Int. Conf. on Image and Graphics, Xi’an, China, Sept.19-22, 2009.
  4. Ping Fan, Dongmei Jiang, Ilse Ravyse, Fengna Wang, Hichem Sahli. “Dynamic Emotion Recognition Based on Video through Manifolds”, Int. Conf. on Image and Graphics, Xi’an, China, Sept.19-22, 2009.
  5. Danqi Chen, Dongmei Jiang, Ilse Ravyse, Hichem Sahli. “Audio-Visual Emotion Recognition Based on a DBN Model with Constrained Asynchrony”, Int. Conf. on Image and Graphics, Xi’an, China, Sept.19-22, 2009.
  6. Peiyan Song, Dongmei Jiang, Fengna Wang. “Articulatory Feature Based On Audio2visual Speech Recognition Model”, Application Research of Computers, 26(7),pp.2481-2483, 2009. (in Chinese)
  7. Yonghui Meng, Dongmei Jiang, Zhonghua Fu, Lei Xie. “A Novel Speech/Music Segmentation and Discrimination Algorithm”. Computer Engineering & Science, 31(4), pp.106-109, 2009. (in Chinese)
  8. Jie Bai, Dongmei Jiang. “Norma lized Amplitude Quotient Fea ture in Emotion Recogn ition”, Computer Simulation, 26(2),183-186,2009. (in Chinese)
  9. Fengna Wang, Dongmei Jiang, Peiyan Song. “Novel Articulatory Feature Based Dynamic Bayesian Network Model for Speech Recognition”, Computer Engineering and Applications, 45(8),pp.178-181,2009. (in Chinese)
  10. Cuihong Ren, Dongmei Jiang, Zhonghua Fu. “Speech Enhancement Based on α-Order Generalized Minimum Mean Square Error Estimators”. Microel Ectronics & Compu Ter, 26(3),pp.76-80, 2009. (in Chinese)
  11. Peiyan Song. <An Articulatory Feature Based Audio-Visual Speech Recognition Model>, Master Thesis, Northwestern Polytechnical University, March 2009. (in Chinese)
  12. Fengna Wang. < Research on Dynamic Bayesian Network Based Models for Speech Recognition>. Master Thesis, Northwestern Polytechnical University, March 2009. (in Chinese)

08

  1. Dongmei Jiang, Ilse Ravyse, Hichem Sahli, Yanning Zhang。 “Accurate Visual Speech Synthesis Based On Diviseme Unit Selection And Concatenation”, Proceedings of the IEEE 10th Workshop on Multimedia Signal Processing (MMSP2008), Oct 8-10, 2008, Cairns, Queensland, Australia, pp. 906-909. (EI: 090111832239).
  2. Jie Bai, Dongmei Jiang. “Emotion Recognition Based on Normalized Amplitude Quotient”, Application Research of Computers, 25(11), pp.3243-3246,2008. (in Chinese)

07

  1. Dongmei Jiang, Guoyun Lv, Ilse Ravyse, Xiaoyue Jiang, Yanning Zhang, Hichem Sahli and Rongchun Zhao. “Audio Visual Speech Recognition and Segmentation Based on DBN Models”. Robust Speech Recognition and Understanding, pp:139-156, ISBN-ISSN: 978-3-902613-08-0, 2007.
  2. Guoyun Lv, Dongmei Jiang, RongChun Zhao, Yunshu Hou. “Multi-stream Asynchrony Modeling for Audio-Visual Speech Recognition”. IEEE International Symposium on Multimedia 2007 (ISM2007), Dec. 10-12, 2007, pp.37-44, Taiwan, China. (EI:083311448445, ISTP:00253310700006
  3. Guoyun Lv, Dongmei Jiang, Rongchun Zhao. “Single Stream DBN Model Based on Triphone for Large Vocabulary Continuous Speech Recognition”. IEEE International Symposium on Multimedia 2007 (ISM2007) & Workshop on Multimedia Audio and Speech Processing: advancing the state-of-the-art, Dec. 10-12, 2007, pp.240-245, Taiwan, China.(EI 083411477168
  4. Guoyun Lv, Dongmei Jiang, Rongchun Zhao, Xiaoyue Jiang, H. Sahli. “Multi-Stream Asynchrony Dynamic Bayesian Network Model for Audio-Visual Continuous Speech Recognition”. 14th International Conference on systems, Signals and Image Processing (IWSSIP 2007) and 6th Eurasip conference Focused on Speech and Image Processing, Multimedia Communications and Services (ECSIPMCS 2007), June 27-30, 2007, vol. 1, pp.437-440, Maribor, Slovenia.(EI:083111421079,ISTP:000253680600046)
  5. Ali Sun, Dongmei Jiang, Guoyun Lv, Hichem Sahli, Werner. Verhelst. “Research on DBN Based Continuous Speech Recognition and Segmentation”. In Application Research of Computers. 24(10), pp.104-106, 2007。(in Chinese)
  6. Pengjuan Guo, Dongmei Jiang, Hichem Sahli, Werner. Verhelst. “Speech Emotion Recognition Based on Pitch Features”. In Application Research of Computers. 24(10), pp.101-103, 2007。(in Chinese)

06

  1. Guoyun Lv, Dongmei Jiang, Pengjuan Guo, Ali Sun,Rongchun Zhao, H.Sahli, W.Verhelst, "Single Stream DBN model for Continuous Speech Recognition and Phone Segmentation", 2006 International Symposium on Distributed Computing and Applications to Business, Engineering and Science (DCABES 2006), Hangzhou, China, vol.1:277-280, 2006. (ISTP:000242373900063)
  2. Ilse Ravyse, Dongmei Jiang, Xiaoyue Jiang, Guoyun Lv, Yunshu Hou, Hichem Sahli, Werner Verhelst, Rongchun Zhao. “DBN Based Models for Audio-Visual Speech Analysis and Recognition”. Proc. Int. Conf. PCM2006, LNCS, Advances in Multimedia Information Processing, pp. 19-30, 2006. (EI:065010297204
Current Positions & Academic Activities
 

Current Positions & Academic Activities:

  • Spokeswoman of the Joint Lab between Northwestern Polytechinical University, China and Vrije Universiteit Brussel, Belgium.
  • Organizing Committee Chair of the Eighth National Conference on Signal and Information Processing, Xiamen, China, July 19-22, 2009.
  • PC Member for HHME2009, Xi’an, China, Oct.29-31, 2009
  • Pubilication Chair for ICIG2009, Xi’an, China, Sept.19-22, 2009.
  • Orgnizing Committee Member for ACCV2009, Xi’an, China, Sept.24-27, 2009
  • Orgnizing Commitee Member for CVWorkshop2009, Beijing China, July 8-10, 2009.
Contact us
 
Mail address: Box 886, School of Computer Science, Northwestern Polytechnical University
Tel: +86(0)29-88491532
E-mail: jiangdm@nwpu.edu.cn
 
 

 

 


 

Dongmei Jiang School of Computer Science Northwestern pol

Hit Counter