POLQA

Presented by: Senior Engineer, TIS – Member of Staff of OPTICOM GmbH Joachim POMY Moscow, 27-29 April 2011

“ “ “ “ “ “

Moscow, 27-29 April 2011

“ “ “ “ “ “


“ “ “ “ “ “


“ ”

No external funding or debt

” ”

Noise-to-Mask Ratio (NMR) 1988 Spin-Off from Fraunhofer-Institute (Home of mp3)

“ “

“ (1996), and now

(1999), (2010)

(2000),

“

“ “


(2004),

(2008),

“

“

the next-generation mobile voice quality testing standard P.863 ”

stands for ‚Perceptual Objective Listening Quality Assessment‚

“

Standardised as Draft ‘PESQ’

, following the history of P.861 ’PSQM’ and P.862

“

Specially developed for HD Voice, 3G and 4G/LTE, VoIP

“

Offers a new level of benchmarking accuracy

“ A joint development of the POLQA consortium in the ITU-T


“ “ “ “ “ “

“ “ “ “ “ “


Evolution of ITU-T Recommendations for Voice Quality Testing (P.86x - Full Reference MOS-LQO) ) B W S ( d n z a H b - k e 4 d i 1 w r e p u S ) B W ( d z n H a k b - 7 e d i W ) B N ( d z n H a k b - 4 . w 3 o r r a N

e c i o V D H

POLQA P.863 (draft) ??/2010

P.862.3

PESQ Application Guide 11/2005

S T O P

PSQM

PESQ

ITU-T P.861

ITU-T P.862

08/1996 (Withdrawn)

Speech Codecs Fixed Delay

1996

VoIP

Wide-band Extension to 7 kHz

P.862.1

P.862.2

11/2003

11/2005

MOS Mapping for Mobile Network Benchmarking

2005

2000

2G

Speech Codecs E2E Network Quality Variable Delay and Time Scaling Level & Linear Filtering Eff ects Acoustical Interfaces POTS and HD Voice (NB and WB/SWB) VQE Enhanced Networks Enhanced Accuracy of MOS P rediction

PESQ MOS-LQO

02/2001

Speech Codecs Variable Delay E2E Network Quality

PESQWB

3G

2010 3.5G

NGN UC 4G/LTE

Evolution of N etwork Technologies available at the time of development, i.e. included use cases for e ach Recommendation


e d . m o c i t p o . w w w –

0 1 0 2 H b m G M O C I T P O ©

Requirement Specification P.OLQA May 2008 Call for Proponents

Six model candidates announced First set of Superwideband Database for training purposes

July 2008

Statistical Evaluation procedure for P.OLQA

Start of model training Submission of mo del candidates to ITU-T

February 2009 July 2009

Second set of speech databases for evaluation purposes

Evaluation of model candidates Report to ITU-T SG12 Models from OPTICOM, SwissQual and TNO are selected to form th e new Rec. P.OLQA with a joint model

Consen t and Appr oval of P.OLQA (P.863) Characterization phase


May 2010

September 2010

“

“ ” ” ” ” ” ” ” ” ” ”


“ “ “ “ “ “



“

“ “

“

“

“

“

“

“

“

“

“

“

“


“

narrow-band narrow-band Averaged Averagedrmse* rmse* wideband wideband Averaged Averagedrmse* rmse*

PESQ PESQ P.862.1 P.862.1 0.1857 0.1857 PESQ PESQ P.862.2 P.862.2 0.3450 0.3450

rmse* rmse* POLQA POLQA Improvm. Improvm. 0.1363 0.1363

27% 27%

POLQA POLQA

Improvm. Improvm.

0.1506 0.1506

56% 56%

 1  Perror i²    N  d N 

rmse*  

Where….

Perror (i)  max( 0, MOSLQS (i)  MOSLQO(i)  ci95 (i)) Moscow, 27-29 April 2011

PESQ Performance - NB_8kHz504_SWISSQUAL, rmse* = 0.4204

5

4.5

4 ) 2 6 8 . P 3.5 ( . d n o 3 C O Q L - 2.5 S O M 2

1.5

1 1

1.5

2

2.5

3

3.5

4

4.5

5

MOS-LQS Cond.

27% improvement*

POLQA Performance - NB_8kHz504_SWISSQUAL, rmse* = 0.2311

5

4.5

4 ) 3 6 8 . 3.5 P ( . d n o 3 C O Q L - 2.5 S O M 2

*Narrowband average rmse*

1.5

1 1

1.5

2

2.5

3

MOS-LQS Cond.


3.5

4

4.5

5

improvement observed for all ITU tests

PESQ Performance - WB_16kHz204_FTDT, rmse* = 0.4221 5

4.5

4 ) 2 6 8 . P 3.5 ( . d n o 3 C O Q L - 2.5 S O M 2

1.5

1 1

1.5

2

2.5

3

3.5

4

4.5

5

MOS-LQS Cond.

56% average Improvement*

POLQA Performance - WB_16kHz204_FTDT, rmse* = 0.2319

5

4.5

4 ) 3 6 8 . 3.5 P ( . d n o 3 C O Q L - 2.5 S O M 2

1.5

*Wideband Average Improvement

1 1

1.5

2

2.5

3

MOS-LQS Cond.


3.5

4

4.5

5

observed for all ITU tests

PESQ Performance - WB_PSY_402_POLQA, rmse* = 0.3245

5

4.5

4 ) 2 6 8 . P 3.5 ( . d n o 3 C O Q L - 2.5 S O M 2

1.5

1 1

1.5

2

2.5

3

3.5

4

4.5

5

MOS-LQS Cond.

56% average Improvement*

POLQA Performance - WB_PSY_402_POLQA, rmse* = 0.1839

5

4.5

4 ) 3 6 8 . 3.5 P ( . d n o 3 C O Q L - 2.5 S O M 2

1.5

*Wideband average rmse* improvement

1 1

1.5

2

2.5

3

MOS-LQS Cond.


3.5

4

4.5

5

observed for all ITU tests

“ “ “ “ “ “


“

5 “

5

5 Clean speech, 50…14000Hz

Clean speech, 300..3400Hz Clean speech, 50…7000Hz (WB)

4

AMR 12.2kBit/s Clean speech, 300…3400Hz (NB) GSM HR

3

“ “

2

1 Moscow, 27-29 April 2011

1

1

“ “ “ “ “ “


| f s,Ref  f s,Deg, est | f s,Ref

0 1 0 2 , H b m G M O C I T P O ©

 1%

• •


0 1 0 2 , H b m G M O C I T P O ©


Very different to PESQ

© OPTICOM GmbH, 2010


Level too high or too low

x

x

0

Strong linear filtering

x

x

0

Noise in the reference signal

x

x

0

High timbre in the reference signal

x

x

0

Level variation

x

x

poor

SWB noise on NB/WB signal

x

x

0


Sample Rate

48kHz

8, 16, 48kHz

Ref. Bandwidth

50..14000Hz

300..3400Hz

Ref. Level

-26dBov (73/79dBSPL)

-26dBov (79dBSPL)

Deg. Level

-21..-46dBov

-26dBov


“ “ “ “ “ “


“ “ “

 


“ for Windows, Linux “ for Symbian, Android, ... “ incl. POLQA+PESQ+ECHO

“

Voice, Video, or Voice+Video

“ “

incl. POLQA+PESQ+ECHO


Europe, Middle East:

USA, Canada:

Asia-Pac: China Taiwan Korea


“ “ “ “ “ “ ”


Years of profitable Business Experience Years of Scientific Expertise International Standards (= 100% Conformance) Essential Patents and License Agreements Excellent Reference Customer Base :


“ “ “ “ “ “


Originally a working title of a new objective ‚instrumental‛ approach for prediction of Listening Quality, ITU-T SG12 / Question 9

Lead study group on quality of service and quality of experience

Subcommittee of ITU-T Study Group 12, dealing with perception-based objective methods for voice, audio and visual quality measurements in telecommunication services

Perceptual experiments where the human listeners and viewers in those experiments are named ‚subjects‛.

Instrumental prediction of quality. Measures made model a certain type of perceptual (subjective) experiment.



© OPTICOM GmbH, 2010


“

e B n d o S

B d

Completely Masked Partially Masked

“Smearing”

Bark

Bark

Convert to Loudness

e n o S

Bark

) n g o n i s i s n e e r p p r p a u h S S (

e B n d o S

Bark

“ “



POLQA

Recommend Documents