ProjectUse AcrVQWin to run subjective experiments for video quality in Windows

Use AcrVQWin to run subjective experiments for video quality in Windows

We hereby provide AcrVQWin as freeware:

Zip file containing software and manual - license terms
Getting started with AcrVQWin 1.0 (PDF Manual)

Download here

Introduction

The program AcrVQWin (version 1.0) (Jonsson and Brunnström, 2007) is a program to run subjective experiments for video quality in Windows environment, using the Absolute Category Rating (ACR) method (ITU-T, 1999). It implements the test procedure in the Video Quality Experts Group (VQEG) MultiMedia phase I testplan. (Hands and Brunnström, 2006). It was used by all laboratories performing test in VQEG MultiMedia phase I (VQEG, 2008). It is designed to present the video on a computer screen, using the picture sizes QCIF, CIF and VGA, synchronized with the refresh of the display. This is implemented using DirectX.

Hardware Requirements

As the program uses a multithread function it requires a computer with two processors or a processor that supports multithreading. At least 256 MByte video memory. A primary memory of at least 512 MB. There are graphics card with 256 MByte video memory that is not sufficient. This could also be a driver issue.

Software description

The description below is a citation from the VQEG MultiMedia phase I Final report (VQEG, 2008):

“PCs were used to store and play the video content, using special purpose software, developed by Acreo (AcrVQWin version 1.0). This software was used by all test laboratories. The playback of a video clip was performed by pre-loading the clips in the memory of the PC’s graphics card. This was done to ensure that no frame drops occurred and that the update of each played frame happened in synchronization with the display update. The tests included a mixture of 25 frames per second (fps) and 30 fps. The subjective results were stored directly on the same PCs that were used to present the video. To the right you see the voting dialog in the subjective test software.

The most common LCD computer monitors have 60 Hertz (Hz) as their update frequency. The test plan, therefore, specified the monitor to be set to 60 Hz. Each frame was shown during two update frequency periods to obtain a frame rate of 30 fps. 25 fps was obtained using a modified 2-3 pulldown sequence. For example, each set of five frames was displayed according to the following number of screen updates: 2, 3, 2, 3 and 2.

To minimize waiting for the subjects, the next PVS video sequence was loaded during voting time using multi-threading programming techniques. The ACR rating scales were presented on the LCD after each video clip, using a dialog box as shown in Figure 3. A setup file was used to change the language of the text in the dialog box to that used by the testing laboratories in the different countries. Subjects provided their vote responses using the mouse of the PC. In each subjective test, the presentation order of test sequences was fully randomized between subjects with the exception that two PVSs originating from the same SRC were not allowed to be played next to each other, as specified in the test plan. After the vote was given and the OK button was pressed, the next PVS was automatically played. The software indicated when half of the PVSs had been rated, allowing the subjects to take a break.

The subjective test software (AcrVQWin) was controlled using a setup file, which the operator selected at startup. The setup file specified the particular PVSs and other startup parameters. Before the actual test, a practice session was performed to familiarize the viewer with the test procedure and the range of qualities used in the test.”

References

Hands, D. and Brunnström, K., (2006). Multimedia Group Testplan, Video Quality Experts Group, www.vqeg.org.

ITU-T, (1999). Subjective video quality assessment methods for multimedia applications, (ITU-T Rec. P.910), International Telecommunication Union, Telecommunication standardization sector,

Jonsson, J. and Brunnström, K., (2007). Getting started with AcrVQWin , (acr022250), Acreo AB, Kista, Sweden.

VQEG, (2008). Final report from the Video Quality Experts Group on the validation of objective models of multimedia quality assessment, phase I, (VQEG Final Report of MM Phase I Validation Test), Video Quality Experts Group (VQEG).

Term of usage

This software is provided as is. No warranty or support is given. It may be used for research purposes only.

Known problems with version 1.0

The text string to the parameter “SubjectId” should not contain the character ‘/’

Contacting us

If you have any questions or would like to use this software,please, do not hesitate to contact us:
Kjell Brunnström, kjell.brunnstrom [at] acreo.se