Coding multimedia communications
Lisimachos P. Kondi
Dr. Kondi is specialized in coding for multimedia communications, especially, bandwidth limited systems, such as wireless communications. He heads the Multimedia Communications Laboratory. The laboratory conducts advanced research and development in the areas of error-resilient image and video compression techniques, joint source/channel coding, digital image restoration, resolution enhancement and multimedia transmission over wireless channels and IP networks.
Video Transmission Over Wireless Channels:
Multimedia transmission over wireless channels has been considered as a research area of increasing interest. Highly error-prone and largely varying channel conditions in such applications require us to design a joint source-channel coding scheme to efficiently utilize the available target bit rate, while keeping the bit error probability sufficiently low.
The problem of optimal (in joint source-channel coding sense) video transmission over wireless channels is considered here. Two types of channels are considered: Flat Rayleigh fading channel using Binary Phase Shift Keying (BPSK) modulation and Direct Sequence Code Division Multiple Access (DS-CDMA) channels. A scalable video codec (MPEG4/3-D SPIHT/H.264) is used in conjunction with unequal error protection is used for simulation purposes. In case of DS-CDMA channels, each user transmitting video is considered to occupy more than one CDMA channels.
We are also working on the issues of transmitting video over Orthigonal Frequency Division Multiple Access (OFDMA) channels with joint source-channel-power optimization.
Resolution Enhancement of Video Sequences (Super-resolution):
In many imaging systems, the resolution of the detector array of the camera is not sufficiently high for a particular application. Furthermore, the capturing process introduces additive noise and the point spread function of the lens and the effects of the finite size of the photo-detectors further degrade the acquired video frames.
The goal of resolution enhancement is to estimate a high-resolution image from a sequence of low-resolution images while also compensating for the above-mentioned degradations. Resolution enhancement capitalizes on the subpixel motion that exists between video frames. We have developed resolution enhancement techniques that are based on Maximum A Posteriori (MAP) estimation.
Two sets of images are provided: One in JPEG and one in TIFF (without compression). JPEG files are smaller but compression introduces undesired artifacts. File data_truck_original shows the original 128x128 image taken with an infrared camera. File data_truck20_ratio4 shows the result of our super-resolution algorithm. It is a 512x512 image obtained by combining 20 128x128 images.
For comparison purposes, file bilinear_ratio4 shows a bilinear interpolation of the original 128x128 image. It can be seen that the result of the super-resolution algorithm is much more impressive than simple interpolation.
Original Image (128X128):
Bilinear Ratio: data_truck20_ratio4:
Scalable Video Compression:
A scalable video encoder produces a bitstream that can be partitioned into layers that form a hierarchy. One of the layers is called the base layer and the other layers are called enhancement layers.
The base layer is required for decoding while the enhancement layers can be decoded along with the base layer to provide enhanced video quality. Scalable video coding is useful in video transmission over heterogeneous networks, such as the Internet, and transmission over noisy channels, in conjunction with unequal error protection.
We have developed a single-pass scalable video encoder with is based on the partitioning of the Discrete Cosine Transform (DCT) coefficients. The partitioning is optimal in the Operational Rate-Distortion sense. The proposed algorithm employs Lagrangian optimization and Dynamic Programming.
Shape Coding:
The MPEG-4 video compression standard allows the transmission of objects. Each object consists of shape and texture information.
We have developed techniques for shape coding which outperform the techniques proposed by MPEG-4. The shape is approximated using a polygon or a higher order curve (B spline). We have also considered the topic of jointly optimal shape and texture coding.
Enhancing spatial resolution of video frames using optic flow computation:
The term "optic flow" refers to a visual phenomena experienced during motion. Essentially, optic flow is a visual displacement flow field that can be used to explain changes in a sequence of images.
Optic flow computation can be used to estimate high resolution images from a set of low resolution observations. The critical requirement for this estimation process is that the observations must contain different but related views of a scene. For dynamic scenes such as video, the necessary shifts are introduced by the relative displacement / motion of objects from one frame to another.
Further, in motion compensated DCT based video encoders, a large chunk of the complexity is attributed to the motion estimation, process where large search windows are traversed in order to obtain the best motion vectors for an inter frame. Optic flow computation can also be used to determine the direction of displacement of macroblocks from one frame to another, thus minimizing search windows during motion estimation.
