A free-viewpoint video broadcasting architecture for mobile long-term evolution networks

Abstract

Recent advances in mobile phone technology, cellular networks, and multimedia coding techniques, have paved the way for a wide range of innovative multimedia services that can be delivered to mobile devices. One interesting application is freeviewpoint television (FTV), which is expected to be the ultimate three-dimensional television (3DTV) innovation in the visual media market. FTV is a system for viewing natural video which allows the user to interactively select the viewing position of a scene. However, this service is characterised by large bandwidth requirements and hence, its deployment over cellular networks is not a trivial task. Indeed, current mobile FTV frameworks are not suitable for broadcast applications. Moreover, the deployment of FTV over a specific cellular technology has never been explored. This dissertation aims at developing a feasible free-viewpoint video (FVV) broadcasting architecture, for the next generation long-term evolution (LTE) cellular networks. The 3D scene is represented using the captured multi-view sequences and corresponding depth maps, with the latter representing the scene geometry and ultimately used for view synthesis. In a strive to minimise bandwidth requirements while ensuring that the computational work load on the mobile terminal is kept at a low level, a solution is employed in which depth estimation is performed at the transmitting end, and 3D warping-based view synthesis on the mobile terminal. This approach also eliminates the need of uplink feedback channels for viewpoint requests. For practical transmission, the individual colour and depth compressed streams are assembled into a single bitstream using a novel colour plus depth bit-stream format. The dissertation also looks at analysing the effect of handovers. Thus, a frame jitter algorithm is also developed. Through a series of objective tests, this study reveals that for a given satisfactory level of rendering quality, the proposed framework achieves a bit-rate reduction of 70.6% over the reference frameworks in literature. Furthermore, the results of a subjective framework evaluation indicate that the proposed framework provides better quality of experience (QoE) for patterns containing free-view motion. The reference framework is only preferred for fixed viewpoints. Thus, it can be concluded that the proposed architecture can be successfully deployed in the LTE mobile infrastructure, since besides requiring practical LTE bit-rates, it guarantees an acceptable QoE for typical free-view patterns.