S CIENCE S O CI FIC A LT ION C OMP PROTOT U T ING Y PING
The Social Multimedia Experience Christian Timmerer and Benjamin Rainer, Alpen-Adria-Universität Klagenfurt
Inter-destination multimedia synchronization and quality of experience are critical to the success of social TV, which integrates television viewing with social networking.
SOCIAL TV Beginning with this issue of Computer, I'm taking over the Social Computing column from John Riedl, who, sadly, passed away last year. I'm an assistant professor in the Department of Information Technology (ITEC) and a member of the Multimedia Communications Group at Alpen-Adria-Universität Klagenfurt, Austria. I'm also an associate editor of Computing Now and an editor of the Special Technical Community on Social Networking (STCSN), of which I was the inaugurating chair. My research focuses on immersive multimedia communication, streaming, adaptation, and quality of experience (QoE), which form the subject of this first column. The Social Computing column aims to explore the latest developments within the field from the point of view of both researchers and practitioners. Topics of interest include, but are not limited to, social networking services, applications, and tools; social search; social multimedia and social communications; mobile social networks; social network analysis and visualization; standardization trends and federated social Web initiatives; social computing business models; and societal aspects such as privacy and data protection. Interested readers who wish to contribute to this column should contact me directly at
[email protected]. Christian Timmerer, column editor
M
ultimedia has become pervasive in our daily lives. We consume but also create and share different types of user-generated and (semi) professional multimedia content— images, video, and audio—in various contexts using a plethora of devices.
0018-9162/14/$31.00 © 2014 IEEE
r3soc.indd 67
The emergence of the consumer as producer, or prosumer, has led to a new form of computing: social multimedia computing aims to bridge the gap between multimedia and social media by creating content not for merely passive consumption but to foster the creation of communities and to facilitate active participation by and interaction among users.1
Published by the IEEE Computer Society
Multimedia content quality is increasingly improving thanks to well-constructed and overprovisioned networking infrastructures that enable a broad range of new services and applications in both wired and wireless environments. One such service, social TV, supports the integration of television or TV-like content with social networks.2 Social TV makes TV viewing an online social event: a group of users, often geographically distributed, consume the same multimedia content—for example, a live stream of a sports event—while sharing their experience within a common social networking platform via text, voice, or video telephony. Figure 1 shows the general architecture of a social TV system. Existing applications typically embed a second screen for social networks, such as Twitter and Facebook, directly into the TV content screen. This is often realized through hashtags, and the media and communication synchronization requirements are rather loose or unspecified. Coordinating media playout and real-time communication among
MARCH 2014
67
2/20/14 5:26 PM
S O CI A L C OMP U T ING
Synchronized playout
Online community
Satellite broadcast
Consumer Consumer Consumer
Heterogeneous broadband networks (Semi)professional (wired, wireless) and amateur content acquisition
Live sports events • Olympic Games • NFL Super Bowl • FIFA World Cup •…
Geographically distributed consumers accessing the same content while communicating with one another Figure 1. Enabling a community of geographically distributed consumers to view different types of TV content captured by and delivered to them via heterogeneous networks and devices, and to simultaneously discuss and comment on that content, requires synchronizing media playout and real-time communication.
numerous users in different locations and with different networking capabilities is critical to social TV’s success. Thus, two central components of any social TV system are inter-destination multimedia synchronization (IDMS) and end users’ quality of experience (QoE).
INTER-DESTINATION MULTIMEDIA SYNCHRONIZATION IDMS research has gained momentum in recent years with proposed implementation schemes for numerous use cases—including gaming, e-learning, e-health, and real-time entertainment, just to name a few— and standardization efforts.3 Most solutions are tailored to specific scenarios, such as multiplayer online games or collaborative work, but all of them use one of three approaches to manage timing and control signals to achieve IDMS among participating clients: server/ client (also known as master/slave), in which a dedicated, predefined server handles timing and control information for the clients; synchronization maestro, which separates the content/service provider from timing and control signaling
68
r3soc.indd 68
instances; and distributed control, in which the clients select timing and control reference points among themselves, with the possibility of no central instance. Standardization efforts focus on enabling IDMS within the Real-time Transport Protocol (RTP) family and related architectures, as defined by the European Telecommunications Standards Institute and the Internet Engineering Task Force. Services and applications based on these standards assume that clients are clock-synchronized—for example, using the Network Time Protocol—and follow a synchronization maestro scheme using defined data formats. In principle, a typical IDMS system must be able to identify the synchronization point and threshold that determine asynchronism, signal timing and control information among the participating entities, and adapt media playout to establish or restore synchronism. Asynchronism in the media playout becomes noticeable once a real-time communication channel is available among the participating users and, hence, significantly impacts their QoE. For example, if a
user whose client media playout is ahead of others belonging to the same session comments on an important event during a live TV stream—such as a goal in a soccer match or touchdown in a football game—before other users have had a chance to see it on their respective clients, those users could become annoyed enough to drop out of the session. To address the problems caused by latency, jitter, and packet loss, researchers have developed various adaptive media playout (AMP) schemes to synchronize the audio visual features of multimedia content across different network conditions.4
QUALITY OF EXPERIENCE The European Network on Quality of Experience in Multimedia Systems and Services (www.qualinet. eu) defines QoE as “the degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and/or enjoyment of the application or service in the light of the user’s personality and current state.” As Figure 2 shows, various factors impact QoE; these aren’t limited to technical factors but also include social and psychological factors, commonly referred to as context. The default means to determine QoE for a given service or application is subjective quality assessments following predefined, sometimes standardized procedures. The results of such evaluations, which are typically conducted within a controlled laboratory environment and follow specific recommendations, are reliable if done properly. However, the procedure is time-consuming and costly, and involves a small number of participants. Alternatively, crowdsourcingbased QoE evaluations are less expensive and offer a larger and more diverse group of participants, but they also raise new challenges. From a technical perspective, such
COMPUTER
2/20/14 5:26 PM
evaluations can’t control for device, network, and interaction capabilities, which can vary widely among households. Assessing the reliability of results is also difficult, as subjects’ motives for participating unknown. Nevertheless, QoE crowdtesting seems suitable in the social TV context, and researchers are starting to develop best practices.5
M
ost current IDMS schemes are optimized for push-based RTP streaming with a well-established session between the client and server. However, multimedia services are increasingly moving toward pull-based HTTP streaming using existing infrastructures. In these so-called over-the-top (OTT) services, clients individually request multimedia content segments without an established client/server session, as HTTP is stateless. Thus, solutions are needed that let clients organize themselves to achieve IDMS while minimizing the impact on end users’ QoE. QoE evaluation for social TV services is still in its infancy and highly dependent on both content (for example, genre) and context (for example, device type and social interaction possibilities). Crowdsourcing-based approaches might be difficult or impossible to implement because they require multiple participants to communicate with one another simultaneously. A possible solution is to employ human computation– like games with a purpose (GWAP), wherein users play a given game in a social networking context and provide QoE ratings as a side effect. Emerging networking technologies, such as 5G, will facilitate the development and deployment of social TV systems by more efficiently supporting device-to-device communication. However, they will also raise new challenges—especially with respect to heterogeneity
Network
Device
Content format
Technical factors Task application
Quality of experience (QoE)
Social and psychological factors Environment Content
Context
User expectations
Figure 2. Users’ quality of experience (QoE) is impacted not only by technical factors but also social and psychological factors, commonly referred to as context. (Figure adapted from T. Ebrahimi, “Quality of Multimedia Experience: Past, Present and Future,” keynote address with slideshow at the 2009 ACM Conference on Multimedia, Beijing, China, 22 Oct 2010; www.slideshare.net/touradj_ebrahimi/qoe.)
of devices and platforms—to enabling a truly social multimedia experience.
References 1. Y. Tian et al., “Social Multimedia Computing,” Computer, Aug. 2010, pp. 27–36. 2. P. Cesar and D. Geerts, “Past, Present, and Future of Social TV: A Categorization,” Proc. 2011 IEEE Consumer Communications and Networking Conf. (CCNC 11), 2011, pp. 347–351. 3. M. Montagud et al., “Interdestination Multimedia Synchronization: Schemes, Use Cases and Standardization,” Multimedia Systems, Nov. 2012, pp. 459–482. 4. B. Rainer and C. Timmerer, “Adaptive Media Playout for Inter-Destination Media Synchronization,” Proc. 5th Int’l Workshop Quality of Multimedia Experience (QoMEX 13), 2013, pp. 44–45. 5. T. Hossfeld et al., “Best Practices for QoE Crowdtesting: QoE Assessment with Crowdsourcing,” IEEE Trans. Multimedia, Feb. 2014, pp. 541–558.
Christian Timmerer, Social Computing column editor, is a senior assistant professor at Alpen-AdriaUniversität Klagenfurt, Austria. Contact him at christian.timmerer@ itec.aau.at or follow him on Twitter at http://twitter.com/timse7. Benjamin Rainer is a research assistant at Alpen-Adria-Universität Klagenfurt, Austria. Contact him at
[email protected].
The IEEE Computer Society’s Special Technical Community on Social Networking aims to be the online portal for researchers and practitioners in social computing, social networking, and related fields, fostering communication and interaction among members of this rapidly growing global community. Further information about the STCSN and its goals and members are available at www.computer.org/ stcsn. Come and join now!
r3soc.indd 69
View publication stats
MARCH 2014
69
2/20/14 5:26 PM
