Consortium
Project Information
Brunel Univesrity
Web site: http://www.3dvivant.eu
Centre for Research & Technology / Hellas Informatics & Telematics Istitute
Contract No: IST-7-248420-STREP
Institut für Rundfunktechnik
Start Date:
1st March 2010
Duration:
36 months
Holografika RAI Research Centre Rundfunk BerlinBrandenburg Instituto de Telecomunicações European Broadcast Union Arnold & Richter Cine Technik
Programme: FP7-ICT-2009-4
Coordinator: Brunel University
3D liVe Immerse Video Audio iNteractive mulTimedia
October 2010
Contents Editorial Editorial
1
Consortium
1
Indicative Use Cases
2
System Architecture
4
Examples of 3D Holoscopic Images
6
Dissemination
7
Contact
Dr. Amar Aggoun Brunel University School of Engineering and Design e-mail:
[email protected]
Dr. Emmanuel Tsekleves Brunel University School of Engineering and Design e-mail: Emmanuel.Tsekleves@ brunel.ac.uk
Welcome to the second newsletter from the European Project 3D VIVANT. 3D VIVANT is a research project co-funded by the EU 7th Framework and started its activities on the 1st of March 2010. In the period, since the previous newsletter, the consortium has completed the work under work package 2, entitled “Use Cases, Architecture and System Design”. A number of scenarios and use cases have been produced to showcase the proposed project technology. Also the system architecture of all project components has been defined and the user acceptance metrics have been identified. 3D VIVANT has been disseminated at various international conferences and exhibitions in this period, including the International Broadcast Exhibition 2010 in Amsterdam, Netherland, the Infocomm 2010 in Las Vegas, USA and the Asia and Entertainment Industry Fair 2010 in Korea and several more.
Project Coordinator & Technical Manager:
Administrative Manager:
2nd Newsletter
More information can be found on the project web site: http://www.3dvivant.eu
3D VIVANT is a STREP funded by the European Commission under the 7th Framework Programme
Page 1
Indicative Service Scenarios
2nd Newsletter - October 2010
Introduction One of the main objectives of the 3D VIVANT project, apart from creating a new visual sensation, is to make holoscopic content interactive by means of hyperlinks associated to 3D objects segmented and extracted from the video material. Furthermore, it will enable multimodal story-telling, by allowing mixing of 3D holoscopic content from a variety of sources, be it real or virtual. The envisaged service scenarios within the context of this project are based on typical genres of TV shows. A scenario tells a story in order to describe a service from an end–user’s perspective. It also describes roughly the production background from the professional user’s perspective. Thus, it describes examples of what happens when someone interacts with the system. These scenarios are presented below, from the en-user perspective. Magazine Scenario - “quergelesen” “quergelesen” is a TV magazine show about books for children aged approximately 8 to 14 years. The Lehmann family are keen readers and regularly watch “quergelesen”. The children enjoy watching the book presentations and it helps them to discover new books to read. Their parents recognise the
value of the show as it encourages their children to read more books. Typically, the family sits in various positions around the living room and watches the show on their new holoscopic TV screen. The 3D aspects of the show make for even more exciting viewing for the children. The Bauer family also likes to watch the TV show, but they watch it on their 2D Hybrid Broadcast Broadband TV (HbbTV). By starting the HbbTV application, the family has the option of accessing further information related to the show and hyperlinked objects.
The “quergelesen” show Presenter
The main usage is to watch the transmission on a 3D holoscopic display set on which the embedded hyperlinking functionality allows the user to follow proper links to get further information about the objects which appear in the video. Compatible formats, such as 2D video, are broadcast through traditional channels; the use of HbbTV sets facilitates the user with the hyperlinking functionalities. The audiovisual material is suitably formatted to be streamed on a web server and the appropriate browser/player allows the user to follow the embedded links.
The “quergelesen” show demonstarting mixing of real and vrtual content
3D liVe Immerse Video Audio iNteractive mulTimedia http://www.3dvivant.eu
Page 2
Indicative Service Scenarios
2nd Newsletter - October 2010
Stop-motion - “Sandmännchen” Sandmännchen (the sandman) is a daily children’s show that has been broadcast on German television for over 50 years. It comprises of five minute long shows that are designed to appeal to young children. This scenario is envisaged to be exploited on both broadcast TV and online. In the broadcast version of the scenario, the basic exploitation of the audiovisual product is on a holoscopic 3D TV, used like a usual TV set (i.e., passively). In the online version, through a specific PC player it is possible to both watch the 3D audiovisual material and to follow the hyperlinks associated to the 3D objects embedded in the clip. These links lead to additional information on the subject in the form of further video and audio files and text. Apart from hyperlinking, 3D objects can be used as samples to find similar objects in large 3D object databases. A simpler exploitation mode through a PC platform consists of watching a streamed online version of the programme. In addition, it is possible to watch the programme on a 2D HD HbbTV set. Tommy and his father access the 3D Sandmännchen portal where one can watch all 3D episodes. Tommy chooses yesterday’s chapter. Now, he and his father can see marked 3D objects and, by clicking the slightly flashing hot-air balloon, they get
access to a website which shows a collection of Sandmännchen vehicles. Now, Tommy can search for his favourite vehicles in Sandmännchen’s collection. He can have a look at all the fancy cars, planes and other vehicles which his beloved figure has used over the years. When Tommy clicks the marked house in this 3D episode, he returns to the 3D portal. When Tommy clicks the marked Sandmännchen figure he can access the basic Sandmännchen website and play the easy games there..
There is also a video-on-demand and online version of this scenario. In the online version Francesco watches TV programmes using an interactive TV set which enables him to follow the hyperlinks associated to selected objects. This way, he can explore the web site associated to the programme where he can find further information both on the selected object itself and about the historical period and the social and economical environment where the object was produced and used.
Museum Scenario The programme is a documentary show, depicting the evolution of the electronic technology from telegraphy to holoscopy. The story depicts the historical and technological evolution, with references to important historical facts and to the way of living in the corresponding periods. Francesco likes watching the weekly programme dedicated to technology because it is not limited to the description of equipment and their technical characteristics. On the contrary, the authors try to connect the technology with the historical period and the economical and social environments, pointing out the mutual influences. Thus, in the due time, he sits on his armchair in front of the holoscopic TV set and attentively watches the programme.
The “Sandmännchen” show demonstrating mixing of real and vrtual stop-motion content
3D liVe Immerse Video Audio iNteractive mulTimedia http://www.3dvivant.eu
Page 3
Use Case & System Architecture
2nd Newsletter - October 2010
Use Case In 3D VIVANT there are two kinds of actors interacting with the system, producers and viewers. Producers more or less correspond to TV professionals such as film producers and cameramen, while viewers correspond to home users watching 3D holoscopic content and listening to 3D audio. Producers will be able to record real world objects and 3D scenes and create holoscopic content. Apart from recording 3D video, producers will be also able to record 3D sound and synchronize the audio and video streams for high quality audiovisual experience. Producers will be also able to process recorded footage and mix holoscopic content coming from real and virtual sources, to arrange and edit the separate recorded sound tracks, to set hyperlinks in holoscopic video and to code holoscopic content and spatial audio. Holoscopic content and spatial audio created and edited by producers will be made available to viewers. They will be able to watch holoscopic content on various display devices, such as a light-field display, an integral imaging display and an autostereoscopic display as well as to listen to spatial audio synchronized to video, thus feeling a complete 3D audio-visual experience. 3D VIVANT will allow non-linear video viewing and viewer interactivity. Viewers will be able to follow hyperlinks while watching holoscopic video. Viewers will be also able to retrieve objects
and footage stored in databases, that is similar to samples (queries) provided by them to the system.
capture holoscopic content capture spatial audio
synchronize audio and video
view holoscopic content
listen to spatial audio
edit holoscopic content navigate in hypervideo
Producer
Viewer
edit spatial audio
create hyperlinks
search in holoscopic content
code holoscopic content-spatial audio
The 3D VIVANT Use Case Diagram
3D liVe Immerse Video Audio iNteractive mulTimedia http://www.3dvivant.eu
Page 4
Use Case & System Architecture
2nd Newsletter - October 2010
System Architecture The different modules that comprise the 3D VIVANT system are illustrated in the figure below as a conceptual block diagram. As illustrated, five modules can be identified: the Content Generation Unit, Content Processing & Editing Unit, Content Coding Unit, Content Search & Retrieval Unit and Content Terminal. Users interact with the system through the Content Generation Unit and the Content Terminal. As the names suggest, the Content Generation Unit allows users to record holoscopic content and 3D sound, while the Content Terminal provides the means to display holoscopic content and play back 3D sound. The Content Terminal also serves as a front end to others modules, and allows users to process and retrieve holoscopic content, which are capabilities provided by the Content Processing and Editing Unit and the Content Search and Retrieval Unit respectively. Finally, the Content Coding Unit provides coding functionalities that are necessary for storing and transmitting efficiently this new form of audiovisual content. 3D VIVANT will allow non-linear video viewing and viewer interactivity. Viewers will be able to follow hyperlinks while watching holoscopic video. Viewers will also be able to retrieve objects and footage stored in databases, that is similar to samples (queries) provided by them to the system.
Content Generation Unit
Content Coding Unit
Content Processing & Editing Unit Producers
Viewers Content Terminal
Content Search & Retrieval Unit
Conceptual overall architecture of the 3D VIVANT system
3D liVe Immerse Video Audio iNteractive mulTimedia http://www.3dvivant.eu
Page 5
Examples of 3D Holoscopic Images Examples of 3D Holoscopic Still Images
2nd Newsletter - October 2010
The consortium led by Brunel is working towards the generation of the first 3D holoscopic video. Examples of previously captured 3D Holoscopic still images are shown here. These have been captured using a number of different parameters. The image in the middle column of the page showcases a 3D holoscopic still image that features horizontal parallax whereas the ones on the right column features 3D holoscopic still images with full parallax (both horizontal and vertical).
3D holoscopic image of a fruitball featuring full parallax with a 847 µm lenticular pitch
To view these images and get the full 3D effect, a micro-lens array that matches the one used for the capturing of these images must be overlaid in front of the display unit or the printed image.
3D holoscopic image of a horseman featuring horizontal parallax with a 600 µm lenticular pitch
3D holoscopic image of soft toys featuring full parallax with a 250 µm lenticular pitch
3D liVe Immerse Video Audio iNteractive mulTimedia http://www.3dvivant.eu
Page 6
2nd Newsletter - October 2010
Dissemination 3D VIVANT at IBC 2010
3D VIVANT at Infocomm 2010
3D VIVANT participated in this year’s International Broadcasting Convention (IBC) Exhibition in Amsterdam, Netherlands from the 9th - 14th September 2010. IBC is the leading international forum for the electronic media industry.
Holografika participated in this year’s Infocomm 2010 exhibition in Las Vegas from 9th-11th of June, which attracted more than 32,000 visitors.
The 3D VIVANT exhibition stand received more than 300 visitors over five days and distributed over 150 project brochures. A lot of interest was shown by leading electronics manufacturing companies (Phillips, Samsung, LG), broadcast production companies (HBO, BBC) as well as broadcasting standard representatives (ETSI, DVB). Most visitors commented on the dynamics and advantages of the proposed 3D holoscopic imaging technology, which offers fatigue-free viewing, single camera-single aperture capture under natural light and many emphasised its future potential in substituting current 3D technologies (stereo and multiview). The project co-ordinator, Dr Aggoun, was also interviewed by a Polish broadcaster about the 3D VIVANT project (see photo). A conference paper related to the 3D VIVANT project was also published under the IBC proceedings.
Holografika showed 3D holoscopic content produced by Brunel on its HoloVizio 240P in the 3D Pavillion. In addition to this a presentation was given about the 3D VIVANT project by the company CEO, Mr Tibor Balogh, in the framework of the 3D Comm Seminar organised by Insight Media/Infocomm. Other Dissemination Events & Conferences The project has been also presented at the following conferences and events:
The 3D VIVANT project co-ordinator being interviewed at the 3D VIVANT IBC stand
- 9th Euro-American Workshop on Information Optics in Helsinki, Finland in July 2010, by Brunel. - Asia Content & Entertainment Industry Fair 2010, 3D Stereoscopic World Conference, in Gwangju, Korea, in September 2010 by Holografika. - 2nd Intelligent Imaging Programme Meeting, in London, UK in September 2010, by Brunel. - Future 4G Mobile Networks Workshop, in Lisbon, Portugal in October 2010 by IT.
A 3D holoscopic animation demonstarted on the HoloVizio 240P by the Holografika CEO at Infocomm 2010
3D liVe Immerse Video Audio iNteractive mulTimedia http://www.3dvivant.eu
Page 7
