Nektarios Moumoutzis scopes the work and describes the current activity of WP3 - Audio/Visual and Non-traditional Objects.

Cluster Organization

The cluster is co-ordinated by Professor Stavros Christodoulakis of the Technical University of Crete, Greece and Professor Alberto Del Bimbo of the University of Florence, Italy. The cluster comprises seventeen organizations, (see the list of the organizations and contact details)

Objective

The major objective of the Audiovisual and Non-Traditional Objects Cluster of DELOS II NoE (http://delos.music.tuc.gr) is to focus on research activities related to the management of the lifecycle of Audiovisual and Non-Traditional Objects including:

Capturing of digital objects in order to integrate them into digital libraries.

Analysis of the content of these objects emphasizing metadata extraction and the correlation between them.

Management of audiovisual metadata and content including the use of databases, indexing mechanisms, and support for versioning.

User interactions exploiting the semantics of content for efficient retrieval and dissemination.

Transformation of the digital objects and synthesis of new objects to support services such as multi-channel delivery, personalized browsing and repurposing (e.g. creation of educational experiences).

Advanced applications

Scope of the Work

The scope of the work of the cluster is two-fold: The first part is to develop common foundations for:

1. Metadata capturing for audiovisual content: In particular, the cluster is investigating the state of the art of automatic extraction of audiovisual metadata and produces new models and experiments with prototype systems for automatic and semi-automatic extraction models in domain-specific and context-specific audiovisual application environments.

2. Universal access and interactions with audiovisual libraries: The cluster is looking into the state of the art of audiovisual information access and interaction exploring models based on the integration of existing frameworks of multimedia content (e.g. MPEG-7, TV_Anytime) and of domain-specific extensions of those models in order to enhance the effectiveness of retrieval and provide value-added services. Moreover it is also exploring models of user interaction based on the concept of context and context-based retrieval in audiovisual digital library applications. The cluster is also studying models and interfaces for advanced applications of audiovisual digital libraries. It intends to develop new solutions and conduct experiments with prototype systems for processing, semantic transcoding, interaction and presentation of audiovisual content for different delivery media and interaction devices so as to adapt the users' preferences regarding content to the specific characteristics of the delivery media and the users' terminals.

3. Management of audiovisual content:The cluster is also looking at state-of-the-art database models and data structures appropriate to the storage, retrieval and dissemination of audiovisual metadata and will conduct experiments with prototype systems. It is also investigating state-of-the-art models for profiling and stereotyping users of the most important multimedia applications and current dissemination strategies.

The second part of the scope of work within the cluster is to support common research activities in specific areas identified by the partners, taking into account the current trends and the future directions in the field emphasizing the development of innovative solutions for emerging digital libraries which are integrating Audiovisual and Non-Traditional Objects.

Cluster-supported Research Activities

Research activities identified by the technical annex and currently being investigated include:

1. Research on the development of automatic annotation algorithms at the semantic level, taking into account both higher ontologies supported by multimedia content standards and domain Ontologies addressing the needs of specific application domains such as sports, education, cultural heritage, etc.

2. Research on universal access and interaction with audiovisual libraries including the development of interaction models that capture issues related to the use of digital objects by final users; context models that represent contextual information related to user intentions, physical location of the user, history of interactions etc.; multimodal interfaces integrating new means of user interactions including natural language and speech; models for interoperability of applications of non-traditional objects following diverse standards or Ontologies.

3. Research on the management of audiovisual content in digital libraries including semantic retrieval languages, models for segmentation and summarization as well as models for multimedia databases and multimedia P2P computing.

4. Research on techniques for 3D objects modeling and retrieval.

5. Research on the integration of semantic video annotation and transcoding so as to obtain a semantic transcoding system; extension to real time applications.

6. Joint activity on the development of advanced multimedia demonstrators and test-beds in order to provide a reference point for the development of innovative tools and mechanisms for digital library services by ensuring a common evaluation framework.

Cluster Activities So Far

During the eight months from the official start of the project, an effective management structure for the cluster has been established which began with the cluster kick-off meeting in January 2004. The meeting also nominated a Steering Committee for the cluster. In addition, the skill sets of cluster members have been identified in detail and mapped to the activities of the cluster.

Cluster workshops have been scheduled for the dissemination of cluster results and for the discussion and development of new proposals, solutions and achievements for audiovisual digital libraries. An internal workshop will be held in Chania, Crete, during November 2004.

Cluster 3 - Audio/Visual and Non-Traditional Objects - is organizing a joint workshop with cluster 4 - User Interfaces and Visualization - entitled "Seventh International Workshop of the EU Network of Excellence DELOS on AUDIO-VISUAL CONTENT AND INFORMATION VISUALIZATION IN DIGITAL LIBRARIES (AVIVDiLib'05)". Detailed information about this workshop can be found at here.

To facilitate communication between the cluster members as well as the dissemination of cluster information to the other Delos II NoE members and to the outside world, a site was established at the address http://delos.music.tuc.gr. The site is being updated on a daily basis with new material and news related to the activities of the cluster.

The co-ordination with the other relevant Delos II NoE clusters is a matter of major importance. To support closer co-ordination and collaboration, liaison partners have been nominated as follows:

Prof. Hans Schek with cluster 1 - Digital library architecture

Prof. Stavros Christodoulakis with cluster 2 - Information access and personalization - and with cluster 5 - Knowledge extraction and semantic interoperability

Prof. Alberto Del Bimbo with cluster 4 - User interfaces and visualization

A major activity since the beginning of the project has been the development of State of the Art reports in the major axis of interest of the cluster. These are:

State of the Art report of Metadata Extraction for images, videos, 3D objects, audio, domain-specific ontologies and representation frameworks

State of the Art report of Audiovisual Content-based Retrieval, Information Universal Access & Interaction including Datamodels & Languages

State of the Art report of Audiovisual Metadata Management

The State of the Art reports are also intended to map the expertise and recent research of cluster members in the international research framework.

An important task under way is the development of a website offering the demonstrators and test datasets to be used during the project. Demonstrators can be on-line or off-line. With the former, a link to the demonstrator is given ro provideimmediate use. With the latter, case a link is provided to downloadable software which users can then install on their machines. Different types of test datasets will be chosen to be made available for evaluation purposes, including datasets for images, videos, 3D graphics, audio, music, etc.

Plans for the Next Months

The cluster is currently finalizing the State of the Art reports including previous research by the cluster in the field. The reports will include a section on the future directions that will guide the work of the cluster.

In order to schedule the work in the next 18-month phase of the project and take decisions regarding the funding of the participating partners, the cluster co-ordinators have issued a request to the cluster members for joint research proposals . The proposals will be evaluated on the basis of their interest and will serve to identify the work to be done over the next 18 months. The next 18-month plan will also take account of the Technical Annex, the State of the Art reports and the proposals submitted.

Author Details

Nektarios Moumoutzis
Project Co-ordinator & Researcher
Laboratory of Distributed Multimedia Information Systems & Applications
Technical University of Crete (MUSIC/TUC)
Chania
Crete
Greece

url: http://www.music.tuc.gr/Staff/Staff.htm
E-mail:
Telephone: +30-821-64803

---

Publication date: October 2004
File last modified: Monday, 22-May-2006

The Delos Newsletter is published by the Delos Network of Excellence
and is edited by Richard Waller of UKOLN, University of Bath, UK.
   

PDF version of the whole issue