automatic image caption creation
Funded by: EU 6th Framework, 2006-2009
Principal Investigators: Dr Ross Purves and Dr Mark Sanderson
My role: I led research and development in a workpackage concerned with extracting and integrating information about tourist sites from the web.
Outputs: I developed a Java web service for extracting semi-structured facts from the web, written up as Salway et al (2010).
in Words (TIWO)
Funded by: EPSRC GR/R67194/01, £55,315, January 2002-2005.
Principal Investigator: Dr Andrew Salway
Aim: To develop a computational understanding of narrative in multimedia systems.
Application: Technology to assist the production of audio description, and technology to reuse audio description for video retrieval and browsing applications for films.
Partners: Royal National Institute of the Blind, the BBC, ITFC and Softel
Outputs: Five publications to date. Novel technologies for browsing and analysing films. Technology transfer is under discussion with BBC Access Services, BBC Archive and Philips. Three PhD dissertations.
Funded by: various Canadian agencies and international companies, 2002-2005.
Project Leader: Sara Diamond (then Artistic Director, Banff New Media Institute; now President Ontario College of Art and Design).
Aim: Language visualisation to study and enhance collaboration in online chat environments.
My role: I led workshops with scientists, artists and engineers on corpus-based analysis of online conversations and consulted on technologies for text classification, neural networks and software development strategies.
Output: Software for online chat with enhanced language visualisation.
Collaboration with: Dr Radan Martinec, University of the Arts, London
Aim: To develop a semiotic theory of image-text relations that is grounded in semiotic theory and that can be applied in the development of multimedia computing systems for image retrieval, hypermedia browsing and multimedia generation.
Output: Publication in Journal of Visual Communication.
Extraction from the Tate Gallery website
Collaboration with: Rachel Bhandari, Tate Gallery
Aim: To identify linguistic patterns in a 1,000,000 word corpus of painting captions and artist biographies that could be exploited for automatic image indexing.
Output: Seminar at Tate Britain and a refereed international conference paper.
Annotation and Browsing of Dance Videos
Collaboration with: Department of Dance Studies, Uni. of Surrey
Aim: to analyse the language used by experts to describe and interpret dance sequences, and to extract information from such texts in order to index digital dance video .
Output: PhD dissertation, 'Video Annotation: the role of collateral text.' A prototype dance video annotation and browsing system.
Information System (SOCIS)
Funded by: EPSRC GR/M89041/01, £326,316, 1999-2002
Principal Investigators: Prof. Khurshid Ahmad (Uni. of Surrey) and Prof. Yorick Wilks (Uni. of Sheffield)
Aim: To explore the link between language and vision computationally.
Application: Automatic generation of representations of digital images from the spoken descriptions of experts, leading to an image retrieval system for crime-scene investigations.
My role: I contributed to the proposal when I was a PhD student. In 2000 I was responsible for managing the project RA and PhD students, chairing project round table meetings with UK scene-of-crime officers and managers, and conducting data elicitation exercises.
Design with Information Systems (Safe-DIS)
Funded by: EPSRC / DTI GR/H95235, £220,000, 1994-7
Principal Investigators: Prof. K. Ahmad (Uni. of Surrey) and Dr R. Price (Wallingford Software)
Aim / Application: To enhance the role of information systems in making safety-related information available to design engineers in a timely and effective manner.
My role: I worked as Research Assistant with responsibility for project management (there was another RA and a PhD student) and for organising meetings with an industry round table. The project was awarded top marks for ‘Management and Use of Resources’. My research focus was on a corpus-based study of the ‘language of safety’.