Memories for life Managing information over a human lifetime

ID: Fitzgibbon-Reiter (2003) PDF: (afstuderen:Fitzgibbon-Reiter (2003) - Memories for life Managing information over a human lifetime.pdf|PDF)

===Notes *M’n afstudeervoorstel zou uit behapbare examplars kunnen bestaan *Toch iets met zwervers ofzo. Vla-Cam. Docu gemaakt met de Vla-Cam. *Na je leven je memories meenemen in je graf. *Misschien moeten we alles maar in texfiles doen maar dan weet je nog niet in welke encoding en welk medium *Alle herinneringen moeten zelf proberen te surviven als memen ofzo. Dus de digitale herinneringen moeten zich “tonen” aan de mens. Aan jou maar ook aan anderen. Ze willen vermenigvuldigd worden. Zich verspreiden als een virus. *Eigenlijk is alles video. Een foto is alleen maar 1 frame daarvan. *Firefox plugin voor text snapshots evt zelfs images. Liefst html of pdf output ofzo.

===Memories for life, managing information over a human lifetime Andrew Fitzgibbon (awf@robots.ox.ac.uk) Ehud Reiter (ereiter@csd.abdn.ac.uk) May 22, 2003

People are capturing and storing an ever-increasing amount of information about themselves, including emails, web browsing histories, digital images, and audio recordings. This tsunami of data presents numerous challenges to computer science, including: how to physically store such digital memories over decades; how to protect privacy, especially when data such as photos may involve more than one person; how to extract useful knowledge from this rich library of information; how to use this knowledge effectively, for example in knowledge-based systems; and how to effectively present memories and knowledge to different kinds of users. The unifying grand challenge is to manage this data, these digital memories, for the benefit of human life and for a lifetime.

The digital archive of even one person in the year 2019 is likely to consist of petabytes of linked images, documents and audio; the potential for extracting useful knowledge from this archive is stupendous, and only limited by our imagination.

For example, a short-term challenge could be to develop a model of a users literacy level by analysing examples of what he or she reads and writes, and linguistically simplify web pages based on this model; this would help the 20% of the UK population with poor literacy.

#The computer and programs which operate on the data will change frequently over a human lifetime, but the data must outlast the systems which analyse it. #Questions will be asked of the data which were not predicted when the data was indexed, so the indexing strategies must change over time. #Security research must face the challenge of protecting information over decades, in a way that is robust to advances in computational power or mathematical knowledge, but without imposing untenable constraints on the users activity; and also the challenge of rigorously proving to a sceptical public that their memories are secure from hackers, amoral companies, and Big Brother governments.

should courts or the police have the right to access memories that are relevant to a legal case or criminal investigation? Should people who are included in another person’s memories (in a digital photograph, for example) have any control over how these memories are used? Should aggregate information from memories be made available for medical and other kinds of scientific research? Such issues must be resolved in a way that is satisfactory to the community..

The challenge will be met when the majority of people can efficiently manage their information stream, and when all of us can benefit from our digital memories.

===Examplars Exemplars of the Challenge This proposal develops this programme using illustrative “exemplars”: specialist systems or tools which encapsulate the essence of the challenge. Each exemplar has access to all data stored about an individual, or to a subset pertinent to some aspect of the individual’s life, such as their personal or professional activities.

#The deep, persistent model of the user which is inherent in the digital memories, but which will be differently mined by each challenge; #Sensory interaction between the user and computer which adjusts to the abilities of each user, including visual, aural and haptic interactions, and which allows all people access to digital information; #Extraction of deep structure from the repository of memories, first to index the information, and then to present new views of the knowledge embedded therein; #Adaptation of the representations to allow tasks whose specifications continually evolve, and for which the appropriate algorithms and data structures can not be known at the time the representation is first designed.

*Multimedia searching *Electronic GP *Stories from a life *Persopnal Simplified Webpages *Newspaper (Develop “smart electronic paper” that lets anyone (even people without formal IT education) write down thoughts, scribblings, drawings, or whatever, and have these incorporated into the person’s digital memories.) … newpaper also lets people display and annotate existing documents, including their other memories. *Intelligent Mathematics Tutor for Children *Create a model of the child’s mathematical knowledge, and use this to drive an intelligent tutoring system. Personalise examples and feedback *Aid for Elderly with Short-Term Memory Problems *Virtual Memories: For example, a 3D birthday party, or an action replay of one’s greatest sporting moment.

===Overcoming limits. *Database systems: We need to store many different types of data (text, audio, visual, log files) over a very long period of time (a lifetime and beyond). How can we do this in a manner which easily adapts to new hardware and software, which easily allows new types of information to be integrated when technology advances, *Security: How can we protect people’s privacy, especially when one person’s “memories” contain information about someone else? How can we prove both to the scientific community and to the general public that memories are secure from attackers? *Operating systems: A person’s memories will contain petabytes of data and last for decades. How should this data be distributed across physical filestores, in a way that maximises accessibility and reliability (including reliability in the face of “once in a century” disasters)? *Artificial intelligence: How can we interpret audio and visual data, with a minimal amount of annotation and guidance from the person? How can we learn useful generalisations from the interpreted data, and how can we represent and reason with these generalisations? *User modelling: How can we represent people’s knowledge, experiences, beliefs, emotions, intents, abilities, and so forth in a coherent and unified fashion? How should we update such models when people change? *Human-computer interaction: What is the best way from an HCI perspective for annotating and searching memories? How will new sensor technologies such as haptic interfaces be integrated? how can we support people with disabilities or skill impairments, or people in stressful environments? *Graphics and virtual reality: Given such a rich source of knowledge about the world, how can we use it to build virtual models of the world that integrate all the different types of sensory information in a coherent and consistent way?

===Paradigm shift *Paradigm shift: from the concept of a “computer” forlife, to a “memory” for life *we must stop thinking of information in terms of disjoint data types (images, audio recordings, text files, web pages), and instead think holistically of information as giving different perspectives on people, events, and the world. *the data collected will itself be an invaluable resource for the cognitive sciences **example, detailed long-term data about individuals would be a tremendous resource for longitudinal studies about child development, disease progression, and so forth, **managing and analysing large data sets would be very helpful to many fields of science, ranging from genomics to economics. We live in an era where the amount of scientific data is exploding exponentially; managing and analysing this data is a challenge for all of science.

===“Does it appeal to the imagination of the general public?”=== *our memories are what define us, and we believe that with careful presentation, the general public would be excited and inspired by our vision of computing with digital memories. *the media consider this topic to be of widespread interest. People will certainly be concerned about the extent to which information about them is captured, and about who owns and has access rights to this information. Such issues are already being widely debated in the popular press. This is a public policy issue, not a scientific one, but people may be hostile to this challenge unless they feel confident that they have sufficient control over their memories.

===“Does it avoid duplicating evolutionary development of commercial products?” *Although most commercial R&D is focused on short-term incremental improvements to search technology, Microsoft in particular is also working on longer-term research in this area, in the MyLifeBits project mentioned above. *any exemplar will involve many false starts, and will require many minds working in parallel to achieve results.

===“When was it first proposed as a challenge? Why has it been so difficult so far? Why is it now expected to be feasible in a ten to fifteen year timescale? ” An argument could be made that this was first proposed by Vannevar Bush in the 1940s. But it is only now that technology (disk capacity, sensors, processor speed) permits the acquisition and storage of large diverse collections of digital memories.

===“What are the most likely reasons for failure?” *Outright failure is unlikely—information management will always improve. It may prove, however, that some tasks cannot be solved without human cognition, and cannot be usefully automated. *significant risk is perhaps public hostility if the privacy implications are not carefully addressed.

===Long-term benefits “What kind of long-term benefits to science, industry, or society may be expected from the project even if it is only partial successful?” Each of the above exemplars and scientific challenges should produce useful science and technology even if only partially successful.

===“Does it have international scope?” This challenge is of interest to researchers around the world. Indeed, DARPA in the US currently has a “Lifelog” programme2 which is similar to Memories for Life in many ways, although focused more on the next 5 years than the next 20 years. (fotograferende japanners)

===First steps *… several contributors to our web discussion suggested that we consider creating an example Memories for Life corpus,