In my postdoc at the Idiap research institute (Switzerland), I have been working on unsupervised activity mining from video data. The goal was to automatically mine, without supervision, the recurrent activities that are present in a video. One application, that we considered in the VANAHEIM European project by which I was funded, consists in pre-filtering the hundreds of surveillance cameras present in a metro network. We used our approach to find recurrent activities that we consider as the “normality”. When a new video stream is processed, anything that cannot be explained by these normal activities is then considered as worth showing to a human operator for further analysis.

In this context, I've been developing new probabilistic temporal models using non-parametric Bayesian methods. Inspired by “topic models”, the models we proposed have a strong temporal component: they are able to recover some temporal sequence information within each topic. Not only these models can separate and identify the different activities but they can also temporally localize the occurrences of the activities. The use of non-parametric for these model makes it possible to integrate, in a probabilistically sound manner, the automatic tuning of model parameters directly inside the model. Our model is for example able to automatically estimate the optimal number of activities and their occurrence count.

The developed models are not tied to video data and are actually very adequate for any temporal documents particularly if these are caused by multiple mixed but not necessarily synchronized phenomenons. The methods have for example been applied to audio data to find recurrent noise patterns. Mining activities in multimodal data (such as video + audio) or in data from other sensors using these methods is one of the direction for future work.

This software creation is my first for a company. Written before any solid formation in computer science, this project was a real project with client relationships, requirements analysis and full development of the application. This application is written in C++ for Windows (98 and XP) and weights around 15k C++ SLOC (source lines of codes), resources and custom graphical interface descriptions excluded.

Initial development has been followed by some actions of corrective maintenance and some important evolutions. The software is currently in its third major version and is used daily for risk studies to construct and secure roads and habitations. Maintenance, spread over more than 6 years, provides an important feedback on the design. Well and wrongly designed parts of the software are revealed by maintenance: a complicated correction or a major evolution can be made trivial by good previous design while a simple correction can be very long due to wrong design decisions.

This project stemmed from the success of the previous one, two years before. For certain studies, a 3D digital terrain model is necessary to ensure proper simulation results for example when the terrain features rocky “couloirs”. For this new software, the decision was taken to start from scratch and use Java, Swing and a proper MVC (Model/View/Controller) architecture. The software weights 36k Java SLOC, is developed under linux and mainly used under Windows. Initial development of this software was conducted in cooperation with one other software engineer.

As with the 2D version, initial development was followed by corrective maintenance and evolutions. The software is use daily for studies and allow to export 3D graphs for the reports that the geologist engineers write. This second real size project was the occasion to work in collaboration with someone on both the design and the code, and to apply and deepen my skills in software architecture, Java and Swing. Requiring 3D rendering, this project needed to evaluate and choose among 3D APIs: Java3D was chosen over raw OpenGL as it corresponded better to our requirements in this project. The simulation of 3D rockfalls is more complicated and time consuming than 2D simulation. Effort has been put on optimizing this 3D simulation and it is eventually as fast as the 2D version. In this project also, maintenance gave an extraordinary feedback on the various design decisions.

During my Ph.D. thesis, I took the initiative to develop a graphical user interface to help in the understanding and the management of service oriented architectures to simplify the use of the OMiSCID middleware. This development had three main goals: helping individual service designers in the PRIMA research group, being a demonstration platform and favoring capitalization and reuse in the group.

The OMiSCID Gui can be used to list services and their interconnections. It can be automatically and dynamically updated and extended by new modules contributed by service designers. The Gui has an high adoption rate in the PRIMA group and is widely used and extended by service designers. The ease of installation, extension and update is a powerful way of sharing service graphical interfaces in the research group. The site of OMiSCID Gui, http://omiscid.gforge.inria.fr/omiscidgui.html, is the reference on it.

The OMiSCID Gui is designed as a Netbeans Platform application (equivalent/concurrent-of Eclipse rich client platform) that uses standard Java Swing as its toolkit. Manipulating the Netbeans Platform gave me the occasion to understand how powerful and modular it is. Once mastered, such platform allow to easily build modular application that can be extended dynamically by anyone. I ported OMiSCID Gui (and the core library) to the Apache Maven build system. This gave me the occasion to discover this build system that is widespread and powerful but hard to grasp in some particular cases. I also contributed back to the community an article on a new design pattern (applied to the Netbeans platform) http://netbeans.dzone.com/nb-selector-based-lookup The article was commented by the initial architect of Netbeans.

I followed various formations in pedagogy and taught during 4 year at the Ensimag french engineering school in computer science and applied mathematics. I concentrated my formations on pure pedagogy and particularly on socio-constructivism with the “scientific debate” method. Constructivism aims at making students construct sense around what they learn. The “scientific debate” teaching method consist in organizing a debate among students on a well selected problematic notion of a course. During the debate, students make their own opinion, learn to express and challenge it, learn to accept the opinion of the others and finally raise their motivation for the notion that the teachers will present afterwards.

Concerning the scientific debate, I participated in a workshop aiming at accompanying young teachers in the practice of the scientific debate in their courses. The principle of this workshop was to prepare, put in practice with video recording and analyze some scientific debate sessions between our students. The whole group of teachers participated in the preparation and the analyze of all sessions of scientific debate. We analyzed how the workshop operated and drew some positive overall conclusions. This analyze was formalized as an article published in a french colloquium on pedagogy.

My Ph. D. thesis in the PRIMA research group from INRIA is entitled “Semantic Description of Services and Service Factories for Ambient Intelligence” and is in the domain of software architecture for Ambient Intelligence. Ambient Intelligence tries to bring intelligence to the ever growing network of computing devices surrounding us. This domain is highly interdisciplinary and involves many research fields. Advances in Ambient Intelligence are conditioned by the proper interaction, capitalization and integration of all disciplines.

A first facet of the contributions of this thesis concerns the usability of service oriented architecture: we introduced a simple service oriented middleware and a extensible graphical user interface dedicated to the visualization and the interaction with services. A second contribution consists in introducing a design method that reuses concepts from SOA but insists on the usability by non software-engineering specialists to favor early integration. With this method we introduces the concept of a “service factory” that emerges as a necessary construct from our analysis. A service factory represents a family of services that can be infinite and the factory can instantiate any of these services on request. This thesis also proposes a language and a runtime execution environment for our method building on top of our SOA middleware.

In addition to my thesis subject, I installed, configured and promoted various tools to improve communication in the research group: an internal wiki, a subversion server, an central agenda merging people agendas, etc. I also stimulated the organization of presentations and meetings, and the creation of discussion groups.

My master in the PRIMA research group at INRIA was an occasion to discover machine learning. My subject consisted in modeling, using machine learning, normal outputs of a visual tracking system to detect configuration problems and enable automatic reconfiguration of the system to dynamically adapt to changing operating condition. My master generated a publication with my supervisors.

I discovered various aspects of machine learning: supervised or unsupervised, generative or discriminant, incremental or not, various degree of generalization, etc. I implemented and experimented with numerous machine learning methods: k-means, EM, hierarchical clustering, SVM, LDA, AdaBoost, Neural Networks, Neural Gaz, etc.

A three month contract between my master and my Ph. D. thesis gave me the occasion to apply the machine learning skills I acquired: I implemented and wrote two deliverables on error detection and correction for the CAVIAR European project This three month period generated a deposit of software protection and, later, a publication.

I contributed to the “bépo” project in its first years of existence and followed its evolution since then. My contributions were mainly in the form of ideas, particularly on how to structure the evolution of the project, and then in the form of source code. The project aims at creating a keyboard layout optimized for the french language by using the method from August Dvorak who introduced the “Dvorak Simplified Keyboard” http://en.wikipedia.org/wiki/Dvorak_Simplified_Keyboard. Keyboards using the “bépo” layout can now be bought online. The project main page is at http://bepo.fr/ and a interesting reading about the original Dvorak layout can be found at http://dvzine.org/.