An Augmented Reality App to Annotate Art

[This post is based on the Bachelor project by Jurjen Braam and reuses content from his thesis]

The value of Augmented Reality applications has been shown for a number of different tasks. Most of these show that AR applications add to the immersiveness of an experience. For his Bachelor Project, VU student Jurjen Braam researched to what extent AR technology makes sense for the task of annotating artworks.

To this end, Jurjen built a mobile application which allows experts or laypeople to add textual annotations to artworks in three different modes. One mode doesnt show the artwork, but allows for textual input, the 2nd mode shows the work in an image and allows for localised annotations. The last mode is the AR mode, which projects the artwork in the physical space, using the device camera and screen.

Three modes of the Application (Text, 2D, AR)

Jurjen evaluated the three modes through a small user study, which showed that immersion and enjoyment was highest in the AR mode but that this mode was least efficient. Also, participants indicated that for annotation tasks, larger screens would be preferable.

User evaluation in action

This research was a unique endeavour combining a proven technology (AR) and well-known task (Annotation) which identified interesting possibilities for follow-up research.

Share This:

Source: Victor de Boer

Posted in Staff Blogs, Victor de Boer

An Augmented Reality App to Annotate Art

[This post is based on the Bachelor project by Jurjen Braam and reuses content from his thesis]

The value of Augmented Reality applications has been shown for a number of different tasks. Most of these show that AR applications add to the immersiveness of an experience. For his Bachelor Project, VU student Jurjen Braam researched to what extent AR technology makes sense for the task of annotating artworks.

To this end, Jurjen built a mobile application which allows experts or laypeople to add textual annotations to artworks in three different modes. One mode doesnt show the artwork, but allows for textual input, the 2nd mode shows the work in an image and allows for localised annotations. The last mode is the AR mode, which projects the artwork in the physical space, using the device camera and screen.

Three modes of the Application (Text, 2D, AR)

Jurjen evaluated the three modes through a small user study, which showed that immersion and enjoyment was highest in the AR mode but that this mode was least efficient. Also, participants indicated that for annotation tasks, larger screens would be preferable.

User evaluation in action

This research was a unique endeavour combining a proven technology (AR) and well-known task (Annotation) which identified interesting possibilities for follow-up research.

Share This:

Source: Victor de Boer

Posted in Staff Blogs, Victor de Boer

W4RA work featured in BBC article

Screenshot of the bBC article
Screenshot of the bBC article “‘Siri, will talking ever top typing?’ By Padraig Belton”

A BBC web article “‘Siri, will talking ever top typing?’ By Padraig Belton features our W4RA work done on voice interfaces for farmer information in Northern Ghana.

Francis Dittoh talks about the need for specific information for farmers in their own language and discusses ongoing research into our Kasadaka system. Anna Bon talks more about the web of voices. Very nice to see our work recognized by international media!

Read more at http://www.bbc.com/news/business-43409952

Share This:

Source: Victor de Boer

Posted in Staff Blogs, Victor de Boer

W4RA work featured in BBC article

Screenshot of the bBC article
Screenshot of the bBC article “‘Siri, will talking ever top typing?’ By Padraig Belton”

A BBC web article “‘Siri, will talking ever top typing?’ By Padraig Belton features our W4RA work done on voice interfaces for farmer information in Northern Ghana.

Francis Dittoh talks about the need for specific information for farmers in their own language and discusses ongoing research into our Kasadaka system. Anna Bon talks more about the web of voices. Very nice to see our work recognized by international media!

Read more at http://www.bbc.com/news/business-43409952

Share This:

Source: Victor de Boer

Posted in Staff Blogs, Victor de Boer

Field trip to Ghana, feb 2018

As part of the ongoing W4RA efforts, the VU ICT4D team visited West-Africa once more. This time, we visited Mali, Burkina Faso and Ghana. I personally went to Ghana to talk to external PhD candidate Francis Dittoh and his colleagues at the University of Development Studies (UDS) and the SARI institute in Tamale, in Northern Ghana. Francis and myself talking to a shea-butter guideAfter first flying to Accra, I first was able to have a short meeting with my good friend Nana Baah Gyan, who is now an ICT for Development expert in that city. I then flew to Tamale,. where we met up with Francis to talk about his PhD work on information systems for rural farmers.

After colleagues Anna Bon and Hans Akkermans arrived from Burkina Faso, we met with UDS vice-chancellor as well as the Dean of the Faculty of Engineering to  discuss the ongoing collaboration of W4RA and VU with Prof Saa Dittoh, and Francis. We hope to formalize these collaborations in a Memorandum of Understanding between the two universities. We also discuss the ambition of UDS to set up a curriculum in software engineering as part of the undergraduate programs. Such a programme would include a yearly community service courses, where students go into the field. This is very interesting for us as it aligns well with the goals of the ICT4D course at VU. The programmes also include a French language course to allow for smoother cooperation with other Sahel countries (specifically Burkina Faso).The UDS mission statement

We also visited the Savanna Agriculture Research Institute (CSIR-SARI). The situation is comparable to that in Burkina. SARI governs the use of hybrid seeds, which cannot be reused year by year but result in a higher yield. SARI is interested in educating farmers on how to handle these seeds (fertilization, planting etc). There are three classes of seeds: 1) breeder seeds – grown at research institutes, 2) foundation seeds and 3) certified seeds – produced by ~1000 farmers monitored by ~10 seed companies, and certified by an external agency.SARI research institute

Potentially interesting for us is their connection to rural farmers. SARI now mostly does this through extension workers from the ministry of Agriculture. However, because of budget cuts, these are now reduced to roughly 1 extension worker to 2,500 farmers. Therefore, SARI is open to the idea of commucating with farmers directly, for example through a voice-based system. An interesting opportunity could be two-track strategy with a smartphone app for extension workers to allow them to do their work better and a companion voice app for farmers. SARI is very much interested in developing applications in a co-creation process. This would match the research proposal that Francis has submitted to SARI and UDS.

A personal highlight was the 10hr road trip that Francis offered me to join instead of taking a flight back to Accra. Ghana is a beautiful country that changes before your eyes on such a trip. My sincere hope is that the proposed collaborations will lead to many more visits to this great country. 

Share This:

Source: Victor de Boer

Posted in Staff Blogs, Victor de Boer

Field trip to Ghana, feb 2018

As part of the ongoing W4RA efforts, the VU ICT4D team visited West-Africa once more. This time, we visited Mali, Burkina Faso and Ghana. I personally went to Ghana to talk to external PhD candidate Francis Dittoh and his colleagues at the University of Development Studies (UDS) and the SARI institute in Tamale, in Northern Ghana. Francis and myself talking to a shea-butter guideAfter first flying to Accra, I first was able to have a short meeting with my good friend Nana Baah Gyan, who is now an ICT for Development expert in that city. I then flew to Tamale,. where we met up with Francis to talk about his PhD work on information systems for rural farmers.

After colleagues Anna Bon and Hans Akkermans arrived from Burkina Faso, we met with UDS vice-chancellor as well as the Dean of the Faculty of Engineering to  discuss the ongoing collaboration of W4RA and VU with Prof Saa Dittoh, and Francis. We hope to formalize these collaborations in a Memorandum of Understanding between the two universities. We also discuss the ambition of UDS to set up a curriculum in software engineering as part of the undergraduate programs. Such a programme would include a yearly community service courses, where students go into the field. This is very interesting for us as it aligns well with the goals of the ICT4D course at VU. The programmes also include a French language course to allow for smoother cooperation with other Sahel countries (specifically Burkina Faso).The UDS mission statement

We also visited the Savanna Agriculture Research Institute (CSIR-SARI). The situation is comparable to that in Burkina. SARI governs the use of hybrid seeds, which cannot be reused year by year but result in a higher yield. SARI is interested in educating farmers on how to handle these seeds (fertilization, planting etc). There are three classes of seeds: 1) breeder seeds – grown at research institutes, 2) foundation seeds and 3) certified seeds – produced by ~1000 farmers monitored by ~10 seed companies, and certified by an external agency.SARI research institute

Potentially interesting for us is their connection to rural farmers. SARI now mostly does this through extension workers from the ministry of Agriculture. However, because of budget cuts, these are now reduced to roughly 1 extension worker to 2,500 farmers. Therefore, SARI is open to the idea of commucating with farmers directly, for example through a voice-based system. An interesting opportunity could be two-track strategy with a smartphone app for extension workers to allow them to do their work better and a companion voice app for farmers. SARI is very much interested in developing applications in a co-creation process. This would match the research proposal that Francis has submitted to SARI and UDS.

A personal highlight was the 10hr road trip that Francis offered me to join instead of taking a flight back to Accra. Ghana is a beautiful country that changes before your eyes on such a trip. My sincere hope is that the proposed collaborations will lead to many more visits to this great country. 

Share This:

Source: Victor de Boer

Posted in Staff Blogs, Victor de Boer

Trip Report: Royal Society & KNAW – Responsible Data Science

Last week, I had the pleasure to be able to attend a bilateral meeting between the Royal Society and the KNAW. The aim was to strengthen the relation between the UK and Dutch scientific communities. The meeting focused on three scientific areas: quantum physics & technology; nanochemistry; and responsible data science. I was there for the latter. The event was held at Chicheley Hall which is a classic baroque English country house (think Pride & Prejudice). It’s a marvelous venue – very much similar in concept to Dagstuhl (but with an English vibe) where you are really wholly immersed in academic conversation.

.IMG_0290

One of the fun things about the event was getting a glimpse of what other colleagues from other technical disciplines are doing. It was cool to see Prof. Bert Weckhuysen enthusiasm for using imaging technologies to understand catalysts at the nanoscale. Likewise, seeing both the progress and the investment (!) in quantum computing from Prof. Ian Walmsley was informative. I also got an insider intro to the challenges of engineering a quantum computer from Dr. Ruth Oulton.

The responsible data science track had ~15 people. What I liked was that the organizers not only included computer scientists but also legal scholars, politicians, social scientists, philosophers and policy makers. The session consisted primarily of talks but luckily everyone was open to discussion throughout. Broadly, responsible data science covers the ethics of the practice and implications of data science or put another way:

Call to arms on Responsible Data Science by Jeroen van den Hoven "if we don't do it, somebody else will do it to us" @hoven_j #rdsuknl pic.twitter.com/48xb8SvR5S

— Frank van Harmelen (@FrankVanHarmele) February 21, 2018

For more context, I suggest starting with two sources: 1) The Dutch consortium on responsible data science 2) the paper 10 Simple Rules for Responsible Big Data Research. I took away two themes both from the track as well as my various chats with people during coffee breaks, dinner and the bar.

1) The computer science community is engaging

It was apparent through out the meeting that the computer science community is confronting the challenges head on. A compelling example was the talk by Dr. Alastair Beresford from Cambridge about Device Analyzer a system that captures the activity of user’s mobile phones in order to provide data to improve device security, which it has:

I'm at a joint @royalsociety @_knaw symposium on responsible data science. Interesting opening talk from @arberesford about their Android DeviceAnalyzer – and the huge vulnerabilities found! https://t.co/Kpg3lZl0d1 #RDSUKNL #fb pic.twitter.com/CqVG9PlteM

— Julian Huppert (@julianhuppert) February 21, 2018

He talked compellingly about the trade-offs between consent and privacy and how the project tries to manage these issues. In particular, I thought how they handle data sharing with other researchers was interesting. It reminded me very much of how the Dutch Central Bureau of Statistics manages microdata on populations.

Another example was the discussion by Prof. Maarten De Rijke on the work going on with diversity for recommender and search systems. He called out the Conference on Fairness, Accountability, and Transparency (FAT*) that was happening just after this meeting, where the data science community is engaging on these issues. Indeed, one of my colleagues was tweeting from that meeting:

Huge Kudos to https://t.co/uUaHfDb28i for reporting & helping fix this space. IBM replicated their results internally and released a new, improved API! 👏 #FAT2018

— Corey Harper (@chrpr) February 24, 2018

Julian Huppert, former MP, discussed the independent review board setup up by DeepMind Health to enable transparency about their practices. He is part of that board.  Interestingly, Richard Horton, Editor of the Lancet is also part of that board Furthermore, Prof. Bart Jacobs discussed the polymorphic encryption based privacy system he’s developing for a collaboration between Google’s Verily and Radboud University around Parkinson’s disease. This is an example that  even the majors are engaged around these notions of responsibility. To emphasize this engagement notion even more, during the meeting a new report on the Malicious Uses of AI came out from a number or well-known organizations.

One thing that I kept thinking is that we need more assets or concrete artifacts that data scientists can apply in practice.

So my question is how do I build values into a standard development life cycle? Need actionable artifacts Things like https://t.co/VG63JibOau are important for practice #rdsuknl @hoven_j

— Paul Groth (@pgroth) February 21, 2018

For example, I like the direction outlined in this article from Dr. Virginia Dignum about defining concrete principles using a design for values based approach. See TU Delft’s Design for Values Institute for more on this kind of approach.

2) Other methods needed

As data scientists, we tend to want to use an experimental / data driven approach even to these notions surrounding responsibility.

Computer/data scientists tend to look at news #diversity as a statistical measure of similarity/serendipity etc, while social scientists often look at as a precondition for democracy –@nhelberger #rdsuknl #filterbubble #fakenews

— IB (@1Br0wn) February 21, 2018

Even though I think there’s absolutely a role here for a data driven approach, it’s worth looking at other kinds of more qualitative methods, for example, by using survey instruments or an ethnographic approach or even studying the textual representation of the regulatory apparatus.  For instance, reflecting on the notion of Thick Data is compelling for data science practice. This was brought home by Dr. Ian Brown in his talk on data science and regulation which combined both an economic and survey view:

Personally, I tried to bring some social science literature to bear when discussing the need for transparency in how we source our data. I also argued for the idea that adopting a responsible approach is also actually good for the operational side of data science practice:

#rdsuknl @pgroth on reusing datasets pic.twitter.com/DI4LoiLUPW

— Virginia Dignum (@vdignum) February 22, 2018

While I think it’s important for computer scientists to look at different methods, it’s also important for other disciplines to gain insight into the actual process of data science itself as Dr. Linnet Taylor grappled within in her talk about observing a data governance project.

Overall, I enjoyed both the setting and the content of the meeting. If we can continue to have these sorts of conversations, I think the data science field will be much better placed to deal with the ethical and other implications of our technology.

Random Thoughts

  • Peacocks!
  • Regulating Code – something for the reading list
  • Somebody remind me to bring a jacket next time I go to an English Country house!
  • I always love it when egg codes get brought up when talking about provenance.
  • I was told that I had a “Californian conceptualization” of things – I don’t think it was meant as a complement – but I’ll take it as such 🙂
  • Interesting pointer to work by Seda Gurses about in privacy and software engineering from @1Br0wn
  • Lots of discussion of large internet majors and monopolies. There’s lots of academic work on this but I really like Ben Thompson’s notion of aggregator’s as the way to think about them.
  • Merkle trees are great – but blockchain is a nicer name 😉

 

Source: Think Links

Posted in Paul Groth, Staff Blogs

Trip Report: Royal Society & KNAW – Responsible Data Science

Source: Think Links

Posted in Paul Groth, Staff Blogs

A Voice Service Development Kit for the Kasadaka platform

[This post is written by André Baart and describes his MSc thesis]

While the internet usage in the developing world is still low, the adoption of simple mobile phones is widespread. A way to offer the advantages of the internet to these populations is voice-based information systems. The KasaDaka voice-services platform is aimed at providing voice-services in the context of ICT for Development (ICT4D). The platform is based on a Raspberry Pi and a GSM modem, which enables affordable voice-service hosting, using the locally available GSM network. The platform takes into account the special requirements of the ICT4D context, such as limited internet connectivity and low literacy rates.

This research focuses on lowering the barrier to entry of voice-service development, by reducing the skill set needed to do so. A Voice Service Development Kit (VSDK) is developed that allows the development of voice-services by deploying and customizing provided building-blocks. These building blocks each represent a type of interaction that is often found in voice-services. (for example a menu, user voice input or the playback of a message) The researcher argues that the simplification of voice-service development is an essential step towards sustainable voice-services in the ICT4D context; As this increases the potential number of local voice-service developers, hremoving the dependency on foreign (and thus expensive) developers and engineers. This simplification should ideally be achieved by providing a graphical interface to voice-service development.

The VSDK was evaluated during the ICT4D course at the Vrije Universiteit Amsterdam, where students built applications for various ICT4D use-cases using the VSDK. Afterwards a survey was conducted, which provided insight on the students’ experiences with voice-service development and the VSDK. From the results of the evaluation is concluded that the building-block approach to voice-service development used in the VSDK, is successful for the development of simple voice-services. It allows newcomers to (voice-service) development, to quickly develop (simple) voice-services from a graphical interface, without requiring programming experience.

The VSDK combined with the existing KasaDaka platform provides a good solution to the hosting and development of voice-services in the ICT4D context.

More details can be found in the complete thesis.A slidedeck is included below. You can find the VSDK code on Andre’s Github: http://github.com/abaart/KasaDaka-VSDK

 

Share This:

Source: Victor de Boer

Posted in Staff Blogs, Victor de Boer

A Voice Service Development Kit for the Kasadaka platform

[This post is written by André Baart and describes his MSc thesis]

While the internet usage in the developing world is still low, the adoption of simple mobile phones is widespread. A way to offer the advantages of the internet to these populations is voice-based information systems. The KasaDaka voice-services platform is aimed at providing voice-services in the context of ICT for Development (ICT4D). The platform is based on a Raspberry Pi and a GSM modem, which enables affordable voice-service hosting, using the locally available GSM network. The platform takes into account the special requirements of the ICT4D context, such as limited internet connectivity and low literacy rates.

This research focuses on lowering the barrier to entry of voice-service development, by reducing the skill set needed to do so. A Voice Service Development Kit (VSDK) is developed that allows the development of voice-services by deploying and customizing provided building-blocks. These building blocks each represent a type of interaction that is often found in voice-services. (for example a menu, user voice input or the playback of a message) The researcher argues that the simplification of voice-service development is an essential step towards sustainable voice-services in the ICT4D context; As this increases the potential number of local voice-service developers, hremoving the dependency on foreign (and thus expensive) developers and engineers. This simplification should ideally be achieved by providing a graphical interface to voice-service development.

The VSDK was evaluated during the ICT4D course at the Vrije Universiteit Amsterdam, where students built applications for various ICT4D use-cases using the VSDK. Afterwards a survey was conducted, which provided insight on the students’ experiences with voice-service development and the VSDK. From the results of the evaluation is concluded that the building-block approach to voice-service development used in the VSDK, is successful for the development of simple voice-services. It allows newcomers to (voice-service) development, to quickly develop (simple) voice-services from a graphical interface, without requiring programming experience.

The VSDK combined with the existing KasaDaka platform provides a good solution to the hosting and development of voice-services in the ICT4D context.

More details can be found in the complete thesis.A slidedeck is included below. You can find the VSDK code on Andre’s Github: http://github.com/abaart/KasaDaka-VSDK

 

Share This:

Source: Victor de Boer

Posted in Staff Blogs, Victor de Boer