“Real-world data sets are multimodal.”
For me, this was the most-important single line said during the opening speech, which reflects why we want to model real-world data sets as Knowledge Graphs rather than as (relational) tables.
Just shy of a month ago, I visited the lovely city of Porto, Portugal to attend the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases. I was there to present my PhD research on Deep Learning from Knowledge Graphs, which had been accepted as a poster. While there, I additionally participated in the 2nd workshop on Linked Data for Knowledge Discovery, as well as attend many of the interesting key notes and sessions.
With this trip being my first experience with a large and international conference, I was baffled by the immense organisation behind it all. More than 500 people attended which, together, resulted in 265 (out of 802) accepted papers. These were divided between four tracks, emphasizing either fundamental research, industrial applications, journal papers, or demos. In addition, 12 workshop were being held, as well as 10 tutorials. Despite all of this, the conference seemed to go without too much of a hitch.
As during previous installments, Deep Learning was a frequently-seen subject and still very much a debated topic. Whereas some thought of it as the holy grail of the field, I spoke to others who were sceptical about it and who thought it was merely the current hype. Still, most people agreed that its many recent success stories should count for something. One strong believer of Deep Learning appears to be NVIDIA, who had a large booth at a prime spot: the coffee machine. There, they showed of their “easy to use” frameworks for running Deep Learners on a GPU farm. Interestingly, two of these frameworks are available freely and as open source. The first being CuDNN, which is a plugin that offers a simple Deep-Learning solution to existing frameworks such as Caffe, Theano, and Torch. For a more low-level framework, NVIDIA developed DIGITS. While, of course, these frameworks are purely intended to increase the sail of NVIDIA gpus, I find it interesting that such large corporations invest in novel research areas.
Nine key notes were held in total. Kicking off the conference was Pedro Domingos from the University of Washington. His talk, which I found to be the most interesting of all key notes, focussed on his current work on Sum-Product Networks (SPN). SPNs are said to offer a tractable means for learning from rich probabilistic models. Being fairly simple in concept, they are defined recursively as either a univariate distribution or a product over or weighted sum of other SPNs. This allows them to compute all marginal and conditional MAP states in linear time, contrasting starkly with commonly used models such as Bayesian Networks and Markov Random Fields. As such, he argued, SPNs are the most powerful models capable of Deep Learning. Whether this may be true or not, I am fascinated by the concept behind these SPNs, and am eager to try my hand at it.
Another very interesting keynote was given by Wei Fan from Columbia University, who spoke about smart agents which aid individuals with medical or general healthcare issues. Near all of those people seek for information on the internet, often resulting in a wrong diagnosis or treatment. To combat this, Wei Fan and his team have been working on an easy-to-access medical Q&A-system, available as an app for your smartphone. By using Natural-Language Processing and a Deep Learning architecture, the system, known as mRobot, is able to answers a wide range of questions. More impressive is its capability of requesting more information from the user (e.g. ‘Where does it hurt?’) when the likely cause of the medical issue is still unknown. When it is known, the given advice may vary from a prescription to the advice to go see a physician.
The Doctoral Consortium session was held on the Monday morning, prior to the official opening ceremony. Out of the 30 Ph.D. candidates attending, only 6 had been given the opportunity to present their work on stage. Having received the somewhat-disappointing news that I wasn’t amongst those 6 almost three months before, I remember doubting whether to attend the conference at all. Luckily, I decided to go regardless, as I experienced the poster session to be far more interactive and useful than any of the 6 speakers must have experienced their talks. Whereas I had several really-interesting discussions about my research with several individuals, ranging from fellow peers to a professor, I noticed none of that during the talks. Despite the host’s best efforts, only few interesting questions were posed by the audience. I must admit not having asked anything myself either, primarily because I was not that well versed with any of the topics on that level. Perhaps it would be a better idea to divide the Doctoral Consortium in short 5-minute overviews each, with a minute reserved for questions.
Originally, we were supposed to remove our poster (2mb) at noon, when the Doctoral Consortium session ended. Fortunately, we were told after the talks that we could leave them up for the remainder of the day, thus exposing our research to all those attending the official opening and/or the first keynote. Following advice from a colleague, I had brought several printed copies of my paper with me to give to anyone who was interested. At the end of the day, I had given 16 copies away; a good advice indeed! Hopefully, I will hear back from some of them.
The Linked Data for Knowledge Discovery (LD4KD) workshop was a small half-day workshop hosted by Mathieu d’Aquin and his Ph.D. Ilaria Tiddi, both from the Open University. In all, the workshop attracted about a dozen participants. Moreover, a mere 4 papers had been submitted, of which only two were accepted. Because of this, I was, at first, rather pessimistic about the usefulness of it all. However, I quickly noticed that it, in fact, made room for an interesting discussion to spur, making the workshop much more interesting than simply a miniature presentation session.
During his opening talk, Mathieu d’Aquin spoke about the currently-existing problem of bridging the gap between the Semantic Web (SW) and Machine Learning (ML) communities. His experience with both communities allowed him to see the differences in research questions, goals, and approaches that both communities tend to favor when it comes to ML on the SW. Whereas the SW community views ML as an interesting approach to answer new and yet-unsolvable questions, the ML community regard RDF data as simply a different sort of data. However, both communities, Mathieu believed, are still too constrained by the methodologies they are accustomed to. Hence, semantics remain largely unexploited, as well as background knowledge from implicitly or explicitly linked sources. Going one step further, he ended his talk with the proposed notion of embedding the ML methods within the SW itself.
Next up was Ilaria Tiddi, who had been evaluating the currently-available Knowledge Discovery (KD) tools that work on SW data. Sadly, many were found to be abandoned or out-of-date, with most others being still very experimental. A nice exception was the LOD extension to Rapidminer, which is still being actively developed. I had arrived to the same conclusion several months before, when I explored many of such tools within the context of the ARIADNE project.
Out of the two presented papers, the one from Denis Krompass and Volker Tresp on ensemble learning for link prediction was the most interesting to me. By combining three different latent-feature models, each capturing different aspects of the data, they were able to improve the best single predictor up to 11%. Two of the models used, namely RESCAL tensor factorization and Google’s Knowledge Vault Multiway Neural Network (MNN), exploited background knowledge into their algorithms via semantics and graph structure, respectively. In addition, Google’s MNN employs several techniques similar to those I am currently exploring. Suffice it to say, I was looking forward to hearing more on this topic.
Having initially doubted to attend the conference, I am very happy to have gone in the end. Not only was the Doctoral Consortium session far more useful than I would have thought for someone without a talk, but I ended up speaking to quite a number of interesting people. Similarly, the small number of participants in the KD4LD workshop made the prominent individuals much more easy to approach and to go into an interesting discussion with. Having shared my thoughts with fellow researchers working on similar topics really strengthened my motivation to continue with my research.
Finally, I have to stress how friendly the local residents were. Not once have I felt uncomfortable being in Porto. This, in addition to the lovely city, tasty food, and wonderful climate, convinced me that I will certainly have to go back there again some day.