Van Gogh Comes to Life: The Power of Machine Learning

Researchers at CONNECT, the Science Foundation Ireland research centre for Future Networks at Trinity College Dublin along with Qatar Computing Research Institute, have developed a Machine Learning technique capable of producing head portraits in the style of a given painter, including Vincent Van Gogh.

The technique, unveiled today in Trinity to mark Science Week, uses Deep Learning which is a form of Machine Learning, to extract details from the photograph of an individual and combine these with details from a sample of an artist’s work. A painting is then produced of that individual in the style of the artist.

The technique has been developed by Dr Ahmed Selim at the CONNECT Centre in Trinity College Dublin in collaboration with Dr Mohammed Elgharib, a Trinity graduate now working at Qatar Computing Research Institute, and Professor Linda Doyle, Director of CONNECT and Professor of Engineering and the Arts at Trinity. Their work featured at this year’s SIGGRAPH conference in California, the world’s largest and most prestigious international conference on Computer Graphics and Interactive Techniques.

Professor Linda Doyle said:

“Ahmed has combined cutting edge, Machine Learning techniques with some particularly creative insights to produce an output that is streets ahead of any of his competitors.

“This technique uses complex algorithms to combine information about the identity of an individual with information from the style of an artist to produce a faithful head portrait. In essence, the machine is being taught to do the job of the painter.

“The underlying mathematics can also be extended to other domains in the areas of health and telecommunications.”

“There is tremendous commercial potential for a technique like this. The gift market is an obvious one. Portrait painting is a popular genre.”

Dr Ahmed Selim said:

“There is tremendous potential for the use of the technique in social networks, gaming, animation and the film industry. We have extended our technique to work with video and we are exploring the ability of producing style transfer in real time.”

“Automated portrait painting is particularly challenging. There are already generic painting techniques available but these often deform facial features. Our technique avoids these pitfalls completely. It better captures the painterly qualities of the painting while at the same time better capturing the identity of the individual to be painted.”