The wind that blows in the trees is felt in your hands. It can only be the beginning of the year because the almond trees are in bloom, reminiscent of a blizzard in some Nordic kingdom. Fingertips cool. In addition to the smell, in the mouth there is an almond flavor. Visiting a museum or an exhibition, for example, about the legend of almond blossom, throws us into the picture. It sounds like fiction, but it's reality. It is the combination of creativity and technology “made in Algarve”. A company in Loulé came up with the idea, a group of engineers and other specialists from the University of Algarve developed the technology. The M5SAR has arrived to mess with the senses.
There is an application and a small device that connects to the cell phone. Software e hardware they were designed for use in museums and exhibitions, but they can be adapted to products, brands and countless uses.
João Rodrigues is one of the minds that broke stone to develop Paulo Bica's idea. One is a professor and vice-rector at the University of Algarve, the other leads SPIC, an academic partner company in various projects.
This time, the challenge was to give museums a tool that would take advantage of the cell phones that are in everyone's pockets and offer more than the usual audio guides or pamphlets. The system was developed in order to combine the integration of augmented reality, image recognition technology and a sensory device coupled to a mobile phone.
“The idea was to have a more interactive and personalized experience” says the specialist in electronics and computer engineering. “The person signing up for the application defines their characteristics and preferences and the system manages to define the ideal path. If we are dealing with an art expert, you will want more detailed information on some pieces and just point your cell phone at the painting or object and this data will be made available to you. If you are a child or someone with limited mobility, the route may be different. Depending on the visitor's interests, the pieces to be highlighted can also change”.
The user has full autonomy to define the path he wants and reinforce knowledge through digital content. “If the person says they want to see pieces from the Roman period and the application realizes through the georeferencing that they are making stops in the Greek estate room, the route is automatically adapted”, adds João Rodrigues.
By also pointing your phone at the pieces, you are given more information about what you are seeing. The more pieces you point, the better the route is adjusted. It is a real-time interaction, through sensitive and cognitive use. With the recognition of two-dimensional and 3D images, a kind of instantaneous dialogue is achieved between the work and those who visit it.
There are also doors that open with Augmented Reality technology, which gives the user additional practical, theoretical, aesthetic and symbolic information. Access to information content can include data about the work, the artist, the technique or the historical context, through text, photography and audiovisual materials.
Another advantage of this sensory guide to museums is that no one gets lost. “In the case of a family, it is possible for the father and mother to be in a room watching part of the exhibition and the children in another more adapted room, with the parents always knowing where the kids are. The same applies to group visits.”
The microlocation technology through sensors allows to improve the visitor experience. It is possible to interact with the museum rooms at any point, identify and receive automatic notifications of the places where you go.
Visiting an exhibition with some of these solutions can turn the experience into a kind of game or competition. But despite these components being interesting in terms of service offering, the great innovation was to develop a device that, added to a mobile phone, allows creating the sensation of the five senses.
Don't just see. Feel everything around you!
The challenge on the M5SAR page already tells what's to come. If vision and hearing are widely explored on any smartphone, through the videos and audios we access, the AMtop device, which integrates the system designed by the team from the University of Algarve, adds three more human senses.
Sensations of touch, smell and taste are created. It is possible to hold the device and feel the hot and the cold, with micro scent dispersers, fragrances are perceived and it is also possible to have the sensation of taste through vaporized flavors.
João Rodrigues takes the legend of the almond trees in bloom to exemplify: "it is possible, throughout the exhibition, to smell the almonds, the thermal sensation of the cold or, if the piece is linked to flavors, "taste", for example, a almond. There will not be the fruit itself to be crunched, but the device, like an electronic cigarette, has a straw attached and the person has the sensation of the taste”, he explains. The system is individually dosed and meets all hygiene and safety criteria.
The construction of the prototypes involved two components and three hands full of different specialists. There was the lab part, centered on artificial intelligence, hardware and software. I work mainly with electrical, computer and mathematical engineers. Another step involved a more multidisciplinary team, with food engineers, designers and museology.
senses were lacking
SPIC looked at the museums and realized that much of the offering was equal to a big yawn. Technological and interactive infrastructures had a limited option and the orientation of visitors in the space was supported by guides, traditional paper supports or audio guides.
This in a world where everyone has technology at their fingertips on any mobile device. Paulo Bica explains the reasoning “the popularity and evolution of mobile devices could allow the visit to be more autonomous, with several information directories and a more immersive and complete experience”.
Once the product was developed, the market began to be thought about.” The project was aimed at museum units, but sensory technologies, in particular, allow reaching different markets. In addition to prioritizing the museum sector, it has enormous applicability in technological tourist attractions, events, brand experiences”.
Mine blast, facial recognition and digital masks on the market
SIPC invested around 200 thousand euros, half of which supported by community funds in a project that was completed in 2018, with the prototype ready to be launched. The pandemic, however, came to disrupt the plans.
The M5SAR had aimed its sights at museums, but this aspect of the product still needs to be polished. Paulo Bica admits that “there are needs for additional development to create a more functional commercial solution. At the moment, the product is being restructured, based on the most current and optimized technologies”.
Even so, its applicability can be seen in Parque Mineiro de Aljustrel, where several interactive installations were created, including the sensory experience.
“A simulation of a mining explosion was created that allows visitors to be involved in the experience that awakens all the senses”, explains the Spic administrator “the solution features an immersive sound system, with integration of sensors and smoke diffusers and smells. The final result presents a high emotional factor, making the visit more memorable”.
Of the technologies identified and refined during the execution of the project, some are already commercialized. For Vodafone and HBO, SPIC created an interactive structure that processes users' biometric information, via facial recognition, for a kind of match with characters from the game of thrones, through "image recognition".
Then, through a process of Morphing a short video with the transformation of the user's face is obtained, which is sent by e-mail.
Another application of Augmented Reality technology was used in a system created for Fidelidade Seguros that gives the user the possibility to paint the face, with digital masks, in real time, without the inconvenience of paints.
The M5SAR integrated solution was present at the Museum of Faro as proof of concept and demonstration to the public. Currently, the full experience can be performed with SPIC/UAlg .
Comments