To show the possibilities and flexibility of Crunchfish’s gesture control product – XR Skeleton – the company today publishes a video of a prototype in AR, in which interaction with virtual objects and buttons is shown. As previously announced, the company’s gesture control product XR Skeleton has impressive performance and advanced features, which creates the conditions for high-precision interaction with virtual objects in Augmented Reality (AR) as well as Virtual Reality (VR) environments.

XR Skeleton – Interaction with virtual objects

By identifying all the moving parts and contours of the hand and then linking these points together into a skeletal structure, the possibility of high-precision interaction with virtual objects is opened up. In the interaction video which is now available, examples of how you can use the skeleton model to pinch objects, change size and press buttons are shown. An important detail that affects the user experience in a positive way is the flexibility of the gestures. It does not matter if you press the buttons with one finger, two fingers or the whole hand – it works just as well, which shows how intuitive the interaction is. This may seem like an obvious feature, but it is extremely difficult to solve technically without Crunchfish’s skeleton model.

An important difference between AR and VR is that in AR, you always see reality and the user’s real hands. This entails the complexity that the system must keep track of exactly where the hands are in relation to all augmented information, such as the planets in the video above. From the user’s point of view, the experience must be that the objects as well as the hands seem real, and in that case the user must be able to touch and move the objects as if they were real. Enabling this intuitive gesture control is one of Crunchfish XR Skeleton’s absolute strengths.

“We are proud to show our XR Skeleton solution in an AR environment, where the precision and compliance of the interaction is clear. We constantly enhance the solution technically and there is no doubt that we are among the best in the world in the field of gesture control”, says Daniel Milesson, Head of R&D at Crunchfish Gesture Interaction AB

For more information, please contact:
Joakim Nydemark, CEO of Crunchfish Gesture Interaction AB
+46 706 35 16 09
joakim.nydemark@crunchfish.com 

Ulf Rogius Svensson, IR & Marketing Manager
+46 733 26 81 05
ulf.rogius.svensson@crunchfish.com

Västra Hamnen Corporate Finance AB is the Certified Adviser. Email: ca@vhcorp.se. Telephone +46 40 200 250.

About Crunchfish – crunchfish.com/gestures
Crunchfish is a tech company with a patent-pending solution for digital offline payments that can be integrated both with the payment rail or in a mobile wallet. The offline solution is globally scalable and makes digital payments more robust as the risks of disruptions and downtime are eliminated. We have also developed Blippit, an app terminal that connects to a cash register system for both online and offline payments. Crunchfish also develops gesture control of smart AR glasses for the consumer market. Crunchfish has been listed on Nasdaq First North Growth Market since 2016 with headquarters in Malmö, Sweden and with representation in India.

Crunchfish lanserar idag en uppdatering av sin senaste geststyrningsprodukt XR Skeleton med ytterligare förbättrad precision, prestanda och nya features. I XR Skeleton 1.1 har Crunchfish lyckats pressa ihop fler funktioner i ett och samma neurala nätverk och därmed fördubblat produktens precision med en 30-procentig reduktion i processorkraft. XR Skeleton kan med en helt vanlig mobilkamera detektera och följa 21 punkter i handen i tre dimensioner.

Genom att identifiera handens alla rörliga delar och konturer och sedan sammanlänka dessa punkter till en skelettstruktur öppnas möjligheten att använda 3D-modeller som en exakt avbildning av handen och dess position i en förstärkt verklighet. Crunchfishs senaste mjukvara ger en full 3-dimensionell upplevelse trots att en vanlig 2-dimensionell mobilkamera används – se video.

Teknologin bakom mjukvaruprodukten XR Skeleton bygger på Artificiell Intelligens (AI) och användning av avancerade neurala nätverk. Detta ger systemet intelligens att räkna ut hur en hand rör sig i alla tre dimensionerna trots att kameran som används bara kan läsa av handen ”platt”, dvs i två dimensioner. Detta kräver normalt sett flera stora neurala nätverk, vilka är processor- och batterikrävande. I XR Skeleton 1.1 har Crunchfish lyckats pressa ihop fler funktioner i ett och samma neurala nät och därmed ökat prestandan med 30 procent. Samtidigt har precisionen i XR Skeleton 1.1 fördubblats, dvs hur nära var och en av de 21 punkterna på handen är den verkliga leden på handen.

En viktig skillnad mellan augmenterad verklighet (AR) och virtuell verkligen (VR) är att man i AR hela tiden ser verkligheten. Detta medför komplexiteten att all augmenterad information, som exempelvis robothanden i videon ovan, måste följa den verkliga handen med extrem precision för att ge användaren en bra upplevelse. Att åstadkomma detta är en av Crunchfishs absoluta styrkor.

“Sedan förra produktreleasen har vi tagit ytterligare kliv framåt i området att återskapa 3D information utifrån 2D bilder. Detta till stor del genom en avancerad process för framtagning av mycket högkvalitativ data i kombination med djup expertis inom optimering av neurala nätverk för realtidsapplikationer. Med den senaste versionen av XR Skeleton visar vi teamets otroliga förmåga att skapa extremt effektiva lösningar med små resurser”, säger Daniel Milesson, utvecklingschef på Crunchfish Gesture Interaction AB

För ytterligare information, vänligen kontakta:
Joakim Nydemark, VD Crunchfish Gesture Interaction AB
+46 706 35 16 09
joakim.nydemark@crunchfish.com

Ulf Rogius Svensson, IR & Marketing Manager
+46 733 26 81 05
ulf.rogius.svensson@crunchfish.com

Västra Hamnen Corporate Finance AB är Certified Adviser. Epost: ca@vhcorp.se. Telefon +46 40 200 250.

Ladda ner som PDF

Today, Crunchfish launches an update of its latest gesture control product XR Skeleton with further precision and performance improvements as well as features. In XR Skeleton 1.1, Crunchfish has managed to compress more functions in the same neural network and doubled the precision with a 30 percent reduction in processing power. With an ordinary mobile camera, XR Skeleton detects and follows the hand through 21 points in three dimensions.

By identifying all the moving parts and contours of the hand and then linking these points together into a skeletal structure, the possibility of using 3D models as an exact image of the hand and its position in an augmented reality is opened up. Crunchfish’s latest software provides a full 3-dimensional experience despite the use of a standard 2-dimensional mobile camera – watch video.

The technology behind the XR Skeleton software product is based on Artificial Intelligence (AI) and the use of advanced neural networks. This gives the system intelligence to calculate how a hand moves in all three dimensions, even though the camera used can only read the hand “flat”, i.e. in two dimensions. This normally requires several large neural networks, which is demanding both on the processor and the battery. In XR Skeleton 1.1, Crunchfish has managed to compress more functions in the same neural network and thus increased performance by 30 percent. The precision in XR Skeleton 1.1, i.e. how close each of the 21 points on the hand is to the actual joint on the hand, has also been doubled.

An important difference between augmented reality (AR) and virtual reality (VR) is that in AR, you always see reality. This brings the complexity that all augmented information, such as the robot hand in the video above, must follow the real hand with extreme precision to give the user a good experience. Achieving this is one of Crunchfish’s main strengths.

“Since the last product release, we have taken further steps forward in the area of recreating 3D information based on 2D images. This is largely done through an advanced process for producing very high-quality data in combination with deep expertise in optimizing neural networks for real-time applications. With the latest version of XR Skeleton, we show the team’s incredible ability to create extremely efficient solutions with small resources,” says Daniel Milesson, Head of R&D at Crunchfish Gesture Interaction AB.

For more information, please contact:
Joakim Nydemark, CEO Crunchfish Gesture Interaction AB
+46 706 35 16 09
joakim.nydemark@crunchfish.com

Ulf Rogius Svensson, IR & Marketing Manager
+46 733 26 81 05
ulf.rogius.svensson@crunchfish.com

Västra Hamnen Corporate Finance AB is the Certified Adviser. Email: ca@vhcorp.se. Telephone +46 40 200 250.

About Crunchfish – crunchfish.com/gestures
Crunchfish is a tech company with a patent-pending solution for digital offline payments that can be integrated both with the payment rail or in a mobile wallet. The offline solution is globally scalable and makes digital payments more robust as the risks of disruptions and downtime are eliminated. We have also developed Blippit, an app terminal that connects to a cash register system for both online and offline payments. Crunchfish also develops gesture control of smart AR glasses for the consumer market. Crunchfish has been listed on Nasdaq First North Growth Market since 2016 with headquarters in Malmö, Sweden and with representation in India.

Gestures are in focus in this fourth week in our interview season. is focusing on Joachim Samuelsson starts off by talking to our senior research engineer Anders Henrysson who has a PhD in computer vision-based interaction.

Anders and Joachim discuss the intricate process of teaching a computer to recognize a hand, the difference between algorithms and neural nets, how neural nets for hand recognition are able to train themselves from data – and last but not least mountaineering and skydiving that Anders does when he’s not doing his magic at Crunchfish.

You can take a look at this video of our XR Skeleton product using only the camera sensor in a mobile phone – https://lnkd.in/erz6gwm – a really cool project that Anders has been working on.

Interview 10 of 36 in the second season – Cash goes digital, a payment pandemic – out tomorrow morning 6 am CET / 10.30 am IST. https://lnkd.in/e5RHAYD

Joachim Samuelsson rounds off the Alumni week talking to Henrik Winberg, our Head of Global Sales for five years 2015 – January 2020, who currently serves as COO of Dlaboratory Sweden AB, an innovative company that provides power companies with an intelligent analytics platform to manage their grids.

Henrik and Joachim discuss the famous Ketchup effect and how technology should be adjusted to humans instead of the other way around – something that permeates everything we do at Crunchfish. Don’t miss this interesting discussion!
Please visit https://dlaboratory.com/ to find out more about how they’re unlocking valuable insights in power grids with their data-based analytics tool.

Interview 9 of 36 in the second season – Cash goes digital, a payment pandemic – out Friday morning 6 am CET / 10.30 am IST.