03 9008 5950

An app that sees for the blind, shark-detecting drones, and more advances in computer vision

Last year the Waypoint team was fortunate to be in attendance at TEDX Melbourne and witness the enviously talented Marita Cheng take the stage with her business partner Alberto Rizzoli. Marita inspired the crowd with the story of her blind friend and her desire to make his life easier. With the assistance of her business partner, Marita developed an app called Aipoly that can translate what the user’s phone is seeing into spoken words.

The app was developed using Convolutional Neuro Networking, a technology trained on thousands of everyday objects. Used in Marita’s app, it enables blind people to recognise their surroundings. “It learns the features of a dog,” explains Marita, “it learns what a flower looks like, it learns a knife, a fork. Everyday objects.”

In the age of roombas and self-driving cars this premise seems rather simple, but step back 5-10 years and this idea would have been all but impossible. Computer vision is revolutionising huge areas of our tech industry, from improving quality of life for those with poor vision, to increasing safety on our roads with vehicle automation, and having deliveries literally land at your front door.

Computer vision has its humble origins on the factory line, reading those checked and coloured diagrams on your cereal box to ensure everything is aligned and in order. Today, it’s one of the linch pins of our automated future. Computer vision is essentially a machine’s ability to look at a collection of data that makes up a photograph, and distinguish shapes, colours and patterns to tell me what the image’s contents are.

Face detection is a great example of the lengths we have come with computer vision. First made popular as a simple way of getting your subjects in focus when taking photos on your new digital camera, this has evolved to automatically detect a smile and take the photo on your behalf. In those early days, your digital camera was simply looking for a set of expected shadows around the eyes, mouth and chin to determine a face. Bring that forward to today and we have Snapchat tracking your face in 3D space to cover you in dog ears and flower petals, and Facebook looking at facial patterns to the extent it can suggest which friends are in your picture.

Computer vision has evolved to the point where it can process information and scenarios at a rate and accuracy that eclipses our own eye-to-brain connection. This can be found in the exciting frontier of self driving automation. When last discussed, the self-driving Google car calculated 700Mb of data per second. That’s the equivalent of a human listening to a whole album in one second, and being able to rewrite every single note, flourish and detail in absolute accuracy.

On the extent of data calculation found in Google car technology, the founder and CEO of Idealab put it eloquently:

“It is capturing every single thing that it sees moving — cars, trucks, birds, rolling balls, dropped cigarette butts, and fusing all that together to make its decisions while driving. If it sees a cigarette butt, it knows a person might be creeping out from between cars. If it sees a rolling ball it knows a child might run out from a driveway. I am truly stunned by how impressive an achievement this is. I believe that this is an UNDER-hyped revolution in the making.”

Bill Gross, founder and CEO of Idealab.

This technology is unprecedented on our roads and promises a future where independent transit is not dictated by the strength of our eyesight and motor reflexes, whilst providing a level of road safety that will make road fatalities the rarity we’ve always wished them to be.

Drones are another example where computer vision is moving fast. In NSW last year, the trial of a drone that detects sharks was launched in the region. With a high-resolution camera and a surveillance algorithm, the drone can spot potentially troublesome activity in shark-prone waters.

During the Super Bowl halftime show last month, 300 drones danced behind Lady Gaga; flying, falling and flocking in unison under the control of one person and a computer.  

Computer vision is still in its early days, but fast forward a decade and it’s easy to see that current innovations have a bright future, both in business and society.

I encourage you to spare 8 minutes to watch Marita Cheng’s TEDX speech and also check out her app – her use of computer vision technology is truly inspiring.

I’m excited to see what the future will hold.