Video: Aircraft Carrier Crews Guide In Robot Planes With Visible Hand Gestures | Popular Science

Landing airplanes on moving ships is no mean feat, but this will be especially true when the airplanes are unmanned. Along with making decisions, autonomous airplanes will have to heed their human counterparts during aircraft carrier takeoff and landing — but can a robot read and understand arm-waving signals?

The problem is complicated in at least two ways — first, the airplane must determine whether the human’s hands are up or down, elbows in our out. Second, it has to red which gesture the human is making, and what it means. MIT PhD student Yale Song is trying to solve these problems.

Song and fellow scientists recorded various people performing a set of 24 gestures aircraft carrier deck personnel use, including arm waving and folding, and hand movements. They built software that determined each person’s elbow, wrist and hand positions, including whether palms were open and whether thumbs were up or down. They completed that portion of research last year. Then, the team had to classify all these gestures according to their meanings. But this is complicated, because deck signals are a complex ballet of movement — it’s not like a seaman makes one motion and then stops for a beat before starting another. So the algorithm has to determine a gesture’s meaning without a clear beginning, middle and end.

via Video: Aircraft Carrier Crews Guide In Robot Planes With Visible Hand Gestures | Popular Science.

You’ll have to click through on your own to watch the video, as I’m too lazy to embed it.

In many ways, moving a UAS on the flight deck is a much greater challenge than getting it to land or launch. Launch is easy. Once the catapult fires, whatever is hooked up is going flying. It’s just a question of how far. As for landing, the Navy has spent decades building in an Automatic Carrier Landing System for manned aircraft. Adapting it to a UAS sounds pretty straightforward.

But moving aircraft in the dynamic environment “on the roof” is challenging, and there’s probably not a single carrier aviator out there that doesn’t have a tale about being scared out of his wits just taxiing around.

So, the first time a yellow-shirt directs a UAS over the side, will the accident board take his word or the UAS’s word over what happened?