Robert Dahlstrom and his wife own a beach house in Florida, and in 2013, it needed painting. His Internet security company, BluePrint Data in Jacksonville, Fla., was doing well, and he had extra time, so he decided to paint the house himself.

“I painted houses working through college, so I just put up scaffolding and climbed up to paint my own house,” he said. “With a tired arm and a tired neck, I quickly realized I wasn’t in college anymore, and I said to myself, ‘There must be a better way.’”

Now Dahlstrom and others are developing aerial drones and ground-based robots to paint buildings, towers, bridges, ships, dams and other structures not only more efficiently but more safely. In the United States alone, 95 climbers working on cell and other industrial towers died between 2004 and 2012, according to a 2015 Federal Aviation Administration report. Climbers working on towers have a fatality rate that is 10 times that of construction workers, Dahlstrom noted. Moreover, according to the Occupational Safety and Health Administration, the most dangerous thing a worker can do is climb a ladder. In 2014, falls accounted for roughly 40 percent of all the deaths in construction, the most dangerous line of work in the U.S., according to the Department of Labor.

“It’s 2016—why are people putting themselves in danger hanging off buildings and climbing up scaffolds and exposing themselves to inhaling toxic chemicals when we have robotic systems that can do this?” Dahlstrom said.

Colorful New Market

Apellix, the new company that Dahlstrom formed in 2014 to provide painting services with drones, currently focuses on the oil and gas, shipping and infrastructure industries—each of which has a painting market segment worth more than $2 billion. They are part of a global market for paint and coatings valued at $127.3 billion in 2013, with $25.4 billion of that in the U.S. alone, according to a 2014 report from the World Paint & Coatings Industry Association.

Drones have the potential to capture a significant portion of those markets through more cost-effective service.

For instance, when it comes to shipping, a standard cruise or cargo ship has an underwater hull area 60,000 to 80,000 square feet in size that requires more than $100,000 worth of paint. This takes a work crew of about 30 people four days to set up scaffolding, paint, and remove the scaffolding. Every day a ship spends in dry dock being painted can cost more than $1 million in lost revenue.

“If we can have a robotic system do that in one day of dry dock, that’s three days less lost revenue,” he said.

The Worker Bee

Apellix’s Worker Bee drone is a commercial off-the-shelf Turbo Ace Matrix quadcopter connected to a mobile base station via an umbilical cord. The drone has a spray wand to paint walls, while the base station houses the paint reservoir, air compressor and power source. It can reach up to three stories high.

“By taking the battery off, we can take off a couple of extra pounds,” Dahlstrom said.

When he first came up with the idea for a painting drone, “I thought, ‘drones can only fly for about 20 minutes, and paint is heavy, meaning they can only fly with it for about five or 10 minutes. How am I going to get around that?’” Dahlstrom said. “The eureka moment was using an umbilical cord. You can keep the paint and power on the ground, and pump everything on the ground, and the sensors and software would control everything. I did a proof of concept, and have five patents pending now.”

Apellix uses a 3,000-psi Titan Industrial air compressor. “We’re also experimenting with a 7,300-psi compressor for the heavier paint used on industrial ships,” Dahlstrom said. “In terms of paint, it’s just ugly grey and green and black, ‘oops’ paint that was mixed incorrectly or that a homeowner got wrong from Sherwin-Williams within walking distance from my house.”

The drone uses LiDAR for 3-D scanning, optical and ultrasonic sensors for sub-millimeter depth perception to determine distances to walls, and temperature and humidity sensors that can help refine its painting strategies. It also uses short-wave infrared (SWIR) sensors to look for invisible details such as structural weaknesses, which could help paint manufacturers avoid lawsuits.

“For the most part, when it comes to all the paint that’s sold, paint manufacturers don’t know how it’s applied,” Dahlstrom said. “There are really large lawsuits out there, over paint or coatings peeling off and not lasting as long as they should, that blame the paint manufacturers—while the paint manufacturers say that it’s not the paint, but how it’s applied. A robotic system can measure all these environmental variables involved, such as humidity, temperature of the air, temperature of the surface, and add high-resolution video or snapshots to help address these issues. Paint companies might even take a product offering that normally has a five-year warranty and say that if it’s applied properly within specifications, it has a 10-year warranty.”

Right now the Apellix system can paint a little faster than a person. “However, the drone can paint continuously, whereas people have to stop and move themselves to spray again,” Dahlstrom said. “By not having to put up a scaffold and reposition itself, our estimates suggest that it can paint eight to 12 times faster than a person.”

“There’s a huge element of safety with drone painting, and there can be substantial labor savings, and higher consistency and quality,” John Salvatore, CEO of the Carlstar Group and former CEO of BASF Construction Chemicals Americas, said of Apellix. “Put that all together and it’s a hell of a combination.”

Making It Work

Flying while administering paint presents unusual technical challenges. To make sure propeller wash does not affect the paint spray, Apellix tinkered with the size of the drone’s spray wand and propeller blades, as well as the distances it should keep from walls, Dahlstrom added. To avoid problems caused by the paint spraying onto the drone and its propellers, Apellix coats the Worker Bees with Gentoo, a clear super hydrophobic nanoparticle material from business partner UltraTech International in Jacksonville, Fla.

“If paint does get on, you can brush it off with a damp cloth or sponge,” Dahlstrom said.

The fine control of the Worker Bee’s distance from the surface to be painted is also important to success.

“If a spray nozzle is too close to a surface, the paint bounces off, but if it’s too far, it atomizes and not enough catches,” Dahlstrom said. The exact desired distance can vary “depending on paint viscosity, the nature of the surface, humidity, or temperature—the sunny side versus the shady side of a house. Good intuitive painters know this intuitively from doing it years and years, but even then, you can’t expect people to keep spray wands exactly the same distance from a wall the entire time they are painting. Drones can keep the proper distance from the wall and prevent overspray or underspray. We’re replacing human judgment with science.”

One major challenge Apellix faced is how to enable its drones to fly very close to surfaces, and sometimes even touch them.

“The drone sometimes has to have a probe touch the wall to, say, measure the thickness of a deposited film,” Dahlstrom said. The problem there was that when their drone encountered resistance from a wall, it perceived it as resistance from wind and thought it was getting blown off position. To compensate, it accelerated into the “wind.”

“We had a lot of drone crashes,” Dahlstrom said. “But crashing isn’t that bad. We tell our developers it’s good—if they’re not crashing, they’re not iterating fast enough.”

To prevent these crashes, “we reprogrammed the autopilot, and added sensors,” Dahlstrom said. “We have two sensors that give the exact distance to the wall. If one is slightly off, it corrects the angle, so a probe can hit the wall perfectly straight on—if the drone touches the wall at an angle, it can do things you don’t want, like crashing.”

Adding the umbilical cord was also a challenge.

“Drones were never intended to have a material-transfer system put on them,” Dahlstrom said. “The umbilical cord not only drags on the drone as it moves along, but you have this occasionally semi-rigid cord with high-pressure material within it that can actually push the drone around, so we have software to mitigate that from happening.”

Adding an umbilical cord did help solve a different design problem, however—keeping the wind from pushing the drone around.

“We have noticed adding a tether to the middle of the drone helps it hold position well, just as adding a tail to a kite helps it fly much better,” Dahlstrom said.

Software and data from sensors such as accelerometers, barometers, gyroscopes and GPS also help the drone compensate for how wind and spraying can push it around. Dahlstrom said his previous experience in the software industry is key to his drone work now.

“I’ve often reflected on why nobody else tried doing something similar to this beforehand,” Dahlstrom said. He recalled that at an event with other drone companies, others would say “that drones can’t do that. Because my background is in software, I see drones as flying computers, and see them as able to do things that maybe other people don’t see.”

“For instance, some hobbyist quadcopters have a button where you can push a button and it will execute a flip,” he said. “To do that manually takes hundreds of hours of flight time to do skillfully. Once you put it into code, it’s a no-brainer—it can do it consistently and repeatably and accurately.”

“We’re doing things drones were never intended to do,” Dahlstrom said. “They were intended to fly far and fast and not have anything attached to them. Ours are flying close to a surface, and slow, and tethered.”

Autonomous Future

The Apellix system is only semi-autonomous—it can’t do 100 percent of the painting. Instead, pilots remote control the drone, which is throttled to a maximum speed of 3 mph. “You press a button and the software moves it to the optimal distance from the surface, and you move the joystick to paint,” Dahlstrom said.

Still, “this computer-assisted flight can achieve about 80 percent of the benefits of a fully autonomous system and get to the market much quicker and with less expense,” Dahlstrom said. “Even if it was fully autonomous, it’d still need a person there to monitor the system and take over should something go wrong. But we do aim for it to be fully autonomous in the future.”

Although Apellix’s cautious pace of development “might be slower than I think anybody would like, I think getting it right is really important,” Salvatore said. “Getting a consistent spray is important, and a stable drone is really, really important.”

A large part of painting is not simply the act of applying paint, but also preparing surfaces beforehand.

“The prep work still needs to be done by people,” Dahlstrom said. “As the technology improves, maybe some of that prep work can be done by drones as well, such as washing or sandblasting. Right now if the drone is painting a surface and it comes to an area that needs prep, it will skip it in continue spraying. Once the job is mostly complete, then you can send people in, do prep work, and send the drone back to touch up those areas.”

Apellix has nondisclosure agreements signed with some of the world’s largest paint and spray equipment manufacturers, Dahlstrom said. Apellix hopes to have a commercially available spray-painting drone the first half of 2017.

“We get contacted quite often with potential ideas for the drone, anything from painting power transmission towers to cell phone towers and even wind turbines,” Dahlstrom said. “There are about 1 million power transmission towers. Most of them need coatings, and there’s a huge backlog of deferred maintenance. Robotic systems could help catch up on this dangerous work.”

There are many other potential applications for Apellix besides painting.

“We can use compressed air to clean solar panels, or imagine window-washing skyscrapers,” Dahlstrom said. In addition, “Italian energy companies are now contemplating putting a superhydrophobic coating on the blades of wind turbines to keep them clean longer and reduce wind resistance, but it’s a nanoparticle, and to protect yourself, you need a full suit and air hose. Now imagine wearing that while rappelling down from the center of a wind turbine with a spray gun. Instead, fly in a drone.”

Painter Bots

In addition to aerial painting devices, researchers are developing ground-based painting robots. A machine known as PictoBot from Singapore is aimed at spray-painting the interiors of industrial buildings there, painting walls up to three stories high with its robotic arm.

Industrial buildings often possess high ceilings to accommodate bulky industrial equipment and materials. Painting their interiors requires at least two people—one in a safety harness and one operating the lift.

As with Apellix, one of the main benefits of the PictoBot is safety, its developers said. In tests, the 1.5 ton robot also “takes about 25 percent less time than humans at painting, and the painting is very precise,” said I-Ming Chen, director of Nanyang Technological University’s Robotics Research Center in Singapore. It only requires one human supervisor, “and one supervisor can supervise more than one machine,” Chen said. “It can also run without light for 24-hour operation.”

PictoBot consists of a mobile base that has a robot arm mounted on an elevatable platform. At the end of the arm is a spray nozzle, an optical camera that scans the workspace to calculate the trajectory of the paint, and laser and ultrasound sensors for navigation, range-finding and obstacle avoidance. The robot and its 2,000-psi air compressor can work four hours on one charge of its rechargeable lead-acid battery, and its paint tanks can hold 120 liters. It can navigate on the ground and paint autonomously without human intervention.

“There are huge advantages to being on the ground—you don’t have to worry about gravity,” Dahlstrom said. “However, if you’re on the ground, you’re also restricted as to how high you can go.”

PictoBot was developed in a year by scientists from Nanyang Technological University in cooperation with JTC Corporation, Singapore’s principal developer of infrastructure, as well as Singaporean startup Aitech Robotics and Automation.

“Singapore is very small, and we have a shortage of skilled workers, so this was a project to see what kind of innovative applications we can use given our workforce constraints,” Chen said.

One challenge Chen and his colleagues faced while working on PictoBot is how it can sometimes vibrate, affecting the robot arm’s performance. “Some of that vibration we could not eliminate mechanically,” Chen said. “We factored it in as an external disturbance into the trajectory-planning algorithm.”

Chen noted this robot could do more than painting. “This is a platform technology that you can mount something else on for other purposes—for instance, you can paint a very thin layer of cement to make a surface very smooth, so this machine could do surface preparation as well as painting, or spray water to do cleaning,” Chen said. “We also have another robot to do quality assessments of finished buildings—checking defects in construction. And we are working on painting outer walls, although that research is ongoing.”

Drones for Art

In the future, robots may move beyond coating monotonous surfaces to painting more complex images.

“The challenge is that robots right now only paint one single color,” Chen said. “PictoBot uses a water-based paint, and if it finished with one color and wanted to paint with another, it’d need to flush the whole system. To make PictoBot paint two or many different colors at once, like an inkjet painter, you’d need a new painting head. I think that’s doable, but not at the moment.”

One researcher investigating the possibility that drones could be used to draw murals is Paul Kry, an associate professor at McGill University’s School of Computer Science in Montreal. He and his students are programming tiny quadrotor drones to create drawings dot by dot, an artistic technique known as stippling.

Retroflector markers on the palm-sized drones let a camera-equipped computer with motion-capture software pinpoint where each drone is in space. The computer steers the drones back and forth toward and away from a surface to draw.

The drones are equipped with a miniature arm that holds a bit of ink-soaked sponge. They collide with surfaces with their brushes, dabbing until they paint, say, portraits of Alan Turing, Grace Kelly, and Che Guevara, among others. Each drawing is composed of a few hundred to a few thousand black dots of varying sizes.

The researchers were able to get about 70 dots from each drone before they drained their batteries. The scientists also experimented with tethering the drones so they did not need to replace their batteries, and giving them inkwells the drones could get new ink from. Kry noted that even very slight air currents can knock the featherweight drones off course, so they want to scale up to larger drones.

One problem with drawing complex images with quadrotor drones “is that you have to tilt to fly left or right,” Kry said. “Placing a mark or drawing a line on a wall with precise control with only four motors is challenging. Ground-based robots have absolute control, and perhaps can point nozzles in more hard-to-reach places.”

Still, eventually drones could be deployed to paint artwork on hard-to-reach outdoor surfaces, including curved walls, Kry said. Current advertisements often involve placing giant vinyl stickers on walls which do not unroll well on irregular surfaces. “Drones may paint curved shapes, like the ceiling of the Sistine Chapel,” Kry said.

Dahlstrom and Chen both stressed that automation will not put human painters out of jobs.

“Mostly these robots do large areas—if you can do fine painting of every edge and corner, people have value,” Chen said. “Robots don’t really replace human jobs. They just make our jobs easier.”

“We’re moving people from dangerous situations,” Dahlstrom said. “I was talking with the CEO of a company in the oil and gas industry, and he said that in his 30 years in the industry, he had to tell four mothers that their sons are never coming home again. If we can prevent those conversations from happening, we should be doing it.”