Intelligent autonomy at the edge: Sensors making decisions on their own can make life smarter



Robert Ferguson's short drive to work every morning is senselessly delayed while he waits for a traffic light to turn green.

"It's red for me, and there's not a single person coming in any other direction," he says. "I have to calm myself before I get into my car, because I know it's going to happen."

We've all been there, but for the leader of the industrial radar group at our company, it's an especially aggravating experience. Robert knows that with a simple radar chip, that light would know to change colors for him. He can't wait for the day when our TI millimeter-wave (mmWave) sensors enable traffic signals everywhere to make informed, on-the-spot decisions.

The day isn't far off. Millimeter-wave technology is now available in production in mass quantities to bring advanced radar sensors to our industrial and automotive customers. We're the first to offer the world's most precise single-chip CMOS radar sensor. With a microcontroller, a radio-frequency (RF) front end, a hardware accelerator and a programmable digital signal processor (DSP), TI mmWave sensors calculate an object's range, velocity and angle at up to three times higher resolution than any other radar sensor on the market. Then it decides what action to take.

A simpler, single-chip design

Previous generations of radar sensors were bigger, far more complex designs. They featured several discrete components on a single printed circuit board connected by high-speed interfaces to a processor on a second board. Instead, TI mmWave sensors integrate analog and digital circuits onto one chip.

And unlike analog-only radar sensors, these sensors don't send data to a processor, server or the cloud to wait for instructions.

"All the intelligence is in the sensor," Robert said. "This is intelligent autonomy at the edge."

Precise and highly programmable, TI mmWave sensors have the potential to do far more good than to shorten Robert's commute. From making homes more comfortable to improving workplace efficiency to enhancing smart cities, they have far-reaching implications.

Automotive and industrial customers are already specifying and designing our mmWave sensors into their products. So are engineers in agriculture, drone development, healthcare, human-robot collaboration, security and more.

"The fact that our highly programmable sensor is integrated with analog and digital functions makes it easy to use for our customers," said Kishore Ramaiah, a leader in our automotive radar group.

  
See how TI mmWave sensors make life smarter

 Learn more about the world’s most precise millimeter wave sensor available on a single chip.

Ubiquity, not singularity

To be clear, "intelligent autonomy at the edge" is not code for artificial superintelligence. Whether a decision is made on the spot or in the network, "this is still a processing engine that's using hardware, software and logic to communicate decisions to a larger network," Robert said. "Our goal on the industrial side is to improve certain applications by adding sensing. In some cases, we want to augment other sensing technologies – optical solutions or LIDAR – where we believe we bring a value proposition to improve things."

To enable autonomous vehicles, Kishore said, TI mmWave sensors must work in concert with multiple sensing modalities, including cameras, LIDAR and ultrasonic sensors. "One sensing technology might have a drawback that the other sensing technology can help mitigate," he said.

Either way – alone or to augment other sensors – the benefits of real-time decision-making, a low power and a tiny footprint are likely to make TI mmWave sensor use widespread, if not ubiquitous.

We announced mass production of our ultra-wideband automotive and industrial TI mmWave sensors in May and will continue to enhance our product portfolio.

How mmWave sensors change everything

Why would a TI mmWave sensor improve Robert's morning commute better than, say, a camera or a simple motion sensor? First, radar is more robust. Cameras and other technologies can be hindered by environmental conditions. But even in total darkness, rain and extreme temperatures, radar can sense how far away cars are. And integrated digital processing lets the mmWave sensor make decisions.

"Radar can say, 'I detect cars 50 meters away this way and 75 meters away in that direction, so I need to turn that light green and stop all the others,'" Robert said. "Without that processing capability, it would have to send its observations to a control center that would relay back the instruction to change the light."

With that decision-making capability, a TI mmWave sensor makes the decision locally and then relays its choice to the network for tracking.

Consider several other potentially transformative motion-detection applications for mmWave sensors:

  • Eradicating false alarms from video doorbells. "All of us who have these systems also have about 30 seconds of wasted video on our cell phones from instances when the camera detected a tree swaying or sunlight moving," Robert said. But TI mmWave sensors can enable a doorbell to differentiate between a human, an animal and any other moving object before deciding whether to record.
  • Aiding first responders. In office or apartment building incidents, TI mmWave sensors can enable the detection of fine movement through walls, which could help emergency workers rescue people faster. Even unconscious people would be noticed by radar that can sense micrometer movements such as a person's chest expanding and contracting while breathing.
  • Optimizing indoor environments. mmWave sensors can enable smart building systems to autonomously adjust cooling, heating and lighting based on the numbers and flow of people in a room. Sure, cameras can see how many people are in a room, but TI mmWave sensors can assess populations and movements without invading privacy, regardless of darkness and despite doors and walls.
  • Monitoring patients and newborns without contact. Mounted on a ceiling, under a mattress or behind a wall, TI mmWave sensors can enable the monitoring of a patient's heart rate, breathing and other vital signs without touching them. When integrated into medical systems, highly sensitive groups such as infants and burn victims could be monitored while sparing them the additional pain of physical interaction or the impracticality of attaching probes and electrodes.


Automobile autonomy, in more ways than one

Inside and outside automobiles, the applications for TI mmWave sensors are also many and varied. While radar for cars has existed for some time, Kishore said one difference now is that TI mmWave sensors enable cascaded radar.

"We can connect multiple radar transceivers in a cascaded format that make it possible for automobiles to detect objects up to 350 or more meters away," he said. "We will also be able to achieve a level of accuracy of less than 1 degree, which provides LIDAR-like performance."

Kishore predicts that by 2025, millimeter-wave will be a key technology for front radar systems in autonomous vehicles. It can also be placed in multiple locations in and on a car. Among the benefits:

  • Children and pets can be detected in the back seat of a car and remind drivers of the presence of other passengers.
  • Drivers who doze off could be nudged awake by a vibrating seat or steering wheel. mmWave sensors could pick up sleepiness signals even when a driver is wearing sunglasses or when the sun is shining too brightly for a camera to work effectively.
  • Sensors that respond to breathing and heart-rate variability could help rescue suddenly ill drivers by helping the car navigate and alert the system to call emergency services.
  • Car door operating systems could prevent injuries to fingers, collisions with bypassing bicyclists and damage to other parked cars.
  • Because varying temperature pressures or same-frequency noises may cause ultrasonic parking assist sensors to fail, TI mmWave sensors can step in to aid automated parking. "The additional functionality of radar sensors will change the way parking applications are realized. This robustness to challenging environmental conditions is precisely the reason radars are needed for ADAS applications," Kishore said.

Our radar-assisted future

Drones, forklifts and robotic vacuums are just a few of hundreds of other types of equipment set to benefit immediately from intelligence at the edge, with TI mmWave sensors' ability to detect steep drop-offs, power lines and other obstacles. And Kishore says there are still a lot of applications to be explored for integrated mmWave sensors. In automobiles, he imagines mmWave sensor-enabled inter-vehicle communications and road hazard warnings.

As for future industrial uses, Robert envisions more automated warehouses, and, after that, radar-sensor-driven product delivery. He's particularly excited about the potential for intermingling a mmWave sensor’s unique capabilities to enable human-robot collaboration.

"Remember, radar gives you three pieces of data about an object that no other technology provides: range, speed and angle," he said. "So it can detect motion and recognize gestures at the same time."

In an automated factory setting, where alarms sound when humans come within a few meters of hazardous machines, mmWave sensors could sound them sooner if someone is approaching at higher speed. "It gives us the ability to create safety guards to reduce incidents," Robert said. Even automatic doors would be smarter, knowing to open when a person's body angle indicates that they want to exit and to stay closed for a passersby.

Ultimately, the ways TI mmWave sensors can make our world smarter are only as limited as developers' imaginations.

"We make the components that our customers use, and they come up with larger-than-life ideas that we'll see out in the real world," Robert said. "We're at the tip of the iceberg to see the proliferation of this type of technology, and we look forward to seeing what our customers do next."

Check out our white paper: mmWave: Enabling greater intelligent autonomy at the edge.