IBM Research and Partners Help Traffic Run Smoothly in Madrid
Nearly every city plan takes traffic into account to help commuters. But people still face congestion on the road, whether they drive or take public transportation. IBM researchers, such as Paula Ta-Shma, research staff member, IBM Research–Haifa, are working to rectify that, using machine learning and a host of open-source tools.
“IoT is leading the way to new types of data being collected in unprecedented quantities.”
—Paula Ta-Shma, research staff member, IBM Research–Haifa
Ta-Shma is collaborating with COSMOS to use both historical and real-time data to improve traffic monitoring to optimize the use of roadways and vehicles in the city of Madrid.
COSMOS is a European Union-funded research project that comprises use-case partners (including the Madrid Council and the EMT Madrid bus company) and technology partners (including IBM, Atos and the University of Surrey), and aims to have sensors in the Internet of Things (IoT) interact with one another socially—the way people do on social networks.
According to Ta-Shma, that research could have implications well beyond transportation.
Q: How are you using machine learning in the Madrid traffic project?
Paula Ta-Shma (PTS): What we’re doing is collecting traffic data [regarding speed and intensity]. In the case of Madrid traffic, it’s open data published by the Madrid Council. Around 3,000 sensors record traffic in various fixed locations throughout Madrid, and we continuously collect this data and store it long term. At the same time, we have continuous access to the real-time feed of data. What’s really important is making use of both the historical and real-time data. In order to react intelligently in real time, we use machine learning on the historical data to understand the expected traffic behavior for each city location and time period. That way, we know how to react based on context, such as location in the city (busy city intersection or suburban outskirts) and whether it’s a rush-hour period or not.
Q: How will this help the Madrid Council?
PTS: We’re helping them become more efficient in the control rooms where they currently employ a lot of people looking at traffic screens and manually managing traffic. We cluster the traffic into so-called good and bad traffic. This way, when traffic in a certain location moves from good to bad, we can raise an alert that might trigger notification of the Madrid bus company, alert passengers on highways via information panels or call for emergency vehicles.
Our work can help those in the control rooms react faster. The possibility of automating the response with little or no human intervention also exists. For example, one could imagine an automatic system for adjusting traffic-light behavior according to the current traffic conditions.
We have new speed and intensity readings [showing the amount of vehicles] coming in around every five minutes. If we’re looking just at the city of Madrid, the rate of data coming in is moderate. But you can imagine this extended to many other cities or areas around the world.
It’s important that the machine learning for different locations and time periods is done ahead of time. When new data comes in, we’ve already calculated thresholds that tell us if we’re switching from good traffic to bad traffic. And as that comes in, the machine-learning models may need to change over time, so we have a technique that tells us when our clustering gets out of whack and we need to run it again.
We’re also using a dashboard that monitors traffic at different locations in Madrid with the capability to display alerts in locations where the speed or the intensity of traffic goes beyond a certain threshold. That threshold is calculated using machine learning. It’s different for each location and for rush-hour periods versus nighttime periods, and so on.
Q: How do OpenStack Swift and Apache Spark come into play?
PTS: OpenStack Swift is a scalable, low-cost way to store data. This is crucial for IoT data, which is quickly becoming the “biggest” big data ever. That’s because so many things are connecting to the internet and generating data. We’re also making it more efficient to analyze data on OpenStack Swift for our machine-learning computations.
Apache Spark is a cluster-computing framework that enables analytics to be done on the data. So, OpenStack Swift provides the storage, and Apache Spark provides the compute power for the analytics. It’s doing the machine learning and computation on the data.
Q: So, you combined Swift and Spark to work almost as a single entity?
PTS: We’re also using other open-source tools we helped play nicely together. Apache Kafka and Pinterest Secor are used as a message hub and a way to get the data into Swift. Apache Parquet, a special-data format, is good for analytics in general and for IoT data in particular. We use Elasticsearch to make the analytics more efficient, and IBM Node-RED hooks together systems and sensors to improve the management of the IoT.
It’s a complex challenge. Clients don’t just want to run the solution on their laptops. They want it to be deployed in the cloud and scalable so they can start with a small amount of data or number of users and grow elastically. One of the exciting outcomes of our work is our collaboration with another IBM team, led by Naeem Altaf, to port the whole Madrid traffic use case to run on the IBM Bluemix* platform.
Q: How can this technology be used?
PTS: In COSMOS, for example, we did some work on occupancy detection and the ability to save energy by turning appliances off when people have left their home or office. You can potentially do similar work with other types of sensors collecting data.
For example, by putting sensors along pipes and performing anomaly detection, you can detect and send a warning if a pipe leaks. You can even build actuators into your solutions so that when a pipe is leaking, the mains automatically shut off to prevent further damage to your home. If a company has an oil pipeline traveling across the country, you could use similar technology to help avoid an environmental disaster.
Q: Where do you see the IoT heading in the future?
PTS: IoT is leading the way to new types of data being collected in unprecedented quantities. For example, in healthcare, you may use this technology to get more insight into the reasons behind illnesses.
For traffic, we could gain new insight into the conditions and locations under which accidents occur. Having this additional data can provide new opportunities and approaches we may never have considered before.