An edgy way to survey a city.
It’s March, 2017. A dark-yellow minivan, a Volkswagen Caravelle, is riding along the streets of the Russian city of Saratov; on its roof, a strange construction resembles an antenna. Protruding from it, a bundle of wires escapes through the half-open window into the car's interior. The body of the minivan has no inscriptions, no distinctive signs. At traffic lights, every now and again, drivers of other cars try to find out from the minivan passengers what they are doing and what they have attached to the roof. And whether or not it is dangerous.
Behind the wheel of the minivan is Sergey from Saratov. He had received a call from some Moscow-based researchers who were looking for a car they could use to create a mobile center for studying the urban environment, and he agreed to temporarily turn his Volkswagen into a massive detector-on-wheels. Besides him, four others sit inside the car: Maxim Vorotnikov, Dmitry Masaidov, Anton Kozhinskiy, and Timur Cherkasov, who had invited the first three to take part in the project. All of them stare into their computer screens, monitoring the readings from the navigation systems and sensors mounted on the roof of the minivan. Cherkasov, an employee of Strelka KB, commissioned the study and is the project manager. For help with the equipment design and data collection, he called some experts he knew and with whom he had worked before.
DYNAMIC ANALYSIS OF THE ENVIRONMENT
The Volkswagen drives through the streets while simultaneously collecting data on the city’s traffic congestion, environmental conditions, noise levels, and other parameters. The speed of the car ranges from 15 to 60 kilometers per hour. Each outing lasts about an hour and a half — according to their methodology, the team selects the time of peak activity on the roads in both the morning and evening. The minivan travels the same routes at night in order to measure the illumination levels of the streets.
The car itself didn’t need to be converted or updated. The researchers brought a bunch of instruments and fixtures from Moscow, which were assembled into a universal design for surveying a city. For example, they tested a simplified version of the antenna on an off-road vehicle in Moscow, making a few laps around the Izmailovo area. While such a design can be installed on any car, the main thing is that there is enough room inside for all the project’s participants and that it is spacious enough for all the equipment and bundles of wires. Initially, Timur Cherkasov’s team wanted to contact a rental-car company and get a private minibus during their project in Saratov. However, Strelka KB rejected this proposal. Then the researchers began advertising to local residents, seeking the service of drivers who have their own cars. That’s how they found Sergey. This approach allowed the team to save a significant amount of money and avoid any legal risks in the event of the car breaking down. According to Timur Cherkasov, Sergey’s van became a legend in Saratov during the project.
The whole construction mounted on the roof of the minivan was assembled by hand, with the project members independently connecting and welding together the individual parts and devices. Its foundation was an ordinary camera tripod extended by a half-meter carbon tube. At the very top is a GPS antenna. Fastened a little bit lower are two back-to-back hemispherical cameras, which together take 360-degree panoramas of the streets around the van. The rest of the units are located even lower — on a milled plywood board attached to the tripod. The devices here include a unit for processing the GPS signal from the antenna, a mini computer for data processing, and a unit which houses a device to gauge the noise, dust, humidity, light, temperature, and carbon monoxide in the city air. The information from these devices is transmitted to the computers in the car and is immediately plotted on the online map service Mapbox . Data processing takes place in real time, and, on top of that, is done autonomously — solely with the help of the equipment installed on the machine. Achieving this was one of the team’s main tasks. During the processing stage, the scan results were corrected, though the data was in fact plotted on the map instantaneously.
In order to correctly represent the data on the map, it was important to establish accurate geolocation. To correct the coordinates, the researchers climbed onto the roof of one of the multi-story buildings in the center of Saratov and installed a special sensor there. They were able to get there thanks to architect Daniel Yankylevich, who lives in the building. With this, a triangle was formed: the GPS block on the roof of the building ("base"), a GPS unit on the roof of the car (“rover”), and the satellite. Debugging this system took several days, but the result was that Timur Cherkasov, who had taken on this part of the work, managed to achieve a signal error of three centimeters or less. Thus, the data scans of the city were attached to a specific place, which the minibus passed at a certain time. In the final version of the map, the distance between each received data point was two meters.
Next to Timur in the minivan sits Dmitry, a programmer; the screen of his laptop changes with images from the hemispherical web cameras. While the minivan is driving through Saratov, the cameras take a photo every second. The lenses capture everything: cars, pedestrians, buildings, stop signs, and telephone poles. These photos are processed by a neural network, which knows how to recognize the given objects — in this case, people and cars. The title of each picture shows the time at which it was taken. These numbers are synchronized with the GPS coordinates to determine a specific point on the map. The program remembers the route of the car and saves an animated route of the car through the streets, similar to the simplified interface found in taxi applications.
As for the route, the navigator Anton is responsible. He sits in the front seat next to the driver and simultaneously looks at the road and the navigation app on the tablet screen. The study encompasses both the central districts of the city and its periphery. Specifically for the final circuits, Anton developed seven routes, each of which was designed to last for an hour and a half. The Volkswagen drove each of them three times, and the averages of the data were included in the final report. In total, taking into account the test runs, the car-scanner drove several hundred kilometers along the city streets. In the places where the car couldn't travel, the team members took a light version of their apparatus in a backpack. In these cases, the data processing was carried out after its collection.
While the members each had their own role in the car, the team turned out to be multi-functional, and worked together to solve tasks. "Together with Dmitry, we assembled the first prototype - the hardware and software, including the first working neural network, and then split up the duties and checked on each other. The equipment was assembled all together. Maxim designed the supporting structures, fasteners, commutation, debugging, and - together with Anton - made a laser cutting in a fab lab. I created the inner components of each box: I installed the cooling systems, insulation, and created the interfaces using Soviet toggle switches. Nevertheless, we could change roles at any time. I can’t help but mention the help of Vsevolod Okin, who coordinated with us from KB. In many ways, the project took place thanks to him," recalls Timur Cherkasov.
BLASTER ON THE ROOF
Stabilization for the apparatus on the roof and the functionality of all the signals was handled by the engineer, Maxim. Inside the minivan, he kept an eye on the measurements coming in from the sensors. The most important thing is that they remain up-to-date, which means that everything is working properly. Maxim also documented the project members’ every step and created content for the video blog.
After every trip around the city, the apparatus with all of its sensors was removed from the roof of the minivan and taken to a rented apartment where the entire team took up residence. During the initial test week, the system was constantly being optimized. Two more cameras were attached to the tripod. One broadcasted a wide view of the structure into the car and allowed them to see how much it moved due to shaking and the unevenness of the roads. Maxim attached the other one at the very top, next to the GPS-antenna, which acted as a lookout, and a special neural network using sound signals warned of branches and other obstacles.
Several times the entire design had to be completely reassembled: during the system’s testing phase, things would go out of order every now and then. Not long after the tests began, it was clear that the current Internet signal was not enough. The team then promptly acquired a 4G signal amplifier and installed it next to the other sensors. In addition to these complex optimizations, there were also simpler ones: indicator lights for all the sensors, toggle switches, and plastic netting that protected the sensors from strong gusts of wind.
Attracting too much attention from the townspeople was another problem for researchers. Often there would be so many curious people that their questions began hampering the quality of the work. According to the team members, the construction mounted on the roof of the Volkswagen looked like a blaster or some kind of explosive device. "We had the idea of disguising ourselves," Maxim says in one of his videoblogs. "We want to buy some uniforms - like orange construction vests with reflectors - and wear them in order to pass as civil servants. On top of that, I want to paste stickers that say ‘Digital Geodesy’ on the machine, so that we’ll have fewer questions."
The decals with the name of the service on the machine never happened, but buying reflective, lemon-colored vests, like road workers, was still necessary. Working in them made things much easier: the idea of going undercover to divert attention worked.
HUNDREDS OF KILOMETERS
In the final report on the dynamic analysis of the data collected by the car, the raw indicators of the instruments and sensors revealed some supplementary insights. For example, when the noise level peaked at the intersection of 2nd Stantsionnyy Driveway and Degtyarnaya St., it became clear that this was due to the nearby railway station, and when it exceeded 60 dB at the crossing of Traktornaya St. and Astrakhanskaya St., this was because of loud music coming from the cars of local residents. The low illumination on Sibirskaya St. and the area around the Saratov airport is a result of trees and billboards blocking the light from the streetlights there. Based on the evidence from the detectors, there are enough lights in the city, but because of obstacles, they do not always fulfill their function. Despite the fact that Saratov lies on the banks of the Volga, the city’s dust levels are high even in winter and spring. This figure is especially high on the narrow, winding streets in the city center - in some places, the dust level exceeds sanitary standards and could be a health hazard. The concentration of carbon monoxide in the city proved to be in the normal range, although it considerably exceeded the norm at some intersections. The final figures showed 13,300 cars in motion and 9,980 parked cars. At the same time, pedestrians recognized by the neural network totaled up to almost three times less - about 8,000 people.
In total, Timur Cherkasov's team spent about a month in Saratov. For two weeks, the researchers tested their scanner on wheels, and another five days were spent collecting data using the fine-tuned equipment. During this time, the team automated and stabilized the data processing during the trips, and by the end of the study the project members could monitor the processes one at a time.
When this became possible, the rest of the team were engaged in another job. In addition to the dynamic analysis, the team conducted a study on intersection traffic by using a quadcopter. The drone hovered above the intersections at an altitude of about 300 meters and filmed the movement of cars and pedestrians. The flights were managed by Anton and Maxim: they collected data at 34 intersections. In total, they had to perform about 100 drone flights. The neural network was also used to isolate the cars on the video recordings. The images they received from this were also required in order to create three-dimensional models. Thanks to this data, the team created models of the embankment, the station square, and two streets. The team gave all the results of their work to Strelka KB for a pilot course on urban development in Saratov.