Design of Vehicle Big Data Analysis System Based on Structured Data

Abstract

In order to meet the needs of information construction of company and make full use of the increasingly large volume of traffic checkpoint data, a big data analysis system of vehicles based on structured data has been designed and implemented. The checkpoint data provided by the video structured platform was utilized by the system, which employed Kafka as the message bus, Elasticsearch as the distributed platform for storage and search, Redis as the distributed service for model and algorithm, and a warehouse for the real-time transmission, storage, retrieval, and analysis of traffic data. The system integrated multiple related data from this industry, providing more valuable clues. Following deployment and testing, the system was found to meet the business requirements in terms of functionality. In terms of performance, when the total data size reached one billion and the new data per minute reached 100,000, the operation time of the user interface (UI) response and algorithm module reached a second level, meeting the performance requirements. The system can provide the company with a range of practical application services, including accurate results, comprehensive analysis, and timely warnings.