Big Data Consulting for an Internet of Vehicles Company
Summary
An Internet of Vehicles company that collects, stores, and processes IoT data from 600,000 connected vehicles wanted to enhance its big data analytics capabilities beyond what its key-value Apache Cassandra storage could efficiently support. In a three-day on-site engagement, INNERLUXES audited the existing solution, designed a data-lake-based target architecture, compared on-premises and cloud options, and provided Apache Cassandra performance recommendations.
About the Client
The Client is a large Internet of Vehicles company in the EU that delivers fleet and asset management solutions. It collects, stores, and processes IoT data from 600,000 vehicles connected to its systems to enable the users of its solutions to make informed decisions.
The Challenge
Having a robust IoT data collection and storage solution, the Client was looking to enhance its big data analytics capabilities. The existing solution is based on Apache Cassandra; being a key-value store, Cassandra doesn't support efficient big data analytics well enough. For example, to retrieve data for 70,000 cars, the Client currently needs to create 70,000 individual requests that return 70,000 individual reports, which requires extra effort to get a complete picture.
The Solution
INNERLUXES's big data consultants made a three-day on-site visit, where they carefully examined the existing solution — its architecture, relevant documentation, available data sources, and the data management practices currently in use.
Next, the team held a workshop dedicated to the solution-to-be, discussing the expected launch deadline, the existing and preferred technologies, and available licenses. During the workshop, INNERLUXES laid down architecturally significant requirements concerning the solution's availability, performance, security, and scalability, then prioritized each requirement by its impact on the Client's business as critical, high, medium, or low.
Based on the workshop's findings, the consultants designed the architectural concept of the solution-to-be, defining high-level architecture components — for example, integration services and a data lake — and describing their functions.
After presenting the concept, INNERLUXES organized a Q&A session and provided comprehensive answers on any aspect the Client was interested in. For instance, the team explained the advantages and disadvantages of a data lake compared to a data hub, and compared on-premises implementation against cloud options, including different cloud scenarios with Amazon Web Services and Microsoft Azure at the core.
In addition, INNERLUXES's Apache Cassandra consultants provided recommendations on improving Cassandra's performance, covering aspects such as the structure of tables, partition keys, and the format of data to be stored.
The Results
The Client received a visit report that contained:
- Architecturally significant requirements for the solution-to-be and their impact on the business.
- A high-level design of the key architecture components with their functions described.
The Client also received comprehensive answers to its questions concerning the future solution's implementation, the pros and cons of different technologies and architecture components, the nuances of data storage and integration, and other important issues.
Methodologies
Business requirements analysis, functional decomposition, workshop on architecturally significant requirements, Q&A session, comparative analysis of technologies.