eDatX Deep Data Aggregation
eDatX: Aggregation, Storage and Access to the Data
Manage Flow, Quantity and Formats
The eDatX data platform ingests and aggregates data from diverse input sources to provide a manageable, accessible body of data in the cloud from a network of edge devices in a connected car.
Data from the connected car need not be an all-or-nothing proposition. eDatX allows OTA-updatable rules for policy-based data retention and granularity adjustments to provide a comprehensive toolset to gather the right data, in the right amounts, from any number of devices in any number of vehicles.
eDatX seamlessly integrates with an eSync data pipeline, to manage the flow of data from fleets of deployed vehicles. eDatX offers a secure service-oriented architecture to provide access to the data from the many diverse edge devices in a connected car, to drive system efficiencies, learning cycles, and data monetization opportunities in automotive and related industrial settings.
Product Demonstration Video
Built on the eSync Data Pipeline
The eDatX data platform makes use of the capabilities and resources of the Excelfore eSync data pipeline or similar eSync Compliant bi-directional data pipelines. It may also be customized to other data pipeline architectures.
The in-vehicle eDatX Service is an embedded software component that is architected to serve as a micro-service to the eSync Client. The eDatX Service is available for multiple popular operating systems.
Applications and Use Cases
Prognostics
Predict remaining useful life of automotive components. Aggregate operational data from ECUs to enable analytics and fleet level learning. Reduce service costs and improve customer satisfaction.
Predictive Maintenance
Set triggers to alert the server when ECU, HPC or sensor data drifts outside of normal operating bounds, or when combinations of in-range co-variables occur. Initiate remedial actions before error codes indicate device failure.
Accelerating Development
Draw real time operational data from devices in diverse geographies. Access and investigate from remote engineering facilities. Drive updates and changes across distributed test fleets that can be implemented over-the-air in minutes.
Monitoring Usage
Integrate and access sensor and telematics data to reveal patters of use of features and behavioral correlations.
eDatX Data Aggregation
eDatX allows OTA updatable rules/policy based data retention and granularity adjustments to provide a comprehensive toolset to gather the right data, in the right amounts, from any number of vehicles.
eDatX seamlessly integrates with the eSync data pipeline, to manage the flow of data from all electronic devices across any number of vehicles. eDatX offers a secure service oriented architecture to provide access to the data from heterogeneous networks to drive system efficiencies, learning cycles, and data monetization opportunities in the automotive and related markets
Rolling Buffers
The in-vehicle eDatX Service manages rolling buffers for all data-generating devices. These buffers retain all data published through the in-vehicle network for a short period, typically up to one minute. Data is then normally reduced in granularity for transfer to the non-volatile data stores.
When triggered, the full rolling buffer contents can be immediately written to the non-volatile data store for preservation and publication to the server in the cloud. Trigger events are determined by policy, which can be updated through OTA. Typical trigger events include:
- ABS Activation / Airbag Deployment
- Command from Server UI
In-Vehicle Data Stores
The eDatX Service tags and organizes data by topics, and performs basic logical and arithmetic operations to reduce the granularity of data, as it is transferred from the rolling buffers to the data stores. These operations are programmed into the Granularity Reduction Engine and can be modified through OTA. Typical operations include:
- Simple decimation (sampling reduction)
- Rolling average values
- Highest/Lowest values over a data set
The data stores include overwrite protected memory for critical data.
Data Transmission
The eDatX Service in the vehicle relies upon an external data pipeline to establish and transport data in a secure manner to a server in the cloud.
eDatX can be programmed to send data based on:
- Continuous Streaming via MQTT
- Periodic / On-Command Bulk Upload via HTTPS
eDatX Data Ingestion
- Rules-Based Configuration and Data Trimming
- Policy-Based and OTA-Configurable Data Gathering
- Streaming Data Preprocessing and Transformation
- Data Published to Queue
- Data Stored in Topics for a Configurable Retention Period
Data Lake in the Cloud
The eDatX Platform uses multiple data storage formats for data retention in the cloud:
- Hadoop Distributed File System: to Store Raw and Staging Data
- Mongo DB: to Store Diagnostics Data
- Elasticsearch: to Store Structured Data Processed by Aggregation and Learning Services
- Relational Database: to Store User, Role and Configuration Data
Integrates with eSync Data Pipeline
eDatX adds capabilities to an established data pipeline. It relies upon a pipeline for essential features, and provides useful incremental capabilities for local and cloud management of data generated by the sensors and ECUs in a connected car.
The eDatX Platform can be seamlessly integrated with the eSync Data Pipeline. In such an application, the eSync Pipeline provides:
- Secure TLS 1.2 Communication Channel
- Data Transmission over MQTT
- Up to 128KB Packet Size
- Device Authentication
- Scalable In-Vehicle Architecture with eSync Agents to gather raw data from the various ECUs and Sensors
What’s It Like at Excelfore?
It’s all about technical excellence. Software, hardware, electro-mechanical interfaces, human interfaces, scalability, connectivity, security and reliability … at Excelfore you touch the whole project, not just a little piece. It’s all hands-on. Everyone is a contributor. We keep our organization flat, we work directly with our customers, we drive ourselves, we support each other, we innovate, we solve problems.