Integrate with Feast
Feast (Feature Store) is a customizable operational data system, which uses your existing infratstructure to manage and serve machine learning features to real-time models. When integrated with Hazelcast, you can benefit from an online store that supports materializing feature values in a running Hazelcast cluster.
This approach unlocks the following to power your Feast real-world machine learning (ML) requirements:
-
One-to-one mapping of each feature view to a specific IMap data structure
Feast creates a new IMap for each feature, which means that every feature view corresponds to an IMap in the Hazelcast cluster, and the entries in that IMap correspond to features of entitites. Each feature value is stored separately, and can be retrieved individually.
-
Hazelcast’s inherent strengths, such as high availability, fault tolerance, and data distribution
-
Support for a secure TLS/SSL connection to your Hazelcast online store
-
The ability to set Time-to-Live (TTL) for features in your Hazelcast cluster
What is Feast?
Feast is an open-source feature store that can be used on your existing infrastructure to manage and serve ML features to real-time models.
A feature store is a central repository where features can be stored and processed for reuse or sharing. A feature store is used for ML pipelines in much the same way as data warehousing is used for analytics. A feature store supports the discovery, documentation, and reuse of features while ensuring their correctness. This means that you can use exactly the same features in your model training and production, which helps to prevent skewed model predictions.
Feast can help to decouple ML from your data infrastructure. This can be useful in a number of ways; for example, to allow you to move between model types, such as training models to serving models, or from one data infrastructure system to another.
When creating a Feast project, you can define one or more feature views. A feature view is an object that represents a logical group of time-series feature data from the provided raw underlying data, which is known as the data source. If related to a specific object, the feature view has one or more entities.
An entity is a collection of semantically related features. Each entity in a feature store consists of the entity name and an associated value, known as a join key. The name uniquely identifies the entity, and the join key identifies the physical primary key on which feature values are joined during retrieval.
With Feast, you use the following in your ML models:
-
Historical data to support predictions that allow scaling to improve model performance
-
Real-time data to support data-driven insights
-
Pre-computed features that can be served online
For further information on Feast, refer to the Feast documentation.
Why Integrate with Hazelcast?
Feast supports the following:
-
Batch feature creation. The data in the batch - or offline - store can be transformed, for example, using SQL
-
Stream feature creation. Streaming services, such as Kafka, supply the data from which stream features are created
-
Load data from the offline store to the online store through materialization
Integrating Feast with Hazelcast allows you to do the following:
-
Use a Hazelcast cluster for your online data store. An IMap in the cluster contains the feature values, identified by the project and feature view names. You can push data to, and retrieve data from, the IMap to provide a real-time online store
-
Connect a Feast feature server to stream data. This avoids the need for a temporary IMap sink, provides a real-time solution, and ensures that your features are always fresh.
See the tutorials for some worked examples of using Hazelcast with Feast.
Functionality Matrix
The following table shows which Feast functionality is supported by the Hazelcast integration:
| Feast functionality | Supported by Hazelcast? |
|---|---|
Write feature values to the online store |
Yes |
Read feature values to the online store |
Yes |
Update infrastructure, such as tables, in the online store |
Yes |
Teardown infrastructure, such as tables, in the online store |
Yes |
Generate a plan of infrastructure changes |
No |
Support for online transforms |
Yes |
Readable by Python SDK |
Yes |
Readable by Java |
No |
Readable by Go |
No |
Support for entityless feature views |
Yes |
Support for concurrent writing to the same key |
Yes |
Support for TTL at retrieval |
Yes |
Support for deleting expired data |
Yes |
Collocated by feature view |
No |
Collocated by feature service |
No |
Collocated by entity key |
Yes |
