You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have a strange track record of defining relationships between various feast proto objects, namely we sometimes include "child" object protos inside parent objects, while in other cases we only refer to them by name.
For example, FeatureView includes DataSource objects as part of the message, which means that we are effectively keeping identical DataSource objects in the registry backends more than once (incl. additional information like tags). This can also lead to strange data consistency issues, because a user can call registry.apply_data_source on an existing data source which will update only the data source object in the registry and leave the one embedded in FeatureView with stale value. Actual FeatureStore methods will most likely use a stale value for both operations and permission checking.
In other cases, some relationships are defined by name only, for example FeatureView keeps a list of Entity names only, FeatureViewProjection (and FeatureService by association) only stores feature view names, rather than FeatureView proto messages themselves.
While having relationships based on names forces you to do some additional lookups to reconstruct necessary information after retrieval, I think it's still worth it as it will avoid data duplication and consistency/security issues.
P.S. This is almost certainly a very breaking change.
The text was updated successfully, but these errors were encountered:
We have a strange track record of defining relationships between various feast proto objects, namely we sometimes include "child" object protos inside parent objects, while in other cases we only refer to them by name.
For example, FeatureView includes DataSource objects as part of the message, which means that we are effectively keeping identical DataSource objects in the registry backends more than once (incl. additional information like tags). This can also lead to strange data consistency issues, because a user can call
registry.apply_data_source
on an existing data source which will update only the data source object in the registry and leave the one embedded in FeatureView with stale value. Actual FeatureStore methods will most likely use a stale value for both operations and permission checking.In other cases, some relationships are defined by name only, for example FeatureView keeps a list of Entity names only, FeatureViewProjection (and FeatureService by association) only stores feature view names, rather than FeatureView proto messages themselves.
While having relationships based on names forces you to do some additional lookups to reconstruct necessary information after retrieval, I think it's still worth it as it will avoid data duplication and consistency/security issues.
P.S. This is almost certainly a very breaking change.
The text was updated successfully, but these errors were encountered: