Skip to content

Commit

Permalink
included repo directory and feature_store.yaml
Browse files Browse the repository at this point in the history
  • Loading branch information
@samuel100
samuel100 committed Oct 9, 2021
1 parent 3222f5f commit 8f6363f
Show file tree
Hide file tree
Showing 3 changed files with 176 additions and 206 deletions.
112 changes: 46 additions & 66 deletions provider/docs/tutorial/notebooks/01-register-features.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Copyright (c) Microsoft Corporation.\n",
"Licensed under the MIT license.\n",
Expand All @@ -12,64 +13,45 @@
"\n",
"## Configure Feature Repo\n",
"\n",
"The cell below connects to your feature store. The `registry_blob_url` should point to the location on blob where you want your feature repsository to be stored."
],
"metadata": {}
"The cell below connects to your feature store. __You need to update the feature_repo/feature_store.yaml file so that the registry path points to your blob location__"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from feast import FeatureStore, RepoConfig\n",
"from feast.registry import RegistryConfig\n",
"from feast_azure_provider.mssqlserver import MsSqlServerOfflineStoreConfig\n",
"from feast.infra.online_stores.redis import RedisOnlineStoreConfig\n",
"import os\n",
"from feast import FeatureStore\n",
"from azureml.core import Workspace\n",
"\n",
"# update this to your location on blob\n",
"registry_blob_url = \"https://<ACCOUNT_NAME>.blob.core.windows.net/<CONTAINER>/<PATH>/registry.db\"\n",
"\n",
"# access key vault to get secrets\n",
"ws = Workspace.from_config()\n",
"kv = ws.get_default_keyvault()\n",
"\n",
"# update with your connection string\n",
"offline_conn_str = kv.get_secret(\"FEAST-SQL-CONN\")\n",
"online_conn_str = kv.get_secret(\"FEAST-REDIS-CONN\")\n",
"\n",
"# set RegistryConfig\n",
"reg_config = RegistryConfig(\n",
" registry_store_type=\"feast_azure_provider.registry_store.AzBlobRegistryStore\",\n",
" path=registry_blob_url,\n",
")\n",
"\n",
"# set RepoConfig\n",
"repo_cfg = RepoConfig(\n",
" registry=reg_config,\n",
" project=\"production\",\n",
" provider=\"feast_azure_provider.azure_provider.AzureProvider\",\n",
" offline_store=MsSqlServerOfflineStoreConfig(connection_string=offline_conn_str),\n",
" online_store=RedisOnlineStoreConfig(connection_string=online_conn_str),\n",
")\n",
"os.environ['SQL_CONN']=kv.get_secret(\"FEAST-SQL-CONN\")\n",
"os.environ['REDIS_CONN']=kv.get_secret(\"FEAST-REDIS-CONN\")\n",
"\n",
"# connect to feature store\n",
"store = FeatureStore(config=repo_cfg)"
],
"outputs": [],
"metadata": {}
"fs = FeatureStore(\"./feature_repo\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define the data source (offline store)\n",
"\n",
"The data source refers to raw underlying data (a table in Azure SQL DB or Synapse SQL). Feast uses a time-series data model to represent data. This data model is used to interpret feature data in data sources in order to build training datasets or when materializing features into an online store."
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from feast_azure_provider.mssqlserver_source import MsSqlServerSource\n",
"\n",
Expand All @@ -88,12 +70,11 @@
" event_timestamp_column=\"datetime\",\n",
" created_timestamp_column=\"\",\n",
")"
],
"outputs": [],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define Feature Views\n",
"\n",
Expand All @@ -106,12 +87,13 @@
"- Retrieval of features from the online store. Feature views provide the schema definition to Feast in order to look up features from the online store.\n",
"\n",
"__NOTE: Feast does not generate feature values. It acts as the ingestion and serving system. The data sources described within feature views should reference feature values in their already computed form.__"
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from feast import Feature, FeatureView, ValueType\n",
"from datetime import timedelta\n",
Expand Down Expand Up @@ -139,12 +121,11 @@
" batch_source=customer_source,\n",
" ttl=timedelta(days=2),\n",
")"
],
"outputs": [],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Define entities\n",
"\n",
Expand All @@ -158,22 +139,22 @@
"A related concept is an entity key. These are one or more entity values that uniquely describe a feature view record. In the case of an entity (like a driver) that only has a single entity field, the entity is an entity key. However, it is also possible for an entity key to consist of multiple entity values. For example, a feature view with the composite entity of (customer, country) might have an entity key of (1001, 5).\n",
"\n",
"Entity keys act as primary keys. They are used during the lookup of features from the online store, and they are also used to match feature rows across feature views during point-in-time joins."
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from feast import Entity\n",
"driver = Entity(name=\"driver\", join_key=\"driver_id\", value_type=ValueType.INT64)\n",
"customer = Entity(name=\"customer_id\", value_type=ValueType.INT64)"
],
"outputs": [],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Feast `apply()`\n",
"\n",
Expand All @@ -183,41 +164,40 @@
"1. Feast will validate your feature definitions\n",
"1. Feast will sync the metadata about Feast objects to the registry. If a registry does not exist, then it will be instantiated. The standard registry is a simple protobuf binary file that is stored on Azure Blob Storage.\n",
"1. Feast CLI will create all necessary feature store infrastructure. The exact infrastructure that is deployed or configured depends on the provider configuration that you have set in feature_store.yaml."
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"store.apply([driver, driver_fv, customer, customer_fv])"
],
"metadata": {},
"outputs": [],
"metadata": {}
"source": [
"fs.apply([driver, driver_fv, customer, customer_fv])"
]
}
],
"metadata": {
"orig_nbformat": 4,
"interpreter": {
"hash": "1f420f8439dfed2bd66fe971ededeecdddcec354e785e62812183e5ad86a193f"
},
"kernelspec": {
"display_name": "Python 3.8.11 64-bit ('feast-dev': conda)",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.8.11",
"mimetype": "text/x-python",
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"pygments_lexer": "ipython3",
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"file_extension": ".py"
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3.8.11 64-bit ('feast-dev': conda)"
"pygments_lexer": "ipython3",
"version": "3.8.11"
},
"interpreter": {
"hash": "1f420f8439dfed2bd66fe971ededeecdddcec354e785e62812183e5ad86a193f"
}
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
}
Loading

0 comments on commit 8f6363f

Please sign in to comment.