Skip to content

fix: Fix Shared SQL/Snowflake registry crashes on cross-project UDF deserialization#6427

Open
ntkathole wants to merge 1 commit into
feast-dev:masterfrom
ntkathole:fix_shared_project_udf
Open

fix: Fix Shared SQL/Snowflake registry crashes on cross-project UDF deserialization#6427
ntkathole wants to merge 1 commit into
feast-dev:masterfrom
ntkathole:fix_shared_project_udf

Conversation

@ntkathole
Copy link
Copy Markdown
Member

@ntkathole ntkathole commented May 21, 2026

What this PR does / why we need it:

When multiple Feast projects share a single registry (PostgreSQL, S3, or Snowflake), operations like feast apply, REST API listing (/features/all), and UI dashboard crash with ModuleNotFoundError because the registry eagerly deserializes UDFs (via dill.loads()) from all projects — including those whose Python modules aren't installed in the current environment.

This PR fixes the crash at two layers:

Layer 1: Cache building (proto() method) — SQL & Snowflake registries

  • Added a proto_only parameter to _list_objects() that returns raw protobuf objects without calling from_proto(), completely bypassing dill.loads().
  • Refactored proto() to use _list_objects(proto_only=True) when building the in-memory cache, restoring the clean lister-loop pattern.

Layer 2: REST API / gRPC listing — All registry types

  • Threaded a skip_udf parameter through the registry stack: base_registry → caching_registry / registry / snowflake / remote → proto_registry_utils.
  • gRPC handlers (ListFeatureViews, ListStreamFeatureViews, ListOnDemandFeatureViews, ListAllFeatureViews, ListFeatures) pass skip_udf=True so metadata listing never triggers dill.loads().
  • Preserved full API response fidelity: when skip_udf=True, the raw transformation proto is stored on the Python object and used directly in to_proto(), so the response (including UDF source text, binary body, name, mode) is identical to the non-skip path.

@ntkathole ntkathole self-assigned this May 21, 2026
@ntkathole ntkathole requested a review from a team as a code owner May 21, 2026 09:44
@ntkathole ntkathole force-pushed the fix_shared_project_udf branch 3 times, most recently from 6221445 to 8dceb9d Compare May 21, 2026 12:18
@ntkathole ntkathole force-pushed the fix_shared_project_udf branch 3 times, most recently from 3693501 to 71c3c95 Compare May 21, 2026 13:30
…ation in proto cache building

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
@ntkathole ntkathole force-pushed the fix_shared_project_udf branch from 71c3c95 to 263c52e Compare May 21, 2026 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants