Skip to content

[WIP] [Python/Beam] Add F# quotation support#4398

Open
dbrattli wants to merge 8 commits intomainfrom
dbrattli/quoted
Open

[WIP] [Python/Beam] Add F# quotation support#4398
dbrattli wants to merge 8 commits intomainfrom
dbrattli/quoted

Conversation

@dbrattli
Copy link
Collaborator

@dbrattli dbrattli commented Mar 14, 2026

Summary

FYI: This is just an investigation of what it would take to add Quotation support to Fable [WIP]

  • Adds F# code quotation support (<@ ... @>) to the Python and Beam targets
  • Quotation ASTs can be constructed, pattern matched (via Microsoft.FSharp.Quotations.Patterns), and evaluated (via LeafExpressionConverter.EvaluateQuotation)
  • Shared QuotationEmitter.fs transforms quoted Fable.Expr into runtime library calls at compile time
  • Deliberately left out JS to avoid increasing bundle size (FYI: @MangelMaxime)

Supported quotation nodes

Value, Lambda, Let, IfThenElse, Application, Call, Sequential, NewTuple, Operation (all arithmetic/comparison/logical), TypeCast, Get (TupleIndex, UnionField, UnionTag, FieldGet), Set (ValueSet, FieldSet), NewUnion, NewRecord, NewOption, NewList, DecisionTree (simple cases).

Files changed

Area Files
AST Fable.fs — adds Quote of quotedExpr: Expr * isTyped: bool (3 lines)
Frontend FSharp2Fable.fs, FableTransforms.fs, Transforms.Util.fs — capture and propagate Quote
Shared engine QuotationEmitter.fs (new) — transforms quoted Fable.Expr → runtime lib calls
Python Fable2Python.Transforms.fs, Python/Replacements.fs, fable_quotation.py (new)
Beam Fable2Beam.fs, Beam/Replacements.fs, fable_quotation.erl
Other targets Fable2Babel.fs, Fable2Dart.fs, Fable2Rust.fs, Fable2Php.fs — error stubs for Quote
Tests TestQuotation.fs (Python, 24 tests), QuotationTests.fs (Beam, 18 tests)
Changelogs Fable.Cli/CHANGELOG.md, Fable.Compiler/CHANGELOG.md

Design note: AST approach vs original PR #1839

The original quotation PR (#1839) added a complex ExprData record to the AST:

// PR #1839
| Quote of typed: bool * data: ExprData * range: SourceLocation option

type ExprData = {
    typ: Type; variables: VarData[]; values: ValueData[]
    literals: Expr[]; types: Type[]
    members: array<FSharpEntity * Type * MemberInfo * Type[]>
    data: byte[]
}

This PR takes a minimal AST approach instead:

// This PR
| Quote of quotedExpr: Expr * isTyped: bool

The quoted Expr is kept as-is in the Fable AST. All decomposition into runtime calls happens in the transform layer (QuotationEmitter.fs), not the AST. This means:

  • + Only 3 lines added to Fable.AST (no new types, no new Type variants)
  • + No serialization/deserialization layer needed
  • + QuotationEmitter can evolve without breaking the AST contract
  • Quote _ -> Any is less precise than Expr(Some typ) / Expr None

If a more precise type is desired, a | Quotation of gen: Type option could be added to the Type union — but that would be a separate AST change worth discussing.

Test plan

  • All 2103 Python tests pass (./build.sh test python)
  • All 2287 Beam tests pass (./build.sh test beam)
  • .NET tests pass for both Python and Beam test projects
  • Verify no regressions in other targets

🤖 Generated with Claude Code

dbrattli and others added 2 commits March 14, 2026 08:40
…ching, and evaluation

Add support for F# code quotations (`<@ ... @>`) to the Python and Beam targets.
This enables constructing quotation ASTs at runtime, pattern matching over them
using `Microsoft.FSharp.Quotations.Patterns`, and evaluating them via
`LeafExpressionConverter.EvaluateQuotation`.

Architecture:
- Adds `Quote of quotedExpr: Expr * isTyped: bool` to Fable AST (minimal change)
- QuotationEmitter.fs (shared): transforms quoted Fable.Expr into runtime library calls
- Runtime libraries: fable_quotation.py and fable_quotation.erl with dataclass/tuple-based
  AST nodes, pattern match helpers, and an evaluator with operator dispatch

Supported quotation nodes: Value, Lambda, Let, IfThenElse, Application, Call,
Sequential, NewTuple, Operation, TypeCast, Get (TupleIndex, UnionField, UnionTag,
FieldGet), Set (ValueSet, FieldSet), NewUnion, NewRecord, NewOption, NewList.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ern helpers

Beam arrays are stored as process dictionary references, not plain lists.
Added deref/1 helper to convert Refs back to lists before iteration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Contributor

github-actions bot commented Mar 14, 2026

Python Type Checking Results (Pyright)

Metric Value
Total errors 18
Files with errors 4
Excluded files 4
New errors ✅ No
Excluded files with errors (4 files)

These files have known type errors and are excluded from CI. Remove from pyrightconfig.ci.json as errors are fixed.

File Errors Status
temp/tests/Python/test_applicative.py 12 Excluded
temp/tests/Python/test_hash_set.py 3 Excluded
temp/tests/Python/test_nested_and_recursive_pattern.py 2 Excluded
temp/tests/Python/fable_modules/thoth_json_python/encode.py 1 Excluded

dbrattli and others added 4 commits March 14, 2026 09:09
The NewTuple active pattern returns Expr list (FSharpList) on .NET.
Convert the native array to FSharpList in the Python runtime so
exprs.Length works correctly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove unnecessary $ prefix from plain strings and add %s format
specifier to typeToString interpolation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add QuotationEmitter.fs to fable-standalone project file list
- Use Array[Any] instead of list[Any] in fable_quotation.py to match
  Fable's array representation (FSharpArray), fixing pyright errors

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- get_free_vars: walks AST collecting unbound variables
- substitute: replaces variables using a substitution function
- expr_to_string: pretty-prints quotation AST as F#-like source code
  (ToString wiring deferred to separate task — needs to match .NET format)
- Wired up in both Python and Beam Replacements
- Added GetFreeVars test for both targets
- Updated CLAUDE.md to clarify test running

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@dbrattli
Copy link
Collaborator Author

dbrattli commented Mar 14, 2026

Question: Query expression support and FCS inlining

@ncave @MangelMaxime

While implementing quotation support, we investigated adding query { } expression support. We found that FCS fully inlines QueryBuilder methods before Fable sees the AST.

What happens:

  1. query { for x in [1;2;3] do select x } gets desugared by FCS into QueryBuilder.Run(QueryBuilder.For(source, fun x -> QueryBuilder.Select(...)))
  2. FCS then inlines these methods, which internally call LeafExpressionConverter.EvaluateQuotation
  3. By the time Fable's FSharp2Fable processes the AST, the QueryBuilder calls are gone — the Replaced active pattern in FSharp2Fable.Util.fs is never reached
  4. The inlined code becomes a Raise (since EvaluateQuotation can't be resolved at compile time)

We added replacement handlers for Microsoft.FSharp.Linq.QueryBuilder (Select, Where, For, Yield, etc. → Seq module operations), but they're unreachable because the calls are already inlined away.

Question: Does this analysis make sense? Are we right that FCS inlines the QueryBuilder before Fable can intercept it? If so, is there a way in the FCS fork to prevent this inlining, or to add a hook that lets Fable intercept QueryBuilder calls before they're resolved? This would enable mapping query { } to Seq operations on targets like Python and Beam.

@dbrattli dbrattli changed the title [Python/Beam] Add F# quotation support [WIP] [Python/Beam] Add F# quotation support Mar 14, 2026
dbrattli and others added 2 commits March 14, 2026 10:30
…lity

Seq.length expects IEnumerable_1, which Array implements but plain
list does not. Fixes pyright error in transpiled test output.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The list module is auto-generated from List.fs during build. A minimal
.pyi stub lets pyright/Pylance resolve the import in the source tree
without needing a full build.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ncave
Copy link
Collaborator

ncave commented Mar 14, 2026

@dbrattli

The FCS fork that Fable uses is intentionally a very thin layer/facade on top of the actual FCS, with no customization of the F# AST. That helps to shield us from the FCS internal churn and helps with migrating to new FCS versions, but it is true that it is sometimes hard to accurately reconstruct F# features from the F# AST, as exposed by FCS.

For example, in Fable we go through a lot of trouble to try to reconstruct the original F# match expressions, which get compiled into F# AST decision trees and targets by FCS, and that reconstruction back can never be perfect. I don't know if such reconstruction from F# AST is possible for quotations, does the F# AST retain enough fidelity for Fable to reconstruct the original form. Good question to ask in the dotnet/fsharp repo, perhaps.

That said, if you have a proposal for modifying the Fable FCS fork, we technically can make changes, assuming it is not a drastic modification of the F# AST that would impede future maintenance when adopting new FCS versions.

@MangelMaxime
Copy link
Member

Deliberately left out JS to avoid increasing bundle size

In theory, if we don't use a code from Fable it gets removed by bundlers when generating the bundle.

This is for example, the case for things related to decimal which are quite heavy.

So this is probably fine, to add support for JavaScript too.

@dbrattli
Copy link
Collaborator Author

@ncave Thanks for the feedback. I will drop this for now. Probably easier to just make our own query builder outside the compiler if query syntax is needed.

Comment on lines +3757 to +3885
// F# Quotation: FSharpExpr static methods (e.g. Expr.Value, Expr.Lambda, etc.)
let private quotationExprs
(com: ICompiler)
(_ctx: Context)
r
(t: Type)
(i: CallInfo)
(_thisArg: Expr option)
(args: Expr list)
=
match i.CompiledName, _thisArg, args with
| "Value", _, [ value; typeArg ] ->
Helper.LibCall(com, "fable_quotation", "mk_value", t, [ value; typeArg ], ?loc = r)
|> Some
| "Var", _, [ var ] ->
Helper.LibCall(com, "fable_quotation", "mk_var_expr", t, [ var ], ?loc = r)
|> Some
| "Lambda", _, [ var; body ] ->
Helper.LibCall(com, "fable_quotation", "mk_lambda", t, [ var; body ], ?loc = r)
|> Some
| "Application", _, [ func; arg ] ->
Helper.LibCall(com, "fable_quotation", "mk_app", t, [ func; arg ], ?loc = r)
|> Some
| "Let", _, [ var; value; body ] ->
Helper.LibCall(com, "fable_quotation", "mk_let", t, [ var; value; body ], ?loc = r)
|> Some
| "IfThenElse", _, [ guard; thenExpr; elseExpr ] ->
Helper.LibCall(com, "fable_quotation", "mk_if_then_else", t, [ guard; thenExpr; elseExpr ], ?loc = r)
|> Some
| "Call", _, [ instance; methodInfo; argList ] ->
Helper.LibCall(com, "fable_quotation", "mk_call", t, [ instance; methodInfo; argList ], ?loc = r)
|> Some
| "NewTuple", _, [ elements ] ->
Helper.LibCall(com, "fable_quotation", "mk_new_tuple", t, [ elements ], ?loc = r)
|> Some
| "Sequential", _, [ first; second ] ->
Helper.LibCall(com, "fable_quotation", "mk_sequential", t, [ first; second ], ?loc = r)
|> Some
| "get_Type", Some callee, _ ->
Helper.LibCall(com, "fable_quotation", "get_type", t, [ callee ], ?loc = r)
|> Some
| "GetFreeVars", Some callee, _ ->
Helper.LibCall(com, "fable_quotation", "get_free_vars", t, [ callee ], ?loc = r)
|> Some
| "Substitute", Some callee, [ fn ] ->
Helper.LibCall(com, "fable_quotation", "substitute", t, [ callee; fn ], ?loc = r)
|> Some
| "ToString", Some callee, _
| "ToString", Some callee, [ _ ] ->
Helper.LibCall(com, "fable_quotation", "expr_to_string", t, [ callee ], ?loc = r)
|> Some
| _ -> None

// F# Quotation: FSharpVar (.ctor, get_Name, get_Type, get_IsMutable)
let private quotationVars
(com: ICompiler)
(_ctx: Context)
r
(t: Type)
(i: CallInfo)
(thisArg: Expr option)
(args: Expr list)
=
match i.CompiledName, thisArg, args with
| ".ctor", None, [ name; typ; isMutable ] ->
Helper.LibCall(com, "fable_quotation", "mk_var", t, [ name; typ; isMutable ], ?loc = r)
|> Some
| ".ctor", None, [ name; typ ] ->
Helper.LibCall(com, "fable_quotation", "mk_var", t, [ name; typ; makeBoolConst false ], ?loc = r)
|> Some
| "get_Name", Some callee, _ ->
Helper.LibCall(com, "fable_quotation", "var_get_name", t, [ callee ], ?loc = r)
|> Some
| "get_Type", Some callee, _ ->
Helper.LibCall(com, "fable_quotation", "var_get_type", t, [ callee ], ?loc = r)
|> Some
| "get_IsMutable", Some callee, _ ->
Helper.LibCall(com, "fable_quotation", "var_get_is_mutable", t, [ callee ], ?loc = r)
|> Some
| _ -> None

// F# Quotation: PatternsModule (active patterns like ValuePattern, LambdaPattern, etc.)
let private quotationPatterns
(com: ICompiler)
(_ctx: Context)
r
(t: Type)
(i: CallInfo)
(_thisArg: Expr option)
(args: Expr list)
=
match i.CompiledName, args with
| ("ValuePattern" | "|Value|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_value", t, [ expr ], ?loc = r)
|> Some
| ("VarPattern" | "|Var|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_var", t, [ expr ], ?loc = r) |> Some
| ("LambdaPattern" | "|Lambda|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_lambda", t, [ expr ], ?loc = r)
|> Some
| ("ApplicationPattern" | "|Application|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_application", t, [ expr ], ?loc = r)
|> Some
| ("LetPattern" | "|Let|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_let", t, [ expr ], ?loc = r) |> Some
| ("IfThenElsePattern" | "|IfThenElse|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_if_then_else", t, [ expr ], ?loc = r)
|> Some
| ("CallPattern" | "|Call|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_call", t, [ expr ], ?loc = r) |> Some
| ("NewTuplePattern" | "|NewTuple|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_new_tuple", t, [ expr ], ?loc = r)
|> Some
| ("SequentialPattern" | "|Sequential|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_sequential", t, [ expr ], ?loc = r)
|> Some
| ("NewUnionCasePattern" | "|NewUnionCase|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_new_union", t, [ expr ], ?loc = r)
|> Some
| ("NewRecordPattern" | "|NewRecord|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_new_record", t, [ expr ], ?loc = r)
|> Some
| ("TupleGetPattern" | "|TupleGet|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_tuple_get", t, [ expr ], ?loc = r)
|> Some
| ("PropertyGetPattern" | "|PropertyGet|_|"), [ expr ] ->
Helper.LibCall(com, "fable_quotation", "is_field_get", t, [ expr ], ?loc = r)
|> Some
| _ -> None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When looking at the code from both Replacement.fs files, I am wondering if it would be worth sharing the code.

It seems like both code are identical match.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants