timlariviere/fabulous-new

Fabulous v2 - Work in progress

fabulous-new's Issues

[Bug] Investigate bugs when diffing NavigationPage

TDB

[Tooling] Adapt Fabulous.CodeGen to automate creation of widget builders

In v1, we created Fabulous.CodeGen to automate the creation of wrappers around Xamarin.Forms controls.
It works by taking a mapping file (Xamarin.Forms.Core.json) which has the path to dlls and how we should map the types inside those dlls.
It outputs 2 F# files with the attribute keys and wrappers for all mapped controls.

The idea is to reuse Fabulous.CodeGen and adapt it so it can generate both Xamarin.Forms.Core.Attributes.fs and Xamarin.Forms.Core.fs

Doing this will avoid us to implement everything by hand and make a lot more controls available in v2.

[Performance] Optimize code to create the smallest binaries

Xml documentation for extensions

Would be good if we can add support for XML documentation Add Extension https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/xml-documentation#recommended-tags. This will make really easy to use

[Performance, Architecture] implement `dependsOn` from V1 or `memo`

We need a way to cut off diffing. In V1 there was a clever trick that involved getting the location of a function type.

It does seem like a hack to me, but very clever! ^_^.

Unfortunately, we cannot directly port it to v2 due to new reconciler. For example it works with structs vs reference types.

So, we need a similar way to cut off diffing of a branch. Note that in that case diffing of a branch is going to be way slower than a few allocations, thus it is totally ok to use dynamic dispatch + heap allocated values there.

I would propose to store some data in ViewNode (seems like a good place for that) vs global weak map used in V1. That is how React and Elm do that (at least conceptually).

[Architecture] Support virtualized collections

2 ways to represent item collection controls such as ListView, CollectionView, CarouselView, etc.

// Virtualized list
ListView(items, fun item -> item.Id, fun item ->
    TextCell(item.Text)
)

// Explicit list
ListView([
    for item in items do
        TextCell(item.Text)
            .key(item.Id)
])

In v1, we're only supporting explicit lists but it has a lot of constraints.
Virtualization is hard to do and if you happen to have 1 million items in your list, it would completely kill the performance (view function + diffing).

So I think it would be better to create 2 kinds of lists ListView + VirtualizedListView to support better virtualization.

[Performance] Avoid dynamic dispatch for Convert / ConvertValue in AttributeDefinition

One thing I was thinking about if we can make convert functions optional (ValueOption). My understanding that it is id function mist of the time. If that is true, skipping calling it with checking for ValueSome is probably worth it in terms of perf.

BUT! it is probably going to be tricky to type properly, given that then we will have to convert 'modelType -> 'valueType.

So maybe a union can be a solution there.
just a sketch:

type ScalarAttributeDefinition<'inputType, 'modelType, 'valueType> =
  | NoConverters of .... // 'inputType = 'modelType = 'valueType, are all the same
  | JustModelConverted of ... //  'inputType <> 'modelType ,  'valueType ='modelType
  | JustValueConverter of ... // 'inputType = 'modelType ,  'valueType <> 'modelType
  | ModelAndValueConverters of ... // 'inputType <> 'modelType <> 'valueType, are all different

Feels very clunky but possible worth considering

Originally posted by @twop in #33 (comment)

[Xamarin.Forms] Add wrappers for missing types

There are still somethings are not wrapped . Like Effects, Behaviours, VisualStateManagers, TemplatedView , TemplatedPages, StyleWidgets. Would be good to create a separate issue for all them so we can attempt then in the future.

Originally posted by @edgarfgp in #53 (comment)

RelativeLayout missing support for RelativeToView

RelativeToView is missing from v1 and V2 . Would be good to see if there is anyway we can support this as we move forward to stable 2.0

public static Constraint RelativeToView(View view, Func<RelativeLayout, View, double> measure)
{
	var result = new Constraint { _measureFunc = layout => measure(layout, view), RelativeTo = new[] { view } };

	return result;
}

• BoundsConstraint

public static BoundsConstraint FromExpression(Expression<Func<Rectangle>> expression, IEnumerable<View> parents = null)
{
	return FromExpression(expression, false, parents);
}

internal static BoundsConstraint FromExpression(Expression<Func<Rectangle>> expression, bool fromExpression, IEnumerable<View> parents = null)
{
	Func<Rectangle> compiled = expression.Compile();
	var result = new BoundsConstraint
	{
		_measureFunc = compiled,
		RelativeTo = parents ?? ExpressionSearch.Default.FindObjects<View>(expression).ToArray(), // make sure we have our own copy
		CreatedFromExpression = fromExpression
	};

	return result;
}

Implement Style widgets

It would be great to enable styling through specialized Widgets like LabelStyle(), ButtonStyle(), etc.
Those styles would be passed to .styles([...]) modifiers at both the control level and application level.

Label("Hello")
    .style(
        // Local style
        LabelStyle()
            .textColor(Color.Blue)
            .font(FontFamily.OpenSans)
        )
    )
        
Application(
    ContentPage(
        VerticalStackLayout([
            Button("Click me", OnClicked)
            
            Button("Destroy the world", OnDestroyClicked)
                .styleKey("destructive")
        ])
    )
)
    .styles([
        // Global style applied to all buttons
        ButtonStyle()
            .backgroundColor(light = Color.Gray, dark = Color.Yellow)
            
        // Global style only applied to buttons with style key `destructive`
        ButtonStyle("destructive")
            .textColor(Color.Red)
    ])

The idea of those widgets will be to make them map to

<Application>
    <Application.Resources>
        <Style TargetType="Button">
            <Setter PropertyName="BackgroundColor" Value="{AppThemeBinding Light=Color.Gray, Dark=Color.Yellow}" />
        </Style>
        <Style TargetType="Button" Key="destructive">
            <Setter PropertyName="TextColor" Value="Color.Red" />
        </Style>
    </Application.Resources>
</Application>

To implement this, you can take a look at Xamarin.Forms.Core.Attributes.fs and Xamarin.Forms.Core.fs.

I think the setters will be declared as attributes (via a custom defineWidget => defineSetter) and styles like widgets (type [<Struct>] LabelStyle (...)).

If you want to work on this task, put a message after this comment.
Also try putting everything you write in a new file (eg. Xamarin.Forms.Core.Style.fs) so you won't have too many conflicts if we change stuff.

v2 on top of Uno WinUI

As long as I was managed to run Elmish.Uno on WinUI
<TargetFrameworks>net6.0;net6.0-windows10.0.22000.0</TargetFrameworks>
And now we can call WinRT API using F#.
I think that it is a good idea to propose to build Fabulous v2 on top of WinUI targeting WinUI Uno Platform as it has better performance and is better designed than Xamarin.Forms/MAUI

[Architecture] Support composing widgets with different message types

In v1, we were exposing dispatch in the view function and it was up to the developers to map messages to the right root type.

type Msg =
    | SubPageMsg of SubPage.Msg

let view model dispatch =
   SubPage.view model.SubModel (SubPageMsg >> dispatch)

This was possible because we weren't strongly typing the views (everything is ViewElement).

dispatch is a function internal to Fabulous and it's better to not leak it for the view function where it can be misused.
To avoid this, in v2 we're strongly typing the widgets (eg. ContentPage<SubPage.Msg>) and Fabulous takes care of dispatching internally without exposing it to the developers.

But this strong typing prevents composing widgets of different msg types.

NavigationPage([
    MainPage.view model.MainPageModel
    DetailPage.view model.DetailPageModel // Error: Expected to be ContentPage<MainPage.Msg>, but was ContentPage<DetailPage.Msg>
])

So to change the msg type from one to another (ContentPage<MainPage.Msg> => ContentPage<DetailPage.Msg>), having a mapping function would be necessary

type Msg =
    | MainMsg of MainPage.Msg
    | DetailMsg of DetailPage.Msg

NavigationPage([
    Widget.map MainMsg (MainPage.view model.MainPageModel)
    Widget.map DetailMsg (DetailPage.view model.DetailPageModel)
])

Code format check

Hello, I noticed Fantomas was added in #40.
Many thanks for doing this.

I did also see that you are checking whether the code is formatted using dotnet fantomas --check **/*.fs.
Wildcards are not supported there, you best pass along the folder you want to check, included by the recurse flag.
Something like dotnet fantomas --check -r src.

[Xamarin.Forms] Add wrappers for all controls

Would be great to have a control gallery apps to both test out the wrappers and have a sample on how to use them.

Pages:

• ContentPage
• FlyoutPage
• NavigationPage
• TabbedPage
• TemplatedPage(?)

Layouts:

• AbsoluteLayout
• ContentPresenter(?)
• ContentView
• FlexLayout
• Frame
• Grid
• RelativeLayout
• ScrollView
• StackLayout
• TemplatedView(?)

Controls:

• ActivityIndicator
• BoxView
• Button
• CheckBox
• DatePicker
• Editor
• Ellipse
• Entry
• Image
• ImageButton
• Label
• Line
• Path
• Polygon
• Polyline
• ProgressBar
• RadioButton
• Rectangle
• RefreshView
• SearchBar
• Slider
• Stepper
• SwipeView
• Switch
• TimePicker
• WebView

Collections:

• CarouselView
• CollectionView
• IndicatorView
• ListView
• Picker
• TableView (removed because deprecated in MAUI)

Cells:

• TextCell
• ImageCell
• SwitchCell
• EntryCell
• ViewCell

MenuItems:

• MenuItem
• ToolbarItem

GestureRecognizers:

• TapGestureRecognizer
• PinchGestureRecognizer
• PanGestureRecognizer
• SwipeGestureRecognizer
• DragGestureRecognizer
• DropGestureRecognizer

Release 1 - to do list

Based on Reasonable goals of v2, here are the things we still need to do before releasing the first part of v2.

Core:

• #9
• #11
• #25
• #21
• #24

Xamarin.Forms:

• #53

Add debounce function on WidgetsHelpers to help Android events

I think we should add a debounce function like in V1 https://github.com/fsprojects/Fabulous/blob/8bbb610a2bbfe239e175b8e25ac950d55ebc94f5/Fabulous.XamarinForms/src/Fabulous.XamarinForms/ViewHelpers.fs#L66

Write architecture documentation

We need to write all the decisions we made for the architecture of v2, including details on:

• Add glossary of all the internal names (Widgets, Scalar/Widget/WidgetCollection attributes, Program, ViewAdapter, Reconciler, ViewNode, etc.)
• Explain UI concepts (UI tree vs Virtual tree)
• Explain view building process (Widget, attributes, definitions, CE builders)
• Explain diffing process (Reconciler and ViewNode)
• Explain MVU lifecycle (Program, Runners and ViewAdapters)
• Explain feature MapMsg
• Explain feature Memo (mostly copy-paste #36)
• Explain virtualized collections support (done #9)

Expose animation methods as an extensions

We can enable all the animation methods in : https://docs.microsoft.com/en-us/dotnet/api/xamarin.forms.viewextensions?view=xamarin-forms

• ScaleTo
• RelScaleTo
• ScaleXTo
• ScaleYTo
• RotateTo
• RelRotateTo
• RotateXTo
• RotateYTo
• FadeTo
• TranslateTo
• ScrollTo

[Performance] Figure out how to benchmark with "Mono Ahead of Time"

Currently we benchmark with .NET 6 JIT, while it is awesome to have something going, we probably want a closer to Fabulous actual use case: mobile AoT compilation with Mono.

We use https://benchmarkdotnet.org that can support it, I just haven't had enough energy to figure out how. Note that I have Apple M1 chip which can run arm code natively (thus the closest that we can get)

Here is a snippet how to setup Mono https://benchmarkdotnet.org/articles/configs/toolchains.html#monoaotllvm

Alternatively we can have a test mobile project (not even XF) with a simple functionality of Run and Report/See results that we manually deploy to a device.

Note for the benchmark app above we would still use TestUI framework vs Xamarin binding to have a stable reference and "apples to apples" comparison

[Bug] Invalid diffing when adding/removing similar controls

When toggling the button, diffing by mistake change the label of the Toggle button

Simulator.Screen.Recording.-.iPhone.13.Pro.-.2022-03-04.at.11.30.33.mp4

namespace CounterApp

open Fabulous
open Fabulous.XamarinForms

open type Fabulous.XamarinForms.View

module App =
    type Model =
        { Flag: bool }

    type Msg =
        | Toggle
        
    let init () =
        { Flag = false }

    let update msg model =
        match msg with
        | Toggle ->
            { model with Flag = not model.Flag }

    let view model =
        Application(
            ContentPage(
                "CounterApp",
                (VStack() {
                    Button("Toggle", Toggle)
                    
                    if model.Flag then
                        Label("Hello")
                        Button("World", Toggle)
                        
                 })
                    .padding(30.)
                    .centerVertical ()
            )
        )

    let program =
        Program.statefulApplication init update view

[Architecture] Support specifying a key for each widget to help Reconciler understand intent from update to update

[Performance] Implement StackArray of several sizes and use it for storing Attributes and Children

Context

Currently adding an attribute to a widget involves a heap allocation of a new array.

Copy pasting (at the moment of creation of this issue):

static member inline AddScalarAttribute(this: ^T :> IWidgetBuilder, attr: ScalarAttribute) =
    let attribs = this.ScalarAttributes
    let attribs2 = Array.zeroCreate(attribs.Length + 1) // <--- allocating a new one
    Array.blit attribs 0 attribs2 0 attribs.Length
    attribs2.[attribs.Length] <- attr

    let result =
        (^T: (new : ScalarAttribute [] * WidgetAttribute [] * WidgetCollectionAttribute [] -> ^T) (attribs2,
                                                                                                   this.WidgetAttributes,
                                                                                                   this.WidgetCollectionAttributes))

    result

Problems with that approach

1. We allocate a new array on the heap for each attribute. Note that this gives amazing ergonomics due to immutability that allows safe reuse of widgets, so it is there for a good reason!
2. Most of the time (like 99.9%) these prev allocated arrays (let attribs = this.ScalarAttributes) are short lived and apply pressure on GC
3. Cache misses during attribute diffing. E.g. In order to traverse attributes we need to follow the pointer where these attributes are located.

Proposed solution

Use a technique commonly used in high performance computing when you know that arrays are going to be mostly small.

pseudocode:

[<Struct>]
type StackArrayN<'a> = {
  Size: uint
  Item0: 'a // initialized with 'default'
  Item1: 'a
  ....
  ItemN: 'a

  PostN: 'a[] // needs to be initialized with null. e.g. 'default'
}

As you might guessed it is a bit tricky to work with and it needs to be explicitly written for each N. BUT it doesn't allocate anything before we reach N. Which makes traversal and creation very CPU cache friendly (in a happy path)

Luckily we have only a handful of use cases when it is applicable

Usecases

1. Storing Attributes in a Widget, double win for creation and diffing perf
2. Children of Widgets, usually we have at most 1 type of children attributes with some exceptions, thus StackArray1 might be a good candidate for those, although I'm not sure if we can actually do that given the recursive nature of types.
3. Any other small collections

Notes

1. Exact N needs to be determined from some existing code bases. Ideally we would run a project with a build that can capture usage profile. I would aim for p90 if it is not too big of a number. From what I saw it is probably somewhere between 5 and 10. @TimLariviere you are probably more knowledgeable in that regard.

Downsides

1. Storing Widgets will consume more memory. Assuming we use N=5(TBD) for Attributes. For 1000 Widgets (which is a pretty large number) 5 * 32 bytes (upper limit of Attribute size) * 1000 = 150Kb. Totally fine by me
2. It is super cumbersome to work with, how to iterate them with no dynamic dispatch nor allocations?
3. Sorting is probably going to be a pain

Links/Resources

• Writing high performance F# code

[Bug] Diffing ScalarAttributes can lead to IndexOutOfRange

There is a bug when we remove some attributes and add new ones from a control that is reused by Fabulous.
It is specific when the removed attributes have a higher "key" than the new "ones", and there is more new ones than old ones.

It's quite difficult to explain, but here is a repro:

let Attr1 = Attributes.define ... // Key = 0
let Attr2 = Attributes.define ... // Key = 1
let Attr3 = Attributes.define ... // Key = 2
let Attr4 = Attributes.define ... // Key = 3
let Attr5 = Attributes.define ... // Key = 4

if not model.Flag then
    Button()
        .attr4()
        .attr5()

else
    Button()
        .attr1()
        .attr2()
        .attr3()


// Start: model.Flag = false
// When switching model.Flag to true, an index out of range exception is raised inside ScalarAttributesChangesEnumerator

Add platform specific extensions

This will allow us to reduce the use of ViewRef or the need to match against the platform :

iOS : https://docs.microsoft.com/en-us/dotnet/api/xamarin.forms.platformconfiguration.iosspecific?view=xamarin-forms

Droid : https://docs.microsoft.com/en-us/dotnet/api/xamarin.forms.platformconfiguration.androidspecific.appcompat?view=xamarin-forms

[Architecture] Add lifecycle events to widget (mount, dismount, create, destroy)

Is this currently useful for a pre-alpha proof-of-concept UI?

Just curious, I have a project that would benefit from using Fabulous V2 and I'm not worried about bugs, corner cases, etc, just want something basic I can architect around and use with other devs, not in production.

Are y'all at a useful point here for that kind of early integration, with my expectation that there will be bugs and API instability? Or should I just wait?

Expose Widgets testing as part of Fabulous

In V1 we have some testing helper functions ie findViewElement and the ElementViewers to make the view and Widgets testable . https://fsprojects.github.io/Fabulous/Fabulous.XamarinForms/dev-testing.html

Are we doing the same for V2 ?

[Performance; Architecture] Research "Resumable Code" for tree diffing, thus make it streamable

Problem

Currently we calculate diff recursively for the entire tree all at once. As a part of it we use a recursive data structure

and [<Struct; RequireQualifiedAccess>] WidgetChange =
    | Added of added: WidgetAttribute
    | Removed of removed: WidgetAttribute
    | Updated of updated: struct (WidgetAttribute * WidgetDiff)
    | ReplacedBy of replacedBy: WidgetAttribute

and [<Struct; RequireQualifiedAccess>] WidgetCollectionChange =
    | Added of added: WidgetCollectionAttribute
    | Removed of removed: WidgetCollectionAttribute
    | Updated of updated: struct (WidgetCollectionAttribute * WidgetCollectionItemChange [])

and [<Struct; RequireQualifiedAccess>] WidgetCollectionItemChange =
    | Insert of widgetInserted: struct (int * Widget)
    | Replace of widgetReplaced: struct (int * Widget)
    | Update of widgetUpdated: struct (int * WidgetDiff)
    | Remove of removed: int

and [<Struct>] WidgetDiff =
    {
        ScalarChanges: ScalarChange []
        WidgetChanges: WidgetChange []  // <--- children's update
        WidgetCollectionChanges: WidgetCollectionChange [] // <--- children's update
    }

There a a couple of concerns here:

1. It allocates arrays for every tree node
2. It is all at once, thus not stoppable nor there is an opportunity to reuse memory.
3. The diff structures are very short lived, e.g. after calling ApplyDiff they immediately become garbage applying pressure on GC.

Idea to explore

Make calculation of children diff somehow lazy. Note that it potentially ok to have dynamic dispatch (virtual call) if it removes allocations. I haven't researched enough the new feature of "Resumable Code" in F# 6, to understand if that idea is even applicable to us.

But roughly it would be awesome if we can somehow express this (pseudo code):

and [<Struct>] WidgetDiff =
    {
        ScalarChanges: ScalarChange []
        WidgetChanges: LazyCollection<WidgetChange>  // <--- 
        WidgetCollectionChanges: LazyCollection<WidgetCollectionChange> // <---
    }

My intuition tells me that in this form LazyCollection has to be dynamically allocated, thus it is probably not better than status quo.

HOWEVER this is probably an opportunity to express the same recursive Diff via some other data structure. Like so

type WidgetDiff = Stream<struct (ScalarChange [] * WidgetChange[] * WidgetCollectionChange[])>

Where the Stream data structure might be "Resumable Code". If so then we can replace [] with StackArray (stack allocated collection), thus avoiding intermediate allocations. Not sure if it is possible to avoid dynamic dispatch though.

Objective

Research what are the options making diff calculation lazy, and have a PoC ideally backed by benchmark how it might work.

Note that it is probably going to be one of the last perf explorations (at least according to my intuition), because other "[Performance]" tasks probably have more predictable ROI.

[Performance, Architecture] Optimize attributes for small stack allocated values (float, int, bool etc)

Context

As it stands we have boxing/unboxing shenanigans accessing Attribute values.

simplified pseudocode:

let paddingKey = 4424 // opaque int number
let paddingValue = 3.14 // float value to store in attribute

let padding = Attribute(paddingKey,  box paddingValue) // note 'box' here

Why this is problematic:

1. During creation we allocate an object on the heap just to store a small value
2. During diffing we need to do unbox paddingAttr.Value at least twice. And probably one more time to actually apply the diff. Which is costly due to cache misses and runtime checks to call unbox

Proposal:

1. Use only a part of int or uint to store a key and reserve the rest of the bits for internal use
2. Use 2-4 bits to store a backing type of an attribute (float or int), note that it is not the same as user type. E.g. bool can be encoded as int using simply 0 | 1
3. Add two(?) additional fields to Attribute of float and uint64, actual backing type are TBD
4. Use this information for creation and diffing of Attributes to avoid heap allocation and pointer chasing.

simplified pseudocode:

// converters from user facing value to backing value
// in this example it is 'id' function
let convertFrom = fun p -> p
let convertTo = fun p -> p

let paddingAttr = defineFloatStoredAttr("padding", convertFrom, convertTo)
// will set a flag in the key that it is float backed attribute

let padding = paddingAttr.WithValue(3.14) // no boxing nor allocations
// roughly translates to
// {Key = paddingKey ||| FLOAT_ATTR_TYPE; Value = null; FloatVal = 3.14; UintVal = 0}

Now using that knowledge we can optimize diffing like so

if attr.Key &&& FLOAT_ATTR_TYPE then 
   // optimized case
   a.FloatVal = b.FloatVal
else
  // general case, what we have currently
  let comparer =  getComparer(attr.Key) // <-- cache miss
 comparer.Compare(a.Value, b.Value)  // dynamic dispatch + unboxing

Downsides

1. Declaration of Attributes will get a bit tricky, but that complexity will be a burden for us and not by the library users
2. More complexity: special cases for diffing and handling of bitwise operations
3. Attributes will occupy more space on the heap, which will impact memory footprint. Assuming that we have 1k Attributes stored on the heap (to retain UI tree) that translates into (8 bytes + 8 bytes) * 1000 = 16Kb. Probably totally fine for any reasonably scenario.

Notes

1. In Rust unions are represented differently in memory. E.g. size of the union is the size of the largest variant plus index of the variant. In F# it is as optimized, thus we have to apply some low level tricks like that.
2. It is totally possible that we should just use F# union types for Attributes instead. But as far as I know it will be hard to represent the same concept more efficiently due to limitation of overlapping fields (none allowed). Probably worth sketching out to avoid all this perf "magic".

[<Struct>]
type Attribute = 
  | FloatAttr of key: int * floatVal: float
  | BoxedAttr of key: int * boxedVal: obj

// will produce Error: "key" property is duplicated :(
// also not safe, e.g. for the same key value we can have different variants

3. Ideally, we should not exceed 64 bytes storing Attribute values in memory due to cache line size. But it seems we have plenty of space to work with: Even in the worst case scenario: 8 bytes (pointer) + 8 bytes (float) + 8 bytes (uint value) + 8 bytes (key) = 32 bytes

Links/Resources

• Writing high performance F# code

Incremental Diffing?

Hi,

I’ve been casually following the development of Fabulous V2. Awesome work, really impressive stuff. I was just taking a look at the changes in diffing, and I wanted to put something on your radar, and was wondering if it had been considered. From a brief skim of the code, it kind of looks like each control is wrapped in a kind of builder (WidgetBuidler, CollectionBuilder, etc) that is responsible for being able to calculate the different types of things that could’ve changed (ScalarAttribute, WidgetAttribute, etc). And then the diffing basically takes two Widgets, and then diffs the attributes.

One thing I was thinking about is whether something like https://fsprojects.github.io/FSharp.Data.Adaptive/ could be useful for this. The rough outline of how I was thinking it would work is that each control could have an “adaptive” version generated via https://github.com/krauthaufen/Adaptify/, and then the Fabulous DSL would be responsible for updating values via the FSharp.Data.Adaptive API (transact function). The user-facing API/DSL could stay the same as it is now. And then when the program would want to diff, it would be able to get a near-perfect incremental re-computation of whatever you would want to recompute. Unfortunately, this would be a somewhat intrusive change at the moment, since this would change how the actual data about diffs would be stored (rather than simply changing the diffing algorithm). But I figured I would mention it now while there's still a lot of things in flux.

Anyway, food for thought.

[Performance] Add benchmarks to get baseline for future perf works

The goal is to have something that we can rely on while working on perf optimizations. Thus, this has to be done first before other changes