Admin -> General Improvements: a/b/c/d/e/f/etc... testing spec

[bcardarella]

Admin -> General Improvements: a/b/c/d/e/f/etc... testing spec

[leandrocp]

Data Structure

As https://developers.google.com/search/docs/crawling-indexing/website-testing says:

Once you've concluded the test, update your site with the desired content variation(s) and remove all elements of the test as soon as possible, such as alternate URLs or testing scripts and markup. If we discover a site running an experiment for an unnecessarily long time, we may interpret this as an attempt to deceive search engines and take action accordingly.

We understand that variants are a way to temporally overwrite pieces of a page, most likely the template, so our intention is to add this field to the Page schema:

    embeds_many :variants, Variant do
      field :name, :string
      field :weight, :integer
      field :template, :string
    end

New fields may be added in the future but the first version will allow overwriting only templates.

On /admin we'll present each variant under a tab so users can create and delete as they need, but eventually they should apply the changes from a variant back to the main record. This design matches with page snapshots mechanism where we store the page content when it's published, including the variants, also it avoids data migration.

Analytics with Plausible

Plausible does recommend using https://plausible.io/docs/custom-pageview-props for tracking variants which will require
POSTing custom pageview properties from within Beacon, see BeaconCMS/beacon#291 for more info.
Note that Plausible is not required for A/B testing, the design will allow using other providers but Plausible will be the first implementation and should be supported by Beacon.

[leandrocp]

Internal discussion https://dockyard.slack.com/archives/C02NCP1799C/p1689867735078919

[bcardarella]

with Beacon being OSS I'd prefer if we didn't link into private slack chats. Can we at least copy that discussion here?

[leandrocp]

with Beacon being OSS I'd prefer if we didn't link into private slack chats. Can we at least copy that discussion here?

Sure!