Publish to the Edge

[kof]

Publish to the Edge

[SaraVieira]

This is honestly waaay above me, we need andras here, excited to learn about it tho

[andrasbacsai]

1. How do we connect a some-site.wstd.io to go to Cloudflare worker and then render a specific site?

Workers have wildcard domains so we can point any *.wstd.io to a worker. Rendering a site is done on the app side.

2. How do we connect a user's TLD to Cloudflare worker

Do you mean custom domains, that are not wstd.io? We can dynamically add or remove custom domains to a worker through HTTP request to CF API.

3. Do we need to deploy to a worker for each site or do we have one worker that uses some kind of cache for each site and serves it? If it's later, how do we build such a cache, and where do we store it. It has to have very quick access from the worker.

We can have one worker for all sites. For caching, there are two options:

1. CF KV Store -> The only con is that it has 60s TTL.
2. CF Durable Objects -> I used it once, does not have so much experience with it, but with this, we can define cache TTL's.
   Both are low latency, distributed storage for workers.

There is no other way until CF D1 (CF sqlite) is introduced, which is low-latency and does not require remote HTTP requests.

[kof]

Questions with CF APIs:

1. How hard would it be to switch to something else (right now I think CF is great, but you can't bet on this for 3+ years ahead)
2. How would we build it so that a self-hosted user can publish a site on their own server (whatever that is), assumed they don't want CF or edge

[andrasbacsai]

1. How hard would it be to switch to something else (right now I think CF is great, but you can't bet on this for 3+ years ahead)

These features could be easily switchable to a self-hosted/not-worker environment.

2. How would we build it so that a self-hosted user can publish a site on their own server (whatever that is), assumed they don't want CF or edge

I suggest having an env variable, something like IS_ON_EDGE or similar, and we only need to make a small function that decides where the requests should go, which data source (CF KV/durable object store vs. local SQLite).

Wdyt?

[kof]

Yeah, the question is how much work would it be right now to support workers with stores at the same time with regular node + sqlite Also workers are oss now, so self-hosted could use workers or deno deploy too

…

On Mon, Jun 20, 2022, 13:04 Andras Bacsai ***@***.***> wrote: 1. How hard would it be to switch to something else (right now I think CF is great, but you can't bet on this for 3+ years ahead) These features could be easily switchable to a self-hosted/not-worker environment. 1. How would we build it so that a self-hosted user can publish a site on their own server (whatever that is), assumed they don't want CF or edge I suggest having an env variable, something like IS_ON_EDGE or similar, and we only need to make a small function that decides where the requests should go, which data source (CF KV/durable object store vs. local SQLite). Wdyt? — Reply to this email directly, view it on GitHub <#67 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAM4WF7FWKWTQISTFPOZXLVQBF3XANCNFSM5YYRFEMA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[kof]

We can have one worker for all sites. For caching, there are two options:

So this is how it would work?

1. worker receives a request from some dns provider
2. goes to the durable objects store to fetch the cached data for the project
3. If data cache - renders the site with remix and data
4. if not found fetches data over API service, renders, puts the data into durable object store

Now

1. How fast can it fetch lets say 300KB from store when its cached to start rendering?
2. The assumption is that the worker has all the code already ready? I think when in the future user can provide a code component - this won't be easily possible unless the code component will be stored together with the rest of the data in the store
3. Wouldn't it be MUCH faster if we were able to deploy each worker individually together with data, so that it doesn't have to fetch from store? (I literally don't know where CF workers store the code that is supposed to run and what is possible to store there)

[andrasbacsai]

I've done a small research on this.

1. Tested with 522KB data stored in a KV store. KV responses are between ~18-50ms. (Durable Objects has 130KB size limitations - I was not aware of this)
2. Yes, the worker should have all the code. This is just quickly thinking, but maybe we can store the component code in the KV store as well and return on demand? Not sure if could work.
3. KV data stored in centralized data centers, and if requested, cached on edge.

About the minimum 60s cache TTL, we can get rid of it, by versioning the data. So like versioning each deployment of a website created in webstudio.

[kof]

I think we can close this one in favor of the new 2