When a project needs to store large volumes of files managing their storage and house keeping becomes a challenge. When a project has multiple different use cases all storing files in different places with different foldering conventions things can get really messy and can make for some very difficult migration problems.
Note: The easiest expression of this problem is with file storage, however going forwards I'll use the term "assets" as it allows for a larger vision of what we could do with this.
Abstracting storage of assets from the application allows developers to quickly receive, store and retrieve files without having to do any plumbing.
The name
I'm suggesting we call these things AssetCatalogueProvider. Including the work catalogue reminds us that the provider must also support house keeping and listing functions - not just storage and retrieval of single assets.
Storing an asset
$provider = AssetCatalogueProvider::getProvider($category);
$token = $provider->createAssetFromFile($filePath, $name); // Name and category being optional
$token = $provider->createAssetFromStream($handle, $name); // Name not optional?
Here there is a deviance from the normal provider pattern in that we pass a category to getProvider() e.g. "Agent Logos". I imagine there would be a mapping to map categories to concrete provider implementations - so you could for instance use a file based provider for one use case but an S3 provider for another. The concrete implementations could be then be swapped in the application configuration.
The token returned would information that could recover both the category and provider type to allow for staged migrations. A token would be a base64 encoded string which unencoded would look like:
PROVIDER_TYPE|CATEGORY|SOMEINTERNALDETAIL
e.g.
LocalFiles|Agent Logos|pro/property-is-us.png
Retrieving and using an asset
$asset = AssetCatalogueProvider::getAsset($token);
$asset->writeToFile("/my/local/path/".$asset->name);
$asset->getStream();
$asset->getUrl();Note that we don't ask for a provider - the provider we need is in the token, not the current preference for a category. This allows for elegant migration of providers without having to move wholesale (which when terabytes of data are involved is not straightforward).
getAsset() returns an Asset object which will be sub classed by the appropriate provider.
Deleting an asset
$asset = AssetCatalogueProvider::getAsset($token);
$asset->delete();
Migrating assets
When introducing a new asset catalogue provider for a given category existing tokens should still be supported. It's likely however that there will be a commercial motivation to retire the old catalogue so migration must be considered.
To migration from one catalogue to another:
$asset = AssetCatalogueProvider::getAsset($token);
// Get the new catalogue provider
$provider = AssetCatalogueProvider::getProvider($category);
// Migrate the old asset into the new catalogue.
$newToken = $provider->migrateAsset($asset);
// Delete the old asset
$asset->delete();
// Store the new token
Housekeeping
Still need to give some thought to how we keep a handle on total space being used, removal of orphans etc.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent
