roblox/tarmac

Command line tool to manage Roblox assets

Tarmac

---

Heads up! This repository was recently migrated from the rojo-rbx org to the Roblox org. Not much else has changed.

Tarmac is a resource compiler and asset manager for Roblox projects. It helps enable hermetic place builds when used with tools like Rojo.

Tarmac is inspired by projects like Webpack that make it easy to reference assets from code.

Installation

Installing with Foreman

The recommended way to install Tarmac is with Foreman.

Add an entry to the [tools] section of your foreman.toml file:

tarmac = { source = "Roblox/tarmac", version = "0.7.0" }

Installing from GitHub Releases

Pre-built binaries are available for 64-bit Windows, macOS, and Linux from the GitHub releases page.

Installing from Source

Tarmac requires Rust 1.39.0 or newer to build.

You can build the latest release of Tarmac from crates.io:

cargo install tarmac

or build the latest work from the master branch:

cargo install --git https://github.com/Roblox/tarmac

Basic Example

The examples folder contains small, working projects using different features from Tarmac.

Tarmac is configured by a TOML file in the root of a project named tarmac.toml. Tarmac uses this file to determine where to look for assets and what to do with them.

To tell Tarmac to manage PNG files in a folder named assets, you can use:

name = "basic-tarmac-example"

# Most projects will define some 'inputs'.
# This tells Tarmac where to find assets that we'll use in our game.
[[inputs]]
glob = "assets/**/*.png"
codegen = true
codegen-path = "src/assets.lua"
codegen-base-path = "assets"

Run tarmac sync --target roblox to have Tarmac upload any new or updated assets that your project depends on. You may need to pass a .ROBLOSECURITY cookie explicitly via the --auth argument.

Tarmac will generate Lua code in src/assets.lua that looks something like this:

-- This file was @generated by Tarmac. It is not intended for manual editing.
return {
	foo = {
		bar = "rbxassetid://238549023",
		baz = "rbxassetid://238549024",
	}
}

These files will be turned into ModuleScript instances by tools like Rojo. From there, it's easy to load this module and reference the assets within:

local assets = require(script.Parent.assets)

local decal = Instance.new("Decal")
decal.Texture = assets.foo.bar

Command Line Interface

For more information, run tarmac --help.

Global Options

These options can be specified alongside any subcommands and are all optional.

• --help, -h
  • Prints help information about Tarmac and exits.
• --version, -V
  • Prints version information about Tarmac and exits.
• --auth <cookie>
  • Explicitly defines the authentication cookie Tarmac should use to communicate with Roblox.
  • If not specified, Tarmac will attempt to locate one from the local system.
• --verbose, -v
  • Enables more verbose logging. Can be specified up to three times, which will increase verbosity further.

tarmac sync

Detects changes to assets in the local project and attempts to synchronize them with an external service, like the Roblox cloud.

Usage:

tarmac sync [<config-path>] \
	--target <roblox|debug|none>
	--retry <number>
	--retry-delay <60>

To sync the project in your current working directory with the Roblox cloud, use:

tarmac sync --target roblox

To validate that all inputs are already synced, use the none target:

tarmac sync --target none

When tarmac gets rate limited while syncing to Roblox, use the --retry argument to automatically attempt to re-upload. This will tell tarmac how many times it can attempt to re-upload each asset. The --retry-delay sets the number of seconds to wait between each attempt.

tarmac sync --target roblox --retry 3

tarmac upload-image

Uploads a single image as a decal and prints the ID of the resulting image asset to stdout.

Usage:

tarmac upload-image <image-path> \
	--name <asset-name> \
	--description <asset-description>

Example:

tarmac upload-image foo.png --name "Foo" --description "Foo is a placeholder name."

tarmac asset-list

Outputs a list of all of the asset IDs referenced by the project. Each ID is separated by a newline.

Usage:

tarmac asset-list [<config-path>] \
	--output <file-path>

Example:

tarmac asset-list --output asset-list.txt

tarmac create-cache-map

Creates a mapping from asset IDs back to their source files. Also downloads packaged images to a given folder, generating links to those assets as well.

The mapping file is JSON.

Usage:

tarmac create-cache-map [<config-path>] \
	--index-file <file-path> \
	--cache-dir <cache-folder>

Example:

tarmac create-cache-map --index-file assets.json --cache-dir asset-cache

tarmac help

Prints help information about Tarmac itself, or the given subcommand.

Usage:

tarmac help [<subcommand>]

Project Format

• name, string
  • The name of the Tarmac project, used in logging and error reporting.
• max-spritesheet-size, (int, int), optional
  • The maximum spritesheet size that Tarmac should use. Defaults to (1024, 1024), the maximum image size supported by Roblox.
• asset-cache-path, path, optional
  • If defined, Tarmac will re-download uploaded images to a local folder at the given path. Files in this folder not associated with assets in the project will be deleted.
• asset-list-path, path, optional
  • If defined, Tarmac will write a list of asset URLs used by the project to the given file. One URL is printed per line.
• upload-to-group-id, int, optional
  • If defined, Tarmac will attempt to upload all assets to the given Roblox Group. If unable, syncing will fail.
• inputs, list<InputConfig>, optional
  • A list of inputs that Tarmac will process.
• includes, list<path>, optional
  • A list of additional paths to search recursively for additional projects in. The inputs from discovered projects will be merged into this project, and other settings ignored.
  • When a tarmac.toml file is found, Tarmac will include it and its includes and stop traversing that directory.

InputConfig

• glob, string
  • A path glob that should include any files for this input group.
  • Tarmac uses the globset library and supports any syntax it supports.
• codegen, bool, optional
  • Whether Tarmac should generate Lua code for the assets contained in this input group. Defaults to false.
• codegen-path, path, optional
  • If defined and codegen is true, Tarmac will merge all generated Lua code for this input group into a single file.
• codegen-base-path, path, optional
  • Defines the base path for generating Lua code when codegen-path is also defined. Defaults to the directory containing tarmac.toml.

License

Tarmac is available under the MIT license. See LICENSE.txt for details.

tarmac's Issues

Support codegen into larger units

Right now, the plan is to imitate the semantics that tools in the JS ecosystem have for dealing with assets, where we can pretend to import asset files like they're modules:

local CancelIcon = require(Assets.CancelIcon)
local ConfirmIcon = require(Assets.ConfirmIcon)

cancelBox.Texture = CancelIcon
confirmBox.Texture = ConfirmIcon

We do this by generating Lua files next to each asset that we want to be able to import:

Assets/
├─ CancelIcon.png
├─ CancelIcon.lua
├─ ConfirmIcon.png
└─ ConfirmIcon.lua

Another approach would might want to support is grouping together the generated code into one or more batches of files containing assets keyed by name:

local Assets = require(Assets)

cancelBox.Texture = Assets.CancelIcon
confirmBox.Texture = Assets.ConfirmIcon

With a corresponding directory structure:

Assets/
├─ CancelIcon.png
├─ ConfirmIcon.png
└─ init.lua

The configuration for a project like this might include this bit in its tarmac.toml:

[[inputs]]
glob = "Assets/**/*.png"
codegen = { type = "batch-asset-url", output = "Assets/init.lua" }

--require-up-to-date option

It would be useful to have a flag that tries to run a sync operation and fails if anything needs to be synced. This would be used in CI environments to ensure users have deployed any assets that have changed.

9-slice data for images

It's common to use 9-slice images when building Roblox UI. Slice coordinates need to be scaled to account for DPI.

It'd be neat to be able to associate some metadata with each image about whether they're intended to be sliced and how, and have Tarmac automatically scale up and encode that data into the generated code.

Spritesheet packing overlaps images

We use Sheep for packing spritesheets right now, and as such hit a bug with their maxrects packer.

We should consider one of:

• Debugging and contributing a fix to Sheep
• Writing our own spritesheet packing library
• Some middleground, like implementing our own packing algorithm but still using Sheep

Option to create a .d.ts file alongside .lua

It would be useful as a developer who uses roblox-ts if the generated .lua files had an associated .d.ts file with it too.

For the files that are just return "rbxassetid://?????", this should be as easy as export = string or something.

Support codegen that automatically picks correct DPI scale

We're going to support high DPI images for users that manually import them for sure at first. It'll look like this:

local logo = require(Assets.logo)
local logo2x = require(Assets["logo@2x"])

if isHighDpi then
    label.Image = logo2x
else
    label.Image = logo
end

It's then up to the user to pick the correct asset for the current resolution scale. That isn't very fun, and I think Tarmac is in a good position to help.

One approach would be to write a new codegen mode that returns images as a thunk taking the DPI scale, used something like this:

local logo = require(Assets.logo)

-- This isn't a real property yet, but core scripts have access to something similar
label.Image = logo(screenGui.ResolutionScale)

Local development plan

Currently, Tarmac supports uploading to Roblox.com only. For fast local iteration, it would be useful for Tarmac to copy your assets into your local Roblox Studio content folder temporarily.

To do that, we could introduce a new value for --target named something like local-studio:

tarmac sync --target local-studio

This command would:

1. Locate your local Roblox Studio install using roblox-install
2. Perform a sync. For any files that have changed, copy them to $INSTALL/content/.tarmac/<hash>.
3. Perform codegen, using a path like rbxasset://.tarmac/<hash> for any assets that have no asset ID.

AssetNames collide with included projects

Currently, if I include two projects that both have assets/foo.png in them, one of them will overwrite the other. Tarmac assumes that AssetName is unique across an entire project, but does not correctly prefix paths from other projects currently.

Handle Roblox API 429 response

Currently, Tarmac will error when it hits a 429 error response from Roblox with the following error:
thread 'main' panicked at 'not yet implemented: Handle bad responses', src\roblox_web_api.rs:75:13

https://github.com/rojo-rbx/tarmac/blob/2448e694ce8fe6461e4818ee000b001dd1eb95b5/src/roblox_web_api.rs#L75

This is not preferred as it causes tarmac to immediately exit and not create the manifest / cache previous images it has uploaded (so the user can wait and reupload at a later point)

Additional debug with headers responded:
429-tarmac.txt

Inputs from directories above root project cause panics

Due to the way that Tarmac constructs the AssetName type, the folder containing tarmac.toml must be an ancestor of any included inputs.

High DPI unpacked images

#31 was a partial solution to the high DPI problem, and provided new a new code generation style for assets that have high DPI variants available (foo.png, [email protected], etc). However, for images where packable is not set, each high DPI variant still generates code independently.

Tarmac should merge these inputs and generate one file containing all of the high DPI variants.

Option to alpha bleed images

This might be out of scope for Tarmac, but there's never a case where I want an image with transparency uploaded but not alpha bled.

@Quenty released an article titled "Fixing images in Roblox UI". In this article, he explains how due to a bug, images with transparency form ugly pixels around them. The solution is to change the RGB values of the alpha to coincide with the border.

XAXA released a tool for this that I use regularly. It would be very useful if Tarmac could automate this process.

An extra period appears in the file name of the asset modules

As you can see below, An extra period appears in the file name of the asset modules.

Steps to reproduce

1. Set codegen to asset-url inside tarmac.toml
2. Run tarmac sync --target roblox
3. See the folder where the asset files are located

Alpha bleed unpacked images

The support I landed for alpha bleeding only applies to spritesheets since we don't make any attempts to decode single images.

We should apply this transform to individual images as well.

Add dotenv support for loading environment variables

Update asset references in models

We have a clear plan for how to handle assets that are used programmatically, but that's only a small fraction of the way that assets are used in Roblox.

It would be a good idea to build a tool that can traverse models and places to update their asset references when they need to be updated. This might not fit into Tarmac, but Tarmac should be able to produce the data needed to build a tool that can do this.

Handle Roblox API invalid name

Repro: Upload a file named Panda Fox.png, with

tarmac upload-image images/image.png -- name "Panda Fox.png" -vvv

You will get an output similar to this:

Roblox is responding by telling Tarmac that the name is inappropriate, however, tarmac will immediately error, thus cancelling any work it has done

This is problematic as

• It is not a graceful way to handle the error
• All previous images uploaded must be reuploaded (very time consuming)

Option to populate a cache directory

Some consumers of Tarmac, like Roblox, want to have local copies of each uploaded asset, indexed by asset ID.

Tarmac should support an additional flag or separate command to create an output cache out of whatever the current state is by downloading images from Roblox:

# As a flag
tarmac sync --target roblox --cache-dir my-cache/

# As a separate command
tarmac sync --target roblox
tarmac make-cache my-cache/

Upload models

We have tarmac upload-image right now. As a holdover for other tools, it might be useful to introduce tarmac upload-model to upload models.

Unmaintained

Tarmac is a great tool for any Roblox developer, it's a shame that the "official" version doesn't work. For anyone who's looking for a working version of tarmac, you can find a working fork at https://github.com/rojo-rbx/tarmac, it's even in the rojo-rbx organization, which was the original organization that started tarmac.
This issue should probably be pinned unless work is done to fix tarmac, or the repository may as well just be archived and the fork advertised in the README. Otherwise I'm sure a lot of people will end up not using tarmac due to it seeming completely broken.

Incremental sync (watch mode)

Once we have a workflow for local development, allowing Tarmac to watch for files to be changed, added, or deleted would be useful when paired with Rojo.

I propose:

tarmac sync --target local-studio --watch

Add retry behavior to work around rate limiting

The Roblox upload endpoint will rate-limit, which can cause problems when running Tarmac on large sets of assets. This can be especially troublesome when integrating with CI.

The upload API returns a 429 error when rate limiting occurs, and the budget is per-minute. We should be able to provide Tarmac an optional retry count, so that when it receives a 429 it will wait for one minute, then retry with the last file it was attempting to upload.

Add progress bar or similar

There's a few ways this could be done nicely. For example, the current message could be changed like this:
Old: [INFO] Uploading {} to Roblox
New: [INFO] Uploading {} to Roblox (5/250)

Or, we could use a terminal progress bar, like any of the ones from here
I'm happy to submit a PR for this myself. Just waiting for approval from a maintainer so I don't do useless work.

Break apart `sync` command

tarmac sync is very complex. It's effectively the only command in Tarmac.

It would be useful to create smaller commands that contain each auxiliary piece of functionality, which should help us manage complexity.

• tarmac sync will just sync assets to the Roblox cloud
  • The --target parameter should be removed.
• tarmac dev will just sync assets to a local installation to work on in development mode
• tarmac generate-cache, pitched in #15, will generate a cache from the current state
• tarmac create-path-map (#38) will write a map of all assets managed by Tarmac
• tarmac verify-synced will validate that the project is in sync with the Roblox cloud
  • Can we verify moderation status in this subcommand?
  • Can we verify that assets are uploaded to the correct group here?

Support uploading assets to groups

Instead of having images scattered among developer's accounts, it would be useful to require that all images are uploaded to a given group. This makes it easier to take inventory of them, protect them from moderation actions, and monitor them for problems.

Alpha bleeding

In order to improve behavior of images when they're resized, we should run an alpha bleed step on all images.

Improve uploaded spritesheet names

Currently, all spritesheets get uploaded as spritesheet.png. We should include a descriptive name, like from the project name or a name added to the input group. We could also include information like the DPI scale of the input group.

Documentation

Currently, Tarmac is pretty unstable.

Once it's more stable, we should build a documentation site for it to help users adopt it.

Support tagging inputs with more specific type information

Some file types can be uploaded to multiple different kinds of asset.

Model files, for example, have the rbxm extension and can represent models, plugins, packages, animations, and probably more. In order for Tarmac to upload these asset types correctly, we'll need to explicitly tell it what a group of models represent.

Early-abort for spritesheets does not always work

When testing Tarmac against UIBlox, an internal Roblox project, it seems Tarmac always thinks it needs to reupload the generated spritesheets.