rustshop/flakebox

Flakebox is to your Rust project dev environment, what NixOS is to your OS, or home-manager to your home directory.

RustShop

RustShop is an attempt at building a template and utilities to help quickly set up and manage a production grade cloud-based system.

The core target audience are Rust developers that are familiar or willing to learn Nix and have some DevOps interest. However it might be useful for wide group of people:

• rustshop project is a bit like a tutorial for setting up Nix&Rust shop building a AWS/k8s stack.
• rustshop binary is well integrated tool to make working with like terraform, kubectl, kops, helm and other utilities much more convenient (in particular when dealing with multiple accounts/profiles/clusters).
• rustshop is trying to build a well oiled, flexible and extremely powerful IaaC, gitops based cloud software shop template that can be a source of inspiration. A system like this must:
  • Store both all infra and application code under revision control, gitops style.
  • Utilize a customizable (also stored under revision control) merge queue bot, and a CI/CD pipeline.
  • Implement well integrated solutions for all cutting edge best practices.

While the project aim is large in scope, the core technical philosophy is about minimalism and efficiency:

• Pick few powerful and universal tools and stick to them (Rust + Nix).
• Integrate things well, but keep them extremely modular to enable change.
• Don't be afraid to implement smaller, easier to customize and right-sized solutions from scratch, when the mainstream does not fit your goals.

Read more about our technical philosophy.

The basic technologies used are:

• Rust (for all tools, and eventually example applications)
• Nix (for all things build & dev-env & glue)
• AWS (for cloud hosting)
• Terraform (for infrastructure automation)
• Kubernetes (for orchestration)

See Tech Stack for more details. Subscribe to Status Updates to track project progress.

Participating

For fun and to have real-like general direction, RustShop (capitalized) is a pretend (fake) business. Hopefully it will keep the goal and the direction be focused on solving real-like problems.

See Welcome to Rust Shop for more information about the idea and the project.

Welcome to RustShop!

Tired of not being able to find a Rust job? Join RustShop - our ambitious innovative fake company. Pretend you are delivering business value using Rust in your free time for no pay or benefits!

Links

• Onboarding - setting your dev env to work with rustshop tooling
• Infra Bootstrapping - setting up new AWS infrastructure, and bootstrapping basic services using rustshop bootstrap functionality
• Standalone usage - using rustshop binary in an existing org infrastructure
• Tech Stack - general information about technologies used.
• Technical philosophy

flakebox's Issues

Can't build example on aarch64-linux

Running through the example and I got to this part:

flake.nix

{
  description = "A very basic flake";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.05";

  flakebox = {
    url = "github:rustshop/flakebox";
    inputs.nixpkgs.follows = "nixpkgs";
  };

    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flakebox, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        flakeboxLib = flakebox.lib.${system} { };
      in
      {
        devShells = flakeboxLib.mkShells {
          packages = [ ];
        };
      });
}

➜  flakebox-test git:(master) ✗ nix run nixpkgs#nix flake check
warning: Git tree '/Users/tony/Dev/Rust/learning/flakebox-test' is dirty
error: a 'aarch64-darwin' with features {} is required to build '/nix/store/ay9qys7hlr73j73h84s5saracxclck59-flakebox-flakebox-ci-yaml-gen.drv', but I am a 'aarch64-linux' with features {benchmark, big-parallel, gccarch-armv8-a, kvm, nixos-test}

The instructions say: it means your nix is too old. Use nix run nixpkgs#nix flake check instead to run a newest version. With Nix even tools that are not currently installed are easy to use.

But I am using that, so I'm not sure what else could be the problem.

question: how do you specify abi when cross compiling

I want to cross compile to target:
aarch64-unknown-linux-musl

But using when a follow the get started tutorial and do a build with:

nix build .#aarch64-linux.dev.hello-flakebox a get a file of aarch64-unknown-linux-gnu

How do i declare the -musl part?

Cross compilation dependencies

Hi! love what you're building here.

I'm using this to achieve cross compilation from my x86 to a raspberry pi 4 (aarch64). While building works great, when I copy over the executable to my raspberry pi (also running nixos) and run it I get no such file or directory which indicates a problem with linking (according to my google searching).

Here is the file of the output:

❯ file result/bin/solver
result/bin/solver: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/5gdh4mp8rwliq4s33gwcpwzqvsb2xpzr-glibc-aarch64-unknown-linux-gnu-2.38-23/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.10.0, with debug_info, not stripped

And running ldd on the raspberry pi:

❯ ldd solver 
	linux-vdso.so.1 (0x0000007fb649c000)
	libc.so.6 => /nix/store/ilr14j6gzr9l651yb5w367y00cs6pflb-glibc-2.37-45/lib/libc.so.6 (0x0000007fb5ae0000)
	libm.so.6 => /nix/store/ilr14j6gzr9l651yb5w367y00cs6pflb-glibc-2.37-45/lib/libm.so.6 (0x0000007fb5a30000)
	libgcc_s.so.1 => /nix/store/x2f8wiip66lxf6zynscszpm4d990q098-xgcc-12.2.0-libgcc/lib/libgcc_s.so.1 (0x0000007fb59f0000)
	/nix/store/5gdh4mp8rwliq4s33gwcpwzqvsb2xpzr-glibc-aarch64-unknown-linux-gnu-2.38-23/lib/ld-linux-aarch64.so.1 => /nix/store/ilr14j6gzr9l651yb5w367y00cs6pflb-glibc-2.37-45/lib/ld-linux-aarch64.so.1 (0x0000007fb645f000)
❯ objdump -j .interp -s ./solver

./solver:     file format elf64-littleaarch64

Contents of section .interp:
 0270 2f6e6978 2f73746f 72652f35 67646834  /nix/store/5gdh4
 0280 6d703872 776c6971 34733333 67776370  mp8rwliq4s33gwcp
 0290 777a7176 73623278 707a722d 676c6962  wzqvsb2xpzr-glib
 02a0 632d6161 72636836 342d756e 6b6e6f77  c-aarch64-unknow
 02b0 6e2d6c69 6e75782d 676e752d 322e3338  n-linux-gnu-2.38
 02c0 2d32332f 6c69622f 6c642d6c 696e7578  -23/lib/ld-linux
 02d0 2d616172 63683634 2e736f2e 3100      -aarch64.so.1.  

While none of the dependencies are missing, it appears the interpreter is set to /nix/store/5gdh4mp8rwliq4s33gwcpwzqvsb2xpzr-glibc-aarch64-unknown-linux-gnu-2.38-23/lib/ld-linux-aarch64.so.1 (which is on my x86 system), and while ldd states this should link to /nix/store/ilr14j6gzr9l651yb5w367y00cs6pflb-glibc-2.37-45/lib/ld-linux-aarch64.so.1 it doesn't appear to do that when it runs.. Honestly I'm a bit out of my depth on this.

I'm wondering if anyone else has run into this and how it was resolved? It looks like statically linking is one option, but that is only supported by musl not glibc, which flakebox doesn't appear to support as the target is set to:

  aarch64-linux = mkClangToolchain {
    target = "aarch64-unknown-linux-gnu";
    ...

I suppose another option would be to ensure that /nix/store/5gdh4mp8rwliq4s33gwcpwzqvsb2xpzr-glibc-aarch64-unknown-linux-gnu-2.38-23/lib/ld-linux-aarch64.so.1 is indeed present on the raspberry pi system, but I'm not really sure how to do that on a continuous basis, I don't want to have to rebuild the system when I deploy my rust application to it

I also see that the intepreter is set to gnu-2.38-23 but links to 2.37-45 but if it were just a version change, why would it link to that in the first place?

Port Android and iOS xcompilation from Fedimint

Try out enhancing craneLib via new scope.

ipetkov/crane#402 (comment)

Prompt & create cargo build profile `ci`

Rethink filters

I'd like filtering source code to be as easy as possible.

Howerver currently the flakeboxLib.filter.filterSubdirs seems buggy for deeply nested dirs, and I'm not even sure what's the best API should be. Need to rethink.

Add granularity over defaults of `flakebox install`

Hey there, really cool project!

I have a few nitpicks with the opinionated defaults of flakebox install:

• I might be using Gitlab and not requiring the .github CI
• I don't like git hooks and would prefer to disable them
• I may have a custom justfile and want to keep it over the flakebox default one

I'm asking if it's possible to make these settings opt-in via command line arguments.
Adding a prompt selection with y/n questions is also an option.

Include all applicable tips/optimizations from "My ideal Rust workflow"

https://fasterthanli.me/articles/my-ideal-rust-workflow

Lots of them we have already anyway, but worth looking through and making sure there's nothing good we're missing out on.

Backport `CARGO_DENY_COMPILATION` from Fedimint

Include a method to set it without room for typos.

missing `lib.fileset`

I'm following along the tutorial to set up a dev shell and got as far as

{
  description = "A very basic flake";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.05";

    flakebox = {
      url = "github:rustshop/flakebox";
      inputs.nixpkgs.follows = "nixpkgs";
    };

    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flakebox, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        flakeboxLib = flakebox.lib.${system} { };
      in
      {
        devShells = flakeboxLib.mkShells {
          packages = [ ];
        };
      });
}

on running nix run nixpkgs#nix flake check, i encounter the following error:

error:
       … while checking flake output 'devShells'

         at /nix/store/qkig73szmrhgp0qhncxy5vb36lw2g3jj-source/lib.nix:39:17:

           38|               {
           39|                 ${key} = (attrs.${key} or { })
             |                 ^
           40|                   // { ${system} = ret.${key}; };

       … while checking the derivation 'devShells.x86_64-linux.default'

         at /nix/store/0c36hfc4mdihdf87vm3kg14013vnzwi8-source/lib/mkShells.nix:22:3:

           21|   lint = mkLintShell { packages = lintPackages; };
           22|   default = mkDevShell cleanedArgs;
             |   ^
           23|

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: attribute 'fileset' missing

       at /nix/store/0c36hfc4mdihdf87vm3kg14013vnzwi8-source/lib/flakeboxBin.nix:5:8:

            4| let
            5|   fs = lib.fileset;
             |        ^
            6|   fileSet = fs.unions [

Support for including rules from other justfile files

casey/just#1470

Use `typos --force-exclude`

crate-ci/typos#837