tfpk/macrokata

Learn Macros In Rust

MacroKata

Welcome to MacroKata, a set of exercises which you can use to learn how to write macros in Rust. When completing each task, there are three goals:

• Get your code to compile without warnings or errors.
• Get your code to "work correctly" (i.e. produce the same output)
• Importantly, generate the same code as what the sample solution does.

You should complete the kata in order, as they increase in difficulty, and depend on previous kata.

This set of exercises is written for people who have already spent some time programming in Rust. Before completing this, work through a Rust tutorial and build some small programs yourself.

Getting Started

Clone this repository:

$ git clone https://www.github.com/tfpk/macrokata/

You will also need to install the Rust "nightly" toolchain, so that we can show expanded macros:

$ rustup toolchain install nightly

Next, install cargo-expand:

$ cargo install cargo-expand

Build the main binary provided with this repo:

$ cargo build --bin macrokata

You can find the first kata (my_first_macro) inside exercises/01_my_first_macro. Read the first chapter of the book and get started by editing the main.rs file.

To compare your expanded code to the "goal", use the test subcommand:

$ cargo run -- test 01_my_first_macro

You can run your own code as follows:

$ cargo run --bin 01_my_first_macro

How To Learn About Procedural Macros

I was originally planning to expand macrokata into discussing procedural macros as well. As I was researching that, I found dtolnay's superlative Proc Macro Workshop. Jon Gjengset's video on proc-macros is also a phenomenal resource (despite its length).

I've put my attempt to write something like that on hold because I think the above is better in every way. Do file an issue if there's something that we could do here to complement that workshop though.

macrokata's Issues

Exercise 6 typo?

Exercise 6 doesn't seem to compile unless the main() function is modified.

Original main:

Compilable main:

Exercise 9 doesn't compile

Exercise 9 doesn't compile with the given solution, not sure if intended as the book doesn't mention this.

error[E0600]: cannot apply unary operator `-` to type `u32`
  --> exercises/09_ambiguity_and_ordering/main.rs:50:36
   |
50 |         NumberType::NegativeNumber(-$negative)
   |                                    ^^^^^^^^^^ cannot apply unary operator `-`
...
70 |     get_number_type!(-5).show();
   |     -------------------- in this macro invocation
   |
   = note: unsigned values cannot be negated
   = note: this error originates in the macro `get_number_type` (in Nightly builds, run with -Z macro-backtrace for more info)

For more information about this error, try `rustc --explain E0600`.
error: could not compile `macrokata` due to previous error

Exercise 12 idea is vague to grasp

I'm not sure I've got everything correctly here --- it's not easy to reason on something you're still learning --- so bear with me, pls. X)

I decided to try to solve the exercise without using material from the relevant page to see what would happen and what problems do we actually shoot.

So the solution is straightforward af.

macro_rules! coord {
    ($x:expr, $y:expr) => {super::Coordinate{x: $x, y: $y}};
    ($x:expr, $y:expr, $z:expr) => {super::third_dimension::Coordinate{
        x: $x, y: $y, z: $z
    }};
    ($x:expr, $y:expr, $z:expr, $t:expr) => {fourth_dimension::Coordinate{
        x: $x, y: $y, z: $z, t: $t
    }};
}

Beautiful makrokata tells me that I kinda ignored the task, which is fairly expected.
...

The diff is:

@@ -1,5 +1,5 @@
 fn main() {
-    let four_dim = fourth_dimension::Coordinate {
+    let four_dim = crate::fourth_dimension::Coordinate {
         x: 1,
         y: 2,
         z: 3,

Now let's see how miserably we let the exercise down with this awful short-cut.

$ cargo run --bin 12_hygienic_macros
    Finished dev [unoptimized + debuginfo] target(s) in 0.10s
     Running `/home/[REDACTED]/Desktop/macrokata/target/debug/12_hygienic_macros`
Coordinate { x: 1, y: 2 }

This irony bring no offense with it, just to skim the story. I have no idea atm how to remediate the thing, and does it even needs remediation. I'm not really getting this last segment of macrokata yet, but feel like something not entirely right here. So my main question: was this intended, did I something plainly wrong?

Add license

Hello! First of all, thanks for making this. I just went through the exercises and they were very valuable to me! This is the best macro teaching material I've come across thus far.

I think it would be great to add a license to this project.

For example, I hang out on exercism.org quite a bit. I think this project could be added to the Rust track over there mostly as-is, under a new concept for macros. I'm wondering if the creators of these exercises are open to them being shared in such a way. Adding a license could answer such questions.

Add rustup setting to nightly in README

There's lack of rustup default nightly step in 'Getting Started ' section of README, which result in inability of cargo to find certain dependencies (such as clap in 4.0.17 version). After execution of mentioned command everything works fine.

Make Positive Literals Match

Perhaps unrelated, but would it be better for positive literal to be

( $(+)?$positive: literal ) => {...}

So the macro works for +5 as well as 5?

Originally posted by @dong-zh in #9 (comment)

Task 10 is "unsatisfying"

(Spoilers for task 10).

As highlighted in this reddit thread, the solution to Task 10 is not well motivated. It'd be nice if the task made it more "obvious" that the macro needs to call another macro.

Exercise 7 main function different from solution

The exercise has a trailing comma in the key values, while the solution does not.

Solution:

fn main() {
    let value = "my_string";
    let my_hashmap = hashmap!(
        "hash" => "map",
        "Key" => value // <- no comma
    );

    print_hashmap(&my_hashmap);
}

Exercise:

fn main() {
    let value = "my_string";
    let my_hashmap = hashmap!(
        "hash" => "map",
        "Key" => value, // <- comma
    );

    print_hashmap(&my_hashmap);
}

Fails to build on latest rust nightly

The latest rust nightly build does not have the feature proc_macro_span_shrink, and the proc_macro2 crate has updated and does not require that feature anymore. I removed and regenerated Cargo.lock to use macrokata, but this is just a heads-up, other people might run into this issue as well.

Task 11 Doesn't Actually Require Macro Recursion

I'll be thinking about a different task 11 that forces using macro recursion (or making it clearer why it requires using braces... maybe adding in a "curry_log" function or something so people can see the currying working.

cargo run -- test 01_my_first_macro fails

I've got below error for comparing the macro I built with the solution.

cargo run -- test 01_my_first_macro

vscode ➜ ~/macrokata (main) $ cargo run -- test 01_my_first_macro
    Finished dev [unoptimized + debuginfo] target(s) in 0.12s
     Running `target/debug/macrokata test 01_my_first_macro`
Got some errors when expanding the macro:

    Checking macrokata v0.3.1 (/home/vscode/macrokata)
    Finished dev [unoptimized + debuginfo] target(s) in 0.15s

Surprisingly, cargo expand, if run seperately, shows me the expanded macro without any errors:
cargo run -- test 01_my_first_macro

vscode ➜ ~/macrokata (main) $ cargo expand --color "always" --bin 01_my_first_macro
    Checking macrokata v0.3.1 (/home/vscode/macrokata)
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s

#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2021::*;
#[macro_use]
extern crate std;
fn show_output() {
    {
        ::std::io::_print(format_args!("I should appear as the output.\n"));
    }
}
fn main() {
    show_output()
}

Below is the output of cargo expand --color "always" --bin 01_my_first_macro main that's actually executed in

macrokata/src/test.rs

Lines 14 to 22 in bbec43e

	let main_output = Command::new("cargo")
	.arg("expand")
	.arg("--color")
	.arg(color)
	.arg("--bin")
	.arg(&exercise)
	.arg("main")
	.output()
	.unwrap();

vscode ➜ ~/macrokata (main) $ cargo expand --color "always" --bin 01_my_first_macro main
    Checking macrokata v0.3.1 (/home/vscode/macrokata)
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s

fn main() {
    show_output()
}

capturing `Copy` values

I wonder if 11_macro_recursion needs also to mention that capturing by value respects Copy trait, since it's a little bit puzzling (at least for me) that print_curried_argument() get ownership of $argident, and then it's used in following closures. (While solving I used borrow, and your beautiful testing system showed me everything.) X)

IIUC Copy [explains] this nuance. And I don't really know if it deserves to be mentioned in the exercise, or not.
[explains]: https://github.com/rust-lang/rust-by-example/blob/31961fe22521a779070a44a8f30a2b00a20b6212/src/fn/closures/capture.md?plain=1#L71

test subcommand doesn't actually test

It would be nice to auto-compare AST, since doing it manually is quite painful for larger examples.

Confusing paragraph regarding operator precedence in Exercise 4 readme

macrokata/exercises/04_expression_variables/README.md

Line 21 in 7d21847

Macros do not affect the order of operations. If the expression `3 * math!(4,

This says that if 3 * math!(...) expands to 3 * 4 + 2, then the result would evaluate to 14. This hypothetical example confused me because I am not aware of any way to write a macro math! such that 3 * math!(4, plus, 2) expands to 3 * 4 + 2 with a result of 14 instead of 3 * (4 + 2) with a result of 18. The straight-forward attempt results in 18 despite not using parentheses:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=fc8f2afa0be0dc9f79da7022b02127ed

It seems like in practice every macro invocation is isolated from surrounding code by a virtual pair of parentheses.

7: More Repetition, wrong exercise description

In the readme of 7: More Repetition in the exercise it is described that the macro if_any! should be implemented. But it actually should be hashmap!. The description what the macro should do is also wrong.

Probably a copy paste error of 6

Proposal to Translate Repository into Chinese

Hi, I'm a student from China. I just finished these exercises yesterday as my first step in learning Rust macros. Your repo is really helpful for beginners(The little book in Macros is too verbose for those who just want to write some codes using macros system as a start). So I would like to propose translating the contents of this repository into Chinese(mainly the README in \exercises).
I am willing to take on the task of translating the content and maintaining it as updates are made to the repository. Please let me know if this is something you would be interested in and if you have any guidelines or preferences for the translation process.
Thank you for considering my proposal.

Testing

@tfpk , while creating this (exercise) book did you ever thought to add a chapter on Testing ?

https://stackoverflow.com/questions/61171437/is-there-a-way-to-test-macros-in-rust/61171504#comment134610126_61171504

Does task 6 require parantheses?

A minor nit: the expected expansion for task 6 is

fn main() {
    if (false || 0 == 1 || true) {
        print_success();
    }
}

Do we need the parantheses around the if clause? I managed to write the macro without them, so I was a bit confused.

General macrokata feedback

Hello,
Thank you for recommending your project!

I wanted to let you a general feedback on my macrokata experience.

Feedback

Overall, it's a really good step-by-step solution and easy to use that you provide.
It greatly helped me in completing the exercism macro exercise.

The book is a really good idea and explains a lot of things.

I didn't have any trouble understanding the vocabulary but that's because I studied compilation so I know how token trees and parsing work. Your token definition is really effective. I don't think people would understand what an AST is in 10., or you can provide a hyperlink to an article or a definition online.

Speaking about difficulty, I noticed a significant increase on 8. and more. A few more examples or advice could help on the separator placement or the expanded code result. I think that before 8. it is accessible to any rust developer. But you should put a warning in the main README explaining that these exercises are not recommended to beginners in programming.

test command

Can we just type the chapter number ? I'm lazy as any developer so when I saw that I needed to type the entire name of the exercise (am on windows), I was like "why do the titles have long words...", because you do have numbers as your chapters.
And if you fail to find the exercise, it continues with the next sentences and congratulations 🎉 because your stdouts are both empty . You should check the slices length.

How to read the instructions

You say in the main README that we should read the README associated but the code inside says to look at it in the book.
You should add a line just after, saying that you can find the associated chapter in the book with the hyperllink.

Exercise 4

The Macros and the Precedence of Operators paragraph is very short, but maybe too short. You could add a sentence saying:
"... expands to 3 * 4 + 2, so you compute 3 * 4 first, the answer will be 14, not 18 ..."

Exercise 5

I was kinda confused about the order of identifiers used in the main code, was it done on purpose ?
On the first use of the macro, you start by defining row then col but inside you use col on x and row on y.
But on the second use of the macro, you define first x then y and you use x to x and y to y.
I always define in my code the horizontal axis (first dimension) then the vertical (second dimension). I made a mistake on the macro code nesting of the macro. Would it be better to reverse row and col to match the second usage?

Moreover, I failed to understand in the book that the block fragment specifier DOES include the curly braces, so I didn't have to type them in my macro because you say "(anything inside curly braces)".

Exercise 8

I had trouble with the ; placement in the substitution macro part.
I didn't know we could just not put a separator when joining the repetitions back together so I spend a long time moving the character all around.
Can the graph be empty? because the matcher could change a lot then and the final ; be a pain in the 🍑.

Exercise 9

I think the exercise main misses a - minus sign before creating the enum:

macrokata/exercises/09_ambiguity_and_ordering/main.rs

Line 43 in 5c68db2

NumberType::NegativeNumber($negative)

Because the goal of the exercise is only to reorder and remove ambiguity.

Exercise 10

I thought the goal was to use tt but the solution uses ident, is it important?

Exercise 11

The double arrow and underscore are quite complicated to manipulate, but finding the solution is very rewarding.

Exercise 12

At first I didn't understand we could use the $crate metavariable to use the structs. the "item or macro" words in the extract is quite generic and does not tell everything (is a macro an item too?). When I saw the example, I thought it was only macros.

Conclusion

I think like I know enough to make use of them, I may not be an expert in rust macros, but with what I learned I can do pretty extensive things. Thanks!

Expected expansion for exercise 10 is impossible

This is the expected expansion for exercise 10:

fn main()
    let pair: (char, u8) = ('a', 1);
    //        ^ it's impossible to specify this type from inside of a macro
    print_pair(pair);
    let value = "value";
    let my_hashmap = {
        HashMap::from([(String::from("Hash"), "map"), (String::from("Key"), value)])
        //             ^ it's not clear where does String::from() comes from,
        //               and the provided solution doesn't generate it
    };
    print_hashmap(&my_hashmap);
}

Project does not build on stable toolchain

First of all, sorry for that I wasn't specific about the actual issue in #11. The problem was I couldn't build the first exercise following the instruction.
Command: cargo run -- test 01_my_first_macro
Received error:

    Blocking waiting for file lock on package cache
    Updating crates.io index
error: failed to select a version for the requirement `clap = "^4.0.17"`
candidate versions found which didn't match: 3.2.23, 3.2.22, 3.2.21, ...
location searched: crates.io index
required by package `macrokata v0.1.0

Setting toolchain to nightly resolved the issue. However @tfpk said in #11 that it would be inappropriate to force anyone to change the compiler. Is there any other way to build and run exercises without setting toolchain to nightly?

Task 08 starter code missing clippy directive

Task 08 is missing the clippy directive to #[allow(clippy::vec_init_then_push)] above main, leading to linter errors with a correct solution.