Odd behaviour coming from macros

[InnocentusLime]

I tried this code:

struct T(i32);

#[macro_export]
macro_rules! test {
  ($array_type:ty => $($elem:expr),+ $(,)?) => {};
  ($($elem:expr),+) => {};
}

fn main() {
    test!(T(2));
}

I expected it to compile, since T(2) is obviously an expression.

Instead I received the following compiler error:

error: expected type, found 2

The compiler seems to forcefully parse the string with the first rule (although it is clearly inapplicable here), fails and terminates with an error.

What is more surprising is that the following code compiles just fine

struct T;

#[macro_export]
macro_rules! test {
  ($array_type:ty => $($elem:expr),+ $(,)?) => {};
  ($($elem:expr),+) => {};
}

fn main() {
    test!(T);
}

Even the right branch gets picked (the second one)!

Moreover, reordering the branches breaks the following example (which compiles if you swap the test's arms back)

struct T(i32);

#[macro_export]
macro_rules! test {
  ($($elem:expr),+) => {};
  ($array_type:ty => $($elem:expr),+ $(,)?) => {};
}

fn main() {
    test!([&'static T; 2] => T(2));
}

Meta

rustc --version --verbose:

rustc 1.51.0 (2fd73fabe 2021-03-23)
binary: rustc
commit-hash: 2fd73fabe469357a12c2c974c140f67e7cdd76d0
commit-date: 2021-03-23
host: x86_64-unknown-linux-gnu
release: 1.51.0
LLVM version: 11.0.1

Beta and nightly struggle with this code too.

[ehuss]

I believe this is working as intended (for now). Non-terminal metavariables are parsed as a whole, and if it fails, the whole macro is rejected. macro_rules does not support backtracking (see #42838 and some of the issues linked there).

[InnocentusLime]

I feel like it doesn't have much to do with backtracking. Because test!(T) doesn't get rejected. Despite being more ambiguous than T(2). T clearly belongs to the type category too, but Rust picked the expression variant, which means that Rust dropped the first arm and tried the second one. Why isn't it the case for T(2)

Moreover, why Rust is even trying to use the first arm? T(2) shouldn't be classified as a type by the macro to begin with for a variety of reasons:

1. When you write macros, stuff like ($array_type:ty($e:expr)) gets rejected by the macro parser
2. From the reference one can infer that types don't have any syntax like T(U)

[SkiFire13]

From the reference one can infer that types don't have any syntax like T(U)

That's not true, a Type can end up matching a PathIdentSegment optionally followed by a TypePathFn which starts with parenthesis. In fact Fn(T) is a valid type.

[InnocentusLime]

Hm... Indeed then the macro dives into the first arm trying to parse T(2) as a type, but fails... But why does it match if you declare

struct T;

And call test!(T)? As I said, it matches the type grammar too... Which means that Rust is sequentially trying each macro arm (like a match?), but it isn't the case for T(2) for some reason, it drops trying other branches!

I'll reiterate my point. Consider the following code

struct T;
struct U(i32);
struct R(T);

#[macro_export]
macro_rules! test {
  // First arm
  ($array_type:ty => $($elem:expr),+ $(,)?) => {};    
  // Second arm. Should be tried if the first fails, right? Macros are a match-like thing after all
  ($($elem:expr),+) => {println!("Expression branch")};  
}

fn main() {
    // Doesn't get rejected. Rust tries the second arm after realizing the first arm failed
    test!(T); 
    // Gets rejected. Rust should try all the arms, but terminates after failing with the first arm
    test!(U(2));
    // Doesn't get rejected purely because R(T) parses as a type 
    test!(R(T));
}

Rust compiles well the first line of the main, but fails with the second one. They both don't satisfy the first arm of the macro in some way, but the second macro invocation gets a different treatment than the first one.

[ehuss]

test!(T) works because it successfully parses T as a type, and then the => isn't found, so it moves on to the next arm.

It only fails when there is an error matching a nonterminal matching fragment.

[InnocentusLime]

Which brings us back to the backtracking issue, which seems dead... Alright, guess I'll close this issue.

Is there any progress on the original backtracking issue, which is unseen to us?

[ehuss]

I'm not aware of anyone working on it. #33840 (comment) discusses some of the complexity involved in solving it.

[InnocentusLime]

Yeah... Seems like the feature isn't going to be added at least for backtracking and trying other macro arms... Guess I'll look for other ways to solve the issue in the tinyvec