Instead of using AST pretty printing.
This is a step towards removing `token::Interpolated`, which will
eventually (in #124141) be replaced with a token stream within invisible
delimiters.
This changes (improves) the output of the `stringify!` macro in some
cases. This is allowed. As the `stringify!` docs say: "Note that the
expanded results of the input tokens may change in the future. You
should be careful if you rely on the output."
Test changes:
- tests/ui/macros/stringify.rs: this used to test both token stream
pretty printing and AST pretty printing via different ways of invoking
of `stringify!` (i.e. `$expr` vs `$tt`). But those two different
invocations now give the same result, which is a nice consistency
improvement. This removes the need for all the `c2*` macros. The AST
pretty printer now has more thorough testing thanks to #125236.
- tests/ui/proc-macro/*: minor improvements where small differences
between `INPUT (DISPLAY)` output and `DEEP-RE-COLLECTED (DISPLAY)`
output disappear.
`tokenstream::Spacing` appears on all `TokenTree::Token` instances,
both punct and non-punct. Its current usage:
- `Joint` means "can join with the next token *and* that token is a
punct".
- `Alone` means "cannot join with the next token *or* can join with the
next token but that token is not a punct".
The fact that `Alone` is used for two different cases is awkward.
This commit augments `tokenstream::Spacing` with a new variant
`JointHidden`, resulting in:
- `Joint` means "can join with the next token *and* that token is a
punct".
- `JointHidden` means "can join with the next token *and* that token is a
not a punct".
- `Alone` means "cannot join with the next token".
This *drastically* improves the output of `print_tts`. For example,
this:
```
stringify!(let a: Vec<u32> = vec![];)
```
currently produces this string:
```
let a : Vec < u32 > = vec! [] ;
```
With this PR, it now produces this string:
```
let a: Vec<u32> = vec![] ;
```
(The space after the `]` is because `TokenTree::Delimited` currently
doesn't have spacing information. The subsequent commit fixes this.)
The new `print_tts` doesn't replicate original code perfectly. E.g.
multiple space characters will be condensed into a single space
character. But it's much improved.
`print_tts` still produces the old, uglier output for code produced by
proc macros. Because we have to translate the generated code from
`proc_macro::Spacing` to the more expressive `token::Spacing`, which
results in too much `proc_macro::Along` usage and no
`proc_macro::JointHidden` usage. So `space_between` still exists and
is used by `print_tts` in conjunction with the `Spacing` field.
This change will also help with the removal of `Token::Interpolated`.
Currently interpolated tokens are pretty-printed nicely via AST pretty
printing. `Token::Interpolated` removal will mean they get printed with
`print_tts`. Without this change, that would result in much uglier
output for code produced by decl macro expansions. With this change, AST
pretty printing and `print_tts` produce similar results.
The commit also tweaks the comments on `proc_macro::Spacing`. In
particular, it refers to "compound tokens" rather than "multi-char
operators" because lifetimes aren't operators.
This crate is a private implementation detail. We only need to insert it
into the crate graph for linking and should not expose any of its public
API.
Fixes#113533
