Tracking issue for release notes of #130183: Bump Unicode to version 16.0.0

[rustbot]

This issue tracks the release notes text for #130183.

Steps

• Proposed text is drafted by PR author (or team) making the noteworthy change.
• Issue is nominated for release team review of clarity for wider audience.
• Release team includes text in release notes/blog posts.

Release notes text

The responsible team for the underlying change should edit this section to replace the automatically generated link with a succinct description of what changed, drawing upon text proposed by the author (either in discussion or through direct editing).

# Libraries
- [Bump Unicode to version 16.0.0](https://github.com/rust-lang/rust/pull/130183)

Tip

Use the previous releases categories to help choose which one(s) to use.
The category will be de-duplicated with all the other ones by the release team.

More than one section can be included if needed.

Release blog section

If the change is notable enough for inclusion in the blog post, the responsible team should add content to this section.
Otherwise leave it empty.

cc @Marcondiro, @Manishearth -- origin issue/PR authors and assignees for starting to draft text

[Noratrieb]

we didn't have it in the blog post for 15, so I think that's it

[Mark-Simulacrum]

We found a few regressions in Crater:

• https://crater-reports.s3.amazonaws.com/beta-1.83-3/beta-2024-10-19/reg/lady-deirdre-derive-2.1.0/log.txt
• https://crater-reports.s3.amazonaws.com/beta-1.83-3/beta-2024-10-19/reg/lady-deirdre-2.1.0/log.txt

(Might actually be the same one)

Do we think a compat note is merited? Or just the library note is enough?

[Marcondiro]

Might be worth it since crates embedding some Unicode Data (as the one you linked) could brake.

[pushrax]

I think a compat note is merited. Our application hit some errors after upgrading rustc, and it bisected to the unicode upgrade.

The reason is pretty niche; we're running a tokenizer to build bm25 text indexes, and the tokenization behaviour changed in some edge cases involving barely-valid utf8. But I could imagine that this might affect other similar use cases.