bitref: bit-level reference types

[tarcieri]

Adds a crate providing a &BitSlice/&mut BitSlice type which is constructable from &[u8] but provides slicing at the granularity of individual bits. The name of the crate is a play on bitvec, which provides a similar type. However, the implementation in this crate is significantly simpler with a much smaller code surface and minimal use of unsafe code.

The implementation is a generalization of RustCrypto/formats#2300 which sought to implement a similar data structure as a reference type for representing ASN.1 BIT STRINGs. However, using this approach was deferred because the implementation relies on a conversion which is sound under Tree Borrows (as verified by Miri) but unsound under Stacked Borrows as it loses provenance. See rust-lang/unsafe-code-guidelines#134

There are several places such a data structure is potentially useful for RustCrypto projects. Beyond the previously mentioned ASN.1 BIT STRING use case, being able to iterate over bits is useful in many numerical algorithms with applications in cryptography, notably in crypto-bigint and for elliptic curves.

Elliptic curve scalar multiplication is generally implemented as a loop over the bits of a scalar. Having an iterator type for this purpose avoids problems relating to the endianness of how scalars are serialized when implementing generic scalar multiplication algorithms, e.g. wNAF (see RustCrypto/group#12).

Given the current open soundness story, I'm not rushing to use this in crypto-bigint until that changes. Where we could use it today though is as an optional dependency to der, where it can act as an ASN.1 BIT STRING type, but implement ToOwned producing a der::asn1::BitString (which, to make ToOwned work, needs to impl Borrow<BitSlice>).

This would make it optionally possible to use Cow for copy-on-write BIT STRINGs today with BitSlice as the borrowed form, but leaving the preferred default data structure for that purpose as der::asn1::BitStringRef, which is a lifetime-parameterized struct that avoids the open soundness questions around BitSlice.

From there we can see what develops around the soundness story and SB/TB discrepancy, and beyond that new Rust features like custom DSTs which may make expressing structures like this less of a hack.

[thomcc]

It seems dubious to use techniques which don't pass SB in a foundational crate like this. That means none of the users of this (perhaps transitively) can use SB. Given that SB is safer than TB (in that there are things allowed in TB which are likely to be UB eventually, which is not really true for SB), that doesn't seem good.

You also don't need the UnsafeCell<()>. Just () should be fine, since you don't support a &[Cell<u8>]-backed BitSlice with interior mutability (bitvec does, but the difference between it and the normal BitSlice was tracked in PhantomData types alone, so rustc/LLVM couldn't tell, and just saw that there were writes to a pointer which came from an immutable slice.

[tarcieri]

@thomcc this is not nor are there plans for it to be a "foundational crate" until the soundness story is resolved.

As the commit message notes, I am not rushing to use this, and the der crate already provides a BitStringRef<'_> type with equivalent functionality but the inability to use reference patterns with it as it's a lifetime-parameterized struct, and likewise the PR to actually convert BitStringRef to this form was closed explicitly because the soundness story isn't there yet: RustCrypto/formats#2300

[tarcieri]

To expand on that, the main reasons I want to put this under RustCrypto:

1. There are already people using bitvec for cryptography (though we aren't anymore) and this at least offers a less-bad partial replacement which passes under Miri with TB if not SB
2. The der use case could be useful today as an optional, caveat-adorned alternative to der::asn1::BitStringRef<'a>
3. I would aspirationally really like to have something like this for crypto-bigint, and it illustrates a "here's something I'd like to use for cryptography but can't because the soundness story isn't quite there" use case when it comes to nailing down the opsem

I am happy to add an additional note about this all in the README.md, saying we aren't using it anywhere yet because of the open soundness issues.

But if there are continued objections, I can just locate it under @tarcieri rather than @RustCrypto. But I thought I'd try here first.