Conversation
|
Thanks a lot for your extensive work on this!
Could you clarify which parts of your proposal are breaking changes? Or is it possible to keep things completely backwards compatible? My preference would be to get #600 out soon and not rush this complex 57-bit work to be included. I'd rather do another semver-breaking release if necessary. |
|
Thanks for your reply!
Yes. Based on my testing and review so far, when
|
Thanks for the thoughtful reply. I completely understand your preference to get #600 out soon and not rush the complex 57-bit work, and I agree that a separate semver-breaking release may be the safer path for full 57-bit support. That said, I wonder if we could find a narrow compromise. If we can agree that this PR (step 1, and possibly step 2) is sufficiently backwards compatible, and that it won't become an obstacle to any future implementation of full 57-bit support regardless of how that is eventually done, would you be open to including step 1 (and maybe step 2) in this release? I have a fairly immediate need for these features, so it would help me a lot. If necessary, I can also prioritize step 2 and complete it as soon as possible to make this more feasible. Of course, I'm happy to adjust the scope or provide more details to make it safe. I'd appreciate your thoughts. |
phil-opp
left a comment
There was a problem hiding this comment.
I took a quick look, not a full review.
We would also need to adjust the Page code because it also includes code for the address space gap, e.g. in the PageRange iterator impl.
| /// Creates a canonical virtual address by discarding invalid high bits, with the given number of | ||
| /// bits. | ||
| #[inline] | ||
| const fn new_truncate_with_bits<V: VirtAddrValidity>(addr: u64, bits: usize) -> VirtAddrGeneric<V> { |
There was a problem hiding this comment.
This function is safe, but the bits is not checked either, is it? So why does the safety requirement from try_new_with_bits not apply here?
/// The caller must ensure that
bitsis valid for the selected validity policy. This is not
/// checked.
| /// This is [`FixedValidity<48>`] by default and [`FixedValidity<57>`] when the | ||
| /// `default_virt_addr_57` feature is enabled. | ||
| #[cfg(not(feature = "default_virt_addr_57"))] | ||
| pub type DefaultVirtAddrValidity = FixedValidity<48>; |
There was a problem hiding this comment.
This seems dangerous because cargo unifies features. Imagine that you leave the default_virt_addr_57 disabled deliberately, but some other crate in your cargo workspace (maybe even in an unrelated package) enables it. When you build the whole workspace, the feature is now enabled for both packages. But when you build the packages separately (or install them via crates.io), the feature stays disabled for your package.
So cargo features should be additive and not change behavior. Otherwise your code might panic/fail depending on which cargo build command you use.
| /// The default virtual address type. | ||
| /// | ||
| /// This is an alias for [`VirtAddr48`] by default and [`VirtAddr57`] when the | ||
| /// `default_virt_addr_57` feature is enabled. | ||
| #[cfg(feature = "default_virt_addr_57")] | ||
| pub type VirtAddr = VirtAddr57; |
There was a problem hiding this comment.
As noted above, behavior-changing features are dangerous. This here is especially dangerous because it is an alias that seems to indicate backwards compatibility. However, it might silently change types if any other crate in the workspace enables the feature. Given that existing users of x86_64 relied on the fact that VirtAddr is 48 bits, their code will probably break in that case.
So if we add an alias for backwards compatibility we need to make sure that it actually is backwards compatible.
|
Hi @phil-opp , thanks for your review! I've added For the |
They don't really get "support" for it though, they are silently switched to 57-bit addresses (and you don't fully control the switch). But this switch requires some code changes, so you can easily end up with something broken. For example, consider So I don't think that there is a way around making these Virtaddr-dependent structures also generic over the virtual address space size. I.e. types like Page, OffsetPageTable, etc |
|
For the register types, we could probably just always use the 57bit addresses, given that all 48 bit addresses are also valid 57 bit addresses. So no need to make them generic. We should provide From/TryFrom impls for this. If we want to be fancy, we could gate the 57bit writes to registers on some CPUID token that checks whether the CPU supports 5 level paging. I.e. a empty struct that checks CPUID in its constructor and then can be passed to the 57bit write functions. (Apparently it's allowed to write 57bits even if it's not turned on.) |
| /// A [`VirtAddrValidity`] for which arithmetic operations are supported. | ||
| /// | ||
| /// Enabled fixed validity policies always support arithmetic. `RuntimeValidity` supports | ||
| /// arithmetic when the `instructions` feature is enabled and the target is `x86_64`. | ||
| pub(crate) trait ArithmeticValidity: VirtAddrValidity {} |
There was a problem hiding this comment.
Isn't ArithmeticValidity implemented for all validity types? Can we remove ArithmeticValidity?
There was a problem hiding this comment.
It's not implemented for VirtAddrRT when virt_addr_rt is enabled but the target is not x86_64 or instructions are not enabled (which means CR4 cannot be read).
|
|
||
| /// Returns a lazily initialized virtual-address width from the given cache. | ||
| #[inline] | ||
| fn cached_virtual_address_bits_with( |
There was a problem hiding this comment.
| fn cached_virtual_address_bits_with( | |
| fn cached_virtual_address_bits_width( |
Ditto for refetch_virtual_address_bits_with.
| #[inline] | ||
| fn refetch_virtual_address_bits_with(cache: &AtomicU8, read_current_bits: impl FnOnce() -> u8) { |
There was a problem hiding this comment.
Let's inline this function into it's caller. I understand the the idea here is to separate it out to make it testable, but given the simplicity of this logic, I don't think that's necessary.
Ditto for cached_virtual_address_bits_with.
| #[cfg_attr( | ||
| not(all(feature = "instructions", target_arch = "x86_64")), | ||
| doc = r#" | ||
| Address-producing arithmetic is unavailable when the current address-space mode cannot be read: | ||
|
|
||
| ```compile_fail | ||
| use x86_64::{RuntimeValidity, VirtAddrGeneric}; | ||
|
|
||
| let address = VirtAddrGeneric::<RuntimeValidity>::zero(); | ||
| let _ = address + 1u64; | ||
| ``` | ||
| "# | ||
| )] |
There was a problem hiding this comment.
Let's not gate doc comments behind feature gates.
| /// consult the active mode are available wherever the policy is available. Checked construction, | ||
| /// canonicalization, and address-producing arithmetic additionally require the `instructions` | ||
| /// feature and an `x86_64` target, and they must execute in Ring 0. | ||
| #[cfg_attr(feature = "doc_cfg", doc(cfg(feature = "virt_addr_rt")))] |
There was a problem hiding this comment.
| #[cfg_attr(feature = "doc_cfg", doc(cfg(feature = "virt_addr_rt")))] | |
| #[cfg(feature = "virt_addr_rt")] |
Shouldn't we just be doing this?
There was a problem hiding this comment.
The entire mod is #[cfg(feature = "virt_addr_rt")], see src/addr/mod.rs
| /// | ||
| /// This alias is available with the `virt_addr_57` feature. | ||
| #[cfg(feature = "virt_addr_57")] | ||
| #[cfg_attr(feature = "doc_cfg", doc(cfg(feature = "virt_addr_57")))] |
There was a problem hiding this comment.
Doesn't the compiler figure this out itself?
There was a problem hiding this comment.
Sorry but I don't get you. Could you please explain it?
| } | ||
|
|
||
| /// Tries to create a new canonical virtual address. | ||
| /// Tries to create a new canonical virtual address, with provided fixed width. |
There was a problem hiding this comment.
| /// Tries to create a new canonical virtual address, with provided fixed width. | |
| /// Tries to create a new canonical virtual address. |
This is redundant with the sentence below. Let's not focus on the fixed width aspect too much.
Ditto for the other functions.
| #[expect(unused)] | ||
| pub(crate) fn new_with_validity(addr: u64) -> Self { |
There was a problem hiding this comment.
Let's remove this code if it isn't needed yet.
Yes, page table related types do not get 5-level support automatically (but it should not broken existing 4-level page table walking?), and that's why I suggest a following step to adding explicit 5-level page tables/pages in 57-bit virtual address spaces. Most other types like segments, idt, gdt, tss, etc, should work with 57-bit VA without any change. |
That would certainly compile and run. But it would very likely weaken the address-validity checking that the |
Yes, but if the CPU supports it, the registers might still contain a 57bit address even if LA-57 is disabled. AFAIK it's possible to enable LA-57, write a 57 bit address to some register, then disable it again. The register will still contain the 57bit value. Not sure if this is something that we want do handle, but in theory it's possible AFAIK. |
The current feature based approach does this as well, doesn't it? Imagine that your workspace has two crates that both depend on x86_64, one that uses the feature and one that doesn't. Cargo turns the feature on for both of them if you build them together. Now you get weaker checking in one of the crates |
|
In my opinion we have to keep Yes, a semver breaking release allows breaking changes, but such fundamental behavior changes should lead to compile errors. Otherwise users of this library don't notice the change when they update. Imagine someone that's using unsafe code and relies on the canonical address check of us. Now they update, fix all compile errors, run all their tests, and things keep working. So they publish a new version of their library. Then a downstream user of that crate combines it with the new 57bit feature of |
A more realistic case is maybe a virtual machine manager that runs with 4 level paging. Its guest system might run on 5 level paging. When the VMM gets control, some CPU registers (e.g. MSRs) might still contain 5 level addresses. When the VMM tries to read them using the A design that would support this use case would be to always return a And we could add some token struct that is only constructable if the CPU doesn't support 5 level paging (checked via cpuid on construction) and provide non-failing (For convenience, we could also provide a converter that is available when the feature is supported but disabled. The docs for it should clearly state that it might lead to errors/panics in some specific cases, e.g. in the example above.) |
Yes, I completely understand your concern, and in fact I have had similar concerns myself. On the other hand, I once discussed this kind of topic with @Freax13 and reached the opposite conclusion. My understanding is that this is not strictly a matter of right or wrong, but rather a choice: whether to trust that the users can handle such issues properly (not enabling features arbitrarily, and giving understandable compile-time or runtime errors when an unsupported feature is encountered), thereby achieving a simpler and more consistent design, or to insist on completely separating the different types to obtain the best clarity and backwards compatibility, but at the cost of making the implementation of other types and methods more complex. I don't mean to stir up trouble, and I don't have a particularly strong preference, but I understand that now is the time to make a design decision. Perhaps we should discuss this carefully with @Freax13? |
Yes, such cases do exist. I've considered adding a
I don't really understand your design about the "converter", could you please explain it in detail or give me a simple example? |
My point is that this is outside of your control. If any crate in your workspace or dependency tree adds a (transitive) dependency on |
I was thinking about some convenient way to do the VirtAddr57->48 conversion if the feature is disabled. Could be a struct with a non-failing convert_addr method that checks that the feature is disabled on construction. Alternatively, we could just add more ways to construct the token to use the |
|
Brainstorming aside, this is the gist of the approach that I would favor:
This requires users to make some adjustments and add some conversions when updating |
|
By the way, I'm sorry that I'm joining the discussion so late and that I'm raising things again that you two discussed earlier. The past months have been very busy for me and I couldn't find the time to dig into this until now... |
Implement the approach discussed in rust-osdev#605: instead of a cargo feature that changes the meaning of `VirtAddr`, provide the address types of both paging modes as distinct types and let users convert explicitly. Paging modes: - The new sealed `PagingMode` trait describes the two paging modes of the CPU: `FourLevelPaging` (4 levels, 48-bit addresses) and `FiveLevelPaging` (5 levels, 57-bit addresses). It exposes the number of page table levels (`LEVELS`) and the number of valid virtual address bits (`VIRT_ADDR_BITS`). Everything that depends on the paging mode takes it as a type parameter `M` that defaults to `FourLevelPaging`. Address types: - `VirtAddr48` and `VirtAddr57` are aliases of `VirtAddrGeneric<M>`; `VirtAddr` stays an alias for `VirtAddr48`, so existing 48-bit code compiles unchanged. `VirtAddr48` converts into `VirtAddr57` via `From`, the reverse via `TryFrom`. - The successor operations (`Step`) are implemented on the position of an address in the sequence of canonical addresses, which makes the gap handling independent of the paging mode. `Add`/`Sub` keep their plain arithmetic semantics. Hardware boundary: - Values that the CPU writes and that this crate cannot verify (interrupt stack frames, CR2, segment base MSRs, RIP, descriptor table pointers, stored IDT handler addresses, the CET bitmap base) are represented by the new unchecked `RawVirtAddr` type, which converts to either checked type via `try_into_48`/`try_into_57`/ `TryFrom` and from either checked type via `From`. - Write paths accept addresses of both paging modes: the base MSR `write` methods, `Segment64::write_base`, and the CET `write` methods are generic over the paging mode; the interrupt frame constructors and `Entry::set_handler_addr` accept `impl Into<RawVirtAddr>`. - Structures written by the kernel and only read by the CPU (TSS, INVPCID descriptor) use `VirtAddr57`. Paging: - `Page`, `PageRange`, `PageRangeInclusive`, `MapperFlush`, `UnmappedFrame`, and the `Mapper`, `MapperAllSizes`, `Translate`, and `CleanUp` traits have a paging mode parameter defaulting to `FourLevelPaging`. `tlb::flush` and the `invlpgb` builder accept both paging modes. `MappedPageTable` and `RecursivePageTable` keep implementing the traits for `FourLevelPaging` only; 5-level walking can be added without further breaking changes. Tests for the gap arithmetic are written relative to the gap and run for both paging modes, as is the corresponding kani harness. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lqj6m1wpMqQoA5jCa5fhhR
Implement the approach discussed in rust-osdev#605: instead of a cargo feature that changes the meaning of `VirtAddr`, provide both address widths as distinct types and let users convert explicitly. - Make the virtual address type generic over a sealed `VirtAddrWidth` (`Width48`, `Width57`). `VirtAddr48` and `VirtAddr57` are aliases of `VirtAddrGeneric<W>`; `VirtAddr` stays an alias for `VirtAddr48`. `VirtAddr48` converts into `VirtAddr57` via `From`, the reverse via `TryFrom`. Gap handling in `Step`, `align_*`, and range arithmetic is derived from the width. - Use `VirtAddr57` unconditionally wherever the CPU consumes or produces a linear address: `InterruptStackFrameValue`, `TaskStateSegment`, `DescriptorTablePointer`, IDT handler addresses, `Cr2`, `FsBase`, `GsBase`, `KernelGsBase`, `LStar`, `UCet`/`SCet`, `Segment64`, `read_rip`, and `InvPcidCommand::Address`. - Add a virtual address width parameter (defaulting to `Width48`) to `Page`, `PageRange`, `PageRangeInclusive`, `MapperFlush`, `UnmappedFrame`, and the `Mapper`, `MapperAllSizes`, `Translate`, and `CleanUp` traits. `tlb::flush` and the `invlpgb` builder accept both widths. `MappedPageTable` and `RecursivePageTable` keep implementing the traits for 48-bit addresses only. - Add tests for the 57-bit address and page arithmetic. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lqj6m1wpMqQoA5jCa5fhhR
This PR is part of #599 and introduces the generic
VirtAddrGenstruct, the sealedVirtAddrValiditytrait, and the specialized typesVirtAddr48(the 48-bit VA, corresponding to the oldVirtAddr),VirtAddr57(the 57-bit VA; requires thevirt_addr_57feature), andVirtAddrRT(the runtime-validated VA; requires thevirt_addr_rtfeature).VirtAddris now a type alias forVirtAddr48for compatibility. Alternatively, enabling thedefault_virt_addr_57feature makesVirtAddran alias forVirtAddr57, so the 57-bit VA becomes the default. Whendefault_virt_addr_57is enabled, most data structures and functions in this crate automatically use the 57-bit VA, except for thepagingmodule (which requires further consideration and implementation about page levels; I think that work should be done in a separate PR).