google / moveit Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
It might be worthwhile to run the tests with miri to check for undefined behavior. Adding the following job (adapted from Rand) to the GitHub actions should work:
test-miri:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install toolchain
run: |
MIRI_NIGHTLY=nightly-$(curl -s https://rust-lang.github.io/rustup-components-history/x86_64-unknown-linux-gnu/miri)
rustup default "$MIRI_NIGHTLY"
rustup component add miri
- name: Test rand
run: |
cargo miri test
Line 81 in 5f9f8c7
Assigning MaybeUninit::::uninit() to a local variable is usually free, even when size_of::() is large. However, passing it for example to Arc::new causes at least one copy (from the stack to the newly allocated heap memory) even though there is no meaningful data.
AFAIK, there is no way to construct Arc<MaybeUninit<T>>
and Rc<MaybeUninit<T>>
efficiently until feature(new_uninit)
is stable.
Currently Emplace
is implemented for Box<T>
, Rc<T>
, Arc<T>
. It would be great if Box<T, A>
/Rc<T, A>
/Arc<T, A>
were supported too. This functionality could be feature gated, since allocator_api
is currently nightly-only.
Would a PR for such a feature be (possibly) accepted? I don't have the capacity right now, but I may take a stab at it in the future.
TL;DR: An option to compile without feature = "alloc"
would be great :)
There is a feature flag for alloc already but it's not completely implemented.
A modified basic example (using a *mut
self reference) compiles if I #[cfg(feature = "alloc")]
out the use core::alloc::{...}
and impl for Box, Rc, ... in mod.rs
. But there are other parts which might need to be adjusted.
Reduced feature set would be fine IMO as the most basic is already nice enough to just wrap self-referencing types from C FFI on the stack.
I'm not well versed all these macro-magic though :)
Hi,
I'm not affiliated with moovit in any way.
I just wanted to give you a heads up in case there is a problem.
moveit is very close to moovit which could potentially present a problem for your employer, FYI.
Recently, I found myself writing autocxx bindings to "plain old data" types. With the desire to not have to care about pinning/boxing/whatever these types, I figured out that I could extract the value like so, without writing any of my own unsafe code:
fn unwrap_new<T>(obj: impl New<Output = T>) -> T
where
T: Unpin,
{
MoveRef::into_inner(Pin::into_inner(moveit!(obj)))
}
Finding the correct sequence of things to do was not clear from documentation, nor does there appear to be a similar helper inside of moveit itself (or autocxx for that matter), leading me to think that I'm touching a part of the library that is either not well exercised, or I'm doing something that is not recommended. Therefore, I wanted to ask before I push such code into production.
This isn't currently useful on stable, and likely won't be until rust-lang/rust#32838 (more specifically Allocator::grow_zeroed
) and rust-lang/rust#31844 (at least #![feature(min_specialization)]
) land, but an unsafe
trait with a function like
/// Repairs an instance of `Self` located at `moved` back into a valid state.
///
/// # Safety
///
/// The data referenced by `moved` must have been
///
/// - previously located at `old_address`,
/// - must have been initialized as valid instance of `Self` there and
/// - must have been moved to its new location without modification since then.
unsafe fn repair_moved(old_address: NonNull<Self>, moved: Pin<&mut MaybeUninit<Self>>);
would be potentially helpful as optimisation for reallocating pinning collections.
As I said before, I don't expect this to be currently useful, and also would not currently have a use for it myself.
I just came across it while writing a blog post about pinning and wanted to jot down the idea somewhere since it doesn't fit that text.
The traits in this crate will most likely see a huge boost in helpfulness once specialisation allows them to be default-implemented for T: Unpin
. Feel free to ping me when that happens and I should have time to gladly turn this feature into a pull-request (with hopefully better naming-sense, as I'm not sure repair_moved
checks out and is consistent with the other traits in the crate).
Currently to write a constructor/move-constructor/copy-constructor, you have to deal with a lot of unsafe code and boilerplate steps. Each constructor has to do:
The only steps that the type T actually cares about and wants to define are (2) and (5), but today it has to write the whole algorithm in each constructor/move-constructor/copy-constructor. Like so:
fn myctor(dest: Pin<&mut MaybeUninit<<T as SafeCtor>::Output>>) {
unsafe {
let maybe_uninit_ref = Pin::into_inner_unchecked(dest); // 1
let (out, init) = T { ... }; // 2 (using the copy- or moved-from object in copy- or move-constructors)
*maybe_uninit_ref = MaybeUninit::new(out); // 3
let t_ref = &mut *maybe_uninit_ref.as_mut_ptr(); // 4
setup_self_references(t_ref, init); // 5
}
}
I have implemented a set of "SafeCtor" safe traits that a type can implement in order to have minimal unsafe code in their implementations (even none if they do not require moving/offsetting pointers).
These traits expose the step (2) above as construct() -> (Self, Data)
or copy_construct(from: &Self) -> (Self, Data)
or move_construct(from: Pin<&mut Self>) -> (Self, Data)
. These trait functions provide the "constructed-from" object as a reference that can be directly used from safe rust. And they output Data that can be collected from the "constructed-from" object and represented in a useful way for initialization of self-references.
And they expose step (5) above as an initialize(this: &mut Self, d: Data)
that consumes the constructor's side-channel Data to set up any self-referential state in the constructed object.
The other steps (1), (3) and (4) are performed in unsafe implementations in the unsafe CopyCtor
and unsafe MoveCtor
traits, or in a ctor::construct()
function that replaces the direct use of ctor::from_placement_fn()
from the being-constructed type.
@mcy has pointed out initial self-referential object construction could be done by exposing this 2-stage construction paradigm (i.e. steps (2) and (5)) through the constructing function such as ctor::new_with<T>(init: T, setup: FnOnce(Pin<&mut T>))
, instead of using traits and requiring T : SafeCtor<Output = T>
.
It presents a nice advantage: It would force the caller to construct the type T, which is reasonable for initial construction, and it even allows calling different constructors.
But it has a challenge as well, as copy- and move-constructors can not be implemented in the same way:
Here is an example of a self-referential type that points into a buffer. When moving, the move-constructed type must be able to set up a new self-reference at the same offset (or in this case, the next offset).
use moveit::*;
use std::marker::PhantomPinned;
use std::pin::Pin;
use std::ptr::null_mut;
struct Cycle {
num: [i32; 10],
ptr: *mut i32,
_pin: PhantomPinned,
}
impl Cycle {}
struct CycleInit {
offset: usize,
}
impl ctor::SafeCtor for Cycle {
type Output = Self;
type InitializeData = CycleInit;
fn construct() -> (Self, CycleInit) {
(
Cycle {
num: [9, 8, 7, 6, 5, 4, 3, 2, 1, 0],
ptr: null_mut(), // Set by initialize().
_pin: PhantomPinned,
},
CycleInit { offset: 0 },
)
}
fn initialize(this: &mut Self, init: CycleInit) {
this.ptr = unsafe { (&mut this.num).as_mut_ptr().add(init.offset) }
}
}
impl ctor::SafeMoveCtor for Cycle {
fn move_construct(other: Pin<&mut Self>) -> (Self, CycleInit) {
(
Cycle {
num: other.num,
ptr: null_mut(), // Set by initialize().
_pin: PhantomPinned,
},
CycleInit {
// Move-constructing will move the `ptr` to the right one
// so we can observe it.
offset: 1 + unsafe { other.ptr.offset_from(&other.num[0]) as usize },
},
)
}
}
The only unsafe code here is that which deals with the pointer into num
. The same self-referential initialization is shared between each of construction, copy-construction, and move-construction. It's not obvious if forcing that behaviour to be shared is undesirable.
Here are the traits, which would appear in ctor.rs
, as they provide generic implementations of CopyCtor
and MoveCtor
for any type implementing SafeCopyCtor
or SafeMoveCtor
respectively. The SafeCtor
trait provides the self-referential setup step (2nd stage of construction) used in all construction paths, but only provides a single type-construction function without any parameters. A way to provide multiple constructors is needed.
pub trait SafeCtor {
type Output;
type InitializeData;
fn construct() -> (Self::Output, Self::InitializeData);
fn initialize(o: &mut Self::Output, init: Self::InitializeData);
}
pub trait SafeCopyCtor: CopyCtor + SafeCtor {
fn copy_construct(other: &Self::Output) -> (Self::Output, Self::InitializeData);
}
pub trait SafeMoveCtor: MoveCtor + SafeCtor {
fn move_construct(other: Pin<&mut Self::Output>) -> (Self::Output, Self::InitializeData);
}
unsafe impl<T: SafeCopyCtor + SafeCtor<Output = T>> CopyCtor for T {
unsafe fn copy_ctor(src: &<T as SafeCtor>::Output, dest: Pin<&mut MaybeUninit<<T as SafeCtor>::Output>>) {
let dest_uninit = Pin::into_inner_unchecked(dest);
let (out, init) = <T as SafeCopyCtor>::copy_construct(src);
*dest_uninit = MaybeUninit::new(out);
<T as SafeCtor>::initialize(&mut *dest_uninit.as_mut_ptr(), init);
}
}
unsafe impl<T: SafeMoveCtor + SafeCtor<Output = T>> MoveCtor for T {
unsafe fn move_ctor(
mut src: Pin<MoveRef<<T as SafeCtor>::Output>>,
dest: Pin<&mut MaybeUninit<<T as SafeCtor>::Output>>,
) {
let dest_uninit = Pin::into_inner_unchecked(dest);
let (out, init) = <T as SafeMoveCtor>::move_construct(src.as_mut());
*dest_uninit = MaybeUninit::new(out);
<T as SafeCtor>::initialize(&mut *dest_uninit.as_mut_ptr(), init);
}
}
pub fn construct<T: SafeCtor<Output = T>>() -> impl Ctor<Output = T> {
unsafe {
from_placement_fn(|dest| {
let dest_uninit = Pin::into_inner_unchecked(dest);
let (out, init) = <T as SafeCtor>::construct();
*dest_uninit = MaybeUninit::new(out);
<T as SafeCtor>::initialize(&mut *dest_uninit.as_mut_ptr(), init);
})
}
}
Using the safe traits looks much like using the existing ones, except that T::new()
(or similar) is not called directly.
fn main() {
moveit!(let x = new ctor::construct::<Cycle>());
moveit!(let y = new ctor::mov(x));
println!("{}", unsafe { *y.ptr });
}
Perhaps SafeCtor
could be renamed to SelfReferenceSetup
, the contruct()
trait functions removed, and the ctor::construct()
function modified to receive a T
by value instead. That would handle both multiple construction paths, while still sharing the setup of self-references.
>> use std::pin::Pin;
>> let mut pinned = Box::pin(Box::new(123));
>> moveit::ctor::try_assign(pinned.as_mut(), moveit::ctor::from_try_fn(|| Err(())))
Err(())
>> let b: Box<i32> = *Pin::into_inner(pinned);
>> b
-1594092836
>> b
-1594092772
>> b
-1594092836
Essentially, [try_]assign
doesn't guarantee that lvalue
won't be accessible if the ctor panics or (in TryCtor
's case) returns an Err, so we can access lvalue.deref_mut()
even after ptr::drop_in_place
was called on it.
from cargo miri test
---- src/move_ref.rs - move_ref::PinExt::as_move (line 278) stdout ----
Test executable failed (exit code 1).
stderr:
The following memory was leaked: alloc1499 (Rust heap, size: 4, align: 4) {
05 00 00 00 โ ....
}
DerefMove
allows to get a MoveRef
from self
, and that's not pinned anymore. If self
is a Pin
type then this is unsound
use std::marker::PhantomPinned;
use std::pin::Pin;
use moveit::move_ref::DerefMove;
use moveit::slot;
struct Unmovable {
_marker: PhantomPinned,
}
impl Unmovable {
fn assert_pinned(self: Pin<&mut Self>) {
println!("Pinned")
}
fn assert_unpinned(&mut self) {
println!("Uninned")
}
}
fn main() {
// This also works for creating `unmovable`:
//
// moveit::moveit!(let mut unmovable = moveit::new::of(Unmovable {
// _marker: PhantomPinned,
// }));
let mut unmovable = Box::pin(Unmovable {
_marker: PhantomPinned,
});
unmovable.as_mut().assert_pinned();
slot!(
#[dropping]
drop_slot: _
);
unmovable.deref_move(drop_slot).assert_unpinned();
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.