Error Creating PollEvented for RawFd

[dbcfd]

Version

0.2.18

Platform

19.4.0 Darwin Kernel Version 19.4.0: Wed Mar 4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64 x86_64

Description

I am attempting to create PollEvented from a RawFd, but it looks to be getting an invalid token.

[2020-04-17T17:27:17Z TRACE mio::sys::unix::kqueue] registering; token=Token(34359738368); interests=Readable | Writable | Error | Hup

This is leading to an IO Error for invalid argument

Code occurring inside an async context:

let mio_ev = mio::unix::EventedFd(&args.fd);
debug!("Create event for {:?}", mio_ev);
let ev = tokio::io::PollEvented::new(mio_ev).map_err(|e| {
   error!("Failed to create evented");
   Error::Io(e)
})?;

When I create a mio event manually, I do not have an issue.

let ev = mio::unix::EventedFd(&self.fd);
let ready = mio::Ready::readable();
let opts = mio::PollOpt::edge();
ev.register(&self.poll, mio::Token(0), ready, opts).map_err(Error::Io)?;

[kleimkuhler]

@dbcfd Do you have any additional output from the error? Token is just a wrapped usize so that value should be okay. Also do you have a way to reproduce this?

[dbcfd]

Not a way to reproduce easily, but a way to fix.

PollEvented registers both read and write, but not all file descriptors are read and write. Providing a method to register a PollEvented with just read prevents this error from happening.

index d8535d9a..1521e245 100644
--- a/tokio/src/io/driver/mod.rs
+++ b/tokio/src/io/driver/mod.rs
@@ -241,6 +241,10 @@ impl Inner {
     ///
     /// The registration token is returned.
     pub(super) fn add_source(&self, source: &dyn Evented) -> io::Result<Address> {
+        self.add_source_with_ready(source, mio::Ready::all())
+    }
+
+    pub(super) fn add_source_with_ready(&self, source: &dyn Evented, ready: mio::Ready) -> io::Result<Address> {
         let address = self.io_dispatch.alloc().ok_or_else(|| {
             io::Error::new(
                 io::ErrorKind::Other,
@@ -253,7 +257,7 @@ impl Inner {
         self.io.register(
             source,
             mio::Token(address.to_usize()),
-            mio::Ready::all(),
+            ready,
             mio::PollOpt::edge(),
         )?;

diff --git a/tokio/src/io/poll_evented.rs b/tokio/src/io/poll_evented.rs
index 298e6e58..19b56019 100644
--- a/tokio/src/io/poll_evented.rs
+++ b/tokio/src/io/poll_evented.rs
@@ -175,14 +175,18 @@ where
     /// from a future driven by a tokio runtime, otherwise runtime can be set
     /// explicitly with [`Handle::enter`](crate::runtime::Handle::enter) function.
     pub fn new(io: E) -> io::Result<Self> {
-        let registration = Registration::new(&io)?;
+        PollEvented::new_with_ready(io, mio::Ready::all())
+    }
+
+    pub fn new_with_ready(io: E, ready: mio::Ready) -> io::Result<Self> {
+        let registration = Registration::new_with_ready(&io, ready)?;
         Ok(Self {
             io: Some(io),
             inner: Inner {
                 registration,
                 read_readiness: AtomicUsize::new(0),
                 write_readiness: AtomicUsize::new(0),
-            },
+            }
         })
     }

diff --git a/tokio/src/io/registration.rs b/tokio/src/io/registration.rs
index 4df11999..425ed2f5 100644
--- a/tokio/src/io/registration.rs
+++ b/tokio/src/io/registration.rs
@@ -63,12 +63,19 @@ impl Registration {
     /// from a future driven by a tokio runtime, otherwise runtime can be set
     /// explicitly with [`Handle::enter`](crate::runtime::Handle::enter) function.
     pub fn new<T>(io: &T) -> io::Result<Registration>
+    where
+        T: Evented,
+    {
+        Registration::new_with_ready(io, mio::Ready::all())
+    }
+
+    pub fn new_with_ready<T>(io: &T, ready: mio::Ready) -> io::Result<Registration>
     where
         T: Evented,
     {
         let handle = Handle::current();
         let address = if let Some(inner) = handle.inner() {
-            inner.add_source(io)?
+            inner.add_source_with_ready(io, ready)?
         } else {
             return Err(io::Error::new(
                 io::ErrorKind::Other,

I then create PollEvented with tokio::io::PollEvented::new_with_ready(ev, mio::Ready::readable()) and there is no error.

[dbcfd]

I believe the PR referenced above can verify the fix with sudo cargo test stream::tests::packets_from_lookup -- --exact.

[carllerche]

What type of resource does the FD reference?

[dbcfd]

In this case it's an interface opened by libpcap for packet capture.

[carllerche]

@dbcfd could you help provide an example of how to repro? I would like to see the steps to get the RawFd in this case (syscalls, ...). I think we need to move away from PollEvented if we want to support uring.

[dbcfd]

1. Create a handle for receiving packets, can use something like https://github.com/jmmk/rustcap/tree/master/pcap-sys
2. Get the raw file descriptor from that handle (https://www.tcpdump.org/manpages/pcap_get_selectable_fd.3pcap.html)
3. Attempt to register a mio RawFd using that raw file descriptor. This will fail.