File::open() on directories does not return Err(), leads to breakage with BufReader

[kentfredric]

I tripped onto some odd behaviour.

I was under the assumption calling File::open() on a directory would Err(), and that I could use that to handle a user specifying a path that was not a file, ( instead of falling prey to race conditions by stat-ing first and then opening second ).

However, ... bad things happened instead.

use std::path::PathBuf;
use std::fs::File;
use std::io::BufRead;
use std::io::BufReader;

fn main() {
    let file = File::open(PathBuf::from("/tmp")).unwrap();
    let buf  = BufReader::with_capacity(100, file);
    let _lines: Vec<std::io::Result<String>> = buf.lines().collect();
}

The last of these lines will run forever, with strace reporting:

read(3, 0x564803253b80, 100)            = -1 EISDIR (Is a directory)

Over and over again ad-infinitum with no end in sight.

Somehow I had a variation of this go crazy and eat 200% of my ram, but I'm having a hard time reproducing that exact case (Though it may have been related to the target-directory in question also being massive in my case).

Digging shows related bug #43504

The profound question that remains unanswered is: "Why is calling File::open on a directory fine?"

And the residual question is "How does one invoke File::open in such a way that it refuses to work on directories".

And importantly, how do I achieve that portably?

And if none of these concerns can be mitigated, the very least that could be done is have this giant foot-gun documented somewhere in File::open.

( As far as I can divine, there's no useful behaviour to be found by allowing File::open to work on directories, as standard read() semantics simply don't work on directory filehandles, you have to use readdir() )

[sfackler]

File::open works on directories because the underlying system call open(2) works on directories. You can use O_DIRECTORY to trigger an error if the path is not a directory, but I don't know if there's a way of doing the opposite.

Your program is going crazy because you're collecting a vector of io::Errors.

[kentfredric]

Perhaps, BufRead et. al needs to internally track if the last call returned EISDIR, and then return None() afterwards?

Trying to continue calling read() after EISDIR is kinda nonsensical.

[kentfredric]

( That is, EISDIR should be considered equivalent to "Error + EOF" , because a subsequent read() cannot return any more data )

[sfackler]

I'm not sure we want to try to catalog the list of all possible terminal errors on every platform.

[kentfredric]

Are there platforms where invoking read() on a filehandle opened for a directory does something useful?

There may be other function calls useful on directories opened as filehandles, but read()?

[mgorny]

Yes. NetBSD supports reading directory entries this way.

[the8472]

As far as I can divine, there's no useful behaviour to be found by allowing File::open to work on directories

On linux there are many uses. It gives you a directory file descriptor (accessible via AsRawFd) which can be used for various other syscalls (not necessarily in the standard library) such as fiemap or the *at syscall family such as renameat.

And the residual question is "How does one invoke File::open in such a way that it refuses to work on directories".

Get the metadata of the file after opening it and check its type. This is often necessary anyway if you only want to operate on regular files and not block devices for example.

[jonas-schievink]

Re-categorizing as a documentation issue. File::open should mention that it (might) work on directories.