FUSE for Cygwin - was: Re: Fork and Windows Heap

[Bill Zissimopoulos <billziss at navimatics dot com>]

[I apologize if my responses to the list appear to break the mailing list's
threading model. I am not actually subscribed to the list and I respond to
individual messages using my mail app.]

Hello, Herbert:

Herbert Stocker wrote:
> > On 16.06.2016 08:37, Bill Zissimopoulos wrote:
> >
> > I have a Windows solution developed using Visual Studio
> > and the Windows Driver Kit that I am building a FUSE
> > compatibility layer for.
> 
> this is a GREAT piece of work.
> Although i did not dive into kernel development, i do understand that
> writing a file system driver for Windows is surely lots of painful work,
> with its undocumented and grown API.
>
> You should write a book documenting all the knowledge you gathered
> through your research, if the only one available is from '97 as you say.
>
> Really, great work.
> Also that you add the FUSE API.

Thank you for your very kind words.

> So, how is your architecture of the FUSE part of WinFsp?
> Are you porting libfuse to Cygwin, so that a Cygwin process can link to
> it and instead of receiving requests via /dev/fuse fro the Linux kernel,
> your libfuse will receive IRPs through your user mode DLL from your
> Windows kernel driver FSD?

WinFsp consists of a kernel-mode file system driver (FSD) and a user mode
dynamic link library (DLL). The FSD registers two devices in the NTOS
(Windows
kernel) namespace with names: \Device\WinFsp.Disk and \Device\WinFsp.Net.
The
DLL interacts with these devices using CreateFile and DeviceIoControl.

The FSD receives IRP's by the kernel and if it cannot handle them itself,
it
posts them to an I/O queue. At a later time the user mode file system
process
receives the IRP's from the queue using DeviceIoControl. The user mode file
system handles the IRP's and then send the responses back to the FSD, which
then completes the IRP's.

All this is of course nicely wrapped by an API so that the user-mode file
system developer never sees IRP's. Instead IRP's are translated to callback
calls with familiar names and semantics such as Create, Open, Read, Write,
etc.

The FUSE layer is a thin layer on top of this api. For example, the WinFsp
Create callback maps to FUSE mkdir or create, the Open callback maps to
FUSE
opendir or open, etc.

I am not porting libfuse to Windows. This is a complete reimplementation.

For the full details on the WinFsp design please see this document:
    http://www.secfs.net/winfsp/develop/design/

For details on the FUSE port please see the WinFsp blog:
    http://www.secfs.net/winfsp/blog/

> Are you planning to have FUSE file systems be ported to native Windows
> apps, i.e. don't link with cygwin1.dll, or will they run in a Cygwin
> environment?

WinFsp provides three (3) different modes of integration:

(1) Its own native API, which allows a user mode file system to do almost
anything NTFS can do (with a few exceptions). It also allows for the
Read, Write and ReadDirectory callbacks to be asynchronous (return
STATUS_PENDING). I recommend this API for file systems that want maximum
performance and integration with Windows.

(2) A FUSE (high-level) API for Win32 (i.e. non-Cygwin) applications. This
is useful if one has a FUSE file system that they can easily port to
Windows or they do not want any Cygwin dependencies.

(3) A FUSE (high-level) API for Cygwin. As you correctly note many FUSE
file systems are too POSIX specific (as they were born on Linux/OSX) and
they only make sense in a Cygwin environment.

I expect that most FUSE file systems will probably go for option (3)
initially. Then they may want to consider going to (2) if they can
easily port their core file system code to Windows. Then they may even
consider (1) if they have needs that the FUSE API cannot support (e.g.
full support for Windows ACL's).

> If porting to native app, this would be easier for the user to install
> them, but:
>  - FUSE has bindings to many languages like Perl, PHP, etc. Will these
>    work then?

I expect that parts of the bindings will have to be rewritten. WinFsp does
not pretend to be FUSE, it only has a FUSE API for someone that writes C
or C++ and has a compiler that understands #include <fuse.h>

> How are you dealing with limitations of the Windows file system as seen
> from a POSIX perspective? You say you can't support hard links.

Windows hard links are rather un-POSIX like and rarely used on Windows.
After considering the required changes on the FSD for a feature that is
so rarely used I opted against supporting them.

I expect that POSIX file systems with hard link support need to take some
care when exposed through WinFsp. But I admit that I have not thought too
hard about this problem and I do not have any solid recommendations or
solutions at this point.

> How will a Cygwin program see symlinks exported by the FUSE file system?

WinFsp will present symlinks as reparse points. Cygwin already knows how
to deal with NTFS reparse points and it will present them as POSIX
symlinks.
I may even relax the whole âyou have to be admin to create a symlinkâ
nonsense.

Please note that WinFsp does (not) yet support reparse points. There is no
big obstacle in the way, just a question of time.

> And pipes?

Pipes in Windows are not stored in the file system, rather they are
implemented by a special Windows file system called NPFS. OTOH POSIX
FIFO's are special files. I am unsure at this time how Cygwin represents
FIFO's.

[Quick experiment:

$ mkfifo foo; cmd /c dir 'foo*' | grep foo; rm foo
06/16/2016  11:02 PM               130 foo.lnk
]

Ok, so they are shortcuts. Naturally they are supported.

> You write that you are mapping
>  - characters not supported by Win32 but by POSIX via the Unicode
>    private use page
>  - Security apspects (SID vs. uid/gid, ACL)
> between POSIX and Windows and that you do it like Cygwin/SFU/SFM is
> doing it.
>
> But if that's done by your code, a Cygwin process may see slightly
> different mappings through WinFsp and through Cygwin. Won't that be a
> potential for Bugs (misbehaving apps) or even for security issues?

I agree that the potential is there. But I did not want WinFsp to depend
on Cygwin so reimplementation was the best option for me. Additionally
Cygwin does not expose this functionality to third party apps AFAIK.

> i'm looking forward to trying it out this weekend and to see it go
> from alpha state to stable.

Thank you for your support :)

> (And i'm curious about the differences of disk based and network based
> file systems. Is it just the handling as seen from the user or also
> the semantics.)

The main difference is how "mounting" happens. Also the Windows explorer
provides different behavior for a disk and network file system. The actual
user mode file system sees no difference whether it is mounted as a disk
or a network file system.

Ok, a quick primer on "mounting" on Windows.

A file system is implemented as a special device that is unnamed (i.e.
does not have a name in the NTOS namespace). Nevertheless this file system
device must somehow receive file system related requests. This is done by
associating (mounting) this device with either a disk device (e.g.
a hard drive) or with the MUP (Multiple UNC Provider).

When the file system device gets associated with a disk device it becomes
a disk file system. When it gets associated with the MUP it becomes a
network file system.

Usually we assign a drive letter X: to a new file system and that is how
we access files on it. But with MUP there is another option: using the
\\Server\Share notation. This is the first important difference between
a disk and network file system.

[There is actually another option for accessing files on a file system.

Try this from a cmd.exe prompt:
    type \\?\GLOBALROOT\Device\HarddiskVolume2\Cygwin64\Cygwin.bat

This assumes that your C: drive points to \Device\HarddiskVolume2 and
that you have Cygwin installed under C:\Cygwin64.]

The second important difference between disk and network file systems
is Windows Explorer support. Windows explorer allows you to map
network drives, list them, etc.

I am using this with WinFsp to spawn file system instances straight
from explorer. For example mapping the network drive Z: to
\\sshfs\billziss@myhost will spawn a new instance of SSHFS that
connects to billziss@myhost. [Please note that while WinFsp already
implements this support, my SSHFS port is not there yet.]

For this reason alone, I expect that most user mode file systems
will be packaged to work as network file systems.

Bill