This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: FUSE for Cygwin


Hi Bill,

this is now my proposal of an alternative mode for WinFsp to support
Cygwin based FUSE file systems. I'll call it mode (4).
It's actually my initial idea that i have in mind for some time but
did not propose to implement because i have some constraints that
prevent me from completing it.

Maybe i should first note that i'm not part of Cygwin, i'm just a
user of Cygwin who is happy to have Cygwin. However i know how to
code.

To repeat your 3 modes of operation (in my words):
(1) WinFsp's own API. The file system is a Windows process that
    can use all file system features that Windows (and WinFsp)
    provides.

(2) A FUE API for Win32. The file system is a native Windows process
    but receives requests as defined in fuse.h . It eases porting
    for the file system and does not require Cygwin be installed.

(3) The file system is a Cygwin process and thus can use all POSIX
    features of Cygwin. It needs no porting or only little.

What i don't like on (3) is that when a Cygwin process accesses the
FUSE file system there are two Cygwin processes whose communication
is translated from Posix to Win32 and then back (which is done again
for the response).

You sketched this as follows:
Let's examine the lifetime of a call to creat(). Suppose a Cygwin process
does creat("/cygdrive/z/foo*bar"). In the following OP is the "originating
process", CW is the "Cygwin runtime", NT is NTOS, WD is the "WinFsp FSD",
WL is the "WinFsp DLL", FL is the "FUSE layer", and FS is the "user mode
FUSE file system".

OP: creat("/cygdrive/z/foo*bar")
CW: NtCreateFile(L"<DEVICE>\\foo\xf02abar")     <--- Cygwin translation
NT: IRP_MJ_CREATE L"\\foo\xf02abar"
WD: FspFsctlTransactCreateKind L"\\foo\xf02abar"
WL: FSP_FILE_SYSTEM_INTERFACE::Create L"\\foo\xf02abar"
FL: fuse_operations::create "/foo*bar"          <--- WinFsp/FUSE
translation
FS: somehow satisfies fuse_operations::create
[snip return path]


So my proposal is basically this:

(4) The file system is a Cygwin process and Cygwin is extended to
    pass file system requests to that process if they fall into the
    respective path (where the file system is mounted.)

    This way the file system is exported to Cygwin only. But another
    tool exports this to the Win32 world.

Initially i thought for the second part we could use that part of
VirtualBox Guest Additions that makes the host file system available
in a windows guest. But now we have WinFsp for this...

For the first part of (4),
Cygwin does have a place that translates Unix paths to Win32 paths.
This part follows the settings in /etc/fstab so that arbitrary Win32
paths can be "mounted" to arbitrary Posix paths with settings applied.
This could be extended so that processes that link to a Cygwin version
of libfuse can register for certain paths.

Cygwin does not have a "kernel", but a Cygwin1.dll which is loaded
into every Cygwin process. On places where it does path lookups or
interferes with files (read, write, extended attributes, etc) it would
require some means to communicate with the file system process and
to wait for the response.

This would make Cygwin processes accessing the file system 1st class
citicens while Windows processes wouldn't lose compared to mode (3).
In mode (3) Cygwin processes would be 2nd class, as i'll explain below.

But the drawback of my approach is that it requires to reimplement
FUSE for Cygwin. If i could, i would volunteer to work on it. i like
Cygwin and i like FUSE, so i sure would like to work on that.

Mode (4) would avoid the following issues coming from the double
translation of mode (3):

A) Besides the double double translations of every request, there
   is also the need for four Kernel/Usermode transitions, which take
   their time.

B) We could have hardlinks for the Cygwin world only, and you have no
   need to implement them in FSD or the DLL. Though you could add that
   feature later not not at all.

   (Or would that be an issue e.g. with oplocs?)

C) i guess this way we could support fuse with all other languages
   besides C/C++.

D) Symlinks won't go through reparse points.
   You'd have no need to implement that or to follow Cygwin should
   Cygwin improve its way of translation.
   BTW, are code pages involved there?

E) Same for uid/gid to SID mapping.
   No need to implement or follow Cygwin.
   And how about the case where a uid/gid has no correspoinding SID?
   Can this happen?

F) Pipes:
> [Quick experiment:
>
> $ mkfifo foo; cmd /c dir 'foo*' | grep foo; rm foo
> 06/16/2016  11:02 PM               130 foo.lnk
> ]
>
> Ok, so they are shortcuts. Naturally they are supported.

   I think they are not.
   The mkfifo system call will have Cygwin create a .lnk file and
   WinFsp will forward it as such to the file system process. The
   sytem calls readdir or open will then have the file system
   process tell WinFsp that there is a .lnk file and Cygwin will
   translate this back to a fifo, so in this sense it does work.

   But the file system will see a file (with name *.lnk) where it
   should see a pipe (mknod call with 'mode' set to S_IFIFO).
   IMHO one could say this is a break of the FUSE API.

   Practically it will break:
    - file systems that special-treat pipe files (or .lnk files).

    - If one uses sshfs to connect to a Linux based server and
      issues the command mkfifo foo from Cygwin, the server will
      end up with a .lnk file instead of a pipe special file.

    - Imagine something like mysqlfs, which stores the stuff in a
      database. When you run SQL statements to analyze the data
      in the file system, you won't see the pipes as such. Or if
      you open the file system from Linux you'll see the .lnk
      files.

   To fix this with mode (3) you'd have to recognize these .lnk
   files and forward them to the file system as pipes, and you'd
   have to generate .lnk files on the fly when the file system
   says there is a pipe file (e.g. on readdir).

G) Case sensitivity.
   Windows is usually case insensitive but case
   preserving on file names. But there is a registry setting that
   allows one to activate case sensitivity. Then programs that use
   the Win32 API in a certain way can have case sensitivity.
   Cygwin has an option to do so, too. It is an option in the
   /etc/fstab file. (And i use it for all drives.)

   With this you can say   touch a A   in the Cygwin command line
   and you'll see two files in Windows Explorer. Double clicking one
   of them will always open the first file though.

   i guess FUSE file systems ported to mode (3) or (2) will expect
   file names to be case sensitive.

https://cygwin.com/cygwin-ug-net/using-specialnames.html#pathnames-casesensitive

   What happens if you use sshfs to connect to a Linux box and then
   do the following in cmd.exe:
     # echo Hallo >Foo.txt
     # type foo.Txt

H) Currently you have either "disk style" file system that you mount
   as a drive letter or a "network style" file sytem that you mount
   via UNC path, e.g. \\sshfs\name@host . Drive letters are a name
   space of only 26 entries which is very narrow, but you can
   also mount them to empty directories on other drives (which must
   support reparse points i think). I don't know if that's possible
   with UNC paths too.

   That means that Cygwin processes will see them as
   //sshfs/name@host and have to use symlinks or /etc/fstab settings
   to "mount" them elsewhere.

It looks like mode (4) is much more work because it requires FUSE
to be reimplemented/ported to Cygwin. But it's lesser work on WinFsp
and would have no issues due to double translations. It would make
Cygwin processes 1st class on FUSE file systems.

So my question is to Corinna, would it be possible and feasible
to implement this in Cygwin?


best regards,

Herbert


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]