Re: native Linux userland in Windows 10

[Andrey Repin <anrdaemon at yandex dot ru>]

Greetings, Warren Young!

> On Apr 15, 2016, at 4:04 AM, Andrey Repin <anrdaemon@yandex.ru> wrote:
>> 
>>> And yet, despite the free
>>> availability of top-quality VM technology, Cygwin continues to thrive.
>> 
>> Because interoperability.

> So the ability to run native Linux ELF binaries without recompilation on
> Windows is not âinteroperabilityâ?

Following your logic, starting a Linux VM to run diff on two files counts as
"interoperability".
Not as long as I can't run them alongside native windows binaries.

>>> UfW wonât fill 100% of the use cases of Cygwin from day 1, but for a lot of
>>> cases, the two will be interchangeable,
>> 
>> Name me a few.

> I can name a bunch, including some where UfW comes out ahead:

> 1. The Cygwin solution has some fatal flaw, either due to a missing POSIX
> facility in cygwin1.dll or an attempt to bypass the DLL, going straight to
> the Windows API under Cygwin, which results in mismatched semantics.

> One such example is the recent complaint thread about Gitâs path handling,
> which wouldnât even come up under UfW, because it isnât Windows at all.  Git
> under UfW has exactly the same semantics as under Linux, where presumably
> the semantics are perfect, since Linux is gitâs natural environment.

That's unconfirmed suspicions. We haven't seen the how fork(2) implemented yet
to judge the level of "superiority".
And the git issues seems to have been fixed by repairing directory permissions.
I give you that UfW will likely not have permission issues, since it
essentially runs as "root", but this in itself is a fatal flaw, when it comes
to portability of practices.

> Another example is CRLF and UTF-8 handling.  Any program running under
> Cygwin that bypasses its POSIX interfaces (e.g. calling ReadFile() instead
> of read(2)) will likely do strange things with LF-terminated UTF-8 text
> files,

That's not possible. There's no special rules regarding LF vs. CRLF in UTF-8.
You're making things up.

> (e.g. Cygwinâs âtextâ mount option hack, the crnl-glob feature in Fossil to
> make it ignore CRs when determining if a file is âbinaryâ, etc.) donât need
> to exist under UfW.

Again, unconfirmed suspicions, and text mounts were discouraged for the last
ten years at least, on my memory.

> Still another example is the unfortunate mismatches in Windows vs POSIX
> file locking semantics, as you recently noted in your recent thread
> complaining about another âuselessâ feature, Cygwinâs flock()
> implementation.  Again, the insides of the UfW box are completely POSIX, not
> Windows at all, so presumably locking semantics are Linux-native (i.e.
> advisory by default, rather than mandatory),

Which means that such a lock could still be broken from another [user's] UfW
process, or a native Windows application.

> too, so there is no mismatch
> between Windows and POSIX semantics.  Here, the wall between NT subsystems
> helps: you canât have a Windows app and a UfW app fighting for lock control
> of a single file, since Windows apps canât even see inside the UfW filesystem.

Wishful thinking. There's no separate filesystem for UfW.

> (You could have such a fight through the /mnt/[driveletter] door, but
> thatâs like arguing that the availability of NFS or SMB locking prevents
> Linux locking semantics from working correctly.  Conflicts can only occur in
> the shared filesystem.)

For all that I know, CIFS locks are managed on server, not on client.
I don't know about NFS, though.

> 2. Many people using Cygwin merely want a given non-portable Unixy tool
> (e.g. git) to work on Windows.  The choice between installing Cygwin git vs
> installing UfW from the Windows app store and then saying âapt install gitâ
> is a âsix of one, half a dozen of the otherâ alternative, as long as youâre
> only interested in managing a Windows git tree from the command line. 

> If you need non-CLI things to work, such as like git gdiff integration
> with, say, Beyond Compare, then of course youâd have to use Cygwin git or
> similar, such as the MinGW-based âGit for Windowsâ offering instead of UfW git.

I can see it working for git, or other self-contained application.
Or an application that provide network service (MySQL, Apache).
But not for apps that have a high usage frequency and short lifetime.
I.e. diff, in my case.

> 3. Youâre using Cygwin merely to give Windows an interactive POSIX
> userland, so you have things like grep

Or grep.

> and sed for operation on Windows
> filesystem objects.  The same ability to say something like âsed -i -e
> s/foo/bar $(grep -Rsl foo)â in Cygwin works just as well under UfW.

It doesn't.
To get it running in UfW, I first need to get to it somehow, and then I need
to somehow pass the paths, in case it don't grok redirects well. (In case of
diff, that's precisely the case.)

> It works better under UfW, in fact, since the recursive filesystem search
> will probably be a lot faster under UfW, since all file access isnât going
> through an in-userspace POSIX translation layer.

That's a case I rarely use myself. I have better tools to search across the
filesystem.

> This is what I meant when I described UfW as an apology to Windows users
> for the complete misinterpretation of request vs intent when the PC
> poweruser crowd demanded a better shell and Microsoft gave us PowerShell
> instead of what we actually wanted, which is something more like Bash.

bash in itself isn't especially feature-rich. Yes, it has nice extensions, but
few people actually use more than a few of them.
What makes it seemingly so powerful is the POSIX ecosystem.
Quoting and redirection rules in particular, compounded by the set of tools
that POSIX mandates to be present.
Which makes coupling different tools together less of a pain.
In the case of UfW, this is not going to be an advantage, as it is a parallel
world to the Win32 subsystem.

> 4. Youâre using Cygwin on Windows to test software that normally builds and
> runs on Linux on a PC where installing Linux or a VM manager isnât an
> option.  (e.g. A typical corporate locked-down desktop PC.)  Given a choice
> between Cygwin and UfW, both will work; UfW will probably work better, since
> your actual deployment environment needs ELF executables, so you can even
> build distribution tarballs under UfW, whereas under Cygwin, youâd have to
> do cross-compilation, which adds a bunch of complexity.  (Custom compilers, separate include and lib trees, etc.)

> In fact, in such an environment, UfW might have a distinct advantage, being
> available through the Windows Store.  A typical corporate PC lock-down
> policy might not restrict installation from the Windows Store (such apps
> being pre-vetted, signed, and therefore âsafeâ) but might prevent
> installation of Cygwin, since Cygwin is third-party and doesnât normally
> install on a per-user basis.

Corporate permissions (and social issues overall) is a topic for different
discussion.
Under normal circumstances, such issues are resolved by a simple [written]
request to the associated department, outlining requirements and necessities
to perform your daily job.

> 5. Anything where the Ubuntu package repo has software not present in the
> Cygwin or Cygwin Ports repo.  For example, the Cygwin package repo only has
> iperf2, which is protocol-incompatible with iperf3.  If you need to do a
> bandwidth test against a remote system running iperf3, installing UfW so you
> can âapt install iperf3â is probably easier than porting iperf3 to Cygwin
> and maintaining a private copy.

> (Indeed, iperf3 doesnât build OOTB on Cygwin due to an API conflict with
> newlib, a problem that doesnât impact glibc based systems like UfW.)

Seeing as newlib moves forward, I think that's more a question of effort,
rather than possibility.
Regardless, I got your point, and I covered it earlier.
Any self-contained application will likely success. But that's all.

> 6. A lot of Unix software runs like a pig under Cygwin due to the
> user-space gyrations Cygwin must go through to implement POSIX semantics
> under Windows.

> The classic example of this is fork(), which makes a lot of software (e.g.
> Apache, shell scripts, etc.) much faster on Linux or in native Windows ports
> than under Cygwin.

And we haven't seen the results of fork implementation in UfW yet. Or did we?
Either way, this is a guesswork, and needs real ground before any discussion
could go on in this area.

> 7. Running âtoo-portableâ software.

> A lot of things ported to Cygwin have in-tree support for building natively
> on Windows as well.  (e.g. Perl.)  Often such software has â#ifdef WINDOWSâ
> conditional code in it, so that it behaves differently when run under Cygwin
> than when run under Linux,

I'm not familiar with "e.g. Perl", so I'll give you that.

> 8. All of us who greatly prefer installing software via a command line
> package manager (e.g. apt, pkg, brew, yumâ) rather than a GUI (e.g.
> setup.exe) will probably be happier under UfW than on Cygwin.  âapt install
> fooâ is a whole lot easier than Cmd-E, navigate to the Cygwin download
> folder, and click-click-click for 2-3 minutes to navigate the GUI just to install one piece of software.

I don't antagonize GUI and CLI. CLI perfect when you know the result in
advance and fully expect the system to "do the right thing" (f.e.
"apt install whatever").
But when you need to visualize a complicated chain of dependencies, a nice GUI
tool, such as aptitude or synaptic, suits the job better, than console fiddling.

> 9. A lot of software for Linux simply isnât portable enough to build under
> Cygwin, and there is no native Windows port.  Such software may do low-level
> unportable things, include assembly language hacks, etc. that donât work on
> Windows, so your only alternative heretofore to run such software on a
> Windows box was to run a Linux VM.  (e.g. Node.js, the Oracle JVM (as
> opposed to Cygwinâs current JVM alternative, gcc-gcj), Valgrind, etc.)

And on this point, I'm HIGHLY sceptical. To make such tools work, the kernel
API needs to be implemented on such a level, that it basically becomes a VM,
nigh compatible with the rest of operating system.

> Bottom line, for many use cases, UfW might well completely displace Cygwin.

From nine cases you presented, four are arguable the variations of the other.

> That is, my objection is to the absolute nature of your statement.  If you
> make such a statement relative to yourself and I wonât have any argument
> against the pronouncement.

Certainly. My point being, if I need an isolated, self-contained application,
I chose native program for the job. Though it is long since I moved all of my
tools to dedicated Linux server. I can't possible recall any program that
stands out from general utility pack, that I can give up for UfW.
Assuming I want to install OS that blatantly spying on me, of course.

> Is UfW problem-free?  Certainly not.  But does it have reduced problems
> relative to Cygwin?  In many cases, yes.

I'd say, it have different problems, in addition to some issues shared across
both. (I.e. path conversion.)

>> Always working as root? What could possible go wrong?

> Thatâs a temporary issue, a limitation of UfWâs current beta nature. 
> Microsoft have publicly stated that they will be reworking it to allow it to
> run under your normal userâs permissions and to implement sudo before they
> release it to the public.

That's more the way I'd like it to be.

> Iâm sure the âeverything runs as rootâ nature of it is for the same reason
> that during the bootstrapping process for any OS, you normally do everything
> as root.  User permission management is one of the last things you port over
> when bootstrapping a new OS, once all the tooling is ported over. 


-- 
With best regards,
Andrey Repin
Thursday, April 21, 2016 17:50:15

Sorry for my terrible english...