[Discuss] systemd sucks and containers suck and language-specific package repos suck

[markw at mohawksoft.com (markw at mohawksoft.com)]

> markw at mohawksoft.com wrote:
>> > Kent Borg wrote:
>> >> And it reinforces my dislike of systemd, and suspicion that Docker is
>> >> the
>> >> first thing people throw at any problem making the starting size and
>> >> complexity always large whether it needs to be or not.
>> >
>> >
>> > VM images as deployment mechanisms are the direct descendant of
>> > "it works on my laptop... let's put my laptop in the rack".
>>
>> This is a point I STRONGLY disagree with. A VM is a very powerful tool,
>> it
>> allows you to run fundamentally different environments on an otherwise
>> well managed uniform server environment.
>
> ... which is evidence that the development process is broken, because
> instead of running portable software, a VM emulates a whole machine
> (understandable in short term use) and brings along a whole OS and
> a dependency structure and (hopefully) tools for manipulating that
> structure which are *different* from your standard. If they weren't
> different, you wouldn't need it.

In an ideal world, this is kind of true, but also not really. Let me first
articulate what is wrong with the software development model in one word:
"libraries"

The "shared library" was a fantastic idea way back when systems had a few
megabytes of storage. All programs HAD to share code or you couldn't fit
anything on a disk. It was a brilliant solution to a very difficult
problem, but make no mistake, it was a compromise to truly portable code.
It was always a hindrance to interoperability because of the independence
of versions. Static libraries are nice, but they too have a version issue.

Every developer, Linux, UNIX, Windows, or what ever has had to fight
version dependencies either statically or dynamically. Windows "DLL Hell"
was not limited to Windows. Its a mess and it has always been a mess. How
many times have you had to fight different versions of shared libraries
needed by other shared libraries? How many times have you had to fight to
get the proper static library versions in order to compile?

I had to write a statically linked executable to be called by a PAM module
because I needed elliptic curve encryption on SSL, but the version of
CentOS I was using do not support them. I couldn't like with the newer
version of the .so without breaking the PAM system. That's just one
example and I suspect everyone reading this understands the point.

In this case, VMs and containers solve the problem. By no means do I
assert that they do it efficiently or even well, but they do solve the
problem in a performant way without rewriting everything we've been doing
for 40 years.

Secondly, VMs provide a level of abstraction, security, and resource
allocation.

Abstraction, I can run a Windows server (or any OS/system) VM on a well
managed Linux server farm. I can freeze it, move it to a different system,
start it, and perform maintenance or work on the previous machine. If the
back-end is a high speed SAN (and most are) its almost instantaneous
meaning that clients can barely see any interruption. (Iscsi and fibre
channel clients have configurable retry timeouts that can work with
fail-over semantics of a server)

I've worked on large data cluster systems for decades and VMs are the
single most important development in software administration ever. By
encapsulating and abstracting the "service" from the "server" you can
manage your infrastructure so much more cleanly.

Security is a huge advantage for VMs. A VM only has privilege to the
resources it has. If I spin up a postgresql service in a VM and analytical
service in another VM, but leave the postgresql system isolated from the
analytical front end, the postgresql database is more protected and the
analytical front end can be restarted and reinstalled if it is
compromised.

Resource allocation is big too. You can "rent" a VM to a customer. with X
number of CPUs, Y amount of RAM, and a limited amount of storage. Customer
needs more? Provision more. Its easy. Its not like you have to install a
new machine. On top of that, if the customer has a few CPUs idle and
unused RAM, you can use those on other loads.

>
>> People, I think, misunderstand VMs on a conceptual level. Hear me out.
>> Think about the VM as a packaging vehicle. I can test and deploy whole
>> environments as a single step. I can shut down VM 1.0, spin up VM 1.1,
>> in
>> one step. Don't think of a VM as "laptop in a rack" think of it as a
>> self
>> contained way to deploy whole systems.
>
> We have users, permissions, ports and namespaces to separate software
> running on a given system.

That is true, but unix style users and permissions are very limited in
their scope. The standard user, group, all does not scale. You really need
to set up SELINUX policies to do real system security management. Names
spaces are less secure than a VM because the work in the same kernel space
and are subject to all the unpatched kernel CVEs.

A VM has its own kernel and implements its own security independent of the
host. While there is no such thing as an absolutely secure system, that
doesn't mean steps can't be made to make it more difficult or practically
impossible to exploit.

>
> What you are describing is figuratively "it didn't work with this laptop,
> let's try a different one".
>
> A VM for testing can actually be useful, because it provides the
> simulation of an entire machine, controlled from top to bottom, without
> the resource expenditure of an entire very capable machine. So you can
> offer lots of VMs for a lower cost per-"machine" at the time when you
> need lots of low cost machines: testing and development.
>
> At deployment time, however, software should be able to take advantage
> of the available hardware capabilities.

There are things called "paravirtualized drivers" that allow VMs to access
hardware directly. A VM running a native instruction set has virtually no
impact on performance. Azure even offers network offloading ala mellenox
drives if you install the mellanox paravirtual driver. Devices shared to
the VM through a paravirtualized driver access the device directly.

I can't think of any large production environment today that is not based
on VMs. I haven't set up a server system that isn't based on VMs in almost
20 years. With KVM on linux being so good and free, there is no point.

>
>> With overly file systems, you can have a fully read-only VM image that
>> can
>> be impervious to attacks.
>
> It turns out that reading the wrong data out to a hostile user is
> an attack, and one which a read-only VM image does not solve. It
> is also the case that nearly all "read-only" VM images are not
> read-only. They are, at best, selective in the places that they are
> allowed to write.

That's not true. Most large server clusters are connected to a SAN of some
kind. The VM image is presented via fibre or iscsi and can be marked
read-only at the SAN. Which is how you do it in a secure environment.

>
> In other words, UNIX permissions and mount options can do the same thing,
> cheaper.

"Cheaper" is a dangerous word. To what are you referring? Cost of the
machine? Cost of maintenance? Cost of interconnects? The SLA cost if
something goes wrong. In the end it is an economics decision that leads
the tech decision. VMs are safer, easier to maintain, easier to upgrade,
easier to fail-over, and easier to back up.
>
>
> -dsr-
>