Changing out the Operating system underneath Habitat


#1

If any of you are like me, you’ve read all of the Habitat marketing and have bought into this idea of decoupling your app from your Operating System. It’s a great goal and a lofty claim.
So I set out to test it.

Using Terraform, I built out a three node postgresql cluster on Ubuntu 16.04. The key here was I created a separate datastore, and mounted it under /hab, so I have my OS and everything underneath /hab is effectively isolated.

Next I built a CentOS 7.2 VM. Note that they have different Kernels:

root@hab-psql-tf-stable-1:~# uname -a
Linux hab-psql-tf-stable-1 4.4.0-92-generic #115-Ubuntu SMP Thu Aug 10 09:04:33 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
root@hab-psql-tf-stable-1:~# cat /etc/*release*
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
[root@hab-psql1-pt2 ~]# uname -a
Linux hab-psq1-pt2 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@hab-psql1-pt2 ~]# cat /etc/*release*
CentOS Linux release 7.2.1511 (Core)
Derived from Red Hat Enterprise Linux 7.2 (Source)

I chose ubuntu 16.04 and CentOS 7.2 for no particular reason other than it’s what I clicked on in OpenStack.

So I unmounted then mounted my volume on my new CentOS 72 image, created my systemd init script aaaand:

Mar  8 17:45:21 hab-psql1-pt2 systemd: Starting Habitat Supervisor...
Mar  8 17:45:21 hab-psql1-pt2 systemd: Failed at step EXEC spawning /bin/hab: No such file or directory

Doh. Oh, link /bin/hab to /hab/pkgs/core/hab/0.77.0/20190301212334/bin/hab, problem solved, right?

Well… sort of:

hab-sup(MR): Can't create directory /hab/svc/postgresql: hab-sup(ER)[components/sup/src/error.rs:460:46]: Package requires user hab to exist, but it doesn't
hab-sup(MR): If this service is running as non-root, you'll need to create /hab/svc/postgresql and give the current user write access to it

Ok, easy fix, create the hab user and group, start my service again, and

[root@hab-psq1-pt2 ~]# hab svc status
package                                type        desired  state  elapsed (s)  pid   group
core/postgresql/9.6.11/20190305213451  standalone  up       up     3            3012  postgresql.default

And, we can confirm on our master node:

Before the change

postgres=# SELECT client_addr, state FROM pg_stat_replication;
 client_addr  |   state
--------------+-----------
 172.16.10.11 | streaming
 172.16.10.37 | streaming

During the change:

postgres=# SELECT client_addr, state FROM pg_stat_replication;
 client_addr  |   state
--------------+-----------
 172.16.10.37 | streaming

After the change:

postgres=# SELECT client_addr, state FROM pg_stat_replication;
 client_addr  |   state
--------------+-----------
 172.16.10.37 | streaming
 172.16.10.18 | streaming

Once I mounted my /hab partition, created my systemd init script, linked /bin/hab to the right location, and created my hab user, I was off and cooking with gas having successfully changed out the OS underneath the application.


#2

Awesome! This is definitely nice to see!

I love how when I build my apps, I am making a /hab filesystem. Everything that my app needs to run is located under there versus everywhere like how yum and apt package managers allow.


#3

This is a super cool test and great to see!


#4

Amazing work here! Thank you for testing/proving this out, and for sharing it back.