Habitat Wrapper Plans


#1

A while back I opened a PR on a core-plan to make some changes to the default config. It was suggested at that time to use a “wrapper plan” as the core plans aren’t designed to hit every use case. While I can totally see how that would become a nightmare, there isn’t (as far as I can find) a good guide on how to write a wrapper plan. So I stumbled my way through and am hoping we might refine the process!

So without further ado:

Writing a Wrapper Plan

Intro

So you want to modify the configuration of a Core Plan, but you don’t want to be responsible for maintaining the whole package, or you’d like to take advantage of the automatic dependency rebuilding without affecting how your configuration is written.

Enter wrapper plans. Similar in principle to Chef wrapper cookbooks, wrapper plans allow us to leverage core plans as our upstream while maintaining our own specific set of configuration files.

Example Use Case

Let’s take core/vault for example. For test use cases, the core plan is great. But we want to hook Vault up to a Consul cluster. As written, core/vault can’t accommodate us. We could just clone down the core/vault plan, fork it, and make our changes there, but when the next version of Vault is released, we’ll be responsible for updating the plan, testing the new version, and then rolling it out. Wrapper plans eliminate the need for updating the plan every time a new version is released.

Process

Obtain the core plan

First, let’s clone down the core/vault plan, and put it in our own working directory.

cd ~
mkdir hab-dev
git clone https://github.com/habitat-sh/core-plans.git
cp -r core-plans/vault hab-dev/vault-wrapper
cd hab-dev/vault-wrapper

Plan.sh updates

Great! Now we need to update our pkg_origin, pkg_name , and pkg_maintainer (and I’ve gone back and forth a bunch now… should we update the version too? to what?)

- pkg_origin=core
- pkg_name=vault
...
- pkg_maintainer='The Habitat Maintainers <humans@habitat.sh>'
+ pkg_origin=qbrd
+ pkg_name=vault-wrapper
...
+ pkg_maintainer='QubitRenegade <qbrd@qubitrenegade.com>'

Next we’ll update our plan dependency:

- pkg_deps=()
+ pkg_deps=(core/vault)

We also need to update and any do_*() functions to return 0 so they are a no-op.

- do_unpack() {
-   cd "${HAB_CACHE_SRC_PATH}" || exit
-   unzip "${pkg_filename}" -d "${pkg_name}-${pkg_version}"
- }
+ do_unpack() {
+   return 0
+ }

Finally, since we want to bind this service to another, we need to add our bind contract to the plan.

+ pkg_binds_optional=(
+   [consul]="port-http"
+ )
Full Plan.sh
pkg_origin=qbrd
pkg_name=vault
pkg_version=1.0.2
pkg_description="A tool for managing secrets."
pkg_maintainer='QubitRenegade qbrd@qubitrenegade.com'
pkg_license=("MPL-2.0")
pkg_upstream_url=https://www.vaultproject.io/
pkg_deps=(core/vault)
pkg_svc_user=hab
pkg_svc_group=hab
pkg_exports=(
  [port]=listener.port
)
pkg_exposes=(port)

pkg_binds_optional=(
  [consul]="port-http"
)

do_unpack() {
  return 0
}

do_build() {
  return 0
}

do_install() {
  return 0
}

Hook Updates

Next we need to update any hooks.

First we’ll update the run hook to refer to the core/vault package binary:

- exec vault server -dev-listen-address=0.0.0.0:8200 -dev
...
- exec vault server -config={{pkg.svc_config_path}}/settings.hcl
+ exec {{pkgPathFor "core/vault"}}/bin/vault server -dev-listen-address=0.0.0.0:8200 -dev
...
+ exec {{pkgPathFor "core/vault"}}/bin/vault server -config={{pkg.svc_config_path}}/settings.hcl
Full run hook
#!/bin/bash -xe

exec 2>&1

DEVMODE={{cfg.dev.mode}}

if [ "$DEVMODE" = true ] ; then
  exec {{pkgPathFor "core/vault"}}/bin/vault server -dev-listen-address=0.0.0.0:8200 -dev
else
  exec {{pkgPathFor "core/vault"}}/bin/vault server -config={{pkg.svc_config_path}}/settings.hcl
fi

In the case of vault, we’re going to just remove the init hook since we’re not modifying the local filesystem. This may be different for other wrapper plans.

$ rm hooks/init

Config Updates

Next we want to update our configuration file. Because we made our pkg_binds optional, we might want to manually specify the Consul address (i.e.: 127.0.0.1:8500). Since this is more or less a full rewrite the full config is included below.

Full settings.hcl
ui = {{cfg.server.ui}}

storage "consul" {
  {{#if bind.consul-client}}
    {{#if cfg.backend.use-https }}
  scheme = "https"
  address = "{{bind.consul.leader.sys.ip}}:{{bind.consul.leader.cfg.ports.https}}"
    {{ else }}
  scheme = "http"
  address = "{{bind.consul.leader.sys.ip}}:{{bind.consul.leader.cfg.ports.http}}"
    {{/if}}
  {{ else }}
  scheme = "{{cfg.backend.scheme}}"
  address = "{{cfg.backend.address}}"
  {{/if}}

  path = "{{cfg.backend.path}}"
  service = "{{cfg.backend.service-name}}"
  {{#if cfg.backend.token }}
  token = "{{cfg.backend.token}}"
  {{/if}}
}

listener "{{cfg.listener.type}}" {
  address = "{{cfg.listener.location}}:{{cfg.listener.port}}"
  tls_disable = {{cfg.listener.tls_disable}}
}

Update default.toml

Finally, we need to add our new settings to the default.toml.

Full default.toml
# switch this to true you want to start in DEVMODE
# https://www.vaultproject.io/intro/getting-started/dev-server.html
[dev]
mode = false

[backend]
use-https = false
storage = "file"
path = "vault/"
service-name = "vault"

[listener]
type = "tcp"
location = "127.0.0.1"
port = "8200"
tls_disable = "1"

[server]
ui = true

Conclusion

I put together a GitHub repo with all of this code and built the plan here.
Overall, I like the idea of a wrapper plan, and think there is a lot of potential here to make wrapper plans an extremely powerful tool in the Habitat arsenal.


Thoughts? I feel like there’s an opportunity here to make the process more “user friendly”, but is there anything I’m missing in how to make a “wrapper plan”? Anything else you would include? Anything you wouldn’t include?

Thanks!
-Q


#2

You’ve pretty much covered it! @predominant wrote up a good guide on wrapper plans a while back too https://grahamweldon.com/post/2018/09/habitat-configuration-wrapper-plans/

In some of my wrapper plans, I also pull in other deps. Core plans are often missing good health-check hooks, so I often add core/curl and core/jq-static to be able to implement my own. I maintain my own wrappers for etcd, memcached, postgresql, rabbitmq, and vault.

Personally, I think this pattern should be documented in the Habitat docs and promoted heavily as a use pattern. It’s really the only sane way to deploy core plans into production.


#3

The should be moved into the wiki at the very least. Great guide!


#4

Agree, and it would make a great blog post too. Our blog is open source and pull requests are welcome! Instructions are in the readme: https://github.com/habitat-sh/habitat/tree/master/www/source/blog


#5

Wrapping plans is a solid pattern. Thanks for writing this up @qubitrenegade. Personally I’d like to see some more first class features implemented that make this experience more straightforward. It would be lovely for us to be able to, say, inherent the lifecycle hooks from an underlying package if theres no necessity to change them. Stuff like that.


#6

@ht154 I feel like there’s a difference between what @predominant wrote and what I wrote… but I struggle to articulate why… I guess my goal is to leverage the core plan while injecting my own configuration, whereas @predominant is wrapping a plan for a more “specific” reason?

I think the big difference is the hooks… because the packages are loaded into the $PATH anyway perhaps {{pkgPathFor "core/vault"}} is really overkill and I could be ok not modifying them, just including them in my plan.

It would maybe be nice to hab svc load core/vault in my run hook? Not sure how that might look…

@tashadrew alright, if you think it’s worthy. :slight_smile: Link to my blog? Lil’ quid pro quo?

@eeyun that is exactly the kind of feedback I was looking to elicit. e.g.: it would be really nice to reuse the init and run hooks because I have nothing to change there.

Though, that raises the question, if I were to overwrite the run hook, would I have to implement the whole thing or just modify the parts I want (kinda like how I can create a user.toml and specify foo = "bar" and {{cfg.biz}} won’t be overridden)? I think the former would be much easier to implement, i.e.: “does the wrapper plan have a run hook? no - use parent plan hook, yes - use wrapper plan hook”. Same thing with pkg_ vars. I’m really only overriding pkg_origin and pkg_name, it might be nice to be able to go pkg_inherits="$(pkg_parent 'core/valut')" ...