How to set up the supervisor to automatically manage private packages

packages
supervisor
builder

#1

I think I must be missing something very obvious and fundamental here, but I haven’t been able to find anything that would work reading the docs or tutorials. And I surely can’t be the only one who tries to work with private packages and a supervisor. My main issue is that I have been unable to set up a supervisor to automatically deploy new packages from a private origin.

I am able to install a private package either by setting the habitat token as an env variable export HAB_AUTH_TOKEN=_mytoken or by specifying the auth option hab pkg install --auth _mytoken. This also works for commands like hab pkg search. Whenever a package has been installed, I am able to load that into the supervisor. However in that case the supervisor does not pick up any new packages that were promoted to the channel as described in the tutorials.

It seems like the commands related to the supervisor or services (hab sup and hab svc) do not pick up the environment variable and they do not provide a way to specify a token and therefore are unable to poll or request the private packages.

This is what I observed:

hab studio enter

hab --version
hab 0.57.0/20180614230004

export HAB_AUTH_TOKEN=_mytoken

hab pkg search private_origin/my_package #successfully lists private packages (from stable and unstable channel)

hab svc load private_origin/my_package
» Installing private_origin/my_package
☁ Determining latest version of private_origin/my_package in the 'stable' channel
✗✗✗
✗✗✗ [Err: 0] hab-sup(ER)[components/sup/src/error.rs:455:9]: Package not found. If you are attempting to install from a local depot with an upstream configured, try again in a few seconds.
✗✗✗


hab pkg install private_origin/my_package #installs latest stable package
hab svc load private_origin/my_package --strategy at-once --channel stable 
# now the package is loaded, however new packages in the stable channel are not picked up without a explicit prior hab pkg install 

So, what do I need to configure to be able to make the supervisor or hab svc pick up privately listed packages?


#2

Hmm, I tried to dig into this some more, but it kind of just got more confusing:

If I launch the studio, export my token and try to load a service using the full package specifier, the service loads successfully. I can even load a package, which is not in the specified channel.

hab studio enter
export HAB_AUTH_TOKEN=_mytoken

# fails:
hab svc load private_origin/my_package 

# loads and runs the specified package from unstable(!) channel:
hab svc load private_origin/my_package/0.1.0/20180608041107 --channel stable 

#successfully loads and runs the package from stable channel
hab svc load private_origin/my_package/0.1.0/20180622182548 --channel stable --strategy at-once 

# now, promoting a newer package does not trigger a reload

#3

I know that when you fully specify a package (with origin/name/version/build) it’s not supposed to matter what channels it’s in

I imagine that feeding a fully-specified package name to hab svc load would prevent automatic updates as no other build would ever match that identifier. Can you get your service loaded with just origin/name and your desired --channel passed to hab svc load?


#4

Like hab svc load private_origin/my_package --channel unstable?

That depends: If I start with a new machine or studio: No. Same error as without specifying the channel:

» Installing private_origin/my_package
☁ Determining latest version of private_origin/my_package in the 'unstable' channel
✗✗✗
✗✗✗ [Err: 0] hab-sup(ER)[components/sup/src/error.rs:455:9]: Package not found. If you are attempting to install from a local depot with an upstream configured, try again in a few seconds.
✗✗✗

However, if I install a package via hab pkg install, then hab svc load private_origin/my_package loads the package that has been installed previously.

But automatically reloading a service with a new package when it is available, does not work in any case.

It seems like (but that is just a guess) the hab svc resource seems to only be able to query local packages, but is not able to reach out to the builder (default public builder) and query for private packages. Same goes for the supervisor. hab pkg however seems to be able to do that without issues


#5

Chris is correct, btw, that when you install a package using a fully qualified identifier, the channel gets ignored, because by definition there can be only 1 of those packages. That would also prevent automatic updates as well.

From what I can tell, as long as HAB_AUTH_TOKEN exists in the environment that the supervisor starts up in, updating private packages should work.

I think part of the problem might be that you’re doing this inside of a studio. When you enter a studio, a supervisor is started up in the background, and at that point, your auth token isn’t set.

What about something like (outside of a studio):

env HAB_AUTH_TOKEN=mytoken hab sup run
hab svc load myorigin/my_private_package

Alternatively, if you really want to do this inside of a studio, start the studio like so:

env HAB_STUDIO_SUP=false hab studio enter

which will prevent the supervisor from starting automatically when you enter the studio. Then, once inside, start the supervisor manually after exporting your auth token, like above.


#6

Thank you very much. Now everything makes sense. Should have thought about that though… But I am able to load private packages now and they automatically update, once there is a newer one. Now I can finally go figure out why I am having the exact same issue when setting up a server via chef.