Canceling all dispatching Builder builds


#1

So you’ve gotten yourself into a state where you have WAY more builds going on Builder than you meant to have and the number of waiting jobs is increasing exponentially…do not fear, some SQL and bash are here to save the day!

First, ssh into the builder live datastore node and get to a psql prompt as instructed in this wiki post.

Now, get a list of all the groups that are currently dispatching with this query:

select id from groups where group_state = 'Dispatching';

Copy that list and save it to a text file on your workstation - for example, let’s say I save it as “groups_to_cancel.txt”.

Now, copy this script to your workstation:

#!/bin/bash

list=$(cat $1)
echo $1
for ident in $list; do
  hab bldr job cancel $ident
done

Save it, let’s say I saved it as “cancel_script.sh”.

Now, invoke the script and pass it the file with the list of groups in it:

./cancel_script.sh to_cancel.txt groups_to_cancel.txt

NOTE: You will need to confirm “Yes” for each group it cancels.

And watch the number of waiting jobs go down!


On-call engineering duties