Need to build apache with mod_authnz_ldap


#1

I need to build httpd with mod_authnz_ldap. Having looked into it, it is apparent that in order to do so, I also need apr-util to have been built with ldap support.

I’ve looked at the plan for apr-util and compared it to https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/apr-util and see that I need --with-ldap. I’ve added this to my plan, and a dependency on core/openldap but when I build it, I get the following error:

/hab/cache/src/apr-util-1.6.1/include/apr_ldap.h:79:18: fatal error: lber.h: No such file or directory

What library within core will provide this, or do I need to build this too?

Furthermore, once I’ve managed to build these libraries (apr-util with ldap, and whatever it depends on) what’s the best way to ensure my plan uses these packages rather than the ones from core?


#2

Ok there are few things here worth talking through.

Furthermore, once I’ve managed to build these libraries (apr-util with ldap, and whatever it depends on) what’s the best way to ensure my plan uses these packages rather than the ones from core?

So the first thing (the thing easiest to clear up) is that you can always build/upload packages for your personal origin which means you can always include those packages as deps in your own packages. So say your origin was called foo you would create foo/apr-util that included the appropriate flag and in the package that needed the updated apr-util you would modify your deps to look like pkg_deps=(foo/apr-util). In order to test this locally, you’ll want to enter your interactive studio from a directory one step above ALL the directories for the interdependent bits of code you’re planning to build.

What this looks like in practice is perhaps you have a plans dir and inside that directory each of your package definitions have their own directories, like so:

$ cd plans
$ ls plans
apr-util httpd openldap

Solving the issue of libraries not being appropriately linked might take a little more effort. Give me some time and I’ll see if I can’t figure out why core/openldap isn’t providing the libraries to be appropriately linked.


#3

I should get a chance to look at this this afternoon, but for starters, having looked into the openldap source code, that header file is certainly provided in the distribution used by core/openldap.

The broader question is why it’s not being found. Check the output of ./configure --with-ldap and post it here.

If you could show the plan.sh as well, that would help others to try to reproduce your issue.

EDIT: The differences in my plan vs upstream, for reproduction:

$ git diff
diff --git a/apr-util/plan.sh b/apr-util/plan.sh
index a222c46..56a628b 100644
--- a/apr-util/plan.sh
+++ b/apr-util/plan.sh
@@ -1,5 +1,5 @@
 pkg_name=apr-util
-pkg_origin=core
+pkg_origin=atalanta
 pkg_version=1.6.1
 pkg_license=('Apache-2.0')
 pkg_maintainer="The Habitat Maintainers <humans@habitat.sh>"
@@ -7,7 +7,7 @@ pkg_description="Apache Portable Runtime util"
 pkg_upstream_url="https://apr.apache.org/"
 pkg_source=https://archive.apache.org/dist/apr/${pkg_name}-${pkg_version}.tar.bz2
 pkg_shasum=d3e12f7b6ad12687572a3a39475545a072608f4ba03a6ce8a3778f607dd0035b
-pkg_deps=(core/gcc-libs core/glibc core/apr core/expat)
+pkg_deps=(core/gcc-libs core/glibc core/apr core/expat core/openldap)
 pkg_build_deps=(core/gcc core/make)
 pkg_bin_dirs=(bin)
 pkg_include_dirs=(include)
@@ -17,6 +17,8 @@ pkg_pconfig_dirs=(lib/pkgconfig)
 do_build() {
   ./configure --prefix="${pkg_prefix}" \
               --with-apr="$(pkg_path_for core/apr)" \
-              --with-expat="$(pkg_path_for core/expat)"
+              --with-expat="$(pkg_path_for core/expat)" \
+             --with-ldap
+  attach
   make
 }

#4

Here’s what I see from ./configure output:

checking for ldap_init in -lldap50... no
checking for ldap_init in -lldapssl41... no
checking for ldap_init in -lldapssl40... no
checking for ldap_init in -lldapssl30... no
checking for ldap_init in -lldapssl20... no
checking for ldap_init in -lldapsdk... no
checking for ldap_init in -lldapsdk... no
checking for ldap_init in -lldap... yes
checking for ldapssl_client_init in -lldap... no
checking for ldapssl_client_deinit in -lldap... no
checking for ldapssl_add_trusted_cert in -lldap... no
checking for ldap_start_tls_s in -lldap... yes
checking for ldap_sslinit in -lldap... no
checking for ldapssl_init in -lldap... no
checking for ldapssl_install_routines in -lldap... no
  setting LDADD_ldap to "-lldap -llber"
checking for ber_init in -llber... yes
checking lber.h usability... yes
checking lber.h presence... yes
checking for lber.h... yes
checking for ldap.h... yes
checking ldap_ssl.h usability... no
checking ldap_ssl.h presence... no
checking for ldap_ssl.h... no
checking for LDAP toolkit... OpenLDAP
checking style of ldap_set_rebind_proc routine... three```

#5

And this is the failure when I run make test:

/bin/sh /hab/pkgs/core/apr/1.6.3/20171115103100/build-1/libtool --silent --mode=compile gcc -pthread  -I/hab/pkgs/core/glibc/2.22/20170513201042/include -I/hab/pkgs/core/file/5.24/20170513201915/include -I/hab/pkgs/core/gcc/5.2.0/20170513202244/include -I/hab/pkgs/core/make/4.2.1/20170513214620/include -DHAVE_CONFIG_H  -DLINUX -D_REENTRANT -D_GNU_SOURCE  -I/hab/pkgs/core/glibc/2.22/20170513201042/include -I/hab/pkgs/core/file/5.24/20170513201915/include -I/hab/pkgs/core/gcc/5.2.0/20170513202244/include -I/hab/pkgs/core/make/4.2.1/20170513214620/include -I/hab/cache/src/apr-util-1.6.1/include -I/hab/cache/src/apr-util-1.6.1/include/private  -I/hab/pkgs/core/apr/1.6.3/20171115103100/include/apr-1  -I/hab/pkgs/core/expat/2.2.2/20171015103808/include  -o ldap/apr_ldap_stub.lo -c ldap/apr_ldap_stub.c && touch ldap/apr_ldap_stub.lo
In file included from ldap/apr_ldap_stub.c:20:0:
/hab/cache/src/apr-util-1.6.1/include/apr_ldap.h:79:18: fatal error: lber.h: No such file or directory
compilation terminated.

So apr_ldap.h, tries to include lber.h, but doesn’t find it.

lber.h is present in core/openssl:

[25] apr-util(do_build)> pwd
/hab/pkgs/core/openldap/2.4.45/20180103174008
[26] apr-util(do_build)> find . -name lber.h
./include/lber.h

.


#6

I haven’t had a chance to dig into this yet. But typically when you see an underlying library that is not getting linked appropriately by a consuming package (e.g. foo/bar needs lib_baz which should be provided by foo/baz) its more often than not an issue with the build of the package being consumed (in our example foo/baz). The easiest thing it could be (and in this case i’d almost guarantee it isn’t) is that pkg_lib_dirs didn’t get set in the package being consumed.

After that, it’s an issue of making sure the linker knows those libraries exist and sometimes that means having to add some custom configuration to the build to force it to bubble those libraries up to the linker.

I’m not sure how helpful any of this is, I will also hopefully get to take a look at this later today and we can track effort here!


#7

OK, so the configure log shows how it found the library:

configure:11491: checking lber.h usability
configure:11491: gcc -c -I/hab/pkgs/core/glibc/2.22/20170513201042/include -I/hab/pkgs/core/apr/1.6.3/20171115103100/include -I/hab/pkgs/core/expat/2.2.2/20171015103808/include -I/hab/pkgs/core/openldap/2.4.45/20180103174008/include -I/hab/pkgs/core/gcc/5.2.0/20170513202244/include -I/hab/pkgs/core/make/4.2.1/20170513214620/include -pthread -I/hab/pkgs/core/glibc/2.22/20170513201042/include -I/hab/pkgs/core/apr/1.6.3/20171115103100/include -I/hab/pkgs/core/expat/2.2.2/20171015103808/include -I/hab/pkgs/core/openldap/2.4.45/20180103174008/include -I/hab/pkgs/core/gcc/5.2.0/20170513202244/include -I/hab/pkgs/core/make/4.2.1/20170513214620/include -DLINUX -D_REENTRANT -D_GNU_SOURCE conftest.c >&5
configure:11491: $? = 0
configure:11491: result: yes
configure:11491: checking lber.h presence
configure:11491: gcc -E -I/hab/pkgs/core/glibc/2.22/20170513201042/include -I/hab/pkgs/core/apr/1.6.3/20171115103100/include -I/hab/pkgs/core/expat/2.2.2/20171015103808/include -I/hab/pkgs/core/openldap/2.4.45/20180103174008/include -I/hab/pkgs/core/gcc/5.2.0/20170513202244/include -I/hab/pkgs/core/make/4.2.1/20170513214620/include -DLINUX -D_REENTRANT -D_GNU_SOURCE conftest.c
configure:11491: $? = 0
configure:11491: result: yes
configure:11491: checking for lber.h
configure:11491: result: yes

#8

Output of make --dry-run

I don’t see /hab/pkgs/core/openldap included anywhere:


#9

So, if I manually add the include, this works:

[16] apr-util(do_build)> /bin/sh /hab/pkgs/core/apr/1.6.3/20171115103100/build-1/libtool --silent --mode=compile gcc -pthread -I/hab/pkgs/core/openldap/2.4.45/20180103174008/include -I/hab/pkgs/core/glibc/2.22/20170513201042/include -I/hab/pkgs/core/file/5.24/20170513201915/include -I/hab/pkgs/core/gcc/5.2.0/20170513202244/include -I/hab/pkgs/core/make/4.2.1/20170513214620/include -DHAVE_CONFIG_H -DLINUX -D_REENTRANT -D_GNU_SOURCE -I/hab/pkgs/core/glibc/2.22/20170513201042/include -I/hab/pkgs/core/file/5.24/20170513201915/include -I/hab/pkgs/core/gcc/5.2.0/20170513202244/include -I/hab/pkgs/core/make/4.2.1/20170513214620/include -I/hab/cache/src/apr-util-1.6.1/include -I/hab/cache/src/apr-util-1.6.1/include/private -I/hab/pkgs/core/apr/1.6.3/20171115103100/include/apr-1 -I/hab/pkgs/core/expat/2.2.2/20171015103808/include -o ldap/apr_ldap_stub.lo -c ldap/apr_ldap_stub.c && touch ldap/apr_ldap_stub.lo
[17] apr-util(do_build)> echo $?
0

Note the addition of -I/hab/pkgs/core/openldap/2.4.45/20180103174008/include


#10

So looking into the generated Makefile, I could see that openldap wasn’t included. This is fixed by adding a build dependency on core/pkg-config, which seems to tie it together properly.

A make --dry-run now shows the openldap include directory on all the compile operations and all but one link operations.

The make now fails at the very last step, with:

 /bin/sh /hab/pkgs/core/apr/1.6.3/20171115103100/build-1/libtool --silent --mode=link gcc  -pthread  -I/hab/pkgs/core/glibc/2.22/20170513201042/include -I/hab/pkgs/core/file/5.24/20170513201915/include -I/hab/pkgs/core/gcc/5.2.0/20170513202244/include -I/hab/pkgs/core/make/4.2.1/20170513214620/include   -lgcc_s -L/hab/pkgs/core/gcc-libs/5.2.0/20170513212920/lib -L/hab/pkgs/core/glibc/2.22/20170513201042/lib -L/hab/pkgs/core/file/5.24/20170513201915/lib -L/hab/pkgs/core/gcc/5.2.0/20170513202244/lib -L/hab/pkgs/core/expat/2.2.2/20171015103808/lib -release 1 -module -rpath /hab/pkgs/atalanta/apr-util/1.6.1/20180308191935/lib/apr-util-1 -o ldap/apr_ldap.la ldap/apr_ldap_init.lo ldap/apr_ldap_option.lo ldap/apr_ldap_rebind.lo -lldap -llber
/hab/pkgs/core/binutils/2.25.1/20170513201927/bin/ld.bfd.real: cannot find -lldap
/hab/pkgs/core/binutils/2.25.1/20170513201927/bin/ld.bfd.real: cannot find -llber

I think -lldap and -llber are added to LDADD_ldap. Not sure if this is needed, as my recollection of LDADD is that it’s used in the case where you wish to link against libraries not found my ./configure. If I delete this from the makefile manually, and run make, the build completes… but make test fails, as the -lldap -llber arguments seem to reappear.


#11

So current status is:

pkg_name=apr-util
pkg_origin=atalanta
pkg_version=1.6.1
pkg_license=('Apache-2.0')
pkg_maintainer="The Habitat Maintainers <humans@habitat.sh>"
pkg_description="Apache Portable Runtime util"
pkg_upstream_url="https://apr.apache.org/"
pkg_source=https://archive.apache.org/dist/apr/${pkg_name}-${pkg_version}.tar.bz2
pkg_shasum=d3e12f7b6ad12687572a3a39475545a072608f4ba03a6ce8a3778f607dd0035b
pkg_deps=(core/gcc-libs core/glibc core/apr core/expat core/openldap)
pkg_build_deps=(core/gcc core/make core/pkg-config)
pkg_bin_dirs=(bin)
pkg_include_dirs=(include)
pkg_lib_dirs=(lib)
pkg_pconfig_dirs=(lib/pkgconfig)

do_build() {
  ./configure --prefix="${pkg_prefix}" \
              --with-apr="$(pkg_path_for core/apr)" \
              --with-expat="$(pkg_path_for core/expat)" \
              --with-ldap-include="$(pkg_path_for core/openldap)/include" \
              --with-ldap
  attach
  make
}

I need to edit the Makefile to change LDADD_ldap = -lldap -llber to LDADD_ldap =, then make completes and the plan comes together ™.

Open questions:

  1. Are we safe just removing -lldap -llber?
  2. If so, what’s the best way to get the configure script not to append them?
  3. Or… should we just mutate the Makefile with sed

#12

I think if I were doing this plan I’d go with option 3. However, if the intention is to contribute this back to core we should probably have a larger discussion around why the makefile isn’t working for us and look for additional underlying issues.


#13

I think it’s a problem worth chasing down. This should definitely ‘just work’ and anyone else using this plan, and wanting to add LDAP support will expect it to ‘just work’.