Building install tarballs for ubuntu containers

Posted in software by Christopher R. Wirz on Fri May 25 2018

Docker provides OS-level virtualization to allow software to run in isolation through containers. Containers are a bundle of software and configuration that are fairly well defined... But there is a way to make sure they are better defined and can deploy off-network (typical when you can't connect certain assets to your work network).

Note: This example does not address the breaking changes between Ubuntu 16.04 and 18.04, but does show how to overcome similar events in the future using Docker

Take for example the beginning of the following docker file:

FROM ubuntu:latest

RUN apt-get update && apt-get clean all

RUN apt-get update && \
	apt-get install -y \
	lsb-release \
    curl \
    texinfo \
    g++ \
    bison \
    flex \
    automake \
    libtool \
    autoconf \
    gcc \
	cmake \
	menson \
    git \
    make \


RUN ./	

This doesn't seem to be a problem, but I notice that tools like Menson Build tend to move on to the next version of the operating system - and have breaking changes (for valid reasons). So you're telling me that Docker's promise of being able to run docker build . doesn't hold up to the claim of being future proof? Yes; if you write your Dockerfiles as described above.

Okay, so let's take a more consistent approach (and comment the Dockerfile):

# The bundled installs work on ubuntu bionic
FROM ubuntu:18.04

# make a folder for the installs
RUN mkdir -p installs

# Copy the file from the folder into the installs folder of the container
COPY installs.tar.gz /installs/

# Go to the installs folder, extract the installs, install them, 
# leave the folder, then delete the install folder
RUN	cd installs && \
	tar zxvf installs.tar.gz && \
	dpkg -i *.deb && \
	cd .. && \
	rm -rf installs

# Copy my program into the base of the container

# Run my program 
RUN ./	

While that will work really well, it does take an additional step to package all the programs, and their dependencies, into a single bundle. Lets even take it a step further by saying out host system is amd64 and our target system is arm64.

First, let's let apt know that we want to support arm.

sudo dpkg --add-architecture arm64
dpkg --print-foreign-architectures

Now let's add the package repository. There is a somewhat general way of doing this:

sudo apt-get install -y software-properties-common lsb-release

add-apt-repository "deb [arch=arm64] $(lsb_release -sc) main restricted universe multiverse"
add-apt-repository "deb [arch=arm64] $(lsb_release -sc)-updates main restricted universe multiverse"

I've heard there is mixed success with this. We can always just edit the sources manually

sudo vi /etc/apt/sources.list

And then we add the following lines: This step is going to be specific to ubuntu 18.04 (bionic), but if you are reading this in the future or past, you'll have a name other than "bionic".

# source urls for arm64
deb [arch=arm64] bionic main restricted
deb-src bionic main restricted
deb [arch=arm64] bionic-updates main restricted
deb-src bionic-updates main restricted
deb [arch=arm64] bionic universe
deb-src bionic universe
deb [arch=arm64] bionic-updates universe
deb-src bionic-updates universe
deb [arch=arm64] bionic multiverse
deb-src bionic multiverse
deb [arch=arm64] bionic-updates multiverse
deb-src bionic-updates multiverse

Good, we've either updated the sources list or ran add-apt-repository. Now it's time to update apt-get.

sudo apt-get autoclean; sudo apt-get update; sudo apt-get dist-upgrade

With that out of the way, it's time to download all the packages.

I have seen people claim to do it like this

for i in $(sudo apt-cache depends vim:arm64 | grep -E 'Depends|Recommends|Suggests' | cut -d ':' -f 2,3 | sed -e s/'<'/''/ -e s/'>'/''/); do sudo apt-get download $i 2>>errors.txt; done

but I personally have mixed success. Instead, I recommend using apt-rdepends. I'll show you the example for vim (getting the packages for arm):

sudo apt-get install apt-rdepends
for i in $(apt-rdepends vim:arm64 |grep -v "^ " |grep -v "^libc-dev$"); do sudo apt-get download $i:arm64 2>>errors.txt; done
sudo apt-get download vim:arm64
sudo chmod 0777 *.deb

Now we can bundle everything up

tar -czvf vim_arm64.tar.gz *.deb
sudo chmod 0777 vim_arm64.tar.gz

Congrats! You should have vim_arm64.tar.gz which you can load on the target system. On the target system, we can run something like

tar zxvf vim_arm64.tar.gz
sudo dpkg -i *.deb

Or you can get more complicated with a series of install files (which could make life easier in the Dockerfile).

# go through the files in the current directory
for file in `pwd`/*.tar.gz
	# check for files you don't want to install
	if [ "${file}" == `pwd`"/to_be_ignored.tar.gz" ]
		echo "Skipping ${file}"
	echo "Installing ${file}"
	# extract the archive
	tar zxvf $file
	# install the deb files
	sudo dpkg -i *.deb
	# remove the deb files
	rm *.deb