Building install tarballs for ubuntu containers

Docker provides OS-level virtualization to allow software to run in isolation through containers. Containers are a bundle of software and configuration that are fairly well defined... But there is a way to make sure they are better defined and can deploy off-network (typical when you can't connect certain assets to your work network).

Take for example the beginning of the following docker file:

FROM ubuntu:latest

RUN apt-get update && apt-get clean all

RUN apt-get update && \
	apt-get install -y \
	lsb-release \
    curl \
    texinfo \
    g++ \
    bison \
    flex \
    automake \
    libtool \
    autoconf \
    gcc \
	cmake \
	menson \
    git \
    make \
	pkg-config

COPY my_program.sh /

RUN ./my_program.sh	

This doesn't seem to be a problem, but I notice that tools like Menson Build tend to move on to the next version of the operating system - and have breaking changes (for valid reasons). So you're telling me that Docker's promise of being able to run docker build . doesn't hold up to the claim of being future proof? Yes; if you write your Dockerfiles as described above.

Okay, so let's take a more consistent approach (and comment the Dockerfile):

# The bundled installs work on ubuntu bionic
FROM ubuntu:18.04

# make a folder for the installs
RUN mkdir -p installs

# Copy the file from the folder into the installs folder of the container
COPY installs.tar.gz /installs/

# Go to the installs folder, extract the installs, install them, 
# leave the folder, then delete the install folder
RUN	cd installs && \
	tar zxvf installs.tar.gz && \
	dpkg -i *.deb && \
	cd .. && \
	rm -rf installs

# Copy my program into the base of the container
COPY my_program.sh /

# Run my program 
RUN ./my_program.sh	

While that will work really well, it does take an additional step to package all the programs, and their dependencies, into a single bundle. Lets even take it a step further by saying out host system is amd64 and our target system is arm64.

First, let's let apt know that we want to support arm.

sudo dpkg --add-architecture arm64
dpkg --print-foreign-architectures

Now let's add the package repository. There is a somewhat general way of doing this:

sudo apt-get install -y software-properties-common lsb-release

add-apt-repository "deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports $(lsb_release -sc) main restricted universe multiverse"
add-apt-repository "deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports $(lsb_release -sc)-updates main restricted universe multiverse"

I've heard there is mixed success with this. We can always just edit the sources manually

sudo vi /etc/apt/sources.list

And then we add the following lines: This step is going to be specific to ubuntu 18.04 (bionic), but if you are reading this in the future or past, you'll have a name other than "bionic".

# source urls for arm64
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ bionic main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic main restricted
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ bionic-updates main restricted
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-updates main restricted
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ bionic universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic universe
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ bionic-updates universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-updates universe
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ bionic multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic multiverse
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ bionic-updates multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ bionic-updates multiverse

Good, we've either updated the sources list or ran add-apt-repository. Now it's time to update apt-get.

sudo apt-get autoclean; sudo apt-get update; sudo apt-get dist-upgrade

With that out of the way, it's time to download all the packages.

I have seen people claim to do it like this

for i in $(sudo apt-cache depends vim:arm64 | grep -E 'Depends|Recommends|Suggests' | cut -d ':' -f 2,3 | sed -e s/'<'/''/ -e s/'>'/''/); do sudo apt-get download $i 2>>errors.txt; done

but I personally have mixed success. Instead, I recommend using apt-rdepends. I'll show you the example for vim (getting the packages for arm):

sudo apt-get install apt-rdepends
for i in $(apt-rdepends vim:arm64 |grep -v "^ " |grep -v "^libc-dev$"); do sudo apt-get download $i:arm64 2>>errors.txt; done
sudo apt-get download vim:arm64
sudo chmod 0777 *.deb

Now we can bundle everything up

tar -czvf vim_arm64.tar.gz *.deb
sudo chmod 0777 vim_arm64.tar.gz

Congrats! You should have vim_arm64.tar.gz which you can load on the target system. On the target system, we can run something like

tar zxvf vim_arm64.tar.gz
sudo dpkg -i *.deb

Or you can get more complicated with a series of install files (which could make life easier in the Dockerfile).

#!/bin/bash
# go through the files in the current directory
for file in `pwd`/*.tar.gz
do
	# check for files you don't want to install
	if [ "${file}" == `pwd`"/to_be_ignored.tar.gz" ]
	then
		echo "Skipping ${file}"
		continue;
	fi
	echo "Installing ${file}"
	# extract the archive
	tar zxvf $file
	# install the deb files
	sudo dpkg -i *.deb
	# remove the deb files
	rm *.deb
done