Lab 5: Software Development Tools

CMSC 240
Fall 2024
Due: December 6th, 2024 by 23:59:59pm

Introduction

Web sites and many other applications are often developed using both a development
and a production environment. The production environment runs the stable, tested
version of the software that customers, clients, or other developers see. The
development environment contains software that is being actively developed. This
allows an organization to develop and test new features or software patches before they
are pushed out to place where a failure could cause data loss or other problems.

Once development code has been thoroughly tested and is ready for deployment, we
need an easy way to move it to the production environment. In the past, this was often
done manually, which sometimes created problems when new software didn’t integrate
properly with existing (older) code or when the wrong files got copied to the wrong
places.

This is particularly difficult when a software package has many dependencies: other
pieces of software that it needs in order to function, such as code libraries, utility
software, or management scripts. Software that works fine with one version of these
tools may not work at all with a newer (or older!) version. This means that even if
software works perfectly in a development environment, it may break when moved to
production – even with careful testing.

In order to avoid these problems, most organizations make use of two technologies:
virtual machines and containers.

A virtual machine is a program that simulates a computer system. This has two
advantages:
First, we can run software that is incompatible with a particular machine on a
virtual host that simulates one that is compatible.

Second, software running in a virtual environment is largely isolated from the

system it is running on. A security flaw or other software bug that affects
those programs does not affect the other systems running on the computer system.

Virtual machines do have one major drawback: they use lots of resources. Instead of a
single operating system, we now need two: one for the host computer and one for the
virtual computer. Virtualization also introduces additional memory and computational
overhead. While modern hardware has many features that reduce this overhead, in
practice the additional memory and computation time required often impacts the
efficiency of the software.

An alternative to virtual machines is to use containers. A container is an isolated

environment within an operating system. Software in a container cannot access the
files, user accounts, or resources of other containers on the system (or the host system)
except in very specific, well defined ways.
Both virtual machines and containers support the use of images. These are single files
that contain a complete copy of a virtual machine or container. Creating an image of a
virtual machine or container allows us to reproduce that environment on another
computer – or to make multiple copies on the same computer. This can help address
the dependency problem. Instead of deploying our development version in a new
environment with new dependencies, we can test it inside a container and then deploy
the entire container (dependencies and all) to production.

Another advantage of containers is that once someone has created an image file, they
can share it with others. This means that someone else can install, configure, and
deploy some software for us – and all we have to do is download the image and run it
inside a container.

Step 0. Getting Started

Log in to one of the lab systems and create a folder named “Lab5”. Make it your current
working directory with “cd”.

We are going to need to connect to some network services in the lab for this project. If
you are working remotely, you will need to use SSH tunneling when you connect, since
the lab is behind a firewall.

Read the section on “Tunneling” at https://www.cs.longwood.edu/RemoteLogin.html

and then configure your SSH client to forward port 3000 on the server to port 3000 on
your laptop and port 10224 on the server to port 10224 on your laptop.

Step 1. Running a Container

One of the most popular container frameworks is Docker. One powerful feature of
Docker is the “Docker Hub” – a web site that contains hundreds of free container images
you can download and use. Let’s give this a try. Type:
 docker run --name 240game -p 3000:3000 zsoltm/word-game:0.0.3

This will download and run a container image named “word-game” from Docker Hub
(“zsoltm” is the name of a user who uploaded this container to the web site – and also
the name of the repository to which he or she publishes images).

All docker commands start with the word “docker” followed by an action. In this case,
the action is “run”. We add some arguments to configure how Docker runs this
container:

--name 240game Sets the name of the container to

make managing it easier

-p 3000:3000 Maps network port 3000 inside the

container to the host’s port 3000
Step 2. Testing the Container

In order to test this, we need to connect to port 3000 on one of the lab systems. If you
are in the lab or have set up port forwarding, this is easy. Just open up a web browser
and browse to http://localhost:3000.

You should see a word game. It’s hard! Feel free to play a round to see how you do.

Notice how easy it was to install this game! One Docker command was all it took to
launch a complete web application, written in NodeJS and AngularJS, inside a container
environment.

Not only didn’t you have to write the code for this – you didn’t have to install a web
server, configure networking for it (other than possibly forwarding the network port),
edit any configuration files, create a user account for the server, set permissions, or
install any libraries, tools, or other dependencies. This was all taken care of by the
author of the container image.

Step 3. Stopping the Container

Notice that your terminal window seems “frozen”. It is running the word game and you
can’t do anything else with it. To fix this, we need to stop the docker container. Open
up a second terminal window and type:

docker stop 240game

Notice that we are using “240game” (the name we assigned to the container when we
ran it). We could instead have used the unique identifier docker assigns to each
container when it is launched – but using the name is simpler.

Note: If you forgot to give your docker container a name, Docker gives it one for you. It
selects an adjective_noun combination from a set of silly names to generate a unique and
random string for each new container. You can type docker ps to see a list of running
containers and their ids.

Step 4. Building a Container

While Docker Hub has hundreds (thousands!) of images, for most applications we want
to either create our own container or customize an existing one in some way. Docker
allows us to do this by creating a Dockerfile – a text document that serves as a recipe for
a container.
Let’s build a container that will host a simple web site.

First we are going to create a folder to hold our files. In the Lab5 folder, type:

mkdir html

Do NOT use “cd” to change to this directory yet. Stay in your Lab5 folder.

Then create a file named “Dockerfile” (it MUST be spelled and capitalized exactly like
this!):

vim Dockerfile

Now add the following (don’t forget to go to insert mode first!)

FROM nginx
COPY html /usr/share/nginx/html

Save and quit.

The FROM directive says to start from the nginx web server container and use it as a
base. Docker will download this container from the Docker Hub web site.

The COPY directive says to copy everything from our html/ folder into the container
when we build it. We will place these files in /usr/share/nginx/html inside the container,
which is where the web server is configured to look for pages.

We can now build the container with:

docker build -t myweb .

Pay close attention! That little “dot” at the end is important. It says to look for the
Dockerfile in the current directory! This will take a little while to run – Docker has to
download the nginx container from DockerHub and then customize it by following the
commands you wrote in the Dockerfile.

You can now run your container with:

docker run --rm -it --name web -p 10224:80 myweb

And test it by browsing to http://localhost:10224. Of course, we haven’t put any pages

there yet, so will only see a placeholder page for now.
You can stop the container with CTRL-C in your terminal window when you are done.

Here are some other important directives you can put in a Dockerfile (you will need
these later!):

FROM Sets the base container

COPY Copies files from the host system into the container image
RUN Runs any Unix command inside the container
WORKDIR Sets the current working directory (and creates it, if needed)
ENV Sets an environment variable within the container
CMD Defines the command to run when the container is launched

Probably the most important is the RUN command. One really common way to use this
is to install new files. For example, “RUN apt-get -y install vim” would install vim in the
container environment.

Step 5. Making a Web Page

Let’s make a simple web page. First, change to the html folder:

cd html

Then create a file named index.html:

vim index.html

Add this to your file (do NOT copy/paste this – type it in!):

<!DOCTYPE html>
<html lang=en>
<head>
<meta charset=utf-8>
<meta name=viewport content="width=device-width; initial-scale=1">
<link rel=stylesheet href="style.css">
</head>

<body>
<h1>What I’ve learned in CMSC 240</h1>
<h2>Name</h2>
<p>I learned what I love and hate about Linux.</p>
</body>
</html>

Save and quit.

Then rebuild your docker container with:

docker build -t myweb ..

Note that this time we used TWO dots because the current working directory is the html
folder, but the Dockerfile is in the parent folder (Lab5).

Rerun your container with:

docker run --rm -it --name web -p 10224:80 myweb

Notice that it doesn’t matter what folder we’re in when we do a “docker run”. The
docker containers are stored in a special system folder when you build them and the
docker daemon can find them from anywhere.

Change the text in the paragraph to describe something you’ve learned. Replace
“Name” with your name.

Step 6. Some Style

Right now, the HTML looks pretty ugly. Let’s fix that by adding some CSS to fix the
spacing, colors, and fonts.

I’ve created a style sheet that should work well as a base. You can download it with:

wget https://www.cs.longwood.edu/240/style.css

You can either put style.css in your html folder or add a RUN command to the Dockerfile
to automatically download it.

Rebuild your docker container and check that the new style has been applied.

Step 7. Building Software

Note: The idea for this part of the lab comes from my friend Carlisle Childress. He developed
a workshop for students in the Linux Users Group (LUG) at VCU, which I have adapted for use
here with his permission. Thanks, Carlisle!

Rogue is one of the earliest Unix “Dungeon Crawling” games [ see the article at
https://en.wikipedia.org/wiki/Rogue_(video_game) for more details]. It was so popular
that we now call an entire category of computer and board games “rogue-like” games.
We are going to download, compile, and install it.

First, download the source code with this command:

wget -c https://www.cs.longwood.edu/240/rogue5.4.4-rgm-r1.0.2-src.tar.gz

Now make a directory to hold the source code and extract the tarball there:

mkdir src
cd src
tar xvf ../rogue5.4.4-rgm-r1.0.2-src.tar.gz

This creates a directory named rogue5.4.4-rgm-r1.0.2 in your src directory. Change to

the new directory (remember to use TAB completion so you don’t have to type the whole
filename!) and then type “ls” to see the contents of the directory.

You should see a list of about fifty-six different files, some of which are scripts.

Modern software often uses tools such as “Cmake” to generate a Makefile. However,
older code instead uses the “GNU build system (often called the “autotools”). Building
software that uses the autotools requires a three step process:

1. We run a configure script. This script checks to see whether you have the required
dependencies for building the software (such as libraries, software tools, or CPU
features). It adjusts the Makefile for the project to enable or disable the parts of the
program that depend on these and configures them to use the correct file paths and
other settings.

2. We run “make” to compile the project.

3. We run “make install” to install the software on our system.

Run the configure script by typing:

mkdir $HOME/Lab5
./configure --prefix=$HOME/Lab5 2>&1 | tee configure.log

Configure produces a lot of output and you may need to do some troubleshooting. The
2>&1 is a fancy way of combining the standard output and standard error into one
stream. The tee command then displays this stream to the display, but ALSO writes it to
a file named “configure.log”.

Configure scripts are usually very flexible and can take dozens or even hundreds of
different options. However, for this lab, the only option you need to provide is the
prefix option. The prefix tells configure where to put the installed software. If you don’t
specify a prefix, configure will assume you want the binaries to go in /usr/local/bin and
any related libraries to go to the /usr/local/lib directory. You may remember that this is
where we’re supposed to put manually installed software (as opposed to /usr/bin and
/usr/lib where the system package manager puts software).

This is a pretty good default, but requires administrative access (and you don’t have
admin privileges). So instead, we are using the $HOME environmental variable to tell
the script to install the files in a Lab5 directory under your home directory.

When you run configure, it will check to see if all the libraries and helper programs
rogue needs are installed. One dependency of rogue is the “ncurses” library. If ncurses
isn’t installed, the script will fail with an error. In this case, I’ve carefully made sure that
the correct libraries are already available.

Once you’ve run ./configure and generated a Makefile, you can type “make” to compile
the project. Do that now:

make 2>&1 | tee make.log

Assuming you encountered no compilation errors, you can now use “make install” to
install the game:

make install 2>&1 | tee make_install.log

The make install command will place the executable binary and man pages in the
location specified in the prefix option ($HOME/Lab5).

You can read more about the GNU build system here:

http://en.wikipedia.org/wiki/GNU_build_system
http://en.wikipedia.org/wiki/Configure_script
http://en.wikipedia.org/wiki/Makefile
http://en.wikipedia.org/wiki/Make_(software)

To run rogue program, you can type:

~/Lab5/bin/rogue

Use Shift-Q or Ctrl-C to exit.

Of course, typing the complete path is a bit of a pain. Also, we should make sure the
manpage for rogue is properly installed.

Go back to the “Lab5/src” folder:

cd ..

And create a file named “rogue_env.sh” with the following contents:

#!/usr/bin/bash

export PATH=$HOME/Lab5/bin:$PATH
export MANPATH=$HOME/Lab5/share/man:$MANPATH

Make it executable and run the source command:

chmod +x rogue_env.sh
source rogue_env.sh

The which command will verify that rogue is now in your path:

which rogue

and now the game can be run by issuing the command from any directory (note that
you don't need to put ./ in front of it!):

rogue

There is a mistake with the location of the rogue man page (the way this works has
changed since version 5.4.4 was written), but it can be easily corrected.

cd $HOME/Lab5/share/man
mkdir man6
mv rogue.6 man6

and now the rogue man page can be accessed with:

man rogue
Step 8. Combining what you’ve learned

That may have seemed like a lot, but really, there were just three main commands:
./configure
make
make installed

These three commands are enough to install most Linux software, assuming all the
dependencies are properly configured.

Let’s create a Dockerfile that automates this entire process. In your Lab5/src folder,
create a Dockerfile that:

1. Begins from the base container “debian:bookworm”

2. Runs apt-get -y update to download the latest packages.

3. Uses apt-get to install dependencies. You will need at least:

build-essential wget libncurses-dev

4. Set your working directory to /rogue

5. Use wget to download the source code for rogue

6. Change your working directory to /rogue/rogue5.4.4-rgm-r1.0.2

7. Run configure (You do not need to set a prefix, we’ll use the default path)

8. Run make and make install (you do not need to do any redirections for logging)

9. Set the command to run to /usr/local/bin/rogue

You can build your docker container with:

docker build -t rogue .

And run it with:

docker run --rm -it --name rogue rogue

Submitting

Make a tarball of your entire Lab5 directory:

tar czvpf Lab5-lastname.tar.gz Lab5

Then log into http://marmorstein.org/~robert/submit/ and upload the tarball as usual.