Overview of Emulab

Terminology

• NS script: Emulab uses the "NS" ("Network Simulator") format to describe network topologies. This is substantially the same Tcl-based format used by ns-2. Since Emulab offers emulation, rather than simulation, these files are interpreted in a somewhat different manner than ns-2. Therefore, some ns-2 functionality may work differently than you expect, or may not be implemented at all.
• project: a logical entity grouping related users, images, experiments and data.
• experiment: An experiment defined using an NS script.
• swap-in: Swapping in an experiment causes Emulab to attempt to build a topology and install nodes based on the description provided in the experiments NS file. After a successful swap-in, all the nodes defined in your NS file have been installed with the OS specified (or default, if none was specified).
• swap-out: Swapping out an experiment causes the network topology to be torn down and the nodes to be released and reinstalled with the default OS image. After a swap-out, the experiment is no longer consuming resources.
• Terminate: Terminate means DELETE. Terminating an experiment DELETES the experiment and its NS file/emulab logs.
• Max Duration: The maximum duration that an experiment can remain swapped in.
• *Idle-Swap": Emulab will automatically swap an experiment out if it detects that it has been idle for this long.
• control-plane: The Ethernet network that is used by emulab to image and control the experimental nodes. Also for users to access the nodes (e.g. via ssh).
• *data-plane": A separate network that is reserved for experimental purposes.

Emulab works by installing pre-made OS images on multiple nodes over a control network. The nodes are customized when they boot the OS image for the first time. This process is fully automated by Emulab, the user just defines which nodes types they want and how many in an NS file. A basic Emulab installation consists of two control servers (boss and ops) one or more physical networks and collections of nodes. A user creates and experiment, describing nodes and network topology in NS file. An NS file generally specifies hostnames, OS images and node types. Users can also define networks and special links between nodes. Note: As of July 2015, network control in NARWHAL Emulab is not complete, so make-lan and related NS commands will fail.

When an experiment is "swapped in", the Emulab boss node gets to work creating the experiment as defined in the NS file. Nodes are assigned randomly from the available pool. If a requested OS for a node is not in the default image, the node will be reinstalled. The boss server uses a multicast tool called frisbee to transmit new OS images to the nodes. As nodes are made ready with an image containing the requested OS, they are rebooted into the target OS. Emulab control software running on the target OS sets up mount points, starts the Emulab event system and runs any startup commands specified in the NS file. Daemons continue running on the nodes and report state back to boss. Finally, DNS entries are created using the format $nodename.$experiment_name.$project_name.narwhal.pdl.local.cmu.edu. Nodes are accessible anywhere on the PDL network or through the PDL central Proxy Servers. Once your swapin is successful, you can log into your nodes from the Emulab "ops" node, ops.narwhal.pdl.cmu.edu. Emulab manages a public/private SSH key pair in your home directory, which is used by your nodes to authenticate you. Do not directly modify your SSH keys directly. Instead, use the Emulab web interface to manage your keys. Also, do not modify the contents of $HOME/.ssl, as it is also used by Emulab. If you were given local_root or group_root privileges, you will have full sudo privileges in the experiment nodes.

WARNING: Do not store private keys in your home directory that can be used to access other resources, like your workstation.

When an experiment is "swapped out", the nodes are rebooted and reimaged with the default image. No information from the local disks is saved.

"Terminate" means "delete" When an experiment is "terminated", it and all associated files are deleted--including log files. This means that the admins can't review the log file if you terminated the experiment. If you have problems swapping an experiment do not terminate it if you want us to take a look at it.

To get your first experiment running, log into the web interface on boss:

• https://narwhal.pdl.cmu.edu/
• Join a project. (Experimentation->Join an Existing Project). If you are collaborating with someone, please join an existing experiment. If you wish to start a new project, please ask your advisor to request it. They can do so here: https://narwhal.pdl.cmu.edu/newproject.php3.
• Create an experiment. (Experimentation->Begin an Experiment)
• Choose the project & group.
• Write a useful description.
• Choose an NS file (or specify one on ops)
• Choose reasonable timeouts for Max Duration and Idle-Swap. Be courteous to other researchers.
• Select "Skip Linktest"
• Select "Do Not Swap In"
• Click Submit

Once your experiment is set up, click "Swap Experiment In" to start it. (WARNING: "Terminate Experiment" will swap out and delete your experiment and all of its log files.) You can modify your experiment at any time, even when it is swapped in.

NS File Example:

This is a basic NS file with two nodes. It will use the default OS image for each node. (default for rr nodes is UBUNTU14-64-NFS)

set ns [new Simulator]
source tb_compat.tcl

#node1
set node1 [$ns node]
# this tells emulab what node class to use
node1 add-desire rr 1.0
# the node OS can be set using text or using variables defined above
#UBUNTU14-64-PROBE is the default OS.  This is here for illustration:

#node2
set node2 [$ns node]
node2 add-desire rr 1.0

$ns rtproto Static
$ns run

This is a more advanced NS file for a hypothetical Hadoop cluster with 3 master nodes and 5 slaves. A custom OS image is specified. The number of slaves can be changed while the experiment is swapped in.

set ns [new Simulator]
source tb_compat.tcl

# for convenience
set opt(OSIMAGE) CDH5
set opt(SLAVES) 5

#manager
set manager [$ns node]
node1 add-desire rr 1.0
# the node OS can be set using text or using variables defined above
tb-set-node-os manager CDH5MGR
tb-set-node-startcmd manager "/proj/pdl/cdh5/start manager"

#primary namenode
set pnn [$ns node]
node1 add-desire rr 1.0
tb-set-node-os pnn $opt(OSIMAGE)
tb-set-node-startcmd pnn "/proj/pdl/cdh5/start namenode"

#jobtracker
set jt [$ns node]
node1 add-desire rr 1.0
tb-set-node-os jt $opt(OSIMAGE)
tb-set-node-startcmd jt "/proj/pdl/cdh5/start jobtracker"

for {set i 1} {$i <= $opt(SLAVES)} {incr i} {
    set c$i [$ns node]
    c$i add-desire rr 1.0
    tb-set-node-os c$i $opt(OSIMAGE)
    tb-set-node-failure-action c$i "nonfatal"
    tb-set-node-startcmd c$i "/proj/pdl/cdh5/start computenode"

}

$ns rtproto Static
$ns run

More information about NS commands can be found on the Emulab wiki: https://wiki.emulab.net/wiki/nscommands