Colosseum Development Guide
Warning
This page is deprecated and needs to be updated
This document provides resources to guide the user in developing software to successfully interact with the hardware on the SRN and the interfaces required by Colosseum. Specific documentation and guides for developing software for those resources is beyond the scope of this guide, and the reader is referred to the references listed at the end of this document for more information.
APIs
Radio Command and Control (C2) API describes the Command and Control API that user containers must support for Batch Mode, Scrimmages, and Events
Collaboration Protocol Specification describes the information exchange to be used collaboratively by CIRNs as well as the procedures for providing updates to the Collaboration Protocol.
SRN Configuration
For users seeking to replicate the SRN configuration locally, see the instructions for Setting up and using a local SRN.
SRN Base Container Software Specifications
Base Container Software List
Software |
Version |
Notes |
|---|---|---|
Ubuntu |
Ubuntu 16.04.3 LTS (Xenial) |
|
Linux Kernel |
4.4.0-109-generic |
|
Nvidia Driver |
387.26 |
|
CUDA |
9.1.85-1_amd64 |
|
GNURadio |
3.7.10 |
git clone –recursive –branch v3.7.10 https://github.com/gnuradio/gnuradio.git |
UHD |
3.9.5 |
git clone –branch release_003_009_005 https://github.com/EttusResearch/uhd.git |
ColosseumCLI |
2.2.3 |
|
gpsd |
3.17 |
git clone –branch release-3.17 https://git.savannah.gnu.org/git/gpsd.git |
Base Container Default Password
One time root password for both machines is ChangeMe. Users can change their Colosseum passwords when logged into either the SSH Gateway or the File Proxy using the unix passwd utility. This will change your Colosseum account password which is used both on the Colosseum website and to access the File Proxy server within the Colosseum. Currently, password changes through the user website are not supported.
Prerequisites
Users must first Upload their SSH Public Keys.
Users should have configured their SSH client following SSH Proxy Setup.
Users must know their current password.
Password Change Instructions
Users should ssh into the SSH gateway from their local machine using:
~$ ssh gw.colosseum.net
user@gw:~/$
Then, type passwd at the command line to begin the password change. Enter your current Colosseum password, then enter your new password twice:
user@gw:~/$ passwd
Enter login(LDAP) password:
New password:
Re-enter new password:
If the password change is successful, the user will see the following response:
LDAP password information changed for user
passwd: password updated successfully
See the man page for passwd:
man passwd
ColosseumCLI
The base containers have the ColosseumCLI pre-installed. To install ColosseumCLI in a different container, see instructions for Installing or Updating ColosseumCLI into a container: ColosseumCLI
SRN Hardware Resources
This section details how users can access the attached SDR, GPU, and memory from within the LXC container development environment.
Standard Radio Node (SRN) provides a platform for software defined radio and machine learning applications
SRN Host Hardware Specifications
The SRN has 3 key hardware components:
Dell R730 Server
Ettus X310 USRP Software Defined Radio
NVIDIA K40 GP-GPU
Hardware |
Dell PowerEdge R730 Server (210-ACXU) PE R730/xd Motherboard MLK (329-BCZK) |
Processors |
Intel Xeon E5-2650 v4 2.2GHz,30M Cache,9.60GT/s QPI,Turbo,HT,12C/24T (105W) Max Mem 2400MHz (338-BJDV) Intel Xeon E5-2650 v4 2.2GHz,30M Cache,9.60GT/s QPI,Turbo,HT,12C/24T (105W) Max Mem 2400MHz (338-BJDW) |
Graphics Processing Unit |
NVIDIA Tesla K40M GPU (490-BBSQ) R730 GPU Installation Kit (490-BCDP) |
Memory |
128GB [16GB RDIMM, 2400MT/s, Dual Rank, x8 Data Width (370-ACNX) x8 2400MT/s RDIMMs (370-ACPH)] Performance Optimized (370-AAIP) |
Hard Drives |
Chassis with up to 8, 3.5” Hard Drives, Software RAID (350-BBEM) Bezel (350-BBEJ) 1TB 7.2K RPM SATA 6Gbps 3.5in Hot-plug Hard Drive,13G (400-AEEZ) 2x |
Networking Adapters |
R730/xd PCIe Riser 2, Center (330-BBCO) R730 PCIe Riser 3, Left (330-BBCQ) R730 PCIe Rise 1 Filler Blank, Right (374-BBHS) Qlogic 57810 Dual Port 10Gb Base-T Network Adapter (540-BBBD) Qlogic 57800 2x10Gb BT + 2x1Gb BT Network Daughterboard (540-BBBZ) iDRAC8 Enterprise, integrated Dell Remote Access Controller, Enterprise (385-BBHO) |
10GbE SFP+ Network Card |
|
USRP Base Unit |
Ettus X310 |
USRP Bandwidth |
100Mhz Total / 80Mhz usable |
USRP FPGA Resources |
XILINX Kintex 7 – 410T Logic Cells: 406K Memory: 28,620 Kb Multipliers: 1540 Clock Rate: 200Mhz Streaming Bandwidth per Channel (16-bit): 200MS/s |
USRP Daughterboard |
UBX 160LP: Modified Ettus UBX 160 for reduced power output and increased RX/TX isolation |
USRP Networking |
10G ethernet |
USRP Reference |
Internal 10 MHz https://www.ettus.com/content/files/X300_X310_Spec_Sheet.pdf |
Ettus USRP X310 SDR
The USRP attached to each SRN is connected by a 10Gbps Ethernet interface which is used for control and data transfer to and from the USRP. In the baseline configuration within the SRN, the USRP is controlled by UHD over the Ethernet interface (see References for more details on using UHD with the USRP X310). Within the Base LXC container, the USRP Ethernet interface is mapped as usrp0 and can be reached from the SRN at the default IP of 192.168.40.2.
See USRPs for information on interacting with the USRP X310. For additional information on using the USRPs or the UHD driver, see the information listed below in the References section.
Radio Command and Control (C2) API
At this time we do not intend to connect the SRN USRPs to an external precision 10MHz reference or 1PPS.
Important
DO NOT USE THE uhd_image_loader UTILITY PROVIDED IN UHD TO FLASH THE FPGA. SEE DIRECTIONS IN USRPs
NVIDIA GPU
Each SRN contains NVIDIA GPU resources which may be used by the users within their radio application, should they choose to do so. The specifics of developing software to leverage NVIDIA GPUs are beyond the scope of this document, and the reader is referred to the References section below. Within the Base LXC Container, the following GPU devices are mapped and available as:
/dev/nvidia0
/dev/nvidia-uvm
/dev/nvidiactl
See GPUs for more information on interacting with the NVIDIA GPU. For additional information on using or the NVIDIA GPU or CUDA, see the information listed below in the References section.
On-Board Memory
Each SRN is outfitted with on-board RAM, which the user may use as needed. Of course, portions of the available RAM will be used to run the SRN host operating system and user LXC container, but the remainder may be used to run supporting software, radio applications, or mapped to a RAM disk local to the SRN. A RAM disk may be useful to support fast file read/write operations during radio application execution. By default, there are no RAM disks configured in the Base LXC container, but users may allocate memory as desired to create RAM disks.
To mount a RAM disk, first create a directory at a desired location (in this example, /media/ramdisk), if one does not already exist. Note that folder permissions may require sudo privileges to create a folder, which is the case in this example.
sudo mkdir /media/ramdisk
If needed, change the owner and permissions of the directory to allow non-sudo access to the mounted disk.
sudo chown user:user /media/ramdisk
sudo chmod +rw /media/ramdisk
Then, mount a tmpfs disk (in this example, the disk size is 1 GB) to that location.
sudo mount -t tmpfs -o size=1024M tmpfs /media/ramdisk
This is a temporary storage drive. Users should take measures to save data (or the entire ramdisk) prior to SRN deallocation should they need to maintain any data.
Traffic Generation
Should users want to test traffic input to their radio designs prior to when the Colosseum traffic generation system is online, they may use networking tools available within the container to generate basic IP traffic. For some recommendations on how to test network connectivity into your application, see Traffic Generation.