============================================================================== Instructions for OpenSPARC T2 Version 1.0 Legion ============================================================================== ## ## Getting Started with Legion ## ## Legion is a fast instruction accurate simulator which provides a rapid ## means of developing and testing software functionality in the absence ## of actual hardware. Legion provides the fastest simulation environment ## for developing and testing SPARC Software and is the primary simulation ## environment for the Niagara/Niagara2 firmware and software teams. ## [0] System Requirements: - Sparc system running Solaris 8 or higher - 700MB of free disk space - Root access to the system (if you want to modify the contents of the disk image) [1] Compiling Legion: [1.1] Compiling Legion from source Legion source code is located under $SIM_ROOT/legion/src Tool Requirements: - cc compiler - gmake These are the steps needed to compile legion from source: 1) Source the OpenSPARC_SAM.cshrc provided with this bundle 2) cd legion 3) mkdir build 4) cd build 5) sh ../src/configure.opensparc 6) gmake -s This will generate a sim binary and a plugins directory within the build directory. NOTE - bin/build_legion.sh script provided does all the steps described above. Copy (or create a symbolic link) of sim binary and plugins directory in to $SIM_ROOT/S10image directory. [2] What is included: sim - Legion Simulator binary plugins - Legion simulation plugins directory reset.bin - Reset binary for basic processor configuration q.bin - Hypervisor binary q - Non Stripped Hypervisor binary (used for debugging) openboot.bin - Openboot binary disk1.img - Solaris 10 disk image (512MB, approx 250MB of free space) netcons - tool used to connect to legion console socket 1c1t/ - 1 core, 1 thread per core 1c2t/ - 1 core, 2 threads per core 1c8t/ - 1 core, 8 threads per core 2c8t/ - 2 cores, 8 threads per core 4c8t/ - 4 cores, 8 threads per core 8c8t/ - 8 cores. 8 threads per core Each of the above configs directories includes the following files: legion-guest-domain0.md - Guest Machine Description (used by OBP/OS) legion-hv.md - Hypervisor Machine Description legion.conf - Legion config file [3] Environment requirements: - Make sure your DISPLAY variable is set so that you can run an xterm and have it display on your desktop. - Your path should include the following : /usr/openwin/bin/xterm # or the path to the xterm of your choice /usr/lib/cpp # or the path where cpp lives on your machine . # Current directory [4] Running Legion: [4.1] Starting Legion: Legion need to be run with the -a option and needs to have a config file specified. The config file is called 'legion.conf' and there is a specific config file for each config (see list above). All of these configs will boot on a single processor sparc machine although the boot times will take longer for the larger configs. Legion is fully multi-threaded so it runs larger configs faster on host machines with more cpus. ./sim -a 1c1t/legion.conf [4.2] Log file of Console output: Legion maintains a log file of the Guest Console. This output is written to a file called guest1.log. The name of this file can be changed by modifying the filename in the legion config files (legion.conf). All output is appended to this file. [4.3] Booting Solaris from the ok prompt: Once you get the ok prompt, you can : [-] View the number of processors in your config: ok .cpu-state [-] Boot from the disk by typing 'boot vdisk' or 'boot vdisk -vV' for a verbose boot or boot without any Solaris services for a quicker boot time by 'boot vdisk -m milestone = none'. ok boot vdisk -or- ok boot vdisk -vV -or- ok boot vdisk -m milestone=none [-] There is no root password on this disk so just login as root and hit return. [4.3] Killing the simulation When you are done with the simulation, you can terminate it by: - Ctrl-C in the window where you ran legion - ~z in the Guest console window [5] Debug Tools [5.1] Verbose mode: Legion can be started in verbose mode using the -v or --verbose flag # ./sim -v -a 1c1t/legion.conf [5.2] Debug mode: Legion can be started in debug mode which will dump out debug state while the simulation is running. An example is if you want to see the %pc and all sparc instruction that the simulator is executing along with the register and relevant processor state at each instn you would use -debug 0x2 # ./sim -debug 0x2 -a 1c1t/legion.conf The list of debug flags that you can pass is: 0000.0000.0000.0001 Parsing messages, initialization, 0000.0000.0000.0002 PC & Instruction & Registers 0000.0000.0000.0004 PC & Instruction 0000.0000.0000.8000 Hypervisor Calls 0000.0000.0001.0000 Performance Counters 0000.0000.0002.0000 Exceptions, XIR 0000.0000.0004.0000 hpstate changes 0000.0000.0008.0000 MMU 0000.0000.0010.0000 MMU miss 0000.0000.0020.0000 Memory controllers 0000.0000.0040.0000 TSTATE, etc. 0000.0000.0080.0000 Error injection 0000.0000.0100.0000 SSI interrupts 0000.0000.0200.0000 Little Endian 0000.0000.0400.0000 Alignment errors 0000.0000.0800.0000 Mondo 0000.0000.1000.0000 SOFTINT 0000.0000.2000.0000 TICK 0000.0000.4000.0000 Instruction decode 0000.0000.8000.0000 Device 0000.0001.0000.0000 Execution Loop 0000.0002.0000.0000 Error Traps You can combine multiple debug flags by OR'ing them together. So if you want to see all the hypervisor calls (0x8000) along with the parsing information (0x1), you can run it as follows: # ./sim -debug 0x8001 -a 1c1t/legion.conf [5.3] Dynamic tilde escape sequences: Once you have started the simulation you can use the ~ (tilde) escape sequence in your Legion Guest Console window to dump out some processor state. The list of tilde commands supported is as follows: # BREAK ~ generate tilde z exit simulator i dump I-TLB contents for CPUs in same domain d dump D-TLB contents for CPUs in same domain b toggle the debug output enable bits n dump the current and delta instruction count for each CPU ? this message So if you want to turn on the %pc and instruction tracing after you have started the simulation, you can type ~b in the Legion Guest console window. Typing this gain will turn off the extra debug output. If you want to view the contents of the I-TLB (Instruction TLB) you can dump it out for all cpus by typing ~i in the Guest Console window. The output will appear in the window where you started legion. [6] Modifying the Disk Image [6.1] Adding/Removing files to/from the disk If you want to add/remove/modify files on the disk image provided, you will need root access on a system. Here are the instructions for doing this on a Solaris system. # Use the loopback file driver to moun the disk image: /usr/sbin/lofiadm -a /export/legion/disk1.img # Mount this device onto a mount point: # where /dev/lofi/1 is the value returned from lofiadm /usr/sbin/mount /dev/lofi/1 /mnt cd /mnt You can now add/remove files to/from your disk image If you get permission problems when accessing the files: 1) As root, chown/chmod the file/dir that you are modifying 2) only modify files as the root-user on your host machine Once you are done, you can umount the disk image and boot the simulator. It's always advisable to unmount the disk image before booting it in the simulator. /usr/sbin/umount /mnt /usr/sbin/lofiadm -d /dev/lofi/1 [6.2] Making your disk writable By default, legion mounts it's disk image as read-only. That means that any changes to the disk while running legion will not be written back to the disk image file on the host machine. If you want to preserve the changes to the disk, you will have to mount the disk under legion in write mode (or shared). To do this: - edit your legion config file and locate the block where the disk is mounted (search for virtual_disk). - change the rom keyword to shared - now when you boot legion, any changes to the file system within the simulation will be reflected on the disk image file. You can get log files off the disk image my lofi mounting it as described above. WARNING!!!: When mounting a disk as shared, you must gracefully exit the simulation to avoid file system corruption on the disk image. To do this, sync;halt Solaris within the simulation and wait until you return to the ok prompt before terminating the simulation by typing ~z in the Guest Console window. WARNING!!! [7] Limitations of this Environment: - Legion only implements enough of the processor state to boot the firmware and software needed to boot Solaris. There are many areas of the processor that are not needed in this boot flow and are most likely not implemented. Areas like caches, RAS, are not implemented in legion. - You cannot telnet into or out-of the simulation at this time.