==========================================================================================================================================
================================================================================================================
The purpose of this document is to create a “cook book” for SAM setup and tracing. This document will go over all the necessary steps on how to get SAM up and running so you can start tracing. Also note that SAM and blaze refer to the same thing and are used interchangeably in this document. The document describes SAM and it's features followed by descriptions on how to run SAM. Tracing and validation guidelines are explained in great detail followed by execution driven simulation. SAM combined with ValueSim can be hooked up to a cycle accurate simulator and used in execution driven mode.
==========================================================================================================================================
SAM is a full system (CPU, memory, network, hard disk, timer) simulator developed at SUN and currently maintained by the Advanced Technology group (ATG). It simulates various SPARC V9-based systems running the Solaris operating system. It is primarily used for collecting architectural traces from workloads such as TPCC, SPECWeb, ICPERF, SPECJAppServer, ICPERF, SPECJbb, SPEC CPU, etc. SAM is also being used in RTL verification and in system bringup. It boots a fake boot prom or Sun-Fire OBP, and then the real OS. Since SAM-v5 runs off-the-shelf OS, most high-level functionality built on top is available in the simulation including TCP/IP, file systems, kernel modules, multiple-users, NIS, etc.
SAM simulates a system consisting of the following:
One or more SPARC CPUs.
Physical memory (RAM)
Bridge
Switch
A serial console device
The disk subsystem (e.g. SCSI or FibreChannel)
The network cards (e.g. gem or cassini)
SAM-v5 is a new SAM revision with enhanced user interface for system configuration and new expandable and flexible infrastructure. Note that some devices are system specific and so are modeled accordingly and used in that particular SAM system. Further details in the sections that follow.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SAM-Huron (Sun Fire T2000) system configuration uses the Niagara (UltraSPARC-T2) chip as the CPU. SAM Huron consists of SAM system simulator plus the Vonk processor reference model. The following is the component list of modules in SAM Huron:
Vonk – Niagara 2 Processor Model
PCIE-PCIE Switch
PCIE-PCI Bridge
SCSI Controller
FC Controller
SCSI Disk
FC Disk
Networking – GE/CE/E1000g adapters
Miscellaneous – LL, RTC, Serial etc.
For further details please refer to https://systemsweb.sfbay.sun.com/archperf/TOI/talk-2007/qe-sam-toi.odp
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
==========================================================================================================================================
We have the following four SPARC CPU models currently modelled in SAM:
Niagara II, Huron Systems (Vonk N2)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Riesling is the SPARC CPU architecture model in the SAM Ontario (T1) full-system simulator. Vonk is used for both Niagara II and ROCK CPU models. Vonk N2 is the SPARC CPU architecture model in the SAM Huron (T2) full-system simulator. Vonk ROCK is the SPARC CPU architecture model in the SAM Supernova full-system simulator. Riesling/Vonk is the overall name for the Sun UltraSPARC processor golden reference model product, it implements the full UltraSPARC architecture specification. On top of the UltraSPARC specification, Riesling/Vonk implements the deltas from that specification as mentioned in each product specific PRM (Programming Reference Manual), but usually only the parts that are related to processor functions.
Riesling/Vonk always implements a full system model in turn of CPU architecture. A Riesling/Vonk system consists of a number of Riesling/Vonk CPUs. A Riesling/Vonk CPU consists of a number of Riesling/Vonk cores. A Riesling/Vonk core consists of a number of Riesling/Vonk strands. The structure and functionality of the Riesling/Vonk system that is provided in the Riesling/Vonk library reflects the hardware it functionally models. Riesling/Vonk also provides a simple memory model. A system is a combination of CPU model(s) and memory model(s). Together the CPU models and memory model(s) in the Riesling/Vonk library allow for a standalone simulator of CPUs with memory to be built out of the box. Riesling/Vonk itself does not provide device model functionality.
Riesling/Vonk does not model time. A single step (cycle) in Riesling/Vonk means: Fetch, Decode, Execute, and Retire the architectural state, in one single operation. In essence, Riesling/Vonk is a functional simulator, it does not provide any performance indication of the modeled CPU architecture. Caches are not modeled in Riesling/Vonk. Caches are functionally not required by the UltraSPARC specification. This means that cache coherency is also not modeled in Riesling/Vonk. Riesling/Vonk generally does not model functions related to errors and diagnostics. However, some functions related to those areas are implemented due to special modeling requirements.
Riesling/Vonk library is a collection of C++ classes. The classes are constructed in a way that mimics hardware structure. For Riesling, default UltraSPARC Ontario system configuration (one CPU, eight cores per CPU, four strands per core) is implemented in the library, users have the capability to extend the classes to create their own T1-like systems. For Vonk N2, default UltraSPARC Huron system configuration (one CPU, eight cores per CPU, eight strands per core) is implemented in the library, users have the capability to extend the classes to create their own T2-like systems.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Vcpu hides the details of the cpu specific implementation. It presents an abstract interface to the "virtual cpu".
for cpu with multiple strands it represents each strands
for cpu without strands it represent the cpu
Vcpu interface has a few components:
control interface
system interface ( Memory and IO )
trace interface
For further details on usage and other features refer to https://systemsweb.sfbay.sun.com/archperf/TOI/Oct2006/VcpuInterface.html
==========================================================================================================================================
To run SAM, launch the SAM binary from the SAM run directory. Ofcourse before running SAM Chplus we need a solaris root image, configuration files etc. The following sections explain in detail the steps that need to be done. Most of the steps are common to running SAM irrespective of whether it is SAM Chplus or SAM Huron etc.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
You will need at least Solaris 2.8 and above with at least 512 MB
of RAM on each machine. However, to simulate machine the host side
would consume about ¾th of the total memory size.
So, size accordingly. Some planning and set-up is necessary before
running SAM. This involves specifying the system configuration,
setting up all necessary input files, and making sure the host
machine and the working directory on which the simulation is run are
adequate for the task.
There are two principle ways to run
SAM: restore from a pre-existing checkpoint, and start a fresh run
from scratch. Most often, you will restore from existing checkpoints
in order to collect traces or run simulations. Restoring from a
checkpoint is easiest because the system configuration is
pre-determined and you simply need to sym-link or copy over the
necessary files into the working directory before running the
simulation. For a run from scratch, it is advisable to start with an
existing configuration as a template, and modify the necessary
configuration parameters (such as cpu count and frequency, device
hierarchy etc.). When restoring from a checkpoint, there are other
considerations that apply (see Lazy Restore).
In both cases,
the system-configuration file (usually called config.rc) is a good
starting point. This file specifies the type and number of simulated
CPUs, the amount of simulated memory, the number of simulated
devices, pointers to the mapping of real disk images to simulated
disks etc. This file also specifies the number of host threads onto
which the simulated CPUs are mapped. You should always ensure that
the number of host threads used is at least equal to the number of
real available CPUs on the host system. Furthermore, some
applications like execution-driven simulation may require a single
host thread for all the simulated CPUs.
By default, SAM will
try to allocate as much real memory as the simulated RAM. If you
decide to go with this default, ensure that the host machine has
enough RAM+SWAP (and also enough free swap) to accommodate this
amount. It is possible to map simulated memory to a disk file rather
than physical memory, in which case you need to allow enough space in
the working directory for the memory-mapped file. When the simulated
application writes to the simulated disks, the updates are written to
files called "overlays" in the working directory. You need
to anticipate the needs of the program and allow enough free space in
the working directory for the overlays.
For example, if the
simulated application is a database that performs 1000 writes of 8KB
per simulated second and you are simulating 1200 seconds, you need
enough space for 1.2M updates, or 1.2M x 8KB = 9.6 GB of free space
for overlays.
The latest release is in
/import/archperf/pkgs/sam/latest
For insturctions on how to bringover sam and build it yourself refer to: Getting and running SAM
For the list of files needed for SAM Chplus run directory refer to: Getting and running SAM
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
You will need to do serveral steps to create the root image before running Blaze. Also, you will need to login as root before proceeding. Note that root.img and swap.img are part of the file list needed to run SAM.
For further details on creating a root image refer to: #HOWTO: create a root disk image file from a reference system|outline
When we use fakeprom to build the SAM device tree we need to make modifications to the root image and for details on modifying the root image refer to: #HOWTO: modify reference root image into SAM root image|outline
When you modify root image you will notice a driver by name ll being loaded. This driver is used to give access to local host file system from SAM simulated environment.
Blaze supports LLFS (localhost lookup file system) letting you read/write files on your host machine directly from the simulated host. Thus, the simhost can access /import/arch-trace25, /import/arch-trace30/ and /tmp/logman on your host. More importantly, LLFS lets you run SAM as a user program and have access to all NFS files on SWAN, which is where your benchmark programs probably reside.
We map / on the host to /ll/root in simhost via LLFS. E.g. to access your home directory from /home/xyz. You would go to /ll/root/home/xyz in SAM.
As a added convenience, we (use symbolic links to) map the following directories in the sim host to the same directory on the underlying host:
/home, /net, /vol, /ws, /import, /usr/ccs, /usr/share, /usr/shared, /usr/dist
Thus, in the simhost, you can get to your home directory via /home/xyz as well as /ll/root/home/xyz. Tycho wrote the nifty LLFS (local lookup file system). Tycho implemented LLFS correctly, by implementing the Solaris LL file system, the Solaris LL device driver and the LL device (in SAM). /home /net /vol /ws /import /usr/ccs /usr/share /usr/shared /usr/dist.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
We use the same dd procedure for collecting data disk images. The database disks can be under SVM (Solaris Volume Manager) or under Veritas. For SVM you can use metadb to get a listing. For Veritas you can use vxprint to get a list of all disks under Veritas. You can then select the disks you want to take disk images of i.e. only those belonging to the database of interest.
For information on setting scsidisk.init with respect to SVM and Veritas refer to: SCSI init file Setup
For information on setting fcdisk.init with respect to SVM and Veritas refer to: FC init file Setup
Also during root image modification note that SVM requires /etc/path_to_inst file entries for the database disks. The disks should also have the same minor numbers as they had on the host reference machine. For further details on this refer to: #HOWTO: modify reference root image into SAM root image|outline
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Go to the directory where you want the SAM to be installed and run the following command. Note that SAM chplus, SAM vonk etc. are all in one workspace.
# bringover -p /import/archperf/ws/sam -w ./<samdir> .
# cd <samdir>
# make install-chplus
The above command will build the Chplus version of SAM. Similarly you can build for N2 using make install-n2 command. Refer to Makefile.targetspecs for a list of all target builds. The above command will create a directory install-chplus. You will find the following directories under install-chplus directory: bin, doc, etc, lib, man, pfe, rtl. The bin directory has the sam binary and the lib directory has all the loadable modules for devices.
Before you can run SAM the run directory must contain the following files:
bin/sam: sam binary
lib/*.so: the lib directory contains all the loadable modules (*.so)
config.rc: the rc file where system configuration is specified. RC file Setup
scsidisk.init: there could be more than one of these files. SCSI init file Setup
fcdisk.init: there could be more than one of these files. FC init file Setup
fakeprom or samprom: this file is needed when we use the fakeprom device tree (only used in sam chplus based systems), not needed if we boot using OBP
ufsboot: this file has to be copied from the root image /tmp/root-image/platform/sun4u/ufsboot
root.img: link to or a copy of the modified root image. Disk Image file from a Reference System
swap.img: link to or a copy of swap image. Disk Image file from a Reference System
Without networking:
# bin/sam -w -c config.rc
The -w automatically brings up the console window of the simulated machine. Make sure you have set your DISPLAY environment variable correctly.
At blaze prompt: run
With networking:
Make sure you have created a file called hostname.ge0 in /etc/system directory on root image and the file should have the machine name in it.
It could be ge0 or ce0 depending the NIC card gem or cassini being used. It could be ge0 or ge1 depending on the name specified in rc file for the device.
Start switchsim on any machine using command: # /import/blaze-trace/blz-binaries/switch-v1.6 -r ip. Switch does the following:
connects multiple blaze simulated environments together
models network delay
sync multiple blazes on time boundaries
behaves as a traditional switch in a real network
Start blaze and connect to switch
# bin/sam -w -c config.rc At blaze prompt: “simge connect <machine name>” where <machine name> is the machine on which switchsim is running At blaze prompt: “sync connect <machine name>” where <machine name> is the machine on which switchsim is running At blaze prompt: “conf” ensure correct values for all parameters, if not, change using “conf <parameter> <value>” At blaze prompt: run
Bringup ge/ce NIC
After boot login as root in the simulated machine and then run the following commands in the simulated machine:
devfsadm -i ge
ifconfig ge0 plumb
ifconfig ge0 inet <ip address> netmask <net mask>
ifconfig ge0 ether <mac address>, mac address is currently not being used in SAM
ifconfig ge0 up
Start other blazes and connect them using steps 2 and 3 above.
Check to see if the simulated machines are communicating using traditional ping etc.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Here is a sample TPCC rc file. I included TPCC because it covers all the facets of the setup.
############################################### start of rc file ###########################################################
conf ramsize 65536M
conf mmutype cheetahplus
conf bootpath /pci@1f,600000/scsi@1/disk@d,0:a
conf cpu_per_thread 1
conf stickfreq 12500000
load 0x0e00000 fakeprom 0xffd00000
load 0x00120000 ufsboot 0x00120000
write word 0x00 0x120000
write word 0x04 0x120000
write word 0x08 0x34ffff
write word 0x0c 0x120000
write word 0x10 0x2
write word 0x38 0x1234
write byte 0x3c 0x80
sysconf -p ./
sysconf cpu name=cpu0 cpu-type=SUNW,UltraSPARC-III+ clock-frequency=3000000000 dtlb-entries=32 itlb-entries=32
sysconf cpu name=cpu1 cpu-type=SUNW,UltraSPARC-III+ clock-frequency=3000000000 dtlb-entries=32 itlb-entries=32
sysconf cpu name=cpu2 cpu-type=SUNW,UltraSPARC-III+ clock-frequency=3000000000 dtlb-entries=32 itlb-entries=32
sysconf cpu name=cpu3 cpu-type=SUNW,UltraSPARC-III+ clock-frequency=3000000000 dtlb-entries=32 itlb-entries=32
sysconf cpu name=cpu4 cpu-type=SUNW,UltraSPARC-III+ clock-frequency=3000000000 dtlb-entries=32 itlb-entries=32
sysconf cpu name=cpu5 cpu-type=SUNW,UltraSPARC-III+ clock-frequency=3000000000 dtlb-entries=32 itlb-entries=32
sysconf cpu name=cpu6 cpu-type=SUNW,UltraSPARC-III+ clock-frequency=3000000000 dtlb-entries=32 itlb-entries=32
sysconf cpu name=cpu7 cpu-type=SUNW,UltraSPARC-III+ clock-frequency=3000000000 dtlb-entries=32 itlb-entries=32
conf mips 300
setreg pc 0xe00020
setreg npc 0xe00024
sysconf excalibur excalibur0
sysconf pci_bus pciA31 bridge=schizo31 -d0
sysconf pci_bus pciB31 bridge=schizo31 -d0
sysconf pci_bus pciA30 bridge=schizo30 -d0
sysconf pci_bus pciB30 bridge=schizo30 -d0
sysconf pci_bus pciA29 bridge=schizo29 -d0
sysconf pci_bus pciB29 bridge=schizo29 -d0
sysconf pci_bus pciA28 bridge=schizo28 -d0
sysconf pci_bus pciB28 bridge=schizo28 -d0
sysconf schizo schizo31 aid=31 pciA=pciA31 pciB=pciB31 -d0
sysconf schizo schizo30 aid=30 pciA=pciA30 pciB=pciB30 -d0
sysconf schizo schizo29 aid=29 pciA=pciA29 pciB=pciB29 -d0
sysconf schizo schizo28 aid=28 pciA=pciA28 pciB=pciB28 -d0
sysconf scsi scsi0 bus=pciA31 schizo=schizo31 dev=1 targets=scsidisk.init boothba -d0
sysconf rtc rtc0 bus=pciA31 dev=3 fun=0
sysconf gem ge1 bus=pciB31 dev=4 fun=0
sysconf serial console bus=pciB31 dev=5 fun=0 sam-console
sysconf ll PBBll bus=pciB31 dev=6 fun=0
sysconf fc PBBfc01 bus=pciA30 schizo=schizo30 dev=1 targets=fcdisk1.multi1.init
sysconf fc PBBfc02 bus=pciA30 schizo=schizo30 dev=2 targets=fcdisk1.multi2.init
sysconf fc PBBfc03 bus=pciA30 schizo=schizo30 dev=3 targets=fcdisk1.multi3.init
sysconf fc PBBfc04 bus=pciA30 schizo=schizo30 dev=4 targets=fcdisk1.multi4.init
sysconf fc PBBfc05 bus=pciB30 schizo=schizo30 dev=5 targets=fcdisk1.multi5.init
sysconf fc PBBfc06 bus=pciB30 schizo=schizo30 dev=6 targets=fcdisk1.multi6.init
sysconf fc PBBfc07 bus=pciB30 schizo=schizo30 dev=7 targets=fcdisk1.multi7.init
sysconf fc PBBfc08 bus=pciB30 schizo=schizo30 dev=8 targets=fcdisk1.multi8.init
sysconf fc PBBfc09 bus=pciA29 schizo=schizo29 dev=1 targets=fcdisk1.multi9.init
sysconf fc PBBfc10 bus=pciA29 schizo=schizo29 dev=2 targets=fcdisk2.multi1.init
sysconf fc PBBfc11 bus=pciA29 schizo=schizo29 dev=3 targets=fcdisk2.multi2.init
sysconf fc PBBfc12 bus=pciA29 schizo=schizo29 dev=4 targets=fcdisk2.multi3.init
sysconf fc PBBfc13 bus=pciB29 schizo=schizo29 dev=5 targets=fcdisk2.multi4.init
sysconf fc PBBfc14 bus=pciB29 schizo=schizo29 dev=6 targets=fcdisk2.multi5.init
sysconf fc PBBfc15 bus=pciB29 schizo=schizo29 dev=7 targets=fcdisk2.multi6.init
sysconf fc PBBfc17 bus=pciB29 schizo=schizo29 dev=8 targets=fcdisk2.multi7.init
sysconf fc PBBfc18 bus=pciA28 schizo=schizo28 dev=1 targets=fcdisk2.multi8.init
sysconf fc PBBfc19 bus=pciA28 schizo=schizo28 dev=2 targets=fcdisk3.multi1.init
sysconf fc PBBfc20 bus=pciA28 schizo=schizo28 dev=3 targets=fcdisk3.multi2.init
sysconf fc PBBfc21 bus=pciA28 schizo=schizo28 dev=4 targets=fcdisk3.multi3.init
sysconf fc PBBfc22 bus=pciB28 schizo=schizo28 dev=5 targets=fcdisk3.multi4.init
sysconf fc PBBfc23 bus=pciB28 schizo=schizo28 dev=6 targets=fcdisk3.multi5.init
sysconf fc PBBfc24 bus=pciB28 schizo=schizo28 dev=7 targets=fcdisk3.multi6.init
sysconf fc PBBfc25 bus=pciB28 schizo=schizo28 dev=8 targets=fcdisk3.multi7.init
sysconf fc PBBfc26 bus=pciB31 schizo=schizo31 dev=8 targets=fcdisk3.multi8.init
console-send "kernel/sparcv9/unix -vV\n"
################################################### end of rc file ###########################################################
Here is an explanation of each of the above lines:
conf ramsize 65536M – size of RAM or Memory on simulated machine
conf mmutype cheetahplus – mmu type for simulated machine conf bootpath /pci@1f,600000/scsi@1/disk@d,0:a – specifies the path of the boot device (root image), in this case c1t13d0s0
conf cpu_per_thread 1 – blaze is a multi-threaded (MP-on_MP) application. In this case the rc file specifies 8 cpus and the
cpu_per_thread of 1 means blaze creates 8 threads for 8 cpus i.e. 1 thread to handle each cpu. If the cpu_per_thread value
were 2 blaze will create 4 threads to handle 8 cpus i.e. 2 cpus per thread. Apart from the cpu threads blaze creates 1 thread
per disk controller (scsi and/or fc). The UI in blaze runs on a separate thread.
conf stickfreq 12500000 – run the binary /home/vsreeni/bin/stick on the reference machine and the output is the stick frequency.
Solaris uses STICK and STICK_COMPARE registers to keep track of time. stickfreq value defines the frequency at which the STICK
register update happens.
load 0x0e00000 fakeprom 0xffd00000 – where 0xffd00000 is the start adress in fakeprom (always verify this using
dis fakeprom, if different then change this. Fakeprom is loaded at address 0x0e00000 in memory of simulated machine.
load 0x00120000 ufsboot 0x00120000 – explanation same as for fakeprom. Just as for fakeprom you need to verify
start address of ufsboot by doing dis ufsboot.
sysconf -p ./ - indicates the directory in which the loadable modules are present in this case ./ i.e. current directory. In
latest blaze I think this is not changeable i.e. the search path for loadable modules is hard-coded to ./lib
sysconf cpu name=cpu0 cpu-type=SUNW,UltraSPARC-III+ clock-frequency=3000000000 dtlb-entries=32 itlb-entries=32 – cpu specification
conf mips 300 – Make sure this conf line is always specified after the cpu specification lines in the rc file. Useful in cpi
calculation. CPI is calculated by the formula: CPU clock frequency in Mhz / mips. For example in the above rc file the CPI will be
3000 / 300 = 10.
setreg pc 0xe00020 – this does not change unless for some reason fakeprom had to be loaded at an address different from 0x0e00000
in memory of simulated machine
setreg npc 0xe00024 – explanation same as for setreg pc above.
sysconf excalibur excalibur0 – loading of excalibur module sysconf pci_bus pciA31 bridge=schizo31 -d0 – pci bus specification and the bridge it is attached. -d0 specifies the debug level.
sysconf schizo schizo31 aid=31 pciA=pciA31 pciB=pciB31 -d0 – specification for schizo bridge chip and the names of pci buses
attached to the bridge.
sysconf scsi scsi0 bus=pciA31 schizo=schizo31 dev=1 targets=scsidisk.init boothba -d0 – specification for scsi controller, states
that scsi controller scsi0 is on bus pciA31 of schizo31 bridge chip. All the disks attached to this controller are specified in
scsidisk.init. #5.HOWTO: setup run directory scsi init file|outline sysconf rtc rtc0 bus=pciA31 dev=3 fun=0 – specification for real time clock device module
sysconf gem ge1 bus=pciB31 dev=4 fun=0 – specification for gem device module which is ethernet card
sysconf serial console bus=pciB31 dev=5 fun=0 sam-console – specification for serial console, the console of the simulated machine
sysconf ll PBBll bus=pciB31 dev=6 fun=0 – specification for access to local file system, ll module
sysconf fc PBBfc01 bus=pciA30 schizo=schizo30 dev=1 targets=fcdisk1.multi1.init – similar to scsi controller this is a
specification fc controller PCCfc01 which is attached to bus pciA30 of schizo30 bridge chip. The fc disks attached to this
controller are in fcdisk1.multi1.init. #6.HOWTO: setup run directory fc init file|outline There can be a maximum of 8 devices attached to each bus of schizo bridge chip i.e. a total maximum of 16 devices per schizo.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Here is a sample TPCC and a sample SAP scsi disk init file.
################################## start of TPCC scsidisk.init file ##############################
t13d0s0 ./root.img.fc
t13d0s1 ./swap.img
t0d0s2 /net/cop4/Simics/install/import/pauly-girl1/pauly.home2.s0 vtoc
t0d0s3 /net/cop4/Simics/install/import/pauly-girl1/pauly.home.s3
t0d0s4 /net/cop4/Simics/install/import/pauly-girl1/pauly.rootdg.s4 ################################# end of TPCC scsidisk.init file ################################# The controller number is assigned automatically. The format is: <logical name without controller number> <path to disk image> [,vtoc] If vtoc is specified then blaze uses the vtoc to read the disk image. If you specify vtoc make sure the slice is always 2 in the logical name. The above example does not work if any of the disks are under Solaris Volume Manager (SVM) control. The example below for SAP takes care of SVM issues. ################################# start of SAP scsidisk.init file ################################ t1d0s0 ./root.img t1d0s1 ./swap.img t2d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c2t5d0s0 vtoc serial-no="3BT0667C000010431BQJ" revision-id="5221" product-id="ST318404LSUN18G" vendor-id="SEAGATE" ################################ end of SAP scsidisk.init file #################################### As you can see in the above SAP scsidisk.init file example the line for t2d0s2 has lots of extra parameters that are required for SVM to correctly access disks under SVM control. The information needed can be gathered on the referenence machine using prtconf -v | grep c2t5d0s0. In this case c2t5d0s0 is the logical name of the disk on the reference machine.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Here is a sample TPCC and a sample fcdisk init file.
################################# start of TPCC fcdisk1.multi1.init file ##################################
/dev/dsk/t0d0s2 /net/cop4/Simics/install/import/10k10g/c11t0d0s2 0 vtoc 21000004cf27c89e 21000004cf27c89e
/dev/dsk/t1d0s2 /net/cop4/Simics/install/import/10k10g/c11t10d0s2 0 vtoc 21000004cf27efd7 21000004cf27efd7
/dev/dsk/t2d0s2 /net/cop4/Simics/install/import/10k10g/c11t16d0s2 0 vtoc 21000004cf27c8e1 21000004cf27c8e1
/dev/dsk/t3d0s2 /net/cop4/Simics/install/import/10k10g/c11t17d0s2 0 vtoc 21000004cf27f29a 21000004cf27f29a
/dev/dsk/t4d0s2 /net/cop4/Simics/install/import/10k10g/c11t18d0s2 0 vtoc 21000004cf27c26c 21000004cf27c26c
/dev/dsk/t5d0s2 /net/cop4/Simics/install/import/10k10g/c11t19d0s2 0 vtoc 21000004cf27e26c 21000004cf27e26c
/dev/dsk/t6d0s2 /net/cop4/Simics/install/import/10k10g/c11t1d0s2 0 vtoc 21000004cf27c16f 21000004cf27c16f
/dev/dsk/t7d0s2 /net/cop4/Simics/install/import/10k10g/c11t20d0s2 0 vtoc 21000004cf27c74a 21000004cf27c74a
/dev/dsk/t8d0s2 /net/cop4/Simics/install/import/10k10g/c11t21d0s2 0 vtoc 21000004cf27cc06 21000004cf27cc06
/dev/dsk/t9d0s2 /net/cop4/Simics/install/import/10k10g/c11t22d0s2 0 vtoc 21000004cf27c64c 21000004cf27c64c
/dev/dsk/t10d0s2 /net/cop4/Simics/install/import/10k10g/c11t23d0s2 0 vtoc 21000004cf27c6ee 21000004cf27c6ee
/dev/dsk/t11d0s2 /net/cop4/Simics/install/import/10k10g/c11t24d0s2 0 vtoc 21000004cf27c5af 21000004cf27c5af
/dev/dsk/t12d0s2 /net/cop4/Simics/install/import/10k10g/c11t25d0s2 0 vtoc 21000004cf27f9dd 21000004cf27f9dd
/dev/dsk/t13d0s2 /net/cop4/Simics/install/import/10k10g/c11t26d0s2 0 vtoc 21000004cf27cd8b 21000004cf27cd8b
/dev/dsk/t14d0s2 /net/cop4/Simics/install/import/10k10g/c11t2d0s2 0 vtoc 21000004cf27e712 21000004cf27e712 ################################ end of TPCC fcdisk1.multi1.init file ###################################### The controller is assigned automatically. The format is <logical name without controller number> <path to disk image> 0 vtoc <portname> <nodename>. <portname> and <nodename> are identical and are obtained by doing a ls -al on the logical name in /dev/dsk directory for each of the data disks. For example: lrwxrwxrwx 1 root root 85 Jul 20 11:57 c12t1d0s2 -> ../../devices/ssm@0,0/pci@19,700000/pci@3/ SUNW,qlc@4/fp@0,0/ssd@w50020f2300006207,0:c portname = nodename = 50020f2300006207 The above example works when the data disks are under Veritas control. We need extra data when data disks are under SVM control. Given below is an SAP fc init file as an example. ############################################## start of SAP fcdisk.data1.init file ########################################## t0d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c4t2d0s0 0 vtoc 210000203733adb7 200000203733adb7 revision-id="0929" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.33.ad.b7 node-wwn=20.00.00.20.37.33.ad.b7 t1d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c4t3d0s0 0 vtoc 21000020372b7c40 20000020372b7c40 revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.2b.7c.40 node-wwn=20.00.00.20.37.2b.7c.40 t2d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c4t4d0s0 0 vtoc 21000020372b81f6 20000020372b81f6 revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.2b.81.f6 node-wwn=21.00.00.20.37.2b.81.f6 t3d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c4t5d0s0 0 vtoc 21000020372b77ad 20000020372b77ad revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.2b.77.ad node-wwn=20.00.00.20.37.2b.77.ad t4d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c4t6d0s0 0 vtoc 21000020372b806e 20000020372b806e revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.2b.80.6e node-wwn=20.00.00.20.37.2b.80.6e t5d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c4t7d0s0 0 vtoc 21000020372b6ee1 20000020372b6ee1 revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.2b.6e.e1 node-wwn=20.00.00.20.37.2b.6e.e1 t6d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c4t8d0s0 0 vtoc 21000020372b81cd 20000020372b81cd revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.2b.81.cd node-wwn=20.00.00.20.37.2b.81.cd t15d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c4t9d0s0 0 vtoc 21000020372b8284 20000020372b8284 revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.2b.82.84 node-wwn=20.00.00.20.37.2b.82.84 t8d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c6t2d0s0 0 vtoc 2100002037331084 2000002037331084 revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.33.10.84 node-wwn=20.00.00.20.37.33.10.84 t9d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c6t3d0s0 0 vtoc 210000203722e506 200000203722e506 revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.22.e5.06 node-wwn=20.00.00.20.37.22.e5.06 t10d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c6t4d0s0 0 vtoc 2100002037330d60 2000002037330d60 revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.33.0d.60 node-wwn=20.00.00.20.37.33.0d.60 t11d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c6t5d0s0 0 vtoc 2100002037330d78 2000002037330d78 revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.33.0d.78 node-wwn=20.00.00.20.37.33.0d.78 t12d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c6t6d0s0 0 vtoc 2100002037330fef 2000002037330fef revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.33.0f.ef node-wwn=20.00.00.20.37.33.0f.ef t13d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c6t7d0s0 0 vtoc 21000020373318c6 20000020373318c6 revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.33.18.c6 node-wwn=20.00.00.20.37.33.18.c6 t14d0s2 /net/cop4/Simics/mnts/mnt24/sap_2005_new/c6t8d0s0 0 vtoc 2100002037330fb7 2000002037330fb7 revision-id="0728" product-id="ST39102FCSUN9.0G" vendor-id="SEAGATE" port-wwn=21.00.00.20.37.33.0f.b7 node-wwn=20.00.00.20.37.33.0f.b7 ############################################ end of SAP fcdisk.data1.init file ############################################# As you see above for SVM to correctly access data disks under SVM control we need extra parameters. The information needed can be gathered on the referenence machine using prtconf -v | grep c6t8d0s0. In this case c6t8d0s0 is the logical name of the disk on the reference machine which is the last in the above list.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
You have to have root access to that host system -
Login as root and run ”prtvtoc” on the disk where the root partition resides to get the disk configuration,
You can know where the root partition is by looking at /etc/vfstab and see the device logical link that is mounted at /. For example if the root partition is c0t1d0s0 do a prtvtoc on slice 2 of the disk i.e. c0t1d0s2.
#prtvtoc /dev/rdsk/c0t1d0s2
* /dev/rdsk/c0t1d0s2 partition map
*
* Dimensions:
* 512 bytes/sector
* 248 sectors/track
* 19 tracks/cylinder
* 4712 sectors/cylinder
* 7508 cylinders
* 7506 accessible cylinders
*
* Flags:
* 1: unmountable
* 10: read-only
*
* First Sector Last
* Partition Tag Flags Sector Count Sector Mount Directory
0 2 00 0 31273544 31273543 /
1 3 01 31273544 4094728 35368271
2 5 00 0 35368272 35368271
From above vtoc table, the root partition starts from 0 (first sector) and total in 31273544 sector counts. Slice 2 of a disk is a special slice that points to the entire disk and so when you do dd your input is slice 2 of the disk that holds the root partition. Using the following command to create a root disk image:
dd if=/dev/dsk/c0t1d0s2 of=/blaze/aa/lren/env09-root iseek=0 count=31273544
The general format is the following:
dd if=”partition 2 of the disk with root resides, partition 2 points to the entire disk” of=”disk image file name” iseek=”first sector” count=”sector count”
Follow above steps 1 and 2 for the swap partition as well to generate swap disk image file.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Before making following changes to your root disk image, mount the disk file as a lofi device, by doing
lofiadm -a "your root image file"
mount /lofi/dev/x /tmp/root-image
where x is the lofi device number from "lofiadm -a" above command and /tmp/root-image is where your root image will be mounted. Be very careful when modifying root image because you are root on the machine on which you have the root image mounted. For example if you want to delete /etc/path_to_inst remember that it is /tmp/root-image/etc/path_to_inst that needs to be deleted. Before doing rm or mv or editing any file always do pwd to make sure you are in /tmp/root-image/...
Please note,
you need to "umount" and "lofiadm -d" after you are done with the changes
/tmp/root-image will be referenced in this document as the the device your root image
Install drivers.
Copy following drivers from /import/blazetrace/blz-binaries/drivers/ to /tmp/root-image/kernel/drv/sparcv9,
pcisimd
pcisimc
bid
Solaris 9
Copy following drivers from /import/blazetrace/blz-binaries/drivers/ to /tmp/root-image/kernel/drv/sparcv9
ll
Copy following driver from /import/blazetrace/blz-binaries/drivers/ to /tmp/root-image/kernel/fs/sparcv9
llfs
Create /tmp/root-image/usr/lib/fs/llfs directory and copy /import/blazetrace/blz-binaries/drivers/mount into that directory.
Solaris 10
Copy following drivers from /import/blazetrace/blz-binaries/drivers/S10_63/ to /tmp/root-image/kernel/drv/sparcv9
ll
Copy following driver from /import/blazetrace/blz-binaries/drivers/S10_63/ to /tmp/root-image/kernel/fs/sparcv9
llfs
Create /tmp/root-image/usr/lib/fs/llfs directory and copy /import/blazetrace/blz-binaries/drivers/S10_63/mount into that directory.
Create device nodes
cd /tmp/root-image/devices
mkdir pci@1f,600000
cd pci@1f,600000
You need to do the following for root and swap only. Let us assume here that root is at /dev/dsk/c0t0d0s0 and swap is at /dev/dsk/c0t0d0s1.
mkdir scsi@0
mknod scsi@0:devctl c 50 0
mknod scsi@0:scsi c 50 1
cd scsi@0
mknod sd@0,0:a b 32 0
mknod sd@0,0:a,raw c 32 0
mknod sd@0,0:b b 32 1
mknod sd@0,0:b,raw c 32 1
scsi@0 is for c0 and it stands for controller 0. sd@0,0 is for t0 and d0 i.e. target 0 and disk 0. The :a stands for s0 or slice 0 and :b stands for s1 or slice 1. The 50 stands for major device number for scsi controller and this is used by solaris to load the appropriate device driver in this case glm. Look at /etc/name_to_major file and grep for 50. The 32 stands for major device number for scsi disk. You will find sd entry in /etc/name_to_major as well. Look at man pages for further details. Minor number calculation:
minor number = (controller_id * 120) + (target_id * 8) + (partition_id)
Examples:
c1t0d0s0: minor number = 1*120 + 0*0 + 0 = 120
c1t1d0s3: minor number = 1*120 + 1*8 + 3 = 131
Create dev links
Here is the example to link c0t0d0s0 and c0t0d0s1 to their device
nodes. If there are existing names, please do mkdir OFF and mv them
into that directory before performing the following links.
cd /tmp/root-image/dev/dsk
ln -s ../../devices/pci@1f,600000/scsi@0/sd@0,0:a c0t0d0s0
ln -s ../../devices/pci@1f,600000/scsi@0/sd@0,0:b c0t0d0s1
cd /tmp/root-image/dev/rdsk
ln -s ../../devices/pci@1f,600000/scsi@0/sd@0,0:a,raw c0t0d0s0
ln -s ../../devices/pci@1f,600000/scsi@0/sd@0,0:b,raw c0t0d0s1
Note that although you only create device nodes in step 2 for root and image you have to create these /dev/dsk links or logical names for all disks that you need for simulation for example data disks from a database. We will talk about this later in this document.
Create llfs file system mount point
cd /tmp/root-image/
mkdir ll
cd ll
mkdir root
This is used to access the local file system which will get mounted under /ll/root on the simulated machine during boot.
Administrative works in /etc directory
Save or remove host system specific files.
The list of files here is system dependent. Your target system might not have more or less than what are mentioned here.
cd /tmp/root-image/etc
mkdir OFF
mv defaultdomain dumpadm hostname* hosts mnttab path_to_inst OFF
cd /tmp/root-image/etc/rcS.d
mkdir OFF
mv S50devfsadm OFF
Modifications for /etc files
/tmp/root-image/etc/system - add following line
set pcisch:pci_buserr_interrupt=0
/tmp/root-image/etc/vfstab - for example,
#device device mount FS fsck mount mount
#to mount to fsck point type pass at boot options
#
fd - /dev/fd fd - no -
/proc - /proc proc - no -
/dev/dsk/c0t0d0s1 - - swap - no -
/dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 no -
%/ - /ll/root llfs - yes -
And any additional disks you would like to map, for example
/dev/dsk/c0t2d0s0 - /export/apps ufs 1 no -
/tmp/root-image/etc/name_to_major – find a available major number for Blaze specific devices and add
pcisimd 251
pcisimc 252
bid 253
ll 254
The device numbers assigned here need not be 251, 252 etc. Just go to the end of the file name_to_major
and start assigning numbers greater than the highest number you see in the file.
/tmp/root-image/etc/shadow – change the login password for root login
root::11431::::::
/tmp/root-image/etc/minor_perm – add the following line
pcisimc:* 0660 root sys
/tmp/root-image/etc/iu.ap - add pcisimc here otherwise the text output wont line up on Blaze console.
pcisimc 0 0 ldterm ttcompat
Blaze specific files
/tmp/root-image/etc/hosts – you need a unique name and IP address within the subnet where you are going to run Blaze
for example,
#
# Blaze internet hosts file
#
127.0.0.1 localhost loghost
129.144.10.170 moan
/tmp/root-image/etc/path_to_inst – the path_to_inst for blasé only need one line as following,
#path_to_inst_bootstrap_1
If you have database disks under SVM in your simulation setup then you need to refer to the following
section to modify path_to_inst file:
/tmp/root-image/etc/mnttab – make it a empty file for Blaze.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Certain benchmarks takes weeks to run. It is always a good idea to generate checkpoints along the way. If multiple SAM simulations are connected using switchsim and they are running in time-sync mode do the following to generate checkpoint:
At switchsim prompt: sync off
At each SAM prompt: stop
At each SAM prompt: dump <checkpointdir>
Where <checkpointdir> is the path to where the checkpoint will be stored. Make sure you have enough disk space in the partition of the <checkpointdir>. Once checkpoint is generated do the following to continue running:
At each SAM prompt: sync on
At switchsim prompt: sync on
It is not advisable to run multiple SAMs without time-sync being “on” in switchsim.
If your simulation involves only one SAM then to generate the checkpoint do the following:
At SAM prompt: stop
At SAM prompt: dump <checkpointdir>
To restore SAM from a checkpoint add “-R <checkpointdir>” to the SAM run command. Make sure your config.rc is the same as what was used during checkpoint generation.
Once your benchmark is warmed and ready to be traced you will generate a pretracing checkpoint. This is a very important checkpoint which will not only be used to generate traces but also to do execution-driven simulation by attaching SAM to a cycle-accurate CPU simulator which restores from the same checkpoint.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Frequently, you will want to restore from a large checkpoint with
a huge amount of memory and built-up disk state, but only run for a
short amount of time. Because of the short simulation duration, you
will not read every memory page or every disk block. The lazy restore
feature allows you to restore from a large checkpoint and run the
simulation with a smaller memory and disk footprint.
For
example, consider a realistic database simulation with 64GB of
simulated memory, and 10 TB of disk images with 500 GB of accumulated
disk updates (overlays).
When this checkpoint is restored,
SAM does not make a copy of the entire accumulated overlay files.
Instead, it memory-maps the overlay from the checkpoint, and only
writes newer updates to the overlay files in the working directory.
This feature is always enabled on SAM.
A similar optimization
is available for the simulated memory, where you can restore the
simulation on a machine with less available memory than simulated.
This is achieved by specifying the -nonemreserve option
to the SAM command-line.
Keep in mind that in both these
cases, SAM does not reserve memory or disk space on the real machine
for the corresponding simulated RAM/disk, and if the simulation runs
long enough, a failure can happen as no more space is available to
simulate more updates to memory or disk.
==========================================================================================================================================
To run SAM, launch the SAM binary from the SAM run directory. Ofcourse before running SAM Chplus we need a solaris root image, configuration files etc. The following sections explain in detail the steps that need to be done. Most of the steps in here are the same as for SAM Chplus. That is the reason you will links back to SAM Chplus documentation in most places. I have provided explanation for steps that are unique to SAM Huron in this section. For Getting Started, Creating a Root Image, Data Disk Images, SCSI init file Setup, FC init file Setup, Disk Image file from a Reference System, Check-pointing using Dump/Restore and Lazy Restore refer to SAM Chplus documentation.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Go to the directory where you want the SAM to be installed and run the following command. Note that SAM chplus, SAM vonk etc. are all in one workspace.
# bringover -p /import/archperf/ws/sam -w ./<samdir> .
# cd <samdir>
# make install-n2
The above command will build the Huron version of SAM. The above command will create a directory install-n2. You will find the following directories under install-chplus directory: bin, doc, etc, lib, man, pfe, rtl. The bin directory has the sam binary and the lib directory has all the loadable modules for devices.
Before you can run SAM Huron the run directory must contain the following files:
bin/sam: sam binary
lib/*.so: the lib directory contains all the loadable modules (*.so)
config.rc: the rc file where system configuration is specified. RC file Setup
scsidisk.init: there could be more than one of these files. SCSI init file Setup
1c1t-hv.bin:
1c1t-md.bin:
q.bin:
reset.bin:
openboot.bin:
nvram.bin:
root.img: link to or a copy of the modified root image. Disk Image file from a Reference System
swap.img: link to or a copy of swap image. Disk Image file from a Reference System
Without networking:
# bin/sam -c config.rc
The -w is not needed here because it is the default. The -w automatically brings up the console window of the simulated machine. Make sure you have set your DISPLAY environment variable correctly.
At blaze prompt: run
In the simulated window do the following:
After getting the ok prompt select ll device by typing
dev /pci@0/pci@0/pci108e,0@2
Type the following to create needed properties (spaces are important below)
" ll" encode-string " name" property
2 encode-int " #size-cells" property
3 encode-int " #address-cells" property
At ok prompt type to start booting
boot /pci@0/pci@0/scsi@1/disk -vV
With networking:
Make sure you have created a file called hostname.ge0 in /etc/system directory on root image and the file should have the machine name in it.
It could be ge0 or ce0 depending the NIC card gem or cassini being used. It could be ge0 or ge1 depending on the name specified in rc file for the device.
Start switchsim on any machine using command: # /import/blaze-trace/blz-binaries/switch-v1.6 -r ip. Switch does the following:
connects multiple blaze simulated environments together
models network delay
sync multiple blazes on time boundaries
behaves as a traditional switch in a real network
Start blaze and connect to switch
# bin/sam -c config.rc At blaze prompt: “simge connect <machine name>” where <machine name> is the machine on which switchsim is running At blaze prompt: “sync connect <machine name>” where <machine name> is the machine on which switchsim is running At blaze prompt: “conf” ensure correct values for all parameters, if not, change using “conf <parameter> <value>” At blaze prompt: run
In the simulated window do the following:
After getting the ok prompt select ll device by typing
dev /pci@0/pci@0/pci108e,0@2
Type the following to create needed properties (spaces are important below)
" ll" encode-string " name" property
2 encode-int " #size-cells" property
3 encode-int " #address-cells" property
At ok prompt type to start booting
boot /pci@0/pci@0/scsi@1/disk -vV
Bringup ge/ce NIC
After boot login as root in the simulated machine and then run the following commands in the simulated machine:
devfsadm -i ge
ifconfig ge0 plumb
ifconfig ge0 inet <ip address> netmask <net mask>
ifconfig ge0 ether <mac address>, mac address is currently not being used in SAM
ifconfig ge0 up
Start other blazes and connect them using steps 2 and 3 above.
Check to see if the simulated machines are communicating using traditional ping etc.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Here is a sample TPCE rc file. I included TPCE because it covers all the facets of the setup.
############################################### start of rc file ###########################################################
conf ramsize 65536M
conf mips 1000
conf cpu_per_thread 1
conf stickfreq 1417000000
load bin 1c2t-hv.bin 0x1f12080000
load bin 1c2t-md.bin 0x1f12000000
load bin ./nvram.bin 0x1f11000000
sysconf -p ./
sysconf cpu name=cpu0 cpu-type=SUNW,UltraSPARC-N2 clock-frequency=1417000000 id=0
sysconf cpu name=cpu1 cpu-type=SUNW,UltraSPARC-N2 clock-frequency=1417000000 id=1
sysconf ioram reset start_pa=0xfff0000000 size=0x10000 file=reset.bin sparse
sysconf ioram hv start_pa=0xfff0010000 size=0x70000 file=q.bin sparse
sysconf ioram obp start_pa=0xfff0080000 size=0x80000 file=openboot.bin sparse
sysconf serial_4v hypervisor-console base=0xfff0c2c000 size=0x1000
sysconf serial_4v guest-console base=0x9f10000000 size=0x51 pop log=1c2t.guest.log
sysconf tod_4v tod base=0xfff0c1fff8 size=0x8 tod=01010000002000 -d2
sysconf n2_ncu ncu
sysconf n2_piu piu bus=pciea ncu=ncu
sysconf pcie_bus pciea bridge=piu
sysconf bridge b0 bus=pciea dev=0 fun=0 secbus=pcia
sysconf bridge b2 bus=pciea dev=0 fun=2 secbus=pcib
sysconf pci_bus pcia bridge=b0
sysconf pci_bus pcib bridge=b2
sysconf scsi scsi0 bus=pcia dev=1 fun=0 targets=scsidisk1.init
sysconf ll ll0 bus=pcia dev=2 fun=0
sysconf gem ge0 bus=pcia dev=3 fun=0
sysconf fc fc1 bus=pcib dev=1 fun=0 targets=fcdisk2.1.init
sysconf fc fc2 bus=pcib dev=2 fun=0 targets=fcdisk2.2.init
sysconf fc fc3 bus=pcib dev=3 fun=0 targets=fcdisk2.3.init
sysconf fc fc4 bus=pcib dev=4 fun=0 targets=fcdisk2.4.init
sysconf fc fc5 bus=pcib dev=5 fun=0 targets=fcdisk2.5.init
sysconf fc fc6 bus=pcib dev=6 fun=0 targets=fcdisk2.6.init
sysconf fc fc7 bus=pcib dev=7 fun=0 targets=fcdisk2.7.init
sysconf fc fc8 bus=pcib dev=8 fun=0 targets=fcdisk2.8.init
sysconf fc fc9 bus=pcib dev=9 fun=0 targets=fcdisk2.9.init
################################################### end of rc file ###########################################################
Here is an explanation of each of the above lines:
conf ramsize 65536M – size of RAM or Memory on simulated machine
conf mips 1000 – Useful in cpi calculation. CPI is calculated by the formula: CPU clock frequency in Mhz / mips. For example
in the above rc file the CPI will be 1417 / 1000 = 1.417. conf cpu_per_thread 1 – blaze is a multi-threaded (MP-on_MP) application. In this case the rc file specifies 8 cpus and the
cpu_per_thread of 1 means blaze creates 8 threads for 8 cpus i.e. 1 thread to handle each cpu. If the cpu_per_thread value
were 2 blaze will create 4 threads to handle 8 cpus i.e. 2 cpus per thread. Apart from the cpu threads blaze creates 1 thread
per disk controller (scsi and/or fc). The UI in blaze runs on a separate thread.
conf stickfreq 1417000000 – run the binary /home/vsreeni/bin/stick on the reference machine and the output is the stick frequency.
Solaris uses STICK and STICK_COMPARE registers to keep track of time. stickfreq value defines the frequency at which the STICK
register update happens. Note that for Huron based systems stickfreq and cpufreq are the same.
load bin 1c2t-hv.bin 0x1f12080000 - guest partition description, currently from Legion workspace
load bin 1c2t-md.bin 0x1f12000000 – machine description, currently from Legion workspace
load bin ./nvram.bin 0x1f11000000 - a zero filled file, obtained by modifying an existing nvram file.
sysconf -p ./ - indicates the directory in which the loadable modules are present in this case ./ i.e. current directory. In
latest blaze I think this is not changeable i.e. the search path for loadable modules is hard-coded to ./lib
sysconf cpu name=cpu0 cpu-type=SUNW,UltraSPARC-N2 clock-frequency=1417000000 id=0 – cpu specification
sysconf ioram reset start_pa=0xfff0000000 size=0x10000 file=reset.bin sparse - reset code, HV workspace
sysconf ioram hv start_pa=0xfff0010000 size=0x70000 file=q.bin sparse - hypervisor, HV workspace
sysconf ioram obp start_pa=0xfff0080000 size=0x80000 file=openboot.bin sparse - OBP, OBP workspace
sysconf serial_4v hypervisor-console base=0xfff0c2c000 size=0x1000 - specification for serial console, the hypervisor console
of the simulated machine
sysconf serial_4v guest-console base=0x9f10000000 size=0x51 pop log=1c2t.guest.log - specification for serial console, the
console of the simulated machine
sysconf tod_4v tod base=0xfff0c1fff8 size=0x8 tod=01010000002000 -d2 - specification for time of day device module. -d2
specifies the debug level.
sysconf n2_ncu ncu – non-cacheable unit
sysconf n2_piu piu bus=pciea ncu=ncu – pcie interface unit
sysconf pcie_bus pciea bridge=piu - pci bus specification and the bridge it is attached
sysconf bridge b0 bus=pciea dev=0 fun=0 secbus=pcia - specification for bridge chip and the names of pci buses attached to
the bridge.
sysconf pci_bus pcia bridge=b0 - pci bus specification and the bridge it is attached.
sysconf scsi scsi0 bus=pcia dev=1 fun=0 targets=scsidisk1.init - specification for scsi controller, states that scsi controller
scsi0 is on bus pcia. All the disks attached to this controller are specified in scsidisk.init.
#5.HOWTO: setup run directory scsi init file|outline sysconf ll ll0 bus=pcia dev=2 fun=0 - specification for access to local file system, ll module
sysconf gem ge0 bus=pcia dev=3 fun=0 - specification for gem device module which is ethernet card
sysconf fc fc1 bus=pcib dev=1 fun=0 targets=fcdisk2.1.init - similar to scsi controller this is a
specification fc controller PCCfc01 which is attached to bus pciA30 of schizo30 bridge chip. The fc disks attached to this
controller are in fcdisk1.multi1.init. #6.HOWTO: setup run directory fc init file|outline -----------------------------------------------------------------------------------------------------------------------------------------------------------
Before making following changes to your root disk image, mount the disk file as a lofi device, by doing
lofiadm -a "your root image file"
mount /lofi/dev/x /tmp/root-image
where x is the lofi device number from "lofiadm -a" above command and /tmp/root-image is where your root image will be mounted. Be very careful when modifying root image because you are root on the machine on which you have the root image mounted. For example if you want to delete /etc/path_to_inst remember that it is /tmp/root-image/etc/path_to_inst that needs to be deleted. Before doing rm or mv or editing any file always do pwd to make sure you are in /tmp/root-image/...
Please note,
you need to "umount" and "lofiadm -d" after you are done with the changes
/tmp/root-image will be referenced in this document as the the device your root image
Install drivers.
Copy following drivers from /import/blazetrace/blz-binaries/drivers/ to /tmp/root-image/kernel/drv/sparcv9,
pcisimd
pcisimc
bid
Solaris 11
Copy following drivers from /import/blazetrace/blz-binaries/drivers/S11/ to /tmp/root-image/kernel/drv/sparcv9
ll
Copy following driver from /import/blazetrace/blz-binaries/drivers/S11/ to /tmp/root-image/kernel/fs/sparcv9
llfs
Create /tmp/root-image/usr/lib/fs/llfs directory and copy /import/blazetrace/blz-binaries/drivers/S11/mount into that directory.
Create llfs file system mount point
cd /tmp/root-image/
mkdir ll
cd ll
mkdir root
This is used to access the local file system which will get mounted under /ll/root on the simulated machine during boot.
Administrative works in /etc directory
Save or remove host system specific files.
The list of files here is system dependent. Your target system might not have more or less than what are mentioned here.
cd /tmp/root-image/etc
mkdir OFF
mv defaultdomain dumpadm hostname* hosts mnttab path_to_inst OFF
cd /tmp/root-image/etc/rcS.d
mkdir OFF
mv S50devfsadm OFF
Modifications for /etc files
/tmp/root-image/etc/system - add following line
set pcisch:pci_buserr_interrupt=0
/tmp/root-image/etc/vfstab - for example,
#device device mount FS fsck mount mount
#to mount to fsck point type pass at boot options
#
fd - /dev/fd fd - no -
/proc - /proc proc - no -
/dev/dsk/c0t0d0s1 - - swap - no -
/dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 no -
%/ - /ll/root llfs - yes -
And any additional disks you would like to map, for example
/dev/dsk/c0t2d0s0 - /export/apps ufs 1 no -
/tmp/root-image/etc/name_to_major – find a available major number for Blaze specific devices and add
pcisimd 251
pcisimc 252
bid 253
ll 254
The device numbers assigned here need not be 251, 252 etc. Just go to the end of the file name_to_major
and start assigning numbers greater than the highest number you see in the file.
/tmp/root-image/etc/shadow – change the login password for root login
root::11431::::::
/tmp/root-image/etc/minor_perm – add the following line
pcisimc:* 0660 root sys
/tmp/root-image/etc/iu.ap - add pcisimc here otherwise the text output wont line up on Blaze console.
pcisimc 0 0 ldterm ttcompat
Blaze specific files
/tmp/root-image/etc/hosts – you need a unique name and IP address within the subnet where you are going to run Blaze
for example,
#
# Blaze internet hosts file
#
127.0.0.1 localhost loghost
129.144.10.170 moan
/tmp/root-image/etc/path_to_inst – the path_to_inst for blasé only need one line as following,
#path_to_inst_bootstrap_1
If you have database disks under SVM in your simulation setup then you need to refer to the following
section to modify path_to_inst file:
/tmp/root-image/etc/mnttab – make it a empty file for Blaze.
==========================================================================================================================================
This is one of the primary functions of SAM i.e. to be able to generate traces which contain both user and kernel instructions. Traces are used by cycle accurate simulators to make performance projections. This sections explains how to gather a trace, generate and gather statistics, running various scripts as part of trace validation and reporting of trace.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The RST tracer module rstracer.so is a shared object located in the SAM directory. You must dynamically link/load it with SAM via. RSTracer registers call back functions such as, instructions, Traps, TLB update and DMAs. RST writes the record to an output trace file.
After you restore from pretracing checkpoint do the following at SAM prompt:
sam: stop
sam: mod load analyzer ./rstracer.so
sam: run
Always make sure you stop before loading the RSTracer. Ofcourse SAM is in stop state after you restore from pretracing checkpoint but make it a habit to double-check everything.
Once we restore from pretracing checkpoint we are ready to start tracing. It is not sufficient to just load the rstracer. We then have to specify certain parameters to the rstracer. You can type “help rstrace” at SAM prompt (once rstracer is loaded) to get help on parameters.
You can collect either a single trace or a series of K sample traces each NN instructions long, with each trace separated by PP instructions.
The trace will be a compressed RST trace. We apply RST compression and then gzip compression, which typically gives a 30X compression. If you insist on consuming incredible amounts of disk space, you can collect an uncompressed trace. Additionally, it is faster to collect a compressed trace than an uncompressed trace. In SAM v5 by default a value trace is collected.
By default, a trace name “bigeasy” will be the name of your file. For example, bigeasy-2002-04-15_16- 16-12-10707-1:0.1.rz2.gz. Clearly, the date and time and the PID of sam is put into the filename. Additionally, there is a tracing request number, the current sample and the total number of samples to the trace name in a format. Each time you specify a (set of samples) trace, the trace request number is bumped.
rstrace -o <file> -n <insts-per-cpu> -d <initial-delay-per-cpu> -x <ntraces> -p <period-per-cpu>
All durations are in terms of per-cpu instruction counts.
For further details on tracing and validation of traces refer to: http://traces.sfbay/twiki/pub/Traces/Documentation/Tracing_doc041406.pdf
The pdf document also contains a sample validation report. You can
access a number of validation reports from http://traces.sfbay.
==========================================================================================================================================
This section has a few howto items and these are not needed for the normal running and tracing in SAM simulation environment. These are more like conveniences.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
In the default console window that opens up using the -w option do the following:
#export TERM=vt100
#stty rows 24
#stty cols 80
#/ll/root/import/archperf/local/bin/screen
Once it starts, you can use "ctrl-a c" to create new screens, and "ctrl-a n" to move to the next one.
==========================================================================================================================================