\ ========== Copyright Header Begin ==========================================
\ Hypervisor Software File: mmu.fth
\ Copyright (c) 2006 Sun Microsystems, Inc. All Rights Reserved.
\ - Do no alter or remove copyright notices
\ - Redistribution and use of this software in source and binary forms, with
\ or without modification, are permitted provided that the following
\ - Redistribution of source code must retain the above copyright notice,
\ this list of conditions and the following disclaimer.
\ - Redistribution in binary form must reproduce the above copyright notice,
\ this list of conditions and the following disclaimer in the
\ documentation and/or other materials provided with the distribution.
\ Neither the name of Sun Microsystems, Inc. or the names of contributors
\ may be used to endorse or promote products derived from this software
\ without specific prior written permission.
\ This software is provided "AS IS," without a warranty of any kind.
\ ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,
\ INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A
\ PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE HEREBY EXCLUDED. SUN
\ MICROSYSTEMS, INC. ("SUN") AND ITS LICENSORS SHALL NOT BE LIABLE FOR
\ ANY DAMAGES SUFFERED BY LICENSEE AS A RESULT OF USING, MODIFYING OR
\ DISTRIBUTING THIS SOFTWARE OR ITS DERIVATIVES. IN NO EVENT WILL SUN
\ OR ITS LICENSORS BE LIABLE FOR ANY LOST REVENUE, PROFIT OR DATA, OR
\ FOR DIRECT, INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL OR PUNITIVE
\ DAMAGES, HOWEVER CAUSED AND REGARDLESS OF THE THEORY OF LIABILITY,
\ ARISING OUT OF THE USE OF OR INABILITY TO USE THIS SOFTWARE, EVEN IF
\ SUN HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
\ You acknowledge that this software is not designed, licensed or
\ intended for use in the design, construction, operation or maintenance of
\ ========== Copyright Header End ============================================
id: @(#)mmu.fth 1.32 05/02/16
copyright: Copyright 2005 Sun Microsystems, Inc. All Rights Reserved
copyright: Use is subject to license terms.
\ We define this now, until mixed-page support is ready
\ set-tte-soft ( tte virt -- tte' )
\ map-page ( pa.lo pa.hi va -- )
\ map-pages ( pa.lo pa.hi virtual size -- )
\ unmap-pages ( va len -- )
\ >mmu-boundaries ( va len size -- va' len' )
\ >page-boundaries ( va len -- va' len' )
#vabits d# 64 = if -1 else 1 #vabits lshift 1- then constant va-mask
pageshift tteshift - constant vptshift
0 1 #vabits pageshift - tteshift + lshift - constant vpt-base
0 vpt-base - constant vpt-size
\ For example, in a 44 bit va space, the vpt-base is at ffff.fffc.0000.0000
\ The vpt address of a tte mapping any va is found by:
\ apply va-mask to the va
\ shift page bits off va (all adrs in same page correspond to same tte)
\ shift tte bits back on va (each tte is 8 bytes in vpt)
\ The above method is used to get the tte for both non-vpt and vpt addresses.
\ The lower pages of the vpt itself are mapped by tte's higher in the vpt.
\ The next level of pages in the vpt holding tte's for the first level are
\ themselves mapped by tte's in the vpt-root page at ffff.ffff.ffff.c000.
\ The vpt-root page is not mapped by a vpt entry, it is locked in the dtlb.
\ These two defers are executed during the performance-critical path
\ of mapping, but are not re-vectored at run-time. They appear to
\ be being defined here as DEFERs either to workaround the lack of
\ forward-definitions, or because their resolutions differ by platform.
\ Arguably, they should be PATCHed-over where they're resolved...
defer allocate-page ( -- pa.lo pa.hi )
defer set-tte-soft ( tte virt -- tte' ) ' drop is set-tte-soft
\ This is a code saving measure, it does not obey any of the standard
\ ABI conventions as it trashes the locals!! Do not call this routine
\ from outside this file.
tos sc2 scr and \ VA & va-mask
scr pageshift scr srlx \ VA -> vpt offset
sc1 sc3 sc1 add \ vpt-segment
sc2 sc3 sc2 add \ vpt-region
\ this routine returns 0 for an invalid VA.
\ the VPT is scanned from region->segment->pgmap before the
code pgmap@ ( va -- tte|0 )
%g0 %g0 tos add \ (delay) Invalid
%g0 %g0 tos add \ (delay) Invalid
scr %g0 tos ldx \ get tte
\ get vpt address of tte for va
\ this routine is the primitive pagemap stuffer.
\ If it returns 0 then this segment and region are fine.
\ if the region needs creation then the tte,va,3 and true are returned.
\ if the segment needs updating then tte,va,2,true are returned
\ returning non-zero means it required high level assistance.
code ((pgmap!)) ( tte va -- 0|tte,va,2,-1|tte,va,3,-1 )
%g0 3 sc4 add \ (delay) map-region required.
%g0 1 tos sub \ (delay) map needs assistance
%g0 2 sc4 add \ (delay) map-segment required.
%g0 1 tos sub \ (delay) map needs assistance
\ Pop the tte and stuff the entry
%g0 %g0 tos add \ All done.
sc4 scr %g0 stx \ store the tte.
((pgmap!)) if ( tte va index )
over swap ( tte va va index )
0 ?do >vpt loop ( tte va va' )
allocate-page ( tte va va' pa.lo pa.hi )
>tte >tte-soft ( tte va va' tte' )
\ From this point onwards we are presenting the higher level MMU routines.
\ No-one should be calling routines defined before this point - except
\ root vpt page which is locked in tlb never misses
0 >vpt >vpt >vpt constant vpt-root
\ The final 2drop is patched at RUNTIME by enable-map-flushing
\ this code is in a performance path which is why it is not
\ using a variable/value/defer to switch this feature on/off.
: (common-mapper) ( va tte len -- )
r@ 0 ?do 2dup i + (pgmap!) pagesize +loop ( tte va )
\ patched to flush-tlb-range
\ The virtual memory node calls into this routine to 'modify' entries..
pagesize (common-mapper) ( )
\ round pa.lo and va down
code >tte-boundaries ( pa.lo pa.hi va round -- va pa.lo pa.hi )
sc2 scr sc2 andn \ pa.lo'
\ round the region to enclose the appropriate page restrictions.
code >mmu-boundaries ( va len size -- va' len' )
sc1 tos sc2 andn \ round-down
scr tos scr add \ add size-1
scr tos scr andn \ round-down
: >page-boundaries ( va len -- va' len' ) pagesize >mmu-boundaries ;
: map-page ( pa.lo pa.hi va -- )
pagesize >tte-boundaries ( va' pa.lo' pa.hi )
\ this assumes you have correctly aligned all the args!!
: (map-pages) ( pa.lo pa.hi va len -- )
>r pagesize >tte-boundaries ( va pa.lo pa.hi )
rot r> ( pa.lo pa.hi va len )
bounds ?do ( pa.lo pa.hi )
2dup i -rot ( pa.lo pa.hi va pa.lo pa.hi )
>tte ( pa.lo pa.hi va tte )
over set-tte-soft ( pa.lo pa.hi va tte' )
swap (pgmap!) ( pa.lo pa.hi )
swap pagesize + swap ( pa.lo' pa.hi )
pagesize +loop ( pa.lo' pa.hi )
dup pgmap@ 0 >tte-valid invert and over pgmap! ( virt )
depend-load MMU-8K-ONLY? ${BP}/cpu/sparc/ultra/mmu-policy/8k-pages.fth
\ Call this once OBPs internal mappings are complete, this will then
\ enable the tlb flushing code for mapping.
\ We use patch because this code is all in the performance path.
: enable-map-flushing ( -- )
['] flush-tlb-range ['] 2drop ( acf1 acf2 )
2dup ( acf1 acf2 acf1 acf2 )
['] (common-mapper) (patch ( acf1 acf2 )
alias deallocate-segment drop ( vadr -- )
alias ?allocate-segment drop ( vadr -- )
dup tte>size ( vadr tte size )
rot dup ." VA:" .x cr ( tte size vadr )
swap 1- and ( tte offset )
over .tlb cr ( tte offset )
over tte> drop or ( tte padr )
swap valid-tte? if ( padr )
\ We set this to change the default translation behaviour so we can
\ reuse the same vpt-walker for .trans and the translation property.
\ This defer is set by any map activity; it is critical to performance.
\ Speeding up the application of IS to a kernel native defer shows up
\ here as a significant performance win.
defer vpt-data-fn ( ?? va len tte -- ?? ) ' 3drop is vpt-data-fn
/prev-vpt-data ualloc user prev-vpt-data
: prev-tte-invalid ( -- ) 0 prev-vpt-data >vpt-tte x! ;
: prev-tte-valid? ( -- flag ) prev-vpt-data >vpt-tte x@ valid-tte? ;
i x@ dup tte>size swap ( va size tte )
2 pick 2 pick 1- and if ( va size tte )
\ tte-size overlaps va so we merge
prev-tte-valid? if ( va size tte )
prev-vpt-data >r ( va size tte )
2 pick r@ >vpt-va x@ - ( va size tte n )
over tte> drop ( va size tte n pa.lo )
r@ >vpt-tte x@ tte> drop - ( va size tte n n )
\ merge sizes, because va and pa are contiguous.
vpt-data@ vpt-data-fn ( va size tte )
\ save this va,tte for merging
vpt-data@ vpt-data-fn ( )
: (vpt-segment) ( va -- )
\ walk the vpt and execute acf with va,len,tte on the stack
\ beginning at the vpt-root, iterate over each tte in that page;
\ for each valid tte, go to the page it maps and iterate over those tte's;
\ for each valid tte in those pages, again descend to the mapped page
\ and iterate over those tte's; these last map the remaining address space
vpt-root pagesize bounds do ( )
projects / OpenSPARC-T2-SAM / blob