This is a list of the changes to the vm system as of 14 Jan 94:
authorDavid Greenman <davidg@Root.COM>
Fri, 14 Jan 1994 16:27:33 +0000 (16:27 +0000)
committerDavid Greenman <davidg@Root.COM>
Fri, 14 Jan 1994 16:27:33 +0000 (16:27 +0000)
1) Upages are now in process address space.  They can be paged out when
the process is swapped out.  The majority of changes are in
vm_glue.c and pmap.c.  Upages are always recorded to be modified.

2) Page table pages are not high level vm wired except to insert pages
pte's into the pmap.  The pmap subsystem has a wiring mechanism of
its own now, and releases page table pages to the vm system when the
number of pages pointed to by the page table page drops to zero.
This mechanism allows the physical memory for process page tables
to be reclaimed for use by other processes during memory shortages.
The majority of changes are in trap.c, pmap.c, kern__physio.c,
procfs_subr.c.  Page table pages are never recorded to be modified.

3) Page table pages are faulted in by referring to the page table page
from kernel address space.  Temporary wiring of the page table page
is performed by removing the page from the inactive and active
queues.  This is much more efficient than many other techniques
(explicit vm_fault's.)

4) Pmap.c has major sections rewritten for efficiency.  String operations
are used when possible.  Also, pmap_remove (called often) has
been rewritten to be *much* more efficient.

5) Pmap_attributes has been removed.  The side effect of storing the
modified and used (referenced) bits is difficult to control.  Mods
have been made to the rest of the vm system to not assume carryover
of the modified bits.  It is believed that this fixes a problem
where pages were often written to the backing store unnecessarily.

6) Pageout queue management is *much* more efficient:

6a) Pages that are deactivated are removed from pmaps.  This allows the
system to fault immediately when a page is used again.  Thereby
providing LRU heuristics used in [6b].

6b) The active page queue is logically segmented into slices, and a
page has to be scanned as unused for multiple passes to be deactivated.
Pages that are used are placed immediately at the end of the active
queue, thereby deferring deactivation for a "long time".

6c) A limited form of biasing page inactivation towards processes exceeding
RSS limits has been included.  Hooks have been added to support hard
RSS limits that may be added in the future.

7) Vm_fault now supports pagers that can do multiple block input.  The
logic makes sure that the readahead/behind falls in the same object
and pager (used in [8] and [9].)  Readaheads/behinds are inhibited
under very low memory conditions.  It is left to the pager to decide
whether to actually perform the readahead/behind.

8) The swap_pager supports contiguous block allocations and multi-block
input can be peformed.

9) The vnode_pager bypasses the buffer-caching mechanism, thereby
conserving buffer cache and eliminating page memory copies under
most circumstances.  Also, the vnode_pager supports contiguous
readaheads/behinds.

10) The vm_object_collapse code has be improved significantly, and the
swap_pager allocations have been modified to support the improved
operation.  (This improves the swap space allocation problem.)
Under rare circumstances it is still possible to run into the
VM swap space deallocation problems related to the object
paging_in_progress flag, but a fix is in the works ( and is
relatively simple, but tedious.)

11) The swap space full condition is handled in a rather severe way.
If a process tries to allocate another page in its working
set when swap space is zero and no backing store exists, it is killed.
(I know that this is not good, but it is worse to hang the system.)

12) Because of the swap space being allocated in blocks, it is possible
for a page that is allocated on swap space to never be used.  An
optimization has been made to the swap pager to reclaim the space,
thereby almost eliminating internal fragmentation of the swap
blocks.

13) The algorithms associated with vm_pageout have been improved to
decrease significantly the amount of CPU time used.  (Wakeups
of the pageout daemon have been minimized.)
13a) vm_page_alloc woke-up the page daemon MUCH too often.
13b) the swap pager output completion woke-up the page daemon too often.
13c) the probability of a page at the beginning of a queue needing
to be (deactivated or freed) is higher

14) The system has been modified to support discontiguous allocation of
physical memory.  This allows use of memory below 640K when the kernel
is loaded at 1MB.  Now, the system is configured to load only at 1MB,
because the cost is minimal (1/4 page table page, instead of 640K) and
every kernel loads at the same address.

15) There now is an internal option to tune the location of the kernel in
the virtual address space.  Additionally, the default base kernel
virtual address is now 0xF0000000, moved from 0xFE000000, allowing
up to a 256MB kernel virtual address space.  Currently we support
a usable kernel address space of (60-1)MB (as determined by the actual
number of allocated kernel ptes (15).)  (Actually, now the kernel uses
virtual addresses starting at 0xF0100000 because it is loaded at
1MB.)

John Dyson
David Greenman

21 files changed:
sys/vm/device_pager.c
sys/vm/lock.h
sys/vm/swap_pager.c
sys/vm/swap_pager.h
sys/vm/vm_fault.c
sys/vm/vm_glue.c
sys/vm/vm_init.c
sys/vm/vm_kern.c
sys/vm/vm_map.c
sys/vm/vm_map.h
sys/vm/vm_mmap.c
sys/vm/vm_object.c
sys/vm/vm_object.h
sys/vm/vm_page.c
sys/vm/vm_page.h
sys/vm/vm_pageout.c
sys/vm/vm_pageout.h
sys/vm/vm_pager.c
sys/vm/vm_pager.h
sys/vm/vnode_pager.c
sys/vm/vnode_pager.h

index 89f9a22..01ce730 100644 (file)
@@ -84,6 +84,7 @@ struct pagerops devicepagerops = {
        dev_pager_alloc,
        dev_pager_dealloc,
        dev_pager_getpage,
        dev_pager_alloc,
        dev_pager_dealloc,
        dev_pager_getpage,
+       0,
        dev_pager_putpage,
        dev_pager_haspage
 };
        dev_pager_putpage,
        dev_pager_haspage
 };
@@ -137,7 +138,7 @@ dev_pager_alloc(handle, size, prot, foff)
        /*
         * Offset should be page aligned.
         */
        /*
         * Offset should be page aligned.
         */
-       if (foff & (NBPG-1))
+       if (foff & (PAGE_SIZE-1))
                return(NULL);
 
        /*
                return(NULL);
 
        /*
@@ -258,6 +259,7 @@ dev_pager_getpage(pager, m, sync)
        vm_offset_t offset, paddr;
        vm_page_t page;
        dev_t dev;
        vm_offset_t offset, paddr;
        vm_page_t page;
        dev_t dev;
+       int s;
        int (*mapfunc)(), prot;
 
 #ifdef DEBUG
        int (*mapfunc)(), prot;
 
 #ifdef DEBUG
@@ -290,7 +292,9 @@ dev_pager_getpage(pager, m, sync)
        vm_page_lock_queues();
        vm_page_free(m);
        vm_page_unlock_queues();
        vm_page_lock_queues();
        vm_page_free(m);
        vm_page_unlock_queues();
+       s = splhigh();
        vm_page_insert(page, object, offset);
        vm_page_insert(page, object, offset);
+       splx(s);
        PAGE_WAKEUP(m);
        if (offset + PAGE_SIZE > object->size)
                object->size = offset + PAGE_SIZE;      /* XXX anal */
        PAGE_WAKEUP(m);
        if (offset + PAGE_SIZE > object->size)
                object->size = offset + PAGE_SIZE;      /* XXX anal */
index 85782a4..e393e14 100644 (file)
@@ -176,10 +176,10 @@ void              lock_clear_recursive();
  * Try to get semi-meaningful wait messages into thread_sleep...
  */
 extern void thread_sleep_(int, simple_lock_t, const char *);
  * Try to get semi-meaningful wait messages into thread_sleep...
  */
 extern void thread_sleep_(int, simple_lock_t, const char *);
-#if __GNUC__ > 2
-#  define thread_sleep(a,b,c) thread_sleep_((a), (b), __FUNCTION__)
+#if __GNUC__ >= 2
+#define thread_sleep(a,b,c) thread_sleep_((a), (b), __FUNCTION__)
 #else
 #else
-#  define thread_sleep(a,b,c) thread_sleep_((a), (b), "vmsleep")
+#define thread_sleep(a,b,c) thread_sleep_((a), (b), "vmslp")
 #endif
 #define thread_sleep_new thread_sleep_
 extern void thread_wakeup(int);
 #endif
 #define thread_sleep_new thread_sleep_
 extern void thread_wakeup(int);
index 442a03a..a22f0d2 100644 (file)
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
- *     from: Utah $Hdr: swap_pager.c 1.4 91/04/30$
- *     from: @(#)swap_pager.c  7.4 (Berkeley) 5/7/91
- *     $Id: swap_pager.c,v 1.10 1993/12/22 12:51:53 davidg Exp $
- */
-
-/*
- * Quick hack to page to dedicated partition(s).
- * TODO:
- *     Add multiprocessor locks
- *     Deal with async writes in a better fashion
+ * from: Utah $Hdr: swap_pager.c 1.4 91/04/30$
+ * from: @(#)swap_pager.c      7.4 (Berkeley) 5/7/91
+ *
+ * $Id$
  */
 
 /*
  */
 
 /*
- * From John Dyson:
- *
- * Enhancements to page in multiple pages
- * Efficiency improvements in pageout code
- * Changes to allocation algorithm to allow more
- *  dense allocation of swap space.
- * Addition of a routine to allow disk-based copying
- *  to allow vm_object_collapse to work better.
- *
- * TODO:
- *  Make allocation more intelligent re: text space
- *   because of rounding problems with the allocation of
- *   blocks of swap space, it is possible to allocate swap
- *   space for text.  Add some hooks to find out if portions of  an object
- *   will ever need swap space???????
+ * Mostly rewritten by John Dyson with help from David Greenman, 12-Jan-1994
  */
 
 #include "param.h"
 #include "proc.h"
 #include "buf.h"
  */
 
 #include "param.h"
 #include "proc.h"
 #include "buf.h"
+#include "kernel.h"
 #include "systm.h"
 #include "specdev.h"
 #include "vnode.h"
 #include "malloc.h"
 #include "systm.h"
 #include "specdev.h"
 #include "vnode.h"
 #include "malloc.h"
+#include "queue.h"
 #include "rlist.h"
 #include "rlist.h"
-#include "kernel.h"
 
 #include "vm_param.h"
 #include "queue.h"
 
 #include "vm_param.h"
 #include "queue.h"
 #include "swap_pager.h"
 #include "vm_map.h"
 
 #include "swap_pager.h"
 #include "vm_map.h"
 
-#define NPENDINGIO     64
-
-struct pagerops swappagerops = {
-       swap_pager_init,
-       swap_pager_alloc,
-       swap_pager_dealloc,
-       swap_pager_getpage,
-       swap_pager_putpage,
-       swap_pager_haspage
-};
+#ifndef NPENDINGIO
+#define NPENDINGIO     96
+#endif
 
 extern int nswbuf;
 int nswiodone;
 extern int vm_pageout_rate_limit;
 static int cleandone;
 
 extern int nswbuf;
 int nswiodone;
 extern int vm_pageout_rate_limit;
 static int cleandone;
+extern int hz;
 int swap_pager_full;
 extern vm_map_t pager_map;
 int swap_pager_full;
 extern vm_map_t pager_map;
-void swap_pager_finish();
+extern int vm_pageout_pages_needed;
 
 struct swpagerclean {
        queue_head_t            spc_list;
 
 struct swpagerclean {
        queue_head_t            spc_list;
@@ -115,6 +90,7 @@ struct swpagerclean {
 } swcleanlist [NPENDINGIO] ;
 
 typedef        struct swpagerclean     *swp_clean_t;
 } swcleanlist [NPENDINGIO] ;
 
 typedef        struct swpagerclean     *swp_clean_t;
+
 extern vm_map_t kernel_map;
 
 /* spc_flags values */
 extern vm_map_t kernel_map;
 
 /* spc_flags values */
@@ -126,22 +102,46 @@ queue_head_t      swap_pager_done;        /* list of compileted page cleans */
 queue_head_t   swap_pager_inuse;       /* list of pending page cleans */
 queue_head_t   swap_pager_free;        /* list of free pager clean structs */
 queue_head_t   swap_pager_list;        /* list of "named" anon regions */
 queue_head_t   swap_pager_inuse;       /* list of pending page cleans */
 queue_head_t   swap_pager_free;        /* list of free pager clean structs */
 queue_head_t   swap_pager_list;        /* list of "named" anon regions */
+queue_head_t   swap_pager_un_list;     /* list of "unnamed" anon pagers */
+#define        SWAP_FREE_NEEDED        0x1     /* need a swap block */
+int swap_pager_needflags;
+
+static queue_head_t *swp_qs[]={
+       &swap_pager_list, &swap_pager_un_list, (queue_head_t *) 0
+};
+
+struct pagerops swappagerops = {
+       swap_pager_init,
+       swap_pager_alloc,
+       swap_pager_dealloc,
+       swap_pager_getpage,
+       swap_pager_getmulti,
+       swap_pager_putpage,
+       swap_pager_haspage
+};
+
+extern int nswbuf;
 
 int npendingio = NPENDINGIO;
 int swiopend;
 int pendingiowait;
 int require_swap_init;
 
 int npendingio = NPENDINGIO;
 int swiopend;
 int pendingiowait;
 int require_swap_init;
-
-int swap_wakeup;
+void swap_pager_finish();
 int dmmin, dmmax;
 int dmmin, dmmax;
+extern int vm_page_count;
+
+struct buf * getpbuf() ;
+void relpbuf(struct buf *bp) ;
 
 void
 swap_pager_init()
 {
        register int i;
 
 void
 swap_pager_init()
 {
        register int i;
+       extern int dmmin, dmmax;
 
        dfltpagerops = &swappagerops;
        queue_init(&swap_pager_list);
 
        dfltpagerops = &swappagerops;
        queue_init(&swap_pager_list);
+       queue_init(&swap_pager_un_list);
 
        /*
         * Initialize clean lists
 
        /*
         * Initialize clean lists
@@ -152,14 +152,12 @@ swap_pager_init()
 
        require_swap_init = 1;
 
 
        require_swap_init = 1;
 
-
        /*
         * Calculate the swap allocation constants.
         */
 
        dmmin = CLBYTES/DEV_BSIZE;
        /*
         * Calculate the swap allocation constants.
         */
 
        dmmin = CLBYTES/DEV_BSIZE;
-
-       dmmax = btodb( SWB_NPAGES*NBPG) * 8;
+       dmmax = btodb(SWB_NPAGES*NBPG)*2;
 
 }
 
 
 }
 
@@ -179,13 +177,19 @@ swap_pager_alloc(handle, size, prot)
        int waitok;
        int i,j;
                        
        int waitok;
        int i,j;
                        
-       ifrequire_swap_init) {
+       if (require_swap_init) {
                register swp_clean_t spc;
                register swp_clean_t spc;
-               ifnpendingio > nswbuf)
+               if (npendingio > nswbuf)
                        npendingio = nswbuf;
                        npendingio = nswbuf;
+               if (npendingio > vm_page_count / 32)
+                       npendingio = vm_page_count / 32;
+               /*
+                * kva's are allocated here so that we dont need to keep
+                * doing kmem_alloc pageables at runtime
+                */
                for (i = 0, spc = swcleanlist; i < npendingio ; i++, spc++) {
                for (i = 0, spc = swcleanlist; i < npendingio ; i++, spc++) {
-                       spc->spc_kva = kmem_alloc_pageable( pager_map, NBPG);
-                       if!spc->spc_kva)
+                       spc->spc_kva = kmem_alloc_pageable(pager_map, NBPG);
+                       if (!spc->spc_kva)
                                break;
                        spc->spc_flags = 0;
                        queue_enter(&swap_pager_free, spc, swp_clean_t, spc_list);
                                break;
                        spc->spc_flags = 0;
                        queue_enter(&swap_pager_free, spc, swp_clean_t, spc_list);
@@ -211,6 +215,9 @@ swap_pager_alloc(handle, size, prot)
                }
        }
 
                }
        }
 
+       if (swap_pager_full)
+               return(NULL);
+
        /*
         * Pager doesn't exist, allocate swap management resources
         * and initialize.
        /*
         * Pager doesn't exist, allocate swap management resources
         * and initialize.
@@ -233,13 +240,13 @@ swap_pager_alloc(handle, size, prot)
        if (swp->sw_blocks == NULL) {
                free((caddr_t)swp, M_VMPGDATA);
                free((caddr_t)pager, M_VMPAGER);
        if (swp->sw_blocks == NULL) {
                free((caddr_t)swp, M_VMPGDATA);
                free((caddr_t)pager, M_VMPAGER);
-               return(FALSE);
+               return(NULL);
        }
        bzero((caddr_t)swp->sw_blocks,
              swp->sw_nblocks * sizeof(*swp->sw_blocks));
 
        }
        bzero((caddr_t)swp->sw_blocks,
              swp->sw_nblocks * sizeof(*swp->sw_blocks));
 
-       for(i=0;i<swp->sw_nblocks;i++) {
-               for(j=0;j<SWB_NPAGES;j++)
+       for (i = 0; i < swp->sw_nblocks; i++) {
+               for (j = 0; j < SWB_NPAGES; j++)
                        swp->sw_blocks[i].swb_block[j] = SWB_EMPTY;
        }
 
                        swp->sw_blocks[i].swb_block[j] = SWB_EMPTY;
        }
 
@@ -260,6 +267,7 @@ swap_pager_alloc(handle, size, prot)
        } else {
                swp->sw_flags = 0;
                queue_init(&pager->pg_list);
        } else {
                swp->sw_flags = 0;
                queue_init(&pager->pg_list);
+               queue_enter(&swap_pager_un_list, pager, vm_pager_t, pg_list);
        }
        pager->pg_handle = handle;
        pager->pg_ops = &swappagerops;
        }
        pager->pg_handle = handle;
        pager->pg_ops = &swappagerops;
@@ -270,9 +278,11 @@ swap_pager_alloc(handle, size, prot)
 }
 
 /*
 }
 
 /*
- * return the address on disk and the validity of the
- * data on disk.
+ * returns disk block associated with pager and offset
+ * additionally, as a side effect returns a flag indicating
+ * if the block has been written
  */
  */
+
 static int *
 swap_pager_diskaddr(swp, offset, valid)
        sw_pager_t swp;
 static int *
 swap_pager_diskaddr(swp, offset, valid)
        sw_pager_t swp;
@@ -282,7 +292,7 @@ swap_pager_diskaddr(swp, offset, valid)
        register sw_blk_t swb;
        int ix;
 
        register sw_blk_t swb;
        int ix;
 
-       ifvalid)
+       if (valid)
                *valid = 0;
        ix = offset / (SWB_NPAGES*NBPG);
        if (swp->sw_blocks == NULL || ix >= swp->sw_nblocks) {
                *valid = 0;
        ix = offset / (SWB_NPAGES*NBPG);
        if (swp->sw_blocks == NULL || ix >= swp->sw_nblocks) {
@@ -290,15 +300,20 @@ swap_pager_diskaddr(swp, offset, valid)
        }
        swb = &swp->sw_blocks[ix];
        ix = (offset % (SWB_NPAGES*NBPG)) / NBPG;
        }
        swb = &swp->sw_blocks[ix];
        ix = (offset % (SWB_NPAGES*NBPG)) / NBPG;
-       ifvalid)
+       if (valid)
                *valid = swb->swb_valid & (1<<ix);
        return &swb->swb_block[ix];
 }
 
                *valid = swb->swb_valid & (1<<ix);
        return &swb->swb_block[ix];
 }
 
+/*
+ * Utility routine to set the valid (written) bit for
+ * a block associated with a pager and offset
+ */
 static void
 static void
-swap_pager_setvalid(swp, offset)
+swap_pager_setvalid(swp, offset, valid)
        sw_pager_t swp;
        vm_offset_t offset;
        sw_pager_t swp;
        vm_offset_t offset;
+       int valid;
 {
        register sw_blk_t swb;
        int ix;
 {
        register sw_blk_t swb;
        int ix;
@@ -309,7 +324,10 @@ swap_pager_setvalid(swp, offset)
 
        swb = &swp->sw_blocks[ix];
        ix = (offset % (SWB_NPAGES*NBPG)) / NBPG;
 
        swb = &swp->sw_blocks[ix];
        ix = (offset % (SWB_NPAGES*NBPG)) / NBPG;
-       swb->swb_valid |= (1 << ix);
+       if (valid)
+               swb->swb_valid |= (1 << ix);
+       else
+               swb->swb_valid &= ~(1 << ix);
        return;
 }
 
        return;
 }
 
@@ -317,49 +335,135 @@ swap_pager_setvalid(swp, offset)
  * this routine frees swap blocks from a specified pager
  */
 void
  * this routine frees swap blocks from a specified pager
  */
 void
-swap_pager_freespace(vm_pager_t pager, vm_offset_t start, vm_offset_t size) {
-       
+swap_pager_freespace(pager, start, size)
+       vm_pager_t pager;
+       vm_offset_t start;
+       vm_offset_t size;
+{
        sw_pager_t swp = (sw_pager_t) pager->pg_data;
        vm_offset_t i;
        int s;
 
        s = splbio();
        sw_pager_t swp = (sw_pager_t) pager->pg_data;
        vm_offset_t i;
        int s;
 
        s = splbio();
-       for(i=start;i<round_page(start+size-1);i+=NBPG) {
-               int *addr = swap_pager_diskaddr( swp, i, 0);
-               ifaddr && *addr != SWB_EMPTY) {
+       for (i = start; i < round_page(start + size - 1); i += NBPG) {
+               int *addr = swap_pager_diskaddr(swp, i, 0);
+               if (addr && *addr != SWB_EMPTY) {
                        rlist_free(&swapmap, *addr, *addr + btodb(NBPG) - 1);
                        *addr = SWB_EMPTY;
                        rlist_free(&swapmap, *addr, *addr + btodb(NBPG) - 1);
                        *addr = SWB_EMPTY;
+                       swap_pager_full = 0;
                }
        }
        splx(s);
 }
 
                }
        }
        splx(s);
 }
 
+/*
+ * swap_pager_reclaim frees up over-allocated space from all pagers
+ * this eliminates internal fragmentation due to allocation of space
+ * for segments that are never swapped to. It has been written so that
+ * it does not block until the rlist_free operation occurs; it keeps
+ * the queues consistant.
+ */
 
 /*
 
 /*
- * This routine copies pages from one pager to another and destroys
- * the original pager
+ * Maximum number of blocks (pages) to reclaim per pass
  */
  */
-int
-swap_pager_copy(vm_pager_t srcpager, vm_offset_t srcoffset,
-               vm_pager_t dstpager, vm_offset_t dstoffset,
-               vm_offset_t offset) {
+#define MAXRECLAIM 256
+
+void
+swap_pager_reclaim()
+{
+       vm_pager_t p;
+       sw_pager_t swp;
+       int i, j, k;
+       int s;
+       int reclaimcount;
+       static int reclaims[MAXRECLAIM];
+       static int in_reclaim;
+
+       s = splbio();
+       if (in_reclaim) {
+               tsleep((caddr_t) &in_reclaim, PSWP, "swrclm", 0);
+               splx(s);
+               return;
+       }
+       in_reclaim = 1;
+       reclaimcount = 0;
+
+       /* for each pager queue */
+       for (k = 0; swp_qs[k]; k++) {
+
+               p = (vm_pager_t) queue_first(swp_qs[k]);
+               while (reclaimcount < MAXRECLAIM &&
+                       !queue_end(swp_qs[k], (queue_entry_t) p)) {
+
+                       /*
+                        * see if any blocks associated with a pager has been
+                        * allocated but not used (written)
+                        */
+                       swp = (sw_pager_t) p->pg_data;
+                       for (i = 0; i < swp->sw_nblocks; i++) {
+                               sw_blk_t swb = &swp->sw_blocks[i];
+                               for (j = 0; j < SWB_NPAGES; j++) {
+                                       if (swb->swb_block[j] != SWB_EMPTY &&
+                                               (swb->swb_valid & (1 << j)) == 0) {
+                                               reclaims[reclaimcount++] = swb->swb_block[j];
+                                               swb->swb_block[j] = SWB_EMPTY;
+                                               if (reclaimcount >= MAXRECLAIM)
+                                                       goto rfinished;
+                                       }
+                               }
+                       }
+                       p = (vm_pager_t) queue_next(&p->pg_list);
+               }
+       }
+       
+rfinished:
+
+/*
+ * free the blocks that have been added to the reclaim list
+ */
+       for (i = 0; i < reclaimcount; i++) {
+               rlist_free(&swapmap, reclaims[i], reclaims[i] + btodb(NBPG) - 1);
+               wakeup((caddr_t) &in_reclaim);
+               swap_pager_full = 0;
+       }
+
+       splx(s);
+       in_reclaim = 0;
+       wakeup((caddr_t) &in_reclaim);
+}
+               
+
+/*
+ * swap_pager_copy copies blocks from one pager to another and
+ * destroys the source pager
+ */
+
+void
+swap_pager_copy(srcpager, srcoffset, dstpager, dstoffset, offset)
+       vm_pager_t srcpager;
+       vm_offset_t srcoffset;
+       vm_pager_t dstpager;
+       vm_offset_t dstoffset;
+       vm_offset_t offset;
+{
        sw_pager_t srcswp, dstswp;
        sw_pager_t srcswp, dstswp;
-       sw_blk_t srcbp, dstbp;
-       int i,j;
+       vm_offset_t i;
        int s;
 
        srcswp = (sw_pager_t) srcpager->pg_data;
        dstswp = (sw_pager_t) dstpager->pg_data;
 
        s = splbio();
        int s;
 
        srcswp = (sw_pager_t) srcpager->pg_data;
        dstswp = (sw_pager_t) dstpager->pg_data;
 
        s = splbio();
-
        if (srcswp->sw_flags & SW_NAMED) {
                queue_remove(&swap_pager_list, srcpager, vm_pager_t, pg_list);
                srcswp->sw_flags &= ~SW_NAMED;
        if (srcswp->sw_flags & SW_NAMED) {
                queue_remove(&swap_pager_list, srcpager, vm_pager_t, pg_list);
                srcswp->sw_flags &= ~SW_NAMED;
+       } else {
+               queue_remove(&swap_pager_un_list, srcpager, vm_pager_t, pg_list);
        }
        
        }
        
-       whilesrcswp->sw_poip) {
-               tsleep((caddr_t)&swap_wakeup, PVM, "wpgout", 0); 
+       while (srcswp->sw_poip) {
+               tsleep((caddr_t)srcswp, PVM, "spgout", 0); 
        }
        splx(s);
 
        }
        splx(s);
 
@@ -369,53 +473,65 @@ swap_pager_copy(vm_pager_t srcpager, vm_offset_t srcoffset,
 /*
  * clear source block before destination object
  */
 /*
  * clear source block before destination object
  */
-       for(i=0;i<offset+srcoffset;i+=NBPG) {
-               int *addr = swap_pager_diskaddr( srcswp, i, 0);
-               if( addr && *addr != SWB_EMPTY)
+       for (i = 0; i < offset + srcoffset; i += NBPG) {
+               int *addr = swap_pager_diskaddr(srcswp, i, 0);
+               if (addr && *addr != SWB_EMPTY) {
                        rlist_free(&swapmap, *addr, *addr + btodb(NBPG) - 1);
                        rlist_free(&swapmap, *addr, *addr + btodb(NBPG) - 1);
+                       *addr = SWB_EMPTY;
+                       swap_pager_full = 0;
+               }
        }
 /*
  * transfer source to destination
  */
        }
 /*
  * transfer source to destination
  */
-       for(i=0;i<dstswp->sw_osize;i+=NBPG) {
+       for (i = 0; i < dstswp->sw_osize; i += NBPG) {
                int srcvalid, dstvalid;
                int srcvalid, dstvalid;
-               int *srcaddrp = swap_pager_diskaddr( srcswp, i+offset+srcoffset,
+               int *srcaddrp = swap_pager_diskaddr(srcswp, i + offset + srcoffset,
                        &srcvalid);
                int *dstaddrp;
                        &srcvalid);
                int *dstaddrp;
-               ifsrcaddrp && *srcaddrp != SWB_EMPTY) {
-                       ifsrcvalid) {
-                               dstaddrp = swap_pager_diskaddr( dstswp, i+dstoffset, &dstvalid);
-                               if!dstvalid && dstaddrp && *dstaddrp != SWB_EMPTY) {
+               if (srcaddrp && *srcaddrp != SWB_EMPTY) {
+                       if (srcvalid) {
+                               dstaddrp = swap_pager_diskaddr(dstswp, i + dstoffset, &dstvalid);
+                               if (!dstvalid && dstaddrp && *dstaddrp != SWB_EMPTY) {
                                        rlist_free(&swapmap, *dstaddrp, *dstaddrp + btodb(NBPG) - 1);
                                        *dstaddrp = SWB_EMPTY;
                                        rlist_free(&swapmap, *dstaddrp, *dstaddrp + btodb(NBPG) - 1);
                                        *dstaddrp = SWB_EMPTY;
+                                       swap_pager_full = 0;
                                }
                                }
-                               ifdstaddrp && *dstaddrp == SWB_EMPTY) {
+                               if (dstaddrp && *dstaddrp == SWB_EMPTY) {
                                        *dstaddrp = *srcaddrp;
                                        *srcaddrp = SWB_EMPTY;
                                        *dstaddrp = *srcaddrp;
                                        *srcaddrp = SWB_EMPTY;
-                                       swap_pager_setvalid( dstswp, i + dstoffset);
+                                       swap_pager_setvalid(dstswp, i + dstoffset, 1);
                                }
                        }
                                }
                        }
-                       if( *srcaddrp != SWB_EMPTY)
+                       if (*srcaddrp != SWB_EMPTY) {
                                rlist_free(&swapmap, *srcaddrp, *srcaddrp + btodb(NBPG) - 1);
                                rlist_free(&swapmap, *srcaddrp, *srcaddrp + btodb(NBPG) - 1);
+                               *srcaddrp = SWB_EMPTY;
+                               swap_pager_full = 0;
+                       }
                }
        }
 
 /*
  * deallocate the rest of the source object
  */
                }
        }
 
 /*
  * deallocate the rest of the source object
  */
-       for(i=dstswp->sw_osize + offset + srcoffset;i<srcswp->sw_osize;i+=NBPG) {
-               int *srcaddrp = swap_pager_diskaddr( srcswp, i, 0);
-               if( srcaddrp && *srcaddrp != SWB_EMPTY)
+       for (i = dstswp->sw_osize + offset + srcoffset; i < srcswp->sw_osize; i += NBPG) {
+               int *srcaddrp = swap_pager_diskaddr(srcswp, i, 0);
+               if (srcaddrp && *srcaddrp != SWB_EMPTY) {
                        rlist_free(&swapmap, *srcaddrp, *srcaddrp + btodb(NBPG) - 1);
                        rlist_free(&swapmap, *srcaddrp, *srcaddrp + btodb(NBPG) - 1);
+                       *srcaddrp = SWB_EMPTY;
+                       swap_pager_full = 0;
+               }
        }
                                
        splx(s);
 
        free((caddr_t)srcswp->sw_blocks, M_VMPGDATA);
        }
                                
        splx(s);
 
        free((caddr_t)srcswp->sw_blocks, M_VMPGDATA);
+       srcswp->sw_blocks = 0;
        free((caddr_t)srcswp, M_VMPGDATA);
        free((caddr_t)srcswp, M_VMPGDATA);
+       srcpager->pg_data = 0;
        free((caddr_t)srcpager, M_VMPAGER);
 
        free((caddr_t)srcpager, M_VMPAGER);
 
-       return 1;
+       return;
 }
 
 
 }
 
 
@@ -437,14 +553,16 @@ swap_pager_dealloc(pager)
        if (swp->sw_flags & SW_NAMED) {
                queue_remove(&swap_pager_list, pager, vm_pager_t, pg_list);
                swp->sw_flags &= ~SW_NAMED;
        if (swp->sw_flags & SW_NAMED) {
                queue_remove(&swap_pager_list, pager, vm_pager_t, pg_list);
                swp->sw_flags &= ~SW_NAMED;
+       } else {
+               queue_remove(&swap_pager_un_list, pager, vm_pager_t, pg_list);
        }
        /*
         * Wait for all pageouts to finish and remove
         * all entries from cleaning list.
         */
 
        }
        /*
         * Wait for all pageouts to finish and remove
         * all entries from cleaning list.
         */
 
-       whileswp->sw_poip) {
-               tsleep((caddr_t)&swap_wakeup, PVM, "wpgout", 0); 
+       while (swp->sw_poip) {
+               tsleep((caddr_t)swp, PVM, "swpout", 0); 
        }
        splx(s);
                
        }
        splx(s);
                
@@ -456,20 +574,24 @@ swap_pager_dealloc(pager)
         */
        s = splbio();
        for (i = 0, bp = swp->sw_blocks; i < swp->sw_nblocks; i++, bp++) {
         */
        s = splbio();
        for (i = 0, bp = swp->sw_blocks; i < swp->sw_nblocks; i++, bp++) {
-               for(j=0;j<SWB_NPAGES;j++)
+               for (j = 0; j < SWB_NPAGES; j++)
                if (bp->swb_block[j] != SWB_EMPTY) {
                        rlist_free(&swapmap, (unsigned)bp->swb_block[j],
                                (unsigned)bp->swb_block[j] + btodb(NBPG) - 1);
                if (bp->swb_block[j] != SWB_EMPTY) {
                        rlist_free(&swapmap, (unsigned)bp->swb_block[j],
                                (unsigned)bp->swb_block[j] + btodb(NBPG) - 1);
+                       bp->swb_block[j] = SWB_EMPTY;
+                       swap_pager_full = 0;
                }
        }
        splx(s);
                }
        }
        splx(s);
+
        /*
         * Free swap management resources
         */
        free((caddr_t)swp->sw_blocks, M_VMPGDATA);
        /*
         * Free swap management resources
         */
        free((caddr_t)swp->sw_blocks, M_VMPGDATA);
+       swp->sw_blocks = 0;
        free((caddr_t)swp, M_VMPGDATA);
        free((caddr_t)swp, M_VMPGDATA);
+       pager->pg_data = 0;
        free((caddr_t)pager, M_VMPAGER);
        free((caddr_t)pager, M_VMPAGER);
-       swap_pager_full = 0;
 }
 
 int
 }
 
 int
@@ -480,8 +602,7 @@ swap_pager_getmulti(pager, m, count, reqpage, sync)
        int reqpage;
        boolean_t sync;
 {
        int reqpage;
        boolean_t sync;
 {
-       
-       return swap_pager_io( (sw_pager_t) pager->pg_data, m, count, reqpage, B_READ);
+       return swap_pager_io((sw_pager_t) pager->pg_data, m, count, reqpage, B_READ);
 }
 
 int
 }
 
 int
@@ -519,19 +640,19 @@ swap_pager_putpage(pager, m, sync)
 }
 
 static inline int
 }
 
 static inline int
-swap_pager_block_index( swp, offset)
+swap_pager_block_index(swp, offset)
        sw_pager_t swp;
        vm_offset_t offset;
 {
        sw_pager_t swp;
        vm_offset_t offset;
 {
-       return  offset / (SWB_NPAGES*NBPG);
+       return (offset / (SWB_NPAGES*NBPG));
 }
 
 static inline int
 }
 
 static inline int
-swap_pager_block_offset( swp, offset)
+swap_pager_block_offset(swp, offset)
        sw_pager_t swp;
        vm_offset_t offset;
 {      
        sw_pager_t swp;
        vm_offset_t offset;
 {      
-       return offset % (SWB_NPAGES*NBPG);
+       return (offset % (SWB_NPAGES*NBPG));
 }
 
 static boolean_t
 }
 
 static boolean_t
@@ -548,8 +669,8 @@ _swap_pager_haspage(swp, offset)
        }
        swb = &swp->sw_blocks[ix];
        ix = (offset % (SWB_NPAGES*NBPG)) / NBPG;
        }
        swb = &swp->sw_blocks[ix];
        ix = (offset % (SWB_NPAGES*NBPG)) / NBPG;
-       ifswb->swb_block[ix] != SWB_EMPTY) {
-               if( swb->swb_valid & (1<<ix))
+       if (swb->swb_block[ix] != SWB_EMPTY) {
+               if (swb->swb_valid & (1 << ix))
                        return TRUE;
        }
 
                        return TRUE;
        }
 
@@ -561,40 +682,49 @@ swap_pager_haspage(pager, offset)
        vm_pager_t pager;
        vm_offset_t offset;
 {
        vm_pager_t pager;
        vm_offset_t offset;
 {
-       return _swap_pager_haspage( (sw_pager_t) pager->pg_data, offset);
+       return _swap_pager_haspage((sw_pager_t) pager->pg_data, offset);
 }
 
 static void
 }
 
 static void
-swap_pager_freepage( vm_page_t m) {
+swap_pager_freepage(m)
+       vm_page_t m;
+{
        PAGE_WAKEUP(m);
        vm_page_free(m);
 }
 
 static void
        PAGE_WAKEUP(m);
        vm_page_free(m);
 }
 
 static void
-swap_pager_ridpages( vm_page_t *m, int count, int reqpage) {
+swap_pager_ridpages(m, count, reqpage)
+       vm_page_t *m;
+       int count;
+       int reqpage;
+{
        int i;
        int i;
-       for(i=0;i<count;i++)
-               if( i != reqpage)
-                       swap_pager_freepage( m[i]);
+       int s;
+
+       s = splhigh();
+       for (i = 0; i < count; i++)
+               if (i != reqpage)
+                       swap_pager_freepage(m[i]);
+       splx(s);
 }
 
 int swapwritecount=0;
 
 }
 
 int swapwritecount=0;
 
-void swap_pager_iodone(struct buf *bp);
-
 void
 void
-swap_pager_iodone1(struct buf *bp) {
+swap_pager_iodone1(bp)
+       struct buf *bp;
+{
        bp->b_flags |= B_DONE;
        bp->b_flags &= ~B_ASYNC;
        wakeup((caddr_t)bp);
        bp->b_flags |= B_DONE;
        bp->b_flags &= ~B_ASYNC;
        wakeup((caddr_t)bp);
-       if(bp->b_flags & B_READ) == 0)
+       if ((bp->b_flags & B_READ) == 0)
                vwakeup(bp);
 }
 /*
  * Scaled down version of swap().
  * BOGUS:  lower level IO routines expect a KVA so we have to map our
  * provided physical page into the KVA to keep them happy.
                vwakeup(bp);
 }
 /*
  * Scaled down version of swap().
  * BOGUS:  lower level IO routines expect a KVA so we have to map our
  * provided physical page into the KVA to keep them happy.
- *
  */
 int
 swap_pager_io(swp, m, count, reqpage, flags)
  */
 int
 swap_pager_io(swp, m, count, reqpage, flags)
@@ -606,22 +736,18 @@ swap_pager_io(swp, m, count, reqpage, flags)
        register struct buf *bp;
        register sw_blk_t swb;
        register int s;
        register struct buf *bp;
        register sw_blk_t swb;
        register int s;
-       int i;
-       int ix;
+       int i, ix;
        boolean_t rv;
        vm_offset_t kva, off;
        swp_clean_t spc;
        int cluster;
        vm_offset_t paging_offset;
        vm_object_t object;
        boolean_t rv;
        vm_offset_t kva, off;
        swp_clean_t spc;
        int cluster;
        vm_offset_t paging_offset;
        vm_object_t object;
-       int reqaddr;
-       int mydskregion;
+       int reqaddr, mydskregion;
        extern int dmmin, dmmax;
 
        extern int dmmin, dmmax;
 
-
        spc = NULL;
 
        spc = NULL;
 
-
        object = m[reqpage]->object;
        paging_offset = object->paging_offset;
        /*
        object = m[reqpage]->object;
        paging_offset = object->paging_offset;
        /*
@@ -631,42 +757,47 @@ swap_pager_io(swp, m, count, reqpage, flags)
         * with the page.
         */
        off = m[reqpage]->offset + paging_offset;
         * with the page.
         */
        off = m[reqpage]->offset + paging_offset;
-       ix = swap_pager_block_index( swp, off);
+       ix = swap_pager_block_index(swp, off);
        if (swp->sw_blocks == NULL || ix >= swp->sw_nblocks) {
                /* printf("swap pager: out of range\n"); */
        if (swp->sw_blocks == NULL || ix >= swp->sw_nblocks) {
                /* printf("swap pager: out of range\n"); */
-               swap_pager_ridpages( m, count, reqpage);
+               swap_pager_ridpages(m, count, reqpage);
                return(VM_PAGER_FAIL);
        }
        
 
        swb = &swp->sw_blocks[ix];
        off = swap_pager_block_offset(swp, off) / NBPG;
                return(VM_PAGER_FAIL);
        }
        
 
        swb = &swp->sw_blocks[ix];
        off = swap_pager_block_offset(swp, off) / NBPG;
-       if ((flags & B_READ) &&
-           ((swb->swb_block[off] == SWB_EMPTY) ||
-               (swb->swb_valid & (1 << off)) == 0)) {
-               swap_pager_ridpages( m, count, reqpage);
-               return(VM_PAGER_FAIL);
-       }
-
        reqaddr = swb->swb_block[off];
 
        /* make sure that our I/O request is contiguous */
        reqaddr = swb->swb_block[off];
 
        /* make sure that our I/O request is contiguous */
-       if ( flags & B_READ) {
-               int first=0, last=count;
+       if (flags & B_READ) {
+               int first = 0, last = count;
                int failed = 0;
                int reqdskregion = reqaddr / dmmax;
                int failed = 0;
                int reqdskregion = reqaddr / dmmax;
-               for(i=reqpage-1;i>=0;--i) {
-                       int *tmpaddr = swap_pager_diskaddr(swp, m[i]->offset + paging_offset,0);
-                       if( tmpaddr == 0 || failed ||
-                               *tmpaddr != reqaddr + btodb((i-reqpage)*NBPG) ) {
+               int valid;
+
+               if (reqaddr == SWB_EMPTY ||
+                       (swb->swb_valid & (1 << off)) == 0) {
+                       swap_pager_ridpages(m, count, reqpage);
+                       return(VM_PAGER_FAIL);
+               }
+                               
+               /*
+                * search backwards for the first contiguous page to transfer
+                */
+               for (i = reqpage - 1; i >= 0; --i) {
+                       int *tmpaddr = swap_pager_diskaddr(swp,
+                               m[i]->offset + paging_offset,&valid);
+                       if (tmpaddr == 0 || failed || !valid ||
+                               *tmpaddr != reqaddr + btodb((i - reqpage) * NBPG)) {
                                failed = 1;
                                swap_pager_freepage(m[i]);
                                m[i] = 0;
                                failed = 1;
                                swap_pager_freepage(m[i]);
                                m[i] = 0;
-                               iffirst == 0)
+                               if (first == 0)
                                        first = i + 1;
                        } else {
                                mydskregion = *tmpaddr / dmmax;
                                        first = i + 1;
                        } else {
                                mydskregion = *tmpaddr / dmmax;
-                               ifmydskregion != reqdskregion) {
+                               if (mydskregion != reqdskregion) {
                                        failed = 1;
                                        swap_pager_freepage(m[i]);
                                        m[i] = 0;
                                        failed = 1;
                                        swap_pager_freepage(m[i]);
                                        m[i] = 0;
@@ -674,38 +805,40 @@ swap_pager_io(swp, m, count, reqpage, flags)
                                }
                        }
                }
                                }
                        }
                }
+               /*
+                * search forwards for the last contiguous page to transfer
+                */
                failed = 0;
                failed = 0;
-               for(i=reqpage+1;i<count;i++) {
+               for (i = reqpage + 1; i < count; i++) {
                        int *tmpaddr = swap_pager_diskaddr(swp, m[i]->offset + paging_offset,0);
                        int *tmpaddr = swap_pager_diskaddr(swp, m[i]->offset + paging_offset,0);
-                       if( tmpaddr == 0 || failed ||
-                               *tmpaddr != reqaddr + btodb((i-reqpage)*NBPG) ) {
+                       if (tmpaddr == 0 || failed || !valid ||
+                               *tmpaddr != reqaddr + btodb((i - reqpage) * NBPG) ) {
                                failed = 1;
                                swap_pager_freepage(m[i]);
                                m[i] = 0;
                                failed = 1;
                                swap_pager_freepage(m[i]);
                                m[i] = 0;
-                               iflast == count)
+                               if (last == count)
                                        last = i;
                        } else {
                                mydskregion = *tmpaddr / dmmax;
                                        last = i;
                        } else {
                                mydskregion = *tmpaddr / dmmax;
-                               ifmydskregion != reqdskregion) {
+                               if (mydskregion != reqdskregion) {
                                        failed = 1;
                                        swap_pager_freepage(m[i]);
                                        m[i] = 0;
                                        failed = 1;
                                        swap_pager_freepage(m[i]);
                                        m[i] = 0;
-                                       iflast == count)
+                                       if (last == count)
                                                last = i;
                                }
                        }
                }
                count = last;
                                                last = i;
                                }
                        }
                }
                count = last;
-               iffirst != 0) {
-                       for(i=first;i<count;i++) {
-                               m[i-first] = m[i];
+               if (first != 0) {
+                       for (i = first; i < count; i++) {
+                               m[i - first] = m[i];
                        }
                        count -= first;
                        reqpage -= first;
                }
        }
        
                        }
                        count -= first;
                        reqpage -= first;
                }
        }
        
-
        /*
         * For reads (pageins) and synchronous writes, we clean up
         * all completed async pageouts.
        /*
         * For reads (pageins) and synchronous writes, we clean up
         * all completed async pageouts.
@@ -720,102 +853,112 @@ swap_pager_io(swp, m, count, reqpage, flags)
         * are available, we try again later.
         */
        else if (swap_pager_clean(m[reqpage], B_WRITE)) {
         * are available, we try again later.
         */
        else if (swap_pager_clean(m[reqpage], B_WRITE)) {
-               swap_pager_ridpages( m, count, reqpage);
-               return VM_PAGER_FAIL;
+               swap_pager_ridpages(m, count, reqpage);
+               return VM_PAGER_TRYAGAIN;
        }
 
        spc = NULL;     /* we might not use an spc data structure */
        kva = 0;
 
        }
 
        spc = NULL;     /* we might not use an spc data structure */
        kva = 0;
 
-       if( (flags & B_READ) && count > 1) {
-               kva = kmem_alloc_pageable( pager_map, count*NBPG);
-       }
-               
-       if( !kva) {
        /*
        /*
-        * get a swap pager clean data structure, block until we get it
+        * we allocate a new kva for transfers > 1 page
+        * but for transfers == 1 page, the swap_pager_free list contains
+        * entries that have pre-allocated kva's
         */
         */
-               /*
-                * Note: The expected wakeup comes from swap_pager_iodone
-                * (a routine called at interrupt time.)
-                */
-               s = splbio();
-
-               if (queue_empty(&swap_pager_free))
-                       (void) swap_pager_clean(NULL, B_WRITE);
-
-               while (queue_empty(&swap_pager_free)) { 
-                       tsleep((caddr_t)&swap_wakeup, PVM, "swpfre", 0);
-                       /*
-                        * Grr...it would be best to call the clean not at
-                        *      splbio..but it's necessary to stay at splbio
-                        *      until the queue_empty check is complete.
-                        */
-                       (void) swap_pager_clean(NULL, B_WRITE);
-               }
-               splx(s);
-
-               queue_remove_first(&swap_pager_free, spc, swp_clean_t, spc_list);
-               for(i=0;i<count;i++) {
-                       if( i != reqpage) {
-                               swap_pager_freepage( m[i]);
+       if ((flags & B_READ) && count > 1) {
+               kva = kmem_alloc_pageable(pager_map, count*NBPG);
+       }
+               
+       if (!kva) {
+               for (i = 0; i < count; i++) {
+                       if (i != reqpage) {
+                               swap_pager_freepage(m[i]);
                                m[i] = 0;
                        }
                }
                count = 1;
                m[0] = m[reqpage];
                reqpage = 0;
                                m[i] = 0;
                        }
                }
                count = 1;
                m[0] = m[reqpage];
                reqpage = 0;
+       /*
+        * get a swap pager clean data structure, block until we get it
+        */
+               if (queue_empty(&swap_pager_free)) {
+                       s = splbio();
+                       (void) swap_pager_clean(NULL, B_WRITE);
+                       while (queue_empty(&swap_pager_free)) { 
+                               swap_pager_needflags |= SWAP_FREE_NEEDED;
+                               tsleep((caddr_t)&swap_pager_free,
+                                       PVM, "swpfre", 0);
+                               (void) swap_pager_clean(NULL, B_WRITE);
+                       }
+                       splx(s);
+               }
+               queue_remove_first(&swap_pager_free, spc, swp_clean_t, spc_list);
                kva = spc->spc_kva;
        }
        
 
        /*
         * Determine swap block and allocate as necessary.
                kva = spc->spc_kva;
        }
        
 
        /*
         * Determine swap block and allocate as necessary.
+        * We try to get SWB_NPAGES first, but then we punt and try
+        * to get one page.  If that fails, we look at the allocation
+        * data structures to find unused but allocated pages in other
+        * pagers allocations.
         */
        if (reqaddr == SWB_EMPTY) {
                int blk;
         */
        if (reqaddr == SWB_EMPTY) {
                int blk;
-               for(i=0;i<SWB_NPAGES;i++)
-                       if( swb->swb_block[i] != SWB_EMPTY)
+               int tries;
+               int ntoget;
+
+               tries = 0;
+               s = splbio();
+               for (i = 0; i < SWB_NPAGES; i++) {
+                       if (swb->swb_block[i] != SWB_EMPTY)
                                break;
                                break;
-               if( i == SWB_NPAGES &&
-                       rlist_alloc(&swapmap, btodb( SWB_NPAGES*NBPG),&blk)) {
-                       for(i=0;i<SWB_NPAGES;i++)
-                               swb->swb_block[i] = blk + btodb(NBPG)*i;
-               } else if( !rlist_alloc(&swapmap, btodb( NBPG), &swb->swb_block[off])) {
-                               if( spc)
+               }
+
+               ntoget = (i == SWB_NPAGES) ? SWB_NPAGES : 1;
+retrygetspace:
+               if (ntoget == SWB_NPAGES &&
+                       rlist_alloc(&swapmap, btodb(ntoget * NBPG),&blk)) {
+                       for (i = 0; i < ntoget; i++)
+                               swb->swb_block[i] = blk + btodb(NBPG) * i;
+               } else if (!rlist_alloc(&swapmap, btodb(NBPG), &swb->swb_block[off])) {
+                               if (++tries == 1) {
+                                       swap_pager_reclaim();
+                                       goto retrygetspace;
+                               }
+                               if (spc)
                                        queue_enter(&swap_pager_free, spc, swp_clean_t, spc_list);
                                        queue_enter(&swap_pager_free, spc, swp_clean_t, spc_list);
-                               ifswap_pager_full == 0) 
+                               if (swap_pager_full == 0) 
                                        printf("swap_pager: out of swap space !!!\n");
                                swap_pager_full = 1;
                                        printf("swap_pager: out of swap space !!!\n");
                                swap_pager_full = 1;
-                               swap_pager_ridpages( m, count, reqpage);
-                               return(VM_PAGER_FAIL);
+                               swap_pager_ridpages(m, count, reqpage);
+                               splx(s);
+                               return(VM_PAGER_TRYAGAIN);
                }
                }
+               splx(s);
                swap_pager_full = 0;
        }
 
                swap_pager_full = 0;
        }
 
-       for(i=0;i<count;i++) {
-               pmap_enter(vm_map_pmap( pager_map), kva+NBPG*i,
+       for (i = 0; i < count; i++) {
+               pmap_enter(vm_map_pmap(pager_map), kva + NBPG * i,
                        VM_PAGE_TO_PHYS(m[i]), VM_PROT_ALL, TRUE);
        }
                                
                        VM_PAGE_TO_PHYS(m[i]), VM_PROT_ALL, TRUE);
        }
                                
-       off = swap_pager_block_offset(swp, m[0]->offset+paging_offset) / NBPG;
+       off = swap_pager_block_offset(swp, m[0]->offset + paging_offset) / NBPG;
 
 
-/*
-       if( flags & B_READ)
+#ifdef DEBUG
+       if (flags & B_READ && count > 1)
                printf("obj: 0x%x off: 0x%x poff: 0x%x off: 0x%x, sz: %d blk: %d op: %s\n",
                        object, m[0]->offset, paging_offset, off, count, swb->swb_block[off], flags&B_READ?"r":"w");
                printf("obj: 0x%x off: 0x%x poff: 0x%x off: 0x%x, sz: %d blk: %d op: %s\n",
                        object, m[0]->offset, paging_offset, off, count, swb->swb_block[off], flags&B_READ?"r":"w");
-*/
+#endif
 
        s = splbio();
        /*
         * Get a swap buffer header and perform the IO
         */
 
        s = splbio();
        /*
         * Get a swap buffer header and perform the IO
         */
-       while (bswlist.av_forw == NULL) {
-               bswlist.b_flags |= B_WANTED;
-               tsleep((caddr_t)&bswlist, PSWP+1, "wswbuf", 0); 
-       }
-       bp = bswlist.av_forw;
-       bswlist.av_forw = bp->av_forw;
+       bp = getpbuf();
        bp->b_flags = B_BUSY | (flags & B_READ);
        bp->b_proc = &proc0;    /* XXX (but without B_PHYS set this is ok) */
        bp->b_un.b_addr = (caddr_t) kva;
        bp->b_flags = B_BUSY | (flags & B_READ);
        bp->b_proc = &proc0;    /* XXX (but without B_PHYS set this is ok) */
        bp->b_un.b_addr = (caddr_t) kva;
@@ -844,14 +987,18 @@ swap_pager_io(swp, m, count, reqpage, flags)
                queue_enter(&swap_pager_inuse, spc, swp_clean_t, spc_list);
                swb->swb_valid |= (1 << off);
        } else {
                queue_enter(&swap_pager_inuse, spc, swp_clean_t, spc_list);
                swb->swb_valid |= (1 << off);
        } else {
-               if( (flags & B_READ) == 0)
+               if ((flags & B_READ) == 0) {
                        swb->swb_valid |= (1 << off);
                        swb->swb_valid |= (1 << off);
+                       swp->sw_poip++;
+               } else {
+                       swp->sw_piip++;
+               }
                bp->b_flags |= B_CALL;
                bp->b_iodone = swap_pager_iodone1;
        }
        VOP_STRATEGY(bp);
        if ((flags & (B_READ|B_ASYNC)) == B_ASYNC ) {
                bp->b_flags |= B_CALL;
                bp->b_iodone = swap_pager_iodone1;
        }
        VOP_STRATEGY(bp);
        if ((flags & (B_READ|B_ASYNC)) == B_ASYNC ) {
-               if(bp->b_flags & B_DONE) == B_DONE) {
+               if ((bp->b_flags & B_DONE) == B_DONE) {
                        swap_pager_clean(NULL, flags);
                }
                splx(s);
                        swap_pager_clean(NULL, flags);
                }
                splx(s);
@@ -862,61 +1009,65 @@ swap_pager_io(swp, m, count, reqpage, flags)
        }
        rv = (bp->b_flags & B_ERROR) ? VM_PAGER_FAIL : VM_PAGER_OK;
        bp->b_flags &= ~(B_BUSY|B_WANTED|B_PHYS|B_DIRTY|B_CALL|B_DONE);
        }
        rv = (bp->b_flags & B_ERROR) ? VM_PAGER_FAIL : VM_PAGER_OK;
        bp->b_flags &= ~(B_BUSY|B_WANTED|B_PHYS|B_DIRTY|B_CALL|B_DONE);
-       bp->av_forw = bswlist.av_forw;
-       bswlist.av_forw = bp;
-       if( bswlist.b_flags & B_WANTED) {
-               bswlist.b_flags &= ~B_WANTED;
-               wakeup((caddr_t)&bswlist);
-       }
 
 
+       if (bp->b_flags & B_READ) {
+               --swp->sw_piip;
+               if (swp->sw_piip == 0)
+                       wakeup((caddr_t) swp);
+       } else {
+               --swp->sw_poip;
+               if (swp->sw_poip == 0)
+                       wakeup((caddr_t) swp);
+       }
+               
        if (bp->b_vp)
                brelvp(bp);
 
        if (bp->b_vp)
                brelvp(bp);
 
+       relpbuf(bp);
+
        splx(s);
 
        splx(s);
 
-       pmap_remove(vm_map_pmap( pager_map), kva, kva+count*NBPG);
+       pmap_remove(vm_map_pmap(pager_map), kva, kva + count * NBPG);
 
        if ((flags & B_READ) == 0 && rv == VM_PAGER_OK) {
                m[reqpage]->flags |= PG_CLEAN;
                pmap_clear_modify(VM_PAGE_TO_PHYS(m[reqpage]));
 
        if ((flags & B_READ) == 0 && rv == VM_PAGER_OK) {
                m[reqpage]->flags |= PG_CLEAN;
                pmap_clear_modify(VM_PAGE_TO_PHYS(m[reqpage]));
+               if (pmap_is_referenced(VM_PAGE_TO_PHYS(m[reqpage])))
+                       vm_page_activate(m[reqpage]);
+               pmap_clear_reference(VM_PAGE_TO_PHYS(m[reqpage]));
        }
 
        }
 
-       ifspc) {
+       if (spc) {
                queue_enter(&swap_pager_free, spc, swp_clean_t, spc_list);
        } else {
                queue_enter(&swap_pager_free, spc, swp_clean_t, spc_list);
        } else {
-               for(i=0;i<count;i++) {
-#ifdef PMAP_ATTRIBUTES
-                       /*
-                        * mark the page as unmodified
-                        */
-                       pmap_clear_cached_attributes(VM_PAGE_TO_PHYS(m[i]));
-#else
+               for (i = 0; i < count; i++) {
                        pmap_clear_modify(VM_PAGE_TO_PHYS(m[i]));
                        pmap_clear_modify(VM_PAGE_TO_PHYS(m[i]));
-#endif
                        m[i]->flags |= PG_CLEAN;
                        m[i]->flags &= ~PG_LAUNDRY;
                        m[i]->flags |= PG_CLEAN;
                        m[i]->flags &= ~PG_LAUNDRY;
-                       ifi != reqpage) {
+                       if (i != reqpage) {
                                /*
                                 * whether or not to leave the page activated
                                 * is up in the air, but we should put the page
                                 * on a page queue somewhere. (it already is in
                                 * the object).
                                 */
                                /*
                                 * whether or not to leave the page activated
                                 * is up in the air, but we should put the page
                                 * on a page queue somewhere. (it already is in
                                 * the object).
                                 */
-                               if( i < count/2 && i > reqpage)
+                               if (i < (reqpage + 3))
                                        vm_page_activate(m[i]);
                                else
                                        vm_page_deactivate(m[i]); 
                                        vm_page_activate(m[i]);
                                else
                                        vm_page_deactivate(m[i]); 
+
                                /*
                                 * just in case someone was asking for this
                                 * page we now tell them that it is ok to use
                                 */
                                /*
                                 * just in case someone was asking for this
                                 * page we now tell them that it is ok to use
                                 */
+                               m[i]->flags &= ~PG_FAKE;
                                PAGE_WAKEUP(m[i]);
                        }
                }
 /*
  * and free the kernel virtual addresses
  */
                                PAGE_WAKEUP(m[i]);
                        }
                }
 /*
  * and free the kernel virtual addresses
  */
-               kmem_free( pager_map, kva, count*NBPG);
+               kmem_free_wakeup(pager_map, kva, count * NBPG);
        }
        return(rv);
 }
        }
        return(rv);
 }
@@ -930,7 +1081,7 @@ swap_pager_clean(m, rw)
        register int s;
 
        tspc = NULL;
        register int s;
 
        tspc = NULL;
-       if( queue_empty( &swap_pager_done))
+       if (queue_empty(&swap_pager_done))
                return FALSE;
        for (;;) {
                s = splbio();
                return FALSE;
        for (;;) {
                s = splbio();
@@ -940,9 +1091,9 @@ swap_pager_clean(m, rw)
                 */
                spc = (swp_clean_t) queue_first(&swap_pager_done);
                while (!queue_end(&swap_pager_done, (queue_entry_t)spc)) {
                 */
                spc = (swp_clean_t) queue_first(&swap_pager_done);
                while (!queue_end(&swap_pager_done, (queue_entry_t)spc)) {
+                       pmap_remove(vm_map_pmap(pager_map), spc->spc_kva, ((vm_offset_t) spc->spc_kva) + NBPG);
                        swap_pager_finish(spc);
                        swap_pager_finish(spc);
-                       queue_remove(&swap_pager_done, spc,
-                                    swp_clean_t, spc_list);
+                       queue_remove(&swap_pager_done, spc, swp_clean_t, spc_list);
                        goto doclean;
                }
 
                        goto doclean;
                }
 
@@ -953,7 +1104,6 @@ swap_pager_clean(m, rw)
                splx(s);
                break;
 
                splx(s);
                break;
 
-
                /*
                 * The desired page was found to be busy earlier in
                 * the scan but has since completed.
                /*
                 * The desired page was found to be busy earlier in
                 * the scan but has since completed.
@@ -963,7 +1113,6 @@ doclean:
                        tspc = NULL;
                }
                spc->spc_flags = 0;
                        tspc = NULL;
                }
                spc->spc_flags = 0;
-               pmap_remove(vm_map_pmap( pager_map), spc->spc_kva, ((vm_offset_t) spc->spc_kva)+NBPG);
                queue_enter(&swap_pager_free, spc, swp_clean_t, spc_list);
                ++cleandone;
                splx(s);
                queue_enter(&swap_pager_free, spc, swp_clean_t, spc_list);
                ++cleandone;
                splx(s);
@@ -983,7 +1132,6 @@ swap_pager_finish(spc)
        if (--object->paging_in_progress == 0) 
                thread_wakeup((int) object);
 
        if (--object->paging_in_progress == 0) 
                thread_wakeup((int) object);
 
-
        /*
         * If no error mark as clean and inform the pmap system.
         * If error, mark as dirty so we will try again.
        /*
         * If no error mark as clean and inform the pmap system.
         * If error, mark as dirty so we will try again.
@@ -997,28 +1145,38 @@ swap_pager_finish(spc)
                pmap_clear_modify(VM_PAGE_TO_PHYS(m));
                m->flags |= PG_CLEAN;
        }
                pmap_clear_modify(VM_PAGE_TO_PHYS(m));
                m->flags |= PG_CLEAN;
        }
+
+       if (pmap_is_referenced(VM_PAGE_TO_PHYS(m))) 
+               vm_page_activate(m);
+
        pmap_clear_reference(VM_PAGE_TO_PHYS(m));
        PAGE_WAKEUP(m);
        /*
         * if we need memory desperately, then free it now
         */
        pmap_clear_reference(VM_PAGE_TO_PHYS(m));
        PAGE_WAKEUP(m);
        /*
         * if we need memory desperately, then free it now
         */
-       if((m->flags & PG_CLEAN) &&
-               vm_page_free_count <= vm_pageout_free_min) {
-               pmap_page_protect( VM_PAGE_TO_PHYS(m), VM_PROT_NONE);
+       if (vm_page_free_count < vm_page_free_reserved &&
+               (m->flags & PG_CLEAN) && m->wire_count == 0 &&
+               !pmap_is_wired(VM_PAGE_TO_PHYS(m))) {
+               pmap_page_protect(VM_PAGE_TO_PHYS(m), VM_PROT_NONE);
                vm_page_free(m);
                vm_page_free(m);
-               thread_wakeup((int) &vm_pages_needed);
        }
        --nswiodone;
 
        return;
 }
 
        }
        --nswiodone;
 
        return;
 }
 
+/*
+ * swap_pager_iodone
+ */
 void
 swap_pager_iodone(bp)
        register struct buf *bp;
 {
        register swp_clean_t spc;
        daddr_t blk;
 void
 swap_pager_iodone(bp)
        register struct buf *bp;
 {
        register swp_clean_t spc;
        daddr_t blk;
+       int s;
+
+       s = splbio();
        spc = (swp_clean_t) bp->b_spc;
        queue_remove(&swap_pager_inuse, spc, swp_clean_t, spc_list);
        queue_enter(&swap_pager_done, spc, swp_clean_t, spc_list);
        spc = (swp_clean_t) bp->b_spc;
        queue_remove(&swap_pager_inuse, spc, swp_clean_t, spc_list);
        queue_enter(&swap_pager_done, spc, swp_clean_t, spc_list);
@@ -1029,29 +1187,77 @@ swap_pager_iodone(bp)
        }
        spc->spc_bp = NULL;
 
        }
        spc->spc_bp = NULL;
 
-       if(bp->b_flags & B_READ) == 0)
+       if ((bp->b_flags & B_READ) == 0)
                vwakeup(bp);
                
        bp->b_flags &= ~(B_BUSY|B_WANTED|B_PHYS|B_DIRTY|B_ASYNC);
        if (bp->b_vp) {
                brelvp(bp);
        }
                vwakeup(bp);
                
        bp->b_flags &= ~(B_BUSY|B_WANTED|B_PHYS|B_DIRTY|B_ASYNC);
        if (bp->b_vp) {
                brelvp(bp);
        }
-       bp->av_forw = bswlist.av_forw;
-       bswlist.av_forw = bp;
-       if( bswlist.b_flags & B_WANTED) {
-               bswlist.b_flags &= ~B_WANTED;
-               wakeup((caddr_t)&bswlist);
-       }
+
+       relpbuf(bp);
        nswiodone++;
        nswiodone++;
+       if (--spc->spc_swp->sw_poip == 0) {
+               wakeup((caddr_t)spc->spc_swp);
+       }
+
+       if ((swap_pager_needflags & SWAP_FREE_NEEDED) ||
+           queue_empty(&swap_pager_inuse)) { 
+               swap_pager_needflags &= ~SWAP_FREE_NEEDED;
+               wakeup((caddr_t)&swap_pager_free);
+       }
 
 
-       if( (--spc->spc_swp->sw_poip == 0) ||
-               queue_empty( &swap_pager_inuse)) { 
-               wakeup( (caddr_t)&swap_wakeup);
+       if (vm_pageout_pages_needed) {
+               wakeup((caddr_t)&vm_pageout_pages_needed);
        }
 
        }
 
-       if( queue_empty( &swap_pager_inuse) ||
-               queue_empty( &swap_pager_free) ||
-               nswiodone >= npendingio / 2 ) { 
-               thread_wakeup((int) &vm_pages_needed);
+       if (queue_empty(&swap_pager_inuse) ||
+           (vm_page_free_count < vm_page_free_min &&
+           nswiodone + vm_page_free_count >= vm_page_free_min) ) {
+               wakeup((caddr_t)&vm_pages_needed);
        }
        }
+       splx(s);
 }
 }
+
+/*
+ * allocate a physical buffer 
+ */
+struct buf *
+getpbuf() {
+       int s;
+       struct buf *bp;
+
+       s = splbio();
+       /* get a bp from the swap buffer header pool */
+       while (bswlist.av_forw == NULL) {
+               bswlist.b_flags |= B_WANTED;
+               tsleep((caddr_t)&bswlist, PVM, "wswbuf", 0); 
+       }
+       bp = bswlist.av_forw;
+       bswlist.av_forw = bp->av_forw;
+
+       splx(s);
+
+       bzero(bp, sizeof *bp);
+       return bp;
+}
+
+/*
+ * release a physical buffer
+ */
+void
+relpbuf(bp)
+       struct buf *bp;
+{
+       int s;
+
+       s = splbio();
+       bp->av_forw = bswlist.av_forw;
+       bswlist.av_forw = bp;
+       if (bswlist.b_flags & B_WANTED) {
+               bswlist.b_flags &= ~B_WANTED;
+               wakeup((caddr_t)&bswlist);
+       }
+       splx(s);
+}
+
index 310319a..2505375 100644 (file)
@@ -36,7 +36,7 @@
  * SUCH DAMAGE.
  *
  *     from: @(#)swap_pager.h  7.1 (Berkeley) 12/5/90
  * SUCH DAMAGE.
  *
  *     from: @(#)swap_pager.h  7.1 (Berkeley) 12/5/90
- *     $Id: swap_pager.h,v 1.2 1993/10/16 16:20:21 rgrimes Exp $
+ *     $Id: swap_pager.h,v 1.5 1993/12/22 12:51:57 davidg Exp $
  */
 
 /*
  */
 
 /*
  * however, due to the allocation spilling into non-swap pager backed memory,
  * suggest keeping SWB_NPAGES small (1-4).  If high performance is manditory
  * perhaps up to 8 pages might be in order????
  * however, due to the allocation spilling into non-swap pager backed memory,
  * suggest keeping SWB_NPAGES small (1-4).  If high performance is manditory
  * perhaps up to 8 pages might be in order????
+ * Above problem has been fixed, now we support 16 pages per block.  Unused
+ * space is recovered by the swap pager now...
  */
  */
-#define SWB_NPAGES 1
+#define SWB_NPAGES 8
 struct swblock {
        unsigned int swb_valid;         /* bitmask for valid pages */
        int      swb_block[SWB_NPAGES]; /* unfortunately int instead of daddr_t */
 struct swblock {
        unsigned int swb_valid;         /* bitmask for valid pages */
        int      swb_block[SWB_NPAGES]; /* unfortunately int instead of daddr_t */
@@ -69,6 +71,7 @@ struct swpager {
        sw_blk_t     sw_blocks; /* pointer to list of swap blocks */
        short        sw_flags;  /* flags */
        short        sw_poip;   /* pageouts in progress */
        sw_blk_t     sw_blocks; /* pointer to list of swap blocks */
        short        sw_flags;  /* flags */
        short        sw_poip;   /* pageouts in progress */
+       short        sw_piip;   /* pageins in progress */
 };
 typedef struct swpager *sw_pager_t;
 
 };
 typedef struct swpager *sw_pager_t;
 
index 854e4be..8c15e35 100644 (file)
@@ -1,6 +1,10 @@
 /* 
  * Copyright (c) 1991 Regents of the University of California.
  * All rights reserved.
 /* 
  * Copyright (c) 1991 Regents of the University of California.
  * All rights reserved.
+ * Copyright (c) John S. Dyson
+ * All rights reserved.
+ * Copyright (c) David Greenman
+ * All rights reserved.
  *
  * This code is derived from software contributed to Berkeley by
  * The Mach Operating System project at Carnegie-Mellon University.
  *
  * This code is derived from software contributed to Berkeley by
  * The Mach Operating System project at Carnegie-Mellon University.
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
- *     from: @(#)vm_fault.c    7.6 (Berkeley) 5/7/91
- *     $Id: vm_fault.c,v 1.10 1993/12/21 05:51:00 davidg Exp $
- */
-
-/*
+ *     @(#)vm_fault.c  7.6 (Berkeley) 5/7/91
+ *
+ *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  *
  * any improvements or extensions that they make and grant Carnegie the
  * rights to redistribute these changes.
  */
  * any improvements or extensions that they make and grant Carnegie the
  * rights to redistribute these changes.
  */
+/*
+ * $Id$
+ */
 
 /*
  *     Page fault handling module.
  */
 
 #include "param.h"
 
 /*
  *     Page fault handling module.
  */
 
 #include "param.h"
-#include "systm.h"
-#include "proc.h"              /* XXX - just to get curproc */
 
 #include "vm.h"
 #include "vm_page.h"
 #include "vm_pageout.h"
 
 #include "vm.h"
 #include "vm_page.h"
 #include "vm_pageout.h"
-#include "vm_user.h"           /* make sure we match prototype */
+#include "proc.h"
+#include "resource.h"
+#include "resourcevar.h"
 
 
-vm_statistics_data_t vm_stat;
+#define VM_FAULT_READ_AHEAD 8
+#define VM_FAULT_READ_AHEAD_MIN 4
+#define VM_FAULT_READ_BEHIND 1
+#define VM_FAULT_READ (VM_FAULT_READ_AHEAD+VM_FAULT_READ_BEHIND+1)
+extern int swap_pager_full;
+extern int vm_pageout_proc_limit;
 
 
+vm_statistics_data_t vm_stat;
 /*
  *     vm_fault:
  *
 /*
  *     vm_fault:
  *
@@ -119,6 +130,9 @@ vm_fault(map, vaddr, fault_type, change_wiring)
        boolean_t               page_exists;
        vm_page_t               old_m;
        vm_object_t             next_object;
        boolean_t               page_exists;
        vm_page_t               old_m;
        vm_object_t             next_object;
+       vm_page_t               marray[VM_FAULT_READ];
+       int                     reqpage;
+       int                     spl;
 
        vm_stat.faults++;               /* needs lock XXX */
 /*
 
        vm_stat.faults++;               /* needs lock XXX */
 /*
@@ -147,11 +161,15 @@ vm_fault(map, vaddr, fault_type, change_wiring)
 
 #define        UNLOCK_THINGS   {                               \
        object->paging_in_progress--;                   \
 
 #define        UNLOCK_THINGS   {                               \
        object->paging_in_progress--;                   \
+       if (object->paging_in_progress == 0)            \
+               wakeup((caddr_t)object);                \
        vm_object_unlock(object);                       \
        if (object != first_object) {                   \
                vm_object_lock(first_object);           \
                FREE_PAGE(first_m);                     \
                first_object->paging_in_progress--;     \
        vm_object_unlock(object);                       \
        if (object != first_object) {                   \
                vm_object_lock(first_object);           \
                FREE_PAGE(first_m);                     \
                first_object->paging_in_progress--;     \
+               if (first_object->paging_in_progress == 0) \
+                       wakeup((caddr_t)first_object);  \
                vm_object_unlock(first_object);         \
        }                                               \
        UNLOCK_MAP;                                     \
                vm_object_unlock(first_object);         \
        }                                               \
        UNLOCK_MAP;                                     \
@@ -162,6 +180,7 @@ vm_fault(map, vaddr, fault_type, change_wiring)
        vm_object_deallocate(first_object);             \
 }
 
        vm_object_deallocate(first_object);             \
 }
 
+
     RetryFault: ;
 
        /*
     RetryFault: ;
 
        /*
@@ -170,8 +189,8 @@ vm_fault(map, vaddr, fault_type, change_wiring)
         */
 
        if ((result = vm_map_lookup(&map, vaddr, fault_type, &entry,
         */
 
        if ((result = vm_map_lookup(&map, vaddr, fault_type, &entry,
-                       &first_object, &first_offset,
-                       &prot, &wired, &su)) != KERN_SUCCESS) {
+           &first_object, &first_offset,
+           &prot, &wired, &su)) != KERN_SUCCESS) {
                return(result);
        }
        lookup_still_valid = TRUE;
                return(result);
        }
        lookup_still_valid = TRUE;
@@ -247,25 +266,14 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                         *      wait for it and then retry.
                         */
                        if (m->flags & PG_BUSY) {
                         *      wait for it and then retry.
                         */
                        if (m->flags & PG_BUSY) {
-#ifdef DOTHREADS
-                               int     wait_result;
-
-                               PAGE_ASSERT_WAIT(m, !change_wiring);
+                               int s;
                                UNLOCK_THINGS;
                                UNLOCK_THINGS;
-                               thread_block("pagein");
-                               wait_result = current_thread()->wait_result;
-                               vm_object_deallocate(first_object);
-                               if (wait_result != THREAD_AWAKENED)
-                                       return(KERN_SUCCESS);
-                               goto RetryFault;
-#else
-                               PAGE_ASSERT_WAIT(m, !change_wiring);
-                               UNLOCK_THINGS;
-                               thread_wakeup((int)&vm_pages_needed);/* XXX! */
-                               thread_block("pagein");
+                               if (m->flags & PG_BUSY) {
+                                       m->flags |= PG_WANTED;
+                                       tsleep((caddr_t)m,PVM,"vmpfw",0);
+                               }
                                vm_object_deallocate(first_object);
                                goto RetryFault;
                                vm_object_deallocate(first_object);
                                goto RetryFault;
-#endif
                        }
 
                        if (m->flags & PG_ABSENT)
                        }
 
                        if (m->flags & PG_ABSENT)
@@ -277,6 +285,7 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                         */
 
                        vm_page_lock_queues();
                         */
 
                        vm_page_lock_queues();
+                       spl = vm_disable_intr();
                        if (m->flags & PG_INACTIVE) {
                                queue_remove(&vm_page_queue_inactive, m,
                                                vm_page_t, pageq);
                        if (m->flags & PG_INACTIVE) {
                                queue_remove(&vm_page_queue_inactive, m,
                                                vm_page_t, pageq);
@@ -291,6 +300,7 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                                m->flags &= ~PG_ACTIVE;
                                vm_page_active_count--;
                        }
                                m->flags &= ~PG_ACTIVE;
                                vm_page_active_count--;
                        }
+                       vm_set_intr(spl);
                        vm_page_unlock_queues();
 
                        /*
                        vm_page_unlock_queues();
 
                        /*
@@ -302,9 +312,31 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                }
 
                if (((object->pager != NULL) &&
                }
 
                if (((object->pager != NULL) &&
-                               (!change_wiring || wired))
+                   (!change_wiring || wired))
                    || (object == first_object)) {
 
                    || (object == first_object)) {
 
+#if 0
+                       if (curproc && (curproc->p_rlimit[RLIMIT_RSS].rlim_max <
+                           curproc->p_vmspace->vm_pmap.pm_stats.resident_count * NBPG)) {
+                               UNLOCK_AND_DEALLOCATE;
+                               wakeup(&vm_pages_needed);
+                               vm_pageout_proc_limit = 1;
+                               tsleep(&vm_pageout_proc_limit, PVM, "vmlimt", 0);
+                               goto RetryFault;
+                       }
+#endif
+                               
+                       if (swap_pager_full && !object->shadow && (!object->pager || 
+                               (object->pager && object->pager->pg_type == PG_SWAP &&
+                               !vm_pager_has_page(object->pager, offset+object->paging_offset)))) {
+                               if (vaddr < VM_MAXUSER_ADDRESS && curproc && curproc->p_pid >= 48) /* XXX */ {
+                                       UNLOCK_AND_DEALLOCATE;
+                                       printf("Process %d killed by vm_fault -- out of swap\n", curproc->p_pid);
+                                       psignal(curproc, SIGKILL);
+                                       return KERN_RESOURCE_SHORTAGE;
+                               }
+                       }
+
                        /*
                         *      Allocate a new page for this object/offset
                         *      pair.
                        /*
                         *      Allocate a new page for this object/offset
                         *      pair.
@@ -319,15 +351,29 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                        }
                }
 
                        }
                }
 
-               if ((object->pager != NULL) &&
-                               (!change_wiring || wired)) {
+               if ((object->pager != NULL) && (!change_wiring || wired)) {
                        int rv;
                        int rv;
+                       int faultcount;
+                       int reqpage;
 
                        /*
                         *      Now that we have a busy page, we can
                         *      release the object lock.
                         */
                        vm_object_unlock(object);
 
                        /*
                         *      Now that we have a busy page, we can
                         *      release the object lock.
                         */
                        vm_object_unlock(object);
+                       /*
+                        * now we find out if any other pages should
+                        * be paged in at this time
+                        * this routine checks to see if the pages surrounding this fault
+                        * reside in the same object as the page for this fault.  If
+                        * they do, then they are faulted in also into the
+                        * object.  The array "marray" returned contains an array of vm_page_t structs
+                        * where one of them is the vm_page_t passed to the routine.  The reqpage
+                        * return value is the index into the marray for the vm_page_t passed to the
+                        * routine.
+                        */
+                       faultcount = vm_fault_additional_pages(first_object, first_offset, m,
+                               VM_FAULT_READ_BEHIND, VM_FAULT_READ_AHEAD, marray, &reqpage);
 
                        /*
                         *      Call the pager to retrieve the data, if any,
 
                        /*
                         *      Call the pager to retrieve the data, if any,
@@ -335,7 +381,13 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                         */
                        UNLOCK_MAP;
 
                         */
                        UNLOCK_MAP;
 
-                       rv = vm_pager_get(object->pager, m, TRUE);
+                       if (faultcount != 1) {
+                               rv = faultcount ?
+                                   vm_pager_getmulti(object->pager, marray, faultcount, reqpage, TRUE):
+                                   VM_PAGER_FAIL;
+                       } else {
+                               rv = vm_pager_get(object->pager, m, TRUE);
+                       }
                        if (rv == VM_PAGER_OK) {
                                /*
                                 *      Found the page.
                        if (rv == VM_PAGER_OK) {
                                /*
                                 *      Found the page.
@@ -411,6 +463,8 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                         */
                        if (object != first_object) {
                                object->paging_in_progress--;
                         */
                        if (object != first_object) {
                                object->paging_in_progress--;
+                               if (object->paging_in_progress == 0)
+                                       wakeup((caddr_t) object);
                                vm_object_unlock(object);
 
                                object = first_object;
                                vm_object_unlock(object);
 
                                object = first_object;
@@ -422,20 +476,24 @@ vm_fault(map, vaddr, fault_type, change_wiring)
 
                        vm_page_zero_fill(m);
                        vm_stat.zero_fill_count++;
 
                        vm_page_zero_fill(m);
                        vm_stat.zero_fill_count++;
-                       m->flags &= ~(PG_FAKE | PG_ABSENT);
+                       m->flags &= ~(PG_FAKE|PG_ABSENT);
                        break;
                }
                else {
                        vm_object_lock(next_object);
                        break;
                }
                else {
                        vm_object_lock(next_object);
-                       if (object != first_object)
+                       if (object != first_object) {
                                object->paging_in_progress--;
                                object->paging_in_progress--;
+                               if (object->paging_in_progress == 0)
+                                       wakeup((caddr_t) object);
+                       }
                        vm_object_unlock(object);
                        object = next_object;
                        object->paging_in_progress++;
                }
        }
 
                        vm_object_unlock(object);
                        object = next_object;
                        object->paging_in_progress++;
                }
        }
 
-       if ((m->flags & (PG_ABSENT|PG_ACTIVE|PG_INACTIVE)) || !(m->flags & PG_BUSY))
+       if ((m->flags & (PG_ABSENT|PG_ACTIVE|PG_INACTIVE) != 0) ||
+               (m->flags & PG_BUSY) == 0)
                panic("vm_fault: absent or active or inactive or not busy after main loop");
 
        /*
                panic("vm_fault: absent or active or inactive or not busy after main loop");
 
        /*
@@ -483,7 +541,7 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                         */
 
                        vm_page_copy(m, first_m);
                         */
 
                        vm_page_copy(m, first_m);
-                       first_m->flags &= ~(PG_FAKE | PG_ABSENT);
+                       first_m->flags &= ~(PG_FAKE|PG_ABSENT);
 
                        /*
                         *      If another map is truly sharing this
 
                        /*
                         *      If another map is truly sharing this
@@ -499,6 +557,12 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                         */
 
                        vm_page_lock_queues();
                         */
 
                        vm_page_lock_queues();
+
+                       if ((m->flags & PG_CLEAN) && pmap_is_modified(VM_PAGE_TO_PHYS(m)))
+                               m->flags &= ~PG_CLEAN;
+
+                       if ((m->flags & PG_CLEAN) == 0)
+                               m->flags |= PG_LAUNDRY;
                        vm_page_activate(m);
                        pmap_page_protect(VM_PAGE_TO_PHYS(m), VM_PROT_NONE);
                        vm_page_unlock_queues();
                        vm_page_activate(m);
                        pmap_page_protect(VM_PAGE_TO_PHYS(m), VM_PROT_NONE);
                        vm_page_unlock_queues();
@@ -508,6 +572,8 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                         */
                        PAGE_WAKEUP(m);
                        object->paging_in_progress--;
                         */
                        PAGE_WAKEUP(m);
                        object->paging_in_progress--;
+                       if (object->paging_in_progress == 0)
+                               wakeup((caddr_t) object);
                        vm_object_unlock(object);
 
                        /*
                        vm_object_unlock(object);
 
                        /*
@@ -529,6 +595,8 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                         *      paging_in_progress to do that...
                         */
                        object->paging_in_progress--;
                         *      paging_in_progress to do that...
                         */
                        object->paging_in_progress--;
+                       if (object->paging_in_progress == 0)
+                               wakeup((caddr_t) object);
                        vm_object_collapse(object);
                        object->paging_in_progress++;
                }
                        vm_object_collapse(object);
                        object->paging_in_progress++;
                }
@@ -584,40 +652,18 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                        copy_m = vm_page_lookup(copy_object, copy_offset);
                        if (page_exists = (copy_m != NULL)) {
                                if (copy_m->flags & PG_BUSY) {
                        copy_m = vm_page_lookup(copy_object, copy_offset);
                        if (page_exists = (copy_m != NULL)) {
                                if (copy_m->flags & PG_BUSY) {
-#ifdef DOTHREADS
-                                       int     wait_result;
-
                                        /*
                                         *      If the page is being brought
                                         *      in, wait for it and then retry.
                                         */
                                        /*
                                         *      If the page is being brought
                                         *      in, wait for it and then retry.
                                         */
-                                       PAGE_ASSERT_WAIT(copy_m, !change_wiring);
+                                       PAGE_ASSERT_WAIT(copy_m, !change_wiring); 
                                        RELEASE_PAGE(m);
                                        copy_object->ref_count--;
                                        vm_object_unlock(copy_object);
                                        UNLOCK_THINGS;
                                        RELEASE_PAGE(m);
                                        copy_object->ref_count--;
                                        vm_object_unlock(copy_object);
                                        UNLOCK_THINGS;
-                                       thread_block("pagein");
-                                       wait_result = current_thread()->wait_result;
+                                       thread_block("fltcpy");
                                        vm_object_deallocate(first_object);
                                        vm_object_deallocate(first_object);
-                                       if (wait_result != THREAD_AWAKENED)
-                                               return(KERN_SUCCESS);
                                        goto RetryFault;
                                        goto RetryFault;
-#else
-                                       /*
-                                        *      If the page is being brought
-                                        *      in, wait for it and then retry.
-                                        */
-                                       PAGE_ASSERT_WAIT(copy_m, !change_wiring);
-                                       RELEASE_PAGE(m);
-                                       copy_object->ref_count--;
-                                       vm_object_unlock(copy_object);
-                                       UNLOCK_THINGS;
-                                       thread_wakeup((int)&vm_pages_needed);
-                               /* XXX  ^^^^^*/
-                                       thread_block("pagein");
-                                       vm_object_deallocate(first_object);
-                                       goto RetryFault;
-#endif
                                }
                        }
 
                                }
                        }
 
@@ -639,8 +685,7 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                                 *      found that the copy_object's pager
                                 *      doesn't have the page...
                                 */
                                 *      found that the copy_object's pager
                                 *      doesn't have the page...
                                 */
-                               copy_m = vm_page_alloc(copy_object,
-                                                               copy_offset);
+                               copy_m = vm_page_alloc(copy_object, copy_offset);
                                if (copy_m == NULL) {
                                        /*
                                         *      Wait for a page, then retry.
                                if (copy_m == NULL) {
                                        /*
                                         *      Wait for a page, then retry.
@@ -714,10 +759,20 @@ vm_fault(map, vaddr, fault_type, change_wiring)
                                 *    pmaps use it.)
                                 */
                                vm_page_lock_queues();
                                 *    pmaps use it.)
                                 */
                                vm_page_lock_queues();
+
+                               vm_page_activate(old_m);
+
+                               if ((old_m->flags & PG_CLEAN) &&
+                                       pmap_is_modified(VM_PAGE_TO_PHYS(old_m)))
+                                       old_m->flags &= ~PG_CLEAN;
+
+                               if ((old_m->flags & PG_CLEAN) == 0)
+                                       old_m->flags |= PG_LAUNDRY;
+
                                pmap_page_protect(VM_PAGE_TO_PHYS(old_m),
                                                  VM_PROT_NONE);
                                copy_m->flags &= ~PG_CLEAN;
                                pmap_page_protect(VM_PAGE_TO_PHYS(old_m),
                                                  VM_PROT_NONE);
                                copy_m->flags &= ~PG_CLEAN;
-                               vm_page_activate(copy_m);       /* XXX */
+                               vm_page_activate(copy_m);
                                vm_page_unlock_queues();
 
                                PAGE_WAKEUP(copy_m);
                                vm_page_unlock_queues();
 
                                PAGE_WAKEUP(copy_m);
@@ -811,8 +866,9 @@ vm_fault(map, vaddr, fault_type, change_wiring)
 
        /* XXX This distorts the meaning of the copy_on_write bit */
 
 
        /* XXX This distorts the meaning of the copy_on_write bit */
 
-       if (prot & VM_PROT_WRITE)
+       if (prot & VM_PROT_WRITE) {
                m->flags &= ~PG_COPY_ON_WRITE;
                m->flags &= ~PG_COPY_ON_WRITE;
+       }
 
        /*
         *      It's critically important that a wired-down page be faulted
 
        /*
         *      It's critically important that a wired-down page be faulted
@@ -831,6 +887,7 @@ vm_fault(map, vaddr, fault_type, change_wiring)
         *      page back on the active queue until later so
         *      that the page-out daemon won't find us (yet).
         */
         *      page back on the active queue until later so
         *      that the page-out daemon won't find us (yet).
         */
+
        pmap_enter(map->pmap, vaddr, VM_PAGE_TO_PHYS(m), prot, wired);
 
        /*
        pmap_enter(map->pmap, vaddr, VM_PAGE_TO_PHYS(m), prot, wired);
 
        /*
@@ -865,7 +922,8 @@ vm_fault(map, vaddr, fault_type, change_wiring)
  *
  *     Wire down a range of virtual addresses in a map.
  */
  *
  *     Wire down a range of virtual addresses in a map.
  */
-void vm_fault_wire(map, start, end)
+void
+vm_fault_wire(map, start, end)
        vm_map_t        map;
        vm_offset_t     start, end;
 {
        vm_map_t        map;
        vm_offset_t     start, end;
 {
@@ -899,7 +957,8 @@ void vm_fault_wire(map, start, end)
  *
  *     Unwire a range of virtual addresses in a map.
  */
  *
  *     Unwire a range of virtual addresses in a map.
  */
-void vm_fault_unwire(map, start, end)
+void
+vm_fault_unwire(map, start, end)
        vm_map_t        map;
        vm_offset_t     start, end;
 {
        vm_map_t        map;
        vm_offset_t     start, end;
 {
@@ -948,13 +1007,13 @@ void vm_fault_unwire(map, start, end)
  *             entry corresponding to a main map entry that is wired down).
  */
 
  *             entry corresponding to a main map entry that is wired down).
  */
 
-void vm_fault_copy_entry(dst_map, src_map, dst_entry, src_entry)
+void
+vm_fault_copy_entry(dst_map, src_map, dst_entry, src_entry)
        vm_map_t        dst_map;
        vm_map_t        src_map;
        vm_map_entry_t  dst_entry;
        vm_map_entry_t  src_entry;
 {
        vm_map_t        dst_map;
        vm_map_t        src_map;
        vm_map_entry_t  dst_entry;
        vm_map_entry_t  src_entry;
 {
-
        vm_object_t     dst_object;
        vm_object_t     src_object;
        vm_offset_t     dst_offset;
        vm_object_t     dst_object;
        vm_object_t     src_object;
        vm_offset_t     dst_offset;
@@ -1037,5 +1096,186 @@ void vm_fault_copy_entry(dst_map, src_map, dst_entry, src_entry)
                PAGE_WAKEUP(dst_m);
                vm_object_unlock(dst_object);
        }
                PAGE_WAKEUP(dst_m);
                vm_object_unlock(dst_object);
        }
+}
+
 
 
+/*
+ * looks page up in shadow chain
+ */
+int
+vm_fault_page_lookup(object, offset, rtobject, rtoffset, rtm)
+       vm_object_t object;
+       vm_offset_t offset;
+       vm_object_t *rtobject;
+       vm_offset_t *rtoffset;
+       vm_page_t *rtm;
+{
+       vm_page_t m;
+       vm_object_t first_object = object;
+
+       *rtm = 0;
+       *rtobject = 0;
+       *rtoffset = 0;
+
+       
+       while (!(m=vm_page_lookup(object, offset))) {
+               if (object->pager) {
+                       if (vm_pager_has_page(object->pager, object->paging_offset+offset)) {
+                               *rtobject = object;
+                               *rtoffset = offset;
+                               return 1;
+                       }
+               }
+                       
+               if (!object->shadow)
+                       return 0;
+               else {
+                       offset += object->shadow_offset;
+                       object = object->shadow;
+               }
+       }
+       *rtobject = object;
+       *rtoffset = offset;
+       *rtm = m;
+       return 1;
 }
 }
+
+/*
+ * This routine checks around the requested page for other pages that
+ * might be able to be faulted in.
+ *
+ * Inputs:
+ *     first_object, first_offset, m, rbehind, rahead
+ *
+ * Outputs:
+ *  marray (array of vm_page_t), reqpage (index of requested page)
+ *
+ * Return value:
+ *  number of pages in marray
+ */
+int
+vm_fault_additional_pages(first_object, first_offset, m, rbehind, raheada, marray, reqpage)
+       vm_object_t first_object;
+       vm_offset_t first_offset;
+       vm_page_t m;
+       int rbehind;
+       int raheada;
+       vm_page_t *marray;
+       int *reqpage;
+{
+       int i;
+       vm_page_t tmpm;
+       vm_object_t object;
+       vm_offset_t offset, startoffset, endoffset, toffset, size;
+       vm_object_t rtobject;
+       vm_page_t rtm;
+       vm_offset_t rtoffset;
+       vm_offset_t offsetdiff;
+       int rahead;
+       int treqpage;
+
+       object = m->object;
+       offset = m->offset;
+
+       offsetdiff = offset - first_offset;
+
+       /*
+        * if the requested page is not available, then give up now
+        */
+
+       if (!vm_pager_has_page(object->pager, object->paging_offset+offset))
+               return 0;
+
+       if (!object->pager->pg_ops->pgo_getmulti) {
+               *reqpage = 0;
+               marray[0] = m;
+               return 1;
+       }
+
+       rahead = raheada;
+       if (rahead > (vm_page_free_count - vm_page_free_reserved)) {
+               rahead = vm_page_free_count - vm_page_free_reserved;
+               rbehind = 0;
+       }
+
+       if (rahead <= 0) {
+               *reqpage = 0;
+               marray[0] = m;
+               return 1;
+       }
+
+       /*
+        * scan backward for the read behind pages --
+        * in memory or on disk not in same object
+        */
+       toffset = offset - NBPG;
+       startoffset = offset - rbehind*NBPG;
+       while (((int)(toffset+NBPG)) >= 0 && toffset >= startoffset) {
+               if (!vm_fault_page_lookup(first_object, toffset - offsetdiff, &rtobject, &rtoffset, &rtm) ||
+                   rtm != 0 || rtobject != object) {
+                       startoffset = toffset + NBPG;
+                       break;
+               }
+               toffset -= NBPG;
+       }
+
+       /*
+        * scan forward for the read ahead pages --
+        * in memory or on disk not in same object
+        */
+       toffset = offset + NBPG;
+       endoffset = offset + (rahead+1)*NBPG;
+       while (toffset < object->size && toffset < endoffset) {
+               if (!vm_fault_page_lookup(first_object, toffset - offsetdiff, &rtobject, &rtoffset, &rtm) ||
+                   rtm != 0 || rtobject != object) {
+                       endoffset = toffset;
+                       break;
+               }
+               toffset += NBPG;
+       }
+
+       /* calculate number of bytes of pages */
+       size = (endoffset - startoffset) / NBPG;
+
+       /* calculate the page offset of the required page */
+       treqpage = (offset - startoffset) / NBPG;
+               
+       /* see if we have space */
+       if (vm_page_free_count >= vm_page_free_reserved + size) {
+               bzero(marray, (rahead + rbehind + 1) * sizeof(vm_page_t));
+               /*
+                * get our pages and block for them
+                */
+               for (i = 0; i < size; i++) {
+                       if (i != treqpage)
+                               rtm  = vm_page_alloc(object, startoffset + i * NBPG);
+                       else
+                               rtm = m;
+                       marray[i] = rtm;
+               }
+
+               for (i = 0; i < size; i++) {
+                       if (marray[i] == 0)
+                               break;
+               }
+
+               if (i < size) {
+                       for (i = 0; i < size; i++) {
+                               if (i != treqpage && marray[i])
+                                       FREE_PAGE(marray[i]);
+                       }
+                       *reqpage = 0;
+                       marray[0] = m;
+                       return 1;
+               } 
+
+               *reqpage = treqpage;
+               return size;
+       }
+       wakeup((caddr_t) &vm_pages_needed);
+       *reqpage = 0;
+       marray[0] = m;
+       return 1;
+}
+
index d603505..476c252 100644 (file)
@@ -1,6 +1,8 @@
 /* 
  * Copyright (c) 1991 Regents of the University of California.
  * All rights reserved.
 /* 
  * Copyright (c) 1991 Regents of the University of California.
  * All rights reserved.
+ * Copyright (c) John S. Dyson
+ * All rights reserved.
  *
  * This code is derived from software contributed to Berkeley by
  * The Mach Operating System project at Carnegie-Mellon University.
  *
  * This code is derived from software contributed to Berkeley by
  * The Mach Operating System project at Carnegie-Mellon University.
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
- *     from: @(#)vm_glue.c     7.8 (Berkeley) 5/15/91
- *     $Id: vm_glue.c,v 1.13 1993/12/12 12:27:23 davidg Exp $
- */
-
-/*
+ *     @(#)vm_glue.c   7.8 (Berkeley) 5/15/91
+ *
+ *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  * 
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  * 
 #include "resourcevar.h"
 #include "buf.h"
 #include "user.h"
 #include "resourcevar.h"
 #include "buf.h"
 #include "user.h"
+#include "kernel.h"
 
 #include "vm.h"
 #include "vm_page.h"
 #include "vm_kern.h"
 #include "machine/stdarg.h"
 
 
 #include "vm.h"
 #include "vm_page.h"
 #include "vm_kern.h"
 #include "machine/stdarg.h"
 
-static void swapout(struct proc *);
-
+extern char kstack[];
 int    avefree = 0;            /* XXX */
 int    readbuffers = 0;        /* XXX allow kgdb to read kernel buffer pool */
 int    avefree = 0;            /* XXX */
 int    readbuffers = 0;        /* XXX allow kgdb to read kernel buffer pool */
+/* vm_map_t upages_map; */
 
 
+void swapout(struct proc *p);
 int
 kernacc(addr, len, rw)
        caddr_t addr;
 int
 kernacc(addr, len, rw)
        caddr_t addr;
@@ -101,9 +103,6 @@ kernacc(addr, len, rw)
         * or worse, inconsistencies at the pmap level.  We only worry
         * about the buffer cache for now.
         */
         * or worse, inconsistencies at the pmap level.  We only worry
         * about the buffer cache for now.
         */
-       if (!readbuffers && rv && (eaddr > (vm_offset_t)buffers &&
-                  saddr < (vm_offset_t)buffers + MAXBSIZE * nbuf))
-               rv = FALSE;
        return(rv == TRUE);
 }
 
        return(rv == TRUE);
 }
 
@@ -126,10 +125,12 @@ useracc(addr, len, rw)
         * only used (as an end address) in trap.c.  Use it as an end
         * address here too.
         */
         * only used (as an end address) in trap.c.  Use it as an end
         * address here too.
         */
-       if ((vm_offset_t) addr >= VM_MAXUSER_ADDRESS
+       if ((vm_offset_t) addr >= VM_MAXUSER_ADDRESS 
            || (vm_offset_t) addr + len > VM_MAXUSER_ADDRESS
            || (vm_offset_t) addr + len > VM_MAXUSER_ADDRESS
-           || (vm_offset_t) addr + len <= (vm_offset_t) addr)
+           || (vm_offset_t) addr + len <= (vm_offset_t) addr) {
+               printf("address wrap\n");
                return (FALSE);
                return (FALSE);
+       }
 
        rv = vm_map_check_protection(&curproc->p_vmspace->vm_map,
            trunc_page(addr), round_page(addr+len), prot);
 
        rv = vm_map_check_protection(&curproc->p_vmspace->vm_map,
            trunc_page(addr), round_page(addr+len), prot);
@@ -152,13 +153,13 @@ chgkprot(addr, len, rw)
                       round_page(addr+len), prot, FALSE);
 }
 #endif
                       round_page(addr+len), prot, FALSE);
 }
 #endif
-
 void
 vslock(addr, len)
        caddr_t addr;
        u_int   len;
 {
        vm_map_pageable(&curproc->p_vmspace->vm_map, trunc_page(addr),
 void
 vslock(addr, len)
        caddr_t addr;
        u_int   len;
 {
        vm_map_pageable(&curproc->p_vmspace->vm_map, trunc_page(addr),
+                       /* round_page(addr+len-1), FALSE); */
                        round_page(addr+len), FALSE);
 }
 
                        round_page(addr+len), FALSE);
 }
 
@@ -171,7 +172,7 @@ vsunlock(addr, len, dirtied)
 #ifdef lint
        dirtied++;
 #endif lint
 #ifdef lint
        dirtied++;
 #endif lint
-       vm_map_pageable(&curproc->p_vmspace->vm_map, trunc_page(addr),
+               vm_map_pageable(&curproc->p_vmspace->vm_map, trunc_page(addr),
                        round_page(addr+len), TRUE);
 }
 
                        round_page(addr+len), TRUE);
 }
 
@@ -192,16 +193,16 @@ vm_fork(p1, p2, isvfork)
        int isvfork;
 {
        register struct user *up;
        int isvfork;
 {
        register struct user *up;
-       vm_offset_t addr;
+       vm_offset_t addr, ptaddr;
+       int i;
+       struct vm_map *vp;
 
 
-#ifdef i386
        /*
         * avoid copying any of the parent's pagetables or other per-process
         * objects that reside in the map by marking all of them non-inheritable
         */
        (void)vm_map_inherit(&p1->p_vmspace->vm_map,
        /*
         * avoid copying any of the parent's pagetables or other per-process
         * objects that reside in the map by marking all of them non-inheritable
         */
        (void)vm_map_inherit(&p1->p_vmspace->vm_map,
-               UPT_MIN_ADDRESS-UPAGES*NBPG, VM_MAX_ADDRESS, VM_INHERIT_NONE);
-#endif
+               UPT_MIN_ADDRESS - UPAGES * NBPG, VM_MAX_ADDRESS, VM_INHERIT_NONE);
        p2->p_vmspace = vmspace_fork(p1->p_vmspace);
 
 #ifdef SYSVSHM
        p2->p_vmspace = vmspace_fork(p1->p_vmspace);
 
 #ifdef SYSVSHM
@@ -212,15 +213,38 @@ vm_fork(p1, p2, isvfork)
        /*
         * Allocate a wired-down (for now) pcb and kernel stack for the process
         */
        /*
         * Allocate a wired-down (for now) pcb and kernel stack for the process
         */
-#ifdef notyet
-       addr = kmem_alloc_pageable(kernel_map, ctob(UPAGES));
-       vm_map_pageable(kernel_map, addr, addr + ctob(UPAGES), FALSE);
-#else
-       addr = kmem_alloc(kernel_map, ctob(UPAGES));
-       if (!addr)
-               panic("vm_fork: failed to allocate UPAGES");
-#endif
-       up = (struct user *)addr;
+
+       /* addr = UPT_MIN_ADDRESS - UPAGES*NBPG; */
+       addr = (vm_offset_t) kstack;
+
+       vp = &p2->p_vmspace->vm_map;
+
+       /* ream out old pagetables and kernel stack */
+       (void)vm_deallocate(vp, addr, UPT_MAX_ADDRESS - addr);
+
+       /* get new pagetables and kernel stack */
+       (void)vm_allocate(vp, &addr, UPT_MAX_ADDRESS - addr, FALSE);
+
+       /* force in the page table encompassing the UPAGES */
+       ptaddr = trunc_page((u_int)vtopte(addr));
+       vm_map_pageable(vp, ptaddr, ptaddr + NBPG, FALSE);
+
+       /* and force in (demand-zero) the UPAGES */
+       vm_map_pageable(vp, addr, addr + UPAGES * NBPG, FALSE);
+
+       /* get a kernel virtual address for the UPAGES for this proc */
+       up = (struct user *)kmem_alloc_pageable(kernel_map, UPAGES * NBPG);
+
+       /* and force-map the upages into the kernel pmap */
+       for (i = 0; i < UPAGES; i++)
+               pmap_enter(vm_map_pmap(kernel_map),
+                       ((vm_offset_t) up) + NBPG * i,
+                       pmap_extract(vp->pmap, addr + NBPG * i),
+                       VM_PROT_READ|VM_PROT_WRITE, 1);
+
+       /* and allow the UPAGES page table entry to be paged (at the vm system level) */
+       vm_map_pageable(vp, ptaddr, ptaddr + NBPG, TRUE);
+
        p2->p_addr = up;
 
        /*
        p2->p_addr = up;
 
        /*
@@ -239,16 +263,7 @@ vm_fork(p1, p2, isvfork)
            ((caddr_t)&up->u_stats.pstat_endcopy -
             (caddr_t)&up->u_stats.pstat_startcopy));
 
            ((caddr_t)&up->u_stats.pstat_endcopy -
             (caddr_t)&up->u_stats.pstat_startcopy));
 
-#ifdef i386
-       { u_int addr = UPT_MIN_ADDRESS - UPAGES*NBPG; struct vm_map *vp;
-
-       vp = &p2->p_vmspace->vm_map;
-
-       /* ream out old pagetables and kernel stack */
-       (void)vm_deallocate(vp, addr, UPT_MAX_ADDRESS - addr);
-       (void)vm_allocate(vp, &addr, UPT_MAX_ADDRESS - addr, FALSE);
-       }
-#endif
+       
        /*
         * cpu_fork will copy and update the kernel stack and pcb,
         * and make the child ready to run.  It marks the child
        /*
         * cpu_fork will copy and update the kernel stack and pcb,
         * and make the child ready to run.  It marks the child
@@ -267,6 +282,7 @@ void
 vm_init_limits(p)
        register struct proc *p;
 {
 vm_init_limits(p)
        register struct proc *p;
 {
+       int tmp;
 
        /*
         * Set up the initial limits on process VM.
 
        /*
         * Set up the initial limits on process VM.
@@ -279,8 +295,11 @@ vm_init_limits(p)
         p->p_rlimit[RLIMIT_STACK].rlim_max = MAXSSIZ;
         p->p_rlimit[RLIMIT_DATA].rlim_cur = DFLDSIZ;
         p->p_rlimit[RLIMIT_DATA].rlim_max = MAXDSIZ;
         p->p_rlimit[RLIMIT_STACK].rlim_max = MAXSSIZ;
         p->p_rlimit[RLIMIT_DATA].rlim_cur = DFLDSIZ;
         p->p_rlimit[RLIMIT_DATA].rlim_max = MAXDSIZ;
+       tmp = ((2 * vm_page_free_count) / 3) - 32;
+       if (vm_page_free_count < 512)
+               tmp = vm_page_free_count;
        p->p_rlimit[RLIMIT_RSS].rlim_cur = p->p_rlimit[RLIMIT_RSS].rlim_max =
        p->p_rlimit[RLIMIT_RSS].rlim_cur = p->p_rlimit[RLIMIT_RSS].rlim_max =
-               ptoa(vm_page_free_count);
+               ptoa(tmp);
 }
 
 #include "../vm/vm_pageout.h"
 }
 
 #include "../vm/vm_pageout.h"
@@ -293,6 +312,64 @@ int        swapdebug = 0;
 #define SDB_SWAPOUT    4
 #endif
 
 #define SDB_SWAPOUT    4
 #endif
 
+void
+faultin(p)
+struct proc *p;
+{
+       vm_offset_t i;
+       vm_offset_t vaddr, ptaddr;
+       vm_offset_t v, v1;
+       struct user *up;
+       int s;
+       int opflag;
+
+       if ((p->p_flag & SLOAD) == 0) {
+               int rv0, rv1;
+               vm_map_t map;
+
+               opflag = p->p_flag;
+               p->p_flag |= SLOCK;
+
+               map = &p->p_vmspace->vm_map;
+               /* force the page table encompassing the kernel stack (upages) */
+               ptaddr = trunc_page((u_int)vtopte(kstack));
+               vm_map_pageable(map, ptaddr, ptaddr + NBPG, FALSE);
+
+               /* wire in the UPAGES */
+               vm_map_pageable(map, (vm_offset_t) kstack,
+                       (vm_offset_t) kstack + UPAGES * NBPG, FALSE);
+
+               /* and map them nicely into the kernel pmap */
+               for (i = 0; i < UPAGES; i++) {
+                       vm_offset_t off = i * NBPG;
+                       vm_offset_t pa = (vm_offset_t)
+                               pmap_extract(&p->p_vmspace->vm_pmap, 
+                               (vm_offset_t) kstack + off);
+                       pmap_enter(vm_map_pmap(kernel_map),
+                               ((vm_offset_t)p->p_addr) + off,
+                                       pa, VM_PROT_READ|VM_PROT_WRITE, 1);
+               }
+
+               /* and let the page table pages go (at least above pmap level) */
+               vm_map_pageable(map, ptaddr, ptaddr + NBPG, TRUE);
+
+               s = splhigh();
+
+               if (p->p_stat == SRUN)
+                       setrq(p);
+
+               p->p_flag |= SLOAD; 
+
+               /* undo the effect of setting SLOCK above */
+               p->p_flag &= ~SLOCK;
+               p->p_flag |= opflag & SLOCK;
+               splx(s);
+
+       }
+
+}
+       
+int swapinreq;
 /*
  * Brutally simple:
  *     1. Attempt to swapin every swaped-out, runnable process in
 /*
  * Brutally simple:
  *     1. Attempt to swapin every swaped-out, runnable process in
@@ -300,7 +377,7 @@ int swapdebug = 0;
  *     2. If not enough memory, wake the pageout daemon and let it
  *        clear some space.
  */
  *     2. If not enough memory, wake the pageout daemon and let it
  *        clear some space.
  */
-void                           /* XXX should be __dead, too */
+void
 sched()
 {
        register struct proc *p;
 sched()
 {
        register struct proc *p;
@@ -308,9 +385,10 @@ sched()
        struct proc *pp;
        int ppri;
        vm_offset_t addr;
        struct proc *pp;
        int ppri;
        vm_offset_t addr;
-       vm_size_t size;
 
 
+       /* printf("vm_page_free_count: %d\n", vm_page_free_count); */
 loop:
 loop:
+       vmmeter();
 #ifdef DEBUG
        if (!enableswap) {
                pp = NULL;
 #ifdef DEBUG
        if (!enableswap) {
                pp = NULL;
@@ -336,8 +414,8 @@ noswap:
         * Nothing to do, back to sleep
         */
        if ((p = pp) == NULL) {
         * Nothing to do, back to sleep
         */
        if ((p = pp) == NULL) {
-         tsleep((caddr_t)&proc0, PVM, "sched", 0);
-         goto loop;
+               sleep((caddr_t)&proc0, PVM);
+               goto loop;
        }
 
        /*
        }
 
        /*
@@ -345,24 +423,16 @@ noswap:
         * This part is really bogus cuz we could deadlock on memory
         * despite our feeble check.
         */
         * This part is really bogus cuz we could deadlock on memory
         * despite our feeble check.
         */
-       size = round_page(ctob(UPAGES));
-       addr = (vm_offset_t) p->p_addr;
-       if (vm_page_free_count > atop(size)) {
-#ifdef DEBUG
-               if (swapdebug & SDB_SWAPIN)
-                       printf("swapin: pid %d(%s)@%x, pri %d free %d\n",
-                              p->p_pid, p->p_comm, p->p_addr,
-                              ppri, vm_page_free_count);
-#endif
-               vm_map_pageable(kernel_map, addr, addr+size, FALSE);
-               (void) splclock();
-               if (p->p_stat == SRUN)
-                       setrq(p);
-               p->p_flag |= SLOAD;
-               (void) spl0();
+       (void) splhigh();
+       if (((vm_page_free_count + vm_page_inactive_count) >=
+           (vm_page_inactive_target + vm_page_free_reserved)) ||
+           (vm_page_free_count >= vm_page_free_min)) {
+               spl0();
+               faultin(p);
                p->p_time = 0;
                goto loop;
                p->p_time = 0;
                goto loop;
-       }
+       } 
+       ++swapinreq; 
        /*
         * Not enough memory, jab the pageout daemon and wait til the
         * coast is clear.
        /*
         * Not enough memory, jab the pageout daemon and wait til the
         * coast is clear.
@@ -372,7 +442,6 @@ noswap:
                printf("sched: no room for pid %d(%s), free %d\n",
                       p->p_pid, p->p_comm, vm_page_free_count);
 #endif
                printf("sched: no room for pid %d(%s), free %d\n",
                       p->p_pid, p->p_comm, vm_page_free_count);
 #endif
-       (void) splhigh();
        VM_WAIT;
        (void) spl0();
 #ifdef DEBUG
        VM_WAIT;
        (void) spl0();
 #ifdef DEBUG
@@ -383,8 +452,9 @@ noswap:
 }
 
 #define        swappable(p) \
 }
 
 #define        swappable(p) \
-       (((p)->p_flag & (SSYS|SLOAD|SKEEP|SWEXIT|SPHYSIO)) == SLOAD)
+       (((p)->p_flag & (STRC|SSYS|SLOAD|SLOCK|SKEEP|SWEXIT|SPHYSIO)) == SLOAD)
 
 
+extern int vm_pageout_free_min;
 /*
  * Swapout is driven by the pageout daemon.  Very simple, we find eligible
  * procs and unwire their u-areas.  We try to always "swap" at least one
 /*
  * Swapout is driven by the pageout daemon.  Very simple, we find eligible
  * procs and unwire their u-areas.  We try to always "swap" at least one
@@ -399,34 +469,41 @@ swapout_threads()
        register struct proc *p;
        struct proc *outp, *outp2;
        int outpri, outpri2;
        register struct proc *p;
        struct proc *outp, *outp2;
        int outpri, outpri2;
+       int tpri;
        int didswap = 0;
        extern int maxslp;
        int didswap = 0;
        extern int maxslp;
+       int s;
 
 #ifdef DEBUG
        if (!enableswap)
                return;
 #endif
 
 #ifdef DEBUG
        if (!enableswap)
                return;
 #endif
+               
        outp = outp2 = NULL;
        outp = outp2 = NULL;
-       outpri = outpri2 = 0;
+       outpri = outpri2 = INT_MIN;
        for (p = allproc; p != NULL; p = p->p_nxt) {
                if (!swappable(p))
                        continue;
                switch (p->p_stat) {
                case SRUN:
        for (p = allproc; p != NULL; p = p->p_nxt) {
                if (!swappable(p))
                        continue;
                switch (p->p_stat) {
                case SRUN:
-                       if (p->p_time > outpri2) {
+                       if (p->p_pri <= PUSER) /* possible deadlock unless this check */
+                               continue;
+                       if ((tpri = p->p_time + p->p_nice * 8) > outpri2) {
                                outp2 = p;
                                outp2 = p;
-                               outpri2 = p->p_time;
+                               outpri2 = tpri;
                        }
                        continue;
                        
                case SSLEEP:
                case SSTOP:
                        }
                        continue;
                        
                case SSLEEP:
                case SSTOP:
-                       if (p->p_slptime > maxslp) {
+                       if (p->p_pri <= PRIBIO) /* possible deadlock unless this check */
+                               continue;
+                       if (p->p_slptime > maxslp || p->p_pri == PWAIT) {
                                swapout(p);
                                didswap++;
                                swapout(p);
                                didswap++;
-                       } else if (p->p_slptime > outpri) {
+                       } else if ((tpri = p->p_slptime + p->p_nice * 8) > outpri) {
                                outp = p;
                                outp = p;
-                               outpri = p->p_slptime;
+                               outpri = tpri ;
                        }
                        continue;
                }
                        }
                        continue;
                }
@@ -437,25 +514,36 @@ swapout_threads()
         * if we are real low on memory since we don't gain much by doing
         * it (UPAGES pages).
         */
         * if we are real low on memory since we don't gain much by doing
         * it (UPAGES pages).
         */
-       if (didswap == 0 &&
-           vm_page_free_count <= atop(round_page(ctob(UPAGES)))) {
-               if ((p = outp) == 0)
+       if (didswap == 0 && (swapinreq && 
+                       vm_page_free_count < vm_page_free_min)) {
+               if ((p = outp) == 0 &&
+                       (vm_page_free_count < vm_page_free_reserved ))
                        p = outp2;
 #ifdef DEBUG
                if (swapdebug & SDB_SWAPOUT)
                        printf("swapout_threads: no duds, try procp %x\n", p);
 #endif
                        p = outp2;
 #ifdef DEBUG
                if (swapdebug & SDB_SWAPOUT)
                        printf("swapout_threads: no duds, try procp %x\n", p);
 #endif
-               if (p)
+               if (p) {
                        swapout(p);
                        swapout(p);
+                       didswap = 1;
+               }
+       }
+       if (didswap) {
+               if (swapinreq)
+                       wakeup((caddr_t)&proc0);
+               swapinreq = 0;
        }
 }
 
        }
 }
 
-static void
+void
 swapout(p)
        register struct proc *p;
 {
        vm_offset_t addr;
 swapout(p)
        register struct proc *p;
 {
        vm_offset_t addr;
-       vm_size_t size;
+       struct pmap *pmap = &p->p_vmspace->vm_pmap;
+       vm_map_t map = &p->p_vmspace->vm_map;
+       vm_offset_t ptaddr;
+       int i;
 
 #ifdef DEBUG
        if (swapdebug & SDB_SWAPOUT)
 
 #ifdef DEBUG
        if (swapdebug & SDB_SWAPOUT)
@@ -463,38 +551,23 @@ swapout(p)
                       p->p_pid, p->p_comm, p->p_addr, p->p_stat,
                       p->p_slptime, vm_page_free_count);
 #endif
                       p->p_pid, p->p_comm, p->p_addr, p->p_stat,
                       p->p_slptime, vm_page_free_count);
 #endif
-       size = round_page(ctob(UPAGES));
-       addr = (vm_offset_t) p->p_addr;
-       p->p_stats->p_ru.ru_nswap++ ;           /* record in resource stats */
-#ifdef notyet
-#ifdef hp300
-       /*
-        * Ugh!  u-area is double mapped to a fixed address behind the
-        * back of the VM system and accesses are usually through that
-        * address rather than the per-process address.  Hence reference
-        * and modify information are recorded at the fixed address and
-        * lost at context switch time.  We assume the u-struct and
-        * kernel stack are always accessed/modified and force it to be so.
-        */
-       {
-               register int i;
-               volatile long tmp;
 
 
-               for (i = 0; i < UPAGES; i++) {
-                       tmp = *(long *)addr; *(long *)addr = tmp;
-                       addr += NBPG;
-               }
-               addr = (vm_offset_t) p->p_addr;
-       }
-#endif
-       vm_map_pageable(kernel_map, addr, addr+size, TRUE);
-       pmap_collect(vm_map_pmap(&p->p_vmspace->vm_map));
-#endif
+
        (void) splhigh();
        p->p_flag &= ~SLOAD;
        if (p->p_stat == SRUN)
                remrq(p);
        (void) spl0();
        (void) splhigh();
        p->p_flag &= ~SLOAD;
        if (p->p_stat == SRUN)
                remrq(p);
        (void) spl0();
+
+       p->p_flag |= SLOCK;
+/* let the upages be paged */
+       pmap_remove(vm_map_pmap(kernel_map),
+               (vm_offset_t) p->p_addr, ((vm_offset_t) p->p_addr) + UPAGES * NBPG);
+
+       vm_map_pageable(map, (vm_offset_t) kstack,
+               (vm_offset_t) kstack + UPAGES * NBPG, TRUE);
+
+       p->p_flag &= ~SLOCK;
        p->p_time = 0;
 }
 
        p->p_time = 0;
 }
 
@@ -513,33 +586,28 @@ assert_wait(event, ruptible)
 #endif
        curproc->p_thread = event;
 }
 #endif
        curproc->p_thread = event;
 }
-#endif /* assert_wait */
+#endif
 
 void
 
 void
-thread_block(const char *wmesg)
+thread_block(const char *msg)
 {
 {
-       int s = splhigh();
-
-       if (curproc->p_thread) {
-               tsleep((caddr_t)curproc->p_thread, PVM, wmesg, 0);
-       }
-       splx(s);
+       if (curproc->p_thread)
+               tsleep((caddr_t)curproc->p_thread, PVM, msg, 0);
 }
 
 }
 
+
 void
 thread_sleep_(event, lock, wmesg)
        int event;
        simple_lock_t lock;
        const char *wmesg;
 {
 void
 thread_sleep_(event, lock, wmesg)
        int event;
        simple_lock_t lock;
        const char *wmesg;
 {
-       int s = splhigh();
 
        curproc->p_thread = event;
        simple_unlock(lock);
        if (curproc->p_thread) {
                tsleep((caddr_t)event, PVM, wmesg, 0);
        }
 
        curproc->p_thread = event;
        simple_unlock(lock);
        if (curproc->p_thread) {
                tsleep((caddr_t)event, PVM, wmesg, 0);
        }
-       splx(s);
 }
 
 #ifndef thread_wakeup
 }
 
 #ifndef thread_wakeup
@@ -547,16 +615,14 @@ void
 thread_wakeup(event)
        int event;
 {
 thread_wakeup(event)
        int event;
 {
-#if 0
-       int s = splhigh();      /* XXX is this really necessary??? */
-#endif
-
        wakeup((caddr_t)event);
        wakeup((caddr_t)event);
-#if 0
-       splx(s);
-#endif
 }
 }
-#endif /* thread_wakeup */
+#endif
+
+/*
+ * DEBUG stuff
+ */
+
 
 /*
  * DEBUG stuff
 
 /*
  * DEBUG stuff
index f74b970..00bbdb9 100644 (file)
@@ -34,7 +34,7 @@
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_init.c     7.3 (Berkeley) 4/21/91
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_init.c     7.3 (Berkeley) 4/21/91
- *     $Id$
+ *     $Id: vm_init.c,v 1.2 1993/10/16 16:20:28 rgrimes Exp $
  */
 
 /*
  */
 
 /*
@@ -81,7 +81,8 @@
  *     The start and end address of physical memory is passed in.
  */
 
  *     The start and end address of physical memory is passed in.
  */
 
-void vm_mem_init()
+void
+vm_mem_init()
 {
        extern vm_offset_t      avail_start, avail_end;
        extern vm_offset_t      virtual_avail, virtual_end;
 {
        extern vm_offset_t      avail_start, avail_end;
        extern vm_offset_t      virtual_avail, virtual_end;
index 0341a97..89031b8 100644 (file)
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
- *     from: @(#)vm_kern.c     7.4 (Berkeley) 5/7/91
- *     $Id: vm_kern.c,v 1.6 1993/12/19 00:56:01 wollman Exp $
- */
-
-/*
+ *     @(#)vm_kern.c   7.4 (Berkeley) 5/7/91
+ *
+ *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  *
@@ -62,7 +60,6 @@
  *
  * any improvements or extensions that they make and grant Carnegie the
  * rights to redistribute these changes.
  *
  * any improvements or extensions that they make and grant Carnegie the
  * rights to redistribute these changes.
- *
  */
 
 /*
  */
 
 /*
  */
 
 #include "param.h"
  */
 
 #include "param.h"
-#include "systm.h"
-#include "proc.h"              /* XXX - for curproc */
-#include "syslog.h"
 
 #include "vm.h"
 #include "vm_page.h"
 #include "vm_pageout.h"
 #include "vm_kern.h"
 
 #include "vm.h"
 #include "vm_page.h"
 #include "vm_pageout.h"
 #include "vm_kern.h"
+#include "machine/pmap.h"
 
 vm_map_t       kernel_map;
 vm_map_t       mb_map;
 
 vm_map_t       kernel_map;
 vm_map_t       mb_map;
@@ -85,6 +80,7 @@ vm_map_t      kmem_map;
 vm_map_t       phys_map;
 vm_map_t       buffer_map;
 
 vm_map_t       phys_map;
 vm_map_t       buffer_map;
 
+
 /*
  *     kmem_alloc_pageable:
  *
 /*
  *     kmem_alloc_pageable:
  *
@@ -92,7 +88,8 @@ vm_map_t      buffer_map;
  *     map must be "kernel_map" below.
  */
 
  *     map must be "kernel_map" below.
  */
 
-vm_offset_t kmem_alloc_pageable(map, size)
+vm_offset_t
+kmem_alloc_pageable(map, size)
        vm_map_t                map;
        register vm_size_t      size;
 {
        vm_map_t                map;
        register vm_size_t      size;
 {
@@ -120,7 +117,8 @@ vm_offset_t kmem_alloc_pageable(map, size)
  *     Allocate wired-down memory in the kernel's address map
  *     or a submap.
  */
  *     Allocate wired-down memory in the kernel's address map
  *     or a submap.
  */
-vm_offset_t kmem_alloc(map, size)
+vm_offset_t
+kmem_alloc(map, size)
        register vm_map_t       map;
        register vm_size_t      size;
 {
        register vm_map_t       map;
        register vm_size_t      size;
 {
@@ -129,6 +127,7 @@ vm_offset_t kmem_alloc(map, size)
        register vm_offset_t    offset;
        extern vm_object_t      kernel_object;
        vm_offset_t             i;
        register vm_offset_t    offset;
        extern vm_object_t      kernel_object;
        vm_offset_t             i;
+       unsigned v;
 
        size = round_page(size);
 
 
        size = round_page(size);
 
@@ -185,6 +184,7 @@ vm_offset_t kmem_alloc(map, size)
         */
 
        vm_object_lock(kernel_object);
         */
 
        vm_object_lock(kernel_object);
+
        for (i = 0 ; i < size; i+= PAGE_SIZE) {
                vm_page_t       mem;
 
        for (i = 0 ; i < size; i+= PAGE_SIZE) {
                vm_page_t       mem;
 
@@ -204,6 +204,16 @@ vm_offset_t kmem_alloc(map, size)
 
        (void) vm_map_pageable(map, (vm_offset_t) addr, addr + size, FALSE);
 
 
        (void) vm_map_pageable(map, (vm_offset_t) addr, addr + size, FALSE);
 
+       vm_map_lock(map);
+       for (i = 0; i < size; i += PAGE_SIZE) {
+               vm_page_t m;
+               vm_object_lock(kernel_object);
+               m = vm_page_lookup(kernel_object, offset + i);
+               vm_object_unlock(kernel_object);
+               pmap_enter(map->pmap, addr + i, VM_PAGE_TO_PHYS(m),
+                          VM_PROT_DEFAULT, TRUE);
+       }
+       vm_map_unlock(map);
        /*
         *      Try to coalesce the map
         */
        /*
         *      Try to coalesce the map
         */
@@ -220,13 +230,15 @@ vm_offset_t kmem_alloc(map, size)
  *     with kmem_alloc, and return the physical pages
  *     associated with that region.
  */
  *     with kmem_alloc, and return the physical pages
  *     associated with that region.
  */
-void kmem_free(map, addr, size)
+void
+kmem_free(map, addr, size)
        vm_map_t                map;
        register vm_offset_t    addr;
        vm_size_t               size;
 {
        (void) vm_map_remove(map, trunc_page(addr), round_page(addr + size));
        vm_map_simplify(map, addr);
        vm_map_t                map;
        register vm_offset_t    addr;
        vm_size_t               size;
 {
        (void) vm_map_remove(map, trunc_page(addr), round_page(addr + size));
        vm_map_simplify(map, addr);
+       wakeup((caddr_t)map);
 }
 
 /*
 }
 
 /*
@@ -242,7 +254,8 @@ void kmem_free(map, addr, size)
  *     min, max        Returned endpoints of map
  *     pageable        Can the region be paged
  */
  *     min, max        Returned endpoints of map
  *     pageable        Can the region be paged
  */
-vm_map_t kmem_suballoc(parent, min, max, size, pageable)
+vm_map_t
+kmem_suballoc(parent, min, max, size, pageable)
        register vm_map_t       parent;
        vm_offset_t             *min, *max;
        register vm_size_t      size;
        register vm_map_t       parent;
        vm_offset_t             *min, *max;
        register vm_size_t      size;
@@ -267,9 +280,11 @@ vm_map_t kmem_suballoc(parent, min, max, size, pageable)
                panic("kmem_suballoc: cannot create submap");
        if ((ret = vm_map_submap(parent, *min, *max, result)) != KERN_SUCCESS)
                panic("kmem_suballoc: unable to change range to submap");
                panic("kmem_suballoc: cannot create submap");
        if ((ret = vm_map_submap(parent, *min, *max, result)) != KERN_SUCCESS)
                panic("kmem_suballoc: unable to change range to submap");
+
        return(result);
 }
 
        return(result);
 }
 
+#if 0
 /*
  *     vm_move:
  *
 /*
  *     vm_move:
  *
@@ -288,7 +303,8 @@ vm_map_t kmem_suballoc(parent, min, max, size, pageable)
  *
  *     Returns new destination address or 0 (if a failure occurs).
  */
  *
  *     Returns new destination address or 0 (if a failure occurs).
  */
-vm_offset_t vm_move(src_map,src_addr,dst_map,num_bytes,src_dealloc)
+vm_offset_t
+vm_move(src_map, src_addr, dst_map, num_bytes, src_dealloc)
        vm_map_t                src_map;
        register vm_offset_t    src_addr;
        register vm_map_t       dst_map;
        vm_map_t                src_map;
        register vm_offset_t    src_addr;
        register vm_map_t       dst_map;
@@ -345,7 +361,7 @@ vm_offset_t vm_move(src_map,src_addr,dst_map,num_bytes,src_dealloc)
                return(dst_start + (src_addr - src_start));
        return(0);
 }
                return(dst_start + (src_addr - src_start));
        return(0);
 }
-
+#endif
 /*
  * Allocate wired-down memory in the kernel's address map for the higher
  * level kernel memory allocator (kern/kern_malloc.c).  We cannot use
 /*
  * Allocate wired-down memory in the kernel's address map for the higher
  * level kernel memory allocator (kern/kern_malloc.c).  We cannot use
@@ -373,23 +389,26 @@ kmem_malloc(map, size, canwait)
        vm_offset_t             addr;
        vm_page_t               m;
        extern vm_object_t      kmem_object;
        vm_offset_t             addr;
        vm_page_t               m;
        extern vm_object_t      kmem_object;
+       int                     result;
 
 
-       if (map != kmem_map && map != mb_map)
-               panic("kern_malloc_alloc: map != {kmem,mb}_map");
+       if (map != kmem_map && map != mb_map && map != buffer_map)
+               panic("kern_malloc_alloc: map != {kmem,mb,buffer}_map");
 
        size = round_page(size);
        addr = vm_map_min(map);
 
 
        size = round_page(size);
        addr = vm_map_min(map);
 
-       if (vm_map_find(map, NULL, (vm_offset_t)0,
-                       &addr, size, TRUE) != KERN_SUCCESS) {
-               if (!canwait) {
-                       if (map == kmem_map)
-                               panic("kmem_malloc: kmem_map too small");
-                       else if (map == mb_map)
-                               log(LOG_WARNING,
-                                       "kmem_malloc: mb_map too small (can't wait)\n");
+       result = vm_map_find(map, NULL, (vm_offset_t)0, &addr, size, TRUE);
+       if (result != KERN_SUCCESS) {
+               printf("vm_map_find failure: %d\n", result);
+               if (canwait) {
+                       return(0);
+               } else {
+                       if (map != buffer_map) {
+       /*                      panic("kmem_malloc: kmem_map too small"); */
+                       }
+                       return 0;
                }
                }
-               return 0;
+               panic("kmem_malloc: not enough map entries or map too small\n");
        }
 
        /*
        }
 
        /*
@@ -438,12 +457,9 @@ kmem_malloc(map, size, canwait)
                        vm_object_unlock(kmem_object);
                        vm_map_delete(map, addr, addr + size);
                        vm_map_unlock(map);
                        vm_object_unlock(kmem_object);
                        vm_map_delete(map, addr, addr + size);
                        vm_map_unlock(map);
+                       thread_wakeup((int)&vm_pages_needed);
                        return(0);
                }
                        return(0);
                }
-#if 0
-               vm_page_zero_fill(m);
-#endif
-               m->flags &= ~PG_BUSY;
        }
        vm_object_unlock(kmem_object);
 
        }
        vm_object_unlock(kmem_object);
 
@@ -470,6 +486,8 @@ kmem_malloc(map, size, canwait)
                vm_object_unlock(kmem_object);
                pmap_enter(map->pmap, addr + i, VM_PAGE_TO_PHYS(m),
                           VM_PROT_DEFAULT, TRUE);
                vm_object_unlock(kmem_object);
                pmap_enter(map->pmap, addr + i, VM_PAGE_TO_PHYS(m),
                           VM_PROT_DEFAULT, TRUE);
+               PAGE_WAKEUP(m);
+               vm_page_wire(m);
        }
        vm_map_unlock(map);
 
        }
        vm_map_unlock(map);
 
@@ -484,7 +502,8 @@ kmem_malloc(map, size, canwait)
  *     has no room, the caller sleeps waiting for more memory in the submap.
  *
  */
  *     has no room, the caller sleeps waiting for more memory in the submap.
  *
  */
-vm_offset_t kmem_alloc_wait(map, size)
+vm_offset_t
+kmem_alloc_wait(map, size)
        vm_map_t        map;
        vm_size_t       size;
 {
        vm_map_t        map;
        vm_size_t       size;
 {
@@ -509,15 +528,14 @@ vm_offset_t kmem_alloc_wait(map, size)
                lock_clear_recursive(&map->lock);
                if (result != KERN_SUCCESS) {
 
                lock_clear_recursive(&map->lock);
                if (result != KERN_SUCCESS) {
 
-                       if ( (vm_map_max(map) - vm_map_min(map)) < size ) {
+                       if ((vm_map_max(map) - vm_map_min(map)) < size) {
                                vm_map_unlock(map);
                                return(0);
                        }
 
                                vm_map_unlock(map);
                                return(0);
                        }
 
-                       assert_wait((int)map, TRUE);
                        vm_map_unlock(map);
                        thread_wakeup((int)&vm_pages_needed); /* XXX */
                        vm_map_unlock(map);
                        thread_wakeup((int)&vm_pages_needed); /* XXX */
-                       thread_block("vmalloc");
+                       tsleep((caddr_t)map, PVM, "kmemaw", 0);
                }
                else {
                        vm_map_unlock(map);
                }
                else {
                        vm_map_unlock(map);
@@ -528,6 +546,7 @@ vm_offset_t kmem_alloc_wait(map, size)
        return(addr);
 }
 
        return(addr);
 }
 
+#if 0
 /*
  *     kmem_alloc_wired_wait
  *
 /*
  *     kmem_alloc_wired_wait
  *
@@ -535,7 +554,8 @@ vm_offset_t kmem_alloc_wait(map, size)
  *     has no room, the caller sleeps waiting for more memory in the submap.
  *
  */
  *     has no room, the caller sleeps waiting for more memory in the submap.
  *
  */
-vm_offset_t kmem_alloc_wired_wait(map, size)
+vm_offset_t
+kmem_alloc_wired_wait(map, size)
        vm_map_t        map;
        vm_size_t       size;
 {
        vm_map_t        map;
        vm_size_t       size;
 {
@@ -560,15 +580,14 @@ vm_offset_t kmem_alloc_wired_wait(map, size)
                lock_clear_recursive(&map->lock);
                if (result != KERN_SUCCESS) {
 
                lock_clear_recursive(&map->lock);
                if (result != KERN_SUCCESS) {
 
-                       if ( (vm_map_max(map) - vm_map_min(map)) < size ) {
+                       if ((vm_map_max(map) - vm_map_min(map)) < size) {
                                vm_map_unlock(map);
                                return(0);
                        }
 
                                vm_map_unlock(map);
                                return(0);
                        }
 
-                       assert_wait((int)map, TRUE);
                        vm_map_unlock(map);
                        thread_wakeup((int)&vm_pages_needed); /* XXX */
                        vm_map_unlock(map);
                        thread_wakeup((int)&vm_pages_needed); /* XXX */
-                       thread_block("kmalloc");
+                       tsleep((caddr_t)map, PVM, "kmemaww", 0);
                }
                else {
                        vm_map_unlock(map);
                }
                else {
                        vm_map_unlock(map);
@@ -578,6 +597,7 @@ vm_offset_t kmem_alloc_wired_wait(map, size)
 
        return(addr);
 }
 
        return(addr);
 }
+#endif
 
 /*
  *     kmem_free_wakeup
 
 /*
  *     kmem_free_wakeup
@@ -585,16 +605,17 @@ vm_offset_t kmem_alloc_wired_wait(map, size)
  *     Returns memory to a submap of the kernel, and wakes up any threads
  *     waiting for memory in that map.
  */
  *     Returns memory to a submap of the kernel, and wakes up any threads
  *     waiting for memory in that map.
  */
-void   kmem_free_wakeup(map, addr, size)
+void
+kmem_free_wakeup(map, addr, size)
        vm_map_t        map;
        vm_offset_t     addr;
        vm_size_t       size;
 {
        vm_map_lock(map);
        (void) vm_map_delete(map, trunc_page(addr), round_page(addr + size));
        vm_map_t        map;
        vm_offset_t     addr;
        vm_size_t       size;
 {
        vm_map_lock(map);
        (void) vm_map_delete(map, trunc_page(addr), round_page(addr + size));
-       thread_wakeup((int)map);
        vm_map_unlock(map);
        vm_map_simplify(map, addr);
        vm_map_unlock(map);
        vm_map_simplify(map, addr);
+       thread_wakeup((int)map);
 }
 
 /*
 }
 
 /*
@@ -603,7 +624,8 @@ void        kmem_free_wakeup(map, addr, size)
  *     Initialize the kernel's virtual memory map, taking
  *     into account all memory allocated up to this time.
  */
  *     Initialize the kernel's virtual memory map, taking
  *     into account all memory allocated up to this time.
  */
-void kmem_init(start, end)
+void
+kmem_init(start, end)
        vm_offset_t     start;
        vm_offset_t     end;
 {
        vm_offset_t     start;
        vm_offset_t     end;
 {
index 57eac46..0e4f563 100644 (file)
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_map.c      7.3 (Berkeley) 4/21/91
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_map.c      7.3 (Berkeley) 4/21/91
- *     $Id: vm_map.c,v 1.8 1993/11/25 01:39:04 wollman Exp $
- */
-
-/*
+ *     $Id: vm_map.c,v 1.9 1993/12/19 00:56:03 wollman Exp $
+ *
+ *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  *
 /*
  *     Virtual memory mapping module.
  */
 /*
  *     Virtual memory mapping module.
  */
-
 #include "ddb.h"
 #include "param.h"
 #include "ddb.h"
 #include "param.h"
-#include "systm.h"
 #include "malloc.h"
 #include "vm.h"
 #include "vm_page.h"
 #include "vm_object.h"
 #include "malloc.h"
 #include "vm.h"
 #include "vm_page.h"
 #include "vm_object.h"
+#include "systm.h"
 
 /*
  *     Virtual memory maps provide for the mapping, protection,
 
 /*
  *     Virtual memory maps provide for the mapping, protection,
 vm_offset_t    kentry_data;
 vm_size_t      kentry_data_size;
 vm_map_entry_t kentry_free;
 vm_offset_t    kentry_data;
 vm_size_t      kentry_data_size;
 vm_map_entry_t kentry_free;
+int kentry_count;
 vm_map_t       kmap_free;
 vm_map_t       kmap_free;
+static vm_offset_t mapvm=0;
+static int mapvmpgcnt=0;
+extern vm_map_t                kernel_map, kmem_map, pager_map;
+extern int vm_page_count;
+void vm_map_save_pmap(vm_map_t map, vm_map_entry_t entry) ;
 
 
-void vm_map_startup()
+void
+vm_map_startup()
 {
        register int i;
        register vm_map_entry_t mep;
 {
        register int i;
        register vm_map_entry_t mep;
@@ -161,11 +166,13 @@ void vm_map_startup()
         * Form a free list of statically allocated kernel map entries
         * with the rest.
         */
         * Form a free list of statically allocated kernel map entries
         * with the rest.
         */
+       kentry_count = 0;
        kentry_free = mep = (vm_map_entry_t) mp;
        i = (kentry_data_size - MAX_KMAP * sizeof *mp) / sizeof *mep;
        while (--i > 0) {
                mep->next = mep + 1;
                mep++;
        kentry_free = mep = (vm_map_entry_t) mp;
        i = (kentry_data_size - MAX_KMAP * sizeof *mp) / sizeof *mep;
        while (--i > 0) {
                mep->next = mep + 1;
                mep++;
+               kentry_count++;
        }
        mep->next = NULL;
 }
        }
        mep->next = NULL;
 }
@@ -181,6 +188,16 @@ vmspace_alloc(min, max, pageable)
        int pageable;
 {
        register struct vmspace *vm;
        int pageable;
 {
        register struct vmspace *vm;
+       int s;
+
+       if (mapvmpgcnt == 0 && mapvm == 0) {
+               mapvmpgcnt = (vm_page_count * sizeof(struct vm_map_entry) + NBPG - 1) / NBPG;
+               s = splhigh();
+               mapvm = kmem_alloc_pageable(kmem_map, mapvmpgcnt * NBPG);
+               splx(s);
+               if (!mapvm)
+                       mapvmpgcnt = 0;
+       }
 
        MALLOC(vm, struct vmspace *, sizeof(struct vmspace), M_VMMAP, M_WAITOK);
        bzero(vm, (caddr_t) &vm->vm_startcopy - (caddr_t) vm);
 
        MALLOC(vm, struct vmspace *, sizeof(struct vmspace), M_VMMAP, M_WAITOK);
        bzero(vm, (caddr_t) &vm->vm_startcopy - (caddr_t) vm);
@@ -192,22 +209,20 @@ vmspace_alloc(min, max, pageable)
 }
 
 void
 }
 
 void
-vmspace_free(vm)
+_vmspace_free(vm)
        register struct vmspace *vm;
 {
 
        register struct vmspace *vm;
 {
 
-       if (--vm->vm_refcnt == 0) {
-               /*
-                * Lock the map, to wait out all other references to it.
-                * Delete all of the mappings and pages they hold,
-                * then call the pmap module to reclaim anything left.
-                */
-               vm_map_lock(&vm->vm_map);
-               (void) vm_map_delete(&vm->vm_map, vm->vm_map.min_offset,
-                   vm->vm_map.max_offset);
-               pmap_release(&vm->vm_pmap);
-               FREE(vm, M_VMMAP);
-       }
+       /*
+        * Lock the map, to wait out all other references to it.
+        * Delete all of the mappings and pages they hold,
+        * then call the pmap module to reclaim anything left.
+        */
+       vm_map_lock(&vm->vm_map);
+       (void) vm_map_delete(&vm->vm_map, vm->vm_map.min_offset,
+           vm->vm_map.max_offset);
+       pmap_release(&vm->vm_pmap);
+       FREE(vm, M_VMMAP);
 }
 
 /*
 }
 
 /*
@@ -217,13 +232,13 @@ vmspace_free(vm)
  *     the given physical map structure, and having
  *     the given lower and upper address bounds.
  */
  *     the given physical map structure, and having
  *     the given lower and upper address bounds.
  */
-vm_map_t vm_map_create(pmap, min, max, pageable)
+vm_map_t
+vm_map_create(pmap, min, max, pageable)
        pmap_t          pmap;
        vm_offset_t     min, max;
        boolean_t       pageable;
 {
        register vm_map_t       result;
        pmap_t          pmap;
        vm_offset_t     min, max;
        boolean_t       pageable;
 {
        register vm_map_t       result;
-       extern vm_map_t         kernel_map, kmem_map;
 
        if (kmem_map == NULL) {
                result = kmap_free;
 
        if (kmem_map == NULL) {
                result = kmap_free;
@@ -272,19 +287,71 @@ vm_map_init(map, min, max, pageable)
  *     Allocates a VM map entry for insertion.
  *     No entry fields are filled in.  This routine is
  */
  *     Allocates a VM map entry for insertion.
  *     No entry fields are filled in.  This routine is
  */
-vm_map_entry_t vm_map_entry_create(map)
+static struct vm_map_entry *mappool;
+static int mappoolcnt;
+void vm_map_entry_dispose(vm_map_t map, vm_map_entry_t entry);
+
+vm_map_entry_t
+vm_map_entry_create(map)
        vm_map_t        map;
 {
        vm_map_entry_t  entry;
        vm_map_t        map;
 {
        vm_map_entry_t  entry;
-       extern vm_map_t         kernel_map, kmem_map, mb_map, buffer_map, pager_map;
+       int s;
+       int i;
+#define KENTRY_LOW_WATER 64
+#define MAPENTRY_LOW_WATER 64
 
 
-       if (map == kernel_map || map == kmem_map || map == mb_map
-               || map == buffer_map || map == pager_map) {
-               if (entry = kentry_free)
-                       kentry_free = kentry_free->next;
-       } else
+       /*
+        * This is a *very* nasty (and sort of incomplete) hack!!!!
+        */
+       if (kentry_count < KENTRY_LOW_WATER) {
+               if (mapvmpgcnt && mapvm) {
+                       vm_page_t m;
+                       if (m = vm_page_alloc(kmem_object, mapvm-vm_map_min(kmem_map))) {
+                               int newentries;
+                               newentries = (NBPG/sizeof (struct vm_map_entry));
+                               vm_page_wire(m);
+                               m->flags &= ~PG_BUSY;
+                               pmap_enter(vm_map_pmap(kmem_map), mapvm,
+                                       VM_PAGE_TO_PHYS(m), VM_PROT_DEFAULT, 1);
+
+                               entry = (vm_map_entry_t) mapvm;
+                               mapvm += NBPG;
+                               --mapvmpgcnt;
+
+                               for (i = 0; i < newentries; i++) {
+                                       vm_map_entry_dispose(kernel_map, entry);
+                                       entry++;
+                               }
+                       }
+               }
+       }
+
+       if (map == kernel_map || map == kmem_map || map == pager_map) {
+
+               if (entry = kentry_free) {
+                       kentry_free = entry->next;
+                       --kentry_count;
+                       return entry;
+               }
+
+               if (entry = mappool) {
+                       mappool = entry->next;
+                       --mappoolcnt;
+                       return entry;
+               }
+
+       } else {
+               if (entry = mappool) {
+                       mappool = entry->next;
+                       --mappoolcnt;
+                       return entry;
+               }
+                       
                MALLOC(entry, vm_map_entry_t, sizeof(struct vm_map_entry),
                       M_VMMAPENT, M_WAITOK);
                MALLOC(entry, vm_map_entry_t, sizeof(struct vm_map_entry),
                       M_VMMAPENT, M_WAITOK);
+       }
+dopanic:
        if (entry == NULL)
                panic("vm_map_entry_create: out of map entries");
 
        if (entry == NULL)
                panic("vm_map_entry_create: out of map entries");
 
@@ -296,18 +363,29 @@ vm_map_entry_t vm_map_entry_create(map)
  *
  *     Inverse of vm_map_entry_create.
  */
  *
  *     Inverse of vm_map_entry_create.
  */
-void vm_map_entry_dispose(map, entry)
+void
+vm_map_entry_dispose(map, entry)
        vm_map_t        map;
        vm_map_entry_t  entry;
 {
        vm_map_t        map;
        vm_map_entry_t  entry;
 {
-       extern vm_map_t         kernel_map, kmem_map, mb_map, buffer_map, pager_map;
+       extern vm_map_t         kernel_map, kmem_map, pager_map;
+       int s;
 
 
-       if (map == kernel_map || map == kmem_map || map == mb_map
-               || map == buffer_map || map == pager_map) {
+       if (map == kernel_map || map == kmem_map || map == pager_map ||
+               kentry_count < KENTRY_LOW_WATER) {
                entry->next = kentry_free;
                kentry_free = entry;
                entry->next = kentry_free;
                kentry_free = entry;
-       } else
+               ++kentry_count;
+       } else {
+               if (mappoolcnt < MAPENTRY_LOW_WATER) {
+                       entry->next = mappool;
+                       mappool = entry;
+                       ++mappoolcnt;
+                       return;
+               }
+                       
                FREE(entry, M_VMMAPENT);
                FREE(entry, M_VMMAPENT);
+       }
 }
 
 /*
 }
 
 /*
@@ -336,7 +414,8 @@ void vm_map_entry_dispose(map, entry)
  *     Creates another valid reference to the given map.
  *
  */
  *     Creates another valid reference to the given map.
  *
  */
-void vm_map_reference(map)
+inline void
+vm_map_reference(map)
        register vm_map_t       map;
 {
        if (map == NULL)
        register vm_map_t       map;
 {
        if (map == NULL)
@@ -354,7 +433,8 @@ void vm_map_reference(map)
  *     destroying it if no references remain.
  *     The map should not be locked.
  */
  *     destroying it if no references remain.
  *     The map should not be locked.
  */
-void vm_map_deallocate(map)
+void
+vm_map_deallocate(map)
        register vm_map_t       map;
 {
        register int            c;
        register vm_map_t       map;
 {
        register int            c;
@@ -419,8 +499,9 @@ vm_map_insert(map, object, offset, start, end)
         *      existing entry, this range is bogus.
         */
 
         *      existing entry, this range is bogus.
         */
 
-       if (vm_map_lookup_entry(map, start, &temp_entry))
+       if (vm_map_lookup_entry(map, start, &temp_entry)) {
                return(KERN_NO_SPACE);
                return(KERN_NO_SPACE);
+       }
 
        prev_entry = temp_entry;
 
 
        prev_entry = temp_entry;
 
@@ -430,8 +511,9 @@ vm_map_insert(map, object, offset, start, end)
         */
 
        if ((prev_entry->next != &map->header) &&
         */
 
        if ((prev_entry->next != &map->header) &&
-                       (prev_entry->next->start < end))
+                       (prev_entry->next->start < end)) {
                return(KERN_NO_SPACE);
                return(KERN_NO_SPACE);
+       }
 
        /*
         *      See if we can avoid creating a new entry by
 
        /*
         *      See if we can avoid creating a new entry by
@@ -529,7 +611,8 @@ vm_map_insert(map, object, offset, start, end)
  *     result indicates whether the address is
  *     actually contained in the map.
  */
  *     result indicates whether the address is
  *     actually contained in the map.
  */
-boolean_t vm_map_lookup_entry(map, address, entry)
+boolean_t
+vm_map_lookup_entry(map, address, entry)
        register vm_map_t       map;
        register vm_offset_t    address;
        vm_map_entry_t          *entry;         /* OUT */
        register vm_map_t       map;
        register vm_offset_t    address;
        vm_map_entry_t          *entry;         /* OUT */
@@ -621,10 +704,10 @@ vm_map_find(map, object, offset, addr, length, find_space)
        register vm_offset_t    end;
        int                     result;
 
        register vm_offset_t    end;
        int                     result;
 
-       start = *addr;
-
        vm_map_lock(map);
 
        vm_map_lock(map);
 
+       start = *addr;
+
        if (find_space) {
                /*
                 *      Calculate the first possible address.
        if (find_space) {
                /*
                 *      Calculate the first possible address.
@@ -716,7 +799,8 @@ vm_map_find(map, object, offset, addr, length, find_space)
  *             removing extra sharing maps
  *             [XXX maybe later] merging with a neighbor
  */
  *             removing extra sharing maps
  *             [XXX maybe later] merging with a neighbor
  */
-void vm_map_simplify_entry(map, entry)
+void
+vm_map_simplify_entry(map, entry)
        vm_map_t        map;
        vm_map_entry_t  entry;
 {
        vm_map_t        map;
        vm_map_entry_t  entry;
 {
@@ -790,8 +874,8 @@ void vm_map_simplify_entry(map, entry)
  *     This routine is called only when it is known that
  *     the entry must be split.
  */
  *     This routine is called only when it is known that
  *     the entry must be split.
  */
-static
-void _vm_map_clip_start(map, entry, start)
+void
+_vm_map_clip_start(map, entry, start)
        register vm_map_t       map;
        register vm_map_entry_t entry;
        register vm_offset_t    start;
        register vm_map_t       map;
        register vm_map_entry_t entry;
        register vm_offset_t    start;
@@ -802,7 +886,7 @@ void _vm_map_clip_start(map, entry, start)
         *      See if we can simplify this entry first
         */
                 
         *      See if we can simplify this entry first
         */
                 
-       vm_map_simplify_entry(map, entry);
+       /* vm_map_simplify_entry(map, entry); */
 
        /*
         *      Split off the front portion --
 
        /*
         *      Split off the front portion --
@@ -835,6 +919,7 @@ void _vm_map_clip_start(map, entry, start)
  *     it splits the entry into two.
  */
 
  *     it splits the entry into two.
  */
 
+void _vm_map_clip_end();
 #define vm_map_clip_end(map, entry, endaddr) \
 { \
        if (endaddr < entry->end) \
 #define vm_map_clip_end(map, entry, endaddr) \
 { \
        if (endaddr < entry->end) \
@@ -845,8 +930,8 @@ void _vm_map_clip_start(map, entry, start)
  *     This routine is called only when it is known that
  *     the entry must be split.
  */
  *     This routine is called only when it is known that
  *     the entry must be split.
  */
-static
-void _vm_map_clip_end(map, entry, end)
+inline void
+_vm_map_clip_end(map, entry, end)
        register vm_map_t       map;
        register vm_map_entry_t entry;
        register vm_offset_t    end;
        register vm_map_t       map;
        register vm_map_entry_t entry;
        register vm_offset_t    end;
@@ -1134,6 +1219,7 @@ vm_map_pageable(map, start, end, new_pageable)
 {
        register vm_map_entry_t entry;
        vm_map_entry_t          temp_entry;
 {
        register vm_map_entry_t entry;
        vm_map_entry_t          temp_entry;
+       extern vm_map_t kernel_map;
 
        vm_map_lock(map);
 
 
        vm_map_lock(map);
 
@@ -1315,7 +1401,8 @@ vm_map_pageable(map, start, end, new_pageable)
  *     The map in question should be locked.
  *     [This is the reason for this routine's existence.]
  */
  *     The map in question should be locked.
  *     [This is the reason for this routine's existence.]
  */
-void vm_map_entry_unwire(map, entry)
+void
+vm_map_entry_unwire(map, entry)
        vm_map_t                map;
        register vm_map_entry_t entry;
 {
        vm_map_t                map;
        register vm_map_entry_t entry;
 {
@@ -1328,20 +1415,23 @@ void vm_map_entry_unwire(map, entry)
  *
  *     Deallocate the given entry from the target map.
  */            
  *
  *     Deallocate the given entry from the target map.
  */            
-void vm_map_entry_delete(map, entry)
+void
+vm_map_entry_delete(map, entry)
        register vm_map_t       map;
        register vm_map_entry_t entry;
 {
        register vm_map_t       map;
        register vm_map_entry_t entry;
 {
+       int prev_ref_count;
        if (entry->wired_count != 0)
                vm_map_entry_unwire(map, entry);
                
        vm_map_entry_unlink(map, entry);
        map->size -= entry->end - entry->start;
 
        if (entry->wired_count != 0)
                vm_map_entry_unwire(map, entry);
                
        vm_map_entry_unlink(map, entry);
        map->size -= entry->end - entry->start;
 
-       if (entry->is_a_map || entry->is_sub_map)
+       if (entry->is_a_map || entry->is_sub_map) {
                vm_map_deallocate(entry->object.share_map);
                vm_map_deallocate(entry->object.share_map);
-       else
+       } else {
                vm_object_deallocate(entry->object.vm_object);
                vm_object_deallocate(entry->object.vm_object);
+       }
 
        vm_map_entry_dispose(map, entry);
 }
 
        vm_map_entry_dispose(map, entry);
 }
@@ -1364,6 +1454,7 @@ vm_map_delete(map, start, end)
        register vm_map_entry_t entry;
        vm_map_entry_t          first_entry;
 
        register vm_map_entry_t entry;
        vm_map_entry_t          first_entry;
 
+
        /*
         *      Find the start of the region, and clip it
         */
        /*
         *      Find the start of the region, and clip it
         */
@@ -1421,15 +1512,20 @@ vm_map_delete(map, start, end)
                 *      it.
                 */
 
                 *      it.
                 */
 
-               if (object == kernel_object || object == kmem_object)
+               if (object == kernel_object || object == kmem_object) {
                        vm_object_page_remove(object, entry->offset,
                                        entry->offset + (e - s));
                        vm_object_page_remove(object, entry->offset,
                                        entry->offset + (e - s));
-               else if (!map->is_main_map)
+               } else if (!map->is_main_map) {
                        vm_object_pmap_remove(object,
                                         entry->offset,
                                         entry->offset + (e - s));
                        vm_object_pmap_remove(object,
                                         entry->offset,
                                         entry->offset + (e - s));
-               else
+               } else {
+               /*
+                * save the pmap info
+                */
+                       vm_map_save_pmap(map, entry);
                        pmap_remove(map->pmap, s, e);
                        pmap_remove(map->pmap, s, e);
+               }
 
                /*
                 *      Delete the entry (which may delete the object)
 
                /*
                 *      Delete the entry (which may delete the object)
@@ -1445,6 +1541,36 @@ vm_map_delete(map, start, end)
        return(KERN_SUCCESS);
 }
 
        return(KERN_SUCCESS);
 }
 
+/*
+ *     vm_map_save_pmap
+ *     
+ *     Save the recorded pmap information into the vm_page_t
+ *     data structures.
+ *
+ */
+void
+vm_map_save_pmap(map, entry)
+       vm_map_t map;
+       vm_map_entry_t entry;
+{
+       vm_map_t tmpm;
+       vm_map_entry_t tmpe;
+       if (entry == 0) {
+               return;
+       } else if (entry->is_sub_map || entry->is_a_map) {
+               tmpm = entry->object.share_map;
+               tmpe = tmpm->header.next;
+               while (tmpe != &tmpm->header) {
+                       vm_map_save_pmap(tmpm, tmpe);
+                       tmpe = tmpe->next;
+               };
+                       
+       } else if (entry->object.vm_object) {
+               vm_object_save_pmap_attributes(entry->object.vm_object,
+                       entry->offset, entry->offset + (entry->end - entry->start));
+       }
+}
+       
 /*
  *     vm_map_remove:
  *
 /*
  *     vm_map_remove:
  *
@@ -1474,7 +1600,8 @@ vm_map_remove(map, start, end)
  *     privilege on the entire address region given.
  *     The entire region must be allocated.
  */
  *     privilege on the entire address region given.
  *     The entire region must be allocated.
  */
-boolean_t vm_map_check_protection(map, start, end, protection)
+boolean_t
+vm_map_check_protection(map, start, end, protection)
        register vm_map_t       map;
        register vm_offset_t    start;
        register vm_offset_t    end;
        register vm_map_t       map;
        register vm_offset_t    start;
        register vm_offset_t    end;
@@ -1524,7 +1651,8 @@ boolean_t vm_map_check_protection(map, start, end, protection)
  *     Copies the contents of the source entry to the destination
  *     entry.  The entries *must* be aligned properly.
  */
  *     Copies the contents of the source entry to the destination
  *     entry.  The entries *must* be aligned properly.
  */
-void vm_map_copy_entry(src_map, dst_map, src_entry, dst_entry)
+void
+vm_map_copy_entry(src_map, dst_map, src_entry, dst_entry)
        vm_map_t                src_map, dst_map;
        register vm_map_entry_t src_entry, dst_entry;
 {
        vm_map_t                src_map, dst_map;
        register vm_map_entry_t src_entry, dst_entry;
 {
@@ -1552,13 +1680,15 @@ void vm_map_copy_entry(src_map, dst_map, src_entry, dst_entry)
         *      this sharing map belongs in).
         */
 
         *      this sharing map belongs in).
         */
 
-       if (dst_map->is_main_map)
+       if (dst_map->is_main_map) {
+               vm_map_save_pmap(dst_map, dst_entry);
                pmap_remove(dst_map->pmap, dst_entry->start, dst_entry->end);
                pmap_remove(dst_map->pmap, dst_entry->start, dst_entry->end);
-       else
+       } else {
                vm_object_pmap_remove(dst_entry->object.vm_object,
                        dst_entry->offset,
                        dst_entry->offset +
                                (dst_entry->end - dst_entry->start));
                vm_object_pmap_remove(dst_entry->object.vm_object,
                        dst_entry->offset,
                        dst_entry->offset +
                                (dst_entry->end - dst_entry->start));
+       }
 
        if (src_entry->wired_count == 0) {
 
 
        if (src_entry->wired_count == 0) {
 
@@ -1662,9 +1792,7 @@ void vm_map_copy_entry(src_map, dst_map, src_entry, dst_entry)
  *     fragmentation.]
  */
 int
  *     fragmentation.]
  */
 int
-vm_map_copy(dst_map, src_map,
-                         dst_addr, len, src_addr,
-                         dst_alloc, src_destroy)
+vm_map_copy(dst_map, src_map, dst_addr, len, src_addr, dst_alloc, src_destroy)
        vm_map_t        dst_map;
        vm_map_t        src_map;
        vm_offset_t     dst_addr;
        vm_map_t        dst_map;
        vm_map_t        src_map;
        vm_offset_t     dst_addr;
@@ -1691,14 +1819,12 @@ vm_map_copy(dst_map, src_map,
         *      XXX While we figure out why src_destroy screws up,
         *      we'll do it by explicitly vm_map_delete'ing at the end.
         */
         *      XXX While we figure out why src_destroy screws up,
         *      we'll do it by explicitly vm_map_delete'ing at the end.
         */
-
        old_src_destroy = src_destroy;
        src_destroy = FALSE;
 
        /*
         *      Compute start and end of region in both maps
         */
        old_src_destroy = src_destroy;
        src_destroy = FALSE;
 
        /*
         *      Compute start and end of region in both maps
         */
-
        src_start = src_addr;
        src_end = src_start + len;
        dst_start = dst_addr;
        src_start = src_addr;
        src_end = src_start + len;
        dst_start = dst_addr;
@@ -1708,7 +1834,6 @@ vm_map_copy(dst_map, src_map,
         *      Check that the region can exist in both source
         *      and destination.
         */
         *      Check that the region can exist in both source
         *      and destination.
         */
-
        if ((dst_end < dst_start) || (src_end < src_start))
                return(KERN_NO_SPACE);
 
        if ((dst_end < dst_start) || (src_end < src_start))
                return(KERN_NO_SPACE);
 
@@ -1716,7 +1841,6 @@ vm_map_copy(dst_map, src_map,
         *      Lock the maps in question -- we avoid deadlock
         *      by ordering lock acquisition by map value
         */
         *      Lock the maps in question -- we avoid deadlock
         *      by ordering lock acquisition by map value
         */
-
        if (src_map == dst_map) {
                vm_map_lock(src_map);
        }
        if (src_map == dst_map) {
                vm_map_lock(src_map);
        }
@@ -1737,7 +1861,6 @@ vm_map_copy(dst_map, src_map,
         *      about protection, but instead about whether the region
         *      exists.]
         */
         *      about protection, but instead about whether the region
         *      exists.]
         */
-
        if (src_map->is_main_map && dst_map->is_main_map) {
                if (!vm_map_check_protection(src_map, src_start, src_end,
                                        VM_PROT_READ)) {
        if (src_map->is_main_map && dst_map->is_main_map) {
                if (!vm_map_check_protection(src_map, src_start, src_end,
                                        VM_PROT_READ)) {
@@ -1768,7 +1891,6 @@ vm_map_copy(dst_map, src_map,
         *      until we have done the first clip, as the clip
         *      may affect which entry we get!
         */
         *      until we have done the first clip, as the clip
         *      may affect which entry we get!
         */
-
        (void) vm_map_lookup_entry(src_map, src_addr, &tmp_entry);
        src_entry = tmp_entry;
        vm_map_clip_start(src_map, src_entry, src_start);
        (void) vm_map_lookup_entry(src_map, src_addr, &tmp_entry);
        src_entry = tmp_entry;
        vm_map_clip_start(src_map, src_entry, src_start);
@@ -1781,7 +1903,6 @@ vm_map_copy(dst_map, src_map,
         *      If both source and destination entries are the same,
         *      retry the first lookup, as it may have changed.
         */
         *      If both source and destination entries are the same,
         *      retry the first lookup, as it may have changed.
         */
-
        if (src_entry == dst_entry) {
                (void) vm_map_lookup_entry(src_map, src_addr, &tmp_entry);
                src_entry = tmp_entry;
        if (src_entry == dst_entry) {
                (void) vm_map_lookup_entry(src_map, src_addr, &tmp_entry);
                src_entry = tmp_entry;
@@ -1791,7 +1912,6 @@ vm_map_copy(dst_map, src_map,
         *      If source and destination entries are still the same,
         *      a null copy is being performed.
         */
         *      If source and destination entries are still the same,
         *      a null copy is being performed.
         */
-
        if (src_entry == dst_entry)
                goto Return;
 
        if (src_entry == dst_entry)
                goto Return;
 
@@ -1799,19 +1919,16 @@ vm_map_copy(dst_map, src_map,
         *      Go through entries until we get to the end of the
         *      region.
         */
         *      Go through entries until we get to the end of the
         *      region.
         */
-
        while (src_start < src_end) {
                /*
                 *      Clip the entries to the endpoint of the entire region.
                 */
        while (src_start < src_end) {
                /*
                 *      Clip the entries to the endpoint of the entire region.
                 */
-
                vm_map_clip_end(src_map, src_entry, src_end);
                vm_map_clip_end(dst_map, dst_entry, dst_end);
 
                /*
                 *      Clip each entry to the endpoint of the other entry.
                 */
                vm_map_clip_end(src_map, src_entry, src_end);
                vm_map_clip_end(dst_map, dst_entry, dst_end);
 
                /*
                 *      Clip each entry to the endpoint of the other entry.
                 */
-
                src_clip = src_entry->start + (dst_entry->end - dst_entry->start);
                vm_map_clip_end(src_map, src_entry, src_clip);
 
                src_clip = src_entry->start + (dst_entry->end - dst_entry->start);
                vm_map_clip_end(src_map, src_entry, src_clip);
 
@@ -1824,7 +1941,6 @@ vm_map_copy(dst_map, src_map,
                 *      If both entries refer to a VM object, we can
                 *      deal with them now.
                 */
                 *      If both entries refer to a VM object, we can
                 *      deal with them now.
                 */
-
                if (!src_entry->is_a_map && !dst_entry->is_a_map) {
                        vm_map_copy_entry(src_map, dst_map, src_entry,
                                                dst_entry);
                if (!src_entry->is_a_map && !dst_entry->is_a_map) {
                        vm_map_copy_entry(src_map, dst_map, src_entry,
                                                dst_entry);
@@ -1839,7 +1955,6 @@ vm_map_copy(dst_map, src_map,
                        /*
                         *      We have to follow at least one sharing map.
                         */
                        /*
                         *      We have to follow at least one sharing map.
                         */
-
                        new_size = (dst_entry->end - dst_entry->start);
 
                        if (src_entry->is_a_map) {
                        new_size = (dst_entry->end - dst_entry->start);
 
                        if (src_entry->is_a_map) {
@@ -1868,7 +1983,6 @@ vm_map_copy(dst_map, src_map,
                                 *      Note that we can only do so if the
                                 *      source and destination do not overlap.
                                 */
                                 *      Note that we can only do so if the
                                 *      source and destination do not overlap.
                                 */
-
                                new_dst_end = new_dst_start + new_size;
 
                                if (new_dst_map != new_src_map) {
                                new_dst_end = new_dst_start + new_size;
 
                                if (new_dst_map != new_src_map) {
@@ -1893,7 +2007,6 @@ vm_map_copy(dst_map, src_map,
                        /*
                         *      Recursively copy the sharing map.
                         */
                        /*
                         *      Recursively copy the sharing map.
                         */
-
                        (void) vm_map_copy(new_dst_map, new_src_map,
                                new_dst_start, new_size, new_src_start,
                                FALSE, FALSE);
                        (void) vm_map_copy(new_dst_map, new_src_map,
                                new_dst_start, new_size, new_src_start,
                                FALSE, FALSE);
@@ -1907,7 +2020,6 @@ vm_map_copy(dst_map, src_map,
                /*
                 *      Update variables for next pass through the loop.
                 */
                /*
                 *      Update variables for next pass through the loop.
                 */
-
                src_start = src_entry->end;
                src_entry = src_entry->next;
                dst_start = dst_entry->end;
                src_start = src_entry->end;
                src_entry = src_entry->next;
                dst_start = dst_entry->end;
@@ -1917,7 +2029,6 @@ vm_map_copy(dst_map, src_map,
                 *      If the source is to be destroyed, here is the
                 *      place to do it.
                 */
                 *      If the source is to be destroyed, here is the
                 *      place to do it.
                 */
-
                if (src_destroy && src_map->is_main_map &&
                                                dst_map->is_main_map)
                        vm_map_entry_delete(src_map, src_entry->prev);
                if (src_destroy && src_map->is_main_map &&
                                                dst_map->is_main_map)
                        vm_map_entry_delete(src_map, src_entry->prev);
@@ -1926,10 +2037,11 @@ vm_map_copy(dst_map, src_map,
        /*
         *      Update the physical maps as appropriate
         */
        /*
         *      Update the physical maps as appropriate
         */
-
        if (src_map->is_main_map && dst_map->is_main_map) {
        if (src_map->is_main_map && dst_map->is_main_map) {
-               if (src_destroy)
+               if (src_destroy) {
+                       vm_map_save_pmap(src_map, src_entry);
                        pmap_remove(src_map->pmap, src_addr, src_addr + len);
                        pmap_remove(src_map->pmap, src_addr, src_addr + len);
+               }
        }
 
        /*
        }
 
        /*
@@ -1937,9 +2049,10 @@ vm_map_copy(dst_map, src_map,
         */
 
        Return: ;
         */
 
        Return: ;
-
-       if (old_src_destroy)
+       
+       if (old_src_destroy) {
                vm_map_delete(src_map, src_addr, src_addr + len);
                vm_map_delete(src_map, src_addr, src_addr + len);
+       }
 
        vm_map_unlock(src_map);
        if (src_map != dst_map)
 
        vm_map_unlock(src_map);
        if (src_map != dst_map)
@@ -2101,6 +2214,23 @@ vmspace_fork(vm1)
        return(vm2);
 }
 
        return(vm2);
 }
 
+/*
+ *     vmspace_deallocate
+ *
+ *     clean up old parent vmspace references
+ *
+ */
+
+void
+vmspace_free(struct vmspace *vm) {
+
+       if (vm == 0 || --vm->vm_refcnt != 0) {
+               return;
+       }
+       _vmspace_free(vm);
+}
+
+
 /*
  *     vm_map_lookup:
  *
 /*
  *     vm_map_lookup:
  *
@@ -2124,8 +2254,7 @@ vmspace_fork(vm1)
  *     remain the same.
  */
 int
  *     remain the same.
  */
 int
-vm_map_lookup(var_map, vaddr, fault_type, out_entry,
-                               object, offset, out_prot, wired, single_use)
+vm_map_lookup(var_map, vaddr, fault_type, out_entry, object, offset, out_prot, wired, single_use)
        vm_map_t                *var_map;       /* IN/OUT */
        register vm_offset_t    vaddr;
        register vm_prot_t      fault_type;
        vm_map_t                *var_map;       /* IN/OUT */
        register vm_offset_t    vaddr;
        register vm_prot_t      fault_type;
@@ -2177,8 +2306,9 @@ vm_map_lookup(var_map, vaddr, fault_type, out_entry,
                 *      Entry was either not a valid hint, or the vaddr
                 *      was not contained in the entry, so do a full lookup.
                 */
                 *      Entry was either not a valid hint, or the vaddr
                 *      was not contained in the entry, so do a full lookup.
                 */
-               if (!vm_map_lookup_entry(map, vaddr, &tmp_entry))
+               if (!vm_map_lookup_entry(map, vaddr, &tmp_entry)) {
                        RETURN(KERN_INVALID_ADDRESS);
                        RETURN(KERN_INVALID_ADDRESS);
+               }
 
                entry = tmp_entry;
                *out_entry = entry;
 
                entry = tmp_entry;
                *out_entry = entry;
@@ -2343,7 +2473,8 @@ vm_map_lookup(var_map, vaddr, fault_type, out_entry,
  *     (according to the handle returned by that lookup).
  */
 
  *     (according to the handle returned by that lookup).
  */
 
-void vm_map_lookup_done(map, entry)
+void
+vm_map_lookup_done(map, entry)
        register vm_map_t       map;
        vm_map_entry_t          entry;
 {
        register vm_map_t       map;
        vm_map_entry_t          entry;
 {
@@ -2373,7 +2504,8 @@ void vm_map_lookup_done(map, entry)
  *             at allocation time because the adjacent entry
  *             is often wired down.
  */
  *             at allocation time because the adjacent entry
  *             is often wired down.
  */
-void vm_map_simplify(map, start)
+void
+vm_map_simplify(map, start)
        vm_map_t        map;
        vm_offset_t     start;
 {
        vm_map_t        map;
        vm_offset_t     start;
 {
@@ -2409,11 +2541,13 @@ void vm_map_simplify(map, start)
                if (map->first_free == this_entry)
                        map->first_free = prev_entry;
 
                if (map->first_free == this_entry)
                        map->first_free = prev_entry;
 
-               SAVE_HINT(map, prev_entry);
-               vm_map_entry_unlink(map, this_entry);
-               prev_entry->end = this_entry->end;
-               vm_object_deallocate(this_entry->object.vm_object);
-               vm_map_entry_dispose(map, this_entry);
+               if (!this_entry->object.vm_object->paging_in_progress) {
+                       SAVE_HINT(map, prev_entry);
+                       vm_map_entry_unlink(map, this_entry);
+                       prev_entry->end = this_entry->end;
+                       vm_object_deallocate(this_entry->object.vm_object);
+                       vm_map_entry_dispose(map, this_entry);
+               }
        }
        vm_map_unlock(map);
 }
        }
        vm_map_unlock(map);
 }
@@ -2422,24 +2556,32 @@ void vm_map_simplify(map, start)
 /*
  *     vm_map_print:   [ debug ]
  */
 /*
  *     vm_map_print:   [ debug ]
  */
-void vm_map_print(map, full)
+void
+vm_map_print(map, full)
        register vm_map_t       map;
        boolean_t               full;
 {
        register vm_map_entry_t entry;
        extern int indent;
        register vm_map_t       map;
        boolean_t               full;
 {
        register vm_map_entry_t entry;
        extern int indent;
+       static int nmaps;
 
 
+       if (indent == 0)
+               nmaps = 0;
        iprintf("%s map 0x%x: pmap=0x%x,ref=%d,nentries=%d,version=%d\n",
                (map->is_main_map ? "Task" : "Share"),
                (int) map, (int) (map->pmap), map->ref_count, map->nentries,
                map->timestamp);
 
        iprintf("%s map 0x%x: pmap=0x%x,ref=%d,nentries=%d,version=%d\n",
                (map->is_main_map ? "Task" : "Share"),
                (int) map, (int) (map->pmap), map->ref_count, map->nentries,
                map->timestamp);
 
+/*
        if (!full && indent)
                return;
        if (!full && indent)
                return;
+*/
 
        indent += 2;
        for (entry = map->header.next; entry != &map->header;
                                entry = entry->next) {
 
        indent += 2;
        for (entry = map->header.next; entry != &map->header;
                                entry = entry->next) {
+               nmaps++;
+               if (full || indent == 2) {
                iprintf("map entry 0x%x: start=0x%x, end=0x%x, ",
                        (int) entry, (int) entry->start, (int) entry->end);
                if (map->is_main_map) {
                iprintf("map entry 0x%x: start=0x%x, end=0x%x, ",
                        (int) entry, (int) entry->start, (int) entry->end);
                if (map->is_main_map) {
@@ -2452,8 +2594,10 @@ void vm_map_print(map, full)
                        if (entry->wired_count != 0)
                                printf("wired, ");
                }
                        if (entry->wired_count != 0)
                                printf("wired, ");
                }
+               }
 
                if (entry->is_a_map || entry->is_sub_map) {
 
                if (entry->is_a_map || entry->is_sub_map) {
+               if (full || indent == 2)
                        printf("share=0x%x, offset=0x%x\n",
                                (int) entry->object.share_map,
                                (int) entry->offset);
                        printf("share=0x%x, offset=0x%x\n",
                                (int) entry->object.share_map,
                                (int) entry->offset);
@@ -2467,7 +2611,8 @@ void vm_map_print(map, full)
                        }
                                
                }
                        }
                                
                }
-               else {
+               else if (full || indent == 2) {
+                       
                        printf("object=0x%x, offset=0x%x",
                                (int) entry->object.vm_object,
                                (int) entry->offset);
                        printf("object=0x%x, offset=0x%x",
                                (int) entry->object.vm_object,
                                (int) entry->offset);
@@ -2487,5 +2632,8 @@ void vm_map_print(map, full)
                }
        }
        indent -= 2;
                }
        }
        indent -= 2;
+
+       if (indent == 0)
+               printf("nmaps=%d\n", nmaps);
 }
 }
-#endif /* defined(DEBUG) || (NDDB > 0) */
+#endif
index 6d5bb7a..f0b2d52 100644 (file)
@@ -34,7 +34,7 @@
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_map.h      7.3 (Berkeley) 4/21/91
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_map.h      7.3 (Berkeley) 4/21/91
- *     $Id: vm_map.h,v 1.2 1993/10/16 16:20:36 rgrimes Exp $
+ *     $Id: vm_map.h,v 1.3 1993/12/19 00:56:05 wollman Exp $
  */
 
 /*
  */
 
 /*
@@ -231,10 +231,6 @@ extern void vm_fault_copy_entry(vm_map_t, vm_map_t, vm_map_entry_t,
 /* XXX: number of kernel maps and entries to statically allocate */
 #define MAX_KMAP       10
 
 /* XXX: number of kernel maps and entries to statically allocate */
 #define MAX_KMAP       10
 
-#ifdef OMIT
-#define MAX_KMAPENT     500
-#else   /* !OMIT*/
-#define MAX_KMAPENT     1000   /* 15 Aug 92*/
-#endif  /* !OMIT*/
+#define MAX_KMAPENT    128
 
 #endif _VM_MAP_
 
 #endif _VM_MAP_
index ed5e78d..e957f30 100644 (file)
@@ -37,7 +37,7 @@
  *
  *     from: Utah $Hdr: vm_mmap.c 1.3 90/01/21$
  *     from: @(#)vm_mmap.c     7.5 (Berkeley) 6/28/91
  *
  *     from: Utah $Hdr: vm_mmap.c 1.3 90/01/21$
  *     from: @(#)vm_mmap.c     7.5 (Berkeley) 6/28/91
- *     $Id: vm_mmap.c,v 1.18 1993/12/24 10:31:55 davidg Exp $
+ *     $Id: vm_mmap.c,v 1.19 1993/12/24 10:37:31 davidg Exp $
  */
 
 /*
  */
 
 /*
@@ -709,7 +709,7 @@ vm_mmap(map, addr, size, prot, maxprot, flags, handle, foff)
         * We only need to set max_protection in case it's
         * unequal to its default, which is VM_PROT_DEFAULT.
         */
         * We only need to set max_protection in case it's
         * unequal to its default, which is VM_PROT_DEFAULT.
         */
-       if(maxprot != VM_PROT_DEFAULT) {
+       if (maxprot != VM_PROT_DEFAULT) {
                rv = vm_map_protect(map, *addr, *addr+size, maxprot, TRUE);
                if (rv != KERN_SUCCESS) {
                        (void) vm_deallocate(map, *addr, size);
                rv = vm_map_protect(map, *addr, *addr+size, maxprot, TRUE);
                if (rv != KERN_SUCCESS) {
                        (void) vm_deallocate(map, *addr, size);
index da81f2d..789f492 100644 (file)
@@ -35,9 +35,8 @@
  *
  *     from: @(#)vm_object.c   7.4 (Berkeley) 5/7/91
  *     $Id: vm_object.c,v 1.13 1993/12/22 12:51:59 davidg Exp $
  *
  *     from: @(#)vm_object.c   7.4 (Berkeley) 5/7/91
  *     $Id: vm_object.c,v 1.13 1993/12/22 12:51:59 davidg Exp $
- */
-
-/*
+ *
+ *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  *
  * rights to redistribute these changes.
  */
 
  * rights to redistribute these changes.
  */
 
-/*
- * Significant modifications to vm_object_collapse fixing the growing
- * swap space allocation by John S. Dyson, 18 Dec 93.
- */
-
 /*
  *     Virtual memory object module.
  */
 
 /*
  *     Virtual memory object module.
  */
 
+#include "ddb.h"
 #include "param.h"
 #include "param.h"
-#include "systm.h"
 #include "malloc.h"
 #include "malloc.h"
+#include "systm.h"
 
 #include "vm.h"
 #include "vm_page.h"
 
 #include "vm.h"
 #include "vm_page.h"
-#include "ddb.h"
+#include "proc.h"
+
 
 static void _vm_object_allocate(vm_size_t, vm_object_t);
 
 static void _vm_object_allocate(vm_size_t, vm_object_t);
-static void vm_object_deactivate_pages(vm_object_t);
+void vm_object_deactivate_pages(vm_object_t);
 static void vm_object_cache_trim(void);
 static void vm_object_remove(vm_pager_t);
 
 static void vm_object_cache_trim(void);
 static void vm_object_remove(vm_pager_t);
 
@@ -112,6 +108,7 @@ static void vm_object_remove(vm_pager_t);
  *
  */
 
  *
  */
 
+
 queue_head_t   vm_object_cached_list;  /* list of objects persisting */
 int            vm_object_cached;       /* size of cached list */
 simple_lock_data_t     vm_cache_lock;  /* lock for object cache */
 queue_head_t   vm_object_cached_list;  /* list of objects persisting */
 int            vm_object_cached;       /* size of cached list */
 simple_lock_data_t     vm_cache_lock;  /* lock for object cache */
@@ -122,25 +119,64 @@ simple_lock_data_t        vm_object_list_lock;
                                        /* lock for object list and count */
 
 vm_object_t    kernel_object;          /* the single kernel object */
                                        /* lock for object list and count */
 
 vm_object_t    kernel_object;          /* the single kernel object */
-vm_object_t    kmem_object;
-
+vm_object_t    kmem_object;            /* the kernel malloc object */
 struct vm_object       kernel_object_store;
 struct vm_object       kmem_object_store;
 
 struct vm_object       kernel_object_store;
 struct vm_object       kmem_object_store;
 
-#define        VM_OBJECT_HASH_COUNT    157
 
 
-int            vm_cache_max = 100;     /* can patch if necessary */
+#ifdef VSMALL
+#define        VM_OBJECT_HASH_COUNT    127     
+int            vm_cache_max = 256;     /* can patch if necessary */
+#else
+#define        VM_OBJECT_HASH_COUNT    521
+int            vm_cache_max = 2048;    /* can patch if necessary */
+#endif
+
 queue_head_t   vm_object_hashtable[VM_OBJECT_HASH_COUNT];
 
 long   object_collapses = 0;
 long   object_bypasses  = 0;
 
 queue_head_t   vm_object_hashtable[VM_OBJECT_HASH_COUNT];
 
 long   object_collapses = 0;
 long   object_bypasses  = 0;
 
+/*
+ * internal version of vm_object_allocate
+ */
+static inline void
+_vm_object_allocate(size, object)
+       vm_size_t               size;
+       register vm_object_t    object;
+{
+       queue_init(&object->memq);
+       vm_object_lock_init(object);
+       object->ref_count = 1;
+       object->resident_page_count = 0;
+       object->size = size;
+       object->can_persist = FALSE;
+       object->paging_in_progress = 0;
+       object->copy = NULL;
+
+       /*
+        *      Object starts out read-write, with no pager.
+        */
+
+       object->pager = NULL;
+       object->internal = TRUE;        /* vm_allocate_with_pager will reset */
+       object->paging_offset = 0;
+       object->shadow = NULL;
+       object->shadow_offset = (vm_offset_t) 0;
+
+       simple_lock(&vm_object_list_lock);
+       queue_enter(&vm_object_list, object, vm_object_t, object_list);
+       vm_object_count++;
+       simple_unlock(&vm_object_list_lock);
+}
+
 /*
  *     vm_object_init:
  *
  *     Initialize the VM objects module.
  */
 /*
  *     vm_object_init:
  *
  *     Initialize the VM objects module.
  */
-void vm_object_init()
+void
+vm_object_init()
 {
        register int    i;
 
 {
        register int    i;
 
@@ -158,7 +194,8 @@ void vm_object_init()
                        kernel_object);
 
        kmem_object = &kmem_object_store;
                        kernel_object);
 
        kmem_object = &kmem_object_store;
-       _vm_object_allocate(VM_KMEM_SIZE + VM_MBUF_SIZE, kmem_object);
+       _vm_object_allocate(VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS,
+                       kmem_object);
 }
 
 /*
 }
 
 /*
@@ -167,56 +204,39 @@ void vm_object_init()
  *     Returns a new object with the given size.
  */
 
  *     Returns a new object with the given size.
  */
 
-vm_object_t vm_object_allocate(size)
+static struct vm_object *objpool;
+static int objpoolcnt;
+
+vm_object_t
+vm_object_allocate(size)
        vm_size_t       size;
 {
        register vm_object_t    result;
        vm_size_t       size;
 {
        register vm_object_t    result;
-
-       result = (vm_object_t)
-               malloc((u_long)sizeof *result, M_VMOBJ, M_WAITOK);
+       int s;
+
+       if (objpool) {
+               result = objpool;
+               objpool = result->copy;
+               --objpoolcnt;
+       } else {
+               result = (vm_object_t)
+                       malloc((u_long)sizeof *result, M_VMOBJ, M_WAITOK);
+       }
+               
 
        _vm_object_allocate(size, result);
 
        return(result);
 }
 
 
        _vm_object_allocate(size, result);
 
        return(result);
 }
 
-static void
-_vm_object_allocate(size, object)
-       vm_size_t               size;
-       register vm_object_t    object;
-{
-       queue_init(&object->memq);
-       vm_object_lock_init(object);
-       object->ref_count = 1;
-       object->resident_page_count = 0;
-       object->size = size;
-       object->can_persist = FALSE;
-       object->paging_in_progress = 0;
-       object->copy = NULL;
-
-       /*
-        *      Object starts out read-write, with no pager.
-        */
-
-       object->pager = NULL;
-       object->pager_ready = FALSE;
-       object->internal = TRUE;        /* vm_allocate_with_pager will reset */
-       object->paging_offset = 0;
-       object->shadow = NULL;
-       object->shadow_offset = (vm_offset_t) 0;
-
-       simple_lock(&vm_object_list_lock);
-       queue_enter(&vm_object_list, object, vm_object_t, object_list);
-       vm_object_count++;
-       simple_unlock(&vm_object_list_lock);
-}
 
 /*
  *     vm_object_reference:
  *
  *     Gets another reference to the given object.
  */
 
 /*
  *     vm_object_reference:
  *
  *     Gets another reference to the given object.
  */
-void vm_object_reference(object)
+inline void
+vm_object_reference(object)
        register vm_object_t    object;
 {
        if (object == NULL)
        register vm_object_t    object;
 {
        if (object == NULL)
@@ -238,8 +258,9 @@ void vm_object_reference(object)
  *
  *     No object may be locked.
  */
  *
  *     No object may be locked.
  */
-void vm_object_deallocate(object)
-       register vm_object_t    object;
+void
+vm_object_deallocate(object)
+       vm_object_t     object;
 {
        vm_object_t     temp;
 
 {
        vm_object_t     temp;
 
@@ -259,11 +280,11 @@ void vm_object_deallocate(object)
                vm_object_lock(object);
                if (--(object->ref_count) != 0) {
 
                vm_object_lock(object);
                if (--(object->ref_count) != 0) {
 
+                       vm_object_unlock(object);
                        /*
                         *      If there are still references, then
                         *      we are done.
                         */
                        /*
                         *      If there are still references, then
                         *      we are done.
                         */
-                       vm_object_unlock(object);
                        vm_object_cache_unlock();
                        return;
                }
                        vm_object_cache_unlock();
                        return;
                }
@@ -275,48 +296,25 @@ void vm_object_deallocate(object)
                 */
 
                if (object->can_persist) {
                 */
 
                if (object->can_persist) {
-#ifdef DIAGNOSTIC
-                       register vm_page_t      p;
-
-                       /*
-                        * Check for dirty pages in object
-                        * Print warning as this may signify kernel bugs
-                        * pk@cs.few.eur.nl     - 4/15/93
-                        */
-                       p = (vm_page_t) queue_first(&object->memq);
-                       while (!queue_end(&object->memq, (queue_entry_t) p)) {
-                               VM_PAGE_CHECK(p);
-
-                               if (pmap_is_modified(VM_PAGE_TO_PHYS(p)) ||
-                                                               !(p->flags & PG_CLEAN)) {
-
-                                       printf("vm_object_dealloc: persistent object %x isn't clean\n", object);
-                                       goto cant_persist;
-                               }
-
-                               p = (vm_page_t) queue_next(&p->listq);
-                       }
-#endif /* DIAGNOSTIC */
 
                        queue_enter(&vm_object_cached_list, object,
                                vm_object_t, cached_list);
                        vm_object_cached++;
                        vm_object_cache_unlock();
 
 
                        queue_enter(&vm_object_cached_list, object,
                                vm_object_t, cached_list);
                        vm_object_cached++;
                        vm_object_cache_unlock();
 
-                       vm_object_deactivate_pages(object);
+                       /* vm_object_deactivate_pages(object); */
                        vm_object_unlock(object);
 
                        vm_object_cache_trim();
                        return;
                }
                        vm_object_unlock(object);
 
                        vm_object_cache_trim();
                        return;
                }
-       cant_persist:;
 
                /*
                 *      Make sure no one can look us up now.
                 */
                vm_object_remove(object->pager);
                vm_object_cache_unlock();
 
                /*
                 *      Make sure no one can look us up now.
                 */
                vm_object_remove(object->pager);
                vm_object_cache_unlock();
-
+       
                temp = object->shadow;
                vm_object_terminate(object);
                        /* unlocks and deallocates object */
                temp = object->shadow;
                vm_object_terminate(object);
                        /* unlocks and deallocates object */
@@ -324,18 +322,19 @@ void vm_object_deallocate(object)
        }
 }
 
        }
 }
 
-
 /*
  *     vm_object_terminate actually destroys the specified object, freeing
  *     up all previously used resources.
  *
  *     The object must be locked.
  */
 /*
  *     vm_object_terminate actually destroys the specified object, freeing
  *     up all previously used resources.
  *
  *     The object must be locked.
  */
-void vm_object_terminate(object)
+void
+vm_object_terminate(object)
        register vm_object_t    object;
 {
        register vm_page_t      p;
        vm_object_t             shadow_object;
        register vm_object_t    object;
 {
        register vm_page_t      p;
        vm_object_t             shadow_object;
+       int s;
 
        /*
         *      Detach the object from its shadow if we are the shadow's
 
        /*
         *      Detach the object from its shadow if we are the shadow's
@@ -345,10 +344,10 @@ void vm_object_terminate(object)
                vm_object_lock(shadow_object);
                if (shadow_object->copy == object)
                        shadow_object->copy = NULL;
                vm_object_lock(shadow_object);
                if (shadow_object->copy == object)
                        shadow_object->copy = NULL;
-#if 0
+/*
                else if (shadow_object->copy != NULL)
                        panic("vm_object_terminate: copy/shadow inconsistency");
                else if (shadow_object->copy != NULL)
                        panic("vm_object_terminate: copy/shadow inconsistency");
-#endif
+*/
                vm_object_unlock(shadow_object);
        }
 
                vm_object_unlock(shadow_object);
        }
 
@@ -357,10 +356,12 @@ void vm_object_terminate(object)
         *      with the object.
         */
 
         *      with the object.
         */
 
+       s = splhigh();
        while (object->paging_in_progress != 0) {
        while (object->paging_in_progress != 0) {
-               vm_object_sleep((int)object, object, FALSE);
+               vm_object_sleep(object, object, FALSE);
                vm_object_lock(object);
        }
                vm_object_lock(object);
        }
+       splx(s);
 
 
        /*
 
 
        /*
@@ -380,6 +381,7 @@ void vm_object_terminate(object)
                VM_PAGE_CHECK(p);
 
                vm_page_lock_queues();
                VM_PAGE_CHECK(p);
 
                vm_page_lock_queues();
+               s = vm_disable_intr();
                if (p->flags & PG_ACTIVE) {
                        queue_remove(&vm_page_queue_active, p, vm_page_t,
                                                pageq);
                if (p->flags & PG_ACTIVE) {
                        queue_remove(&vm_page_queue_active, p, vm_page_t,
                                                pageq);
@@ -393,6 +395,7 @@ void vm_object_terminate(object)
                        p->flags &= ~PG_INACTIVE;
                        vm_page_inactive_count--;
                }
                        p->flags &= ~PG_INACTIVE;
                        vm_page_inactive_count--;
                }
+               vm_set_intr(s);
                vm_page_unlock_queues();
                p = (vm_page_t) queue_next(&p->listq);
        }
                vm_page_unlock_queues();
                p = (vm_page_t) queue_next(&p->listq);
        }
@@ -413,6 +416,7 @@ void vm_object_terminate(object)
                vm_object_page_clean(object, 0, 0);
                vm_object_unlock(object);
        }
                vm_object_page_clean(object, 0, 0);
                vm_object_unlock(object);
        }
+
        while (!queue_empty(&object->memq)) {
                p = (vm_page_t) queue_first(&object->memq);
 
        while (!queue_empty(&object->memq)) {
                p = (vm_page_t) queue_first(&object->memq);
 
@@ -427,8 +431,9 @@ void vm_object_terminate(object)
         *      Let the pager know object is dead.
         */
 
         *      Let the pager know object is dead.
         */
 
-       if (object->pager != NULL)
+       if (object->pager != NULL) {
                vm_pager_deallocate(object->pager);
                vm_pager_deallocate(object->pager);
+       }
 
 
        simple_lock(&vm_object_list_lock);
 
 
        simple_lock(&vm_object_list_lock);
@@ -440,7 +445,13 @@ void vm_object_terminate(object)
         *      Free the space for the object.
         */
 
         *      Free the space for the object.
         */
 
-       free((caddr_t)object, M_VMOBJ);
+       if (objpoolcnt < 64) {
+               object->copy = objpool;
+               objpool = object;
+               ++objpoolcnt;
+               return;
+       } else
+               free((caddr_t)object, M_VMOBJ);
 }
 
 /*
 }
 
 /*
@@ -460,32 +471,55 @@ vm_object_page_clean(object, start, end)
        register vm_offset_t    end;
 {
        register vm_page_t      p;
        register vm_offset_t    end;
 {
        register vm_page_t      p;
+       int s;
+       int size;
 
        if (object->pager == NULL)
                return;
 
 
        if (object->pager == NULL)
                return;
 
+       if (start != end) {
+               start = trunc_page(start);
+               end = round_page(end);
+       }
+       size = end - start;
+
 again:
 again:
+       s = splimp();
        p = (vm_page_t) queue_first(&object->memq);
        p = (vm_page_t) queue_first(&object->memq);
-       while (!queue_end(&object->memq, (queue_entry_t) p)) {
-               if (start == end ||
-                   p->offset >= start && p->offset < end) {
-                       if ((p->flags & PG_CLEAN) && pmap_is_modified(VM_PAGE_TO_PHYS(p)))
+       while (!queue_end(&object->memq, (queue_entry_t) p) && ((start == end) || (size != 0) ) ) {
+               if (start == end || (p->offset >= start && p->offset < end)) {
+                       if (pmap_is_wired(VM_PAGE_TO_PHYS(p)) ||
+                               p->flags & PG_BUSY)
+                               goto next;
+
+                       size -= PAGE_SIZE;
+
+                       if ((p->flags & PG_CLEAN)
+                                && pmap_is_modified(VM_PAGE_TO_PHYS(p)))
                                p->flags &= ~PG_CLEAN;
                                p->flags &= ~PG_CLEAN;
+
+                       if (p->flags & PG_ACTIVE)
+                               vm_page_deactivate(p);
                        pmap_page_protect(VM_PAGE_TO_PHYS(p), VM_PROT_NONE);
                        pmap_page_protect(VM_PAGE_TO_PHYS(p), VM_PROT_NONE);
-                       if (!(p->flags & PG_CLEAN)) {
+                       if ((p->flags & PG_CLEAN) == 0) {
                                p->flags |= PG_BUSY;
                                object->paging_in_progress++;
                                vm_object_unlock(object);
                                (void) vm_pager_put(object->pager, p, TRUE);
                                vm_object_lock(object);
                                object->paging_in_progress--;
                                p->flags |= PG_BUSY;
                                object->paging_in_progress++;
                                vm_object_unlock(object);
                                (void) vm_pager_put(object->pager, p, TRUE);
                                vm_object_lock(object);
                                object->paging_in_progress--;
-                               p->flags &= ~PG_BUSY;
+                               if (object->paging_in_progress == 0)
+                                       wakeup((caddr_t) object);
                                PAGE_WAKEUP(p);
                                PAGE_WAKEUP(p);
+                               splx(s);
                                goto again;
                        }
                }
                                goto again;
                        }
                }
+next:
                p = (vm_page_t) queue_next(&p->listq);
        }
                p = (vm_page_t) queue_next(&p->listq);
        }
+       splx(s);
+       wakeup((caddr_t)object);
 }
 
 /*
 }
 
 /*
@@ -496,17 +530,20 @@ again:
  *
  *     The object must be locked.
  */
  *
  *     The object must be locked.
  */
-static void
+void
 vm_object_deactivate_pages(object)
        register vm_object_t    object;
 {
        register vm_page_t      p, next;
 
 vm_object_deactivate_pages(object)
        register vm_object_t    object;
 {
        register vm_page_t      p, next;
 
+       int s = splimp();
        p = (vm_page_t) queue_first(&object->memq);
        while (!queue_end(&object->memq, (queue_entry_t) p)) {
                next = (vm_page_t) queue_next(&p->listq);
                vm_page_lock_queues();
        p = (vm_page_t) queue_first(&object->memq);
        while (!queue_end(&object->memq, (queue_entry_t) p)) {
                next = (vm_page_t) queue_next(&p->listq);
                vm_page_lock_queues();
-               if (!(p->flags & PG_BUSY))
+               if ((p->flags & (PG_INACTIVE|PG_BUSY)) == 0 &&
+                       p->wire_count == 0 &&
+                       (object->can_persist || !pmap_is_wired(VM_PAGE_TO_PHYS(p))))
                        vm_page_deactivate(p);  /* optimisation from mach 3.0 -
                                                 * andrew@werple.apana.org.au,
                                                 * Feb '93
                        vm_page_deactivate(p);  /* optimisation from mach 3.0 -
                                                 * andrew@werple.apana.org.au,
                                                 * Feb '93
@@ -514,12 +551,13 @@ vm_object_deactivate_pages(object)
                vm_page_unlock_queues();
                p = next;
        }
                vm_page_unlock_queues();
                p = next;
        }
+       splx(s);
 }
 
 /*
  *     Trim the object cache to size.
  */
 }
 
 /*
  *     Trim the object cache to size.
  */
-static void
+void
 vm_object_cache_trim()
 {
        register vm_object_t    object;
 vm_object_cache_trim()
 {
        register vm_object_t    object;
@@ -539,7 +577,6 @@ vm_object_cache_trim()
        vm_object_cache_unlock();
 }
 
        vm_object_cache_unlock();
 }
 
-
 /*
  *     vm_object_shutdown()
  *
 /*
  *     vm_object_shutdown()
  *
@@ -552,7 +589,9 @@ vm_object_cache_trim()
  *     race conditions!
  */
 
  *     race conditions!
  */
 
-void vm_object_shutdown()
+#if 0
+void
+vm_object_shutdown()
 {
        register vm_object_t    object;
 
 {
        register vm_object_t    object;
 
@@ -563,7 +602,6 @@ void vm_object_shutdown()
 
        vm_object_cache_clear();
 
 
        vm_object_cache_clear();
 
-       printf("free paging spaces: ");
 
        /*
         *      First we gain a reference to each object so that
 
        /*
         *      First we gain a reference to each object so that
@@ -594,7 +632,7 @@ void vm_object_shutdown()
        }
        printf("done.\n");
 }
        }
        printf("done.\n");
 }
-
+#endif
 /*
  *     vm_object_pmap_copy:
  *
 /*
  *     vm_object_pmap_copy:
  *
@@ -604,12 +642,19 @@ void vm_object_shutdown()
  *
  *     The object must *not* be locked.
  */
  *
  *     The object must *not* be locked.
  */
-void vm_object_pmap_copy(object, start, end)
+void
+vm_object_pmap_copy(object, start, end)
        register vm_object_t    object;
        register vm_offset_t    start;
        register vm_offset_t    end;
 {
        register vm_page_t      p;
        register vm_object_t    object;
        register vm_offset_t    start;
        register vm_offset_t    end;
 {
        register vm_page_t      p;
+       vm_offset_t amount;
+
+       start = trunc_page(start);
+       end = round_page(end);
+
+       amount = ((end - start) + PAGE_SIZE - 1) / PAGE_SIZE;
 
        if (object == NULL)
                return;
 
        if (object == NULL)
                return;
@@ -620,6 +665,9 @@ void vm_object_pmap_copy(object, start, end)
                if ((start <= p->offset) && (p->offset < end)) {
                        pmap_page_protect(VM_PAGE_TO_PHYS(p), VM_PROT_READ);
                        p->flags |= PG_COPY_ON_WRITE;
                if ((start <= p->offset) && (p->offset < end)) {
                        pmap_page_protect(VM_PAGE_TO_PHYS(p), VM_PROT_READ);
                        p->flags |= PG_COPY_ON_WRITE;
+                       amount -= 1;
+                       if (amount <= 0)
+                               break;
                }
                p = (vm_page_t) queue_next(&p->listq);
        }
                }
                p = (vm_page_t) queue_next(&p->listq);
        }
@@ -634,12 +682,14 @@ void vm_object_pmap_copy(object, start, end)
  *
  *     The object must *not* be locked.
  */
  *
  *     The object must *not* be locked.
  */
-void vm_object_pmap_remove(object, start, end)
+void
+vm_object_pmap_remove(object, start, end)
        register vm_object_t    object;
        register vm_offset_t    start;
        register vm_offset_t    end;
 {
        register vm_page_t      p;
        register vm_object_t    object;
        register vm_offset_t    start;
        register vm_offset_t    end;
 {
        register vm_page_t      p;
+       vm_offset_t size = ((end - start) + PAGE_SIZE - 1) / PAGE_SIZE;
 
        if (object == NULL)
                return;
 
        if (object == NULL)
                return;
@@ -647,13 +697,50 @@ void vm_object_pmap_remove(object, start, end)
        vm_object_lock(object);
        p = (vm_page_t) queue_first(&object->memq);
        while (!queue_end(&object->memq, (queue_entry_t) p)) {
        vm_object_lock(object);
        p = (vm_page_t) queue_first(&object->memq);
        while (!queue_end(&object->memq, (queue_entry_t) p)) {
-               if ((start <= p->offset) && (p->offset < end))
+               if ((start <= p->offset) && (p->offset < end)) {
+                       if ((p->flags & PG_CLEAN) 
+                                && pmap_is_modified(VM_PAGE_TO_PHYS(p))) {
+                               p->flags &= ~PG_CLEAN;
+                       }
+                       if ((p->flags & PG_CLEAN) == 0)
+                               p->flags |= PG_LAUNDRY;
                        pmap_page_protect(VM_PAGE_TO_PHYS(p), VM_PROT_NONE);
                        pmap_page_protect(VM_PAGE_TO_PHYS(p), VM_PROT_NONE);
+                       if (--size <= 0) break;
+               }
                p = (vm_page_t) queue_next(&p->listq);
        }
        vm_object_unlock(object);
 }
 
                p = (vm_page_t) queue_next(&p->listq);
        }
        vm_object_unlock(object);
 }
 
+void
+vm_object_save_pmap_attributes(vm_object_t object,
+       vm_offset_t start, vm_offset_t end) {
+
+       vm_page_t p;
+
+       if (!object)
+               return;
+
+       if (object->shadow) {
+               vm_object_save_pmap_attributes(object->shadow,
+                       object->shadow_offset + start,
+                       object->shadow_offset + end);
+       }
+
+       p = (vm_page_t) queue_first(&object->memq);
+       while (!queue_end(&object->memq, (queue_entry_t) p)) {
+               if ((start <= p->offset) && (p->offset < end)) {
+                       if ((p->flags & PG_CLEAN) 
+                                && pmap_is_modified(VM_PAGE_TO_PHYS(p))) {
+                               p->flags &= ~PG_CLEAN;
+                       }
+                       if ((p->flags & PG_CLEAN) == 0)
+                               p->flags |= PG_LAUNDRY;
+               }
+               p = (vm_page_t) queue_next(&p->listq);
+       }
+}
+
 /*
  *     vm_object_copy:
  *
 /*
  *     vm_object_copy:
  *
@@ -665,8 +752,8 @@ void vm_object_pmap_remove(object, start, end)
  *     May defer the copy until later if the object is not backed
  *     up by a non-default pager.
  */
  *     May defer the copy until later if the object is not backed
  *     up by a non-default pager.
  */
-void vm_object_copy(src_object, src_offset, size,
-                   dst_object, dst_offset, src_needs_copy)
+void
+vm_object_copy(src_object, src_offset, size, dst_object, dst_offset, src_needs_copy)
        register vm_object_t    src_object;
        vm_offset_t             src_offset;
        vm_size_t               size;
        register vm_object_t    src_object;
        vm_offset_t             src_offset;
        vm_size_t               size;
@@ -677,6 +764,8 @@ void vm_object_copy(src_object, src_offset, size,
        register vm_object_t    new_copy;
        register vm_object_t    old_copy;
        vm_offset_t             new_start, new_end;
        register vm_object_t    new_copy;
        register vm_object_t    old_copy;
        vm_offset_t             new_start, new_end;
+       vm_offset_t             src_offset_end;
+       vm_offset_t             tmpsize;
 
        register vm_page_t      p;
 
 
        register vm_page_t      p;
 
@@ -695,6 +784,7 @@ void vm_object_copy(src_object, src_offset, size,
         *      default pager, we don't have to make a copy
         *      of it.  Instead, we set the needs copy flag and
         *      make a shadow later.
         *      default pager, we don't have to make a copy
         *      of it.  Instead, we set the needs copy flag and
         *      make a shadow later.
+        *      DYSON: check for swap(default) pager too....
         */
 
        vm_object_lock(src_object);
         */
 
        vm_object_lock(src_object);
@@ -702,18 +792,12 @@ void vm_object_copy(src_object, src_offset, size,
        /*
         *      Try to collapse the object before copying it.
         */
        /*
         *      Try to collapse the object before copying it.
         */
-       vm_object_collapse(src_object);
 
 
+       vm_object_collapse(src_object);
 
 
-/*
- * we can simply add a reference to the object if we have no
- * pager or are using the swap pager or we have an internal object
- *
- * John Dyson 24 Nov 93
- */
        if (src_object->pager == NULL ||
        if (src_object->pager == NULL ||
-           (src_object->pager->pg_type == PG_SWAP) ||
-               src_object->internal) {
+           src_object->pager->pg_type == PG_SWAP ||  
+           src_object->internal) {
 
                /*
                 *      Make another reference to the object
 
                /*
                 *      Make another reference to the object
@@ -723,12 +807,16 @@ void vm_object_copy(src_object, src_offset, size,
                /*
                 *      Mark all of the pages copy-on-write.
                 */
                /*
                 *      Mark all of the pages copy-on-write.
                 */
+               tmpsize = size;
+               src_offset_end = src_offset + size;
                for (p = (vm_page_t) queue_first(&src_object->memq);
                for (p = (vm_page_t) queue_first(&src_object->memq);
-                    !queue_end(&src_object->memq, (queue_entry_t)p);
+                    !queue_end(&src_object->memq, (queue_entry_t)p) && tmpsize > 0;
                     p = (vm_page_t) queue_next(&p->listq)) {
                        if (src_offset <= p->offset &&
                     p = (vm_page_t) queue_next(&p->listq)) {
                        if (src_offset <= p->offset &&
-                           p->offset < src_offset + size)
+                           p->offset < src_offset_end) {
                                p->flags |= PG_COPY_ON_WRITE;
                                p->flags |= PG_COPY_ON_WRITE;
+                               tmpsize -= PAGE_SIZE;
+                           }
                }
                vm_object_unlock(src_object);
 
                }
                vm_object_unlock(src_object);
 
@@ -849,10 +937,13 @@ void vm_object_copy(src_object, src_offset, size,
         *      Mark all the affected pages of the existing object
         *      copy-on-write.
         */
         *      Mark all the affected pages of the existing object
         *      copy-on-write.
         */
+       tmpsize = size;
        p = (vm_page_t) queue_first(&src_object->memq);
        p = (vm_page_t) queue_first(&src_object->memq);
-       while (!queue_end(&src_object->memq, (queue_entry_t) p)) {
-               if ((new_start <= p->offset) && (p->offset < new_end))
+       while (!queue_end(&src_object->memq, (queue_entry_t) p) && tmpsize > 0) {
+               if ((new_start <= p->offset) && (p->offset < new_end)) {
                        p->flags |= PG_COPY_ON_WRITE;
                        p->flags |= PG_COPY_ON_WRITE;
+                       tmpsize -= PAGE_SIZE;
+               }
                p = (vm_page_t) queue_next(&p->listq);
        }
 
                p = (vm_page_t) queue_next(&p->listq);
        }
 
@@ -874,7 +965,8 @@ void vm_object_copy(src_object, src_offset, size,
  *     are returned in the source parameters.
  */
 
  *     are returned in the source parameters.
  */
 
-void vm_object_shadow(object, offset, length)
+void
+vm_object_shadow(object, offset, length)
        vm_object_t     *object;        /* IN/OUT */
        vm_offset_t     *offset;        /* IN/OUT */
        vm_size_t       length;
        vm_object_t     *object;        /* IN/OUT */
        vm_offset_t     *offset;        /* IN/OUT */
        vm_size_t       length;
@@ -919,7 +1011,8 @@ void vm_object_shadow(object, offset, length)
  *     Set the specified object's pager to the specified pager.
  */
 
  *     Set the specified object's pager to the specified pager.
  */
 
-void vm_object_setpager(object, pager, paging_offset,
+void
+vm_object_setpager(object, pager, paging_offset,
                        read_only)
        vm_object_t     object;
        vm_pager_t      pager;
                        read_only)
        vm_object_t     object;
        vm_pager_t      pager;
@@ -931,6 +1024,9 @@ void vm_object_setpager(object, pager, paging_offset,
 #endif lint
 
        vm_object_lock(object);                 /* XXX ? */
 #endif lint
 
        vm_object_lock(object);                 /* XXX ? */
+       if (object->pager && object->pager != pager) {
+               panic("!!!pager already allocated!!!\n");
+       }
        object->pager = pager;
        object->paging_offset = paging_offset;
        vm_object_unlock(object);                       /* XXX ? */
        object->pager = pager;
        object->paging_offset = paging_offset;
        vm_object_unlock(object);                       /* XXX ? */
@@ -941,14 +1037,15 @@ void vm_object_setpager(object, pager, paging_offset,
  */
 
 #define vm_object_hash(pager) \
  */
 
 #define vm_object_hash(pager) \
-       (((unsigned)pager)%VM_OBJECT_HASH_COUNT)
+       ((((unsigned)pager) >> 5)%VM_OBJECT_HASH_COUNT)
 
 /*
  *     vm_object_lookup looks in the object cache for an object with the
  *     specified pager and paging id.
  */
 
 
 /*
  *     vm_object_lookup looks in the object cache for an object with the
  *     specified pager and paging id.
  */
 
-vm_object_t vm_object_lookup(pager)
+vm_object_t
+vm_object_lookup(pager)
        vm_pager_t      pager;
 {
        register queue_t                bucket;
        vm_pager_t      pager;
 {
        register queue_t                bucket;
@@ -986,7 +1083,8 @@ vm_object_t vm_object_lookup(pager)
  *     the hash table.
  */
 
  *     the hash table.
  */
 
-void vm_object_enter(object, pager)
+void
+vm_object_enter(object, pager)
        vm_object_t     object;
        vm_pager_t      pager;
 {
        vm_object_t     object;
        vm_pager_t      pager;
 {
@@ -1022,7 +1120,7 @@ void vm_object_enter(object, pager)
  *     is locked.  XXX this should be fixed
  *     by reorganizing vm_object_deallocate.
  */
  *     is locked.  XXX this should be fixed
  *     by reorganizing vm_object_deallocate.
  */
-static void
+void
 vm_object_remove(pager)
        register vm_pager_t     pager;
 {
 vm_object_remove(pager)
        register vm_pager_t     pager;
 {
@@ -1049,8 +1147,8 @@ vm_object_remove(pager)
  *     vm_object_cache_clear removes all objects from the cache.
  *
  */
  *     vm_object_cache_clear removes all objects from the cache.
  *
  */
-
-void vm_object_cache_clear()
+void
+vm_object_cache_clear()
 {
        register vm_object_t    object;
 
 {
        register vm_object_t    object;
 
@@ -1089,18 +1187,12 @@ boolean_t       vm_object_collapse_allowed = TRUE;
  *     Requires that the object be locked and the page
  *     queues be unlocked.
  *
  *     Requires that the object be locked and the page
  *     queues be unlocked.
  *
- *     This routine substantially modified by John S. Dyson
- *     Dec 18,1993.  Many restrictions have been removed from
- *     this routine including:
- *
- *     1)      The object CAN now have a pager
- *     2)      Backing object pager space is now removed if not needed
- *     3)      Bypasses now check for existance of pages on paging space
+ *     This routine has significant changes by John S. Dyson
+ *     to fix some swap memory leaks.  18 Dec 93
  *
  */
  *
  */
-
-
-void vm_object_collapse(object)
+void
+vm_object_collapse(object)
        register vm_object_t    object;
 
 {
        register vm_object_t    object;
 
 {
@@ -1165,7 +1257,7 @@ void vm_object_collapse(object)
                /*
                 * we can deal only with the swap pager
                 */
                /*
                 * we can deal only with the swap pager
                 */
-               if(object->pager && 
+               if ((object->pager && 
                        object->pager->pg_type != PG_SWAP) ||
                    (backing_object->pager && 
                        backing_object->pager->pg_type != PG_SWAP)) {
                        object->pager->pg_type != PG_SWAP) ||
                    (backing_object->pager && 
                        backing_object->pager->pg_type != PG_SWAP)) {
@@ -1222,17 +1314,12 @@ void vm_object_collapse(object)
                                        vm_page_unlock_queues();
                                } else {
                                    pp = vm_page_lookup(object, new_offset);
                                        vm_page_unlock_queues();
                                } else {
                                    pp = vm_page_lookup(object, new_offset);
-                                   if (pp != NULL) {
+                                   if (pp != NULL || (object->pager && vm_pager_has_page(object->pager,
+                                               object->paging_offset + new_offset))) {
                                        vm_page_lock_queues();
                                        vm_page_free(p);
                                        vm_page_unlock_queues();
                                        vm_page_lock_queues();
                                        vm_page_free(p);
                                        vm_page_unlock_queues();
-                                   }
-                                   else {
-                                       /*
-                                        *      Parent now has no page.
-                                        *      Move the backing object's page
-                                        *      up.
-                                        */
+                                   } else {
                                        vm_page_rename(p, object, new_offset);
                                    }
                                }
                                        vm_page_rename(p, object, new_offset);
                                    }
                                }
@@ -1242,39 +1329,47 @@ void vm_object_collapse(object)
                         *      Move the pager from backing_object to object.
                         */
 
                         *      Move the pager from backing_object to object.
                         */
 
-                       if( backing_object->pager) {
-                               if( object->pager) {
+                       if (backing_object->pager) {
+                               backing_object->paging_in_progress++;
+                               if (object->pager) {
+                                       vm_pager_t bopager;
                                        object->paging_in_progress++;
                                        object->paging_in_progress++;
-                                       backing_object->paging_in_progress++;
                                        /*
                                         * copy shadow object pages into ours
                                         * and destroy unneeded pages in shadow object.
                                         */
                                        /*
                                         * copy shadow object pages into ours
                                         * and destroy unneeded pages in shadow object.
                                         */
+                                       bopager = backing_object->pager;
+                                       backing_object->pager = NULL;
+                                       vm_object_remove(backing_object->pager);
                                        swap_pager_copy(
                                        swap_pager_copy(
-                                               backing_object->pager, backing_object->paging_offset,
+                                               bopager, backing_object->paging_offset,
                                                object->pager, object->paging_offset,
                                                object->shadow_offset);
                                        object->paging_in_progress--;
                                                object->pager, object->paging_offset,
                                                object->shadow_offset);
                                        object->paging_in_progress--;
-                                       ifobject->paging_in_progress == 0)
+                                       if (object->paging_in_progress == 0)
                                                wakeup((caddr_t)object);
                                                wakeup((caddr_t)object);
-                                       backing_object->paging_in_progress--;
-                                       if( backing_object->paging_in_progress == 0)
-                                               wakeup((caddr_t)backing_object);
                                } else {
                                } else {
+                                       object->paging_in_progress++;
                                        /*
                                         * grab the shadow objects pager
                                         */
                                        object->pager = backing_object->pager;
                                        object->paging_offset = backing_object->paging_offset + backing_offset;
                                        /*
                                         * grab the shadow objects pager
                                         */
                                        object->pager = backing_object->pager;
                                        object->paging_offset = backing_object->paging_offset + backing_offset;
+                                       vm_object_remove(backing_object->pager);
+                                       backing_object->pager = NULL;
                                        /*
                                         * free unnecessary blocks
                                         */
                                        /*
                                         * free unnecessary blocks
                                         */
-                                       swap_pager_freespace( object->pager, 0, object->paging_offset);
+                                       swap_pager_freespace(object->pager, 0, object->paging_offset);
+                                       object->paging_in_progress--;
+                                       if (object->paging_in_progress == 0)
+                                               wakeup((caddr_t)object);
                                }
                                }
-                               vm_object_remove( backing_object->pager);
+                               backing_object->paging_in_progress--;
+                               if (backing_object->paging_in_progress == 0)
+                                       wakeup((caddr_t)backing_object);
                        }
 
                        }
 
-                       backing_object->pager = NULL;
 
                        /*
                         *      Object now shadows whatever backing_object did.
 
                        /*
                         *      Object now shadows whatever backing_object did.
@@ -1348,10 +1443,8 @@ void vm_object_collapse(object)
 
                                if (p->offset >= backing_offset &&
                                    new_offset <= size &&
 
                                if (p->offset >= backing_offset &&
                                    new_offset <= size &&
-                                   ((pp = vm_page_lookup(object, new_offset)) == NULL ||
-                                       (pp->flags & PG_FAKE)) &&
-                                       (!object->pager ||
-                                       !vm_pager_has_page( object->pager, object->paging_offset+new_offset))) {
+                                   ((pp = vm_page_lookup(object, new_offset)) == NULL || (pp->flags & PG_FAKE)) &&
+                                       (!object->pager || !vm_pager_has_page(object->pager, object->paging_offset+new_offset))) {
                                        /*
                                         *      Page still needed.
                                         *      Can't go any further.
                                        /*
                                         *      Page still needed.
                                         *      Can't go any further.
@@ -1389,7 +1482,7 @@ void vm_object_collapse(object)
                         *      will not vanish; so we don't need to call
                         *      vm_object_deallocate.
                         */
                         *      will not vanish; so we don't need to call
                         *      vm_object_deallocate.
                         */
-                       ifbacking_object->ref_count == 1)
+                       if (backing_object->ref_count == 1)
                                printf("should have called obj deallocate\n");
                        backing_object->ref_count--;
                        vm_object_unlock(backing_object);
                                printf("should have called obj deallocate\n");
                        backing_object->ref_count--;
                        vm_object_unlock(backing_object);
@@ -1404,7 +1497,6 @@ void vm_object_collapse(object)
        }
 }
 
        }
 }
 
-
 /*
  *     vm_object_page_remove: [internal]
  *
 /*
  *     vm_object_page_remove: [internal]
  *
@@ -1413,26 +1505,47 @@ void vm_object_collapse(object)
  *
  *     The object must be locked.
  */
  *
  *     The object must be locked.
  */
-void vm_object_page_remove(object, start, end)
+void
+vm_object_page_remove(object, start, end)
        register vm_object_t    object;
        register vm_offset_t    start;
        register vm_offset_t    end;
 {
        register vm_page_t      p, next;
        register vm_object_t    object;
        register vm_offset_t    start;
        register vm_offset_t    end;
 {
        register vm_page_t      p, next;
+       vm_offset_t size;
+       int cnt;
+       int s;
 
        if (object == NULL)
                return;
 
 
        if (object == NULL)
                return;
 
-       p = (vm_page_t) queue_first(&object->memq);
-       while (!queue_end(&object->memq, (queue_entry_t) p)) {
-               next = (vm_page_t) queue_next(&p->listq);
-               if ((start <= p->offset) && (p->offset < end)) {
-                       pmap_page_protect(VM_PAGE_TO_PHYS(p), VM_PROT_NONE);
-                       vm_page_lock_queues();
-                       vm_page_free(p);
-                       vm_page_unlock_queues();
+       start = trunc_page(start);
+       end = round_page(end);
+       size = end-start;
+       if (size > 4*PAGE_SIZE || size >= object->size/4) {
+               p = (vm_page_t) queue_first(&object->memq);
+               while (!queue_end(&object->memq, (queue_entry_t) p) && size > 0) {
+                       next = (vm_page_t) queue_next(&p->listq);
+                       if ((start <= p->offset) && (p->offset < end)) {
+                               pmap_page_protect(VM_PAGE_TO_PHYS(p), VM_PROT_NONE);
+                               vm_page_lock_queues();
+                               vm_page_free(p);
+                               vm_page_unlock_queues();
+                               size -= PAGE_SIZE;
+                       }
+                       p = next;
+               }
+       } else {
+               while (size > 0) {
+                       while (p = vm_page_lookup(object, start)) {
+                               pmap_page_protect(VM_PAGE_TO_PHYS(p), VM_PROT_NONE);
+                               vm_page_lock_queues();
+                               vm_page_free(p);
+                               vm_page_unlock_queues();
+                       }
+                       start += PAGE_SIZE;
+                       size -= PAGE_SIZE;
                }
                }
-               p = next;
        }
 }
 
        }
 }
 
@@ -1458,10 +1571,8 @@ void vm_object_page_remove(object, start, end)
  *     Conditions:
  *     The object must *not* be locked.
  */
  *     Conditions:
  *     The object must *not* be locked.
  */
-boolean_t vm_object_coalesce(prev_object, next_object,
-                       prev_offset, next_offset,
-                       prev_size, next_size)
-
+boolean_t
+vm_object_coalesce(prev_object, next_object, prev_offset, next_offset, prev_size, next_size)
        register vm_object_t    prev_object;
        vm_object_t     next_object;
        vm_offset_t     prev_offset, next_offset;
        register vm_object_t    prev_object;
        vm_object_t     next_object;
        vm_offset_t     prev_offset, next_offset;
@@ -1511,9 +1622,11 @@ boolean_t vm_object_coalesce(prev_object, next_object,
         *      a previous deallocation.
         */
 
         *      a previous deallocation.
         */
 
+/*
        vm_object_page_remove(prev_object,
                        prev_offset + prev_size,
                        prev_offset + prev_size + next_size);
        vm_object_page_remove(prev_object,
                        prev_offset + prev_size,
                        prev_offset + prev_size + next_size);
+*/
 
        /*
         *      Extend the object if necessary.
 
        /*
         *      Extend the object if necessary.
@@ -1526,11 +1639,31 @@ boolean_t vm_object_coalesce(prev_object, next_object,
        return(TRUE);
 }
 
        return(TRUE);
 }
 
+/*
+ * returns page after looking up in shadow chain
+ */
+vm_page_t
+vm_object_page_lookup(object, offset)
+       vm_object_t object;
+       vm_offset_t offset;
+{
+       vm_page_t m;
+       if (!(m=vm_page_lookup(object, offset))) {
+               if (!object->shadow)
+                       return 0;
+               else
+                       return vm_object_page_lookup(object->shadow, offset + object->shadow_offset);
+       }
+       return m;
+}
+
 #if defined(DEBUG) || (NDDB > 0)
 /*
  *     vm_object_print:        [ debug ]
  */
 #if defined(DEBUG) || (NDDB > 0)
 /*
  *     vm_object_print:        [ debug ]
  */
-void vm_object_print(object, full)
+void
+vm_object_print(object, full)
        vm_object_t     object;
        boolean_t       full;
 {
        vm_object_t     object;
        boolean_t       full;
 {
index 765e83a..06fe937 100644 (file)
@@ -34,7 +34,7 @@
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_object.h   7.3 (Berkeley) 4/21/91
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_object.h   7.3 (Berkeley) 4/21/91
- *     $Id: vm_object.h,v 1.4 1993/11/25 01:39:11 wollman Exp $
+ *     $Id: vm_object.h,v 1.5 1993/12/19 00:56:09 wollman Exp $
  */
 
 /*
  */
 
 /*
@@ -158,16 +158,16 @@ void              vm_object_print();
 #define        vm_object_lock_init(object)     { simple_lock_init(&(object)->Lock); (object)->LockHolder = 0; }
 #define        vm_object_lock(object)          { simple_lock(&(object)->Lock); (object)->LockHolder = (int) current_thread(); }
 #define        vm_object_unlock(object)        { (object)->LockHolder = 0; simple_unlock(&(object)->Lock); }
 #define        vm_object_lock_init(object)     { simple_lock_init(&(object)->Lock); (object)->LockHolder = 0; }
 #define        vm_object_lock(object)          { simple_lock(&(object)->Lock); (object)->LockHolder = (int) current_thread(); }
 #define        vm_object_unlock(object)        { (object)->LockHolder = 0; simple_unlock(&(object)->Lock); }
-#define        vm_object_lock_try(object)      (simple_lock_try(&(object)->Lock) ? ( ((object)->LockHolder = (int) current_thread()) , TRUE) : FALSE)
+#define        vm_object_lock_try(object)      (simple_lock_try(&(object)->Lock) ? (((object)->LockHolder = (int) current_thread()) , TRUE) : FALSE)
 #define        vm_object_sleep(event, object, interruptible) \
 #define        vm_object_sleep(event, object, interruptible) \
-                                       { (object)->LockHolder = 0; thread_sleep((event), &(object)->Lock, (interruptible)); }
+                                       { (object)->LockHolder = 0; thread_sleep((int)(event), &(object)->Lock, (interruptible)); }
 #else  /* VM_OBJECT_DEBUG */
 #define        vm_object_lock_init(object)     simple_lock_init(&(object)->Lock)
 #define        vm_object_lock(object)          simple_lock(&(object)->Lock)
 #define        vm_object_unlock(object)        simple_unlock(&(object)->Lock)
 #define        vm_object_lock_try(object)      simple_lock_try(&(object)->Lock)
 #define        vm_object_sleep(event, object, interruptible) \
 #else  /* VM_OBJECT_DEBUG */
 #define        vm_object_lock_init(object)     simple_lock_init(&(object)->Lock)
 #define        vm_object_lock(object)          simple_lock(&(object)->Lock)
 #define        vm_object_unlock(object)        simple_unlock(&(object)->Lock)
 #define        vm_object_lock_try(object)      simple_lock_try(&(object)->Lock)
 #define        vm_object_sleep(event, object, interruptible) \
-                                       thread_sleep((event), &(object)->Lock, (interruptible))
+                                       thread_sleep((int)(event), &(object)->Lock, (interruptible))
 #endif /* VM_OBJECT_DEBUG */
 
 extern void vm_object_page_clean(vm_object_t, vm_offset_t, vm_offset_t);
 #endif /* VM_OBJECT_DEBUG */
 
 extern void vm_object_page_clean(vm_object_t, vm_offset_t, vm_offset_t);
index dd75b77..ed8d027 100644 (file)
@@ -34,7 +34,7 @@
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_page.c     7.4 (Berkeley) 5/7/91
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_page.c     7.4 (Berkeley) 5/7/91
- *     $Id: vm_page.c,v 1.6 1993/12/21 05:51:04 davidg Exp $
+ *     $Id: vm_page.c,v 1.7 1994/01/03 07:58:07 davidg Exp $
  */
 
 /*
  */
 
 /*
@@ -75,6 +75,7 @@
 #include "vm_map.h"
 #include "vm_page.h"
 #include "vm_pageout.h"
 #include "vm_map.h"
 #include "vm_page.h"
 #include "vm_pageout.h"
+#include "proc.h"
 
 /*
  *     Associated with page of user-allocatable memory is a
 
 /*
  *     Associated with page of user-allocatable memory is a
@@ -107,6 +108,8 @@ int vm_page_active_count;
 int    vm_page_inactive_count;
 int    vm_page_wire_count;
 int    vm_page_laundry_count;
 int    vm_page_inactive_count;
 int    vm_page_wire_count;
 int    vm_page_laundry_count;
+int    vm_page_count;
+extern int vm_pageout_pages_needed;
 
 int    vm_page_free_target = 0;
 int    vm_page_free_min = 0;
 
 int    vm_page_free_target = 0;
 int    vm_page_free_min = 0;
@@ -122,7 +125,8 @@ int vm_page_free_reserved = 0;
  *
  *     Sets page_shift and page_mask from page_size.
  */
  *
  *     Sets page_shift and page_mask from page_size.
  */
-void vm_set_page_size()
+void
+vm_set_page_size()
 {
        page_mask = page_size - 1;
 
 {
        page_mask = page_size - 1;
 
@@ -144,21 +148,55 @@ void vm_set_page_size()
  *     for the object/offset-to-page hash table headers.
  *     Each page cell is initialized and placed on the free list.
  */
  *     for the object/offset-to-page hash table headers.
  *     Each page cell is initialized and placed on the free list.
  */
-vm_offset_t vm_page_startup(start, end, vaddr)
-       register vm_offset_t    start;
-       vm_offset_t     end;
+
+vm_offset_t
+vm_page_startup(starta, enda, vaddr)
+       register vm_offset_t    starta;
+       vm_offset_t     enda;
        register vm_offset_t    vaddr;
 {
        register vm_offset_t    mapped;
        register vm_page_t      m;
        register queue_t        bucket;
        register vm_offset_t    vaddr;
 {
        register vm_offset_t    mapped;
        register vm_page_t      m;
        register queue_t        bucket;
-       vm_size_t               npages;
+       vm_size_t               npages, page_range;
        register vm_offset_t    new_start;
        int                     i;
        vm_offset_t             pa;
        register vm_offset_t    new_start;
        int                     i;
        vm_offset_t             pa;
+       int nblocks;
+       vm_offset_t             first_managed_page;
+       int                     size;
 
        extern  vm_offset_t     kentry_data;
        extern  vm_size_t       kentry_data_size;
 
        extern  vm_offset_t     kentry_data;
        extern  vm_size_t       kentry_data_size;
+       extern vm_offset_t phys_avail[];
+/* the biggest memory array is the second group of pages */
+       vm_offset_t start;
+       vm_offset_t biggestone, biggestsize;
+
+       vm_offset_t total;
+
+       total = 0;
+       biggestsize = 0;
+       biggestone = 0;
+       nblocks = 0;
+       vaddr = round_page(vaddr);
+
+       for (i = 0; phys_avail[i + 1]; i += 2) {
+               phys_avail[i] = round_page(phys_avail[i]);
+               phys_avail[i+1] = trunc_page(phys_avail[i+1]);
+       }
+               
+       for (i = 0; phys_avail[i + 1]; i += 2) {
+               int size = phys_avail[i+1] - phys_avail[i];
+               if (size > biggestsize) {
+                       biggestone = i;
+                       biggestsize = size;
+               }
+               ++nblocks;
+               total += size;
+       }
+
+       start = phys_avail[biggestone];
 
 
        /*
 
 
        /*
@@ -187,22 +225,23 @@ vm_offset_t vm_page_startup(start, end, vaddr)
         *      Note:
         *              This computation can be tweaked if desired.
         */
         *      Note:
         *              This computation can be tweaked if desired.
         */
-
        vm_page_buckets = (queue_t) vaddr;
        bucket = vm_page_buckets;
        if (vm_page_bucket_count == 0) {
                vm_page_bucket_count = 1;
        vm_page_buckets = (queue_t) vaddr;
        bucket = vm_page_buckets;
        if (vm_page_bucket_count == 0) {
                vm_page_bucket_count = 1;
-               while (vm_page_bucket_count < atop(end - start))
+               while (vm_page_bucket_count < atop(total))
                        vm_page_bucket_count <<= 1;
        }
 
                        vm_page_bucket_count <<= 1;
        }
 
+
        vm_page_hash_mask = vm_page_bucket_count - 1;
 
        /*
         *      Validate these addresses.
         */
 
        vm_page_hash_mask = vm_page_bucket_count - 1;
 
        /*
         *      Validate these addresses.
         */
 
-       new_start = round_page(((queue_t)start) + vm_page_bucket_count);
+       new_start = start + vm_page_bucket_count * sizeof(struct queue_entry);
+       new_start = round_page(new_start);
        mapped = vaddr;
        vaddr = pmap_map(mapped, start, new_start,
                        VM_PROT_READ|VM_PROT_WRITE);
        mapped = vaddr;
        vaddr = pmap_map(mapped, start, new_start,
                        VM_PROT_READ|VM_PROT_WRITE);
@@ -210,7 +249,7 @@ vm_offset_t vm_page_startup(start, end, vaddr)
        bzero((caddr_t) mapped, vaddr - mapped);
        mapped = vaddr;
 
        bzero((caddr_t) mapped, vaddr - mapped);
        mapped = vaddr;
 
-       for (i = vm_page_bucket_count; i--;) {
+       for (i = 0; i< vm_page_bucket_count; i++) {
                queue_init(bucket);
                bucket++;
        }
                queue_init(bucket);
                bucket++;
        }
@@ -221,8 +260,6 @@ vm_offset_t vm_page_startup(start, end, vaddr)
         *      round (or truncate) the addresses to our page size.
         */
 
         *      round (or truncate) the addresses to our page size.
         */
 
-       end = trunc_page(end);
-
        /*
         *      Pre-allocate maps and map entries that cannot be dynamically
         *      allocated via malloc().  The maps include the kernel_map and
        /*
         *      Pre-allocate maps and map entries that cannot be dynamically
         *      allocated via malloc().  The maps include the kernel_map and
@@ -248,8 +285,7 @@ vm_offset_t vm_page_startup(start, end, vaddr)
        new_start = start + (vaddr - mapped);
        pmap_map(mapped, start, new_start, VM_PROT_READ|VM_PROT_WRITE);
        bzero((caddr_t) mapped, (vaddr - mapped));
        new_start = start + (vaddr - mapped);
        pmap_map(mapped, start, new_start, VM_PROT_READ|VM_PROT_WRITE);
        bzero((caddr_t) mapped, (vaddr - mapped));
-       mapped = vaddr;
-       start = new_start;
+       start = round_page(new_start);
 
        /*
         *      Compute the number of pages of memory that will be
 
        /*
         *      Compute the number of pages of memory that will be
@@ -257,43 +293,52 @@ vm_offset_t vm_page_startup(start, end, vaddr)
         *      of a page structure per page).
         */
 
         *      of a page structure per page).
         */
 
-       vm_page_free_count = npages =
-               (end - start + sizeof(struct vm_page))/(PAGE_SIZE + sizeof(struct vm_page));
+       npages = (total - (start - phys_avail[biggestone])) / (PAGE_SIZE + sizeof(struct vm_page));
+       first_page = phys_avail[0] / PAGE_SIZE;
 
 
+       page_range = (phys_avail[(nblocks-1)*2 + 1] - phys_avail[0]) / PAGE_SIZE;
        /*
         *      Initialize the mem entry structures now, and
         *      put them in the free queue.
         */
 
        /*
         *      Initialize the mem entry structures now, and
         *      put them in the free queue.
         */
 
-       m = vm_page_array = (vm_page_t) vaddr;
-       first_page = start;
-       first_page += npages*sizeof(struct vm_page);
-       first_page = atop(round_page(first_page));
-       last_page  = first_page + npages - 1;
+       vm_page_array = (vm_page_t) vaddr;
+       mapped = vaddr;
 
 
-       first_phys_addr = ptoa(first_page);
-       last_phys_addr  = ptoa(last_page) + page_mask;
 
        /*
         *      Validate these addresses.
         */
 
 
        /*
         *      Validate these addresses.
         */
 
-       new_start = start + (round_page(m + npages) - mapped);
+       new_start = round_page(start + page_range * sizeof (struct vm_page));
        mapped = pmap_map(mapped, start, new_start,
                        VM_PROT_READ|VM_PROT_WRITE);
        start = new_start;
 
        mapped = pmap_map(mapped, start, new_start,
                        VM_PROT_READ|VM_PROT_WRITE);
        start = new_start;
 
+       first_managed_page = start / PAGE_SIZE;
+
        /*
         *      Clear all of the page structures
         */
        /*
         *      Clear all of the page structures
         */
-       bzero((caddr_t)m, npages * sizeof(*m));
-
-       pa = first_phys_addr;
-       while (npages--) {
-               m->phys_addr = pa;
-               queue_enter(&vm_page_queue_free, m, vm_page_t, pageq);
-               m++;
-               pa += PAGE_SIZE;
+       bzero((caddr_t)vm_page_array, page_range * sizeof(struct vm_page));
+
+       vm_page_count = 0;
+       vm_page_free_count = 0;
+       for (i = 0; phys_avail[i + 1] && npages > 0; i += 2) {
+               if (i == biggestone) 
+                       pa = ptoa(first_managed_page);
+               else
+                       pa = phys_avail[i];
+               while (pa < phys_avail[i + 1] && npages-- > 0) {
+                       ++vm_page_count;
+                       ++vm_page_free_count;
+                       m = PHYS_TO_VM_PAGE(pa);
+                       m->flags = 0;
+                       m->object = 0;
+                       m->phys_addr = pa;
+                       queue_enter(&vm_page_queue_free, m, vm_page_t, pageq);
+                       pa += PAGE_SIZE;
+               }
        }
 
        /*
        }
 
        /*
@@ -312,8 +357,13 @@ vm_offset_t vm_page_startup(start, end, vaddr)
  *
  *     NOTE:  This macro depends on vm_page_bucket_count being a power of 2.
  */
  *
  *     NOTE:  This macro depends on vm_page_bucket_count being a power of 2.
  */
-#define vm_page_hash(object, offset) \
-       (((unsigned)object+(unsigned)atop(offset))&vm_page_hash_mask)
+inline const int
+vm_page_hash(object, offset)
+       vm_object_t object;
+       vm_offset_t offset;
+{
+       return ((unsigned)object + offset/NBPG) & vm_page_hash_mask;
+}
 
 /*
  *     vm_page_insert:         [ internal use only ]
 
 /*
  *     vm_page_insert:         [ internal use only ]
@@ -322,15 +372,18 @@ vm_offset_t vm_page_startup(start, end, vaddr)
  *     table and object list.
  *
  *     The object and page must be locked.
  *     table and object list.
  *
  *     The object and page must be locked.
+ * interrupts must be disable in this routine!!!
  */
 
  */
 
-void vm_page_insert(mem, object, offset)
+void
+vm_page_insert(mem, object, offset)
        register vm_page_t      mem;
        register vm_object_t    object;
        register vm_offset_t    offset;
 {
        register queue_t        bucket;
        int                     spl;
        register vm_page_t      mem;
        register vm_object_t    object;
        register vm_offset_t    offset;
 {
        register queue_t        bucket;
        int                     spl;
+       vm_page_t tmpm;
 
        VM_PAGE_CHECK(mem);
 
 
        VM_PAGE_CHECK(mem);
 
@@ -344,16 +397,15 @@ void vm_page_insert(mem, object, offset)
        mem->object = object;
        mem->offset = offset;
 
        mem->object = object;
        mem->offset = offset;
 
+       tmpm = vm_page_lookup(object, offset);
        /*
         *      Insert it into the object_object/offset hash table
         */
 
        bucket = &vm_page_buckets[vm_page_hash(object, offset)];
        /*
         *      Insert it into the object_object/offset hash table
         */
 
        bucket = &vm_page_buckets[vm_page_hash(object, offset)];
-       spl = splimp();
        simple_lock(&bucket_lock);
        queue_enter(bucket, mem, vm_page_t, hashq);
        simple_unlock(&bucket_lock);
        simple_lock(&bucket_lock);
        queue_enter(bucket, mem, vm_page_t, hashq);
        simple_unlock(&bucket_lock);
-       (void) splx(spl);
 
        /*
         *      Now link into the object's list of backed pages.
 
        /*
         *      Now link into the object's list of backed pages.
@@ -377,9 +429,12 @@ void vm_page_insert(mem, object, offset)
  *     table and the object page list.
  *
  *     The object and page must be locked.
  *     table and the object page list.
  *
  *     The object and page must be locked.
+ *
+ * interrupts must be disable in this routine!!!
  */
 
  */
 
-void vm_page_remove(mem)
+void
+vm_page_remove(mem)
        register vm_page_t      mem;
 {
        register queue_t        bucket;
        register vm_page_t      mem;
 {
        register queue_t        bucket;
@@ -387,19 +442,19 @@ void vm_page_remove(mem)
 
        VM_PAGE_CHECK(mem);
 
 
        VM_PAGE_CHECK(mem);
 
-       if (!(mem->flags & PG_TABLED))
+       if (!(mem->flags & PG_TABLED)) {
+               printf("page not tabled?????\n");
                return;
                return;
+       }
 
        /*
         *      Remove from the object_object/offset hash table
         */
 
        bucket = &vm_page_buckets[vm_page_hash(mem->object, mem->offset)];
 
        /*
         *      Remove from the object_object/offset hash table
         */
 
        bucket = &vm_page_buckets[vm_page_hash(mem->object, mem->offset)];
-       spl = splimp();
        simple_lock(&bucket_lock);
        queue_remove(bucket, mem, vm_page_t, hashq);
        simple_unlock(&bucket_lock);
        simple_lock(&bucket_lock);
        queue_remove(bucket, mem, vm_page_t, hashq);
        simple_unlock(&bucket_lock);
-       (void) splx(spl);
 
        /*
         *      Now remove from the object's list of backed pages.
 
        /*
         *      Now remove from the object's list of backed pages.
@@ -413,6 +468,7 @@ void vm_page_remove(mem)
         */
 
        mem->object->resident_page_count--;
         */
 
        mem->object->resident_page_count--;
+       mem->object = 0;
 
        mem->flags &= ~PG_TABLED;
 }
 
        mem->flags &= ~PG_TABLED;
 }
@@ -426,7 +482,8 @@ void vm_page_remove(mem)
  *     The object must be locked.  No side effects.
  */
 
  *     The object must be locked.  No side effects.
  */
 
-vm_page_t vm_page_lookup(object, offset)
+vm_page_t
+vm_page_lookup(object, offset)
        register vm_object_t    object;
        register vm_offset_t    offset;
 {
        register vm_object_t    object;
        register vm_offset_t    offset;
 {
@@ -439,22 +496,22 @@ vm_page_t vm_page_lookup(object, offset)
         */
 
        bucket = &vm_page_buckets[vm_page_hash(object, offset)];
         */
 
        bucket = &vm_page_buckets[vm_page_hash(object, offset)];
+       spl = vm_disable_intr(); 
 
 
-       spl = splimp();
        simple_lock(&bucket_lock);
        mem = (vm_page_t) queue_first(bucket);
        while (!queue_end(bucket, (queue_entry_t) mem)) {
                VM_PAGE_CHECK(mem);
                if ((mem->object == object) && (mem->offset == offset)) {
                        simple_unlock(&bucket_lock);
        simple_lock(&bucket_lock);
        mem = (vm_page_t) queue_first(bucket);
        while (!queue_end(bucket, (queue_entry_t) mem)) {
                VM_PAGE_CHECK(mem);
                if ((mem->object == object) && (mem->offset == offset)) {
                        simple_unlock(&bucket_lock);
-                       splx(spl);
+                       vm_set_intr(spl); 
                        return(mem);
                }
                mem = (vm_page_t) queue_next(&mem->hashq);
        }
 
        simple_unlock(&bucket_lock);
                        return(mem);
                }
                mem = (vm_page_t) queue_next(&mem->hashq);
        }
 
        simple_unlock(&bucket_lock);
-       splx(spl);
+       vm_set_intr(spl); 
        return(NULL);
 }
 
        return(NULL);
 }
 
@@ -466,18 +523,22 @@ vm_page_t vm_page_lookup(object, offset)
  *
  *     The object must be locked.
  */
  *
  *     The object must be locked.
  */
-void vm_page_rename(mem, new_object, new_offset)
+void
+vm_page_rename(mem, new_object, new_offset)
        register vm_page_t      mem;
        register vm_object_t    new_object;
        vm_offset_t             new_offset;
 {
        register vm_page_t      mem;
        register vm_object_t    new_object;
        vm_offset_t             new_offset;
 {
+       int spl;
        if (mem->object == new_object)
                return;
 
        vm_page_lock_queues();  /* keep page from moving out from
                                   under pageout daemon */
        if (mem->object == new_object)
                return;
 
        vm_page_lock_queues();  /* keep page from moving out from
                                   under pageout daemon */
+       spl = vm_disable_intr(); 
        vm_page_remove(mem);
        vm_page_insert(mem, new_object, new_offset);
        vm_page_remove(mem);
        vm_page_insert(mem, new_object, new_offset);
+       vm_set_intr(spl);
        vm_page_unlock_queues();
 }
 
        vm_page_unlock_queues();
 }
 
@@ -489,26 +550,40 @@ void vm_page_rename(mem, new_object, new_offset)
  *
  *     Object must be locked.
  */
  *
  *     Object must be locked.
  */
-vm_page_t vm_page_alloc(object, offset)
+vm_page_t
+vm_page_alloc(object, offset)
        vm_object_t     object;
        vm_offset_t     offset;
 {
        register vm_page_t      mem;
        int             spl;
 
        vm_object_t     object;
        vm_offset_t     offset;
 {
        register vm_page_t      mem;
        int             spl;
 
-       spl = splimp();                         /* XXX */
+       spl = vm_disable_intr();
        simple_lock(&vm_page_queue_free_lock);
        if (    object != kernel_object &&
                object != kmem_object   &&
        simple_lock(&vm_page_queue_free_lock);
        if (    object != kernel_object &&
                object != kmem_object   &&
-               vm_page_free_count <= vm_page_free_reserved) {
+               curproc != pageproc && curproc != &proc0 &&
+               vm_page_free_count < vm_page_free_reserved) {
 
                simple_unlock(&vm_page_queue_free_lock);
 
                simple_unlock(&vm_page_queue_free_lock);
-               splx(spl);
+               vm_set_intr(spl);
+               /*
+                * this wakeup seems unnecessary, but there is code that
+                * might just check to see if there are free pages, and
+                * punt if there aren't.  VM_WAIT does this too, but
+                * redundant wakeups aren't that bad...
+                */
+               if (curproc != pageproc)
+                       wakeup((caddr_t) &vm_pages_needed);
                return(NULL);
        }
        if (queue_empty(&vm_page_queue_free)) {
                simple_unlock(&vm_page_queue_free_lock);
                return(NULL);
        }
        if (queue_empty(&vm_page_queue_free)) {
                simple_unlock(&vm_page_queue_free_lock);
-               splx(spl);
+               vm_set_intr(spl);
+               /*
+                * comment above re: wakeups applies here too...
+                */
+               wakeup((caddr_t) &vm_pages_needed);
                return(NULL);
        }
 
                return(NULL);
        }
 
@@ -516,27 +591,20 @@ vm_page_t vm_page_alloc(object, offset)
 
        vm_page_free_count--;
        simple_unlock(&vm_page_queue_free_lock);
 
        vm_page_free_count--;
        simple_unlock(&vm_page_queue_free_lock);
-       splx(spl);
 
        mem->flags = PG_BUSY|PG_CLEAN|PG_FAKE;
        vm_page_insert(mem, object, offset);
        mem->wire_count = 0;
 
        mem->flags = PG_BUSY|PG_CLEAN|PG_FAKE;
        vm_page_insert(mem, object, offset);
        mem->wire_count = 0;
+       vm_set_intr(spl);
 
 
-       /*
-        *      Decide if we should poke the pageout daemon.
-        *      We do this if the free count is less than the low
-        *      water mark, or if the free count is less than the high
-        *      water mark (but above the low water mark) and the inactive
-        *      count is less than its target.
-        *
-        *      We don't have the counts locked ... if they change a little,
-        *      it doesn't really matter.
-        */
+/*
+ * don't wakeup too often, so we wakeup the pageout daemon when
+ * we would be nearly out of memory.
+ */
+       if (curproc != pageproc &&
+               (vm_page_free_count < vm_page_free_reserved))
+               wakeup((caddr_t) &vm_pages_needed);
 
 
-       if ((vm_page_free_count < vm_page_free_min) ||
-                       ((vm_page_free_count < vm_page_free_target) &&
-                       (vm_page_inactive_count < vm_page_inactive_target)))
-               thread_wakeup((int)&vm_pages_needed);
        return(mem);
 }
 
        return(mem);
 }
 
@@ -548,9 +616,14 @@ vm_page_t vm_page_alloc(object, offset)
  *
  *     Object and page must be locked prior to entry.
  */
  *
  *     Object and page must be locked prior to entry.
  */
-void vm_page_free(mem)
+void
+vm_page_free(mem)
        register vm_page_t      mem;
 {
        register vm_page_t      mem;
 {
+       int     spl;
+
+       spl = vm_disable_intr();
+
        vm_page_remove(mem);
        if (mem->flags & PG_ACTIVE) {
                queue_remove(&vm_page_queue_active, mem, vm_page_t, pageq);
        vm_page_remove(mem);
        if (mem->flags & PG_ACTIVE) {
                queue_remove(&vm_page_queue_active, mem, vm_page_t, pageq);
@@ -564,22 +637,46 @@ void vm_page_free(mem)
                vm_page_inactive_count--;
        }
 
                vm_page_inactive_count--;
        }
 
-       if (!(mem->flags & PG_FICTITIOUS)) {
-               int     spl;
 
 
-               spl = splimp();
+       if (!(mem->flags & PG_FICTITIOUS)) {
                simple_lock(&vm_page_queue_free_lock);
                simple_lock(&vm_page_queue_free_lock);
+               if (mem->wire_count) {
+                       vm_page_wire_count--;
+                       mem->wire_count = 0;
+               }
                queue_enter(&vm_page_queue_free, mem, vm_page_t, pageq);
 
                vm_page_free_count++;
                simple_unlock(&vm_page_queue_free_lock);
                queue_enter(&vm_page_queue_free, mem, vm_page_t, pageq);
 
                vm_page_free_count++;
                simple_unlock(&vm_page_queue_free_lock);
-/*
- * Other processes than pageproc can free memory, pageproc does the wakeup
- * and so should other processes that free memory. 
- */
-               wakeup((caddr_t) &vm_page_free_count);
-               splx(spl);
+               vm_set_intr(spl);
+
+               /*
+                * if pageout daemon needs pages, then tell it that there
+                * are some free.
+                */
+               if (vm_pageout_pages_needed)
+                       wakeup((caddr_t)&vm_pageout_pages_needed);
+
+               /*
+                * wakeup processes that are waiting on memory if we
+                * hit a high water mark.
+                */
+               if (vm_page_free_count == vm_page_free_min) {
+                       wakeup((caddr_t)&vm_page_free_count); 
+               }
+               
+               /*
+                * wakeup scheduler process if we have lots of memory.
+                * this process will swapin processes.
+                */
+               if (vm_page_free_count == vm_page_free_target) {
+                       wakeup((caddr_t)&proc0);
+               }
+
+       } else {
+               vm_set_intr(spl);
        }
        }
+       wakeup((caddr_t) mem);
 }
 
 /*
 }
 
 /*
@@ -591,10 +688,13 @@ void vm_page_free(mem)
  *
  *     The page queues must be locked.
  */
  *
  *     The page queues must be locked.
  */
-void vm_page_wire(mem)
+void
+vm_page_wire(mem)
        register vm_page_t      mem;
 {
        register vm_page_t      mem;
 {
+       int spl;
        VM_PAGE_CHECK(mem);
        VM_PAGE_CHECK(mem);
+       spl = vm_disable_intr();
 
        if (mem->wire_count == 0) {
                if (mem->flags & PG_ACTIVE) {
 
        if (mem->wire_count == 0) {
                if (mem->flags & PG_ACTIVE) {
@@ -612,6 +712,7 @@ void vm_page_wire(mem)
                vm_page_wire_count++;
        }
        mem->wire_count++;
                vm_page_wire_count++;
        }
        mem->wire_count++;
+       vm_set_intr(spl);
 }
 
 /*
 }
 
 /*
@@ -622,18 +723,48 @@ void vm_page_wire(mem)
  *
  *     The page queues must be locked.
  */
  *
  *     The page queues must be locked.
  */
-void vm_page_unwire(mem)
+void
+vm_page_unwire(mem)
        register vm_page_t      mem;
 {
        register vm_page_t      mem;
 {
+       int spl;
        VM_PAGE_CHECK(mem);
 
        VM_PAGE_CHECK(mem);
 
-       mem->wire_count--;
+       spl = vm_disable_intr();
+       if (mem->wire_count != 0)
+               mem->wire_count--;
        if (mem->wire_count == 0) {
                queue_enter(&vm_page_queue_active, mem, vm_page_t, pageq);
                vm_page_active_count++;
                mem->flags |= PG_ACTIVE;
                vm_page_wire_count--;
        }
        if (mem->wire_count == 0) {
                queue_enter(&vm_page_queue_active, mem, vm_page_t, pageq);
                vm_page_active_count++;
                mem->flags |= PG_ACTIVE;
                vm_page_wire_count--;
        }
+       vm_set_intr(spl);
+}
+
+/*
+ *     vm_page_queue_deactivate:
+ *
+ *     Place a page onto the inactive list only.
+ */
+void
+vm_page_queue_deactivate(m)
+       vm_page_t m;
+{
+       int spl;
+       spl = vm_disable_intr();
+       if (!(m->flags & PG_INACTIVE) && m->wire_count == 0 &&
+               !pmap_is_wired(VM_PAGE_TO_PHYS(m))) {
+               if (m->flags & PG_ACTIVE) {
+                       queue_remove(&vm_page_queue_active, m, vm_page_t, pageq);
+                       m->flags &= ~PG_ACTIVE;
+                       vm_page_active_count--;
+               }
+               queue_enter(&vm_page_queue_inactive, m, vm_page_t, pageq);
+               m->flags |= PG_INACTIVE;
+               vm_page_inactive_count++;
+       }
+       vm_set_intr(spl);
 }
 
 /*
 }
 
 /*
@@ -645,9 +776,11 @@ void vm_page_unwire(mem)
  *
  *     The page queues must be locked.
  */
  *
  *     The page queues must be locked.
  */
-void vm_page_deactivate(m)
+void
+vm_page_deactivate(m)
        register vm_page_t      m;
 {
        register vm_page_t      m;
 {
+       int spl;
        VM_PAGE_CHECK(m);
 
        /*
        VM_PAGE_CHECK(m);
 
        /*
@@ -660,7 +793,9 @@ void vm_page_deactivate(m)
         *      Paul Mackerras (paulus@cs.anu.edu.au) 9-Jan-93.
         */
 
         *      Paul Mackerras (paulus@cs.anu.edu.au) 9-Jan-93.
         */
 
-       if (!(m->flags & PG_INACTIVE) && m->wire_count == 0) {
+       spl = vm_disable_intr();
+       if (!(m->flags & PG_INACTIVE) && m->wire_count == 0 &&
+               !pmap_is_wired(VM_PAGE_TO_PHYS(m))) {
                pmap_clear_reference(VM_PAGE_TO_PHYS(m));
                if (m->flags & PG_ACTIVE) {
                        queue_remove(&vm_page_queue_active, m, vm_page_t, pageq);
                pmap_clear_reference(VM_PAGE_TO_PHYS(m));
                if (m->flags & PG_ACTIVE) {
                        queue_remove(&vm_page_queue_active, m, vm_page_t, pageq);
@@ -669,14 +804,43 @@ void vm_page_deactivate(m)
                }
                queue_enter(&vm_page_queue_inactive, m, vm_page_t, pageq);
                m->flags |= PG_INACTIVE;
                }
                queue_enter(&vm_page_queue_inactive, m, vm_page_t, pageq);
                m->flags |= PG_INACTIVE;
+               vm_set_intr(spl);
                vm_page_inactive_count++;
                vm_page_inactive_count++;
-               if (pmap_is_modified(VM_PAGE_TO_PHYS(m)))
+               if ((m->flags & PG_CLEAN) &&
+                       pmap_is_modified(VM_PAGE_TO_PHYS(m)))
                        m->flags &= ~PG_CLEAN;
                        m->flags &= ~PG_CLEAN;
-               if (m->flags & PG_CLEAN)
-                       m->flags &= ~PG_LAUNDRY;
-               else
+               if ((m->flags & PG_CLEAN) == 0)
                        m->flags |= PG_LAUNDRY;
                        m->flags |= PG_LAUNDRY;
-       }
+       /*
+        * let page fault back onto active queue if needed
+        */
+               pmap_page_protect(VM_PAGE_TO_PHYS(m), VM_PROT_NONE);
+       } else if (pmap_is_wired(VM_PAGE_TO_PHYS(m)))  {
+               vm_set_intr(spl);
+               if (!(m->flags & PG_ACTIVE))
+                       vm_page_activate(m);
+       } else {
+               vm_set_intr(spl);
+       } 
+
+}
+
+/*
+ *     vm_page_makefault
+ *
+ *     Cause next access of this page to fault
+ */
+void
+vm_page_makefault(m)
+       vm_page_t m;
+{
+       if ((m->flags & PG_CLEAN) &&
+               pmap_is_modified(VM_PAGE_TO_PHYS(m)))
+               m->flags &= ~PG_CLEAN;
+       if ((m->flags & PG_CLEAN) == 0)
+               m->flags |= PG_LAUNDRY;
+
+       pmap_page_protect(VM_PAGE_TO_PHYS(m), VM_PROT_NONE);
 }
 
 /*
 }
 
 /*
@@ -686,26 +850,37 @@ void vm_page_deactivate(m)
  *
  *     The page queues must be locked.
  */
  *
  *     The page queues must be locked.
  */
-
-void vm_page_activate(m)
+void
+vm_page_activate(m)
        register vm_page_t      m;
 {
        register vm_page_t      m;
 {
+       int spl;
        VM_PAGE_CHECK(m);
 
        VM_PAGE_CHECK(m);
 
+       m->deact = 2;
+
+       spl = vm_disable_intr();
+
        if (m->flags & PG_INACTIVE) {
                queue_remove(&vm_page_queue_inactive, m, vm_page_t,
                                                pageq);
                vm_page_inactive_count--;
                m->flags &= ~PG_INACTIVE;
        if (m->flags & PG_INACTIVE) {
                queue_remove(&vm_page_queue_inactive, m, vm_page_t,
                                                pageq);
                vm_page_inactive_count--;
                m->flags &= ~PG_INACTIVE;
+                       
        }
        if (m->wire_count == 0) {
                if (m->flags & PG_ACTIVE)
                        panic("vm_page_activate: already active");
 
        }
        if (m->wire_count == 0) {
                if (m->flags & PG_ACTIVE)
                        panic("vm_page_activate: already active");
 
-               queue_enter(&vm_page_queue_active, m, vm_page_t, pageq);
                m->flags |= PG_ACTIVE;
                m->flags |= PG_ACTIVE;
+               queue_enter(&vm_page_queue_active, m, vm_page_t, pageq);
+               queue_remove(&m->object->memq, m, vm_page_t, listq);
+               queue_enter(&m->object->memq, m, vm_page_t, listq);
                vm_page_active_count++;
                vm_page_active_count++;
+
        }
        }
+
+       vm_set_intr(spl);
 }
 
 /*
 }
 
 /*
@@ -716,7 +891,8 @@ void vm_page_activate(m)
  *     be used by the zero-fill object.
  */
 
  *     be used by the zero-fill object.
  */
 
-boolean_t vm_page_zero_fill(m)
+boolean_t
+vm_page_zero_fill(m)
        vm_page_t       m;
 {
        VM_PAGE_CHECK(m);
        vm_page_t       m;
 {
        VM_PAGE_CHECK(m);
@@ -730,8 +906,8 @@ boolean_t vm_page_zero_fill(m)
  *
  *     Copy one page to another
  */
  *
  *     Copy one page to another
  */
-
-void vm_page_copy(src_m, dest_m)
+void
+vm_page_copy(src_m, dest_m)
        vm_page_t       src_m;
        vm_page_t       dest_m;
 {
        vm_page_t       src_m;
        vm_page_t       dest_m;
 {
index 919ed83..0cbd549 100644 (file)
@@ -34,7 +34,7 @@
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_page.h     7.3 (Berkeley) 4/21/91
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_page.h     7.3 (Berkeley) 4/21/91
- *     $Id: vm_page.h,v 1.5 1993/12/19 00:56:11 wollman Exp $
+ *     $Id: vm_page.h,v 1.6 1993/12/21 05:51:05 davidg Exp $
  */
 
 /*
  */
 
 /*
@@ -120,7 +120,8 @@ struct vm_page {
        vm_offset_t     offset;         /* offset into that object (O,P) */
 
        unsigned int    wire_count;     /* how many wired down maps use me? */
        vm_offset_t     offset;         /* offset into that object (O,P) */
 
        unsigned int    wire_count;     /* how many wired down maps use me? */
-       unsigned int    flags;          /* bit encoded flags */
+       unsigned short  flags;          /* bit encoded flags */
+       unsigned short  deact;          /* deactivation count */
 
        vm_offset_t     phys_addr;      /* physical address of page */
 };
 
        vm_offset_t     phys_addr;      /* physical address of page */
 };
@@ -129,7 +130,7 @@ typedef struct vm_page      *vm_page_t;
 
 #if    VM_PAGE_DEBUG
 #define        VM_PAGE_CHECK(mem) { \
 
 #if    VM_PAGE_DEBUG
 #define        VM_PAGE_CHECK(mem) { \
-               if ( (((unsigned int) mem) < ((unsigned int) &vm_page_array[0])) || \
+               if ((((unsigned int) mem) < ((unsigned int) &vm_page_array[0])) || \
                     (((unsigned int) mem) > ((unsigned int) &vm_page_array[last_page-first_page])) || \
                     ((mem->flags & PG_ACTIVE) && (mem->flags & PG_INACTIVE)) \
                    ) panic("vm_page_check: not valid!"); \
                     (((unsigned int) mem) > ((unsigned int) &vm_page_array[last_page-first_page])) || \
                     ((mem->flags & PG_ACTIVE) && (mem->flags & PG_INACTIVE)) \
                    ) panic("vm_page_check: not valid!"); \
@@ -267,5 +268,14 @@ extern vm_offset_t pmap_phys_ddress(int);
 extern boolean_t pmap_testbit(vm_offset_t, int);
 extern void pmap_changebit(vm_offset_t, int, boolean_t);
 
 extern boolean_t pmap_testbit(vm_offset_t, int);
 extern void pmap_changebit(vm_offset_t, int, boolean_t);
 
+
+/*
+ * these macros are *MUCH* faster on a 386/486 type machine
+ * eventually they need to be implemented correctly and put
+ * somewhere in the machine dependant stuff.
+ */
+#define vm_disable_intr() (disable_intr(), 0)
+#define vm_set_intr(spl) enable_intr()
+
 #endif /* KERNEL */
 #endif /* _VM_PAGE_ */
 #endif /* KERNEL */
 #endif /* _VM_PAGE_ */
index 91d8f0d..b53bf6c 100644 (file)
@@ -1,6 +1,10 @@
 /* 
  * Copyright (c) 1991 Regents of the University of California.
  * All rights reserved.
 /* 
  * Copyright (c) 1991 Regents of the University of California.
  * All rights reserved.
+ * Copyright (c) 1994 John S. Dyson
+ * All rights reserved.
+ * Copyright (c) 1994 David Greenman
+ * All rights reserved.
  *
  * This code is derived from software contributed to Berkeley by
  * The Mach Operating System project at Carnegie-Mellon University.
  *
  * This code is derived from software contributed to Berkeley by
  * The Mach Operating System project at Carnegie-Mellon University.
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
- *     from: @(#)vm_pageout.c  7.4 (Berkeley) 5/7/91
- *     $Id: vm_pageout.c,v 1.8 1993/12/21 05:51:06 davidg Exp $
- */
-
-/*
+ *     @(#)vm_pageout.c        7.4 (Berkeley) 5/7/91
+ *
+ *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  *
@@ -62,6 +64,8 @@
  *
  * any improvements or extensions that they make and grant Carnegie the
  * rights to redistribute these changes.
  *
  * any improvements or extensions that they make and grant Carnegie the
  * rights to redistribute these changes.
+ *
+ * $Id$
  */
 
 /*
  */
 
 /*
  */
 
 #include "param.h"
  */
 
 #include "param.h"
-#include "systm.h"
 
 #include "vm.h"
 #include "vm_page.h"
 #include "vm_pageout.h"
 
 #include "vm.h"
 #include "vm_page.h"
 #include "vm_pageout.h"
+#include "malloc.h"
+#include "proc.h"
+#include "resource.h"
+#include "resourcevar.h"
 #include "vmmeter.h"
 
 #include "vmmeter.h"
 
-simple_lock_data_t     vm_pages_needed_lock;
+extern vm_map_t kmem_map;
 int    vm_pages_needed;                /* Event on which pageout daemon sleeps */
 int    vm_pageout_free_min = 0;        /* Stop pageout to wait for pagers at this free level */
 
 int    vm_page_free_min_sanity = 40;
 int    vm_pages_needed;                /* Event on which pageout daemon sleeps */
 int    vm_pageout_free_min = 0;        /* Stop pageout to wait for pagers at this free level */
 
 int    vm_page_free_min_sanity = 40;
+int    vm_pageout_pages_needed = 0;    /* flag saying that the pageout daemon needs pages */
+int    vm_page_pagesfreed;
+
+extern int npendingio;
+extern int hz;
+int    vm_pageout_proc_limit;
+extern int nswiodone;
+
+#define MAXREF 32767
+
+/*
+ * vm_pageout_clean:
+ *     cleans a vm_page
+ */
+int
+vm_pageout_clean(m, wait) 
+       register vm_page_t m;
+       int wait;
+{
+       /*
+        *      Clean the page and remove it from the
+        *      laundry.
+        *
+        *      We set the busy bit to cause
+        *      potential page faults on this page to
+        *      block.
+        *
+        *      And we set pageout-in-progress to keep
+        *      the object from disappearing during
+        *      pageout.  This guarantees that the
+        *      page won't move from the inactive
+        *      queue.  (However, any other page on
+        *      the inactive queue may move!)
+        */
+
+       register vm_object_t    object;
+       register vm_pager_t     pager;
+       int                     pageout_status;
+
+       object = m->object;
+       if (!object) {
+               printf("pager: object missing\n");
+               return 0;
+       }
+
+       /*
+        *      Try to collapse the object before
+        *      making a pager for it.  We must
+        *      unlock the page queues first.
+        *      We try to defer the creation of a pager
+        *      until all shadows are not paging.  This
+        *      allows vm_object_collapse to work better and
+        *      helps control swap space size.
+        *      (J. Dyson 11 Nov 93)
+        */
+
+       if (!object->pager &&
+               vm_page_free_count < vm_pageout_free_min)
+               return 0;
+
+       if (!object->pager &&
+               object->shadow &&
+               object->shadow->paging_in_progress)
+               return 0;
+
+       if (object->shadow) {
+               vm_offset_t offset = m->offset;
+               vm_object_collapse(object);
+               if (!vm_page_lookup(object, offset))
+                       return 0;
+       }
+
+waitagain:
+       if (!wait && (m->flags & PG_BUSY)) {
+               return 0;
+       } else if (m->flags & PG_BUSY) {
+               int s = splhigh();
+               m->flags |= PG_WANTED;
+               tsleep((caddr_t)m, PVM, "clnslp", 0);
+               splx(s);
+               goto waitagain;
+       }
+
+       m->flags |= PG_BUSY;
+
+       pmap_page_protect(VM_PAGE_TO_PHYS(m), VM_PROT_READ);
+       pmap_clear_reference(VM_PAGE_TO_PHYS(m));
+
+       vm_stat.pageouts++;
+
+       object->paging_in_progress++;
+
+       /*
+        *      If there is no pager for the page,
+        *      use the default pager.  If there's
+        *      no place to put the page at the
+        *      moment, leave it in the laundry and
+        *      hope that there will be paging space
+        *      later.
+        */
+
+       if ((pager = object->pager) == NULL) {
+               pager = vm_pager_allocate(PG_DFLT, (caddr_t)0,
+                       object->size, VM_PROT_ALL, 0);
+               if (pager != NULL) {
+                       vm_object_setpager(object, pager, 0, FALSE);
+               }
+       }
+       if ((pager && pager->pg_type == PG_SWAP) || 
+               vm_page_free_count >= vm_pageout_free_min) {
+               pageout_status = pager ?
+                       vm_pager_put(pager, m, (((object == kernel_object) || wait) ? TRUE: FALSE)) :
+                       VM_PAGER_FAIL;
+       } else
+               pageout_status = VM_PAGER_FAIL;
+
+       switch (pageout_status) {
+       case VM_PAGER_OK:
+               m->flags &= ~PG_LAUNDRY;
+               break;
+       case VM_PAGER_PEND:
+               m->flags &= ~PG_LAUNDRY;
+               break;
+       case VM_PAGER_BAD:
+               /*
+                * Page outside of range of object.
+                * Right now we essentially lose the
+                * changes by pretending it worked.
+                */
+               m->flags &= ~PG_LAUNDRY;
+               m->flags |= PG_CLEAN;
+               pmap_clear_modify(VM_PAGE_TO_PHYS(m));
+               break;
+       case VM_PAGER_FAIL:
+               /*
+                * If page couldn't be paged out, then
+                * reactivate the page so it doesn't
+                * clog the inactive list.  (We will
+                * try paging out it again later).
+                */
+               vm_page_activate(m);
+               break;
+       case VM_PAGER_TRYAGAIN:
+               break;
+       }
+
+
+       /*
+        * If the operation is still going, leave
+        * the page busy to block all other accesses.
+        * Also, leave the paging in progress
+        * indicator set so that we don't attempt an
+        * object collapse.
+        */
+       if (pageout_status != VM_PAGER_PEND) {
+               if (pmap_is_referenced(VM_PAGE_TO_PHYS(m))) {
+                       pmap_clear_reference(VM_PAGE_TO_PHYS(m));
+                       vm_page_activate(m);
+               }
+               PAGE_WAKEUP(m);
+               if (--object->paging_in_progress == 0)
+                       wakeup((caddr_t) object);
+       }
+       return (pageout_status == VM_PAGER_PEND ||
+               pageout_status == VM_PAGER_OK) ? 1 : 0;
+}
+
+/*
+ *     vm_pageout_object_deactivate_pages
+ *
+ *     deactivate enough pages to satisfy the inactive target
+ *     requirements or if vm_page_proc_limit is set, then
+ *     deactivate all of the pages in the object and its
+ *     shadows.
+ *
+ *     The object and map must be locked.
+ */
+int
+vm_pageout_object_deactivate_pages(map, object, count)
+       vm_map_t map;
+       vm_object_t object;
+       int count;
+{
+       register vm_page_t      p, next;
+       int rcount;
+       int s;
+       int dcount;
+
+       dcount = 0;
+/*
+ * if the object is used by more than one process and we are not enforcing
+ * RSS size, then dont deactivate the pages.  Note that the check for
+ * ref_count is not really a check for more than one process, but is
+ * close enough.
+ */
+
+#if 0
+       if (object->ref_count > 1)
+                       return 0;
+#endif
+       /*
+        * deactivate the pages in the objects shadow
+        */
+
+       if (object->ref_count)
+               count /= object->ref_count;
+
+       if (count == 0)
+               count = 1;
+
+       if (object->shadow)
+               dcount += vm_pageout_object_deactivate_pages(map, object->shadow, count);
+
+       /*
+        * scan the objects entire memory queue
+        */
+       rcount = object->resident_page_count;
+       p = (vm_page_t) queue_first(&object->memq);
+       while ((rcount-- > 0) && !queue_end(&object->memq, (queue_entry_t) p) ) {
+               next = (vm_page_t) queue_next(&p->listq);
+               vm_page_lock_queues();
+               /*
+                * if a page is active, not wired and is in the processes pmap,
+                * then deactivate the page.
+                */
+               if ((p->flags & (PG_ACTIVE|PG_BUSY)) == PG_ACTIVE &&
+                       p->wire_count == 0 &&
+                       !pmap_is_wired(VM_PAGE_TO_PHYS(p)) &&
+                       pmap_page_exists(vm_map_pmap(map), VM_PAGE_TO_PHYS(p))) {
+                       if (!pmap_is_referenced(VM_PAGE_TO_PHYS(p))) {
+                               vm_page_pageout_deactivate(p);
+                               /*
+                                * see if we are done yet
+                                */
+                               if (p->flags & PG_INACTIVE) {
+                                       if ((p->flags & PG_CLEAN) == 0)
+                                               vm_pageout_clean(p, 0);
+                                       --count;
+                                       ++dcount;
+                                       if (count <= 0 &&
+                                               vm_page_inactive_count > vm_page_inactive_target) {
+                                                       vm_page_unlock_queues();
+                                                       return dcount;
+                                       }
+                               }
+                               
+                       } else {
+                               p->deact = 2;
+                               pmap_clear_reference(VM_PAGE_TO_PHYS(p));
+                               queue_remove(&object->memq, p, vm_page_t, listq);
+                               queue_enter(&object->memq, p, vm_page_t, listq);
+                               queue_remove(&vm_page_queue_active, p, vm_page_t, pageq);
+                               queue_enter(&vm_page_queue_active, p, vm_page_t, pageq);
+                       }
+               }
+               vm_page_unlock_queues();
+               p = next;
+       }
+       return dcount;
+}
 
 
-int    vm_page_pagesfreed;             /* Pages freed by page daemon */
+
+/*
+ * deactivate some number of pages in a map, try to do it fairly, but
+ * that is really hard to do.
+ */
+
+void
+vm_pageout_map_deactivate_pages(map, entry, count)
+       vm_map_t map;
+       vm_map_entry_t entry;
+       int *count;
+{
+       vm_map_t tmpm;
+       vm_map_entry_t tmpe;
+       vm_object_t obj;
+       if (*count <= 0)
+               return;
+       vm_map_reference(map);
+       if (!lock_try_read(&map->lock)) {
+               vm_map_deallocate(map);
+               return;
+       }
+       if (entry == 0) {
+               tmpe = map->header.next;
+               while (tmpe != &map->header && *count > 0) {
+                       vm_pageout_map_deactivate_pages(map, tmpe, count);
+                       tmpe = tmpe->next;
+               };
+       } else if (entry->is_sub_map || entry->is_a_map) {
+               tmpm = entry->object.share_map;
+               tmpe = tmpm->header.next;
+               while (tmpe != &tmpm->header && *count > 0) {
+                       vm_pageout_map_deactivate_pages(tmpm, tmpe, count);
+                       tmpe = tmpe->next;
+               };
+       } else if (obj = entry->object.vm_object) {
+               *count -= vm_pageout_object_deactivate_pages(map, obj, *count);
+       }
+       lock_read_done(&map->lock);
+       vm_map_deallocate(map);
+       return;
+}
 
 /*
  *     vm_pageout_scan does the dirty work for the pageout daemon.
 
 /*
  *     vm_pageout_scan does the dirty work for the pageout daemon.
@@ -90,40 +398,110 @@ int       vm_page_pagesfreed;             /* Pages freed by page daemon */
 void
 vm_pageout_scan()
 {
 void
 vm_pageout_scan()
 {
-       register vm_page_t      m;
-       register int            page_shortage;
-       register int            s;
-       register int            pages_freed;
-       int                     free;
+       vm_page_t       m;
+       int             page_shortage, maxscan, maxlaunder;
+       int             pages_freed, free, nproc, nbusy, tries;
+       vm_page_t       next;
+       struct proc     *p;
+       vm_object_t     object;
+       int             s;
+
+       tries = 0;
 
        /*
 
        /*
-        *      Only continue when we want more pages to be "free"
+        * deactivate objects with ref_counts == 0
         */
         */
+       object = (vm_object_t) queue_first(&vm_object_list);
+       while (!queue_end(&vm_object_list, (queue_entry_t) object)) {
+               if (object->ref_count == 0)
+                       vm_object_deactivate_pages(object);
+               object = (vm_object_t) queue_next(&object->object_list);
+       }
 
 
-       s = splimp();
-       simple_lock(&vm_page_queue_free_lock);
+       /*
+        * now check malloc area or swap processes out if we are in low
+        * memory conditions
+        */
        free = vm_page_free_count;
        free = vm_page_free_count;
-       simple_unlock(&vm_page_queue_free_lock);
-       splx(s);
-
-       if (free < vm_page_free_target) {
-#ifdef OMIT
-               swapout_threads();
-#endif /* OMIT*/
-
+       if (free <= vm_page_free_min) {
                /*
                 *      Be sure the pmap system is updated so
                 *      we can scan the inactive queue.
                 */
                /*
                 *      Be sure the pmap system is updated so
                 *      we can scan the inactive queue.
                 */
-
                pmap_update();
                pmap_update();
+
+               /*
+                * swap out inactive processes
+                */
+               swapout_threads();
+
+#if 0
+               /*
+                * see if malloc has anything for us
+                */
+               if (free <= vm_page_free_reserved)
+                       malloc_gc(); 
+#endif
        }
 
        }
 
+rerun:
+#if 1
        /*
        /*
-        *      Acquire the resident page system lock,
-        *      as we may be changing what's resident quite a bit.
+        * next scan the processes for exceeding their rlimits or if process
+        * is swapped out -- deactivate pages 
         */
         */
-       vm_page_lock_queues();
+
+rescanproc1a:
+       for (p = allproc; p != NULL; p = p->p_nxt)
+               p->p_flag &= ~SPAGEDAEMON;
+
+rescanproc1:
+       for (p = allproc; p != NULL; p = p->p_nxt) {
+               vm_offset_t size;
+               int overage;
+               vm_offset_t limit;
+
+               /*
+                * if this is a system process or if we have already
+                * looked at this process, skip it.
+                */
+               if (p->p_flag & (SSYS|SPAGEDAEMON|SWEXIT)) {
+                       continue;
+               }
+
+               /*
+                * if the process is in a non-running type state,
+                * don't touch it.
+                */
+               if (p->p_stat != SRUN && p->p_stat != SSLEEP) {
+                       continue;
+               }
+
+               /*
+                * get a limit
+                */
+               limit = p->p_rlimit[RLIMIT_RSS].rlim_cur;
+               if (limit > p->p_rlimit[RLIMIT_RSS].rlim_max)
+                       limit = p->p_rlimit[RLIMIT_RSS].rlim_max;
+                       
+               /*
+                * let processes that are swapped out really be swapped out
+                * set the limit to nothing (will force a swap-out.)
+                */
+               if ((p->p_flag & SLOAD) == 0)
+                       limit = 0;
+
+               size = p->p_vmspace->vm_pmap.pm_stats.resident_count * NBPG;
+               if (size >= limit) {
+                       overage = (size - limit) / NBPG;
+                       vm_pageout_map_deactivate_pages(&p->p_vmspace->vm_map,
+                               (vm_map_entry_t) 0, &overage);
+                       p->p_flag |= SPAGEDAEMON;
+                       goto rescanproc1;
+               }
+               p->p_flag |= SPAGEDAEMON;
+       }
+#endif
 
        /*
         *      Start scanning the inactive queue for pages we can free.
 
        /*
         *      Start scanning the inactive queue for pages we can free.
@@ -133,45 +511,75 @@ vm_pageout_scan()
         */
 
        pages_freed = 0;
         */
 
        pages_freed = 0;
-       m = (vm_page_t) queue_first(&vm_page_queue_inactive);
-       while (!queue_end(&vm_page_queue_inactive, (queue_entry_t) m)) {
+       maxlaunder = 2 * (vm_page_free_target - vm_page_free_count);
+rescan_all:
+       maxscan = vm_page_inactive_count;
+rescan:
+       while (maxscan-- > 0) {
                vm_page_t       next;
 
                vm_page_t       next;
 
-               s = splimp();
-               simple_lock(&vm_page_queue_free_lock);
-               free = vm_page_free_count;
-               simple_unlock(&vm_page_queue_free_lock);
-               splx(s);
+               if (vm_page_free_count >= vm_page_free_target) {
+                       break;
+               }
 
 
-               if (free >= vm_page_free_target)
+               /*
+                * see if any swapping I/O is complete
+                */
+               if (nswiodone) {
+                       swap_pager_clean(NULL, NULL);
+                       goto rescan_all;
+               }
+
+               m = (vm_page_t) queue_first(&vm_page_queue_inactive);
+               if (queue_end(&vm_page_queue_inactive, (queue_entry_t) m)) {
                        break;
                        break;
+               }
+
+               /*
+                * don't let async queue modifications hang us
+                * remove us from the inactive queue and set the page
+                * non-inactive.
+                */
+
+               /*
+                * dont mess with busy pages
+                */
+               if (m->flags & PG_BUSY) {
+                       queue_remove(&vm_page_queue_inactive, m, vm_page_t, pageq);
+                       queue_enter(&vm_page_queue_inactive, m, vm_page_t, pageq);
+                       continue;
+               }
+
+               /*
+                * dont mess with a page that is pmap wired -- just activate it
+                */
+               if (pmap_is_wired(VM_PAGE_TO_PHYS(m))) {
+                       vm_page_activate(m);
+                       continue;
+               }
 
 
+               /*
+                * if page is clean and but the page has been referenced,
+                * then reactivate the page, but if we are very low on memory
+                * or the page has not been referenced, then we free it to the
+                * vm system.
+                */
                if (m->flags & PG_CLEAN) {
                if (m->flags & PG_CLEAN) {
-                       next = (vm_page_t) queue_next(&m->pageq);
-                       if (pmap_is_referenced(VM_PAGE_TO_PHYS(m))) {
+                       if ((vm_page_free_count > vm_pageout_free_min)
+                               && pmap_is_referenced(VM_PAGE_TO_PHYS(m))) {
+                               pmap_clear_reference(VM_PAGE_TO_PHYS(m));
                                vm_page_activate(m);
                                vm_page_activate(m);
-                               vm_stat.reactivations++;
+                               ++vm_stat.reactivations;
+                               continue;
                        }
                        else {
                        }
                        else {
-                               register vm_object_t    object;
-                               object = m->object;
-                               if (!vm_object_lock_try(object)) {
-                                       /*
-                                        *      Can't lock object -
-                                        *      skip page.
-                                        */
-                                       m = next;
-                                       continue;
-                               }
                                pmap_page_protect(VM_PAGE_TO_PHYS(m),
                                                  VM_PROT_NONE);
                                pmap_page_protect(VM_PAGE_TO_PHYS(m),
                                                  VM_PROT_NONE);
-                               vm_page_free(m);        /* will dequeue */
-                               pages_freed++;
-                               vm_object_unlock(object);
+                               vm_page_free(m);
+                               ++pages_freed;
+                               continue;
                        }
                        }
-                       m = next;
-               }
-               else {
+               } else if ((m->flags & PG_LAUNDRY) && maxlaunder > 0) {
                        /*
                         *      If a page is dirty, then it is either
                         *      being washed (but not yet cleaned)
                        /*
                         *      If a page is dirty, then it is either
                         *      being washed (but not yet cleaned)
@@ -180,237 +588,201 @@ vm_pageout_scan()
                         *      cleaning operation.
                         */
 
                         *      cleaning operation.
                         */
 
-                       if (m->flags & PG_LAUNDRY) {
-                               /*
-                                *      Clean the page and remove it from the
-                                *      laundry.
-                                *
-                                *      We set the busy bit to cause
-                                *      potential page faults on this page to
-                                *      block.
-                                *
-                                *      And we set pageout-in-progress to keep
-                                *      the object from disappearing during
-                                *      pageout.  This guarantees that the
-                                *      page won't move from the inactive
-                                *      queue.  (However, any other page on
-                                *      the inactive queue may move!)
-                                */
-
-                               register vm_object_t    object;
-                               register vm_pager_t     pager;
-                               int                     pageout_status;
+                       if (vm_pageout_clean(m,0)) {
+                               --maxlaunder;
+                       }
+               } 
+               if ((m->flags & PG_INACTIVE) == PG_INACTIVE) {
+                       queue_remove(&vm_page_queue_inactive, m, vm_page_t, pageq);
+                       queue_enter(&vm_page_queue_inactive, m, vm_page_t, pageq);
+               }
+       }
 
 
-                               object = m->object;
-                               if (!vm_object_lock_try(object)) {
-                                       /*
-                                        *      Skip page if we can't lock
-                                        *      its object
-                                        */
-                                       m = (vm_page_t) queue_next(&m->pageq);
-                                       continue;
-                               }
+       /*
+        * If we did not free any pages, but we need to do so, we grow the
+        * inactive target.  But as we successfully free pages, then we
+        * slowly shrink the inactive target.
+        */
+       if (pages_freed == 0 && vm_page_free_count < vm_page_free_min)
+               vm_page_inactive_target += (vm_page_free_target - vm_page_free_count);
+       else if (pages_freed > 0) {
+               vm_page_inactive_target = vm_page_free_target;
+       }
 
 
+       /*
+        *      Compute the page shortage.  If we are still very low on memory
+        *      be sure that we will move a minimal amount of pages from active
+        *      to inactive.
+        */
 
 
-                               /*
-                                *      Try to collapse the object before
-                                *      making a pager for it.  We must
-                                *      unlock the page queues first.
-                                *      We try to defer the creation of a pager
-                                *      until all shadows are not paging.  This
-                                *      allows vm_object_collapse to work better and
-                                *      helps control swap space size.
-                                *      (J. Dyson 18 Dec 93)
-                                */
-                               vm_page_unlock_queues();
+restart_inactivate_all:
 
 
+       page_shortage = vm_page_inactive_target - vm_page_inactive_count;
+       page_shortage -= vm_page_free_count;  
+
+       if (page_shortage <= 0) {
+               if (pages_freed == 0  /* &&
+                       ((vm_page_free_count + vm_page_inactive_count) <
+                               (vm_page_free_min + vm_page_inactive_target)) */ ) {
+                       page_shortage = 1;
+               } else {
+                       page_shortage = 0;
+               }
+       }
 
 
-                               if( !object->pager &&
-                                       (object->shadow && object->shadow->paging_in_progress ) ||
-                                       (vm_page_free_count < vm_pageout_free_min)) {
-                                       vm_object_unlock(object);
-                                       m = (vm_page_t) queue_next(&m->pageq);
-                                       continue;
-                               }
+       maxscan = vm_page_active_count;
 
 
+       /*
+        * deactivate pages that are active, but have not been used
+        * for a while.
+        */
+restart_inactivate:
+       m = (vm_page_t) queue_first(&vm_page_queue_active);
+       while (page_shortage > 0 && maxscan-- > 0) {
 
 
-                               vm_object_collapse(object);
+               if (queue_end(&vm_page_queue_active, (queue_entry_t) m)) {
+                       break;
+               }
 
 
-                               pmap_page_protect(VM_PAGE_TO_PHYS(m),
-                                                 VM_PROT_NONE);
-                               m->flags |= PG_BUSY;
-                               vm_stat.pageouts++;
+               next = (vm_page_t) queue_next(&m->pageq);
 
 
-                               object->paging_in_progress++;
-                               vm_object_unlock(object);
+               /*
+                * dont mess with pages that are busy
+                */
+               if (m->flags & PG_BUSY) {
+                       m = next;
+                       continue;
+               }
 
 
-                               /*
-                                *      Do a wakeup here in case the following
-                                *      operations block.
-                                */
-                               thread_wakeup((int) &vm_page_free_count);
+       /*
+        *      Move some more pages from active to inactive.
+        */
 
 
+               /*
+                * see if there are any pages that are able to be deactivated
+                */
+               if (!pmap_is_wired(VM_PAGE_TO_PHYS(m))) {
+                       /*
+                        * the referenced bit is the one that say that the page
+                        * has been used.
+                        */
+                       if (!pmap_is_referenced(VM_PAGE_TO_PHYS(m))) {
                                /*
                                /*
-                                *      If there is no pager for the page,
-                                *      use the default pager.  If there's
-                                *      no place to put the page at the
-                                *      moment, leave it in the laundry and
-                                *      hope that there will be paging space
-                                *      later.
+                                * if the page has not been referenced, call the
+                                * vm_page_pageout_deactivate routine.  It might
+                                * not deactivate the page every time.  There is
+                                * a policy associated with it.
                                 */
                                 */
-
-                               if ((pager = object->pager) == NULL) {
-                                       pager = vm_pager_allocate(PG_DFLT,
-                                                                 (caddr_t)0,
-                                                                 object->size,
-                                                                 VM_PROT_ALL,
-                                                                 0);/* start @0*/
-                                       if (pager != NULL) {
-                                               vm_object_setpager(object,
-                                                       pager, 0, FALSE);
-                                       }
-                               }
-                               pageout_status = pager ?
-                                       vm_pager_put(pager, m, FALSE) :
-                                       VM_PAGER_FAIL;
-                               vm_object_lock(object);
-                               vm_page_lock_queues();
-                               next = (vm_page_t) queue_next(&m->pageq);
-
-                               switch (pageout_status) {
-                               case VM_PAGER_OK:
-                               case VM_PAGER_PEND:
-                                       m->flags &= ~PG_LAUNDRY;
-                                       break;
-                               case VM_PAGER_BAD:
+                               if (vm_page_pageout_deactivate(m)) {
                                        /*
                                        /*
-                                        * Page outside of range of object.
-                                        * Right now we essentially lose the
-                                        * changes by pretending it worked.
-                                        * XXX dubious, what should we do?
+                                        * if the page was really deactivated, then
+                                        * decrement the page_shortage
                                         */
                                         */
-                                       m->flags &= ~PG_LAUNDRY;
-                                       m->flags |= PG_CLEAN;
-                                       pmap_clear_modify(VM_PAGE_TO_PHYS(m));
-                                       break;
-                               case VM_PAGER_FAIL:
-                                       /*
-                                        * If page couldn't be paged out, then
-                                        * reactivate the page so it doesn't
-                                        * clog the inactive list.  (We will
-                                        * try paging out it again later).
-                                        */
-                                       vm_page_activate(m);
-                                       break;
+                                       if ((m->flags & PG_ACTIVE) == 0) {
+                                               --page_shortage;
+                                               m = next;
+                                               continue;
+                                       }
                                }
                                }
-
-                               pmap_clear_reference(VM_PAGE_TO_PHYS(m));
-
+                       } else {
                                /*
                                /*
-                                * If the operation is still going, leave
-                                * the page busy to block all other accesses.
-                                * Also, leave the paging in progress
-                                * indicator set so that we don't attempt an
-                                * object collapse.
+                                * if the page was recently referenced, set our
+                                * deactivate count and clear reference for a future
+                                * check for deactivation.
                                 */
                                 */
-                               if (pageout_status != VM_PAGER_PEND) {
-                                       m->flags &= ~PG_BUSY;
-                                       PAGE_WAKEUP(m);
-                                       object->paging_in_progress--;
-                               }
-                               thread_wakeup((int) object);
-                               vm_object_unlock(object);
-                               m = next;
+                               m->deact = 2;
+                               pmap_clear_reference(VM_PAGE_TO_PHYS(m));
+                               queue_remove(&m->object->memq, m, vm_page_t, listq);
+                               queue_enter(&m->object->memq, m, vm_page_t, listq);
+                               queue_remove(&vm_page_queue_active, m, vm_page_t, pageq);
+                               queue_enter(&vm_page_queue_active, m, vm_page_t, pageq);
                        }
                        }
-                       else
-                               m = (vm_page_t) queue_next(&m->pageq);
-               }
+               } 
+               m = next;
        }
        }
-       
-       /*
-        *      Compute the page shortage.  If we are still very low on memory
-        *      be sure that we will move a minimal amount of pages from active
-        *      to inactive.
-        */
-
-       page_shortage = vm_page_inactive_target - vm_page_inactive_count;
-       page_shortage -= vm_page_free_count;
-
-       if ((page_shortage <= 0) && (pages_freed == 0))
-               page_shortage = 1;
 
 
-       while (page_shortage > 0) {
-               /*
-                *      Move some more pages from active to inactive.
-                */
+       vm_page_pagesfreed += pages_freed;
+}
 
 
-               if (queue_empty(&vm_page_queue_active)) {
-                       break;
-               }
-               m = (vm_page_t) queue_first(&vm_page_queue_active);
+/*
+ * optionally do a deactivate if the deactivate has been done
+ * enough to justify it.
+ */
+int
+vm_page_pageout_deactivate(m)
+       vm_page_t m;
+{
+       if (m->deact > 2)
+               m->deact = 2;
+       if (m->deact == 0 /* || vm_page_free_count < vm_page_free_reserved */) {
                vm_page_deactivate(m);
                vm_page_deactivate(m);
-               page_shortage--;
+               return 1;
+       } else {
+               vm_page_makefault(m);
+               --m->deact;
+               return 0;
        }
        }
-
-       vm_page_pagesfreed += pages_freed;
-       vm_page_unlock_queues();
 }
 
 /*
  *     vm_pageout is the high level pageout daemon.
  */
 
 }
 
 /*
  *     vm_pageout is the high level pageout daemon.
  */
 
-void vm_pageout()
+void
+vm_pageout()
 {
 {
+       extern npendingio, swiopend;
        (void) spl0();
 
        /*
         *      Initialize some paging parameters.
         */
 
        (void) spl0();
 
        /*
         *      Initialize some paging parameters.
         */
 
-       if (vm_page_free_min == 0) {
-               vm_page_free_min = vm_page_free_count / 20;
-               if (vm_page_free_min < 3)
-                       vm_page_free_min = 3;
-
-               if (vm_page_free_min > vm_page_free_min_sanity)
-                       vm_page_free_min = vm_page_free_min_sanity;
-       }
-
-       if (vm_page_free_reserved == 0) {
-               if ((vm_page_free_reserved = vm_page_free_min / 2) < 10)
-                       vm_page_free_reserved = 10;
-       }
-       if (vm_pageout_free_min == 0) {
-               if ((vm_pageout_free_min = vm_page_free_reserved / 2) > 10)
-                       vm_pageout_free_min = 10;
-       }
-
-       if (vm_page_free_target == 0)
-               vm_page_free_target = (vm_page_free_min * 4) / 3;
-
-       if (vm_page_inactive_target == 0)
-               vm_page_inactive_target = vm_page_free_min * 2;
-
-       if (vm_page_free_target <= vm_page_free_min)
-               vm_page_free_target = vm_page_free_min + 1;
-
-       if (vm_page_inactive_target <= vm_page_free_target)
-               vm_page_inactive_target = vm_page_free_target + 1;
+vmretry:
+       vm_page_free_min = npendingio/3;
+#ifdef VSMALL
+       vm_page_free_min = 8;
+#endif
+       vm_page_free_reserved = 14;
+       if (vm_page_free_min < 8)
+               vm_page_free_min = 8;
+       if (vm_page_free_min > 32)
+               vm_page_free_min = 32;
+       vm_pageout_free_min = 3;
+       vm_page_free_target = 2*vm_page_free_min + vm_page_free_reserved;
+       vm_page_inactive_target = 3*vm_page_free_min + vm_page_free_reserved;
+       vm_page_free_min += vm_page_free_reserved;
+
+       swiopend = npendingio;
+               
 
        /*
         *      The pageout daemon is never done, so loop
         *      forever.
         */
 
 
        /*
         *      The pageout daemon is never done, so loop
         *      forever.
         */
 
-       simple_lock(&vm_pages_needed_lock);
+               
        while (TRUE) {
        while (TRUE) {
-               thread_sleep((int) &vm_pages_needed, &vm_pages_needed_lock,
-                            FALSE);
-               cnt.v_scan++;
+               
+               splhigh();
+               if (vm_page_free_count > vm_page_free_min) {
+                       wakeup((caddr_t) &vm_page_free_count);
+                       tsleep((caddr_t)&vm_pages_needed, PVM, "psleep", 0);
+               } else {
+                       if (nswiodone) {
+                               spl0();
+                               goto dosync;
+                       }
+                       tsleep((caddr_t)&vm_pages_needed, PVM, "psleep1", 5);
+               }
+                       
+               spl0();
+
+               vm_pager_sync();
                vm_pageout_scan();
                vm_pageout_scan();
+       dosync:
                vm_pager_sync();
                vm_pager_sync();
-               simple_lock(&vm_pages_needed_lock);
-               thread_wakeup((int) &vm_page_free_count);
+               cnt.v_scan++;
+               wakeup((caddr_t) kmem_map);
        }
 }
        }
 }
index fef6ce4..a806059 100644 (file)
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
- *     from: @(#)vm_pageout.h  7.3 (Berkeley) 4/21/91
- *     $Id: vm_pageout.h,v 1.2 1993/10/16 16:20:49 rgrimes Exp $
- */
-
-/*
+ *     @(#)vm_pageout.h        7.3 (Berkeley) 4/21/91
+ *
+ *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  *
  * Copyright (c) 1987, 1990 Carnegie-Mellon University.
  * All rights reserved.
  *
@@ -63,9 +61,7 @@
  * any improvements or extensions that they make and grant Carnegie the
  * rights to redistribute these changes.
  */
  * any improvements or extensions that they make and grant Carnegie the
  * rights to redistribute these changes.
  */
-
-#ifndef _VM_VM_PAGEOUT_H_
-#define _VM_VM_PAGEOUT_H_ 1
+#include <sys/systm.h>
 
 /*
  *     Header file for pageout daemon.
 
 /*
  *     Header file for pageout daemon.
@@ -76,7 +72,7 @@
  */
 
 extern int     vm_pages_needed;        /* should be some "event" structure */
  */
 
 extern int     vm_pages_needed;        /* should be some "event" structure */
-extern simple_lock_data_t      vm_pages_needed_lock;
+simple_lock_data_t     vm_pages_needed_lock;
 
 
 /*
 
 
 /*
@@ -86,11 +82,21 @@ extern simple_lock_data_t   vm_pages_needed_lock;
 /*
  *     Signal pageout-daemon and wait for it.
  */
 /*
  *     Signal pageout-daemon and wait for it.
  */
+#define VM_WAIT vm_wait()
+inline static void vm_wait() {
+       extern struct proc *curproc, *pageproc;
+       extern int vm_pageout_pages_needed;
+       int s;
+       s = splhigh();
+       if (curproc == pageproc) {
+               vm_pageout_pages_needed = 1;
+               tsleep((caddr_t) &vm_pageout_pages_needed, PSWP, "vmwait", 0);
+               vm_pageout_pages_needed = 0;
+       } else {
+               wakeup((caddr_t) &vm_pages_needed);
+               tsleep((caddr_t) &vm_page_free_count, PVM, "vmwait", 0);
+       }
+       splx(s);
+}
+
 
 
-#define        VM_WAIT         { \
-                       simple_lock(&vm_pages_needed_lock); \
-                       thread_wakeup((int)&vm_pages_needed); \
-                       thread_sleep((int)&vm_page_free_count, \
-                               &vm_pages_needed_lock, FALSE); \
-                       }
-#endif /* _VM_VM_PAGEOUT_H_ */
index c23b001..48e4ae8 100644 (file)
@@ -34,7 +34,7 @@
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_pager.c    7.4 (Berkeley) 5/7/91
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_pager.c    7.4 (Berkeley) 5/7/91
- *     $Id: vm_pager.c,v 1.7 1993/12/19 23:24:17 wollman Exp $
+ *     $Id: vm_pager.c,v 1.8 1993/12/21 05:51:06 davidg Exp $
  */
 
 /*
  */
 
 /*
@@ -94,7 +94,7 @@ struct pagerops *dfltpagerops = NULL; /* default pager */
  * Kernel address space for mapping pages.
  * Used by pagers where KVAs are needed for IO.
  */
  * Kernel address space for mapping pages.
  * Used by pagers where KVAs are needed for IO.
  */
-#define PAGER_MAP_SIZE (256 * PAGE_SIZE)
+#define PAGER_MAP_SIZE (1024 * PAGE_SIZE)
 vm_map_t pager_map;
 vm_offset_t pager_sva, pager_eva;
 
 vm_map_t pager_map;
 vm_offset_t pager_sva, pager_eva;
 
@@ -146,6 +146,26 @@ vm_pager_deallocate(pager)
        VM_PAGER_DEALLOC(pager);
 }
 
        VM_PAGER_DEALLOC(pager);
 }
 
+int
+vm_pager_getmulti(pager, m, count, reqpage, sync)
+       vm_pager_t      pager;
+       vm_page_t       m;
+       int             count;
+       int             reqpage;
+       boolean_t       sync;
+{
+       extern boolean_t vm_page_zero_fill();
+       extern int vm_pageout_count;
+       int i;
+
+       if (pager == NULL) {
+               for (i=0;i<count;i++)
+                       vm_page_zero_fill(m+i);
+               return VM_PAGER_OK;
+       }
+       return(VM_PAGER_GET_MULTI(pager, m, count, reqpage, sync));
+}
+
 int
 vm_pager_get(pager, m, sync)
        vm_pager_t      pager;
 int
 vm_pager_get(pager, m, sync)
        vm_pager_t      pager;
index 32899be..51aa200 100644 (file)
@@ -36,7 +36,7 @@
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_pager.h    7.2 (Berkeley) 4/20/91
  * SUCH DAMAGE.
  *
  *     from: @(#)vm_pager.h    7.2 (Berkeley) 4/20/91
- *     $Id: vm_pager.h,v 1.2 1993/10/16 16:20:51 rgrimes Exp $
+ *     $Id: vm_pager.h,v 1.3 1993/12/19 00:56:14 wollman Exp $
  */
 
 /*
  */
 
 /*
@@ -67,6 +67,7 @@ struct        pagerops {
        vm_pager_t      (*pgo_alloc)();         /* allocate pager */
        void            (*pgo_dealloc)();       /* disassociate */
        int             (*pgo_getpage)();       /* get (read) page */
        vm_pager_t      (*pgo_alloc)();         /* allocate pager */
        void            (*pgo_dealloc)();       /* disassociate */
        int             (*pgo_getpage)();       /* get (read) page */
+       int             (*pgo_getmulti)();      /* get (read) multiple pages */
        int             (*pgo_putpage)();       /* put (write) page */
        boolean_t       (*pgo_haspage)();       /* does pager have page? */
 };
        int             (*pgo_putpage)();       /* put (write) page */
        boolean_t       (*pgo_haspage)();       /* does pager have page? */
 };
@@ -77,15 +78,18 @@ struct      pagerops {
  * BAD specified data was out of the accepted range
  * FAIL        specified data was in range, but doesn't exist
  * PEND        operations was initiated but not completed
  * BAD specified data was out of the accepted range
  * FAIL        specified data was in range, but doesn't exist
  * PEND        operations was initiated but not completed
+ * TRYAGAIN operation will be successful in the future
  */
 #define        VM_PAGER_OK     0
 #define        VM_PAGER_BAD    1
 #define        VM_PAGER_FAIL   2
 #define        VM_PAGER_PEND   3
  */
 #define        VM_PAGER_OK     0
 #define        VM_PAGER_BAD    1
 #define        VM_PAGER_FAIL   2
 #define        VM_PAGER_PEND   3
+#define        VM_PAGER_TRYAGAIN 4
 
 #define        VM_PAGER_ALLOC(h, s, p)         (*(pg)->pg_ops->pgo_alloc)(h, s, p)
 #define        VM_PAGER_DEALLOC(pg)            (*(pg)->pg_ops->pgo_dealloc)(pg)
 #define        VM_PAGER_GET(pg, m, s)          (*(pg)->pg_ops->pgo_getpage)(pg, m, s)
 
 #define        VM_PAGER_ALLOC(h, s, p)         (*(pg)->pg_ops->pgo_alloc)(h, s, p)
 #define        VM_PAGER_DEALLOC(pg)            (*(pg)->pg_ops->pgo_dealloc)(pg)
 #define        VM_PAGER_GET(pg, m, s)          (*(pg)->pg_ops->pgo_getpage)(pg, m, s)
+#define        VM_PAGER_GET_MULTI(pg, m, c, r, s)      (*(pg)->pg_ops->pgo_getmulti)(pg, m, c, r, s)
 #define        VM_PAGER_PUT(pg, m, s)          (*(pg)->pg_ops->pgo_putpage)(pg, m, s)
 #define        VM_PAGER_HASPAGE(pg, o)         (*(pg)->pg_ops->pgo_haspage)(pg, o)
 
 #define        VM_PAGER_PUT(pg, m, s)          (*(pg)->pg_ops->pgo_putpage)(pg, m, s)
 #define        VM_PAGER_HASPAGE(pg, o)         (*(pg)->pg_ops->pgo_haspage)(pg, o)
 
index 744ef36..00bf1cd 100644 (file)
@@ -2,6 +2,7 @@
  * Copyright (c) 1990 University of Utah.
  * Copyright (c) 1991 The Regents of the University of California.
  * All rights reserved.
  * Copyright (c) 1990 University of Utah.
  * Copyright (c) 1991 The Regents of the University of California.
  * All rights reserved.
+ * Copyright (c) 1993 John S. Dyson
  *
  * This code is derived from software contributed to Berkeley by
  * the Systems Programming Group of the University of Utah Computer
  *
  * This code is derived from software contributed to Berkeley by
  * the Systems Programming Group of the University of Utah Computer
@@ -36,7 +37,7 @@
  * SUCH DAMAGE.
  *
  *     from: @(#)vnode_pager.c 7.5 (Berkeley) 4/20/91
  * SUCH DAMAGE.
  *
  *     from: @(#)vnode_pager.c 7.5 (Berkeley) 4/20/91
- *     $Id: vnode_pager.c,v 1.6 1993/12/19 23:24:18 wollman Exp $
+ *     $Id: vnode_pager.c,v 1.7 1993/12/21 05:51:07 davidg Exp $
  */
 
 /*
  */
 
 /*
  *     fix credential use (uses current process credentials now)
  */
 
  *     fix credential use (uses current process credentials now)
  */
 
+/*
+ * MODIFICATIONS:
+ * John S. Dyson  08 Dec 93
+ *
+ * This file in conjunction with some vm_fault mods, eliminate the performance
+ * advantage for using the buffer cache and minimize memory copies.
+ *
+ * 1) Supports multiple - block reads
+ * 2) Bypasses buffer cache for reads
+ * 
+ * TODO:
+ *
+ * 1) Totally bypass buffer cache for reads
+ *    (Currently will still sometimes use buffer cache for reads)
+ * 2) Bypass buffer cache for writes
+ *    (Code does not support it, but mods are simple)
+ */
+
 #include "param.h"
 #include "param.h"
-#include "systm.h"
 #include "proc.h"
 #include "malloc.h"
 #include "vnode.h"
 #include "proc.h"
 #include "malloc.h"
 #include "vnode.h"
 #include "mount.h"
 
 #include "vm_param.h"
 #include "mount.h"
 
 #include "vm_param.h"
+#include "vm.h"
 #include "lock.h"
 #include "queue.h"
 #include "vm_prot.h"
 #include "vm_object.h"
 #include "vm_page.h"
 #include "vnode_pager.h"
 #include "lock.h"
 #include "queue.h"
 #include "vm_prot.h"
 #include "vm_object.h"
 #include "vm_page.h"
 #include "vnode_pager.h"
+#include "vm_map.h"
+#include "vm_pageout.h"
+#include "buf.h"
+#include "specdev.h"
 
 struct pagerops vnodepagerops = {
        vnode_pager_init,
        vnode_pager_alloc,
        vnode_pager_dealloc,
        vnode_pager_getpage,
 
 struct pagerops vnodepagerops = {
        vnode_pager_init,
        vnode_pager_alloc,
        vnode_pager_dealloc,
        vnode_pager_getpage,
+       vnode_pager_getmulti,
        vnode_pager_putpage,
        vnode_pager_haspage
 };
 
        vnode_pager_putpage,
        vnode_pager_haspage
 };
 
-static int vnode_pager_io(vn_pager_t, vm_page_t, enum uio_rw);
+static int vnode_pager_io(vn_pager_t vnp, vm_page_t *m, int count, int reqpage, 
+       enum uio_rw rw);
+struct buf * getpbuf() ;
+void relpbuf(struct buf *bp) ;
+
+extern vm_map_t pager_map;
 
 queue_head_t   vnode_pager_list;       /* list of managed vnodes */
 
 
 queue_head_t   vnode_pager_list;       /* list of managed vnodes */
 
@@ -210,6 +239,19 @@ vnode_pager_dealloc(pager)
        free((caddr_t)pager, M_VMPAGER);
 }
 
        free((caddr_t)pager, M_VMPAGER);
 }
 
+int
+vnode_pager_getmulti(pager, m, count, reqpage, sync)
+       vm_pager_t pager;
+       vm_page_t *m;
+       int count;
+       int reqpage;
+       boolean_t sync;
+{
+       
+       return vnode_pager_io((vn_pager_t) pager->pg_data, m, count, reqpage, UIO_READ);
+}
+
+
 int
 vnode_pager_getpage(pager, m, sync)
        vm_pager_t pager;
 int
 vnode_pager_getpage(pager, m, sync)
        vm_pager_t pager;
@@ -217,11 +259,17 @@ vnode_pager_getpage(pager, m, sync)
        boolean_t sync;
 {
 
        boolean_t sync;
 {
 
+       int err;
+       vm_page_t marray[1];
 #ifdef DEBUG
        if (vpagerdebug & VDB_FOLLOW)
                printf("vnode_pager_getpage(%x, %x)\n", pager, m);
 #endif
 #ifdef DEBUG
        if (vpagerdebug & VDB_FOLLOW)
                printf("vnode_pager_getpage(%x, %x)\n", pager, m);
 #endif
-       return(vnode_pager_io((vn_pager_t)pager->pg_data, m, UIO_READ));
+       if (pager == NULL)
+               return FALSE;
+       marray[0] = m;
+
+       return vnode_pager_io((vn_pager_t)pager->pg_data, marray, 1, 0, UIO_READ);
 }
 
 boolean_t
 }
 
 boolean_t
@@ -231,19 +279,17 @@ vnode_pager_putpage(pager, m, sync)
        boolean_t sync;
 {
        int err;
        boolean_t sync;
 {
        int err;
+       vm_page_t marray[1];
 
 #ifdef DEBUG
        if (vpagerdebug & VDB_FOLLOW)
                printf("vnode_pager_putpage(%x, %x)\n", pager, m);
 #endif
        if (pager == NULL)
 
 #ifdef DEBUG
        if (vpagerdebug & VDB_FOLLOW)
                printf("vnode_pager_putpage(%x, %x)\n", pager, m);
 #endif
        if (pager == NULL)
-               return 0;
-       err = vnode_pager_io((vn_pager_t)pager->pg_data, m, UIO_WRITE);
-       if (err == VM_PAGER_OK) {
-               m->flags |= PG_CLEAN;                   /* XXX - wrong place */
-               pmap_clear_modify(VM_PAGE_TO_PHYS(m));  /* XXX - wrong place */
-       }
-       return(err);
+               return FALSE;
+       marray[0] = m;
+       err = vnode_pager_io((vn_pager_t)pager->pg_data, marray, 1, 0, UIO_WRITE);
+       return err;
 }
 
 boolean_t
 }
 
 boolean_t
@@ -345,6 +391,7 @@ vnode_pager_setsize(vp, nsize)
         * File has shrunk.
         * Toss any cached pages beyond the new EOF.
         */
         * File has shrunk.
         * Toss any cached pages beyond the new EOF.
         */
+       nsize = round_page(nsize);
        if (nsize < vnp->vnp_size) {
                vm_object_lock(object);
                vm_object_page_remove(object,
        if (nsize < vnp->vnp_size) {
                vm_object_lock(object);
                vm_object_page_remove(object,
@@ -420,78 +467,481 @@ vnode_pager_uncache(vp)
        return(uncached);
 }
 
        return(uncached);
 }
 
-static int
-vnode_pager_io(vnp, m, rw)
-       register vn_pager_t vnp;
+
+void
+vnode_pager_freepage(m)
        vm_page_t m;
        vm_page_t m;
+{
+       PAGE_WAKEUP(m);
+       vm_page_free(m);
+}
+
+/*
+ * calculate the linear (byte) disk address of specified virtual
+ * file address
+ */
+vm_offset_t
+vnode_pager_addr(vp, address)
+       struct vnode *vp;
+       vm_offset_t address;
+{
+       int rtaddress;
+       int bsize;
+       vm_offset_t block;
+       struct vnode *rtvp;
+       int err;
+       int vblock, voffset;
+
+       bsize = vp->v_mount->mnt_stat.f_bsize;
+       vblock = address / bsize;
+       voffset = address % bsize;
+
+       err = VOP_BMAP(vp,vblock,&rtvp,&block);
+
+       rtaddress = block * DEV_BSIZE + voffset;
+
+       return rtaddress;
+}
+
+/*
+ * interrupt routine for I/O completion
+ */
+void
+vnode_pager_iodone(bp)
+       struct buf *bp;
+{
+       bp->b_flags |= B_DONE;
+       wakeup((caddr_t)bp);
+}
+
+/*
+ * vnode_pager_io:
+ *     Perform read or write operation for vnode_paging
+ *
+ *     args:
+ *               vnp -- pointer to vnode pager data structure
+ *                      containing size and vnode pointer, etc 
+ *
+ *               m   -- pointer to array of vm_page_t entries to
+ *                      do I/O to.  It is not necessary to fill any
+ *                      pages except for the reqpage entry.  If a
+ *                      page is not filled, it needs to be removed
+ *                      from its object...
+ *
+ *               count -- number of pages for I/O
+ *
+ *               reqpage -- fault requested page for I/O
+ *                          (index into vm_page_t entries above)
+ *
+ *               rw -- UIO_READ or UIO_WRITE
+ *
+ * NOTICE!!!! direct writes look like that they are close to being
+ *            implemented.  They are not really,  several things need
+ *            to be done to make it work (subtile things.)  Hack at
+ *            your own risk (direct writes are scarey).
+ */
+
+int
+vnode_pager_io(vnp, m, count, reqpage, rw)
+       register vn_pager_t vnp;
+       vm_page_t *m;
+       int count, reqpage;
        enum uio_rw rw;
 {
        enum uio_rw rw;
 {
+       int i,j;
        struct uio auio;
        struct iovec aiov;
        vm_offset_t kva, foff;
        struct uio auio;
        struct iovec aiov;
        vm_offset_t kva, foff;
-       int error, size;
+       int size;
        struct proc *p = curproc;               /* XXX */
        struct proc *p = curproc;               /* XXX */
+       vm_object_t object;
+       vm_offset_t paging_offset;
+       struct vnode *dp, *vp;
+       vm_offset_t mapsize;
+       int bsize;
+       int errtype=0; /* 0 is file type otherwise vm type */
+       int error = 0;
+
+       object = m[0]->object;  /* all vm_page_t items are in same object */
+       paging_offset = object->paging_offset;
 
 
-#ifdef DEBUG
-       if (vpagerdebug & VDB_FOLLOW)
-               printf("vnode_pager_io(%x, %x, %c): vnode %x\n",
-                      vnp, m, rw == UIO_READ ? 'R' : 'W', vnp->vnp_vp);
-#endif
-       foff = m->offset + m->object->paging_offset;
        /*
        /*
-        * Return failure if beyond current EOF
+        * get the UNDERLYING device for the file
         */
         */
-       if (foff >= vnp->vnp_size) {
-#ifdef DEBUG
-               if (vpagerdebug & VDB_SIZE)
-                       printf("vnode_pager_io: vp %x, off %d size %d\n",
-                              vnp->vnp_vp, foff, vnp->vnp_size);
-#endif
-               return(VM_PAGER_BAD);
+       vp = vnp->vnp_vp;
+       bsize = vp->v_mount->mnt_stat.f_bsize;
+       VOP_BMAP(vp, 0, &dp, 0);
+
+       /*
+        * trim off unnecessary pages
+        */
+       for (i = reqpage - 1; i >= 0; --i) {
+               if (m[i]->object != object) {
+                       for (j = 0; j <= i; j++)
+                               vnode_pager_freepage(m[j]);
+                       for (j = i + 1; j < count; j++) {
+                               m[j - (i + 1)] = m[j];
+                       }
+                       count -= i + 1;
+                       reqpage -= i + 1;
+                       break;
+               }
        }
        }
-       if (foff + PAGE_SIZE > vnp->vnp_size)
-               size = vnp->vnp_size - foff;
-       else
-               size = PAGE_SIZE;
+       for (i = reqpage + 1; i < count; i++) {
+               if ((m[i]->object != object) ||
+                       (m[i]->offset + paging_offset >= vnp->vnp_size)) {
+                       for (j = i; j < count; j++)
+                               vnode_pager_freepage(m[j]);
+                       count = i;
+                       break;
+               }
+       }
+
+       /*
+        * we only do direct I/O if the file is on a local
+        * BLOCK device and currently if it is a read operation only.
+        */
+
+       kva = 0;
+       mapsize = 0;
+       if (rw == UIO_READ && dp->v_type == VBLK &&
+               vp->v_mount->mnt_stat.f_type == MOUNT_UFS) {
+               /*
+                * we do not block for a kva, notice we default to a kva conservative behavior
+                */
+               kva = kmem_alloc_pageable(pager_map, (mapsize = count*NBPG));
+       }
+
+       if (!kva) {
+               /*
+                * here on I/O through VFS
+                */
+
+               for (i = 0; i < count; i++) {
+                       foff = m[i]->offset + paging_offset;
+       /*
+        * Return failure if beyond current EOF
+        */
+                       if (foff >= vnp->vnp_size) {
+                               if (i == reqpage) {
+                                       errtype = 1;
+                                       error = VM_PAGER_BAD;
+                                       i += 1;
+                               }
+                               for (j = i; j < count; j++) {
+                                       if (j != reqpage) {
+                                               vnode_pager_freepage(m[j]);
+                                               m[j] = 0;
+                                       } else {
+                                               errtype = 1;
+                                               error = VM_PAGER_BAD;
+                                       }
+                               }
+                               break;
+                       }
+                       if (foff + NBPG > vnp->vnp_size)
+                               size = vnp->vnp_size - foff;
+                       else
+                               size = NBPG;
        /*
         * Allocate a kernel virtual address and initialize so that
         * we can use VOP_READ/WRITE routines.
         */
        /*
         * Allocate a kernel virtual address and initialize so that
         * we can use VOP_READ/WRITE routines.
         */
-       kva = vm_pager_map_page(m);
-       aiov.iov_base = (caddr_t)kva;
-       aiov.iov_len = size;
-       auio.uio_iov = &aiov;
-       auio.uio_iovcnt = 1;
-       auio.uio_offset = foff;
-       auio.uio_segflg = UIO_SYSSPACE;
-       auio.uio_rw = rw;
-       auio.uio_resid = size;
-       auio.uio_procp = (struct proc *)0;
-#ifdef DEBUG
-       if (vpagerdebug & VDB_IO)
-               printf("vnode_pager_io: vp %x kva %x foff %x size %x",
-                      vnp->vnp_vp, kva, foff, size);
-#endif
-       if (rw == UIO_READ)
-               error = VOP_READ(vnp->vnp_vp, &auio, 0, p->p_ucred);
-       else
-               error = VOP_WRITE(vnp->vnp_vp, &auio, 0, p->p_ucred);
-#ifdef DEBUG
-       if (vpagerdebug & VDB_IO) {
-               if (error || auio.uio_resid)
-                       printf(" returns error %x, resid %x",
-                              error, auio.uio_resid);
-               printf("\n");
+                       kva = vm_pager_map_page(m[i]);
+                       aiov.iov_base = (caddr_t)kva;
+                       aiov.iov_len = size;
+                       auio.uio_iov = &aiov;
+                       auio.uio_iovcnt = 1;
+                       auio.uio_offset = foff;
+                       auio.uio_segflg = UIO_SYSSPACE;
+                       auio.uio_rw = rw;
+                       auio.uio_resid = size;
+                       auio.uio_procp = (struct proc *)0;
+                       if (rw == UIO_READ) {
+                               error = VOP_READ(vp, &auio, IO_PAGER, p->p_ucred);
+                       } else {
+                               error = VOP_WRITE(vp, &auio, IO_PAGER, p->p_ucred);
+                       }
+                       if (!error) {
+                               register int count = size - auio.uio_resid;
+       
+                               if (count == 0)
+                                       error = EINVAL;
+                               else if (count != NBPG && rw == UIO_READ)
+                                       bzero((caddr_t)kva + count, NBPG - count);
+                       }
+                       vm_pager_unmap_page(kva);
+               }
+       } else {
+
+               /*
+                * here on direct device I/O
+                */
+               int first=0, last=count;
+               int reqaddr, firstaddr;
+               int block, offset;
+               
+               struct buf *bp;
+               int s;
+               int failflag;
+
+               foff = m[reqpage]->offset + paging_offset;
+
+               /*
+                * This pathetic hack gets data from the buffer cache, if it's there.
+                * I believe that this is not really necessary, and the ends can
+                * be gotten by defaulting to the normal vfs read behavior, but this
+                * might be more efficient, because the will NOT invoke read-aheads
+                * and one of the purposes of this code is to bypass the buffer
+                * cache and keep from flushing it by reading in a program.
+                */
+               /*
+                * calculate logical block and offset
+                */
+               block = foff / bsize;
+               offset = foff % bsize;
+               s = splbio();
+
+               /*
+                * if we have a buffer in core, then try to use it
+                */
+               while (bp = incore(vp, block)) {
+                       int amount;
+                                       
+                       /*
+                        * wait until the buffer is avail or gone
+                        */
+                       if (bp->b_flags & B_BUSY) {
+                               bp->b_flags |= B_WANTED;
+                               tsleep ((caddr_t)bp, PVM, "vnwblk", 0);
+                               continue;
+                       }
+
+                       amount = NBPG;
+                       if ((foff + amount) > vnp->vnp_size)
+                               amount = vnp->vnp_size - foff;
+
+                       /*
+                        * make sure that this page is in the buffer
+                        */
+                       if ((amount > 0) && (offset + amount) <= bp->b_bcount) {
+                               bp->b_flags |= B_BUSY;
+                               splx(s);
+
+                               /*
+                                * map the requested page
+                                */
+                               pmap_enter(vm_map_pmap(pager_map),
+                                       kva, VM_PAGE_TO_PHYS(m[reqpage]),
+                                       VM_PROT_DEFAULT, TRUE);
+                               /*
+                                * copy the data from the buffer
+                                */
+                               bcopy(bp->b_un.b_addr + offset, (caddr_t)kva, amount);
+                               if (amount < NBPG) {
+                                       bzero((caddr_t)kva + amount, NBPG - amount);
+                               }
+                               /*
+                                * unmap the page and free the kva
+                                */
+                               pmap_remove(vm_map_pmap(pager_map), kva, kva + NBPG);
+                               kmem_free_wakeup(pager_map, kva, mapsize);
+                               /*
+                                * release the buffer back to the block subsystem
+                                */
+                               bp->b_flags &= ~B_BUSY;
+                               wakeup((caddr_t)bp);
+                               /*
+                                * we did not have to do any work to get the requested
+                                * page, the read behind/ahead does not justify a read
+                                */
+                               for (i = 0; i < count; i++) {
+                                       if (i != reqpage) {
+                                               vnode_pager_freepage(m[i]);
+                                               m[i] = 0;
+                                       }
+                               }
+                               /*
+                                * sorry for the goto
+                                */
+                               goto finishup;
+                       }
+                       /*
+                        * buffer is nowhere to be found, read from the disk
+                        */
+                       break;
+               }
+               splx(s);
+
+               foff = m[reqpage]->offset + paging_offset;
+               reqaddr = vnode_pager_addr(vp, foff);
+               /*
+                * Make sure that our I/O request is contiguous.
+                * Scan backward and stop for the first discontiguous
+                *      entry or stop for a page being in buffer cache.
+                */
+               failflag = 0;
+               for (i = reqpage - 1; i >= 0; --i) {
+                       int myaddr;
+                       if (failflag ||
+                          incore(vp, (foff + (i - reqpage) * NBPG) / bsize) ||
+                          (myaddr = vnode_pager_addr(vp, m[i]->offset + paging_offset))
+                                       != reqaddr + (i - reqpage) * NBPG) {
+                               vnode_pager_freepage(m[i]);
+                               m[i] = 0;
+                               if (first == 0)
+                                       first = i + 1;
+                               failflag = 1;
+                       }
+               }
+
+               /*
+                * Scan forward and stop for the first discontiguous
+                * entry or stop for a page being in buffer cache.
+                */
+               failflag = 0;
+               for (i = reqpage + 1; i < count; i++) {
+                       int myaddr;
+                       if (failflag ||
+                          incore(vp, (foff + (i - reqpage) * NBPG) / bsize) ||
+                          (myaddr = vnode_pager_addr(vp, m[i]->offset + paging_offset))
+                                       != reqaddr + (i - reqpage) * NBPG) {
+                               vnode_pager_freepage(m[i]);
+                               m[i] = 0;
+                               if (last == count)
+                                       last = i;
+                               failflag = 1;
+                       }
+               }
+
+               /*
+                * the first and last page have been calculated now, move input
+                * pages to be zero based...
+                */
+               count = last;
+               if (first != 0) {
+                       for (i = first; i < count; i++) {
+                               m[i - first] = m[i];
+                       }
+                       count -= first;
+                       reqpage -= first;
+               }
+
+
+               /*
+                * calculate the file virtual address for the transfer
+                */
+               foff = m[0]->offset + paging_offset;
+               /*
+                * and get the disk physical address (in bytes)
+                */
+               firstaddr = vnode_pager_addr(vp, foff);
+
+               /*
+                * calculate the size of the transfer
+                */
+               if ((m[count - 1]->offset + paging_offset) + NBPG > vnp->vnp_size)
+                       size = vnp->vnp_size - foff;
+               else
+                       size = count * NBPG;
+
+
+               /*
+                * and map the pages to be read into the kva
+                */
+               for (i = 0; i < count; i++)
+                       pmap_enter(vm_map_pmap(pager_map),
+                               kva + NBPG * i, VM_PAGE_TO_PHYS(m[i]),
+                               VM_PROT_DEFAULT, TRUE);
+               VHOLD(vp);
+               bp = getpbuf();
+
+               /* build a minimal buffer header */
+               bzero((caddr_t)bp, sizeof(struct buf));
+               bp->b_flags = B_BUSY | B_READ | B_CALL;
+               bp->b_iodone = vnode_pager_iodone;
+               /* B_PHYS is not set, but it is nice to fill this in */
+               bp->b_proc = &proc0;
+               bp->b_un.b_addr = (caddr_t) kva;
+               bp->b_blkno = firstaddr / DEV_BSIZE;
+               bp->b_vp = dp;
+       
+               /* Should be a BLOCK or character DEVICE if we get here */
+               bp->b_dev = dp->v_rdev;
+               bp->b_bcount = NBPG * count;
+
+               /* do the input */
+               VOP_STRATEGY(bp);
+
+               /* we definitely need to be at splbio here */
+
+               while ((bp->b_flags & B_DONE) == 0) {
+                       tsleep((caddr_t)bp, PVM, "vnread", 0);
+               }
+               splx(s);
+               if ((bp->b_flags & B_ERROR) != 0)
+                       error = EIO;
+
+               if (!error) {
+                       if (size != count * NBPG)
+                               bzero((caddr_t)kva + size, NBPG * count - size);
+               }
+               HOLDRELE(vp);
+
+               pmap_remove(vm_map_pmap(pager_map), kva, kva + NBPG * count);
+               kmem_free_wakeup(pager_map, kva, mapsize);
+
+               /*
+                * free the buffer header back to the swap buffer pool
+                */
+               relpbuf(bp);
+
        }
        }
-#endif
-       if (!error) {
-               register int count = size - auio.uio_resid;
 
 
-               if (count == 0)
-                       error = EINVAL;
-               else if (count != PAGE_SIZE && rw == UIO_READ)
-                       bzero((caddr_t)(kva + count), PAGE_SIZE - count);
+finishup:
+       if (rw == UIO_READ)
+       for (i = 0; i < count; i++) {
+               /*
+                * we dont mess with pages that have been already
+                * deallocated....
+                */
+               if (!m[i]) 
+                       continue;
+               pmap_clear_modify(VM_PAGE_TO_PHYS(m[i]));
+               m[i]->flags |= PG_CLEAN;
+               m[i]->flags &= ~PG_LAUNDRY;
+               if (i != reqpage) {
+                       /*
+                        * whether or not to leave the page activated
+                        * is up in the air, but we should put the page
+                        * on a page queue somewhere. (it already is in
+                        * the object).
+                        */
+                       /*
+                        * just in case someone was asking for this
+                        * page we now tell them that it is ok to use
+                        */
+                       if (!error) {
+                               if (i <= reqpage + 3)
+                                       vm_page_activate(m[i]);
+                               else
+                                       vm_page_deactivate(m[i]);
+                               PAGE_WAKEUP(m[i]);
+                               m[i]->flags &= ~PG_FAKE;
+                       } else {
+                               vnode_pager_freepage(m[i]);
+                       }
+               }
+       }
+       if (!error && rw == UIO_WRITE) {
+               pmap_clear_modify(VM_PAGE_TO_PHYS(m[reqpage]));
+               m[reqpage]->flags |= PG_CLEAN;
+               m[reqpage]->flags &= ~PG_LAUNDRY;
+       }
+       if (error) {
+               printf("vnode pager error: %d\n", error);
        }
        }
-       vm_pager_unmap_page(kva);
+       if (errtype)
+               return error;
        return (error ? VM_PAGER_FAIL : VM_PAGER_OK);
 }
        return (error ? VM_PAGER_FAIL : VM_PAGER_OK);
 }
index dd9b37e..78723e3 100644 (file)
@@ -36,7 +36,7 @@
  * SUCH DAMAGE.
  *
  *     from: @(#)vnode_pager.h 7.1 (Berkeley) 12/5/90
  * SUCH DAMAGE.
  *
  *     from: @(#)vnode_pager.h 7.1 (Berkeley) 12/5/90
- *     $Id: vnode_pager.h,v 1.2 1993/10/16 16:21:03 rgrimes Exp $
+ *     $Id: vnode_pager.h,v 1.3 1993/11/07 17:54:32 wollman Exp $
  */
 
 #ifndef        _VNODE_PAGER_
  */
 
 #ifndef        _VNODE_PAGER_
@@ -63,6 +63,7 @@ void          vnode_pager_init();
 vm_pager_t     vnode_pager_alloc();
 void           vnode_pager_dealloc();
 int            vnode_pager_getpage();
 vm_pager_t     vnode_pager_alloc();
 void           vnode_pager_dealloc();
 int            vnode_pager_getpage();
+int            vnode_pager_getmulti();
 int            vnode_pager_putpage();
 boolean_t      vnode_pager_haspage();
 
 int            vnode_pager_putpage();
 boolean_t      vnode_pager_haspage();