Memory Management Short Notes

Basic Concepts
A computer's memory system is core to its functionality and operations. It refers to the data
storage parts in a computer that can store instructions and data over various time durations.
The primary components of a computer's memory system are the CPU registers, cache, main
memory (RAM), and secondary storage devices.
The CPU contains a small set of registers that provide direct access for storing and operating
on data needed currently. Cache memory is a very high-speed SRAM memory located closer
to the CPU core and used as a buffer for frequently accessed data and instructions. Main
memory or RAM provides generally fast access, allows random access due to which data items
can be read or written in any order. RAM being volatile loses its data when power is switched
off. Secondary storage devices like hard disks, magnetic tapes and SSDs serve as permanent
large capacity storage, maintain data persistently when powered off, but provide relatively
slower access. The backing storage acts as a reservoir supplying data and instructions to main
memory which in turn supplies currently active data to cache and registers as needed by the
CPU for ongoing computations. Memory organization and management thus has significant
influence on performance.

Logical Address Space and Logical Address

The logical address space is defined as the total memory size that is available to a process for
its execution logically. Even if the physical memory is lesser, the process can have the illusion
as if a very large memory space is available through techniques like virtual memory. The CPU
generates logical addresses when accessing instructions and data in memory during a
program's execution. The logical address size depends on the size of the logical address space.
For example, if the logical space is 128MB (2^27 bytes) and word size is 4 bytes (2^2 bytes),
then the Logical Address = log2(L) = log2(2^27/2^2) = 25 bits.

Physical Address Space and Physical Address

The physical address space refers to the actual main memory size in the computer. For
instance, if the physical memory is 64KB (2^16 bytes) and word size is 8 bytes (2^3 bytes),
then Physical Address Space (in words) = (2^16)/(2^3) = 2^13 words. Hence, Physical Address
= log2(N) = log2(2^13) = 13 bits. The physical address is the actual address of a memory
location in the main memory.

Mapping Between Logical and Physical Address

The CPU accesses memory locations using logical addresses. The Memory Management Unit
(MMU) in the operating system is responsible for mapping logical to physical addresses.
Mapping is done using data structures like page tables. The physical address is obtained by
replacing the page number portion of the logical address with the corresponding frame
number in which the page is stored. The offset portion remains unchanged. This scheme
provides virtual memory management support.
The CPU generates logical addresses to access instructions and data. However, the actual
physical memory in a system has physical addresses assigned to each location. There needs to
be a mapping mechanism to convert logical to physical addresses. The logical address is
divided into two parts - a page number and offset within the page. For example, for a 16-bit
logical address, upper 8 bits could be the page number and lower 8 bits the offset.
The Memory Management Unit (MMU) handles the logical to physical translation using
mapping information stored in a page table. Page table contains one entry for every page, and
an entry consists of the frame number and other metadata like protection bits, dirty bit etc.
So, when the CPU provides the logical address, the MMU extracts the page number, and uses
it as an index into the page table. From the page table entry, it finds out the frame number in
physical memory where that page is placed.
The offset part of the logical address is directly copied and appended to this frame number to
generate the final physical address. The byte/word at the physical address is accessed to
complete the access. This separation of logical address space from physical address space via
paging/mapping allows overcommitting physical memory, Demand paging from disk to
physical memory and easier relocation/allocation benefits for the Operating System via virtual
memory management.
The mapping of logical to physical addresses using a page table as described requires the
Memory Management Unit (MMU) to access the correct page table and entry every time. To
speed up this translation, dedicated hardware registers are employed. The base register
contains the starting physical address of the page table in main memory. The logical page
number needs to be added to this base address to index into the page table and find the
corresponding entry. The limit register indicates the length of the page table or the maximum
permissible logical page number. The MMU ensures that the page number used as index lies
within the limits set by this register. This avoids segmentation faults. The base and limit
registers help locate the relevant page table entry faster by avoiding recalculating the page
table location each time. Appropriately setting these registers forms part of the context
switching when switching between processes. The registers provide mapping flexibility and
speed up virtual address translation.

Memory allocation
Contiguous Memory Allocation refers to the technique of allocating physical memory to
processes in a manner where all memory allocated to a process is stored in contiguous blocks.
The contiguous allocation can be done in two ways:

Fixed Partitioning:
In fixed partitioning, the main memory is divided into partitions of fixed sizes that do not
change dynamically. The partition sizes are set by the operating system during OS installation.
When a process arrives, it is allocated memory from a partition large enough to accommodate
it. If no suitable partition is available, the process waits until one frees up.

Problems with Fixed Partitioning:

Internal Fragmentation: Since partition sizes are fixed, there may be waste/unused space left
in a partition if the process allocated is smaller than the partition size. This unused space
within a partition is termed as internal fragmentation.
External Fragmentation: Over time, when processes complete and partitions get freed non-
contiguously, free memory gets fragmented across memory and is unavailable for allocation
to large processes even if sum of individual free spaces is enough. This is called external
fragmentation.
Degree of Multiprogramming is limited. The number of processes that can be accommodated
simultaneously depends strictly on the initially defined fixed partitions.
Variable Partition Dynamic Allocation:
To overcome limitations of Fixed Partitioning, partitions can be created dynamically to be just
large enough to hold processes. When a process terminates, its memory is deallocated, and
the free space merged into contiguous spaces maintained using algorithms like first-fit, best-
fit etc.
First Fit:

• The memory is divided into fixed-sized partitions.

• When a process arrives, the allocator assigns the first available partition that is large
enough to accommodate the process.
• Simple and fast but can lead to fragmentation.
Best Fit:

• Similar to first fit, but the allocator selects the smallest available partition that fits the
process's size.
• Reduces external fragmentation but may lead to small, unusable holes.
Worst Fit:

• Allocates the largest available partition to the arriving process.

• Tends to produce larger fragments and is generally less efficient than first fit and best
fit.
Simulations have shown that both first fit and best fit are better than worst fit in terms of
decreasing time and storage utilization. Neither first fit nor best fit is clearly better than the
other in terms of storage utilization, but first fit is generally faster.

Problems with Dynamic Allocation:

While internal fragmentation is reduced, external fragmentation still persists unless
compaction is done periodically. Keeping track of free partitions also adds OS overhead.

Compaction:
The process of defragmentation in memory by shuffling allocated and free partitions so all
free memory gets into one contiguous block is called compaction. After compaction, the free
space is available to accommodate any large process requests. However, compaction is a
complex and compute-intensive process. Doing compactions too frequently degrades
performance.

Paging
Paging Principles:
Paging is a memory management technique that allows allocation of non-contiguous memory
to processes. In paging, the logical memory space of a process is broken into blocks of equal
size called pages. The physical main memory is divided into frames of equal size. When a
process is executed, its pages are loaded into available free frames in physical memory.
Paging enables allocating free frames spread across memory to a process unlike contiguous
allocation schemes. It also facilitates swapping to disk when free frames aren’t available,
enabling higher multiprogramming levels and process loads.

Page Table and Address Translation:

A page table is used to keep track of the frame locations of each page of a process. The page
table resides in the main memory. A page table consists of page entries, with each entry
storing one page's frame location.
The logical address generated by CPU has two parts - a page number and offset within page.
The Memory Management Unit uses the page number to index into the page table and fetch
the corresponding frame number. This frame number and offset are concatenated to generate
the physical address sent to memory.

Page Allocation:
Initially all frames are marked free. For each process to be executed, its first page is allocated
a free frame, the frame is marked allocated, and entry created in the page table. This
continues until frames exist or process pages get exhausted. If free frames aren't available,
pages are swapped to disk using algorithms like FIFO, LRU etc. When a process terminates, its
frames are reset to available.
Hardware Support:
Paging is facilitated by page tables residing in memory and TLB registers to cache page table
entries, enabling faster lookups. Registers also exist tracking information like page table
length, access bits per page, dirty bits etc. which assist page replacement algorithms and
speedups. The hardware raises page faults on invalid accesses for OS to handle via transfers
from disk.

Protection and Sharing:

Paging inherently provides memory protection from errant processes attempting to overwrite
other process spaces or kernel space, by validating page table controlled access permissions
per process. Page sharing across cooperative processes also naturally falls out where multiple
processes map a shared code/data page to the same physical frame simultaneously.

Disadvantages and Challenges:

Internal fragmentation exists within partially filled last pages. Page tables can be large, wasting
substantial memory. Locating free frames and managing page tables dynamically adds
overheads. Excess paging to disk due to unavailability of free frames causes performance
issues. Optimizing page size, replacement, allocation algorithms are key challenges.
Virtual Memory
Virtual memory is an abstraction that provides each process with the illusion of having a very
large contiguous logical memory. It enables having many actively executing processes that are
larger than total physical memory available. This is facilitated by having only some of the
process pages resident in main memory at a time, and the remaining ones mapped to
secondary storage locations.
The execution shifts pages back and forth between disk and memory as needed through
demand paging. Page faults trigger page swaps to bring required pages into available memory
frames transparently. The hardware Memory Management Unit handles mapping logical to
physical addresses for seamless access across the actual physical memory available.

Hardware and Control Structures:

Specialized hardware support registers and Translation Lookaside Buffers speed up the logical
to physical address translations. These store page table, process state and control info about
frames, permissions, access bits etc.
The OS handles allocation and de-allocation of frames across processes, manages secondary
storage assignments, responds to page faults by bringing in required pages from disk, decides
which pages to replace using algorithms when mem is scarce - all through integrated data
structures like page tables, queues, bitmaps etc.

Locality of Reference:
Programs and code exhibit locality of reference - a tendency that over small time intervals,
limited sets of instructions and data are actively referenced repeatedly before switching to
another set. Virtual memory systems optimize for such behaviour. Recently used active page
working sets stay resident near the CPU minimizing access costs due to locality.

Page Faults:
Attempting to access an address mapping to a non-resident page triggers a page fault -
handled by fetching the required page containing the address from disk into a free frame,
updating mappings and resuming the faulting instruction. Multiple page faults lead to
performance degradation.

Working Set:
The actively referenced pages over recent instruction windows constitute the working set.
Virtual memory subsystems strive to ensure the working set pages are memory resident
through prefetching and replacement of inactive pages.

Dirty Bit and Page:

Modified pages are marked dirty - indicating pending synchronization of changed contents
with secondary storage version of the page. This avoids overwriting secondary storage's page
with unchanged data and optimizations like copy-on-write when possible.
Demand Paging:
Demand paging is a key concept in virtual memory systems. It refers to the technique of
loading process pages into frames in main memory only on demand i.e. when they are actually
required during program execution.
When a process starts executing, none of its pages are loaded upfront. The CPU generates the
logical address space with virtual addresses assuming all pages available. The OS sets page
table mappings to mark all pages absent/invalid initially. The first page access faults and raises
an exception that is handled by bringing that page from secondary storage into a free frame.
The mappings are updated accordingly. Thus pages are paged in from disk on demand.

Page Replacement Algorithms:

When a requested page is not present in memory and no free frames exist, the OS has to
select an existing allocated page, write it back to disk if dirty and free its frame to
accommodate the new page. This leads to the following class of page replacement algorithms:

Optimal Algorithm:
This theoretical algorithm selects the page that will not be used for the longest period in the
future. While providing optimal benchmarks, it is impossible to implement in real-world
scenarios due to the need for future knowledge of page accesses.

First in First Out (FIFO):

This algorithm selects the oldest page in memory based on the order of insertion.
Implementation is straightforward using a queue, but it is not adaptable to changing access
patterns, and it may suffer from the "aging" problem where long-resident pages are penalized.

Second Chance (SC) / Clock Algorithm:

The Second Chance algorithm combines aspects of FIFO with a check of an access bit. It
considers the FIFO order but gives a second chance to pages that have been referenced. This
helps prevent the eviction of frequently accessed pages, making it better than pure FIFO in
certain scenarios.

Not Recently Used (NRU):

NRU prioritizes pages based on whether they have been referenced recently, using access bits.
It combines information from the history of accesses and recent past to categorize pages. This
algorithm is designed to strike a balance between recency and frequency of access in making
eviction decisions.

Least Recently Used (LRU):

LRU selects the page that has not been referenced for the longest time. It is considered close
to optimal in terms of minimizing page faults. While conceptually strong, efficient hardware
implementations of LRU can be challenging, leading to various approximations and heuristics
in practical systems. Fundamentally trade-offs exist between complexity of trying to predict
working set pages that will not be reused shortly vs easily implemented algorithms like FIFO
that disregard access patterns.
Implementing LRU purely in software can add substantial overheads. TLB reach field
algorithms provide hardware LRU approximations by capturing temporal order information
across smaller time spans. Another hybrid technique incorporates software simulated LRU
over large intervals combined with hardware implemented access bit marking across smaller
intervals.
Simulation across longer intervals prone to errors making Pure LRU costly. Combining software
and hardware support provides a robust solution balancing accuracy and overheads - like
Second Chance augmenting FIFO using hardware access bits as example.
TLB reach algorithms assign an ordinal value to page table entries mapping to order of
translation lookups such that least valued entry is LRU candidate for replacement.
Performance counters allow smoothing estimates across random fluctuations.

Optimality vs Overheads:
Simulation policies provide a spectrum with varying overheads and optimality tradeoffs to
choose from based on constraints and hardware available. Getting page replacements wrong
leads to thrashing severely impacting performance. The policies continue to be an active
research area to accommodate modern application working sets.