Advanced Systems Programming H (2021-2022)
Lecture 5: Resource Ownership and Memory Management
This lecture begins to discuss the various options available for resource ownership and memory management in modern programming languages. It discusses how running programs are arranged in memory, and outlines the need for automatic memory management. It then discusses two approaches to memory management: reference counting and region-based memory management, considering their various costs and benefits. The region-based approach used in Rust is shown to have several important benefits, in terms of efficient and deterministic memory management, but at the cost of making certain classes of program more complex to express.
Part 1: Memory
The first part of this lecture reviews of a process is stored in memory, and what different types of memory exist within a running process, as a prelude to the discussion of automatic memory management. It also discusses how memory-unsafe behaviour can lead to security vulnerabilities through buffer overflows attacks.
00:00:00.200 In this week’s lecture, I want to
00:00:02.200 talk about resource ownership and memory management.
00:00:06.000 In this first part, I’ll talk about
00:00:07.866 how a process is stored in memory,
00:00:10.100 and what memory has to be managed.
00:00:12.400 Then, in the following parts,
00:00:14.300 I’ll start our discussion of different
00:00:16.166 approaches to memory management,
00:00:18.166 starting with reference counting and region-based
00:00:20.766 memory management, and how these interact with
00:00:23.366 resource management.
00:00:26.600 Let’s start by talking about memory.
00:00:30.600 To understand memory management, you must first
00:00:33.400 understand what memory needs to be managed.
00:00:35.800 In all modern operating systems, every process
00:00:38.300 is given a virtual address space in
00:00:40.700 which it executes. Each process thinks it’s
00:00:43.066 the only one running on the machine,
00:00:45.466 and has access to the full range
00:00:47.033 of memory addresses.
00:00:49.000 The underlying hardware translates these virtual addresses
00:00:52.000 into physical addresses that represent particular locations
00:00:55.000 in real memory, and makes sure that
00:00:57.966 the different processes are isolated from each other.
00:01:01.300 The virtual address space that a process
00:01:03.566 sees is divided into several different parts.
00:01:06.933 At the bottom of memory, starting with
00:01:09.000 the lowest numbered address sits the program
00:01:11.033 text itself: the machine runs that represents
00:01:14.466 the running program. That’s immediately followed by
00:01:17.800 the static data and space for any
00:01:19.900 global variables.
00:01:22.000 Following this in memory is the heap:
00:01:24.100 memory allocated using malloc(), or similar mechanisms.
00:01:27.900 Following that is the memory used to
00:01:30.200 hold shared libraries and memory mapped files.
00:01:33.000 Then the stack space, growing down from
00:01:35.100 the top of user accessible memory.
00:01:37.533 Then, finally, the operating system kernel itself
00:01:40.566 occupies the top part of the address space.
00:01:43.433 How much memory is allocated to each
00:01:45.600 of these, and what virtual addresses each
00:01:47.900 sits between, depend on whether you have
00:01:50.900 a 32-bit or a 64-bit machine,
00:01:53.433 and on what operating system you’re running.
00:01:55.900 But all the systems follow the same basic approach.
00:02:01.566 The program text, data, and global variables
00:02:04.966 occupy the lowest part of the address space.
00:02:08.100 There are four parts to this.
00:02:10.966 The very lowest page of memory,
00:02:12.800 usually the first 4096 bytes starting at
00:02:16.133 address zero, is reserved. In languages like
00:02:20.166 C, a null pointer is represented by
00:02:22.400 address zero, so the hardware virtual memory
00:02:25.000 controller is programmed to prohibit access to
00:02:27.766 that address, and to adjacent addresses,
00:02:30.366 as a way of trapping null-pointer dereferences.
00:02:33.800 Trying to access this memory will trap
00:02:36.133 into the operating system, which will kill
00:02:38.100 the process with a segmentation violation error.
00:02:42.000 Following this reserved page, sits the compiled
00:02:44.700 program text. That is, the machine code
00:02:47.966 representing the program. This occupies the lowest
00:02:51.266 part of the address space, starting just above address zero.
00:02:55.800 Next comes the data segment.
00:02:58.233 This comprises string literals and static global variables.
00:03:02.266 Data and global variables where the value
00:03:04.233 is known at compile time.
00:03:06.500 These are stored in the compiled binary,
00:03:08.866 along with the executable machine code,
00:03:11.266 and loaded into memory following the program text.
00:03:15.000 Then, space is allocated for the BSS segment.
00:03:18.966 This provides reserved space for uninitialised
00:03:21.933 global variables defined in the program.
00:03:24.600 The name BSS stands for “block started
00:03:27.400 by symbol”, and is a historical relic.
00:03:30.833 The program text and data segments are
00:03:33.333 fixed. They comprise the compiled program code
00:03:36.033 and data known at compile time.
00:03:38.666 Similarly, the size of the BSS segment
00:03:41.133 is known at compile time.
00:03:43.500 In older operating systems,
00:03:45.533 the program and data always start at a fixed
00:03:48.000 location at the start of the memory.
00:03:50.500 In modern systems,
00:03:52.033 they’re still loaded near the start of the memory,
00:03:54.766 but the actual starting address is randomised
00:03:56.833 each time the program runs.
00:03:59.000 Why is it randomised?
00:04:01.066 It’s a security measure.
00:04:02.800 It makes it harder for code executed as part of
00:04:05.366 a buffer overflow attack to call into
00:04:07.800 other parts of the program, since it
00:04:09.966 can’t know where they’ll be located in memory.
00:04:14.633 As a program executes, it needs space
00:04:17.333 to hold the function parameters, return addresses,
00:04:19.800 and local variables.
00:04:21.833 These are stored on the stack.
00:04:24.900 The stack occupies the top of user
00:04:27.000 accessible memory, starting at a random address
00:04:30.000 for security, and grows downwards.
00:04:33.566 Each time a function is called,
00:04:35.366 starting with main(), the parameters to that
00:04:38.133 function, the return address, and a pointer
00:04:40.766 to the previous stack frame are pushed
00:04:42.566 onto the stack. The occupy the next
00:04:45.466 addresses below the previous top of the stack.
00:04:48.866 And when the function starts to execute,
00:04:51.200 space for its local variables is similarly
00:04:53.433 allocated on the stack.
00:04:56.033 If that function calls other functions,
00:04:58.166 the stack grows, as the new stack frames are created.
00:05:02.000 And when a function returns, the stack
00:05:04.500 shrinks and the memory it used on
00:05:06.233 the stack is automatically reclaimed.
00:05:08.866 The compiler generates the code to manage
00:05:11.333 the stack, as part of the code
00:05:13.133 it generates for each function.
00:05:15.333 It knows how the return address, parameters, and stack
00:05:18.500 pointer are represented, and how to grow
00:05:21.000 and shrink the stack.
00:05:23.000 And the operating system generates the stack
00:05:25.033 frame for main() when the program starts,
00:05:27.900 with a return address pointing to the process cleanup code.
00:05:32.000 To the programmer, the stack is managed automatically.
00:05:35.600 Ownership of the stack memory follows function invocation.
00:05:41.500 This slide shows an example of a simple C program.
00:05:45.100 It comprises a main() function that inspects
00:05:47.566 the values given as command line arguments,
00:05:49.966 and either prints a greeting or a usage message.
00:05:53.833 When this program starts to execute,
00:05:56.366 and main() is called, the stack contents
00:05:59.000 are as shown in the green region
00:06:01.033 on the right of the slide.
00:06:03.466 The stack contains the arguments to main(),
00:06:06.300 argc and argv, the address to return
00:06:10.066 to once main() completes, and the local variables for main.
00:06:14.833 When execution reaches the printf() line,
00:06:17.966 shown in red, and the function is
00:06:20.233 called, a new stack frame is created.
00:06:24.266 This new stack frame holds the arguments
00:06:26.533 for printf(), the address to return to
00:06:29.466 once the printf() call finishes, and a
00:06:31.733 pointer to the previous stack frame.
00:06:34.633 The address of the previous stack frame
00:06:36.533 is stored for ease of debugging,
00:06:38.266 so stack traces can be printed and
00:06:40.500 so debuggers can trace program execution.
00:06:44.200 As printf() starts to execute, space is
00:06:46.733 allocated on the stack for any local variables it uses.
00:06:50.566 And, if printf() itself calls any functions,
00:06:53.266 the stack will continue to grow as needed.
00:06:57.333 This suggests the classic buffer overflow attack.
00:07:01.333 If the language is not type safe,
00:07:03.566 and doesn’t check array bounds,
00:07:05.600 then the call to printf() can be arranged
00:07:07.700 so that it writes past the bounds of a local
00:07:09.600 variable stored on the stack.
00:07:12.000 When this happens, it overwrites whatever is
00:07:14.633 stored in the next higher addresses on the stack.
00:07:18.366 What would that be?
00:07:20.166 Well, maybe some other local variables,
00:07:23.200 but also the pointer to the previous stack frame,
00:07:26.000 then the return address of the function.
00:07:29.000 The goal is to overwrite the return address,
00:07:31.700 and some of the following memory.
00:07:33.933 The attacker tries to fill the memory
00:07:36.066 following the return address with the code
00:07:37.866 they want to execute. And to overwrite
00:07:40.500 the return address so that it points to that code.
00:07:44.266 When the function then returns, rather than
00:07:47.233 return to the correct place, it returns
00:07:49.733 instead to the return address written during
00:07:51.766 the overflows, and executes the attacker’s code.
00:07:55.533 It’s a technique known as stack smashing.
00:07:59.033 The link to the Phrack article on
00:08:01.233 the slide is the classic explanation of this technique.
00:08:05.766 Modern systems have workarounds, of course.
00:08:09.900 They randomise the location of the stack
00:08:11.833 each time a program runs, to make
00:08:13.833 it harder to known what value to
00:08:15.300 overwrite the return address with.
00:08:18.000 And they arrange for the virtual memory hardware to mark
00:08:20.566 the stack memory as non-executable,
00:08:22.800 so the system will refuse to execute the code
00:08:25.466 if the return address is successfully overwritten.
00:08:29.000 These techniques help.
00:08:30.933 The classic stack smashing attack I’ve described
00:08:34.000 won’t work on a modern system.
00:08:36.933 But they don’t entirely solve the problem.
00:08:39.666 Newer attacks, such as return oriented programming,
00:08:43.100 can defeat the protections in some cases.
00:08:46.000 The real solution, of course, is to
00:08:48.533 use a language that checks array bounds,
00:08:51.333 and prevents buffer overflows in the first place.
00:08:56.000 Back to memory management.
00:08:58.400 In addition to the program code and the stack,
00:09:01.333 the heap holds explicitly allocated memory.
00:09:06.733 This is memory allocated by calls to
00:09:08.666 functions such as malloc() and calloc() in C;
00:09:11.966 boxed memory in Rust;
00:09:13.800 and objects allocated using new() in Java.
00:09:17.500 The heap starts at a low address in memory,
00:09:20.500 following the BSS segment,
00:09:22.266 and grows upwards towards the stack.
00:09:25.100 As you might expect by now,
00:09:26.733 the exact starting address for heap allocations
00:09:28.933 is randomised for security.
00:09:31.733 In general, successive heap allocations will occupy
00:09:34.733 consecutive blocks in memory,
00:09:36.633 although multi-threaded systems will partition the heap
00:09:39.800 so that each thread that its own
00:09:41.433 space in which to make allocations,
00:09:43.300 without needed to coordinate with other threads.
00:09:46.433 And, depending on the processor, small allocations
00:09:49.900 may be rounded up in size to
00:09:51.533 align on a 32 bit or a 64 bit boundary.
00:09:56.500 While the stack is managed automatically as
00:09:58.800 functions are called, the challenge of memory
00:10:01.233 management is primarily about how to manage
00:10:03.333 and reclaim the heap memory.
00:10:06.000 This can be done manually, by calling the free() function.
00:10:10.000 It can be done automatically via reference
00:10:12.533 counting or garbage collection, as we’ll discuss
00:10:15.200 in the next lecture.
00:10:17.000 Or it can be done automatically,
00:10:18.966 based on regions and lifetime analysis,
00:10:21.233 as I’ll discuss in later part of this lecture.
00:10:26.000 In addition to allocating heap memory,
00:10:28.733 most operating systems allow memory mapped files,
00:10:31.900 allowing data on disk to be directly
00:10:34.100 mapped into the address space.
00:10:37.000 On Unix-like systems, such mappings are created
00:10:39.866 using the mmap() system call.
00:10:43.000 The mmap() call returns a pointer to
00:10:45.133 a block of memory that acts as
00:10:46.633 a proxy for the file. Reading from
00:10:49.800 that memory will page into the relevant
00:10:51.533 part of the file into memory, and return its contents.
00:10:54.933 And writes to that block of memory will eventually be
00:10:57.400 paged back out to disk by the operating system.
00:11:01.266 The file is paged in and out
00:11:03.066 of memory as needed, with only the
00:11:05.333 parts of the file being accessed being
00:11:07.200 loaded. It’s an effective way of providing
00:11:10.066 random access to parts of a file.
00:11:13.433 Such memory mapped files generally occupy the
00:11:16.200 space between the heap and the stack.
00:11:19.000 Shared libraries are usually implemented using memory
00:11:21.666 mapped files, and are mapped into this memory space too.
00:11:27.333 Finally, the operating system kernel resides at
00:11:30.033 the top of the address space.
00:11:32.566 Kernel memory isn’t directly accessible to user
00:11:34.900 programs, and attempts to access that memory
00:11:37.566 will result in a segmentation violation.
00:11:41.000 The hardware provides a special machine code
00:11:43.100 instruction, known as syscall on 64-bit Intel
00:11:46.733 processors, that switches the system to kernel
00:11:49.700 mode and executes a particular function in
00:11:52.100 kernel memory, after performing some permission checks.
00:11:56.000 The kernel itself can read and write
00:11:58.566 to the entire address space. The hardware
00:12:01.166 protects the operating system kernel from the
00:12:03.200 user processes, but the operating system has
00:12:05.600 to be able to control those processes
00:12:09.600 That concludes our discussion of how
00:12:11.533 programs are stored in memory.
00:12:13.700 In the next part, I’ll start to
00:12:15.333 talk about memory management, and the use
00:12:17.666 of reference counting as a memory management technique.
Part 2: Automatic Memory Management: Reference Counting
The second part of the lecture introduces the concepts of automatic memory management and its use in systems programs. Then it discusses one of the simplest automatic memory management schemes: reference counting. It outlines how reference counting works, and describes its limitations, costs, and benefits.
00:00:00.400 In this part, I’ll talk about the goals
00:00:02.633 of automatic memory management.
00:00:04.566 And I’ll discuss one of the simplest automatic memory
00:00:06.866 management schemes, reference counting.
00:00:11.000 Automatic memory management
00:00:12.900 is frequently distrusted by systems programmers.
00:00:15.633 There’s a general belief that it has high overheads.
00:00:19.166 That it’s slow, CPU hungry, and wastes memory.
00:00:22.900 And that the use of automatic memory management
00:00:25.466 introduces unpredictability into the timing
00:00:28.200 of time sensitive operations.
00:00:31.000 And to some extent this belief is true.
00:00:34.066 There are many different types of automatic
00:00:36.100 memory management. And some of them do
00:00:38.600 have high CPU costs. Some of them
00:00:41.000 do waste memory. And some do introduce
00:00:43.866 unpredictability into timing.
00:00:46.733 Equally, though, there are very real problems
00:00:49.266 due to manual memory management.
00:00:52.000 Managing memory manually has unpredictable overheads.
00:00:56.066 As a programmer you known when you
00:00:58.266 call malloc() and free(), but you have
00:01:00.466 no idea how long those calls will
00:01:01.933 take to execute. And they can be
00:01:04.166 quite processor intensive, and can take unpredictable
00:01:06.666 amounts of time, in some cases.
00:01:10.000 And there are the numerous, and well
00:01:11.966 known, problems due to memory leaks,
00:01:13.900 memory corruption, buffer overflows, use-after-free bugs,
00:01:16.933 and iterator invalidation, all of which are
00:01:19.833 due to manual memory management.
00:01:23.000 Systems programmers have focussed on the problems
00:01:25.333 of moving to automatic memory management,
00:01:27.466 discounting the problems of the status quo.
00:01:31.166 Furthermore, we’re starting to see automatic memory
00:01:33.966 management techniques that solve some of the
00:01:35.800 problems with older approaches. New garbage collection
00:01:39.566 algorithms have lower overheads and are more
00:01:41.466 predictable. And systems have gotten faster,
00:01:44.633 making the overheads of existing algorithms more acceptable.
00:01:48.300 And different approaches, such as region-based memory
00:01:51.133 management, are starting to see widespread use,
00:01:54.766 and offer more predictability and stronger compile-time
00:01:58.266 behaviour guarantees.
00:02:00.700 For many systems programs, it may be
00:02:02.900 time to reconsider the cost-benefit tradeoff for
00:02:05.266 automatic memory management, to see if the
00:02:07.866 balance has shifted.
00:02:11.500 Systems programs traditionally used a mix of
00:02:13.900 manual and automatic memory management.
00:02:17.600 The memory used for the stack is managed automatically.
00:02:21.666 In the sample code on the slide,
00:02:23.600 for example, the memory for the local
00:02:25.833 variable, di, in the saveDataForKey() function is
00:02:29.266 automatically allocated when the function is called,
00:02:32.900 and automatically freed when the function returns.
00:02:37.000 This is so common that we don’t even think about it.
00:02:40.166 Stack-based memory management, in this form,
00:02:43.466 works extremely well for languages like C
00:02:46.366 and C++ that support complex value types.
00:02:49.400 That is, for languages that can put
00:02:51.666 structs on the stack and pass them by value.
00:02:55.800 It works less well for languages like
00:02:57.600 Java, where objects are heap allocated.
00:03:00.666 In Java, the variable, di, would be
00:03:03.166 a reference to a heap allocated object,
00:03:05.400 which would need to be garbage collected.
00:03:07.833 Only the reference to that object would
00:03:10.066 be stored on the stack. While,
00:03:12.600 in C and C++, the entire object
00:03:14.966 can be efficiently managed on the stack.
00:03:17.566 The stack in C and C++ is
00:03:20.000 an example of successful automatic memory management.
00:03:24.000 The heap, on the other hand,
00:03:25.900 is generally managed manually. Allocation is by
00:03:28.866 a call to malloc(). Deallocation, if the
00:03:31.266 programmer remembers to do so, is by
00:03:33.833 an explicit call to free().
00:03:37.900 It would be good to have an
00:03:39.266 effective and automatic way of managing the
00:03:41.033 heap. Something that’s as simple, efficient,
00:03:44.300 and effective as managing the stack.
00:03:47.266 The goal of automatic memory management is
00:03:49.366 to manage the heap. To find memory
00:03:51.966 that’s no longer in use, and make
00:03:53.900 that space available for reuse. To look
00:03:56.033 for objects that are no longer referenced
00:03:58.166 by the running program, and to free their memory.
00:04:01.000 And to do so efficiently and automatically.
00:04:05.000 And to do so safely.
00:04:06.966 It’s better to waste memory, and to keep an
00:04:09.033 object alive when it shouldn’t be,
00:04:11.200 than to deallocate the memory used by
00:04:13.766 an object that’s potentially still in use.
00:04:16.500 There are three approaches to automatically
00:04:18.800 managing the heap.
00:04:21.000 The first is reference counting, that I’ll
00:04:22.933 discuss in a minute.
00:04:25.000 The second is region-based lifetime tracking,
00:04:27.700 that I’ll discuss in the next part of this lecture.
00:04:31.000 And the third is to use garbage
00:04:32.933 collection, that I’ll discuss in lecture 6.
00:04:37.500 The first of the automatic memory management
00:04:40.100 techniques is reference counting.
00:04:43.500 Reference counting is simple and easy to understand.
00:04:47.166 When allocating memory for an object,
00:04:49.433 the allocation contains additional space for a
00:04:51.500 reference count that’s stored along with the object.
00:04:54.900 That is, every object has a hidden
00:04:57.566 extra field, large enough to hold an
00:04:59.866 integer value, that’s stored alongside the object
00:05:03.100 and managed by the runtime system,
00:05:04.966 invisibly to the programmer.
00:05:07.333 This extra field contains a reference count.
00:05:10.400 It counts how many other objects have
00:05:12.833 a pointer, a reference, to this object.
00:05:15.333 When a heap allocated object is created,
00:05:18.033 the reference count is set to one,
00:05:20.133 and a reference to the object is returned.
00:05:22.866 When a new reference, a new pointer,
00:05:25.366 to the object is made, then the
00:05:28.066 reference count is increased by one.
00:05:30.233 When a reference is removed or changed,
00:05:32.766 so that it no longer points to
00:05:34.866 the object, then the reference count is decreased by one.
00:05:38.300 If the reference count reaches zero,
00:05:40.866 then there are no references left that
00:05:43.466 point to the object, and it may
00:05:44.966 be reclaimed, and it’s memory deallocated.
00:05:48.000 If an object that contains references to
00:05:50.300 other objects is reclaimed, this removes a
00:05:52.966 reference to those objects. Doing so potentially
00:05:56.233 causes their reference counts to go to
00:05:58.433 zero, triggering further reclamation.
00:06:01.066 The reference counts are maintained automatically.
00:06:04.100 In a compiled language with reference counting,
00:06:07.133 such as the Objective C runtime used
00:06:09.766 in iPhones, then the compiler generates the
00:06:12.500 code to manage references and reclaim objects
00:06:14.966 when pointers are manipulated. In an interpreted
00:06:18.533 language with reference counting, such as Python
00:06:21.400 or Ruby, the interpreter updates the references
00:06:24.533 and reclaims the objects.
00:06:28.500 The key benefit of reference counting is
00:06:31.066 that it’s predictable and understandable.
00:06:34.000 It’s easy to explain how reference counting works.
00:06:37.533 It’s easy to understand when memory is
00:06:39.866 reclaimed, and what actions in a program
00:06:42.333 might trigger memory to be reclaimed.
00:06:45.000 It’s easy to understand what is the
00:06:46.766 overhead and what is the cost.
00:06:49.233 It’s one additional integer per object;
00:06:51.166 an increment of a reference count when
00:06:53.466 taking a pointer to an object;
00:06:55.200 and a decrement of a reference count,
00:06:57.033 an if statement, and a potential call
00:06:58.900 to free() when removing a reference.
00:07:01.933 The behaviour is intuitive to programmers –
00:07:04.233 and that counts for a lot.
00:07:06.466 Reference counting is also incremental.
00:07:09.266 Memory is reclaimed in small bursts, and the costs
00:07:12.133 that occur are an occasional small overhead
00:07:14.666 on pointer operations. There are few long
00:07:17.233 bursts of memory management activity, and it’s
00:07:19.800 clear what might trigger such bursts.
00:07:24.000 Reference counting also has some costs.
00:07:27.600 Cyclic data structures can be produced that
00:07:30.100 contain mutual references. That is, a set
00:07:33.100 of objects can reference each other in
00:07:35.233 a loop, such that the objects are
00:07:37.666 all reachable from some other object,
00:07:39.166 so the reference count never reaches zero,
00:07:41.733 but not from the rest of the program.
00:07:44.566 This can lead to a disconnected set
00:07:46.466 of objects, unreachable from the rest of
00:07:48.400 the code, that aren’t reclaimed because they
00:07:50.833 have non-zero reference counts.
00:07:53.000 That is, it can leak memory.
00:07:56.400 The programmer needs to notice, and explicitly
00:07:59.066 set one of the references to null,
00:08:00.933 to break the cycle before removing the
00:08:03.066 last other pointer to the objects.
00:08:05.000 Only then will the objects be reclaimed.
00:08:08.266 Reference counting stores a reference count alongside
00:08:11.100 each object, as an additional hidden field.
00:08:14.100 If the objects can be accessed concurrently,
00:08:16.633 it may also need a mutex for each object.
00:08:19.666 This uses additional memory. And can be
00:08:22.566 quite a high overhead if the objects are small.
00:08:25.733 And the processor cost of updating the
00:08:27.633 references can be moderately high, if the
00:08:30.133 program manipulates pointers a lot, of if
00:08:32.000 the objects don’t live long.
00:08:35.233 Despite these limitations,
00:08:36.900 reference counting is widely used.
00:08:39.800 It’s common in scripting languages, such as
00:08:42.500 Python or Ruby, where the overhead of
00:08:44.866 using an interpreted language far outweighs the
00:08:47.133 overhead of using reference counting.
00:08:50.000 It’s also used for some aspects of systems programming.
00:08:54.166 Applications for the iPhone and Mac are
00:08:56.700 typically written in the Objective C language.
00:08:59.700 This is an object oriented extension to
00:09:02.100 C that uses reference counting to manage
00:09:03.966 the memory allocated to objects. And it’s
00:09:06.966 actually highly optimised and reasonably high performance.
00:09:09.966 Reference counting has always been fast enough
00:09:12.333 for user interface code and much application
00:09:14.766 logic. It’s not used for low-level,
00:09:17.933 in-kernel, code, where performance is critical, though.
00:09:22.766 That concludes our discussion of reference counting.
00:09:25.766 It’s a simple, and reasonably effective automatic
00:09:28.400 memory management scheme, that’s quite widely used.
00:09:31.600 In the next part, I’ll move on to discuss
00:09:34.300 an alternative known as region-based memory management.
Part 3: Region-based Memory Management
The third part of the lecture introduces the concept of region-based memory management. It describes the concepts and rationale behind this approach, and how it builds on the way the stack is managed. And it describes the benefits and limitations of region-based approaches to memory management, such as that employed by Rust. The concept of ownership tracking is introduced.
00:00:00.066 In this part, I want to introduce
00:00:02.033 region-based memory management, and discuss how it’s
00:00:04.833 used to manage memory in Rust.
00:00:08.000 Region-based memory management is a new way
00:00:10.366 of managing memory that grew out of
00:00:12.266 frustrations with the alternatives.
00:00:15.000 Reference counting, as we saw in the
00:00:16.933 last part, is simple and easy to
00:00:19.466 understand, but has relatively high overhead.
00:00:23.133 It takes space to store the reference
00:00:25.100 counts, and time to update them.
00:00:27.600 And while this overhead is okay for
00:00:29.733 most applications, it’s too much for performance-critical
00:00:32.566 systems code.
00:00:35.000 Garbage collection, which I’ll talk about in
00:00:37.933 detail in the next lecture, has unpredictable
00:00:40.200 timing and high memory overheads.
00:00:42.433 And manual memory management is too error prone.
00:00:46.066 Region-based memory management aims for the middle
00:00:48.366 ground between these.
00:00:50.333 It aims to be safe and offer
00:00:52.666 predictable timing. And it has no run-time
00:00:55.133 cost, compared to manual memory management.
00:00:57.800 But, to achieve this, it forces some
00:01:00.733 changes in the way code is written.
00:01:05.000 So what is region-based memory management?
00:01:07.800 Well, to understand that, let’s first recap
00:01:11.366 the way memory is managed on the stack.
00:01:14.566 The slide shows a simple C program.
00:01:17.900 This comprises two functions: main() and conic_area(),
00:01:21.766 where the conic_area() function implements a mathematical
00:01:25.000 formula to calculate the surface area
00:01:27.133 of a right circular cone.
00:01:30.000 There’s one global static variable,
00:01:32.066 holding the value of Pi.
00:01:34.500 The main() function takes no parameters,
00:01:37.066 and hold local variables width, height, and area.
00:01:40.700 And the conic_area() function takes two parameters,
00:01:43.433 w and h, and hold the local
00:01:45.833 variables r and a.
00:01:49.000 The diagram on the right shows the
00:01:50.900 memory regions owned by the program,
00:01:52.933 while the conic_area() function is executing.
00:01:56.433 We see there’s a region holding the
00:01:58.133 global variables, and sub-regions for the stack
00:02:01.066 frame for main() and for the conic_area()
00:02:03.700 functions. These nest, one within the other,
00:02:06.433 with the global region lasting for the
00:02:09.966 entire duration of the program, and the
00:02:12.666 other regions nesting within, and lasting for
00:02:15.200 part of the time.
00:02:17.000 The lifetime of the local variables and
00:02:19.066 function parameters matches that of the stack
00:02:21.733 frames of the functions holding them.
00:02:24.600 Memory is allocated and deallocated automatically,
00:02:27.866 efficiently, and seamlessly.
00:02:32.000 Essentially, there’s a hierarchy of regions corresponding
00:02:34.500 to the call stack of the program.
00:02:37.166 The lifetime of the global variables is
00:02:39.566 that of the entire program, and the
00:02:42.066 lifetime of the local variables matches the
00:02:44.366 execution time of the functions they live within.
00:02:47.400 And, within each function, we see there
00:02:49.933 may be lexically scoped variables that have
00:02:51.933 a lifetime less than that of the enclosing function.
00:02:55.533 In the code on the slide,
00:02:57.366 for example, we see that the variable
00:02:59.666 i, the index variable in the for
00:03:01.700 loop, lives only for the duration of that for loop.
00:03:06.000 Each of these variables lives within a
00:03:08.566 region, scoped by the program code,
00:03:10.766 and is automatically allocated when entering that
00:03:13.166 region, and deallocated when the program leaves the region.
00:03:16.466 This works seamlessly.
00:03:19.100 So much so that we take it for granted when writing code.
00:03:24.433 There’s one limitation of this stack-based approach
00:03:27.166 to memory management.
00:03:29.100 It requires data to be allocated on the stack.
00:03:33.000 In the example on this slide,
00:03:35.333 the variable tmp lives on the stack
00:03:37.466 frame for the hostname_matches() function. Space for
00:03:41.166 the variable is created when the function
00:03:43.033 starts to execute, and freed when the function concludes.
00:03:47.533 But that’s only the memory that holds tmp.
00:03:50.633 That is, the memory for the local
00:03:52.666 variable holding the pointer is managed automatically.
00:03:56.666 The heap allocated memory, the memory allocated
00:03:59.700 by the call to malloc(), is not freed automatically.
00:04:04.000 When tmp goes out of scope,
00:04:06.233 the space holding the pointer is freed,
00:04:08.433 but the value it points to is not.
00:04:11.766 The program doesn’t call free() to deallocate
00:04:14.066 that memory, and it doesn’t return the
00:04:16.333 pointer so some other function can do so.
00:04:19.966 It leaks memory. The pointer is gone,
00:04:23.533 so there’s no remaining reference to the
00:04:25.466 allocated memory, so no way to free that memory.
00:04:30.566 Stack-based memory management is clearly effective,
00:04:33.533 but has limited applicability.
00:04:35.933 Can we extend it to manage the heap?
00:04:39.033 Yes. We can arrange for the compiler
00:04:41.866 to track the lifetime of data,
00:04:44.000 and in the same way that it
00:04:45.733 inserts code to manage the stack,
00:04:47.466 it can insert code to manage heap
00:04:48.933 memory based on object lifetimes.
00:04:51.833 If we arrange the code so that
00:04:53.700 there’s a single clear owner of every
00:04:55.500 data value, then we can arrange it
00:04:57.833 so that if a pointer goes out
00:04:59.100 of scope, then the value it points
00:05:01.133 to is also freed.
00:05:03.666 This requires us to track ownership of
00:05:05.533 all data objects, but as we saw
00:05:07.833 in the last lecture, Rust does this.
00:05:11.333 The idea then it to define the
00:05:13.533 Box-of-T type, which is a value that’s
00:05:16.366 stored on the stack that holds a
00:05:18.033 reference to data of type T that’s
00:05:21.433 allocated on the heap.
00:05:23.800 Calling Box::new() allocates and populates heap memory,
00:05:28.400 and returns a pointer to that memory
00:05:30.600 that’s stored in a local variable on
00:05:32.066 the stack. The resulting local variable is
00:05:35.666 a normal variable, with lifetime matching that
00:05:38.166 of the stack frame in which it resides.
00:05:41.366 And the heap allocated object has lifetime
00:05:43.633 matching that of the Box too.
00:05:46.566 When the local variable holding the Box
00:05:48.466 goes out of scope, its destructor is
00:05:50.800 automatically called by the compiler. The destructor
00:05:54.433 frees the heap allocated memory. This is
00:05:57.766 the basis for region based memory management in Rust.
00:06:01.400 It’s highly efficient. And it safety manages
00:06:05.033 heap memory. But it loses generality,
00:06:08.266 since it ties the lifetime of heap
00:06:10.333 allocation to that of stack frames.
00:06:13.833 If you’ve programmed C++, this approach will
00:06:16.766 be familiar as the RAII – Resource
00:06:19.866 Acquisition Is Initialization – design pattern.
00:06:23.566 Python also does something similar in its “with” clauses.
00:06:30.133 Rust takes the pattern further though.
00:06:33.766 To be effective, region-based memory management has
00:06:36.666 to not just tie object lifetimes to
00:06:39.033 stack frames, but also to track how
00:06:42.033 objects are passed between stack frames.
00:06:45.133 And do so in a way that
00:06:46.733 maintains the single owner invariant for every object.
00:06:49.700 At every point, the compiler must know
00:06:52.666 what region of code owns each piece
00:06:54.800 of data. And as data is passed
00:06:57.266 between functions, as ownership of the data
00:07:00.266 is passed between functions, the compiler must
00:07:02.666 track that data ownership.
00:07:05.300 And, when the data item finally goes
00:07:07.866 out of scope, after its ownership
00:07:10.500 has been passed between various functions,
00:07:12.433 only then should it deallocate the heap memory.
00:07:17.166 As we saw in the last lecture,
00:07:19.066 Rust tracks ownership of data items as
00:07:21.366 they’re passed between functions.
00:07:24.066 The code on the slide is the
00:07:25.966 Rust equivalent of the C program we
00:07:27.933 saw earlier, to calculate the conic area.
00:07:31.300 If we look at the lifetimes of
00:07:32.900 the variables in the function area_of_cone(),
00:07:35.566 we see that the variable, r,
00:07:37.466 has lifetime matching that of the function’s
00:07:39.566 stack frame. It’s allocated when the function
00:07:42.666 starts to execute, and goes out of
00:07:45.000 scope and is freed when the function finishes.
00:07:48.533 The variable, a, though is created within
00:07:51.400 the area_of_cone() but then returned from that
00:07:54.566 function. Ownership of the value is passed
00:07:57.666 from that function to its caller,
00:07:59.800 the main() function. And from there,
00:08:02.200 it’s passed as an argument to the
00:08:04.633 println!() macro, that consumes the value.
00:08:07.500 The key point is that the lifetime
00:08:09.466 of the value initially stored in the
00:08:11.166 local variable, a, in the area_of_cone() function,
00:08:14.566 outlives that function.
00:08:17.133 And the compiler tracks this lifetime.
00:08:19.733 It knows that the ownership has changed.
00:08:22.466 That the value has been passed out
00:08:24.533 of the function into main(), and is
00:08:26.333 later consumed by the println!() call.
00:08:30.100 We see that ownership of the return
00:08:32.233 value is moved to the calling function.
00:08:35.333 The value is moved into the stack
00:08:37.066 frame of the calling function, and assigned
00:08:39.466 to a local variable. And the original
00:08:42.066 value, in the called function’s stack frame,
00:08:44.666 is destroyed when that function returns.
00:08:48.000 If the value being moved is a
00:08:49.500 Box, what’s moved is the Box itself.
00:08:52.433 That is, the smart pointer to the
00:08:54.333 heap allocated memory is moved into the
00:08:56.400 stack frame of the caller. The value
00:08:59.200 on the heap that it references is unchanged.
00:09:02.766 Similarly, if a reference is returned,
00:09:05.100 it’s the reference that’s moved. The referenced
00:09:08.100 value is unchanged.
00:09:11.500 Any variable not returned by a function
00:09:13.966 goes out of scope and is destroyed
00:09:15.800 when the function returns.
00:09:18.466 What does this mean? The destructor for
00:09:21.466 the type, the drop() method, is called.
00:09:24.566 Then the memory holding the object is deallocated.
00:09:28.100 If the value going out of scope
00:09:29.966 is a Box, the destructor for the
00:09:32.133 box calls the destructor for the heap
00:09:34.666 allocated object, then frees the heap memory.
00:09:37.300 Since every object goes out of scope
00:09:39.233 eventually, when it reaches the end of
00:09:40.933 its lifetime, since ensures that all heap memory is freed.
00:09:47.266 Since the Rust compiler tracks the lifetime
00:09:49.933 of every object, it can prevent common
00:09:52.000 lifetime-related errors.
00:09:54.466 For example, the Rust program shown on
00:09:56.900 the top of this slide is a
00:09:58.700 function that tries to return a reference
00:10:00.433 to a local variable. This isn’t possible,
00:10:03.533 since the local variable ceases to exist
00:10:05.166 once the function returns.
00:10:08.000 The compiler notices this, and as we
00:10:10.900 see, the code doesn’t compile. You can’t
00:10:14.333 return a reference to, that is,
00:10:16.800 borrow, a value that doesn’t exist any more.
00:10:19.733 The equivalent C program, shown on the
00:10:22.766 bottom of the slide, will compile just
00:10:24.733 fine, but crash at runtime when the
00:10:26.833 returned pointer is used.
00:10:29.800 Good C compilers warn about this,
00:10:32.000 in simple cases like this example,
00:10:34.433 but will compile more complex variants of
00:10:36.866 the code without warnings.
00:10:40.500 Similarly, because the Rust compiler tracks object
00:10:43.400 lifetimes, it can prevent you from accessing
00:10:45.800 memory once it’s been freed.
00:10:48.766 The Rust code shown at the top
00:10:50.800 of the slide shows an example.
00:10:53.066 It imports and explicitly calls the drop()
00:10:55.300 function, to force the local variable,
00:10:57.900 x, to be deallocated.
00:11:01.000 Calling drop() manually is never explicitly needed
00:11:03.666 in Rust, but is possible, and has
00:11:06.666 the same effect as calling free() to
00:11:08.433 deallocate memory in a C program.
00:11:12.000 The Rust code on the slide,
00:11:13.866 that deallocates memory then tries to print
00:11:16.266 out the contents of the just-deallocated memory,
00:11:19.133 fails to compile. The compiler knows that
00:11:22.633 the drop() call consumes the object and
00:11:25.033 that its lifetime ends. It knows that
00:11:27.266 the value cannot then be accessed.
00:11:30.200 The equivalent C program, shown at the
00:11:32.733 bottom, will compile without warnings, but has
00:11:36.066 undefined behaviour at runtime since it accesses
00:11:39.166 previously freed memory.
00:11:43.366 In addition to returning ownership of data,
00:11:46.533 functions can take ownership of a value.
00:11:49.366 When a value is passed to a
00:11:51.400 function, that function takes ownership of that
00:11:53.966 value. We see this in the code
00:11:56.233 on the slide, where the consume() function
00:11:58.633 takes ownership of the local variable, a,
00:12:00.800 from the main() function, in the
00:12:02.633 form of its parameter, x.
00:12:05.233 And, as usual, once the consume() function
00:12:07.700 ends, the values of any data it
00:12:09.633 owns that are not returned, as destroyed.
00:12:13.000 We see this when we try to
00:12:14.500 compile the code. The main() function can’t
00:12:17.500 access the local variable, a, to print
00:12:19.766 its length, since it doesn’t own the
00:12:21.500 value anymore. It gave ownership to the
00:12:25.100 consume() function, and that function destroyed the
00:12:27.233 value and didn’t pass it back.
00:12:29.733 Accordingly, it’s no longer accessible in main()
00:12:32.500 and the code won’t compile.
00:12:36.566 That concludes this introduction to region-based
00:12:39.066 memory management.
00:12:40.900 The key insight is that if every
00:12:43.200 value has a unique owner, and the
00:12:45.666 compiler tracks ownership and the lifetime of
00:12:48.233 those values, then every value can be
00:12:50.700 automatically freed at the end of its lifetime.
00:12:54.066 This is the key difference between Rust
00:12:56.400 and other programming languages. The Rust compiler
00:12:59.533 knows when a value has been consumed,
00:13:01.966 when it’s reached the end of its
00:13:04.100 lifetime, and will enforce this at compile time.
00:13:07.000 Other languages either don’t track lifetimes,
00:13:09.700 like C, and potentially allow use after
00:13:12.533 free bugs, dangling references, and so on.
00:13:15.900 Or they use some form of garbage
00:13:18.166 collection, to track liveness of data at
00:13:20.200 run time. They’re either unsafe, or have
00:13:23.533 run time overhead.
00:13:26.000 Rust is both safe, and avoids the
00:13:28.433 run-time overheads.
00:13:30.000 But to achieve this, Rust restricts the
00:13:32.300 set of programs that can be expressed,
00:13:34.600 as we’ll see in the next part of this lecture.
Part 4: Resource Management
The final part of the lecture discusses how region-based memory management can be extended to include resource management. It further expands on the concept of ownership tracking, introduced in the previous part of the lecture, and shows how Rust supports borrowing of data to make region-based memory management feasible. It outlines the limitations of region-based memory management and how it limits the classes of program that can be written, and discusses the trade-off this imposes. And it reviews how region-based management can be used to enforce deterministic resource clean-up.
00:00:00.366 In this final part of the lecture,
00:00:02.600 I’ll talk about how Rust extends ownership
00:00:05.166 with the idea of borrowing, and how
00:00:07.333 this can apply to management of resources
00:00:09.233 other than memory.
00:00:12.000 We saw in the previous part that
00:00:14.000 functions can pass ownership of data between
00:00:16.433 them. We saw how functions take ownership
00:00:19.533 of data they’re passed, and how they
00:00:22.033 can return ownership of data to their caller.
00:00:24.600 We discussed how this relates to memory
00:00:26.533 management, with memory being freed when an
00:00:28.666 object goes out of scope at the
00:00:30.133 end of a function. And, we showed
00:00:32.166 how, with the aid of the single
00:00:33.633 ownership rules, Rust can leverage this to
00:00:36.433 provide an automatic region-based memory management scheme.
00:00:40.833 This can work, but repeatedly passing and
00:00:43.700 returning ownership of data to and from
00:00:46.233 functions is inconvenient. To make this easier,
00:00:49.966 Rust augments the ownership rules with the
00:00:52.166 idea of borrowed data.
00:00:55.666 Functions in Rust can take references to
00:00:58.133 data as parameters. For example, in the
00:01:01.700 slide we see the function borrow() that
00:01:04.100 takes a mutable reference to a vector,
00:01:06.700 x, as its parameter, and modifies the
00:01:09.766 vector, by pushing a new element onto the end.
00:01:13.700 In the main() function, we create a
00:01:15.866 vector, add some data to it,
00:01:18.033 call the borrow() function, and print the
00:01:19.900 length of the vector.
00:01:22.433 Unlike the case we saw in the
00:01:24.166 last part, where the function consumed the
00:01:26.366 vector, in this case the vector is
00:01:28.633 still accessible after the call to borrow().
00:01:31.966 Why is this?
00:01:34.000 Well, because borrow() is passed data by reference.
00:01:38.166 The borrow() function is passed, takes ownership
00:01:41.766 of, a reference to the vector.
00:01:44.066 It doesn’t take ownership of the vector.
00:01:46.933 When the borrow() function returns, the lifetime
00:01:50.200 of the parameter x ends, and it
00:01:52.466 is reclaimed in the usual way.
00:01:55.300 The function has taken ownership of that
00:01:57.033 parameter, and hasn’t returned ownership to the
00:01:59.200 caller, so the data is freed.
00:02:02.233 But, what it took ownership of was
00:02:04.266 a reference to the vector, not the
00:02:06.233 vector itself. So what is freed is
00:02:09.066 the reference, not the vector. Ownership of
00:02:12.666 the vector remains with the main() function at all times.
00:02:16.433 This is known as borrowing a reference.
00:02:20.533 It’s not shown on the slide,
00:02:22.233 but a function that takes a reference
00:02:23.800 as a parameter can return that reference
00:02:25.800 to the caller. This is safe,
00:02:28.566 because the reference has to be accessible
00:02:30.500 to the caller – otherwise, how could
00:02:32.200 it pass it to the function –
00:02:33.700 so the returned reference will also be accessible.
00:02:38.700 In this example, the borrow() function changes
00:02:41.466 the contents of the vector. It pushes
00:02:44.433 a new element onto the end of the vector.
00:02:47.466 And in this case, it’s safe for it to do so.
00:02:51.000 But this is not always safe.
00:02:54.000 For example, if main() was iterating over
00:02:56.533 the contents of the vector, and passed
00:02:59.033 a mutable reference to that vector to
00:03:01.100 a function that modified it, while that
00:03:03.866 iteration was in progress, this might lead
00:03:06.466 to elements being skipped or duplicated,
00:03:09.000 or to a result to be calculated
00:03:11.066 with inconsistent data.
00:03:13.866 This is a problem known as iterator invalidation.
00:03:18.800 To avoid iterator invalidation, and other problems,
00:03:22.166 references in Rust come in two different
00:03:24.333 types, and have restrictions on how they can be used.
00:03:29.066 Rust has two different types of pointer;
00:03:31.333 two different types of reference.
00:03:34.100 The first is written &T, and is
00:03:37.366 a shared reference to an immutable object
00:03:39.833 of type T.
00:03:42.133 The second is written &mut T,
00:03:45.633 and is a unique reference to a
00:03:47.833 mutable object of type T.
00:03:50.866 The Rust compiler and runtime system work
00:03:53.366 together to control how references can be
00:03:55.166 used, and to track ownership of references
00:03:58.000 and the referenced values.
00:04:01.000 There are three fundamental rules.
00:04:04.566 An object of type T can be
00:04:07.000 referenced by one or more references of
00:04:09.800 type &T. Or it can be referenced
00:04:13.333 by exactly one reference of type &mut T.
00:04:16.533 But it’s not possible to have
00:04:18.333 both mutable and immutable references to the
00:04:20.766 same object.
00:04:23.000 If an object is defined to be
00:04:24.966 immutable, it’s not possible to take a
00:04:27.466 mutable reference to it.
00:04:30.000 And it an object is defined to
00:04:31.800 be mutable, then taking an immutable reference
00:04:34.666 to that object makes it immutable for
00:04:36.700 the duration of the immutable reference.
00:04:41.066 These restrictions complicate how pointers work in
00:04:43.800 Rust. And they limits the set of
00:04:46.333 programs it’s possible to write.
00:04:49.000 But they allow functions to safely borrow
00:04:51.500 objects, without needing to give away ownership.
00:04:55.933 In a Rust program, to be able
00:04:58.300 to change an object, you must either
00:05:00.600 own the object, and it not be
00:05:02.333 marked as immutable.
00:05:04.000 Or you must own the only &mut
00:05:06.433 reference to it.
00:05:09.000 These rules prevent iterator invalidation.
00:05:12.966 Iterators in Rust are designed to take
00:05:15.066 an immutable reference to the object over
00:05:17.266 which they iterate. This guarantees that the
00:05:20.266 object can’t change, and that no mutable
00:05:22.733 references to the object can exist.
00:05:25.966 If the object being iterated over cannot
00:05:27.933 change, the iterator cannot be invalidated.
00:05:31.666 The compiler checks and enforces these rules.
00:05:37.000 By tracking ownership, and controlling how pointers
00:05:40.633 are used, Rust turns various issues that
00:05:43.466 are run-time bugs in other languages into
00:05:46.233 compile-time errors.
00:05:49.000 Rust prevents use-after-free bugs by making it
00:05:52.433 impossible to return references to data to
00:05:55.266 outlive the data being referenced.
00:05:58.000 Rust prevents iterator invalidation, by making it
00:06:01.100 impossible to change the object being iterated
00:06:03.500 over whilst the iterator exists.
00:06:06.500 And Rust prevents race conditions between multiple
00:06:09.000 threads, because the rules about pointers make
00:06:11.366 it impossible for two threads to each
00:06:13.533 have a mutable reference to the same object.
00:06:16.900 All these behaviours are checked at compile time.
00:06:21.466 And Rust has efficient run-time behaviour.
00:06:25.000 The Rust compiler generates exactly the same
00:06:27.533 code to allocate and free memory as
00:06:29.766 would a correctly written C program using
00:06:31.866 malloc() and free(). The difference is that
00:06:35.166 the compiler ensures the malloc() and free()
00:06:37.733 calls are in the correct places.
00:06:40.000 As a result the timing and memory
00:06:43.100 usage of Rust code is a predictable
00:06:45.833 as a correctly written C program.
00:06:48.866 Rust is deterministic in when memory is
00:06:51.100 both allocated and freed.
00:06:55.766 The region-based approach to memory management used
00:06:58.233 by Rust has some limitations though.
00:07:02.000 Primarily, Rust ensures correctness by limiting the
00:07:05.266 types of program it’s possible to write.
00:07:08.666 The rules about ownership and borrowing
00:07:10.800 make it impossible to express certain
00:07:12.833 data structures in safe Rust.
00:07:15.600 For example, it’s not possible to write
00:07:18.366 code that uses data structures that contain
00:07:20.500 reference cycles.
00:07:23.000 The canonical example of this, is that
00:07:25.300 it’s not possible to write a doubly
00:07:26.933 linked list in safe Rust. If you
00:07:30.100 look at the example of the slide,
00:07:32.166 where you have a list containing elements
00:07:33.833 a, b, and c, you’ll see that
00:07:36.366 it’s not possible to add an element,
00:07:37.933 d, to the end.
00:07:40.266 You can take an immutable reference to
00:07:42.133 element c, and add it to element
00:07:44.333 d. This works, since Rust allows two
00:07:47.533 immutable references to C to exist –
00:07:50.266 one from element b and one from element d.
00:07:54.000 What you can’t do, though, is take
00:07:56.500 a mutable reference to element c,
00:07:58.200 in order to modify it to add
00:07:59.900 a reference to element d. This is
00:08:03.000 because there’s already an immutable reference to
00:08:05.266 element c, held by element b,
00:08:07.966 and you can’t have both mutable and
00:08:09.733 immutable references to the same object.
00:08:13.133 The restrictions on references that prevent race
00:08:15.833 conditions, iterator invalidation, and so on,
00:08:19.000 also prevent cyclic data structures.
00:08:22.000 And it’s not just doubly linked lists.
00:08:25.000 They prevent any data structure that contains
00:08:27.266 a loop of pointers.
00:08:29.533 It’s a fundamental limitation of the way
00:08:31.466 references work in Rust. It trades expressive
00:08:34.933 power for safety.
00:08:38.000 The designers of Rust recognised this,
00:08:40.100 and added an escape hatch.
00:08:43.000 Rust also has a third type of
00:08:45.133 reference, known as the raw pointer.
00:08:48.700 Raw pointers work just like pointers in
00:08:50.900 C. They allow you to circumvent the
00:08:53.533 restrictions that Rust imposes on mutable and
00:08:56.066 immutable references, and make it possible to
00:08:59.366 write cyclic data structures in exactly the
00:09:01.666 same way you would do so in C.
00:09:04.333 Of course, by allowing you to circumvent
00:09:06.666 these restrictions, you lose the safety guarantees
00:09:09.400 of Rust. A Rust program that uses
00:09:12.733 raw pointers is as likely to suffer
00:09:14.766 from memory safety issues, such as use
00:09:17.200 after free bugs, iterator invalidation, and race
00:09:20.000 conditions, as is a C program.
00:09:24.000 To make this clear, Rust requires use
00:09:27.033 of raw pointers to be explicitly labelled
00:09:29.200 as unsafe in the code. It warns
00:09:32.366 the programmer, and makes it easy to find such uses.
00:09:36.633 There’s a second limitation of safe Rust,
00:09:39.366 which is that it can’t express shared
00:09:41.266 ownership of mutable data.
00:09:44.000 Shared ownership of immutable data is straight-forward.
00:09:48.200 It’s okay for many different references to
00:09:50.600 point to an object that cannot change.
00:09:53.466 Shared ownership of mutable data is problematic,
00:09:56.466 though, because it potentially open up race
00:09:59.066 conditions. This is why Rust only allows
00:10:02.300 a single mutable reference to a value
00:10:04.200 to exist at once.
00:10:06.866 But, occasionally, rarely, you need shared mutable state.
00:10:12.066 If you do, Rust has a RefCell-of-T
00:10:15.133 type. This wraps some value of type
00:10:18.400 T, and dynamically enforces the borrowing rules.
00:10:22.100 That is, it allows callers to borrow,
00:10:24.800 or mutably borrow, the wrapped value,
00:10:27.600 and enforces at run time that there
00:10:29.833 can only be immutable borrows, or a
00:10:32.366 single mutable borrow, but not both.
00:10:35.666 The RefCell type provides essentially the same
00:10:38.700 guarantees as the regular Rust borrowing rules,
00:10:41.366 but enforced at run time rather than
00:10:43.633 at compile time. This means that attempts
00:10:46.533 to take, for example, several mutable references
00:10:49.366 to the wrapped object, will cause a
00:10:51.366 panic when the code executes, rather than
00:10:53.666 failing to compile.
00:10:56.000 The RefCell is safe,
00:10:57.633 in that it never causes undefined behaviour,
00:11:00.433 but it can cause a run time failure.
00:11:05.266 Possibly the biggest issue with region-based memory
00:11:08.200 management, though, is that it forces the
00:11:10.866 programmer to consider object ownership explicitly,
00:11:13.966 and early in the design.
00:11:17.000 To what extent this is a problem
00:11:19.233 depends, I think, on your background.
00:11:22.500 If you’re experienced at programming in C,
00:11:24.866 this tends to be a non-issue.
00:11:27.500 Writing correct C programs also requires careful
00:11:30.633 consideration of data ownership, to know what
00:11:33.366 functions malloc() data and what functions free()
00:11:35.966 that data. Rust essentially codifies and enforces
00:11:39.433 the rules that well-written C programs tend
00:11:42.100 to adopt anyway.
00:11:44.700 If you’re experience is rather with a
00:11:46.500 language like Python or Java, where the
00:11:49.366 use of a garbage collector or reference
00:11:51.266 counting hides a lot of the complexity
00:11:53.166 of memory management, then needing to start
00:11:55.833 thinking about ownership is more of a
00:11:57.666 burden. In this case, programming effectively in
00:12:01.400 Rust may require a shift in your thinking.
00:12:06.733 That concludes our introduction to region-based
00:12:09.100 memory management.
00:12:10.666 This is one of the more unusual features of Rust.
00:12:14.566 It provides an efficient
00:12:16.166 and predictable way of managing memory,
00:12:18.266 and offers strong correctness guarantees that can
00:12:20.566 prevent many common bugs.
00:12:23.300 But it does so by constraining the
00:12:25.333 types of program that can be written,
00:12:27.000 and by making the programmer explicitly consider
00:12:29.300 data ownership.
00:12:31.233 It’s a trade-off that works well for
00:12:33.233 some types of problem, but less well for others.
00:12:39.300 The ownership rules of Rust can also
00:12:41.766 be useful for resource management.
00:12:46.000 As we’ve seen, Rust tracks data ownership,
00:12:49.166 and deterministically frees heap allocated memory when
00:12:52.300 references to that memory go out of scope.
00:12:55.533 Particular types can use this ownership tracking
00:12:58.166 to implement custom destructors that provide deterministic
00:13:01.366 clean-up of resources they own.
00:13:04.433 If a type implements the Drop trait,
00:13:06.766 then the runtime will call the drop()
00:13:08.800 method on instances of that type at
00:13:10.966 the end of their lifetime, when they
00:13:12.700 go out of scope. This allows those
00:13:15.900 instances to clean-up after themselves.
00:13:19.000 For example, the File class implements the
00:13:21.866 Drop trait to close the underlying file
00:13:24.366 when file objects go out of scope.
00:13:28.000 Python has special syntax for this,
00:13:30.200 as shown on the slide. In Rust,
00:13:33.466 much like in C++, the cleanup happens
00:13:36.700 naturally when the object goes out of
00:13:38.300 scope, without the explicit syntax.
00:13:43.000 Finally, as we saw in the discussion
00:13:45.666 of struct-based state machines in Lecture 4,
00:13:48.866 ownership tracking allows state transitions to be enforced.
00:13:53.500 A state machine can be defined using
00:13:55.666 structs to represent the state, and to
00:13:58.333 hold any needed state variables. And state
00:14:01.433 transitions can be implemented by defining functions
00:14:04.033 on those structs that consume self,
00:14:06.166 and return a new state.
00:14:08.966 The slide shows an example, where methods
00:14:11.966 login() and disconnect() are implemented on an
00:14:15.066 UnauthenticatedConnection struct.
00:14:18.366 Note that both of these take self
00:14:20.866 as their first parameter, not &self.
00:14:24.066 That is, they take ownership of the
00:14:26.400 struct on which they’re implemented, rather than
00:14:28.600 borrow that struct.
00:14:31.300 This means they consume self. They destroy
00:14:34.333 the object on which they’re called.
00:14:37.300 That is, they force the object representing
00:14:39.600 the state to be destroyed. And they
00:14:42.266 return ownership of a new object,
00:14:44.333 representing the new state of the system.
00:14:47.566 This forces the cleanup of any data
00:14:49.600 held by that state that’s not explicitly
00:14:51.700 copied to the new state. It enforces
00:14:54.500 a clean state transition.
00:14:57.600 The blog post linked talks about these
00:14:59.666 issues further, and I encourage you to read it.
00:15:04.766 That concludes the first part of our
00:15:06.466 discussion of memory and resource management.
00:15:09.866 We’ve looked at how processes are stored in memory.
00:15:12.466 How reference counting works,
00:15:14.333 and what are its strengths and weaknesses.
00:15:17.033 And how region based memory management is
00:15:19.133 used in Rust as an alternative.
00:15:22.333 In the next lecture,
00:15:23.566 we’ll move on to discuss garbage collection.
Lecture 5 focussed on resource ownership and memory management. It started by discussing how a process is laid out in memory, and what memory needs to be managed. Then, it moved on to discuss the problem of, and need for, automatic memory management in general terms, highlighting that systems programmers tend to distrust automatic memory management, yet suffer from the problems due to manual memory management.
The lecture then discussed reference counting. This is widely used in both scripting languages, such as Python or Ruby, and in the Objective-C runtime used on Apple's iPhone and Mac operating systems. It noted that reference counting is relatively easy to understand, incremental, and predictable, but suffers from relatively high overhead with small or short-lived objects, and cannot collect cyclic data structures.
The lecture then focussed on the region-based memory management scheme used by Rust. It noted that the language tracks ownership of data as it moves between functions, and deterministically cleans up heap allocated memory when the lifetime of an object ends. Region based memory management achieves good, predictable, performance, and provides a number of nice guarantees around absence of race conditions, use-after-free bugs, iterator invalidation, and so on. But it comes at the cost of making the type system considerably more complex, and in limiting the types of programs that can be expressed.
Discussion will focus, primarily, on how the region based memory management scheme works, and on its advantages and disadvantages. The goal is to understand whether you think the benefits outweigh the complexity introduced.