Advanced Systems Programming H (2021-2022)

Lecture 1: Introduction

Lecture 1 provides an introduction to the course. It reviews the course aims and objectives, intended learning outcomes, and timetable; and it outlines the means by which the course will be assessed. This lecture is comprised on one part.

Part 1: Advanced Topics in Systems Programming

Part 1 of the lecture comprises a brief introduction to the course.

Slides for part 1


00:00:00.333 Welcome to Advanced Systems Programming.

00:00:02.800 For those who don’t know me, my name is Colin Perkins

00:00:06.100 and I’m the lecturer and coordinator for this course.

00:00:08.633 In this first lecture I’ll introduce the course materials,

00:00:12.600 and review the aims, objectives, and administration

00:00:15.400 of the course.


00:00:18.933 As I said, I'm Colin Perkins.

00:00:21.300 I'll be the lecturer for this course.


00:00:23.133 My contact details are on the slide

00:00:25.533 and if you have questions about the material

00:00:27.466 covered in the course I'm happy to answer them via email.


00:00:30.533 We also have office hours and discussion

00:00:33.033 sessions throughout the course,

00:00:34.533 and a chat room in Microsoft teams.


00:00:36.866 And in normal times

00:00:38.500 you can also find my office in Lilybank Gardens.


00:00:42.900 The lecture slides, recordings, transcripts, lab

00:00:45.833 exercises, and other course material are on Moodle.

00:00:48.700 And the material is also on my website at csperkins.org/teaching.

00:00:53.333 The version on my website also has lecture transcripts

00:00:56.433 that are difficult upload to Moodle.


00:01:00.066 So what’s this course, about?


00:01:04.000 Well, over the last 10 to 15 years or so

00:01:07.300 we’ve seen a significant shift in the technology landscape.

00:01:11.466 We’ve gone from a world dominated by desktop PCs running Windows,

00:01:16.100 with a relatively small number of servers, often located on site,

00:01:19.766 which support those PCs,

00:01:22.133 to a world dominated by smartphones and cloud computing.


00:01:27.166 Devices are now much more likely to be mobile and battery powered,

00:01:30.600 and connected to a network

00:01:33.400 that lets them offload computation and data

00:01:35.566 to remote cloud computing resources

00:01:38.100 running in dedicated data centers.


00:01:40.966 This changes the way we use those devices,

00:01:44.233 and the type of applications we can use.


00:01:46.900 And part of that shift is that we’re increasingly

00:01:49.900 using real time applications like telephony,

00:01:53.200 video conferencing, or streaming video,

00:01:55.800 on our smartphones, tablets,

00:01:57.666 and other general purpose computing devices.

00:02:00.833 And these applications are powered

00:02:03.266 by an increasing number of servers.


00:02:06.133 These servers are not battery powered or mobile,

00:02:09.166 but are also, increasingly, power aware as data centre

00:02:12.800 operators strive to reduce the electricity

00:02:15.100 and air conditioning costs of operating

00:02:16.800 their services, and to reduce their environmental impact.


00:02:22.033 And the devices, we use her also increasingly concurrent.

00:02:26.166 In part, this is because a data centre is a

00:02:29.266 natural home for concurrency.

00:02:30.833 Running millions of instances of the same service

00:02:33.666 to meet the needs of different users.

00:02:35.900 But it's also, in part, because of the

00:02:37.933 need to control power consumption

00:02:39.900 has pushed mobile CPUs in the

00:02:42.000 direction of multi-core support.

00:02:44.233 These days, everything is a multi-core system.


00:02:49.566 Despite these massive shifts,

00:02:52.133 the underlying operating system and software stack

00:02:54.800 has changed relatively little.

00:02:57.333 The operating system on my smartphone

00:03:00.133 runs the same underlying software stack

00:03:02.633 as the desktop workstation I used in 1992,

00:03:05.900 for example.

00:03:07.300 It uses the same operating system kernel,

00:03:10.166 it's programmed using the same programming language

00:03:13.133 and shares many of the same software libraries,

00:03:16.166 API's and features.


00:03:18.366 The Code has evolved and changed, of course,

00:03:21.300 but the organization, architecture, and underlying approaches

00:03:25.033 to writing the system remain surprisingly similar.


00:03:29.400 The technology, and programming model,

00:03:32.066 is increasingly limiting.


00:03:37.366 In this course we'll explore some new techniques

00:03:40.466 and approaches that aim to support safer,

00:03:42.766 and more effective, systems programming.


00:03:46.100 We'll talk about advances in programming languages

00:03:49.100 that allow us to retain the advantages of systems

00:03:51.866 programming languages, such as C,

00:03:54.100 in terms of control over data

00:03:56.033 representations and high performance,

00:03:58.133 while being much safer and easier to program.


00:04:02.333 And we'll talk about how operating systems

00:04:04.300 might change, to better support writers of

00:04:06.633 device drivers, level networking code,

00:04:09.333 and so on,

00:04:10.633 with the goal of improving the safety

00:04:12.300 and security of the overall system.


00:04:18.533 The aims of the course are to explore the features

00:04:21.200 of modern programming languages and operating systems

00:04:23.966 that can ease the challenges of systems programming,

00:04:26.866 focusing on type systems and runtime support.


00:04:30.733 We'll review the literature on systems programming

00:04:34.033 and operating systems design.

00:04:35.700 discuss limitations of deployed systems,

00:04:38.866 and consider how systems programming might evolve

00:04:41.400 to address the challenges of supporting

00:04:43.133 modern computing systems.


00:04:46.133 In particular, the course will aim to highlight

00:04:48.933 how modern programming languages

00:04:50.766 and new approaches to software development

00:04:53.133 can help ensure system correctness and security.


00:04:57.866 Fundamentally, we'll be looking at how systems programming is

00:05:00.900 changing to support new applications and new use cases.


00:05:06.600 What are the expected learning outcomes?


00:05:11.333 Well, first and most importantly,

00:05:14.466 by the end of the course, you should understand

00:05:16.800 the advantages and disadvantages of using C

00:05:19.666 as a systems programming language.

00:05:22.300 You should be able to discuss the benefits of using a modern,

00:05:25.900 strongly typed, systems programming language

00:05:28.100 as an alternative.

00:05:29.666 And you should have an understanding of the concepts

00:05:32.466 behind type-driven development,

00:05:34.300 and the role of the type system in ensuring code correctness.


00:05:40.333 Second, you should have some understanding of

00:05:42.866 the challenges in writing secure systems code.

00:05:46.233 In particular, you should understand some of the

00:05:48.966 issues around memory safety and parsing

00:05:51.300 untrusted network input.

00:05:53.300 And you should understand why these tasks are difficult in C.


00:05:59.533 Third, you should have some understanding

00:06:03.266 of the basics of how garbage collection,

00:06:05.300 reference counting, region based memory management,

00:06:08.866 and manual memory management work.

00:06:10.900 And you should have some idea when it's appropriate

00:06:13.566 to use each approach,

00:06:14.766 and what are the trade offs

00:06:16.200 between them.


00:06:19.100 You should understand why multi-core systems

00:06:22.266 have become popular, and the hardware trends

00:06:24.766 that are driving this.


00:06:26.400 And you should have an understanding of different

00:06:28.633 approaches to concurrent programming,

00:06:30.633 including multi threading,

00:06:32.533 actor based systems,

00:06:34.033 concurrent transactions,

00:06:35.533 and coroutines.

00:06:37.166 And understand a little of what are the trade-offs

00:06:39.866 between these different approaches to concurrent programming.


00:06:44.000 And finally, you should be able to write simple programs

00:06:47.333 in a modern systems programming language

00:06:49.466 to demonstrate your understanding.


00:06:54.566 There are a number of prerequisites for this course.


00:06:57.966 Firstly, you're expected to be familiar with the C programming language.

00:07:03.033 You won't need to write any C programs in this course,

00:07:07.333 but you will need to read and understand C code.


00:07:11.400 One of the aims of this course

00:07:13.633 is to make you aware of the strengths,

00:07:15.366 and weaknesses of C as a programming language,

00:07:17.900 and to compare C to other languages.

00:07:21.366 To do this, you need to understand C programming.


00:07:26.333 You also need to understand the basics

00:07:27.966 of operating systems and networking.

00:07:30.833 This is a systems programming course,

00:07:33.233 and operating systems and networks

00:07:35.033 are two of the main areas where systems programming is used.


00:07:40.300 And you need a conceptual understanding of functional programming.

00:07:44.100 You won’t need to read more than a few lines of Haskell code,

00:07:48.033 and you certainly wouldn't need to write Haskell programs.

00:07:52.233 But the ideas behind Haskell, and functional programming, are important.

00:07:57.233 We’ll be talking a lot about how functional programming concepts,

00:08:01.366 and some of the ideas around modern type systems,

00:08:04.133 both of which were pioneered in Haskell,

00:08:06.466 can be used in more mainstream systems programming tasks.


00:08:12.600 These prerequisites broadly match the systems programming,

00:08:16.033 operating systems, network systems,

00:08:18.666 and functional programming programming courses

00:08:21.300 in the Glasgow computing science degree.


00:08:24.400 You can take this course, if you haven't done those courses,

00:08:27.233 but you might need to do some catching up in your own time.


00:08:33.466 This course is structured as 10 lectures

00:08:35.866 and five lab exercises.

00:08:38.300 The lectures run weekly,

00:08:40.366 with a new topic each week for 10 weeks.

00:08:42.733 The lectures are pre-recorded,

00:08:44.900 and there’s a group discussion slot

00:08:46.500 and a Q-and-A slot each week.


00:08:49.633 The labs are also organized weekly,

00:08:51.666 but some exercises are expected

00:08:53.600 to take one week to complete,

00:08:55.366 while others will take two weeks.

00:08:57.333 The labs finish after nine weeks.


00:09:03.533 Lecture recordings will be made available ahead of time,

00:09:06.733 and will comprise one-to-two hours worth

00:09:08.800 of material each week

00:09:10.166 split into several shorter parts.


00:09:13.866 Each lecture will be accompanied

00:09:15.366 by a set of discussion questions

00:09:17.133 that'll be available on moodle and on the course website.

00:09:20.500 These discussion questions are not assessed,

00:09:23.766 and intended rather to help you understand the material.


00:09:28.800 We have a live session on zoom,

00:09:30.966 scheduled from 10 to 11 on Monday mornings.

00:09:33.633 The sessions are intended for discussion


00:09:36.400 of the lecture material and questions.

00:09:38.566 You should watch the lecture videos,

00:09:40.700 and think about the discussion questions,

00:09:42.766 before those timetable sessions.


00:09:46.033 This discussion slot is your main opportunity to ask

00:09:49.166 questions about the material,

00:09:50.666 and I would encourage you to come prepared

00:09:52.866 to use the slot for discussion.

00:09:55.266 The more questions and discussion we have,

00:09:58.200 the more useful, it will be to everybody.


00:10:03.733 Immediately following that discussion slot,

00:10:06.300 from 11-12 on Monday mornings,

00:10:08.933 is a scheduled question-and-answer drop-in session.

00:10:12.800 This is your time to ask questions

00:10:14.866 and discuss the materials with me

00:10:16.800 in a one-to-one zoom call.

00:10:18.700 And to ask any questions you don't want to

00:10:20.466 ask in the open discussion session.


00:10:26.033 Finally, there'll be weekly labs during the first nine weeks

00:10:29.400 of the course.


00:10:31.700 As with the lectures,

00:10:33.033 the lab materials will be made available in advance

00:10:35.100 of the timetabled sessions

00:10:36.866 And the exercises are to be completed in your own time.


00:10:41.933 The timetabled sessions, from 1-2 on Mondays,

00:10:45.333 are for live support with the lab exercises.

00:10:48.533 You should try to solve the exercises,

00:10:51.066 and think of questions you might need to ask,

00:10:53.100 before the timetabled slots,

00:10:54.966 so we can effectively help you.


00:10:57.500 There’ll be groups set up on Microsoft teams

00:11:00.266 for discussion of the labs.


00:11:06.133 This course will use the Rust programming language

00:11:09.233 to illustrate type-driven design

00:11:10.900 and ownership tracking for systems programming.

00:11:13.733 You're expected to learn the basics of how to

00:11:16.700 program in Rust as part of this course.


00:11:20.766 The two books listed on the slide

00:11:23.033 provide good introductions to Rust

00:11:24.866 to complement the material we’ll cover in the lectures.


00:11:28.100 The first book, The Rust Programming Language,

00:11:31.800 by Steve Klabnick and Carol Nichols,

00:11:34.033 is the standard tutorial introduction to Rust.

00:11:37.333 It’s available for free on the Rust language website,

00:11:40.966 or you can buy it as a bound paper book, if you like.


00:11:46.900 The second book, Programming Rust,

00:11:49.366 by Jim Blandy and Jason Orendorff,

00:11:51.933 is more expensive to buy,

00:11:53.933 but I think it has the better explanations

00:11:56.133 of many of the concepts.

00:11:57.800 The link on the slides offers free access to this

00:12:01.100 book online via the University library.


00:12:05.000 I strongly recommend you read one of these books

00:12:07.700 to build on the lectures and lab exercises.


00:12:11.533 In addition to the two books

00:12:13.666 on the Rust programming language,

00:12:15.600 lectures will also refer to various research

00:12:18.133 papers and blog posts.


00:12:20.133 You're expected to read this material.


00:12:24.166 For all the research papers, either a URL

00:12:28.133 or a digital object identifier, a DOI,

00:12:30.933 will be provided.

00:12:32.300 If a DOI is given, go to dx.doi.org

00:12:37.166 enter the DOI, and it will redirect you to the paper.


00:12:41.100 All the papers mentioned in the course

00:12:44.300 are accessible at no cost from the campus network

00:12:46.933 or via the university’s VPN service.

00:12:49.933 In most cases, you should also be able to get access

00:12:53.666 by following the institutional login

00:12:55.600 option on the publisher's website

00:12:57.566 from your home network.


00:13:00.600 Under no circumstances should you

00:13:02.766 pay to access any of these papers.

00:13:04.866 You can't get access for free,

00:13:07.400 ask me, and I'll send you a copy.


00:13:10.600 Indeed, as a general principle

00:13:13.400 don't pay for access to any research papers.

00:13:16.433 The authors of the research papers

00:13:19.433 do not get paid if you buy

00:13:21.566 a copy of their paper from the publisher.

00:13:24.000 In fact, most research paper authors

00:13:26.700 actually pay to publish their papers.

00:13:29.600 If you want a copy of a research paper,

00:13:32.700 and you can't find it for free online,

00:13:34.733 then email the author and ask.

00:13:36.933 They'll always almost always just send you a copy by return.


00:13:41.833 Now, as I said earlier,

00:13:43.666 you're expected to read these papers.

00:13:46.100 When you do, read them to understand the concepts and ideas,

00:13:50.133 not the details.

00:13:51.633 Don't worry if you don't understand

00:13:53.800 everything in these papers.

00:13:55.300 Skip over details and parts you don't understand.


00:13:58.633 You're looking to get a feel for the ideas,

00:14:01.700 not to understand every detail.

00:14:04.000 Some of the ideas mentioned in the papers will be good;

00:14:07.700 some less so.

00:14:09.266 Some will be interesting, but perhaps impractical.

00:14:12.566 And what's impractical today might become important tomorrow

00:14:16.633 due to changes in technology and/or society.


00:14:20.700 The goal of the papers, is to help you think

00:14:23.900 more deeply about the subject

00:14:25.700 They're not necessarily there to suggest

00:14:28.133 particular desirable outcomes,

00:14:30.000 they're there to suggest interesting ideas.


00:14:33.533 If you've not read many research papers before

00:14:37.300 the link on the page,

00:14:40.133 on how to how to read a research paper, might be useful.


00:14:48.366 The course follows the usual model

00:14:50.600 with an exam worth 80% of the marks

00:14:52.866 and the coursework being worth the remaining 20%.


00:14:56.066 The assessed coursework will comprise two exercises,

00:14:59.966 each of which is worth 10% of the marks.


00:15:03.800 The first assessed exercise will be an essay question

00:15:06.866 about memory management.

00:15:08.733 It will be made available at the start of lecture five,

00:15:11.433 and will be due at the start of lecture seven.


00:15:14.766 The second assessed exercise

00:15:16.766 will be concurrent programming exercise

00:15:18.733 using the Rust programming language.

00:15:20.866 It will be made available at the start of lecture seven,

00:15:24.100 and will be due at the start of lecture nine.


00:15:27.300 There'll also be an exam

00:15:29.533 worth 80% of the marks for the course,

00:15:32.033 sometime in April or May.

00:15:35.000 Past papers, and a sample exam paper with worked answers,

00:15:38.566 are available on moodle.


00:15:41.033 The material covered in the lectures and labs is examinable.

00:15:45.466 Importantly, though, so too are the ideas discussed

00:15:49.000 in the cited to the research papers and blog posts.


00:15:52.433 The goal is to test your understanding of the concepts,

00:15:56.166 not to test your memory of the details.

00:15:58.466 Reading widely, and broadly understanding

00:16:01.333 the material, is important.

00:16:03.733 If you only watch the lectures, and skip the reading,

00:16:07.066 you'll miss important ideas.


00:16:11.333 That concludes these introductory remarks.


00:16:14.600 The main lecture content starts in week two.

00:16:17.500 Please make sure you watch lecture two

00:16:20.266 and think about the discussion questions before then

00:16:22.733 and come prepared to discuss.

00:16:25.566 The labs start in week one of the semester.

00:16:29.000 Please review the lab one hand out,

00:16:31.500 and start on the exercises.


The discussion session in week 1 will review the course structure, goals, and logistics. There will be time to answer ask any questions about the course, lectures, and labs.