Advanced Systems Programming H (2021-2022)

Lecture 10: Future Directions

The final lecture concludes by reviewing the course material, and discussing some possible future directions for systems programming.

Part 1: Future Directions

The final part of the course talks about future directions and open issues. It notes that there are still open questions around how best to support concurrent programming, data centre-scale computing, and large-scale distributed operating systems. It also highlights the benefits of modern programming languages for developing programs that run within a single system, and notes the challenges in scaling out to massively distributed systems.

Slides for part 1


00:00:00.200 In this last lecture, I’d like to

00:00:02.000 talk about some open issues and future

00:00:03.733 directions for systems programming.


00:00:08.466 So, in this course we've discussed a

00:00:11.200 number of advanced topics in the systems

00:00:13.733 programming space.


00:00:15.900 We spent some time talking about what

00:00:18.300 is systems programming, and what distinguishes a

00:00:21.000 systems program from an applications program,

00:00:24.966 focusing on control over memory layout,

00:00:29.633 and performance needs, and so on


00:00:33.533 We spent a fair amount of time

00:00:34.866 talking about type systems, and type based

00:00:37.333 modelling and design, and thinking about the

00:00:41.233 benefits of advanced type systems for building

00:00:44.933 systems programs.


00:00:46.766 Thinking about some of the constraints and

00:00:48.600 issues around modelling the problem space using

00:00:52.800 types, and using that to help robustness

00:00:55.433 and security of programs.


00:00:59.133 We focused a lot on resource ownership

00:01:02.200 and memory management, and tracking how the

00:01:06.166 program works with, controls, and owns resources,

00:01:10.833 including memory, but also including state machines,

00:01:14.833 and the state variables in state machines.


00:01:17.600 And we spoke about different approaches to

00:01:19.400 memory management, the reference counting approaches,

00:01:23.666 garbage collection, and region-based memory management.


00:01:28.566 And we spent a fair amount of

00:01:29.866 time talking about concurrency.


00:01:32.633 Talking about message passing approaches to concurrency.


00:01:37.200 Talking about transactions, talking about co-routines and

00:01:40.333 asynchronous programming. And we spent some time

00:01:44.000 discussing why the common approach of using multithreading,

00:01:48.166 locks, and shared mutable state, is not

00:01:52.400 an effective approach, and doesn't scale out

00:01:55.866 to larger systems, and is very hard to get right.


00:02:00.900 And, we just finished up by talking

00:02:03.033 about some of the security considerations in

00:02:05.166 these systems, and thinking about

00:02:08.733 how we can use advanced type systems

00:02:12.500 to make explicit the assumptions in the

00:02:15.766 code, and reduce the risk of inconsistencies

00:02:21.033 which lead to security breaches. And we

00:02:24.233 spent some time talking about how memory

00:02:26.666 unsafety is the primary cause of security vulnerabilities.


00:02:32.600 What did we learn from this?


00:02:36.700 Well, I think one of the key

00:02:39.500 distinguishing features with systems programs, is that

00:02:42.833 they need low-level control over the data layout.


00:02:47.266 In order to effectively write device drivers,

00:02:50.133 in order to effectively write network protocols,

00:02:52.866 or file I/O,

00:02:55.766 and in order to make effective use

00:02:58.000 of memory, and fit within the cache,

00:03:00.933 and optimise the layout of data structures

00:03:04.666 to fit within the cache,

00:03:07.533 we need to be able to control

00:03:09.400 the way data is laid out in memory.


00:03:12.833 One of the key features of systems

00:03:15.266 programs, is that they give that control.


00:03:18.233 And going forward, I expect systems programming

00:03:20.466 languages to continue to do that.


00:03:23.300 We need to be able to specify

00:03:25.633 exactly how a data structure is laid

00:03:28.000 out in memory, and to be able

00:03:29.800 to control whether it is allocated on

00:03:31.533 the stack or on the heap.


00:03:33.933 And to be able to box,

00:03:35.533 or unbox, data structures as we need

00:03:37.766 to, to get good performance.


00:03:43.066 In traditional systems programming languages,

00:03:46.000 that control has

00:03:49.400 come at the expense a lot of safety.


00:03:54.433 C provides you with a tremendous amount

00:03:57.833 of control over the way data is

00:03:59.866 laid out in memory, with tremendous amount

00:04:02.700 of control over representations, and pointer use,

00:04:06.266 and so on, but in a way

00:04:08.033 that's quite unsafe, and quite hard to get right.


00:04:12.100 More modern languages,

00:04:14.500 Rust being the example we’re using in

00:04:17.633 this course, but certainly not the only

00:04:20.166 language in the space,

00:04:22.433 provide this type of behaviour in a much

00:04:28.100 safer way. They give you control over data layout,

00:04:31.800 but they also provide memory safety,

00:04:34.366 they check it array bounds they check

00:04:36.033 pointers are used correctly, and they can

00:04:39.100 avoid problems like use-after-free bugs, data races,

00:04:42.866 iterator invalidation, and so on, and get

00:04:46.166 the control over data layout whilst maintaining

00:04:48.933 a lot more safety,

00:04:51.400 a lot more robust behaviours.


00:04:56.566 I think it's clear that concurrency is

00:04:59.366 important, especially as Moore's law and Dennard

00:05:03.300 scaling break down, and we get increasingly

00:05:07.033 concurrent systems.


00:05:12.166 And we spoke about several different approaches

00:05:14.100 to concurrency, but I think it's clear

00:05:16.166 they've all got trade-offs, and none of

00:05:18.200 them are a one size fits all

00:05:20.800 approach to building concurrent systems.


00:05:26.500 The transactional memory approach, I think,

00:05:28.833 works wonderfully in Haskell, but it's not

00:05:32.366 clear it scales out to other languages.


00:05:35.533 The message passing approach works wonderfully in Erlang,

00:05:41.100 but fits less well, less naturally,

00:05:44.566 into many other languages, and has some

00:05:47.733 difficult safety issues in some cases.


00:05:52.200 And the asynchronous coroutine based approach

00:05:56.033 leads to strange behaviours with starvation and blocking.


00:05:59.966 There’s constraints, and it’s not clear

00:06:02.666 we have the right answer.


00:06:06.166 And we’ve shown that type systems,

00:06:07.900 I think, can help model the problem domain.

00:06:09.833 I think that's one of the key takeaways.


00:06:12.400 We can model the assumptions, we can

00:06:14.933 help ensure the system is safe and correct.


00:06:21.366 And we can make the systems much

00:06:24.233 safer, and avoid a lot of common

00:06:26.166 problems, by encoding more information into the

00:06:28.400 types, and letting the compiler help us

00:06:30.666 check the systems for correctness.


00:06:37.200 So that's where we are at today.


00:06:39.633 That, I think, is the current state-of-the-art,

00:06:41.933 and what we have learned from building

00:06:44.166 systems programs today.


00:06:47.333 Where we going in the future?


00:06:51.166 Well, as I mentioned, I think we

00:06:53.000 still don't have good solutions for concurrent programming.


00:06:57.733 Transactional memory, I think, is very promising,

00:07:00.833 but doesn't fit well with impure imperative

00:07:05.033 languages, and most of the programs we

00:07:07.966 write today are written in impure imperative languages.


00:07:13.500 Message passing is quite a natural approach,

00:07:18.500 and fits nicely into mainstream languages,

00:07:23.100 and solves some of the problems with

00:07:26.233 shared mutable state, and threads and locks,

00:07:29.300 but it's got problems with providing back

00:07:32.133 pressure, it can have race conditions and

00:07:34.933 deadlocks, and it's difficult to structure programs,

00:07:39.366 structure large message passing systems, without losing

00:07:42.333 track of the message flows.


00:07:45.800 And asynchronous code introduces subtle

00:07:48.800 blocking and starvation bugs.


00:07:52.433 And it's not really clear what's the right solution.


00:07:55.666 It’s not clear if the problem domain

00:07:58.533 is sufficiently broad that we need different

00:08:01.533 types of solution for different problem domains,

00:08:04.800 and we are just going to end

00:08:06.400 up with three-or-four different ways of doing

00:08:08.566 concurrency, and you pick the appropriate one

00:08:11.066 for your domain.


00:08:12.700 Or if there's a more general solution

00:08:15.666 waiting to be discovered.


00:08:18.766 And I think none of these approaches

00:08:20.366 scale out to GPU programming, where you're

00:08:23.100 doing the same operation on 10s,

00:08:26.133 or 100s, of thousands of cores simultaneously.


00:08:30.200 And, maybe, there are approaches with implicit

00:08:32.300 parallelism. Maybe there are data parallel array

00:08:35.800 types that can solve this problem.


00:08:38.000 We haven't really spoken about GPU programming

00:08:40.400 in this course. But, again, there’s a

00:08:43.500 different type of concurrency, which again seems

00:08:45.866 to need different solutions.


00:08:54.100 It's not clear that we have good

00:08:56.866 approaches to making use of data centre

00:08:59.800 scale computing.


00:09:02.200 It’s not clear that we know how

00:09:03.766 to scale systems programming

00:09:06.200 to massive data centres, or to large

00:09:09.366 scale distributed edge compute data centres.


00:09:13.800 It’s not clear that we know how

00:09:16.366 to do distributed computations across millions of cores.


00:09:25.266 We do know how to build large-scale

00:09:28.500 Linux clusters, but we do this by

00:09:33.666 assuming a reliable, robust, homogeneous system.


00:09:38.566 And we do this by essentially automating

00:09:40.600 traditional systems administration.


00:09:44.133 We have DevOps tools, like Ansible,

00:09:48.100 and Salt Stack, and the like,

00:09:51.666 but all they’re doing is shell scripting

00:09:54.333 and editing config files.


00:09:56.566 It's the same type of approach,

00:09:58.833 just automating the ssh into a bunch

00:10:02.033 of machines and edit some config files.


00:10:04.200 And there has to be a better

00:10:05.966 way of doing that, a better way

00:10:08.033 of structuring the system.


00:10:10.900 And we know how to use these

00:10:12.666 data centres, but we know how to

00:10:14.033 use them to run millions of instances of the same thing.


00:10:19.866 It's not clear that we are good at

00:10:25.200 distributing code into these environments.


00:10:31.166 We can make it work

00:10:33.533 for restricted tasks and in homogenous data centres.


00:10:38.700 But as compute becomes more ubiquitous,

00:10:41.266 as it moves out of the controlled

00:10:42.966 data centre and towards the edges of the network,

00:10:46.433 where we need to run millions of

00:10:48.733 tasks over a ubiquitous computing environment that

00:10:51.600 spreads around the world, and is much

00:10:54.400 more heterogeneous, and much more variable,

00:10:57.200 in its configuration, it's not clear we

00:10:59.900 know how to do that that.


00:11:01.433 It’s not clear we have good approaches to doing that.


00:11:09.366 It’s not clear that we necessarily have

00:11:13.266 the right operating systems abstraction for data

00:11:16.833 centres, or for ubiquitous computing,

00:11:20.500 edge computing, environments.


00:11:23.466 But I think are some interesting ideas in this space.


00:11:29.133 The system known as BarrelFish, for example,

00:11:33.033 and the paper on the slide talks about this,

00:11:36.866 has a goal of building a message-passing

00:11:40.533 operating system kernel for multi core systems.


00:11:45.000 And the idea here, is if you

00:11:46.833 have a system with multiple processors in

00:11:49.233 it, each processor core runs its own

00:11:52.066 instance of the operating system, and they

00:11:54.733 communicate with each other via message passing.


00:11:58.200 And there's no use of shared memory,

00:12:00.900 other than to implement the message passing

00:12:02.966 system as an optimisation, and there's no

00:12:05.666 main CPU in a system to act

00:12:07.600 as a central coordinator.


00:12:09.966 So if you have a quad core

00:12:13.300 phone, for example, then each of the

00:12:16.366 cores on the phone would be running

00:12:17.966 a separate instance of this operating system,

00:12:19.900 communicating internally via message passing.


00:12:25.466 And it's taking message passing as the

00:12:28.066 fundamental primitive. It's taking this idea to

00:12:30.166 its extreme, to separate every CPU into

00:12:33.500 its own message passing core.


00:12:36.566 And it's very much a research prototype,

00:12:38.866 to test an idea. It's not something

00:12:41.133 that you could currently make a robust

00:12:42.833 product out. But I think it's an

00:12:44.833 interesting approach, and I think it leads

00:12:47.033 to an interesting discussion about what's the

00:12:49.733 boundary of an operating system?


00:12:53.133 What's the distinction between a distributed multi-kernel,

00:12:57.300 multiple operating system kernels running on a

00:13:00.666 single machine exchanging messages with threads,

00:13:03.566 and one running across the network?


00:13:06.666 And should there, or does there even

00:13:09.266 need to be, such an assumption,

00:13:13.366 such a distinction?


00:13:17.100 Can we just run this type of

00:13:20.100 multi kernel across a data centre,

00:13:22.233 across an edge computing environment, for example?


00:13:26.033 Where are the boundaries between systems?


00:13:28.566 And what should be the boundary between systems?


00:13:31.700 So I think there's a bunch of

00:13:32.866 interesting questions in this space, that have

00:13:34.966 not yet been fully explored. And I

00:13:37.933 hope we’ll see some innovation in the

00:13:39.633 way we build operating systems going forward.


00:13:46.366 And we're increasingly seeing innovation in programming

00:13:50.733 languages. We spent a long time stuck

00:13:56.400 with C and C++, as a sort of local maxima.


00:14:02.000 But we're starting to see more research

00:14:05.400 ideas coming into mainstream languages.


00:14:10.133 And in Rust, as we've spoken about

00:14:12.666 in most of this course, and in

00:14:15.533 Swift, we're beginning to see abstraction,

00:14:18.333 and resource management, and

00:14:20.733 more expressive type systems, coupled with low-level

00:14:25.100 control, giving more powerful, safer,

00:14:31.200 low-level programming environments.


00:14:33.733 And languages like Erlang and Go,

00:14:36.100 are exploring the concurrency and fault tolerance

00:14:39.100 and communication space. And there are others

00:14:41.900 which are bubbling up, and people are

00:14:43.833 starting to explore how things can work.


00:14:48.166 And I think it’s long overdue that

00:14:50.700 we're seeing this development in the underlying

00:14:54.133 languages, and we're beginning to have more

00:14:57.266 expressive ways of describing what our program should do.


00:15:04.066 We should clearly, I think, going forward

00:15:06.766 choose the language for our systems based

00:15:09.733 on the problem domain, based on the

00:15:11.733 tooling based, based on our expertise.


00:15:14.300 And it's not clear that single language

00:15:17.066 fits the entire space.


00:15:20.200 Rust, for example, works extraordinarily well for

00:15:24.166 building operating systems.


00:15:26.200 But languages like Erlang or Go are

00:15:29.600 possibly better suited to

00:15:34.066 large-scale web application development running in a

00:15:39.866 data centre, for example.


00:15:43.533 We've got good solutions for programming single

00:15:46.600 systems. I think Rust is great for

00:15:49.466 programming within a single system.


00:15:52.033 We're starting to learn about how to

00:15:54.933 scale to large clusters, to ubiquitous computing,

00:15:58.600 to edge computing environments, and I think

00:16:00.800 Erlang and Go are starting to have

00:16:02.533 some good ideas in this space,

00:16:04.300 but we don't have a language that

00:16:05.700 fills all of these spaces yet.


00:16:12.833 The systems we use, Unix and C,

00:16:16.533 made sense when we were programming simple

00:16:20.200 single core systems.


00:16:22.966 And they form on the basis of

00:16:24.400 pretty-much every computing system we use today.


00:16:30.133 But the system we see on the left,

00:16:34.166 programmers at a PDP 10 with a

00:16:37.133 teletype, is not the data centre environment

00:16:40.133 we see on the right, where a

00:16:41.366 lot of modern software lives.


00:16:45.500 And I do think we need new

00:16:49.600 approaches to build software that is massively

00:16:51.933 concurrent and massively distributed.


00:16:55.733 Languages like Rust solve part of this

00:16:57.966 problem space, and give us low-level performance,

00:17:01.333 they give us an effective type system,

00:17:03.566 they give us abstractions for writing the

00:17:05.666 low-level code running on a single machine.


00:17:08.866 And that's essential, because no-one can hold

00:17:11.366 all those details in their head,

00:17:13.933 and we need this abstraction, we need

00:17:15.866 the compiler help to write this sort of code correctly.


00:17:19.833 We need the tooling to check our

00:17:22.200 invariants, document our assumptions, ensure consistency.


00:17:27.533 But once we start scaling out to

00:17:30.000 data centres, and to the sort of

00:17:32.866 future ubiquitous computing, edge computing, world it

00:17:35.866 seems we are likely to get,

00:17:38.233 it's not clear that we have the

00:17:39.933 right DevOps, infrastructure as code, self-managing,

00:17:44.033 autonomic management systems, and the right approaches

00:17:47.366 for programming at such large scales.


00:17:55.033 And that is, essentially, all I want to say.


00:18:00.900 The course, obviously, will have some assessment at the end.


00:18:06.566 There will be a final exam,

00:18:08.866 worth 80% of the marks, which will

00:18:11.700 happen in the April/May time frame.


00:18:15.000 There is a sample exam paper,

00:18:17.400 with some worked answers, available on Moodle,

00:18:21.000 to give you an idea for the

00:18:22.633 feel of the questions.


00:18:26.633 The idea is very much to test

00:18:30.033 your understanding, rather than testing your memory.


00:18:34.133 Especially with the type of open-book exam,

00:18:38.466 which it’s necessary to offer with online

00:18:41.800 assessment, there's no point asking questions that

00:18:45.300 you can just Google the answer for,

00:18:47.233 and you can just look up in the notes.


00:18:49.433 We're looking to test your

00:18:50.666 deeper understanding of the material.


00:18:53.266 So think read-over the material, watch the

00:18:57.300 lectures, look at the material from the

00:19:00.166 papers, and the labs, and so on,

00:19:02.866 and think about it, learn about and

00:19:05.266 understand the material deeply.


00:19:08.300 And look out for questions

00:19:10.200 testing that deeper understanding.


00:19:15.400 There’s no specific revision lecture scheduled,

00:19:19.100 but we do have the Teams chat

00:19:21.333 and email, if you have questions about

00:19:23.200 the material. And the marks for the

00:19:27.166 assessed coursework it will be available shortly,

00:19:31.266 and I'm sorry it's taken a while to get to those.


00:19:37.700 To finish up,

00:19:39.700 if you're interested further in these topics,

00:19:43.100 if you're continuing to Level 5 and

00:19:46.100 are interested in doing an MSI project

00:19:48.433 in this space, please contact me,

00:19:51.200 send me email and we'll talk.


00:19:54.200 I’m very keen to work with motivated

00:19:56.566 students to develop

00:19:58.566 MSci projects looking at, perhaps, networking and

00:20:03.133 transport protocol APIs using some of these

00:20:06.000 modern languages and modern approaches.

00:20:09.300 I’m keen to look at kernel bypass networking,

00:20:15.266 kernel and user space networking APIs that

00:20:20.400 avoid a lot of the overheads in the current approaches.


00:20:23.933 I'm keen to look into systems programming

00:20:26.300 models for large-scale edge compute systems,

00:20:29.733 and try to understand how we should

00:20:32.166 best be developing and programming such systems.


00:20:35.333 So, if any of this is of

00:20:36.566 interest to you, and you're staying on

00:20:38.966 for Level 5 and looking for an

00:20:40.900 MSci project, then comes talk to me.


00:20:45.933 And that's it.


00:20:48.366 A final paper to highlight

00:20:52.966 some of the issues around systems programming,

00:20:56.366 which you will hopefully find interesting.


00:20:59.433 I hope you found some of the

00:21:01.333 material in the course interesting. Thank you

00:21:04.000 for your attention.


00:21:06.566 And if there are any further questions,

00:21:08.333 please do get in touch. Thank you.


Lecture 10 summarised the course content and briefly discussed future directions for systems programming. It noted that there are still no good, general purpose, solutions for concurrent programming, and highlighted some issues with programming large scale systems in data centres and edge computing environments.

Discussion will focus on future directions for systems programming, and on summarising key points of the material from the course as a whole.