Advanced Systems Programming H (2021-2022)
Lecture 10: Future Directions
The final lecture concludes by reviewing the course material, and discussing some possible future directions for systems programming.
Part 1: Future Directions
The final part of the course talks about future directions and open issues. It notes that there are still open questions around how best to support concurrent programming, data centre-scale computing, and large-scale distributed operating systems. It also highlights the benefits of modern programming languages for developing programs that run within a single system, and notes the challenges in scaling out to massively distributed systems.
00:00:00.200 In this last lecture, I’d like to
00:00:02.000 talk about some open issues and future
00:00:03.733 directions for systems programming.
00:00:08.466 So, in this course we've discussed a
00:00:11.200 number of advanced topics in the systems
00:00:13.733 programming space.
00:00:15.900 We spent some time talking about what
00:00:18.300 is systems programming, and what distinguishes a
00:00:21.000 systems program from an applications program,
00:00:24.966 focusing on control over memory layout,
00:00:29.633 and performance needs, and so on
00:00:33.533 We spent a fair amount of time
00:00:34.866 talking about type systems, and type based
00:00:37.333 modelling and design, and thinking about the
00:00:41.233 benefits of advanced type systems for building
00:00:44.933 systems programs.
00:00:46.766 Thinking about some of the constraints and
00:00:48.600 issues around modelling the problem space using
00:00:52.800 types, and using that to help robustness
00:00:55.433 and security of programs.
00:00:59.133 We focused a lot on resource ownership
00:01:02.200 and memory management, and tracking how the
00:01:06.166 program works with, controls, and owns resources,
00:01:10.833 including memory, but also including state machines,
00:01:14.833 and the state variables in state machines.
00:01:17.600 And we spoke about different approaches to
00:01:19.400 memory management, the reference counting approaches,
00:01:23.666 garbage collection, and region-based memory management.
00:01:28.566 And we spent a fair amount of
00:01:29.866 time talking about concurrency.
00:01:32.633 Talking about message passing approaches to concurrency.
00:01:37.200 Talking about transactions, talking about co-routines and
00:01:40.333 asynchronous programming. And we spent some time
00:01:44.000 discussing why the common approach of using multithreading,
00:01:48.166 locks, and shared mutable state, is not
00:01:52.400 an effective approach, and doesn't scale out
00:01:55.866 to larger systems, and is very hard to get right.
00:02:00.900 And, we just finished up by talking
00:02:03.033 about some of the security considerations in
00:02:05.166 these systems, and thinking about
00:02:08.733 how we can use advanced type systems
00:02:12.500 to make explicit the assumptions in the
00:02:15.766 code, and reduce the risk of inconsistencies
00:02:21.033 which lead to security breaches. And we
00:02:24.233 spent some time talking about how memory
00:02:26.666 unsafety is the primary cause of security vulnerabilities.
00:02:32.600 What did we learn from this?
00:02:36.700 Well, I think one of the key
00:02:39.500 distinguishing features with systems programs, is that
00:02:42.833 they need low-level control over the data layout.
00:02:47.266 In order to effectively write device drivers,
00:02:50.133 in order to effectively write network protocols,
00:02:52.866 or file I/O,
00:02:55.766 and in order to make effective use
00:02:58.000 of memory, and fit within the cache,
00:03:00.933 and optimise the layout of data structures
00:03:04.666 to fit within the cache,
00:03:07.533 we need to be able to control
00:03:09.400 the way data is laid out in memory.
00:03:12.833 One of the key features of systems
00:03:15.266 programs, is that they give that control.
00:03:18.233 And going forward, I expect systems programming
00:03:20.466 languages to continue to do that.
00:03:23.300 We need to be able to specify
00:03:25.633 exactly how a data structure is laid
00:03:28.000 out in memory, and to be able
00:03:29.800 to control whether it is allocated on
00:03:31.533 the stack or on the heap.
00:03:33.933 And to be able to box,
00:03:35.533 or unbox, data structures as we need
00:03:37.766 to, to get good performance.
00:03:43.066 In traditional systems programming languages,
00:03:46.000 that control has
00:03:49.400 come at the expense a lot of safety.
00:03:54.433 C provides you with a tremendous amount
00:03:57.833 of control over the way data is
00:03:59.866 laid out in memory, with tremendous amount
00:04:02.700 of control over representations, and pointer use,
00:04:06.266 and so on, but in a way
00:04:08.033 that's quite unsafe, and quite hard to get right.
00:04:12.100 More modern languages,
00:04:14.500 Rust being the example we’re using in
00:04:17.633 this course, but certainly not the only
00:04:20.166 language in the space,
00:04:22.433 provide this type of behaviour in a much
00:04:28.100 safer way. They give you control over data layout,
00:04:31.800 but they also provide memory safety,
00:04:34.366 they check it array bounds they check
00:04:36.033 pointers are used correctly, and they can
00:04:39.100 avoid problems like use-after-free bugs, data races,
00:04:42.866 iterator invalidation, and so on, and get
00:04:46.166 the control over data layout whilst maintaining
00:04:48.933 a lot more safety,
00:04:51.400 a lot more robust behaviours.
00:04:56.566 I think it's clear that concurrency is
00:04:59.366 important, especially as Moore's law and Dennard
00:05:03.300 scaling break down, and we get increasingly
00:05:07.033 concurrent systems.
00:05:12.166 And we spoke about several different approaches
00:05:14.100 to concurrency, but I think it's clear
00:05:16.166 they've all got trade-offs, and none of
00:05:18.200 them are a one size fits all
00:05:20.800 approach to building concurrent systems.
00:05:26.500 The transactional memory approach, I think,
00:05:28.833 works wonderfully in Haskell, but it's not
00:05:32.366 clear it scales out to other languages.
00:05:35.533 The message passing approach works wonderfully in Erlang,
00:05:41.100 but fits less well, less naturally,
00:05:44.566 into many other languages, and has some
00:05:47.733 difficult safety issues in some cases.
00:05:52.200 And the asynchronous coroutine based approach
00:05:56.033 leads to strange behaviours with starvation and blocking.
00:05:59.966 There’s constraints, and it’s not clear
00:06:02.666 we have the right answer.
00:06:06.166 And we’ve shown that type systems,
00:06:07.900 I think, can help model the problem domain.
00:06:09.833 I think that's one of the key takeaways.
00:06:12.400 We can model the assumptions, we can
00:06:14.933 help ensure the system is safe and correct.
00:06:21.366 And we can make the systems much
00:06:24.233 safer, and avoid a lot of common
00:06:26.166 problems, by encoding more information into the
00:06:28.400 types, and letting the compiler help us
00:06:30.666 check the systems for correctness.
00:06:37.200 So that's where we are at today.
00:06:39.633 That, I think, is the current state-of-the-art,
00:06:41.933 and what we have learned from building
00:06:44.166 systems programs today.
00:06:47.333 Where we going in the future?
00:06:51.166 Well, as I mentioned, I think we
00:06:53.000 still don't have good solutions for concurrent programming.
00:06:57.733 Transactional memory, I think, is very promising,
00:07:00.833 but doesn't fit well with impure imperative
00:07:05.033 languages, and most of the programs we
00:07:07.966 write today are written in impure imperative languages.
00:07:13.500 Message passing is quite a natural approach,
00:07:18.500 and fits nicely into mainstream languages,
00:07:23.100 and solves some of the problems with
00:07:26.233 shared mutable state, and threads and locks,
00:07:29.300 but it's got problems with providing back
00:07:32.133 pressure, it can have race conditions and
00:07:34.933 deadlocks, and it's difficult to structure programs,
00:07:39.366 structure large message passing systems, without losing
00:07:42.333 track of the message flows.
00:07:45.800 And asynchronous code introduces subtle
00:07:48.800 blocking and starvation bugs.
00:07:52.433 And it's not really clear what's the right solution.
00:07:55.666 It’s not clear if the problem domain
00:07:58.533 is sufficiently broad that we need different
00:08:01.533 types of solution for different problem domains,
00:08:04.800 and we are just going to end
00:08:06.400 up with three-or-four different ways of doing
00:08:08.566 concurrency, and you pick the appropriate one
00:08:11.066 for your domain.
00:08:12.700 Or if there's a more general solution
00:08:15.666 waiting to be discovered.
00:08:18.766 And I think none of these approaches
00:08:20.366 scale out to GPU programming, where you're
00:08:23.100 doing the same operation on 10s,
00:08:26.133 or 100s, of thousands of cores simultaneously.
00:08:30.200 And, maybe, there are approaches with implicit
00:08:32.300 parallelism. Maybe there are data parallel array
00:08:35.800 types that can solve this problem.
00:08:38.000 We haven't really spoken about GPU programming
00:08:40.400 in this course. But, again, there’s a
00:08:43.500 different type of concurrency, which again seems
00:08:45.866 to need different solutions.
00:08:54.100 It's not clear that we have good
00:08:56.866 approaches to making use of data centre
00:08:59.800 scale computing.
00:09:02.200 It’s not clear that we know how
00:09:03.766 to scale systems programming
00:09:06.200 to massive data centres, or to large
00:09:09.366 scale distributed edge compute data centres.
00:09:13.800 It’s not clear that we know how
00:09:16.366 to do distributed computations across millions of cores.
00:09:25.266 We do know how to build large-scale
00:09:28.500 Linux clusters, but we do this by
00:09:33.666 assuming a reliable, robust, homogeneous system.
00:09:38.566 And we do this by essentially automating
00:09:40.600 traditional systems administration.
00:09:44.133 We have DevOps tools, like Ansible,
00:09:48.100 and Salt Stack, and the like,
00:09:51.666 but all they’re doing is shell scripting
00:09:54.333 and editing config files.
00:09:56.566 It's the same type of approach,
00:09:58.833 just automating the ssh into a bunch
00:10:02.033 of machines and edit some config files.
00:10:04.200 And there has to be a better
00:10:05.966 way of doing that, a better way
00:10:08.033 of structuring the system.
00:10:10.900 And we know how to use these
00:10:12.666 data centres, but we know how to
00:10:14.033 use them to run millions of instances of the same thing.
00:10:19.866 It's not clear that we are good at
00:10:25.200 distributing code into these environments.
00:10:31.166 We can make it work
00:10:33.533 for restricted tasks and in homogenous data centres.
00:10:38.700 But as compute becomes more ubiquitous,
00:10:41.266 as it moves out of the controlled
00:10:42.966 data centre and towards the edges of the network,
00:10:46.433 where we need to run millions of
00:10:48.733 tasks over a ubiquitous computing environment that
00:10:51.600 spreads around the world, and is much
00:10:54.400 more heterogeneous, and much more variable,
00:10:57.200 in its configuration, it's not clear we
00:10:59.900 know how to do that that.
00:11:01.433 It’s not clear we have good approaches to doing that.
00:11:09.366 It’s not clear that we necessarily have
00:11:13.266 the right operating systems abstraction for data
00:11:16.833 centres, or for ubiquitous computing,
00:11:20.500 edge computing, environments.
00:11:23.466 But I think are some interesting ideas in this space.
00:11:29.133 The system known as BarrelFish, for example,
00:11:33.033 and the paper on the slide talks about this,
00:11:36.866 has a goal of building a message-passing
00:11:40.533 operating system kernel for multi core systems.
00:11:45.000 And the idea here, is if you
00:11:46.833 have a system with multiple processors in
00:11:49.233 it, each processor core runs its own
00:11:52.066 instance of the operating system, and they
00:11:54.733 communicate with each other via message passing.
00:11:58.200 And there's no use of shared memory,
00:12:00.900 other than to implement the message passing
00:12:02.966 system as an optimisation, and there's no
00:12:05.666 main CPU in a system to act
00:12:07.600 as a central coordinator.
00:12:09.966 So if you have a quad core
00:12:13.300 phone, for example, then each of the
00:12:16.366 cores on the phone would be running
00:12:17.966 a separate instance of this operating system,
00:12:19.900 communicating internally via message passing.
00:12:25.466 And it's taking message passing as the
00:12:28.066 fundamental primitive. It's taking this idea to
00:12:30.166 its extreme, to separate every CPU into
00:12:33.500 its own message passing core.
00:12:36.566 And it's very much a research prototype,
00:12:38.866 to test an idea. It's not something
00:12:41.133 that you could currently make a robust
00:12:42.833 product out. But I think it's an
00:12:44.833 interesting approach, and I think it leads
00:12:47.033 to an interesting discussion about what's the
00:12:49.733 boundary of an operating system?
00:12:53.133 What's the distinction between a distributed multi-kernel,
00:12:57.300 multiple operating system kernels running on a
00:13:00.666 single machine exchanging messages with threads,
00:13:03.566 and one running across the network?
00:13:06.666 And should there, or does there even
00:13:09.266 need to be, such an assumption,
00:13:13.366 such a distinction?
00:13:17.100 Can we just run this type of
00:13:20.100 multi kernel across a data centre,
00:13:22.233 across an edge computing environment, for example?
00:13:26.033 Where are the boundaries between systems?
00:13:28.566 And what should be the boundary between systems?
00:13:31.700 So I think there's a bunch of
00:13:32.866 interesting questions in this space, that have
00:13:34.966 not yet been fully explored. And I
00:13:37.933 hope we’ll see some innovation in the
00:13:39.633 way we build operating systems going forward.
00:13:46.366 And we're increasingly seeing innovation in programming
00:13:50.733 languages. We spent a long time stuck
00:13:56.400 with C and C++, as a sort of local maxima.
00:14:02.000 But we're starting to see more research
00:14:05.400 ideas coming into mainstream languages.
00:14:10.133 And in Rust, as we've spoken about
00:14:12.666 in most of this course, and in
00:14:15.533 Swift, we're beginning to see abstraction,
00:14:18.333 and resource management, and
00:14:20.733 more expressive type systems, coupled with low-level
00:14:25.100 control, giving more powerful, safer,
00:14:31.200 low-level programming environments.
00:14:33.733 And languages like Erlang and Go,
00:14:36.100 are exploring the concurrency and fault tolerance
00:14:39.100 and communication space. And there are others
00:14:41.900 which are bubbling up, and people are
00:14:43.833 starting to explore how things can work.
00:14:48.166 And I think it’s long overdue that
00:14:50.700 we're seeing this development in the underlying
00:14:54.133 languages, and we're beginning to have more
00:14:57.266 expressive ways of describing what our program should do.
00:15:04.066 We should clearly, I think, going forward
00:15:06.766 choose the language for our systems based
00:15:09.733 on the problem domain, based on the
00:15:11.733 tooling based, based on our expertise.
00:15:14.300 And it's not clear that single language
00:15:17.066 fits the entire space.
00:15:20.200 Rust, for example, works extraordinarily well for
00:15:24.166 building operating systems.
00:15:26.200 But languages like Erlang or Go are
00:15:29.600 possibly better suited to
00:15:34.066 large-scale web application development running in a
00:15:39.866 data centre, for example.
00:15:43.533 We've got good solutions for programming single
00:15:46.600 systems. I think Rust is great for
00:15:49.466 programming within a single system.
00:15:52.033 We're starting to learn about how to
00:15:54.933 scale to large clusters, to ubiquitous computing,
00:15:58.600 to edge computing environments, and I think
00:16:00.800 Erlang and Go are starting to have
00:16:02.533 some good ideas in this space,
00:16:04.300 but we don't have a language that
00:16:05.700 fills all of these spaces yet.
00:16:12.833 The systems we use, Unix and C,
00:16:16.533 made sense when we were programming simple
00:16:20.200 single core systems.
00:16:22.966 And they form on the basis of
00:16:24.400 pretty-much every computing system we use today.
00:16:30.133 But the system we see on the left,
00:16:34.166 programmers at a PDP 10 with a
00:16:37.133 teletype, is not the data centre environment
00:16:40.133 we see on the right, where a
00:16:41.366 lot of modern software lives.
00:16:45.500 And I do think we need new
00:16:49.600 approaches to build software that is massively
00:16:51.933 concurrent and massively distributed.
00:16:55.733 Languages like Rust solve part of this
00:16:57.966 problem space, and give us low-level performance,
00:17:01.333 they give us an effective type system,
00:17:03.566 they give us abstractions for writing the
00:17:05.666 low-level code running on a single machine.
00:17:08.866 And that's essential, because no-one can hold
00:17:11.366 all those details in their head,
00:17:13.933 and we need this abstraction, we need
00:17:15.866 the compiler help to write this sort of code correctly.
00:17:19.833 We need the tooling to check our
00:17:22.200 invariants, document our assumptions, ensure consistency.
00:17:27.533 But once we start scaling out to
00:17:30.000 data centres, and to the sort of
00:17:32.866 future ubiquitous computing, edge computing, world it
00:17:35.866 seems we are likely to get,
00:17:38.233 it's not clear that we have the
00:17:39.933 right DevOps, infrastructure as code, self-managing,
00:17:44.033 autonomic management systems, and the right approaches
00:17:47.366 for programming at such large scales.
00:17:55.033 And that is, essentially, all I want to say.
00:18:00.900 The course, obviously, will have some assessment at the end.
00:18:06.566 There will be a final exam,
00:18:08.866 worth 80% of the marks, which will
00:18:11.700 happen in the April/May time frame.
00:18:15.000 There is a sample exam paper,
00:18:17.400 with some worked answers, available on Moodle,
00:18:21.000 to give you an idea for the
00:18:22.633 feel of the questions.
00:18:26.633 The idea is very much to test
00:18:30.033 your understanding, rather than testing your memory.
00:18:34.133 Especially with the type of open-book exam,
00:18:38.466 which it’s necessary to offer with online
00:18:41.800 assessment, there's no point asking questions that
00:18:45.300 you can just Google the answer for,
00:18:47.233 and you can just look up in the notes.
00:18:49.433 We're looking to test your
00:18:50.666 deeper understanding of the material.
00:18:53.266 So think read-over the material, watch the
00:18:57.300 lectures, look at the material from the
00:19:00.166 papers, and the labs, and so on,
00:19:02.866 and think about it, learn about and
00:19:05.266 understand the material deeply.
00:19:08.300 And look out for questions
00:19:10.200 testing that deeper understanding.
00:19:15.400 There’s no specific revision lecture scheduled,
00:19:19.100 but we do have the Teams chat
00:19:21.333 and email, if you have questions about
00:19:23.200 the material. And the marks for the
00:19:27.166 assessed coursework it will be available shortly,
00:19:31.266 and I'm sorry it's taken a while to get to those.
00:19:37.700 To finish up,
00:19:39.700 if you're interested further in these topics,
00:19:43.100 if you're continuing to Level 5 and
00:19:46.100 are interested in doing an MSI project
00:19:48.433 in this space, please contact me,
00:19:51.200 send me email and we'll talk.
00:19:54.200 I’m very keen to work with motivated
00:19:56.566 students to develop
00:19:58.566 MSci projects looking at, perhaps, networking and
00:20:03.133 transport protocol APIs using some of these
00:20:06.000 modern languages and modern approaches.
00:20:09.300 I’m keen to look at kernel bypass networking,
00:20:15.266 kernel and user space networking APIs that
00:20:20.400 avoid a lot of the overheads in the current approaches.
00:20:23.933 I'm keen to look into systems programming
00:20:26.300 models for large-scale edge compute systems,
00:20:29.733 and try to understand how we should
00:20:32.166 best be developing and programming such systems.
00:20:35.333 So, if any of this is of
00:20:36.566 interest to you, and you're staying on
00:20:38.966 for Level 5 and looking for an
00:20:40.900 MSci project, then comes talk to me.
00:20:45.933 And that's it.
00:20:48.366 A final paper to highlight
00:20:52.966 some of the issues around systems programming,
00:20:56.366 which you will hopefully find interesting.
00:20:59.433 I hope you found some of the
00:21:01.333 material in the course interesting. Thank you
00:21:04.000 for your attention.
00:21:06.566 And if there are any further questions,
00:21:08.333 please do get in touch. Thank you.
Lecture 10 summarised the course content and briefly discussed future directions for systems programming. It noted that there are still no good, general purpose, solutions for concurrent programming, and highlighted some issues with programming large scale systems in data centres and edge computing environments.
Discussion will focus on future directions for systems programming, and on summarising key points of the material from the course as a whole.