Parallel processing and virtual machines
Doing many things at once
- Modern computing relies on parallelism — many processors working together.
- Flynn's taxonomy classifies the styles.
- And a virtual machine runs a whole computer in software.
Flynn's taxonomy
- SISD — one instruction, one data stream (a traditional single core).
- SIMD — one instruction works on many data items at once (GPUs, vector units) — great for images and arrays.
- MISD — several operations on the same data (rare, mostly theoretical).
- MIMD — many processors run different instructions on different data (multi-core CPUs, clusters) — the most general.
Practice
SIMD means:
Single Instruction, Multiple Data — e.g. a GPU running the same operation on thousands of pixels.
Practice
A multi-core CPU running different programs on each core is an example of:
Multiple Instruction, Multiple Data — independent instruction streams on independent data.
Massively parallel computers
- A massively parallel system uses thousands of processors on a fast network, each with its own distributed memory, exchanging data by messages.
- It is MIMD, needs special software (MPI, CUDA), and powers the largest supercomputers — climate simulation, ML training, astrophysics.
Practice
A massively parallel computer uses:
Massively parallel systems (supercomputers) link thousands of processors with distributed memory over a fast network.
Virtual machines
- A virtual machine (VM) is a software emulation of a whole computer.
- A system VM runs a complete guest OS; a hypervisor creates and manages several VMs. Uses: run different OSes on one machine, server consolidation, sandboxing risky software.
- A process (language) VM runs portable bytecode (the JVM, .NET CLR): portability, runtime safety, and JIT for speed — at the cost of an extra layer.
Practice
A hypervisor is used to:
A hypervisor manages system VMs, letting several guest operating systems share one physical machine.
Practice
A process (language) virtual machine such as the JVM:
A process VM runs one program's portable bytecode with runtime safety and JIT — not a whole OS.
You've got it
Key idea
- Flynn: SISD, SIMD (one instruction, many data — GPUs), MISD, MIMD (multi-core)
- massively parallel = thousands of processors with distributed memory (supercomputers)
- a system VM + hypervisor runs whole guest OSes (sandboxing, consolidation)
- a process VM runs portable bytecode (JVM) — "write once, run anywhere"