Parallel and Distributed Computing
Introduced| English |
|---|
| sequential computing/siːˈkwenʃl kəmˈpjuːtɪŋ/ |
| Parallel computing/ˈpærəlel kəmˈpjuːtɪŋ/ |
| processors/ˈprəʊsesəz/ |
| Distributed computing/ˈdɪstrɪbjuːtɪd kəmˈpjuːtɪŋ/ |
| speedup/ˈspiːdʌp/ |
One step at a time
- Traditionally a computer runs steps one at a time.
- This is sequential computing 顺序计算: each step must finish before the next begins.
- It is simple, but it can be slow for big jobs.
- To go faster, we can do several steps at once.
In sequential computing:
One step at a time, in order.
Parallel and distributed
- Parallel computing 并行计算 uses several processors 处理器 in one machine to run parts of a task at the same time.
- Distributed computing 分布式计算 spreads a task across many separate computers, often in different places, connected by a network.
- Parallel = many processors, one machine. Distributed = many machines.
- Both aim to finish a large task faster.
Sequential, parallel, or distributed?
Sequential runs one step at a time; parallel uses several processors in one machine at once; distributed spreads a task across many separate computers.
Match each term to its meaning.
Parallel is one machine; distributed is many machines.
Speedup and its limit
- The benefit of parallel computing is measured by speedup 加速比:
- $\text{speedup} = \dfrac{\text{time using one processor}}{\text{time using several processors}}$.
- But any part that must run sequentially cannot be sped up by adding processors.
- So the sequential portion sets a floor — a hard cap on the total speedup.
A task takes 100 s on one processor and 40 s on several. What is the speedup?
speedup = 100 / 40 = 2.5×.
If 20 s of a 100 s task must run sequentially, what is the maximum possible speedup?
Even with infinite processors, 20 s remains: 100 / 20 = 5×.
Why can adding more processors never remove the speedup cap?
The sequential portion sets a floor on the total time.
Distributed computing suits huge jobs too big for one machine, like indexing the whole web.
The work splits naturally across thousands of computers.
When distributed shines
- Distributed solutions shine on huge jobs — enormous data sets, or a search engine indexing the whole web.
- The work is too big for one machine and splits naturally across thousands.
Speedup with a cap. A 100-second task has 80 s parallel and 20 s sequential. On 4 processors the parallel part is $80 / 4 = 20$ s, plus 20 s sequential = 40 s. Speedup $= 100 / 40 = 2.5\times$. Even with a million processors, the 20 s sequential remains, so speedup can never beat $100 / 20 = 5$.
Sequential computing runs one step at a time. Parallel computing uses several processors in one machine at once; distributed computing spreads work across many machines. Parallel gain is measured as speedup (one-processor time ÷ many-processor time), but the sequential part caps it (here, 5×).