Data integrity — validation and verification
Introduced| English | Meaning |
|---|---|
| integrity/ɪnˈteɡrɪti/ | The accuracy, completeness and protection of data against unauthorised or unintended alteration. |
| validation/ˌvælɪˈdeɪʃn/ | Checking whether data meets specified rules for acceptable values or format. |
| verification/ˌverɪfɪˈkeɪʃn/ | Checking that data has been entered or copied accurately from its source, rather than deciding whether the source itself is correct. |
| range check/reɪndʒ tʃek/ | A validation test that checks whether a value lies within specified permitted limits. |
| check digit/tʃek ˈdɪdʒɪt/ | The remainder when the digit sum is divided by 10, appended to the number. |
| double entry/ˈdʌbl ˈentri/ | double entry |
| parity check/ˈpærɪti tʃek/ | An error-detection method that checks whether the number of set bits matches an agreed odd or even parity. |
| checksum/ˈtʃeksəm/ | A value computed from data to help detect corruption; different data can sometimes produce the same value. |
| cyclic redundancy check/ˈsaɪklɪk rɪˈdʌndənsi tʃek/ | An error-detection method that compares a check value calculated using polynomial division with the value sent or stored alongside the data. |
| parity block check/ˈpærɪti blɒk tʃek/ | Parity checks applied across both rows and columns of a block of bits, which can help locate a single-bit error. |
Every number passed every check
- In September 1999 NASA's Mars Climate Orbiter fired its engine to enter orbit and was never heard from again. It had flown 170 km lower than planned and burned up.
- One team's software produced thrust figures in pound-force seconds; the receiving software expected newton-seconds. Every value was a well-formed number in a sensible range. Every value was wrong by a factor of 4.45.
- No format check could have caught it, because the data was not malformed. It was simply not what it claimed to be. $327 million.
- This lesson is the two families of checks that guard integrity 完整性: validation, which asks whether data is sensible, and verification, which asks whether it was copied correctly. And the gap between them that Mars fell through.
Two questions
- Data has integrity when it is accurate, consistent and complete.
- Validation 验证 is an automatic check that entered data is reasonable and follows set rules. It asks: is this sensible?
- Verification 核对 checks that data has been entered or transferred correctly, by comparing it with the source or with a recomputed value. It asks: was this copied correctly?
- Neither proves the data is true. Both are needed.
Computing concept lab
Classify concrete examples by the computing idea they demonstrate.
Validation methods
| Check | Rule | Example |
|---|---|---|
| range check 范围检查 | between two limits | a month is 1 to 12 |
| limit check | on one side of a single limit | age ≥ 18 |
| existence (lookup) check | the value is in a list or table | the product code exists |
| length check | the right number of characters | a postcode of 6 characters |
| type / character check | the right kind of data | a phone field allows only digits |
| format check | matches a pattern | an email contains @ |
| presence check | required fields are not empty | a name is entered |
| check digit 校验位 | an extra digit computed from the others | ISBNs, card numbers |
Checking that a month entered is between 1 and 12 is a:
Difficulty 1/5 · 100 coins + a small bonus on first completion today
A range check confirms a value lies within sensible limits.
Match each validation check to its purpose.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
Presence = not empty; format = matches a pattern; check digit = computed to catch mistyped digits.
An extra digit computed from the others to catch a mistyped number (as on ISBNs and card numbers) is a ______ digit.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
A check digit is recomputed from the rest and compared, catching most single-digit and transposition errors.
Worked example: name the check the pseudocode performs
| Pseudocode | Check |
|---|---|
IF x < 0 OR x > 10 THEN OUTPUT "Invalid" |
range check: between two limits |
IF x = "" THEN OUTPUT "Invalid" |
presence check: not empty |
IF NOT(x = "Red" OR x = "Yellow" OR x = "Blue") THEN OUTPUT "Invalid" |
existence (lookup) check: one of a list |
IF LENGTH(x) <> 6 THEN OUTPUT "Invalid" |
length check: six characters |
IF MID(x, 1, 1) < "A" OR MID(x, 1, 1) > "Z" THEN OUTPUT "Invalid" |
format check: the first character is a letter |
- A car registration of one letter, three digits and two letters needs a format check on each position and a length check for six characters. A date of birth needs a format check (
DD/MM/YYYY), a range check (the month is 1 to 12, the year not in the future) and a presence check.
Match each line of pseudocode to the validation check it performs.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
Empty test = presence; character count = length; two limits = range; a list of allowed values = existence or lookup.
Worked example: a check digit, and what it misses
- A simple scheme: the check digit is the remainder when the digit sum is divided by 10, appended to the number.
4162has digit sum 13, so it is stored as41623. - A user types
41523: the sum is now 12, the check digit should be 2, so the error is caught. - A user types
14623: the first two digits are swapped, but the sum is still 13, so the check digit matches and the error is not caught. Real schemes (ISBN, card numbers) weight each position so that a swap changes the result.
What validation cannot do
- Validation catches data that is wrongly formatted or out of range.
- It cannot catch data that has the right format and is factually wrong.
Bobtyped forBibpasses every check; so did every thrust figure sent to Mars. - For that you need verification against the source, and a person who reads what they typed.
Why can validation still let bad data through?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Validation checks the form, not the truth — a correctly formatted but wrong value passes.
Verification during data entry
- Double entry 双重录入: the data is entered twice, by the same person or by two people, and the computer compares the two versions and reports any difference. A new password is the everyday case.
- Visual check: the person compares what is on the screen with the original source document and corrects any difference before saving.
- Both protect integrity by making sure the stored data matches the source.
Asking a user to type a new password twice is an example of:
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Double entry compares two typed copies to verify the data was entered correctly.
Verification during data transfer
- Bits can flip in transit. A parity check 奇偶校验 adds one extra bit to each byte so that the number of 1s is even (even parity) or odd; the receiver recounts, and a single flipped bit fails the count.
- A checksum 校验和 is a value calculated from a block of data by an algorithm and sent with it; the receiver recalculates it and compares. Transferring video files from a camera to a server works this way: a mismatch means retransmit.
- A cyclic redundancy check 循环冗余校验 (CRC) is a stronger checksum using polynomial division, catching many more error patterns.
One extra bit makes the count of 1s even; a flipped bit breaks it
Compute, send, recompute, compare
A parity check works by:
Difficulty 1/5 · 100 coins + a small bonus on first completion today
The parity bit makes the count of 1s even or odd; the receiver re-counts to detect a single-bit error.
Which are verification methods used during data transfer? Select all that apply.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
Parity and checksums recompute a value at the receiving end. Double entry and a visual check happen during data entry.
A byte parity check detects every possible transmission error in that byte.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Two flipped bits leave the count of 1s unchanged, so they pass. Parity reliably catches single-bit errors only.
Worked example: a parity block locates the bit
- A parity block check 奇偶块校验 arranges the bytes in a grid: each byte carries a row parity bit, and one extra parity byte holds the column parity of the bytes above it.
- Four bytes are sent with even parity, followed by the parity byte. Count the 1s in every row and column: byte 3 has five and column 4 has three, both odd.
- The bit where that row and that column cross is the one that changed, so it is reset from 1 to 0. A single parity bit detects an error; a block locates and corrects it.
One row wrong, one column wrong: the bit is at the crossing
Put the steps of a parity block check in order.
Difficulty 3/5 · 200 coins + a small bonus on first completion today
Rows detect, columns detect, the crossing locates, and the flip corrects. That is what a block adds over a single parity bit.
Worked example: which check catches which error
- A user types their date of birth as
31/02/2009, then types their email address twice. Which check catches which error, and what is the difference? - Validation: a format or range check rejects
31/02/2009, because February never has 31 days. The computer tests the value against a rule. - Verification: double entry compares the two email addresses and reports a mismatch if they differ. The computer compares two copies, and it would accept a wrong address typed identically twice.
- Validation asks whether the data is sensible; verification asks whether it was entered correctly.
Sensible, or copied correctly: two different questions
Which statement is correct?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Validation tests sensibility against rules; verification tests that the data was entered/transferred without change.
Verification proves the data was copied or entered without change — but not that the value is factually correct.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Double entry / checksums prove the data arrived unchanged; only a human or a cross-check can confirm it is actually true.
Marks that slip away
- A check digit is validation on entry; parity and a checksum verify a transfer. Do not file them under the wrong heading.
- Neither check proves the data is true. Validation says "reasonable"; verification says "matches the source".
- A byte parity check misses two flipped bits in the same byte, because they cancel. Say "single-bit errors".
- Encryption is a security measure. It does nothing for integrity.
You've got it
- validation = automatic rules on entry: range, limit, existence, length, type, format, presence, check digit; it cannot catch right-format-but-wrong values
- verification on entry = double entry and a visual check; on transfer = parity (byte or block), checksum, CRC
- a parity block locates a flipped bit at the crossing of the failing row and column
- validation asks "sensible?"; verification asks "copied correctly?"; neither asks "true?"