Program Correctness, The specification

Question

From Wikipedia: In theoretical computer science, correctness of an algorithm is asserted when it is said that the algorithm is correct with respect to a specification.

But the problem is that to get the "appropriate" specification is not a trivial task, and there is no 100% correct method (as far as i know) to get the right one, it just an estimation, so if we are going to take a predicate as a specification just because it "looks" like the "one", why not taking the program as correct just because it "looks" correct?

Answer 1

First off, you're absolutely right: you're on to a real concern. Formal verification transfers the problem of confidence in program correctness to the problem of confidence in specification correctness, so it is not a silver bullet.

There are several reasons why this process can still be useful, though.

1. Specifications are often simpler than the code itself. For instance, consider the problem of sorting an array of integers. There are fairly sophisticated sorting algorithms that do clever things to improve performance. But the specification is fairly simple to state: the output must be in increasing order, and must be a permutation of the input. Thus, it is arguably easier to gain confidence in the correctness of the specification than in the correctness of the code itself.

2. There is no single point of failure. Suppose you have one person write down a specification, and another person write the source code, and then formally verify that the code meets the spec. Then any undetected flaw would have to be present in both the spec and the code. In some cases, for some types of flaws, this feels less likely: it's less likely that you'd overlook the flaw when inspecting the spec and overlook the flaw when inspecting the source code. Not all, but some.

3. Partial specs can be vastly simpler than the code. For instance, consider the requirement that the program is free of buffer overrun vulnerabilities. Or, the requirement that there are no array index out-of-bounds errors. This is a simple spec that is fairly obviously a good and useful thing to be able to prove. Now you can try to use formal methods to prove that the entire program meets this spec. That might be a fairly involved task, but if you are successful, you gain increased confidence in the program.

4. Specs might change less frequently than code. Without formal methods, each time we update the source code, we have to manually check that the update won't introduce any bugs or flaws. Formal methods can potentially reduce this burden: suppose the spec doesn't change, so that software updates involve only changes to the code and not changes to the spec. Then for each update, you are relieved of the burden to check whether the spec is still correct (it hasn't changed, so there's no risk new bugs have been introduced in the spec) and of the burden to check whether the code is still correct (the program verifier checks that for you). You still need to check that the original spec is correct, but then you don't need to keep checking it each time a developer commits a new patch/update/change. This can potentially reduce the burden of checking correctness as code is maintained and evolves.

Finally, remember that specs typically are declarative and can't necessarily be executed nor compiled directly to code. For instance, consider sorting again: the spec says that the output is increasing and is a permutation of the input, but there is no obvious way to "execute" this spec directly and no obvious way for a compiler to automatically compile it to code. So just taking the spec as correct and executing it often isn't an option.

Nonetheless, the bottom line remains the same: formal methods are not a panacea. They simply transfer the (very hard) problem of confidence in code correctness to the (merely hard) problem of confidence in spec correctness. Bugs in the spec are a real risk, they are common, and they can't be overlooked. Indeed, the formal methods community sometimes separates the problem into two pieces: verification is about ensuring the code meets the spec; validation is about ensuring the spec is correct (meets our needs).

You might also enjoy Formal program verification in practice and Why aren't we researching more towards compile time guarantees? for more perspectives with some bearing on this.

Answer 2

D.W.'s answer is great, but I'd like to expand on one point. A specification is not just a reference against which the code is verified. One of the reasons to have a formal specification is to validate it by proving some fundamental properties. Of course, the specification cannot be completely validated — the validation would be as complex as the specification itself, so it would be an endless process. But validation allows us to get a stronger guarantee on some critical properties.

For example, suppose you're designing a car autopilot. This is a pretty complex thing involving a lot of parameters. Correctness properties of the autopilot include things like “the car will not crash against a wall” and “the car will drive where it's told to go”. A property like “the car will not crash against a wall” is really really important, so we'd like to prove that. Since the system operates in the physical world, you'll need to add some physical constraints; the actual property of the computational system will be something like “under these assumptions regarding materials science, and these assumptions regarding the perception of obstacles by the car's sensors, the car will not crash against a wall”. But even so, the result is a relatively simple property that is clearly desirable.

Could you prove this property from the code? Ultimately, that's what's going on, if you're following a fully formal approach¹. But the code has a lot of different parts; the brakes, the cameras, the engine, etc. are all controlled autonomously. A correctness property of the brakes would be something like “if the ‘apply brakes’ signal is on then the brakes are applied”. A correctness property of the engine would be “if the clutch signal is off then the engine isn't driving the wheels”. It takes a very high-level view to put them all together. A specification creates an intermediate layers where the different components of the system can be articulated together.

In fact, such a complex system as a car autopilot would have several levels of specifications with varying amounts of refinements. A refinement approach is often used in the design: start with some high-level properties like “the car will not crash against a wall”, then figure out that this requires sensors and brakes and work out some basic requirements for the sensors, the brakes and the pilot software, then refine again those basic requirements into a design of the component (for the sensor, I'm going to need a radar, a DSP, an image processing library, …), etc. In a formal development process, each level of specification is proven to meet the requirements set by the level above it, all the way from the highest-level properties down to the code.

It's impossible to be sure that the specification is correct. For example, if you got the physics wrong, the brakes might not be effective even though the math relating the brake code to the formal requirements is correct. It's no good to prove that the breaks are effective with 500kg of load if you actually have 5000kg. But it's easier to see that 500kg is wrong than to see inside the brakes code that they won't be good enough for the physical parameters of the car.

¹ _{The opposite of a fully formal approach is “I guess this works, but I can't be sure”. When you're betting your life on it, that doesn't seem so great.}