Where Did My Error Code Go?

By: 
Ryan Lavering

October 01, 2013

The other day, I helped a customer debug an interesting problem with one of his test cases. No matter what he did to set up input values, the code stubbornly refused to traverse one of the error checking cases in his function. The example code below is similar to what we were trying to test:

#define PERIPHERAL_A (0x8345678)
#define SENSOR_DATA (0x456789AB)

void sysOutput(const char *line);
void processSensorData(long data);

/* Returns next value held in the given I/O stream
* params:  key: I/O stream id; value: [output] returns next I/O value
* return:  length of value, in bytes;  -1 on failure   */
int getNextValue(int key, long *value);

/* Output all pending values for the given I/O stream
to the system I/O
* params:  key: I/O stream id  */
void commandReceived(int key)
{
 long periph_a_cmd;
 long sensor_data;
 unsigned short status;

 status = getNextValue(PERIPHERAL_A, &periph_a_cmd);
  if (status != -1)
 {
  sysOutput("Peripheral A: Input received\n");
  status = getNextValue(SENSOR_DATA,&sensor_data);
  if (status == -1)
  {
   /* VECTORCAST NOTE:
   * Can you spot the problem?
   */
   sysOutput("Error: No sensor data available!\n");
   return;
  }
  processSensorData(sensor_data);
 }
 else
 {
  sysOutput("No command found\n");
 }
}

Writing a test case for the inner if statement (the TRUE, TRUE case) was pretty easy. We simply specified a list of return values for the getNextValue() function of "0,-1". The first time the stub is hit it will return 0, then -1 for the second and any subsequent calls. We also set up two expected string values for the sysOutput() stub to look for the appropriate log messages. That should hit the "Error: No sensor data available!" case, right? That's what we thought.

When we ran the test case, however, our expected value for the second sysOutput() call was never used, so VectorCAST reported an Unused Expected Value error. Looking at the code coverage for the test case, it was easy to see why our expected value wasn't hit: The error condition never executed! Instead, we got coverage for the processSensorData() call, which should not have been hit at all.

After verifying that the test case logic was correct, and that the return values from getNextValue() were correct in the Test Case Execution Report, we were left scratching our heads. The code just wasn't doing what it seemed to say it was.

Having failed to identify the problem from inspection, we decided to investigate the behavior in the debugger. Fortunately VectorCAST made this simple -- you can debug any test case just by right clicking and selecting Execute with Debug => Without Coverage.

We already knew that the behavior we wanted to inspect was part of the function under test, so we added a breakpoint to the commandReceived() function (GDB: "b commandReceived") and ran to hit the breakpoint. Once the breakpoint hit, we stepped once to get past the SBF (stub by function) block that VectorCAST inserted, and were in the commandReceived() code.

A few steps and inspected values later we saw the problem: after the second call to getNextValue() 'status' was 65535, not -1! The 'status' variable is declared as an unsigned short. We were looking at an integer promotion bug. getNextValue() is returning the -1 value correctly, but when it is stored into 'status' it is converted to an unsigned short value of 65535. Then, when the code checks (status == -1) in the second if statement, 'status' must be promoted to a 32-bit value to match the 32-bit constant -1, and since it is unsigned the conversion results in a value of 65535, not -1 as the developer intended.

So, long story short, as written there is no way to get to the error condition code we were looking for. Testing with VectorCAST not only located a bug in the source code, but made it easy for the tester to determine what happened, debug the problem, and deliver a comprehensive bug report to the development team. As a bonus, since we know the test case is correct, once the code is fixed this test case should start passing automatically without any additional work on the part of the tester.