Things an "Expert" C Developer Knows: Undefined Behavior

This is the first post in my series: Things an "Expert" C Developer Knows


The 2016 SEI CERT C Coding Standard [free download] gives  this definition of Undefined Behavior:

Behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which the C Standard imposes no requirements.

An example of an action that causes undefined behavior is signed integer overflow. Professor John Regehr at U of Utah has a blog post that gives several possible examples of what C compilers *could* do with this C statement in a program:

printf ("%d\n", (INT_MAX+1) < 0);

The possibilities Regehr lists include printing 0, 1, 42 or formatting your hard drive. While the latter won't actually happen, undefined behavior can cause data corruption and/or security vulnerabilities.

The CERT C Coding Standard lists. in Appendix C, all undefined behaviors in the current C standard: C11 (more formally known as: ISO/IEC 9899:2011) There are 203.

Since I've mentioned the C11 standard here, I should probably have a Things an "Expert" C Developer Knows post about C standards.

Things an "Expert" C Developer Knows...

In January, my engagement with my current client will be taking a real change in direction. Rather than being a software craftsmanship coach, I will be joining a large team as an embedded C developer.

In that vein, I submitted a talk to my employer's internal conference in February about things that developers need to know if they are going to say they *really* know C.

In preparation for that, watch for a series of posts from me on: "Things an "Expert" C Developer Knows"

Always TDD? But Certainly Not for Startups, Right?

I just read "Uncle Bob is Smoking His Socks", in which the author, Joe Rounceville, starts by taking issue with Uncle Bob's statement that you are not a professional software developer if you don't TDD. I understand how that statement makes people bristle. Rounceville goes on to suggest that there are contexts (in particular lean startup) that don't merit TDD. He goes as far as to suggest that TDD in that context might be "gold plating."

Of course, startups need to be ultra-agile and able to get fast feedback on experiments by getting work out in front of customers rapidly. There's an element of racing against time when a company has to achieve success with only a certain amount of capital to burn through. Startups need to be able to pivot quickly in response to what they learn in their experiments.

But people who argue against "always TDD" seem to think that they save time up front by not using TDD. Among experienced TDD developers, I have not found that to be the case. And well test-driven code allows for rapid pivots because it's easy to understand and different concerns in the code are not tangled-up among each other.

Another risk for startups that decide not to TDD is that prototypes tend to be pushed into production as-is. Many young companies have met serious pains when their initial rushed-to-market release could not handle the load that came with the product's rapid success. And there's the old saw that if you don't have the time to write it correctly the first time, you are not going to have the time to re-write it later.

The relationship between TDD and startups is tricky. Learning TDD takes time. It's not a simple matter to start using TDD if you aren't good at it. Developing software with TDD when you are new to it is slower than developing without TDD. There is a non-trivial learning curve. As a developer, I would not want to try to get good at TDD while working for a startup. And I personally would not want to join a startup if I wasn't already strong in TDD. And because of the ever-changing nature of the market, if I have a startup idea and I want to get good at TDD before starting on it, I'm likely to miss my window of opportunity. And if I develop my startup without TDD, I'm likely to end up with a hard to change and hard to maintain project that's prone to defects. So while Uncle Bob's statement about "always TDD" may sound dogmatic, I find the statement that "startups don't need / can't afford TDD" to be false.

The Importance of Saying "I don't know and..."

As a technical coach, especially when I am embedded in a team, it is hard to overemphasize the importance of using the phrase "I don't know," plus "let me go find out." or "I'll ask someone who I think can help us."

Being able to say "I don't know" is an important skill for developers if they want to be on a high-performing team. Without it, the team risks creating an environment of fear and delayed learning. So when I say "I don't know ..." as a coach, I am modeling behavior that will help devs succeed. And adding the second phrase models taking initiative. While many, perhaps most, devs are good at finding answers to things they don't know, there can still be cultural and other reasons why a developer might not step out of their usual domain.

If you have coaching responsibilities, either formal or informal, and you don't say "I don't know..." maybe ask yourself if you are missing an opportunity to model important behavior.

Guest Post: Thoughts on Re-evaluating Spring as a Framework for Building Well Maintainable Applications

I'm hosting this post for Brandon Zeeb, another software craftsman at Pillar

Thoughts on Re-evaluating Spring as a Framework for Building Well Maintainable Applications

Spring Boot is good for the general populous.  Not sure I’d recommend it for any greenfield development.

Spring Boot wraps up a lot of auto configuration that most developers, in my experience, do not understand.  Spring Boot enables 3-6 annotations, 2-3 of which you cannot enable outside of Spring Boot.  Those 3-4 common annotations explode out into at least 10 more annotations.  In total, all those annotations create a large number of beans very few developers know exist.  Only if you had been using Spring since version 2.0 and had followed it's growth through MVC (and its ViewResolvers) into the RESTy side (with its MessageConverters) will you know what it's actually doing.  In short, very few developers I've found can actually be effective within Spring.  If they try to find the right place for a thing, they spend more time reading documentation to find the right extension point (what base class does my @Configuration need to extend so that I can put this bean in the ContentNegotiationManager!?!?) and if they work around the framework they pay later when Spring works against them

If you're writing small micro services and you finish them and forget them I don't see a problem.

Though these days there are many more frameworks which are slimmer and offer a more obvious and easily testable environment which makes, in my view, Spring less relevant. (edited)

In short, I'm looking at more ala cart solutions.  An example might be using Guice (for IoC) with Vert.x 3.  There is pretty minimal frameworking, whatever your app does, you see very clearly.  The examples online don't show what I believe to be the best use of Vert.x (and Guice), they put all the codes in one file and say "hey look how easy this is!".  The Java APIs of Vert.x are incredibly clean, all interfaces, easy as hell to mock, which ties in well to properly abstracting and dividing the application into components

The built-in Hazelcast solves a lot of problems in architecture N-tier apps have.


Brandon Zeeb:
Twitter: @phasebash
Github.com/phasebash

SonarLint for Eclipse Does Not Analyze Test Files by Default

I say unit tests are code like any other code. As such, they deserve to be refactored and kept clean.

I learned today that, by default, SonarLint for Eclipse disagrees. If you select Window->Preferences->Sonar Lint, you will see that "Test file regular expressions" contains "**/*Test.*,**/test/**/*":

For SonarLint to behave the way I want, which is to treat any file whose name ends in ".java" as code, I have to clear the regex.

Legacy Code: Characterize or Test Protect?

In the context of working with legacy code, I often hear the terms "Characterization Tests" and "Test Protecting". Here is how I think about each term:


Characterization Tests: a suite of tests that I build against legacy code that uses sufficient representative inputs to convince myself that I have characterized the behavior of the object I am testing. This suite also documents the behavior.

Test Protecting: building a suite of tests against legacy code that covers the object under test sufficiently that I am confident that the suite protects me from inadvertently changing the object's behavior during refactoring or when I add features.


One thing that is interesting to me is that the exact same suite of tests can both characterize and protect an object's behavior.

I was running a workshop today and introducing the concept of test protecting. Here are the steps I introduced today. If you are new to test protecting, they should be a good starting point. If you've done some test protecting, I hope you'll find some new nuance to your understanding. If you're already awesome at test protecting, maybe you'll want to teach from my steps, or comment with your insights about test protecting.

Steps I Teach to Test Protect Legacy Code:

Select a class to protect. 

Usually this is the class that I need to fix a bug in or add a feature to. Or maybe I have a few spare cycles and I want to add coverage to a frequently edited class. I do not select a rarely-modified class to test protect - that would be a waste of time (and thereby money, mine or my employer or client's)

Create a test class.

Legacy code, by one definition is code that doesn't have tests, so it follows that a legacy class won't have a test corresponding class. Create a test file with a name and location that follows your unit testing framework's conventions.

(Commit to your version control.)

Choose a public method to start with.

Often legacy classes are large and complex enough that it is difficult to test protect them well through only the public methods. I see several options (of varying risk) in that case:

1. Change private methods to public
2. Change private methods to package protected
3. Don't protect private methods that are difficult to test through public methods.
4. Refactor the class so that it is easier to test.
5. Use a testing tool that allows you to test non-public methods.
6. Protect as much as you can in a reasonable amount of time, assess risks posed by unprotected code, then decide on your next action.

The only answer that I am comfortable with is number 6.

Write a test that exercises the method you selected.

If you are at least somewhat familiar with the behavior of the method you selected, think about edge case inputs and happy and sad path inputs. Choose one to start with. Some devs like to start with edge cases, some like to start happy or sad path. I'm not sure it matters.

Give your test an absurd assertion against the result.

This step is as much like characterization as it is like test protecting. I want to be sure my test fails before it passes, so I choose a value that is unlikely to be correct. For instance, if the method I am exercising is:

int lookupDogAverageWeightLbs(string breed);

then I will choose a number like 5000:

assertEquals(5000, lookupDogAverageWeightLbs("Goldendoodle"));

Run the test so it fails.

In the previous example, your test should fail with a message something like:

Expected [5000] got [40].

If it doesn't, check your assumptions and look for mistakes in your exercising of the method or in how you wrote your assertion.

Fix the test so it passes.

When we test drive, we assert what the expected value should be, run it to see it fail, and then write the least amount of code needed to make the test pass. In test protection, the existing implementation is the standard behavior, so we fix our expectation to make the test pass. From absurd fail message above, you can easily see the actual result delivered by the implementation and fix the expectation with it so that the test passes:

assertEquals(40, lookupDogAverageWeightLbs("Goldendoodle"));

(Commit to your version control.)

Examine the test coverage of the method you are exercising.

If your coverage tool doesn't automatically run when you run tests (why doesn't it?), then run your coverage tool. Find which lines of your exercised method are not executed by your first test. If there are none, your method is test protected.

Write another test that hopefully increases coverage. 

Write a new test and give your exercised method new input data. Use inputs that, based on whatever understanding you have of the method so far, will exercise a non-covered line. Sometimes the name of the method will give you an idea, sometimes you will have gleaned some tentative understanding by scanning the implementation code while looking at coverage, and sometimes you still won't have a clue. Pick new inputs however you can and run your test.

Get the new test passing.

You might have needed to write a green test in order to increase the method's coverage. That's ok. In test driving, a test that doesn't go red before going green is worthless and time-wasting at best and possibly dangerous at worst. But in test protecting, such a test is only worthless if it does not add to the method's coverage. If your new test didn't increase the method's coverage, change your inputs until it does. This is another difference from when we test drive. When we test drive, we choose a set of inputs that fail and implement code to make it pass. We change the inputs only if the ones we chose did not make the test fail.

(Commit to your version control.)

Check the coverage of your method again.

If you have not covered all the lines in the method, jump back to "Write another test that hopefully increases coverage." If you have covered all the lines in the method, start over again with"Choose a public method to start with." If you've covered all the lines in the class, you have test protected the class. Congratulations!

Make sure new tests are run in CI.

If there are already other test classes in your project, your continuous integration server may already be configured to run tests in all test classes. If not, adjust your CI server's configuration to be sure that it runs your new tests. Kick off a build and inspect the build output to be sure your new tests ran.