To Type or Not to Type
Reading time: 4 minutes
Programmers have strong opinions on many things, one of which is the use of strong typing in programming languages. On the one hand are people who claim that strong typing makes developers' intentions clearer and catches errors before code is even run. On the other hand are those who say that strong typing makes code harder to modify, and that focusing on getting types correct distracts from focusing on getting the rest of the program correct.
What does the evidence say? In 2010, Stefan Hanenberg reported the results of an experiment in which 49 students were randomly divided into two groups and asked to build a scanner and parser using either a strongly-typed or a freely-typed version of a language called Purity that none of them had seen before. The main result was that—under the conditions of his experiment—the existence of a static type system did not have a positive impact on development time or quality. In fact, the subjects who used the dynamic type version of Purity were significantly faster in developing a scanner, but there was no statistically significant difference with respect to quality of the final product. (See this page for more details and discussion.)
But that's far from being the end of the story. A 2014 study by
Hanenberg and others looked at more than just "time to write first version":
This paper describes an experiment that tests whether static type
systems improve the maintainability of software systems, in terms of
understanding undocumented code, fixing type errors, and fixing
semantic errors. The results show rigorous empirical evidence that
static types are indeed beneficial to these activities, except when
fixing semantic errors. We further conduct an exploratory analysis
of the data in order to understand possible reasons for the effect
of type systems on the three kinds of tasks used in this
experiment. From the exploratory analysis, we conclude that
developers using a dynamic type system tend to look at different
files more frequently when doing programming tasks—which is a
potential reason for the observed differences in time.
Later work has strengthened the case for strong typing. In 2015, Hanenberg and Fischer published another study whose abstract is worth reproducing in full (not least because the paper itself is hidden behind the Great Paywall of Academia):
Recent empirical studies that compared static and dynamic type
systems on API usability showed a positive impact of static type
systems on developer productivity in most cases. Nevertheless, it is
unclear how large this effect is in comparison to other factors. One
obvious factor in programming is tooling: It is commonly accepted
that modern IDEs have a large positive impact on developers,
although it is not clear which parts of modern IDEs are responsible
for that. One possible—and for most developers obvious
candidate—is code completion. This paper describes a 2×2
typed alternative TypeScript with and without code completion in MS
Visual Studio. While the experiment shows (in correspondence to
previous experiments) a large positive effect of the statically
typed language TypeScript, the code completion effect is not only
marginal, but also just approaching statistical significance. This
seems to be an indicator that the effect of static type systems is
larger than often assumed, at least in comparison to code
There is evidence for the benefits of strong typing from other sources too. The most recent addition to the debate is from Ray et al, who looked at 729 GitHub projects with 29,000 authors and 80 million lines of code in 17 languages. They found that, "...strong typing is modestly better than weak typing, and among functional languages, static typing is also somewhat better than dynamic typing." However, they caution that, "...the modest effects arising from language design are overwhelmingly dominated by the process factors such as project size, team size, and commit size. However, we hasten to caution the reader that even these modest effects might quite possibly be due to other, intangible process factors, e.g., the preference of certain personality types for functional, static and strongly typed languages."
Microsoft's Dan Luu wrote a good summary of this research back in 2014. The most important thing for programmers to take away isn't the particular results, but the fact that we can and should answer questions like these the same way we answer questions about public health: by gathering and examining data. Some of that may come from controlled experiments like Hanenberg et al's, and some may be "found data" from sources like GitHub, but no matter what the source is, careful analysis will get us closer to a useful truth than throwing anecdotes at one another.
Here at Rangle, we take pride in our evidence-based development process. We keep track of test code coverage, the rate at which we're burning through user stories, and how often the customer says "not done" when we think something is, because we believe that basing decisions on facts helps us build better software faster. The modest-but-real benefits found by the studies cited above aren't enough to make us switch all of our work to strongly-typed languages, any more than the health benefits of being vegetarian are going to make us say "no" to the steak frites at Jules Bistro, but evidence like this does shape our own decisions and what we recommend to clients.