Book Review: Test-Driven Development with Mockito

Test-Driven Development with Mockito provides a general introduction to test-driven development (TDD) before looking at application of Mockito to implement test doubles as part of test-driven development. Test-Driven Development with Mockito is written by Sujoy Acharya, is published by Packt Publishing with a publication date of November 2013, and has about 150 substantive pages of substantive content divided into 9 chapters and 2 appendices. The subtitle of Test-Driven Development with Mockito is "Learn how to apply Test-Driven Development and the Mockito framework in real life projects, using realistic, hands-on examples."

Preface

Test-Driven Development with Mockito's Preface opens with a very brief description of test-driven development and how Mockito plays a role in test-driven development. The preface summarizes each chapter and appendix before recommending what a reader of that book should use while reading the book: J2SE 5, the Eclipse IDE (Kepler 4.3 recommended), and Mockito.

One of the most important parts of a Packt Publishing Preface is the section on "Who this book is for." In this case, the paragraph in this section states:

This book is for developers who want to develop software according to Test Driven Development using Mockito and to leverage various Mockito features. Developers don't need prior knowledge of TDD, Mockito, or JUnit. It is ideal for developers who have some experience in Java application development as well as some basic knowledge of unit testing, but it covers the basic fundamentals of TDD and JUnit testing to get you acquainted with these concepts before you use them.

Chapter 1: Getting Familiar with TDD

As the Preface stated, Test-Driven Development with Mockito does cover "basic fundamentals of TDD" and begins this in the first chapter. This chapter starts out at a very basic level, defining a test in Java development and listing some "available code-driven unit-testing frameworks for Java." In this chapter, we learn that JUnit 4.0 will be used in examples in the book. An example is then presented using Eclipse and JUnit in which the test class is written before the production class. This transitions into more details on the historical background of TDD and lists benefits of TDD.

Chapter 1 continues by looking into how TDD fits into overall software development and how refactoring plays a role in test-driven development. In many ways this first chapter is what you'd expect from a book on test-driven development. The code-based examples of test-driven development do a nice job of illustrating basic techniques and advantages associated with test-driven development. However, there seemed to be some things that seemed a bit out of place in this chapter. In particular, the chapter's extremely brief coverage of "Object-function programming languages such as Scala and Groovy" seemed irrelevant to the rest of the chapter and added little value. As with most books on TDD, this chapter of this book had a paragraph that seemed a bit preachy.

Chapter 2: Refactoring – Roll the Dice

The second chapter of Test-Driven Development with Mockito, as its name implies, focuses on refactoring. The chapter begins by reviewing benefits of refactoring and then discusses "trigger points for refactoring" and when not to refactor. Some relatively lengthy code listings are provided to show some code in desperate need of refactoring with text explaining issues with the code and possible ways to refactor that code. This section of the book also introduces the Eclipse plugin CodePro Analytix.

The second chapter moves onto coverage of code smells. Duplicate code with switch statements is shown as a code smell and there is discussion and more code listings on how to refactor this using the strategy pattern and polymorphic behavior. Other code smells that are covered in this chapter along with potential remedies include duplicate code, comments, long methods, long method parameter lists, large classes, dead code, and overengineering.

This second chapter is a reasonable introduction to code smells and how to use refactoring and other techniques to deal with them. This chapter mentions common object-oriented and software development principles such as YAGNI, DRY, open-closed principle, and strategy pattern. A single chapter of a book cannot compete with an entire seminal treatment of a subject, but this chapter does provide a good illustration of how refactoring fits into TDD.

Chapter 3: Applying TDD

The third chapter of Test-Driven Development with Mockito begins with a bold statement, "Test-Driven Development (TDD) is the new way of programming." The chapter then outlines the simple steps that are taken to implement TDD and is the first chapter in the book to really dive into TDD. The chapter concludes with a lengthy example including code listings and Eclipse screen snapshots to demonstrate JUnit and TDD in practice.

Chapter 4: Understanding the Difference between Inside-out and Outside-in

Chapter 4 of Test-Driven Development with Mockito compares and contrasts TDD's two "common" styles: Outside-In and Inside-Out. The chapter provides details regarding the outside-in approach before listing advantages and disadvantages of that outside-in approach. The same is then done for inside-out, though the coverage is much shorter than for outside-in.

Chapter 5: Test Doubles

Test-Driven Development with Mockito's fifth chapter covers test doubles. The chapter very briefly discusses why test doubles are useful (I don't think this part is particularly strong) before categorizing the types of test doubles (dummy, stub, mock, and fake). The coverage of these types of test doubles is much better than the general introduction to test doubles and provides nice code examples and discussion on each and how they are different.

Chapter 6: Mockito Magic

Chapter 6 of Test-Driven Development with Mockito is the book's first significant look at Mockito (although Mockito was mentioned in Chapter 5 for the example of the mock category of test double). The chapter begins with an overview of Mockito and explains Mockito's primary roles in test-driven development: "Mockito plays a key role to mock out external dependencies" and helps achieve the desired "qualities of unit testing."

Chapter 6 continues by demonstrating adding Mockito to Eclipse. There are numerous screen snapshots of Mockito being used within Eclipse (these are in color in the electronic edition of the book). The chapter introduces key annotations and functionality provided by Mockito and demonstrates using Mockito to address things commonly tested using mocks. A reader of this book who's primary interest in the book is learning Mockito would find this to be one of his or her favorite chapters of the book.

Chapter 7: Leveraging the Mockito Framework in TDD

Test-Driven Development with Mockito's seventh chapter begins with a brief discussion of classical TDD versus mockist TDD. The chapter is then introduced with the statement that it will "use Mockist TDD style to mock external dependencies and explore Test-Driven Development." The entire chapter is then devoted to a fairly realistic example of building an "online portal for the aspiring game developers" with test-driven development techniques and using Mockito to mock out the external interfaces.

Chapter 8: World of Patterns

Chapter 8 of Test-Driven Development with Mockito opens with reference to Robert Martin's "three important characteristics of a bad design": rigidity, fragility, and immobility. The chapter then covers "higher level design guidelines" of modularity, high cohesion, and low coupling and "lower level design principles" of Open/Closed Principle, Dependency Inversion Principle, Integration Segregation Principle, Single Responsibility Principle, Liskov Substitution Principle, and the Law of Demeter.

The next section of Chapter 8 discusses design patterns. The section provides an overview of what design patterns are, how they are categorized, and provides a couple of examples of applying design patterns.

Chapter 9: TDD, Legacy Code, and Mockito

The ninth chapter of Test-Driven Development with Mockito discusses what constitutes legacy code and then emphasizes: "Any code with no unit test is legacy code." Chapter 9 looks at some common less-than-testable features of legacy code and how to refactor that code for greater testability. The chapter then moves onto how Mockito can make it easier to refactor code to be more testable.

Appendix A: TDD Tools and Frameworks

Appendix A covers Eclipse IDE and JUnit 4 unit testing framework. It discusses using keyboard shortcuts in Eclipse (including "The mother of all shortcuts is Ctrl + Shift + L" to see all shortcuts) and emphasizes those shortcuts useful in refactoring and TDD. There are some nice color screen snapshots showing shortcuts applied in this section. Appendix A's coverage of JUnit begins with the statement, "Inheritance in Java is not a smart thing to implement" to explain advantages of JUnit 4.x (which supports annotations) over earlier versions which required inheritance. Appendix A's coverage of JUnit covers fundamentals of JUnit and demonstrates using Eclipse shortcuts in JUnit unit test development.

Appendix B: Agile Practices

Appendix B discusses continuous integration (CI) in general before covering Jenkins as a particular continuous integration tool. The author begins continuous integration coverage with the statement, "In a good software development team, we'd find TDD as well as CI." There are numerous color screen snapshots depicting configuration of Jenkins. Appendix B also discusses the "Agile development methodologies Scrum and Kanban." After briefly discussing Agile in general and providing a reference to the Agile Manifesto, Appendix B focuses on Scrum and Kanban specifically.

General Observations

• Test-Driven Development with Mockito attempts to cover numerous topics within its roughly 150 pages.
  • The title nicely captures two of the most significant topics of the book: Test-Driven Development and Mockito. It is important to note that this book is on both subjects, among others, and is not exclusively a book on test-driven development or exclusively a book on Mockito.
  • Topics related (in some cases less tightly than others) to test-driven development that are covered in this book include refactoring, object-oriented design practices and principles, and agile methodologies.
  • The advantage of this coverage of a wide set of somewhat related topics is that a developer new to test-driven development can get an introduction to a wide variety of related topics. The disadvantage of this approach is that there is less depth on any one subject that books more tightly focused might have. To this point, note how long each of the books listed below is in terms of the single topic each book covers. Each of these books is lengthier on a single topic than Test-Driven Development with Mockito is for all of the topics. Test-Driven Development with Mockito allows a reader to get a taste of each of these topics and how they fit together in a single book, but thorough coverage of the topics would need to be gained by moving onto these specifically focused books.
    • Martin Fowler's Refactoring: Improving the Design of Existing Code has over 450 pages devoted to refactoring.
    • Kent Beck's Test-Driven Development: By Example has well over 200 pages.
    • Design Patterns: Elements of Reusable Object-Oriented Software has over 400 pages.
    • Robert Martin's Clean Code: A Handbook of Agile Software Craftsmanship is over 450 pages in length.
    • Tomasz Kaczanowski's Practical Unit Testing with JUnit and Mockito has over 400 pages.
    • Lasse Koskela's Effective Unit Testing: A Guide for Java Developers has almost 250 pages.
• Test-Driven Development with Mockito has several nice images. I especially like the screen snapshots that depict Eclise refactoring keyboard shortcuts and the screen snapshots that depict Jenkins configuration. An advantage of my reviewing the PDF provided by Packt Publishing is that these images are in full color.
• The code listings are useful in illustrating the points being made, but the lack of color syntax (even in the electronic PDF version) make it more difficult to read the code quickly and easily. I have become spoiled by my IDE (and even my vim!) and online code listings featuring color syntax. Reading code without color syntax reminds me of the "old days," but the nostalgia is not worth the extra effort required to read the code.
• Test-Driven Development with Mockito has some typos in it and also has some awkward and incomplete sentences. However, none of these were significant enough to prevent me from picking up the concept being covered.
• Although the flow of topics covered in Test-Driven Development with Mockito generally made sense, there were some aspects of the arrangement of the various concepts that were a bit awkward. In particular, there were a couple concepts that were presented in almost a "side note" manner at first and were not very clearly explained, but then were much more clearly explained in the later sections that focused on that topic.
• As is the case with just about every TDD-related book I have read, Test-Driven Development with Mockito is a highly opinionated book. Although I generally agreed with most of the opinions, I felt some of the stated opinions were backed up better than others and some I don't feel quite as strongly about or think are quite as certain as the author does. That being stated, I do like to see opinions expressed and I generally agreed with them in principle. The more focused books I mentioned in an earlier bullet would be useful for the reader of Test-Driven Development with Mockito who wanted to better understand the "why" behind some of the opinions expressed in this book.

Conclusion and Recommendation

Test-Driven Development with Mockito will be most beneficial to readers who have little to no familiarity with the numerous concepts introduced in the book (test-driven development, refactoring, object design, JUnit, Mockito, etc.). The book provides an overview of each of these and how they are used together. Developers who have read other books on these subjects may not find the introductory information in Test-Driven Development with Mockito as useful or as new to them. Developers wanting in-depth coverage of a topic might prefer a book focused on that particular topic. The book is most useful for those using the Eclipse IDE, but many of the sections are useful for developers who don't use Eclipse.

Better JUnit-based Unit Tests with NetBeans 7.4 beta Hints

In my last post, I wrote about hints provided in NetBeans 7.4 beta that improve a developer's ability to avoid nasty runtime issues with Java exception handling. In this post, I look at how two more hints provided by NetBeans 7.4 beta can be used to make unit tests more correct and more clear during unit test execution. These are the "Inconvertible parameters of Assert.assertEquals" and "Incorrect order of parameters of Assert.assertEquals" hints.

As far as I can tell from anecdotal evidence and from talking to other Java developers, JUnit remains the most widely used unit testing framework in the Java environment. Most of these JUnit users are intimately familiar with JUnit's Assert class and its many overloaded assertEquals methods. NetBeans 7.4 beta now provides two hints to make it easier to use these assertEquals methods appropriately.

Although many of the Assert.assertEquals() methods have very specific data types for the "expected" and "actual" parameters to be asserted as equal, there is a version that accepts two Objects and this means two parameters of different types that cannot possibly be considered "equal" can still be passed to that method. There is no way for the compiler to prevent that, but NetBeans 7.4 beta includes the "Inconvertible parameters of Assert.assertEquals" hint to address that particular case. Without such a hint, one is more likely to not realize the error of his or her ways until he or she runs the JUnit test and sees the failed assertion.

One of the most common issues I've run into with JUnit (and one of the reasons I love Hamcrest's fluent API) is that I just cannot seem to remember with confidence which order the parameters to the assertEquals methods are. I have a 50/50 chance of being correct by guessing. Modern Java IDEs such as NetBeans help greatly when writing new JUnit code because their method completion features will indicate that the "expected" parameter is specified first and the "actual" parameter is specified second. More commonly, this is a problem when reading code rather than when writing it because there is no method completion helping me read the code. NetBeans 7.4 beta addresses this by highlighting the situation in which I have mixed up the parameters' order via the "Incorrect order of parameters of Assert.assertEquals" hint. With that hint enabled (which it is by default), I can quickly recognize out-of-order parameters prior to runtime and even without method completion.

Both of the hints discussed above can be demonstrated in a very simple unit test class.

Portion of CalculatorTest.java

/**
 * Test Calculator.sum(int ...).
 */
@Test
public void TestSumIntegers()
{
   final Calculator calculator = new Calculator();
   Assert.assertEquals(calculator.add(1, 2, 3, 4, 5), 15);
   Assert.assertEquals("15", calculator.add(1, 2, 3, 4, 5));
}

The code the above unit test method is testing is not important for this discussion. Rather, the focus is on the use of Assert.assertEquals in two cases. Both cases as shown above are incorrect and force demonstration of the two previously discussed NetBeans hints. The first attempt at asserting two objects are equal places the parameters in the wrong order. The "expected" value (hard-coded 15) should be listed first followed by the "actual" value calculated by the method under test. The second attempt at asserting two objects are equal will always fail because the types don't match: the first parameter is a String and the second parameter is an integer. In both of these cases, the unit test code compiles without complaint. However, both assertions will ALWAYS fail when running the unit tests. In fact, these tests results may inadvertently be interpreted as problems with the code being tested until someone looks at the test failures in greater depth.

The next two screen snapshots demonstrate NetBeans 7.4 beta flagging both problematic unit testing assertion statements.

There is one caveat to note regarding the "Incorrect order of parameters of Assert.assertEquals" hint. It works well when the assertion statement is like the one shown in my example: a hard-coded expected value is provided as the "actual" value along with an obviously calculated value as the "expected" value. The next screen snapshot illustrates this point; only the statement I earlier showed is flagged by the hint and other ways of comparing actual to expected are not flagged, even when the order is incorrect.

The last shown screen snapshot demonstrates that the NetBeans hint is only able to detect an incorrect order of assertEquals parameters (should be expected before actual rather than actual before expected) in the case where those values are directly accessed in the statement (the actual calculation is performed for the first [expected] parameter and the expected hard-coded value is provided for the second [actual] parameter).

The two hints covered in this blog post make it easier to detect problems with the frequently used JUnit Assert.assertEquals methods that might not be detected until analysis of unit test run results without the hints. Although the two issues these hints warn developers about are typically fairly easy to detect and fix, detecting and fixing these problems is still more difficult and time-consuming than having the NetBeans IDE tell you they are wrong before you even run the test.

Book Review: TestNG Beginner's Guide

I was happy to accept Packt Publishing's invitation to review the recently released (July 2013) book TestNG Beginner's Guide because I welcomed the opportunity to learn more about TestNG. I have read about and played a little bit with TestNG before, but the vast majority of my Java unit testing experience has been with JUnit.

Author Varun Menon opens TestNG Beginner's Guide with the following statement in the Preface, "Currently, TestNG is the most widely used testing framework in the software industry." This is definitely not what I've found in my experience where JUnit has been used far more than TestNG in the different Java projects that I've worked on. The Preface lists four things needed for getting most use out of TestNG Beginner's Guide: Java JDK, Eclipse, Linux or Windows, "basic knowledge of Java and testing."

Chapter 1: Getting Started

The first chapter of TestNG Beginner's Guide opens with a brief introduction to testing and automated testing. The chapter then introduces TestNG and provides a brief history and overview of the features of that open source unit testing framework. The chapter describes how to acquire TestNG via download (testng-6.8.jar in the book) and explains five ways it can be run: command line, Eclipse plugin, IntelliJ IDEA plugin, Ant, and Maven. The chapter spends multiple pages and several screen snapshots demonstrating configuration of TestNG within Eclipse via the plugin update site http://beust.com/eclipse. It demonstrates creating an Eclipse project, associating the TestNG library with the Eclipse project, writing a TestNG-based test class, and executing the TestNG test class in Eclipse. There are numerous screen snapshots in this Equinox-heavy portion of the chapter. Similar instructions on using TestNG with Eclipse can be found in the TestNG Eclipse documentation. The TestNG documentation includes a section on running TestNG from the command line.

Chapter 2: Understanding testng.xml

The second chapter of TestNG Beginner's Guide focuses on the testng.xml file, which it describes as "a configuration file for TestNG" that is "used to define test suites and tests in TestNG" and is "used to pass parameters to test methods." The chapter focuses on using testng.xml to specify test suites. It covers use of TestNG within Eclipse and from the command line. This chapter contains some of the concepts central to using TestNG. The examples shown are Eclipse-oriented, but XML snippets are shown as well, which is useful for those not using Eclipse. As testng.xml is a core part of TestNG, it is not surprising that Chapter 2 is a core part of TestNG Beginner's Guide.

Chapter 3: Annotations

The third chapter of TestNG Beginner's Guide, like the second chapter, focuses on a core part of TestNG. In this case, it's on TestNG's annotations. The chapter introduces Java annotations and standard Java annotations (@Override, @Deprecated, and @SuppressWarnings) before covering TestNG's specific annotations in a table with annotation name and brief description.

Chapter 3 of TestNG Beginner's Guide examines TestNG's annotations in more detail after the initial listing of them in the table with annotations names and descriptions. This more detailed examination includes code uses of these annotations. Annotations covered in this chapter include the @Test annotation (including specification of timeouts and expected exceptions), several of the wide variety of "before" and "after" TestNG annotations (such as @BeforeClass and @AfterMethod), @Optional, @Parameters, and @DataProvider. This, along with Chapter 2, is another "core" chapter of the book because annotations are core to TestNG. The third chapter also includes additional references to testng.xml.

Chapter 4: Groups

The fourth chapter of TestNG Beginner's Guide covers TestNG Groups. Coverage of test groups covers grouping of tests, running those groups of tests, specifying tests as parts of more than one group, using regular expressions to specify which groups to run, and grouping test groups within groups (metagroups). As with many of the other chapters, this chapter includes numerous screen snapshots showing Eclipse used with the different concepts, but code snippets are also included.

Chapter 5: Dependencies

TestNG Beginner's Guide's fifth chapter is on dependencies, a TestNG mechanism that allows tests to be executed in a particular order via expression of dependencies between tests and other tests or groups of tests. Screen snapshots of Eclipse and code listings are used to illustrate creating a dependency between one test on another, between one test on multiple other tests, between one test and a test in another class in the same inheritance hierarchy, and between a test and a group. The chapter also covers use of regular expressions with test dependencies and using XML to specify test dependencies.

Chapter 6: The Factory Annotation

The sixth chapter of TestNG Beginner's Guide introduces TestNG factories. The chapter explains the use of factories to "define and create tests dynamically at runtime." Through more screen snapshots of Eclipse and mode code listings, the chapter introduces the @Factory annotation and shows how it is used. This chapter also introduces the @DataProvider annotation and demonstrates using these two annotations together. The chapter also includes explanation of how @Factory and @DataProvider differ and when each is most appropriate.

Chapter 7: Parallelism

The seventh chapter of TestNG Beginner's Guide covers parallelism, or running tests with multiple threads. The chapter includes a brief section describing two reasons why being able to run tests in parallel can be advantageous.

Chapter 8: Using Build Tools

The eighth chapter of TestNG Beginner's Guide begins by outlining the "advantages of automating a build" and highlights Ant, Maven, and Gradle as the "major build tools that are being used by the software industry when it comes to Java or Java related languages." The rest of the chapter focuses on incorporating TestNG with Ant and with Maven using TestNG's custom Ant task and Maven plug-ins.

Chapter 9: Logging and Reports

Chapter 9 of TestNG Beginner's Guide introduces TestNG support for reporting and logging with an introductory paragraph on how reporting and logging are generally beneficial. The chapter introduces the interfaces ITestListener and IReporter and very briefly differentiates between these and where each is more appropriate. Like several other chapters, the ninth chapter mixes screen snapshots of Eclipse with code listings to demonstrate how to write a custom logger implementing the ITestListener interface, how to write a custom reporter implementing the IReporter interface, using TestNG-provided XML and HTML reports, and generating a JUnit HTML report with Ant. This chapter also introduces ReportNG, which is described on its website as "a simple HTML reporting plug-in for the TestNG unit-testing framework" that "is intended as a replacement for the default TestNG HTML report." This chapter also introduces Reporty-ng, formerly known as TestNG-XSLT.

Chapter 10: Creating a Test Suite through Code

The tenth chapter of TestNG Beginner's Guide covers "how to configure and run [a] test suite through code" to help "in configuring and running tests at runtime." In short, this chapter covers how to run TestNG tests programmatically rather than via specification in a testng.xml file. The chapter introduces and describes "the API classes" XmlSuite, XmlTest, XmlClass, XmlPackage, XmlDependencies, and a few more. The chapter adapts examples that were used earlier in the book to demonstrate using testng.xml for configuration but instead specify configuration in the code itself.

Chapter 11: Migrating from JUnit

The eleventh chapter of TestNG Beginner's Guide was one that I most looked forward to as I scanned the table of contents of the book. I find documentation on migration of one product (or version) to another product (or version) useful for two reasons. First, if I know the product or version being migrated from, I can better understand how the product or version being migrated to implements the same functionality. It helps me to learn the new product or version. Second, if I decide to adopt the new product or version, the migration guide is helpful in transitioning to that new product or version.

The section of the TestNG documentation called Migrating from JUnit states, "TestNG can automatically recognize and run JUnit tests, so you can use TestNG as a runner for all your existing tests and write new tests using TestNG." TestNG Beginner's Guide begins Chapter 11 with coverage of executing JUnit tests in TestNG. The chapter continues with a demonstration of running JUnit and TestNG tests together via testng.xml specification and then demonstrates "running JUnit tests along with TestNG [tests] through Ant."

Chapter 11 also covers true migration of JUnit-based unit tests to TestNG-based unit tests (as opposed to simply running JUnit-based tests in TestNG). This section includes an extremely handy table that compares the feature of TestNG to use compared to the feature of JUnit 3 and JUnit 4 to fulfill the same use case. This table is one of my favorite features of TestNG Beginner's Guide. The table validated my own internal mapping that I had been creating in my mind of TestNG concepts to JUnit concepts. The chapter also discusses how assert statements are handled differently in JUnit versus TestNG and this reminded me of why I prefer Hamcrest-based assertions with their high readability over straight JUnit assertions in which I must remember which of the two is an expected value and which is the actual value.

Chapter 12: Unit and Functional Testing

The final chapter of TestNG Beginner's Guide focuses on using TestNG for unit and functional testing. The chapter provides basic definitions of unit testing and functional testing before examining each approach in more detail via examples.

The unit testing section of Chapter 12 provides a table of assertions supported by TestNG and demonstrates use of an assertion in a code sample. That section also focuses on mocking with TestNG. Although this chapter specifically mentions jMock, Mockito, EasyMock, PowerMock, and JMockit as a "few of the mocking utilities," the chapter's focus is on jMock and Mockito.

After contrasting functional testing against unit testing, Chapter 12 includes a "list of functional testing automation tools that are available in the market" including Selenium, Sikuli, and SilkTest. Selenium is the product in the functional test coverage that gets the same focus that jMock and Mockito got in the unit test coverage.

The final paragraph of the Summary of Chapter 12 is also the summary paragraph for the entire book.

Overall Positives

• For Beginners - As its title TestNG For Beginners correctly advertises, this book is aimed at beginners. I found my experience with JUnit to make it very easy to understand the concepts covered in this book, but I think most Java developers, even with less JUnit experience, would find this book highly approachable.
• Screen snapshots and Images - TestNG Beginner's Guide features numerous screen snapshots that can be helpful in learning how to use TestNG, particularly in conjunction with the Eclipse plugin. The one downside is that many of the images are Eclipse-specific. This is a positive for Eclipse users, but may be a bit of a negative for users of other IDEs.
• Code Listings - TestNG Beginner's Guide includes numerous code examples and listings. Because they are listed for nearly every covered topic, users of IDEs other than Eclipse should still be able to use this book.

Overall Negatives

• Editing and Polish - One of the weaknesses of several (not all) of the Packt titles I have reviewed has been occasional convoluted sentence structure and weird grammar and incorrect spelling. This particular book has some strange grammar in a couple places as well as several spelling errors. Examples of spelling errors include @SupressWarnings (only one "p") and "TestNg" rather than "TestNG" on the same page 52. The good news is that these are mostly just minor distractions or annoyances and rarely take away from understanding of the meaning.
• For Beginners - This is an advantage for new Java developers and developers new to unit testing, but is a bit of a disadvantage for more experienced developers (especially with JUnit). The book does a sound job of covering the most important features of TestNG, but there is very little coverage of why these features are useful. The simplicity and clarity of the examples comes at the price of the examples not providing very realistic ideas of how these TestNG features might be used in real code.
• Eclipse-centric - The book provides enough code listings to largely overcome this limitation and this is not a limitation, of course, for Eclipse users.

Other Observations

The title says it all: TestNG Beginner's Guide is for those new to TestNG.

As already mentioned this book is targeted most closely at Eclipse users. That stated, I think developers with at least some Java experience will find it easy to pick up the concepts and even implement the code listings in their favorite IDE even when that IDE is not Eclipse. Complete code examples are typically provided and most of the many Eclipse screen snapshots are showing reported results which will be shown on the command line or in other IDEs.

The ordering of the chapters is not the same order I'd have chosen, but this is not a big deal. For the most part, one could read the chapters in the order one prefers or even skip chapters on subjects that are not of interest or which the reader is already familiar. The chapters rely slightly on each other, but for the most part could be read fairly independently of one another. There are references to earlier parts of the book in several chapters, but these references are often not necessary to understand the newly covered topic.

Most of the material covered in TestNG Beginner's Guide is covered online in TestNG documentation and in blog post. I have linked to specific sections or pages of these resources several times in my review of this book. Cédric Beust's blog is another obvious source of information on TestNG. The TestNG site includes the More TestNG Reading page with links to numerous additional online TestNG resources. Although there is little in TestNG Beginner's Guide that is not available online, this book provides organization for the introductory material and can provide context that is especially helpful when accessing seemingly unrelated blog postings.

TestNG Beginner's Guide is close to 250 pages of content plus more pages for table of contents, index, and so forth. Much of the space on these pages consists of code listings and screen snapshots, so the overall prose content is well below 250 pages. The examples are very simple, making it easy to focus on the TestNG feature being demonstrated, but also meaning that more subtle understanding of when certain features would be useful might be missed, especially by those not familiar with unit testing.

Conclusion

TestNG Beginner's Guide is squarely aimed at developers new to TestNG. Its twelve chapters cover the basics of TestNG and are approachable for anyone with some basic Java ability, but are especially easy to understand with some Java and JUnit experience. The book is probably too introductory for advanced TestNG users, but its title suggests that this is intentional. The only caveat I'd offer to a person new to TestNG who is trying to decide whether to obtain this book is to consider whether the book helps fill a need that is not covered adequately by the broad TestNG documentation or blog posts such as those by mkyong. I personally find some advantages to learning from books in conjunction with online resources, but I understand that this is a matter of taste.

Book Review: Effective Unit Testing: A Guide for Java Developers

In the Preface for Effective Unit Testing: A Guide for Java Developers, author Lasse Koskela states that although the impetus for Effective Unit Testing was to "write a Java edition of Roy Osherove's book, The Art of Testing with Examples in .NET," Effective Unit Testing ended up "having very little in common with Roy's book." Koskela further explains in the Preface that "this book is for the Java programmer," but adds that "writing good tests is a language-agnostic problem" and he recommends his book even for developers using languages other than Java.

Koskela does a nice job in the "Preface" of succinctly summarizing what the book is and isn't when he says, "I didn't want to give you a tutorial on JUnit or my favorite mock object library" and "I've tried to minimize the amount of technology-specific advice." This is important to note because Effective Unit Testing is not the book you'll want if you're looking for a book covering intricate details of JUnit, TestNG, Mockito, EasyMock, Hamcrest or other commonly used Java unit testing frameworks and tools. Instead, the focus of Effective Unit Testing: A Guide for Java Developers is on more general concepts of unit testing in general (and in Java in particular) that might be implemented with any variety of different tools. That being stated, Koskela doesn't ignore these tools completely and does sprinkle JUnit code throughout the book along with code samples based on other unit testing tools such as JMock, Mockito, and Hamcrest.

Chapter 1: The Promise of Good Tests

In Chapter 1 of Effective Unit Testing, Koskela mixes brief historical testing anecdotes with basic introductory material and often-cited reasons on why unit tests and automated tests are important (identifying bugs, improving design, avoiding scope creep, and learning from the test-writing experience). Koskela talks about why units tests are more effective when used for design in addition to being used for quality assurance.

The "Factors of Productivity" section of Chapter 1 is useful for understanding certain measures by which one might determine whether unit tests are "effective." These include execution speed (performance), readability, reliability, and trustworthiness. Achieving these characteristics of effective unit tests is the focus of the book. A couple concluding sections of the first chapter focus on using tests for design and employing behavior-driven development (BDD).

Among other introductory details, he articulates why "100% code coverage isn't the goal." Several well-known testing-related terms ["test-infected", "test-driven development" (TDD), and "accidental complexity"] are also introduced in this chapter along with references for additional details. In this first chapter, the author also introduces his "The Law of Two Plateaus" to differentiate between using unit tests solely for quality assurance versus using them for design in addition to quality assurance.

Chapter 1 is mostly introductory and probably doesn't hold a lot of new insights for someone who has worked with Java extensively and/or has written unit tests extensively. However, it does manage in 12 pages to meet the author's goal for it and the other two chapters (Chapter 2 and Chapter 3) of Part 1 of providing a "shared context" for the remainder of the book.

Chapter 2: In Search of Good

Chapter 2 delves deeper into the question of "What makes a test 'good'?" Koskela is quick to point out that there is some subjectiveness to this ("some of the quality of test code is in the eye of the beholder" as is the case for any code) and that different contexts can affect whether a particular unit test is good or not.

Koskela discusses in this chapter how virtues of regular source code are often valued virtues of test code. For example, he discusses that readable test code is maintainable test code and that appropriate structure plays a big role in making tests understandable. Koskela devotes a section of Chapter 2 to a smaller but significant issue I've seen repeatedly in writing and maintaining my own and others' unit tests: unit test methods that advertise testing something they don't really test (perhaps because they are named poorly) can be very costly. I like what Koskela titles that section, "It's not good if it's testing the wrong things." This sounds obvious, but there is deep truth to that simple statement.

Another section of the second chapter focuses on the principle that, for good tests, "independent tests run easily in solitude." Koskela provides a list of dependencies (such as "randomness" and "persistence") that to him are code smells indicating that something might be wrong with the unit test code. I like his "litmus test for a project's test infrastructure" to satisfy the following scenario: "Can I check out a fresh copy from version control to a brand new computer I just unboxed, run a single command, lean back, and watch a full suite of automated tests run and pass?" This section does a nice job of covering why it's important that tests are independent and do not rely on being called in a certain order. In addition to pointing out some unit test dependency smells in this section, Kosekla also provides some specific approaches that might be taken to address these.

One of the sections of the second chapter of Effective Unit Testing looks at why testing the wrong thing or even testing nothing at all ("happy tests") are problematic. There is coverage of why tests "need to be repeatable" along with references to Java-specific examples of tests that introduce things outside of the testing developer's control into the tests.

The last section of Chapter 2 (not counting the "Summary") introduces test doubles, the subject of Chapter 3. Koskela defines test doubles as "an umbrella term for ... stubs, fakes, or mocks." He adds that "test doubles" are "objects that you substitute for the real implementation for testing purposes." Koskela groups test doubles with testing frameworks and build tools as his "top three tools of the trade for software developers writing automated tests."

Chapter 3: Test Doubles

The third chapter is the concluding chapter and my favorite chapter of Part 1 ("Foundations"). The chapter is devoted to coverage of "test doubles", a term and concept introduced in Gerard Meszaros's xUnit Test Patterns: Refactoring Test Code. Koskela outlines five reasons developers might use test doubles, including "the most fundamental of the reasons for employing a test double - to isolate the code you want to test from its surroundings." After listing these five reasons for use of test doubles, Koskela describes each of these motivating reasons in greater detail. He then describes each of the types of test doubles and compares their strengths and weaknesses. Koskela's section "Guidelines for Using Test Doubles" introduces his "logic and heuristics" for "picking the [test double] option that results in the most readable test". These include five considerations plus the simplifying rule: "stub queries; mock actions" (attributed to J.B. Rainsberger, author of JUnit Recipes).

The third chapter of Effective Unit Testing also touches on organizing unit tests with the Arrange-Act-Assert convention and likens this to behavior-driven development's Given-When-Then vocabulary. This chapter also demonstrates principles of the chapter with brief forays into JMock and Mockito code examples and a reference to J.B. Rainsberger's blog post JMock v. Mockito, but Not to the Death.

Chapter 4: Readability

The fourth chapter of Effective Unit Testing is the first chapter of Part 2 ("Catalog"). As with all three chapters in Part 2, Chapter 4 looks at "test smells" that might indicate tests that are less effective than they could be. In this chapter's case, the test smells are those most closely associated with problems related to readability of unit tests.

Koskela starts Chapter 4 by articulating the difference between reading test code and running test code: "Reading the tests ... should provide the programmer with an understanding of what the code should do. Running those tests should tell the programmer what the code actually does."

The test smells that Koskela associates most closely with the "readability" portion of his Test Smells Catalog are:

• Primitive Assertions
  • Assertion that "uses more primitive elements than the behavior it's checking"
  • Analogous to the primitive obsession code smell
• Bitwise Assertions
  • Special case of Primitive Assertions that uses bitwise operators for "optimized test assertions" at the cost of readability and understandability
• Hyperassertions
  • Assertion that that "becomes brittle and hides its intent under its overwhelming breadth and depth"
• Incidental Details
  • Incidental details make it difficult to identify the "intent, purpose, and meaning" of a unit test
• Split Personality
  • Unit test "embodies multiple tests" rather than obeying unit test mantra "A test should only check one thing and check it well"
• Split Logic
  • Test code scattered over multiple files
• Magic Numbers
  • Using numeric and String literals rather than using constants and variables with readable names
• Setup Sermon
  • Too much code (often refactored from tests suffering incidental details smell) in the setup method
• Overprotective Tests
  • Application of unnecessary/redundant guard or test assertions when condition would fail anyway

In each test smell case, Koskela provides examples of these test smells along with one or more ways (description and code examples) of addressing the smells.

Hamcrest is introduced in Chapter 4 as a partial solution to addressing test smells. There is also a unit test example that is built for testing JRuby source code.

In the fourth chapter, Koskela provides some memorable quotes. He articulates an opinion that I've long held regarding unit test code: "When weighing alternatives to expressing intent in test code, you should keep in mind that the nature and purpose of tests puts a higher value on readability and clarity than, say, code duplication or performance." He also writes, "A test that has never failed is of little value - it's probably not testing anything. On the other end of the spectrum, a test that always fails is a nuisance." Koskela also explains that "A test should have only one reason to fail" and explains that this is related to the Single Responsibility Principle.

Chapter 5: Maintainability

Chapter 5 continues the coverage of test smells, but moves from the focus of Chapter 4 on "readability" to focus instead of "maintainability." As he did in Chapter 4, Koskela enumerates several test smells most closely associated with test maintainability and uses code examples to demonstrate these smells and how to address these smells.

Chapter 5 focuses on the following "maintainability" test smells:

• Duplication
  • Needless repetition that increases places where same change must be made and increases risk of not changing all necessary code
  • Duplication can be structural or semantic or both
• Conditional Logic
  • "Conditional execution structures such as if, else, for, while, and switch" reduce the ability to use tests to "understand what the code does and what it should do"
• Flaky Test
  • Tests that "fail intermittently," typically due to multithreading or race conditions
• Crippling File Path
  • Absolute paths, especially hard-coded absolute paths, prevent unit tests from being run on others' machines
• Persistent Temp Files
  • Files that are generated by unit tests may be less temporary than one realizes and interfere with later tests
• Sleeping Snail
  • Use of Thread.sleep(long) calls in unit test methods
• Pixel Perfection
  • Specialized version of Primitive Assertion and Magic Numbers test smells applied to exactly matching graphic representations in unit tests
• Parameterized Mess
  • Result of overusing and abusing parameterized tests
• Lack of Cohesion in Methods
  • "Test methods in a test class are only interested in some of the fixture's objects"

As he did for the test smells covered in Chapter 5, Koskela provides code-based examples of each test smell discussed in Chapter 5 along with code-based examples of how to address each of the code smells.

Chapter 6: Trustworthiness

Chapter 6 finishes off Part 2 and the Catalog of Test Smells. Chapter 6's focus is on test smells closely associated with the degree of reliability and trustworthiness of tests. The test smells covered in this chapter are:

• Commented-out Tests
  • Commenting out tests' implementations so they appear to pass when they really aren't run at all ("poor man's version control")
  • Removing @Test annotation from JUnit 4-based unit test method has same negative effect
  • Use of @Ignore (JUnit) is similar
• Misleading Comments
  • Confuse what the tests are really testing (a famous code smell as well as test smell)
• Never-failing Tests
  • "A test that can never fail is probably worse than not having that test ... Tests are supposed to fail when they should."
• Shallow Promises
  • A "tests that does much less than what it says it does - or does nothing at all"
  • One type of "shallow promise" test smell (commenting out the body of the test method but not its signature and @Test annotation) misleads people to think test of something significant passed
  • Lack of assertions in a test method is another specific example of this test smell
  • Method name implying different functionality tested than what is actually tested is third example
• Lowered Expectations
  • "Tests that are overly robust - they don't fail when they should," usually because "assertions are too vague"
• Platform Prejudice
  • "Failure to treat all platforms equal"
• Conditional Tests
  • "... conditional tests in our tests ... [are] bad in general"
  • "All branches in a test method should have a chance to fail"

Koskela uses code samples to illustrate the Chapter 6 code smells and what unit tests look like once the test smells are addressed with his recommendations. Chapter 6 wraps up Part 2 on the Catalog of Test Smells.

Chapter 7: Testable Design

Proponents of Test-Driven Development and even many proponents of any type of unit testing argue that unit testing is as much about design as it is about testing. As mentioned earlier, Koskela wrote about the Law of Two Plateaus in Chapter 1 and explained that significantly more benefit can be obtained from unit tests when they are used for more than quality assurance and are used in the design process. Chapter 7 returns to this idea and focuses on what it means to be a "testable design." Koskela defines testable design as "[making] it easy to instantiate classes, substitute implementations, simulate different scenarios, and invoke particular execution paths from our test code."

Koskela describes several design principles that lead to testable design: modular design, SOLID principles [Single Responsibility Principle, Open Closed Principle, Liskov Substitution Principle, Interface Segregation Principle, Dependency Inversion Principle]. Koskela discusses these principles of object-oriented design and how software that adheres to them is inherently more testable./p>

Koskela articulates an advantage of writing unit tests early that I've definitely observed: "The act of writing tests before the implementation they call for is essentially a way to ensure that you're taking the client's view on the code you're shaping."

Section 7.2 of Effective Unit Testing looks at several "testability issues" and then Section 7.3 addresses these testability issues with "guidelines for testable design." It is worth nothing here that Section 7.2 and Section 7.3 (and really the entire book) make the assumption that these Java unit tests are run without use of reflection or bytecode manipulation. Using these tools, such as provided by PowerMock, would be one way to address some of these issues, but do add complexity to the unit tests.

I expected Section 7.3 to be the most controversial part of the book for me. It's not that I expected the book to be controversial; rather, it is the topic that is controversial. It has always bothered me when I have to change my production code design for no other reason than to accommodate a unit test. In practical terms, I often cave in and do this because the negative effect on my production code design is less significant than the benefit of unit testing, but I still don't have to like doing it.

In Section 7.3, Koskela describes some of the commonly-held assertions about testable Java code, including avoiding complex private methods, avoiding final methods, avoiding static methods, avoiding logic in constructors, avoiding the singleton, favoring composition over inheritance, and wrapping external libraries. I started reading this section expecting to think, "Yeah, but ...." Instead, I found myself largely agreeing with Koskela's arguments based on my own experience. His pragmatic attitude made his assertions in this section agreeable and realistic. For example, he advises to avoid "complex private methods" rather than simply stating avoid private methods altogether. I've always liked when good design principles are also good for testability and Koskela manages to couch most of these "testable design principles" as such.

Chapter 8: Writing Tests in Other JVM Languages

I'm a big fan of Groovy (127 blog posts to date labeled Groovy) and am increasingly interested in Scala, so I looked forward to reading Chapter 8 on unit testing with alternative JVM languages. Koskela provides a brief history of languages other than Java on the JVM and covers some benefits common to these alternative languages.

Although Koskela mentions Scala, Clojure, JRuby, Jython and even non-JVM Ruby in Chapter 8, the lion's share of the chapter is on testing with Groovy and Groovy-based tools. In particular, the chapter covers using Groovy for testing directly as well as using Groovy-based BDD testing frameworks easyb and Spock Framework.

Chapter 9: Speeding Up Test Execution

There are numerous factors that can demotivate Java developers when it comes to unit testing. Perhaps none is more significant than slowly executing tests. If running the tests starts to take too long, the developer might lose interest in running them as often. As they are used (run) less, the developer may start to question the investment of time and energy in writing and maintaining them. Also, as they are run less, it becomes increasingly likely that problems the test would flag will go on longer before being caught.

Koskela covers ideas for improving unit test execution performance in the ninth and final chapter of Effective Unit Testing. He begins by looking at why it is important to have fast tests (both builds and test execution) and follows that with looking at strategies to make the builds and execution of tests quicker.

Chapter 9 includes brief coverage of using Ant and Maven for building unit tests and how to profile build performance with both of those tools. Until reading this chapter I was not aware of Ant's built-in ProfileLogger (since 1.8) or Ant-Contrib. This chapter also demonstrates how to use Koskela's maven-build-utils extension. Ant's JUnitReport task and Maven's maven-surefire-plugin are also demonstrated in this chapter.

Besides covering build and execution profiling tools and how to use them to identify the tests that really need to have their performance addressed, Koskela also provides several tactical approaches one can use to improve test execution efficiency. These are solid ideas that I don't consider premature optimization because they are, for the most part, simply good ideas in general that potentially improve performance without sacrificing readability or maintainability of the tests. Another thing I like about these tactical approaches is that many of them have been covered in a slightly different perspective earlier in the book. This ending chapter now brings those previously discussed principles back into the discussion for improving test performance while at the same time addressing test smells.

All of Koskela's tactical suggestions for better performing unit tests make sense to me, but I particularly liked his coverage on database access in unit tests because I so often see this violated at extreme cost. I like his emphasized statement: "friends don't let friends use a database in their unit tests." He explains (and I agree) that integration tests are more appropriate for testing of actual database access.

There is a table in Chapter 9 (Table 9-1) that summarizes approaches one might take to address the two primary external constraints on unit test execution performance (CPU and I/O). Koskela then moves onto detailing how to implement some of these mitigation approaches. Several tools are covered in this chapter including some Linux tools, but two of the most interesting and new to me were Koskela's descriptions of how to use Amazon Web Services and GridGain to improve unit test building performance.

Appendices

Appendix A is a 7-page "JUnit Primer" that covers basics of JUnit with focus on assertions and use of Hamcrest matchers. Appendix B is a little over 6 pages on "Extending JUnit" with focus on runners and rules.

Test Smells

For me, the heart of Effective Unit Testing is Part 2 (plus Chapter 7 and Chapter 9 of Part 3 which I highlight next) that catalogs the test smells. This Part 2 is great at outlining issues I've run into with unit tests that can make unit testing more painful than it needs to be. It is also easy to see that there are trade-offs between the test smells such that the very approach that alleviates one test smell may increase risk of introducing another test smell. One example is the trade-off between duplication and readability in unit tests. The author talks about many of these trade-offs and provides rough guidelines of how to decide which way to lean without going too far in either direction. In the case of unit test readability at the cost of some duplication, I like stucampbell's take on this: "I'm more likely to refactor duplicated code for setting up state. But less likely to refactor the part of the test that actually exercises the code."

Testable Design and JUnit Performance

Like all three chapters in Part 2 on test smells, two of the chapters in Part 3 (Chapter 7 on testable design and Chapter 9 on unit test performance) are also chapters that I plan to re-read in the future. There is a lot packed into these chapters that directly address common unit testing issues as well as sparking ideas about other approaches that could be used to improve unit testing.

The Audience

As one would expect from a book with the title Effective Unit Testing, this book does indeed provide guidance on what separates effective unit tests from less effective unit tests. Not only does it cover practices one should use and should avoid, but it introduces terminology and cites well-known resources in the unit testing literature. As such, it is not only an appropriate book for someone with basic familiarity with unit testing who wants to improve their unit tests, but is also highly relevant to those new to unit testing in general and unit testing in Java in particular who want an overview of unit testing in Java. This book doesn't have enough details to be the only source of information for someone new to unit testing in Java, but does give the overall high-level view that can provide context for reading literature more specifically focused on frameworks and other unit testing tools.

Part 1 will be least interesting to those with significant experience with unit testing, but will be of high value to those who are new to unit testing or non-coding managers and leads who want an overview of why unit testing is important and what kinds of high-level things can make unit tests less effective or more effective. Although Part 1 was the least interesting to me, I still felt it was well-written and met the author's stated goal of providing a common context for the remainder of the book.

Part 2 and Part 3 are more technically detailed than Part 1. I like the overall descriptions of test smells and approaches to scrub these test smells in Part 2. I also really like the coverage in Part 3 of ideas for improving test build and execution performance and related to how to design code that is more testable.

What This Book is Not

As I have stated in this post and as the author reminds the reader of Effective Unit Testing, this book is not the book one should use to learn details of JUnit or other testing framework. There is just enough coverage of these to illustrate more general points, but the focus is on unit testing principles rather than on unit testing implementation libraries and frameworks. Although someone new to unit testing Java applications could find this book useful (particularly Part 1), some knowledge of JUnit would be beneficial (particularly for Part 2 and Part 3).

The Book Advantage

I have found that different mediums have different advantages when it comes to conveying information and learning. Although many of the principles found in this book are available online in various forms, the strength of the book is the author's organization and articulation of the ideas in a single coherent source. I think the book is well worth its price when I think of the time it would take to collect and organize these ideas from online sources. Furthermore, the author provides examples from his own and friends' experiences to add a "real life" feeling to it all. A well-written book's advantage over blog posts, forum threads, and the like is the ability to cohesively and coherently cover a topic with breadth and depth. This book does just that for the topic of unit testing in Java. To me this book may not offer a lot of new high-level concepts (although it does offer some new to me low-level details), but it articulates well the ideas and practices that seem to be emerging from collective experience writing unit tests for Java-based applications.

Breadth of References

Another thing I liked about Effective Unit Testing is the abundance of reference to sources with additional details on the ideas, concepts, tools, and frameworks referenced in the book. Some might argue that a downside of the book is that most of the concepts are not new. I actually argue the opposite: because these are practices for effective unit tests, one would expect them to be based on what more than one person has found to be effective through hard experience. When I am reading a book on "effective" anything, I'm not looking for something that is simply "new" or "different"; I am looking for things I should generally do and generally not do and why. By referencing others' work in unit testing as well as describing his own efforts in this area, Koskela increases the credibility of his book. I also liked the mixing of alternative languages, frameworks, and operating systems in the examples.

Breadth of Coverage

In 201 pages (not counting appendices, preface, etc.), Koskela articulates and demonstrates what has taken me years of experience to learn from writing and maintaining unit tests and from reading about unit testing. Even though the high-level concepts were not really new to me, I still learned several tactical approaches from this book. Besides learning some new tactics to employ to implement the concepts of unit testing in Java that I thought I already knew, this book has sparked additional ideas for improving my unit tests and has reinvigorated my interest in writing better unit tests.

Pragmatic Advice

I liked Koskela's pragmatism in this book. Some unit testing enthusiasts (or test-infected Java developers as he calls them in the book) can be overbearing in their enthusiasm and evangelism to the point where it's difficult to believe their claims. Koskela is obviously enthusiastic about unit testing, but seems to keep in mind one important truth: unit tests exist to benefit the quality of the design and code (the production code does not exist to benefit the unit tests). Koskela points out, for example, that there are cases where redundant unit test code might be easier to read and maintain than rigorously implementing DRY principles within test code at any cost.

Conclusion

I was a little apprehensive when I purchased Effective Unit Testing as part of Manning's MEAP, but I am glad that I did. This book delivered what I hoped for and I found Part 2 and Chapter 7 and Chapter 9 of Part 3 to be particularly useful for someone in my situation (relatively experienced Java developer looking for ideas to improve his or her unit testing). Although there wasn't a lot new to me at the highest level, there were a lot of interesting lower-level details that were new to me or were presented from a unique and interesting perspective. I also liked having these ideas I had from my own experiences laid out and articulated for me in printed form and I liked having these concepts being developed by the Java unit testing community all codified in a single book. The book packs a lot into 201 pages of regular text and the writing style is easy to read and understand. It is easy for me to recommend this book to Java developers who feel they have room to improve in writing of unit tests of Java applications. I also know many Java developers (including myself) who could benefit from reading this book.

Hamcrest Containing Matchers

The Hamcrest 1.3 Javadoc documentation for the Matchers class adds more documentation for several of that class's methods than were available in Hamcrest 1.2. For example, the four overloaded contains methods have more descriptive Javadoc documentation as shown in the two comparison screen snapshots shown next.

Although one can figure out how the "contains" matchers work just by trying them out, the Javadoc in Hamcrest 1.3 makes it easier to read how they work. Most Java developers probably think of behavior like that of String.contains(CharSequence) or Collection.contains(Object) when they think of a contains() method. In other words, most Java developers probably think of "contains" as describing if the String/Collection contains the provided characters/objects among other possible characters/objects. However, for Hamcrest matchers, "contains" has a much more specific meaning. As the Hamcrest 1.3 documentation makes much clearer, the "contains" matchers are much more sensitive to number of items and order of items being passed to these methods.

My examples shown here use JUnit and Hamcrest. It is important to emphasize here that Hamcrest's JAR file must appear on the unit tests' classpath before JUnit's JAR file or else I must use the "special" JUnit JAR file built for use with the standalone Hamcrest JAR. Using either of these approaches avoids the NoSuchMethodError and other errors (suc as org.hamcrest.Matcher.describeMismatch error) resulting from mismatched versions of classes. I have written about this JUnit/Hamcrest nuance in the blog post Moving Beyond Core Hamcrest in JUnit.

The next two screen snapshots indicate the results (as shown in NetBeans 7.3) of the unit test code snippets I show later in the blog to demonstrate Hamcrest containing matchers. The tests are supposed to have some failures (7 tests passing and 4 tests failing) to make it obvious where Hamcrest matchers may not work as one expects without reading the Javadoc. The first image shows only 5 tests passing, 2 tests failing, and 4 tests causing errors. This is because I have JUnit listed before Hamcrest on the NetBeans project's "Test Libraries" classpath. The second image shows the expected results because the Hamcrest JAR occurs before the JUnit JAR in the project's "Test Libaries" classpath.

For purposes of this demonstration, I have a simple contrived class to be tested. The source code for that Main class is shown next.

Main.java

package dustin.examples;

import java.util.Collections;
import java.util.HashSet;
import java.util.Set;

/**
 * Main class to be unit tested.
 * 
 * @author Dustin
 */
public class Main
{
   /** Uses Java 7's diamond operator. */
   private Set<String> strings = new HashSet<>();

   public Main() {}

   public boolean addString(final String newString)
   {
      return this.strings.add(newString);
   }

   public Set<String> getStrings()
   {
      return Collections.unmodifiableSet(this.strings);
   }
}

With the class to be tested shown, it is now time to look at building some JUnit-based tests with Hamcrest matchers. Specifically, the tests are to ensure that Strings added via the class's addString(String) method are in its underlying Set and accessible via the getStrings() method. The unit test methods shown next demonstrate how to use Hamcrest matchers appropriately to determine if added Strings are contained in the class's underlying Set

Using Hamcrest contains() Matcher with Single String in Set Works

   /**
    * This test will pass because there is only a single String and so it will
    * contain that single String and order will be correct by implication.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsForSingleStringSoWorks()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, contains("Java"));
   }

The unit test shown above passes because the Set only has one String in it and so the order and number of Strings tested with the contains matcher matches.

Using Hamcrest Contains with Same Number of Elements Works if Order Matches

   /**
    * The "contains" matcher expects exact ordering, which really means it should
    * not be used in conjunction with {@code Set}s. Typically, either this method
    * will work and the method with same name and "2" on end will not work or
    * vice versa.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsForMultipleStringsNotWorks1()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, contains("Java", "Groovy"));
   }

   /**
    * The "contains" matcher expects exact ordering, which really means it should
    * not be used in conjunction with {@code Set}s. Typically, either this method
    * will work and the method with same name and "1" on end will not work or
    * vice versa.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsForMultipleStringsNotWorks2()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, contains("Groovy", "Java"));
   }

The two example unit tests shown above and their resultant output of running those test as shown in the previous screen snapshot show that as long as the number of arguments to the contains() matcher are the same as the number of Strings in the collection being tested, the match may work if the elements tested are in exactly the same order as the elements in the collection. With an unordered Set, this order cannot be relied upon, so contains() is not likely to be a good matcher to use with a unit test on a Set of more than one element.

Using Hamcrest Contains with Different Number of Elements Never Works

   /**
    * Demonstrate that contains will NOT pass when there is a different number
    * of elements asked about contains than in the collection.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsNotWorksDifferentNumberElements1()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, contains("Java"));
   }

   /**
    * Demonstrate that contains will NOT pass when there is a different number
    * of elements asked about contains than in the collection even when in
    * different order.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsNotWorksDifferentNumberElements2()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, contains("Groovy"));
   }

As the JUnit test results indicate, these two unit tests never pass because the number of elements being tested for in the Set is fewer than the number of elements in the Set. In other words, this proves that the contains() matcher does not test simply for a given element being in a collection: it tests for all specified elements being present and in the specified order. This might be too limiting in some cases, so now I'll move onto some other matches Hamcrest provides for determining if an element is contained in a particular collection.

Using Hamcrest's containsInAnyOrder() Matcher

The containsInAnyOrder matcher is not as strict as the contains() matcher: it allows tested elements to be in any order within the containing collection to pass.

   /**
    * Test of addString and getStrings methods of class Main using Hamcrest
    * matcher containsInAnyOrder.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsInAnyOrder()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultCSharp = subject.addString("C#");
      final boolean resultGroovy = subject.addString("Groovy");
      final boolean resultScala = subject.addString("Scala");
      final boolean resultClojure = subject.addString("Clojure");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, containsInAnyOrder("Java", "C#", "Groovy", "Scala", "Clojure"));
   }

   /**
    * Use containsInAnyOrder and show that order does not matter as long as
    * all entries provided are in the collection in some order.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsInAnyOrderAgain()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, containsInAnyOrder("Java", "Groovy"));
      assertThat(strings, containsInAnyOrder("Groovy", "Java"));
   }

The two unit tests shown immediately above both pass despite the Strings being tested being provided to the containsInAnyOrder() matcher in a different order than what they could exist in for both collections. However, the less strict containsInAnyOrder() matcher still requires all elements of the containing collection to be specified to pass. The following unit test does not pass because this condition is not met.

   /**
    * This will fail because containsInAnyOrder requires all items to be matched
    * even if in different order. With only one element being tried and two
    * elements in the collection, it will still fail. In other words, order
    * does not matter with containsInAnyOrder, but all elements in the collection
    * still need to be passed to the containsInAnyOrder matcher, just not in the
    * exact same order.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsInAnyOrderDiffNumberElements()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, containsInAnyOrder("Java"));
   }

Hamcrest hasItem() and hasItems() Matchers Work As Sounds

As shown in the next two unit test methods (both of which pass), the Hamcrest hasItem() (for single item) and hasItems (for multiple items) successfully tests whether a collection has the one or more than one specified items respectively without regard for order or number of specified items. This really works more like most Java developers are used to "contains" working when working with Strings and collections.

   /**
    * Demonstrate hasItem() will also work for determining a collection contains
    * a particular item.
    */
   @Test
   public void testAddStringAndGetStringsWithHasItem()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, hasItem("Groovy"));
      assertThat(strings, hasItem("Java"));
   }

   /**
    * Demonstrate that hasItems works for determining that a collection has one
    * or more items and that the number of items and the order of the items
    * is not significant in determining pass/failure.
    */
   @Test
   public void testAddStringAndGetStringsWithHasItems()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, hasItems("Groovy", "Java"));
      assertThat(strings, hasItems("Java", "Groovy"));
      assertThat(strings, hasItems("Groovy"));
      assertThat(strings, hasItems("Java"));
   }

Hamcrest isIn() Matcher Tests Containment from Other Direction

The just-discussed hasItem() and hasItems() matchers are less strict than contains() and even less strict than containsInAnyOrder() and are often what one wants when one wants to simply ensure that one or multiple items are somewhere in a collection without concern about the item's order in that collection or that other possible items are in that collection. One other way to use Hamcrest to determine the same relationship, but from the opposite perspective, is to use isIn matcher. The isIn matcher determines if an item is somewhere with the collection provided to the matcher without regard for that item's order in the collection or whether or not there are other items in that containing collection.

   /**
    * Use isIn matcher to test individual element is in provided collection.
    */
   @Test
   public void testAddStringAndGetStringsWithIsIn()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat("Groovy", isIn(strings));
      assertThat("Java", isIn(strings));
   }

Conclusion

Hamcrest provides a rich set of matchers that can be used to determine if specified elements reside within a specified collection. Here are important points to keep in mind when deciding to apply these and determining which to use:

• Ensure that the Hamcrest JAR is on the test classpath before the JUnit JAR.
• Use contains when you want to ensure that the collection contains all specified items and no other items and you want the collection to contain the items in the specified order.
  • Generally avoid using contains() matcher with Sets because they are unordered by nature.
• Use containsInAnyOrder matcher when you still want to strictly test for presence of exactly same items in collection as specified in test, but don't care about the order (applicable for Sets).
• Use hasItem() and hasItems() matchers to ask a collection if it contains, possibly among other unlisted items and in no particular order, the specified item or items.
• Use isIn() matcher to ask if a particular item is in the specified collection with no regard for whether other items are in that collection or what order that item is in within the containing collection.