Unit Testing and
Test Automation
NOVATaig
May 13, 2009
Jim Moore, ClassFive LLC, CTO
www.classfive.com
Date
Topics
•Concepts and terminology of Unit Testing
•Benefits and Limitations of Unit Test
•See sample unit tests and test fixtures
•Related technologies and methodologies
•Discuss where Unit Test fits into overall
testing methodology
•Share knowledge
2
About Jim & ClassFive
Jim – Software Engineer – Multimedia,
Office Automation, Point of Sale, Training,
Consulting – 20 years exp
ClassFive - Focused on small to medium
associations for IT consulting and
providing Association Management systems
and collabortaion tools
3
Why Unit Testing for ClassFive?
“Begin with the end in mind”
-- Stephen Covey (The 7 Habits of Highly Effective People)
Helpful toolset/methodology to help
creating a new product codebase that is
highly stable, changeable and usable.
Something we can confidently sell and still
sleep at night.
4
Challenges starting Unit Test
•We don’t have time to do this!
•It won’t make meaningful difference.
•My code doesn’t need tests!
•Just make it work now!
•How many unit tests do we need?
•What toolset to use?
•How far to go? (e.g. Mock objects, CI)
•Deadlines vs process improvement
•Can we reduce our testing staff if we unit
test?
5
Challenges starting Unit Test – Part 2
Hey!
What is this Unit Testing thing – and what
does it do?
6
This is a Unit
public class MyMath
{
public int Add(int i, int j)
{
return i + j;
}
}
7
Definition: Unit
The smallest piece of code software that
can be tested in isolation.
8
This is a Unit Test
public void MyAddTest()
{
MyMath m = new MyMath();
Assert.AreEqual(m.Add(2, 3), 5);
}
9
Definition: Unit Test
A unit test is a piece of code (usually a
method) that invokes another piece of
code and checks the correctness of some
assumptions afterward. If the assumptions
turn out to be untrue, the unit test has
failed.
10
Unit Tests – Independence
•Units are tested independent of each
other and their environment.
•Tests should only test one thing.
•Try to keep a traceable connection
between potential bugs and test code.
11
Why Unit Test?
• Implicit documentation
• Simplifies Integration
• Improves code design – code becomes highly
decoupled due to 2 users : application and test
harness – code resists transition from 1 user to
2 users, the rest are easy
• Facilitates Change
• Allows programmer to refactor code at a later
date and show individual parts are correct
• Continuous unit tests support sustained maintenance
12
Limitations of Unit Testing
•Don’t expect Unit Testing to solve all
problems or catch all defects – not silver
bullet
•Will not catch integration errors – focuses
on individual units
•Only 1 pare of an effective testing
methodology – cover all bases
•Requires rigorous discipline
•Requires sustainable process of
construction and remediation
13
What is Quality?
14
Quality – definition
Value to some person (Jerry Weinberg).
15
More about Quality
•What is quality software?
•Who decides?
•Pretty front vs. termites in the framing.
•Technical quality vs. Business value
16
Elements of a Good Unit Test Fixture
•Smell test (first reaction)
•Unit test hierarchy should parallel units
•Naming convention clear and consistent
•Good coverage of units in module
•Low to no % of unintended failures
•Test intended failures, not just expected
behavior
17
Unit Test Quality Types (Good, Bad,
Ugly)
Good – effective, well designed tests
Bad – ineffective (wrong) tests (who cares
about design?)
Ugly – effective, poorly designed tests
18
Elements of a Good Unit Test
•Automatic
• Invocation
• Checking
•Thorough
• Depends on needs of project
•Repeatable
•Independent
•Professional
• Test code is “real code”
(Source: Pragmatic Unit Testing with C#)
19
Software Development Testing Stages
Unit Testing – testing of individual
components
Integration Testing – testing to expose
problems from combination of components
System Testing – test complete system
prior to delivery
Acceptance Testing – by users to check
system from satisfaction of requirements
20
Unit Test vs. Acceptance Test
•Unit Test – satisfy programmer that
software does what programmer thinks it
does
•Acceptance Test – satisfy customer that
software provides business value and
makes them willing to pay for software
21
What is a Unit Test Framework?
22
What is a Unit Test Framework
•Automatically run a set of Unit Tests
•Collects Unit Tests into cohesive groups
(e.g. by module)
•Collects Unit Test results into a
consolidated report
•Provides setup and teardown support to
unit tests
23
What is a Test Harness?
Program that executes set of Test
Fixtures, collects results and produces
combined report.
24
What is a Test Fixture?
•Code that executes tests
•Phases:
• Setup Environment
• Execute Test
• Teardown Environment
25
Sample Unit code
using System;
namespace MyApp
{
public class MyMath
{
public int Add(int i, int j)
{
return i + j;
}
}
}
26
•Sample Test Fixture
using System;
using NUnit.Framework;
using MyApp;
namespace MyAppTest
{
[TestFixture]
public class Class1
{
[Test]
public void MyAddTest( )
{
MyMath m = new MyMath( );
Assert.AreEqual(m.Add(2, 3), 5);
}
}
}
27
Unit Test Frameworks for All
•Java – JUnit, Cactus, EasyMock
•Ruby – Test:Unit, RSpec
•.Net - Nunit, TestDriven.Net, NMock
•Javacript – JSUnit, JSpec,
•Python – PyUnit
List of frameworks can be found at :
http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks
28
StringTemplate Unit Tests
StringTemplate is template library (e.g.
merge data into template file)
•Substantial set of unit tests
•Originally written in Java (and ANTLR)
•Ported to C# and Python
29
StringTemplate Unit Test Example – C#
C# version
[Test]
public virtual void testSingleExprTemplateArgument()
{
string templates = ""
+ "group test;" + NL
+ "test(name) ::= \"<bold(name)>\"" + NL
+ "bold(item) ::= \"*<item>*\"" + NL;
StringTemplateGroup group = new StringTemplateGroup(new
StringReader(templates), typeof(AngleBracketTemplateLexer));
StringTemplate e = group.GetInstanceOf("test");
e.SetAttribute("name", "Ter");
string expecting = "*Ter*";
string result = e.ToString();
Assert.AreEqual(expecting, result);
}
30
StringTemplate Unit Test Example Java
public void testSingleExprTemplateArgument() throws Exception {
String templates =
"group test;" +newline+
"test(name) ::= \"<bold(name)>\""+newline+
"bold(item) ::= \"*<item>*\""+newline
;
StringTemplateGroup group =
new StringTemplateGroup(new StringReader(templates),
AngleBracketTemplateLexer.class);
StringTemplate e = group.getInstanceOf("test");
e.setAttribute("name", "Ter");
String expecting = "*Ter*";
String result = e.toString();
assertEquals(expecting, result);
}
31
Common Unit Test Patterns
•Correct results
•Boundary conditions correct
•Inverse relationships checked (e.g. use
result to check itself)
•Cross check results by other means
•Force error condition to occur
•Was performance within bounds (e.g.
impact of data set size, network latency)
32
Common Unit Test Patterns
Boundary Conditions:
•Format conformance
•Result ordering
•Input data range
•External references required state
•Resource existence (e.g database, file)
•Correct number items (e.g. 0, 1, more than 1)
•Time (e.g. timeout length, tolerance of delay)
33
What are Mock Objects?
34
Mock Objects
Simulated objects that mimic the behavior
of real objects in controlled ways.
35
What is Refactoring?
36
Refactoring
The process of changing module’s internal
structure without modifying its external
functional behavior or existing functionality.
This is done to:
• Improve internal non-functional properties of
the software (e.g. readability)
• Simplify code structure (e.g. adhere to a given
programming paradigm),
• Improve maintainability
• Improve performance
• Improve extensibility
37
What is Test Driven Development?
38
Test Driven Development (TDD)
A software development technique that uses short
development iterations based on pre-written test cases
that define desired improvements or new functions.
Each iteration produces code necessary to pass that
iteration's tests. Finally, the programmer or team
refactors the code to accommodate changes.
A key TDD concept is that preparing tests before
coding facilitates rapid feedback changes.
39
TDD Steps
•Write tests that fail
•Implement modules to make test pass.
•Run all tests to ensure nothing broken
•Refactor to keep code clean
•Continue development
40
What is Continuous Integration?
41
Continuous Integration
Continuous Integration is a software
development practice where members of a
team integrate their work frequently,
usually each person integrates at least
daily - leading to multiple integrations per
day.
Each integration is verified by an
automated build (including test) to detect
integration errors as quickly as possible.
42
Continuous Integration Practices
* Maintain a Single Source Repository.
* Automate the Build
* Make Your Build Self-Testing
* Everyone Commits Every Day
* Every Commit Should Build the Mainline on an Integration
Machine
* Keep the Build Fast
* Test in a Clone of the Production Environment
* Make it Easy for Anyone to Get the Latest Executable
* Everyone can see what's happening
* Automate Deployment
(Souce: http://martinfowler.com/articles/continuousIntegration.html)
43
Continuous Integration Tools
Version Control: Subversion, CVS, Git,
SourceSafe
Build Tools: Ant, NAnt, MSBuild, Make
CI Server: CruiseControl, CruisControl.Net,
TeamCity
Unit Test: NUnit, JUnit, NMock, EasyMock, etc
44
Unit Test As Software Test Automation
•Unit test execution best when part of an
automated build system.
•Unit Test practice can directly affect the
quality of the code produced.
•Good set of automated tests allow for
incremental creation of code quality.
•Unit Test starts with immature codebase
while other automation requires mature
codebase and design
45
Summary
•Unit Tests are independent tests of small
chunks of code created during development.
•There are good Unit Test frameworks and
harnesses available for many languages and
environments.
•Unit Testing can directly affect the quality of
code during construction.
•Unit Testing is only one part of an effective
test automation methodology.
46
Q&A
47
Thanks for coming!
Jim Moore
CTO
ClassFive, LCC
[email protected]
http://www.classfive.com/
www.classfive.com
Descargar

Unit Testing 101