辅导COMP3053、讲解Java编程语言、辅导ART Harness、Java讲解

COMP3053.SQA Coursework03

TDD to create ART Harness; MT (30%)

Learning Aims:

This courseworkis designed to give youexperience writing high-qualityobject orientedcode using

Test-Driven Development(TDD); to give you a chance to implementsome Adaptive Random Testing

algorithms; andto get you thinking about testing when faced with the Oracle Problem --- using

Metamorphic Testing (MT).

Background:

You have been working at the prestigious software development companyTopQualitySoftware Inc.

Since your arrival there, you have beenpromoted to “technicallead,”andhave been receiving

increasing amounts of responsibility.

TopQualitySoftware’s Chief Technology Officer (CTO), MsBig, hasspoken with youa number of

times about theimportance of software quality and appropriatedocumentation. As part of this,she

has decided to give youa new project:

“I want you, using TDD,to build me a test harness. Thetest harness willwork from the

commandline. It will take a string of arguments, usingflags if necessary,and then

performrandom testing or adaptive random testing on a specified program.For now, all

the programs the harness will be used on will have reasonably simple, numeric,

parameters.”

Your friend, MrNice, was very excited to hear about this new project. Because he is a QA expert,

and an advocateof not only Random Testing, but also Adaptive RandomTesting, he hada lot of

very helpful advice foryou:

“This is a great idea.I thinkyou should research a little about pseudo-random number

generators (PRNGs), especially how to use them in Java,and theimportanceof the seed.

It is probably a good idea to research how to scale the output of Java’sPRNG todifferent

ranges,withoutcausinga negative impact on the quality of thepseudo-random number

sequence. Thinkcarefully aboutwhat you’ll record in any datafile yourharnesswill

generate.”

You arefamiliar with the DART,RRT, and MART versions of ART, and knowthevariouscontrol

parameters eachwould require. Mr Nice,is alsovery enthusiastic about this project. He tells you

that it is really important that you get this project right. Perhaps this will be your ticket to becoming

the vice-president! As an expert in ARThe has some more very helpful advice for you:

“In addition toall the things you needto do for an ordinary random testingharness, you

should choose aversionof ART that will work well. Personally,I like theRestriction-based

ART implementation (RART, or RRT), but you can decide which oneto implement

yourself. Remember thateach ART implementationhas different parameters thatwill

need to be passed as arguments on the command line! You’ll probably need to include

some additionalinformation in the datafile, too.”

You arevery eager to make surethat this project is a success,and youhave been studying

Metamorphic Testing (MT) to seeif there is anyway to test theharnessonce completed.

Basic/Outline Harness Interaction:

After some careful consideration, you come up with an outline plan for howa simple random

testing(not ART) test harness could work. Fromthe command line, it would take a sequenceof

arguments introduced byflags. You havedecidedthat the flags (and their arguments) needed are

as follows:

-p <program under test>

-o <oracle>

-s <seed>

-n <number of test cases to generate>

-a <number of arguments/parameters the program under test takes>

-r <series of lower andupper bounds for the arguments/parameters, in a[lwr, upr) range,

meaninggreaterthan orequal to lwr, but strictly lessthan upr>

Assuming the harness is called TestHarness, andthat both the program under test and the

oracle are in the same folder (and called PUT and Oracle,respectively), then a typical call to the

harnessmight be:

%TestHarness -p PUT -o Oracle -s 7 -n 10000 -a 2 -r 0.0 1.0 -1.0 0.0

Which would mean that TestHarness wouldgenerate 10000 test cases using the PRNG with the

seed 7;each test case would bea 2-tuple with the first number in the range [0.0, 1.0), and the

second number in the range [-1.0, 0.0);these test cases would be appliedto the programsPUT and

Oracle,with the outputs from both compared to see if they are the same or not … if the output is

not thesame, that would mean that the harness had found a failure/fault/bug.

After some further consideration, you have decided that, in addition to those already specified in

the ordinary test harness, the following additional flags (and their arguments) would be needed if

you were to implement ALL four of the ART algorithms that you know:

-m [RT|DART|MART|RRT]

-k <Candidate Set size,as an integer: for DART>

-P <Partitioning schema, as a string: for MART>

-R <Target Exclusion Ratio, as a floating point value: for RRT>

Of course, you only need to implement one version of ART!

Again assuming the harness is called TestHarness, and that both the program under test and

the oracle are in the same folder (and called PUT and Oracle, respectively), then a call to the

harness to run random testing (RT) might be:

%TestHarness –m RT –p PUT –o Oracle –s 7 –n 10000 –a 2 –r 0.0 1.0 -1.0 0.0

Which would mean that TestHarness wouldgenerate 10000 test cases using thePRNG with the

seed 7; each test case would be a 2-tuple with the first number in the range [0.0, 1.0), and the

second number in the range [-1.0, 0.0);these test cases would be applied to the programs PUT and

Oracle,with the outputs from both compared to see if they are the same or not … if the output is

not thesame, that would mean that the harness had found a failure/fault/bug.

A similar call to the harness, but thistime to use theDART implementation of ART, with a candidate

set size of 10, might be:

%TestHarness –m DART –k 10 –p PUT –o Oracle –s 7 –n 10000 –a 2 –r 0.0 1.0

-1.0 0.0

You have also decided that whenthe test harness is run, the outputs will be put into a newly

createdfile called “TestResults.dat”.You need to think very carefully about what should go into this

file, and what should happen if the file already exists.

Finally, you should be careful to buildthe harness in such a way that it reacts appropriately to

incorrect input. Error messagesor exceptions may be a good idea.

Metamorphic Testing:

In addition to the TDD-supported unit testing and regression testing that you shouldperform, you

want to use your knowledge of Metamorphic Testing (MT) to further test the harness. You should

identify five (5) metamorphic relations (MRs) for the harness.

Deliverables/Submissions:

§ JavaCode:

You should use Java to produce both theunit tests (forTDD) and the production(main)

code. All the Java code(including any additional libraries) should be put into a zip file (not a

package).

§ (Brief) UserManual

You should prepare a very briefUser Manual, nomore than about250words. This should

explainclearlyto any users how to setup, prepare, and use the test harness.

§ CodeReport

You should write a report aboutthe code used in the test harness.This isa document that

would go to a maintenance team,so it should include details like algorithms used, external

dependencies, known issues, etc. This should not be more than about 500 words.

§ QA Report detailing testing

You should write reportabout the QA process followed to produce the final test harness.

This should include allthe steps takento ensure the quality of the harness, including a

listingof (at least) five MRs used for Metamorphic Testing. This should not be more than

about 500 words.

Submission:

All four deliverables should beput into a .zip file and submitted through Moodle, before the

deadline.

Assessment:

Submissions will be manually marked andgraded,with the weighting of the parts asfollows.

- 60% for the Java code

- 10% for User Manual

- 15% for Code Report

- 15% for QA Report

Deadline:

The deadline will be Friday, 4 pm, December 14th, 2018.