Tapper is an infrastructure.
It consists of applications, tools and protocols for testing software and evaluating the results. One initial main focus was on testing Operating Systems in virtualization environments. It is now a modular infrastructure for lots of other, related scenarios, like benchmarking or build systems.
There are 3 important layers:
Reports Framework
Test Suites
Automation System
The layers can work completely autonomously, though can also be connected together and are targeted to be stacked in this order:
The Report Framework is for receiving and evaluating test reports sent by any source. The only requirement is that the result reports are using TAP, the Test Anything Protocol.
The Test Suites are either directed test suites or wrappers around existing test projects doing conversion of their results into TAP. These are the TAP producers that create reports and send them to the Reports Framework.
The Automation System is taking care of setting up machines, installing dependencies and test suites and running the latter. It can set up virtualized environments.
To fully exploit the system you need to learn:
Connect and prepare a new machine into the infrastructure
Write tests using the Test Anything Protocol (TAP)
Write preconditions to describe automation tasks
Review results via Web interface
Evaluate results via Report Query interface
There is a central server controlling the automation by running the Master Control Program, aka. MCP. Usually it also centralizes several other services: it is the TFTP server for network booting, runs the daemons of the reports framework (reports receiver, remote api) and the web application, including the mysql databases, and also serves the file repository via NFS.
When machines run automated tests, these test program runs are controlled by a Program Run Control, aka. PRC. In virtualization scenarios, each host and guest has its own PRC, numbered PRC0 (for the host), PRC1 (1st guest), PRC2 (2nd guest), etc.
The Reports Receiver means the daemons that accept reports. We often run them on the same machine as the MCP and the Web framework, but that's not neccessary.
Similar to the reports receiver is the Reports API which is the daemon for all more complex interfaces, like uploading files, downloading files, querying the reports. Similar to reports API we often run them on the same machine as the MCP and the Web application, but that's not neccessary.
The Web User Interface is an independent web application. Similar to the reports receiver and the reports API it can run anywhere, either standalone or in Apache, via mod_perl, FCGI, etc.. The only common thing for all those central applications (MCP, reports receiver, reports api, web application) is the config to use the same databases.
The Reports DB contains all data that are reported. It's the base for the reports receiver, the reports API, the web application.
The Testrun DB is the DB for the automation layer. It contains hosts, testrun specifications and scheduling information.
A Testrun is a request to the automation layer to set up a host machine and run a workload on it. It consists of ``preconditions'' and scheduling information (host name, wanted host features, scheduling queue).
Preconditions are specifications that describe how to set up a host. They are the essential part of a testrun.
A Report is the reported result of any workload, regardless of how it was produced (automatically, by a tes suite, manually via echo and netcat). Its only requirement is to be formatted in TAP (the Test Anything Protocol), or as TAP archive.
The Test Anything Protocol aka. TAP is the syntax to describe test results.
A TAP archive is a .tar.gz file that contains files of TAP. It's the result of a test suite that can consist of many parts compressed into a single file.