NFV splits responsibility for application characteristics between NFVI and VNF vendors. Hence testing that demarcation point becomes essential in all phases:
Essential Integration Point between vendors (orange dots)
The following statement serves as a base for NFV strategy for identifying performance responsibilities between NFV and VNF vendors;
“Verify that your NFV-Infrastructure fulfills its performance KPIs, and it becomes VNF vendors’ responsibility to fulfill theirs."
With that as a starting point, Voereir has spent a significant effort in finding the KPIs that will determine the VNFs’ performance.
We call these a VNF’s essential NFVI KPIs. These will be different for different VNFs, and the definition of these KPIs can be quite complex.
Today we work with around 200 essential KPI definitions. Compliance to the SLA of these KPIs becomes the operator's commitment to the VNF vendors.
Consequently, NFVI KPIs must be verified and guaranteed for Operators to be able to require VNFs’ SLA compliance from the VNF vendors. NFVI SLA compliance tests are needed in pre-operation validation as well as during the full Life Cycle. This put massive demand on testing and makes test automation necessary. Therefore, we developed Touchstone.
Touchstone is a Test Suite Manager. Touchstone enables test organizations to configure,
execute and interpret results from test suites in a homogenous way,
independent of the test framework used for test case development, and compare the results with predefined SLAs, or previous executions of that specific Test Suite.
A tester can entirely rely on the pre-configured Test Suites delivered as part of a Touchstone installation or create his Test Suites with his configuration of Test Cases. Touchstone is provided with a set of pre-integrated open source Test Suites and Test Suites developed by VoerEir, all in all, several thousands of automated Test Cases with a focus on certifying and validating an NFV Infrastructure. Touchstone’s field of application spans over Functional, Performance, Robustness and Stability testing with a specific emphasis on NFVI performance KPI validation.