sonata-nfv/tng-tests

5GTANGO Integration tests repository

tng-tests's Issues

Create a new freestyle job in Jenkins for test 4.1

https://jenkins.sonata-nfv.eu/job/test-test_sla_e2e-4.1/

Slice -SLA within Network Slice 3.3.1

The Slice1 has the SLA1 applied, and the Slice2 has the SLA2 applied.
Task List;

• Create SP folder
• Create README
• Create .robot
• Create Jenkins job

Add package NSID1V_OSM

Add package NSID2C

Add package NSID1V

name for disk image missing

Add package NSID2V

Add SLA test cases in:

https://github.com/sonata-nfv/tng-tests/wiki/test_cases

**test_sp05_test_migration_of_state (k8s)**

Short description: This test ensures that a state migration of a CNF is consistent.
Tool: MANO
Collected metric: test execution time, test instantiation time, test migration time
Priority: medium
Complexity: medium
Responsible: PANOS K (SYN), Eleni (UBI), Thomas (IMEC)

test_sdk08_benchmarker_performance

Tool: tng-sdk-benchmark
Short description: Benchmarking performance in terms of executed benchmarks per time
Parameters: Number of tested configurations
Collected metric: Reconfiguration time
Priority: high
Complexity: medium
Responsible: Manuel (UPB)

---

Data should be already available from collaboration with Ubitech. Just need to filter, polish and plot it.

rate_sp_kpi

• endpoint is one of the collected KPIs dimensions, so we can filter by it

client needs a testing_taged pair of a service and a test

Graylogs output into Robot

Add Graylogs error messages as an artifact if the Robot test fails.

Error when requesting map_package_on_service function from tnglib

robot test fails on map_package_on_service function
when called from /test_runtime_policy_lifecycle/test.robot file

This is the output as seen at command line after test execution:

==============================================================================
Test :: Test suite for enforcing a Runtime Policy to the SP platform          
==============================================================================
Setting the SP Path                                                   | PASS |
------------------------------------------------------------------------------
Upload the Package                                                    | FAIL |
KeyError: 'name'

Can we fix this? Am i requesting correctly the map_package_on_service funtion?
Thanks in advance :-)

Slice - Network Service Composition 3.1.1

Slice that is comprised of 3 NS with 1 VNFs each, linked among them

Task list:

• Create SP folder
• Create README
• Create .robot
• Create Jenkins job

[Rate limit] Check rate limit responses when under the rate limit

1. Set the SP
2. Create a user with a role (eg. Dev)
3. Login
4. Get REQUESTS (2 reqs/1sec)
5. Check if 2 x 200 OK received (should be true)
6. Logout
7. Delete User

Create a new freestyle job in Jenkins for test sla 4.2

Manual scaling - 1 VNF 5.1

The manual scaling consists to scale out/in one VNF using the gk.
Task List;

• Create SP folder
• Create README
• Create .robot
• Create Jenkins job

Template .robot: https://github.com/sonata-nfv/tng-tests/wiki/methodology.md#test-case-descriptor
SP folder location: tng-test/SP/test-manual-scale-5-1

Create a folder for the test in the route /test/SP/test_name

Simple Service Lifecycle - Multi VIM and the use of the WIM

Task list:

• Create SP folder
• Create README
• Create .robot
• Create Jenkins job

test_sdk04_test_creation

Tool: tng-sdk-test
Short description: Test execution times (on-emulator)
Parameters: set of dummy tests
Collected metric: test execution time, service instantiation time
Priority: high
Complexity: medium
Responsible: Askhat (IMEC)

---

Unexpected field

The field slad is not in the schema sla schema

tng-tests/slas/silver_sla_example.yml

Line 6 in 6e09fe0

slad:

**

Slice - Network Service Composition 3.1.2

Slice that is comprised of 3 NS, 2 with 1 VNFs and 1 with 1 CNF, linked among them

Task list:

• Create SP folder
• Create README
• Create .robot
• Create Jenkins job

Create a new freestyle job in Jenkins for test sla 4.4

test_sdk02_packager

Tool: tng-sdk-packager
Short description: Package different packages.
Parameters: package size (no. VNF/files 1-100 VNF (maybe 1000)), package kind (5GTANGO, OSM), validation (yes/no)
Collected metric: time to package, memory used, package size
Priority: high
Complexity: low
Responsible: Manuel (UPB)

---

Slice - Share Network Service 3.2.1

Bothe slices are composed by 3 NSs, which they are all composed of VNFs.
NS1_1 --> shared_NS2 --> NS3_1
NS1_2 --> shared_NS2 --> NS3_1

Task list:

• Create SP folder
• Create README
• Create .robot
• Create Jenkins job

test_sdk07_vim-emu-scalability

Tool: vim-emu
Short description: Test scalability of vim-emu
Parameters: Number of emulated PoPs, number of deployed services
Collected metric: Boot times, memory, etc.
Priority: high
Complexity: low
Responsible: Manuel (UPB)

---

Note: Tests have already performed and results are available from previous Journal publication. We will just re-use those numbers for D6.3.

errors_sp_kpi

• endpoint and error are collected KPIs dimensions, so we can filter by it

**test_sp01_test_integration_monitoring_vims**

Short description: Check the integration between monitoring manager and the VIMs
Tool: monitoring engine
Collected metric: test execution time
Priority: medium
Complexity: low
Responsible: PANOS K (SYN)

Simple Service Lifecycle - Multi VIM Network Service with 2 VMs deployed in 2 VIMs

Task list:

• Create SP folder
• Create README
• Create .robot
• Create Jenkins job

test_sdk03_validation

Tool: tng-sdk-validate
Short description: Validate a project before packaging.
Parameters: Size (no. VNF), descriptor complexity (low/high), validation level (s,i,t), custom rules (number of rules)
Collected metric: time to validate, memory used by tool, (detection rate or similar? not sure if this works, to be decided by tool owner)
Priority: high
Complexity: medium
Responsible: Miguel, Ana (Quobis)

---

Add package NSID3V (scalable NS)

Here I will need help!

Slice - Share Network Service 3.2.2

The shared NSS is a CNF where the slices are (NS1(VNF) - NS2(CNF) - NS3(VNF)) and (NS1(CNF) - NS2(VNF) - NS3(CNF))

Task list:

• Create SP folder
• Create README
• Create .robot
• Create Jenkins job

Create a new freestyle job in Jenkins for test sla 4.3

Develop the Test Cases

test_sdk05_test_packaging

Tool: tng-sdk-test
Short description: How long does test creation for VnV take?
Parameters: -
Collected metric: Packing time
Priority: high
Complexity: low
Responsible: Askhat (IMEC)

---

request_latency_sp_kpi

• endpoint is one of the collected KPIs dimensions, so we can filter by it

Define devops workflow

Complete the information in the wikipage defining the DevOps workflow for the tests in each environment

Manual scaling - 1 CNF 5.2

The manual scaling consists to scale out/in one CNF using the gk.
Task List;

• Create SP folder
• Create README
• Create .robot
• Create Jenkins job

Template .robot: https://github.com/sonata-nfv/tng-tests/wiki/methodology.md#test-case-descriptor
SP folder location: tng-test/SP/test-manual-scale-5-2

Add package NSID4VC: Mixed NS (CNF / VNF)

Create a file .robot with the definition of the sla test

tng-tests documentation

Complete the devops testing methodology @felipevicens plus any additional information. There are some TST folders under the packages folder that are confusing. The wiki page is available.

Simple Service Lifecycle - Multi VIM Network Service with 2 containers deployed in 2 VIMs

Task list:

• Create SP folder
• Create README
• Create .robot
• Create Jenkins job

Create a README.md with the test description

test_sdk01_project_management

Tool: tng-sdk-project
Short description: Create project with different sizes.
Parameters: Number of VNFs (1-1000)
Collected metric: time to create, memory used by tool
Priority: high
Complexity: low
Responsible: Stefan (UPB)

---

**test_sp02_test_service_reconfiguration_(OS)**

Short description: Ensure that a service reconfiguration is triggered and executed correctly based on custom metrics from Openstack.
Tool: monitoring engine, MANO, Policy Manager
Collected metric: test execution time, test instantiation time, test scale time
Priority: high
Complexity: medium
Responsible: PANOS K (SYN), Eleni (UBI), Thomas (IMEC)

Simple Service Lifecycle - NSID1C1V Network service that deploys 1 container in one VIM and one VM in other VIM

Task list:

• Create SP folder
• Create README
• Create .robot
• Create Jenkins job

**test_sp03_test_service_reconfiguration_(k8s)**

Short description: Ensure that a service reconfiguration is triggered and executed correctly based on custom metrics from K8s.
Tool: monitoring engine, MANO, Policy Manager
Collected metric: test execution time, test instantiation time, test scale time
Priority: high
Complexity: medium
Responsible: PANOS K (SYN), Eleni (UBI), Thomas (IMEC)

Graylogs messages in Robot

Add Graylogs error messages as an artifact if the Robot test fails.

Add package NSID1C

**test_sp04_test_migration_of_state (OS)**

Short description: This test ensures that a state migration of a VNF is consistent.
Tool: MANO
Collected metric: test execution time, test instantiation time, test migration time
Priority: medium
Complexity: medium
Responsible: PANOS K (SYN), Eleni (UBI), Thomas (IMEC)