Setting up a test lab can make both automated and manual testing go more smoothly. The physical setup of a test lab is important to the lab's function in the testing process. This document is a guide to setting up the computers, cables, and other parts that you need to create a test lab for use with Eggplant Automation Cloud.
What Is a Test Lab?
A test lab (or device lab) is a central set of test devices and environments that can be easily accessed by testers. A test lab has many benefits:
- Ensures that testers have access to the devices they need. For example, new models are typically added to the lab faster than they would be procured by individual testers.
- Ensures that devices are "clean", have the correct versions of operating systems and browsers, and have the correct version of the application under test.
- Reduces the total number of devices required. For example, for old models that are only needed occasionally for defect validation, only one device would be needed in the test lab to make the device available to all testers.
- Reduces the effort of test device management.
Eggplant Automation Cloud is a software product that lets you quickly set up a centralized test lab of real devices that your automation and manual testers can connect to. Eggplant Automation Cloud is a private device cloud solution, inside your firewall and under your control, and can include any mobile, desktop, or server test devices that you need – from iOS and Android mobile devices, to Windows laptops running Chrome or Firefox, to Linux database servers.
Note that this document assumes you are familiar with Eggplant Automation Cloud and are now considering the physical setup of your test lab. If you would like to learn more about Eggplant Automation Cloud, see the Eggplant Automation Cloud documentation or contact us.
Logical Architecture of the Eggplant Automation Cloud Test Lab
It is important to understand the logical architecture and operation of an Eggplant Automation Cloud test lab before planning the physical setup. The architecture is shown in the diagram below.
At a high level, the test client (e.g., Eggplant Functional) connects to the Eggplant Automation Cloud server, which then forwards this connection to the appropriate test device. If the test device is a desktop, then Eggplant Automation Cloud forwards the connection directly to that computer (which must be running the appropriate agent for the test client to connect to). If the test device is a mobile device, then the test client connects to a "gateway" machine ("Host computer for mobile" in the diagram) to which the test device is tethered. The "gateway" software installed on the "gateway" machine interacts directly with the test device.
Physical Setup
The key steps to set up an Eggplant Automation Cloud test lab are:
- Install Eggplant Automation Cloud on a server.
- Configure your network so that clients (testers with Eggplant Functional or Eggplant Manual) can connect to Eggplant Automation Cloud.
- Connect your non-mobile test devices to Eggplant Automation Cloud. This means making them accessible via a network from the Eggplant Automation Cloud server and then adding them to the Eggplant Automation Cloud configuration.
- Connect your mobile test devices to Eggplant Automation Cloud. This involves:
- Setting up the gateway machines with the gateway software.
- Making the gateway machines accessible via a network from the Eggplant Automation Cloud server.
- Tethering the mobile test devices to the gateway machines.
- Adding the devices to the Eggplant Automation Cloud configuration.
The key hardware questions when setting up the Eggplant Automation Cloud test lab are:
- Which devices should I have in the test lab?
- What kind of gateway machines should I have? How many devices can be tethered to each gateway machine?
- How should the devices physically be stored?
The following sections answer these questions.
Device Selection
The answer to "Which devices?" can change rapidly. It depends somewhat on your geography and sector (especially the priority of different devices), but typically teams need a mix of Apple and Android devices, with a variety of hardware models, OS versions, and browser versions. Several websites exist which can give you information about the current market share of different device configurations, and therefore which configurations you need to include in your test lab to get appropriate coverage. For example:
Gateway Machines
Mobile devices must be tethered to a gateway machine. This is because getting the level of control needed to remotely test a mobile device can only be done via USB. More specifically, it cannot be done via an agent running on the mobile device itself. So your device lab needs gateway machines; but what kind of machines and how many?
Android devices (and most other devices) can be tethered to a PC (Windows or Linux) or Mac. Apple devices, however, can only be reliably tethered to a Mac. Since every test team needs to be able to test both Android and Apple devices, and teams want to use the same type of machine for all gateways, gateway machines must be Macs. There are two options—a Mac mini or a Mac Pro.
A Mac mini can typically have up to 6 test devices tethered to it. A Mac mini has 4 USB ports, so a powered USB hub is used to allow 6 tethered test devices. Note that the USB hub must be USB-2 (or higher) and be a powered USB hub.
A Mac Pro can typically have up to 18 test devices tethered to it. Again, powered USB-2+ hubs are used to increase the ‘fan in’ of the Mac Pro, and these USB-2 hubs are connected to the Thunderbolt ports on the Mac Pro via a Thunderbolt-to-USB adapter. Note that a Mac Pro has 3 separate USB buses. The 18 devices should be spread evenly across these buses (i.e. 6 per USB bus).
When we talk about how many test devices can be tethered to a single gateway machine, we are really talking about the number of test devices that can be tested simultaneously via a single gateway machine. Test devices that are not being used are irrelevant. Typically we must assume that all test devices tethered to a gateway machine might be used at the same time, but if you know that only some of these devices can be used at the same time, then you can increase the number of devices proportionally. For example, if you know that only half the test devices connected to a Mac Pro might be used at the same time, then you could tether ABC*2 test devices to that gateway machine.
Note that we have made some usage assumptions when measuring these figures. Certain kinds of tests use more of the USB bandwidth (e.g., for video streaming), and so if these tests are commonly running on several devices then fewer devices might have to be connected to the gateway; similarly some tests use less of the USB bandwidth (e.g. slow forms) and so in such cases more devices might be able to be connected to the gateway.
Physical Storage
Physical storage is a key consideration. There are several options depending on the size of your lab and your requirements.
Magnets
Magnets can be used to attach test devices to a wall or board.
They key advantage of this approach is that all the devices can be easily be watched while tests are executing, and it is tidy, which is why it is TestPlant’s recommended approach for small test labs. The downside of the approach is that it does not scale well to test labs with 20+ devices (and a usable wall must be available).
The picture below shows a typical small test lab with 9 mobile devices (and two watches) connected to 4 eggBoxes.
Device Rack
A device rack is a more scalable approach that can be used for test labs with 10s and 100s of test devices. Typically the gateway machines would also be placed inside the device rack.
Note that the built-in USB hubs of such racks are typically relatively slow, so it is better to connect the test devices to the gateway machines using the approaches described in Section 3.2.
The picture below shows an example device rack from Tripp-Lite.
There are also products to enable device storage within your existing racks; e.g., the 3U shelf below.
eggCrate
An eggCrate (shown below) is a purpose-built device rack for 10 devices. It can be used for small test labs when magnets are not appropriate.
An eggCrate storage system
Examples of Test Labs
The following examples show the physical setup of two test labs. The size and configuration of each lab reflects the needs of the organizations that built the labs.
Eggplant's Demo Lab
The picture below shows Eggplant’s own demo lab. All of Eggplant’s customer demonstrations and technical support investigations are done using this lab. Specific details:
- Magnets are used to store 9 mobile test devices.
- Test devices are connected to 3 Mac minis. Since there are fewer than 4 test devices per Mac mini, there is no need for any USB-3 hubs.
- Eggplant Automation Cloud runs on the 4th Mac mini.
The test lab setup at Testplant
Large Mobile Test Lab
The picture below shows a large mobile device test lab. Specific details:
- 40 mobile test devices connected to a Mac Pro.
- Device racks from Tripp-Lite.
Eggplantsoftware.com | Documentation Home | User Forums | Support | Copyright © 2022 Eggplant