Performance Testing via Artillery.io

Java-based tools can be complicated and time-consuming to work with. Artillery.io is a Node.js tool with a basicСLI, which is a far simpler way to run performance tests.

Out of the box, Artillery.io will support:

However, you can use Artillery.io to test any stack, thanks to the plugin interface. Here are a few examples:

Unlike popular competitors’ testing tools, Artillery.io writes tests in YAML format which makes them readable and easy to understand. Artillery also supports a JSON format for testing.

With Artillery’s scenarios and phases, you can build more flexible tests and cover multiple use cases in parallel. Also, it takes you just a few steps and conditions to set up tests for complex user behaviours. Finally, it’s easy to reuse scenarios for comparison and future testing.

Artillery.io seamlessly integrates with CI (for example, check the integration guide for GitHub Actions), making the process very organized. It also works with Docker, so you can run your Artillery tests inside a Docker container.

After running tests, you can create a JSON report through the --output flag. Also, you can generate an HTML report out of JSON to save it as a file or export it to your browser. Find details on reporting in the Scenario section below.

Artillery provides numerous documents to browse for instructions and other useful info. This helps you get the performance testing software up and running much smoother than with other similar tools. Also, you can look up support from other developers on community forums.

To install the latest version, enter:

To run a fast test, enter:

With this command, the test will run 15 virtual users with each of them making 30 HTTP GET requests to the URL address that you specified.

This command also supports other flags to configure your fast tests:

Below is a list of possible configurations of the run command:

For standard test scenarios, you should create a ‘my.yml’ file with config.

The my.yml file should contain ‘config:’ paired with the target URL. To set up environments for your tests, you can write environments into the command after the target URL has been entered.

Artillery.io allows you to enter multiple sequential load options for the application. The testing phase consists of several aspects:

If you only have one target URL, the different phases of the performance testing are placed right after it. If there are multiple environments, then you should add phases to each environment variable.

In the Artillery npm utility, you can find lots of plugins that can help you in your performance testing. Install artillery-plugin-expect to compare the expected result with the actually received result.

Then, after the phases inside the config, enter:

Now, with the config completed, we can finally write some tests!

All tests should be written in the scenarios section and should contain:

To run tests, you should enter:

To run a test and then generate a report in a .txt file, run:

After generating a .txt report, to generate a second report in HTML, run the command:

Here’s the report example:

With Artillery.io, you can use basic authentication or get authorized by tokens uploading CSV with your credentials. Just pick an option that suits your project best.

Check their official documentation to find information on authentication and many other aspects of Artillery.io.

Let’s first revise how you can get authorized by tokens. To upload a CSV file with credentials, you should add the following lines into the config file:

See an example with a CSV file:

The second way to get authorized is through environment variables. And we have several options for this. The first one is to execute an export command in your console as below:

And after it’s done, you can use your variable inside your yml file:

Another option is to write a new script with an environment variable in your package.json file to execute your tests without typing the export variable in the terminal each time. It will look like this:

Then all you need to do is run this script in the terminal:

Finally, you can upload CSV to pass parameters for your test requests: GET, POST, PUT, etc.

Check an example with POST: