Melis Okanovic

Part 2: UI Test Automation in Elixir with Wallaby Wallaby is a functional testing library for Phoenix and Elixir web applications. It allows you to write and run automated tests for your application's UI, ensuring that it works as expected. With Wallaby, you can test the interactions and behaviors of your application's UI components, including buttons, links, forms, and more. It is easy to use and integrates well with other testing tools, making it an excellent choice for testing UI in Elixir applications. Setting Up Wallaby for Your Elixir Application Setting up Wallaby for your Elixir application is straightforward. First, you need to add Wallaby to your project's dependencies in your mix.exs file. Next, you need to create a test/support/conn_case.ex file, which will contain the setup and teardown code for your tests. Finally, you need to create a test file in the test directory and start writing your tests. def deps do [ {:wallaby, "~> 0.30.0", runtime: false, only: :test} ] end Unlike the cabbage dependency from earlier, the only difference is runtime: false. The runtime: false option specifies that this dependency should only be used during the test environment and not during runtime. This is because wallaby is a testing tool, and should not be included in the application's runtime environment. Configure the driver: # Chrome - default config :wallaby, driver: Wallaby.Chrome, screenshot_on_failure: true, otp_app: :hospital_management, screenshot_dir: "test/screenshots/" Wallaby.Chrome - sets the default driver, which means that it will use the Chrome web browser to interact with the application being tested. screenshot_on_failure: true - specifies that a screenshot of the browser window should be taken and saved to disk when a test fails. otp_app: :hospital_management - specifies the OTP application name for the project, which is used to find the config.exs file for the project. screenshot_dir: "test/screenshots/" - specifies the directory where the screenshots should be saved. Then ensure that Wallaby is started in your test_helper.exs: {:ok, _} = Application.ensure_all_started(:wallaby) This code uses the Application module in Elixir to ensure that the :wallaby OTP application is started before continuing execution. The :wallaby application is responsible for starting the browser driver that will be used by Wallaby to simulate user interactions with a web application. The Application.ensure_all_started/1 function is called with the argument :wallaby, which is the name of the OTP application to be started. If the application is already running, this function has no effect, but if it is not running, it will be started. The function returns a tuple containing either {:ok, pid} or {:error, reason} depending on whether the application was successfully started or not. In this case, the code is using pattern matching to capture only the :ok atom from the tuple and ignores the second element of the tuple. If the :wallaby application fails to start for any reason, the function will return a tuple with the :error atom as the first element, which will cause the pattern matching to fail and an exception will be raised. This is why it's important to use pattern matching, in this case, to ensure that the application was started successfully before continuing execution. Make sure that the sandbox is enabled: config :hospital_management_web, :sandbox, Ecto.Adapters.SQL.Sandbox A sandbox is a way to create a temporary database environment for testing purposes. In this case, the Ecto.Adapters.SQL.Sandbox module is being used to set up the sandbox. The config/3 function from the Kernel module is being called to configure the :sandbox environment for the :hospital_management_web application. The configuration options for the sandbox can be specified as a keyword list. In this case, the only option being set is Ecto.Adapters.SQL.Sandbox, which tells Ecto to use the Ecto.Adapters.SQL.Sandbox module to create the sandbox. The :sandbox environment is typically used for testing, so this configuration will be included in a config/test.exs file for the application. Finally, in your test_helper.exs you can provide some configuration to Wallaby. At minimum, you need to specify a :base_url, so Wallaby knows how to resolve relative paths. Application.put_env(:wallaby, :base_url, HospitalManagementWeb.Endpoint.url) This is setting a configuration option for the Wallaby testing framework in the current Elixir application. Specifically, it uses the Application.put_env/3 function to set the :base_url option for the :wallaby application. The :base_url option is used to specify the base URL for the web application that is being tested with Wallaby. In this case, the value being set for the option is the result of calling the HospitalManagementWeb.Endpoint.url function. This function is likely defined in the Phoenix web framework and returns the URL for the current endpoint. By setting the :base_url option for the :wallaby application, Wallaby will use this URL as the base URL for all of its tests. This means that any relative URLs used in the tests will be relative to this base URL. Writing Test Scenarios with Wallaby Writing test scenarios with Wallaby is straightforward. Wallaby supports a wide range of test scenarios, including clicking buttons, filling out forms, and checking that elements are displayed correctly. With Wallaby, you can easily test the interactions and behaviors of your application's UI components, ensuring that they work as expected for your users. In this scenario, the patient is registered, and their access to the UI is achieved using login credentials. The appointment scheduling page will show the fake test data available to the patient to select from. The confirmation of the appointment will be with fake test data, and the patient should receive a confirmation of the appointment details via a mock email or SMS service rather than an actual email or SMS service. Feature: Hospital Management Service This feature describes the capability of the Hospital Management Service to allow patients to schedule appointments via the user interface (UI). Scenario: Schedule an appointment online Given the patient has access to the Hospital Management application When the patient navigates to the appointment scheduling page And selects the date and time for their appointment And selects the doctor they wish to see And confirms the details of their appointment Then the appointment should be successfully scheduled And the patient should receive a confirmation of the appointment details via email or SMS. Let's implement this scenario with Elixir:   defgiven ~r/^the patient has access to the Hospital Management application$/, %{session: session} = _state do # Perform any necessary setup, such as logging in or creating a test patient account session = session |> visit("/") |> fill_in("username", with: "testuser") |> fill_in("password", with: "testpassword") |> click_button("Log In") |> assert_text("Welcome, Test User") {:ok, %{session: session}} end It first navigates to the homepage, fills in the "username" and "password" fields, clicks the "Log In" button, and then checks that the page displays the expected welcome message. Finally, it returns an {:ok, %{session: session}} tuple with the updated session. defwhen ~r/^the patient navigates to the appointment scheduling page$/, %{session: session} = state do session = session |> visit("/appointments/new") |> assert_title("Schedule an Appointment") {:ok, %{session: session}} end In the implementation, the session is first used to visit the "/appointments/new" page, and then assert that the title of the page is "Schedule an Appointment". The assert_title/2 function is used for this purpose, which takes two arguments: the session, and the expected title. Finally, the updated session is returned as part of the response, along with the state, wrapped in a tuple: "{:ok, %{session: session}}". defwhen ~r/^the patient selects the date and time for their appointment$/, %{ appointment_date: appointment_date, appointment_time: appointment_time, session: session } = state do session = session |> fill_in("appointment_date", with: appointment_date) |> fill_in("appointment_time", with: appointment_time) {:ok, %{session: session}} end This step uses the fill_in function to populate the "appointment_date" and "appointment_time" fields on the appointment scheduling form with the values provided in the appointment_date and appointment_time variables. These values were extracted from the Gherkin step using a regex and passed as part of the state. The updated session is returned as part of the state. defwhen ~r/^the patient selects the doctor they wish to see$/, %{doctor_name: doctor_name, session: session} = state do session = session |> select(doctor_name, from: "doctor_id") {:ok, %{session: session}} end This step is part of an appointment scheduling process, and it handles the action of selecting a doctor that the patient wishes to see. The doctor_name and session are passed as parameters to this function from the previous step. The doctor_name is the name of the doctor that the patient wishes to see, and it is used to select the corresponding doctor from a dropdown menu using the select function provided by Wallaby. The from parameter is used to identify the dropdown menu element where the doctor's name is selected. The updated session with the selected doctor is returned as part of the state in the response. defwhen ~r/^the patient confirms the details of their appointment$/, %{session: session} = state do session = session |> click_button("Confirm Appointment") |> assert_text("Appointment scheduled successfully.") {:ok, %{session: session}} end This step corresponds to the "And confirms the details of their appointment" step. In this step, we are simulating the patient confirming the details of their appointment by clicking on the "Confirm Appointment" button in the UI. After clicking the button, we assert that the confirmation message "Appointment scheduled successfully" is displayed in the UI by calling the assert_text function. The session parameter is passed along with the state, and we are returning the updated state containing the updated session value. defthen ~r/^the appointment should be successfully scheduled$/, %{session: session} = state do # Assertion check to verify that the appointment is properly saved and displayed on the patient's appointment list session |> assert_text("Appointment scheduled successfully.") {:ok, %{session: session}} end This step verifies that the appointment scheduling was successful by checking if the text "Appointment scheduled successfully." is present on the page. This assertion confirms that the appointment creation was successful and no error occurred during the process. If the text is not present, the test will fail, and the test runner will display an appropriate error message. defthen ~r/^the appointment should be successfully scheduled$/, %{session: session} = state do # Verify that the appointment is properly saved and displayed on the patient's appointment list assert session |> visit("/appointments") |> has_table_row?(expected_row: ["Doctor Name", "Appointment Date and Time", "Appointment Type"]) {:ok, %{session: session}} end This step verifies that the appointment is properly saved and displayed on the patient's appointment list. It first visits the "/appointments" page, then asserts that a table row with expected values ("Doctor Name", "Appointment Date and Time", "Appointment Type") exists on the page using the has_table_row? function. If the assertion passes, it returns {:ok, %{session: session}} with the updated session. Pros and Cons: Pros: Improved collaboration: Utilizing Gherkin tests for UI testing with Wallaby facilitates better collaboration with non-technical stakeholders through its easy-to-read plain text format. Clear and concise requirements: Gherkin's "Given-When-Then" format in frontend tests provides clear and well-outlined requirements for each scenario, aiding in comprehensive test coverage and understanding. Modular tests: Gherkin tests can be modular in the frontend, allowing for reuse across different UI testing scenarios and projects, resulting in time and effort savings. Supports test automation: Gherkin tests in the frontend can be automated using tools like Cucumber and Wallaby, enabling automated execution of repetitive tests and improved test coverage. Cons: Additional complexity: Implementing Gherkin tests in the frontend can introduce additional complexity, such as defining step definitions, managing feature files, and keeping tests in sync with application changes. Steep learning curve: Writing and maintaining Gherkin tests for frontend/UI testing may require a certain level of technical expertise, which can be a barrier for non-technical team members. Limited flexibility: Frontend Gherkin tests are constrained to the scenarios and steps specified in the feature files, which may limit testing capabilities for certain complex UI scenarios. Maintenance overhead: As the frontend evolves, Gherkin tests must be updated to align with the changes, potentially requiring significant effort and time for extensive frontend applications. Smaller community: Elixir and Wallaby have a smaller community compared to more mainstream programming languages and libraries, which might result in limited support and resources for frontend Gherkin testing. With Wallaby, you can write and run automated tests for your application's UI, ensuring that it works as expected. Wallaby is easy to use, integrates well with other testing tools, and supports a wide range of test scenarios, making it an excellent choice for testing UI in Elixir applications. In summary, implementing Gherkin tests in an Elixir application can bring several benefits such as improved collaboration, clear requirements, and test automation. However, it can also add additional complexity and maintenance overhead and may require a certain level of technical expertise. Ultimately, the decision to use Gherkin with Elixir should be based on the specific needs and requirements of the project. In case you missed part 1 of this blog, you can read it here.

What is Gherkin? An Introduction to Behavior-Driven Development (BDD) with Gherkin In this two-part blog, we will explore the power of Gherkin, a domain-specific language designed for behavior-driven development, and how it can be leveraged in both backend and frontend testing. Gherkin offers a simple and structured syntax that enables teams to describe the behavior of a system in a way that is easily comprehensible to both technical and non-technical stakeholders. It uses keywords like "Given," "When," "Then," and "And" to define different parts of a scenario. "Given" outlines the initial state of the system, "When" describes the action being taken, and "Then" specifies the expected outcome. Additional steps can be added using the "And" keyword. The structure of a Gherkin scenario consists of a Feature, which describes the functionality being tested, and one or more Scenarios that detail specific test cases for that functionality. Each Scenario is defined using Gherkin keywords, making it easy to read and understand. One of the key advantages of Gherkin scenarios is their ability to automatically generate test cases, saving time and increasing efficiency. Their user-friendly format allows everyone on the team to contribute to the testing process, including non-technical stakeholders like business analysts and project managers. Furthermore, Gherkin scenarios can also serve as documentation for the system's behavior, making them valuable for future reference and onboarding new team members. We will delve into the pros and cons of implementing Gherkin tests in both backend and frontend development. Whether you're new to BDD or looking to enhance your testing approach, this blog will provide insights to help you make informed decisions and improve collaboration within your team. Part 1: Backend Test Automation in Elixir There are several libraries available for using Gherkin in different programming languages. In Elixir, one of the most popular libraries for Gherkin is Cabbage. Cabbage is similar to Cucumber, a well-known Gherkin library in the Ruby world, and provides the necessary tools to parse Gherkin feature files, execute the scenarios, and run the tests. With Cabbage, you can write feature files using Gherkin syntax and then define step definitions in Elixir to implement the scenarios. One of the advantages of using Cabbage is its integration with other testing frameworks such as ExUnit and Erlang's Common Test. This allows you to easily incorporate Gherkin scenarios into your existing test suite and run them alongside other tests. Another benefit of using Cabbage is its flexibility in defining step definitions. You can define step definitions using regular expressions or functions, which gives you the ability to write step definitions that are more concise and expressive. However, it's worth noting that Cabbage is not the only library available for using Gherkin in Elixir. Other options include Hound and BDDex, which provide similar functionality for working with Gherkin feature files in Elixir projects. Ultimately, the choice of the library will depend on the specific needs of your project and the preferences of your development team. In this backend test automation approach, we will rely on the utilization of factory methods to create consistent and reliable test data. By leveraging these factory methods, we can ensure that our tests are executed on the application level, thoroughly validating the behavior and functionality of our backend systems. This allows us to simulate real-world scenarios and ensure the robustness of our applications. We'll explore the utilization of Gherkin scenarios with the Cabbage library, revealing how they can enhance the efficiency and reliability of your testing process, all while harnessing the capabilities of the Elixir programming language. Using Gherkin with Cabbage in an Elixir project is relatively straightforward. Here is an example of how you can set up and use Gherkin with Cabbage in an Elixir project: 1. Add Cabbage as a dependency in your project: You will need to add Cabbage to your list of dependencies in the mix.exs file. def deps do [{:cabbage, "~> 0.3.0", only: :test}] end :cabbage is the name of the dependency. This is the name that will be used to refer to the dependency in the project code. "~> 0.3.0" is a version requirement for the dependency. It specifies that the project requires a version of cabbage that is greater than or equal to 0.3.0, but less than 0.4.0. The ~> operator is used to specify a compatible version range. only: :test is an option that specifies that the dependency should only be included when running tests. This means that the dependency will not be included in the project when it is compiled or released. 2. Create a features directory: This is where you will store your Gherkin feature files. It is a good practice to keep your feature files organized by creating subdirectories for different features. 3. Write your feature files: Once you have created a Gherkin file using a ".feature" extension, you can start writing scenarios. Each scenario should start with the Scenario keyword, followed by a descriptive name for the scenario. The purpose of the scenario is to describe a particular feature or functionality being tested. Within the scenario, you can use the Given-When-Then keywords to define the preconditions, actions, and expected outcomes for the scenario. In this example, we can see the 'Hospital Management Service' Feature with the scenario of adding a patient to the system. Feature: Hospital Management Service The Hospital Management Service feature allows users to easily add new patients to the system, providing essential information such as name, age, gender, address, and contact details. The system securely saves the patient's record in the database and returns a success message to confirm the successful addition. This feature streamlines patient management and enhances overall operational efficiency in the hospital. Scenario: Register a new patient in the hospital management system Given the following list of patients and their information | Name | Age | Gender | Address | Contact | | John Doe | 32 | Male | 123 Main St, Anytown USA | 555-555-5555 | | Jane Doe | 28 | Female | 456 Park Ave, Anytown USA | 555-555-5556 | | Bob Smith | 40 | Male | 789 Elm St, Anytown USA | 555-555-5557 | When a hospital worker submits a request for adding 'John Doe, Jane Doe, Bob Smith' as a list of patients Then users should be registered in the Hospital Management System And there should be a message indicating that 'Patients were successfully added' Given - all prerequisites for the test should be satisfied, in this case, we have all the necessary information that the system needs to enter patients. When -  it is used to initiate an action, in our case the user initiates saving the user to the hospital system. Then - the final step serves to verify the desired test output. A success message is returned by the system that the patients were successfully saved. And - in this case is actually an additional Then step, which performs the second test output verification, in our case the users were successfully saved to the hospital database. 4. Write step definitions: Match the steps in your feature files and define the actions that should be taken when each step is executed. In the following example, steps will be written in Elixir.  defgiven ~r/^the following list of patients and their information$/, %{table: patient_info}, %{context: context} = state do patient_info = patient_info |> Enum.map(fn row -> row |> Map.take(["Name", "Age", "Gender", "Address", "Contact"]) |> Map.new() end) context = context |> Context.set_patient_info(patient_info) {:ok, %{context: context}} end This step definition takes a table of patient information and stores it in the context. The table is transformed into a list of maps with only the necessary fields, and the resulting patient list is stored in the Context using the Context.set_patient_info function. The Context is used to store the state of the operation being performed, which is being updated as various functions are called, and operations are performed on the schedule. defwhen ~r/^a hospital worker submits a request for adding 'John Doe, Jane Doe, Bob Smith' as a list of patients$/, %{context: context} = state do {:ok, patient} = Context.get_first_patient(context) response = HospitalManagement.add_patient_to_system(patient) context = context |> Context.set_last_response(response) {:ok, %{context: context}} end This step definition simulates sending a request to add the patient to the system using the HospitalManagement.add_patient_to_system function. It gets the first patient from the patient list stored in the context using the Context.get_first_patient function. The resulting response is stored in the context using the Context.set_last_response function. defthen ~r/^users should be registered in the Hospital Management System$/, %{context: context} = state do response = Context.get_last_response(context) assert response.status == :ok assert response.message == "Patient added successfully." {:ok, %{context: context}} end This step definition asserts that the last response received from the system has a status of :ok and a message of "Patient added successfully." It uses Context.get_last_response function to get the last response stored in the context. defthen ~r/^there should be a message indicating that 'Patients were successfully added'$/, %{context: context} = state do {:ok, patient} = Context.get_first_patient(context) assert HospitalManagement.patient_exists_in_database?(patient) {:ok, %{context: context}} end This step definition asserts that the patient's information has been saved to the database using the HospitalManagement.patient_exists_in_database function. It gets the first patient from the patient list stored in the context using the Context.get_first_patient function. Pros and Cons: Pros: Improved collaboration: Implementing Gherkin tests in the backend allows for better collaboration with non-technical stakeholders, such as business analysts and project managers, as the plain text format is easy to read and understand. Clear and concise requirements: Gherkin's "Given-When-Then" format helps to clearly outline the requirements for each scenario in the backend. This ensures that all necessary scenarios are covered and easy to comprehend and maintain. Modular tests: Gherkin tests can be modular in the backend, allowing for reuse across different applications and environments, and saving time and effort when testing similar functionalities in various projects. Supports test automation: Backend Gherkin tests can be automated using tools like Cucumber and ExUnit, enabling efficient execution of repetitive tests, reducing human error, and improving test coverage. Cons: Additional complexity: The implementation of Gherkin tests in the backend can introduce extra complexity, requiring the definition of step definitions, maintenance of feature files, and alignment with application changes. Steep learning curve: Writing and maintaining Gherkin tests for the backend may demand a certain level of technical expertise, which could be challenging for non-technical stakeholders unfamiliar with these tools. Limited flexibility: Backend Gherkin tests are constrained to the scenarios and steps defined in the feature files, potentially making it difficult to test complex scenarios that do not fit the step-by-step format. Maintenance overhead: As the backend application evolves, Gherkin tests must be updated accordingly, which can be time-consuming, especially for large and complex applications.

What Is Jest? Jest is an open JavaScript testing library from Facebook. It is mainly used for white box (unit/integration) testing purposes, but it can also be utilized for black box testing techniques (API testing, UI testing..etc.). It has good cross-browser compatibility and is widely used with Selenium for automated testing. It has recently gained much popularity for both front-end and back-end testing. Jest is essentially a framework rather than a library. There's even a command-line interface (CLI) tool available. For instance, you can use the CLI tool to execute only those tests that meet a pattern. Aside from that, it has many more features, which you may read about in the CLI documentation.  This means that Jest offers a test runner, assertion library, CLI tool, and great support for different mocking techniques.  Describe Blocks A test suite is defined via a describe block. A test suite is a collection of one or more tests that are functionally related. describe('verify something', () => { //your code here }); The describe block is used to group test cases into logical categories. For instance, suppose we wish to collect all of the tests for a particular class. New described blocks can be nested within an existing described block. To keep the example going, you can add a describe  block that wraps all tests for a specific function in this class. "It" or "Test" Tests A single test is described by an it block. A test is a single functional unit that you want to put under inspection. In addition, the test keyword is used to begin a new test case specification. It keyword is just another way of saying test, I prefer to use it since it provides for a more natural linguistic flow when developing tests.  it('Validate test', async() => { //your test here }); Using it block, we can separate the various tests used in the described block that are part of one test suite/test scenario. In this way, we have a better insight into the running tests, and in the test report itself, we have an overview of which part of the test failed. Assertions/Matchers Let's have a look at the matchers that Jest reveals next. We use expect keyword to make an assertion. We want to compare the result of our test to a value that we anticipate the function returning. it('Validate test', async() => { expect(locator()).toBe('available'); }); With the expect() keyword, as the name suggests, we expect a certain output to match the desired one. We use expect in combination with matcher, and general syntax would be expect(actualValue).matcher(expectedValue) Setup and Teardown It's critical that we know how to prepare for and clean up a test. Let's say we use a database, for example, in one of our tests. We don't want to run a function for each test to set up and clean up the database. To avoid code duplication, we can use the beforeAll and afterAll hooks to fix this problem. You can use both functions to run logic before or after each test suite. Running functions before and after the test can be helpful in automated testing, from loading data from the database to complex setup scripts, as well as cleaning up after tests. The beforeAll and afterAll hooks enable this. With beforeAll , we can predefine all the data we need in the test suite and load it before starting tests. With the help of afterAll , we can write functions that will be executed after all tests, in most cases, it is used to clean (delete) the data used in the tests.  Similar to beforeAll and afterAll , we also have beforeEach and afterEach The main difference compared to beforeAll and afterAll is that these hooks are used when we want to run specific functions before and after each test within the test suite. Unlike beforeAll and afterAll , which are executed once before the tests and once after all the tests in the test suite, these hooks are run multiple times, depending on the number of tests. It is important to note that the functionality of beforeEach and afterEach also depends on their place in the test suite itself. If, for example, we set beforeEach before the Describe block, that hook will be executed once before each Describe block, if we put it inside the describe block, before the It block, beforeEach will be executed before each it block independently. Finally, besides using  beforeAll / beforeEach, afterAll/ AfterEach blocks, we can write our custom setup and teardown scripts which will be executed before all and after all test suites we have.  We can control the order of their execution in various ways, one of them is by defining the npm script (package.json), which defines the order. These scripts are used in beforeAll and afterAll hooks. This allows us to place the desired data in a database we will use later in the defined tests. In this way, we can also load users into the database via APIs and all the data we want to test. This avoids unnecessary duplication of processes we would have to define for each test suite. For example, if we have a set of users who use certain data in the system, we can perform different tests for the same users without loading them every time the test suite is run. This is especially useful in sequential execution. Testing asynchronous code To test asynchronous code, we can use async and await functions. Use the async keyword in front of the function supplied to it to create an async test. it('Validate test, async() => { let response = await api.response(); expect(response).toEqual(userData); }); Jest has all the building blocks needed to ensure you can quickly and efficiently write your test specs, no matter what type of testing you are doing. If you liked this blog, read more about QA/Test Automation.