Playwright (playwright github) is a framework for web test automation developed and open-sourced by Microsoft. Is this a good option to consider for your project test automation? To answer this question correctly we should try to answer what is a good automation tool?
Professionals discussing this topic use several metrics, I would stick to the following Reliability, Ease of use, Parallelism, Maintenance, Reporting and Integration. Let’s discuss them wrt to Playwright.
Nothing is more painful than an unreliable automation framework. It doesn’t matter if the tool is AI ML enabled, super expensive, has 24-hour support, has a great function library, amazing execution speed, or grid support.
If the framework does not provide consistent results for the test execution, it is useless, as simple as that.
How can we assess the reliability of a tool? The best approach is to pick the tool and script a few complex flows from your project. Run them quite a number of times in different environments if possible. Finally, compare the tool-related failures like not being able to find locators, timing issues, unexpected slowness during the execution, etc
A framework which supports quick script recording, code generation, parametrisation, test case scripting in new age languages, and flexibility to troubleshoot the failures can be said to be easy to use.
Playwright has a good codegen, Inspector and Traceviewer which aids in quick test creation, locator assistance and troubleshooting.
However IMO for serious test automation, we rarely use record and play and generally make use of a design pattern along with a Page Object Model (POM) based test automation framework.
We need to run the cases reliably but we also want to see the results quickly. A good automation framework should have the capability to run cases in parallel, same case in parallel, across suites, etc.
Playwright supports parallel execution in a headless and head mode. It also supports connecting to a Selenium grid.
Though Playwright has features to generate CI reports in dot, line, list mode, junit and HTML reports, I still see a need for more inbuilt reporting. But it doesn’t prevent implementing custom reporting making use of Playwright’s reporting library.
5. Integration
Playwright supports good integration and provides an option to make use of NVM packages. We use the library for linting, connecting to CICD, pulling cases from our test management tool for execution, pushing reports to our test management tool, etc.
6. Maintainable
Maintainability depends on how we architect our test automation framework. A POM based is generally easy to maintain as the locators and other files are centralised. It also supports a high reusability of the UI behaviours.
We found Playwright finding locators pretty well, faster and reliably than other tools (Selenium, Mabl) that we tried.
