Table of Contents
Artificial intelligence (AI) has revolutionized how development teams work, and quality assurance (QA) professionals and Product Owners are no exception. In the context of QA in the age of AI, tasks such as creating user stories, defining use cases, and generating test cases, which were once tedious and manual processes, can now be automated. AI completes these tasks with enhanced consistency and accuracy in just a few seconds.
In this article, I will share my experience in how, with minimal training, AI has successfully automated the generation of use cases from user stories and the creation of manual tests in Xray for Jira. Additionally, we will explore the potential of QA in the age of AI and the next steps for adopting this technology in the short term.
QA in The Age of AI: Writing Manual Tests
It all started with the need to implement a manual test library that would offer traceability, maintainability, and scalability. Up to that point, I found many user stories that lacked well-defined use cases or only contained basic listings within their description.
The tool of choice was Xray, a Jira plugin that allows you to manage both manual and automatic tests, linking them to stories, epics, and other elements as needed.
If you’ve ever had to write a manual test in Xray or another similar tool, you’ll know that the process is repetitive and tedious, especially when you have to permute case studies or cover flowcharts. It can also be difficult to know where to cut each test since the interesting thing is to be able to reuse them in multiple developments and keep them updated with new functionality changes.
This problem led to debates and complaints within the team, seeing the overload of work involved in embarking from scratch on such an ambitious process, and above all, who would take on the workload? POs? QAs? Was there a better way to do it?
At this point, knowing that Xray, like other testing tools, offers the possibility of massively importing tests and being used to using AI in my day-to-day work, I asked myself: would it be possible to automate the creation of tests from the user story?
QA in The Age of AI: First Steps
After five prompts with ChatGPT, and with the Xray API documentation in hand, I managed to generate a JSON with use cases in Gherkin format, parameterizing iterable fields (such as countries or devices), along with a CSV containing these iterations.
By simply saving these files, Xray was able to import multiple perfectly structured test cases with their iterations. In addition, the Xray API allows sorting these cases by folder, adding tags, linking them to other tickets, and updating existing tests… so it could still improve the result.
To further optimize the process, I asked the AI to automatically generate the JSON with the cases and the CSV with iterations whenever it received a user story.
We are currently developing a system based on AI roles:
- Role PO: Creates detailed and well-structured stories. Keeping always the same format
- QA Role: Receives the story, generates or modifies the tests, and organizes them in X-ray.
The earlier we get into the automation flow, the better the result will be.
In addition, we are exploring a visual interface that allows to review, edit, and discard tests before sending them to Jira/Xray. Also offer the option to give feedback to the AI so we can train it to fine tune the results.
The Next Level: AI Integrated Into the Development Ecosystem
This advance is already a great time saver, but the real revolution will come with the integration of AI’s with access to Jira, Confluence, and Xray. This will allow:
- Build a centralized test library, acting as the source of truth for the product.
- Reduce redundancy, optimizing the number of tests and eliminating unnecessary cases.
- Automatically detect which regression tests are needed based on code changes.
- Implement BDD in the project with minimum effort.
How Far Do We Want to Go?
AI is changing the way QA’s work, freeing us from mechanical tasks and allowing us to focus on strategy and quality. The ultimate vision in QA in the age of AI is that of a semi-automated system, where we simply indicate a need, and AI, with full context, generates:
- The user story and acceptance criteria.
- The use cases and test cases.
- The test set design and regression updates.
All of that under the light supervision of the people who would traditionally do this work manually, QA’s and PO’s. It is after all a matter of stopping being pawns and starting to be supervisors.
Links of interest and example of test generation given 2 prompts:
PROMPTS:
Request 1:
Given this user story:
Description
AS a user of a digital calculator
I WANT to perform basic operations such as addition, subtraction, and multiplication.
TO get accurate results for my daily calculations.
Context
The calculator should allow users to enter two numbers and select an operation (addition, subtraction or multiplication). The result of the operation must be displayed on the screen. The functionality should be intuitive and accessible to users of all ages.
I need you to generate cases with the format GIVEN WHEN THEN, also some AND if necessary, and put them in the format:
- Test Title 1
GIVEN preconditions
WHEN action 1
AND action 2
THEN result
Response:
Here are some test cases in the requested format:
001. Perform a sum
GIVEN the calculator is on
AND the user has entered the number 5
AND the user has entered the number 3
WHEN the user selects the addition operation
THEN the result shown on the display is 8
002. Performing a subtraction
GIVEN the calculator is turned on
AND the user has entered the number 10
AND the user has entered the number 4
WHEN the user selects the subtraction operation
THEN the result shown on the display is 6
003. Performing a multiplication
GIVEN the calculator is on
AND the user has entered the number 7
AND the user has entered the number 6
WHEN the user selects the multiplication operation
THEN the result shown on the display is 42
004. Handling negative numbers in addition
GIVEN the calculator is turned on
AND the user has entered the number -2
AND the user has entered the number 3
WHEN the user selects the addition operation
THEN the result shown on the display is 1
005. Handling negative numbers in subtraction
GIVEN the calculator is turned on
AND the user has entered the number 5
AND the user has entered the number -3
WHEN the user selects the subtraction operation
THEN the result shown on the display is 8
006. Handling negative numbers in multiplication
GIVEN the calculator is on
AND the user has entered the number -4
AND the user has entered the number 2
WHEN the user selects the multiplication operation
THEN the result shown on the display is -8
These test cases cover the basic operations of addition, subtraction, and multiplication, including the handling of negative numbers.
Request 2:
Now I want you to generate a JSON to be able to import the tests in X-ray. I will give you examples of both structure and format and where we place the given, when then. In this case, I want all cases in the same test with multiple steps.
[
{
“testtype": ‘manual’,
“fields": {
“summary": ‘Brief description of test’,
“project": { ‘key’: ‘ONLINE’ }
},
“steps": [
{
“action": ‘*Scenario title* GIVEN initial condition WHEN action performed’,
“data": ‘’,
“result": ”THEN expected result.”
},
{
“action": ‘*Scenario Title 2* GIVEN other initial condition WHEN other action performed’,
“data": ‘’,
“result": ”THEN another expected result.”
}
]
}
]
Result:
Here you have the JSON with all test cases combined into a single test with multiple steps, in the right format for importing into X-ray:
```json
[
{
“testtype": ‘Manual’,
“fields": {
“summary": { ‘calculator basic operations’,
“project": { ‘key’: ‘ONLINE’ }
},
“steps": [
{
“action": ‘*Perform a sum* GIVEN the calculator is on AND the user has entered the number 5 AND the user has entered the number 3 WHEN the user selects the sum operation.’, ‘action’: ‘*Perform a sum* GIVEN the calculator is on AND the user has entered the number 5 AND the user has entered the number 3 WHEN the user selects the sum operation’,
“data": ‘’,
“result": ”THEN the result shown on the display is 8”
},
{
“action": ‘*Perform a subtraction* GIVEN the calculator is on AND the user has entered the number 10 AND the user has entered the number 4 WHEN the user selects the subtraction operation’,
“data": ‘’,
“result": ”THEN the result shown on the display is 6”
},
{
“action": ‘*Perform a multiplication* GIVEN the calculator is on AND the user has entered the number 7 AND the user has entered the number 6 WHEN the user selects the multiplication operation’,
“data": ‘’,
“result": ”THEN the result shown on the display is 42”
},
{
“action": ‘*Negative number handling in addition* GIVEN the calculator is on AND the user has entered the number -2 AND the user has entered the number 3 WHEN the user selects the addition operation’,
“data": ‘’,
“result": ”THEN the result shown on the display is 1”
},
{
“action": ‘*Handling negative numbers in subtraction* GIVEN the calculator is on AND the user has entered the number 5 AND the user has entered the number -3 WHEN the user selects the subtraction operation’,
“data": ‘’,
“result": ”THEN the result shown on the display is 8”
},
{
“action": ‘*Negative number handling in multiplication* GIVEN the calculator is on AND the user has entered the number -4 AND the user has entered the number 2 WHEN the user selects the multiplication operation’,
“data": ‘’,
“result": ”THEN the result shown on the display is -8”
}
]
}
]
```This JSON combines all test cases into a single test, with each step detailing the actions and expected results in the format GIVEN, WHEN, THEN.
Picture 1: Import of the JSON generated by the AI in Xray
Picture 2: Final visualization of the tests in Xray after import
