Unit 13.3: Software Testing

[Audio] Unit 13.3: Software Testing Arnas Budrys - 19130367.

[Audio] Meeting about the DVLA testing: This email shows how I have asked my client about meeting up with me to look at the DVLA program which I have sent to them. This emails shows the reply from the client stating that they will be able to meet up with me on a specified day in the BL01 room..

[Audio] Pie Charts for Pass, Merit & Distinction tests: This page shows all of the Pass, Merit and Distinction Pie charts based on how many tests have passed and how many tests have failed. I have used Excel to make these pie charts by using the Insert, then 3D pie button. I then inserted the data such as 18 for Passed and 2 for failed to insert them within the pie chart to show what the tests have outputted. The reason why there are no skipped tests is because I didn’t skip any and there was only either passed or failed within these pie charts..

[Audio] Section 1 P5: Within the pass criteria of my DLVA test, I have conducted 20 tests, 2 of which had failed meaning that 90% of the tests have passed and 10% of the tests have failed. Moreover, for the P5 section of the software testing some of the tests that I have carried out are testing whether or not the login screen accepts the first and second name, another test which I have conducted is to test whether or not the question screens appear when needed such as on each question screen to ensure that the user is able to answer all of the required questions. These examples of the tests that I have carried out are necessary to ensure that the fundamental aspect of the program works as initially intended to meet with the user requirements which is to make sure that the program works well enough to able to at least submit the final score into a database and to be able to input your full name alongside being able to answer all of the necessary questions. Although, there were only some testing methodologies used such as Functional testing, performance testing, and unit testing etc, there were still more that could’ve been used to enhance the testing of the software itself, the way it would've enhanced the software is that it would’ve led to more tests being carried out which would ensure that more errors and bugs would be found and therefore fixed. The following slides will show all of the passed and failed tests, which is shown to tell you what errors I had found and what is needed to fix them..

[Audio] Passed Tests for Pass section (P5). Functional Test 1: This test verifies the functionality of the first name and second name text boxes. By inputting "Bob" into the forename box and "Ross" into the surname box, the program should accept these inputs as they contain only alphabetic characters. Additionally, the program should generate a new username by combining the first three letters of the surname and the first letter of the forename. Functional Test 2: This test examines the behaviour of the forename and surname boxes when hyphens are included alongside the names. Inputting "- " alongside a name should be allowed, according to the program specifications, and should permit the user to proceed to the next step. System Test 1: Focused on the first name box, this test involves inputting "Bob1" to assess the program's response. The program should immediately notify the user that only letters, spaces, hyphens, apostrophes, and backspace keys are permissible, disallowing the input of numbers. System Test 2: Similar to System Test 1, this test targets the second name box. Inputting "Ross#" into the surname box should prompt a message indicating that only specific characters (letters, spaces, hyphens, apostrophes, and backspace keys) are allowed, rejecting special characters. Unit Test 1: This test ensures that the first question screen appears promptly after inserting the forename and surname, immediately following the login screen. Clicking "Start Test" should initiate the DVLA exam and display the first question for the user to answer. Unit Test 2: Following the completion of the first question, this test confirms the appearance of the second question screen upon clicking "Next Question" on the first question screen. This sequential progression illustrates the proper functioning of the test interface..

[Audio] Unit Test 3: This test confirms the appearance of the third question screen after the second question screen. By clicking "Next Question" on the second question screen, the user should be directed to the third question screen. This sequence ensures the proper flow of questions within the test. Unit Test 4: Following the completion of the third question, this test validates the appearance of the fourth question screen upon clicking "Next Question" on the third question screen. The fourth question screen should be displayed, maintaining the sequential progression of the test. Unit Test 5: After completing the fourth question, this test ensures the fifth question screen appears upon clicking "Next Question" on the fourth question screen. The fifth question screen should be presented to the user, maintaining the sequential flow of questions within the test. Unit Test 6: Upon completing all questions, this test verifies the appearance of the score screen. After the fifth question screen, the score screen should display, prompting the user to click "End Test" to view their final score. Clicking "Finish" on the score screen should lead the user to see their score out of 5 and percentage out of 100, with options to return to the main menu or save their test score. Performance Test 1: This test assesses the performance of the system by verifying the immediate appearance of a pop-up message upon answering the first question correctly. Clicking "Next Question" after answering correctly should trigger a message confirming the correctness of the answer, allowing the user to proceed. Performance Test 2: Similar to Performance Test 1, this test evaluates the system's performance by ensuring the immediate appearance of a pop-up message upon answering the second question correctly. Clicking "Next Question" after answering correctly should trigger a message confirming the correctness of the answer, maintaining the smooth flow of the test..

[Audio] Performance Test 3: This test evaluates the system's performance by verifying the immediate appearance of a pop-up message upon answering the third question correctly. Clicking "Next Question" after answering correctly should prompt a message confirming the correctness of the answer, stating "Well done that is correct," allowing the user to proceed. Performance Test 4: Similar to Performance Test 3, this test assesses the system's performance by ensuring the immediate appearance of a pop-up message upon answering the fourth question correctly. Clicking "Next Question" after answering correctly should trigger a message confirming the correctness of the answer, stating "Well done that is correct," maintaining the smooth flow of the test. Performance Test 5: Following the completion of the fifth question, this test verifies the appearance of a pop-up message upon clicking the "Finished" button on the fifth question screen. The message should confirm the correctness of the answer, stating "Well done that is correct," allowing the user to move on. System Test 1: This test ensures that the final score, consisting of both a score out of 5 and a percentage out of 100, is displayed on the score screen. The user can end the test by clicking "End Test" on the fifth and final score screen page, where the system calculates the number of correct answers and presents them accordingly. System Test 2: This test evaluates the functionality of the system in determining the user's score out of 5, displayed on the score screen. After completing the test, the user can view their score out of 5, which is determined based on the number of correct answers submitted. Functional Test: This test checks the functionality of the "Go to Main Menu" button, ensuring it successfully navigates the user back to the main menu after clicking. Clicking "Go to Main Menu" on the score screen should immediately return the user to the main menu, allowing them to restart the test or exit the DVLA exam entirely..

[Audio] Failed Tests for Pass section: Unit Test: This test focuses on the splash screen functionality of the DVLA program. Upon opening the application, a splash screen should appear for 5 seconds. However, it was observed that the splash screen appeared at the top left-hand side of the screen instead of the middle, indicating a minor error in the program's display. Functional Test: This test examines the functionality of the "Save Test Score" button and its interaction with the local database. Upon reaching the score screen after completing the final question, the user should be able to click "Save Test Score" to store their test data in the local database for future viewing. If an error occurs and the user attempts to click "Save Test Score," an error message should be displayed, preventing the user from saving their score onto the local database..

[Audio] Evaluation: Overall within the Pass section of P3 it was mostly successful as out of 20 tests which were done, only 2 of the tests had failed meaning that it was a major success. Some of the tests that were carried out were Functional, unit, and performance tests such as testing that only characters from A-Z are allowed, and that special characters such as “#” or “@” should not be allowed. Another example, for a failed test this time is that there is an error message when the user tries to click on “Save to Database” after trying to submit their final score and percentage. However, considering these tests, there could've been more testing methods used and even more tests carried out than the 20 that I have done to make sure that the program has been tested to the extreme, instead of only doing a limited amount of tests which could be detrimental to the program itself. Overall I believe that the pass tests for the Pass criteria went successfully as no tests were skipped, and in addition there was only 2 failed tests meaning that most of the program was tuned in regards to fundamental aspects of the program such as the questions screens and the login screen. Moreover, even though there was 2 failed tests, they were not as important as they are insignificant in comparison to other passed tests..

[Audio] Section 2 M3: Within the M3 section which is the Merit criteria, I have additionally also created another 20 tests which will be shown after this slide. The reason these tests were carried out for the merit criteria is because to show more tests that can be included to then be able to improve the overall testing within the entire DVLA program. Moreover, I will once again provide all of the evidence through the screenshots of all the passed tests and failed tests which I carried out to ensure that the DVLA program is as efficient as possible in terms of actually finding problems within the program and then fixing them immediately which would all be explained within the screenshots. Some of the tests that I have carried out in the merit section, are similar in a lot of ways to the pass section. Some of the tests that I have carried out in M3 are: testing what will happen when all of the answers are answered on different question screens as this shouldn’t be possible because most of the questions only require one answer instead of all of them which should come up as an error. Another test is to test if the “QUIT” button works on all of the screens. Lastly, a failed test which I have screenshotted is to test what would happen if a user happened to leave either the First name box or the Surname box empty as the program should only accept letters instead of accepting a blank name. Some of these tests such as the test regarding the user’s first name and surname is a type of validation test meaning that the user should not be able to insert invalid data into these text boxes as doing so should be an error which is why most of these tests were carried out to ensure that the user’s inputs are validated and correct. Lastly, in regards to validation another test that was done to ensure that there is a validation of data is that there should be a pop-up message to alert the user if they have got the answer incorrect and would therefore take off a point in the user’s overall score at the end..

[Audio] Passed Tests for Merit section (M3): Functional Test 1: This test aims to assess the behavior of the DVLA program when a user attempts to select all answer options on the first question page screen. The user should only be allowed to select one answer, and attempting to select all options should be denied. After attempting this action, a message should appear stating "Well done that is correct" to confirm the correct behavior. Functional Test 2: Similar to Functional Test 1, this test focuses on the second question page screen. The user should only be allowed to select one answer, and selecting all options should be denied. After attempting this action, a message should appear confirming the correct behavior. Functional Test 3: Following the pattern of previous tests, this test examines the behavior of the DVLA program on the third question page screen. Again, the user should only be allowed to select one answer, and selecting all options should be denied. After attempting this action, a message should appear indicating the correct behavior. Functional Test 4: This test assesses the behavior of the DVLA program on the fourth question page screen, with a slight variation. While only one answer should be allowed, on this specific question, two answers can be chosen. Attempting to select all options should still be denied, and after attempting this action, a message should confirm the correct behavior. Functional Test 5: This test evaluates the consistency of screen sizes throughout the mock theory test exam. After entering their first and second names in the main menu and starting the test, the user should progress through all questions and check if the screen sizes remain consistent. Any discrepancies, such as a larger screen size on Question 5, should be identified and noted for correction. Functional Test 6: This test focuses on the functionality of the "Quit" button, which should be present on all question screens. The user should be able to click the "Quit" button throughout the test to return to the main menu. However, this action should only navigate the user back to the main menu during the test; on the score screen, clicking "Quit" should allow the user to leave the program entirely. The "Quit" button's presence and functionality should be consistent across all question screens..

[Audio] System Test 1: This test examines the behavior of the DVLA program when the user answers the first question correctly. After entering their first and second names and selecting the correct answer (Option 2), the user should receive an instant pop-up or message confirming that they have answered correctly. This message should prompt them to proceed to the next question. System Test 2: Similar to System Test 1, this test assesses the behavior of the DVLA program when the user answers the second question correctly. After entering their first and second names and selecting the correct answer (Option 4), the user should receive an immediate pop-up or message confirming their correct answer, allowing them to move on to the next question. System Test 3: Continuing the pattern, this test evaluates the DVLA program's behavior when the user answers the third question correctly. After entering their first and second names and selecting the correct answer (Option 1), the user should receive an instant pop-up or message confirming their correct answer, enabling them to proceed to the next question. System Test 4: This test focuses on the behavior of the DVLA program when the user answers the fourth question correctly. After entering their first and second names and selecting the correct answers (Option 2 & 5), the user should receive an immediate pop-up or message confirming their correct answers, allowing them to move on to the next question. System Test 5: This test examines the DVLA program's response when the user answers the first question incorrectly. After selecting incorrect answers (Options 1, 3, & 4), the user should receive a pop-up or message informing them of their incorrect choice and displaying the correct answer (Option 2). Despite the incorrect response, the user should still be able to proceed to the next question. System Test 6: Similar to System Test 5, this test evaluates the DVLA program's behavior when the user answers the second question incorrectly. After selecting incorrect answers (Options 1, 2, & 3), the user should receive a pop-up or message indicating their incorrect choice and displaying the correct answer (Option 4). Despite the mistake, the user should still be able to proceed to the next question..

[Audio] System Test 7: This test assesses the behavior of the DVLA program when the user answers the third question incorrectly. After selecting incorrect answers (Options 2, 3, & 4), the user should receive a pop-up or message indicating their incorrect choices and displaying the correct answer (Option 1). Despite the mistake, the user should still be able to proceed to the next question. System Test 8: Similar to System Test 7, this test evaluates the DVLA program's response when the user answers the fifth question incorrectly. After selecting incorrect answers (Options 1, 2, 3, & 6), the user should receive a pop-up or message informing them of their incorrect choices and displaying the correct answers (Options 3 & 4). Despite the mistake, the user should still be able to proceed to the next question..

[Audio] Failed Tests for Merit Section (M3): Functional Test 1: This test checks the behavior of the DVLA program when the "Surname" box is left empty in the main menu. After running the program and leaving the "Surname" box empty while filling in the "Forename" box, the program should display a message showing only a part of the Forename. However, the error lies in allowing access to the test without filling in both the Forename and Surname. Functional Test 2: Similar to Test 1, this test examines the program's response when the "Forename" box is left empty in the main menu. After running the program and leaving the "Forename" box empty while filling in the "Surname" box, the program should display a message showing only a part of the Surname. Again, the error lies in allowing access to the test without filling in both the Forename and Surname. Functional Test 3: This test involves leaving both the "Forename" and "Surname" boxes empty in the main menu to observe the program's response. After clicking the "Start test" button, a popup message should appear displaying a blank username, indicating that both the Forename and Surname were left empty. The program shouldn't allow proceeding to the test without filling in both boxes. Functional Test 4: This test evaluates the program's behavior when the user attempts to select all answers on the 5th question page screen. Only one answer should be allowed, but on this specific question, two answers are permissible. The program should validate and ensure that only one or two answers are chosen, depending on the question's requirements. System Test 9: This test examines the program's response when the user answers the fifth question correctly. After entering the first and second names, and selecting the correct answers for the fifth question, a popup or message should appear, indicating that the answer is correct. This confirms that the program correctly recognizes the correct answer. System Test 10: This test assesses the program's response when the user answers the fourth question incorrectly. After selecting incorrect answers (Options 1, 3, 4, & 6), a popup or message should appear, informing the user of their incorrect choices and displaying the correct answers (Options 2 & 5). Despite the mistake, the user should still be able to proceed to the next question..

[Audio] Evaluation: Within my overall evaluation regarding the merit criteria of the DVLA program, after carrying out a further 20 tests, 6 of which have failed meaning that 70% of the tests have passed throughout the merit tests in total. Some of the tests which I have conducted in regards to the merit section is to test each question screen to see what will happen if you press a certain amount of answers instead of only selecting one or two as for example the first 3 questions within the DVLA exam, only one answer can be accepted which means that the data of these questions was validated correctly, whereas with the 4th and 5th question screens all of the answers could be chosen instead of only one. However, for the last two questions the answer requires the user to select two answers so there must have been an error within the validation when creating the questions. In addition, some of the tests which I have created are to test whether or not all of the screens including the question screens are same or similar in size as for the 5th question screen the size does appear relatively larger in comparison to other screens which is an error that I have noted down as there can be seen a lot more white background on this screen than on any other which seems unnecessary and also inconsistent. Another test, in this case a failed test which I have thought of is to see what would happen if the user had only put in their first name and left their surname blank and vise versa which when inputted and then starting the exam shouldn’t allow the user to continue however, the error is that the program does in fact let the user continue after leaving either their first name or last name blank. However, to actually improve the overall testing for the merit section of the program I could’ve implemented many more tests whilst also adding more testing methodologies to enhance the effectiveness and efficiency of the tests regarding the DVLA program. Although, what the merit section provides is another 20 tests, there was still a lack of different methodologies such as Acceptance or Usability tests which will appear within the Distinction section later on..

[Audio] Section 3 – D3: This section which is the Distinction criteria is the last part of the testing process in regards to the DVLA program as I have added another 41 following tests in addition to the 40 which I have previously talked about regarding Pass and Merit. What’s different about the Distinction tests is that in addition to Functional, System, Unit and performance tests I have additionally added another two testing methodologies which are Acceptance tests and Usability tests which add more testing capabilities to the entirety of the DVLA program itself. This was due to the fact that now I am able to test more in regards to the end-user which is the person trying out and using the program, and the client who tests out the program even before it being published to users. This was needed to ensure that there wouldn’t be any fatal errors in regards to the software from when the user and client is testing out every single different feature of the DVLA exam such as testing whether or not the database works when trying to input a score and percentage or testing if the question screens work as intended. The reason why Usability and Acceptance tests were used, is firstly because Usability tests ensure that the initial plan for the DVLA program’s functions, features and overall purpose are also similar to what the user would want from the program by looking at how the software works and therefore users would be able to learn what is good and bad from the program by observing if each different feature works as intended. Conversely, in regards to the different pathways of the software program which are carried out to understand what would happen if a user were to input a combination of answers into the software to see what the result of that would be as the intended result could be something different than from the actual result of the answers which were selected. Therefore pathways were introduced into the DVLA exam to ensure that any combination of answers would never result in an error and would either notify the user if they have got a question correct or incorrect. On the other hand, Acceptance testing which was uniquely added into the Distinction section of the tests is needed at the end of all of the tests which was carried out because similarly to the usability tests where it allows the end user or client to gather large amount of information from the DVLA program in terms of what went well and what needs refinements or improvements as the Acceptance testing is a conversation between me and the client/end-user in which they have reviewed having asked them if they are positive or negative about the DVLA program as a whole and whether it meets all of the necessary requirements which were set in the beginning of the project. In conclusion, the acceptance tests refers to how I am able to communicate via e-mail with the client/end-user by providing them with the DVLA program which I have made so that they can comment on it to tell me the positive and negative aspects of the program. Overall, the tests that I have carried out in the Distinction criteria is testing whether or not for all of the question screens that the pathways work correctly and don’t provide the user with any errors in regards to what answers they have chosen, I have also tested acceptance testing to ensure whether the system meets the user requirements which may be to have all of the question.

[Audio] Passed Tests for Distinction section (D3) Functional Test 1: This test checks the scoring mechanism of the DVLA program when specific answers are chosen for each question. The user inputs answers corresponding to the pathway (1, 1, 1, 1, 1), resulting in a score of 1 out of 5 and a percentage of 20%. This confirms that the scoring system is correctly evaluating the user's responses. Functional Test 2: Similar to Test 1, this test evaluates the scoring mechanism when different answers are selected. The user inputs answers corresponding to the pathway (2, 2, 2, 2, 2), resulting in a score of 3 out of 5 and a percentage of 60%. This indicates that the scoring system accurately calculates the user's performance based on their responses. Functional Test 3: This test assesses the scoring mechanism when another set of answers is chosen. The user inputs answers corresponding to the pathway (3, 3, 3, 3, 3), resulting in a score of 4 out of 5 and a percentage of 80%. This confirms that the scoring system appropriately evaluates the user's performance. Functional Test 4: Similar to the previous tests, this test examines the scoring mechanism with a different set of answers. The user inputs answers corresponding to the pathway (4, 4, 4, 4, 4), resulting in a perfect score of 5 out of 5 and a percentage of 100%. This demonstrates that the scoring system accurately reflects the user's performance. Functional Test 5: This test evaluates the scoring mechanism when alternating between two different answers. The user inputs answers corresponding to the pathway (1, 2, 1, 2, 1, 2, 1), resulting in a score of 1 out of 5 and a percentage of 20%. This confirms that the scoring system correctly evaluates the user's performance based on the chosen pathway. Functional Test 6: Similar to Test 5, this test assesses the scoring mechanism with a different alternating pathway. The user inputs answers corresponding to the pathway (2, 1, 2, 1, 2, 1, 2), resulting in a score of 2 out of 5 and a percentage of 40%. This indicates that the scoring system accurately calculates the user's performance based on the chosen pathway..

[Audio] Functional Test 1: In this test, the user inputs answers 3 and 2 for all questions. The expected outcome is a score of 2 out of 5 and a percentage of 40% due to the pattern followed in answering the questions. Functional Test 2: Here, the user inputs answers 2 and 3 for all questions. The expected outcome is a score of 3 out of 5 and a percentage of 60% based on the pattern followed in answering the questions. System Test: In this test, the user inputs all correct answers for each question. The expected outcome is a score of 5 out of 5 and a percentage of 100%. Usability Test 1: This test ensures that users can start the DVLA theory exam program by logging in with their first and second names and clicking "Start Test". The expected outcome is a pop-up message displaying the user's full name and taking them to the first question screen. Usability Test 2: Here, the user begins the DVLA test by entering their name and answering the first question. The expected outcome is smooth navigation to the first question screen and receiving feedback on their answer, followed by the ability to proceed to the next question..

[Audio] Acceptance Test: The acceptance test involves gathering feedback from the end user to assess their overall satisfaction with the DVLA program. This feedback can be collected through various channels like phone, email, or social media. The expected outcome is to receive both positive and negative feedback, addressing any concerns raised by the user to improve the program. Usability Test: In the usability test, the end user begins the DVLA test by entering their name and answering the first question. They should be able to navigate through the test smoothly, receiving feedback on their answers and progressing to the next question. The expected outcome is for the user to successfully complete the test without encountering any major usability issues..

[Audio] Usability Test 1: The end user completes all the questions and reaches the final score screen by clicking the "End Test" button. They should see their score out of 5 and the corresponding percentage. The expected outcome is for the score and percentage to be displayed accurately, showing which questions were answered correctly and incorrectly. Usability Test 2: The end user can quit the DVLA program either from the main menu or the score screen by clicking the "Quit" button. They will then be prompted to confirm by typing "QUIT" and clicking "OK". The expected outcome is for the program to shut down after confirmation, preventing accidental exits..

[Audio] Usability Test 3: The user must locate the "Quit" button on the fifth question screen, usually positioned in the top left corner. Clicking it should instantly navigate the user back to the main menu screen. The expected outcome is for the user to easily find and click the "Quit" button, leading them back to the main menu. Usability Test 4: Upon reaching the final score screen, the user will press the "Return to main menu" button located at the bottom right-hand side of the screen. This action should promptly transport the user back to the main menu screen. The expected outcome is for the user to locate and click the "Return to main menu" button, successfully returning them to the main menu..

[Audio] Failed Tests for Distinction section (D3) Functional Test 5: First, input all necessary answers for the fourth test scenario. Choose the fourth answer for the first three questions, and for questions 4 and 5, select answers 5 and 6. Upon completing all questions, the expected result is a score of 1 out of 5 and a percentage of 20%. However, the actual outcome shows an erroneous score of 6 out of 5 and a percentage of 120%, indicating a bug where selecting the fourth answer for the first three questions adds extra points. Functional Test 6: Input all necessary answers for answers 3 and 4. For questions 1 to 3, select answers 3 and 4 alternately. For questions 4 and 5, select answers 4 and 3 respectively. The expected outcome is a score of 2 out of 5 and a percentage of 40%. However, the actual result shows a score of 6 out of 5 and a percentage of 120%, revealing the same bug observed in the previous test..

[Audio] Functional Test 7: Begin by inputting answers 2 and 1 for questions 1 to 3, and for questions 4 and 5, choose answers 6 and 5 respectively. After logging in with correct first and second names, proceed to answer the questions following the pathway (2,1,2,6,5,6,5). Upon completing the test, the expected outcome is a score of 1 out of 5 and a percentage of 20%. However, the actual result shows a score of 10 out of 5 and a percentage of 200%, indicating a major error. System Test 2: Input answers 3 and 2 for questions 1 to 3, and for questions 4 and 5, choose answers 5 and 6 respectively. After logging in with correct first and second names, proceed to answer the questions following the pathway (3,2,3,5,6,5,6). Upon completing the test, the expected outcome is a score of 0 out of 5 and a percentage of 0%. However, the actual result shows a score of 10 out of 5 and a percentage of 200%, indicating a major error..

[Audio] Usability Test 1: After completing the DVLA mock exam, the user attempts to view their score and percentage through the local database located in the same zip file as the exam. The expected outcome is successful access to the database and viewing of saved scores. However, upon trying to open the database, an error message appears, stating it was created from a previous version of the application, hindering the user's ability to access their scores. Usability Test 2: Upon completing the DVLA mock exam, the user proceeds to the score screen to save their score and percentage to a local database. They click the "Save data" button expecting a prompt confirming successful data saving. However, upon clicking, an error occurs, causing the entire DVLA program to shut down unexpectedly, despite both the program and database being located within the same zip file..

[Audio] Evaluation: As I have previously stated that 41 tests were carried out in the entirety of the Distinction criteria, most of which were in fact only the system tests and usability tests as those were very important when it came to the overall testing of the DVLA program. The reason as to why System tests were so important in the distinction criteria is because they provide the user to use pathways without there being any errors or inconveniences as answering each different question from the 5 question screens in different orders may lead to a much more different outcome than expected and could lead the program to an error. On the other hand, usability tests were used a lot in the Distinction section because they were needed as now I am able to test all of the functionality of the software by observing how the end-user attempts to complete certain tasks on it such as answering each different question and inserting unique characters into the forename and surname boxes. All of this then helped me to understand what was needed to be changed in order to meet the client requirements. Secondly, I have added a final test to the Distinction testing which is the Acceptance test regarding the conversation between me and the client/end-user as I have asked them to review my DVLA exam and to provide me with the necessary information regarding what the program must improve on, how it can be improved and if there are any positive features that they have found to then communicate that to me. In regards to all of the tests that I have implemented, the system tests have shown the most failed test out of all of the methodologies as there had appeared to be a major problem when the user starts to open and reopen the software multiple times to see what the end result would be i.e. the final score and percentage as if the user were to repeat the exam over and over again their scores would add up and overlap with each other which is a major error in the system. An example of this is that the score should be out of 5 but if you were to do the exam again there is a chance you could get 7/5 or 10/5 for example which shouldn’t happen as it goes above 5. Comparing the Distinction section to the Pass and Merit, there is a major improvement to the overall testing and what testing methodologies were used as firstly, the usability tests were carried out to ensure that all of the testing in regards to the functionality, usability etc are working correctly and as intended and that the acceptance testing part of the program was used to be able to communicate with the client and or end-user to acknowledge and faults within the program that need immediate attention to ensure that the user doesn’t face any bugs or glitches that shouldn’t be there such as there being a bigger screen on the 5th question screen which is an error. Lastly, the pathways should work as intended in terms of what the user inputs for their answers in accordance to the questions as if this wasn’t fixed then the user wouldn’t even be able to answer all of the questions. Lastly, what the Distinction section of the software testing has provided me is because I have included way more tests I have then consequently received more results in terms of what the program needs improvements on in regards to the failed test which I have found many of,.

[Audio] Unit Tests The unit tests within my DVLA program were used throughout all of the 81 tests that I have conducted which comparing all of the testing methodologies, unit tests are one of the lesser important tests in regards to the DVLA program as a whole. Some examples of unit tests are: Testing the splash screen which appears on the top left-hand side of the scree instead of the middle which is a minor error. Testing how the program should allow the user to enter both their forename and surname into these text boxes and they would receive a message saying what their new username is by a combination of their first 3 letters of their surname and first letter of their forename. Testing how if you type in any special characters within your first name or second name the program will not allow this as you are only allowed to use letters, spaces, hyphens, apostrophes and backspace keys..

[Audio] Passed Unit Tests Unit Test 1: The first question screen should appear immediately after entering the forename and surname on the login screen. To initiate the DVLA exam, the user must click the "Start Test" button and then proceed to answer the first question presented. Unit Test 2: After completing the first question screen, the user should click the "Next Question" button to progress. This action will lead to the appearance of the second question screen. Unit Test 3: Following the completion of the second question screen, the user should click the "Next Question" button again to proceed. This action will cause the third question screen to appear..

[Audio] Failed Unit Tests The purpose of this test is to verify the functionality of the splash screen in the DVLA program. Open DVLA Program: Open the DVLA program application. Splash Screen Appearance: Upon opening the application, a splash screen should appear on the screen for 5 seconds. Result: The splash screen appeared as expected; however, it was positioned on the top left-hand side of the screen instead of the center, indicating a minor error in the placement..

[Audio] Evaluation Overall there was only 7 Unit tests that were made throughout all of the 81 tests throughout the software testing, 6 of which had passed and therefore 1 had failed. The 6 that have passed were tests that included how there should be a message that should appear showing “Well done that is correct” after answering a question correctly. On the other hand the failed test that had occurred is the splash screen showing incorrectly as it appeared in the top-left hand side of the screen instead of the middle like every other screen such as the question screens appearing correctly..

[Audio] Functional tests Functional tests in terms of the DVLA program as a whole are almost constantly used throughout the software testing as functional tests are needed to ensure that whether or not the functionality aspect of the program works as intended such as if the buttons work like the “Exit” button which should make the user leave the program entirely, or the “Next Question” button which when pressed should allow the user to move onto the next question, so without these buttons the program would not work effectively and the user would struggle to even complete the exam. Some examples of actual functional tests that I have done are: The system will determine how many answers you have gotten right and how many you have gotten wrong and will present this in a score out of 5 and a percentage out of 100%. After the user has clicked on the “Go to Main Menu” it should immediately bring them back to the main menu so they can start the test over again or leave the DVLA exam entirely if they wanted to..

[Audio] Functional Passed Tests This test examines the functionality of the first name and surname text boxes. For the first scenario, the user inputs "Bob" into the forename box and "Ross" into the surname box. The program should accept these inputs as they contain no invalid characters. In the second scenario, the user tests the text boxes by entering a name alongside hyphens. The input includes "-" in both forename and surname fields. The program should permit the use of hyphens in names, allowing users to proceed to the first question. Overall, the DVLA program successfully allows users to input names with hyphens, meeting the intended functionality..

[Audio] More Passed Tests Functional Test - Single Answer Selection: This test examines the behavior when all answers on the 4th question screen are selected, which should be denied as only one answer should be allowed. The user runs the DVLA program, enters their forename and surname, starts the test, and attempts to select all answers on the 4th question screen. Expected behavior dictates that only one answer should be selectable on the 4th question screen. This ensures that the test validates the restriction of selecting only one answer. Upon attempting to select all answers on the 4th question screen, the user should receive a message indicating that only one answer can be chosen. Functional Test - Question Screen Sizes: This test assesses whether all question screen sizes are correct as the user progresses through the mock theory test exam. The user inputs their first and second names in the main menu, starts the test, and progresses through all questions, checking the screen sizes. Expected results require consistent screen sizes throughout all question screens and the score screen. However, upon completing the 4th question and proceeding to the 5th question, the screen appears larger with unnecessary white spaces, indicating an inconsistency in screen size. Functional Test - "Quit" Button: This test evaluates the functionality of the "Quit" button available on all question screens within the DVLA theory test program. The user opens the DVLA program, enters their first and second names, starts the test, and immediately presses the "Quit" button on all question screens. Expected functionality allows the user to click the "Quit" button on each question screen, enabling them to return to the main menu to either restart the exam or rectify any mistakes. The "Quit" button, located in the top left corner of each question screen, serves to facilitate the user's exit from the DVLA exam, providing flexibility to restart or exit the program entirely..

[Audio] Failed Tests Functional Test - Empty Surname Box: This test examines the behavior when leaving the "Surname" box empty within the main menu. The user runs the DVLA program, leaves the "Surname" box empty but fills in the Forename box, and proceeds to start the test. Expected behavior includes receiving a message showing only a small part of the Forename upon starting the test, indicating an error as access to the test should require filling both Forename and Surname. Upon entering the Forename as "Ross" and starting the test, the message incorrectly displays "Hello, your new username is R" which accepts the Forename without requiring the Surname. Functional Test - Empty Forename Box: This test assesses the behavior when leaving the "Forename" box empty within the main menu. The user runs the DVLA program, leaves the "Forename" box empty but fills in the Surname box, and proceeds to start the test. Expected behavior involves receiving a message showing only a small part of the Surname upon starting the test, indicating an error as access to the test should require filling both Forename and Surname. Upon entering the Surname as "Ross" and starting the test, the message incorrectly displays "Hello, your new username is "Ros" which accepts the Surname without requiring the Forename. Functional Test - Empty Forename and Surname Boxes: This test explores the behavior when leaving both the "Forename" and "Surname" boxes empty within the main menu. The user runs the DVLA program, leaves both the "Forename" and "Surname" boxes empty, and clicks on the "Start test" button. Expected behavior includes a popup message displaying a blank username as both the Forename and Surname have been left empty. The DVLA program allows the user to keep both Forename and Surname completely empty, providing a blank username prompt..

[Audio] Evaluation Overall, there were only 14 functional tests that were carried out in the entirety of the software testing, first of all 9 tests had passed and 5 had failed meaning that there were still many errors that I need to address in regards to functional testing within the program. A major reason for so many errors is because of there being poor validation in terms of how it allows the user to proceed without putting their name or only putting either only their forename or surname by itself which is an error. Some of the validation that wasn’t working properly could be seen in the 4th and 5th question screens as the program allows the user to select every single answer when you are only supposed to select 2 answers so that is another validation error. And because you can select all the answers you will get the question correct which shouldn’t happen as you have also selected some wrong answers. However some passed tests that were successful were how The user should only be accepted to answer only 1 answer instead of all of them within the 1st question screen as only one answer is applicable. This therefore will validate and ensure that only 1 answer can be chosen instead of more than one..

[Audio] System Tests System tests are in many ways similar to Functional tests as in how it tests the system in terms of how different components of the software work with each other with should align with the user requirements. Many of the tests that had to do with the system were to do with all of the different pathways which the user had to follow in terms of how there are many possibilities for how you can answer each of the questions in different orders as there are many results for each different pathway which must be configured in a way which will not cause any issues or bugs. Furthermore, some examples of the passed and failed tests in system testing are: The program should immediately present the user with a message box that states that only letters, spaces, hyphens, apostrophes and backspace keys are allowed and cannot accept numbers. The system will determine how many answers you have gotten right and how many you have gotten wrong and will present this in a score out of 5 and a percentage out of 100%. After the user has finally inserted their answer and presses the “Continue” button to proceed, a pop up or message screen should appear very instantly which will tell the user that they have got the answer correctly and so that they can then move onto the next question within the DVLA mock theory program..

[Audio] System Passed Tests System Test - First Name Box (Numeric Input): This test evaluates the behavior of the first name box when numeric input is entered. The user inputs "Bob1" into the first name box. Expected behavior includes the program presenting a message box indicating that only letters, spaces, hyphens, apostrophes, and backspace keys are allowed, and numbers are not accepted. The program correctly restricts the input of numbers in the first name box, displaying a message indicating the allowed characters. System Test - Second Name Box (Special Character Input): This test examines the behavior of the second name box when a special character is entered. The user inputs "Ross#" into the surname box. Expected behavior involves the program presenting a message box indicating that only letters, spaces, hyphens, apostrophes, and backspace keys are allowed, and special characters are not accepted. The program correctly restricts the input of special characters in the second name box, displaying a message indicating the allowed characters..

[Audio] System Test - Third Question Correct Answer Notification: This test verifies the functionality of the system to display a message when the third question is answered correctly. The user starts the DVLA program, enters their first and second name, and answers the third question correctly (Answer 1). Expected behavior includes a popup or message appearing immediately after clicking the "Continue" button, indicating that the answer is correct and allowing the user to proceed to the next question. The screenshot provided confirms that after selecting Option 1 for the third question, the user received a message stating "Well done that is correct!" System Test - Fourth Question Correct Answer Notification: This test ensures the system correctly notifies the user when they answer the fourth question correctly. The user initiates the DVLA program, inputs their first and second name, and selects the correct answers for the fourth question (Options 2 & 5). Expected behavior involves a popup or message appearing promptly upon clicking the "Continue" button, confirming that the answer is correct and enabling the user to progress to the next question. The provided screenshot demonstrates that after choosing Options 2 & 5 for the fourth question, the user received a message indicating "Well done that is correct!".

[Audio] System Failed Tests System Test - Pathway (3,4,3,4,5,4,5): This test ensures the system accurately evaluates the user's answers according to the specified pathway. The user enters their first and second name correctly, then answers the questions following the pathway (3,4,3) for the first three questions and (3,4,3,4,5,4,5) for the subsequent ones. Expected outcome: The system should display a score of 1 out of 5 questions correct and a percentage of 20%. Actual outcome: The user received a score of 8 out of 5 and a percentage of 160%, indicating an error in the system's evaluation. System Test - Pathway (4,3,4,5,4,5,4): This test verifies the system's ability to assess answers according to the specified pathway. The user correctly inputs their first and second name and answers the questions following the pathway (4,3,4) for the first three questions and (4,3,4,5,4,5,4) for the remaining ones. Expected outcome: The system should display a score of 0 out of 5 questions correct and a percentage of 0%. Actual outcome: The user received a score of 8 out of 5 and a percentage of 160%, indicating an error in the system's evaluation. System Test - Pathway (1,2,1,5,6,5,6): This test validates the system's ability to evaluate answers based on the specified pathway. The user correctly inputs their first and second name and answers the questions following the pathway (1,2,1) for the first three questions and (1,2,1,5,6,5,6) for the rest. Expected outcome: The system should display a score of 1 out of 5 questions correct and a percentage of 20%. Actual outcome: The user received a score of 9 out of 5 and a percentage of 180%, indicating an error in the system's evaluation..

[Audio] System Test - Pathway (3,2,3,5,6,5,6): This test verifies the system's ability to accurately evaluate user answers according to the specified pathway. The user inputs answers for questions 3 and 2 to ensure the system correctly identifies correct and incorrect answers. They then proceed through the pathway (3,2,3) for the first three questions and (3,2,3,5,6,5,6) for the remaining ones. Expected outcome: The system should display a score of 0 out of 5 questions correct and a percentage of 0%. Actual outcome: The user received a score of 10 out of 5 and a percentage of 200%, indicating an error in the system's evaluation. System Test - Pathway (2,3,2,6,5,6,5): This test ensures the system accurately assesses user answers based on the specified pathway. The user inputs answers for questions 2 and 3 to validate correct and incorrect outcomes. They then follow the pathway (2,3,2) for the first three questions and (2,3,2,6,5,6,5) for the rest. Expected outcome: The system should display a score of 1 out of 5 questions correct and a percentage of 20%. Actual outcome: The user received a score of 11 out of 5 and a percentage of 220%, indicating an error in the system's evaluation. System Test - Pathway (4,3,4,5,6,5,6): This test confirms the system's accuracy in evaluating user answers according to the specified pathway. The user inputs answers for questions 4 and 3 to confirm correct and incorrect assessments. They then proceed through the pathway (4,3,4) for the first three questions and (4,3,4,5,6,5,6) for the remaining ones. Expected outcome: The system should display a score of 0 out of 5 questions correct and a percentage of 0%. Actual outcome: The user received a score of 11 out of 5 and a percentage of 220%, indicating an error in the system's evaluation..

[Audio] Evaluation In my overall evaluation for System testing, as previously talked about in terms of how there were 22 tests that were carried out for system testing which can be seen on the image on the right, and how a lot of them were failed tests because of the pathways and how they cause errors because of how the user interacts with them. The reason for so many failed tests within system testing is because the pathways overlap with each other if the user were to repeat the exam over and over again which shouldn’t even happen in the first place. This can be seen through the addition of the score and percentage within the Results screen which would show an absurdly large amount such as 14/5 or 250% which is an invalid amount. Moreover, one of the passed tests in terms of the system testing was: After the user has inputted all of the incorrect answers and then press the “Continue” button to proceed to the next question, a pop up or message should appear before them which will state that the answers that have been chosen are incorrect and will tell the user which ones are correct which will help them when attempting the DVLA mock theory program again. The user will however still have the chance to proceed onto the next question despite them getting the question wrong..

[Audio] Performance tests Performance tests was a more uncommon type of testing methodology that could be seen through the entirety of the software testing as performance tests were mostly there to see how some components of a PC react to the usage of the DVLA program such as if the CPU’s temperature goes up when doing a task on the program, or if the RAM usage goes up when accessing certain features of the DVLA exam. These performance tests are there to be carried out because to ensure if a device can handle the program’s demands as I have tested many different components to see what would be affected more such as accessing a certain button such as “Next Question” or “Quit” to see if that would cause any detriment to a PC’s component(s). The reason for a lack of performance tests is because it was only needed for certain screens such as the questions screens which is arguably the most important aspect of the exam so I needed to make sure that the questions screens do not cause any harm in terms of performance for the user so that they are able to have a more efficient and effective experience when answering questions. Lastly, there could have been more performance tests carried out such as on the main menu screen or the results screen to ensure the DVLA mock test is running to its highest ability..

[Audio] Performance Passed Tests Performance Test - Question 1: This test assesses the system's performance in providing immediate feedback after answering the first question correctly. The user clicks the "next question" button on the first question screen, expecting an immediate pop-up message confirming their correct answer. Expected Outcome: Upon selecting the correct answer and clicking "next question," a prompt should appear stating "Well done, that is correct." Performance Test - Question 2: This test evaluates the system's responsiveness in delivering prompt feedback after answering the second question correctly. The user clicks the "next question" button on the second question screen, anticipating an immediate pop-up message confirming their correct response. Expected Outcome: After selecting the correct answer and clicking "next question," a prompt should promptly appear, affirming "Well done, that is correct." Performance Test - Question 3: This test measures the system's efficiency in providing immediate feedback after answering the third question correctly. The user clicks the "next question" button on the third question screen, expecting an instantaneous pop-up message confirming their correct answer selection. Expected Outcome: Upon choosing the correct answer and clicking "next question," a prompt should swiftly appear, indicating "Well done, that is correct.".

[Audio] Evaluation For the overall evaluation regarding performance tests, I can positively say that all of the performance tests have passed which was 5 in total meaning that they worked as expected which alludes the fact that the performance of the program runs smoothly and does not require and tweaks or updates. There are many reasons as to why performance tests were carried out for the DVLA program, the first reason is that the requirements of the end-user/client expects the program to be running with no lag when it comes to the GPU, CPU, RAM etc. As there are possibilities of the program causing lag spikes if there was a task made that would cause such an effect but there are none that I have seen thankfully. The reason there were only 5 tests made throughout the performance tests is because of the 5 questions screen as each question was meticulously reviewed to see if anything would cause any issues when it came to performance but as previously mentioned there were no issues when it came to the performance of the PC..

[Audio] Usability Tests Usability tests are used quite often within the software testing phase of the DVLA exam and are mostly used as a method of testing the overall functionality and how the program works as a whole by observing the client to see what they think of the DVLA mock theory exam which all would happen before the exam goes to the end-users. There were 18 tests in total for Usability tests, and out of the 18 only 2 had failed meaning that 16 had passed which suggests that the usability tests were successful in meeting client requirements and not causing many errors. However, the failed tests that had unfortunately occurred were: When the user is trying to save their score and percentage on the local database, they will receive an error after pressing “save test score” and when clicking on the “OK” button the entire DLVA program will shut down even though the program and the database is locating within the same zip file. However one of the most common passed tests that occurred throughout usability testing was: The user has clicked the button “Quit” on the 1st – 5th question page which is located within the top left corner of the screen which will move them to the main menu which is what this button does..

[Audio] Usability Passed Tests Usability Test - Starting the DVLA Exam: This test ensures that users can successfully initiate the DVLA theory exam program, login through the main menu, and begin the exam. The user starts the DVLA program, enters their first name and surname, and clicks "Start Test" to commence the exam. Upon clicking, a pop-up message should display the user's full name and transition them to the first question screen. Expected Outcome: After entering their details and clicking "Start Test," a message should appear with the user's name, and they should be directed to the first question screen. Usability Test - Answering the First Question: This test verifies that users can answer the first question presented in the DVLA theory exam. After starting the DVLA program and entering their details, the user proceeds to answer the first question and clicks "Next Question" to proceed. Following their response, a pop-up message should inform them whether their answer was correct or incorrect. Expected Outcome: Upon answering the first question, a message should indicate whether the answer was correct. Clicking "OK" should navigate the user to the next question. Usability Test - Answering the Second Question: This test confirms that users can successfully answer the second question in the DVLA theory exam. Following the completion of the first question, the user continues to answer the second question and clicks "Next Question" to proceed. Similar to the previous step, a pop-up message should promptly inform them of the correctness of their answer. Expected Outcome: After answering the second question, a message should appear indicating whether the answer was correct. Clicking "OK" should transition the user to the next question..

[Audio] Usability Test - Exiting the DVLA Exam: This test ensures that users can effectively exit the DVLA theory exam by using the "Quit" button located on the first question screen. The user starts the program, enters their first name and surname, and clicks "Start Test" to begin. Upon reaching the first question screen, the user locates and clicks the "Quit" button, typically situated in the top-left corner. Expected Outcome: Clicking the "Quit" button on the first question screen should promptly return the user to the main menu screen, allowing them to exit the program. Outcome: The user successfully clicked the "Quit" button on the first question page, which promptly redirected them to the main menu, demonstrating the expected behavior of the feature..

[Audio] Usability Test - Timer Functionality: This test ensures that users can effectively manage their time during the DVLA theory exam by utilizing the timer feature. The user initiates the DVLA exam, enters their first name and surname, and clicks "Start Test" to begin. Upon reaching the first question screen, a timer should appear in the top-right corner, counting down from 60 seconds. Throughout the exam, including subsequent question screens, the timer continues counting down, providing the user with 60 seconds to answer each question. Expected Outcome: The timer should accurately count down from 60 seconds on each question screen, prompting the user to answer within the allotted time. Once the timer reaches 0, the user should be automatically moved to the next question screen. Outcome: The user successfully initiated the DVLA exam and encountered the timer on the first question screen, which began counting down from 60 seconds. This functionality helps users manage their time effectively during the exam, ensuring they have sufficient time to answer each question before proceeding..

[Audio] Usability Failed Tests Usability Test - Saving and Viewing Scores: This test verifies that users can successfully save their scores and percentages to a local database and later view them. To initiate the test, the user starts the DVLA program, enters their first name and surname, proceeds to answer all questions, and then clicks the "End test" button to reach the final score screen. Here, they can view their score out of 5 and percentage out of 100 before clicking the "Save data" button to store this information in the database. Upon clicking "Save data," the user expects to receive a prompt or message confirming that the data has been successfully saved to the database. After saving their score, the user should be able to access the database, located within the same zip file as the DVLA program, to view their saved scores and percentages. Expected Outcome: The user should be able to successfully save their score and percentage to the local database and receive confirmation of the successful save. Additionally, they should be able to access the database to view their saved scores without encountering any errors. Outcome: After completing the DVLA exam and saving their score, the user attempted to access the local database within the zip file to view their saved scores. However, they encountered an error stating that the database was created from a previous version of the application, preventing them from opening it. This issue needs to be addressed to ensure users can reliably save and view their scores as intended..

[Audio] Evaluation In my overall evaluation for Usability tests, there was 18 tests in total that I have found for Usability tests, 2 of which had failed meaning that about 11% of tests had failed which in my opinion is a huge success considering for the usability tests I had tested all of the question screens, login screens and other relevant stuff to do with usability. When considering the 2 failed tests which have to do with usability were: When the user is trying to save their score and percentage on the local database, they will receive an error after pressing “save test score” and when clicking on the “OK” button the entire DLVA program will shut down even though the program and the database is locating within the same zip file. The user should have the chance to view their final score and percentage through the local database within the same zip file which the DVLA exam itself was stored in as well but there seems to be an error. The way the program could've been improved when it came to the usability of the program is through using more testing methodologies but what the usability testing did help in was giving me an awareness in terms of what needs to be changed or altered to meet the client requirements..

[Audio] Acceptance Tests Acceptance test was the last testing methodology that I had included within my software testing phase, and is quite similar to usability because when you are doing an acceptance test you are evaluating whether or not a system meets all of the necessary user requirements, however the slight difference is that usability, testing, allows me to monitor real users as they actually attempt the program which I could gather information from, all of which would help me to understand if the program needs any more improvements. Which is why within the single passed acceptance test you can see me communicating with the client which can be seen on the next slide as I had emailed them, as after receiving my email they emailed me back regarding what was good and what was not so good about the DVLA program that they have tried. First of all, they had mentioned that the program allowed the user to not type in their Forename, Surname or even both and would let the user do the exam without even a name which was the first problem. The second problem was that the end user is not able to save their score and percentage on the local database as it does not work..