The Usability of Training Management System for Teachers’ Professional Development: A Preliminary Study

The usability study of the Ministry of Education’s Training Management System (SPLKPM) is a preliminary study conducted by the researcher to determine the rationale and analysis of research needs regarding system usage behaviour among teachers. The purpose of this study is to assess the system's usability in the management of in-service teacher training. SPLKPM serves as a platform that records and stores training data and credit point reporting for all Ministry of Education (MoE) staff members based on 13 activities outlined in continuous professional development credit points (MyPBB). The respondents for this study were in-service teachers who are also regular users of SPLKPM. A total of 32 respondents provided feedback that was randomly distributed through several mediums; emails and mobile messaging apps. The study found that the existing system has a usability score of 45.9, compared to the average of 68, indicating that it requires improvement and that a comprehensive analysis of the factors that influence system use should be done. In conclusion, most teachers believe that the system needs to be improved to facilitate more frequent recording and use.


Introduction
Organizational practices have been significantly influenced by developments in information technologies as well as other aspects such as information interchange, rising societal expectations, current management perspectives, and implementation of these technologies (Forrester, 2019). Managing an educational institution effectively involves a lot of data and information to be recorded, analysed, and managed. Raw data or information comes in various forms from numerous sources processed using automated systems as well as manual or legacy means (Musti, 2020). Teachers, on the other hand, must keep up with the rapid adoption of new technologies in education by acquiring the essential knowledge and skills through effective training management (EPU, 2021). An effective training management system relies on teachers' ability to dynamically organise and reorganise learning information for several purposes, including reflection and transparency in personal and professional growth (San Jose, 2017). The greater the usability of the system, the easier it will be for the user to find and access particular records (Hassan et al., 2010). To facilitate understanding, the term Training Management System (SPLKPM) is frequently used interchangeably with other terms in this study, namely the Continuous Professional Development (CPD) system. Teachers' accomplishments and commitment to professional development activities are documented in a web-based professional development system that serves as a central repository for generating teacher professional development profiles (MoE, 2016). To that end, this study seeks to provide a need analysis for such a large-scale study from the perspective of teachers, who will serve as key respondents.

System usability
A well-designed information system will provide valuable insights into resource allocation, budgeting, and spending. In the educational sector, however, many critical aspects such as operational challenges, cyber security, standards, system hierarchy/categorization, changing rules and regulations and their impact on information systems, factors/reasons for failures, overall planning, and best practices in managing information systems are not well presented (Musti, 2020). The term "usability" and "success" refer to a system's level of user satisfaction, effectiveness, and efficiency as determined by the user's experience with the system (Ferreira et al., 2020;Hassan et al., 2010;Lyerla et al., 2018;Nielsen, 1992;Smailes et al., 2019). User experience (UX) is a new concept that emphasises the emotional aspects of system use, including implications for organizations, user documentation, support and maintenance, and long-term use (Viitanen et al., 2011). According to the International Organization for Standardization (ISO 9241-11:2018), usability is the degree to which a system, product, or service may be utilised effectively, efficiently, and satisfactorily by specified users in a stated context of use (Bevan et al., 2016;Estdale & Georgiadou, 2018;Ferreira et al., 2020;Smailes et al., 2019).
To enhance the system's usability, organizations must prioritise user demands through policies and procedures that promote user-centered design, while also allocating funds and resources to address newly identified user needs (Smailes et al., 2019). Numerous studies have been conducted with the goal of increasing the system's usability and enabling user interaction (Daradkeh, 2019;Munaiseche & Liando, 2016;Wattearachchi et al., 2019). Additionally, empirical research has been conducted on the use of e-learning (Gunesekera et al., 2019;Taat & Francis, 2020;Wang et al., 2019) and its effect on the usability mechanism (Ferreira et al., 2020). The availability of research on the usability of the CPD system at this time, however, is still lacking, particularly in the context of an education management information system, which can serve as a reference in the future. Additionally, Gunesekera et al. (2019) assert that efficiency, ease of use, memorability, and effectiveness are critical usability factors that have received less attention in the literature yet have a significant impact on user satisfaction and system usage continuity.

Teachers' Training Management System
In recent years, Malaysia's education system has succeeded in developing Education Management Information System (EMIS) to ensure that the data management system for schools, teachers, and students is always complete and up to date. The EMIS ecosystem consists of 239 systems and applications for pre-school through secondary school, as well as 110 systems and applications for post-secondary education. In the Ministry of Education (MoE), that division that has its own management information system is in charge of running that system (United Nations International Children's Emergency Fund [UNICEF], 2020). The training management system (SPLKPM) is one of the systems that manage the professional development of teachers in particular and MoE staff members (non-teachers) in general. Using SPLKPM, teachers can keep track of all the information and data related to their professional development, including the courses and workshops they have attended, the self-directed learning they have completed, and the current credit points they have earned. SPLKPM also helps teachers manage their professional development based on the CPD activities they have participated in across various platforms (MoE, 2016).
Documenting CPD is critical for teachers and school leaders to plan their professional development, further education, and career promotion while keeping track of their progress and competencies over time (Kwok & Hui, 2017;MoE, 2014MoE, , 2016Nur Leenna et al., 2020;Segaran & Hasim, 2021;Song, 2021). However, the utilisation rate of this system is relatively low due to the inefficiency of centralized data storage procedures and unstructured documentation (MoE, 2014), as well as a lack of teacher participation in CPD activities (Ab Rahim et al., 2021), particularly in self-initiated professional development (SI-PD). SI-PD or self-directed learning in other terms, can motivate teachers to use the system on a regular basis, allowing for better use of the current system and more ordered management of teachers' CPD activities (Segaran & Hasim, 2021;Song, 2021). For an electronic CPD system (e-CPD) to succeed, administrators and teachers must work together and provide ongoing technical support and involvement in professional development activities (Razak & Yusop, 2013). However, increasing user participation in government decision-making and access to information, as well as the use of systems, remains a difficult challenge (Huang & Benyoucef, 2014).

Methodology
The objective of this study is to determine the system's usability for managing in-service teacher training. At the preliminary stage, the participants involved consisted of inservice teachers who serve as Training Coordinating Officers (SULDP) in their respective schools. A questionnaire was developed to get the participants' initial assessment of the dependability of the usability perspective. The questionnaire was divided into two sections, referred to as Section A and Section B. Specifically, Section A examines the system's usability for teachers as users, who are presented with 10 statements relating to various areas of usability, which are graded on a 5-point Likert scale ranging from strongly disagree (1) to strongly agree (5). The System Usability Scale (SUS) instrument, developed by John Brooke in 1996(Brooke, 1996, is used in this section's questionnaire. The 10 items contain both positive and negative statements, which adds another layer of complexity, and the responses to the questions show the degree of agreement or disagreement, which is equivalent to one another (Brooke, 2013) as depicted in Table 1. SUS is a "quick and dirty" survey scale that allows a usability practitioner to evaluate the usability of a given product or service in a short and straightforward manner. For the most part, researchers have used the SUS to assist them to assess the usability and user satisfaction of a particular system they are testing out (Bangor et al., 2008;Brooke, 2013). When it comes to selecting a usability questionnaire, the researcher has a variety of options such as (1) the System Usability Scale (SUS) (Brooke, 1996), (2) the American Customer Satisfaction Index (ACSI) (Anderson & Fornell, 2000), (3) the Website Analysis Measurement Inventory (WAMMI), (Kirakowski & Cierlik, 1998), (4) a five-scale questionnaire (Van Schaik & Ling, 2005), (5) the Website User Satisfaction Questionnaire (Muylle, Moenaert, & Despontin, 2004), (6) SUMI (Software Usability Measurement Inventory) (Kirakowski & Corbett, 1993) and (7) The Usability Metric for User Experience (UMUX) (Finstad, 2010;Lewis et al., 2013). SUS was chosen because it is concise, applicable to a wide variety of situations and systems, most notably government websites, and provides a broad indication of the overall level of usability (Elling et al., 2012). I think that I would need the support of a technical person to be able to use this system. 5 I found the various functions in this system were well integrated. 6 I thought there was too much inconsistency in this system. 7 I would imagine that most people would learn to use this system very quickly. 8 I found the system very cumbersome to use. 9 I felt very confident using the system. 10 I needed to learn a lot of things before I could get going with this system. Source: Finstad (2006)

Result
The study's analysis was conducted manually using Microsoft Excel software to determine the system's usability score. The system's grading mechanism is based on a scoring schedule that Lewis and Sauro (2018) have updated. The SUS's final score is between 0 and 100, with higher values suggesting greater usability (Mol et al., 2020). Lewis (2018) emphasised that while the SUS scoring system requires scores for all ten elements if an item is left blank, a raw score of three (the five-point centre) should be assigned. Subtract 1 from the raw score for things with odd numbers and 5 from the raw score for items with even numbers. To obtain the normal SUS ranking, the sum of the adjusted scores is calculated and then multiplied by 2.5 (Brooke, 2013). The full curving grading scale is shown in Table 2, together with the number of SUS scores assigned to each grade and the corresponding percentile range. Source: Lewis and Sauro (2018) After analysis, the ultimate score achieved for this study was 45.9, which corresponds to Grade F with a percentile range of 0-14. These results indicate that the system still has room for improvement. However, the score is meaningless until a comparison is made. Lewis (2018) noted that comparing scores to criteria enables evaluation of how excellent or bad a score is, within allowed generalisation bounds. The results of the SUS score for SPLKPM are shown in Table 3. Section B includes four subjective questions, commonly referred to as open-ended questions, that require teachers to have a clear understanding of SPLKPM. Open-ended questions at the completion of the SUS will assist in defining the method's issues that must be addressed (Klug, 2017). Brooke (2013) found that subjective opinions were significantly more influential in the case of office systems, but subjective satisfaction assessments are pertinent to whether users are more inclined to recommend the system to others. This questionnaire was analysed using a basic technique called thematic analysis, which was implemented in Microsoft Excel.
Question 1: What are the weaknesses of the existing training management system that need to be improved in order to increase the ability and interest of teachers to use this training management system more efficiently?
In response to question 1, the majority of teachers claimed that the system was not userfriendly. The responses provided are various, but all result in technological difficulties and ineffective operational methods. There are too many blank spots to fill, the box options are rather perplexing, the directions are vague, and the terminologies are unfamiliar. These are the responses of 18 teachers; seven others stated otherwise, and the remainder did not respond.
Question 2: What functions are required in the existing training management system to make it easier for teachers to record training?
Most teachers do not precisely describe what the desired function is due to a lack of understanding of the operating system. They do recommend, however, that every current system function may make it easier for teachers to record training and grasp the instructions and concepts. This perspective represents 15 of them, while the remainder expresses doubt or a lack of knowledge of the system's functional requirements.
Question 3: What steps need to be taken by the authorities to encourage teachers to frequently use the training management system to improve their professionalism?
Four themes emerged from the respondents' varied responses. Ten teachers felt that the variety of courses or training available may be expanded, as well as promotions to motivate teachers who complete the training to maximise their use of the system. Meanwhile, the remaining nine would like to see the MoE refine its role in delivering policies, procedures, and steps that would help make the system more transparent and understandable. Whereas the remaining six viewpoints advise that MoE is more assertive in its monitoring of system usage, and five opinions ask for system enhancement and access. The remainder remains unresolved.

Question 4: In your opinion, what is the reason why teachers do not want or less use the existing system to record training?
Overload work and time constraints cause teachers to use less of the system to record training (11). Apart from saying that the existing system is overly technical and difficult to use (7), teachers frequently believe that there is no urgent need for them to record training using the system (6). Other problems, such as laziness (4) and lack of ICT skills (3), contribute to a lack of active usage of the system among teachers.

Conclusion
Based on the findings, the usability (SUS) score of 45.9 was found to be significantly lower than the average SUS score of 68. These results have led to the need for improvements to the current system and a more comprehensive study needs to be conducted. Furthermore, based on a review of open-ended questionnaires, the majority of teachers claimed that the system was not user-friendly. This is backed by the research of Rahman et al. (2019), who suggested a more user-friendly teacher training system. Aside from that, teachers agreed that the variety of courses or training offered might be expanded, as well as promotions to motivate teachers who complete the training to make the most of this system. Additionally, they require MoE to refine its role in delivering policies, processes, and steps, as well as to be more proactive in monitoring system usage. The initial study also revealed a lack of use of the system due to workload factors and time constraints to undertake CPD activities, resulting in a lack of CPD activities, particularly self-initiated professional development (SI-PD) being registered in the system. The study's findings are consistent with the OECD's International Teaching and Learning Study (TALIS), which highlighted schedule and workload conflicts, a lack of incentives, costs, and significance in terms of the value of collaboration, time allocation, and financial support among teachers (OECD, 2014). To summarise, the majority of teachers agree that the system should be enhanced to make it easier to record and use.
This study still has limitations in terms of the number of samples that are not representative of the entire population given the short duration of its implementation. Furthermore, this study is preliminary with the aim of obtaining a needs analysis for future studies. This study could be beneficial at an early stage to justify the reliability of the instrument. Therefore, there is a need for the instrument to be tested to ensure that it is reliable and can be used in actual fieldwork. For future studies, sampling techniques should be carried out more carefully taking into account environmental and geographical factors, as well as the actual population.