Evaluating Microsoft office 365 implementation in learning at state senior and vocational high schools in Pohuwato

This research aims to evaluate the implementation of Microsoft Office 365 in learning at state senior and vocational high schools in Pohuwato. The approach used in this descriptive research was a mixed method with an explanatory design. The research population comprised 402 people, reduced to 40 using Slovin’s formula. Primary data were collected through questionnaire distribution, interview, observation, and documentation. The data analysis technique was the descriptive analysis using the CIPP evaluation model. The results suggested that (1) based on the context evaluation, teachers actively looked for information about Microsoft Office 365 and participated in training regularly or through partners, (2) based on the input evaluation, teachers prepared many different pivotal aspects to input learning using Microsoft Office 365. The primary inputs were lesson plans and other instruments, (3) based on the process evaluation, the implementation of Microsoft Office 365 in learning was effective. The COVID-19 pandemic made teachers make learning innovations, and (4) based on the product evaluation, teachers could make a positive learning product or content material that fostered students to be active in either online, offline, or hybrid learning and escalated their learning outcomes.


Introduction
Implementing and developing a curriculum using information technology at schools are two strategic steps for improving education in Indonesia.The roles of teachers in using applications for e-learning activities are crucial, specifically in optimizing learning strategies.Microsoft Office 365 is one of the requisite applications for learning.And yet, we have to evaluate the use of the application to optimize its future implementation.Evaluation is research to collect, analyze, and present information useful as the evaluated object and assess it by comparing it with evaluation indicators (Wirawan, 2011:7).This research uses the CIPP (Context-Input-Process-Product) evaluation model.This research was undertaken at state senior and vocational high schools in Pohuwato.We decided on the research area based on the problems found in the preliminary observation.It demonstrated sufficient competencies of teachers in using applications to carry out e-learning activities.The causes were that teachers did not optimize the potency of applications used in learning activities and that teachers preferred giving tasks to providing detailed material explanations when a diverse array of applications in the form of videos or PowerPoint were available.One of the familiar applications employed by teachers at state senior and vocational high schools in Pohuwato was Microsoft Office 365.The application allows teachers to induce a virtual classroom.Microsoft Office 365 is an effective tool for online or hybrid learning.
In implementing information, technology, and communication-based learning, several challenges remain in its management.For example, the information, technology, and communication hardware and software procurement costs are still high.Unfortunately, the majority of our society cannot afford them.Besides, information technology infrastructures are still poor, not to mention government preparedness.Furthermore, no educator resources that can make effective and efficient information technology-based learning are available.That being so, all parties should heighten teacher competencies in using information technology in learning.This is development research based on previous research, e.g., that by Kartini (2021) exhibiting students perceived that Microsoft Teams gave them happiness and opportunities to stay at home with long-distance learning and Nafisah & Fitrayati (2021) indicating the effective use of Microsoft Teams in learning and significant influences of Microsoft Teams on learning outcomes.And yet, this research offers a novelty.We focus on the CIPP evaluation model implemented in all learning stages and the mixed method with an explanatory model.

Methodology
The research areas were all state senior and vocational high schools in Pohuwato, and the targets were the teachers.The research was conducted in February-July 2022.This descriptive research used a mixed method with an explanatory design.The research population consisted of 402 respondents whittled down to 40 using Slovin's formula.The primary data were collected through questionnaire distribution, observation, and documentation.The data analysis technique was descriptive with a CIPP evaluation model.

Results
The descriptive analysis results of all performance indicators examined at all state senior and vocational high schools in Pohuwato are as follows.

Context Evaluation
Six aspects were engaged in the context evaluation of Microsoft Office 365 implementation in learning at all state senior and vocational high schools in Pohuwato.The results are pointed out in Figure 1.

Figure 1 Context Evaluation
Referring to Figure 1, we interpreted the results as presented in Table 2.

Input Evaluation
Nine aspects were involved in the input evaluation of Microsoft Office 365 implementation in learning at all state senior and vocational high schools in Pohuwato.The results are suggested in Figure 2.

Figure 2 Input Evaluation
Referring to Figure 2, we interpreted the results as demonstrated in Table 2.
Table 2 exhibited the mean score of the input evaluation of Microsoft Office 365 implementation in learning at all state senior and vocational high schools in Pohuwato.The mean score, i.e., 91.48%, indicated an effective implementation with a follow-up of maintaining the effectiveness.Here, teachers from all state senior and vocational high schools in Pohuwato had prepared manifold salient aspects as the input to learning using Microsoft Office 365.The three prime inputs were lesson plans, assessment instruments, and adequate competencies of teachers to design learning content growing student interest and allowing them to achieve well.

Process Evaluation
Five aspects were engaged in the process evaluation of Microsoft Office 365 implementation in learning at all state senior and vocational high schools in Pohuwato.The results are pointed out in Figure 3.

Figure 3 Process Evaluation
Referring to Figure 2, we interpreted the results presented in Table 3.
Table 3 pointed out the mean score of the process evaluation of Microsoft Office 365 implementation in learning at all state senior and vocational high schools in Pohuwato.The mean score, i.e., 92.74%, presented an effective implementation with a follow-up of maintaining the effectiveness.As suggested by the mean score, learning implementation using Microsoft Office 365 at all state senior and vocational high schools in Pohuwato ran effectively.
The COVID-19 pandemic made teachers innovate in learning.It engendered good learning implementation pursuant to the set targets and plans.

Product Evaluation
Five aspects were involved in the product evaluation of Microsoft Office 365 implementation in learning at all state senior and vocational high schools in Pohuwato.The results are shown in Figure 4.

Figure 4 Product Evaluation
Referring to Figure 4, we interpreted the results as suggested in Table 4.
Table 4 pointed out the mean score of the product evaluation of Microsoft Office 365 implementation in learning at all state senior and vocational high schools in Pohuwato.The mean score, i.e., 93.46%, demonstrated an effective implementation with a follow-up of maintaining the effectiveness.Building on the results, teachers at all state senior and vocational high schools were able to generate a learning material product or content integrated with Microsoft Office 365.The learning material content had a positive effect on students.Delivered online, hybrid, or offline, it increased student activeness in learning and motivated students to understand learning material concepts and contexts well (exhibited by better learning outcomes).

Discussion
Learning technology encompassed theories and practices of designing, developing, leveraging, managing, and evaluating learning processes and sources.Implementing and developing a curriculum using information and technology provided at schools were two strategic attempts for better future education in Indonesia.The futuristic curriculum of information technology was implemented following the global trend.Instead, it was a strategic effort to scale up educational service quality delivered to society (Harianti, 2007:2).Responding to the advanced information, technology, and communication, the National Education Department made a learning curriculum emphasizing information, technology, and communication for all levels.
The analysis results of evaluating Microsoft Office 365 implementation in learning at all state senior and vocational high schools in Pohuwato are pointed out in Figure 5.As a result, a better learning process could be induced.Teacher competencies were the key supporters of using applications in online learning processes.With relevant competencies, they could efficiently design innovative learning stages, i.e., planning stages ideal by planning guidelines.Teachers with appropriate competencies tended not to feel satisfied with the current achievement of learning implementation.Hence, following up on the issue, they would use various applications to advocate learning programs.As a result, students could understand learning concepts and contexts easily.
Below, we proposed a detailed explanation of evaluating Microsoft Office 365 implementation at all state senior and vocational high schools in Pohuwato.

Context Evaluation of Microsoft Office 365 Implementation in Learning
From the context evaluation, Microsoft Office 365 implementation at all state senior and vocational high schools was effective by 94.92% and hence, needed a low-priority follow-up.Predicated on the descriptive statistic analysis results and all informant statements, teachers from all state senior and vocational high schools actively looked for information concerning Microsoft Office 365.Furthermore, they actively participated in regular training or sought information by asking their teacher partners.In so doing, they could implement Microsoft Office 365 and produce more innovative technology-based learning.
It resonated with Arikunto (2008:46-47), that explained the CIPP evaluation model in detail.A context evaluation aimed to describe the environment, unmet needs, the served population and samples, and objectives.Nadhifah (2008:2) conveyed educators were mandated to develop the ability of the graduates of educational levels that included all aspects of their life, namely knowledge (cognitive aspects) covering being knowledgeable and capable, competency (psychomotor aspects), that was being creative, and attitude (affective aspects) embracing being faithful, pious, noble, healthy, independent, and democratic.Achieving those objectives required innovative and creative learning processes.
In tandem with curriculum development and advanced information, technology, and communication, teachers as the curriculum designers, developers, and implementing actors should be competent and conduct learning processes in accordance with the science and technology development.
It was aligned with Ibrahim & Suardiman (2014), that indicated a positive impact of e-learning on student learning motivation and achievement.Notwithstanding this, we still need to evaluate the online learning implementation to acquire clear, data-based remedial endeavors responding to the evaluation of teacher competencies in online learning.

Input Evaluation of Microsoft Office 365 Implementation in Learning
From the input evaluation, Microsoft Office 365 implementation at all state senior and vocational high schools was effective by 91.48%, and that being so, should be maintained.Based on the descriptive statistic analysis results and all informant statements, teachers from all state senior and vocational high schools actively prepared imperative aspects as input to Microsoft Office 365 learning.The inputs were lesson plans, assessment instruments, and adequate competencies of teachers to design attractive learning content allowing students to attain desired learning outcomes.
It was commensurate with Midaus (2008:128-130) that a special input evaluation aimed to help teachers determine programs resulting in needed changes.Input evaluation targeted finding barriers and available resource potencies.It enabled clients to study alternates related to organizational needs and targets.In other words, input evaluation helped clients avoid futile innovations predicted to fail or use excessive resources in vain.
It comported with Ma'sum (2017) stating the implementation stages of learning quality improvement management: 1) formulating the vision, mission, objective, and target of schools (situational objectives of schools), 2) socializing the Madrasa-Based Quality Improvement Management (MPMBM) concept, 3) identifying real challenges madrasas had to confront to identify functions required to realize the target, and 4) performing a SWOT analysis.This online learning was undertaken during the COVID-19 pandemic.Rosali (2020) remarked that student activeness with online learning was different from that with offline, in which students were more active.

Input Evaluation of Microsoft Office 365 Implementation in Learning
From the process evaluation, Microsoft Office 365 implementation at all state senior and vocational high schools was effective by 92.74%, and that being so, should be maintained.Building on the descriptive statistic analysis results and all informant statements, Microsoft Office 365 implementation in learning at all state senior and vocational high schools was effective.The COVID-19 pandemic had made teachers innovate in learning.It resulted in good learning implementation in accordance with the determined targets and planning.
The COVID-19 pandemic had made the central government temporarily close schools since March 2020.The local government responded to this policy by implementing long-distance learning (PJJ).Regrettably, in its local implementation, education was facing challenges during the pandemic era, specifically insufficient facilities and infrastructures, e.g., electricity and internet.It conformed to Mahmudi (2011:123) that a process evaluation focused on implementing the set plan.The output was suggestions for managers and the staff concerning the compatibility between the plan, the timeline that had been made, and resource use efficiency.If the plan needed modification or development, the evaluation process would demonstrate where it should be carried out.Additionally, the evaluation aimed to assess to what extent participants accepted the program and succeeded in implementing their roles and to provide complete notes of the plan implementation and the comparison with the objectives.

Product Evaluation of Microsoft Office 365 Implementation in Learning
From the process evaluation, Microsoft Office 365 implementation at all state senior and vocational high schools was effective by 93.46%, and that being so, should be maintained.Building on the descriptive statistic analysis results and all informant statements, teachers from all state senior and vocational high schools could make a learning material product or content integrated with Microsoft Office 365.The learning material content had a positive influence on students.It made them more active in online, hybrid, or offline learning and could prompt them to understand learning material concepts and content well (pointed out by augmented learning outcomes).
It was congruent with Ashadi & Suhaeb (2020) that this application could ease teachers and students in performing indepth learning processes.It allowed students or teachers to submit, distribute, and assess tasks at home or anywhere without time or learning hour constraints.A learning outcome indicated student competencies after they received learning experiences.Learning experiences could be using new learning media in learning processes to help students achieve learning objectives optimally.
It was consistent with Arif (2013) that technology-based educational product quality was affected by the degree to which institutions could optimally manage all potencies: education workers, students, learning processes, learning facilities, finance, and the relationship with society.Technology development yielded changes in teaching and learning processes.The Internet grew broader and more sophisticated to enable efficient learning.Online learning was needed as a facility or tool to buoy the current learning process.One of the technology-based media most people used was a mobile phone.Gheytasi et al. ( 2015) remarked that students frequently interacting using applications on mobile phones could understand reading text content easily.A variety of learning media were available but poorly used by teachers.One of the learning media was Microsoft Office 365-based.

Conclusion
Building on the results and discussion, we could draw the following conclusions.


The context evaluation results presented the activeness of teachers from all state senior and vocational high schools in Pohuwato in finding information about Microsoft Office 365, participating in regular training, and asking for information from other teachers.It allowed them to use Microsoft Office 365 more optimally and design more innovative technology-based learning.


The input evaluation results showed that teachers from all state senior and vocational high schools in Pohuwato prepared significant aspects as input to learning using Microsoft Office 365.The main inputs were lesson plans, many different assessment instruments, and adequate teacher competencies to design learning content attractive to students, enabling them to attain the expected learning outcomes.


The process evaluation results suggested the effectiveness of Microsoft Office 365 implementation in learning at all state senior and vocational high schools in Pohuwato.The COVID-19 pandemic urged teachers to make innovations in learning, thereby making learning implementation processes run well in accordance with the determined target and planning.


The product evaluation results demonstrated the ability of teachers from all state senior and vocational high schools in Pohuwato to make a learning material product or content integrated with Microsoft Office 365.The learning material content had a positive impact on students.It made students more active in online, offline, or hybrid learning and propelled them to understand learning material concepts and contexts well (exhibited by elevated learning outcomes).

Suggestion
Predicated on the results and conclusions, we proposed the following suggestions.


Capacity and commitment-building programs should be given to teachers from all state senior and vocational high schools in Pohuwato regarding using Microsoft Office 365 in learning and evaluation in accordance with applicable regulations.Routine socialization, seminars, and concrete measures should be delivered to teachers, making them more creative and information technology literate, especially during the COVID-19 pandemic, when information and technology are of importance for education.


Teachers from all state senior and vocational high schools in Pohuwato should be more creative in making learning content that can enhance student interest in learning and escalate their learning outcomes. Microsoft Office 365 can heighten student spirit in class.Still, the cost of accessing the application license is high, requiring a solution to realize learning effectiveness that needs information and technology.

Disclosure of conflict of interest
No conflict of interest.

Figure 5
Figure 5The IPA Analysis Results of Evaluating Microsoft Office 365 Implementation Figure5presents an effective Microsoft Office 365 implementation in learning at all state senior and vocational high schools in Pohuwato.Accordingly, teachers from the schools could use information technology, that in our case, Microsoft Office 365, in making lesson plans.The lesson plans acted as learning steps and guidelines for carrying out learning.As a result, a better learning process could be induced.Teacher competencies were the key supporters of using applications in online learning processes.With relevant competencies, they could efficiently design innovative learning stages, i.e., planning stages ideal by planning guidelines.Teachers with appropriate competencies tended not to feel satisfied with the current achievement of learning implementation.Hence, following up on the issue, they would use

Table 1
presented the mean score of the context evaluation of Microsoft Office 365 implementation in learning at all state senior and vocational high schools in Pohuwato.The mean score, i.e., 94.92%, showed an effective implementation with a low-priority follow-up.The teachers at the schools actively looked for information concerning Microsoft Office 365 through regular training or by asking their partners.It improved their competencies of leveraging Microsoft Office 365 in innovative technology-based learning.

Table 2
Input Evaluation Source: Processed data, 2022

Table 3
Process Evaluation Source: Processed data, 2022

Table 4
Product Evaluation Source: Processed data, 2022