Development and Validation of Open Ended Based on Worksheet for Growing Higher Level Thinking Skills of Students

This research aims to develop open ended-based worksheet in improving students' high-level thinking skills on static fluid materials. The method used in this research is Research and Development (R&D) with 3 steps namely, the initial stage, then the design and development of the product, and finally the product testing. The data analysis technique for product effectiveness is with the main field trials, namely (1) N-gain analysis, (2) paired t-test, (3) ANCOVA, and finally (4) effect size test. From the test results obtained information that conducted 10 students showed the results of the development worksheets obtained the average worth of development products by 90% this value is included in the very high category, while the student response of the average of 88% and also the readability response of 89% in the very high category. In the results of the n-gain analysis also show the value is0.70, that mean medium category. From the results of this study, it can be concluded that worksheets based on open ended can improve students' higher-order thinking skills. So, in further development we suggest that open ended-based on worksheet is not only applied to static fluid material, but also in other materials.


Introduction
The latest educational challenge is that every educational institution must be able to produce graduates who have 21st century competence. 21st century competence is expected to be able to work together (collaborative), able to think critically (creative thinking), creative thinking (creative thinking), and skilled in communication (communication and problem solving). These competencies are known as 21 Century Skills (Trilling & Hood, 1999).
School is a place to forge students in developing thinking skills, interacting to communicate, collaborating and a place to obtain accurate information to shape students' thinking skills (Bacanli et al., 2011). Thinking skills are divided into basic level thinking skills which include remembering, understanding and applying and high level thinking skills which include analyzing (Balsa, 2019), evaluating, and creating skills (Anderson & Krathwohl, 2001). Besides having basic level thinking skills (lower order thinking, LOT), students must also have higher order thinking skills (higher order thinking, HOT).
Based on one international study that measures students' mathematical and scientific achievements, TIMSS (Trends in Mathematics and Science Study) held by the IEA (The International Association for the Evaluation of Educational Achievement) in 2011 shows that Indonesia is ranked 40 out of 42 countries surveyed in the field of science (Novoa et al., 2019). The 2015 TIMSS results also ranked Indonesia 45th out of 48 participants in the field of science. The fields of science tested are Earth Sciences, Physics, Chemistry, and Biology. In the field of Physics, Indonesia scored 397. This value is below the international average, which is 500. Based on percentage data for science content and cognitive domains especially Physics (Pahrudin et al., 2019), the percentage of participants from Indonesia who answered correctly on the problem of understanding was higher than the problem of application and reasoning (Moreno & Trejo, In addition to the use of appropriate learning strategies, the use of teaching materials must also be appropriate so that students' higher-order thinking skills can grow (Syahrir et al., 2018). Teaching material is a set of teaching material / substance (teaching material) that is arranged systematically, showing a complete figure of the competencies that students will master in learning activities (Pannen, 2001). Forms of teaching materials can take the form of visual, audio, audio visual and interactive media. One of the visual form teaching materials has an important role in learning activities is the Student Worksheet (Kaymakci, 2012).
Based on the results of the needs analysis questionnaire in the field study it was found that as many as 50% of teachers answered that the worksheets that were already available did not meet teacher expectations in the learning process (Jaimes, 2019). Also, as many as 50% of 2 teachers answered that Static Fluid material has the opportunity to use open ended problems (Kalyanasundaram & Madhavi, 2019). While the results of filling out the questionnaire by students, as many as 73% of students had difficulty understanding Static Fluid material, and as many as 80% of students revealed that teaching materials in the form of worksheet were useful to facilitate learning physics.
The material presented in the developed student worksheet is static fluid material. Static fluid is a physical material that has many applications in everyday life. In addition, static fluid material can be taught by using open ended problems (Sriyakul et al., 2019). Romli et al., (2018)  This study has two objectives, the first is to develop learning resources in the form of open ended-based worksheets, especially in Static Fluid material. Second, implementing learning by using open ended-based worksheet to describe students' high-level thinking skills in learning.

Research Design
The research design used is development research. The research procedure was carried out using research and development steps according to Gall, Gall and Borg (2003). The development procedure is divided into three stages of development, namely the preliminary study stage, the planning and development stage, and the field test stage. The design of the product trial using the test-posttest with control group design is described in Table 1. Data on the results of students 'high-level thinking skills in the form of quantitative data as seen from the students' pretest and posttest scores.

Research Sample
To get a sample with the desired characteristics, the sampling technique using purposive sampling technique, the school was chosen based on the researchers' consideration of the quality and location of the school. Research sample to get data needs analysis in schools, researchers involved 15 students and 2 high school teachers to fill the questionnaire. The product trial in this study involved 50th grade XI high school students in Bandar Lampung, of which 25 students were taught using open ended based worksheet and as a comparison conventional worksheet was applied to 25 students in the same school and grade level.

Research Instrument and Procedure
This research has three stages of research; the first stage is a preliminary study. This preliminary study consists of observation and data collection needs analysis for teachers and students using a questionnaire. The second stage is product planning and development. This stage has developed a product in the form of worksheet based on open ended, before using this worksheet product the content and construct validated by three experts. Validation is done by filling out the questionnaire with a Likert scale and then analyzed with descriptive analysis. Cognitive test instruments to measure students' higher-order thinking skills. The last stage, an open ended worksheet-based product implementation is carried out in the experimental class.
The research instrument used was a questionnaire to collect needs analysis data in the field. The questionnaire consisted of 14 questions for the teacher and 14 questions for students, each of which consisted of answers to strongly agree, agree, and disagree. Then, for the construct validation test questionnaire instrument and the content is used to determine the feasibility level of the worksheet developed. For the higher-order thinking skills test consists of 6 reasonable choice multiple questions and 6 descriptions. Before using all research instruments, validity and reliability have been tested.

Data Analysis
Data analysis results of worksheet validity by calculating the average score of each validator, then changing the average score of the value into a value with criteria. The reference for changing the score to a scale of five according to Tegeh (2014) can be seen in the following Table 2. Data analysis to determine the increase in students' higher order thinking skills used Analysis of covariance (ANCOVA) and strengthened by the effect size test. Data analysis is also supported by the analysis of the average gain normalized score which aims to find out the level of effectiveness of open ended physics worksheets as a source of physics learning for students. The normalized gain calculation results are then interpreted using the classification from Hake (2001) as presented in Table 3. Table 3.N-Gain Classification g value Interpretation g > 0,7 High 0,3 < g £ 0,7 Medium g £ 0,3 Low

Findings / Results
The worksheets that were developed had a content section divided into 3 parts, namely, the first meeting on the topic of Hydrostatic Pressure, the second meeting on Pascal law, and the last meeting on the law of Archimedes. At the first meeting, presented the problem of divers, which then described the hydrostatic pressure experienced by the diver. In addition, it is completed with practicum activities whose purpose is to prove the hypotheses that have been made by students, then compare whether they are in accordance with existing theories (Mikhailovna et al., 2019;Nashir et al., 2020). The second activity discusses Pascal's law which is associated with the phenomenon of a car being lifted in a car wash using a hydraulic jack. The last meeting, discussed about the law of Archimedes associated with the phenomenon of sinking or floating of an object that is in liquid. In addition, the worksheet is equipped with static fluid phenomena in the surrounding environment through "Let's Observe" activities that are presented open ended, Making hypotheses through "Let's Investigate" activities, fostering higher-order thinking skills through "Let's Discuss" activities in answering questions The discussion is presented in an open ended. Static fluid phenomena presented in the worksheet can be seen in Figure 1.

Figure 1. (left) Divers, (center) Cars washed, and (right) Ships
The results of expert assessment of content validity and construct validity against the worksheet that have been developed can be seen in Table 4.  Kusuma et al., (2017). The results of the validity and reliability of these instruments have shown to be valid and reliable for use.
The results of the analysis of students' higher-order thinking skills in the experimental (Munifah et al., 2019a) and control classes can be described in the following Table 4 and Table 5. The average gain value in the experimental class (0.69) is higher than the control class (0.28). Analysis of covariance (ANCOVA) is used to determine the differences between the two classes by using the initial creative thinking skills test as a covariate and thinking skills test Final creative as the dependent variable, as shown in Table 6. Based on the results (sig. <0.05), shows that there are significant differences between the two classes, where students who learn to use open ended worksheet have better creative thinking skills than students who use Conventional worksheet. The results of increasing creative thinking skills after learning between control classes and experimental classes are strengthened by the results of the effect size test in Figure 2, Based on Figure 2 it appears that the effect of open endedbased worksheet results in the development of higher-order thinking skills students obtain an effect-size r of 0.708933 or in the medium size category, meaning that learning using open ended-based worksheet has a moderate effect on students' higher-order thinking skills (Munifah et al., 2019b).

Discussion
Open ended-based worksheets are able to achieve the goal of learning physics in the material fluid in a predetermined time allocation both in terms of cognitive, affective, and psychomotor. This shows that in the open ended worksheetbased design, students are asked to understand the appropriate static fluid phenomena described in the worksheet, students then answer problems related to the phenomenon open ended (Munifah et al., 2019c). Teaching materials that contain phenomena of environmental problems that are around students can improve student learning outcomes (Sarwanto et al., 2015).
After that in worksheet students are required to conduct experiments related to the phenomena displayed to prove their hypotheses about the problems presented, this is done to reach the psychomotor realm with students having to cooperate with each other and must interact in groups to achieve affective aspects, then students are asked to analyze the results of an experiment and link the results of an experiment with existing physical theories through group discussions, in the end students will be tested through high-level thinking skills instruments to reach the cognitive realm (Sulaiman & Ani, 2019). Experiments in physics material make it easier for students to understand concepts, and enable students to find principles or knowledge for themselves, besides experiments in physics can practice critical thinking skills (Purwanto et al., 2012).
Learning using open ended worksheet can eventually become a means to train students' science process skills because there is integration between the use of open ended problems and the components to be improved in higher-order thinking skills (Syazali et al., 2019a).
Worksheets developed based on open ended have a learning process that requires students to think at a high level, it can be seen how students actively think and discuss since the initial stages of observing static fluid phenomena presented in open ended, formulating hypotheses, continuing to conduct experiments, analyzing experimental results to answer questions, and provide conclusions in each section of the activity (Maskur et al., 2020). The open ended step requires students' skills to submit hypotheses, analyze, and create conclusions so that they can develop higher-order thinking skills (Syazali et al., 2019b). This is in line with research (Yonata, 2013) which explains that higher-order thinking skills can be trained through activities to formulate problems, make hypotheses, presentations that can facilitate students to conduct question and answer activities, and evaluate the process of finding solutions to problems. Meanwhile, according to Oktaviani and Dance's research (2017) problem solving skills can be improved by implementing open ended problems in learning. The steps above are similar to higher level thinking skills indicators namely: analyzing, evaluating, and creating (Hopson et al., 2001).
Following students' answers in completing the analytical problems given can be seen in Figure 3. Analysis skills must be possessed by students; lack of analysis skills will be bad for students in the form of student learning outcomes that are far from the learning objectives (Novita et al., 2016). Yet according to Rose and Nicholl (2002) people who have good analytical skills who can master the 21st century. Students are also tested the given evaluation questions can be seen in Figure 4.

Figure 4. Evaluation Skills Problem
Overall student skills after using open ended-based worksheet in evaluating are already good, it can be seen from the students' answers in answering questions that measure evaluation skills. In addition to analyzing skills, evaluation skills are also included in the indicators of student success in higher-order thinking. Students' skills in evaluating are trained in each of the Let's Discuss activities presented in the Worksheet.
Students 'skills in evaluating are very necessary because they can foster students' skills in critical thinking as part of higher level thinking skills (Hartinah et al., 2020). Skills in answering evaluation questions will build skills in assessing the credibility of statements or other presentations by assessing or describing one's perceptions, experiences, situations, beliefs, decisions and using the logical power of expected inferential relationships or actual inferential relationships between statements, questions, j0 '= descriptions and other forms of representation (Susilowati et al., 2017). Following are the answers of students in solving the problems presented above can be seen in Figure 5.

Figure 5. Student Answers in the question Evaluating Questions
The highest skill in the indicator of higher order thinking skills is the skill of creating. The following is a matter of thinking at a high level, creating as in Figure 6 below. Following are the answers of students in solving the problems presented above can be seen in Figure 7.

Figure 7. Student Answers Regarding the Creation Problem
Based on the results of students' answers in tests of high-level thinking skills, it appears that students are able to correctly answer questions that are indicative of analyzing and evaluating (Yu et al., 2019;Yusupbekov et al., 2020). However, no student is able to perfectly answer the question of creating (Usmanjonovna et al., (2020). These findings reinforce the results of the effectiveness of open ended worksheet-based worksheet test having N Gain and mediumsized effect sizes on students' high-level thinking skills .
In teaching Physics, the teacher must try to minimize the difficulties of students in learning by creating interesting learning situations for students. One effort that can be done is to choose the right learning strategy (Sagala et al., 2019). Scaffolding is a form of cognitive apprenticeship that can be chosen to improve student learning . Scaffolding is the right step taken to reduce the degree of freedom in doing a task so students can concentrate on skills that are difficult to have. In essence, scaffolding seeks to enhance learning through social interaction by involving the negotiation of content, understanding , and learning needs (Sumarni et al., 2019). Theoretically, scaffolding will improve the quality of the Physics learning process which in turn will improve student learning achievement (Habibi et al., 2019). Some of the advantages of using scaffolding include being able to improve student investigation and performance, keep students from feeling failed, and can bridge students' learning difficulties (Nurulsari eal., 2017;Rahman et al., 2015).

Conclusion
In this study, data were obtained that identified potential problems at school. This problem shows that the development of teaching materials in the form of worksheets can improve students' higher-order thinking skills. In the next stage, namely the design and development of open-based worksheet products, the validation results obtained by expert lecturers with a validation value of 88% and 84% of the construction results. In addition, the limited trial given to 10 students found that worksheets can improve the ability to develop ways of thinking that are practical and effective in learning. This value is indicated by the average feasibility of development products, which is 90% (very high category), average response value of 88% (very good category), readability response value 89% (very good category), and N-gain value amounted to 0.70 (medium category).While the results of the main trials conducted in the experimental class showed the results that worksheets can increase the achievement of learning increases by 89%. This value is very effective in improving students' higher-order thinking skills. As shown in the results of the N-Gain analysis with normalization (g) in the experimental class of (g = 0, 69), this value is higher than the control class of (g = 0.28). The results of paired sample t-test and ANCOVA analysis also showed that there was a significant difference from the average results of the students' high-level thinking ability tests (p <0.05). So, from the results of this study, it can be stated that the open ended-based worksheet has achieved the research goal of being able to increase high-level capabilities.

Recommendation
In further development, we suggest that open ended-based on worksheet is not only applied to static fluid material, but also in other materials.