A Study to Evaluate the Effectiveness of Art Creativity in China: Torrance Test Creative Thinking

Torrance Tests of Creative Thinking is an important method for testing creativity in education. It was developed in 1966 and has gone through five revisions. It has become a highly reliable and widely used important measure of creativity. With society's development, students' creativity has received widespread attention. Educators are also very interested in the creativity of university students. Nevertheless, in the teaching process, it is found that the creativity of university students' works is not high, especially for university students majoring in animation; if there is no creativity, there will be a lack of good work. The study adopts qualitative research by interviewing 12 Chinese university teachers on their experience of using TTCT-Figural. Furthermore, special interviews on its sub-item functions, usage methods, and applicability are also conducted. The MAXQDA software will summarize the data. Thus, the results show that TTCT-Figural has high applicability and practical significance to measure and evaluate university students' artistic creativity. At the same time, it also provides valuable experience for further improving university students' artistic creativity in animation courses.


Introduction
Creative thinking expresses a person's ability to generate many ideas or solutions in problem-solving.It leads to creative, high-quality solutions or practical products, mainly in art, music, science, or mathematics (Guilford, 1950;1956;Livne & Milgram, 2006;Torrance, 1962).Creativity has typically been given little attention in educational assessments before.Nevertheless, with the development of the economy and education, creativity has changed from a background player to a key player.Most innate theories also regard creativity as a core component (Kaufman et al., 2012).The education system now pays more and more attention to developing students' creativity to recognize the increasing value of creativity (Halverson et al., 2016).
Early creativity education is similar to elite education, which mainly aims at top students or people with special skills in a particular area.It is specifically concentrated in special fields such as art and technology.With the deepening of research, scholars have gradually realized that everyone has creativity.Furthermore, creativity can be cultivated, and creativity education can be developed on a massive scale (Guilford, 1956).
Although research on creativity education started relatively late in China, with the rapid development of the social economy, the research on creativity education is also developing rapidly.It is also because the cultivation of creativity has attracted more and more attention from educational circles, and at the same time, realized the importance of cultivating creativity (Li et al., 2015).Fostering creativity as a central issue in animation education in the context of creative industries is not adequately addressed.It has caused a cognitive gap in the understanding of the field of animation education in China.According to Ma's (2015) research, there is a decline in the needed competencies of animation students, as revealed by the survey.Therefore, it is also paying more and more attention to the cultivation of college students in China, and university students have also become the main group of people who develop creativity.Especially for university students majoring in animation, the demand for creativity is getting higher and higher, and it is no longer possible to evaluate their creativity simply by scoring their work.It is very important to evaluation criteria for animation creativity evaluation should be standardized, and professional evaluation standards should be used to complete the evaluation of animation creativity.Alencar and Fleith (2003) state that "the generation of a new product, idea, original invention, re-elaboration, or improvement of existing products or ideas" is one of the primary features covered in the majority of definitions of creativity.The definitions of creativity offered by the other authors also incorporate the concepts of novelty and unique goods.Guilford (1967), for instance, emphasises "divergent production."According to Torrance (1976), creativity is the ability to recognise gaps, suggest other approaches to issues, generate original ideas, reorganise them, and infer creative connections between ideas.According to Sternberg (2005), originality, oddity, and nonfamiliarity are the main ingredients of creativity.Though creativity appears to consist of several components that make assessment challenging and illusive, the TTTC, which has been applied globally, is widely acknowledged as one of the most effective methods of measuring creativity (Almeida et al., 2008).

Literature Review
The earliest creativity measurement tool was compiled based on the intellectual structure theory of Guilford, a professor of psychology at the University of Southern California.Guilford is an American psychologist whose research focuses on psychometrics, personality, and intelligence.Guilford's intellectual structure mainly includes flexibility, fluency, and originality.It mainly examines divergent thinking abilities.These three dimensions are the basis for Guilford's Children's Creativity Test (Zhao et al., 2015).
Dr. Torrance developed the Torrance Test of Creativity Thinking (TTCT) as a highly reliable and widely used creativity test (Zhao et al., 2015).TTCT builds on the work of J.P. Guilford.While Guilford's work laid the groundwork for the TTCT, the two are different: Guilford's work focuses on developing a theory of intelligence, while the TTCT is a test for assessing creativity (Zhao et al., 2015).
TTCT is widely used, and people of different ages can use it for testing.Specifically, the test is mainly used to examine the divergent thinking ability of the subjects.The main scoring basis is the fluency, flexibility, originality, and elaboration of the subject's answers (Zhao et al., 2015).
Given the characteristics of the above methods of evaluating creativity for university students, a more suitable method for measuring creativity is TTCT.It is a test that assesses, recognizes, and potentially fosters multiple approaches to creative thinking.It mainly examines the four scales of fluency, elaboration, flexibility, and originality, which are the most important for evaluating creativity (Theoharopoulou et al., 2020).
The TTCT remains a highly reliable and one of the most widely used tests of creativity for people of all ages.Torrens saw divergent thinking as the foundation of creativity and constructed tests emphasizing its assessment.Of all creativity tests, the TTCT is the longest-running, most carefully researched, and most widely used in educational settings of all creativity tests (Theoharopoulou et al., 2020).

Methodology
Although TTCT has been recognized as an effective method to detect creativity, few people have explained whether it can be used in the professional field of animation.This study mainly uses qualitative research methods to explain the applicability of TTCT in the professional field of animation.Because qualitative research methods can better explain the applicability of TTCT to the creativity scoring criteria of animation students.Large samples are often needed for quantitative research, whereas smaller samples are needed for qualitative research (Fainshmidt et al., 2020).As a result, the maximum variation technique will be utilised to determine the size of the interview participants rather than relying just on the saturation of the replies.This is due to the fact that the participants' diversity means that each participating organisation and person has a unique narrative to share.The implementation research by Palinkas et al. (2015) also suggests it.
The saturation point is considered to have been achieved when the respondent's replies become redundant, at which point the interview is terminated (Alam, 2021).Typically, eight to ten people are employed in small groups when concerns are being studied (Fetters & Molina-Azorin, 2017).Thus, based on literature review, the number of recruits was limited to 12 professionals (Alasmari & Zhang, 2019).
The 12 professionals are all lecturers, associate professors and professors from CUCN animation major, and they have all used TTCT in their courses.The researcher completed the interviews based on the respondents' experience and summarized the demographic data of the respondents, including gender, age, Teaching age, Job title, Used TTCT, Usage count, Course type, and Target population statistics.
After completing the interview，using MAXQDA software version 2022, all interviews with respondents related to the research question were included in the analysis and measured using the same criteria to obtain the results (Rädiker & Kuckartz, 2020).This research involves applicability to university students, proposition availability, assessment process, and score criterion to analyze whether TTCT applies to university students.

Demographic Profile of Interviewee
In this study, the number of people interviewed face-to-face was limited to 12 to ensure the validity of the data.
According to the standard of systematic interview analysis, it also protects the respondents' privacy.First, the respondents are coded with basic coding (Rädiker & Kuckartz, 2020).
Table 1 shows the codes of 12 interviewees: IN1 to IN12, and their gender, age, teaching experience, and basic status.Table 2 shows the demographic characteristics of the respondents are displayed more comprehensively.
To better match the proportion of respondents in the survey phase, the age of the interviewees was also controlled to 35-45 years old, with the ratio of male to female is 7:5 and the ratio of male to female teachers in the whole school (535:760) is similar.

Coding Process
Coding transforms raw qualitative data into something communicable and trustworthy (Linneberg & Korsgaard, 2019).This study adopted the inductive coding method because it is suitable for exploratory research and can help researchers grasp the relevance of research objects (Linneberg & Korsgaard, 2019).
The first-level code of this qualitative research is classified into four research variables: applicability to university students, proposition availability, assessment process, and score criterion.In addition, according to the method of inductive coding, the content mentioned in the interview record related to this study was summarized as a sub-code (Table 3).The table also counts the frequency of sub-codes and the most representative representations of the respondents.
Word clouds are a popular visual representation when summarizing the frequency of textual information.This study also used different arrangements of words in visual areas to express potential themes (Hearst et al., 2019).
Table 3 is a word cloud generated according to the frequency of sub-codes in interviews.
Since topics that appear only once or twice are not representative, only sub-codes that appear more than three times are reserved in the word cloud (Hearst et al., 2019).The scoring method is not uniform 9 There are differences in the scoring methods of the four variables.For me, it has brought some difficulties when counting the scores, and I feel that it is not convenient enough.I think the scoring method can be unified according to the characteristics of the profession.(IN5)

Less efficient 9
When I was scoring, I felt that the efficiency was a bit low, which may have something to do with the inconsistent scoring methods.It would be convenient if the scoring methods were the same.(IN11) Easy to score 8 On the whole, this scoring method is still simple, mainly based on counting.This is still going well.(IN2) Freq.= Frequency of occurrence

Result
This qualitative interview mentioned applicability to university students, proposition availability, assessment process, and score criterion many times.Inapplicability to university students is sorted from the times they are mentioned: "easy to use," "students are willing to participate," "student recognition," and "modify the use." These themes show the applicability of TTCT in the evaluation of the creativity of college students.Among them, more attention is paid to whether TTCT is easy to use and whether students are willing to participate.In the interview records, the respondents are most interested in whether TTCT is easy to use; students are willing to participate.It was mentioned 18 times, more than any other mentioned form.
It is the question that the respondents are most concerned about, "I think TTCT works great, and I've used it many times with good results" (IN2)."The students showed great interest in participating in the TTCT process.They were very happy and found it interesting" (IN6).
Other respondents also made it clear that they consider ease of use and the willingness of students to participate to be the most important.For example, they mentioned that "the ease of use of TTCT is a big reason why I use it" (IN3)."It is critical for me that students are willing to take this assessment" (IN4)."I am more concerned about the use of TTCT and whether students are willing to participate.If all are satisfied, I think it will go smoothly" (IN7).
At the same time, student recognition is significant for the credibility of the TTCT test."After completing a test once, I asked the students for their opinion.Most students feel that this testing method is fair.The few students who they recognized as unconstrained thinking have done very well" (IN10)."Students have said that this is a good way to see how others are creative" (IN7)."I have found that some students appreciate this approach; they will appreciate each other's creative performance" (IN9)."This way of student recognition can immerse students in it and get more accurate results" (IN11).
In addition, respondents will mention modifying the use, which can be considered that some items need to be adjusted during testing to achieve better results.
"I still have some problems after using it.I feel that some problems cannot be used directly.I changed it, and the students feel pretty good after using it" (IN5)."I think some items can be adjusted to highlight the key points" (IN12)."After adjusting some items, the effect is obviously much better" (IN10).
In proposition availability, they are sorted by the number of times mentioned: easy to do, material diversity, and lack of challenge.During the interviews, the researchers found that most respondents were concerned about the ease of administration of the test."It is still very easy for students to take this test.It provides a lot of materials, and these materials also provide a lot of creative directions, which can also achieve the purpose of testing students' creativity" (IN1)."The direction of this test proposition is very clear and easy to complete" (IN9)."I proceeded in the order of the propositions, and each item was easy to complete" (IN4).On the one hand, "there are quite a lot of materials provided by this test, which is also helpful to students.However, I think some parts of the material can be reduced" (IN12).The respondent (IN7) said, "Material diversity of TTCT gave a lot of references.""With so much material to inspire, students can get a lot of inspiration" (IN2).However, some respondents said that "they feel that there are too many materials, and I think it can be reduced appropriately so that it can better reflect the creativity of students" (IN4).
On the other hand, the respondents believe that in proposition availability, "during this test, it is not challenging for some students, because there is no uniform requirement for the final figure.Some respondents may be needed to judge the creativity of the animation major, and it is best to combine it with the major" (IN10).At the same time, "we believe that the creation based on existing materials is very open" (IN1).Respondent (IN2) stated that "the final graphics were not aimed at the animation profession." In the assessment process, they are sorted from the number of times mentioned, namely relaxed environment, no professionalism, highlight professional figure, and unified operation.This way of testing makes it easy for students."I found that students were very relaxed when taking this test.I also asked the students, and they said that this test is not as stressful as other exams; it is easy.I think it is a good way for them to use their creativity in this relaxed environment" (IN9).Respondent (IN7) also felt that this kind of testing created a relaxing environment, just like doing it in a game."It is a method different from traditional tests, which can be done in a very relaxed state, and students also find it very interesting" (IN10).
However, the respondents also found some phenomena based on the assignments submitted by the students.Such as, "during the test, students may not be very obvious about their professionalism.The figural designed by many students are very simple, and it seems that any major can achieve it" (IN7).In addition, "there is lack of professionalism in student work" (IN9)."Although the graphics in the final design are wild and unconstrained, there are not many features of animation design" (IN3).
Some respondents pointed out that they should highlight professional figural for the problems encountered in the test."During the test, I found that the expression of animation professional figural should be emphasized, so that the creativity of the animation direction can be expressed better, which will be better.It is suggested that the final patterns of the three types of exam questions should be close to the animation character design" (IN6)."If the figural of the animation major can be emphasized, I think such a test can better reflect the professionalism" (IN4)."We can judge the creativity of students through the figural of the animation major" (IN10).
At the same time, some respondents also suggested that "during the test, I think the four variables of this test are quite convenient.We will evaluate according to these four variables, and the operation is still unified" (IN8).The best thing about this test is that there are four clear variables.It is very uniform to evaluate creativity according to such a unified requirement" (IN12).It can be seen that this unified operation can clarify the criteria for consideration, which is better.
In the score criterion, they are sorted from the number of times they are mentioned, namely difficult to score individual variables, the scoring method is not uniform, less efficient, and easy to score.Some respondents said that "when counting scores, I found that it is difficult to count the grades of individual variables.Because it involves a lot of figural work, I need to spend a lot of time to count" (IN4)."Sometimes individual variables are more difficult to calculate because the scoring criteria for the four variables are different" (IN5)."There are a large number of graphic designs that require statistical scores, which indeed brings certain difficulties" (IN9).At the same time, the respondents found that "there are differences in the scoring methods of the four variables.For me, it has brought some difficulties when counting the scores, and I feel that it is not convenient enough.I think the scoring method can be unified according to the characteristics of the profession" (IN5)."When counting scores, if the four variables can be standardized, it will bring convenience to scoring" (IN10)."If the final examination questions are unified according to the characteristics of the animation major, it is also reasonable to unify the scoring standards for the four variables" (IN12).Also, "when I was scoring, I felt that the efficiency was a bit low, which may have something to do with the inconsistent scoring methods.It would be convenient if the scoring methods were the same" (IN11)."The efficiency is relatively low in terms of the examiner's scoring.(IN2).
"On the whole, this scoring method is still simple, mainly based on counting.It is still going well" (IN2)."It is relatively easy to score according to the number of creative points, and it also objectively reflects the quality of the work" (IN6)."The advantage of the scoring method is that it is easy to calculate the score and there is no subjective scoring by the examiner" (IN9).

Conclusion
From the result, we show four codes: "applicability to university students," "proposition availability," "assessment process," and "score criterion."According to the "applicability to student" aspect of TTCT, professional lecturers who have used this method multiple times have recognized it as a reliable way to measure the creativity of college students.As a result, it has become frequently used.Both lecturers and students find the TTCT easy to use, and students are happy and willing to take the test because they perceive it as fair.Furthermore, students show interest because they see the work of some of the most creative students.However, professional lecturers also found some problems using it and finally modified the TTCT to get a good response.
In "proposition availability," TTCT is relatively easy for students to complete the test content because it provides diverse materials that inspire students.However, professional lecturers think that some materials can be reduced, and some difficulties can be increased.During the test, the professional lecturers believed that the test questions would not be challenging for university students, and the final figure was not unified, so it should be combined with the animation major.It is a reasonable suggestion because the main target group is animation majors, and it should be more in line with their professional characteristics.
When students take this test in the "assessment process," they say it differs from other courses' exams.The assessment is conducted in a relaxed environment where students can freely express their creativity.This approach is also acknowledged as an effective assessment method.At the same time, the four variables in TTCT present the advantages of convenience and unity.
Although everyone recognized the examination method, professional teachers were concerned about whether it could better reflect its professionalism.They think that if there is no clear goal to express creativity according to the original figural, other majors can also complete it, then the expression of creativity for animation majors is insufficient.Therefore, professional teachers said they should emphasize the expression of animation professional figural, and the final figural direction of the three test questions can be set as animation character design.This method clarifies TTCT's consideration for animation majors, and it is a feasible method.
On the "score criterion," TTCT has attracted the attention of professional lecturers, although this scoring method is straightforward, mainly based on counting.However, some variables in the scoring method are not unified, which brings difficulties and reduces efficiency.Therefore, it can be counted uniformly according to professional characteristics, which is advantageous for observing the performance of overall creativity.
In summary, the results show that TTCT is an effective method suitable for measuring the creativity of Chinese university students, especially animation students.
Although the research results show that TTCT is an important method for measuring Chinese university students' creativity, this study also has limitations.This research mainly aims at university students majoring in animation in China, and it cannot explain the specific situation of university students majoring in other majors.In future studies, the impact of TTCT on university students of other majors can be studied.At the same time, we can also think about how to make more specific modifications to the TTCT assessment questions and evaluation methods so that TTCT is more suitable for measuring the creativity of university students.

Table 1 :
Interviewee Profile and Coding

Table 2 :
Demographic Characteristic of Interviewee

Table 3 :
Word Cloud Generated of Sub-codes in Interviews