Leveraging Computational Thinking in the Era of Generative AI – Communications of the ACM

Computational thinking is shaped in terms of the questions “What can humans do better than computers? And what can computers do better than humans?” (Wing, 2006), which a flood of papers have sought to answer and define which human cognitive and social skills are required for problem-solving processes in general, and for solving computational problems in particular. Today, the following skills are recognized as key components of computational thinking: problem formulation; problem decomposition; organization and logical analysis of data; data representation using models and simulations; abstraction; suggestion and assessment of multiple solutions to a given problem; implementation of the chosen solution; and generalization (e.g., Wing, 2010). Computational thinking skills are recognized today as important not only in the context of CS, but also as important skills for everyone in the 21^st century, and can be applied in various contexts (Günbatar, 2019; Wing, 2017).

Prompt Engineering

Prompt engineering refers to the “systematic practice of constructing prompts to improve the generated output of a generative model” (Dang et al., 2022). Prompt engineering is a new field of research, asking what the best practices are for shaping prompts for GenAI tasks in general, and for programming tasks in particular (Oppenlaender et al., 2023). From an educational perspective, researchers ask how to teach prompt engineering, and how to evaluate the results of such teaching and learning processes (Denny et al., 2023, 2024). Currently, for the majority of end users, writing effective prompts is largely a trial-and-error process (Dang et al., 2022). Massive online open courses (MOOCs) suppliers, such as Coursera, now offer courses in prompt engineering (https://www.coursera.org/learn/prompt-engineering-for-web-developers) that include topics such as breaking tasks into smaller steps, prompt iterativity, generating code with pseudocode, and test and edge cases. Figure 1 presents ChatGPT’s advice for the best practices for prompting.

Figure 1. Best practices for prompting: ChatGPT’s answer

GenAI apps are becoming an inevitable technology in the field of CS education (Armoni and Hazzan, in press). Students use them to learn, debug, and solve short coding exercises, wherein these apps excel. We suggest GenAI apps facilitate new opportunities for learning and teaching (Denny et al., 2022; Yilmaz and Karaoglan Yilmaz, 2023), and that therein, the importance of computational thinking increases due to its strong interaction with prompt engineering, as illustrated in Figure 2.

It has been found that learning how to prompt enhances the learning of several computational thinking skills. Specifically, students who use GenAI should describe clearly the steps needed to solve a problem (Denny et al., 2023), focus on the algorithm to determine its subprograms, and combine the sub-solutions into the entire solution (Yilmaz and Karaoglan Yilmaz, 2023). Efficient prompting can accordingly be viewed as writing pseudocode (Denny et al., 2022). After all, shaping prompts to produce specific results is a problem-solving process, and hence requires computational thinking.

To test the degree of resemblance between computational thinking and prompt engineering, we replaced “computational thinking” with “prompt engineering” in Wing’s paper (Wing, 2006), with the following results:

• Prompt engineering “is interpreting code as data and data as code.” (p. 33) as the prompt data should be treated as code, since efficient prompting is writing pseudocode; on the other hand, the prompt pseudocode is data for the LLM.
• Prompt engineering “is using abstraction and decomposition when attacking a large complex task.” (p. 33) as to implement large complex systems, the task should be broken down into small sub-tasks, each of them then articulated by a prompt.
• Prompt engineering “is having the confidence [that] we can safely use, modify, and influence a large complex system without understanding its every detail.” (p. 33), as using pseudocode, instead of a specific programming language, enables to perform [sic] the task without the need to master the details of a specific language.
• Finally, prompt engineering “is search, search, and more search, resulting in a list of Web pages, a strategy for winning a game, or a counterexample.” (p. 34) as efficient prompting involves experimenting with different prompts in learning processes and with iterative refinements.

Undergraduate CS Students’ Expressions of the Interrelationships between Computational Thinking and Prompt Engineering

Yael Erez, co-author of this blog, teaches an Introduction to Computer Science (CS1) course to CS undergraduate students. In the Fall 2023 semester, several tasks that involved problem-solving with the help of GenAI apps, such as ChatGPT, were assigned to the students. Two of these assignments are detailed in brief in Table 1.

Data collected using these assignments demonstrate the increased importance of computational thinking for prompt engineering:

• When the students fix the incorrect cyclic shift with the help of ChatGPT (Assignment #1), not only does assessment of the app solutions enhance their computational thinking skills, such as critical thinking and program comprehension, but the computational thinking skill of problem formulation is required for prompt engineering.
• When the students use the app to implement the contact list (Assignment #2), not only do the complexity and length of the implementation compel them, as part of the prompt engineering task, to decompose the problem into sub-problems, thereby enhancing their learning of the computational thinking skill of decomposition, but computational thinking skills such as successive refinement are required for prompt engineering.

While the above examples do not require the students to write code on their own, suggesting that GenAI apps may eliminate the need to master a specific language syntax, these tasks underscore the importance of students’ computational thinking, as the following data illustrates.

After each assignment submission, the students were asked to answer a questionnaire containing an open question: “Write down three new ideas that you learned from solving the question.” Many of the students’ answers to this question referred to the skills needed for solving programming problems with ChatGPT, demonstrating the strong interrelationships between computational thinking and prompt engineering.

One answer to the above-mentioned question was “Formulating the problem explicitly,” and another similar answer was “Accuracy in formulating requests,” indicating the importance of the computational thinking skill of problem formulation, and the importance of clarity and specificity for prompt engineering. Other students answered, “This time, unlike previous times, I sent it a few details each time and was able to lead it to solving the problem faster,” and “Reducing the problem to small tasks,” describing the problem decomposition phase of computational thinking and the iterative refinements of prompt engineering. Another student referred to system design and implementation, describing the process of “First understand the problem, and then direct the tools to solve the problem.” The computational thinking skills of critical thinking and solution evaluation required for prompt engineering were cited by another student, who wrote, “I knew what ChatGPT’s ability is, it doesn’t always answer correctly.”

Effective prompt engineering requires computational thinking skills such as problem formulation, problem decomposition, data organization and analysis, and suggestion and assessment of multiple solutions to a given problem. Accordingly, not only does experiencing prompt engineering improve our computational thinking skills, but computational thinking is required for prompt engineering.

These interactions between prompt engineering and computational thinking underscore the importance of computational thinking in the era of GenAI. While GenAI apps may facilitate programming tasks, even for complex systems, thereby eliminating the need to master a specific language syntax, they still lack computational thinking. Therefore, humans’ computational thinking is required in the GenAI era.

While we introduced the interactions between computational thinking and prompt engineering mainly in the context of CS, GenAI tools are available to all, and are widely used in fields other than CS, and by humans who are not computer scientists. For example, Koby Mike, co-author of this blog, teaches data science to social science graduate students. In these courses, following the incorporation of ChatGPT and prompt engineering in the course, while the volume of Python programming in the curriculum dramatically decreased, the volume of computational thinking expanded. This finding is aligned with Wing’s statement “Computational thinking is a fundamental skill for everyone, not just for computer scientists.” Continuing the exercise of replacing “computational thinking” with “prompt engineering” in Wing’s paper, evoking a discussion about computational thinking (Wing, 2006), the wide use of GenAI tools suggests that prompt engineering is a fundamental skill for everyone, not just for computer scientists.

When prompting efficiently, computational thinking is applied and improved, even for everyday tasks such as planning a trip, solving riddles, or knowledge expansion. If Wing wrote her viewpoint today, it would likely include the following: Computational thinking is required for having effective conversations with LLMs to solve problems efficiently and quickly. Hence Wing’s message to “make computational thinking commonplace” (Wing, 2006) has become a reality.

References

Armoni, Y. and Hazzan, O. (in press). The Inevitability of AI Technology in Education: Futurism Perspectives on Education in 10-20 years.

Dang, H., Mecke, L., Lehmann, F., Goller, S., and Buschek, D. (2022). How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Models.

Denny, P., Becker, B. A., Leinonen, J., and Prather, J. (2023). Chat Overflow: Artificially Intelligent Models for Computing Education – renAIssance or apocAIypse? Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1, 3–4. https://doi.org/10.1145/3587102.3588773

Denny, P., Kumar, V., and Giacaman, N. (2022). Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language (arXiv:2210.15157). arXiv. http://arxiv.org/abs/2210.15157

Denny, P., Leinonen, J., Prather, J., Luxton-Reilly, A., Amarouche, T., Becker, B. A., and Reeves, B. N. (2024). Prompt Problems: A New Programming Exercise for the Generative AI Era. Proceedings of the 55^th ACM Technical Symposium on Computer Science Education V. 1, 296–302. https://doi.org/10.1145/3626252.3630909

Günbatar, M. S. (2019). Computational thinking within the context of professional life: Change in CT skill from the viewpoint of teachers. Education and Information Technologies, 24(5), 2629–2652. https://doi.org/10.1007/s10639-019-09919-x

Mike, K., Ragonis, N., Rosenberg-Kima, R. B., and Hazzan, O. (2022). Computational Thinking in the Era of Data Science. Communications, 65(8), 33–35. https://doi.org/10.1145/3545109

Oppenlaender, J., Linder, R., and Silvennoinen, J. (2023). Prompting AI Art: An Investigation into the Creative Skill of Prompt Engineering.

Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. Basic Books.

Wing, J. M. (2006). Computational Thinking. Communications, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215

Wing, J. M. (2010). Computational Thinking: What and Why? http://www.cs.cmu.edu/~CompThink/resources/TheLinkWing.pdf

Yilmaz, R., and Karaoglan Yilmaz, F. G. (2023). The Effect of Generative Artificial Intelligence (AI)-based Tool Use on Students’ Computational Thinking Skills, Programming Self-Efficacy and Motivation. Computers and Education: Artificial Intelligence, 4, 100147. https://doi.org/10.1016/j.caeai.2023.100147

Yael Erez is a lecturer in the Department of Computer Science of the Technion—Israel Institute of Technology, and a faculty member in the Department of Electrical Engineering of the Braude College of Engineering. She has BSc and MSc degrees in Electrical Engineering from the Technion, and is currently a Ph.D. student in the Technion’s Faculty of Education in Science and Technology.

Koby Mike is a postdoctoral fellow in the Gender Studies program at Bar-Ilan University; he earned BSc and MSc degrees in Electrical Engineering from Tel Aviv University, and a Ph.D. in Education in Science and Technology from the Technion.

Orit Hazzan is a professor at the Technion’s Faculty of Education in Science and Technology. Her research focuses on computer science, software engineering, and data science education; for additional details, see https://orithazzan.net.technion.ac.il/.