Skip to main content Skip to main navigation menu Skip to site footer
Articles
DOI: 10.14254/1795-6889.2025.21-1.5
Submitted: 2023-12-30
Published: 2025-05-28

Designing a framework for ethnography-driven prompt engineering in social work

Institute of Sociology and Social Work, Vilnius University
Institute of Computer Science, Vilnius University
Institute of Sociology and Social Work, Vilnius University
Lithuania Business College
Kauno kolegija Higher Education institution
large language models prompt engineering ethnography social work

Abstract

This study proposes a framework for integrating ethnographic principles into prompt engineering (PE) for large language models (LLMs) in social work. While PE has emerged as a key methodology for optimizing LLM outputs, it often lacks grounding in the cultural contexts of end users. By bridging theoretical approaches and methodologies from social and computer sciences, the proposed framework addresses the limitations of a strictly semantic approach in PE. The framework’s theoretical foundations are grounded in empirical data gathered during an ethnographic field study conducted within a social service organization. Ten interviews were analysed following the key stages of the ethnographic analysis method. Key cultural themes composed of multiple semantic relationships were uncovered and then connected to core prompt components. These components were further elaborated into various prompting techniques and developed into a set of prompt templates that can be applied to LLM evaluation and further customization.

Metrics

Metrics Loading ...

References

  1. Amatriain, X. (2024). Prompt design and engineering: Introduction and advanced methods. arXiv preprint arXiv:2401.14423. https://doi.org/10.48550/arXiv.2401.14423
  2. Arksey, H., & O'Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616 DOI: https://doi.org/10.1080/1364557032000119616
  3. Atkinson, P. (2007). Ethnography: Principles in Practice (3rd ed.). Routledge. https://doi.org/10.4324/9780203944769 DOI: https://doi.org/10.4324/9780203944769
  4. Bernard, H. R. (2017). Research methods in anthropology: Qualitative and quantitative approaches (5th ed.). Rowman & Littlefield.
  5. Blomberg, J., Suchman, L., & Trigg, R. (2017). Ethnography and the construction of computer science. In J. M. Suchman, L. Trigg, & J. Blomberg (Eds.), Ethnography and the construction of computer science (pp. 1–15). Springer.
  6. Brown, T. B. (2020). Language models are few-shot learners. arXiv preprint arXiv:2005.14165.
  7. Collins, H., & Yearley, S. (1992). Epistemological chicken. In A. Pickering (Ed.), Science as practice and culture (pp. 301–326). Chicago, IL: University of Chicago Press.
  8. Dicks, B., Coffey, A., & Mason, B. (2005). Qualitative research and hypermedia: Ethnography for the digital age. DOI: https://doi.org/10.4135/9781849209649
  9. Ein-Dor, L., Toledo-Ronen, O., Spector, A., Gretz, S., Dankin, L., Halfon, A., Katz, Y., & Slonim, N. (2024). Conversational prompt engineering. arXiv preprint arXiv:2408.04560. https://doi.org/10.48550/arXiv.2408.04560
  10. Farchi, C., Touzi, B., Farchi, F., & Mousrij, A. (2021). Sustainable performance assessment: A systematic literature review. Journal of Sustainable Development of Transport and Logistics, 6(2), 124-142. https://doi.org/10.14254/jsdtl.2021.6-2.8 DOI: https://doi.org/10.14254/jsdtl.2021.6-2.8
  11. Grint, K., & Woolgar, S. (1997). The machine at work: Technology, work and organization. Cambridge, UK: Polity Press.
  12. Hammersley, M., & Atkinson, P. (2019). Ethnography: Principles in practice (4th ed.). Routledge. https://doi.org/10.1177/1357034X04042943 DOI: https://doi.org/10.4324/9781315146027
  13. Jiang, E., Olson, K., Toh, E., Molina, A., Donsbach, A., Terry, M., & Cai, C. (2022). PromptMaker: Prompt-based prototyping with large language models. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 1–8. https://doi.org/10.1145/3491101.3503564 DOI: https://doi.org/10.1145/3491101.3503564
  14. Kela. (2021). Kelan vuosi 2020 [Kela’s annual report 2020]. https://tietotarjotin.fi/documents/20124/40278/kelan-vuosi-2020.pdf
  15. Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., Santhanam, K., Vardhamanan, S., ... & Potts, C. (2023). Dspy: Compiling declarative language model calls into self-improving pipelines. arXiv preprint arXiv:2310.03714. https://doi.org/10.48550/arXiv.2310.03714
  16. Kling, R., McKim, G., & King, A. (2003). A bit more to it: Scholarly communication forums as socio-technical interaction networks. Journal of the American Society for Information Science and Technology, 54(1), 47–67. https://doi.org/10.1002/asi.10154 DOI: https://doi.org/10.1002/asi.10154
  17. Kling, R., Rosenbaum, H., & Sawyer, S. (2005). Understanding and communicating social informatics: A framework for studying and teaching the human contexts of information and communication technologies. Medford, NJ: Information Today.
  18. Kolková, A., & Ključnikov, A. (2022). Demand forecasting: AI-based, statistical and hybrid models vs practice-based models - the case of SMEs and large enterprises. Economics and Sociology, 15(4), 39-62. doi:10.14254/2071-789X.2022/15-4/2 DOI: https://doi.org/10.14254/2071-789X.2022/15-4/2
  19. Lamb, R., & Sawyer, S. (2005). On extending social informatics from a rich legacy of networks and conceptual resources. Information Technology & People, 18(1), 9–20. https://doi.org/10.1108/09593840510584595 DOI: https://doi.org/10.1108/09593840510584595
  20. Latour, B. (2004). How to talk about the body? The normativity of the body and the normative of the bodies in science. Body and Society, 10(2-3), 205-229. DOI: https://doi.org/10.1177/1357034X04042943
  21. Latour, B. (2005). Reassembling the social: An introduction to actor-network-theory. Oxford University Press. DOI: https://doi.org/10.1093/oso/9780199256044.001.0001
  22. Law, J. (2008). On sociology and STS. The Sociological Review, 56(4), 623–649. https://doi.org/10.1111/j.1467-954X.2008.00808.x DOI: https://doi.org/10.1111/j.1467-954X.2008.00808.x
  23. Liu, V., & Chilton, L. B. (2022). Design guidelines for prompt engineering text-to-image generative models. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, Article 384, 1–23. https://doi.org/10.1145/3491102.3501825 DOI: https://doi.org/10.1145/3491102.3501825
  24. Liu, Y., Deng, G., Li, Y., Wang, K., Zhang, T., Liu, Y., Wang, H., Zheng, Y., & Liu, Y. (2023). Prompt injection attack against LLM-integrated applications. arXiv preprint arXiv:2306.05499. https://arxiv.org/abs/2306.05499
  25. Liu, Y., Jia, Y., Geng, R., Jia, J., & Gong, N. Z. (2024). Formalizing and benchmarking prompt injection attacks and defenses. arXiv preprint arXiv:2310.12815. https://arxiv.org/abs/2310.12815
  26. Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2022). Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786. https://arxiv.org/abs/2104.08786 DOI: https://doi.org/10.18653/v1/2022.acl-long.556
  27. Mahdavi Goloujeh, A., Sullivan, A., & Magerko, B. (2024). The social construction of generative AI prompts. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’24) (pp. 1–7). ACM. https://doi.org/10.1145/3613905.3650947 DOI: https://doi.org/10.1145/3613905.3650947
  28. Meskó, B. (2023). Prompt engineering as an important emerging skill for medical professionals: Tutorial. Journal of Medical Internet Research, 25, e50638. https://doi.org/10.2196/50638 DOI: https://doi.org/10.2196/50638
  29. Murchison, J. (2010). Ethnography essentials: Designing, conducting, and presenting your research. SAGE Publications.
  30. Napieralski, P., Wojciechowski, A., & Korjonen-Kuusipuro, K. (2024). Artificial intelligence for scientific research – large language models in manuscript preparation. Human Technology, 20(3), 416–419. https://doi.org/10.14254/1795-6889.2024.20-3.0 DOI: https://doi.org/10.14254/1795-6889.2024.20-3.0
  31. Norman Donald, A. (2013). The design of everyday things. MIT Press.
  32. O’Dwyer, G. (2020). Swedish municipalities test AI to drive efficiencies and cost savings. https://www.computerweekly.com/news/252480145/Swedish-municipalities-test-AI-to-drive-efficiencies-and-cost-savings
  33. Ranade, N., Saravia, M., & Johri, A. (2024). Using rhetorical strategies to design prompts: A human-in-the-loop approach to make AI useful. AI & Society, 1-22. https://doi.org/10.1007/s00146-024-01905-3 DOI: https://doi.org/10.1007/s00146-024-01905-3
  34. Ratnayake, H., & Wang, C. (2023, November). A prompting framework to enhance language model output. In Australasian Joint Conference on Artificial Intelligence (pp. 66–81). Springer Nature Singapore.
  35. Ratnayake, H., & Wang, C. (2024). A prompting framework to enhance language model output. In T. Liu et al. (Eds.), AI 2023, LNAI 14472 (pp. 66–81). Springer Nature Singapore. https://doi.org/10.1007/978-981-99-8391-9_6 DOI: https://doi.org/10.1007/978-981-99-8391-9_6
  36. Razavi, N., & Sierpinski, G. (2024). An attempt to determine the impact of the implementation of autonomous vehicles on a larger scale on the planning of city transport systems. Journal of Sustainable Development of Transport and Logistics, 9(1), 96-120. https://doi.org/10.14254/jsdtl.2024.9-1.8 DOI: https://doi.org/10.14254/jsdtl.2024.9-1.8
  37. Ringes I. F. (2020) Frida Jobber Døgnet Rundt. Available at: https://memu.no/artikler/frida-jobber-dognet-rundt/
  38. Sanchez, T. (2023). Examining the Text-to-Image Community of Practice: Why and How do People Prompt Generative AIs? In Proceedings of the 15th Conference on Creativity and Cognition (C&C '23) (pp. 43–61). Association for Computing Machinery. https://doi.org/10.1145/3591196.3593051 DOI: https://doi.org/10.1145/3591196.3593051
  39. Sawyer, S., & Tyworth, M. (2006). Social informatics: Principles, theory, and practice. In J. Berleur, M. I. Nurminen, & J. Impagliazzo (Eds.), Social informatics: An information society for all? In remembrance of Rob Kling (pp. 49–62). Springer. https://doi.org/10.1007/978-0-387-37876-3_4 DOI: https://doi.org/10.1007/978-0-387-37876-3_4
  40. Schulhoff, S., Ilie, M., Balepur, N., Kahadze, K., Liu, A., Si, C., ... & Resnik, P. (2024). The prompt report: A systematic survey of prompting techniques. arXiv preprint arXiv:2406.06608.
  41. Seniutis, M., Gružauskas, V., & Sas, A. (2026). Exploring more-than-human ethics of large language models in social service delivery. Accepted for consideration in Ethics and Social Welfare: Special Issue on More-than-human Ethics for Social Work.
  42. Seniutis, M., Sas, A., & Gružauskas, V. (2024). Exploring large language models in social service delivery: Initial results of a systematic review. Human Centric AI: Ethics, Regulation and Safety, Vilnius. https://www.hcai.tf.vu.lt/
  43. Short, C., & Short, J. (2023). The artificially intelligent entrepreneur: ChatGPT, prompt engineering, and entrepreneurial rhetoric creation. Journal of Business Venturing Insights, 19, e00388. https://doi.org/10.1016/j.jbvi.2023.e00388 DOI: https://doi.org/10.1016/j.jbvi.2023.e00388
  44. Spradley, J. P. (2016a). The ethnographic interview (Revised edition). Waveland Press.
  45. Spradley, J. P. (2016b). Participant observation. Waveland Press.
  46. Spradley, J. P., & McCurdy, D. W. (2012). Conformity and conflict: Readings in cultural anthropology (14th ed.). Pearson.
  47. Suchman, L. A. (1987). Plans and situated actions: The problem of human-machine communication. Cambridge University Press.
  48. Tatnall, A. (2005). Actor-network theory and information systems research. In D. L. Brown & D. S. Remenyi (Eds.), Proceedings of the 13th European Conference on Information Systems (ECIS 2005) (pp. 1211–1219). European Conference on Information Systems. DOI: https://doi.org/10.4018/978-1-59140-553-5.ch009
  49. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., & Zhou, D. (2024). Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems (NIPS '22) (Article 1800, pp. 24824–24837). Curran Associates Inc.
  50. White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D. C. (2023). A prompt pattern catalog to enhance prompt engineering with ChatGPT. arXiv preprint arXiv:2302.11382. https://arxiv.org/abs/2302.11382
  51. White, S., Hall, C., & Peckover, S. (2009). The descriptive tyranny of the common assessment framework: Technologies of categorization and professional practice in child Welfare. British Journal of Social Work, 218, 39(7), 1197–1217. DOI: https://doi.org/10.1093/bjsw/bcn053
  52. Winograd, T. (1986). Understanding computers and cognition: A new foundation for design. Addison-Wesley.
  53. Wojciechowski, A., & Korjonen-Kuusipuro, K. (2023). How artificial intelligence affects education?. Human Technology, 19(3), 302–306. https://doi.org/10.14254/1795-6889.2023.19-3.0 DOI: https://doi.org/10.14254/1795-6889.2023.19-3.0
  54. Yüksekgönül, M., Bianchi, F., Boen, J., Liu, S., Huang, Z., Guestrin, C., & Zou, J. (2024). TextGrad: Automatic "differentiation" via text. arXiv preprint arXiv:2406.07496. https://doi.org/10.48550/arXiv.2406.07496
  55. Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023). Why Johnny can’t prompt: How non-AI experts try (and fail) to design LLM prompts. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23), Article 437, 1–21. https://doi.org/10.1145/3544548.3581388 DOI: https://doi.org/10.1145/3544548.3581388
  56. Zhuge, M., Zhao, C., Ashley, D., Wang, W., Khizbullin, D., Xiong, Y., ... & Schmidhuber, J. (2024). Agent-as-a-judge: Evaluate agents with agents. arXiv preprint arXiv:2410.10934. https://arxiv.org/abs/2410.10934

How to Cite

Seniutis, M., Gružauskas, V., Sas, A., Navickas, V., & Švažas, M. (2025). Designing a framework for ethnography-driven prompt engineering in social work. Human Technology, 21(1), 106–127. https://doi.org/10.14254/1795-6889.2025.21-1.5