Content validation for a clinical decision support system in mastology
DOI:
https://doi.org/10.29289/2594539420260008Keywords:
algorithm, artificial intelligence, validation study, primary health care, breast neoplasms, clinical decision support systemsAbstract
To develop and validate the scientific content in mastology of 17 flowcharts as a knowledge base for a clinical decision support system using conversational artificial intelligence for use in primary health care, through expert judgment and calculation of the content validity index (CVI). Methods: This was a methodological content validation study that constitutes the first stage of the doctoral project “Validation of an Electronic Clinical Decision Support System in Mastology”, aimed at general practitioners, gynecologists, and nurses. The flowcharts were developed based on widely validated protocols prioritizing topics frequently encountered in primary health care. Seven experts were selected according to objective criteria. Evaluation was performed using a four-point Likert scale administered via Google Forms®. The CVI for each flowchart was calculated as the proportion of agreement (“strongly agree” + “mostly agree”), with a cutoff point of 0.78 for panels of 6–10 experts. A thematic qualitative analysis of the suggestions was also conducted. Results: All flowcharts achieved CVI≥0.86, and 16 reached a CVI=1.00. Responses were concentrated in agreement categories (42.9–100.0% “totally agree”), with a single instance of “totally disagree” (14.3%). Qualitative suggestions were specific and focused on adding differential diagnoses, improving alignment with the Unified Health System protocols and refining referral priorities. Conclusions: The flowcharts showed high content validity and constitute a reliable scientific knowledge base for conversational artificial intelligence systems based on retrievalaugmented generation applied to breast diseases in primary health care.
Downloads
References
1. Migowski A, Azevedo e Silva G, Dias MBK, Diz MDPE, Sant’Ana DR, Nadanovsky P. Diretrizes para detecção precoce do câncer de mama no Brasil. II – Novas recomendações nacionais, principais evidências e controvérsias. Cad Saúde Pública. 2018;34(6):e00074817. https://doi.org/10.1590/0102-311X00074817
2. Hernández-Leal MJ, Codern-Bové N, Pérez-Lacasta MJ, Cardona A, Vidal-Lancis C, Carles-Lavila M, et al. Development of support material for health professionals who are implementing shared decision-makiing in breast cancer screening: validation using the Delphi techique. BMJ Open. 2022;12:e052566. https://doi.org/10.1136/bmjopen-2021-052566
3. Instituto Nacional de Câncer José de Alencar Gomes da Silva. Parâmetros técnicos para detecção precoce do câncer de mama. Rio de Janeiro: Ministério da Saúde; 2022.
4. Instituto Nacional de Câncer José Alencar Gomes da Silva. Posicionamento sobre a detecção precoce do câncer de mama no Brasil: nota técnica. Rio de Janeiro: Ministério da Saúde; 2023.
5. Korom R, Kiptinness S, Adan N, Said K, Ithuli C, Rotich O, et al. AI-based Clinical Decision Support for Primary Care: A Real-World Study [preprint]. arXiv:2507.16947; 2024. https://doi.org/10.48550/arXiv.2507.16947
6. Instituto Nacional de Câncer José Alencar Gomes da Silva. Ações de Controle do Câncer de Mama - Detecção Precoce. Rio de Janeiro: Ministério da Saúde; 2023.
7. Leite SS, Áfio ACE, Carvalho LV, Silva JM, Almeida PC, Pagliuca LMF. Construção e validação de Instrumento de Validação de Instrumento de Validação de Conteúdo Educativo em Saúde. Rev Bras Enferm. 2018;71(Suppl 4):1732-8. https://doi.org/10.1590/0034-7167-2017-0648
8. Vieira TW, Sakamoto VTM, Moraes LC, Blatt CR, Caregnato RCA. Validation Methods of Nursing care protocols: an integrative review. Rev Bras Enferm. 2020;73(Suppl 5):e20200050. https://doi.org/10.1590/0034-7167-2020-0050
9. Silva FRR, Pereira RA, Souza AC, Gimenes FRE, Simino GPR, Dessote CAM, et al. Construção e validação de cartilha para cuidados paliativos domiciliares após alta hospitalar. Acta Paul Enferm. 2022;35:eAPE028112. http://dx.doi.org/10.37689/acta-ape/2022AO02812
10. Albuquerque MEF, Santos S, Martini LGC, Melo ECA, Noronha JAF, Vieira GACM, et al. Validação de conteúdo de um instrumento para consulta de enfermagem em infecções sexualmente transmissíveis. Rev Baiana Enferm. 2023;37:e52183. https://doi.org/10.18471/rbe.v37.52183
11. Instituto Nacional de Câncer José Alencar Gomes da Silva. Detecção precoce do câncer de mama: ações para gestores e profissionais de saúde. Rio de Janeiro: Ministério da Saúde; 2023.
12. National Comprehensive Cancer Network. NCCN Clinical Practice Guidelines in Oncology: Breast cancer screening and diagnosis. Version 2.2026. Plymouth Meeting: NCCN; 2024.
13. American College of Radiology. ACR BI-RADS atlas: breast imaging reporting and data system. 5a ed. Reston, VA: American College of Radiology, 2013.
14. Elias S, Facina G, Araújo Neto JT. Mastologia: Condutas atuais. São Paulo: Manole, 2016.
15. Cuzick J. IBIS breast cancer risk evaluation tool [cited on 2026 May 30]. Londres: Wolfson Institute of Preventive Medicine, Queen Mary University of London; [s.d.]. Available from: https://ibis.ikonopedia.com
16. Muro N, Larburu N, Kerexeta J, Artola G, Arrue M, et al. Architecture for a Multimodal and Domain-Independent Clinical Decision Support System Software Development Kit. Annu Int Conf IEEE Eng Med Biol Soc. 2019;1399-404. https://doi.org/10.1109/embc.2019.8856459
17. Fehring RJ. Methods to validate nursing diagnosis. Heart Lung. 1987;16(6):625-9.
18. Pasquali L. Psicometria. Rev Esc Enferm USP. 2009;43:922-9. https://doi.org/10.1590/S0080-62342009000500002
19. Gu Y, Andargoli AE, Mackelprang JL, Meyer D. Design and implementation of clinical decision support systems in mental health helpline Services: A systematic review. Int J Med Inform. 2024;186:105416. https://doi.org/10.1016/j.ijmedinf.2024.105416
20. Fernandes TF, Pinho L, Brito MFSF, Lima CCM, Caldeira AP. Elaboração e Validação de Conteúdo de um instrumento sobre as atividades dos Agentes Comunitários de Saúde. Esc Anna Nery. 2022;26:e20220070. https://doi.org/10.1590/2177-9465-EAN-2022-0070pt
21. Ghorayeb A, Darbyshire JL, Wronikowska MW, Watkinson PJ. Design and validation of a new Healthcare Systems Usability Scale (HSUS) fol clinical decision support systems: a mixed-methods approach. BMJ Open. 2023;13(1):e065323. https://doi.org/10.1136/bmjopen-2022-065323
22. Hendriks MP, Verbeek XAAM, Vegchel TV, Sangen MJCV, Strobbe LJA, Merkus JWS, et al. Transformation of the National Breast Cancer Guideline into Data Driven Clinical Decision Trees. JCO Clin Cancer Inform. 2019;3:1-14. https://doi.org/10.1200/cci.18.00150
23. Tsopra R, Fernandez X, Luchinat C, Alberghina L, Lehrach H, Vanoni M, et al. A framework of validating AI in precisiona medicine: considerations from the European ITFoC consortium. BMC Med Inform Decis Mak. 2021;21(1):274. https://doi.org/10.1186/s12911-021-01634-3
24. Mazo C, Kearns C, Mooney C, Gallagher WM. Clinical decision support systems in breast cancer: A systematic review. Cancer (Basel). 2020;12(2):369. https://doi.org/10.3390/cancers12020369
25. Medeiros RKS, Ferreira Júnior MA, Pinto DPSR, Vitor AF, Santos VEP, Barichello E. Modelo de validação de conteúdo de Pasquali nas pesquisas em Enfermagem. Rev Enf Ref. 2015;4(4):127-35.
26. Santos CLJ, Silva AS, Nunes WB, Oliveira JS, Acioly CMC, Ferreira TMC, et al. Validity of a booklet to promote the health of people with diabetes in the face of COVID-19. Rev Bras Enferm. 2023;76(Suppl 1):e20220472.
27. Santos CPRS, Goyanna NF, Corpes EF, Yanez RJV, Mourão Netto JJ, Barbosa RCM, et al. Validação de conteúdo e guia de orientação sobre autocuidado no pós-operatório de câncer de mama. Rev Bras Enferm. 2024;77(4):e20240188. https://doi.org/10.1590/0034-7167-2024-0188pt
28. Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciênc Saúde Colet. 2011;16(7):3061-8. https://doi.org/10.1590/S1413-81232011000800006
29. Pereira FGF, Rocha DJL, Melo GAA, Jaques RMPL, Formiga LMF. Construção e validação de aplicativo digital para ensino de instrumentação cirúrgica. Cogitare Enferm. 2019;24:e58334. https://doi.org/10.5380/ce.v24i0.58334
30. Rostirolla LM, Dalchiavon C, Cardoso JK, Vendruscolo C, Adamy EK. Validação de conteúdo e semântica de manual para coleta de dados na consulta do enfermeiro. OLEL. 2025;23(9):e11534. https://doi.org/10.55905/oelv23n9-113
31. Tibes-Charman C. Tecnologia computacional para gerenciar o cuidado e indicadores relacionados à úlcera por pressão [tese]. Ribeirão Preto: Universidade de São Paulo; 2018.
32. Silva BASA, Silva PRR, Brito WRS, Coimbra MA, Batista JFC, Matos MLSS, et al. Processos de validação de instrumentos para área da saúde. REAS. 2024;24(2):e14695. https://doi.org/10.25248/reas.e14695.2024
33. Silva F. Elaboração e validação de conteúdo de um protocolo clínico em fluxograma para o manejo de pessoas com estomia na atenção primária. In: Anais do XV Congresso Brasileiro de Estomaterapia; 2025. Florianópolis, Brasil. Florianópolis: SOBEST; 2025.
34. Souza FMLC, Santos WN, Dantas JC, Sousa HRA, Moreira OAA, Silva RAR. Desenvolvimento de aplicativo de acompanhamento pré-natal e validação. Acta Paul Enferm. 2022;35:eAPE01861. https://doi.org/10.37689/acta-ape/2022AO01861
35. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. https://doi.org/10.1038/s41746-020-0221-y
36. Amorim TKB, Michaelsen S, Natalio MA. Desenvolvimento e validação de conteúdo do questionário de avaliação do conhecimento sobre AVC para agentes de saúde. Cad Saúde Colet. 2025;33(2):e33020396. https://doi.org/10.1590/1414-462X202533020396
37. Tibes CMS. Aplicativo Móvel para prevenção e classificação de Úlceras por Pressão [dissertação]. São Carlos: Universidade Federal de São Carlos; 2015.
38. Duarte FHS, Araújo NM, Silva SO, Leal NTB, Costa TMS, Alencar IGM, et al. Validação do conteúdo de um recurso audiovisual para pessoas vivendo com HIV. Acta Paul Enferm. 2024;37:eAPE01361. https://doi.org/10.37689/acta-ape/2024AO0001361
39. Muniz MLC, Galindo Neto NM, Sá GGM, Pereira JCN, Nascimento MC, Santos CS. Construção e validação de vídeo educativo para estudantes de enfermagem sobre a parada cardiorrespiratória obstétrica. Esc Anna Nery Rev Enferm. 2022;26:e20210466. https://doi.org/10.1590/2177-9465-EAN-2021-0466pt
40. Soutinho LAR, Machado DA, Marques CHD. Protocolo de rastreio multiprofissional de disfagia em pacientes com infecção HIV: elaboração e validação de conteúdo. CoDAS. 2022;34(2):e20210012. https://doi.org/10.1590/2317-1782/20212021012
41. Vasey B, Ursprung S, Beddoe B, Taylor EH, Marlow N, Bilbro N, et al. Association of Clinician Diagnostic Performance With Machine Learning–Based Decision Support Systems: A Systematic Review. JAMA Netw Open. 2021;4(3):e211276. https://doi.org/10.1001/jamanetworkopen.2021.1276
42. Gernier F, Grellard JM, Dupont C, Castel H, Fernette M, Lahaye F, et al. Impact of web application support versus standard management on adherence with adjuvante hormone therapy in patients treated for breast cancer: the WEBAPPAC study. BMC Cancer. 2023;23(1):736. https://doi.org/10.1186/s12885-023-11242-1
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Beatriz Daou Verenhitach, Simone Elias
This work is licensed under a Creative Commons Attribution 4.0 International License.
