Suggestions
Idioma
Metrics
Journal Information
Visits
18
Original
Full text access

Key elements for successful healthcare procurement practices: insights from a Delphi study

Elementos clave para unas prácticas de contratación pública sanitaria exitosa: conclusiones de un estudio Delphi
Visits
18
Iris Menéndez Fernándeza,
Corresponding author
irismf14@gmail.com

Corresponding author.
, María José Vicente Edoa, José López Puya, Braulio Aguilar Utrillab
a Aragonese Institute of Health Sciences (IACS), Zaragoza, Spain
b Aragonese Health Service (SALUD), Zaragoza, Spain
This item has received
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Figures (6)
fig0005
fig0010
fig0015
fig0020
202611201600577901
fig0025
Tables (2)
Table 1. Final list of criteria and categories that represent the key elements for a successful healthcare public procurement process.
Tables
Table 2. Priority distribution of the criteria.
Tables
Additional material (1)
Abstract
Objective

To identify, through a Delphi consensus study, the key criteria and categories that define a successful and crisis-resilient healthcare public procurement (HCPP) process across the European Union.

Method

Within the European project PROCURE, a Real-Time Delphi study was conducted with a multi-stakeholder expert panel (procurers, suppliers, regulators, policymakers, academics and healthcare professionals) from 10 European countries. An initial questionnaire, developed from a structured literature review, was assessed over two online consultation rounds using a six-point Likert scale, followed by a face-to-face meeting to reach final consensus on the wording of the retained criteria and categories.

Results

The initial questionnaire (9 criteria, 60 categories) was refined into a final consensus-based framework of 9 criteria and 43 categories. Categories relating to tender, monitoring and control of execution, governance and accountability, and emergency preparedness were ranked as the highest priority. Key insights included the need for a specific, EU-driven legal framework for HCPP, and for standardized metrics and definitions for concepts such as sustainability or monitoring and control of execution.

Conclusions

This is, to our knowledge, the first pan-European, multi-stakeholder Delphi consensus framework for HCPP, spanning the full procurement lifecycle with dedicated attention to crisis resilience. It provides a validated, prioritized basis for designing, evaluating or improving HCPP processes, and will inform European policy recommendations and country-specific action plans within the PROCURE project.

.

Keywords:
Delivery of health care
Purchasing
hospital
Delphi technique
Health policy
Consensus
Surveys and questionnaires
Organizational innovation
Resumen
Objetivo

Identificar, mediante un estudio Delphi de consenso, los criterios y las categorías clave que definen un proceso de compra pública sanitaria (CPSS) exitoso y resiliente frente a crisis en la Unión Europea (UE).

Método

En el marco del proyecto europeo PROCURE, se realizó un estudio Delphi en tiempo real con un panel de expertos (compradores, proveedores, reguladores, responsables políticos, académicos y profesionales sanitarios) de 10 países europeos. Un cuestionario inicial, basado en una revisión bibliográfica estructurada, se evaluó en dos rondas de consulta online (escala Likert de seis puntos) y un encuentro presencial para consensuar la redacción de los criterios y las categorías retenidos.

Resultados

El cuestionario inicial (9 criterios, 60 categorías) se refinó hasta un marco final consensuado de 9 criterios y 43 categorías. Las categorías de licitación, seguimiento y control de la ejecución, gobernanza y rendición de cuentas, y preparación ante emergencias, se clasificaron como prioritarias. Entre los principales hallazgos destaca la necesidad de un marco legal específico impulsado por la UE para la CPSS, así como de métricas y definiciones estandarizadas para conceptos como sostenibilidad o seguimiento y control de la ejecución.

Conclusiones

Hasta donde sabemos, este es el primer marco de consenso Delphi paneuropeo y multidisciplinario para la CPSS, que abarca todo el ciclo de contratación con especial atención a la resiliencia frente a crisis. Ofrece una base validada y priorizada para diseñar, evaluar o mejorar procesos de CPSS, y servirá de punto de partida para las recomendaciones políticas europeas y los planes de acción por Estado miembro de PROCURE.

Palabras clave:
Prestación de atención de salud
Contratación en hospitales
Técnica Delphi
Política de salud
Consenso
Encuestas y cuestionarios
Innovación organizativa
Graphical abstract
Full Text
Introduction

Public procurement in the healthcare sector faces growing complexity, driven by the increasing diversity of products and services, new technologies, and global supply chains. The COVID-19 pandemic evinced the vulnerabilities of the process, revealing an urgent need to improve the prevailing practices and procedures.

Problems identified during the pandemic included slow response and lack of flexibility, absent stockpiles, supply chain disruptions, and fraud, corruption and e-procurement risks,1,2 underscoring the lack of common good practices and guidelines and the need for a space for experts to exchange opinions and experience.3

Given healthcare's relevance and its position as the second largest government spending area4, improving a process that has such a significant impact in this field, namely public procurement, is considered a priority. With this aim, the European project PROCURE (Public Procurement Assessment in the Healthcare Sector), the project underpinning this study, was conceived. The initial part of the project, The Observational Study,5 highlighted the differences in procurement practices among European countries.

Existing literature on healthcare public procurement (HCPP), however, remains fragmented and largely descriptive. Prior studies have addressed isolated dimensions of the process, such as the degree of centralization of purchasing structures, corruption and transparency risks, or the underuse of innovation-oriented procurement instruments, without offering an integrated, empirically validated framework of the factors that jointly determine procurement success across the diversity of European health systems.6–8 Moreover, most available guidance stems from narrow-scope policy reports4,9, topic-specific narrative reviews,10 or single-country analyses11, and, to date, no study has systematically elicited and validated, through structured multi-stakeholder expert consensus, a comprehensive and prioritized set of criteria applicable at European Union (EU) level. This gap is particularly salient given the persistent heterogeneity in HCPP practices already documented across Member States by the PROCURE Observational Study.5

Against this background, the present study addresses the following research question: which criteria and categories do European multi-stakeholder experts (procurers, suppliers, regulators, policymakers, academics and healthcare professionals) consider essential for a successful and crisis-resilient HCPP process, and how can these be structured into a validated, consensus-based framework applicable across EU Member States? To answer this question, a Delphi consensus methodology was applied, given its recognized value for building structured, unbiased agreement among heterogeneous expert groups on complex, poorly standardized domains.12 More specifically, a real-time Delphi (RTD) approach was used, a validated variant of the classical Delphi method.13,14

To our knowledge, this is the first study to apply this methodology at pan-European level to HCPP, integrating the perspectives of all key stakeholder groups into a single validated and prioritized framework of success criteria, rather than a descriptive account of pandemic-related challenges or individual lessons learned,2 with dedicated attention to procurement resilience during health emergencies.

Method

A RTD approach was used to identify and reach consensus on the key elements for successful HCPP: a variant of the classical Delphi method in which panelists’ responses and group statistics are displayed in real time during online voting, improving efficiency and facilitating agreement across a geographically dispersed, multidisciplinary expert group.13,14 This approach, illustrated in the graphical abstract, combined two structured online consultation rounds using Likert scale15 questionnaires with a final face-to-face meeting to reach semantic consensus.

Phase 1: the initial Delphi questionnaire

An expert librarian conducted a structured literature search on HCPP (2014-2024; five languages; PubMed/MEDLINE, Embase, Web of Science, Scopus and TripDatabase Recolecta), complemented by a parallel search of national and European organizations (OECD, WHO, EU) and further references contributed by PROCURE Work Package leaders; the complete search strategy and document-type criteria are detailed in Supplementary Material (S1 and Table S1).

A multidisciplinary working group was constituted to review and analyze the retrieved documents. Following a two-hour standardization workshop led by an independent methodologist, each document was assigned to a single reviewer, who completed a standardized data-extraction grid summarizing its content and potential contribution to the questionnaire.

The initial items, criteria and domains were not predefined but emerged from this process: extraction-grid findings, together with the stakeholder-mapping matrix from the earlier Observational Study (WP2), were jointly discussed and synthesized on the collaborative platform Miro, triangulating the evidence through three complementary lenses (resilience-oriented, procurement process-lifecycle, and OECD-aligned); the process-lifecycle lens most closely informed the final questionnaire structure (Fig. 1). The resulting draft statements were reviewed by external experts from CORVERS, SILO and the Council of Europe Development Bank.

Figure 1.

Preparation process and structure of the initial Delphi questionnaire.

Phase 2: setting the expert panel

Recruitment aimed for a diverse panel, a balanced composition considered essential for robust, unbiased Delphi consensus.16 The Delphi leading team defined seven stakeholders profiles: Procurers, suppliers, healthcare regulators, health policy makers, academic, researchers, healthcare professionals and end-users of healthcare services.

The panel combined participants from within the PROCURE consortium and external experts identified through partners’ professional networks, selected to ensure gender balance, geographical representation and stakeholder diversity (Fig. 2; eligibility requirements in Supplementary Material, S3 and Table S2). No patient representative ultimately participated: patients and patient organizations were invited but either did not respond or could not commit to the full process, including the face-to-face meeting.

Figure 2.

Profiles and characteristics of the Delphi expert panel.

There is no international consensus on optimal Delphi panel size17; the ideal size is generally understood to depend on the scope of the problem and available resources.

Phase 3: two online rounds

The online consultation process consisted of two rounds. In the first round (16/09/2024 - 06/10/2024), experts judged the relevance of each category for HCPP using a six-point Likert scale (1=not relevant at all, 6=of most relevance), via the platform ‘SmartDelphi’, designed and customized by Onsanity,18 which displayed each expert's answer alongside the group median in real time, enabling participants to respond in light of the emerging consensus.

The real-time display of group responses during online voting, a feature of the SmartDelphi platform, reflects a deliberate design choice rather than an unintended deviation from classical Delphi principles. This real-time display reflects a deliberate design choice consistent with the RTD variant, introduced to improve efficiency and reduce panel attrition;13,14 methodological comparisons have found that this “feedback effect” does not substantially distort final results relative to conventional multi-round Delphi.14,19 In the present study, this feature was deliberately used to facilitate a more efficient convergence process across a geographically dispersed, multidisciplinary panel, consistent with the RTD rationale.

Item relevance was determined by the median score, and level of agreement by the interquartile range (IQR), following established methodological guidance for Delphi and consensus-panel studies in healthcare.20,21

Given the six-point scale used, IQR thresholds were set more strictly for borderline median scores (4) than for clearly high ones (6), and an IQR of 3 or more (spanning half of the scale's range) was consistently considered indicative of substantial disagreement regardless of median. These thresholds were adapted, with methodological support from the SmartDelphi developers and an in-house statistician, from validated consensus criteria used in prior Delphi/eDelphi studies with comparable Likert scales,22,23 including one from the same research team using a nine-point scale.24

Accordingly, in the first round, three possible scenarios were presented: 1) median score of 6 (high relevance): accepted if IQR was 1-2 (strong agreement); 2) median of 5 (borderline high relevance): accepted as relevant, unless flagged for reassessment when IQR=3 (lack of consensus), or the 4-5 vote proportion exceeded the 5-6 proportion (trend toward lower relevance); and 3) median 4 (borderline low relevance) if the IQR was 0-1, the item was sent for reassessment (relevance unclear) or if the IQR was 2-3, the item was discarded (low relevance and lack of consensus).

A table summarizing the decision rules is presented in Supplementary Material, Table S3.

Items reaching agreement on relevance advanced to a priority-setting stage; those agreed as irrelevant were discarded; unresolved items were carried to the second round. for reassessment.

In the second round (28/10/2024-17/11/2024), experts performed two distinct tasks. The online platform used different colors to help panelists easily identify the status of each item: 1) re-assessment of unresolved categories (a high degree of consensus was reached, IQR 0-2; median ≥5 accepted, median 4 discarded); 2) rating priority for the categories considered relevant in the first round on a 1-6 scale, based on their importance for HCPP (median 5-6=high priority; median 4=medium priority; no category scored below 4).

Phase 4: the face-to-face meeting

The process concluded with a two-day in-person meeting in Zaragoza, Spain (12-13/12/2024), attended by 20 of the 28 experts who completed both online rounds. An internal facilitator with prior Delphi experience led a structured, directive consensus approach, supported by two rapporteurs; representatives from other PROCURE Work Packages attended as non-participating observers. For each criterion and category, panelists reached semantic agreement on final wording, merging, rephrasing or relocating items (additions/deletions were not permitted); a change was adopted with explicit majority approval or, at minimum, no sustained minority objection. Initial and final wording were compared for traceability (Supplementary Material, Table S4).

Ethical considerations

All experts participating in the Delphi panel signed an informed consent form authorizing the processing of their personal data prior to their participation. Throughout the online consultation rounds, individual responses were displayed to the panel only in aggregate, anonymized form (median and interquartile range of the group), so that no participant's identity or individual answers were disclosed to other panelists at any point in the process. This study did not involve patients or clinical intervention and, as a consensus exercise among professional stakeholders, did not require review by a research ethics committee. The study was conducted in accordance with the data protection and ethical requirements established under the PROCURE project's Grant Agreement with the European Health and Digital Executive Agency (HaDEA) (GA number 101128437).

ResultsInitial Delphi questionnaire on HCPP

Based on the initial literature review and expert input, the questionnaire was structured into four main domains, containing a total of 9 criteria and 60 categories (Fig. 1), following the logical lifecycle of HCPP.

Consultation process

Figures 3 and 4 show the progression of categories per criterion across both online rounds and the face-to-face meeting (discarded, re-assessed, and considered relevant). The final list of criteria and categories representing the key elements for successful HCPP comprises 9 criteria and 43 categories (Table 1). Prioritization results (Table 2) ranked monitoring and control of execution and governance and accountability as high priority, while initial analysis and sustainability ranked lowest (medium priority).

Figure 3.

Changes in criteria and category number and composition thorough online rounds and face-to-face meeting.

Figure 4.

Percentage of categories kept and discarded by category.

Table 1.

Final list of criteria and categories that represent the key elements for a successful healthcare public procurement process.

Delphi on public procurement in the healthcare sector
ID  Criteria name  Categories  Priority 
1Initial analysisA multiannual strategy for healthcare public procurement is clearly defined and established at member state level in alignment with EU policies and strategies  High 
There are centralized purchasing strategies developed with input from stakeholders that allow the acquisition of resources to meet the procurement needs  Medium 
The organization has in place updated e-procurement platforms for the public procurement process covering the entire procurement cycle  Medium 
Independent institutions, at European, national and regional level, offer regular information on healthcare public procurement processes  Low 
A healthcare specific EU legal framework for healthcare public procurement has been designed and implemented  High 
2Planning and market consultationHealthcare public procurement objectives and goals are aligned with institutional (i.e., local, national or international) health strategies and policies  Medium 
There are available transparent processes and mechanisms to identify the needs, analyze the market and collect information from the suppliers  Medium 
Open market consultation is used by procurers as a tool to obtain information from the market, design and improve tenders  Medium 
All the relevant stakeholders (i.e., procures, users, patients, health professionals, suppliers, etc.) are involved in the healthcare public procurement process  Medium 
3Governance and accountabilityThere is an appropriate organizational design and budget allocation for the professionalization on public procurement, with political and/or institutional support  Medium 
Adequate mechanisms and reporting system (confidential, online, secured) in place for prevention, identification, correction, prosecution and communication of misuse of power, fraud, bribery and corruption in healthcare public procurement are set up in the organization  High 
There is an established access to data and public information tools to guarantee transparency across the Procurement process (from planning to the execution of the contract, including evaluation)  Medium 
There is a defined policy to ensure acceptability and use of ICT among stakeholders to enable their involvement in the healthcare public procurement process  Medium 
Organizational structures are clearly defined and described in legal or administrative acts (i.e. governance, roles, responsibility assignments, flows of communication and work and accountabilities)  - 
There is a clear description of the specific role and responsibilities of all stakeholders involved and the relationships among them are well-defined  Medium 
Public procurement processes are integrated into each organizational information systems to monitor and evaluate the process (i.e., accessible channels of communication, specific software, AI, real-time availability…) to guarantee transparency along the processes and to avoid internal fragmentation  - 
A comprehensive operational risk management system as well as mitigation measures are in place  - 
4TenderComprehensible and accessible healthcare public procurement processes and documents are set up and properly communicated to the relevant stakeholders to guarantee alignment  High 
At organizational level, a process to manage penalties and breach of contract has been designed and is applied  Medium 
Most economically advantageous tender criteria (i.e., quality, price, social responsibility, environment, health outcomes, security of supply, etc.) are established in healthcare public procurement  High 
The tenders’ platforms are transformed to be open, accessible and easy-to-use (i.e., TED, government procurement portals, private tender portals, industry-specific platforms, etc.)  High 
5Capacity and resourcesTraining needs of stakeholders involved are assessed and taken into account in the development of the training programs in healthcare public procurement  Medium 
Comprehensible and accessible procurement good practices, guidelines, protocols, etc. are developed, updated, efficiently communicated and applied by relevant stakeholders  Medium 
All staff involved in the procurement process possess a wide range of knowledge, skills and competencies for specific tasks in healthcare public procurement  Medium 
There are systems in place to automate the verification of information corresponding to companies (powers of attorney, accounts, social security, treasury, etc.)   
6  Monitoring and control of execution  Risk management mechanisms and strategies are in place to prevent or mitigate breach of contract  Medium 
7EvaluationIndicators take into account value-based measurements: health outcomes, patient reported outcomes, economic (i.e., budget impacts and efficiency), social, environmental and healthcare professional outcomes  Medium 
Outcomes assessment has a people-centered approach and is focused on impact (i.e. consequences derived from the healthcare public procurement process) and overall satisfaction of the stakeholders, including users  Medium 
The outcomes and evaluation framework are shared among stakeholders to facilitate improvement through public and open access collaboration, respecting the principles of transparency and equal treatment  - 
8SustainabilityRelevant environmental and social considerations and awarding criteria, aligned with policies and the scope of the contract, are defined on the tender and evaluated along the execution phase, without being an unfair barrier for the competitors  - 
The healthcare public procurement process can guarantee the viability of the outcomes of the contract in the long term, in a multidimensional approach that integrates environmental, social and economic considerations, promoting sustainable development  Low 
9Emergency situationThe organization establishes clear guidelines and regularly updates a risk management tool (including a contingency plan, modeling to predict) for critical functions  Medium 
In case of an emergency situation that impacts the health system, an existing emergency committee (with defined roles and functions and executive capacity of decision) is activated  High 
There are clear channels established for internal and external communication, with all stakeholders involved, to facilitate the timely and effective dissemination of information during emergencies  Medium 
Mechanisms such as protocols, guidelines, European and cross-border legal frameworks with third countries, are established to rapidly mobilize and deploy essential resources during emergencies  Medium 
Healthcare systems have the resources to maintain stocks of key supplies and established distribution networks to facilitate timely delivery to affected areas  Medium 
As part of the emergency strategy, there are simplified and fast-track protocols for procurers to approach a wide range of suppliers and manufacturers to deal with stock-outs or other contingencies and ensure quality  Medium 
Policies and procedures are in place to address legal, organizational and ethical considerations related to emergency response, such as patient privacy, consent, and confidentiality  Medium 
A continuous evaluation of emergency actions and efforts taken is done to incorporate lessons learned from all stakeholders, to improve procedures and ensure the organizational resources  Medium 
Inter-organizational dialogues and collaboration networks with suppliers, manufacturers and other stakeholders are in place to ensure equitative distributed access to critical resources in times of crisis  Medium 
Transparency mechanisms are in place throughout the process (ex-pos included) when the procurement procedure is carried out without prior publication for reasons of urgency  High 
Regular training sessions are conducted to prepare personnel to effectively respond to emergencies  Medium 
European and cross-border collaboration strategies are enhanced during non-crisis periods in order to facilitate the acquisition process that benefits the economies of scale when a crisis hits  - 

AI: artificial intelligence; ICT: Information and communication technologies; TED: Tenders electronic daily.

Table 2.

Priority distribution of the criteria.

Priority ranking  Criteria  Mean score 
High priority  Criterion 4. Tender  5,34 
Criterion 6. Monitoring and control of execution  5,21 
Criterion 3. Governance and Accountability  5,12 
Criterion 9. Emergency situation  5,07 
Criterion 7. Evaluation  5,05 
Criterion 2. Planning and market consultation  5,03 
Criterion 5. Capacity and resources  5,00 
Criterion 1. Initial analysis  4,84 
Medium priority  Criterion 8. Sustainability  4,36 

Average of medians obtained on the Likert scale for prioritizing the importance of the different statements for public procurement.

Discussion

Results showed high agreement among experts: only 10 of 60 categories were deemed not relevant in the first round, though most underwent rephrasing ranging from minor wording adjustments to substantive conceptual revisions (Supplementary Material, Table S4). Main insights per criterion, situated against prior literature, are summarized below:

  • •

    Criterion 1. Initial analysis, reached consensus with little controversy: experts strongly agreed on the need for a specific, EU-driven legal framework for HCPP, echoing prior calls for regulation distinct from general procurement law,25 but moving beyond earlier policy-level recommendations9 by embedding this need within a broader, empirically validated set of success criteria.

  • •

    Criterion 2. Planning and market consultation, showed high agreement with few changes, stressing stakeholder involvement, alignment with European strategies, and engagement of high-level decision-makers.

  • •

    Criterion 3. Governance and accountability, centered on transparency; debate over whether contract performance and evaluation results should be public was resolved by agreeing that transparency should govern all phases, including evaluation. These results, together with Criterion 5, address the integration lacking in prior work on centralization6 and innovation-oriented instruments7 examined in isolation.

  • •

    Criterion 4. Tender, revealed a recurring tension between supplier and procurer perspectives on “value” (economic vs. patient- and outcome-centered) that shaped much of the face-to-face meeting.

  • •

    Criterion 5. Capacity and resources, centered on comprehensive training extended to all personnel, dedicated budget allocation, and the role of technological resources (particularly automation and AI) in reducing administrative burden, reflected in the retained category on automated verification of company information.

  • •

    Criterion 6. Monitoring and control of execution, illustrates that discarded categories can be as informative as retained ones. Four of five initial categories were dropped online (KPI-based monitoring with performance-conditional payments, a multidimensional data collection plan, follow-up reporting during contract execution, and a dedicated monitoring committee) leaving only a single revised category, “risk management mechanisms and strategies”, after experts considered the original wording (“fixing mechanisms and strategies”) imprecise. Despite this attrition, experts strongly reaffirmed the criterion's importance face-to-face, consistent with its high priority ranking (Table 2), some requesting reinstatement of discarded items. This divergence likely reflects practical complexity rather than low relevance: tools like KPI-linked payments presuppose technical and administrative capabilities not uniformly available across institutions,26 compounded by the recognized absence of standardized enforcement mechanisms in HCPP.27 Additionally, effective monitoring is widely regarded as a managerial and governance responsibility rather than a routine operational task;28 the more deliberative, face-to-face format likely allowed experts to reframe the criterion at this institutional level. This is consistent with evidence that monitoring and enforcement mechanisms in public procurement remain weak in practice despite their recognized importance,27,28 and our study adds empirical, multi-stakeholder confirmation of this gap within the healthcare sector.

  • •

    Criterion 7. Evaluation, was among the most controversial, with debate over whether patient outcomes and economic aspects should carry equal weight; the panel converged on a holistic approach incorporating indicators across the full outcome spectrum, notably proposing a “people-centered” guiding principle. This reflects a broader recognition of the patient perspective that the panel could not directly represent, given the absence of patient participants. The difficulty of recruiting and retaining patients in Delphi and eDelphi consensus exercises is a recognized methodological challenge, particularly in studies addressing structural or administrative processes (such as procurement).29 As a result, the criteria and categories related to patient-centeredness in this study reflect the perspective of professionals advocating on patients’ behalf, rather than patients’ own “lifeworld” experience,30 which may limit the extent to which these criteria fully capture patient priorities.

  • •

    Criterion 8. Sustainability, was also highly controversial, with three of five categories dropped online; diverging interpretations across environmental, social and economic/institutional dimensions underscored the need for common definitions integrating all three pillars, consistent with prior work showing sustainability in public procurement is understood inconsistently31 and that its integration depends on leadership commitment and strategic alignment.32 Our findings extend this literature by empirically demonstrating, through expert consensus, that this ambiguity remains a live, unresolved tension among HCPP practitioners themselves, not only a theoretical concern.

  • •

    Criterion 9. Emergency situation, showed swift consensus, likely reflecting shared COVID-19 experience: organizations must be proactively prepared rather than reactive, with a pre-designated emergency committee holding executive authority, and cross-border, public-private and international cooperation, without compromising equal treatment.

These findings reinforce prior calls for strengthened cross-border and public-private cooperation in health emergencies33 and align with COVID-19 lessons on proactive preparedness.2,3 Unlike prior studies documenting individual practitioner learning or diagnosing pandemic-specific failures,2,3 our study translates this experience into a validated, prioritized set of preparedness criteria intended for structural adoption ahead of future crises. These criteria are directly relevant to ongoing EU efforts to strengthen joint procurement and crisis preparedness through HERA,34 offering an expert-endorsed basis for shared standards, evaluation tools, or national action plans.

Taken together, these results address the gap identified in the Introduction: to our knowledge, this is the first study to apply a structured, multi-stakeholder Delphi consensus methodology to derive and prioritize an integrated, pan-European HCPP framework spanning the full procurement lifecycle, providing both a diagnostic and a prescriptive tool that prior, largely descriptive literature has not offered.9–11

Beyond its academic contribution, this framework has direct practical relevance: strategic procurement practices have been shown to reduce costs for medical supplies while maintaining or enhancing quality,35 an objective the identified criteria can directly support.

A comparable Delphi-RAND methodology from the same research team was previously adopted in real-world pilot actions across several European countries,24 illustrating this type of framework's translation potential, including for ongoing EU crisis-preparedness efforts such as HERA. In the specific context of HCPP, these criteria are directly relevant to ongoing EU joint procurement and crisis-preparedness efforts, offering a validated basis for shared standards, evaluation tools, or national action plans.

Limitations

As a semi-qualitative consensus framework, the Delphi methodology has proven appropriate for capturing diverse perspectives on complex, poorly standardized domains such as HCPP. Nonetheless, several limitations should be considered:

  • •

    Panel composition and representativeness: despite covering seven stakeholder profiles with diverse professional, gender and geographic composition (Fig. 2), no patient or end-user representative ultimately participated; future applications should actively seek direct patient input. External experts were also primarily identified through PROCURE consortium networks, a snowball-type recruitment common in Delphi studies17 that may have favored more homogeneous perspectives than fully independent sampling.

  • •

    Participant attrition: panel size decreased from 29 experts in the first online round to 20 at the face-to-face meeting (Fig. 2), within the range reported for multi-phase Delphi studies but possibly reducing perspective diversity at the final consensus stage.

  • •

    Real-time feedback and potential convergence bias: real-time display of group responses could, in principle, favor premature convergence. However, this reflects a deliberate RTD design choice, and comparative studies have found no substantial distortion of final outcomes relative to conventional Delphi.13,14,19

  • •

    Consensus threshold definition: although grounded in validated Delphi frameworks, the numerical thresholds were adapted to this study's six-point scale rather than adopted unmodified from a single reference, limiting direct comparability with other Delphi studies.

  • •

    Facilitation of the face-to-face meeting: the session was facilitated internally rather than by an external, independently trained facilitator, as recommended in RAND/UCLA guidance;20 although limited to semantic consensus (priority ranking having been completed online), this may have introduced moderator bias.

  • •

    Geographic scope: the panel comprised participants from 10 European countries, reflecting the PROCURE consortium rather than the full diversity of EU healthcare procurement systems; further validation would be needed before generalizing to non-represented countries.

Conclusions

Beyond confirming the relevance of well-established concerns (the need for a specific EU-driven legal framework, and the importance of transparency and stakeholder involvement) this study contributes new, empirically grounded insight into where HCPP stakeholder consensus is strong (governance, tender, capacity-building) and where it remains contested (sustainability, the balance between economic and patient-centered evaluation, monitoring implementation).

This validated, prioritized framework can inform the design or evaluation of HCPP processes at institutional, national and European level, including shared standards and evaluation tools for crisis-resilient procurement relevant to ongoing HERA efforts. Its translation into concrete policy recommendations and Member State action plans is the object of subsequent PROCURE work.

These contributions should be interpreted considering the study's limitations, discussed below, particularly the absence of direct patient representation, participant attrition toward the face-to-face meeting, and the panel's geographic scope (10 European countries), which suggest further validation and contextual adaptation will be needed before generalizing this framework across the full diversity of EU healthcare systems.

Availability of databases and material for replication

Availability of databases and material for replication: please consult PROCURE for more information.

What is known about the topic?

Currently, there is no consensus on public procurement practices, which vary significantly across European Union countries, nor common, consensus-based guidelines for evaluating an ongoing healthcare procurement process or designing a new one.

What does this study add to the literature?

Sharing information and insights among multi-stakeholder experts allowed the identification of the key elements every healthcare public procurement process should have to be successful

What are the implications of the results?

Findings offer a guide to evaluate procurement practices, helping leaders track their readiness and identify improvement opportunities, and will inform tailored recommendations per Member State.

Editor in charge

Salvador Peiró.

Transparency declaration

The corresponding author, on behalf of the other authors guarantee the accuracy, transparency and honesty of the data and information contained in the study, that no relevant information has been omitted and that all discrepancies between authors have been adequately resolved and described.

Authorship contributions

I. Menéndez Fernández coordinated the study technically, contributed to selecting the Delphi panel candidates, communicated with the selected experts, helped design the online consultation platform, followed up the two online rounds, organized the face-to-face meeting and acted as typer during it, processed the results, and drafted the manuscript. M.J. Vicente Edo, as Delphi expert, designed the study, supervised the elaboration of the initial Delphi questionnaire, contributed to selecting the panel candidates, conducted the face-to-face meeting, processed the results, and reviewed the manuscript. J. López Puy and B. Aguilar Utrilla retrieved and analyzed the literature and drafted the initial Delphi questionnaire; J. López Puy also collaborated in defining the expert panel profile and organized the logistics of the face-to-face meeting, while B. Aguilar Utrilla identified SALUD's experts; both acted as observers during the face-to-face meeting and reviewed the manuscript.

Acknowledgments

We appreciate the contribution of María Bezunartea (Navarrabiomed), Carmen de Guerrero (Unizar), Virginia García (CSC) and José María Carrascosa (061) in the elaboration of the initial Delphi questionnaire, and the PROCURE partners who participated in the implementation of the project, especially the coordinators, as well as the Delphi experts who were part of the process.

Funding

The results presented were obtained during the execution of the European project PROCURE (Grant Agreement Number: 101128437), that was co-funded by the European Union, being the European Health and Digital Executive Agency the granting authority under the EU4H Project Grants.

Conflicts of interest

None.

References
[1]
R. Araújo, J.M. Fernandes, L.P. Reis, et al.
Purchasing challenges in times of COVID-19: resilience practices to mitigate disruptions in the health-care supply chain.
J Glob Oper Strateg Sourcing., 16 (2023), pp. 368-396
[2]
C.M. Harland, L. Knight, A.S. Patrucco, et al.
Practitioners’ learning about healthcare supply chain management in the COVID-19 pandemic: a public procurement perspective.
Int J Oper Prod Manag., 41 (2021), pp. 178-189
[3]
E. Kumah, R. Boateng, E. Agyei.
Strategic purchasing in times of public health emergency: lessons from COVID-19.
BMJ Glob Health., 8 (2023),
[4]
Organization for Economic Co-operation and Development.
Implementing the OECD Recommendation on Public Procurement in OECD and Partner Countries: 2020-2024 Report.
OECD Publishing, (2025),
[5]
PROCURE Consortium.
Observational Study on European Healthcare & Procurement Environment.
European Union Project PROCURE, (2024),
[6]
N. Geropoulos, P. Voultsos, M. Geropoulos, et al.
Hybrid model: a promising type of public procurement in the healthcare sector of the European Union.
Front Public Health., 12 (2024),
[7]
M.R. Andrews, P.A. Long, M. Ahlberg, et al.
A European arena for joint innovation in healthcare: The Platform for Innovation of Procurement and Procurement of Innovation (PiPPi).
Front Public Health., 10 (2023),
[8]
P. Nemec, V. Ďuricová, M. Kubak.
Institutions, corruption and transparency in effective healthcare public procurement: evidence from Central and Eastern Europe.
Health Econ Policy Law., 18 (2023), pp. 619-646
[9]
L. Siciliani, M. McKee, A. García-Altés, et al.
Public procurement in healthcare systems. Opinion of the expert panel on Effective Ways of Investing in Health (EXPH).
European Union, (2021), http://dx.doi.org/10.2875/832331
[10]
S.D. Gonçalves, A. Caramelo.
Resilience strategies for healthcare supply chains: lessons from global crises and future directions.
Supply Chain Manag Res., 1 (2026),
[11]
F.J. Esplugues, I. Andújar, J.V. Esplugues.
The impact of EU public procurement regulations on tenders in Spain: a study with adalimumab.
Front Pharmacol., 15 (2024),
[12]
M.K. Murphy, N.A. Black, D.L. Lamping, et al.
Consensus development methods, and their use in clinical guideline development.
Health Technol Assess., 2 (1998), pp. 1-88
[13]
T.J. Gordon, A. Pease.
RT Delphi: an efficient “round-less” almost real time Delphi method.
Technol Forecast Soc Change., 73 (2006), pp. 321-333
[14]
T. Gnatzy, J. Warth, H. von der Gracht, et al.
Validating an innovative real-time Delphi approach — a methodological comparison between real-time and conventional Delphi studies.
Technol Forecast Soc Change., 78 (2011), pp. 1681-1694
[15]
K.A. Batterton, K.N. Hale.
The Likert scale: what it is and how to use it.
Phalanx., 50 (2017), pp. 32-40
[16]
C. Powell.
The Delphi technique: myths and realities.
J Adv Nurs., 41 (2003), pp. 376-382
[17]
F. Hasson, S. Keeney, H. McKenna.
Research guidelines for the Delphi survey technique.
J Adv Nurs., 32 (2000), pp. 1008-1015
[18]
S.L. Onsanity Solutions.
Onsanity, (2022),
[19]
F.A. Quirke, M.R. Battin, C. Bernard, et al.
Multi-round versus real-time Delphi survey approach for achieving consensus in the COHESION core outcome set: a randomised trial.
[20]
K. Fitch, S.J. Bernstein, M.D. Aguilar, et al.
The RAND/UCLA appropriateness method user's manual.
Santa Monica, CA: RAND Corporation, (2001),
[21]
R. Boulkedid, H. Abdoul, M. Loustau, et al.
Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review.
PLoS One., 6 (2011),
[22]
J.M. Monguet, A. Trejo, T. Martí, et al.
Assessment of chronic health care through an Internet consensus tool.
Handbook of research on trends in the diagnosis and treatment of chronic conditions., pp. 424-443
[23]
M. Torres, V. Serra-Sutton, J.B. Soriano, et al.
Consensus on post COVID in the Spanish national health system: results of the CIBERPOSTCOVID eDelphi study.
J Infect Public Health., 16 (2023), pp. 1784-1792
[24]
J. Zaletel, M. Maggini.
Fostering the quality of care for people with chronic diseases, from theory to practice: the development of good practices in disease prevention and care in JA CHRODIS PLUS using JA CHRODIS Recommendations and Quality Criteria.
Int J Environ Res Public Health., 17 (2020), pp. 951
[25]
A. García-Altés, M. McKee, L. Siciliani, et al.
Understanding public procurement within the health sector: a priority in a post-COVID-19 world.
Health Econ Policy Law., 18 (2023), pp. 172-185
[26]
Organization for Economic Co-operation and Development.
Public Procurement Performance: OECD Public Governance Policy Papers.
OECD Publishing, (2023),
[27]
M. Fazekas, J.R. Blum.
Improving public procurement outcomes: review of tools and the state of the evidence base.
World Bank, (2021),
[28]
A.M. Malik, E. Garcia Elorrio.
Governance and management in healthcare organizations: their different roles in driving safety and quality.
Int J Qual Health Care., 35 (2023),
[29]
H. Barrington, B. Young, P.R. Williamson.
Patient participation in Delphi surveys to develop core outcome sets: systematic review.
BMJ Open., 11 (2021),
[30]
M. Niederberger, M. Sonnberger.
The participation of lifeworld experts in Delphi processes: a reflection on method and practice.
MethodsX., 14 (2025), pp. 103274
[31]
A.L.T. dos. Santos, A.C. Reis.
Public procurement from the triple bottom line lens: the identification of sustainability criteria from the international literature review.
Environ Dev Sustain., 26 (2024),
[32]
M. Keil, V. Michaux, M. Ziefle, et al.
Agents of sustainable change: the importance of leadership and responsibility in corporate sustainability.
Front Sustain Organ., 5 (2025),
[33]
C.J. Rutkowski, K. Eboch, A. Carr, et al.
Strategic procurement collaboration for the common good: private and public procurement relationship during a pandemic.
J Public Procurement., 22 (2022), pp. 43-63
[34]
M. Anderson, R. Forman, E. Mossialos.
Navigating the role of the EU Health Emergency Preparedness and Response Authority (HERA) in Europe and beyond.
Lancet Reg Health Eur., 9 (2021),
[35]
A.K. Balogun, H. Ayodimeji-Alaba, K.T. Adebayo, et al.
Strategic procurement practices in healthcare organizations and their impact on Medicaid and Medicare cost containment.
World J Adv Res Rev., 25 (2025), pp. 1863-1872
Copyright © 2026. SESPAS
Download PDF
Idiomas
Gaceta Sanitaria
Article options
Tools
Supplemental materials