Organisational self-identification (OSI) refers to a shared understanding of an organisation’s collective identity. This identity shapes the organisational culture, employee engagement and commitment, and overall performance. While policy documents define rules and interactions, both their substance and tone reinforce or diminish the OSI. However, traditional policy reviews often overlook whether the language and tone of these documents truly reflect organisational values. In this study, we introduce an AI-powered sentiment analysis framework that offers a novel and systematic approach to evaluate OSI alignment in policy texts. Our OSI Sentiment Analyser combines a customised lexicon with rule-based scoring to identify and classify sentiments at multiple levels while ensuring transparency, explainability, and respect for privacy. The analyser was validated across 78 public policies (where it achieved over 90 % agreement with expert judgement) and further tested in two NHS hospital trusts as case studies. The proposed framework not only supports evidence-based policy refinement but also fosters inclusive and value-driven communication. Its modular design promises broad application, from employee surveys to strategic organisational messaging. Accordingly, it advances both theory and practice in organisational identity analytics.
Organisational self-identification (OSI) refers to a shared understanding of an organisation’s collective identity. This concept is fundamental to shaping workplace culture and influences employee engagement, commitment, and overall organisational performance (Albert et al., 2000; Pepple & Ambilichu, 2024; Pepple & Davies, 2019).
Policy documents play a critical role in nurturing a positive OSI because they define relationships and articulate organisational values. Their content and tone can either reinforce or undermine OSI (Cheung et al., 2010; Von Solms & Von Solms, 2004). Research shows that inclusive and positively framed policies enhance employee commitment and satisfaction, whereas rigid or negatively toned language erodes morale and increases disengagement (Chiemeke et al., 2019; Tsai, 2011). Similar to how evaluative tone in public policy affects public attitudes and issue salience (level of attention and engagement), the tone of internal policies shapes employees’ perceptions and responses to organisational directives (Chong & Druckman, 2007; Druckman & McDermott, 2008; Tsai, 2011). Notably, even when the substantive content remains unchanged, tonal variations can lead to markedly different behavioural and emotional outcomes.
Although policy documents are subject to frequent updates, traditional review processes seldom assess the impact of language and tone on employees. Reviews typically focus on legal or procedural compliance, overlooking alignment with organisational identity and ethos (Campbell, 2003; Hinterleitner et al., 2024). This gap can create inconsistencies between stated values and practices and, in turn, affect employee engagement and organisational coherence.
Current advancements in artificial intelligence (AI) present opportunities to bridge these gaps. Therefore, this study introduces an AI-powered OSI Sentiment Analyser designed to evaluate how policy texts reflect OSI constructs and support values-aligned communication (Brown, 2018; Seidl, 2016, 2020). The analyser evaluates policy texts at multiple levels (document, sentence, phrase, and word) and provides interpretable sentiment insights aligned with commitment, engagement, and empowerment. Thus, offering a data-driven process that empowers organisations to enhance clarity and alignment with their identity (Fine & Porteus, 1989; Kim, 2017).
Two hospital trusts participated as case studies in evaluating our proposed system; we refer to the hospitals as Hospital Trust A and Hospital Trust B. We reviewed 12 workforce policy documents using the proposed system. The results indicate that the OSI Analyser identifies OSI-related tone, provides actionable insights to improve inclusive language, and allows for consistent checks for OSI alignment. Furthermore, policy owners reported positive experiences with the analyser, citing ease of use, clarity of outputs, and structured feedback that facilitated confident iterative refinements. In addition, several participants mentioned that the tool encouraged deeper reflection on tone and inclusivity, aspects that are often overlooked in traditional policy reviews.
This study does not assess the impact of policy language on policy end-users, such as employees. Instead, it focuses on the policy review process and the experiences of policy owners and reviewers responsible for maintaining and updating organisational documents. By examining how these stakeholders interact with the AI-powered OSI Sentiment Analyser, this study assesses its effectiveness in facilitating structured, iterative, and value-aligned policy reviews.
This study advances organisational identity research by translating the core constructs of organisational sentiment identity (commitment, engagement, and empowerment) into measurable and explainable indicators within policy texts. By aligning sentiment analysis with OSI‑specific dimensions rather than using a single global sentiment value, the approach offers a structured way to examine how organisational language expresses identity and shapes employees’ sense of belonging, voice, and agency. The OSI Sentiment Analyser provides a transparent, replicable scoring method that preserves explainability and human oversight, laying the foundation for theory-aligned analysis and enabling future hypothesis-driven enquiry into how policy language influences organisational experience.
The remainder of this paper is organised as follows: Section II reviews the relevant literature and key concepts; Section III outlines the research methodology; Section IV presents and discusses the findings; and Section V concludes with implications and recommendations for future research.
Literature reviewIn this section, we examine the existing literature to place our research within the wider conversation regarding organisational self-identity, policy analysis, and sentiment analysis, and to highlight the key concepts and theoretical foundations that underpin our approach.
Organisational self-identificationOrganisational Self-Identity (OSI) is a foundational concept that defines how an organisation perceives itself and seeks to project this perception internally. Rooted in the broader organisational identity (OI) theory first articulated by Albert and Whetten (1985) and later refined by Whetten (2006), OSI incorporates attributes that are central, enduring, and distinctive (CED). These attributes shape an organisation's core values, guide strategic decisions, and distinguish it from others. However, while traditional OI focuses on the organisation and abstract values (mainly related to the organisation), OSI focuses on how the held values are expressed and supported in everyday practice (i.e. organisational and employee interface), such as through internal communication and policy documents. Thus, OSI provides a practical and measurable lens through which to examine how internal language and communication reflect and sustain organisational values, focusing on commitment, engagement, and empowerment as key constructs.
Previous studies have consistently shown that a strong OSI positively influences organisational culture and performance (Goby & Karimova, 2024; Hutchings et al., 2024; Lin, 2004). Other benefits of improvement include employee identification, engagement, and cooperation. Moreover, self-identity, voice, and self-efficacy in employees are key mediators of a healthy organisational climate and staff retention (Peng & Pierce, 2015; Pepple & Davies, 2019; Pepple & Davies, 2020). Therefore, improvements in OSI are associated with improved morale, reduced turnover, and enhanced organisational coherence. The overall result is a healthy and productive workplace with improved organisational outcomes. Furthermore, OSI plays a critical role in organisational change. It shapes how employees interpret and respond to new initiatives based on their perceived alignment with their core identity (Alvesson et al., 2008b; Boonsiritomachai et al., 2022; Cooper & Thatcher, 2010; Sergeeva & Kortantamer, 2021).
Organisations actively pursue and reinforce their self-identity through different strategies (Alvesson et al., 2008; Cooper & Thatcher, 2010). First, they strive to maintain consistent communication of their values and mission. In this regard, policy documents play a role, and both the substance and expression form part of this strategy. Another way to maintain appropriate OSI levels is for leaders to consistently uphold OSI-related values (Seidl, 2016; Van Tonder, 2006). Moreover, cultural practices create a shared sense of belonging to reinforce the key aspects of an organisation’s identity. Furthermore, the strategy of ensuring recruitment prioritises cultural fit, and the socialisation process helps integrate new members into the culture. Altogether, these strategies help embed identity into daily experiences, ensuring coherence and alignment with the organisation's core values and mission (Caprar & Walker, 2024; Lin, 2004).
Considering these understandings, our study introduces an AI-powered system to analyse organisational documents through the lens of the OSI. Applied in two NHS Trusts which are complex, high-impact institutions with urgent workforce challenges, this system aims to help leaders optimise internal communication and better align policy language with values that foster employee belonging, motivation, and commitment.
Policies in organisationsPolicy documents are defined and documented rules, guidelines, and principles that govern an organisation. They form the basis of decision-making and operational procedures. These documents outline relationships and establish standards for conduct, ethics, and responsibility. Therefore, policy documents play a vital role in fostering a healthy organisation and achieving its objectives by promoting consistency, fairness, clarity, unity, and transparency (Cheung et al., 2010; Freeman & Maybin, 2011).
While policy documents are central to the activities of organisations, the effort put into improving how they may be perceived is sometimes inadequate. For example, compliance and risk management policies aid adherence to laws and regulations, reducing legal risks and offering clear guidelines to prevent misconduct and errors that may result in liabilities (Freeman & Maybin, 2011; Sutton, 1999). In such environments, leaders tend to focus on legal or procedural correctness and often lack a structured and accessible approach to assess alignment with the organisation's identity and ethos (Campbell, 2003; Hinterleitner et al., 2024). This misalignment can lead to inconsistencies between an organisation's stated values and perceived practices, which, in turn, affects employee engagement and organisational coherence. Furthermore, as policy documents require updates to reflect evolving operational needs, this may mean that policies with frequent updates may be at a greater risk of misalignment. Therefore, both the substance and expression of policies are crucial. Well-drafted policies promote a positive workplace culture by ensuring transparency, fairness, and accountability (D. K. Cohen & Ball, 1990; Rein, 2017). They align an organisation's culture with its values to encourage a sense of belonging and commitment. When policies are fair and consistently applied, employee engagement and motivation increase (Von Solms & Von Solms, 2004).
Detecting OSI-alignment in policy documentsTo identify sentiments related to OSI (commitment, engagement, and empowerment) in policy documents, we must look beyond positive or negative words. Instead, it involves analysing how their language conveys purpose, inclusivity, and autonomy in ways that align with OSI.
Commitment words include verbs that express dedication and responsibility (De Felice, 2013; Mayfield et al., 2021), such as 'uphold' or 'safeguard' (e.g., stating that the organisation will uphold its standards or safeguard the interests of its stakeholders); nouns like 'mission,' 'values,' and 'purpose' (e.g., referring to alignment with the organisation's mission or a commitment to shared values); and adjectives such as 'enduring' or 'principled', which reinforce the sense of long-term dedication (e.g., mentioning an enduring commitment to ethical practices). Modal verbs and phrases such as 'will continue to' or 'shall maintain' indicate promises about the future (e.g. saying the organisation plans to keep investing in staff development).
Engagement words include verbs that invite participation and collaboration, such as 'consult,' 'collaborate,' or 'contribute' (e.g., saying that staff are encouraged to contribute ideas or that the organisation will consult employees before making changes) (Dumlao & Burack, 2023). Nouns such as 'feedback,' 'dialogue’, and 'forum' indicate mechanisms for involvement. Inclusive pronouns such as 'we' and 'together' and phrases such as 'our community' help increase a sense of shared ownership. Adjectives such as 'inclusive' and 'collaborative' further show an inclusive approach to decision-making.
Empowerment language promotes autonomy and support within an organisation (Nicolaides & Archanjo, 2019). Indicators of this include the use of active verbs such as 'authorise,' 'enable’, and 'delegate' (e.g. stating that employees are authorised to make decisions within their roles or that teams are allowed to lead projects). Nouns such as 'autonomy’, 'authority,' and 'resources' are used to describe the conditions that support empowerment (such as providing resources and training to staff). Descriptive adjectives such as trusted' and 'capable' convey confidence in employees' abilities (for instance, referring to staff as trusted partners in delivering quality service). Additionally, phrases such as 'staff have discretion to' or 'decision-making is delegated to teams' depict empowerment in organisational communication.
When analysing organisational language for commitment, engagement, and empowerment to assess OSI alignment, we need to examine how individual lexicon words interact. The combination and proximity of verbs, nouns, adjectives, and set phrases can significantly strengthen the signal for OSI alignment in the text. Commitment signals are enhanced when verbs like 'uphold' or 'invest' are used with nouns such as 'mission' or 'values.' Additionally, the presence of adjectives such as 'enduring' or 'principled,' especially when linked with future-oriented phrases such as 'will continue to,' further supports a lasting dedication. Engagement signals are stronger when participatory verbs such as 'consult' or 'contribute' are paired with nouns like 'feedback' or 'forum,' along with inclusive pronouns like 'we' or 'together.' Empowerment is most evident when authority-granting verbs such as 'authorise' or 'enable' are combined with nouns like 'autonomy,' 'resources,' or 'decision-making.'
Artificial intelligence for strategic organisational managementThe integration of Artificial Intelligence (AI) in business management is transforming organisational practices and creating new opportunities for strategic innovation and operational improvement (Jorzik et al., 2024; Kulkov et al., 2023). AI is not just automating routine tasks; it is helping businesses evolve in more strategic ways. Today, leaders are finding new ways to improve operations and encourage innovation (Cetindamar et al., 2024; Hmoud & Laszlo, 2019). For example, they use data to better understand employee engagement and support decision-making.
As AI systems take on more analytical and administrative tasks, managers can focus on more important roles such as innovation and strategic planning (Enholm et al., 2021; Sestino & De Mauro, 2022). Recent research has highlighted several aspects of this change. AI capabilities enhance research and development performance by facilitating innovation (both exploratory and exploitative). The gains in turnaround time are highly welcomed by organisations operating in competitive and dynamic environments (V. Kumar et al., 2024). Moreover, AI is enabling new approaches to business innovations, which means that top management needs to gain new competencies and roles to govern these changes effectively (Jorzik et al., 2024).
As a result of the growing human-technology interface, AI literacy among employees is becoming essential (Cetindamar et al., 2024) to identify core capability sets, including technological, work-related, human-machine, and learning-oriented, that shape effective AI interaction in digital workplaces (. Furthermore, this signifies that, with technical robustness, AI applications should be accessible to non-specialists as well.
Additionally, the fairness and ethics of applications are of increasing relevance and attention in the literature. In recruitment, Zheng et al. (2024) found that AI could reduce bias and make processes more objective. However, they highlight the important role of human judgment in understanding complex social interactions.
Therefore, this study contributes to both theory and practice by applying AI to enhance policy alignment with the OSI principles. The goal is to improve technology while also creating a more ethical, transparent, and human-focused workplace (Pepple & Muthuthantrige, 2026).
Sentiment analysisSentiment analysis uses natural language processing (NLP) and machine learning techniques to identify and quantify subjective information (feelings) expressed in text (Kalchbrenner et al., 2014; Liu & Zhang, 2012; Sharma et al., 2024a; Sun et al., 2023; Zhang et al., 2018). By quantifying or classifying (e.g. as positive, negative, or neutral) emotional information, organisations can better understand the sentiments expressed by customers and employees on a large scale. Furthermore, sentiment analysis can be applied in various contexts and text formats, such as customer reviews, social media posts, employee feedback, and public commentary.
Various methods are available for implementing sentiment analysis, each with its own set of trade-offs. The techniques range from simple lexicon-based methods to advanced machine learning and transformer-based methods (Kumar et al., 2025). Lexicon-based methods use dictionaries of words with pre-assigned sentiment scores and classifications (e.g., positive, negative, or neutral sentiments) (Barik & Misra, 2024; Van Der Veen & Bleich, 2025). A reference dictionary was used to perform a series of calculations to determine the sentiment scores of the texts. The calculations can be complex in terms of managing how words relate to influencing the sentiment score. This simple rule-based method has the advantages of interpretability and computational efficiency (Van Der Veen & Bleich, 2025).
Meanwhile, machine learning-based methods identify sentiment by learning patterns from vast samples of pre-labelled data. They tend to require less crafting than lexicon methods and offer gains in accuracy. However, machine learning methods are not transparent and require more computational effort than lexicon-based systems. Machine learning techniques vary and have trade-offs. Advanced machine learning techniques, such as deep learning (neural networks, e.g. CNNs and RNNs), can effectively learn complex patterns in text to better identify sentiment. Similarly, transformer models (e.g. BERT, RoBERTa) achieve state-of-the-art accuracy in sentiment analysis and handle nuanced language well (Mirdan et al., 2025). Generally, machine learning techniques are computationally intensive (Sambo et al., 2020; Zhang et al., 2018) and operate as black boxes, which lack the explainability that is increasingly in demand.
Sentiment analysis has broad applications in organisations (Sharma et al., 2024b; Sweta, 2024). For example, by analysing data from surveys and internal communication platforms, organisations can assess the emotional tone of their workforce, helping managers detect early signs of dissatisfaction, burnout, or disengagement. It is already widely applied in areas such as analysing employee survey feedback to inform HR strategies or tracking sentiment on social media during election campaigns to gauge public opinion words (Chen et al., 2018; Feldman, 2013). These applications help organisations understand employee well-being, engagement, and productivity. Recently, advances in multimodal sentiment analysis, such as the combination of textual, audio, and visual data, have offered the potential for richer insights into the emotional dynamics of the workforce (Das & Singh, 2023).
Prior AI tools for organisational communication include generic sentiment classifiers and writing assistants that optimise style (e.g., confidence and friendliness), and HR analytics dashboards that track workforce sentiment at the aggregate level (Kumar Mishra & Mishra, 2025; Ravichandran et al., 2023). Such tools typically provide global polarity or stylistic guidance without construct-level operationalisation of organisational identity. Our approach is distinctive not by claiming absolute novelty, but by explicitly modelling OSI’s three constructs of OSI and tying lexicon customisation and rule-based adjustments to those constructs so that outputs are interpretable against OSI dimensions (commitment, engagement, empowerment).
In line with contemporary expectations for AI governance, we prioritise explainability, data locality, and human oversight (Abdusattarova, 2025; Bamigbade et al., 2024; Feitsma & Whitehead, 2025). The use of a lexicon-plus-rules pipeline promotes transparent reasoning and allows local deployment, avoiding the need for external data transfer. Additionally, the expert-vetting stage reduces the risk of misinterpretation and addresses contextual blind spots. We place these choices within the ongoing discussions about responsible AI and the importance of institutional trust in decision support tools.
MethodologyThe methodology of this study followed a multistage process. The process included the development of the OSI Analyser tool, its validation, and real-world case studies in two NHS hospital trusts. Furthermore, the methodology aimed to ensure technical accuracy and real-world relevance and provide a detailed evaluation through quantitative and qualitative analyses.
Development of the OSI policy analyserThe OSI Analyser tool for evaluating the alignment of policy documents with OSI principles was developed in accordance with the proposed framework, which is detailed in the later sections.
The OSI sentiment analyser frameworkThe OSI Sentiment Analyser framework consists of five components. This section describes these components and illustrates their interactions and flow (See Fig. 1).
Stage 1 - Configuration and customisation
The foundational stage of the OSI Analyser framework for the definitions and initial customisations. The first step defines the core constructs that the OSI Sentiment Analyser will work with, namely commitment, engagement, and empowerment. These constructs form the lens through which the analyser examines the policy documents to ensure that the detected sentiments reflect the psychological and cultural dimensions of workplace identity.
The next step was to secure a suitable custom lexicon. This process begins with the selection of a reliable, established lexicon from the literature for generic sentiment scoring (Kotelnikova et al., 2022; Liu & Zhang, 2012). To better align the lexicon with OSI-specific sentiments, experts iteratively updated the list. The updates included adding or removing words and updating their sentiment scores to improve how language in policies reflects OSI alignment (through detecting commitment, engagement, and empowerment).
Furthermore, score adjusters were customised to reflect the linguistic patterns in the organisational policy texts. Examples of score adjusters are intensifiers (e.g. exclamation marks, capitalisation, and words that emphasise) and negation (words like not, without). These customisations ensured that the OSI analyser detected sentiments specific to the OSI values more accurately.
Stage 2 - Text preprocessing and structuringThe second component of the OSI Sentiment Analyser framework extracts, preprocesses, and structures documents presented in the typical format of policy documents.
On receiving the documents, the text undergoes automatic preprocessing using established and customised natural language processing (NLP) techniques (Kadhim, 2018; Kathuria et al., 2021). As part of the preprocessing, sentences are broken into meaningful units, such as words, phrases, and sentences, known as tokenisation. Although tokenising words and sentences is generally straightforward, adjusting for context-specific phrases requires domain knowledge. For example, in hospital policies, expressions such as "infection prevention", "patient confidentiality", and "emergency procedures" are identified and treated as distinct semantic units to improve sentiment interpretation.
To focus the analysis on sentiment-carrying words, we drop stop words, which are words like "the" and "is" that carry little to no sentiment. To help the analyser treat different forms of a word consistently, we reduced all words to their base forms using stemming or lemmatisation techniques (Kotelnikova et al., 2022; Liu & Zhang, 2012; Uller et al., 2024). For example, the word "went" was reduced to "go" and the phrase "patients visiting" to "patient visit".
Both processed and original versions of the text were used for scoring. The processed version supports accurate scoring by isolating sentiment-carrying elements, and the original sentences help identify features such as capitalisation or punctuation (e.g. exclamation marks) that are useful for score adjustment and emphasis detection. The result was a structured dataset containing both formats.
Stage 3 - Scoring and classificationThe scoring and classification component of the OSI Sentiment Analyser evaluates the sentiment of policy texts by calculating scores at multiple levels: phrases, sentences, and documents. Additionally, the analyser first evaluates the three OSI constructs: commitment, engagement, and empowerment. These scores form the basis for calculating the OSI sentiment that embodies all constructs. In addition, predefined thresholds determine classifications of positive, neutral, or negative OSI-related sentiments.
For example, using a custom-built OSI lexicon, sentiment scores were first computed at the sentence level. The analyser refines these scores through rule-based adjustments inspired by the VADER model (Barik & Misra, 2024; Hutto & Gilbert, 2014), which accounts for linguistic features such as negation, intensity modifiers, and punctuation. Further adjustments consider sentence-level emphasis, including capitalisation, intensifiers, and down-toners (Cero et al., 2024; Nielsen, 2011).
Once the initial scores were derived, they were normalised to allow for consistent comparison across sentences and constructs. Using the score, each sentence was classified by OSI dimension and then by OSI (all three dimensions). Finally, aggregated scores across all sentences provide a document-level classification, offering a comprehensive view of how policy text supports or undermines organisational self-identity.
The following are the steps and formulas for scoring at different levels.
1. OSI Lexicon Score per Sentence (Base)
Each policy document d was segmented into sentence s∈d. For each OSI category cϵ{Commitment,Engagement,Empowerment}, we computed a raw sentiment score based on an OSI-specific lexicon:
Where:
- •
δc(w) is the sentiment weight of word w for OSI category c,
- •
the sum is taken over words w in sentence s that match the OSI lexicon
2. Rule-Based Adjustments (Inspired by VADER (Hutto & Gilbert, 2014))
Each word’s contribution is adjusted using rule-based modifiers based on
- •
Mod(w,s)ϵR+: degree modifier (e.g., very, highly, slightly), which adjusts the intensity of the contribution of the words. A default value of 1.0 leaves it as is.
- •
Neg(w,s)ϵ{−1,1}: negation term (inverts the polarity if negation words are detected within a window)
- •
Booster damping is applied to prevent an exaggerated effect of multiple intensifiers appearing together or in succession.
Booster dampening It is applied as follows:
Let:
- ○
sb= base sentiment score of the word
- ○
bi = scalar of the i th booster word
- ○
D(bi) = dampened booster weight
- ○
k = total number of booster words modifying the sentiment word
- ○
γ = dampening factor (e.g. 0.95 per booster)
The effect of applying this Booster Dampening is to make words closer to the sentiment word have more influence. Also, with each additional modifier (as I increases) the effect of booster diminishes exponentially.
Explanation:
3. Sentence Level Emphasis Adjustment
Examine each sentence for indication of emphasis and modify the scores accordingly and as follows:
Where:
emph(s)ϵR+: the emphasis multiplier e.g., between 1.1 to 1.5 (+0.2 for exclamation, +0.3 for capitalised sentiment words).
4. Normalise Sentence Score
Normalisation is done to prevent sentence length bias as follows:
Where:
Ns: number of OSI sentiment carrying words in sentence N
5. Sentence Classification by OSI Construct
Each sentence will have classification for each of the OSI construct. Calculations for the classifications are follows:
For each sentence s, let:
- •
OScorei(c): sentiment score for commitment
- •
OScorei(e): sentiment score for engagement
- •
OScorei(p): sentiment score for empowerment
- •
Tp: positive threshold (0.05)
- •
Tn: negative threshold (−0.05)
We classify the sentence for each OSI construct as follows:
6. Overall OSI Classification of Sentences
The sentiment scores by OSI construct were used to determine the overall class for each sentence. We used the hybridisation of two techniques: aggregation by weighted average and majority vote. a) Weighted Average Aggregation
Let weights wc,we,wp∈[0,1] be the importance of each OSI construct
Then classify as:
b) Majority Voting
Check the classifications of the sentences by OSI constructs and classify them using majority vote.
c) We use voting when close to the thresholds.Stage 4 – Results and output
The fourth stage of the framework aggregates sentiment scores to classify them by OSI constructs at multiple levels (document, sentence, phrase, and word) for broad and granular analysis. At each iteration of the review process, the output included different components to support interpretation and refinement. The output is a structured spreadsheet that indexes each sentence as it appears in the original document with the corresponding OSI sentiment scores and classes. This spreadsheet facilitates traceability and comparison across documents and versions. Another form of output is the visualisation of the results, such as sentiment distribution charts and word clouds highlighting key sentiment drivers. Fig. 2 shows pie charts of the distribution of OSI sentiments in Policy A1 before and after an iteration of the review process (See Fig. 2).
Finally, to help identify recurring themes and areas of concern within and across policies, the system groups sentences by sentiment class and analyses them. For example, analysing negative sentences (e.g. frequency of words, word clouds) may help identify topics or phrases of concern. The output of the analyser facilitates deep analysis and simplifies evidence-based revisions.
Stage 5 - Human expert reviewThe final stage of the OSI Sentiment Analyser framework involves human vetting of the results to ensure contextual accuracy and actionable insights. All other steps of the analysis, from text processing to sentiment scoring and classification, are fully automated. In addition to catching errors, reviewers serve as a source of feedback for updating the system. Importantly, the analyser is designed to support rather than replace human judgement, and expert interpretation remains central to ensuring that sentiment insights are understood within their organisational and policy contexts.
Key design decisionsWhile advanced machine learning models, such as those from large language models (LLMs) or commercial generative AI services, can offer impressive accuracy, they often require a large amount of labelled data, lack transparency, and typically require sharing data externally (Liu & Zhang, 2012; Sun et al., 2023; Zhang et al., 2018). These limitations make them less suitable for sensitive organisational contexts where data governance and interpretability are paramount.
For our project, we chose lexicon-based methods because of their transparency, customisability, and ability for local deployment (Cero et al., 2024; Kotelnikova et al., 2022) (Dragut et al., 2010). This approach effectively addresses ethical concerns, does not require training data, and is computationally efficient (Hossen & Dev, 2021; Sambo et al., 2021; Srivastava et al., 2022).
These methods provide effective, explainable, and controllable solutions to the problem. Furthermore, they allow us to create a system that includes a human-in-the-loop approach while ensuring that we do not compromise on privacy or accountability.
Ethical review and approvalThis research project received ethical approval from two esteemed universities in the United Kingdom, ensuring that all procedures met the rigorous standards set forth by their respective ethics committees. This approval reflects a thorough review process that assesses the study's adherence to ethical guidelines, participant safety, and overall integrity of the research methodology.
Validation of the OSI policy analyserThe validation of the OSI Sentiment Analyser involved several stages, iterating through processes to ensure that it was conceptually sound, ethically responsible, and practically reliable. This approach was driven by the need for rigorous validation of AI-based solutions, which cannot be assumed to be inherently safe.
Validation began by modelling OSI-related sentiments–commitment, engagement, and empowerment–based on established theories of workplace psychology. This theoretical foundation guided the development of the custom lexicon and scoring mechanisms, ensuring that the outputs aligned with empirically supported organisational constructs.
Throughout the development, workplace psychologists and domain experts played a central role in refining the sentiment lexicon, adjusting the scoring algorithms, and reviewing the outputs. Their feedback helped ensure that the analyser not only picked up on linguistic sentiment but also considered the broader contextual and organisational factors at play.
Before deploying it in the NHS case studies, we put the analyser through a thorough validation process using 78 publicly available organisational policy documents. Each round consisted of running the analyser, examining its outputs against expert opinions, and tweaking the lexicon and rules based on the differences we found. We focused on key metrics, such as how sentiment was classified (positive, neutral, or negative), how well the tool's findings aligned with expert reviews, and any inconsistencies that needed to be resolved.
The validation involved four iterations of the analysis, expert reviews, and refinement cycles. By the final round, the analyser consistently achieved a high level of agreement (over 90 %) with the expert judgement, and we resolved all major inconsistencies. This saturation in agreement indicates that further iterations are unlikely to yield significant improvements, demonstrating the robustness and reliability of the tool. The comprehensive validation process established a robust foundation for subsequent application in real-world NHS policy reviews, ensuring that the system is both reliable and responsive to organisational needs, with safeguards against oversights and biases common in AI deployments.
Deployment and case studyCase study implementationAs part of the case study, two NHS hospital trusts identified and submitted 12 policy documents that they wished to improve. Each document was reviewed using the OSI framework, which involved the following processes:
- •
Application of the OSI Sentiment Analyser to assess sentiment at the document, sentence, phrase, and word levels.
- •
Outputs from the OSI Analyser were reviewed by subject-matter experts, policy owners, and reviewers engaged in as many review iterations as necessary, using the analyser's feedback and expert recommendations to refine the documents until satisfactory alignment with OSI principles was achieved.
A detailed description of the review process and OSI Sentiment Analyser framework (including configuration, text preprocessing, scoring, classification, and expert review) is provided in the following sections.
Data collection and analysisThroughout the review process, both quantitative and qualitative data were collected and analysed as follows.
- 1.
Quantitative data were obtained by applying the OSI analyser to the policy documents. The data include sentiment scores, classifications, and statistics of the documents at different levels.
- 2.
Qualitative data were obtained from the structured feedback provided by the participants. The participants, who were policy owners and reviewers, shared their experiences in the OSI Analyser-supported review process. We then analysed this feedback to extract common themes, gauge perceptions of the tool's effectiveness, and gain insights into the overall policy review experience.
This mixed data enabled a detailed evaluation of the OSI Analyser's measurable impact, as well as the personal experiences of policy owners and reviewers. By examining how these stakeholders interacted with the AI-powered OSI Analyser, we evaluated its effectiveness in facilitating structured, iterative, and value-driven policy reviews.
Deployment processWe define a guideline to streamline the interactions between the system, policy owner, and human validator (as specified in Step 5 of the framework). The process flow begins when a policy reviewer submits a policy document for analysis. The OSI Analyser then processes the document and produces a detailed sentiment report. Subsequently, a human expert vets the report and provides feedback and recommendations. The policy owner receives the vetted report, reviews it, and can make revisions before resubmitting the document for further analysis, if needed.
After reviewing the report, the policy owner may revise and resubmit the policy document for further iterations. The deployment process has optional feedback loops to enable continuous improvement of the system, accuracy of the analyser, and user experience. One of the loops allows policy owners to provide feedback on the quality and usefulness of the review process itself. Another feature allows the reviewer to provide feedback to the developers. Feedback loops support improvements in accuracy and user experience. The process flow of these interactions is illustrated in Fig. 3. By combining automated efficiency with human oversight, the framework ensures high-quality, sentiment-aware policy analysis aligned with the OSI (See Fig. 3) (Pepple & Muthuthantridge, 2026).
Findings and discussion
This section presents the results of using the OSI Sentiment Analyser in a trial involving two hospital trusts and 12 policy documents. Prior to the trial, we tested the analyser on 78 publicly available documents. However, this report focuses on reviewing the 12 live documents and our direct interactions with policy owners and reviewers.
The 12 policies represent frequently used organisational documents that govern clinical practice, workforce management, and operational procedures within healthcare settings. These include policies on patient observation, rostering, discharge and transfer of care, safe staffing, and flexible working hours. Collectively, these policies reflect the regulatory and procedural landscape shaping staff responsibilities, resource allocation, and patient safety across the organisation.
As the focus of this study is the policy review process, we consider how effectively the proposed system identifies OSI-related sentiments, its utility in helping to align policy documents with OSI, and the convenience and value it adds. The results included quantitative summaries and qualitative insights. Owing to space constraints, we provide detailed results from working on two policy documents. Nevertheless, we present summaries of all 12 reviews.
Summary of policy review outcomesTable 1 presents the sentiment analysis outcomes for the policy documents from two hospital trusts (A and B) before and after review using the OSI Sentiment Analyser. It summarises changes in positive, neutral, and negative OSI sentiment scores, along with the number of iterations and the extent of improvement following full cycle or cycles of the policy review process (See Table 1).
Summary of outcomes of reviewing 12 policy documents. Improvement is the sum of the increase in positive OSI sentiments and the decrease in negative sentiments.
Improving sentiment involves increasing positive sentiments and decreasing negative sentiments. Thus, the analyser calculates the overall improvement in sentiment by summing the rise in positive sentiments and the reduction in negative sentiment scores; the figures are percentages of the total sentiments in the document. Moreover, using these two figures caters to instances where neutral sentiments become positive.
A paired t-test and Wilcoxon signed-rank test were used to compare the initial and post-review OSI scores for each policy. The paired t-test assumes normally distributed differences, which was supported by the Shapiro–Wilk tests (all p > 0.20) (J. Cohen, 2013; Shapiro & Wilk, 1965). The test also provides estimates of mean change, confidence intervals, and effect size. The Wilcoxon test serves as a non-parametric alternative that does not assume normality (Pallant, 2020; Wilcoxon, 1945).
The results showed a significant increase in the Positive OSI (t(11)=5.29, p = 0.00025; Wilcoxon p = 0.00049) and a significant decrease in the Negative OSI (t(11)=−3.91, p = 0.00244; Wilcoxon p = 0.00335), both with large effect sizes. The neutral OSI showed no significant changes (p=0.66). Overall, the tests confirmed a strong and reliable shift from negative to positive sentiment following the review.
Although we present a table of results here, users will receive a report with visualisations and datasets of the analysed texts. The visualisations provide easy-to-understand reporting and monitoring, as shown in Fig. 3. In addition, the policy owner receives a list of recommended actions, which are produced from automated reporting and expert review.
Interpretation of result variation across policiesImprovements in OSI sentiment varied across policies owing to several factors, including policy type, content sensitivity, and the presence of specific OSI dimensions. The following examples from our review process show this variety and its interpretation:
The first example demonstrates an effective policy that aligns with OSI principles: the policy for flexible working for doctors. The analyser found clear signs of empowerment, voice, and value in the participants’ language. Besides the substance of the policy, the results suggest that doctors engaging with this policy would feel supported in collectively deciding on flexible working patterns. In this instance, the result indicated minimal further action and was representative of cases where policies were already well aligned with little room for improvement.
Second, policies with mixed or moderate OSI compliance showed more complex outcomes. For example, in the “Principles of self-development time and flexible working”, the language around the focus on self-development as separate from annual leave was a positive OSI factor. The negative sentiment score increased because of clauses that required doctors to be available during staff crises and language suggesting irresponsibility in self-development. Accordingly, the OSI analysis noted these elements for revision and recommended recognising genuine barriers to returning during emergencies. It also suggests reframing disciplinary language in a more supportive way. This example illustrates how specific policy features and wording can limit improvements and require targeted, context-sensitive changes.
The “Support and guidance for rota changes” policy had a high negative sentiment score; however, expert review found that the negative language was necessary to protect doctors and support work-life balance and well-being of doctors. In these situations, negative sentiment does not indicate misalignment with OSI principles; it reflects essential obligations or safeguards. Here, human judgment is vital for differentiating between problematic and necessary negative language use.
These cases demonstrate that improvement levels vary across policies depending on their purpose, the need for specific language (e.g. safeguarding or compliance), and dominant OSI dimensions. Where negative language is non-negotiable, human judgment is crucial for contextualising the findings. This emphasises the role of the OSI analyser as a decision support tool rather than a simple automated rewriting tool, and indicates that construct-level diagnostics work best when paired with expert review and focused revisions. This approach ensures that sentiment analysis supports value-aligned policy development rather than replacing it.
Case study 1 – Hospital trust A (Policies A1–A8)Hospital Trust A showed consistent and notable improvements across its eight policy documents after applying the OSI Analyser using the framework and proposed process. As detailed in Table 1, the average total sentiment improvement across the policies was approximately 16.5 percentage points; A5 (37.6) and A8 (21.2) are the most significant gains. Improvements include both an increase in positive OSI sentiment and a reduction in negative sentiment.
- •
Positive sentiment increased for all eight policies, with A5 showing the highest increase (15.8 %).
- •
Negative sentiment was significantly reduced, with the highest decrease in A5 (−21.8 %) and A8 (−14.1 %).
- •
Substantial improvements were achieved with only a few review iterations; the mean number of iterations was 1.58, and the median was 1.5.
These outcomes indicate a successful alignment of policy language with the desired organisational self-identity values, enhancing their perceived tone and likely reception by employees.
Case study 2 – Hospital trust B (Policies B1–B4)The OSI sentiment of all the reviewed policy documents for Hospital Trust B improved. However, some policies showed less significant improvements than others. The summaries derived from Table 1 are as follows:
- •
Policies B2 and B3 showed notable gains, with total improvements of 26.8 % and 12.8 %, respectively.
- •
B4 showed a significant positive sentiment increase (9.1 %) but no reduction in negative sentiments. Feedback from the policy owner indicated that some flagged OSI-negative terms must remain in the document because of the issues the policy addresses.
- •
B1 experienced minimal change (2.5 %), indicating limited impact from the review process or the need for further iterations.
Trust B's average total improvement was approximately 12.8 percentage points, slightly lower than Trust A's but still indicating that the OSI analyser was effective.
Emerging themes from qualitative feedbackIn this section, we provide the results of the analysis of the feedback from policy owners and reviewers in the trial. The feedback focused on their experience, such as the perceived effectiveness, relevance, and usability of the OSI Sentiment Analyser.
The analysis of the feedback revealed seven core themes that helped contextualise the quantitative improvements and inform future developments.
1. Alignment with Human Judgement and Trust in Output
Participants expressed a high level of agreement with the analyser's output, indicating that it aligned well with expert intuition and internal reviews. This favourable feedback from users supports the credibility and usefulness of the system in real-world contexts. "It's been really useful for highlighting areas missed by internal reviewers…” "It is interesting to see how well your system identifies sentiments…"
The framework's transparent scoring system, combined with the involvement of domain experts, strengthens user confidence in its reliability. This trust is critical in organisational settings, where policies have significant operational consequences.
2. Clarity and Practical Value of Sentiment Reports
Users described the reports as being clear and actionable. Sentence-level analysis made it easy to target revisions without changing the underlying policy’s intent. One participant explained that the guidance helped reframe language while maintaining organisational objectives. “The specific guidance directs the reframing to promote autonomy, self-efficacy, and self-value while achieving the corporate aim.”
Another noted how encouraging it was to see progress in the right direction and expressed pride in being part of the process: “I am so pleased to see progress in the right direction. I am absolutely honoured to be considered …”
The analyser’s ability to highlight subtle tone shifts was crucial in bridging the gap between policy intent and staff experience.
3. Enhanced Efficiency in Policy Review
The system dramatically accelerated the review process, reducing manual effort and enabling quick iterations. One participant described it as invaluable for identifying issues that internal reviewers had overlooked. “It’s been really useful for highlighting areas missed by internal reviewers…”
What previously took weeks of human-led review could now be completed in minutes, with tone concerns flagged early in the drafting stage. This capability is widely recognised as a key driver of more agile and responsive policymaking.
4. Awareness of Language Impact and Organisational Values
A recurring theme was the greater awareness of how language shapes perception. Crucially, the analyser prompted the authors to think critically about tone and its alignment with organisational values, particularly in creating psychological safety and fostering empowerment. “If policy is written in a corporate way, it may be implemented with that ‘tone” “’ With OSI, the organisation is able to draw its lines, but it promotes values important to staff. Small adjustments can change the tone, leading to a values-based implementation.”
This reinforces the OSI Analyser’s value not only as an editing tool but also as a driver of more inclusive and engaging internal communication practices.
5. Facilitation of Constructive Dialogue and Collaboration
The structured, evidence-based feedback helped shift policy discussions away from subjective opinions and towards shared, value-driven objectives. One participant described this as an example of collaborative work and the use of innovative approaches to support the workforce and, in turn, improve the quality of care delivered to the population. “This is an excellent example of collaborative working practice… Our aim is to use innovative approaches to support the workforce, which in turn will benefit the care we deliver to our population.”
This collaborative orientation promotes institutional learning and a shared commitment to improving the communication culture.
6. Respect for Context and the Role of Human Oversight
Policy reviewers welcomed the inclusion of human validation within the framework and supported the importance of contextual interpretation and expert judgement. This feedback strongly supports a hybrid human–AI model in which automated insights are complemented by a domain-specific understanding. As one participant noted, the analyser promotes values that matter to staff. “…but it promotes values important to staff, and small adjustments can change the tone…”
While automation delivers scale and speed, human insight remains essential for ensuring effectiveness and responsibility.
The emerging themes from the qualitative analysis show that the OSI Analyser is more than just a sentiment-scoring tool. It represents a catalyst for more thoughtful, value-driven communication throughout policy development and review. Participants highlighted its practical benefits, particularly the gains in speed and clarity, which made revisions easier and more targeted. Beyond efficiency, the tool prompted greater awareness of tone and its influence on staff experience, encouraging authors to consider how language aligns with the organisation’s values. In doing so, the analyser supports not only strategic management objectives but also the broader goal of strengthening organisational culture.
Overall, the analysis of the results shows clear improvements in OSI sentiment scores across all 12 policy documents and strong positive qualitative feedback. Alignment with Organisational Self-Identity (OSI) principles improved for all documents, with some by more than 30 percentage points after only a few review cycles. These findings support the tool's ability to deliver fast, targeted refinements. Users attested to the accuracy and usability of the analyser and highlighted the value of sentence-level insights for spotting overlooked issues in tone. The participants praised the OSI analyser for encouraging evidence-based collaboration, transparency, and compatibility with expert oversight. Taken together, these findings position the OSI Sentiment Analyser as a practical, scalable solution for strengthening policy communication and organisational culture.
Furthermore, these themes align closely with the ethical foundations of the system. User comments on trust, clarity, and the importance of human oversight directly support the framework’s emphasis on transparency and responsible AI deployment. In this sense, the case study findings not only demonstrate the tool’s practical value but also empirically substantiate the ethical governance principles that guided its design.
ConclusionsThe findings from this study show that the OSI Sentiment Analyser does more than measure tone in policy documents; it helps organisations understand how their language shapes people’s sense of belonging, voice, and agency. Across the NHS case studies, policy owners not only improved OSI-related sentiment but also reported feeling more confident, informed, and supported during the revision process. These experiences highlight the real value of the tool: it creates space for more thoughtful conversations about how policies affect staff and brings new clarity to the often‑overlooked emotional impact of organisational language.
Our framework contributes to organisational identity theory by operationalising the OSI constructs of commitment, engagement, and empowerment into measurable and explainable sentiment dimensions. By aligning policy language with OSI‑specific markers rather than a single global sentiment value, the analyser provides a structured way to examine how internal texts express identity and how they may strengthen or weaken a sense of connection within the organisation itself. Transparent scoring rules offer researchers a replicable approach, bridging descriptive organisational identity theory with computational analysis in a way that supports scholarly critique and comparison.
Conceptually, this study reframes sentiment analysis as a theory-aligned assessment of OSI constructs. Methodologically, it provides a clear and explainable scoring pipeline supported by human oversight. The OSI Analyser also creates opportunities to test OSI-relevant hypotheses, such as whether empowerment-focused revisions lead to stronger OSI sentiment over time. Taken together, these contributions move the field beyond generic tone detection towards a more meaningful, values-aligned interpretation of organisational communication.
As a practical tool, the analyser offers organisations a reliable and transparent way to review policy language and check whether it reflects the values they aim to uphold. It enables its users to identify key phrases and sentences for refinement and offers practical ways to measure aspects of workplace identity that were once difficult to quantify (Yukalov, 2022). Its modular design implies it has the potential for adaptation across different sectors and document types. Ultimately, the significance of the work lies not only in the analyser itself but also in how it enables OSI-aligned communication. Importantly, the system is designed to complement, rather than replace, human judgement.
Limitations and future workAlthough the analyser demonstrated a clear value, several limitations warrant consideration. First, as a lexicon-based system, it may struggle to capture nuanced or context-dependent expressions of OSI constructs, particularly when empowerment or engagement is conveyed implicitly rather than explicitly. Second, this study evaluated the analyser using only healthcare policies written in English; further sector-specific and cross-linguistic validation is therefore required.
Beyond general methodological constraints, there is a risk that improvements in sentiment may be mistaken for substantive policy changes, which underscores the importance of human oversight. Finally, because the analyser interprets linguistic signals rather than organisational reality, its outputs should be complementary to the broader evaluative processes and not a replacement.
Future development will focus on improving the ability of the OSI to detect subtle and nuanced sentiments that simple lexicon-based methods miss. This may include exploring transformer-based language models while carefully addressing transparency and other related ethical issues. We also plan to adapt the analyser for different languages and contexts to make it more widely applicable. Finally, we aim to conduct a longitudinal study to understand how the tool influences users and organisational culture.
CRediT authorship contribution statementDr. Aliyu Sani Sambo: Writing – review & editing, Writing – original draft, Software, Resources, Investigation, Formal analysis, Data curation, Conceptualization. Dr. Dennis Gabriel Pepple: Writing – review & editing, Writing – original draft, Methodology, Investigation, Formal analysis, Data curation, Conceptualization.
