The AI in HR Reliability Framework: Eight Parameters Every Organisation Must Test Before Depending on AI

When an organisation is evaluating an AI in HR system or an AI product for the Human Resources function, the first question should not be how impressive the technology appears. The first question should be whether the organisation can genuinely depend on it.

Can the organisation rely on the AI system while screening resumes? Can it rely on the system while calculating employee information? Can employees depend on it when asking questions about promotion eligibility? Can HR use its output while making decisions that may affect people?

These questions are important because an AI in HR product does not become dependable merely because it can generate an answer. The organisation must understand how reliably it performs the work assigned to it, where it may fail, and what degree of checks and balances will be required before its output can be used.

For this purpose, I have brought together eight parameters from a Human Resources standpoint into what I call the AI in HR Reliability Framework. These parameters are not presented as ideas that exist nowhere else. The purpose is to bring them together from the perspective of the Human Resources function so that HR practitioners can evaluate whether an AI system is reliable, available and dependable for their actual use cases.

An organisation should know the extent to which every one of these parameters is being met. The purpose is not to assume that every AI product will achieve a perfect score. The purpose is to understand the score, recognise the related risk, and then decide what controls and guardrails are required.

The Eight Parameters of the AI in HR Reliability Framework

Understanding What Each Reliability Parameter Means in HR Practice

The first parameter is repeatability. Let us take a recruitment example. HR gives the AI a resume and a job description and asks whether the resume matches the job description. If the AI says that the resume matches, it should give the same response when the exact task is repeated with the same resume, the same job description and the same circumstances.

Repeating the task should not make the AI change its outcome. If the situation and information are unchanged, the output should also remain unchanged. Otherwise, an HR practitioner cannot know which answer should be relied upon. Repeatability therefore asks a simple but fundamental question: when I give the AI the same activity in the same scenario, does it continue to give me the same result?

The second parameter is accuracy. In simple terms, accuracy can be understood through calculation and factual information. If an HR practitioner asks AI to calculate something, the answer should be correct. If the system holds leave information for different employees, it must provide the correct information for the employee being discussed.

For example, if one employee has 15 days of leave and another has 12, the AI must not interchange their information. When HR asks about the second employee, the AI should provide 12 and not repeat the 15 days applicable to the first employee.

Accuracy means that what exists is what the AI communicates. It should not provide incorrect information or exchange information between people. This becomes particularly important in Human Resources because the system may be handling information belonging to several employees. An answer may look reasonable and still be wrong if it has been linked to the wrong person.

The third parameter is interpretation. Employees will not always ask exactly the same question using exactly the same language. Different employees may use different words while meaning the same thing. Even the same person may ask the same question differently depending on the time, the situation or the way the thought is expressed.

The AI system should understand what the employee is trying to say. If two differently worded questions have the same meaning, the AI should interpret them correctly and avoid producing contradictory outcomes.

For example, an employee may ask, “Am I eligible for promotion?” The same employee may express the question differently by asking what needs to be completed before becoming eligible for promotion. The wording is different, but the subject remains promotion eligibility.

AI should understand that both questions relate to the same underlying issue. It should not interpret one version in a way that produces a yes and another version in a way that produces a no merely because the words have changed. Interpretation is therefore the ability of AI to understand meaning rather than simply react to a rigid phrase.

The fourth parameter is memory. Memory is important because memory supports future accomplishment and provides the basis for learning. When an organisation trains an AI system or engages with it repeatedly, it must know what kind of memory the AI possesses.

Can the AI manage information for 10 employees, 100 employees or 1,000 employees? Can it work with one year of data, 10 years of data, 20 years of data or a much longer period? Can it manage information relating to particular sectors or locations? Can it remember conversations and retain what has been taught through those conversations?

The organisation needs clarity on the scope of the AI’s memory because that scope will affect the output. If the AI remembers only one year of information, HR must know that relevant information from before that period may not be available to it. If the system can manage 10 years of information, the organisation must understand what it needs to keep supplying so that the memory remains useful and updated.

Memory should therefore not be treated as a vague promise that the AI “learns”. HR must know what it remembers, what it does not remember, how much history it can manage and what needs to be continuously provided.

The fifth parameter is capacity. Capacity is the ability of AI to store and process information. This can be compared with storage space on a computer, telephone or cloud environment. Everyone understands that a hard disk or storage account has a defined limit. AI also requires clarity concerning the information it can store and process.

AI may be capable of handling immense quantities of information, but that capability requires the necessary supporting solution. When an organisation purchases an AI product, uses a cloud-based AI system or builds an AI model, it needs to be clear about the capacity being created.

Capacity will influence the use cases the organisation can support. If an AI in HR system is expected to work with large employee populations, long periods of information, detailed records and several HR activities, its ability to store and process that information must be evaluated accordingly.

The organisation must therefore ask what kind of information can be shared with the AI, how much information can be shared, and whether the available capacity matches the proposed Human Resources use case.

The sixth parameter is hallucination. The technical word is hallucination, although in normal language I would describe it as the AI telling lies. I do not mean lying in the legal sense. I mean that the AI gives information that is not factual or provides something that does not exist.

The AI may create its own data, make assumptions or generate scenarios. Memory and capacity can also play a role in hallucination. The organisation must therefore know when hallucination occurs, how it occurs and under what circumstances they are more likely to occur.

Most importantly, the organisation needs systems that can capture hallucinations or other non-factual outputs. This matters because people may depend on the AI system and use its answer while making a decision. If the organisation later discovers that the decision was based on information that never existed, the entire chain of decisions can fall flat.

This makes hallucination more than a technical inconvenience. For HR, it is a reliability issue. If the AI produces a confident answer, the confidence of the wording does not establish that the information is factual. The organisation needs a method for identifying and managing this risk before relying on the output.

The seventh parameter is bias. Bias is different from hallucination. Hallucination relates to information that is non-factual or does not exist. Bias relates to the AI favouring or not favouring an outcome.

Consider an AI recruitment use case. The organisation gives the AI selection criteria based on academic qualifications, years of experience and the present job being performed by a candidate. If exactly the same information is presented for 10 people, the AI should provide the same outcome.

If the outcomes differ despite the instructed criteria being identical, the system may be making assumptions based on other parameters that were never included in the instructions. Those influences could relate to age, gender, ethnic group or other attributes. In that situation, the AI is no longer following the organisation’s instructions exactly. Its output is being affected by bias.

The organisation therefore needs to determine how much bias has been controlled and how much bias remains in the AI system. It needs some way of knowing this rather than assuming that the system is unbiased.

The eighth parameter is consistency. Consistency refers to the ability of AI to remain stable in its outcomes and behaviour. What the system says should align with what it does.

Consistency includes the ability to repeat, remain accurate and interpret correctly, but it also asks whether the AI continues to behave in a dependable way across its work. An AI system cannot be considered reliable if it performs well in one interaction but behaves unpredictably in another comparable interaction.

Taken together, these eight parameters allow an organisation to move beyond a superficial assessment of whether an AI product appears impressive. They help HR examine whether the system can actually be trusted for the work it is expected to perform.

Your Organisation Must Test the AI Product on Its Own Data

Understanding the eight parameters is only the beginning. The organisation must use them to test the AI product it is considering or already using.

The purpose of testing is to make a sound judgement about what the AI is doing and to understand the risks being carried. If an organisation knows that its AI has a 20% repeatability error, it knows that the same task may not produce the same outcome in a meaningful number of cases. If it identifies a 10% error relating to bias, it can decide what checks, balances and acceptable limits are required.

These scores and testing limits should not be accepted merely because the seller states them. The seller may describe the product as having a particular level of accuracy or repeatability, but the organisation must conduct its own exercise.

The important question is not only how the AI product performs generally. The important question is how it performs with the organisation’s data, employees, questions, use cases and operating environment.

The testing must therefore be done by the organisation itself. The organisation should determine its own percentages for repeatability, accuracy, interpretation, memory, capacity, hallucination, bias and consistency.

Once these results are known, the organisation can establish appropriate guardrails. It can decide where a human check is necessary, where repeated testing is required, which use cases carry greater risk, and what limits are acceptable for the organisation.

The objective is not to declare that AI in HR is either completely reliable or completely unreliable. The objective is to know the degree of reliability.

An organisation that knows the limitations of its AI can plan accordingly. An organisation that does not know those limitations may unknowingly depend on outputs that are inconsistent, inaccurate, poorly interpreted, limited by memory or capacity, affected by hallucination, influenced by bias or unstable over time.

That is why the AI in HR Reliability Framework should be used as a practical evaluation tool. It gives HR practitioners a structured way to ask the right questions before depending on an AI product.

AI in HR should not be trusted merely because it responds quickly, produces polished language or appears knowledgeable. Reliability has to be tested.

The organisation must know whether the same situation produces the same answer. It must know whether employee information remains accurate. It must know whether different wording is interpreted correctly. It must know what the AI remembers, how much information it can process, when it may produce non-factual content, whether its outcomes contain bias and whether its behaviour remains consistent.

When an organisation understands these eight parameters and measures them using its own data, it becomes possible to make a far more informed decision about the role AI should play in the Human Resources function.

The final question is therefore not simply, “Does this AI product work?”

The more useful question is:

“To what degree is this AI in HR product repeatable, accurate, interpretable, capable of remembering, able to process the required capacity, controlled for hallucination and bias, and consistent enough for our organisation to depend upon?”

Once the organisation can answer that question with evidence from its own testing, it can establish the guardrails, checks and systems required to use AI in HR in the manner it wants.