Reliability concerns consistency of measurement. In an assessment, that is a broader question than whether a program returns the same number when given the same answers.
Two kinds of consistency
A deterministic scoring system always converts an identical input in the same way. For example, starp's current table always maps 22 correct answers to 100. This is a property of the scoring rules.
Test reliability asks about scores under relevant repeated or equivalent measurement conditions. It depends on response data and the design of the assessment. Fixed code is necessary for consistent calculation, but it cannot supply that evidence on its own.
A thought experiment
Imagine a bathroom scale that adds five units to every reading. It could give very consistent repeated readings while still being systematically off. The analogy shows why consistency and accuracy of interpretation are not identical.
In testing, similarly stable results do not automatically establish that a task measures the intended ability or supports a particular decision. That is why reliability should be considered alongside validity.
Repeating a test introduces complications
People may remember items, recognize a format or adopt a better strategy on a second attempt. Changing the questions can reduce direct recall, but the new version may differ in difficulty. Testing under different time limits creates another complication.
A meaningful reliability estimate therefore describes the sample, the conditions and the type of consistency examined. A bare percentage without that context is difficult to interpret.
What to ask a test provider
Look for a technical explanation of how reliability was estimated and for which scores. A broad total and a short domain score may have different precision. Do not assume evidence for one transfers automatically to the other.
For personal practice, keep a record of conditions and repeated question exposure. That will not turn your notes into a reliability study, but it will help you read score changes more thoughtfully.