Institutional Safety, Governance & Trust / Ethics by Design
SUB-T06-026 · StoryValue-Sensitive Design
The lesson plan looked personalised on screen. In the classroom, Eli saw that personalisation and understanding were not the same thing. Institutional systems show a material need to strengthen value-sensitive design so that governance claims are supported by live controls, evidence and recovery capability.
In Singapore, Eli's team at a regional health service had been asked to explore value-Sensitive Design. The immediate pressure was practical: value-Sensitive Design is often described in policy or documentation but not consistently implemented, observed or evidenced at runtime, creating gaps between institutional claims and actual behaviour. People could see activity, outputs and confident recommendations, but those signals did not establish that capability, safety or agency had improved.
Eli resisted turning the scenario into a success story too early. As a patient advocate, Eli knew that a memorable example can clarify a research problem, but it cannot validate a causal claim. The team therefore framed one answerable question: Which controls, evidence and institutional arrangements make value-sensitive design effective in practice, and how do outcomes vary by sector, system risk, organisational maturity and operating context? The story gave the work human stakes; the question gave it a boundary.
The working hypothesis was specific enough to fail: An explicit, testable and continuously evidenced approach to value-sensitive design, with clear ownership, independent review, runtime telemetry and recovery, will outperform policy-only or periodic compliance approaches. That wording changed the conversation. Instead of asking whether the idea sounded beneficial, the team had to compare conditions, define what improvement meant, and decide what evidence would count against the intervention. They also had to test whether a short-term gain concealed dependence, reduced understanding, new exclusion or a difficult handback when assistance disappeared.
The proposed study centred on value-sensitive design, rights-impact assessment, fairness testing, stakeholder deliberation. The design varied Independent variables: control design, ownership clarity, review independence, telemetry coverage and observed fairness, dignity, harm reduction, public value, trade-off transparency. Subgroup and accessibility analysis were not treated as optional additions. A result that helped an average participant while predictably harming a smaller group would not satisfy the programme's definition of success.
During the imagined pilot, the most useful moment was not a dramatic breakthrough. It was a disagreement. One participant completed the task faster but reported less control; another moved more slowly yet retained the process after support was withdrawn. Eli asked the team to record both observations without choosing a preferred ending. They were scenario prompts, not findings, and they exposed why performance alone could not carry the evaluation.
The team built recovery into the protocol. Participants could challenge a recommendation, inspect relevant reasoning, pause the intervention and resume unaided. Failure scenarios tested changed conditions and incomplete information. Delayed follow-up asked whether any advantage persisted and whether people could still act independently. This made the study less theatrical and more useful: the system had to support correction and handback, not merely produce an impressive first result.
The unknowns remained visible: Effect size, implementation cost, institutional resistance, cross-jurisdiction transfer, public interpretation, optimal review frequency, legacy-system constraints. The principal risks included paper compliance, authority ambiguity, evidence gaps, institutional capture. None could be resolved by the narrative itself. They required sourced literature, approved ethics and accessibility review, a pre-registered protocol, traceable evidence and reproducible analysis.
If the hypothesis is supported, the value could extend beyond one pilot in health and care. Target: improve fairness; dignity; harm reduction; public value; trade-off transparency; stakeholder acceptability while protecting rights, dignity, access, fairness and accountable human authority. The same evidence could inform product requirements, assurance services, training, procurement criteria and policy guidance. If the hypothesis is not supported, that result would still be valuable by preventing a weak approach from scaling behind attractive claims.
At the closing review, Eli replaced the original programme claim with a more honest sentence: “We know what must be tested next.” A runtime-evidenced institutional operating model for value-sensitive design linking ownership, authority, controls, receipts, telemetry, assurance, recovery and lifecycle. For the people represented by the story, progress would not mean a system doing more. It would mean a person remaining more capable when the system stepped back.
Reflection
What did we learn?: The scenario shows why value-Sensitive Design must be evaluated as a human-capability claim, not inferred from activity or short-term output. It also shows why assistance, burden, agency, subgroup effects, handback and recovery belong in the same evaluation.
Why does this matter?: Weak value-sensitive design can lead to unsafe deployment, unlawful or unauthorised action, wasted public resources, loss of rights, poor accountability and declining institutional trust.
What research does this connect to?: This subtopic sits within Ethics by Design and draws on public governance, risk management, assurance, audit, systems engineering, administrative law, ethics, cybersecurity and institutional design. Related subtopics: Ethical Requirements Engineering; Bias and Fairness; Non-Maleficence.
What should happen next?: Complete authoritative standards, legal and literature scan for Value-Sensitive Design; appoint institutional owner; map requirements, controls, evidence and telemetry; define baseline and test scenarios; convene independent, rights and stakeholder review; draft evaluation protocol.
Research connection
Hypothesis: An explicit, testable and continuously evidenced approach to value-sensitive design, with clear ownership, independent review, runtime telemetry and recovery, will outperform policy-only or periodic compliance approaches.
Scientific uncertainty: Effect size; implementation cost; institutional resistance; cross-jurisdiction transfer; public interpretation; optimal review frequency; legacy-system constraints; political and crisis effects.
Variables: Independent variables: control design; ownership clarity; review independence; telemetry coverage; enforcement level; organisational maturity; system risk. Outcomes: fairness; dignity; harm reduction; public value; trade-off transparency; stakeholder acceptability. Confounders: sector, scale, legal context, legacy systems, budget, workforce capability and incident history.
Research methods: Value-sensitive design; rights-impact assessment; fairness testing; stakeholder deliberation; scenario analysis; ethics review; stakeholder interviews; document and control review; fault and incident simulation; longitudinal implementation assessment; methods adapted specifically to Value-Sensitive Design.
Evidence: Authoritative laws, standards and policies; control register; ownership and authority map; pre-registered evaluation plan; runtime logs and receipts; review records; incident and exception data; stakeholder evidence; cost and outcome measures; independent validation.
Frameworks: Value–Requirement–Trade-off–Control–Outcome model applied specifically to Value-Sensitive Design.
Links: UNESCO Recommendation on the Ethics of AI — https://www.unesco.org/en/artificial-intelligence/recommendation-ethics; OECD AI Principles — https://oecd.ai/; Australian Human Rights Commission — https://humanrights.gov.au/; IEEE Ethically Aligned Design — https://standards.ieee.org/industry-connections/ec/autonomous-systems/.
Commercialisation and public value
Products: Governance control library; evidence and receipt ledger; assurance dashboard; policy-as-code module; institutional maturity benchmark; value-sensitive design operating playbook.
Services: Enterprise and public-sector subscriptions; assurance and audit engagements; governance APIs; policy-as-code libraries; certification support; capability training; managed evidence and telemetry services.
Industries: Government; regulators; healthcare; education; justice; infrastructure; financial services; procurement; public administration; critical systems.
Government: Citizens; public servants; executives; boards; regulators; auditors; legal and risk teams; technology teams; service users; civil society; suppliers.
Policy: AI governance; administrative law; public accountability; audit; procurement; standards; rights protection; transparency; records management; regulatory compliance.
Future research: Complete authoritative standards, legal and literature scan for Value-Sensitive Design; appoint institutional owner; map requirements, controls, evidence and telemetry; define baseline and test scenarios; convene independent, rights and stakeholder review; draft evaluation protocol.
Business opportunity: Develop a reusable value-sensitive design framework, control model, evidence pack, benchmark and operating workflow for institutions deploying AI and digital systems.
Scenario narrative — not an empirical finding.